# Phases & Steps

Implementation plan for the RAG Chunking Strategy Benchmarking Framework.
Each step maps back to tasks in [tasks.md](tasks.md).

---

## Phase 1: Document Parsing + Storage

Foundation layer. Everything else depends on this.

| Step | What | Files | Status |
|------|------|-------|--------|
| 1 | SQLite Storage Layer | `src/storage/sqlite.py` | DONE |
| 2 | Qdrant Storage Layer | `src/storage/qdrant.py` | DONE |
| 3 | Document Parser | `src/documents/parser.py` | DONE |
| 4 | Document Routes + Service | `src/documents/routes.py`, `src/documents/service.py`, `src/documents/models.py` | DONE |

**Endpoints delivered:**
- `POST /documents` — upload .docx
- `POST /documents/{id}/process` — run strategies (stubbed)
- `DELETE /documents/{id}` — remove document + vectors
- `GET /strategies` — list all 5 strategies

---

## Phase 2: Chunking Strategies

The core evaluation targets. Each strategy is an independent module.

| Step | What | Files | Status |
|------|------|-------|--------|
| 5 | Base Strategy Interface | `src/chunking/base.py` | TODO |
| 6 | Five Strategy Implementations | `src/chunking/strategies/recursive.py`, `markdown_structure.py`, `semantic.py`, `parent_child.py`, `contextual_structure.py` | TODO |
| 7 | Embedding Service | `src/chunking/embedding.py` | TODO |
| 8 | Chunking Orchestration | `src/chunking/service.py` | TODO |

**ADR alignment:**
- ADR 0001: Unified Chunk model (nullable fields)
- ADR 0002: Hard-fail on contextual enrichment failure
- ADR 0003: Per-strategy failure isolation
- ADR 0005: Hardcoded article-level parents
- ADR 0011: Embed full enriched text for contextual
- ADR 0012: Sentence-level semantic with min chunk size
- ADR 0013: Leaf-level markdown chunks only
- ADR 0015: Direct API usage, no LangChain/LlamaIndex

**Delivers:** The `POST /documents/{id}/process` endpoint goes from stub to real — chunking, embedding, and Qdrant storage for all 5 strategies.

---

## Phase 3: Query Pipeline

Retrieval + generation. The bridge between storage and evaluation.

| Step | What | Files | Status |
|------|------|-------|--------|
| 9 | Query Service | `src/benchmarking/query_service.py` | TODO |
| 10 | Query Routes | `src/benchmarking/routes.py` | TODO |

**Endpoints delivered:**
- `POST /queries` — ask a question against a strategy
- `GET /queries/{id}` — retrieve past query

**Pipeline:** question → embed → vector search (Qdrant) → top_k chunks → gpt-4o-mini → answer + metadata.

---

## Phase 4: Benchmarking + Evaluation

The reason the project exists. Compare strategies head-to-head.

| Step | What | Files | Status |
|------|------|-------|--------|
| 11 | LLM-as-Judge Evaluation | `src/benchmarking/evaluation.py` | TODO |
| 12 | Benchmark Service | `src/benchmarking/benchmark_service.py` | TODO |
| 13 | Benchmark Routes | extends `src/benchmarking/routes.py` | TODO |
| 14 | HTML Report Template | `src/benchmarking/templates/report.html` | TODO |

**Endpoints delivered:**
- `POST /benchmarks` — run benchmark (single or dataset)
- `POST /benchmarks?dry_run=true` — cost estimate only
- `GET /benchmarks/{id}` — retrieve experiment results
- `GET /benchmarks/{id}/report` — download HTML report

**Evaluation metrics (ADR 0004):**
- Context Relevance (1–10)
- Answer Similarity (1–10)
- Faithfulness (1–10)
- Hallucination (bool)

**Modes:**
- Single question
- Dataset (questions.json)
- Dry run (cost estimation)

---

## Phase 5: Wiring + Verification

Polish, integration, and proof that it all works end-to-end.

| Step | What | Files | Status |
|------|------|-------|--------|
| 15 | Wire All Routers + Logging | `src/main.py` | TODO |
| 16 | End-to-End Smoke Test | test suite | TODO |

**Verification checklist:**
- Upload insurance .docx
- Process with all 5 strategies
- Ask 3 test questions
- Run mini benchmark (3 questions × 5 strategies)
- Verify HTML report renders
- Confirm per-strategy failure isolation (simulate one strategy failure)
- Confirm dry-run cost estimation matches actual cost within 10%