Mahdi Bazrafshan
292ae17cfa
Why: - Need implementation plan and task tracking documentation - Need strategy explanations for reference Changes: - phases.md: 5-phase implementation plan with status tracking - tasks.md: 25 tasks mapped to phases and steps - chunking_strategies.md: detailed explanations of all 5 strategies
57 lines
2.6 KiB
Markdown
57 lines
2.6 KiB
Markdown
# Tasks
|
||
|
||
All tasks for the RAG Chunking Strategy Benchmarking Framework.
|
||
Each task maps to a Phase and Step in [phases.md](phases.md).
|
||
|
||
Status legend: `DONE` `IN_PROGRESS` `TODO`
|
||
|
||
## Phase 1 — Document Parsing + Storage
|
||
|
||
| # | Task | Status | Step |
|
||
|---|------|--------|------|
|
||
| 1 | Design FastAPI application architecture and project structure | DONE | — |
|
||
| 2 | Implement DOCX document loading and processing pipeline | DONE | 3 |
|
||
| 3 | Add configurable application settings and experiment parameters | DONE | 1 |
|
||
| 4 | Implement Qdrant vector database integration and collection management | DONE | 2 |
|
||
|
||
## Phase 2 — Chunking Strategies
|
||
|
||
| # | Task | Status | Step |
|
||
|---|------|--------|------|
|
||
| 5 | Create chunking strategy interface and abstraction layer | TODO | 5 |
|
||
| 6 | Implement Structure-Aware Markdown chunking strategy | TODO | 6 |
|
||
| 7 | Implement Recursive chunking strategy | TODO | 6 |
|
||
| 8 | Implement Semantic chunking strategy | TODO | 6 |
|
||
| 9 | Implement Parent-Child chunking strategy | TODO | 6 |
|
||
| 10 | Implement Contextual Structure-Aware chunking strategy | TODO | 6 |
|
||
| 11 | Implement OpenAI embedding service using text-embedding-3-small | TODO | 7 |
|
||
| 12 | Implement strategy-based document processing API | TODO | 8 |
|
||
|
||
## Phase 3 — Query Pipeline
|
||
|
||
| # | Task | Status | Step |
|
||
|---|------|--------|------|
|
||
| 13 | Implement question answering API with configurable chunking strategy selection | TODO | 9–10 |
|
||
|
||
## Phase 4 — Benchmarking + Evaluation
|
||
|
||
| # | Task | Status | Step |
|
||
|---|------|--------|------|
|
||
| 14 | Implement benchmarking pipeline for comparing chunking strategies | TODO | 12 |
|
||
| 15 | Implement RAG evaluation pipeline using GPT-4o-mini | TODO | 11 |
|
||
| 16 | Implement experiment tracking and result storage system | TODO | 12 |
|
||
| 17 | Create question-answer evaluation dataset from insurance regulation document | TODO | 12 |
|
||
| 18 | Implement HTML benchmark report generation system | TODO | 14 |
|
||
| 19 | Design HTML report structure for experiment comparison and visualization | TODO | 14 |
|
||
| 20 | Create background processing jobs for document ingestion and benchmarking | TODO | 12 |
|
||
|
||
## Phase 5 — Wiring + Verification
|
||
|
||
| # | Task | Status | Step |
|
||
|---|------|--------|------|
|
||
| 21 | Add logging, request tracking, and cost monitoring | TODO | 15 |
|
||
| 22 | Create API documentation and Swagger examples | TODO | 15 |
|
||
| 23 | Create automated tests for chunking strategies and RAG workflow | TODO | 16 |
|
||
| 24 | Validate end-to-end benchmarking workflow with insurance regulation dataset | TODO | 16 |
|
||
| 25 | Analyze and compare chunking strategy performance results | TODO | 16 |
|