chunking_strategies_evaluation/docs/tasks.md

Tasks

All tasks for the RAG Chunking Strategy Benchmarking Framework. Each task maps to a Phase and Step in phases.md.

Status legend: DONE IN_PROGRESS TODO

Phase 1 — Document Parsing + Storage

#	Task	Status	Step
1	Design FastAPI application architecture and project structure	DONE	—
2	Implement DOCX document loading and processing pipeline	DONE	3
3	Add configurable application settings and experiment parameters	DONE	1
4	Implement Qdrant vector database integration and collection management	DONE	2

Phase 2 — Chunking Strategies

#	Task	Status	Step
5	Create chunking strategy interface and abstraction layer	DONE	5
6	Implement Structure-Aware Markdown chunking strategy	DONE	6
7	Implement Recursive chunking strategy	DONE	6
8	Implement Semantic chunking strategy	DONE	6
9	Implement Parent-Child chunking strategy	DONE	6
10	Implement Contextual Structure-Aware chunking strategy	DONE	6
11	Implement OpenAI embedding service using text-embedding-3-small	DONE	7
12	Implement strategy-based document processing API	DONE	8

Phase 3 — Query Pipeline

#	Task	Status	Step
13	Implement question answering API with configurable chunking strategy selection	DONE	9–10

Phase 4 — Benchmarking + Evaluation

#	Task	Status	Step
14	Implement benchmarking pipeline for comparing chunking strategies	DONE	12
15	Implement RAG evaluation pipeline using GPT-4o-mini	DONE	11
16	Implement experiment tracking and result storage system	DONE	12
17	Create question-answer evaluation dataset from insurance regulation document	DONE	12
18	Implement HTML benchmark report generation system	DONE	14
19	Design HTML report structure for experiment comparison and visualization	DONE	14
20	Create background processing jobs for document ingestion and benchmarking	TODO	12

Phase 5 — Wiring + Verification

#	Task	Status	Step
21	Add logging, request tracking, and cost monitoring	DONE	15
22	Create API documentation and Swagger examples	DONE	15
23	Create automated tests for chunking strategies and RAG workflow	TODO	16
24	Validate end-to-end benchmarking workflow with insurance regulation dataset	TODO	16
25	Analyze and compare chunking strategy performance results	TODO	16

Phase 6 — Admin Dashboard (UI)

#	Task	Status	Note
26	Design dashboard architecture (ADR-0001 through ADR-0009)	DONE	Babel JSX, persistent tabs, dark theme, /app mount
27	Create frontend skeleton: index.html with tab bar, dark theme, design tokens	DONE	src/static/index.html, 889 lines
28	Wire FastAPI to serve dashboard at /app (StaticFiles mount)	DONE	src/main.py modified
29	Implement admin backend: /admin/health endpoint	DONE	src/admin/routes.py + service.py
30	Implement admin backend: /admin/qdrant/* CRUD endpoints	DONE	Collections list, create, delete, wipe points
31	Implement admin backend: /admin/chunks/{doc_id} chunk preview	DONE	Backend only, no frontend panel
32	Implement admin backend: /admin/questions/* CRUD endpoints	DONE	Backend only, no frontend panel
33	Implement admin backend: /admin/cost-estimate endpoint	DONE	Backend only, no frontend panel
34	Build Documents tab: upload, list, process, delete	DONE	Drag-drop, strategy checkboxes, all wired
35	Build Home tab: status cards + quick start guide	DONE	Documents count, experiments count, server status
36	Build Query tab: form + answer display + chunk table	DONE	Document/strategy/top_k selectors, question textarea
37	Build Benchmarks tab: run form + experiments list	DONE	Strategies checkboxes, cost estimate, report links
38	Build Admin tab: health cards + Qdrant collections panel	DONE	Health status, collection list with delete
39	Build Admin tab: chunk preview panel	TODO	Backend ready, UI not built
40	Build Admin tab: questions dataset management panel	TODO	Backend ready, UI not built
41	Build Admin tab: standalone cost estimator panel	TODO	Backend ready, UI not built
42	Build Query tab: past queries history panel	TODO	Backend storage exists, no UI list
43	Build Benchmarks tab: in-dashboard experiment detail view	TODO	Report link exists, no drill-down UI
44	Build Documents tab: document content preview	TODO	Parsed text stored, no preview panel
45	Add real-time processing progress indicator	TODO	Spinner only, no per-strategy progress
46	Add global error handling and retry logic	TODO	Toast only, no retry button or banner
47	Create docs: cant-do-yet.md (backend ready, no UI)	DONE	docs/cant-do-yet.md
48	Create docs: out-of-scope-v1.md (intentionally excluded)	DONE	docs/out-of-scope-v1.md