chunking_strategies_evaluation/docs/adr/0001-through-0015-architectural-decisions.md

# ADR 0001: Unified Chunk Model Across All Strategies

All chunking strategies return chunks conforming to a single Pydantic model. Strategies that don't produce certain fields (e.g., parent_id for non-parent-child strategies) populate those fields as null. This keeps Qdrant payload schemas consistent and ensures benchmarking metrics compare equivalent structures.

The unified model fields:
- document_name (str)
- chunk_id (str)
- strategy_name (StrategyName enum)
- chunk_index (int) — position in the full document's chunk sequence
- token_count (int) — computed via tiktoken at chunking time
- character_count (int)
- parent_id (Optional[str]) — only populated by parent_child strategy
- enriched_content (Optional[str]) — only populated by contextual_structure strategy

The alternative — strategy-specific chunk shapes — would require per-strategy normalization before storage and make retrieval comparisons unreliable. With a unified model, every strategy's output is directly comparable in Qdrant payload structure, SQLite experiment records, and HTML benchmark reports.

## Considered Options

- Option A: Unified Chunk model with nullable fields for strategy-specific data
- Option B: Strategy-specific chunk shapes, normalized on ingestion

Option A was chosen because it eliminates schema divergence and makes the benchmarking pipeline uniform across strategies.

## Consequences

All five strategies must conform to the same Pydantic model. Adding new fields later is straightforward (add field with default null). Removing fields requires updating all strategies and re-indexing Qdrant collections. The nullable fields approach means strategies only populate what they produce — no forced defaults or sentinel values.

---

# ADR 0002: Hard-Fail on Contextual Enrichment Failure

If the LLM call for contextual enrichment fails (rate limit, service outage, API error), the contextual_structure strategy raises an exception immediately. Chunks are NOT stored in degraded form (without enrichment). This is because a "contextual" chunk without context is just a markdown chunk and would corrupt the benchmark comparison — you'd be measuring markdown chunking quality while labeling it as contextual.

The exception is caught at the strategy orchestration level (chunking/service.py), which marks the strategy as failed in the processing response. Other strategies in the same request proceed independently.

## Considered Options

- Hard-fail: raise exception, mark strategy as failed, store nothing
- Graceful degradation: store chunks without enrichment, mark as degraded
- Retry: retry the LLM call up to N times before failing

Hard-fail was chosen because degraded chunks would produce misleading benchmark results. Retry was rejected because if the LLM is rate-limited, retrying within the same request just delays the inevitable and blocks other strategies.

---

# ADR 0003: Per-Strategy Failure Isolation

Each chunking strategy runs independently within a synchronous processing request. If one strategy fails (e.g., contextual_structure due to LLM unavailability), the other strategies' results are still committed. The processing response includes a strategies_completed list and a strategies_failed list with error details.

The alternative — fail the entire request if any strategy fails — was rejected because it wastes already-computed results and forces the user to re-process strategies that already succeeded.

Note: Although we initially considered background job status tracking (ADR 0009), this isolation principle still applies at the response level — partial results are returned rather than all-or-nothing.

---

# ADR 0004: LLM-Judge Evaluation Without Retrieval Precision/Recall

Benchmark evaluation uses LLM-as-judge scoring rather than retrieval Precision and Recall. The four evaluation metrics are:

1. **Context Relevance** (1-10): Are the retrieved chunks relevant to the question?
2. **Answer Similarity** (1-10): How close is the generated answer to the expected answer?
3. **Faithfulness** (1-10): Is the answer supported by the retrieved chunks?
4. **Hallucination** (bool): Does the answer contain claims not in the retrieved context?

This was chosen because Precision/Recall require ground truth labels mapping questions to "correct" chunks, which don't exist for a single-document benchmark. Since chunks are strategy-specific, the set of "relevant" chunks differs across strategies — requiring per-strategy labeling that would be a separate project.

The architecture does not preclude adding labeled retrieval evaluation later. A labeled ground truth file could be loaded and Precision/Recall computed alongside the LLM-judge metrics without changing any existing code.

---

# ADR 0005: Hardcoded Article-Level Parents in Parent-Child Chunking

The parent-child strategy uses Article as the parent level. This is hardcoded for v1 because we have a single regulation document with clear Article structure within Sections.

Making it configurable (article/section/paragraph) was considered but rejected because:
- It adds a configuration parameter that won't be used for v1
- Each level requires different grouping logic (section contains multiple articles, paragraph is too granular)
- Testing the parameter adds overhead with no practical benefit

The extension point is trivial: add a PARENT_CHILD_PARENT_LEVEL config variable and a conditional branch in the strategy. This is a 30-minute change when a second document type requires it.

---

# ADR 0006: python-docx for DOCX Parsing and Structure Extraction

Document parsing uses python-docx directly rather than routing through pandoc, mammoth, or other converters.

The rationale: the contextual, parent-child, and markdown strategies all need the document's native structure tree (section/article/paragraph hierarchy from heading styles), not just flat text. python-docx provides direct access to paragraph styles and document hierarchy via its Document.paragraphs API, where each paragraph has a style.name property that identifies it as "Heading 1", "Heading 2", "Normal", etc.

External converters (pandoc, mammoth) output flat markdown, which would require re-parsing the markdown text to recover heading levels — a fragile approach that loses style information and edge cases (nested lists, tables, special formatting).

The parsed structure is stored as a JSON-serializable DocumentTree in SQLite, shared across all strategies. Structure extraction happens once per upload regardless of how many strategies are requested.

---

# ADR 0007: Store Parsed Document in SQLite for Re-Processability

After a document is uploaded, its parsed representation (extracted text, DocumentTree JSON, file metadata) is persisted in SQLite alongside experiment data.

This enables re-running individual strategies with different parameters without re-uploading and re-parsing the document. For example: tuning SEMANTIC_THRESHOLD from 0.3 to 0.4 and re-running only the semantic strategy, reading the already-parsed document from SQLite.

Without this, every parameter change would require re-processing the full pipeline from .docx through python-docx parsing — approximately 2-5 seconds of unnecessary work per re-run.

SQLite was chosen over PostgreSQL for v1 because: single-user benchmarking tool, no concurrent access requirements, zero infrastructure setup, and the data volume is small (documents + experiments, not transactional data).

---

# ADR 0008: Domain-Based Directory Structure

The project uses a vertical-slice (domain-based) directory layout with four domains plus a core layer:

```
src/
├── main.py
├── core/           # config, dependencies, exceptions, shared models
├── documents/      # upload, parsing, document tree
├── chunking/       # strategies, orchestration, embedding
├── storage/        # qdrant, sqlite
└── benchmarking/   # queries, benchmarks, evaluation, HTML reports
```

Each domain contains its own routes.py, service.py, and models.py. This means adding a new feature (e.g., a new endpoint) only touches one directory.

The alternative — horizontal/layer-based (api/, services/, models/) — was rejected because adding a new feature requires touching multiple directories (api for routes, services for logic, models for schemas). This creates implicit coupling and makes it unclear where new code belongs.

A core/ directory holds configuration (pydantic-settings), dependency injection (OpenAI/Qdrant singletons), shared models (Chunk, StrategyName), and exception handlers. All domains import upward to core; no domain imports from another domain.

---

# ADR 0009: Synchronous REST Without Background Jobs

All endpoints execute synchronously and return results directly. There is no Job model, no polling endpoint, and no async task queue.

Measured timing for v1 scale:
- Document upload + parsing: < 1 second
- Single strategy processing (chunk + embed + store): 5-15 seconds
- All 5 strategies: 30-60 seconds
- Single query (retrieval + generation): 2-5 seconds
- Dataset benchmark (10 questions x 5 strategies): 2-5 minutes

All of these are acceptable within an HTTP request lifecycle with a loading indicator.

Background job tracking was considered (returns job_id, polls status via GET /jobs/{id}) but rejected because it adds: a Job model, a status state machine (started -> running -> completed/failed), a polling endpoint, and stale job cleanup logic — none of which are needed at v1 scale.

If processing times grow beyond a few minutes (e.g., 100-question datasets with 20+ page documents), async can be introduced via a task queue (Celery or FastAPI BackgroundTasks) without changing the resource model — the POST endpoints would still accept the same request body and return a job_id, just with actual background execution.

---

# ADR 0010: RESTful Resource Model

The API exposes four primary resources following standard REST conventions:

| Resource | Create | Read | Delete | Actions |
|----------|--------|------|--------|---------|
| Documents | POST /documents | — | DELETE /documents/{id} | POST /documents/{id}/process |
| Queries | POST /queries | GET /queries/{id} | — | — |
| Benchmarks | POST /benchmarks | GET /benchmarks/{id} | — | GET /benchmarks/{id}/report |
| Strategies | — | GET /strategies | — | — |

The alternative — verb-based endpoints (POST /ask, POST /benchmark/run) — was rejected because it conflates actions with resources and breaks REST's noun-based resource model.

Key naming decisions:
- "ask" is implemented as POST /queries (a Query is the resource, not the action)
- Dry run uses a query parameter: POST /benchmarks?dry_run=true (same resource, different mode)
- Process is a sub-action on documents, not a standalone endpoint: POST /documents/{id}/process
- Dataset vs single-question is determined by request body, not URL

---

# ADR 0011: Embed Full Enriched Text in Contextual Strategy

For the contextual_structure strategy, the full enriched text is embedded — including Document/Section/Article/Context/Content labels — not just the raw content. This is a deliberate design choice that affects what the benchmark measures.

Enriched text example:
```
Document: Insurance Regulation
Section: Claims and Compensation
Article: 15
Context: This section explains compensation payment rules.
Content: The insurer must pay compensation within 15 days.
```

The whole enriched string is what gets sent to text-embedding-3-small. This means the contextual strategy's vectors encode structural metadata and generated context, not just the regulation text. The hypothesis being tested is: does adding contextual enrichment to the embedding improve retrieval quality compared to raw content embeddings?

The alternative — embed only the Content field, store enriched version in payload for display only — was rejected because it would test whether enrichment helps at answer-generation time (when the LLM sees the enriched payload), but NOT whether it helps at retrieval time (when vector similarity determines which chunks are returned). Since retrieval is the primary bottleneck in RAG, the embedding must reflect the enrichment.

---

# ADR 0012: Sentence-Level Semantic Chunking with Minimum Chunk Size

The semantic strategy operates at sentence granularity, not paragraph granularity. Each sentence gets its own embedding, and cosine similarity between adjacent sentences determines chunk boundaries.

Minimum chunk size is enforced: if a boundary is detected before the minimum is reached (configurable via SEMANTIC_MIN_CHUNK_SIZE, default 3 sentences), accumulation continues. The boundary sentence stays with the previous chunk, not the next.

The alternatives considered were:

1. **Paragraph-level semantic**: Embed each paragraph, cut when paragraph similarity drops. Simpler but less precise — a long paragraph with a mid-paragraph topic shift won't be caught. Chosen against because sentence-level is the canonical approach in the literature and provides the finest boundary detection.

2. **Sentence-level without minimum size**: Allows single-sentence chunks. Rejected because degenerate tiny chunks hurt retrieval quality — a single sentence lacks enough context for the LLM to generate a meaningful answer.

3. **Adaptive threshold**: Instead of a fixed cosine similarity cutoff, use a rolling average that adapts to the document's baseline similarity. More complex to implement and explain. Rejected for v1 because we're testing a single document — the fixed threshold is sufficient and the results are easier to interpret.

The minimum chunk size is a direct consequence of sentence-level granularity — without it, the strategy would produce many unusable single-sentence chunks.

---

# ADR 0013: Leaf-Level Markdown Chunks Without Overlap

The markdown_structure strategy creates chunks only at the deepest header level (leaf chunks). If a document has:

```
## Claims (Section)
### Article 15
### Article 16
```

Chunks are created for Article 15 and Article 16. The "Claims" section itself is NOT a separate chunk — its content is fully contained within its child articles.

The alternative — create chunks at every header level (both Claims and its children) — was rejected because it produces overlapping content. A chunk for "Claims" would contain the text of Articles 15 and 16, and those articles would also have their own chunks. This duplication means:
- The same text appears in multiple chunks, inflating chunk count
- Retrieval could return overlapping content, wasting top_k slots
- Benchmark metrics become less meaningful (more chunks retrieved doesn't mean better coverage)

Leaf-level produces the cleanest, most non-overlapping chunk set for fair comparison with other strategies.

---

# ADR 0014: Strict Unidirectional Import Rules

All domain directories (documents/, chunking/, benchmarking/, storage/) import only from core/. No domain imports from another domain. This is enforced as a project convention.

The dependency graph is strictly:

```
core/          <-  imports nothing from the project
documents/     ->  imports from core/
chunking/      ->  imports from core/
storage/       ->  imports from core/
benchmarking/  ->  imports from core/
```

If benchmarking needs to reference a query or document, it does so through a shared model defined in core/models.py, not by importing chunking/models.py or documents/models.py.

The alternative — allowing cross-domain imports (e.g., benchmarking imports from chunking to call strategy functions directly) — was rejected because it creates circular dependency risk and makes it impossible to test, replace, or refactor one domain without affecting others.

This rule is simple enough to enforce manually in code review. If the project grows, a linter rule (import-linter or similar) can be added.

---

# ADR 0015: Direct API Usage — No LangChain, No LlamaIndex

The project uses OpenAI API, Qdrant SDK, and FastAPI directly. No abstraction frameworks (LangChain, LlamaIndex) are used.

The rationale:
- **Control**: Every API call, every prompt, every embedding is explicit and inspectable. No hidden abstractions making decisions on your behalf.
- **Debuggability**: When a chunk is wrong or an answer is bad, you can trace the exact prompt and parameters. Framework wrappers often obscure the actual API call.
- **Fairness**: Using raw OpenAI SDK means all strategies go through identical API paths. Framework abstractions sometimes add caching, batching, or prompt modification that could bias results.
- **Learning**: For a benchmarking tool, understanding exactly what's happening at each layer is valuable. Frameworks hide the mechanics you're trying to measure.
- **Dependency reduction**: Fewer packages to install, fewer version conflicts, fewer breaking changes from upstream framework updates.

The cost is more boilerplate code for common patterns (API calls, retries, error handling). For a benchmarking tool with well-defined, non-repetitive API usage, this cost is minimal.

The ChunkingStrategy interface is the project's own abstraction — simple, explicit, and sufficient. It does the one thing frameworks do (strategy polymorphism) without the hundreds of things you don't need.