diff --git a/docs/LLD.md b/docs/LLD.md index 2127788..07aa894 100644 --- a/docs/LLD.md +++ b/docs/LLD.md @@ -84,7 +84,7 @@ get_qdrant_client() | `ollama_base_url` | host URL | Local embeds | | `qdrant_url` | `http://localhost:6333` | Vectors | | `top_k` | `5` | Retrieval | -| `neighbor_prev` / `neighbor_next` | `0` / `0` | Expansion defaults | +| `neighbor_prev` / `neighbor_next` | `3` / `3` | Expansion defaults (fixed_size ±3) | | `chunk_size` / `chunk_overlap` | `512` / `50` | fixed_size / recursive targets | | `semantic_threshold` | `0.3` | Fallback if model default missing | | `semantic_min_chunk_size` | `3` | Min units per semantic chunk | diff --git a/docs/final-chunking-strategy-decision.md b/docs/final-chunking-strategy-decision.md index 5e03d69..6fdfdcd 100644 --- a/docs/final-chunking-strategy-decision.md +++ b/docs/final-chunking-strategy-decision.md @@ -1,13 +1,12 @@ # Chunking Strategy — Final Decision -**Date:** 17 August 2026 -**Decision:** Adopt **`fixed_size`** as the chunking Strategy family. -**Not decided here:** Neighbor Expansion level **±N** (follow-up). +**Date:** 17 August 2026 (family); **22 August 2026** (±N locked) +**Decision:** Adopt **`fixed_size`** as the chunking Strategy family with **Neighbor Expansion ±3/3**. **Corpus Embedding Model:** `text-embedding-3-large` **Evaluation set:** 10 Word documents (Decision Board / neighbor-sweep universe) **Questions:** 201 per Candidate (nomic semantic: 195) -**One-line close:** **`fixed_size` wins** on official mean composite (**9.040** vs **8.727**). Pick **±N later**, but **not ±0**. +**One-line close:** **`fixed_size ±3`** is the stabilized Strategy Candidate (**9.040** mean composite vs **8.727** for semantic @ large). **Do not use ±0** as the default — it loses to semantic on mean. Charts below are generated from `data/chunking_benchmark.db` with the same composite as the Decision Board: @@ -123,15 +122,38 @@ One leftover five-strategy Experiment on `customer1` under large is **invalid** **Use `fixed_size` for chunking** under **`text-embedding-3-large`.** +**Use Neighbor Expansion `neighbor_prev=3`, `neighbor_next=3`** (symmetric **±3/3**) for Query and Experiment defaults when running `fixed_size`. + **Do not use `semantic` as the default Strategy** on this evaluation universe. -**Follow-up:** choose Neighbor Expansion **±N** among **±1, ±2, ±3**. Stage 1 auto-pick is **±3**. **±0 is not recommended** if this family decision is to remain valid. +**Do not default to ±0.** Without expansion, semantic @ large beats `fixed_size` on mean composite (8.727 vs 8.606). -Production config change is **out of scope** for this record. +Operational defaults are set in `src/core/config.py`, `.env.example`, and the Dashboard Query/Benchmarks forms. Override per request is still supported. --- -## 8. Evidence (stage 2 cells) +## 8. Addendum — why ±3 (22 August 2026) + +Stage 1 ranked all `fixed_size` Candidates on the 10-doc grid: + +| ±N | Mean composite | vs semantic @ large | +|----|----------------|---------------------| +| ±0 | 8.606 | **loses** (−0.121) | +| ±1 | 8.926 | wins (+0.199) | +| ±2 | 8.987 | wins (+0.260) | +| **±3** | **9.040** | **wins (+0.313)** | + +**±3** is the stage 1 auto-pick and the highest mean composite. **±1** and **±2** are close; **±0** is ruled out. + +Per-document best ±N varies (±3 wins on 5 docs, ±1 on 3, ±2 on 1, ±0 on 1). The **global** default is still **±3** because Decision Board ranks by mean composite across the full set, not by win-count among ±N levels. + +**Caveats kept from stage 2:** semantic still wins head-to-head on 6/10 docs vs ±3 (website, customer1, etc.), but mean composite and judge metrics favor **`fixed_size ±3`**. Retrieval Inspect on outlier docs remains optional follow-up. + +**Production RAG outside this repo** is not changed by this addendum — only benchmarker defaults and this record. + +--- + +## 9. Evidence (stage 2 cells) | Document | `fixed_size ±3` experiment id | `semantic @ large` experiment id | |----------|-------------------------------|----------------------------------|