docs: lock fixed_size ±3 in decision record and lld

Why:
- Close the open ±N follow-up from the strategy finalization memo.

Changes:
- Add addendum with ±0…±3 evidence and binding ±3/3 default.
- Sync LLD config table with new neighbor expansion defaults.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
2026-08-22 13:27:18 +03:30
parent d5ccec2f2c
commit dc76ca67b8
2 changed files with 30 additions and 8 deletions

View File

@@ -1,13 +1,12 @@
# Chunking Strategy — Final Decision
**Date:** 17 August 2026
**Decision:** Adopt **`fixed_size`** as the chunking Strategy family.
**Not decided here:** Neighbor Expansion level **±N** (follow-up).
**Date:** 17 August 2026 (family); **22 August 2026** (±N locked)
**Decision:** Adopt **`fixed_size`** as the chunking Strategy family with **Neighbor Expansion ±3/3**.
**Corpus Embedding Model:** `text-embedding-3-large`
**Evaluation set:** 10 Word documents (Decision Board / neighbor-sweep universe)
**Questions:** 201 per Candidate (nomic semantic: 195)
**One-line close:** **`fixed_size` wins** on official mean composite (**9.040** vs **8.727**). Pick **±N later**, but **not ±0**.
**One-line close:** **`fixed_size ±3`** is the stabilized Strategy Candidate (**9.040** mean composite vs **8.727** for semantic @ large). **Do not use ±0** as the default — it loses to semantic on mean.
Charts below are generated from `data/chunking_benchmark.db` with the same composite as the Decision Board:
@@ -123,15 +122,38 @@ One leftover five-strategy Experiment on `customer1` under large is **invalid**
**Use `fixed_size` for chunking** under **`text-embedding-3-large`.**
**Use Neighbor Expansion `neighbor_prev=3`, `neighbor_next=3`** (symmetric **±3/3**) for Query and Experiment defaults when running `fixed_size`.
**Do not use `semantic` as the default Strategy** on this evaluation universe.
**Follow-up:** choose Neighbor Expansion **±N** among **±1, ±2, ±3**. Stage 1 auto-pick is **±3**. **±0 is not recommended** if this family decision is to remain valid.
**Do not default to ±0.** Without expansion, semantic @ large beats `fixed_size` on mean composite (8.727 vs 8.606).
Production config change is **out of scope** for this record.
Operational defaults are set in `src/core/config.py`, `.env.example`, and the Dashboard Query/Benchmarks forms. Override per request is still supported.
---
## 8. Evidence (stage 2 cells)
## 8. Addendum — why ±3 (22 August 2026)
Stage 1 ranked all `fixed_size` Candidates on the 10-doc grid:
| ±N | Mean composite | vs semantic @ large |
|----|----------------|---------------------|
| ±0 | 8.606 | **loses** (−0.121) |
| ±1 | 8.926 | wins (+0.199) |
| ±2 | 8.987 | wins (+0.260) |
| **±3** | **9.040** | **wins (+0.313)** |
**±3** is the stage 1 auto-pick and the highest mean composite. **±1** and **±2** are close; **±0** is ruled out.
Per-document best ±N varies (±3 wins on 5 docs, ±1 on 3, ±2 on 1, ±0 on 1). The **global** default is still **±3** because Decision Board ranks by mean composite across the full set, not by win-count among ±N levels.
**Caveats kept from stage 2:** semantic still wins head-to-head on 6/10 docs vs ±3 (website, customer1, etc.), but mean composite and judge metrics favor **`fixed_size ±3`**. Retrieval Inspect on outlier docs remains optional follow-up.
**Production RAG outside this repo** is not changed by this addendum — only benchmarker defaults and this record.
---
## 9. Evidence (stage 2 cells)
| Document | `fixed_size ±3` experiment id | `semantic @ large` experiment id |
|----------|-------------------------------|----------------------------------|