docs: record fixed_size as the winning strategy family

Why:
- Close the benchmark with a written family decision backed by Experiment composites.

Changes:
- Add the decision memo and SVG charts generated from SQLite.

Impact:
- Documents fixed_size over semantic under text-embedding-3-large; ±N remains follow-up.

Co-authored-by: Cursor <cursoragent@cursor.com>

docs/final-chunking-strategy-decision.md

@@ -0,0 +1,149 @@
+# Chunking Strategy — Final Decision
+
+**Date:** 17 August 2026  
+**Decision:** Adopt **`fixed_size`** as the chunking Strategy family.  
+**Not decided here:** Neighbor Expansion level **±N** (follow-up).  
+**Corpus Embedding Model:** `text-embedding-3-large`  
+**Evaluation set:** 10 Word documents (Decision Board / neighbor-sweep universe)  
+**Questions:** 201 per Candidate (nomic semantic: 195)
+
+**One-line close:** **`fixed_size` wins** on official mean composite (**9.040** vs **8.727**). Pick **±N later**, but **not ±0**.
+
+Charts below are generated from `data/chunking_benchmark.db` with the same composite as the Decision Board:
+
+```text
+(0.3 × context relevance + 0.4 × answer similarity + 0.3 × faithfulness)
+  × (1 − hallucination rate)
+```
+
+Regenerate figures after new Experiments:
+
+```bash
+.venv/bin/python docs/assets/decision/generate_charts.py
+```
+
+---
+
+## 1. Why `fixed_size` is the winner
+
+Decision Board ranks by **mean composite**, not by “how many documents won.” Stage 1 auto-picks **`fixed_size ±3`** vs **`semantic @ text-embedding-3-large`**. Stage 2 mean composite is **+0.313** for `fixed_size`.
+
+![Stage 1 mean composite by Strategy Candidate](assets/decision/stage1-mean-composite.svg)
+
+| Candidate | Mean composite | Context | Similarity | Faithfulness | Hallucination | Docs |
+|-----------|----------------|---------|------------|--------------|---------------|------|
+| **fixed_size ±3** | **9.040** | 9.40 | 8.95 | 9.47 | 2.2% | 10/10 |
+| fixed_size ±2 | 8.987 | 9.43 | 8.93 | 9.44 | 2.7% | 10/10 |
+| fixed_size ±1 | 8.926 | 9.38 | 8.89 | 9.42 | 3.0% | 10/10 |
+| **semantic @ text-embedding-3-large** | **8.727** | 9.38 | 8.83 | 9.30 | 4.8% | 10/10 |
+| fixed_size ±0 | 8.606 | 9.18 | 8.79 | 9.25 | 4.9% | 10/10 |
+| semantic @ nomic-embed-text-v2-moe | 8.551 | 9.19 | 8.65 | 9.14 | 4.7% | 10/10 |
+
+`fixed_size ±1`, `±2`, and `±3` all beat the best semantic Candidate on **mean**. **`±0` (no Neighbor Expansion) does not** (8.606 vs 8.727). Shipping `fixed_size` without expansion would weaken this family decision.
+
+---
+
+## 2. Judge metrics (stage 2 winners)
+
+`fixed_size ±3` leads on every judge metric, including **lower hallucination**.
+
+![Mean judge metrics for stage 2 winners](assets/decision/stage2-metrics.svg)
+
+---
+
+## 3. Per-document showdown
+
+Mean ranking still favors `fixed_size`. **Head-to-head at ±3: semantic 6 / `fixed_size` 4.** Semantic’s six wins are mostly **small**; `fixed_size`’s four wins are **larger**, especially **fire**.
+
+![Per-document composite: fixed_size ±3 vs semantic @ large](assets/decision/stage2-per-document.svg)
+
+![Signed margin per document (fixed_size − semantic)](assets/decision/stage2-margins.svg)
+
+| Document | Questions | fixed_size ±3 | semantic @ large | Winner |
+|----------|-----------|---------------|------------------|--------|
+| fire.docx | 20 | **8.810** | 6.228 | **fixed_size** (+2.58) |
+| moavenin.docx | 20 | **9.735** | 8.621 | **fixed_size** (+1.11) |
+| general-havades-individuals.doc | 20 | **9.205** | 8.564 | **fixed_size** (+0.64) |
+| havades.docx | 20 | **8.483** | 7.969 | **fixed_size** (+0.51) |
+| bazresi.docx | 16 | 9.366 | **9.405** | semantic (−0.04) |
+| life-time-individual.docx | 20 | 9.445 | **9.505** | semantic (−0.06) |
+| Refah.docx | 20 | 9.680 | **9.815** | semantic (−0.14) |
+| lifetime-compensation.docx | 15 | 8.761 | **9.129** | semantic (−0.37) |
+| customer1.docx | 20 | 8.451 | **8.973** | semantic (−0.52) |
+| website.docx | 30 | 8.466 | **9.058** | semantic (−0.59) |
+
+Win-count would pick semantic. **Official product ranking (mean composite) picks `fixed_size`.** This memo follows the Decision Board rule (ADR-0026).
+
+---
+
+## 4. Full Candidate heatmap
+
+Every cell is a newest single-strategy Experiment under `text-embedding-3-large`. Semantic @ large **collapses on fire**; `fixed_size ±1…±3` stay high across the set.
+
+![Heatmap of composite scores by document and Candidate](assets/decision/heatmap-candidates.svg)
+
+Best `fixed_size` ±N **by document** (does not change the family call):
+
+| ±N | Documents where it is the best fixed_size cell |
+|----|------------------------------------------------|
+| ±3 | bazresi, fire, general-havades-individuals, havades, life-time-individual (5) |
+| ±1 | customer1, moavenin, Refah (3) |
+| ±2 | website (1) |
+| ±0 | lifetime-compensation (1) |
+
+---
+
+## 5. Method
+
+- Corpus Embedding Model locked to **`text-embedding-3-large`**
+- Newest **single-strategy** Experiment per document × Candidate (Decision Board cells)
+- Retrieval `top_k = 5`
+- Stage 1: best `fixed_size` ±N vs best `semantic@Boundary`
+- Stage 2: mean composite + per-document breakdown
+- Experiments dated **9–10 August 2026**
+- `logs/neighbor_sweep.log` recorded mid-run connection errors on some units; **SQLite now has a full 10×4 `fixed_size` grid** — treat the database as source of truth
+
+---
+
+## 6. Excluded strategies
+
+No comparable **10-doc, single-strategy, `text-embedding-3-large`** Experiments exist for the three Strategies below. They are not Decision Board Candidates (ADR-0026).
+
+| Strategy | Why it is not the winner |
+|----------|--------------------------|
+| **recursive** | Not on the Decision Board grid. Informal PDF / `text-embedding-3-small` multi-strategy runs (not comparable to this close-out) were mixed; recursive **does not get Neighbor Expansion** (expansion is `fixed_size` only). |
+| **contextual_retrieval** | Extra LLM call per chunk at process time. Same informal PDF/`small` runs scored **below** recursive and `fixed_size`. |
+| **semantic_parent_child** | Boundary-detection cost plus parent fetch at query. Informal PDF/`small` composites were **much worse** (~1.8–2.8). Fail-hard if Boundary embeds mismatch (ADR-0020). |
+
+One leftover five-strategy Experiment on `customer1` under large is **invalid** (four Strategies scored 0 / errors) and was ignored.
+
+---
+
+## 7. Binding decision
+
+**Use `fixed_size` for chunking** under **`text-embedding-3-large`.**
+
+**Do not use `semantic` as the default Strategy** on this evaluation universe.
+
+**Follow-up:** choose Neighbor Expansion **±N** among **±1, ±2, ±3**. Stage 1 auto-pick is **±3**. **±0 is not recommended** if this family decision is to remain valid.
+
+Production config change is **out of scope** for this record.
+
+---
+
+## 8. Evidence (stage 2 cells)
+
+| Document | `fixed_size ±3` experiment id | `semantic @ large` experiment id |
+|----------|-------------------------------|----------------------------------|
+| bazresi.docx | `6fe08750ee324130b3d46d8b3bc95280` | `6263c4f9e63347cea9eb19f4d36309c0` |
+| customer1.docx | `8c4d888d367b4c0299b2d19dd9654f80` | `0c4572e80902408796f0dd11776e8e55` |
+| fire.docx | `230a3cf57ae74b679ecde83cfeb9b6f5` | `414162efc314406db082ba8537c2423f` |
+| general-havades-individuals.doc | `7ad892cfcd6143109f77a7808c8dfbd9` | `14fa4f872373446c8b042fc8e84e6983` |
+| havades.docx | `04bd45d836a64c3da2e68bb747c0c9b8` | `232238b4d3304243b82b02326ef64617` |
+| life-time-individual.docx | `d682706f54824469b235c099bf5a3d3c` | `bc7d9f1fbe3540c097a5f8cc1a1c44bb` |
+| moavenin.docx | `63f81cf0a632418aab2f93884518ed71` | `4e01960b5db14e81975a4fbc86c4e10c` |
+| Refah.docx | `6304dba39f3f4c05935195c588a36a64` | `b5c630c45615451a93d97d80dee361df` |
+| website.docx | `fb9d47750ca74761a3c82c57aa51030b` | `53256897cfb74dd7a6508e750ff1ba63` |
+| lifetime-compensation.docx | `99736ffbc6da455b8b3ca3a0fb8e5ef7` | `e51c2b57e7364dfa97e5429f615d6e4e` |
+
+HTML reports: `GET /benchmarks/{id}/report`. Dashboard: Decision Tab, Corpus = `text-embedding-3-large`.