docs: lock fixed_size ±3 in decision record and lld
Why: - Close the open ±N follow-up from the strategy finalization memo. Changes: - Add addendum with ±0…±3 evidence and binding ±3/3 default. - Sync LLD config table with new neighbor expansion defaults. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -84,7 +84,7 @@ get_qdrant_client()
|
||||
| `ollama_base_url` | host URL | Local embeds |
|
||||
| `qdrant_url` | `http://localhost:6333` | Vectors |
|
||||
| `top_k` | `5` | Retrieval |
|
||||
| `neighbor_prev` / `neighbor_next` | `0` / `0` | Expansion defaults |
|
||||
| `neighbor_prev` / `neighbor_next` | `3` / `3` | Expansion defaults (fixed_size ±3) |
|
||||
| `chunk_size` / `chunk_overlap` | `512` / `50` | fixed_size / recursive targets |
|
||||
| `semantic_threshold` | `0.3` | Fallback if model default missing |
|
||||
| `semantic_min_chunk_size` | `3` | Min units per semantic chunk |
|
||||
|
||||
@@ -1,13 +1,12 @@
|
||||
# Chunking Strategy — Final Decision
|
||||
|
||||
**Date:** 17 August 2026
|
||||
**Decision:** Adopt **`fixed_size`** as the chunking Strategy family.
|
||||
**Not decided here:** Neighbor Expansion level **±N** (follow-up).
|
||||
**Date:** 17 August 2026 (family); **22 August 2026** (±N locked)
|
||||
**Decision:** Adopt **`fixed_size`** as the chunking Strategy family with **Neighbor Expansion ±3/3**.
|
||||
**Corpus Embedding Model:** `text-embedding-3-large`
|
||||
**Evaluation set:** 10 Word documents (Decision Board / neighbor-sweep universe)
|
||||
**Questions:** 201 per Candidate (nomic semantic: 195)
|
||||
|
||||
**One-line close:** **`fixed_size` wins** on official mean composite (**9.040** vs **8.727**). Pick **±N later**, but **not ±0**.
|
||||
**One-line close:** **`fixed_size ±3`** is the stabilized Strategy Candidate (**9.040** mean composite vs **8.727** for semantic @ large). **Do not use ±0** as the default — it loses to semantic on mean.
|
||||
|
||||
Charts below are generated from `data/chunking_benchmark.db` with the same composite as the Decision Board:
|
||||
|
||||
@@ -123,15 +122,38 @@ One leftover five-strategy Experiment on `customer1` under large is **invalid**
|
||||
|
||||
**Use `fixed_size` for chunking** under **`text-embedding-3-large`.**
|
||||
|
||||
**Use Neighbor Expansion `neighbor_prev=3`, `neighbor_next=3`** (symmetric **±3/3**) for Query and Experiment defaults when running `fixed_size`.
|
||||
|
||||
**Do not use `semantic` as the default Strategy** on this evaluation universe.
|
||||
|
||||
**Follow-up:** choose Neighbor Expansion **±N** among **±1, ±2, ±3**. Stage 1 auto-pick is **±3**. **±0 is not recommended** if this family decision is to remain valid.
|
||||
**Do not default to ±0.** Without expansion, semantic @ large beats `fixed_size` on mean composite (8.727 vs 8.606).
|
||||
|
||||
Production config change is **out of scope** for this record.
|
||||
Operational defaults are set in `src/core/config.py`, `.env.example`, and the Dashboard Query/Benchmarks forms. Override per request is still supported.
|
||||
|
||||
---
|
||||
|
||||
## 8. Evidence (stage 2 cells)
|
||||
## 8. Addendum — why ±3 (22 August 2026)
|
||||
|
||||
Stage 1 ranked all `fixed_size` Candidates on the 10-doc grid:
|
||||
|
||||
| ±N | Mean composite | vs semantic @ large |
|
||||
|----|----------------|---------------------|
|
||||
| ±0 | 8.606 | **loses** (−0.121) |
|
||||
| ±1 | 8.926 | wins (+0.199) |
|
||||
| ±2 | 8.987 | wins (+0.260) |
|
||||
| **±3** | **9.040** | **wins (+0.313)** |
|
||||
|
||||
**±3** is the stage 1 auto-pick and the highest mean composite. **±1** and **±2** are close; **±0** is ruled out.
|
||||
|
||||
Per-document best ±N varies (±3 wins on 5 docs, ±1 on 3, ±2 on 1, ±0 on 1). The **global** default is still **±3** because Decision Board ranks by mean composite across the full set, not by win-count among ±N levels.
|
||||
|
||||
**Caveats kept from stage 2:** semantic still wins head-to-head on 6/10 docs vs ±3 (website, customer1, etc.), but mean composite and judge metrics favor **`fixed_size ±3`**. Retrieval Inspect on outlier docs remains optional follow-up.
|
||||
|
||||
**Production RAG outside this repo** is not changed by this addendum — only benchmarker defaults and this record.
|
||||
|
||||
---
|
||||
|
||||
## 9. Evidence (stage 2 cells)
|
||||
|
||||
| Document | `fixed_size ±3` experiment id | `semantic @ large` experiment id |
|
||||
|----------|-------------------------------|----------------------------------|
|
||||
|
||||
Reference in New Issue
Block a user