docs: record embedding, neighbor, and inspect decisions

Co-authored-by: Cursor <cursoragent@cursor.com>

docs/adr/0023-neighbor-expansion-fixed-size.md

@@ -0,0 +1,46 @@
+# ADR-0023: Neighbor Expansion for fixed_size retrieval
+
+## Status
+
+Accepted (implemented)
+
+## Context
+
+Fixed-size chunks cut text at token windows, so the single highest-scoring chunk often lacks the sentence before/after the answer. Operators want query-time widening: after top-k retrieval, also pull previous and next chunks in document order. Doing this for every Strategy, or reshaping the top-k budget, would muddy Experiment comparisons and hide whether gains come from chunking vs from extra context.
+
+Operators also need to **see** the relationship: all top-k hits and, for each hit, which upper/downer chunks were appended. A flat deduped list alone cannot show per-hit windows when neighbors overlap.
+
+## Decision
+
+### Retrieval behavior
+
+- **Neighbor Expansion** applies only to the `fixed_size` Strategy at query time (not at process/chunking time).
+- Every top-k hit is expanded by configurable `neighbor_prev` and `neighbor_next` counts (same document, by `chunk_index`).
+- Expansion is **additive**: neighbors are merged with top-k, deduped by chunk id, then sorted into **document order** for the LLM. Context may exceed `top_k`.
+- Knobs live on Query and Experiment requests like `top_k`, with config defaults **`0` / `0`** (opt-in). Non-`fixed_size` Strategies ignore the knobs. Missing neighbors at document edges are skipped.
+- Experiments must record `neighbor_prev` / `neighbor_next` for provenance.
+
+### Dual response shape
+
+- **`retrieved_chunks`**: flat list matching what the LLM saw (deduped, document order; entries labeled hit vs neighbor as needed for audit/eval).
+- **`expansion_tree`**: operator-facing grouping — top-k hits in **score order**, each with `neighbors_prev` / `neighbors_next`. When expansion is off (`0`/`0`), the tree still contains the hits with **empty** neighbor arrays.
+- Overlapping windows: the same chunk may appear under **more than one** hit in the Expansion Tree; it still appears **once** in `retrieved_chunks` / the LLM prompt.
+
+### Where the Expansion Tree is shown
+
+- Query result UI
+- Experiment per-question detail in the Dashboard
+- HTML report — **both** managerial and technical views
+
+### Experiment list & Compare provenance
+
+- Experiments list shows a compact **`±P/N`** Neighbors badge (tooltip: `neighbor_prev` / `neighbor_next`; muted when `fixed_size` was not in the run).
+- Compare view repeats the badge on each Experiment column/card.
+- Compare shows a **soft warning** when selected Experiments differ on Neighbor Expansion **or** Embedding Model — intentional A/B is allowed; do not treat mismatched fixed_size scores as identical setups.
+
+## Consequences
+
+- Fixed_size Experiments with expansion enabled may use more tokens than other Strategies at the same `top_k`; that asymmetry is intentional and must be visible in config/reports.
+- Hit-only retrieval metrics remain available by filtering labeled hits in the flat list; unfiltered eval reflects the expanded prompt.
+- UI and reports render the Expansion Tree; they must not assume the flat list alone can reconstruct per-hit appendages after dedupe.
+- Dashboard Compare must keep Neighbor Expansion (and Embedding Model) visible so operators can fairly compare `±0/0` vs `±1/1` vs larger windows.