# ADR-0023: Neighbor Expansion for fixed_size retrieval

## Status

Accepted (implemented)

## Context

Fixed-size chunks cut text at token windows, so the single highest-scoring chunk often lacks the sentence before/after the answer. Operators want query-time widening: after top-k retrieval, also pull previous and next chunks in document order. Doing this for every Strategy, or reshaping the top-k budget, would muddy Experiment comparisons and hide whether gains come from chunking vs from extra context.

Operators also need to **see** the relationship: all top-k hits and, for each hit, which upper/downer chunks were appended. A flat deduped list alone cannot show per-hit windows when neighbors overlap.

## Decision

### Retrieval behavior

- **Neighbor Expansion** applies only to the `fixed_size` Strategy at query time (not at process/chunking time).
- Every top-k hit is expanded by configurable `neighbor_prev` and `neighbor_next` counts (same document, by `chunk_index`).
- Expansion is **additive**: neighbors are merged with top-k, deduped by chunk id, then sorted into **document order** for the LLM. Context may exceed `top_k`.
- Knobs live on Query and Experiment requests like `top_k`, with config defaults **`0` / `0`** (opt-in). Non-`fixed_size` Strategies ignore the knobs. Missing neighbors at document edges are skipped.
- Experiments must record `neighbor_prev` / `neighbor_next` for provenance.

### Dual response shape

- **`retrieved_chunks`**: flat list matching what the LLM saw (deduped, document order; entries labeled hit vs neighbor as needed for audit/eval).
- **`expansion_tree`**: operator-facing grouping — top-k hits in **score order**, each with `neighbors_prev` / `neighbors_next`. When expansion is off (`0`/`0`), the tree still contains the hits with **empty** neighbor arrays.
- Overlapping windows: the same chunk may appear under **more than one** hit in the Expansion Tree; it still appears **once** in `retrieved_chunks` / the LLM prompt.

### Where the Expansion Tree is shown

- Query result UI
- Experiment per-question detail in the Dashboard
- HTML report — **both** managerial and technical views

### Experiment list & Compare provenance

- Experiments list shows a compact **`±P/N`** Neighbors badge (tooltip: `neighbor_prev` / `neighbor_next`; muted when `fixed_size` was not in the run).
- Compare view repeats the badge on each Experiment column/card.
- Compare shows a **soft warning** when selected Experiments differ on Neighbor Expansion **or** Embedding Model — intentional A/B is allowed; do not treat mismatched fixed_size scores as identical setups.

## Consequences

- Fixed_size Experiments with expansion enabled may use more tokens than other Strategies at the same `top_k`; that asymmetry is intentional and must be visible in config/reports.
- Hit-only retrieval metrics remain available by filtering labeled hits in the flat list; unfiltered eval reflects the expanded prompt.
- UI and reports render the Expansion Tree; they must not assume the flat list alone can reconstruct per-hit appendages after dedupe.
- Dashboard Compare must keep Neighbor Expansion (and Embedding Model) visible so operators can fairly compare `±0/0` vs `±1/1` vs larger windows.