Files
chunking_strategies_evaluation/docs/adr/0025-retrieval-inspect.md

1.9 KiB

ADR-0025: Retrieval Inspect (Benchmarks full-page mode)

Status

Accepted (implemented; Expected Answer strip amended)

Context

Neighbor Expansion and Expansion Tree (ADR-0023) store per-question top-k hits with prev/next for Experiments, but operators cannot audit them well: Query is live-only, Experiment expand rows and HTML reports truncate or sample one question, and a new top-level Tab would stretch navigation. Operators need a dedicated, readable surface to choose a question and inspect retrieval in full detail. Operators also need the Expected Answer beside the Generated Answer to judge Similarity without leaving Inspect.

Decision

  • Retrieval Inspect is a full-page mode inside the Benchmarks Tab (not a new top-level Tab, not HTML-report-only).
  • Entry: Inspect action on an Experiment row (alongside Report).
  • Layout: left question rail (status for error / has Expansion Tree) + main pane with Strategy picker (default fixed_size when present), side-by-side Generated | Expected Answer strip with clear column headers (plain text, no word-diff), full-width eval scores under that strip, and a vertical full-text Expansion Tree (prev → hit → next per top-k hit; neighbors empty/N/A for other Strategies).
  • Expected column always renders; empty/missing expected_answer shows a muted “(no expected answer)”.
  • Visual language stays the existing Dashboard theme (ADR-0008); quality comes from hierarchy, spacing, and full text — not a separate brand.
  • Reads existing Experiment per_question payloads (expansion_tree, answers, expected_answer, scores); no new retrieval API required for v1.

Consequences

  • Benchmarks Tab gains a second view mode (list/run vs Inspect); state returns cleanly via Back.
  • HTML report Expansion Tree remains a summary; Inspect is the audit UX.
  • Earlier “no expected-answer compare panel” clause is superseded by the labeled side-by-side strip (not a diff engine).