chunking_strategies_evaluation/docs/adr/0026-decision-board-tab.md

ADR-0026: Decision Board Tab for final Strategy selection

Status

Accepted (implemented)

Context

After Neighbor Expansion Sweeps and semantic Boundary variants, the operator’s real job is choosing between fixed_size and semantic — not ad-hoc Experiment Compare. Compare remains useful for arbitrary side-by-side diffs, but it does not encode two-stage tuning (best variant per family, then family showdown) or a fixed 10-doc evaluation universe.

Decision

• Add a top-level Dashboard Decision Tab (Decision Board).
• Strategy Candidates: fixed_size at ±0…±3, and semantic under each Boundary Embedding Model present in data, under a selected Corpus Embedding Model.
• Auto-discover from single-strategy Experiments on the hardcoded 10-doc set; newest Experiment fills each doc×Candidate cell; optional exclude chips; incomplete cells are — and omitted from that Candidate’s mean/wins.
• Stage 1 ranks by mean composite (same weights as Experiments “Best”), shows win-count, allows manual override.
• Stage 2 duels the two family winners with full metrics + per-doc head-to-head; per-doc matrix lists all Candidates with duel winners highlighted.
• Aggregation is client-side from GET /experiments?limit=… (default list limit raised; max 500).

Considered options

• Enhance Compare only — rejected (wrong job, crowded Benchmarks UX).
• Standalone HTML report only — rejected (weak for override / interactive decision).
• Manual pin of every cell — rejected for v1 (too heavy for ~60 cells).