chore(scripts): add neighbor expansion benchmark sweep

Co-authored-by: Cursor <cursoragent@cursor.com>

.env.example

@@ -5,12 +5,18 @@ OPENAI_API_KEY=sk-...
 QDRANT_URL=http://localhost:6333
 QDRANT_API_KEY=
 
-# Fixed models (not configurable per strategy)
+# Default cloud Embedding Model (registry id / OpenAI model name)
-EMBEDDING_MODEL=text-embedding-3-small
+EMBEDDING_MODEL=text-embedding-3-large
 LLM_MODEL=gpt-4o-mini
 
+# Local Ollama embeddings (Active model switched in Admin)
+OLLAMA_BASE_URL=http://192.168.10.10:11435
+
 # Retrieval defaults
 TOP_K=5
+# Neighbor Expansion for fixed_size only (ADR-0023); 0 = off
+NEIGHBOR_PREV=0
+NEIGHBOR_NEXT=0
 
 # LLM generation parameters
 TEMPERATURE=0.0

CONTEXT.md

@@ -7,11 +7,15 @@ A single-file React app (CDN-loaded, no build step) served by FastAPI at `/app`.
 _Avoid_: Admin panel, web UI, frontend
 
 **Strategy**:
-One of the five chunking algorithms: fixed_size, recursive, semantic, contextual_retrieval, semantic_parent_child.
+One of the five chunking algorithms: fixed_size, recursive, semantic, contextual_retrieval, semantic_parent_child. Semantic and semantic_parent_child determine chunk boundaries from meaning (adjacent-unit similarity), not from fixed sentence/paragraph counts.
 _Avoid_: Method, approach, technique
 
+**Semantic Boundary Detection**:
+Using embeddings of consecutive units (sentences for semantic — Farsi-aware punctuation, with line/paragraph fallback when punctuation yields a single unit; paragraphs for semantic_parent_child) and cosine similarity against a threshold tied to the Boundary Embedding Model to decide where chunks start and end. Distinct from Corpus Embedding (finished-chunk vectors for Qdrant). If boundary embeddings are missing or mismatched, that Strategy fails — there is no fixed-count fallback.
+_Avoid_: Semantic embedding, chunk embedding (ambiguous — that often means storage vectors)
+
 **Experiment**:
-A completed benchmark run — one document, N strategies, M questions, with per-question and aggregate metrics.
+A completed benchmark run — one document, N strategies, M questions, with per-question and aggregate metrics. Always records Corpus Embedding Model and Neighbor Expansion knobs. Records Boundary Embedding Model when semantic Strategies were part of processing/evaluation path for that corpus. Runs under different Boundary/Corpus settings must not be silently compared.
 _Avoid_: Run, trial, benchmark
 
 **Chunk Preview**:
@@ -22,6 +26,14 @@ _Avoid_: Chunk inspection, chunk view
 A persistent top-level navigation section of the Dashboard (Home, Documents, Query, Benchmarks, Admin, PDF). Tabs stay mounted when switching — state survives.
 _Avoid_: Page, route, view
 
+**Retrieval Inspect**:
+A full-page mode inside the Benchmarks Tab for auditing one Experiment’s retrieval: left rail lists questions (status for error / has Expansion Tree); main pane has Strategy picker (defaults to `fixed_size` when present), a side-by-side Generated Answer | Expected Answer strip with eval scores, and a vertical full-text Expansion Tree (previous → hit → next per top-k hit; neighbors empty/N/A for other Strategies). Opened from an Experiment; not a separate top-level Tab. Distinct from live Query and from truncated HTML report Expansion Tree blocks.
+_Avoid_: Query tab, chunk preview, report tree, expansion sample
+
+**Expected Answer**:
+The reference answer for a benchmark question, supplied with the question set and used by evaluation (e.g. Answer Similarity). Shown in Retrieval Inspect beside the Generated Answer. Informal synonym “golden answer” is not product language.
+_Avoid_: Golden answer, ground truth, reference answer (prefer Expected Answer in UI and docs)
+
 **PDF Workspace**:
 The PDF Tab’s end-to-end surface for Text PDF work: upload, process Strategies, Chunk Preview, query, run Experiments, and open reports — scoped to PDF documents only. Documents / Query / Benchmarks Tabs list Word documents only (`.doc`/`.docx`). Inside the Tab, features are collapsible accordion sections (Upload & Process open by default); Admin-only ops (health, Qdrant CRUD) stay on Admin. Built from shared Dashboard section components with a format filter — not a forked UI.
 _Avoid_: PDF page, PDF mode, PDF dashboard
@@ -46,6 +58,46 @@ _Avoid_: OCR check, PDF validation, empty-page filter
 Turning table cells into sequential plain-text blocks in reading order (same contract for DOCX and Text PDF). Strategies never receive grid/markdown-table structure in v1.
 _Avoid_: Table extraction, table parsing, structured tables
 
+**Embedding Model**:
+A registered model identity (Provider, dimension, optional task prefixes, default `semantic_threshold`). Used in one of two roles: Boundary or Corpus. Not an Experiment axis by itself — Strategies are compared under recorded Boundary+Corpus snapshots per Experiment.
+_Avoid_: Embedder, embedding provider (use Provider for that), vector model
+
+**Active Embedding Model**:
+Legacy single global role (ADR-0019). Replaced by two Admin defaults — Default Boundary Embedding Model and Default Corpus Embedding Model — with per-operation overrides. Prefer Boundary / Corpus terms for new work.
+_Avoid_: Current model, selected embedder, runtime model
+
+**Boundary Embedding Model**:
+The Embedding Model used only for Semantic Boundary Detection (`semantic` / `semantic_parent_child`). Does not write Qdrant vectors. Ignored when a Strategy has no boundary step. Admin stores a default; Process may override per run. Snapshotted at process start. Cuts always use this model’s `semantic_threshold` (never the Corpus model’s threshold when the two differ).
+_Avoid_: Semantic embedder, chunking model, similarity model
+
+**Corpus Embedding Model**:
+The Embedding Model used to embed finished chunks into the Model Corpus and to embed queries for search. Storage and query always share this model. Determines which Model Corpus collections are read/written. Admin stores a default; Process, Query, and Experiment may override per run. Snapshotted for that operation.
+_Avoid_: Storage model, retrieval model, Qdrant model, Active Embedding Model
+
+**Provider**:
+Where an Embedding Model runs: Cloud (e.g. OpenAI API) or Local (Ollama on the host). A Provider is a delivery channel, not a model identity.
+_Avoid_: Backend, engine, embedding source
+
+**Model Corpus**:
+The Strategy collections whose vectors were produced by one Corpus Embedding Model. Collection names always include that model’s id (`{strategy}__{model_id}_collection`). Process, query, and Experiment only read/write the Model Corpus of the Corpus Embedding Model in force — other corpora remain untouched.
+_Avoid_: Collection set, vector partition, embedding space
+
+**Embedding Model Registry**:
+The configured catalog of Embedding Models the operator may assign to Boundary or Corpus roles — each entry has a stable identity, Provider, vector dimension, and a default `semantic_threshold` (Admin may override per model in SQLite). Defaults: OpenAI 0.3, Nomic 0.6. Unlisted models are out of scope until added to the registry.
+_Avoid_: Model list, provider catalog, available embeddings
+
+**Neighbor Expansion**:
+Query-time widening of retrieved context for the `fixed_size` Strategy only: for every top-k hit, also include a configurable number of previous and next chunks in document order within the same document. Neighbors are added on top of the top-k set (context may grow beyond k). The same chunk id appears at most once in the LLM context. Final LLM context is sorted by document order (`chunk_index`). Neighbors are not re-ranked as independent hits. Counts (`neighbor_prev` / `neighbor_next`) are set per Query or Experiment like `top_k`; defaults are `0`/`0` (off). Missing neighbors at document edges are skipped. Knobs are ignored for non-`fixed_size` Strategies. Dashboard labels Experiments with a compact `±P/N` badge (tooltip explains prev/next; muted when `fixed_size` was not in the run). Experiments list and Compare show these knobs; Compare warns when selected Experiments differ on Neighbor Expansion, Corpus Embedding Model, or Boundary Embedding Model (when recorded) so operators do not misread cross-run rankings. Distinct from how a Strategy cuts text at process time.
+_Avoid_: Context windowing, chunk padding, sliding window retrieval, adjacent chunk merge
+
+**Expansion Tree**:
+The operator-facing grouping of Neighbor Expansion results, returned alongside the flat `retrieved_chunks` list: top-k hits in score order, each listing previous and next chunks appended for that hit (overlaps may appear under more than one hit). When expansion is off, the tree still lists the hits with empty neighbor arrays. Shown in the Query result UI, Experiment per-question detail, and both managerial and technical HTML report views. Distinct from the flat, deduped, document-ordered context sent to the LLM (each chunk id appears only once there).
+_Avoid_: Retrieved chunks list, neighbor list, expansion map
+
+**Benchmark Sweep**:
+An ordered batch of Experiments that hold Strategy and Corpus Embedding Model fixed while stepping Neighbor Expansion through symmetric levels (±0, ±1, ±2, ±3) — i.e. `(neighbor_prev, neighbor_next)` = `(0,0)`, `(1,1)`, `(2,2)`, `(3,3)` — across one or more documents. Each level is its own Experiment; the Sweep is the sequence, not a single stored row.
+_Avoid_: Batch run, sequential benchmark, benchmark script, neighbor matrix
+
 ## Architecture Decisions
 
 | # | Decision | Status |
@@ -66,3 +118,11 @@ _Avoid_: Table extraction, table parsing, structured tables
  ADR-0014 | Cost Estimator: two number inputs (questions, strategies), Estimate button, result card with total cost in amber, token estimate, 3 breakdown cards (embedding/queries/evaluation). Simple numbers, no tables. | Approved |
  ADR-0016 | Text PDF extraction via PyMuPDF + Heading Reconstruction; Scanned PDFs hard-rejected (Text-layer Gate). See docs/adr/0016-*.md | Approved |
  ADR-0017 | PDF Workspace via shared Dashboard sections + format filter (not a forked UI). See docs/adr/0017-*.md | Approved |
+ ADR-0018 | Model Corpus via model-scoped collections; legacy unscoped → default cloud corpus. See docs/adr/0018-*.md | Superseded by ADR-0021 (naming) |
+ ADR-0019 | Active Embedding Model is global (Admin + registry), not an Experiment axis; provenance required. See docs/adr/0019-*.md | Superseded by ADR-0024 |
+ ADR-0020 | Semantic strategies require Semantic Boundary Detection (fail hard, no fixed-count fallback). See docs/adr/0020-*.md | Approved |
+ ADR-0021 | Always scope Qdrant collection names by Embedding Model id (cloud and local). See docs/adr/0021-*.md | Approved |
+ ADR-0022 | Per-Embedding-Model `semantic_threshold` (Admin override; defaults OpenAI 0.3 / Nomic 0.6). See docs/adr/0022-*.md | Approved |
+ ADR-0023 | Neighbor Expansion for fixed_size (+ Expansion Tree; list/Compare `±P/N` provenance + mismatch warning). See docs/adr/0023-*.md | Approved |
+ ADR-0024 | Boundary vs Corpus Embedding Model roles (Admin defaults + per-op overrides; query locked to Corpus). See docs/adr/0024-*.md | Implemented |
+ ADR-0025 | Retrieval Inspect: Benchmarks full-page mode (question rail + full-text Expansion Tree). See docs/adr/0025-*.md | Approved |

docs/adr/0018-model-corpus-scoped-collections.md

@@ -0,0 +1,17 @@
+# Model Corpus via model-scoped collections
+
+Vectors for Cloud and Local Embedding Models must not share a Qdrant collection: dimensions and embedding spaces differ. Each Embedding Model therefore owns a **Model Corpus** — Strategy collections whose identity includes that model. Process, query, and Experiment only read/write the Active Embedding Model’s corpus; other corpora stay untouched. Pre-existing unscoped collections (e.g. `fixed_size_collection`) are migrated into the default cloud Embedding Model’s corpus so existing OpenAI work is not discarded.
+
+## Considered Options
+
+- **Model-scoped collections** — chosen; non-destructive switch between Providers/models; clear isolation
+- **Wipe-and-rebuild on switch** — simple naming, destructive and easy to forget
+- **Dimension-gated single set** — only allow same-dimension swaps; blocks most OpenAI ↔ Ollama use
+
+Legacy handling (original): **migrate unscoped → default cloud corpus**. **Superseded for naming by ADR-0021** — all models, including default cloud, use `{strategy}__{model_id}_collection`; delete leftover unscoped collections when rebuilding.
+
+## Consequences
+
+- Collection naming and vector size are driven by the Embedding Model Registry entry, not a global `1536` constant
+- Admin Qdrant list shows all corpora, labeled by Embedding Model, with the active corpus emphasized; wipe stays per named collection
+- Switching Active Embedding Model does not delete the other Model Corpus; operator must process documents again under the new model to populate it

docs/adr/0019-active-embedding-model-global.md

@@ -0,0 +1,22 @@
+# Active Embedding Model is global, not an Experiment axis
+
+**Status:** Superseded by [ADR-0024](0024-boundary-vs-corpus-embedding-models.md) — single Active Embedding Model replaced by **Boundary** + **Corpus** roles (query locked to Corpus).
+
+Original decision retained for history:
+
+The platform compares **Strategies** under controlled conditions. Embedding Model is a confounder, not a second experiment dimension: one **Active Embedding Model** applies to process, query, and new Experiments. The operator switches it from Admin among entries in a static **Embedding Model Registry** (Provider, stable id, vector dimension). Selection persists across restarts (config supplies the default only when unset). Every Experiment records which Embedding Model produced it; historical rows without provenance are treated as the default cloud model. A process, query, or Experiment **snapshots** the Active Embedding Model at start so a mid-flight Admin switch cannot mix models inside one operation.
+
+## Considered Options
+
+- **Global Active Embedding Model** — chosen; fair Strategy comparisons; A/B models via separate Experiments
+- **Embedding Model as Experiment axis** — Strategy × model cross-product; richer science, much heavier data model/UI
+- **Per-run choice with no Experiment coupling** — flexible, invites silent unfair comparisons
+
+Switcher: **Admin UI** (not env-only, not read-only status). Catalog: **static registry** (not live Ollama discovery, not free-form). Local Provider host: **`OLLAMA_BASE_URL` in config only**. Cost Estimator: **$0 embedding line when Active Embedding Model is Local**; LLM costs unchanged. This carve-out does not reopen general `/admin/config` (ADR-0007 deferred).
+
+## Consequences
+
+- Introduce Provider adapters (Cloud OpenAI, Local Ollama) behind one embed API; callers bind a registry entry for the operation
+- Dashboard Admin gains an Embedding Model switcher; Benchmarks/reports must show Experiment provenance
+- `docs/out-of-scope-v1.md` “no embedding model from dashboard” is superseded for this focused control only
+- Local Nomic registry entries use task prefixes (`search_document` / `search_query`); Cloud OpenAI entries do not

docs/adr/0020-semantic-boundary-detection-required.md

@@ -0,0 +1,16 @@
+# Semantic strategies require Semantic Boundary Detection
+
+`semantic` and `semantic_parent_child` must cut chunks from meaning: the orchestrator embeds consecutive units (sentences / paragraphs) with the Active Embedding Model snapshot, passes those vectors into `chunk()`, then embeds finished chunks for Qdrant. Fixed-count fallbacks are removed — missing or length-mismatched boundary embeddings fail that Strategy. This completes ADR 0012’s sentence-level design in the process path (and the parent/child analogue) so benchmarks cannot silently measure “every N units” under a semantic name.
+
+## Considered Options
+
+- **Wire both strategies + fail hard** — chosen; same Active model for boundaries and storage; accept double embed cost; re-process to replace old fake-semantic vectors
+- **Wire `semantic` only** — rejected; would leave parent/child on the same lie
+- **Keep fallback with warnings** — rejected; that is how the bug stayed hidden
+- **Always OpenAI for boundaries regardless of Active** — rejected; confounds model A/B experiments
+
+## Consequences
+
+- Process cost rises for these two Strategies (unit embeds + chunk embeds)
+- Prior Experiments / collections produced under the fallback are not true semantic — operator must re-process and re-benchmark
+- Strategy modules raise `ChunkingError` if boundary embeddings are absent or mismatched; orchestration is responsible for supplying them

docs/adr/0021-always-scope-collection-names-by-model.md

@@ -0,0 +1,14 @@
+# Always scope Qdrant collection names by Embedding Model id
+
+Every Model Corpus collection is named `{strategy}__{embedding_model_id}_collection`, including the default cloud model (`text-embedding-3-small`). This supersedes ADR-0018’s exception that left OpenAI on legacy unscoped names (`fixed_size_collection`). Uniform naming makes Admin labeling obvious for both Cloud and Local and avoids a special case that confused operators wiping/rebuilding corpora.
+
+## Considered Options
+
+- **Always include model id** — chosen; clear for cloud and local
+- **Keep legacy unscoped names for default cloud only** — ADR-0018; rejected going forward after a from-scratch rebuild
+
+## Consequences
+
+- Process/query create names like `fixed_size__text-embedding-3-small_collection` and `fixed_size__nomic-embed-text-v2-moe_collection`
+- Old unscoped collections are not written anymore; Admin still labels them as legacy default-cloud if they remain until deleted
+- Operators wiping for a clean slate should delete legacy collections before re-processing

docs/adr/0022-per-embedding-model-semantic-threshold.md

@@ -0,0 +1,22 @@
+# ADR-0022: Per-Embedding-Model semantic_threshold
+
+## Status
+
+Approved
+
+## Context
+
+Semantic Boundary Detection cuts when adjacent-unit cosine similarity falls below a threshold. Cloud OpenAI and local Nomic produce different similarity distributions for the same Farsi text: the global `SEMANTIC_THRESHOLD` (config, historically ~0.3) rarely triggers cuts under Nomic, collapsing `semantic` into a single chunk. Operators need a higher Nomic default without changing OpenAI behavior, and a way to tune without editing `.env` and restarting.
+
+## Decision
+
+- Each Embedding Model Registry entry has a `default_semantic_threshold` (OpenAI `text-embedding-3-small`: **0.3**; Nomic `nomic-embed-text-v2-moe`: **0.6`).
+- Admin may override the effective value per model id in SQLite (`semantic_threshold:{model_id}`).
+- Process snapshots the Active Embedding Model and uses that model’s effective threshold for `semantic` and `semantic_parent_child`.
+- Admin API: list includes `semantic_threshold` / `default_semantic_threshold`; `PUT /admin/embedding-models/{id}/semantic-threshold` persists overrides.
+- Changing the threshold does not rewrite existing corpora — re-process to apply.
+
+## Consequences
+
+- Global `SEMANTIC_THRESHOLD` remains a fallback only when a strategy is called without an explicit threshold (tests/legacy); production process path always passes the model-resolved value.
+- Operators must re-process after tuning; Experiments under different thresholds are not auto-invalidated.

docs/adr/0023-neighbor-expansion-fixed-size.md

@@ -0,0 +1,46 @@
+# ADR-0023: Neighbor Expansion for fixed_size retrieval
+
+## Status
+
+Accepted (implemented)
+
+## Context
+
+Fixed-size chunks cut text at token windows, so the single highest-scoring chunk often lacks the sentence before/after the answer. Operators want query-time widening: after top-k retrieval, also pull previous and next chunks in document order. Doing this for every Strategy, or reshaping the top-k budget, would muddy Experiment comparisons and hide whether gains come from chunking vs from extra context.
+
+Operators also need to **see** the relationship: all top-k hits and, for each hit, which upper/downer chunks were appended. A flat deduped list alone cannot show per-hit windows when neighbors overlap.
+
+## Decision
+
+### Retrieval behavior
+
+- **Neighbor Expansion** applies only to the `fixed_size` Strategy at query time (not at process/chunking time).
+- Every top-k hit is expanded by configurable `neighbor_prev` and `neighbor_next` counts (same document, by `chunk_index`).
+- Expansion is **additive**: neighbors are merged with top-k, deduped by chunk id, then sorted into **document order** for the LLM. Context may exceed `top_k`.
+- Knobs live on Query and Experiment requests like `top_k`, with config defaults **`0` / `0`** (opt-in). Non-`fixed_size` Strategies ignore the knobs. Missing neighbors at document edges are skipped.
+- Experiments must record `neighbor_prev` / `neighbor_next` for provenance.
+
+### Dual response shape
+
+- **`retrieved_chunks`**: flat list matching what the LLM saw (deduped, document order; entries labeled hit vs neighbor as needed for audit/eval).
+- **`expansion_tree`**: operator-facing grouping — top-k hits in **score order**, each with `neighbors_prev` / `neighbors_next`. When expansion is off (`0`/`0`), the tree still contains the hits with **empty** neighbor arrays.
+- Overlapping windows: the same chunk may appear under **more than one** hit in the Expansion Tree; it still appears **once** in `retrieved_chunks` / the LLM prompt.
+
+### Where the Expansion Tree is shown
+
+- Query result UI
+- Experiment per-question detail in the Dashboard
+- HTML report — **both** managerial and technical views
+
+### Experiment list & Compare provenance
+
+- Experiments list shows a compact **`±P/N`** Neighbors badge (tooltip: `neighbor_prev` / `neighbor_next`; muted when `fixed_size` was not in the run).
+- Compare view repeats the badge on each Experiment column/card.
+- Compare shows a **soft warning** when selected Experiments differ on Neighbor Expansion **or** Embedding Model — intentional A/B is allowed; do not treat mismatched fixed_size scores as identical setups.
+
+## Consequences
+
+- Fixed_size Experiments with expansion enabled may use more tokens than other Strategies at the same `top_k`; that asymmetry is intentional and must be visible in config/reports.
+- Hit-only retrieval metrics remain available by filtering labeled hits in the flat list; unfiltered eval reflects the expanded prompt.
+- UI and reports render the Expansion Tree; they must not assume the flat list alone can reconstruct per-hit appendages after dedupe.
+- Dashboard Compare must keep Neighbor Expansion (and Embedding Model) visible so operators can fairly compare `±0/0` vs `±1/1` vs larger windows.

docs/adr/0024-boundary-vs-corpus-embedding-models.md

@@ -0,0 +1,27 @@
+# ADR-0024: Boundary vs Corpus Embedding Model roles
+
+## Status
+
+Implemented. Supersedes ADR-0019’s single global Active Embedding Model for new work.
+
+## Context
+
+Embedding happens in three moments: (1) Semantic Boundary Detection for `semantic` / `semantic_parent_child`, (2) finished-chunk vectors into Qdrant, (3) query vectors for search. ADR-0019 forced one Active Embedding Model for all three so Strategy comparisons stayed fair. Operators now want to vary the model used for cuts independently from the model used for retrieval — but storage and query must remain the same vector space or RAG breaks.
+
+## Decision
+
+- Split into two roles from the Embedding Model Registry:
+  - **Boundary Embedding Model** — Cosine cuts only; ignored by Strategies without boundary detection. Uses that model’s `semantic_threshold`.
+  - **Corpus Embedding Model** — Finished-chunk storage **and** query embedding (always paired). Selects the Model Corpus (`{strategy}__{corpus_model_id}_collection`).
+- **Not** three independent knobs: Query may not diverge from Corpus.
+- Admin stores **Default Boundary** and **Default Corpus** (migrate legacy `active_embedding_model_id` → both defaults).
+- Per-operation overrides (Decision B): Process may set Boundary + Corpus; Query/Experiment set Corpus only (Boundary irrelevant at query time). Snapshot for the operation.
+- Process UI: show Boundary picker only when a semantic Strategy is selected.
+- Provenance: always record Corpus on Query/Experiment; record Boundary when semantic Strategies are in play. Compare warns when Corpus, Boundary (if present), or Neighbors differ.
+- Query/Benchmark Corpus choice should prefer models that have a usable Model Corpus for the target doc/strategy and warn when empty.
+
+## Consequences
+
+- ADR-0019’s single Active switcher is replaced; call sites that `snapshot_active_model()` must become Corpus (and Boundary where needed).
+- Cross-model science is allowed for boundary vs corpus; silent unfair Strategy×model mixes remain out of scope as an Experiment axis.
+- Implementation is live: Admin defaults, Process Boundary+Corpus overrides, Query/Experiment Corpus override, dual provenance, Compare warnings.

docs/adr/0025-retrieval-inspect.md

@@ -0,0 +1,24 @@
+# ADR-0025: Retrieval Inspect (Benchmarks full-page mode)
+
+## Status
+
+Accepted (implemented; Expected Answer strip amended)
+
+## Context
+
+Neighbor Expansion and Expansion Tree (ADR-0023) store per-question top-k hits with prev/next for Experiments, but operators cannot audit them well: Query is live-only, Experiment expand rows and HTML reports truncate or sample one question, and a new top-level Tab would stretch navigation. Operators need a dedicated, readable surface to choose a question and inspect retrieval in full detail. Operators also need the Expected Answer beside the Generated Answer to judge Similarity without leaving Inspect.
+
+## Decision
+
+- **Retrieval Inspect** is a **full-page mode inside the Benchmarks Tab** (not a new top-level Tab, not HTML-report-only).
+- Entry: **Inspect** action on an Experiment row (alongside Report).
+- Layout: **left question rail** (status for error / has Expansion Tree) + main pane with **Strategy picker** (default `fixed_size` when present), **side-by-side Generated | Expected Answer** strip with clear column headers (plain text, no word-diff), **full-width eval scores** under that strip, and a **vertical full-text Expansion Tree** (prev → hit → next per top-k hit; neighbors empty/N/A for other Strategies).
+- Expected column always renders; empty/missing `expected_answer` shows a muted “(no expected answer)”.
+- Visual language stays the existing Dashboard theme (ADR-0008); quality comes from hierarchy, spacing, and full text — not a separate brand.
+- Reads existing Experiment `per_question` payloads (`expansion_tree`, answers, `expected_answer`, scores); no new retrieval API required for v1.
+
+## Consequences
+
+- Benchmarks Tab gains a second view mode (list/run vs Inspect); state returns cleanly via Back.
+- HTML report Expansion Tree remains a summary; Inspect is the audit UX.
+- Earlier “no expected-answer compare panel” clause is superseded by the labeled side-by-side strip (not a diff engine).

docs/out-of-scope-v1.md

@@ -12,7 +12,9 @@ No built-in export of experiment results, query logs, or benchmark comparisons t
 
 ## Custom Chunking Parameters from the UI
 
-You cannot change `chunk_size`, `chunk_overlap`, `semantic_threshold`, or `semantic_min_chunk_size` from the dashboard. These are hardcoded in `src/core/config.py` with defaults (512 tokens, 50 overlap, 0.3 threshold). Changing them requires editing the config and restarting the server. The UI uses whatever the server has configured.
+You cannot change `chunk_size`, `chunk_overlap`, or `semantic_min_chunk_size` from the dashboard. Those remain in `src/core/config.py` (defaults: 512 tokens, 50 overlap, min chunk size from config). Changing them requires editing the config and restarting the server.
+
+**Exception:** per-Embedding-Model `semantic_threshold` is Admin-configurable (registry defaults OpenAI 0.3 / Nomic 0.6; overrides in SQLite). Re-process after changing it. Other Strategy knobs stay out of the UI.
 
 ## Streaming Responses
 
@@ -43,4 +45,4 @@ No full-text search or filtering within the dashboard. You cannot search for a s
 
 ## Configuration Management from the UI
 
-You cannot view or edit the server configuration (OpenAI model, temperature, max_tokens, embedding model, Qdrant URL, database path) from the dashboard. The `/admin/config` endpoint was deferred to V1.1. All configuration is managed via the `.env` file and `src/core/config.py`.
+You cannot view or edit arbitrary server configuration (LLM model, temperature, max_tokens, Qdrant URL, database path, Ollama base URL) from the dashboard. The general `/admin/config` endpoint was deferred to V1.1. **Exception (ADR-0019):** Active Embedding Model may be switched in Admin among entries in the static Embedding Model Registry. Ollama host URL remains env-only (`OLLAMA_BASE_URL`).

scripts/run_neighbor_sweep.py

@@ -0,0 +1,374 @@
+#!/usr/bin/env python3
+"""Run a Benchmark Sweep: fixed_size × text-embedding-3-large × Neighbor ±0..±3.
+
+Calls the running FastAPI app over HTTP (same path as the Dashboard).
+Doc-major order; gap-fills Process (fixed_size only) when SQLite provenance
+says the Model Corpus is not ready; retries each unit twice on failure.
+
+Usage:
+  .venv/bin/python scripts/run_neighbor_sweep.py
+  .venv/bin/python scripts/run_neighbor_sweep.py --base-url http://127.0.0.1:8000
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from dataclasses import dataclass, field
+from typing import Any
+
+import httpx
+
+CORPUS_MODEL_ID = "text-embedding-3-large"
+STRATEGY = "fixed_size"
+TOP_K = 5
+NEIGHBOR_LEVELS = (0, 1, 2, 3)
+MAX_ATTEMPTS = 3  # 1 try + 2 retries
+DEFAULT_BASE_URL = "http://127.0.0.1:8000"
+# Benchmarks with many questions can take a long time (LLM + eval per Q).
+REQUEST_TIMEOUT_S = 60 * 60 * 3  # 3 hours
+
+# App-truth document ↔ questions map (filenames as stored in the app).
+SWEEP_PAIRS: list[tuple[str, str]] = [
+    ("bazresi.docx", "files/bazresi.json"),
+    ("customer1.docx", "files/customer1.json"),
+    ("fire.docx", "files/fire.json"),
+    ("general-havades-individuals.doc", "files/general-havades-individuals.json"),
+    ("havades.docx", "files/havades.json"),
+    ("life-time-individual.docx", "files/life-time-individual.json"),
+    ("moavenin.docx", "files/moavenin.json"),
+    ("Refah.docx", "files/refah.json"),
+    ("website.docx", "files/website.json"),
+    ("lifetime-compensation.docx", "files/lifetime-compensation.json"),
+]
+
+
+@dataclass
+class UnitResult:
+    document: str
+    neighbor: int
+    kind: str  # "process" | "experiment"
+    ok: bool
+    detail: str
+    experiment_id: str | None = None
+    attempts: int = 1
+
+
+@dataclass
+class SweepState:
+    results: list[UnitResult] = field(default_factory=list)
+
+    @property
+    def failed(self) -> list[UnitResult]:
+        return [r for r in self.results if not r.ok]
+
+
+def log(msg: str) -> None:
+    print(msg, flush=True)
+
+
+def is_ready(doc: dict[str, Any]) -> bool:
+    corpus = doc.get("last_corpus_embedding_model_id")
+    counts = doc.get("chunk_counts") or {}
+    fixed = int(counts.get("fixed_size") or 0)
+    return corpus == CORPUS_MODEL_ID and fixed > 0
+
+
+def fetch_documents(client: httpx.Client) -> dict[str, dict[str, Any]]:
+    """Return filename → document dict (paginated)."""
+    by_name: dict[str, dict[str, Any]] = {}
+    offset = 0
+    limit = 100
+    while True:
+        r = client.get("/documents", params={"offset": offset, "limit": limit})
+        r.raise_for_status()
+        data = r.json()
+        items = data.get("items") or []
+        for doc in items:
+            by_name[doc["filename"]] = doc
+        total = int(data.get("total") or 0)
+        offset += len(items)
+        if offset >= total or not items:
+            break
+    return by_name
+
+
+def error_detail(exc: BaseException) -> str:
+    if isinstance(exc, httpx.HTTPStatusError):
+        body = ""
+        try:
+            body = exc.response.text[:500]
+        except Exception:
+            pass
+        return f"HTTP {exc.response.status_code}: {body or exc}"
+    return str(exc)
+
+
+def with_retries(
+    label: str,
+    fn,
+    *,
+    state: SweepState,
+    document: str,
+    neighbor: int,
+    kind: str,
+) -> Any | None:
+    last_err = ""
+    for attempt in range(1, MAX_ATTEMPTS + 1):
+        try:
+            log(f"  [{kind}] {label} (attempt {attempt}/{MAX_ATTEMPTS})")
+            value = fn()
+            return value
+        except Exception as exc:
+            last_err = error_detail(exc)
+            log(f"  ! failed attempt {attempt}/{MAX_ATTEMPTS}: {last_err}")
+            if attempt < MAX_ATTEMPTS:
+                time.sleep(min(5 * attempt, 15))
+    state.results.append(
+        UnitResult(
+            document=document,
+            neighbor=neighbor,
+            kind=kind,
+            ok=False,
+            detail=last_err,
+            attempts=MAX_ATTEMPTS,
+        )
+    )
+    return None
+
+
+def ensure_corpus(
+    client: httpx.Client,
+    doc: dict[str, Any],
+    *,
+    state: SweepState,
+) -> dict[str, Any] | None:
+    """Process fixed_size under CORPUS_MODEL_ID if not ready. Returns updated doc or None."""
+    filename = doc["filename"]
+    if is_ready(doc):
+        log(f"  corpus ready ({CORPUS_MODEL_ID}, fixed_size={doc['chunk_counts'].get('fixed_size')})")
+        return doc
+
+    log(
+        f"  corpus not ready "
+        f"(last_corpus={doc.get('last_corpus_embedding_model_id')!r}, "
+        f"fixed_size={ (doc.get('chunk_counts') or {}).get('fixed_size') }) — processing"
+    )
+
+    def _process() -> dict[str, Any]:
+        r = client.post(
+            f"/documents/{doc['id']}/process",
+            json={
+                "strategies": [STRATEGY],
+                "corpus_model_id": CORPUS_MODEL_ID,
+            },
+        )
+        r.raise_for_status()
+        body = r.json()
+        failed = body.get("strategies_failed") or []
+        if failed:
+            raise RuntimeError(f"process strategies_failed: {json.dumps(failed)}")
+        # Refresh document record
+        docs = fetch_documents(client)
+        updated = docs.get(filename)
+        if updated is None:
+            raise RuntimeError(f"document disappeared after process: {filename}")
+        if not is_ready(updated):
+            raise RuntimeError(
+                "process finished but provenance still not ready: "
+                f"last_corpus={updated.get('last_corpus_embedding_model_id')!r}, "
+                f"chunk_counts={updated.get('chunk_counts')}"
+            )
+        return updated
+
+    result = with_retries(
+        f"process {filename}",
+        _process,
+        state=state,
+        document=filename,
+        neighbor=-1,
+        kind="process",
+    )
+    if result is None:
+        return None
+    state.results.append(
+        UnitResult(
+            document=filename,
+            neighbor=-1,
+            kind="process",
+            ok=True,
+            detail="gap-filled fixed_size",
+            attempts=1,
+        )
+    )
+    return result
+
+
+def run_experiment(
+    client: httpx.Client,
+    *,
+    doc: dict[str, Any],
+    questions_file: str,
+    neighbor: int,
+    state: SweepState,
+) -> None:
+    filename = doc["filename"]
+
+    def _bench() -> dict[str, Any]:
+        r = client.post(
+            "/benchmarks",
+            json={
+                "document_id": doc["id"],
+                "strategies": [STRATEGY],
+                "questions_file": questions_file,
+                "top_k": TOP_K,
+                "neighbor_prev": neighbor,
+                "neighbor_next": neighbor,
+                "corpus_model_id": CORPUS_MODEL_ID,
+            },
+        )
+        r.raise_for_status()
+        return r.json()
+
+    body = with_retries(
+        f"±{neighbor}/{neighbor} on {filename}",
+        _bench,
+        state=state,
+        document=filename,
+        neighbor=neighbor,
+        kind="experiment",
+    )
+    if body is None:
+        return
+
+    exp_id = body.get("experiment_id", "")
+    state.results.append(
+        UnitResult(
+            document=filename,
+            neighbor=neighbor,
+            kind="experiment",
+            ok=True,
+            detail=(
+                f"best={body.get('best_strategy')} "
+                f"latency={body.get('total_latency_seconds')}s "
+                f"cost=${body.get('estimated_cost_usd')}"
+            ),
+            experiment_id=exp_id,
+        )
+    )
+    log(f"  ok experiment_id={exp_id}")
+
+
+def print_summary(state: SweepState) -> None:
+    log("")
+    log("=" * 88)
+    log("BENCHMARK SWEEP SUMMARY")
+    log("=" * 88)
+    log(
+        f"{'document':<40} {'±N':>4} {'kind':<11} {'status':<6} detail"
+    )
+    log("-" * 88)
+    for r in state.results:
+        n = "—" if r.neighbor < 0 else f"±{r.neighbor}"
+        status = "ok" if r.ok else "FAIL"
+        detail = r.experiment_id or r.detail
+        if not r.ok and r.experiment_id is None:
+            detail = r.detail
+        elif r.ok and r.experiment_id:
+            detail = f"id={r.experiment_id} {r.detail}"
+        log(f"{r.document:<40} {n:>4} {r.kind:<11} {status:<6} {detail}")
+
+    failed = state.failed
+    log("-" * 88)
+    exp_ok = sum(1 for r in state.results if r.kind == "experiment" and r.ok)
+    exp_fail = sum(1 for r in state.results if r.kind == "experiment" and not r.ok)
+    log(f"Experiments: {exp_ok} ok, {exp_fail} failed")
+
+    if failed:
+        log("")
+        log("Manual redo hints (failed units):")
+        for r in failed:
+            if r.kind == "process":
+                log(
+                    f"  • Process {r.document}: "
+                    f'POST /documents/{{id}}/process '
+                    f'{{"strategies":["{STRATEGY}"],"corpus_model_id":"{CORPUS_MODEL_ID}"}}'
+                )
+            else:
+                log(
+                    f"  • Experiment {r.document} ±{r.neighbor}: "
+                    f'POST /benchmarks with strategies=["{STRATEGY}"], '
+                    f"neighbor_prev={r.neighbor}, neighbor_next={r.neighbor}, "
+                    f"corpus_model_id={CORPUS_MODEL_ID}, questions_file from SWEEP_PAIRS"
+                )
+            log(f"    error: {r.detail}")
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Run fixed_size Neighbor Expansion Benchmark Sweep")
+    parser.add_argument(
+        "--base-url",
+        default=DEFAULT_BASE_URL,
+        help=f"FastAPI base URL (default: {DEFAULT_BASE_URL})",
+    )
+    args = parser.parse_args()
+    base_url = args.base_url.rstrip("/")
+
+    state = SweepState()
+    log(f"Benchmark Sweep → {base_url}")
+    log(f"Strategy={STRATEGY} corpus={CORPUS_MODEL_ID} top_k={TOP_K} neighbors={list(NEIGHBOR_LEVELS)}")
+    log(f"Documents: {len(SWEEP_PAIRS)}")
+
+    timeout = httpx.Timeout(REQUEST_TIMEOUT_S, connect=30.0)
+    with httpx.Client(base_url=base_url, timeout=timeout) as client:
+        try:
+            health = client.get("/admin/health")
+            health.raise_for_status()
+        except Exception as exc:
+            log(f"Cannot reach app at {base_url}: {error_detail(exc)}")
+            return 2
+
+        docs = fetch_documents(client)
+        log(f"Loaded {len(docs)} documents from app")
+
+        for filename, questions_file in SWEEP_PAIRS:
+            log("")
+            log(f"=== {filename} ({questions_file}) ===")
+            doc = docs.get(filename)
+            if doc is None:
+                log(f"  ! document not found in app — skipping")
+                state.results.append(
+                    UnitResult(
+                        document=filename,
+                        neighbor=-1,
+                        kind="process",
+                        ok=False,
+                        detail="document not found in GET /documents",
+                        attempts=1,
+                    )
+                )
+                continue
+
+            updated = ensure_corpus(client, doc, state=state)
+            if updated is None:
+                log(f"  ! skipping experiments for {filename} (process failed)")
+                continue
+            docs[filename] = updated
+
+            for n in NEIGHBOR_LEVELS:
+                run_experiment(
+                    client,
+                    doc=updated,
+                    questions_file=questions_file,
+                    neighbor=n,
+                    state=state,
+                )
+
+    print_summary(state)
+    return 1 if state.failed else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

src/admin/routes.py

@@ -1,12 +1,30 @@
 """Admin API routes — system health, Qdrant management, chunk preview, questions, cost."""
 
-import os
+from fastapi import APIRouter, HTTPException, UploadFile, File
-from fastapi import APIRouter, UploadFile, File
+from pydantic import BaseModel, Field
+
 from src.admin import service
 
 router = APIRouter(prefix="/admin")
 
 
+class ActiveEmbeddingModelBody(BaseModel):
+    id: str = Field(..., description="Registry Embedding Model id")
+
+
+class RoleEmbeddingModelBody(BaseModel):
+    id: str = Field(..., description="Registry Embedding Model id for Boundary or Corpus role")
+
+
+class SemanticThresholdBody(BaseModel):
+    semantic_threshold: float = Field(
+        ...,
+        gt=0.0,
+        le=1.0,
+        description="Cosine similarity cutoff for Semantic Boundary Detection",
+    )
+
+
 # ── Health ──────────────────────────────────────────────────
 
 @router.get("/health")
@@ -15,6 +33,50 @@ async def health_check():
     return service.get_health()
 
 
+# ── Embedding Models ────────────────────────────────────────
+
+@router.get("/embedding-models")
+async def list_embedding_models():
+    """List Embedding Model Registry and Boundary/Corpus defaults."""
+    return service.list_embedding_models()
+
+
+@router.put("/embedding-models/active")
+async def set_active_embedding_model(body: ActiveEmbeddingModelBody):
+    """Legacy: set Default Corpus Embedding Model."""
+    result = service.set_corpus_embedding_model(body.id)
+    if "error" in result:
+        raise HTTPException(status_code=400, detail=result["error"])
+    return {**result, "active_id": result.get("corpus_id")}
+
+
+@router.put("/embedding-models/corpus")
+async def set_corpus_embedding_model(body: RoleEmbeddingModelBody):
+    """Set Default Corpus Embedding Model (storage + query)."""
+    result = service.set_corpus_embedding_model(body.id)
+    if "error" in result:
+        raise HTTPException(status_code=400, detail=result["error"])
+    return result
+
+
+@router.put("/embedding-models/boundary")
+async def set_boundary_embedding_model(body: RoleEmbeddingModelBody):
+    """Set Default Boundary Embedding Model (semantic cuts)."""
+    result = service.set_boundary_embedding_model(body.id)
+    if "error" in result:
+        raise HTTPException(status_code=400, detail=result["error"])
+    return result
+
+
+@router.put("/embedding-models/{model_id}/semantic-threshold")
+async def update_semantic_threshold(model_id: str, body: SemanticThresholdBody):
+    """Set per-Embedding-Model semantic_threshold (Admin override in SQLite)."""
+    result = service.update_semantic_threshold(model_id, body.semantic_threshold)
+    if "error" in result:
+        raise HTTPException(status_code=400, detail=result["error"])
+    return result
+
+
 # ── Qdrant Collections ─────────────────────────────────────
 
 @router.get("/qdrant/collections")

src/admin/service.py

@@ -24,7 +24,12 @@ QUESTIONS_DIR = PROJECT_ROOT / "files"
 # ── Health ──────────────────────────────────────────────────
 
 def get_health() -> dict[str, Any]:
-    """Check server, Qdrant, and SQLite status."""
+    """Check server, Qdrant, SQLite, OpenAI, and Ollama status."""
+    from src.chunking.embedding import (
+        get_boundary_embedding_model,
+        get_corpus_embedding_model,
+    )
+
     result: dict[str, Any] = {"status": "ok"}
 
     # Check Qdrant
@@ -51,57 +56,228 @@ def get_health() -> dict[str, Any]:
 
     # Check OpenAI
     try:
-        client = get_openai_client()
+        get_openai_client()
-        # Just check the client exists; don't make a real API call
         result["openai_configured"] = bool(settings.openai_api_key)
     except Exception:
         result["openai_configured"] = False
 
+    try:
+        corpus = get_corpus_embedding_model()
+        boundary = get_boundary_embedding_model()
+        result["corpus_embedding_model"] = {
+            "id": corpus.id,
+            "provider": corpus.provider.value,
+            "model_name": corpus.model_name,
+            "dimension": corpus.dimension,
+            "display_name": corpus.display_name,
+        }
+        result["boundary_embedding_model"] = {
+            "id": boundary.id,
+            "provider": boundary.provider.value,
+            "model_name": boundary.model_name,
+            "dimension": boundary.dimension,
+            "display_name": boundary.display_name,
+        }
+        # Legacy alias for older Dashboard code
+        result["active_embedding_model"] = result["corpus_embedding_model"]
+    except Exception as exc:
+        result["corpus_embedding_model"] = None
+        result["boundary_embedding_model"] = None
+        result["active_embedding_model"] = None
+        result["active_embedding_error"] = str(exc)
+
+    result["ollama_base_url"] = settings.ollama_base_url
+    try:
+        import urllib.request
+
+        url = settings.ollama_base_url.rstrip("/") + "/api/tags"
+        with urllib.request.urlopen(url, timeout=2) as resp:
+            result["ollama_reachable"] = resp.status == 200
+    except Exception as exc:
+        result["ollama_reachable"] = False
+        result["ollama_error"] = str(exc)
+
     return result
 
 
+# ── Embedding Models ────────────────────────────────────────
+
+def list_embedding_models() -> dict[str, Any]:
+    """List registry entries and Boundary/Corpus defaults."""
+    from src.chunking.embedding import (
+        get_boundary_embedding_model,
+        get_corpus_embedding_model,
+    )
+    from src.chunking.embedding_models import get_semantic_threshold, list_models
+
+    corpus = get_corpus_embedding_model()
+    boundary = get_boundary_embedding_model()
+    models = []
+    for m in list_models():
+        models.append({
+            "id": m.id,
+            "provider": m.provider.value,
+            "model_name": m.model_name,
+            "dimension": m.dimension,
+            "display_name": m.display_name,
+            "task_prefixes": m.task_prefixes,
+            "default_semantic_threshold": m.default_semantic_threshold,
+            "semantic_threshold": get_semantic_threshold(m.id),
+            "is_corpus_default": m.id == corpus.id,
+            "is_boundary_default": m.id == boundary.id,
+            "is_active": m.id == corpus.id,  # legacy
+        })
+    return {
+        "corpus_id": corpus.id,
+        "boundary_id": boundary.id,
+        "active_id": corpus.id,  # legacy
+        "models": models,
+        "ollama_base_url": settings.ollama_base_url,
+    }
+
+
+def set_active_embedding_model(model_id: str) -> dict[str, Any]:
+    """Legacy: set Corpus default."""
+    return set_corpus_embedding_model(model_id)
+
+
+def set_corpus_embedding_model(model_id: str) -> dict[str, Any]:
+    """Switch the Default Corpus Embedding Model."""
+    from src.chunking.embedding import set_corpus_embedding_model as set_corpus
+
+    try:
+        model = set_corpus(model_id)
+    except KeyError as exc:
+        return {"error": str(exc)}
+
+    return {
+        "corpus_id": model.id,
+        "model": {
+            "id": model.id,
+            "provider": model.provider.value,
+            "model_name": model.model_name,
+            "dimension": model.dimension,
+            "display_name": model.display_name,
+        },
+    }
+
+
+def set_boundary_embedding_model(model_id: str) -> dict[str, Any]:
+    """Switch the Default Boundary Embedding Model."""
+    from src.chunking.embedding import set_boundary_embedding_model as set_boundary
+
+    try:
+        model = set_boundary(model_id)
+    except KeyError as exc:
+        return {"error": str(exc)}
+
+    return {
+        "boundary_id": model.id,
+        "model": {
+            "id": model.id,
+            "provider": model.provider.value,
+            "model_name": model.model_name,
+            "dimension": model.dimension,
+            "display_name": model.display_name,
+        },
+    }
+
+
+def update_semantic_threshold(model_id: str, threshold: float) -> dict[str, Any]:
+    """Persist Admin override of semantic_threshold for a registry Embedding Model."""
+    from src.chunking.embedding_models import (
+        get_model,
+        get_semantic_threshold,
+        set_semantic_threshold,
+    )
+
+    try:
+        model = get_model(model_id)
+        value = set_semantic_threshold(model_id, threshold)
+    except KeyError as exc:
+        return {"error": str(exc)}
+    except ValueError as exc:
+        return {"error": str(exc)}
+
+    return {
+        "id": model.id,
+        "semantic_threshold": value,
+        "default_semantic_threshold": model.default_semantic_threshold,
+        "effective": get_semantic_threshold(model.id),
+    }
+
+
 # ── Qdrant Collections ─────────────────────────────────────
 
 def list_qdrant_collections() -> dict[str, Any]:
-    """List all Qdrant collections with their point counts."""
+    """List all Qdrant collections with point counts and Embedding Model labels."""
+    from src.chunking.embedding import get_active_embedding_model
+
     client = get_qdrant_client()
     collections_data = client.get_collections().collections
+    active = get_active_embedding_model()
 
     result = []
     for col in collections_data:
+        meta = qdrant_store.parse_collection_meta(col.name)
         try:
             info = client.get_collection(collection_name=col.name)
-            result.append({
+            points = info.points_count or 0
-                "name": col.name,
-                "points_count": info.points_count or 0,
-            })
         except Exception as exc:
-            result.append({
+            points = None
-                "name": col.name,
+            err = str(exc)
-                "points_count": None,
+        else:
-                "error": str(exc),
+            err = None
-            })
 
-    return {"collections": result}
+        entry = {
+            "name": col.name,
+            "points_count": points,
+            "strategy": meta.get("strategy"),
+            "embedding_model_id": meta.get("embedding_model_id"),
+            "is_legacy": meta.get("is_legacy", False),
+            "is_active_corpus": meta.get("embedding_model_id") == active.id,
+        }
+        if err:
+            entry["error"] = err
+        result.append(entry)
+
+    # Active Model Corpus first
+    result.sort(key=lambda c: (not c.get("is_active_corpus", False), c["name"]))
+    return {
+        "collections": result,
+        "active_embedding_model_id": active.id,
+    }
 
 
 def create_qdrant_collection(collection_name: str) -> dict[str, Any]:
-    """Create a new Qdrant collection."""
+    """Create a new Qdrant collection using Active Embedding Model dimension."""
+    from src.chunking.embedding import get_active_embedding_model
+
     client = get_qdrant_client()
+    active = get_active_embedding_model()
 
     existing = [c.name for c in client.get_collections().collections]
     if collection_name in existing:
         return {"created": False, "message": f"Collection '{collection_name}' already exists"}
 
+    meta = qdrant_store.parse_collection_meta(collection_name)
+    dimension = active.dimension
+    if meta.get("embedding_model_id"):
+        try:
+            from src.chunking.embedding_models import get_model
+            dimension = get_model(meta["embedding_model_id"]).dimension
+        except KeyError:
+            pass
+
     client.create_collection(
         collection_name=collection_name,
         vectors_config=VectorParams(
-            size=qdrant_store.VECTOR_DIMENSION,
+            size=dimension,
             distance=Distance.COSINE,
         ),
     )
-    logger.info("Created Qdrant collection: %s", collection_name)
+    logger.info("Created Qdrant collection: %s (dim=%d)", collection_name, dimension)
-    return {"created": True, "collection": collection_name}
+    return {"created": True, "collection": collection_name, "dimension": dimension}
 
 
 def delete_qdrant_collection(collection_name: str) -> dict[str, Any]:
@@ -134,8 +310,11 @@ def wipe_qdrant_collection_points(collection_name: str) -> dict[str, Any]:
 # ── Chunk Preview ───────────────────────────────────────────
 
 def preview_chunks(doc_id: str, strategy: str | None = None) -> dict[str, Any]:
-    """Preview chunks for a document. Uses Qdrant scroll to fetch chunks with payload."""
+    """Preview chunks for a document from the Active Embedding Model's corpus."""
+    from src.chunking.embedding import get_active_embedding_model
+
     client = get_qdrant_client()
+    active = get_active_embedding_model()
 
     from qdrant_client.models import Filter, FieldCondition, MatchValue
 
@@ -155,12 +334,12 @@ def preview_chunks(doc_id: str, strategy: str | None = None) -> dict[str, Any]:
 
     results = {}
     for strat_name in strategies_to_search:
-        col_name = f"{strat_name}_collection"
+        col_name = qdrant_store.collection_name(strat_name, active.id)
 
         try:
             existing = [c.name for c in client.get_collections().collections]
             if col_name not in existing:
-                results[strat_name] = {"chunks": [], "count": 0}
+                results[strat_name] = {"chunks": [], "count": 0, "collection": col_name}
                 continue
 
             scroll_filter = Filter(
@@ -188,12 +367,21 @@ def preview_chunks(doc_id: str, strategy: str | None = None) -> dict[str, Any]:
 
             # Sort by chunk_index
             chunks.sort(key=lambda c: c.get("chunk_index") or 0)
-            results[strat_name] = {"chunks": chunks, "count": len(chunks)}
+            results[strat_name] = {
+                "chunks": chunks,
+                "count": len(chunks),
+                "collection": col_name,
+            }
 
         except Exception as exc:
             results[strat_name] = {"error": str(exc), "chunks": [], "count": 0}
 
-    return {"document_id": doc_id, "filename": doc_name, "strategies": results}
+    return {
+        "document_id": doc_id,
+        "filename": doc_name,
+        "embedding_model_id": active.id,
+        "strategies": results,
+    }
 
 
 # ── Questions Dataset ───────────────────────────────────────

src/benchmarking/benchmark_service.py

@@ -14,6 +14,7 @@ from typing import Any
 
 from src.benchmarking.evaluation import evaluate_single
 from src.benchmarking.query_service import run_query
+from src.chunking.embedding_models import Provider
 from src.core.config import settings
 from src.core.exceptions import BenchmarkError
 from src.core.models import StrategyName
@@ -87,15 +88,16 @@ def estimate_cost(
 ) -> dict[str, Any]:
     """Estimate the cost of running a benchmark.
 
-    Args:
+    Local Embedding Models contribute $0 embedding cost (ADR-0019).
-        num_questions: Number of questions
-        num_strategies: Number of strategies
-
-    Returns:
-        Cost estimate dict
     """
+    from src.chunking.embedding import get_corpus_embedding_model
+
+    active = get_corpus_embedding_model()
+
     # Rough estimates based on GPT-4o-mini pricing
-    embedding_cost_per_call = 0.0001
+    embedding_cost_per_call = (
+        0.0 if active.provider == Provider.LOCAL else 0.0001
+    )
     query_cost_per_call = 0.001
     evaluation_cost_per_call = 0.001
 
@@ -118,6 +120,8 @@ def estimate_cost(
         "num_strategies": num_strategies,
         "total_queries": total_queries,
         "total_evaluations": total_evaluations,
+        "embedding_model_id": active.id,
+        "embedding_provider": active.provider.value,
         "estimated_tokens": {
             "input": total_input_tokens,
             "output": total_output_tokens,
@@ -139,6 +143,9 @@ def run_benchmark(
     strategies: list[StrategyName],
     questions: list[dict],
     top_k: int = 5,
+    neighbor_prev: int = 0,
+    neighbor_next: int = 0,
+    corpus_model_id: str | None = None,
 ) -> dict[str, Any]:
     """Run a full benchmark across questions and strategies.
 
@@ -147,16 +154,40 @@ def run_benchmark(
         strategies: List of strategies to test
         questions: List of question dicts
         top_k: Number of chunks to retrieve per query
+        neighbor_prev: Neighbor Expansion prev count (fixed_size only)
+        neighbor_next: Neighbor Expansion next count (fixed_size only)
+        corpus_model_id: Corpus Embedding Model (default Admin Corpus)
 
     Returns:
         Complete benchmark results
     """
+    from src.chunking.embedding import resolve_corpus_model
+
     t_start = time.time()
+    embedding_model = resolve_corpus_model(corpus_model_id)
+    doc = db.get_document(document_id)
+    boundary_id = None
+    if doc and any(
+        s.value in ("semantic", "semantic_parent_child") for s in strategies
+    ):
+        boundary_id = doc.get("last_boundary_embedding_model_id")
+
     logger.info("=" * 80)
     logger.info("[BENCHMARK] Starting benchmark")
     logger.info("[BENCHMARK] Document: %s", document_id)
     logger.info("[BENCHMARK] Strategies: %s", [s.value for s in strategies])
     logger.info("[BENCHMARK] Questions: %d", len(questions))
+    logger.info(
+        "[BENCHMARK] Corpus Embedding Model: %s (%s)",
+        embedding_model.id,
+        embedding_model.provider.value,
+    )
+    logger.info("[BENCHMARK] Boundary (from last process): %s", boundary_id or "—")
+    logger.info(
+        "[BENCHMARK] Neighbor Expansion: prev=%d next=%d",
+        neighbor_prev,
+        neighbor_next,
+    )
 
     per_question_results = []
     total_cost = 0.0
@@ -186,6 +217,9 @@ def run_benchmark(
                     strategy_name=strategy,
                     question=question_text,
                     top_k=top_k,
+                    neighbor_prev=neighbor_prev,
+                    neighbor_next=neighbor_next,
+                    embedding_model=embedding_model,
                 )
                 t_query = time.time() - t0
 
@@ -206,6 +240,7 @@ def run_benchmark(
                 question_results["strategies"][strategy.value] = {
                     "answer": query_result["answer"],
                     "retrieved_chunks": query_result["retrieved_chunks"],
+                    "expansion_tree": query_result.get("expansion_tree") or [],
                     "scores": eval_scores,
                     "latency": {
                         "query_seconds": round(t_query, 3),
@@ -252,11 +287,20 @@ def run_benchmark(
             "strategies": [s.value for s in strategies],
             "num_questions": len(questions),
             "top_k": top_k,
+            "neighbor_prev": neighbor_prev,
+            "neighbor_next": neighbor_next,
+            "embedding_model_id": embedding_model.id,
+            "corpus_embedding_model_id": embedding_model.id,
+            "embedding_provider": embedding_model.provider.value,
+            "boundary_embedding_model_id": boundary_id,
         },
         questions=questions,
         per_question=per_question_results,
         aggregate_metrics=aggregate,
         strategies_used=[s.value for s in strategies],
+        embedding_model_id=embedding_model.id,
+        embedding_provider=embedding_model.provider.value,
+        boundary_embedding_model_id=boundary_id,
     )
 
     logger.info("[BENCHMARK] Completed in %.1fs", t_total)
@@ -267,6 +311,8 @@ def run_benchmark(
         "experiment_id": experiment["id"],
         "document_id": document_id,
         "strategies_used": [s.value for s in strategies],
+        "embedding_model_id": embedding_model.id,
+        "embedding_provider": embedding_model.provider.value,
         "questions_count": len(questions),
         "aggregate_metrics": aggregate,
         "best_strategy": best_strategy,

src/benchmarking/evaluation.py

@@ -66,9 +66,14 @@ def _build_context_for_evaluation(retrieved_chunks: list[dict]) -> str:
     """Build a readable context string from retrieved chunks."""
     parts = []
     for i, chunk in enumerate(retrieved_chunks, 1):
-        score = chunk.get("score", 0)
+        score = chunk.get("score")
         text = chunk.get("text", "")
-        parts.append(f"[Chunk {i} (score: {score:.3f})]\n{text}")
+        role = chunk.get("role") or ("neighbor" if score is None else "hit")
+        if isinstance(score, (int, float)):
+            header = f"[Chunk {i} (score: {score:.3f}, {role})]"
+        else:
+            header = f"[Chunk {i} (score: —, {role})]"
+        parts.append(f"{header}\n{text}")
     return "\n\n".join(parts)
 
 

src/benchmarking/models.py

@@ -4,6 +4,7 @@ from typing import Optional
 
 from pydantic import BaseModel, Field
 
+from src.core.config import settings
 from src.core.models import StrategyName
 
 
@@ -15,16 +16,51 @@ class QueryRequest(BaseModel):
     strategy: StrategyName = Field(description="Chunking strategy to use")
     question: str = Field(description="Question to ask", min_length=1)
     top_k: int = Field(default=5, description="Number of chunks to retrieve", ge=1, le=20)
+    neighbor_prev: int = Field(
+        default_factory=lambda: settings.neighbor_prev,
+        description="Neighbor Expansion: previous chunks per hit (fixed_size only)",
+        ge=0,
+        le=5,
+    )
+    neighbor_next: int = Field(
+        default_factory=lambda: settings.neighbor_next,
+        description="Neighbor Expansion: next chunks per hit (fixed_size only)",
+        ge=0,
+        le=5,
+    )
+    corpus_model_id: Optional[str] = Field(
+        default=None,
+        description="Corpus Embedding Model id (query + search); default = Admin Corpus",
+    )
 
 
 # ── Response ───────────────────────────────────────────────────────
 
 class RetrievedChunk(BaseModel):
-    """A single retrieved chunk with its similarity score."""
+    """A single chunk in the flat LLM/eval context (ADR-0023)."""
+    chunk_id: str
+    score: Optional[float] = None
+    text: str
+    parent_id: Optional[str] = None
+    chunk_index: Optional[int] = None
+    role: Optional[str] = None  # "hit" | "neighbor"
+
+
+class ExpansionNeighbor(BaseModel):
+    """A neighbor chunk in the Expansion Tree."""
+    chunk_id: str
+    text: str
+    chunk_index: Optional[int] = None
+
+
+class ExpansionTreeNode(BaseModel):
+    """One top-k hit with its per-hit Neighbor Expansion window."""
     chunk_id: str
     score: float
     text: str
-    parent_id: Optional[str] = None
+    chunk_index: Optional[int] = None
+    neighbors_prev: list[ExpansionNeighbor] = Field(default_factory=list)
+    neighbors_next: list[ExpansionNeighbor] = Field(default_factory=list)
 
 
 class QueryResponse(BaseModel):
@@ -35,6 +71,9 @@ class QueryResponse(BaseModel):
     question: str
     answer: str
     retrieved_chunks: list[RetrievedChunk]
+    expansion_tree: list[ExpansionTreeNode] = Field(default_factory=list)
+    neighbor_prev: int = 0
+    neighbor_next: int = 0
     latency_breakdown: dict[str, float]
     token_usage: dict[str, int]
     created_at: str
@@ -48,6 +87,7 @@ class QueryDetailResponse(BaseModel):
     question: str
     answer: str
     retrieved_chunks: list[dict]
+    expansion_tree: list[dict] = Field(default_factory=list)
     latency_breakdown: dict[str, float]
     token_usage: dict[str, int]
     created_at: str
@@ -78,6 +118,22 @@ class BenchmarkRequest(BaseModel):
         description="Path to questions JSON file (relative to project root)",
     )
     top_k: int = Field(default=5, description="Number of chunks to retrieve", ge=1, le=20)
+    neighbor_prev: int = Field(
+        default_factory=lambda: settings.neighbor_prev,
+        description="Neighbor Expansion: previous chunks per hit (fixed_size only)",
+        ge=0,
+        le=5,
+    )
+    neighbor_next: int = Field(
+        default_factory=lambda: settings.neighbor_next,
+        description="Neighbor Expansion: next chunks per hit (fixed_size only)",
+        ge=0,
+        le=5,
+    )
+    corpus_model_id: Optional[str] = Field(
+        default=None,
+        description="Corpus Embedding Model id; default = Admin Corpus",
+    )
     dry_run: bool = Field(default=False, description="Only return cost estimate, don't run benchmark")
 
 

src/benchmarking/query_service.py

@@ -16,9 +16,10 @@ from typing import Any
 
 from openai import OpenAI
 
-from src.chunking.embedding import embed_single
+from src.chunking.embedding import embed_single, resolve_corpus_model
+from src.chunking.embedding_models import EmbeddingModelSpec
 from src.core.config import settings
-from src.core.dependencies import get_openai_client
+from src.core.dependencies import get_openai_client, get_qdrant_client
 from src.core.exceptions import QueryError
 from src.core.models import StrategyName
 from src.storage import qdrant as qdr
@@ -41,6 +42,8 @@ def _fetch_parent_chunks(
     child_hits: list[dict],
     strategy: StrategyName,
     document_name: str,
+    *,
+    model_id: str,
 ) -> dict[str, dict]:
     """Fetch parent chunks for child hits in semantic_parent_child strategy.
 
@@ -64,11 +67,10 @@ def _fetch_parent_chunks(
     # Search for parent chunks by their IDs
     parents = {}
     for parent_id in parent_ids:
-        # Use Qdrant scroll to find the parent chunk
         from qdrant_client.models import Filter, FieldCondition, MatchValue
 
-        client = qdr.get_qdrant_client()
+        client = get_qdrant_client()
-        name = qdr.collection_name(strategy)
+        name = qdr.collection_name(strategy, model_id)
 
         try:
             results = client.scroll(
@@ -103,6 +105,8 @@ def _build_context(
     hits: list[dict],
     strategy: StrategyName,
     document_name: str,
+    *,
+    model_id: str,
 ) -> str:
     """Build context string from retrieved chunks.
 
@@ -115,7 +119,9 @@ def _build_context(
 
     if strategy == StrategyName.SEMANTIC_PARENT_CHILD:
         # Fetch parent chunks
-        parents = _fetch_parent_chunks(hits, strategy, document_name)
+        parents = _fetch_parent_chunks(
+            hits, strategy, document_name, model_id=model_id
+        )
 
         for i, hit in enumerate(hits, 1):
             payload = hit.get("payload", {})
@@ -124,7 +130,10 @@ def _build_context(
             parent_id = payload.get("parent_id")
 
             # Add child chunk
-            context_parts.append(f"[Chunk {i} (score: {score:.3f})]")
+            if isinstance(score, (int, float)):
+                context_parts.append(f"[Chunk {i} (score: {score:.3f})]")
+            else:
+                context_parts.append(f"[Chunk {i} (score: —)]")
             context_parts.append(chunk_text)
 
             # Add parent context if available
@@ -143,10 +152,18 @@ def _build_context(
             score = hit.get("score", 0)
             chunk_text = payload.get("text", "")
 
-            logger.info("[CONTEXT] Chunk %d: score=%.3f, text_len=%d, chunk_id=%s",
+            logger.info(
-                       i, score, len(chunk_text), hit.get("chunk_id", "unknown"))
+                "[CONTEXT] Chunk %d: score=%s, text_len=%d, chunk_id=%s",
+                i,
+                f"{score:.3f}" if isinstance(score, (int, float)) else "—",
+                len(chunk_text),
+                hit.get("chunk_id", "unknown"),
+            )
 
-            context_parts.append(f"[Chunk {i} (score: {score:.3f})]")
+            if isinstance(score, (int, float)):
+                context_parts.append(f"[Chunk {i} (score: {score:.3f})]")
+            else:
+                context_parts.append(f"[Chunk {i} (score: —)]")
             context_parts.append(chunk_text)
             context_parts.append("")
 
@@ -201,6 +218,170 @@ def _generate_answer(
         raise QueryError(f"Answer generation failed: {exc}") from exc
 
 
+def _hit_to_chunk_dict(hit: dict, *, role: str) -> dict[str, Any]:
+    payload = hit.get("payload") or {}
+    score = hit.get("score")
+    return {
+        "chunk_id": payload.get("chunk_id", hit.get("chunk_id")),
+        "score": score if role == "hit" else None,
+        "text": payload.get("text", ""),
+        "parent_id": payload.get("parent_id"),
+        "chunk_index": payload.get("chunk_index"),
+        "role": role,
+    }
+
+
+def _neighbor_brief(entry: dict[str, Any]) -> dict[str, Any]:
+    payload = entry.get("payload") or {}
+    return {
+        "chunk_id": payload.get("chunk_id", entry.get("chunk_id")),
+        "text": payload.get("text", ""),
+        "chunk_index": payload.get("chunk_index"),
+    }
+
+
+def apply_neighbor_expansion(
+    hits: list[dict],
+    *,
+    strategy: StrategyName,
+    document_name: str,
+    model_id: str,
+    neighbor_prev: int,
+    neighbor_next: int,
+) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
+    """Build flat retrieved_chunks + Expansion Tree (ADR-0023).
+
+    Expansion runs only for fixed_size when prev/next > 0. Otherwise the tree
+    lists hits with empty neighbor arrays and the flat list is the top-k hits.
+    """
+    expansion_tree: list[dict[str, Any]] = []
+    for hit in hits:
+        payload = hit.get("payload") or {}
+        expansion_tree.append({
+            "chunk_id": payload.get("chunk_id", hit.get("chunk_id")),
+            "score": hit.get("score", 0),
+            "text": payload.get("text", ""),
+            "chunk_index": payload.get("chunk_index"),
+            "neighbors_prev": [],
+            "neighbors_next": [],
+        })
+
+    apply = (
+        strategy == StrategyName.FIXED_SIZE
+        and (neighbor_prev > 0 or neighbor_next > 0)
+        and hits
+    )
+    if not apply:
+        retrieved = [_hit_to_chunk_dict(h, role="hit") for h in hits]
+        return retrieved, expansion_tree
+
+    needed_indices: set[int] = set()
+    hit_indices: list[int | None] = []
+    for hit in hits:
+        payload = hit.get("payload") or {}
+        idx = payload.get("chunk_index")
+        hit_indices.append(idx if idx is not None else None)
+        if idx is None:
+            continue
+        idx = int(idx)
+        for d in range(1, neighbor_prev + 1):
+            needed_indices.add(idx - d)
+        for d in range(1, neighbor_next + 1):
+            needed_indices.add(idx + d)
+
+    # Never fetch negative indices
+    needed_indices = {i for i in needed_indices if i >= 0}
+
+    # Hits may themselves be neighbors of other hits — include them in the lookup map
+    by_index: dict[int, dict[str, Any]] = {}
+    for hit in hits:
+        payload = hit.get("payload") or {}
+        idx = payload.get("chunk_index")
+        if idx is None:
+            continue
+        by_index[int(idx)] = {
+            "chunk_id": payload.get("chunk_id", hit.get("chunk_id")),
+            "score": hit.get("score"),
+            "payload": payload,
+        }
+
+    fetch_indices = [i for i in needed_indices if i not in by_index]
+    if fetch_indices:
+        by_index.update(
+            qdr.get_chunks_by_indices(
+                strategy,
+                document_name,
+                fetch_indices,
+                model_id=model_id,
+            )
+        )
+
+    for tree_node, hit, idx in zip(expansion_tree, hits, hit_indices):
+        if idx is None:
+            continue
+        idx = int(idx)
+        prev_list = []
+        for d in range(neighbor_prev, 0, -1):
+            entry = by_index.get(idx - d)
+            if entry:
+                prev_list.append(_neighbor_brief(entry))
+        next_list = []
+        for d in range(1, neighbor_next + 1):
+            entry = by_index.get(idx + d)
+            if entry:
+                next_list.append(_neighbor_brief(entry))
+        tree_node["neighbors_prev"] = prev_list
+        tree_node["neighbors_next"] = next_list
+
+    # Flat LLM context: hits + all neighbors, dedupe (prefer hit), sort by chunk_index
+    merged: dict[str, dict[str, Any]] = {}
+    for hit in hits:
+        chunk = _hit_to_chunk_dict(hit, role="hit")
+        cid = chunk["chunk_id"]
+        if cid:
+            merged[cid] = chunk
+
+    for node in expansion_tree:
+        for nbr in node["neighbors_prev"] + node["neighbors_next"]:
+            cid = nbr.get("chunk_id")
+            if not cid or cid in merged:
+                continue
+            merged[cid] = {
+                "chunk_id": cid,
+                "score": None,
+                "text": nbr.get("text", ""),
+                "parent_id": None,
+                "chunk_index": nbr.get("chunk_index"),
+                "role": "neighbor",
+            }
+
+    def _sort_key(c: dict[str, Any]) -> tuple:
+        idx = c.get("chunk_index")
+        if isinstance(idx, int):
+            return (0, idx)
+        return (1, str(c.get("chunk_id", "")))
+
+    retrieved = sorted(merged.values(), key=_sort_key)
+    return retrieved, expansion_tree
+
+
+def _chunks_as_context_hits(retrieved_chunks: list[dict[str, Any]]) -> list[dict]:
+    """Adapt flat retrieved_chunks into the hit shape expected by _build_context."""
+    hits = []
+    for c in retrieved_chunks:
+        hits.append({
+            "chunk_id": c.get("chunk_id"),
+            "score": c.get("score") if c.get("score") is not None else 0,
+            "payload": {
+                "chunk_id": c.get("chunk_id"),
+                "text": c.get("text", ""),
+                "parent_id": c.get("parent_id"),
+                "chunk_index": c.get("chunk_index"),
+            },
+        })
+    return hits
+
+
 # ── Main query function ───────────────────────────────────────────
 
 def run_query(
@@ -209,27 +390,44 @@ def run_query(
     strategy_name: StrategyName,
     question: str,
     top_k: int = 5,
+    neighbor_prev: int | None = None,
+    neighbor_next: int | None = None,
+    embedding_model: EmbeddingModelSpec | None = None,
+    corpus_model_id: str | None = None,
 ) -> dict[str, Any]:
     """Run a query against a document using a specific chunking strategy.
 
-    Pipeline: embed → search → answer → store
+    Pipeline: embed → search → Neighbor Expansion (fixed_size) → answer → store
 
     Args:
         document_id: The document to query against.
         strategy_name: Which chunking strategy's collection to search.
         question: The user's question.
         top_k: Number of chunks to retrieve (default 5).
+        neighbor_prev: Prev chunks per hit for fixed_size (default from config).
+        neighbor_next: Next chunks per hit for fixed_size (default from config).
+        embedding_model: Explicit Corpus snapshot (e.g. Experiment);
+            if omitted, resolves corpus_model_id or Admin Corpus default.
+        corpus_model_id: Registry id for Corpus Embedding Model.
 
     Returns:
-        Query result dict with answer, chunks, and metadata.
+        Query result dict with answer, chunks, expansion_tree, and metadata.
     """
     t_start = time.time()
+    if embedding_model is not None:
+        model = embedding_model
+    else:
+        model = resolve_corpus_model(corpus_model_id)
+    prev_n = settings.neighbor_prev if neighbor_prev is None else neighbor_prev
+    next_n = settings.neighbor_next if neighbor_next is None else neighbor_next
     logger.info("=" * 80)
     logger.info("[QUERY] Starting query pipeline")
     logger.info("[QUERY] Document ID: %s", document_id)
     logger.info("[QUERY] Strategy: %s", strategy_name.value)
+    logger.info("[QUERY] Corpus Embedding Model: %s (%s)", model.id, model.provider.value)
     logger.info("[QUERY] Question: %s", question)
     logger.info("[QUERY] Top K: %d", top_k)
+    logger.info("[QUERY] Neighbor Expansion: prev=%d next=%d", prev_n, next_n)
 
     # 1. Load document info
     logger.info("[STEP 1] Loading document info from SQLite...")
@@ -246,35 +444,54 @@ def run_query(
     # 2. Embed the question
     logger.info("[STEP 2] Embedding question...")
     t0 = time.time()
-    question_embedding = embed_single(question)
+    question_embedding = embed_single(question, model=model, purpose="query")
     t_embed = time.time() - t0
     logger.info("[STEP 2] Question embedded in %.2fs", t_embed)
     logger.info("[STEP 2] Embedding dimension: %d", len(question_embedding))
 
     # 3. Vector search in Qdrant
-    logger.info("[STEP 3] Searching Qdrant collection: %s_collection", strategy_name.value)
+    col = qdr.collection_name(strategy_name, model.id)
+    logger.info("[STEP 3] Searching Qdrant collection: %s", col)
     t1 = time.time()
     hits = qdr.search(
         strategy=strategy_name,
         query_vector=question_embedding,
         top_k=top_k,
         document_filter=document_name,
+        model_id=model.id,
     )
     t_search = time.time() - t1
     logger.info("[STEP 3] Search completed in %.2fs", t_search)
     logger.info("[STEP 3] Found %d chunks", len(hits))
 
-    # Log each hit
     for i, hit in enumerate(hits, 1):
         payload = hit.get("payload", {})
         logger.info("[STEP 3] Hit %d: chunk_id=%s, score=%.4f, text_len=%d",
                    i, hit.get("chunk_id", "unknown"), hit.get("score", 0),
                    len(payload.get("text", "")))
 
+    # 3b. Neighbor Expansion (fixed_size) → flat list + Expansion Tree
+    retrieved_chunks, expansion_tree = apply_neighbor_expansion(
+        hits,
+        strategy=strategy_name,
+        document_name=document_name,
+        model_id=model.id,
+        neighbor_prev=prev_n,
+        neighbor_next=next_n,
+    )
+    logger.info(
+        "[STEP 3b] Expansion: %d flat chunks, tree nodes=%d",
+        len(retrieved_chunks),
+        len(expansion_tree),
+    )
+
     # 4. Build context and generate answer
     logger.info("[STEP 4] Building context...")
     t2 = time.time()
-    context = _build_context(hits, strategy_name, document_name)
+    context_hits = _chunks_as_context_hits(retrieved_chunks)
+    context = _build_context(
+        context_hits, strategy_name, document_name, model_id=model.id
+    )
 
     logger.info("[STEP 4] Generating answer...")
     client = get_openai_client()
@@ -284,20 +501,16 @@ def run_query(
 
     t_total = time.time() - t_start
 
-    # 5. Prepare retrieved chunks for storage
+    # 5. Log retrieved chunks
     logger.info("[STEP 5] Preparing retrieved chunks for storage...")
-    retrieved_chunks = []
+    for chunk_data in retrieved_chunks:
-    for hit in hits:
+        logger.info(
-        payload = hit.get("payload", {})
+            "[STEP 5] Chunk: id=%s, role=%s, score=%s, text_len=%d",
-        chunk_data = {
+            chunk_data["chunk_id"],
-            "chunk_id": payload.get("chunk_id", hit.get("chunk_id")),
+            chunk_data.get("role"),
-            "score": hit.get("score", 0),
+            chunk_data.get("score"),
-            "text": payload.get("text", ""),
+            len(chunk_data["text"]),
-            "parent_id": payload.get("parent_id"),
+        )
-        }
-        retrieved_chunks.append(chunk_data)
-        logger.info("[STEP 5] Chunk: id=%s, score=%.4f, text_len=%d",
-                   chunk_data["chunk_id"], chunk_data["score"], len(chunk_data["text"]))
 
     # 6. Store query result in SQLite
     logger.info("[STEP 6] Storing query result in SQLite...")
@@ -314,6 +527,7 @@ def run_query(
         question=question,
         answer=answer,
         retrieved_chunks=retrieved_chunks,
+        expansion_tree=expansion_tree,
         latency_breakdown=latency_breakdown,
         token_usage=token_usage,
     )
@@ -326,9 +540,15 @@ def run_query(
         "query_id": query_record["id"],
         "document_id": document_id,
         "strategy": strategy_name.value,
+        "embedding_model_id": model.id,
+        "corpus_embedding_model_id": model.id,
+        "embedding_provider": model.provider.value,
         "question": question,
         "answer": answer,
         "retrieved_chunks": retrieved_chunks,
+        "expansion_tree": expansion_tree,
+        "neighbor_prev": prev_n,
+        "neighbor_next": next_n,
         "latency_breakdown": latency_breakdown,
         "token_usage": token_usage,
         "created_at": query_record["created_at"],

src/benchmarking/report.py

@@ -175,6 +175,8 @@ def generate_managerial_report(experiment: dict) -> str:
             <div class="meta">
                 {config.get('num_questions', 0)} questions &middot;
                 {len(strategies)} strategies &middot;
+                top_k={config.get('top_k', 5)} &middot;
+                neighbors={config.get('neighbor_prev', 0)}/{config.get('neighbor_next', 0)} &middot;
                 ${estimated_cost:.4f} cost{source_meta}
             </div>
         </header>
@@ -247,6 +249,13 @@ def generate_managerial_report(experiment: dict) -> str:
             {_generate_decision_insights(rankings, aggregate)}
         </div>
 
+        <!-- Expansion Tree -->
+        <div class="section-header">
+            <h2>Neighbor Expansion Tree</h2>
+            <div class="line"></div>
+        </div>
+        {_build_expansion_tree_section(per_question, strategies, compact=True)}
+
         <!-- Quick Links -->
         <div class="section-header">
             <h2>Details</h2>
@@ -306,7 +315,9 @@ def generate_technical_report(experiment: dict) -> str:
             <div class="meta">
                 Experiment: {experiment.get('id', 'N/A')[:16]}... &middot;
                 {total_questions} questions &middot;
-                {len(strategies)} strategies{source_meta}
+                {len(strategies)} strategies &middot;
+                top_k={top_k} &middot;
+                neighbors={config.get('neighbor_prev', 0)}/{config.get('neighbor_next', 0)}{source_meta}
             </div>
         </header>
 
@@ -444,6 +455,13 @@ def generate_technical_report(experiment: dict) -> str:
             </table>
         </div>
 
+        <!-- Expansion Tree -->
+        <div class="section-header">
+            <h2>Neighbor Expansion Tree</h2>
+            <div class="line"></div>
+        </div>
+        {_build_expansion_tree_section(per_question, strategies, compact=False)}
+
         <!-- Quick Links -->
         <div class="section-header">
             <h2>Navigation</h2>
@@ -977,6 +995,112 @@ def _build_token_rows(per_question: list, strategies: list) -> str:
     return rows
 
 
+def _question_has_expansion_tree(qr: dict, strategies: list) -> bool:
+    for strategy in strategies:
+        tree = (qr.get("strategies") or {}).get(strategy, {}).get("expansion_tree") or []
+        if tree:
+            return True
+    return False
+
+
+def _build_expansion_tree_section(
+    per_question: list,
+    strategies: list,
+    *,
+    compact: bool = False,
+) -> str:
+    """Render Expansion Tree HTML for fixed_size (and any strategy that has a tree).
+
+    Compact (managerial) mode shows the first question that has tree data,
+    skipping earlier failed/empty questions.
+    """
+    blocks: list[str] = []
+    first_tree_index: int | None = None
+
+    for qi, qr in enumerate(per_question):
+        if compact and blocks:
+            break
+        if compact and not _question_has_expansion_tree(qr, strategies):
+            continue
+
+        q_id = html.escape(str(qr.get("question_id", "")))
+        q_text = html.escape(str(qr.get("question", ""))[:120])
+        for strategy in strategies:
+            strat = qr.get("strategies", {}).get(strategy, {})
+            tree = strat.get("expansion_tree") or []
+            if not tree:
+                continue
+            if first_tree_index is None:
+                first_tree_index = qi
+            hit_blocks = []
+            for i, node in enumerate(tree):
+                prev_html = "".join(
+                    f'<div style="padding:4px 8px;margin:2px 0;border-left:3px solid var(--text-muted);'
+                    f'color:var(--text-muted);font-size:12px;">'
+                    f'<span class="pill mid">↑ prev {html.escape(str(n.get("chunk_index", "?")))}</span> '
+                    f'{html.escape(str(n.get("text", ""))[:160])}</div>'
+                    for n in (node.get("neighbors_prev") or [])
+                )
+                next_html = "".join(
+                    f'<div style="padding:4px 8px;margin:2px 0;border-left:3px solid var(--text-muted);'
+                    f'color:var(--text-muted);font-size:12px;">'
+                    f'<span class="pill mid">↓ next {html.escape(str(n.get("chunk_index", "?")))}</span> '
+                    f'{html.escape(str(n.get("text", ""))[:160])}</div>'
+                    for n in (node.get("neighbors_next") or [])
+                )
+                score = node.get("score")
+                score_s = f"{score:.3f}" if isinstance(score, (int, float)) else "—"
+                hit_blocks.append(
+                    f'<div style="border:1px solid var(--border);border-radius:8px;'
+                    f'padding:10px;margin:8px 0;background:var(--surface);">'
+                    f'<div style="display:flex;justify-content:space-between;margin-bottom:6px;">'
+                    f'<span class="pill high">Hit #{i + 1}</span>'
+                    f'<span class="pill mid">{score_s}</span></div>'
+                    f'{prev_html}'
+                    f'<div style="padding:6px 8px;margin:2px 0;border-left:3px solid var(--accent);font-size:13px;">'
+                    f'<span class="pill high">● hit {html.escape(str(node.get("chunk_index", "?")))}</span> '
+                    f'{html.escape(str(node.get("text", ""))[:220])}</div>'
+                    f'{next_html}</div>'
+                )
+            blocks.append(
+                f'<div style="margin-bottom:16px;">'
+                f'<div style="font-size:13px;color:var(--text-muted);margin-bottom:6px;">'
+                f'<strong>{q_id}</strong> · {html.escape(str(strategy))} · {q_text}</div>'
+                f'{"".join(hit_blocks)}</div>'
+            )
+
+        if not compact:
+            continue
+        # compact: stop after first question that contributed blocks
+        if blocks:
+            break
+
+    if not blocks:
+        return (
+            '<p style="color:var(--text-muted);font-size:13px;">'
+            "No Expansion Tree data stored for this Experiment "
+            "(Neighbor Expansion off, all questions failed before a tree was saved, "
+            "or no fixed_size results).</p>"
+        )
+
+    note = ""
+    if compact and len(per_question) > 1:
+        if first_tree_index and first_tree_index > 0:
+            note = (
+                '<p style="color:var(--text-muted);font-size:12px;margin-bottom:8px;">'
+                f"Showing first question with Expansion Tree data "
+                f"(skipped {first_tree_index} earlier question(s) with errors or empty trees). "
+                "Open technical view for all questions.</p>"
+            )
+        else:
+            note = (
+                '<p style="color:var(--text-muted);font-size:12px;margin-bottom:8px;">'
+                "Showing first question with Expansion Tree data. "
+                "Open technical view for all questions.</p>"
+            )
+    return note + "".join(blocks)
+
+
 def _pill(score: float) -> str:
     """Create a score pill."""
     if score >= 8:

src/benchmarking/routes.py

@@ -42,6 +42,9 @@ async def create_query(request: QueryRequest):
         strategy_name=request.strategy,
         question=request.question,
         top_k=request.top_k,
+        neighbor_prev=request.neighbor_prev,
+        neighbor_next=request.neighbor_next,
+        corpus_model_id=request.corpus_model_id,
     )
 
     return QueryResponse(
@@ -53,6 +56,9 @@ async def create_query(request: QueryRequest):
         retrieved_chunks=[
             RetrievedChunk(**chunk) for chunk in result["retrieved_chunks"]
         ],
+        expansion_tree=result.get("expansion_tree") or [],
+        neighbor_prev=result.get("neighbor_prev", 0),
+        neighbor_next=result.get("neighbor_next", 0),
         latency_breakdown=result["latency_breakdown"],
         token_usage=result["token_usage"],
         created_at=result["created_at"],
@@ -73,6 +79,7 @@ async def get_query(query_id: str):
         question=result["question"],
         answer=result["answer"],
         retrieved_chunks=result["retrieved_chunks"],
+        expansion_tree=result.get("expansion_tree") or [],
         latency_breakdown=result["latency_breakdown"],
         token_usage=result["token_usage"],
         created_at=result["created_at"],
@@ -130,6 +137,9 @@ async def create_benchmark(request: BenchmarkRequest):
         strategies=request.strategies,
         questions=questions,
         top_k=request.top_k,
+        neighbor_prev=request.neighbor_prev,
+        neighbor_next=request.neighbor_next,
+        corpus_model_id=request.corpus_model_id,
     )
 
     return BenchmarkResponse(
@@ -183,6 +193,11 @@ async def list_experiments(document_id: str | None = None):
     for item in result.get("items", []):
         doc = db.get_document(item.get("document_id", ""))
         item["document_filename"] = doc.get("filename", "Unknown") if doc else "Deleted"
+        questions = item.get("questions") or []
+        item["questions_count"] = (
+            item.get("benchmark_config", {}).get("num_questions")
+            or (len(questions) if isinstance(questions, list) else 0)
+        )
         # Calculate best_strategy from aggregate_metrics
         aggs = item.get("aggregate_metrics", {})
         best_strat, best_score = "N/A", -1
@@ -195,6 +210,11 @@ async def list_experiments(document_id: str | None = None):
                 best_score = adjusted
                 best_strat = strat
         item["best_strategy"] = best_strat
+        # Surface embedding on list even if only in benchmark_config
+        if not item.get("embedding_model_id"):
+            cfg = item.get("benchmark_config") or {}
+            item["embedding_model_id"] = cfg.get("embedding_model_id")
+            item["embedding_provider"] = cfg.get("embedding_provider")
     return result
 
 

src/chunking/base.py

@@ -55,13 +55,36 @@ def build_chunk(
 
 # ── Sentence splitting ────────────────────────────────────────────
 
-_SENTENCE_RE = re.compile(r"(?<=[.!?])\s+(?=[A-Z0-9])")
+# After . ! ? or Persian/fullwidth ؟ ！, split on following whitespace.
+# Does NOT require a Latin capital next (that broke Farsi documents).
+# Periods inside numbers (12.5) are safe because there is no whitespace after.
+_SENTENCE_RE = re.compile(r"(?<=[.!?؟！])\s+")
 
 
 def split_sentences(text: str) -> list[str]:
-    """Split text into sentences using a simple regex heuristic."""
+    """Split text into sentence-like units for Semantic Boundary Detection.
-    sentences = _SENTENCE_RE.split(text.strip())
+
-    return [s.strip() for s in sentences if s.strip()]
+    Primary: punctuation-based splits (English + Farsi terminators).
+    Fallback: if that yields a single unit, use non-empty lines, then
+    blank-line paragraphs — so table/list docs still get multiple units.
+    """
+    text = text.strip()
+    if not text:
+        return []
+
+    sentences = [s.strip() for s in _SENTENCE_RE.split(text) if s.strip()]
+    if len(sentences) > 1:
+        return sentences
+
+    lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
+    if len(lines) > 1:
+        return lines
+
+    paragraphs = [p.strip() for p in re.split(r"\n\s*\n", text) if p.strip()]
+    if len(paragraphs) > 1:
+        return paragraphs
+
+    return sentences if sentences else [text]
 
 
 # ── Abstract base ─────────────────────────────────────────────────

src/chunking/embedding.py

@@ -1,68 +1,219 @@
-"""OpenAI embedding service using text-embedding-3-small.
+"""Embedding service with Cloud (OpenAI) and Local (Ollama) Providers.
 
-All strategies share the same embedding model (fixed, not configurable)
+ADR-0024: two roles — Boundary (semantic cuts) and Corpus (storage + query).
-to ensure fair comparison.  Batch support up to 2048 texts per call.
+Callers snapshot both at operation start so a mid-flight Admin switch cannot mix models.
+Legacy Active Embedding Model maps to Corpus (and migrates into both defaults).
 """
 
 from __future__ import annotations
 
 import logging
+from typing import Literal
 
-from openai import OpenAI
+from src.chunking.embedding_models import (
-
+    EmbeddingModelSpec,
-from src.core.config import settings
+    Provider,
-from src.core.dependencies import get_openai_client
+    apply_task_prefixes,
+    default_model_id,
+    get_model,
+)
+from src.core.dependencies import get_ollama_client, get_openai_client
 from src.core.exceptions import EmbeddingError
+from src.storage import sqlite as db
 
 logger = logging.getLogger(__name__)
 
-# OpenAI batch limit for text-embedding-3-small
+_OPENAI_BATCH_SIZE = 2048
-_BATCH_SIZE = 2048
+_OLLAMA_BATCH_SIZE = 64
+
+_CORPUS_SETTING = "corpus_embedding_model_id"
+_BOUNDARY_SETTING = "boundary_embedding_model_id"
+_LEGACY_ACTIVE_SETTING = "active_embedding_model_id"
+
+Purpose = Literal["document", "query"]
 
 
-def embed_texts(texts: list[str]) -> list[list[float]]:
+def _ensure_role_defaults_migrated() -> None:
-    """Embed a list of texts and return their vectors.
+    """One-shot: legacy Active → Corpus + Boundary when role keys unset."""
+    corpus = db.get_app_setting(_CORPUS_SETTING)
+    boundary = db.get_app_setting(_BOUNDARY_SETTING)
+    if corpus and boundary:
+        return
+    legacy = db.get_app_setting(_LEGACY_ACTIVE_SETTING) or default_model_id()
+    if not corpus:
+        db.set_app_setting(_CORPUS_SETTING, legacy)
+    if not boundary:
+        db.set_app_setting(_BOUNDARY_SETTING, legacy)
 
-    For the contextual_structure strategy, these are the enriched texts
-    (not raw content) — this is by design (ADR 0011).
 
-    Args:
+def _resolve_role(setting_key: str) -> EmbeddingModelSpec:
-        texts: List of strings to embed.
+    _ensure_role_defaults_migrated()
+    stored = db.get_app_setting(setting_key)
+    model_id = stored or default_model_id()
+    try:
+        return get_model(model_id)
+    except KeyError:
+        logger.warning(
+            "Stored Embedding Model '%s' (%s) not in registry; falling back to %s",
+            model_id,
+            setting_key,
+            default_model_id(),
+        )
+        return get_model(default_model_id())
 
-    Returns:
-        List of embedding vectors (same order as input).
 
-    Raises:
+def get_corpus_embedding_model() -> EmbeddingModelSpec:
-        EmbeddingError: If the OpenAI API call fails.
+    """Default Corpus Embedding Model (storage + query)."""
+    return _resolve_role(_CORPUS_SETTING)
+
+
+def get_boundary_embedding_model() -> EmbeddingModelSpec:
+    """Default Boundary Embedding Model (semantic cuts)."""
+    return _resolve_role(_BOUNDARY_SETTING)
+
+
+def set_corpus_embedding_model(model_id: str) -> EmbeddingModelSpec:
+    """Persist Default Corpus. Raises KeyError if unknown."""
+    _ensure_role_defaults_migrated()
+    model = get_model(model_id)
+    db.set_app_setting(_CORPUS_SETTING, model.id)
+    # Keep legacy key in sync for older readers
+    db.set_app_setting(_LEGACY_ACTIVE_SETTING, model.id)
+    logger.info("Corpus Embedding Model set to %s (%s)", model.id, model.provider.value)
+    return model
+
+
+def set_boundary_embedding_model(model_id: str) -> EmbeddingModelSpec:
+    """Persist Default Boundary. Raises KeyError if unknown."""
+    _ensure_role_defaults_migrated()
+    model = get_model(model_id)
+    db.set_app_setting(_BOUNDARY_SETTING, model.id)
+    logger.info("Boundary Embedding Model set to %s (%s)", model.id, model.provider.value)
+    return model
+
+
+def resolve_corpus_model(model_id: str | None = None) -> EmbeddingModelSpec:
+    """Snapshot Corpus for an operation (explicit id or Admin default)."""
+    if model_id:
+        return get_model(model_id)
+    return get_corpus_embedding_model()
+
+
+def resolve_boundary_model(model_id: str | None = None) -> EmbeddingModelSpec:
+    """Snapshot Boundary for an operation (explicit id or Admin default)."""
+    if model_id:
+        return get_model(model_id)
+    return get_boundary_embedding_model()
+
+
+# ── Legacy aliases (Corpus) ───────────────────────────────────────
+
+def get_active_embedding_model() -> EmbeddingModelSpec:
+    """Deprecated: alias for get_corpus_embedding_model (ADR-0024)."""
+    return get_corpus_embedding_model()
+
+
+def set_active_embedding_model(model_id: str) -> EmbeddingModelSpec:
+    """Deprecated: sets Corpus default (and legacy active key)."""
+    return set_corpus_embedding_model(model_id)
+
+
+def snapshot_active_model() -> EmbeddingModelSpec:
+    """Deprecated: snapshot Corpus Embedding Model."""
+    return get_corpus_embedding_model()
+
+
+def embed_texts(
+    texts: list[str],
+    *,
+    model: EmbeddingModelSpec | None = None,
+    purpose: Purpose = "document",
+) -> list[list[float]]:
+    """Embed texts with the given (or Corpus) Embedding Model.
+
+    For contextual_retrieval, pass enriched texts — ADR 0011.
     """
     if not texts:
         return []
 
+    model = model or get_corpus_embedding_model()
+    prepared = apply_task_prefixes(texts, model=model, purpose=purpose)
+
+    try:
+        if model.provider == Provider.CLOUD:
+            return _embed_openai(prepared, model)
+        if model.provider == Provider.LOCAL:
+            return _embed_ollama(prepared, model)
+        raise EmbeddingError(f"Unsupported Provider: {model.provider}")
+    except EmbeddingError:
+        raise
+    except Exception as exc:
+        raise EmbeddingError(f"Embedding failed ({model.id}): {exc}") from exc
+
+
+def embed_single(
+    text: str,
+    *,
+    model: EmbeddingModelSpec | None = None,
+    purpose: Purpose = "query",
+) -> list[float]:
+    """Embed a single text (default purpose=query for retrieval)."""
+    results = embed_texts([text], model=model, purpose=purpose)
+    return results[0]
+
+
+def _embed_openai(texts: list[str], model: EmbeddingModelSpec) -> list[list[float]]:
     client = get_openai_client()
     all_embeddings: list[list[float]] = []
 
-    try:
+    for start in range(0, len(texts), _OPENAI_BATCH_SIZE):
-        for start in range(0, len(texts), _BATCH_SIZE):
+        batch = texts[start : start + _OPENAI_BATCH_SIZE]
-            batch = texts[start:start + _BATCH_SIZE]
+        response = client.embeddings.create(
-            response = client.embeddings.create(
+            model=model.model_name,
-                model=settings.embedding_model,
+            input=batch,
-                input=batch,
+        )
+        sorted_data = sorted(response.data, key=lambda x: x.index)
+        vectors = [item.embedding for item in sorted_data]
+        _validate_dimensions(vectors, model)
+        all_embeddings.extend(vectors)
+        logger.debug(
+            "OpenAI embedded batch %d-%d with %s",
+            start,
+            start + len(batch),
+            model.model_name,
+        )
+
+    return all_embeddings
+
+
+def _embed_ollama(texts: list[str], model: EmbeddingModelSpec) -> list[list[float]]:
+    client = get_ollama_client()
+    all_embeddings: list[list[float]] = []
+
+    for start in range(0, len(texts), _OLLAMA_BATCH_SIZE):
+        batch = texts[start : start + _OLLAMA_BATCH_SIZE]
+        response = client.embeddings.create(
+            model=model.model_name,
+            input=batch,
+        )
+        sorted_data = sorted(response.data, key=lambda x: x.index)
+        vectors = [item.embedding for item in sorted_data]
+        _validate_dimensions(vectors, model)
+        all_embeddings.extend(vectors)
+        logger.debug(
+            "Ollama embedded batch %d-%d with %s",
+            start,
+            start + len(batch),
+            model.model_name,
+        )
+
+    return all_embeddings
+
+
+def _validate_dimensions(vectors: list[list[float]], model: EmbeddingModelSpec) -> None:
+    for i, vec in enumerate(vectors):
+        if len(vec) != model.dimension:
+            raise EmbeddingError(
+                f"Embedding dimension mismatch for {model.id}: "
+                f"expected {model.dimension}, got {len(vec)} (index {i})"
             )
-            # Sort by index to guarantee order matches input
-            sorted_data = sorted(response.data, key=lambda x: x.index)
-            all_embeddings.extend([item.embedding for item in sorted_data])
-
-            logger.debug(
-                "Embedded batch %d-%d (%d texts)",
-                start, start + len(batch), len(batch),
-            )
-
-        return all_embeddings
-    except Exception as exc:
-        raise EmbeddingError(f"Embedding failed: {exc}") from exc
-
-
-def embed_single(text: str) -> list[float]:
-    """Embed a single text (convenience wrapper)."""
-    results = embed_texts([text])
-    return results[0]

src/chunking/embedding_models.py

@@ -0,0 +1,153 @@
+"""Embedding Model Registry — static catalog of Cloud and Local models."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from enum import Enum
+
+
+class Provider(str, Enum):
+    CLOUD = "cloud"
+    LOCAL = "local"
+
+
+@dataclass(frozen=True)
+class EmbeddingModelSpec:
+    """One entry in the Embedding Model Registry."""
+
+    id: str
+    provider: Provider
+    model_name: str
+    dimension: int
+    display_name: str
+    task_prefixes: bool = False
+    # Default Semantic Boundary Detection threshold (Admin may override in SQLite)
+    default_semantic_threshold: float = 0.3
+
+
+# Stable id for legacy unscoped OpenAI corpora / historical Experiment backfill (1536-d).
+LEGACY_CLOUD_MODEL_ID = "text-embedding-3-small"
+# Default Corpus/Boundary when Admin settings are unset (new installs & fallbacks).
+DEFAULT_CLOUD_MODEL_ID = "text-embedding-3-large"
+
+_SEMANTIC_THRESHOLD_SETTING_PREFIX = "semantic_threshold:"
+
+
+def semantic_threshold_setting_key(model_id: str) -> str:
+    return f"{_SEMANTIC_THRESHOLD_SETTING_PREFIX}{model_id}"
+
+
+def _build_registry() -> dict[str, EmbeddingModelSpec]:
+    # Prefer explicit registry entries over EMBEDDING_MODEL aliasing a different
+    # OpenAI model under a single id (wrong dimension / provenance).
+    cloud_small = EmbeddingModelSpec(
+        id=LEGACY_CLOUD_MODEL_ID,
+        provider=Provider.CLOUD,
+        model_name="text-embedding-3-small",
+        dimension=1536,
+        display_name="OpenAI text-embedding-3-small",
+        task_prefixes=False,
+        default_semantic_threshold=0.3,
+    )
+    cloud_large = EmbeddingModelSpec(
+        id=DEFAULT_CLOUD_MODEL_ID,
+        provider=Provider.CLOUD,
+        model_name="text-embedding-3-large",
+        dimension=3072,
+        display_name="OpenAI text-embedding-3-large",
+        task_prefixes=False,
+        default_semantic_threshold=0.3,
+    )
+    local = EmbeddingModelSpec(
+        id="nomic-embed-text-v2-moe",
+        provider=Provider.LOCAL,
+        model_name="nomic-embed-text-v2-moe:latest",
+        dimension=768,
+        display_name="Ollama nomic-embed-text-v2-moe",
+        task_prefixes=True,
+        default_semantic_threshold=0.6,
+    )
+    return {
+        cloud_small.id: cloud_small,
+        cloud_large.id: cloud_large,
+        local.id: local,
+    }
+
+
+def get_registry() -> dict[str, EmbeddingModelSpec]:
+    """Return the Embedding Model Registry (built from config)."""
+    return _build_registry()
+
+
+def get_model(model_id: str) -> EmbeddingModelSpec:
+    """Look up a registry entry by stable id.
+
+    Raises:
+        KeyError: If model_id is not registered.
+    """
+    registry = get_registry()
+    if model_id not in registry:
+        known = ", ".join(sorted(registry))
+        raise KeyError(f"Unknown Embedding Model '{model_id}'. Registered: {known}")
+    return registry[model_id]
+
+
+def list_models() -> list[EmbeddingModelSpec]:
+    """All registered Embedding Models in stable order (default cloud first)."""
+    registry = get_registry()
+    preferred = [
+        DEFAULT_CLOUD_MODEL_ID,
+        LEGACY_CLOUD_MODEL_ID,
+        "nomic-embed-text-v2-moe",
+    ]
+    order = preferred + [mid for mid in registry if mid not in preferred]
+    return [registry[mid] for mid in order if mid in registry]
+
+
+def default_model_id() -> str:
+    """Default Corpus/Boundary Embedding Model id when Admin settings are unset."""
+    return DEFAULT_CLOUD_MODEL_ID
+
+
+def get_semantic_threshold(model_id: str) -> float:
+    """Resolve Semantic Boundary Detection threshold for a model.
+
+    Order: Admin SQLite override → registry default → global SEMANTIC_THRESHOLD.
+    """
+    from src.storage import sqlite as db
+
+    model = get_model(model_id)
+    stored = db.get_app_setting(semantic_threshold_setting_key(model_id))
+    if stored is not None and stored != "":
+        return float(stored)
+    return model.default_semantic_threshold
+
+
+def set_semantic_threshold(model_id: str, threshold: float) -> float:
+    """Persist Admin override for a model's semantic_threshold (0 < t <= 1)."""
+    from src.storage import sqlite as db
+
+    get_model(model_id)  # validate registry id
+    if not (0.0 < threshold <= 1.0):
+        raise ValueError("semantic_threshold must be in (0, 1]")
+    db.set_app_setting(semantic_threshold_setting_key(model_id), str(threshold))
+    return threshold
+
+
+def apply_task_prefixes(
+    texts: list[str],
+    *,
+    model: EmbeddingModelSpec,
+    purpose: str,
+) -> list[str]:
+    """Apply Nomic-style task prefixes when the model requires them.
+
+    purpose: \"document\" → search_document; \"query\" → search_query.
+    """
+    if not model.task_prefixes:
+        return texts
+    if purpose == "query":
+        prefix = "search_query: "
+    else:
+        prefix = "search_document: "
+    return [prefix + t if not t.startswith(prefix) else t for t in texts]

src/chunking/service.py

@@ -4,7 +4,8 @@ Runs selected strategies on a document, embeds chunks, and stores
 them in Qdrant.  Per-strategy failure isolation (ADR 0003): if one
 strategy fails, the others' results are still committed.
 
-This replaces the stub in src/documents/service.py.
+ADR-0024: Boundary Embedding Model for semantic cuts; Corpus Embedding
+Model for finished-chunk vectors and collection scoping.
 """
 
 from __future__ import annotations
@@ -12,16 +13,19 @@ from __future__ import annotations
 import logging
 import time
 
-from src.chunking.base import ChunkingStrategy
+from src.chunking.base import ChunkingStrategy, split_sentences
 from src.chunking.embedding import embed_texts
+from src.chunking.embedding_models import EmbeddingModelSpec, get_semantic_threshold
 from src.chunking.strategies.recursive import RecursiveStrategy
 from src.chunking.strategies.fixed_size import FixedSizeStrategy
 from src.chunking.strategies.semantic import SemanticStrategy
 from src.chunking.strategies.contextual_retrieval import ContextualRetrievalStrategy
-from src.chunking.strategies.semantic_parent_child import SemanticParentChildStrategy
+from src.chunking.strategies.semantic_parent_child import (
+    SemanticParentChildStrategy,
+    split_paragraphs,
+)
 from src.core.exceptions import ChunkingError
 from src.core.models import (
-    Chunk,
     DocumentTree,
     StrategyName,
 )
@@ -30,6 +34,11 @@ from src.storage import sqlite as db
 
 logger = logging.getLogger(__name__)
 
+_BOUNDARY_STRATEGIES = {
+    StrategyName.SEMANTIC,
+    StrategyName.SEMANTIC_PARENT_CHILD,
+}
+
 # ── Strategy registry ─────────────────────────────────────────────
 
 _STRATEGIES: dict[StrategyName, ChunkingStrategy] = {
@@ -48,6 +57,73 @@ def _get_strategy(name: StrategyName) -> ChunkingStrategy:
     return s
 
 
+def _chunk_document(
+    strategy_name: StrategyName,
+    strategy: ChunkingStrategy,
+    *,
+    doc_name: str,
+    tree: DocumentTree,
+    markdown: str,
+    boundary_model: EmbeddingModelSpec,
+) -> list:
+    """Run strategy.chunk(), supplying Boundary embeds when needed."""
+    if strategy_name == StrategyName.SEMANTIC:
+        sentences = split_sentences(markdown)
+        if not sentences:
+            return []
+        threshold = get_semantic_threshold(boundary_model.id)
+        logger.info(
+            "Strategy semantic: embedding %d sentences for boundary detection "
+            "(%s, threshold=%.3f)",
+            len(sentences),
+            boundary_model.id,
+            threshold,
+        )
+        sentence_embeddings = embed_texts(
+            sentences,
+            model=boundary_model,
+            purpose="document",
+        )
+        return strategy.chunk(  # type: ignore[call-arg]
+            doc_name=doc_name,
+            tree=tree,
+            markdown=markdown,
+            sentence_embeddings=sentence_embeddings,
+            semantic_threshold=threshold,
+        )
+
+    if strategy_name == StrategyName.SEMANTIC_PARENT_CHILD:
+        paragraphs = split_paragraphs(markdown)
+        if not paragraphs:
+            return []
+        threshold = get_semantic_threshold(boundary_model.id)
+        logger.info(
+            "Strategy semantic_parent_child: embedding %d paragraphs for boundary detection "
+            "(%s, threshold=%.3f)",
+            len(paragraphs),
+            boundary_model.id,
+            threshold,
+        )
+        paragraph_embeddings = embed_texts(
+            paragraphs,
+            model=boundary_model,
+            purpose="document",
+        )
+        return strategy.chunk(  # type: ignore[call-arg]
+            doc_name=doc_name,
+            tree=tree,
+            markdown=markdown,
+            paragraph_embeddings=paragraph_embeddings,
+            semantic_threshold=threshold,
+        )
+
+    return strategy.chunk(
+        doc_name=doc_name,
+        tree=tree,
+        markdown=markdown,
+    )
+
+
 # ── Single-strategy runner ────────────────────────────────────────
 
 def _run_strategy(
@@ -56,6 +132,9 @@ def _run_strategy(
     doc_name: str,
     tree: DocumentTree,
     markdown: str,
+    *,
+    boundary_model: EmbeddingModelSpec,
+    corpus_model: EmbeddingModelSpec,
 ) -> int:
     """Run one strategy: chunk → embed → store in Qdrant.
 
@@ -64,16 +143,21 @@ def _run_strategy(
     """
     strategy = _get_strategy(strategy_name)
 
-    # Ensure Qdrant collection exists
+    qdr.ensure_collection(
-    qdr.ensure_collection(strategy_name)
+        strategy_name,
+        model_id=corpus_model.id,
+        dimension=corpus_model.dimension,
+    )
 
     t0 = time.time()
 
-    # Step 1: Chunk
+    chunks = _chunk_document(
-    chunks = strategy.chunk(
+        strategy_name,
+        strategy,
         doc_name=doc_name,
         tree=tree,
         markdown=markdown,
+        boundary_model=boundary_model,
     )
 
     if not chunks:
@@ -86,8 +170,6 @@ def _run_strategy(
         strategy_name.value, len(chunks), t_chunk,
     )
 
-    # Step 2: Embed
-    # For contextual strategy, embed enriched_content; for others, embed text
     texts_to_embed = []
     for chunk in chunks:
         if chunk.enriched_content:
@@ -96,16 +178,19 @@ def _run_strategy(
             texts_to_embed.append(chunk.text)
 
     t1 = time.time()
-    embeddings = embed_texts(texts_to_embed)
+    embeddings = embed_texts(
+        texts_to_embed,
+        model=corpus_model,
+        purpose="document",
+    )
     t_embed = time.time() - t1
     logger.info(
-        "Strategy %s: embedded %d texts in %.2fs",
+        "Strategy %s: embedded %d texts in %.2fs (corpus=%s)",
-        strategy_name.value, len(embeddings), t_embed,
+        strategy_name.value, len(embeddings), t_embed, corpus_model.id,
     )
 
-    # Step 3: Upsert to Qdrant
     t2 = time.time()
-    stored = qdr.upsert_chunks(chunks, embeddings)
+    stored = qdr.upsert_chunks(chunks, embeddings, model_id=corpus_model.id)
     t_store = time.time() - t2
     logger.info(
         "Strategy %s: stored %d vectors in %.2fs",
@@ -120,17 +205,39 @@ def _run_strategy(
 def run_strategies(
     doc_id: str,
     strategies: list[StrategyName],
+    *,
+    boundary_model_id: str | None = None,
+    corpus_model_id: str | None = None,
 ) -> tuple[list[dict], list[dict]]:
     """Run multiple strategies on a document with per-strategy failure isolation.
 
     Returns:
         (completed, failed) — lists of result dicts.
     """
+    from src.chunking.embedding import resolve_boundary_model, resolve_corpus_model
+
     doc = db.get_document(doc_id)
     if doc is None:
         raise ChunkingError(f"Document not found: {doc_id}")
 
-    # Parse the stored document tree (may be dict or JSON string)
+    corpus_model = resolve_corpus_model(corpus_model_id)
+    needs_boundary = any(s in _BOUNDARY_STRATEGIES for s in strategies)
+    boundary_model = (
+        resolve_boundary_model(boundary_model_id)
+        if needs_boundary
+        else corpus_model  # unused for non-semantic; keep a valid spec
+    )
+
+    logger.info(
+        "Processing doc %s corpus=%s (%s) boundary=%s (%s) needs_boundary=%s",
+        doc_id,
+        corpus_model.id,
+        corpus_model.provider.value,
+        boundary_model.id if needs_boundary else "—",
+        boundary_model.provider.value if needs_boundary else "—",
+        needs_boundary,
+    )
+
     tree_raw = doc["document_tree"]
     if isinstance(tree_raw, str):
         tree = DocumentTree.model_validate_json(tree_raw)
@@ -151,14 +258,21 @@ def run_strategies(
                 doc_name=doc_name,
                 tree=tree,
                 markdown=markdown,
+                boundary_model=boundary_model,
+                corpus_model=corpus_model,
             )
             elapsed = time.time() - t0
-            completed.append({
+            entry = {
                 "strategy": strategy_name,
                 "status": "completed",
                 "chunks_produced": chunks_produced,
                 "elapsed_seconds": round(elapsed, 2),
-            })
+                "corpus_embedding_model_id": corpus_model.id,
+                "embedding_model_id": corpus_model.id,  # legacy alias
+            }
+            if strategy_name in _BOUNDARY_STRATEGIES:
+                entry["boundary_embedding_model_id"] = boundary_model.id
+            completed.append(entry)
         except Exception as exc:
             logger.error(
                 "Strategy %s failed for doc %s: %s",
@@ -170,10 +284,16 @@ def run_strategies(
                 "error": str(exc),
             })
 
-    # Update chunk counts on the document
     counts = doc.get("chunk_counts", {})
     for result in completed:
         counts[result["strategy"].value] = result["chunks_produced"]
     db.update_chunk_counts(doc_id, counts)
 
+    if completed:
+        db.update_process_embedding_provenance(
+            doc_id,
+            corpus_model_id=corpus_model.id,
+            boundary_model_id=boundary_model.id if needs_boundary else None,
+        )
+
     return completed, failed

src/chunking/strategies/semantic.py

@@ -1,17 +1,15 @@
 """Semantic chunking strategy.
 
-Sentence-level granularity (ADR 0012):
+Sentence-level granularity (ADR 0012 / ADR 0020):
   1. Split markdown into sentences.
-  2. Embed each sentence via OpenAI.
+  2. Orchestrator embeds each sentence (Active Embedding Model).
   3. Compute cosine similarity between adjacent sentences.
-  4. When similarity drops below SEMANTIC_THRESHOLD, create a chunk boundary.
+  4. When similarity drops below the Active Embedding Model's semantic_threshold, create a chunk boundary.
   5. Enforce SEMANTIC_MIN_CHUNK_SIZE (minimum sentences per chunk).
   6. Boundary sentence stays with the previous chunk.
+  7. Orchestrator embeds finished chunks for Qdrant storage.
 
-Note: this strategy requires embeddings at chunk-time.  The chunk()
+Semantic Boundary Detection is required — no fixed-count fallback.
-method returns text chunks WITHOUT embeddings — the embedding step
-happens in the orchestration layer (service.py) which calls the
-embedding service after chunking.
 """
 
 from __future__ import annotations
@@ -20,6 +18,7 @@ import numpy as np
 
 from src.chunking.base import ChunkingStrategy, build_chunk, split_sentences
 from src.core.config import settings
+from src.core.exceptions import ChunkingError
 from src.core.models import Chunk, DocumentTree, StrategyName
 
 
@@ -85,30 +84,33 @@ class SemanticStrategy(ChunkingStrategy):
         tree: DocumentTree,
         markdown: str,
         sentence_embeddings: list[list[float]] | None = None,
+        semantic_threshold: float | None = None,
     ) -> list[Chunk]:
-        """Produce semantic chunks.
+        """Produce semantic chunks via Semantic Boundary Detection.
 
-        If sentence_embeddings is provided (from the orchestration layer),
+        Requires sentence_embeddings aligned 1:1 with split_sentences(markdown).
-        uses them for boundary detection.  Otherwise, falls back to
-        paragraph-level chunking (sentences without similarity-based splits).
         """
         sentences = split_sentences(markdown)
         if not sentences:
             return []
 
-        threshold = settings.semantic_threshold
+        if sentence_embeddings is None or len(sentence_embeddings) != len(sentences):
+            got = 0 if sentence_embeddings is None else len(sentence_embeddings)
+            raise ChunkingError(
+                f"semantic requires sentence embeddings for Semantic Boundary Detection "
+                f"(got {got}, need {len(sentences)}). Fixed-count fallback is disabled (ADR-0020)."
+            )
+
+        threshold = (
+            semantic_threshold
+            if semantic_threshold is not None
+            else settings.semantic_threshold
+        )
         min_size = settings.semantic_min_chunk_size
 
-        if sentence_embeddings and len(sentence_embeddings) == len(sentences):
+        chunk_texts = _group_sentences_into_chunks(
-            chunk_texts = _group_sentences_into_chunks(
+            sentences, sentence_embeddings, threshold, min_size
-                sentences, sentence_embeddings, threshold, min_size
+        )
-            )
-        else:
-            # Fallback: group sentences into fixed-size chunks
-            chunk_texts = []
-            for i in range(0, len(sentences), min_size):
-                group = sentences[i:i + min_size]
-                chunk_texts.append(" ".join(group))
 
         chunks: list[Chunk] = []
         for i, text in enumerate(chunk_texts):

src/chunking/strategies/semantic_parent_child.py

@@ -8,11 +8,13 @@ parent cluster is returned as context — giving the LLM richer
 information than a single paragraph.
 
 No headings or document structure needed — uses meaning instead.
+Semantic Boundary Detection is required — no fixed-count fallback (ADR-0020).
 """
 
 from __future__ import annotations
 
 import logging
+import re
 
 import numpy as np
 
@@ -20,9 +22,9 @@ from src.chunking.base import (
     ChunkingStrategy,
     build_chunk,
     make_chunk_id,
-    count_tokens,
 )
 from src.core.config import settings
+from src.core.exceptions import ChunkingError
 from src.core.models import Chunk, DocumentTree, StrategyName
 
 logger = logging.getLogger(__name__)
@@ -34,10 +36,8 @@ def _cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
     return float(dot / norm) if norm > 0 else 0.0
 
 
-def _split_paragraphs(text: str) -> list[str]:
+def split_paragraphs(text: str) -> list[str]:
     """Split markdown into paragraphs (double newline or single newline)."""
-    import re
-    # Split on double newlines first, then filter empties
     parts = re.split(r"\n\s*\n", text)
     paragraphs = [p.strip() for p in parts if p.strip()]
 
@@ -89,34 +89,35 @@ class SemanticParentChildStrategy(ChunkingStrategy):
         tree: DocumentTree,
         markdown: str,
         paragraph_embeddings: list[list[float]] | None = None,
+        semantic_threshold: float | None = None,
     ) -> list[Chunk]:
-        """Produce parent-child chunks via semantic clustering.
+        """Produce parent-child chunks via Semantic Boundary Detection.
 
-        If paragraph_embeddings is provided (from orchestration layer),
+        Requires paragraph_embeddings aligned 1:1 with split_paragraphs(markdown).
-        uses them for clustering.  Otherwise, groups paragraphs by
-        fixed count.
         """
-        paragraphs = _split_paragraphs(markdown)
+        paragraphs = split_paragraphs(markdown)
         if not paragraphs:
             return []
 
-        threshold = settings.semantic_threshold
+        if paragraph_embeddings is None or len(paragraph_embeddings) != len(paragraphs):
+            got = 0 if paragraph_embeddings is None else len(paragraph_embeddings)
+            raise ChunkingError(
+                f"semantic_parent_child requires paragraph embeddings for "
+                f"Semantic Boundary Detection (got {got}, need {len(paragraphs)}). "
+                f"Fixed-count fallback is disabled (ADR-0020)."
+            )
 
-        if paragraph_embeddings and len(paragraph_embeddings) == len(paragraphs):
+        threshold = (
-            clusters = _cluster_paragraphs(paragraphs, paragraph_embeddings, threshold)
+            semantic_threshold
-        else:
+            if semantic_threshold is not None
-            # Fallback: group every N paragraphs
+            else settings.semantic_threshold
-            group_size = max(3, settings.semantic_min_chunk_size)
+        )
-            clusters = []
+        clusters = _cluster_paragraphs(paragraphs, paragraph_embeddings, threshold)
-            for i in range(0, len(paragraphs), group_size):
-                clusters.append(list(range(i, min(i + group_size, len(paragraphs)))))
 
         chunks: list[Chunk] = []
         chunk_index = 0
 
         for cluster_indices in clusters:
-            # Parent = full cluster text
-            parent_text = "\n\n".join(paragraphs[i] for i in cluster_indices)
             parent_id = make_chunk_id(self.name, doc_name, chunk_index)
 
             # Each paragraph in the cluster is a child

src/core/config.py

@@ -10,15 +10,21 @@ class Settings(BaseSettings):
 
     # OpenAI
     openai_api_key: str
-    embedding_model: str = "text-embedding-3-small"
+    embedding_model: str = "text-embedding-3-large"
     llm_model: str = "gpt-4o-mini"
 
+    # Local embeddings (Ollama) — Admin switches models; host stays in config
+    ollama_base_url: str = "http://192.168.10.10:11435"
+
     # Qdrant
     qdrant_url: str = "http://localhost:6333"
     qdrant_api_key: str | None = None
 
     # Retrieval
     top_k: int = 5
+    # Neighbor Expansion for fixed_size (ADR-0023); 0/0 = off
+    neighbor_prev: int = 0
+    neighbor_next: int = 0
 
     # LLM generation
     temperature: float = 0.0

src/core/dependencies.py

@@ -14,6 +14,13 @@ def get_openai_client() -> OpenAI:
     return OpenAI(api_key=settings.openai_api_key)
 
 
+@lru_cache()
+def get_ollama_client() -> OpenAI:
+    """Return a cached OpenAI-compatible client pointed at Ollama."""
+    base = settings.ollama_base_url.rstrip("/")
+    return OpenAI(base_url=f"{base}/v1", api_key="ollama")
+
+
 @lru_cache()
 def get_qdrant_client() -> QdrantClient:
     """Return a cached Qdrant client singleton."""

src/documents/models.py

@@ -16,6 +16,8 @@ class DocumentResponse(BaseModel):
     paragraph_count: int = 0
     chunk_counts: dict[str, int] = Field(default_factory=dict)
     created_at: str
+    last_corpus_embedding_model_id: Optional[str] = None
+    last_boundary_embedding_model_id: Optional[str] = None
 
 
 class DocumentDetailResponse(DocumentResponse):
@@ -37,6 +39,14 @@ class ProcessRequest(BaseModel):
         description="Which chunking strategies to run (defaults to all 5)",
         min_length=1,
     )
+    boundary_model_id: Optional[str] = Field(
+        default=None,
+        description="Boundary Embedding Model id (semantic cuts); default = Admin Boundary",
+    )
+    corpus_model_id: Optional[str] = Field(
+        default=None,
+        description="Corpus Embedding Model id (storage); default = Admin Corpus",
+    )
 
 
 class StrategyResult(BaseModel):
@@ -52,6 +62,8 @@ class ProcessResponse(BaseModel):
     document_id: str
     strategies_completed: list[StrategyResult]
     strategies_failed: list[StrategyResult]
+    corpus_embedding_model_id: Optional[str] = None
+    boundary_embedding_model_id: Optional[str] = None
 
 
 class DeleteResponse(BaseModel):

src/documents/service.py

@@ -101,7 +101,12 @@ def process_document(
     # Import here to avoid circular imports at module level
     from src.chunking.service import run_strategies
 
-    completed_raw, failed_raw = run_strategies(doc_id, request.strategies)
+    completed_raw, failed_raw = run_strategies(
+        doc_id,
+        request.strategies,
+        boundary_model_id=request.boundary_model_id,
+        corpus_model_id=request.corpus_model_id,
+    )
 
     completed = [
         StrategyResult(
@@ -120,17 +125,28 @@ def process_document(
         for r in failed_raw
     ]
 
+    corpus_id = None
+    boundary_id = None
+    for r in completed_raw:
+        corpus_id = r.get("corpus_embedding_model_id") or corpus_id
+        if r.get("boundary_embedding_model_id"):
+            boundary_id = r["boundary_embedding_model_id"]
+
     return ProcessResponse(
         document_id=doc_id,
         strategies_completed=completed,
         strategies_failed=failed,
+        corpus_embedding_model_id=corpus_id,
+        boundary_embedding_model_id=boundary_id,
     )
 
 
 # ── Delete ─────────────────────────────────────────────────────────
 
 def delete_document(doc_id: str) -> bool:
-    """Delete a document and all its Qdrant vectors."""
+    """Delete a document and all its Qdrant vectors across Model Corpora."""
+    from src.chunking.embedding_models import list_models
+
     doc = db.get_document(doc_id)
     if doc is None:
         return False
@@ -139,7 +155,20 @@ def delete_document(doc_id: str) -> bool:
         if count > 0:
             try:
                 strategy = StrategyName(strategy_name)
-                qdr.delete_document_chunks(strategy, doc["filename"])
+                for model in list_models():
+                    try:
+                        qdr.delete_document_chunks(
+                            strategy,
+                            doc["filename"],
+                            model_id=model.id,
+                        )
+                    except Exception as exc:
+                        logger.warning(
+                            "Failed to delete Qdrant vectors for %s/%s: %s",
+                            strategy_name,
+                            model.id,
+                            exc,
+                        )
             except Exception as exc:
                 logger.warning("Failed to delete Qdrant vectors for %s: %s", strategy_name, exc)
 

src/storage/qdrant.py

@@ -1,75 +1,123 @@
 """Qdrant vector storage layer.
 
-One collection per chunking strategy. Handles collection creation,
+One collection per (Strategy, Embedding Model) — the Model Corpus.
-vector upsert, and similarity search.
+Names are always `{strategy}__{model_id}_collection` (ADR-0021).
+Legacy unscoped `{strategy}_collection` names are recognized for Admin
+labeling only; process/query no longer write them.
 """
 
+from __future__ import annotations
+
 import logging
 import uuid
 from typing import Any
 
-from qdrant_client import QdrantClient
 from qdrant_client.models import (
     Distance,
     FieldCondition,
     Filter,
+    MatchAny,
     MatchValue,
     PointIdsList,
     PointStruct,
     VectorParams,
 )
 
-from src.core.config import settings
+from src.chunking.embedding_models import LEGACY_CLOUD_MODEL_ID
 from src.core.dependencies import get_qdrant_client
 from src.core.exceptions import QdrantError
-from src.core.models import Chunk, ChunkMetadata, StrategyName, chunk_to_metadata
+from src.core.models import Chunk, StrategyName, chunk_to_metadata
 
 logger = logging.getLogger(__name__)
 
-# Embedding dimension for text-embedding-3-small
+# Backward-compat alias — prefer EmbeddingModelSpec.dimension
 VECTOR_DIMENSION = 1536
 
 
-def collection_name(strategy: StrategyName | str) -> str:
+def _sanitize_model_id(model_id: str) -> str:
-    """Convention: {strategy_name}_collection."""
+    return model_id.replace(":", "-").replace("/", "-")
+
+
+def collection_name(strategy: StrategyName | str, model_id: str) -> str:
+    """Model Corpus collection name for a Strategy + Embedding Model.
+
+    Always includes the Embedding Model id (cloud and local).
+    """
     if isinstance(strategy, StrategyName):
         name = strategy.value
     else:
         name = strategy
-    return f"{name}_collection"
+    return f"{name}__{_sanitize_model_id(model_id)}_collection"
+
+
+def parse_collection_meta(name: str) -> dict[str, Any]:
+    """Infer strategy + Embedding Model from a Qdrant collection name."""
+    if not name.endswith("_collection"):
+        return {
+            "strategy": None,
+            "embedding_model_id": None,
+            "is_legacy": False,
+            "recognized": False,
+        }
+    body = name[: -len("_collection")]
+    if "__" in body:
+        strategy, model_id = body.split("__", 1)
+        return {
+            "strategy": strategy,
+            "embedding_model_id": model_id,
+            "is_legacy": False,
+            "recognized": True,
+        }
+    # Pre-ADR-0021 unscoped names (no longer written)
+    return {
+        "strategy": body,
+        "embedding_model_id": LEGACY_CLOUD_MODEL_ID,
+        "is_legacy": True,
+        "recognized": True,
+    }
 
 
 # ── Collection management ──────────────────────────────────────────
 
-def ensure_collection(strategy: StrategyName) -> None:
+def ensure_collection(
-    """Create the collection for a strategy if it doesn't already exist."""
+    strategy: StrategyName,
+    *,
+    model_id: str,
+    dimension: int,
+) -> None:
+    """Create the Model Corpus collection for a strategy if missing."""
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
     try:
         existing = [c.name for c in client.get_collections().collections]
         if name not in existing:
             client.create_collection(
                 collection_name=name,
                 vectors_config=VectorParams(
-                    size=VECTOR_DIMENSION,
+                    size=dimension,
                     distance=Distance.COSINE,
                 ),
             )
-            logger.info("Created Qdrant collection: %s", name)
+            logger.info(
+                "Created Qdrant collection: %s (dim=%d, model=%s)",
+                name,
+                dimension,
+                model_id,
+            )
     except Exception as exc:
         raise QdrantError(f"Failed to create collection '{name}': {exc}") from exc
 
 
-def ensure_all_collections() -> None:
+def ensure_all_collections(*, model_id: str, dimension: int) -> None:
-    """Create collections for all five strategies."""
+    """Create collections for all five strategies under one Embedding Model."""
     for strategy in StrategyName:
-        ensure_collection(strategy)
+        ensure_collection(strategy, model_id=model_id, dimension=dimension)
 
 
-def delete_collection(strategy: StrategyName) -> None:
+def delete_collection(strategy: StrategyName, *, model_id: str) -> None:
-    """Delete a strategy's collection entirely."""
+    """Delete a strategy's collection for an Embedding Model."""
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
     try:
         client.delete_collection(collection_name=name)
         logger.info("Deleted Qdrant collection: %s", name)
@@ -77,10 +125,10 @@ def delete_collection(strategy: StrategyName) -> None:
         raise QdrantError(f"Failed to delete collection '{name}': {exc}") from exc
 
 
-def list_collection_points(strategy: StrategyName) -> int:
+def list_collection_points(strategy: StrategyName, *, model_id: str) -> int:
-    """Return the number of points in a strategy's collection."""
+    """Return the number of points in a strategy's Model Corpus collection."""
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
     try:
         info = client.get_collection(collection_name=name)
         return info.points_count or 0
@@ -90,8 +138,13 @@ def list_collection_points(strategy: StrategyName) -> int:
 
 # ── Upsert ─────────────────────────────────────────────────────────
 
-def upsert_chunks(chunks: list[Chunk], embeddings: list[list[float]]) -> int:
+def upsert_chunks(
-    """Upsert chunks with their embeddings into the appropriate collection.
+    chunks: list[Chunk],
+    embeddings: list[list[float]],
+    *,
+    model_id: str,
+) -> int:
+    """Upsert chunks with their embeddings into the Model Corpus collection.
 
     All chunks must share the same strategy_name (one collection per call).
     Returns the number of points upserted.
@@ -105,7 +158,7 @@ def upsert_chunks(chunks: list[Chunk], embeddings: list[list[float]]) -> int:
 
     strategy = chunks[0].strategy_name
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
 
     points = []
     for chunk, embedding in zip(chunks, embeddings):
@@ -133,13 +186,12 @@ def search(
     query_vector: list[float],
     top_k: int = 5,
     document_filter: str | None = None,
+    *,
+    model_id: str,
 ) -> list[dict[str, Any]]:
-    """Vector similarity search in a strategy's collection.
+    """Vector similarity search in a strategy's Model Corpus collection."""
-
-    Returns a list of {chunk_id, score, payload} dicts, ordered by score.
-    """
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
 
     query_filter = None
     if document_filter:
@@ -167,24 +219,81 @@ def search(
         raise QdrantError(f"Search failed in '{name}': {exc}") from exc
 
 
-# ── Delete by document ─────────────────────────────────────────────
+def get_chunks_by_indices(
+    strategy: StrategyName,
+    document_name: str,
+    chunk_indices: list[int],
+    *,
+    model_id: str,
+) -> dict[int, dict[str, Any]]:
+    """Fetch chunks by document_name + chunk_index.
 
-def delete_document_chunks(strategy: StrategyName, document_name: str) -> int:
+    Returns a map of chunk_index → {chunk_id, score, payload}.
-    """Remove all chunks for a given document from a strategy's collection.
+    Missing indices are omitted (caller skips edges).
-
-    Returns the number of points deleted.
     """
+    if not chunk_indices:
+        return {}
+
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
+    unique_indices = sorted(set(chunk_indices))
+
+    try:
+        results = client.scroll(
+            collection_name=name,
+            scroll_filter=Filter(
+                must=[
+                    FieldCondition(
+                        key="document_name",
+                        match=MatchValue(value=document_name),
+                    ),
+                    FieldCondition(
+                        key="chunk_index",
+                        match=MatchAny(any=unique_indices),
+                    ),
+                ]
+            ),
+            limit=max(len(unique_indices), 1),
+            with_payload=True,
+            with_vectors=False,
+        )
+        found: dict[int, dict[str, Any]] = {}
+        for point in results[0]:
+            payload = point.payload or {}
+            idx = payload.get("chunk_index")
+            if idx is None:
+                continue
+            found[int(idx)] = {
+                "chunk_id": payload.get("chunk_id", str(point.id)),
+                "score": None,
+                "payload": payload,
+            }
+        return found
+    except Exception as exc:
+        raise QdrantError(
+            f"Failed to fetch chunks by index from '{name}': {exc}"
+        ) from exc
+
+
+# ── Delete by document ─────────────────────────────────────────────
+
+def delete_document_chunks(
+    strategy: StrategyName,
+    document_name: str,
+    *,
+    model_id: str,
+) -> int:
+    """Remove all chunks for a given document from a strategy collection."""
+    client = get_qdrant_client()
+    name = collection_name(strategy, model_id)
 
     try:
-        # First find matching point IDs
         results = client.scroll(
             collection_name=name,
             scroll_filter=Filter(
                 must=[FieldCondition(key="document_name", match=MatchValue(value=document_name))]
             ),
-            limit=10_000,  # safety cap
+            limit=10_000,
             with_payload=False,
             with_vectors=False,
         )
@@ -196,16 +305,21 @@ def delete_document_chunks(strategy: StrategyName, document_name: str) -> int:
             collection_name=name,
             points_selector=PointIdsList(points=point_ids),
         )
-        logger.info("Deleted %d points from %s for document '%s'", len(point_ids), name, document_name)
+        logger.info(
+            "Deleted %d points from %s for document '%s'",
+            len(point_ids),
+            name,
+            document_name,
+        )
         return len(point_ids)
     except Exception as exc:
         raise QdrantError(f"Failed to delete from '{name}': {exc}") from exc
 
 
-def delete_all_strategy_chunks(strategy: StrategyName) -> int:
+def delete_all_strategy_chunks(strategy: StrategyName, *, model_id: str) -> int:
     """Delete all points in a strategy's collection (full wipe)."""
     client = get_qdrant_client()
-    name = collection_name(strategy)
+    name = collection_name(strategy, model_id)
     try:
         info = client.get_collection(collection_name=name)
         count = info.points_count or 0

src/storage/sqlite.py

@@ -62,6 +62,7 @@ CREATE TABLE IF NOT EXISTS queries (
     question          TEXT NOT NULL,
     answer            TEXT NOT NULL,
     retrieved_chunks  TEXT NOT NULL DEFAULT '[]',
+    expansion_tree    TEXT NOT NULL DEFAULT '[]',
     latency_breakdown TEXT NOT NULL DEFAULT '{}',
     token_usage       TEXT NOT NULL DEFAULT '{}',
     created_at        TEXT NOT NULL,
@@ -76,17 +77,100 @@ CREATE TABLE IF NOT EXISTS experiments (
     per_question      TEXT NOT NULL DEFAULT '[]',
     aggregate_metrics TEXT NOT NULL DEFAULT '{}',
     strategies_used   TEXT NOT NULL DEFAULT '[]',
+    embedding_model_id TEXT,
+    embedding_provider TEXT,
+    boundary_embedding_model_id TEXT,
     created_at        TEXT NOT NULL,
     FOREIGN KEY (document_id) REFERENCES documents(id) ON DELETE CASCADE
 );
+
+CREATE TABLE IF NOT EXISTS app_settings (
+    key   TEXT PRIMARY KEY,
+    value TEXT NOT NULL
+);
 """
 
 
 def init_db() -> None:
-    """Create tables if they don't exist."""
+    """Create tables if they don't exist and apply light migrations."""
     conn = _connect()
     try:
         conn.executescript(_SCHEMA)
+        _migrate_schema(conn)
+        conn.commit()
+    finally:
+        conn.close()
+
+
+def _migrate_schema(conn: sqlite3.Connection) -> None:
+    """Add Embedding Model provenance columns and backfill legacy experiments."""
+    cols = {
+        row[1]
+        for row in conn.execute("PRAGMA table_info(experiments)").fetchall()
+    }
+    if "embedding_model_id" not in cols:
+        conn.execute("ALTER TABLE experiments ADD COLUMN embedding_model_id TEXT")
+    if "embedding_provider" not in cols:
+        conn.execute("ALTER TABLE experiments ADD COLUMN embedding_provider TEXT")
+    if "boundary_embedding_model_id" not in cols:
+        conn.execute(
+            "ALTER TABLE experiments ADD COLUMN boundary_embedding_model_id TEXT"
+        )
+
+    dcols = {
+        row[1]
+        for row in conn.execute("PRAGMA table_info(documents)").fetchall()
+    }
+    if "last_corpus_embedding_model_id" not in dcols:
+        conn.execute(
+            "ALTER TABLE documents ADD COLUMN last_corpus_embedding_model_id TEXT"
+        )
+    if "last_boundary_embedding_model_id" not in dcols:
+        conn.execute(
+            "ALTER TABLE documents ADD COLUMN last_boundary_embedding_model_id TEXT"
+        )
+
+    qcols = {
+        row[1]
+        for row in conn.execute("PRAGMA table_info(queries)").fetchall()
+    }
+    if "expansion_tree" not in qcols:
+        conn.execute(
+            "ALTER TABLE queries ADD COLUMN expansion_tree TEXT NOT NULL DEFAULT '[]'"
+        )
+
+    # Legacy Experiments without provenance → historical OpenAI small (ADR-0019)
+    from src.chunking.embedding_models import LEGACY_CLOUD_MODEL_ID, Provider
+
+    conn.execute(
+        """UPDATE experiments
+           SET embedding_model_id = ?, embedding_provider = ?
+           WHERE embedding_model_id IS NULL OR embedding_model_id = ''""",
+        (LEGACY_CLOUD_MODEL_ID, Provider.CLOUD.value),
+    )
+
+
+def get_app_setting(key: str) -> str | None:
+    """Read a persisted app setting value."""
+    conn = _connect()
+    try:
+        row = conn.execute(
+            "SELECT value FROM app_settings WHERE key = ?", (key,)
+        ).fetchone()
+        return row["value"] if row else None
+    finally:
+        conn.close()
+
+
+def set_app_setting(key: str, value: str) -> None:
+    """Upsert a persisted app setting."""
+    conn = _connect()
+    try:
+        conn.execute(
+            """INSERT INTO app_settings (key, value) VALUES (?, ?)
+               ON CONFLICT(key) DO UPDATE SET value = excluded.value""",
+            (key, value),
+        )
         conn.commit()
     finally:
         conn.close()
@@ -157,6 +241,27 @@ def update_chunk_counts(doc_id: str, chunk_counts: dict[str, int]) -> None:
         conn.close()
 
 
+def update_process_embedding_provenance(
+    doc_id: str,
+    *,
+    corpus_model_id: str,
+    boundary_model_id: str | None = None,
+) -> None:
+    """Record Corpus/Boundary used on the last successful process run."""
+    conn = _connect()
+    try:
+        conn.execute(
+            """UPDATE documents
+               SET last_corpus_embedding_model_id = ?,
+                   last_boundary_embedding_model_id = ?
+               WHERE id = ?""",
+            (corpus_model_id, boundary_model_id, doc_id),
+        )
+        conn.commit()
+    finally:
+        conn.close()
+
+
 def delete_document(doc_id: str) -> bool:
     """Delete a document and its cascaded queries/experiments."""
     conn = _connect()
@@ -178,6 +283,7 @@ def save_query(
     question: str,
     answer: str,
     retrieved_chunks: list[dict] | None = None,
+    expansion_tree: list[dict] | None = None,
     latency_breakdown: dict[str, float] | None = None,
     token_usage: dict[str, int] | None = None,
 ) -> dict[str, Any]:
@@ -188,10 +294,11 @@ def save_query(
         conn.execute(
             """INSERT INTO queries
                (id, document_id, strategy_name, question, answer,
-                retrieved_chunks, latency_breakdown, token_usage, created_at)
+                retrieved_chunks, expansion_tree, latency_breakdown, token_usage, created_at)
-               VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
+               VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
             (query_id, document_id, strategy_name, question, answer,
              json.dumps(retrieved_chunks or []),
+             json.dumps(expansion_tree or []),
              json.dumps(latency_breakdown or {}),
              json.dumps(token_usage or {}),
              _now()),
@@ -250,22 +357,36 @@ def save_experiment(
     per_question: list[dict] | None = None,
     aggregate_metrics: dict | None = None,
     strategies_used: list[str] | None = None,
+    embedding_model_id: str | None = None,
+    embedding_provider: str | None = None,
+    boundary_embedding_model_id: str | None = None,
 ) -> dict[str, Any]:
-    """Insert a benchmark experiment."""
+    """Insert a benchmark experiment.
+
+    embedding_model_id is the Corpus Embedding Model (ADR-0024).
+    """
+    from src.chunking.embedding_models import DEFAULT_CLOUD_MODEL_ID, Provider
+
     experiment_id = experiment_id or _new_id()
+    model_id = embedding_model_id or DEFAULT_CLOUD_MODEL_ID
+    provider = embedding_provider or Provider.CLOUD.value
     conn = _connect()
     try:
         conn.execute(
             """INSERT INTO experiments
                (id, document_id, benchmark_config, questions, per_question,
-                aggregate_metrics, strategies_used, created_at)
+                aggregate_metrics, strategies_used, embedding_model_id,
-               VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
+                embedding_provider, boundary_embedding_model_id, created_at)
+               VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
             (experiment_id, document_id,
              json.dumps(benchmark_config or {}),
              json.dumps(questions or []),
              json.dumps(per_question or []),
              json.dumps(aggregate_metrics or {}),
              json.dumps(strategies_used or []),
+             model_id,
+             provider,
+             boundary_embedding_model_id,
              _now()),
         )
         conn.commit()
@@ -324,7 +445,7 @@ def delete_experiment(experiment_id: str) -> bool:
 
 # ── Internal helpers ───────────────────────────────────────────────
 
-_JSON_FIELDS = {"chunk_counts", "retrieved_chunks", "latency_breakdown",
+_JSON_FIELDS = {"chunk_counts", "retrieved_chunks", "expansion_tree", "latency_breakdown",
                 "token_usage", "document_tree", "benchmark_config",
                 "questions", "per_question", "aggregate_metrics",
                 "strategies_used"}