2.1 KiB
Active Embedding Model is global, not an Experiment axis
Status: Superseded by ADR-0024 — single Active Embedding Model replaced by Boundary + Corpus roles (query locked to Corpus).
Original decision retained for history:
The platform compares Strategies under controlled conditions. Embedding Model is a confounder, not a second experiment dimension: one Active Embedding Model applies to process, query, and new Experiments. The operator switches it from Admin among entries in a static Embedding Model Registry (Provider, stable id, vector dimension). Selection persists across restarts (config supplies the default only when unset). Every Experiment records which Embedding Model produced it; historical rows without provenance are treated as the default cloud model. A process, query, or Experiment snapshots the Active Embedding Model at start so a mid-flight Admin switch cannot mix models inside one operation.
Considered Options
- Global Active Embedding Model — chosen; fair Strategy comparisons; A/B models via separate Experiments
- Embedding Model as Experiment axis — Strategy × model cross-product; richer science, much heavier data model/UI
- Per-run choice with no Experiment coupling — flexible, invites silent unfair comparisons
Switcher: Admin UI (not env-only, not read-only status). Catalog: static registry (not live Ollama discovery, not free-form). Local Provider host: OLLAMA_BASE_URL in config only. Cost Estimator: $0 embedding line when Active Embedding Model is Local; LLM costs unchanged. This carve-out does not reopen general /admin/config (ADR-0007 deferred).
Consequences
- Introduce Provider adapters (Cloud OpenAI, Local Ollama) behind one embed API; callers bind a registry entry for the operation
- Dashboard Admin gains an Embedding Model switcher; Benchmarks/reports must show Experiment provenance
docs/out-of-scope-v1.md“no embedding model from dashboard” is superseded for this focused control only- Local Nomic registry entries use task prefixes (
search_document/search_query); Cloud OpenAI entries do not