chunking_strategies_evaluation/docs/adr/0018-model-corpus-scoped-collections.md

Model Corpus via model-scoped collections

Vectors for Cloud and Local Embedding Models must not share a Qdrant collection: dimensions and embedding spaces differ. Each Embedding Model therefore owns a Model Corpus — Strategy collections whose identity includes that model. Process, query, and Experiment only read/write the Active Embedding Model’s corpus; other corpora stay untouched. Pre-existing unscoped collections (e.g. fixed_size_collection) are migrated into the default cloud Embedding Model’s corpus so existing OpenAI work is not discarded.

Considered Options

• Model-scoped collections — chosen; non-destructive switch between Providers/models; clear isolation
• Wipe-and-rebuild on switch — simple naming, destructive and easy to forget
• Dimension-gated single set — only allow same-dimension swaps; blocks most OpenAI ↔ Ollama use

Legacy handling (original): migrate unscoped → default cloud corpus. Superseded for naming by ADR-0021 — all models, including default cloud, use {strategy}__{model_id}_collection; delete leftover unscoped collections when rebuilding.

Consequences

• Collection naming and vector size are driven by the Embedding Model Registry entry, not a global 1536 constant
• Admin Qdrant list shows all corpora, labeled by Embedding Model, with the active corpus emphasized; wipe stays per named collection
• Switching Active Embedding Model does not delete the other Model Corpus; operator must process documents again under the new model to populate it