chunking_strategies_evaluation/docs/adr/0020-semantic-boundary-detection-required.md

Semantic strategies require Semantic Boundary Detection

semantic and semantic_parent_child must cut chunks from meaning: the orchestrator embeds consecutive units (sentences / paragraphs) with the Active Embedding Model snapshot, passes those vectors into chunk(), then embeds finished chunks for Qdrant. Fixed-count fallbacks are removed — missing or length-mismatched boundary embeddings fail that Strategy. This completes ADR 0012’s sentence-level design in the process path (and the parent/child analogue) so benchmarks cannot silently measure “every N units” under a semantic name.

Considered Options

• Wire both strategies + fail hard — chosen; same Active model for boundaries and storage; accept double embed cost; re-process to replace old fake-semantic vectors
• Wire semantic only — rejected; would leave parent/child on the same lie
• Keep fallback with warnings — rejected; that is how the bug stayed hidden
• Always OpenAI for boundaries regardless of Active — rejected; confounds model A/B experiments

Consequences

• Process cost rises for these two Strategies (unit embeds + chunk embeds)
• Prior Experiments / collections produced under the fallback are not true semantic — operator must re-process and re-benchmark
• Strategy modules raise ChunkingError if boundary embeddings are absent or mismatched; orchestration is responsible for supplying them