Why:
- ADR-0002's keyword search needs a full-text index on content, which
collection.py deliberately deferred to plan 002. is_active and chunk_index
were unindexed while ingestion was the only reader; every /v1/points read path
filters on them.
Changes:
- content gets a TEXT index with the multilingual tokenizer, which segments
Persian correctly where the word tokenizer mishandles ZWNJ-joined compounds.
No stemmer or stopword list: content is already letter-folded by
normalize_persian_text at ingest, and the ranked Farsi lexical path is the
benchmarked BM25 sparse vector, not this index.
- Tests assert content is TEXT rather than KEYWORD -- a keyword index would only
match an entire chunk verbatim, which never happens and fails silently.
- Adds a test that a missing index is added to an already-live collection.
Impact:
- Requires re-running `python -m src.cli.qdrant_bootstrap`. Payload indexes are
additive, so no collection rebuild and no re-embedding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ali Zarinkolah
5e935e5895