Ali Zarinkolah 5e935e5895 feat(qdrant): index content, is_active, and chunk_index on the chunks collection ...

Why:
- ADR-0002's keyword search needs a full-text index on content, which
  collection.py deliberately deferred to plan 002. is_active and chunk_index
  were unindexed while ingestion was the only reader; every /v1/points read path
  filters on them.

Changes:
- content gets a TEXT index with the multilingual tokenizer, which segments
  Persian correctly where the word tokenizer mishandles ZWNJ-joined compounds.
  No stemmer or stopword list: content is already letter-folded by
  normalize_persian_text at ingest, and the ranked Farsi lexical path is the
  benchmarked BM25 sparse vector, not this index.
- Tests assert content is TEXT rather than KEYWORD -- a keyword index would only
  match an entire chunk verbatim, which never happens and fails silently.
- Adds a test that a missing index is added to an already-live collection.

Impact:
- Requires re-running `python -m src.cli.qdrant_bootstrap`. Payload indexes are
  additive, so no collection rebuild and no re-embedding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-22 13:09:26 +03:30

6.2 KiB

Raw Blame History

View Raw