Why:
- ADR-0002's keyword search needs a full-text index on content, which
collection.py deliberately deferred to plan 002. is_active and chunk_index
were unindexed while ingestion was the only reader; every /v1/points read path
filters on them.
Changes:
- content gets a TEXT index with the multilingual tokenizer, which segments
Persian correctly where the word tokenizer mishandles ZWNJ-joined compounds.
No stemmer or stopword list: content is already letter-folded by
normalize_persian_text at ingest, and the ranked Farsi lexical path is the
benchmarked BM25 sparse vector, not this index.
- Tests assert content is TEXT rather than KEYWORD -- a keyword index would only
match an entire chunk verbatim, which never happens and fails silently.
- Adds a test that a missing index is added to an already-live collection.
Impact:
- Requires re-running `python -m src.cli.qdrant_bootstrap`. Payload indexes are
additive, so no collection rebuild and no re-embedding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Why:
- tests/e2e/ cannot reach fixtures defined in a per-boundary conftest, and
`pytest_plugins` is only honoured in the root conftest.
Changes:
- move the Postgres/MinIO/Qdrant container fixtures into
tests/support/containers.py and register it as a root plugin
- fold MinIO bucket creation into `minio_settings`; an autouse fixture in a
globally registered plugin would pull a container into unit runs
- add a `postgres_settings` fixture so a component can be built from it directly
Impact:
- no behavior change; `pytest -m unit` still needs no Docker
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Why:
- The chunks collection is now created by an explicit deployment step
(qdrant_bootstrap), not at startup, which means a process can boot against
a healthy Qdrant that has no collection at all. /readyz's previous check
only called get_collections(), so it reported ready in that state — the
misconfiguration stayed invisible until the first upload failed with a 502
after already paying for the MinIO write and embedding round trips.
Changes:
- ping_qdrant() now checks collection_exists(collection) instead of just
reachability.
Why:
- Ingested chunks need to become searchable Qdrant points before the upload
response returns, with tenant/domain isolation and a safe re-ingestion
story per ADR-0001/0017.
Changes:
- src/application/points/: index_chunks() is the sole entry point, owning
payload construction, batched/bounded-concurrency upserts
(upsert_concurrency semaphore), and a soft-delete sweep for points a
shorter re-ingestion leaves behind. The sweep runs only after every upsert
in the attempt succeeds, so a failed attempt can leave a stale prefix but
never removes content from a working index.
- PointStorage port (application/ports/) + QdrantPointStorage adapter
(infrastructure/qdrant/points.py), keeping the qdrant_client SDK out of
application code per ADR-0015.
- FakePointStorage test double for exercising the ordering/idempotency
guarantees without a real Qdrant.
Why:
- The chunks collection needs four named vectors (dense_nomic, dense_openai,
sparse, late_interaction) and payload indexes defined at creation time per
ADR-0001; sparse/multivector fields cannot be added to an existing
collection without recreating it, so schema drift here is expensive.
- Creating it at FastAPI startup would mirror the DDL-at-boot anti-pattern
ADR-0009 already rejects for Postgres and ADR-0012 rejects for LangGraph's
setup(), so it is a deployment step instead.
Changes:
- src/infrastructure/qdrant/collection.py: ensure_chunks_collection(),
idempotent and schema-verifying (raises on dimension/modifier mismatch
rather than silently accepting a misconfigured collection).
- src/cli/qdrant_bootstrap.py: the operator entry point
(python -m src.cli.qdrant_bootstrap).
- QdrantSettings gains collection/upsert_batch_size/upsert_concurrency.
Impact:
- Deployments must run the new bootstrap command before the first upload;
see ADR-0001's new "Collection provisioning" section.
