feat(qdrant): provision the chunks collection as an explicit deployment step

Why:
- The chunks collection needs four named vectors (dense_nomic, dense_openai,
  sparse, late_interaction) and payload indexes defined at creation time per
  ADR-0001; sparse/multivector fields cannot be added to an existing
  collection without recreating it, so schema drift here is expensive.
- Creating it at FastAPI startup would mirror the DDL-at-boot anti-pattern
  ADR-0009 already rejects for Postgres and ADR-0012 rejects for LangGraph's
  setup(), so it is a deployment step instead.

Changes:
- src/infrastructure/qdrant/collection.py: ensure_chunks_collection(),
  idempotent and schema-verifying (raises on dimension/modifier mismatch
  rather than silently accepting a misconfigured collection).
- src/cli/qdrant_bootstrap.py: the operator entry point
  (python -m src.cli.qdrant_bootstrap).
- QdrantSettings gains collection/upsert_batch_size/upsert_concurrency.

Impact:
- Deployments must run the new bootstrap command before the first upload;
  see ADR-0001's new "Collection provisioning" section.
This commit is contained in:
Ali Zarinkolah
2026-08-20 18:15:21 +03:30
parent e8fb41af87
commit 5e0addcc55
8 changed files with 456 additions and 2 deletions

View File

@@ -0,0 +1,54 @@
"""Create the `chunks` collection — the Qdrant analogue of `alembic upgrade head`.
uv run python -m src.cli.qdrant_bootstrap
A deployment step, deliberately not part of the FastAPI lifespan: collection
creation is DDL, which ADR-0009 keeps out of application startup for Postgres
and ADR-0012 keeps out of it for LangGraph's `.setup()`. See
`src/infrastructure/qdrant/collection.py` for the full reasoning.
Idempotent and safe to re-run. Exits non-zero if an existing collection
diverges from the pinned schema, rather than leaving a silently degraded
sparse index behind.
"""
import asyncio
import sys
import structlog
from src.config import Settings
from src.infrastructure.observability.logging import configure_logging
from src.infrastructure.qdrant.client import create_client
from src.infrastructure.qdrant.collection import (
CollectionSchemaMismatchError,
ensure_chunks_collection,
)
logger = structlog.get_logger(__name__)
async def bootstrap(settings: Settings | None = None) -> int:
resolved = settings or Settings()
configure_logging(resolved.logging)
client = create_client(resolved.qdrant)
try:
created = await ensure_chunks_collection(client, collection=resolved.qdrant.collection)
except CollectionSchemaMismatchError as exc:
logger.error("qdrant.bootstrap.schema_mismatch", error=str(exc))
return 1
finally:
await client.close()
logger.info(
"qdrant.bootstrap.completed", collection=resolved.qdrant.collection, created=created
)
return 0
def main() -> None:
sys.exit(asyncio.run(bootstrap()))
if __name__ == "__main__":
main()