feat(ingestion): index embedded chunks into Qdrant on upload
Why: - POST /v1/files was reporting chunks_indexed=0/points_created=0 unconditionally — chunks were parsed and embedded but never written to Qdrant, so nothing was actually searchable after upload. Changes: - upload_source_file() now calls index_chunks() after embedding, inside the same INGESTION_TIMEOUT_SECONDS window, and marks the job failed (error_code=index_failed, 502) if it raises. - Job counters (points_created, points_soft_deleted) and the response's chunks_indexed now reflect the real indexing result instead of a hardcoded zero. - Wired PointStorage through AppResources/lifespan/the files router. Impact: - A successful upload is now searchable in Qdrant by the time 201 returns.
This commit is contained in:
17
README.md
17
README.md
@@ -3,6 +3,23 @@
|
||||
Architecture decisions live in [`docs/adr`](docs/adr). The first implementation
|
||||
milestone is documented in the [ingestion vertical-slice plan](docs/plans/001-ingestion-vertical-slice.md).
|
||||
|
||||
## Provisioning the datastores
|
||||
|
||||
Both schema steps run as explicit deployment steps. The application performs no
|
||||
DDL at startup — not for Postgres (ADR-0009) and not for Qdrant (ADR-0001,
|
||||
"Collection provisioning").
|
||||
|
||||
```bash
|
||||
docker compose up -d # Postgres, MinIO, Qdrant
|
||||
uv run alembic upgrade head # Postgres schema
|
||||
uv run python -m src.cli.qdrant_bootstrap # the `chunks` collection
|
||||
uv run fastapi dev src/main.py
|
||||
```
|
||||
|
||||
Both commands are idempotent and safe to re-run. `qdrant_bootstrap` verifies an
|
||||
existing collection against the pinned schema and exits non-zero on a mismatch,
|
||||
rather than leaving a silently degraded sparse index in place.
|
||||
|
||||
## Local Langfuse
|
||||
|
||||
This repo includes a root-level development Compose file for Langfuse:
|
||||
|
||||
Reference in New Issue
Block a user