Ali Zarinkolah
3b9434faf4
Why:
- Ingestion writes points in bulk but nothing could read one back. Plan 002
Phase 2 opens the read surface an admin frontend needs.
Changes:
- GET /v1/points/{point_id}, /v1/points?file_id=..., /v1/points/count,
/v1/points/search, and /v1/files/{file_id}/points, all under points:read --
the scope follows the data, so an upload key does not become a way to read
every chunk of every file.
- The keyword query is Persian-normalized before matching, because ingestion
letter-folds content at ingest and an unfolded query would return an empty
result set silently rather than an error.
- file_id is required on the listing: the cursor is an order_id value and
order_id is only unique within one file.
- PointNotFoundError maps to 404, never 403, so a cross-tenant point id is
indistinguishable from a nonexistent one.
- Route order is load-bearing: /count and /search precede /{point_id}, or
"count" is parsed as a UUID and fails 422.
Impact:
- Requires the content/is_active/chunk_index payload indexes, so a deployed
environment needs qdrant_bootstrap re-run before search works.
- Keyword search returns no relevance score and no ranked order; callers must
not read array position as relevance.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Talie chatbot service
Architecture decisions live in docs/adr. The first implementation
milestone is documented in the ingestion vertical-slice plan.
Day-to-day operation — tuning the ingestion bounds, the proxy timeout
requirement, and how to investigate or retry a failed upload — is the
operator runbook.
Provisioning the datastores
Both schema steps run as explicit deployment steps. The application performs no DDL at startup — not for Postgres (ADR-0009) and not for Qdrant (ADR-0001, "Collection provisioning").
docker compose up -d # Postgres, MinIO, Qdrant
uv run alembic upgrade head # Postgres schema
uv run python -m src.cli.qdrant_bootstrap # the `chunks` collection
uv run fastapi dev src/main.py
Nothing over HTTP can create the first tenant — every /v1 route needs an API
key, and a key cannot exist before its tenant. One command issues both, plus any
domains, printing the key once (only its hash is stored):
uv run python -m src.cli.provision_tenant --slug acme --domain fire
Before a tenant can upload, its domains must be registered — POST /v1/files
rejects an unregistered or disabled domain with 400. The calling backend
manages them over /v1/domains using a key with the domains:write scope:
curl -X POST http://localhost:8000/v1/domains \
-H "Authorization: Bearer $API_KEY" \
-H 'Content-Type: application/json' \
-d '{"domain": "fire", "display_name": "Fire insurance"}'
Both bootstrap commands are idempotent and safe to re-run. qdrant_bootstrap verifies an
existing collection against the pinned schema and exits non-zero on a mismatch,
rather than leaving a silently degraded sparse index in place.
./scripts/smoke.sh verifies the whole path — Compose up, both deployment
steps, provisioning, an upload through the running web process to indexed Qdrant
points. See the runbook.
Local Langfuse
This repo includes a root-level development Compose file for Langfuse:
Start Langfuse locally:
cp .env.langfuse.example .env.langfuse
# edit .env.langfuse and replace CHANGE_ME values
docker compose --env-file .env.langfuse -f docker-compose.langfuse.yml up -d
Open:
http://localhost:3000
If the chatbot app runs on your host machine, configure it with:
LANGFUSE_HOST=http://localhost:3000
If the chatbot app later runs inside the same Compose project/network as Langfuse, configure it with:
LANGFUSE_HOST=http://langfuse-web:3000
A future app stack can be launched together with Langfuse using multiple Compose files:
docker compose \
-f docker-compose.yml \
-f docker-compose.langfuse.yml \
--env-file .env \
--env-file .env.langfuse \
up -d