azk/chatbot_v3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: azk:50f6c33c0cf2cde9d1d4078f3b1a72cc1170bf37
Branches Tags
azk:main
..
compare: azk:e9caeaa4d8a5405913ebc0cbd90c0ec97139176f
Branches Tags
azk:main

3 Commits
50f6c33c0c ... e9caeaa4d8

Author SHA1 Message Date
Ali Zarinkolah
e9caeaa4d8 docs(observability): define Langfuse and structured logging strategy 
Why:
- Establish separate observability systems for LLM tracing, prompt iteration, evaluation, operational logs, and durable application audit records.

Changes:
- Define Langfuse traces, prompt labels, feedback scores, evaluation workflows, redaction rules, and correlation identifiers.
- Define structlog-based JSON logging, request context propagation, event naming, log levels, and privacy requirements.

Impact:
- Langfuse remains the LLM observability plane, while Postgres remains the durable audit and billing source of truth.
- Application logs must avoid secrets and raw sensitive payloads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-10 11:45:19 +03:30
Ali Zarinkolah
96ff2ec137 docs(api): define REST boundary and point management routes 
Why:
- Establish the versioned FastAPI boundary for authentication, tenant isolation, chat runs, file ingestion, point management, and health checks.

Changes:
- Define /v1 routers, bearer-token authentication, scopes, response envelopes, error conventions, and job-shaped ingestion responses.
- Rename the older indicative /chunks routes to /points while preserving the existing Qdrant payload and CRUD semantics.
- Define tenant injection and concurrency requirements at the HTTP boundary.

Impact:
- The REST API is owned by ADR-0008 when older endpoint examples differ.
- Clients should use /v1/points and /v1/threads resource paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-10 11:45:19 +03:30
Ali Zarinkolah
ed96f20ebf docs(data): define Postgres schema and migration conventions 
Why:
- Establish the application-owned relational source of truth for tenants, authentication, ingestion, audit, graph runs, LLM usage, and feedback.

Changes:
- Define SQLAlchemy 2.x and Alembic conventions.
- Specify tenant, API-key, ingestion, audit, graph-run, pricing, usage, and feedback tables.
- Document indexing, retention, privacy, and multitenancy rules.

Impact:
- Postgres schema changes must be implemented through Alembic migrations.
- FastAPI must not run DDL during startup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-08-10 11:45:19 +03:30
5 changed files with 1642 additions and 15 deletions
Expand all files Collapse all files

35
docs/adr/0002-chunk-crud-and-search-api.md
View File

@@ -4,6 +4,11 @@
Accepted Accepted
Route naming note: ADR-0008 owns the REST API surface and renames the
indicative endpoint examples in this ADR from `/chunks/...` to `/points/...`.
The CRUD semantics, Qdrant primitives, payload schema, soft-delete behavior,
and tenant-isolation rules in this ADR remain accepted.
## Context ## Context
Beyond bulk ingestion ([0001](0001-ingestion-pipeline-and-collection-schema.md)), Beyond bulk ingestion ([0001](0001-ingestion-pipeline-and-collection-schema.md)),
@@ -24,16 +29,16 @@ the `chunks` collection and payload schema from ADR-0001:
| Operation | FastAPI endpoint (indicative) | Qdrant primitive | | Operation | FastAPI endpoint (indicative) | Qdrant primitive |
|---|---|---| |---|---|---|
| Create a chunk | `POST /chunks` | upsert (single point) | | Create a point | `POST /points` | upsert (single point) |
| Update a chunk's vectors/content | `PUT /chunks/{chunk_id}` | upsert (`update_only` mode) | | Update a point's vectors/content | `PUT /points/{point_id}` | upsert (`update_only` mode) |
| Partially update payload | `PATCH /chunks/{chunk_id}/payload` | `set_payload` / `overwrite_payload` | | Partially update payload | `PATCH /points/{point_id}/payload` | `set_payload` / `overwrite_payload` |
| Delete one chunk | `DELETE /chunks/{chunk_id}` | delete by ID | | Delete one point | `DELETE /points/{point_id}` | delete by ID |
| Delete many chunks | `DELETE /chunks?file_id=...` | delete by filter | | Delete many points | `DELETE /points?file_id=...` | delete by filter |
| List/paginate chunks, in order | `GET /chunks?file_id=...` | `scroll` with filter + pagination, `order_by: order_id` | | List/paginate points, in order | `GET /points?file_id=...` | `scroll` with filter + pagination, `order_by: order_id` |
| Reorder/insert a chunk | `PATCH /chunks/{chunk_id}/order` | `set_payload` on `order_id` only | | Reorder/insert a point | `PATCH /points/{point_id}/order` | `set_payload` on `order_id` only |
| Count chunks | `GET /chunks/count` | `count` | | Count points | `GET /points/count` | `count` |
| Keyword search | `GET /chunks/search?q=...` | full-text payload index on `content` (match, not semantic) | | Keyword search | `GET /points/search?q=...` | full-text payload index on `content` (match, not semantic) |
| Bulk multi-op edits | `POST /chunks/batch` | Qdrant `points/batch` | | Bulk multi-op edits | `POST /points/batch` | Qdrant `points/batch` |
### Ordering and reordering ### Ordering and reordering
@@ -45,7 +50,7 @@ inserting a chunk only requires assigning it a new value between its two new
neighbors — no renumbering of siblings, and no effect on the chunk's stable neighbors — no renumbering of siblings, and no effect on the chunk's stable
point ID (which is derived from the immutable `chunk_index`, not `order_id`). point ID (which is derived from the immutable `chunk_index`, not `order_id`).
`PATCH /chunks/{chunk_id}/order` does more than set one field, though: since `PATCH /points/{point_id}/order` does more than set one field, though: since
ADR-0001 also maintains `previous_chunk_id`/`next_chunk_id` pointers for O(1) ADR-0001 also maintains `previous_chunk_id`/`next_chunk_id` pointers for O(1)
adjacency lookups, a single reorder touches **up to four chunks** in one adjacency lookups, a single reorder touches **up to four chunks** in one
`points/batch` call: `points/batch` call:
@@ -62,7 +67,7 @@ go stale.
### Keyword search is not semantic search ### Keyword search is not semantic search
`GET /chunks/search` matches against the full-text payload index on `GET /points/search` matches against the full-text payload index on
`content` (and structured filters on `tenant_id`, `domain`, `file_id`, `content` (and structured filters on `tenant_id`, `domain`, `file_id`,
etc.) — it is filter/match-based keyword search, not embedding-based etc.) — it is filter/match-based keyword search, not embedding-based
retrieval. This is deliberately distinct from the hybrid dense+sparse retrieval. This is deliberately distinct from the hybrid dense+sparse
@@ -72,11 +77,11 @@ context) and should not be conflated in the API or in future discussion.
### Delete is soft by default ### Delete is soft by default
`DELETE /chunks/{chunk_id}` and `DELETE /chunks?file_id=...` set `DELETE /points/{point_id}` and `DELETE /points?file_id=...` set
`is_active: false` and `deleted_at` (via `set_payload`) rather than removing `is_active: false` and `deleted_at` (via `set_payload`) rather than removing
the point from Qdrant — consistent with ADR-0001's soft-delete fields. This the point from Qdrant — consistent with ADR-0001's soft-delete fields. This
keeps deleted chunks available for audit and lets `GET /chunks` and keeps deleted points available for audit and lets `GET /points` and
`GET /chunks/search` filter them out by default (`is_active: true` implied `GET /points/search` filter them out by default (`is_active: true` implied
unless the caller explicitly asks to include inactive chunks). A hard delete unless the caller explicitly asks to include inactive chunks). A hard delete
(actual point removal, e.g. `points/delete`) is available separately for (actual point removal, e.g. `points/delete`) is available separately for
compliance-driven purges, not as the default CRUD behavior. Either way, the compliance-driven purges, not as the default CRUD behavior. Either way, the

298
docs/adr/0008-rest-api-and-fastapi-boundary.md Normal file
View File

@@ -0,0 +1,298 @@
# 0008. REST API and FastAPI boundary
## Status
Proposed
## Context
The project now has decisions for ingestion and Qdrant storage
([0001](0001-ingestion-pipeline-and-collection-schema.md)), point-level CRUD
and keyword search ([0002](0002-chunk-crud-and-search-api.md)), agent retrieval
([0003](0003-agent-hybrid-retrieval.md)), conversational graph behaviour
([0006](0006-conversational-agent-graph.md)), and LangGraph thread persistence
([0007](0007-agent-persistence-threads-and-memory.md)). What is still missing
is the HTTP boundary that ties these together.
FastAPI has to do more than expose thin route handlers:
- Authenticate the calling backend/API client with an API key.
- Resolve that API key to an active tenant in Postgres, using the tenant/API-key
tables defined by [ADR-0009](0009-postgres-sqlalchemy-alembic-schema.md).
- Inject `tenant_id` into every Qdrant operation and every LangGraph run, so
callers never submit or override the tenant boundary themselves.
- Receive user chat messages and pass them to the compiled LangGraph graph.
- Receive source files (starting with CSV, but aligned with the docx/xlsx/csv
ingestion decisions from ADR-0001/0004) and kick off ingestion.
- Expose direct management of Qdrant records. ADR-0002 originally sketched
`/chunks/...` endpoint names, but the API is directly managing Qdrant
**points**. The REST surface should use `/points/...` while the payload can
still contain chunk-specific fields such as `chunk_id`, `content`, and
`previous_chunk_id`.
This ADR owns the public REST shape for this FastAPI service. If its endpoint
names differ from indicative endpoint examples in older ADRs, this ADR wins;
the older ADRs still own their data-model and pipeline semantics.
## Decision
### FastAPI application structure
Use a single versioned API prefix, `/v1`, and split the app into routers by
resource:
| Router | Prefix | Responsibility |
|---|---|---|
| `threads` | `/v1/threads` | Chat thread runs, transcript pagination, thread purge. |
| `files` | `/v1/files` | Upload source files and inspect ingestion status. |
| `points` | `/v1/points` | Direct CRUD, ordering, payload updates, keyword search, and batch edits over Qdrant points. |
| `users` | `/v1/users` | Cross-thread memory inspect/reset. |
| `health` | `/healthz`, `/readyz` | Process health and dependency readiness. |
Routers use FastAPI's dependency injection with `Annotated[..., Depends(...)]`.
Shared dependencies live at the router level where they only gate access, and
as typed parameters where the endpoint needs the resolved value. Path
operation functions stay one HTTP operation per function.
The app lifespan owns long-lived resources:
- SQLAlchemy async engine/sessionmaker.
- Qdrant client.
- LangGraph checkpointer/store and the compiled graph (compiled once, reused).
- Any ingestion/retrieval model clients.
No DDL runs at FastAPI startup. Postgres schema changes are Alembic migrations;
LangGraph checkpoint/store setup is a deployment step as decided in ADR-0007.
### Authentication and tenant dependency
Every `/v1` route except health checks requires an API key. The key is supplied
as a bearer token:
```http
Authorization: Bearer <api_key>
```
A FastAPI dependency validates the key against Postgres and returns an
`AuthContext`:
```text
AuthContext(
tenant_id,
tenant_slug,
api_key_id,
scopes,
actor_type="backend" | "admin" | "worker",
)
```
Validation rules:
1. API keys are stored only as hashes, never plaintext.
2. The presented key is split into a lookup prefix/key id plus secret material;
Postgres lookup uses the prefix/id, then compares the stored hash with a
constant-time comparison.
3. The key must be active: tenant active, key not revoked, key not expired.
4. The key must carry the scope required by the router/action.
5. `tenant_id` is derived only from the key's tenant relationship.
The service never accepts `tenant_id` in request bodies or query strings. Any
client-supplied tenant field is rejected as an invalid request. This preserves
the ADR-0001/0002 Qdrant tenant-isolation rule and extends it to the REST
boundary.
Recommended dependency aliases:
```python
SessionDep = Annotated[AsyncSession, Depends(get_db_session)]
AuthContextDep = Annotated[AuthContext, Depends(get_auth_context)]
```
Scope dependencies compose on top of `AuthContextDep`, for example
`require_scope("points:write")`, `require_scope("threads:run")`, and
`require_scope("files:write")`.
### Thread/run endpoints
Chat uses the LangGraph vocabulary already chosen in ADR-0007.
| Endpoint | Purpose |
|---|---|
| `POST /v1/threads/{thread_id}/runs` | Execute one graph run for a new user message. |
| `GET /v1/threads/{thread_id}/messages?limit=20&before=...` | Return a paginated transcript window for the frontend. |
| `DELETE /v1/threads/{thread_id}` | Purge the thread checkpoints for explicit clear/erasure. |
| `POST /v1/threads/{thread_id}/runs/{run_id}/feedback` | Attach feedback to the answer produced by a run. |
`POST /runs` passes this context into LangGraph:
- `thread_id` from the path.
- `tenant_id` from `AuthContext`, not from the caller body.
- `user_id` from the trusted main-backend request payload/header.
- locale/language from the request, with server-side fallback.
The endpoint streams with Server-Sent Events when the caller asks for
streaming. Token events come from LangGraph `stream_mode="messages"`; progress
and terminal status events come from `stream_mode="custom"` or explicit route
wrapping. Terminal statuses are the ADR-0006 statuses: `answered`,
`clarifying`, or `escalate`. When `escalate` is returned, the main backend uses
its existing human-assistant transfer path and stops issuing runs for that
thread.
One in-flight run per thread is allowed. The route takes a Postgres advisory
lock keyed on `thread_id`; a concurrent run returns `409 Conflict`.
### File ingestion endpoints
Files are source documents, not Qdrant points. Uploading a CSV creates or
re-ingests a file and then creates/updates many points via the ADR-0001/0004
ingestion pipeline.
| Endpoint | Purpose |
|---|---|
| `POST /v1/files` | Upload one `.csv`, `.xlsx`, `.docx`, or converted `.doc` source file for ingestion. Initial callers may only use CSV, but the route is named for the generalized source-file concept. |
| `GET /v1/files/{file_id}` | Return ingestion status and source-file metadata. |
| `GET /v1/files/{file_id}/points?limit=...&offset=...` | List active points belonging to a file, ordered by `order_id`. |
| `DELETE /v1/files/{file_id}` | Soft-delete all active points for a file, preserving audit data unless a future compliance hard-delete endpoint is added. |
`POST /v1/files` uses FastAPI `UploadFile` and multipart form fields for
metadata such as `domain`. File validation is server-side:
- Allowlist extensions/content types: `.csv`, `.xlsx`, `.docx`; legacy `.doc`
is accepted only if the conversion pipeline from ADR-0004 is enabled.
- Enforce maximum file size before reading the full body into memory.
- Sniff enough content to reject obvious extension spoofing.
- Derive `tenant_id`, `created_by`, and `updated_by` from `AuthContext`, not
from form fields.
Ingestion may be slow because it parses, chunks, embeds, and writes many
Qdrant points. The REST contract is job-shaped even if the first
implementation runs inline:
```text
202 Accepted -> { file_id, ingestion_job_id, status: "queued" | "running" }
```
A durable worker/job queue can be added later without changing the API
contract.
### Point endpoints replace the older `/chunks` sketches
The Qdrant collection is still named `chunks` in ADR-0001, because each stored
point represents one chunk. The REST resource is nevertheless `/points`,
because callers are managing Qdrant point records and the operations map
almost one-to-one to Qdrant's Points API.
| Endpoint | Qdrant primitive | Notes |
|---|---|---|
| `POST /v1/points` | upsert one point | Manual/admin creation of one chunk-backed point. |
| `GET /v1/points/{point_id}` | retrieve by ID | Returns vectors only when explicitly requested. |
| `PUT /v1/points/{point_id}` | upsert with `update_only` | Replaces content/vectors/payload with optimistic concurrency. |
| `PATCH /v1/points/{point_id}/payload` | `set_payload` / `overwrite_payload` | Payload-only update. |
| `DELETE /v1/points/{point_id}` | payload update or delete | Soft-delete by default (`is_active=false`, `deleted_at`). |
| `GET /v1/points?file_id=...` | `scroll` | List/paginate active points in `order_id` order. |
| `PATCH /v1/points/{point_id}/order` | `points/batch` | Move/insert point and relink neighbors. |
| `GET /v1/points/count` | `count` | Count active points under tenant/domain filters. |
| `GET /v1/points/search?q=...` | full-text payload match | Keyword search over `content`, not semantic retrieval. |
| `POST /v1/points/batch` | `points/batch` | Bulk multi-operation edits. |
Every operation injects a Qdrant filter containing:
- `tenant_id == auth_context.tenant_id`
- `is_active == true` by default on reads
- optional `domain`, `file_id`, and other allowed filters supplied by caller
`point_id` names the Qdrant point ID in the URL. Payload fields keep their
ADR-0001 names (`chunk_id`, `file_id`, `previous_chunk_id`, etc.) so the API
name change does not rename the storage schema.
### Response and error conventions
Use explicit return types or `response_model` to validate and filter public
responses. Do not return raw ORM objects, raw Qdrant responses, or full
LangGraph state.
Error responses use a stable envelope:
```json
{
"error": {
"code": "tenant_not_found | invalid_api_key | missing_scope | validation_error | conflict | not_found | internal_error",
"message": "human-readable summary",
"details": {},
"request_id": "..."
}
}
```
Important status codes:
| Status | Use |
|---|---|
| `202 Accepted` | Ingestion accepted as a job. |
| `400 Bad Request` | Invalid domain/filter combinations or unsupported file type. |
| `401 Unauthorized` | Missing/invalid API key. |
| `403 Forbidden` | Valid key without required scope. |
| `404 Not Found` | Resource absent within this tenant. |
| `409 Conflict` | Concurrent thread run or optimistic-concurrency mismatch. |
| `422 Unprocessable Entity` | Pydantic request validation failure. |
### Observability
Every request gets a `request_id` and structured logs containing route,
status, latency, tenant id, and API key id — never the API key itself. LangGraph
runs include the same request id in `config` metadata so API logs, graph traces,
and Qdrant operations can be correlated.
## Consequences
### Positive
- Tenant isolation starts at the FastAPI dependency boundary, before any route
logic reaches Qdrant or LangGraph.
- `/points` aligns the REST API with what it actually manages — Qdrant point
records — while preserving the chunk payload schema underneath.
- `/threads/{thread_id}/runs` remains compatible with the LangGraph thread/run
model already chosen in ADR-0007.
- Job-shaped file ingestion lets the first implementation be simple while
keeping room for a durable worker without breaking clients.
- Router-level dependencies and typed FastAPI dependencies keep auth, tenant
resolution, sessions, and scopes reusable instead of repeated per endpoint.
### Negative
- The API now has two words for related concepts: `files` are source documents,
`points` are the chunk records produced from them. This is accurate but
requires clear frontend/backend naming.
- `/points` exposes storage terminology to API consumers. That is acceptable
for this internal/admin-oriented service, but it would be less friendly as a
public product API.
- API-key auth in Postgres adds a database lookup to every request unless
short-lived caching is introduced. Caching must preserve revocation semantics.
- A job-shaped ingestion contract needs a job status store even if the initial
implementation processes inline.
- The REST layer now depends on the tenant/API-key, ingestion-job, audit, and
usage tables defined in [ADR-0009](0009-postgres-sqlalchemy-alembic-schema.md).
## Alternatives Considered
- **Keep `/chunks/...` from ADR-0002**: rejected. The storage record being
managed is a Qdrant point, and several operations (`points/batch`, retrieve
by point ID, vector inclusion flags) map directly to Qdrant's Points API.
Keeping `/chunks` would make the REST layer look more domain-friendly but
less honest about what it does.
- **Accept `tenant_id` in every request**: rejected. Tenant is an isolation
boundary, not user input. It must come from the API key's Postgres tenant
relationship.
- **Use `X-API-Key` instead of `Authorization: Bearer`**: rejected as the
default because bearer tokens are standard for service-to-service auth and
work cleanly with middleware, proxies, and generated clients. A compatibility
alias can be added later if an upstream system cannot send `Authorization`.
- **Expose one generic `/v1/qdrant/*` proxy**: rejected. It would leak Qdrant's
full API surface, bypass tenant/scoping rules too easily, and couple clients
to storage operations the service should hide.
- **Synchronous file ingestion only**: rejected as the contract. It is simpler
to implement, but embedding and late-interaction vector generation can be
slow enough to exceed HTTP timeouts. The job-shaped response gives the
implementation room to evolve.
- **Create threads with `POST /v1/threads`**: rejected for now. The main
backend owns conversation/session records, and LangGraph can create a
checkpoint sequence on the first run for a `thread_id`. A create endpoint
would imply metadata this service has explicitly chosen not to store.

444
docs/adr/0009-postgres-sqlalchemy-alembic-schema.md Normal file
View File

@@ -0,0 +1,444 @@
# 0009. Postgres schema with SQLAlchemy 2 and Alembic
## Status
Proposed
## Context
ADR-0008 defines the FastAPI REST boundary: API-key authentication, tenant
resolution from Postgres, source-file ingestion, point management, and chat
thread runs. ADR-0007 defines LangGraph checkpoint persistence and explicitly
avoids making this service the source of truth for chat sessions. We now need
the application-owned Postgres schema that supports those decisions.
The schema has to serve several purposes at once:
- **Multitenancy**: the chatbot serves multiple tenants; every API key belongs
to one tenant and every request resolves to that tenant before touching
Qdrant or LangGraph.
- **Authentication**: each tenant needs API keys. In practice this should mean
*one or more* API keys per tenant, so keys can be rotated, scoped, and
revoked independently.
- **Ingestion records**: when a caller uploads a source file (`.csv`, `.xlsx`,
`.docx`, legacy `.doc` via conversion), the service records the file and the
ingestion job that parsed, chunked, embedded, and wrote Qdrant points.
- **Point/API audit**: point CRUD and batch operations mutate Qdrant, which is
not an audit-log database. The service needs its own record of who changed
what and when.
- **LLM usage and cost**: every model call inside the graph (triage,
contextualize, grade, generate, verify, summary/memory extraction) should be
attributable to a tenant, thread, run, graph node, model, token counts, and
price. For debugging/evals, the system also needs a controlled way to keep
the model input and output.
The project uses SQLAlchemy 2.x and Alembic. The schema should therefore be
specified in SQLAlchemy 2 style (`DeclarativeBase`, `Mapped[...]`,
`mapped_column(...)`) and migrated only through Alembic — never `create_all()`
at FastAPI startup.
## Decision
### SQLAlchemy and migration conventions
Use SQLAlchemy 2.x ORM models with typed mappings:
```python
class Base(DeclarativeBase):
pass
class Tenant(Base):
__tablename__ = "tenants"
id: Mapped[uuid.UUID] = mapped_column(primary_key=True)
slug: Mapped[str] = mapped_column(String(80), unique=True, index=True)
```
Conventions:
- Use SQLAlchemy async sessions in FastAPI (`AsyncSession`) and one session per
request/job unit of work.
- Use Alembic for all DDL. FastAPI startup opens connections and checks
readiness; it does not create or alter tables.
- Prefer UUID primary keys generated by the application. Avoid integer IDs that
leak tenant size and make distributed workers harder to compose.
- Use `timestamptz`/`DateTime(timezone=True)` for all timestamps.
- Use `Numeric(18, 8)` or finer for monetary/cost fields; never floats for
money.
- Store flexible metadata in `JSONB`, but keep relational identifiers and
query-critical fields as typed columns with indexes. If the database column
is named `metadata`, map it with a safe SQLAlchemy attribute such as
`metadata_ = mapped_column("metadata", JSONB, ...)` because `metadata` is
reserved on Declarative models.
- Use string status columns with SQLAlchemy/Pydantic enums and database
`CHECK` constraints rather than PostgreSQL native enums. Status sets change
often during early product work, and native enum migrations are painful.
- Every tenant-owned table has `tenant_id` and an index beginning with
`tenant_id`. Foreign keys include `ondelete` behaviour deliberately, not by
accident.
Postgres uses a shared schema with tenant foreign keys, not database-per-tenant
or schema-per-tenant. This matches ADR-0001's shared Qdrant collection and
keeps tenant count scalable.
### Core tenant and API-key tables
#### `tenants`
One row per customer/tenant.
| Column | Notes |
|---|---|
| `id` | UUID PK. Used as the canonical `tenant_id` injected into Qdrant filters and LangGraph config. |
| `slug` | Stable short name, unique, human-readable. |
| `name` | Display name. |
| `status` | `active` \| `suspended` \| `deleted`. Suspended tenants authenticate to a clear error but cannot run work. |
| `settings` | JSONB for tenant-level feature flags/limits (max upload size, enabled file types, allowed domains, etc.). |
| `created_at`, `updated_at`, `deleted_at` | Audit/soft-delete timestamps. |
#### `tenant_domains`
Optional but recommended. Validates the `domain` values used throughout Qdrant
payloads (`car`, `fire`, etc.) per tenant.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `tenant_id` | FK to `tenants.id`. |
| `domain` | Tenant-local domain key. Unique with `tenant_id`. |
| `display_name` | Human-readable label. |
| `status` | `active` \| `disabled`. |
| `metadata` | JSONB for domain-specific ingestion/retrieval settings. |
This prevents arbitrary caller-supplied domains from silently creating new
partitions in Qdrant.
#### `api_keys`
One tenant can have multiple active keys for rotation and scoped access.
| Column | Notes |
|---|---|
| `id` | UUID PK, also usable as the API key lookup id/prefix. |
| `tenant_id` | FK to `tenants.id`. |
| `name` | Human label, e.g. `main-backend-prod`. |
| `key_prefix` | Short non-secret prefix shown in logs/admin UI, unique. |
| `key_hash` | Hash of the secret key material. Plaintext API keys are never stored. |
| `scopes` | JSONB or text array: `threads:run`, `points:read`, `points:write`, `files:write`, `memory:read`, `admin`. |
| `actor_type` | `backend` \| `admin` \| `worker`. |
| `status` | `active` \| `revoked` \| `expired`. |
| `expires_at`, `revoked_at`, `last_used_at` | Lifecycle timestamps. |
| `created_by`, `created_at`, `updated_at` | Audit fields. |
Authentication dependency in ADR-0008 queries by `key_prefix`/id, verifies
`key_hash` with constant-time comparison, checks tenant/key status and scopes,
and returns `AuthContext`.
### Request and mutation audit tables
#### `api_request_logs`
Append-only request log for authenticated `/v1` calls. This is not a
replacement for structured application logs; it is the durable queryable audit
record.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `tenant_id` | Denormalized from API key for fast tenant queries. |
| `api_key_id` | FK to `api_keys.id`, nullable only for failed auth where key is unknown. |
| `request_id` | Correlation id, unique. |
| `method`, `path_template`, `status_code` | Route identity and result. |
| `scopes_required` | JSONB/text array. |
| `external_user_id` | User id supplied by the main backend, when present. |
| `thread_id` | Present for thread/run routes. No FK — ADR-0007 says this service owns no thread table. |
| `source_ip_hash`, `user_agent` | Optional operational metadata; avoid storing raw IP unless required. |
| `request_summary`, `response_summary` | JSONB summaries, not raw bodies by default. |
| `error_code` | Stable error code from ADR-0008, nullable. |
| `duration_ms`, `created_at` | Timing. |
This table records that an API call happened. Tables below record domain-level
side effects.
#### `point_audit_events`
Append-only audit of `/v1/points` and `/v1/files/{file_id}/points` mutations.
Qdrant remains the storage/search engine; this table records mutation intent
and result.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `tenant_id` | FK to `tenants.id`. |
| `api_request_log_id` | FK to `api_request_logs.id`. |
| `api_key_id` | FK to `api_keys.id`. |
| `operation` | `create` \| `update` \| `payload_patch` \| `soft_delete` \| `hard_delete` \| `reorder` \| `batch`. |
| `point_id` | Qdrant point id, nullable for batch/file-wide operations. |
| `file_id` | Source file id when relevant. |
| `domain` | Qdrant payload domain. |
| `before_version`, `after_version` | Optimistic-concurrency versions when available. |
| `changed_fields` | JSONB list/summary; no large content or vectors. |
| `qdrant_operation_id`, `qdrant_status` | Result returned by Qdrant, if available. |
| `created_at` | Event time. |
### File and ingestion tables
#### `source_files`
One logical source document uploaded by a tenant. Re-ingestion of the same file
creates new jobs against the same or replacement `source_files` row depending
on the `content_hash` policy.
| Column | Notes |
|---|---|
| `id` | UUID PK; this is the `file_id` copied into Qdrant point payloads. |
| `tenant_id` | FK to `tenants.id`. |
| `domain` | Tenant domain, validated by `tenant_domains` where enabled. |
| `source_filename` | Original filename. |
| `source_type` | `csv` \| `xlsx` \| `docx` \| `doc`. |
| `content_sha256` | Hash of the uploaded file bytes for idempotency/change detection. |
| `byte_size` | Upload size. |
| `storage_uri` | Where the original file is stored, if retained. Nullable if not retaining originals. |
| `status` | `active` \| `superseded` \| `soft_deleted` \| `purged`. |
| `created_by_api_key_id`, `created_at`, `updated_at`, `deleted_at` | Audit fields. |
#### `ingestion_jobs`
One attempt to parse/chunk/embed/upsert a source file. This table is required
even if the first implementation processes inline, because ADR-0008's file
upload contract is job-shaped.
| Column | Notes |
|---|---|
| `id` | UUID PK; returned as `ingestion_job_id`. |
| `tenant_id` | FK to `tenants.id`. |
| `source_file_id` | FK to `source_files.id`. |
| `api_request_log_id` | FK to the upload request log. |
| `requested_by_api_key_id` | FK to `api_keys.id`. |
| `status` | `queued` \| `running` \| `succeeded` \| `failed` \| `cancelled`. |
| `chunking_strategy` | `semantic` \| `fixed_size`; matches ADR-0004. |
| `embedding_model_versions` | JSONB map of vector name → model/version. |
| `started_at`, `completed_at` | Lifecycle timestamps. |
| `points_created`, `points_updated`, `points_soft_deleted`, `points_skipped` | Result counters. |
| `error_code`, `error_message` | Failure summary. |
| `metadata` | JSONB for parser/chunker options. |
| `created_at`, `updated_at` | Audit timestamps. |
#### `ingestion_job_events`
Append-only progress/error stream for a job.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `tenant_id` | FK to `tenants.id`. |
| `ingestion_job_id` | FK to `ingestion_jobs.id`. |
| `level` | `info` \| `warning` \| `error`. |
| `stage` | `received` \| `parsed` \| `chunked` \| `embedded` \| `upserted` \| `completed`. |
| `message` | Short human-readable event. |
| `details` | JSONB structured details. |
| `created_at` | Event time. |
### Graph run, LLM usage, and feedback tables
#### `graph_runs`
One row per `POST /v1/threads/{thread_id}/runs`. This is not a session/thread
table: it records one execution for audit, feedback, usage aggregation, and
cost reporting.
| Column | Notes |
|---|---|
| `id` | UUID PK; this is the `run_id` returned by the run endpoint. |
| `tenant_id` | FK to `tenants.id`. |
| `api_request_log_id` | FK to `api_request_logs.id`. |
| `api_key_id` | FK to `api_keys.id`. |
| `thread_id` | LangGraph thread id from the path. No FK. |
| `external_user_id` | User id supplied by the main backend. |
| `status` | `running` \| `answered` \| `clarifying` \| `escalate` \| `failed`. |
| `escalation_reason` | ADR-0006 reason when `status='escalate'`. |
| `input_message_hash` | Hash for idempotency/debug correlation without storing raw text here. |
| `output_message_hash` | Hash of final answer/clarifying/escalation message. |
| `llm_input_tokens`, `llm_output_tokens`, `llm_total_cost` | Denormalized totals from `llm_calls`. |
| `started_at`, `completed_at`, `duration_ms` | Timing. |
| `metadata` | JSONB for graph version, prompt version, retrieved chunk IDs, etc. |
`graph_runs` solves the practical problem left by ADR-0007's feedback endpoint:
feedback needs a stable `run_id`, but this service still does not need a table
that represents chat sessions.
#### `llm_pricing`
Versioned model pricing table so historical cost calculations remain
explainable when model prices change.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `provider` | `anthropic` \| `openai` \| other. |
| `model` | Provider model id. |
| `currency` | Usually `USD`. |
| `input_price_per_1m_tokens`, `output_price_per_1m_tokens` | Numeric. |
| `effective_from`, `effective_to` | Time-bounded price validity. |
| `created_at` | Audit timestamp. |
#### `llm_calls`
One row per provider model call made inside the graph or ingestion pipeline.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `tenant_id` | FK to `tenants.id`. |
| `graph_run_id` | FK to `graph_runs.id`, nullable for ingestion-time LLM calls such as image extraction. |
| `ingestion_job_id` | FK to `ingestion_jobs.id`, nullable for chat-time calls. |
| `api_request_log_id` | FK to the originating request when available. |
| `thread_id`, `external_user_id` | Denormalized for query convenience; nullable outside chat. |
| `node_name` | `triage`, `contextualize`, `grade`, `generate`, `verify`, `summarize`, `memory_extract`, `image_extract`, etc. |
| `provider`, `model`, `model_version` | Provider identity. |
| `pricing_id` | FK to `llm_pricing.id`, nullable if price was configured externally. |
| `input_tokens`, `output_tokens`, `total_tokens` | Provider usage numbers. |
| `input_cost`, `output_cost`, `total_cost`, `currency` | Cost at call time. |
| `latency_ms` | Provider round-trip. |
| `status` | `succeeded` \| `failed` \| `cancelled`. |
| `error_code`, `error_message` | Failure summary. |
| `prompt_version`, `schema_version` | Version of prompt/structured-output schema used. |
| `input_hash`, `output_hash` | Hashes of stored/redacted payloads. |
| `created_at` | Call start time. |
Costs are computed and stored at call time from `llm_pricing` (or explicit
runtime pricing config), not recomputed later from a mutable current price.
#### `llm_call_payloads`
Stores the actual model input/output only when allowed by tenant policy. This
is deliberately separate from `llm_calls` so usage/billing queries never touch
large or sensitive payloads.
| Column | Notes |
|---|---|
| `llm_call_id` | PK/FK to `llm_calls.id`. |
| `tenant_id` | FK to `tenants.id`, repeated for partition/index convenience. |
| `input_redacted` | JSONB/text redacted prompt/messages/tool input. |
| `output_redacted` | JSONB/text redacted model output/tool call result. |
| `input_encrypted`, `output_encrypted` | Optional encrypted raw payload bytes/text if raw retention is enabled. |
| `redaction_version` | Which redaction policy produced the redacted fields. |
| `retention_until` | When payloads must be deleted, independent of usage rows. |
| `created_at` | Timestamp. |
Default policy: store token counts/costs for every call, store **redacted**
input/output for debugging/evals, and store raw encrypted payloads only for
tenants that explicitly enable it. Insurance chat can contain PII and sensitive
claim/coverage information; raw prompt logging cannot be an accidental default.
#### `run_feedback`
Feedback from `POST /v1/threads/{thread_id}/runs/{run_id}/feedback`.
| Column | Notes |
|---|---|
| `id` | UUID PK. |
| `tenant_id` | FK to `tenants.id`. |
| `graph_run_id` | FK to `graph_runs.id`. |
| `external_user_id` | User id from main backend, if present. |
| `rating` | `thumbs_up` \| `thumbs_down` \| numeric score. |
| `reason_codes` | JSONB/text array. |
| `comment` | Optional free text. |
| `created_at` | Timestamp. |
### What is intentionally not modeled
- **No `threads`/`sessions` table.** ADR-0007 remains in force: the main
backend owns session records and LangGraph owns thread checkpoints. This
schema records runs and usage, not conversation ownership.
- **No Postgres copy of Qdrant point content/vectors.** Qdrant remains the
source of truth for point payloads/vectors. Postgres stores source-file,
ingestion, and audit records.
- **No plaintext API keys.** Only hashes and non-secret prefixes.
- **No automatic raw prompt retention.** Raw LLM input/output is opt-in,
encrypted, and retention-limited.
### Indexing and retention
Required indexes:
- `api_keys(key_prefix)` unique; `api_keys(tenant_id, status)`.
- `tenant_domains(tenant_id, domain)` unique.
- `api_request_logs(tenant_id, created_at desc)`, `api_request_logs(request_id)` unique.
- `source_files(tenant_id, domain, created_at desc)`, `source_files(tenant_id, content_sha256)`.
- `ingestion_jobs(tenant_id, status, created_at desc)`, `ingestion_jobs(source_file_id, created_at desc)`.
- `point_audit_events(tenant_id, point_id, created_at desc)`, `point_audit_events(tenant_id, file_id, created_at desc)`.
- `graph_runs(tenant_id, thread_id, started_at desc)`, `graph_runs(tenant_id, external_user_id, started_at desc)`.
- `llm_calls(tenant_id, created_at desc)`, `llm_calls(graph_run_id)`, `llm_calls(ingestion_job_id)`.
- `run_feedback(tenant_id, graph_run_id)`.
Retention:
- Usage/cost rows (`llm_calls`) live longer than payload rows.
- `llm_call_payloads` has the shortest retention and is purged by
`retention_until`.
- API logs and point audit events follow tenant contract/legal retention.
- Deleted tenants are soft-deleted first; hard purge removes API keys, Store
namespaces, checkpointer threads, payload logs, and Qdrant points according
to a separate erasure runbook.
## Consequences
### Positive
- Tenant/API-key authentication has a clear relational source of truth, and
FastAPI dependencies can resolve `AuthContext` with one indexed lookup.
- Ingestion becomes observable and supportable: users can see whether a file
is queued, running, failed, or succeeded, and developers can inspect stage
events without scraping logs.
- Qdrant mutations become auditable even though Qdrant remains the actual
vector/payload store.
- LLM usage is attributable by tenant, thread, run, graph node, model, and
ingestion job, enabling cost reports and per-node optimization.
- Separating `llm_calls` from `llm_call_payloads` keeps billing/analytics fast
and makes sensitive prompt retention a deliberate policy choice.
- `graph_runs` gives the feedback endpoint a stable target while preserving
ADR-0007's decision not to own chat sessions.
### Negative
- This is a larger schema than the minimum needed to answer chat requests.
Implementing all tables up front adds migration and repository code before
the first end-to-end demo.
- There is partial duplication between structured logs and `api_request_logs`;
the former is operational, the latter is durable audit. Both must use the
same `request_id` or they become hard to correlate.
- Storing redacted LLM inputs/outputs still carries privacy risk: redaction can
miss sensitive details, especially in insurance text. Raw encrypted payloads
raise the risk further and need strict access controls.
- Cost calculation depends on pricing data being kept current. If pricing is
wrong at call time, historical costs are wrong unless corrected explicitly.
- Shared-schema tenancy relies on every query and foreign key carrying
`tenant_id`; a missed filter is a data leak. RLS could add defense in depth
later, but it is not part of the initial decision.
## Alternatives Considered
- **Exactly one API key per tenant**: rejected. It makes rotation and scope
separation painful. The requirement is that each tenant can authenticate;
allowing multiple keys per tenant is the safer implementation.
- **Database/schema per tenant**: rejected. It adds migration and operational
overhead per tenant and diverges from ADR-0001's shared Qdrant multitenancy
model. A shared schema with `tenant_id` indexes is simpler and scales better
for this stage.
- **PostgreSQL Row Level Security from day one**: deferred. RLS is useful
defense in depth, but it adds session-variable plumbing and migration/test
complexity. The initial boundary is FastAPI dependency resolution plus
explicit tenant filters and indexes; revisit RLS when the schema stabilizes.
- **Use SQLModel instead of SQLAlchemy ORM**: rejected because the project
explicitly wants SQLAlchemy 2 and Alembic table design. Pydantic request/
response models remain separate from ORM models.
- **Store all HTTP request/response bodies in `api_request_logs`**: rejected.
It would duplicate large payloads, accidentally retain files/prompts, and
raise privacy risk. Store summaries in `api_request_logs`; store controlled
LLM payloads in `llm_call_payloads`; store original files only via
`source_files.storage_uri` if retention policy allows it.
- **Store full Qdrant payloads/vectors in Postgres for audit**: rejected. It
doubles storage and creates two sources of truth. Audit records store change
summaries, ids, versions, and operation outcomes.
- **Compute LLM cost later from token counts**: rejected. Pricing changes over
time. Store the price used and the computed cost with each call so invoices
and reports are reproducible.

455
docs/adr/0010-langfuse-observability-and-prompt-management.md Normal file
View File

@@ -0,0 +1,455 @@
# 0010. Langfuse observability, prompt management, and evaluation workflow
## Status
Proposed
## Context
The chatbot now has architectural decisions for the LangGraph conversation
([0006](0006-conversational-agent-graph.md)), LangGraph thread persistence
([0007](0007-agent-persistence-threads-and-memory.md)), the FastAPI REST
boundary ([0008](0008-rest-api-and-fastapi-boundary.md)), and Postgres tables
for tenants, API keys, ingestion jobs, graph runs, LLM calls, payload retention,
and feedback ([0009](0009-postgres-sqlalchemy-alembic-schema.md)). Those
systems record *what happened* internally, but they do not provide the day-to-day
LLM observability and prompt iteration workflow needed to run an insurance
chatbot safely.
We need to answer questions like:
- Which graph node failed — `triage`, `retrieve`, `generate`, or `verify`?
- Which retrieved chunks did the model see before it answered or escalated?
- Which prompt version produced a bad answer?
- Are prompt changes improving groundedness or just reducing escalations?
- Which tenants, domains, or routes are most expensive?
- Which failures should be fixed by better retrieval, better source content,
better prompts, or stricter handoff policy?
Langfuse is a good fit for the observability and prompt/evaluation layer. It
provides traces, nested observations, generation token/cost tracking, prompt
version management, scores/user feedback, datasets, experiments, evaluators,
and annotation workflows. It should not replace the application database:
Postgres remains the source of truth for API-key authentication, tenant/audit
records, ingestion jobs, graph run rows, billing/cost ledgers, and retention
policy enforcement.
Current Langfuse guidance relevant to this project:
- Use the current Langfuse Python v3 SDK.
- For LangGraph/LangChain, use `from langfuse.langchain import CallbackHandler`
and pass the handler in the graph invocation config.
- Use one trace per user-facing chatbot run.
- Use stable, low-cardinality trace and observation names.
- Model calls should appear as generation observations so model, token, latency,
and cost information can attach correctly.
- Conversation grouping should use Langfuse sessions; in this service,
LangGraph `thread_id` is the Langfuse `session_id`.
- Prompt management uses prompt versions and labels (for example `development`,
`staging`, `production`, tenant-specific labels, and automatically maintained
`latest`). Labels can represent environments, tenants, or experiments.
- Scores represent explicit feedback and implicit/evaluator signals.
- Sensitive data should be masked before export, especially in insurance.
## Decision
### Langfuse is the observability plane, not the transactional database
Adopt Langfuse for:
- traces and nested observations;
- prompt versions, variables, labels, diffs, and trace links;
- user feedback and evaluator scores;
- datasets and experiments;
- annotation queues and systematic error analysis;
- interactive debugging and dashboards.
Keep Postgres ([0009](0009-postgres-sqlalchemy-alembic-schema.md)) as the
internal system of record for:
- tenants and API keys;
- request and point audit records;
- source files and ingestion jobs;
- `graph_runs` and `run_feedback`;
- `llm_calls`, `llm_pricing`, and optional `llm_call_payloads`;
- tenant-specific retention, erasure, and billing/reporting queries.
The two systems are correlated with stable identifiers:
| Identifier | Origin | Use |
|---|---|---|
| `request_id` | FastAPI middleware | Correlate API logs, Postgres audit rows, Langfuse trace metadata. |
| `tenant_id` | API-key dependency | Tenant filtering/cost attribution; never client-supplied. |
| `thread_id` | REST path / LangGraph config | LangGraph thread id and Langfuse `session_id`. |
| `run_id` | `graph_runs.id` | One `POST /v1/threads/{thread_id}/runs` execution. |
| `llm_call_id` | `llm_calls.id` | Optional correlation from a Langfuse generation to Postgres usage ledger. |
| `langfuse_trace_id` | Langfuse | Stored on `graph_runs.metadata` initially, or promoted to a typed column later if frequently queried. |
| `langfuse_observation_id` | Langfuse | Correlated by putting `llm_call_id` in Langfuse observation metadata; add a typed Postgres column later only if needed. |
### SDK and invocation pattern
Use the current Langfuse Python SDK v3 integration for LangGraph/LangChain:
```python
from langfuse import get_client, propagate_attributes
from langfuse.langchain import CallbackHandler
langfuse = get_client()
trace_tags = [environment, "chatbot", "threads:runs"]
trace_metadata = {
"request_id": request_id,
"tenant_id": str(tenant_id),
"api_key_id": str(api_key_id),
"thread_id": thread_id,
"run_id": str(run_id),
"graph_version": graph_version,
"retrieval_config_version": retrieval_config_version,
"prompt_label": prompt_label,
}
with propagate_attributes(
trace_name="chat-run",
user_id=external_user_id,
session_id=thread_id,
tags=trace_tags,
metadata=trace_metadata,
):
langfuse_handler = CallbackHandler()
result = graph.invoke(
graph_input,
config={
"callbacks": [langfuse_handler],
"configurable": {
"thread_id": thread_id,
"tenant_id": str(tenant_id),
"user_id": external_user_id,
},
},
)
```
If the implementation sets Langfuse attributes through LangChain/LangGraph
`config["metadata"]` instead of or in addition to `propagate_attributes`, use the
current integration keys `langfuse_user_id`, `langfuse_session_id`, and
`langfuse_tags` for user, session, and tag propagation.
For non-LangChain/LangGraph work that should appear in the same trace — FastAPI
request spans, Qdrant retrieval, custom reranking metadata, or explicit audit
steps — use Langfuse/OpenTelemetry spans around that work rather than relying
only on automatic LLM callbacks.
Short-lived scripts and tests call `langfuse.flush()` or `langfuse.shutdown()`
before exit. Long-lived FastAPI workers flush during graceful shutdown.
### Trace shape
Create one Langfuse trace per `POST /v1/threads/{thread_id}/runs` call.
Trace-level fields:
| Field | Value |
|---|---|
| name | `chat-run` |
| input | The latest user message, after request-level redaction. |
| output | Final assistant answer, clarifying question, or escalation message. |
| user_id | External `user_id` supplied by the main backend. |
| session_id | LangGraph `thread_id`. |
| tags | Environment, feature (`chatbot`), route (`threads:runs`), tenant segment if safe, terminal status. |
| metadata | `request_id`, `tenant_id`, `api_key_id`, `thread_id`, `run_id`, graph version, retrieval config version, prompt label. |
Observation hierarchy mirrors ADR-0006's graph instead of producing a flat list:
| Observation | Type | Contents |
|---|---|---|
| `load-memory` | span | Store lookup latency and memory keys, not sensitive values. |
| `triage` | generation | Intent classification prompt/output. |
| `contextualize` | generation | Query rewrite input/output. |
| `retrieve` | retriever/span | Qdrant collection, vector names, filters, top-k, returned point/chunk IDs, scores, and source metadata. Avoid raw chunk text unless redacted. |
| `grade` | generation | Relevance verdicts over retrieved chunks. |
| `clarify` | generation | Grounded clarification question when used. |
| `generate` | generation | Final answer generation with citation IDs. |
| `verify` | generation | Groundedness/citation verification result. |
| `escalate` | span | Escalation reason and handoff context summary. |
| `write-memory` | span | Memory write keys and outcome, not sensitive facts. |
Observation names stay stable and low-cardinality. Dynamic values such as tenant
slug, file id, model name, retry number, point id, and user text go in metadata
or input/output fields, not in observation names.
Every generation should include model name, input tokens, output tokens,
latency, and cost when available. Langfuse provides the interactive view;
`llm_calls` in Postgres remains the billing/audit ledger.
### Prompt management and tenant-specific variants
Manage prompts in Langfuse for graph nodes whose behavior will be tuned over
time:
- `insurance-chatbot/triage`
- `insurance-chatbot/contextualize`
- `insurance-chatbot/grade-retrieval`
- `insurance-chatbot/clarify`
- `insurance-chatbot/generate-answer`
- `insurance-chatbot/verify-grounding`
- `insurance-chatbot/summarize-thread`
- `insurance-chatbot/extract-memory`
Prompt text, versions, diffs, labels, and rollback live in Langfuse. Tenant
identity and the policy that chooses which prompt label/name to fetch live in the
application and, where tenant-specific configuration is needed, Postgres
`tenants.settings`.
Prompt names are stable and low-cardinality. The initial tenant-customization
model uses shared prompt names plus tenant-specific labels:
| Label | Use |
|---|---|
| `development` | Local/dev experiments. |
| `staging` | Pre-production validation and experiments. |
| `production` | Default live traffic prompt when no tenant-specific override exists. |
| `tenant-{tenant_slug}-{environment}` | Tenant-specific prompt version for one environment, for example `tenant-acme-production`. |
| `latest` | Automatically points to newest version; useful for discovery, not production runtime. |
Langfuse labels can represent environments, tenants, or experiments. The service
therefore uses labels to support different live prompts per tenant without
creating an application-owned prompt-version table.
Tenant-specific prompt resolution is handled in the application with an explicit
fallback order:
1. derive `tenant_slug` from the authenticated `AuthContext`, never from a
request body/query parameter;
2. derive the environment label from trusted configuration, for example
`production` or `staging`;
3. try the tenant/environment label, for example `tenant-acme-production`;
4. fall back to the environment label, for example `production`;
5. fail closed or use a code-packaged emergency prompt only if the configured
prompt cannot be loaded.
Indicative resolver shape:
```python
prompt_name = "insurance-chatbot/generate-answer"
environment_label = settings.langfuse_prompt_label # e.g. "production"
tenant_label = f"tenant-{auth.tenant_slug}-{environment_label}"
prompt = prompt_resolver.get_with_fallback(
name=prompt_name,
labels=[tenant_label, environment_label],
)
compiled_prompt = prompt.compile(
language=language,
user_message=user_message,
conversation_summary=conversation_summary,
retrieved_chunks=retrieved_chunks,
tenant_policy=tenant_policy,
handoff_policy=handoff_policy,
)
```
The exact SDK call and not-found error handling should follow the current
Langfuse Python SDK, but the architectural rule is stable: select the tenant
label from trusted server-side tenant context, not from caller input.
This keeps prompt content, prompt versions, prompt diffs, and label movement in
Langfuse while keeping tenant identity and resolver policy in the application.
If tenant-specific prompts become large and independently owned, an acceptable
future variant is tenant-namespaced prompt names such as
`insurance-chatbot/tenants/acme/generate-answer` with ordinary `production` and
`staging` labels. The initial design prefers shared prompt names plus
tenant-specific labels because it keeps graph-node prompts easier to compare
across tenants.
Postgres may store prompt resolver configuration in `tenants.settings`, for
example an enabled flag, preferred prompt namespace, explicit label override,
default language, tenant policy reference, or handoff policy reference. Postgres
must not become the source of truth for prompt text/version history; that
belongs in Langfuse.
Prompt variables carry dynamic context, for example:
- `{{language}}`
- `{{user_message}}`
- `{{conversation_summary}}`
- `{{retrieved_chunks}}`
- `{{tenant_policy}}`
- `{{handoff_policy}}`
- `{{citation_contract}}`
- `{{memory_profile}}`
Prompt versions are linked to traces so failures can be traced back to the exact
prompt version that produced them. Rollback is performed by moving the relevant
label — `production` for the default prompt, or `tenant-acme-production` for one
tenant's override — back to a known-good version, not by redeploying code.
Prompt changes are evaluated before promotion using Langfuse datasets and
experiments. A prompt version should not be promoted to `production` or to a
tenant-specific production label only because one manual test looked good.
### Privacy, redaction, and retention
Insurance conversations can contain PII, policy details, claim information, and
financial/health-adjacent facts. Langfuse is therefore sensitive infrastructure.
Rules:
- Mask/redact before export from the application process. Prefer current SDK
masking support such as `mask_otel_spans` where applicable, plus deterministic
field/path redaction for known sensitive fields.
- Never put API keys, database URLs, provider secrets, raw auth headers, or
unredacted request bodies in trace metadata.
- Do not trace raw uploaded files or full document text.
- Retrieval observations should prefer point/chunk IDs, file IDs, scores,
titles, and short redacted snippets over full retrieved chunks.
- Raw LLM input/output retention is governed by ADR-0009's
`llm_call_payloads` tenant policy, not by accidental Langfuse trace capture.
- Langfuse project retention/deletion settings must align with insurance and
tenant contractual retention requirements.
- If self-hosted Langfuse is used later, server-side ingestion masking can add
defense in depth, but it does not replace client-side masking because data
can still reach pre-masking ingestion infrastructure.
### Feedback and scores
Record explicit and implicit feedback as Langfuse scores while mirroring durable
feedback to Postgres `run_feedback` when the application needs it.
Explicit feedback:
| Signal | Langfuse score name | Notes |
|---|---|---|
| Thumbs up/down | `user-thumbs` | Boolean score, attached to the trace/run. |
| Optional comment | score comment/metadata | Also stored in `run_feedback.comment` if enabled. |
| Report bad answer | `user-report` | Useful for annotation queues and support review. |
Implicit feedback:
| Signal | Score/tag | Meaning |
|---|---|---|
| Handoff triggered | `handoff-triggered` | The run ended in `status=escalate`. |
| Regeneration/retry | `user-retry` or metadata | Negative/diagnostic signal. |
| Citation opened | `citation-opened` | User inspected a cited source. |
| Answer copied | `answer-copied` | Weak positive signal if frontend exposes it. |
The FastAPI response for a run should include enough trace/run identity for the
backend or frontend to submit feedback later, but no Langfuse secret key is ever
sent to a browser. Browser-side feedback, if added, may use only a Langfuse
public key or, preferably, route feedback through the backend.
### Evaluation and quality workflow
Use Langfuse beyond basic tracing:
1. **Datasets** — maintain representative insurance questions, expected
citation behavior, ambiguous questions, out-of-scope cases, and handoff
cases.
2. **Experiments** — compare prompt versions, retrieval parameters, reranking
thresholds, model choices, and verification thresholds before deployment.
3. **LLM-as-judge evaluators** — score groundedness, citation correctness,
ambiguity handling, escalation correctness, tone/language, and refusal
quality. Judges are aids, not absolute truth.
4. **Annotation queues** — send low-score traces, user reports, ungrounded
answers, and unnecessary escalations to human review.
5. **Error analysis** — periodically sample traces, open-code failures, cluster
failure modes, and decide whether the fix belongs in retrieval, source
content, prompt text, graph policy, or human handoff rules.
6. **Dashboards** — track per-tenant cost, per-node token spend, latency,
escalation rate, clarification rate, retrieval-insufficient rate, and
verification-failure rate.
### Boundary with ADR-0009 Postgres tables
Langfuse overlaps with parts of ADR-0009, especially LLM-call visibility,
feedback, and run debugging. This overlap is intentional, but the systems have
different authority.
| Need | Langfuse | Postgres | Decision |
|---|---|---|---|
| Prompt text, versions, diffs, labels, and rollback | Yes | No | Use Langfuse as the prompt source of truth. |
| Tenant-specific prompt content | Yes | No | Use Langfuse prompt labels or, later, tenant-namespaced prompt names. |
| Tenant prompt-routing policy | Partly | Yes | Store trusted tenant config in `tenants.settings` or app config; fetch prompt content from Langfuse. |
| Interactive debugging of one bad answer | Yes | Partly | Use Langfuse trace; correlate to `graph_runs.id`. |
| Trace/session grouping | Yes | Partly | Use Langfuse `session_id=thread_id`; keep `thread_id` on `graph_runs` for ledger/audit. |
| LLM token/cost visibility | Yes | Yes | Langfuse for observability; Postgres `llm_calls`/`llm_pricing` for billing and audit. |
| Durable graph-run ledger | No | Yes | Keep `graph_runs`. Langfuse traces are not the application run table. |
| User feedback | Yes | Yes | Store scores in Langfuse and durable feedback in `run_feedback` when needed by the app. |
| Error analysis and annotation queues | Yes | Usually no | Use Langfuse; avoid building an annotation workflow in Postgres initially. |
| Datasets, experiments, and evaluators | Yes | Usually no | Use Langfuse; copy to Postgres only if a product/compliance need appears. |
| Tenants and API keys | No | Yes | Keep `tenants` and `api_keys` in Postgres. |
| API request audit logs | Not enough | Yes | Keep `api_request_logs` in Postgres. |
| Point/file mutation audit | No | Yes | Keep `point_audit_events`, `source_files`, and `ingestion_jobs` in Postgres. |
| Controlled raw/redacted payload retention | Partly | Yes | Postgres `llm_call_payloads` plus tenant policy owns retention; Langfuse must be redacted/configured to comply. |
| Legal retention and tenant erasure policy | Partly | Yes | Postgres owns application policy; Langfuse project retention/deletion settings must align with it. |
Do not remove ADR-0009 tables because Langfuse exists. Langfuse is optimized for
observability, prompt iteration, feedback analysis, datasets, experiments, and
annotation workflows. Postgres is optimized for auth, audit, retention, billing,
and application consistency.
## Consequences
### Positive
- Every chat run becomes inspectable as a graph-shaped trace rather than a flat
log line or a pile of model-call rows.
- Prompt versions can be changed, compared, rolled back, and linked to the
traces they produced without redeploying application code for every prompt
edit.
- Tenant-specific prompt labels let one tenant receive a customized prompt while
other tenants continue using the default `production` version.
- Scores, datasets, experiments, and annotation queues create a path from user
feedback to systematic quality improvement.
- Per-node token/cost/latency data exposes where the graph is expensive or
slow: retrieval, generation, verification, or memory.
- Correlating Langfuse IDs with Postgres IDs preserves both workflows:
interactive debugging and durable audit/billing.
### Negative
- Langfuse adds another service and another set of credentials, retention
policies, access controls, and operational dashboards.
- Prompt management introduces runtime dependency on prompt fetch/caching
behavior. The application needs safe fallbacks if Langfuse is temporarily
unavailable.
- Tenant-specific prompt labels add routing complexity. The resolver must avoid
caller-controlled labels and must have a clear fallback when a tenant override
does not exist.
- Observability can become a privacy leak if masking is incomplete. Insurance
content makes this a high-consequence risk.
- There is intentional duplication between Langfuse generation usage and
Postgres `llm_calls`; implementations must avoid reconciling them as if one
were authoritative for the other's purpose.
- LLM-as-judge evaluators can be wrong. They should guide review and experiments,
not replace human analysis for high-risk policy/coverage answers.
## Alternatives Considered
- **Use Postgres tables only for observability**: rejected. `llm_calls` and
audit tables are good for billing and compliance queries, but poor for
nested trace inspection, prompt-version attribution, annotation queues, and
experiment workflows.
- **Use Langfuse as the only usage/cost store**: rejected. Tenant billing,
erasure, audit, and internal consistency need application-owned tables with
controlled retention and foreign keys to tenants/API keys.
- **Use Langfuse instead of ADR-0009 operational tables**: rejected. Langfuse can
help with traces, prompt versions, scores, datasets, experiments, and
annotation queues, but it should not own API-key authentication, request audit,
point/file mutation audit, ingestion jobs, or the internal billing ledger.
- **Store tenant prompt text and versions in Postgres**: rejected for the initial
design. It would duplicate Langfuse prompt-management features and require the
project to build its own versioning, diffs, labels, rollback, and trace links.
Postgres stores tenant prompt-routing configuration only.
- **Keep prompts only in code**: rejected for prompts that will be tuned often.
Code-only prompts make rollback and prompt/version-to-trace analysis harder.
Small static strings and schema definitions may still live in code.
- **Fetch prompts by `latest` in production**: rejected. `latest` changes every
time a new version is created. Production uses a configured label such as
`production`; rollback is moving the label.
- **Trace full retrieved chunks and raw files for easier debugging**: rejected.
It would make debugging convenient but creates unnecessary privacy and storage
risk. IDs, scores, metadata, and redacted snippets are enough by default.
- **One Langfuse project per tenant**: rejected for the initial design. A single
project per environment with tenant metadata/tags is easier to operate and
compare. Revisit per-tenant projects only if contractual isolation requires
it.
- **Skip Langfuse scores until evals are mature**: rejected. Simple user-thumbs
and handoff-triggered scores are cheap and immediately useful for filtering
traces and building the first annotation queues.

425
docs/adr/0011-python-structured-logging-with-structlog.md Normal file
View File

@@ -0,0 +1,425 @@
# 0011. Python structured logging with structlog and request context
## Status
Proposed
## Context
The service needs application-level Python logging in addition to the durable
Postgres records from ADR-0009 and the LLM/agent traces from ADR-0010.
Postgres audit tables answer durable business questions such as which tenant API
key called an endpoint, which point mutation was requested, which ingestion job
ran, and what usage ledger rows were produced. Langfuse answers LLM observability
questions such as which graph node or prompt version produced an answer.
Structured Python logs answer operational questions while the service is running:
- Which request failed and where?
- Which logs belong to one FastAPI request or LangGraph run?
- Which tenant, API key, thread, run, file, point, or ingestion job was involved?
- Which dependency was slow or unavailable?
- Which fallback path or retry was used?
FastAPI, LangGraph, SQLAlchemy, Qdrant, Langfuse, and HTTP clients can all emit
logs from asynchronous code. Since async tasks can interleave on the same event
loop, relying on process-global mutable variables is unsafe. The logging context
must be request-scoped and safe across async task switching. Python
`contextvars`, exposed through `structlog.contextvars`, provide this behavior.
The user has used a previous `log.py` based on `logging`, `structlog`,
`logging.config.dictConfig`, `ProcessorFormatter`, JSON rendering, stdlib log
capture, and manual `ContextVar` fields such as `session_id` and `property_id`.
This service should keep the same core idea but adapt the field names to the
current architecture:
- `thread_id` instead of `session_id` for LangGraph conversations, matching
ADR-0007 and ADR-0008;
- `tenant_id`, `tenant_slug`, `api_key_id`, and `actor_type` from `AuthContext`;
- `request_id` from FastAPI middleware;
- `run_id` from `graph_runs` for chat runs;
- `file_id`, `point_id`, and `ingestion_job_id` for ingestion and point work;
- `langfuse_trace_id` when available for cross-navigation to ADR-0010 traces.
## Decision
### Use structlog as the application logging interface
Use `structlog` for application logs and integrate it with Python stdlib logging
so framework/library logs are formatted consistently.
Application code imports loggers with:
```python
import structlog
logger = structlog.get_logger(__name__)
```
Log events use stable event names and structured fields:
```python
logger.info(
"graph.run.completed",
status="answered",
duration_ms=duration_ms,
retrieved_chunk_count=len(retrieved_chunks),
)
```
Do not build log messages by interpolating operational metadata into prose.
Prefer fields over long strings because fields are queryable.
### Emit JSON logs by default in production
Production logs are JSON on stdout so process managers, container runtimes, and
log collectors can ingest them directly. Local development may use a colored
console renderer controlled by configuration.
File logging is optional and mainly for local development. If enabled, it must
use explicit rotation settings such as `maxBytes` and `backupCount`. Do not rely
on a default `RotatingFileHandler` with no rotation parameters. In containerized
production, stdout/stderr collection is preferred over writing `logs/app.log`
inside the application container.
### Configure stdlib and structlog together
The logging setup should happen once during process startup, before the FastAPI
app begins serving requests.
Indicative configuration shape:
```python
import logging
import logging.config
import sys
import structlog
def configure_logging(*, log_level: str, json_logs: bool) -> None:
shared_processors = [
structlog.contextvars.merge_contextvars,
structlog.stdlib.add_log_level,
structlog.stdlib.add_logger_name,
structlog.processors.TimeStamper(fmt="iso", utc=True),
structlog.processors.StackInfoRenderer(),
]
structlog.configure(
processors=[
*shared_processors,
structlog.processors.format_exc_info,
structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
],
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
renderer = (
structlog.processors.JSONRenderer()
if json_logs
else structlog.dev.ConsoleRenderer(colors=True)
)
logging.config.dictConfig(
{
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"default": {
"()": structlog.stdlib.ProcessorFormatter,
"processors": [
structlog.stdlib.ProcessorFormatter.remove_processors_meta,
renderer,
],
"foreign_pre_chain": [
structlog.stdlib.ExtraAdder(),
*shared_processors,
],
},
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": log_level,
"formatter": "default",
"stream": sys.stdout,
},
},
"loggers": {
"": {
"handlers": ["console"],
"level": log_level,
"propagate": False,
},
"uvicorn": {
"handlers": ["console"],
"level": log_level,
"propagate": False,
},
"uvicorn.access": {
"handlers": ["console"],
"level": log_level,
"propagate": False,
},
"sqlalchemy.engine": {
"handlers": ["console"],
"level": "WARNING",
"propagate": False,
},
"watchfiles": {
"handlers": ["console"],
"level": "INFO",
"propagate": False,
},
},
}
)
```
Notes:
- Use the logger name `sqlalchemy.engine`, not `sqlalchemy.engin`.
- SQL statement logging is too noisy and can leak values; keep it `WARNING` by
default in production and enable `INFO`/`DEBUG` only in controlled debugging.
- `structlog.stdlib.ExtraAdder()` keeps useful fields from stdlib log records.
- `structlog.contextvars.merge_contextvars` ensures request-bound fields appear
on both structlog and stdlib logs processed through the formatter.
### Bind request context with contextvars
At FastAPI ingress, clear stale context, bind request identifiers, and return the
request id to callers. This makes it possible to select all logs from one request
or one graph run even when async tasks interleave.
Indicative middleware:
```python
from time import perf_counter
from uuid import uuid4
import structlog
from fastapi import Request
from starlette.types import ASGIApp
REQUEST_ID_HEADER = "X-Request-ID"
async def logging_context_middleware(request: Request, call_next: ASGIApp):
structlog.contextvars.clear_contextvars()
request_id = request.headers.get(REQUEST_ID_HEADER) or str(uuid4())
route = request.scope.get("route")
path_template = getattr(route, "path", request.url.path)
structlog.contextvars.bind_contextvars(
request_id=request_id,
method=request.method,
path_template=path_template,
)
logger = structlog.get_logger("app.http")
started = perf_counter()
logger.info("request.started")
try:
response = await call_next(request)
except Exception:
logger.exception(
"request.failed",
duration_ms=round((perf_counter() - started) * 1000, 2),
)
raise
response.headers[REQUEST_ID_HEADER] = request_id
logger.info(
"request.completed",
status_code=response.status_code,
duration_ms=round((perf_counter() - started) * 1000, 2),
)
return response
```
After API-key authentication succeeds, the auth dependency or route handler binds
trusted tenant/auth fields:
```python
structlog.contextvars.bind_contextvars(
tenant_id=str(auth.tenant_id),
tenant_slug=auth.tenant_slug,
api_key_id=str(auth.api_key_id),
actor_type=auth.actor_type,
)
```
Route handlers bind route-specific fields when they become known:
```python
structlog.contextvars.bind_contextvars(
external_user_id=body.user_id,
thread_id=thread_id,
run_id=str(run_id),
)
```
Use these canonical context keys:
| Field | Source | Notes |
|---|---|---|
| `request_id` | FastAPI middleware | Primary log correlation id; also appears in ADR-0008/0009 records. |
| `tenant_id` | `AuthContext` | Trusted server-side tenant id; never request body/query. |
| `tenant_slug` | `AuthContext` | Useful for filtering; avoid if contractual policy treats it as sensitive. |
| `api_key_id` | `AuthContext` | Non-secret id only. Never log raw API keys or auth headers. |
| `actor_type` | `AuthContext` | `backend`, `admin`, or `worker`. |
| `external_user_id` | Main backend | May be high-cardinality; acceptable in logs, not metrics labels. |
| `thread_id` | REST path | LangGraph thread id. |
| `run_id` | `graph_runs.id` | One chat run. |
| `ingestion_job_id` | `ingestion_jobs.id` | File ingestion correlation. |
| `file_id` | `source_files.id` | Source-file correlation. |
| `point_id` | Qdrant point id | Point mutation/read correlation. |
| `langfuse_trace_id` | Langfuse | Cross-link to ADR-0010 trace when available. |
Use `structlog.contextvars.clear_contextvars()` at request/task ingress to avoid
leaking a previous request's context into reused workers.
### Bind context explicitly for jobs and background work
Context variables work across normal async task switching, but background jobs,
worker processes, scheduled jobs, and threadpool work should bind context at
their own entry point from durable identifiers.
Examples:
- ingestion worker binds `tenant_id`, `ingestion_job_id`, `file_id`, and
`request_id` if inherited from the upload request;
- LangGraph run execution binds `thread_id`, `run_id`, `tenant_id`, and
`external_user_id` before invoking the graph;
- point batch workers bind `tenant_id`, `api_request_log_id`, and operation
metadata before processing each batch.
If code crosses a boundary where contextvars may not propagate automatically,
pass the identifiers explicitly and bind them again at the boundary.
### Log levels and event naming
Use log levels consistently:
| Level | Use |
|---|---|
| `DEBUG` | Local diagnostics, disabled by default in production. |
| `INFO` | Normal lifecycle events: request started/completed, graph run completed, ingestion job completed. |
| `WARNING` | Recoverable anomalies: fallback prompt used, retry scheduled, insufficient retrieval before clarification/escalation. |
| `ERROR` | Failed operations requiring attention: unhandled exception, dependency outage, ingestion failure. |
Do not log expected user behavior at `ERROR`. For example, a user asking an
ambiguous question that leads to clarification is an `INFO` event; a retriever
being unavailable is an `ERROR` event.
Use stable dot-separated event names:
- `request.started`
- `request.completed`
- `request.failed`
- `auth.succeeded`
- `auth.failed`
- `graph.run.started`
- `graph.run.completed`
- `graph.run.escalated`
- `retrieval.completed`
- `retrieval.insufficient`
- `llm.call.completed`
- `llm.call.failed`
- `ingestion.job.started`
- `ingestion.job.completed`
- `point.mutation.completed`
Do not include dynamic values in logger names or event names. Put dynamic values
in structured fields.
### Security and privacy rules
Logs must not contain secrets or raw sensitive payloads.
Never log:
- plaintext API keys;
- `Authorization` headers;
- database URLs or provider credentials;
- raw uploaded file contents;
- full retrieved chunks by default;
- raw user messages, raw prompts, or raw model outputs by default;
- embeddings or vectors.
Prefer:
- ids (`request_id`, `thread_id`, `run_id`, `file_id`, `point_id`);
- hashes (`input_message_hash`, `output_message_hash`, `content_sha256`);
- counts, sizes, durations, and status codes;
- short redacted summaries only when useful and allowed by tenant policy.
The same redaction policy used for ADR-0009 `llm_call_payloads` and ADR-0010
Langfuse tracing should guide log redaction. Logging should be safe even when
log aggregation has broader access than the application database.
### Relationship to Postgres and Langfuse
Structured logs complement but do not replace ADR-0009 and ADR-0010.
| Question | System of record |
|---|---|
| What happened operationally inside this process? | Structured logs. |
| Which API key called which endpoint and what durable side effect occurred? | Postgres audit tables from ADR-0009. |
| Which graph node, prompt version, retrieved chunks, and model calls produced an answer? | Langfuse traces from ADR-0010. |
| What should be used for tenant billing and compliance reports? | Postgres `llm_calls`, `llm_pricing`, `api_request_logs`, and audit tables. |
| What should be used for interactive debugging of one LLM answer? | Langfuse trace, linked from logs/Postgres by ids. |
Logs may contain `request_id`, `run_id`, and `langfuse_trace_id` so engineers can
navigate across all three systems.
## Consequences
### Positive
- Logs become queryable by `request_id`, `thread_id`, `run_id`, `tenant_id`,
`file_id`, and `ingestion_job_id`.
- Contextvars prevent async task interleaving from mixing request context.
- Stdlib/framework logs and application logs share one JSON structure.
- Production logs are compatible with common log collectors and container
runtimes.
- Logs, Postgres audit rows, and Langfuse traces can be correlated without
duplicating each system's purpose.
### Negative
- Logging setup is more complex than plain `logging.basicConfig()`.
- Developers must learn to use structured fields instead of prose-only log
messages.
- Context must be rebound at worker/background-task boundaries.
- Too much logging can increase cost and leak sensitive data if redaction rules
are not followed.
- JSON logs are less pleasant locally unless a console renderer is enabled for
development.
## Alternatives Considered
- **Use Python stdlib logging only**: rejected. Stdlib logging can work, but
`structlog` gives cleaner structured context, contextvars integration, and
consistent event dictionaries across application and framework logs.
- **Use manual `ContextVar` fields only**: rejected as the default. Manual
context variables work, but `structlog.contextvars.bind_contextvars()` and
`merge_contextvars` provide a standard way to bind arbitrary request fields
without maintaining one `ContextVar` per field. Manual `ContextVar`s may still
be used for special cases.
- **Use `session_id` as the primary chat correlation field**: rejected for this
service. ADR-0007 standardizes on `thread_id`, which maps to LangGraph threads
and Langfuse sessions. If the main backend calls the same concept a session,
it is translated to `thread_id` at this service boundary.
- **Write only to `logs/app.log`**: rejected for production. File logging is
useful locally, but stdout JSON is the better default for deployed services.
- **Use Langfuse for all observability**: rejected. Langfuse is excellent for
LLM/agent traces, prompt versions, scores, and evals, but it is not a
replacement for process logs covering FastAPI middleware, auth, SQLAlchemy,
Qdrant calls, worker lifecycle, and non-LLM failures.
- **Use Postgres audit tables as logs**: rejected. ADR-0009 tables are durable
business/audit records. They should not receive high-volume operational debug
logs.