docs(observability): define Langfuse and structured logging strategy

Why:
- Establish separate observability systems for LLM tracing, prompt iteration, evaluation, operational logs, and durable application audit records.

Changes:
- Define Langfuse traces, prompt labels, feedback scores, evaluation workflows, redaction rules, and correlation identifiers.
- Define structlog-based JSON logging, request context propagation, event naming, log levels, and privacy requirements.

Impact:
- Langfuse remains the LLM observability plane, while Postgres remains the durable audit and billing source of truth.
- Application logs must avoid secrets and raw sensitive payloads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs/adr/0002-chunk-crud-and-search-api.md

@@ -4,6 +4,11 @@
 
 Accepted
 
+Route naming note: ADR-0008 owns the REST API surface and renames the
+indicative endpoint examples in this ADR from `/chunks/...` to `/points/...`.
+The CRUD semantics, Qdrant primitives, payload schema, soft-delete behavior,
+and tenant-isolation rules in this ADR remain accepted.
+
 ## Context
 
 Beyond bulk ingestion ([0001](0001-ingestion-pipeline-and-collection-schema.md)),
@@ -24,16 +29,16 @@ the `chunks` collection and payload schema from ADR-0001:
 
 | Operation | FastAPI endpoint (indicative) | Qdrant primitive |
 |---|---|---|
-| Create a chunk | `POST /chunks` | upsert (single point) |
-| Update a chunk's vectors/content | `PUT /chunks/{chunk_id}` | upsert (`update_only` mode) |
-| Partially update payload | `PATCH /chunks/{chunk_id}/payload` | `set_payload` / `overwrite_payload` |
-| Delete one chunk | `DELETE /chunks/{chunk_id}` | delete by ID |
-| Delete many chunks | `DELETE /chunks?file_id=...` | delete by filter |
-| List/paginate chunks, in order | `GET /chunks?file_id=...` | `scroll` with filter + pagination, `order_by: order_id` |
-| Reorder/insert a chunk | `PATCH /chunks/{chunk_id}/order` | `set_payload` on `order_id` only |
-| Count chunks | `GET /chunks/count` | `count` |
-| Keyword search | `GET /chunks/search?q=...` | full-text payload index on `content` (match, not semantic) |
-| Bulk multi-op edits | `POST /chunks/batch` | Qdrant `points/batch` |
+| Create a point | `POST /points` | upsert (single point) |
+| Update a point's vectors/content | `PUT /points/{point_id}` | upsert (`update_only` mode) |
+| Partially update payload | `PATCH /points/{point_id}/payload` | `set_payload` / `overwrite_payload` |
+| Delete one point | `DELETE /points/{point_id}` | delete by ID |
+| Delete many points | `DELETE /points?file_id=...` | delete by filter |
+| List/paginate points, in order | `GET /points?file_id=...` | `scroll` with filter + pagination, `order_by: order_id` |
+| Reorder/insert a point | `PATCH /points/{point_id}/order` | `set_payload` on `order_id` only |
+| Count points | `GET /points/count` | `count` |
+| Keyword search | `GET /points/search?q=...` | full-text payload index on `content` (match, not semantic) |
+| Bulk multi-op edits | `POST /points/batch` | Qdrant `points/batch` |
 
 ### Ordering and reordering
 
@@ -45,7 +50,7 @@ inserting a chunk only requires assigning it a new value between its two new
 neighbors — no renumbering of siblings, and no effect on the chunk's stable
 point ID (which is derived from the immutable `chunk_index`, not `order_id`).
 
-`PATCH /chunks/{chunk_id}/order` does more than set one field, though: since
+`PATCH /points/{point_id}/order` does more than set one field, though: since
 ADR-0001 also maintains `previous_chunk_id`/`next_chunk_id` pointers for O(1)
 adjacency lookups, a single reorder touches **up to four chunks** in one
 `points/batch` call:
@@ -62,7 +67,7 @@ go stale.
 
 ### Keyword search is not semantic search
 
-`GET /chunks/search` matches against the full-text payload index on
+`GET /points/search` matches against the full-text payload index on
 `content` (and structured filters on `tenant_id`, `domain`, `file_id`,
 etc.) — it is filter/match-based keyword search, not embedding-based
 retrieval. This is deliberately distinct from the hybrid dense+sparse
@@ -72,11 +77,11 @@ context) and should not be conflated in the API or in future discussion.
 
 ### Delete is soft by default
 
-`DELETE /chunks/{chunk_id}` and `DELETE /chunks?file_id=...` set
+`DELETE /points/{point_id}` and `DELETE /points?file_id=...` set
 `is_active: false` and `deleted_at` (via `set_payload`) rather than removing
 the point from Qdrant — consistent with ADR-0001's soft-delete fields. This
-keeps deleted chunks available for audit and lets `GET /chunks` and
-`GET /chunks/search` filter them out by default (`is_active: true` implied
+keeps deleted points available for audit and lets `GET /points` and
+`GET /points/search` filter them out by default (`is_active: true` implied
 unless the caller explicitly asks to include inactive chunks). A hard delete
 (actual point removal, e.g. `points/delete`) is available separately for
 compliance-driven purges, not as the default CRUD behavior. Either way, the

docs/adr/0008-rest-api-and-fastapi-boundary.md

@@ -0,0 +1,298 @@
+# 0008. REST API and FastAPI boundary
+
+## Status
+
+Proposed
+
+## Context
+
+The project now has decisions for ingestion and Qdrant storage
+([0001](0001-ingestion-pipeline-and-collection-schema.md)), point-level CRUD
+and keyword search ([0002](0002-chunk-crud-and-search-api.md)), agent retrieval
+([0003](0003-agent-hybrid-retrieval.md)), conversational graph behaviour
+([0006](0006-conversational-agent-graph.md)), and LangGraph thread persistence
+([0007](0007-agent-persistence-threads-and-memory.md)). What is still missing
+is the HTTP boundary that ties these together.
+
+FastAPI has to do more than expose thin route handlers:
+
+- Authenticate the calling backend/API client with an API key.
+- Resolve that API key to an active tenant in Postgres, using the tenant/API-key
+  tables defined by [ADR-0009](0009-postgres-sqlalchemy-alembic-schema.md).
+- Inject `tenant_id` into every Qdrant operation and every LangGraph run, so
+  callers never submit or override the tenant boundary themselves.
+- Receive user chat messages and pass them to the compiled LangGraph graph.
+- Receive source files (starting with CSV, but aligned with the docx/xlsx/csv
+  ingestion decisions from ADR-0001/0004) and kick off ingestion.
+- Expose direct management of Qdrant records. ADR-0002 originally sketched
+  `/chunks/...` endpoint names, but the API is directly managing Qdrant
+  **points**. The REST surface should use `/points/...` while the payload can
+  still contain chunk-specific fields such as `chunk_id`, `content`, and
+  `previous_chunk_id`.
+
+This ADR owns the public REST shape for this FastAPI service. If its endpoint
+names differ from indicative endpoint examples in older ADRs, this ADR wins;
+the older ADRs still own their data-model and pipeline semantics.
+
+## Decision
+
+### FastAPI application structure
+
+Use a single versioned API prefix, `/v1`, and split the app into routers by
+resource:
+
+| Router | Prefix | Responsibility |
+|---|---|---|
+| `threads` | `/v1/threads` | Chat thread runs, transcript pagination, thread purge. |
+| `files` | `/v1/files` | Upload source files and inspect ingestion status. |
+| `points` | `/v1/points` | Direct CRUD, ordering, payload updates, keyword search, and batch edits over Qdrant points. |
+| `users` | `/v1/users` | Cross-thread memory inspect/reset. |
+| `health` | `/healthz`, `/readyz` | Process health and dependency readiness. |
+
+Routers use FastAPI's dependency injection with `Annotated[..., Depends(...)]`.
+Shared dependencies live at the router level where they only gate access, and
+as typed parameters where the endpoint needs the resolved value. Path
+operation functions stay one HTTP operation per function.
+
+The app lifespan owns long-lived resources:
+
+- SQLAlchemy async engine/sessionmaker.
+- Qdrant client.
+- LangGraph checkpointer/store and the compiled graph (compiled once, reused).
+- Any ingestion/retrieval model clients.
+
+No DDL runs at FastAPI startup. Postgres schema changes are Alembic migrations;
+LangGraph checkpoint/store setup is a deployment step as decided in ADR-0007.
+
+### Authentication and tenant dependency
+
+Every `/v1` route except health checks requires an API key. The key is supplied
+as a bearer token:
+
+```http
+Authorization: Bearer <api_key>
+```
+
+A FastAPI dependency validates the key against Postgres and returns an
+`AuthContext`:
+
+```text
+AuthContext(
+  tenant_id,
+  tenant_slug,
+  api_key_id,
+  scopes,
+  actor_type="backend" | "admin" | "worker",
+)
+```
+
+Validation rules:
+
+1. API keys are stored only as hashes, never plaintext.
+2. The presented key is split into a lookup prefix/key id plus secret material;
+   Postgres lookup uses the prefix/id, then compares the stored hash with a
+   constant-time comparison.
+3. The key must be active: tenant active, key not revoked, key not expired.
+4. The key must carry the scope required by the router/action.
+5. `tenant_id` is derived only from the key's tenant relationship.
+
+The service never accepts `tenant_id` in request bodies or query strings. Any
+client-supplied tenant field is rejected as an invalid request. This preserves
+the ADR-0001/0002 Qdrant tenant-isolation rule and extends it to the REST
+boundary.
+
+Recommended dependency aliases:
+
+```python
+SessionDep = Annotated[AsyncSession, Depends(get_db_session)]
+AuthContextDep = Annotated[AuthContext, Depends(get_auth_context)]
+```
+
+Scope dependencies compose on top of `AuthContextDep`, for example
+`require_scope("points:write")`, `require_scope("threads:run")`, and
+`require_scope("files:write")`.
+
+### Thread/run endpoints
+
+Chat uses the LangGraph vocabulary already chosen in ADR-0007.
+
+| Endpoint | Purpose |
+|---|---|
+| `POST /v1/threads/{thread_id}/runs` | Execute one graph run for a new user message. |
+| `GET /v1/threads/{thread_id}/messages?limit=20&before=...` | Return a paginated transcript window for the frontend. |
+| `DELETE /v1/threads/{thread_id}` | Purge the thread checkpoints for explicit clear/erasure. |
+| `POST /v1/threads/{thread_id}/runs/{run_id}/feedback` | Attach feedback to the answer produced by a run. |
+
+`POST /runs` passes this context into LangGraph:
+
+- `thread_id` from the path.
+- `tenant_id` from `AuthContext`, not from the caller body.
+- `user_id` from the trusted main-backend request payload/header.
+- locale/language from the request, with server-side fallback.
+
+The endpoint streams with Server-Sent Events when the caller asks for
+streaming. Token events come from LangGraph `stream_mode="messages"`; progress
+and terminal status events come from `stream_mode="custom"` or explicit route
+wrapping. Terminal statuses are the ADR-0006 statuses: `answered`,
+`clarifying`, or `escalate`. When `escalate` is returned, the main backend uses
+its existing human-assistant transfer path and stops issuing runs for that
+thread.
+
+One in-flight run per thread is allowed. The route takes a Postgres advisory
+lock keyed on `thread_id`; a concurrent run returns `409 Conflict`.
+
+### File ingestion endpoints
+
+Files are source documents, not Qdrant points. Uploading a CSV creates or
+re-ingests a file and then creates/updates many points via the ADR-0001/0004
+ingestion pipeline.
+
+| Endpoint | Purpose |
+|---|---|
+| `POST /v1/files` | Upload one `.csv`, `.xlsx`, `.docx`, or converted `.doc` source file for ingestion. Initial callers may only use CSV, but the route is named for the generalized source-file concept. |
+| `GET /v1/files/{file_id}` | Return ingestion status and source-file metadata. |
+| `GET /v1/files/{file_id}/points?limit=...&offset=...` | List active points belonging to a file, ordered by `order_id`. |
+| `DELETE /v1/files/{file_id}` | Soft-delete all active points for a file, preserving audit data unless a future compliance hard-delete endpoint is added. |
+
+`POST /v1/files` uses FastAPI `UploadFile` and multipart form fields for
+metadata such as `domain`. File validation is server-side:
+
+- Allowlist extensions/content types: `.csv`, `.xlsx`, `.docx`; legacy `.doc`
+  is accepted only if the conversion pipeline from ADR-0004 is enabled.
+- Enforce maximum file size before reading the full body into memory.
+- Sniff enough content to reject obvious extension spoofing.
+- Derive `tenant_id`, `created_by`, and `updated_by` from `AuthContext`, not
+  from form fields.
+
+Ingestion may be slow because it parses, chunks, embeds, and writes many
+Qdrant points. The REST contract is job-shaped even if the first
+implementation runs inline:
+
+```text
+202 Accepted -> { file_id, ingestion_job_id, status: "queued" | "running" }
+```
+
+A durable worker/job queue can be added later without changing the API
+contract.
+
+### Point endpoints replace the older `/chunks` sketches
+
+The Qdrant collection is still named `chunks` in ADR-0001, because each stored
+point represents one chunk. The REST resource is nevertheless `/points`,
+because callers are managing Qdrant point records and the operations map
+almost one-to-one to Qdrant's Points API.
+
+| Endpoint | Qdrant primitive | Notes |
+|---|---|---|
+| `POST /v1/points` | upsert one point | Manual/admin creation of one chunk-backed point. |
+| `GET /v1/points/{point_id}` | retrieve by ID | Returns vectors only when explicitly requested. |
+| `PUT /v1/points/{point_id}` | upsert with `update_only` | Replaces content/vectors/payload with optimistic concurrency. |
+| `PATCH /v1/points/{point_id}/payload` | `set_payload` / `overwrite_payload` | Payload-only update. |
+| `DELETE /v1/points/{point_id}` | payload update or delete | Soft-delete by default (`is_active=false`, `deleted_at`). |
+| `GET /v1/points?file_id=...` | `scroll` | List/paginate active points in `order_id` order. |
+| `PATCH /v1/points/{point_id}/order` | `points/batch` | Move/insert point and relink neighbors. |
+| `GET /v1/points/count` | `count` | Count active points under tenant/domain filters. |
+| `GET /v1/points/search?q=...` | full-text payload match | Keyword search over `content`, not semantic retrieval. |
+| `POST /v1/points/batch` | `points/batch` | Bulk multi-operation edits. |
+
+Every operation injects a Qdrant filter containing:
+
+- `tenant_id == auth_context.tenant_id`
+- `is_active == true` by default on reads
+- optional `domain`, `file_id`, and other allowed filters supplied by caller
+
+`point_id` names the Qdrant point ID in the URL. Payload fields keep their
+ADR-0001 names (`chunk_id`, `file_id`, `previous_chunk_id`, etc.) so the API
+name change does not rename the storage schema.
+
+### Response and error conventions
+
+Use explicit return types or `response_model` to validate and filter public
+responses. Do not return raw ORM objects, raw Qdrant responses, or full
+LangGraph state.
+
+Error responses use a stable envelope:
+
+```json
+{
+  "error": {
+    "code": "tenant_not_found | invalid_api_key | missing_scope | validation_error | conflict | not_found | internal_error",
+    "message": "human-readable summary",
+    "details": {},
+    "request_id": "..."
+  }
+}
+```
+
+Important status codes:
+
+| Status | Use |
+|---|---|
+| `202 Accepted` | Ingestion accepted as a job. |
+| `400 Bad Request` | Invalid domain/filter combinations or unsupported file type. |
+| `401 Unauthorized` | Missing/invalid API key. |
+| `403 Forbidden` | Valid key without required scope. |
+| `404 Not Found` | Resource absent within this tenant. |
+| `409 Conflict` | Concurrent thread run or optimistic-concurrency mismatch. |
+| `422 Unprocessable Entity` | Pydantic request validation failure. |
+
+### Observability
+
+Every request gets a `request_id` and structured logs containing route,
+status, latency, tenant id, and API key id — never the API key itself. LangGraph
+runs include the same request id in `config` metadata so API logs, graph traces,
+and Qdrant operations can be correlated.
+
+## Consequences
+
+### Positive
+- Tenant isolation starts at the FastAPI dependency boundary, before any route
+  logic reaches Qdrant or LangGraph.
+- `/points` aligns the REST API with what it actually manages — Qdrant point
+  records — while preserving the chunk payload schema underneath.
+- `/threads/{thread_id}/runs` remains compatible with the LangGraph thread/run
+  model already chosen in ADR-0007.
+- Job-shaped file ingestion lets the first implementation be simple while
+  keeping room for a durable worker without breaking clients.
+- Router-level dependencies and typed FastAPI dependencies keep auth, tenant
+  resolution, sessions, and scopes reusable instead of repeated per endpoint.
+
+### Negative
+- The API now has two words for related concepts: `files` are source documents,
+  `points` are the chunk records produced from them. This is accurate but
+  requires clear frontend/backend naming.
+- `/points` exposes storage terminology to API consumers. That is acceptable
+  for this internal/admin-oriented service, but it would be less friendly as a
+  public product API.
+- API-key auth in Postgres adds a database lookup to every request unless
+  short-lived caching is introduced. Caching must preserve revocation semantics.
+- A job-shaped ingestion contract needs a job status store even if the initial
+  implementation processes inline.
+- The REST layer now depends on the tenant/API-key, ingestion-job, audit, and
+  usage tables defined in [ADR-0009](0009-postgres-sqlalchemy-alembic-schema.md).
+
+## Alternatives Considered
+
+- **Keep `/chunks/...` from ADR-0002**: rejected. The storage record being
+  managed is a Qdrant point, and several operations (`points/batch`, retrieve
+  by point ID, vector inclusion flags) map directly to Qdrant's Points API.
+  Keeping `/chunks` would make the REST layer look more domain-friendly but
+  less honest about what it does.
+- **Accept `tenant_id` in every request**: rejected. Tenant is an isolation
+  boundary, not user input. It must come from the API key's Postgres tenant
+  relationship.
+- **Use `X-API-Key` instead of `Authorization: Bearer`**: rejected as the
+  default because bearer tokens are standard for service-to-service auth and
+  work cleanly with middleware, proxies, and generated clients. A compatibility
+  alias can be added later if an upstream system cannot send `Authorization`.
+- **Expose one generic `/v1/qdrant/*` proxy**: rejected. It would leak Qdrant's
+  full API surface, bypass tenant/scoping rules too easily, and couple clients
+  to storage operations the service should hide.
+- **Synchronous file ingestion only**: rejected as the contract. It is simpler
+  to implement, but embedding and late-interaction vector generation can be
+  slow enough to exceed HTTP timeouts. The job-shaped response gives the
+  implementation room to evolve.
+- **Create threads with `POST /v1/threads`**: rejected for now. The main
+  backend owns conversation/session records, and LangGraph can create a
+  checkpoint sequence on the first run for a `thread_id`. A create endpoint
+  would imply metadata this service has explicitly chosen not to store.

docs/adr/0009-postgres-sqlalchemy-alembic-schema.md

@@ -0,0 +1,444 @@
+# 0009. Postgres schema with SQLAlchemy 2 and Alembic
+
+## Status
+
+Proposed
+
+## Context
+
+ADR-0008 defines the FastAPI REST boundary: API-key authentication, tenant
+resolution from Postgres, source-file ingestion, point management, and chat
+thread runs. ADR-0007 defines LangGraph checkpoint persistence and explicitly
+avoids making this service the source of truth for chat sessions. We now need
+the application-owned Postgres schema that supports those decisions.
+
+The schema has to serve several purposes at once:
+
+- **Multitenancy**: the chatbot serves multiple tenants; every API key belongs
+  to one tenant and every request resolves to that tenant before touching
+  Qdrant or LangGraph.
+- **Authentication**: each tenant needs API keys. In practice this should mean
+  *one or more* API keys per tenant, so keys can be rotated, scoped, and
+  revoked independently.
+- **Ingestion records**: when a caller uploads a source file (`.csv`, `.xlsx`,
+  `.docx`, legacy `.doc` via conversion), the service records the file and the
+  ingestion job that parsed, chunked, embedded, and wrote Qdrant points.
+- **Point/API audit**: point CRUD and batch operations mutate Qdrant, which is
+  not an audit-log database. The service needs its own record of who changed
+  what and when.
+- **LLM usage and cost**: every model call inside the graph (triage,
+  contextualize, grade, generate, verify, summary/memory extraction) should be
+  attributable to a tenant, thread, run, graph node, model, token counts, and
+  price. For debugging/evals, the system also needs a controlled way to keep
+  the model input and output.
+
+The project uses SQLAlchemy 2.x and Alembic. The schema should therefore be
+specified in SQLAlchemy 2 style (`DeclarativeBase`, `Mapped[...]`,
+`mapped_column(...)`) and migrated only through Alembic — never `create_all()`
+at FastAPI startup.
+
+## Decision
+
+### SQLAlchemy and migration conventions
+
+Use SQLAlchemy 2.x ORM models with typed mappings:
+
+```python
+class Base(DeclarativeBase):
+    pass
+
+class Tenant(Base):
+    __tablename__ = "tenants"
+
+    id: Mapped[uuid.UUID] = mapped_column(primary_key=True)
+    slug: Mapped[str] = mapped_column(String(80), unique=True, index=True)
+```
+
+Conventions:
+
+- Use SQLAlchemy async sessions in FastAPI (`AsyncSession`) and one session per
+  request/job unit of work.
+- Use Alembic for all DDL. FastAPI startup opens connections and checks
+  readiness; it does not create or alter tables.
+- Prefer UUID primary keys generated by the application. Avoid integer IDs that
+  leak tenant size and make distributed workers harder to compose.
+- Use `timestamptz`/`DateTime(timezone=True)` for all timestamps.
+- Use `Numeric(18, 8)` or finer for monetary/cost fields; never floats for
+  money.
+- Store flexible metadata in `JSONB`, but keep relational identifiers and
+  query-critical fields as typed columns with indexes. If the database column
+  is named `metadata`, map it with a safe SQLAlchemy attribute such as
+  `metadata_ = mapped_column("metadata", JSONB, ...)` because `metadata` is
+  reserved on Declarative models.
+- Use string status columns with SQLAlchemy/Pydantic enums and database
+  `CHECK` constraints rather than PostgreSQL native enums. Status sets change
+  often during early product work, and native enum migrations are painful.
+- Every tenant-owned table has `tenant_id` and an index beginning with
+  `tenant_id`. Foreign keys include `ondelete` behaviour deliberately, not by
+  accident.
+
+Postgres uses a shared schema with tenant foreign keys, not database-per-tenant
+or schema-per-tenant. This matches ADR-0001's shared Qdrant collection and
+keeps tenant count scalable.
+
+### Core tenant and API-key tables
+
+#### `tenants`
+
+One row per customer/tenant.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. Used as the canonical `tenant_id` injected into Qdrant filters and LangGraph config. |
+| `slug` | Stable short name, unique, human-readable. |
+| `name` | Display name. |
+| `status` | `active` \| `suspended` \| `deleted`. Suspended tenants authenticate to a clear error but cannot run work. |
+| `settings` | JSONB for tenant-level feature flags/limits (max upload size, enabled file types, allowed domains, etc.). |
+| `created_at`, `updated_at`, `deleted_at` | Audit/soft-delete timestamps. |
+
+#### `tenant_domains`
+
+Optional but recommended. Validates the `domain` values used throughout Qdrant
+payloads (`car`, `fire`, etc.) per tenant.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `tenant_id` | FK to `tenants.id`. |
+| `domain` | Tenant-local domain key. Unique with `tenant_id`. |
+| `display_name` | Human-readable label. |
+| `status` | `active` \| `disabled`. |
+| `metadata` | JSONB for domain-specific ingestion/retrieval settings. |
+
+This prevents arbitrary caller-supplied domains from silently creating new
+partitions in Qdrant.
+
+#### `api_keys`
+
+One tenant can have multiple active keys for rotation and scoped access.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK, also usable as the API key lookup id/prefix. |
+| `tenant_id` | FK to `tenants.id`. |
+| `name` | Human label, e.g. `main-backend-prod`. |
+| `key_prefix` | Short non-secret prefix shown in logs/admin UI, unique. |
+| `key_hash` | Hash of the secret key material. Plaintext API keys are never stored. |
+| `scopes` | JSONB or text array: `threads:run`, `points:read`, `points:write`, `files:write`, `memory:read`, `admin`. |
+| `actor_type` | `backend` \| `admin` \| `worker`. |
+| `status` | `active` \| `revoked` \| `expired`. |
+| `expires_at`, `revoked_at`, `last_used_at` | Lifecycle timestamps. |
+| `created_by`, `created_at`, `updated_at` | Audit fields. |
+
+Authentication dependency in ADR-0008 queries by `key_prefix`/id, verifies
+`key_hash` with constant-time comparison, checks tenant/key status and scopes,
+and returns `AuthContext`.
+
+### Request and mutation audit tables
+
+#### `api_request_logs`
+
+Append-only request log for authenticated `/v1` calls. This is not a
+replacement for structured application logs; it is the durable queryable audit
+record.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `tenant_id` | Denormalized from API key for fast tenant queries. |
+| `api_key_id` | FK to `api_keys.id`, nullable only for failed auth where key is unknown. |
+| `request_id` | Correlation id, unique. |
+| `method`, `path_template`, `status_code` | Route identity and result. |
+| `scopes_required` | JSONB/text array. |
+| `external_user_id` | User id supplied by the main backend, when present. |
+| `thread_id` | Present for thread/run routes. No FK — ADR-0007 says this service owns no thread table. |
+| `source_ip_hash`, `user_agent` | Optional operational metadata; avoid storing raw IP unless required. |
+| `request_summary`, `response_summary` | JSONB summaries, not raw bodies by default. |
+| `error_code` | Stable error code from ADR-0008, nullable. |
+| `duration_ms`, `created_at` | Timing. |
+
+This table records that an API call happened. Tables below record domain-level
+side effects.
+
+#### `point_audit_events`
+
+Append-only audit of `/v1/points` and `/v1/files/{file_id}/points` mutations.
+Qdrant remains the storage/search engine; this table records mutation intent
+and result.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `tenant_id` | FK to `tenants.id`. |
+| `api_request_log_id` | FK to `api_request_logs.id`. |
+| `api_key_id` | FK to `api_keys.id`. |
+| `operation` | `create` \| `update` \| `payload_patch` \| `soft_delete` \| `hard_delete` \| `reorder` \| `batch`. |
+| `point_id` | Qdrant point id, nullable for batch/file-wide operations. |
+| `file_id` | Source file id when relevant. |
+| `domain` | Qdrant payload domain. |
+| `before_version`, `after_version` | Optimistic-concurrency versions when available. |
+| `changed_fields` | JSONB list/summary; no large content or vectors. |
+| `qdrant_operation_id`, `qdrant_status` | Result returned by Qdrant, if available. |
+| `created_at` | Event time. |
+
+### File and ingestion tables
+
+#### `source_files`
+
+One logical source document uploaded by a tenant. Re-ingestion of the same file
+creates new jobs against the same or replacement `source_files` row depending
+on the `content_hash` policy.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK; this is the `file_id` copied into Qdrant point payloads. |
+| `tenant_id` | FK to `tenants.id`. |
+| `domain` | Tenant domain, validated by `tenant_domains` where enabled. |
+| `source_filename` | Original filename. |
+| `source_type` | `csv` \| `xlsx` \| `docx` \| `doc`. |
+| `content_sha256` | Hash of the uploaded file bytes for idempotency/change detection. |
+| `byte_size` | Upload size. |
+| `storage_uri` | Where the original file is stored, if retained. Nullable if not retaining originals. |
+| `status` | `active` \| `superseded` \| `soft_deleted` \| `purged`. |
+| `created_by_api_key_id`, `created_at`, `updated_at`, `deleted_at` | Audit fields. |
+
+#### `ingestion_jobs`
+
+One attempt to parse/chunk/embed/upsert a source file. This table is required
+even if the first implementation processes inline, because ADR-0008's file
+upload contract is job-shaped.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK; returned as `ingestion_job_id`. |
+| `tenant_id` | FK to `tenants.id`. |
+| `source_file_id` | FK to `source_files.id`. |
+| `api_request_log_id` | FK to the upload request log. |
+| `requested_by_api_key_id` | FK to `api_keys.id`. |
+| `status` | `queued` \| `running` \| `succeeded` \| `failed` \| `cancelled`. |
+| `chunking_strategy` | `semantic` \| `fixed_size`; matches ADR-0004. |
+| `embedding_model_versions` | JSONB map of vector name → model/version. |
+| `started_at`, `completed_at` | Lifecycle timestamps. |
+| `points_created`, `points_updated`, `points_soft_deleted`, `points_skipped` | Result counters. |
+| `error_code`, `error_message` | Failure summary. |
+| `metadata` | JSONB for parser/chunker options. |
+| `created_at`, `updated_at` | Audit timestamps. |
+
+#### `ingestion_job_events`
+
+Append-only progress/error stream for a job.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `tenant_id` | FK to `tenants.id`. |
+| `ingestion_job_id` | FK to `ingestion_jobs.id`. |
+| `level` | `info` \| `warning` \| `error`. |
+| `stage` | `received` \| `parsed` \| `chunked` \| `embedded` \| `upserted` \| `completed`. |
+| `message` | Short human-readable event. |
+| `details` | JSONB structured details. |
+| `created_at` | Event time. |
+
+### Graph run, LLM usage, and feedback tables
+
+#### `graph_runs`
+
+One row per `POST /v1/threads/{thread_id}/runs`. This is not a session/thread
+table: it records one execution for audit, feedback, usage aggregation, and
+cost reporting.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK; this is the `run_id` returned by the run endpoint. |
+| `tenant_id` | FK to `tenants.id`. |
+| `api_request_log_id` | FK to `api_request_logs.id`. |
+| `api_key_id` | FK to `api_keys.id`. |
+| `thread_id` | LangGraph thread id from the path. No FK. |
+| `external_user_id` | User id supplied by the main backend. |
+| `status` | `running` \| `answered` \| `clarifying` \| `escalate` \| `failed`. |
+| `escalation_reason` | ADR-0006 reason when `status='escalate'`. |
+| `input_message_hash` | Hash for idempotency/debug correlation without storing raw text here. |
+| `output_message_hash` | Hash of final answer/clarifying/escalation message. |
+| `llm_input_tokens`, `llm_output_tokens`, `llm_total_cost` | Denormalized totals from `llm_calls`. |
+| `started_at`, `completed_at`, `duration_ms` | Timing. |
+| `metadata` | JSONB for graph version, prompt version, retrieved chunk IDs, etc. |
+
+`graph_runs` solves the practical problem left by ADR-0007's feedback endpoint:
+feedback needs a stable `run_id`, but this service still does not need a table
+that represents chat sessions.
+
+#### `llm_pricing`
+
+Versioned model pricing table so historical cost calculations remain
+explainable when model prices change.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `provider` | `anthropic` \| `openai` \| other. |
+| `model` | Provider model id. |
+| `currency` | Usually `USD`. |
+| `input_price_per_1m_tokens`, `output_price_per_1m_tokens` | Numeric. |
+| `effective_from`, `effective_to` | Time-bounded price validity. |
+| `created_at` | Audit timestamp. |
+
+#### `llm_calls`
+
+One row per provider model call made inside the graph or ingestion pipeline.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `tenant_id` | FK to `tenants.id`. |
+| `graph_run_id` | FK to `graph_runs.id`, nullable for ingestion-time LLM calls such as image extraction. |
+| `ingestion_job_id` | FK to `ingestion_jobs.id`, nullable for chat-time calls. |
+| `api_request_log_id` | FK to the originating request when available. |
+| `thread_id`, `external_user_id` | Denormalized for query convenience; nullable outside chat. |
+| `node_name` | `triage`, `contextualize`, `grade`, `generate`, `verify`, `summarize`, `memory_extract`, `image_extract`, etc. |
+| `provider`, `model`, `model_version` | Provider identity. |
+| `pricing_id` | FK to `llm_pricing.id`, nullable if price was configured externally. |
+| `input_tokens`, `output_tokens`, `total_tokens` | Provider usage numbers. |
+| `input_cost`, `output_cost`, `total_cost`, `currency` | Cost at call time. |
+| `latency_ms` | Provider round-trip. |
+| `status` | `succeeded` \| `failed` \| `cancelled`. |
+| `error_code`, `error_message` | Failure summary. |
+| `prompt_version`, `schema_version` | Version of prompt/structured-output schema used. |
+| `input_hash`, `output_hash` | Hashes of stored/redacted payloads. |
+| `created_at` | Call start time. |
+
+Costs are computed and stored at call time from `llm_pricing` (or explicit
+runtime pricing config), not recomputed later from a mutable current price.
+
+#### `llm_call_payloads`
+
+Stores the actual model input/output only when allowed by tenant policy. This
+is deliberately separate from `llm_calls` so usage/billing queries never touch
+large or sensitive payloads.
+
+| Column | Notes |
+|---|---|
+| `llm_call_id` | PK/FK to `llm_calls.id`. |
+| `tenant_id` | FK to `tenants.id`, repeated for partition/index convenience. |
+| `input_redacted` | JSONB/text redacted prompt/messages/tool input. |
+| `output_redacted` | JSONB/text redacted model output/tool call result. |
+| `input_encrypted`, `output_encrypted` | Optional encrypted raw payload bytes/text if raw retention is enabled. |
+| `redaction_version` | Which redaction policy produced the redacted fields. |
+| `retention_until` | When payloads must be deleted, independent of usage rows. |
+| `created_at` | Timestamp. |
+
+Default policy: store token counts/costs for every call, store **redacted**
+input/output for debugging/evals, and store raw encrypted payloads only for
+tenants that explicitly enable it. Insurance chat can contain PII and sensitive
+claim/coverage information; raw prompt logging cannot be an accidental default.
+
+#### `run_feedback`
+
+Feedback from `POST /v1/threads/{thread_id}/runs/{run_id}/feedback`.
+
+| Column | Notes |
+|---|---|
+| `id` | UUID PK. |
+| `tenant_id` | FK to `tenants.id`. |
+| `graph_run_id` | FK to `graph_runs.id`. |
+| `external_user_id` | User id from main backend, if present. |
+| `rating` | `thumbs_up` \| `thumbs_down` \| numeric score. |
+| `reason_codes` | JSONB/text array. |
+| `comment` | Optional free text. |
+| `created_at` | Timestamp. |
+
+### What is intentionally not modeled
+
+- **No `threads`/`sessions` table.** ADR-0007 remains in force: the main
+  backend owns session records and LangGraph owns thread checkpoints. This
+  schema records runs and usage, not conversation ownership.
+- **No Postgres copy of Qdrant point content/vectors.** Qdrant remains the
+  source of truth for point payloads/vectors. Postgres stores source-file,
+  ingestion, and audit records.
+- **No plaintext API keys.** Only hashes and non-secret prefixes.
+- **No automatic raw prompt retention.** Raw LLM input/output is opt-in,
+  encrypted, and retention-limited.
+
+### Indexing and retention
+
+Required indexes:
+
+- `api_keys(key_prefix)` unique; `api_keys(tenant_id, status)`.
+- `tenant_domains(tenant_id, domain)` unique.
+- `api_request_logs(tenant_id, created_at desc)`, `api_request_logs(request_id)` unique.
+- `source_files(tenant_id, domain, created_at desc)`, `source_files(tenant_id, content_sha256)`.
+- `ingestion_jobs(tenant_id, status, created_at desc)`, `ingestion_jobs(source_file_id, created_at desc)`.
+- `point_audit_events(tenant_id, point_id, created_at desc)`, `point_audit_events(tenant_id, file_id, created_at desc)`.
+- `graph_runs(tenant_id, thread_id, started_at desc)`, `graph_runs(tenant_id, external_user_id, started_at desc)`.
+- `llm_calls(tenant_id, created_at desc)`, `llm_calls(graph_run_id)`, `llm_calls(ingestion_job_id)`.
+- `run_feedback(tenant_id, graph_run_id)`.
+
+Retention:
+
+- Usage/cost rows (`llm_calls`) live longer than payload rows.
+- `llm_call_payloads` has the shortest retention and is purged by
+  `retention_until`.
+- API logs and point audit events follow tenant contract/legal retention.
+- Deleted tenants are soft-deleted first; hard purge removes API keys, Store
+  namespaces, checkpointer threads, payload logs, and Qdrant points according
+  to a separate erasure runbook.
+
+## Consequences
+
+### Positive
+- Tenant/API-key authentication has a clear relational source of truth, and
+  FastAPI dependencies can resolve `AuthContext` with one indexed lookup.
+- Ingestion becomes observable and supportable: users can see whether a file
+  is queued, running, failed, or succeeded, and developers can inspect stage
+  events without scraping logs.
+- Qdrant mutations become auditable even though Qdrant remains the actual
+  vector/payload store.
+- LLM usage is attributable by tenant, thread, run, graph node, model, and
+  ingestion job, enabling cost reports and per-node optimization.
+- Separating `llm_calls` from `llm_call_payloads` keeps billing/analytics fast
+  and makes sensitive prompt retention a deliberate policy choice.
+- `graph_runs` gives the feedback endpoint a stable target while preserving
+  ADR-0007's decision not to own chat sessions.
+
+### Negative
+- This is a larger schema than the minimum needed to answer chat requests.
+  Implementing all tables up front adds migration and repository code before
+  the first end-to-end demo.
+- There is partial duplication between structured logs and `api_request_logs`;
+  the former is operational, the latter is durable audit. Both must use the
+  same `request_id` or they become hard to correlate.
+- Storing redacted LLM inputs/outputs still carries privacy risk: redaction can
+  miss sensitive details, especially in insurance text. Raw encrypted payloads
+  raise the risk further and need strict access controls.
+- Cost calculation depends on pricing data being kept current. If pricing is
+  wrong at call time, historical costs are wrong unless corrected explicitly.
+- Shared-schema tenancy relies on every query and foreign key carrying
+  `tenant_id`; a missed filter is a data leak. RLS could add defense in depth
+  later, but it is not part of the initial decision.
+
+## Alternatives Considered
+
+- **Exactly one API key per tenant**: rejected. It makes rotation and scope
+  separation painful. The requirement is that each tenant can authenticate;
+  allowing multiple keys per tenant is the safer implementation.
+- **Database/schema per tenant**: rejected. It adds migration and operational
+  overhead per tenant and diverges from ADR-0001's shared Qdrant multitenancy
+  model. A shared schema with `tenant_id` indexes is simpler and scales better
+  for this stage.
+- **PostgreSQL Row Level Security from day one**: deferred. RLS is useful
+  defense in depth, but it adds session-variable plumbing and migration/test
+  complexity. The initial boundary is FastAPI dependency resolution plus
+  explicit tenant filters and indexes; revisit RLS when the schema stabilizes.
+- **Use SQLModel instead of SQLAlchemy ORM**: rejected because the project
+  explicitly wants SQLAlchemy 2 and Alembic table design. Pydantic request/
+  response models remain separate from ORM models.
+- **Store all HTTP request/response bodies in `api_request_logs`**: rejected.
+  It would duplicate large payloads, accidentally retain files/prompts, and
+  raise privacy risk. Store summaries in `api_request_logs`; store controlled
+  LLM payloads in `llm_call_payloads`; store original files only via
+  `source_files.storage_uri` if retention policy allows it.
+- **Store full Qdrant payloads/vectors in Postgres for audit**: rejected. It
+  doubles storage and creates two sources of truth. Audit records store change
+  summaries, ids, versions, and operation outcomes.
+- **Compute LLM cost later from token counts**: rejected. Pricing changes over
+  time. Store the price used and the computed cost with each call so invoices
+  and reports are reproducible.

docs/adr/0010-langfuse-observability-and-prompt-management.md

@@ -0,0 +1,455 @@
+# 0010. Langfuse observability, prompt management, and evaluation workflow
+
+## Status
+
+Proposed
+
+## Context
+
+The chatbot now has architectural decisions for the LangGraph conversation
+([0006](0006-conversational-agent-graph.md)), LangGraph thread persistence
+([0007](0007-agent-persistence-threads-and-memory.md)), the FastAPI REST
+boundary ([0008](0008-rest-api-and-fastapi-boundary.md)), and Postgres tables
+for tenants, API keys, ingestion jobs, graph runs, LLM calls, payload retention,
+and feedback ([0009](0009-postgres-sqlalchemy-alembic-schema.md)). Those
+systems record *what happened* internally, but they do not provide the day-to-day
+LLM observability and prompt iteration workflow needed to run an insurance
+chatbot safely.
+
+We need to answer questions like:
+
+- Which graph node failed — `triage`, `retrieve`, `generate`, or `verify`?
+- Which retrieved chunks did the model see before it answered or escalated?
+- Which prompt version produced a bad answer?
+- Are prompt changes improving groundedness or just reducing escalations?
+- Which tenants, domains, or routes are most expensive?
+- Which failures should be fixed by better retrieval, better source content,
+  better prompts, or stricter handoff policy?
+
+Langfuse is a good fit for the observability and prompt/evaluation layer. It
+provides traces, nested observations, generation token/cost tracking, prompt
+version management, scores/user feedback, datasets, experiments, evaluators,
+and annotation workflows. It should not replace the application database:
+Postgres remains the source of truth for API-key authentication, tenant/audit
+records, ingestion jobs, graph run rows, billing/cost ledgers, and retention
+policy enforcement.
+
+Current Langfuse guidance relevant to this project:
+
+- Use the current Langfuse Python v3 SDK.
+- For LangGraph/LangChain, use `from langfuse.langchain import CallbackHandler`
+  and pass the handler in the graph invocation config.
+- Use one trace per user-facing chatbot run.
+- Use stable, low-cardinality trace and observation names.
+- Model calls should appear as generation observations so model, token, latency,
+  and cost information can attach correctly.
+- Conversation grouping should use Langfuse sessions; in this service,
+  LangGraph `thread_id` is the Langfuse `session_id`.
+- Prompt management uses prompt versions and labels (for example `development`,
+  `staging`, `production`, tenant-specific labels, and automatically maintained
+  `latest`). Labels can represent environments, tenants, or experiments.
+- Scores represent explicit feedback and implicit/evaluator signals.
+- Sensitive data should be masked before export, especially in insurance.
+
+## Decision
+
+### Langfuse is the observability plane, not the transactional database
+
+Adopt Langfuse for:
+
+- traces and nested observations;
+- prompt versions, variables, labels, diffs, and trace links;
+- user feedback and evaluator scores;
+- datasets and experiments;
+- annotation queues and systematic error analysis;
+- interactive debugging and dashboards.
+
+Keep Postgres ([0009](0009-postgres-sqlalchemy-alembic-schema.md)) as the
+internal system of record for:
+
+- tenants and API keys;
+- request and point audit records;
+- source files and ingestion jobs;
+- `graph_runs` and `run_feedback`;
+- `llm_calls`, `llm_pricing`, and optional `llm_call_payloads`;
+- tenant-specific retention, erasure, and billing/reporting queries.
+
+The two systems are correlated with stable identifiers:
+
+| Identifier | Origin | Use |
+|---|---|---|
+| `request_id` | FastAPI middleware | Correlate API logs, Postgres audit rows, Langfuse trace metadata. |
+| `tenant_id` | API-key dependency | Tenant filtering/cost attribution; never client-supplied. |
+| `thread_id` | REST path / LangGraph config | LangGraph thread id and Langfuse `session_id`. |
+| `run_id` | `graph_runs.id` | One `POST /v1/threads/{thread_id}/runs` execution. |
+| `llm_call_id` | `llm_calls.id` | Optional correlation from a Langfuse generation to Postgres usage ledger. |
+| `langfuse_trace_id` | Langfuse | Stored on `graph_runs.metadata` initially, or promoted to a typed column later if frequently queried. |
+| `langfuse_observation_id` | Langfuse | Correlated by putting `llm_call_id` in Langfuse observation metadata; add a typed Postgres column later only if needed. |
+
+### SDK and invocation pattern
+
+Use the current Langfuse Python SDK v3 integration for LangGraph/LangChain:
+
+```python
+from langfuse import get_client, propagate_attributes
+from langfuse.langchain import CallbackHandler
+
+langfuse = get_client()
+trace_tags = [environment, "chatbot", "threads:runs"]
+trace_metadata = {
+    "request_id": request_id,
+    "tenant_id": str(tenant_id),
+    "api_key_id": str(api_key_id),
+    "thread_id": thread_id,
+    "run_id": str(run_id),
+    "graph_version": graph_version,
+    "retrieval_config_version": retrieval_config_version,
+    "prompt_label": prompt_label,
+}
+
+with propagate_attributes(
+    trace_name="chat-run",
+    user_id=external_user_id,
+    session_id=thread_id,
+    tags=trace_tags,
+    metadata=trace_metadata,
+):
+    langfuse_handler = CallbackHandler()
+    result = graph.invoke(
+        graph_input,
+        config={
+            "callbacks": [langfuse_handler],
+            "configurable": {
+                "thread_id": thread_id,
+                "tenant_id": str(tenant_id),
+                "user_id": external_user_id,
+            },
+        },
+    )
+```
+
+If the implementation sets Langfuse attributes through LangChain/LangGraph
+`config["metadata"]` instead of or in addition to `propagate_attributes`, use the
+current integration keys `langfuse_user_id`, `langfuse_session_id`, and
+`langfuse_tags` for user, session, and tag propagation.
+
+For non-LangChain/LangGraph work that should appear in the same trace — FastAPI
+request spans, Qdrant retrieval, custom reranking metadata, or explicit audit
+steps — use Langfuse/OpenTelemetry spans around that work rather than relying
+only on automatic LLM callbacks.
+
+Short-lived scripts and tests call `langfuse.flush()` or `langfuse.shutdown()`
+before exit. Long-lived FastAPI workers flush during graceful shutdown.
+
+### Trace shape
+
+Create one Langfuse trace per `POST /v1/threads/{thread_id}/runs` call.
+
+Trace-level fields:
+
+| Field | Value |
+|---|---|
+| name | `chat-run` |
+| input | The latest user message, after request-level redaction. |
+| output | Final assistant answer, clarifying question, or escalation message. |
+| user_id | External `user_id` supplied by the main backend. |
+| session_id | LangGraph `thread_id`. |
+| tags | Environment, feature (`chatbot`), route (`threads:runs`), tenant segment if safe, terminal status. |
+| metadata | `request_id`, `tenant_id`, `api_key_id`, `thread_id`, `run_id`, graph version, retrieval config version, prompt label. |
+
+Observation hierarchy mirrors ADR-0006's graph instead of producing a flat list:
+
+| Observation | Type | Contents |
+|---|---|---|
+| `load-memory` | span | Store lookup latency and memory keys, not sensitive values. |
+| `triage` | generation | Intent classification prompt/output. |
+| `contextualize` | generation | Query rewrite input/output. |
+| `retrieve` | retriever/span | Qdrant collection, vector names, filters, top-k, returned point/chunk IDs, scores, and source metadata. Avoid raw chunk text unless redacted. |
+| `grade` | generation | Relevance verdicts over retrieved chunks. |
+| `clarify` | generation | Grounded clarification question when used. |
+| `generate` | generation | Final answer generation with citation IDs. |
+| `verify` | generation | Groundedness/citation verification result. |
+| `escalate` | span | Escalation reason and handoff context summary. |
+| `write-memory` | span | Memory write keys and outcome, not sensitive facts. |
+
+Observation names stay stable and low-cardinality. Dynamic values such as tenant
+slug, file id, model name, retry number, point id, and user text go in metadata
+or input/output fields, not in observation names.
+
+Every generation should include model name, input tokens, output tokens,
+latency, and cost when available. Langfuse provides the interactive view;
+`llm_calls` in Postgres remains the billing/audit ledger.
+
+### Prompt management and tenant-specific variants
+
+Manage prompts in Langfuse for graph nodes whose behavior will be tuned over
+time:
+
+- `insurance-chatbot/triage`
+- `insurance-chatbot/contextualize`
+- `insurance-chatbot/grade-retrieval`
+- `insurance-chatbot/clarify`
+- `insurance-chatbot/generate-answer`
+- `insurance-chatbot/verify-grounding`
+- `insurance-chatbot/summarize-thread`
+- `insurance-chatbot/extract-memory`
+
+Prompt text, versions, diffs, labels, and rollback live in Langfuse. Tenant
+identity and the policy that chooses which prompt label/name to fetch live in the
+application and, where tenant-specific configuration is needed, Postgres
+`tenants.settings`.
+
+Prompt names are stable and low-cardinality. The initial tenant-customization
+model uses shared prompt names plus tenant-specific labels:
+
+| Label | Use |
+|---|---|
+| `development` | Local/dev experiments. |
+| `staging` | Pre-production validation and experiments. |
+| `production` | Default live traffic prompt when no tenant-specific override exists. |
+| `tenant-{tenant_slug}-{environment}` | Tenant-specific prompt version for one environment, for example `tenant-acme-production`. |
+| `latest` | Automatically points to newest version; useful for discovery, not production runtime. |
+
+Langfuse labels can represent environments, tenants, or experiments. The service
+therefore uses labels to support different live prompts per tenant without
+creating an application-owned prompt-version table.
+
+Tenant-specific prompt resolution is handled in the application with an explicit
+fallback order:
+
+1. derive `tenant_slug` from the authenticated `AuthContext`, never from a
+   request body/query parameter;
+2. derive the environment label from trusted configuration, for example
+   `production` or `staging`;
+3. try the tenant/environment label, for example `tenant-acme-production`;
+4. fall back to the environment label, for example `production`;
+5. fail closed or use a code-packaged emergency prompt only if the configured
+   prompt cannot be loaded.
+
+Indicative resolver shape:
+
+```python
+prompt_name = "insurance-chatbot/generate-answer"
+environment_label = settings.langfuse_prompt_label  # e.g. "production"
+tenant_label = f"tenant-{auth.tenant_slug}-{environment_label}"
+
+prompt = prompt_resolver.get_with_fallback(
+    name=prompt_name,
+    labels=[tenant_label, environment_label],
+)
+compiled_prompt = prompt.compile(
+    language=language,
+    user_message=user_message,
+    conversation_summary=conversation_summary,
+    retrieved_chunks=retrieved_chunks,
+    tenant_policy=tenant_policy,
+    handoff_policy=handoff_policy,
+)
+```
+
+The exact SDK call and not-found error handling should follow the current
+Langfuse Python SDK, but the architectural rule is stable: select the tenant
+label from trusted server-side tenant context, not from caller input.
+
+This keeps prompt content, prompt versions, prompt diffs, and label movement in
+Langfuse while keeping tenant identity and resolver policy in the application.
+If tenant-specific prompts become large and independently owned, an acceptable
+future variant is tenant-namespaced prompt names such as
+`insurance-chatbot/tenants/acme/generate-answer` with ordinary `production` and
+`staging` labels. The initial design prefers shared prompt names plus
+tenant-specific labels because it keeps graph-node prompts easier to compare
+across tenants.
+
+Postgres may store prompt resolver configuration in `tenants.settings`, for
+example an enabled flag, preferred prompt namespace, explicit label override,
+default language, tenant policy reference, or handoff policy reference. Postgres
+must not become the source of truth for prompt text/version history; that
+belongs in Langfuse.
+
+Prompt variables carry dynamic context, for example:
+
+- `{{language}}`
+- `{{user_message}}`
+- `{{conversation_summary}}`
+- `{{retrieved_chunks}}`
+- `{{tenant_policy}}`
+- `{{handoff_policy}}`
+- `{{citation_contract}}`
+- `{{memory_profile}}`
+
+Prompt versions are linked to traces so failures can be traced back to the exact
+prompt version that produced them. Rollback is performed by moving the relevant
+label — `production` for the default prompt, or `tenant-acme-production` for one
+tenant's override — back to a known-good version, not by redeploying code.
+
+Prompt changes are evaluated before promotion using Langfuse datasets and
+experiments. A prompt version should not be promoted to `production` or to a
+tenant-specific production label only because one manual test looked good.
+
+### Privacy, redaction, and retention
+
+Insurance conversations can contain PII, policy details, claim information, and
+financial/health-adjacent facts. Langfuse is therefore sensitive infrastructure.
+
+Rules:
+
+- Mask/redact before export from the application process. Prefer current SDK
+  masking support such as `mask_otel_spans` where applicable, plus deterministic
+  field/path redaction for known sensitive fields.
+- Never put API keys, database URLs, provider secrets, raw auth headers, or
+  unredacted request bodies in trace metadata.
+- Do not trace raw uploaded files or full document text.
+- Retrieval observations should prefer point/chunk IDs, file IDs, scores,
+  titles, and short redacted snippets over full retrieved chunks.
+- Raw LLM input/output retention is governed by ADR-0009's
+  `llm_call_payloads` tenant policy, not by accidental Langfuse trace capture.
+- Langfuse project retention/deletion settings must align with insurance and
+  tenant contractual retention requirements.
+- If self-hosted Langfuse is used later, server-side ingestion masking can add
+  defense in depth, but it does not replace client-side masking because data
+  can still reach pre-masking ingestion infrastructure.
+
+### Feedback and scores
+
+Record explicit and implicit feedback as Langfuse scores while mirroring durable
+feedback to Postgres `run_feedback` when the application needs it.
+
+Explicit feedback:
+
+| Signal | Langfuse score name | Notes |
+|---|---|---|
+| Thumbs up/down | `user-thumbs` | Boolean score, attached to the trace/run. |
+| Optional comment | score comment/metadata | Also stored in `run_feedback.comment` if enabled. |
+| Report bad answer | `user-report` | Useful for annotation queues and support review. |
+
+Implicit feedback:
+
+| Signal | Score/tag | Meaning |
+|---|---|---|
+| Handoff triggered | `handoff-triggered` | The run ended in `status=escalate`. |
+| Regeneration/retry | `user-retry` or metadata | Negative/diagnostic signal. |
+| Citation opened | `citation-opened` | User inspected a cited source. |
+| Answer copied | `answer-copied` | Weak positive signal if frontend exposes it. |
+
+The FastAPI response for a run should include enough trace/run identity for the
+backend or frontend to submit feedback later, but no Langfuse secret key is ever
+sent to a browser. Browser-side feedback, if added, may use only a Langfuse
+public key or, preferably, route feedback through the backend.
+
+### Evaluation and quality workflow
+
+Use Langfuse beyond basic tracing:
+
+1. **Datasets** — maintain representative insurance questions, expected
+   citation behavior, ambiguous questions, out-of-scope cases, and handoff
+   cases.
+2. **Experiments** — compare prompt versions, retrieval parameters, reranking
+   thresholds, model choices, and verification thresholds before deployment.
+3. **LLM-as-judge evaluators** — score groundedness, citation correctness,
+   ambiguity handling, escalation correctness, tone/language, and refusal
+   quality. Judges are aids, not absolute truth.
+4. **Annotation queues** — send low-score traces, user reports, ungrounded
+   answers, and unnecessary escalations to human review.
+5. **Error analysis** — periodically sample traces, open-code failures, cluster
+   failure modes, and decide whether the fix belongs in retrieval, source
+   content, prompt text, graph policy, or human handoff rules.
+6. **Dashboards** — track per-tenant cost, per-node token spend, latency,
+   escalation rate, clarification rate, retrieval-insufficient rate, and
+   verification-failure rate.
+
+### Boundary with ADR-0009 Postgres tables
+
+Langfuse overlaps with parts of ADR-0009, especially LLM-call visibility,
+feedback, and run debugging. This overlap is intentional, but the systems have
+different authority.
+
+| Need | Langfuse | Postgres | Decision |
+|---|---|---|---|
+| Prompt text, versions, diffs, labels, and rollback | Yes | No | Use Langfuse as the prompt source of truth. |
+| Tenant-specific prompt content | Yes | No | Use Langfuse prompt labels or, later, tenant-namespaced prompt names. |
+| Tenant prompt-routing policy | Partly | Yes | Store trusted tenant config in `tenants.settings` or app config; fetch prompt content from Langfuse. |
+| Interactive debugging of one bad answer | Yes | Partly | Use Langfuse trace; correlate to `graph_runs.id`. |
+| Trace/session grouping | Yes | Partly | Use Langfuse `session_id=thread_id`; keep `thread_id` on `graph_runs` for ledger/audit. |
+| LLM token/cost visibility | Yes | Yes | Langfuse for observability; Postgres `llm_calls`/`llm_pricing` for billing and audit. |
+| Durable graph-run ledger | No | Yes | Keep `graph_runs`. Langfuse traces are not the application run table. |
+| User feedback | Yes | Yes | Store scores in Langfuse and durable feedback in `run_feedback` when needed by the app. |
+| Error analysis and annotation queues | Yes | Usually no | Use Langfuse; avoid building an annotation workflow in Postgres initially. |
+| Datasets, experiments, and evaluators | Yes | Usually no | Use Langfuse; copy to Postgres only if a product/compliance need appears. |
+| Tenants and API keys | No | Yes | Keep `tenants` and `api_keys` in Postgres. |
+| API request audit logs | Not enough | Yes | Keep `api_request_logs` in Postgres. |
+| Point/file mutation audit | No | Yes | Keep `point_audit_events`, `source_files`, and `ingestion_jobs` in Postgres. |
+| Controlled raw/redacted payload retention | Partly | Yes | Postgres `llm_call_payloads` plus tenant policy owns retention; Langfuse must be redacted/configured to comply. |
+| Legal retention and tenant erasure policy | Partly | Yes | Postgres owns application policy; Langfuse project retention/deletion settings must align with it. |
+
+Do not remove ADR-0009 tables because Langfuse exists. Langfuse is optimized for
+observability, prompt iteration, feedback analysis, datasets, experiments, and
+annotation workflows. Postgres is optimized for auth, audit, retention, billing,
+and application consistency.
+
+## Consequences
+
+### Positive
+- Every chat run becomes inspectable as a graph-shaped trace rather than a flat
+  log line or a pile of model-call rows.
+- Prompt versions can be changed, compared, rolled back, and linked to the
+  traces they produced without redeploying application code for every prompt
+  edit.
+- Tenant-specific prompt labels let one tenant receive a customized prompt while
+  other tenants continue using the default `production` version.
+- Scores, datasets, experiments, and annotation queues create a path from user
+  feedback to systematic quality improvement.
+- Per-node token/cost/latency data exposes where the graph is expensive or
+  slow: retrieval, generation, verification, or memory.
+- Correlating Langfuse IDs with Postgres IDs preserves both workflows:
+  interactive debugging and durable audit/billing.
+
+### Negative
+- Langfuse adds another service and another set of credentials, retention
+  policies, access controls, and operational dashboards.
+- Prompt management introduces runtime dependency on prompt fetch/caching
+  behavior. The application needs safe fallbacks if Langfuse is temporarily
+  unavailable.
+- Tenant-specific prompt labels add routing complexity. The resolver must avoid
+  caller-controlled labels and must have a clear fallback when a tenant override
+  does not exist.
+- Observability can become a privacy leak if masking is incomplete. Insurance
+  content makes this a high-consequence risk.
+- There is intentional duplication between Langfuse generation usage and
+  Postgres `llm_calls`; implementations must avoid reconciling them as if one
+  were authoritative for the other's purpose.
+- LLM-as-judge evaluators can be wrong. They should guide review and experiments,
+  not replace human analysis for high-risk policy/coverage answers.
+
+## Alternatives Considered
+
+- **Use Postgres tables only for observability**: rejected. `llm_calls` and
+  audit tables are good for billing and compliance queries, but poor for
+  nested trace inspection, prompt-version attribution, annotation queues, and
+  experiment workflows.
+- **Use Langfuse as the only usage/cost store**: rejected. Tenant billing,
+  erasure, audit, and internal consistency need application-owned tables with
+  controlled retention and foreign keys to tenants/API keys.
+- **Use Langfuse instead of ADR-0009 operational tables**: rejected. Langfuse can
+  help with traces, prompt versions, scores, datasets, experiments, and
+  annotation queues, but it should not own API-key authentication, request audit,
+  point/file mutation audit, ingestion jobs, or the internal billing ledger.
+- **Store tenant prompt text and versions in Postgres**: rejected for the initial
+  design. It would duplicate Langfuse prompt-management features and require the
+  project to build its own versioning, diffs, labels, rollback, and trace links.
+  Postgres stores tenant prompt-routing configuration only.
+- **Keep prompts only in code**: rejected for prompts that will be tuned often.
+  Code-only prompts make rollback and prompt/version-to-trace analysis harder.
+  Small static strings and schema definitions may still live in code.
+- **Fetch prompts by `latest` in production**: rejected. `latest` changes every
+  time a new version is created. Production uses a configured label such as
+  `production`; rollback is moving the label.
+- **Trace full retrieved chunks and raw files for easier debugging**: rejected.
+  It would make debugging convenient but creates unnecessary privacy and storage
+  risk. IDs, scores, metadata, and redacted snippets are enough by default.
+- **One Langfuse project per tenant**: rejected for the initial design. A single
+  project per environment with tenant metadata/tags is easier to operate and
+  compare. Revisit per-tenant projects only if contractual isolation requires
+  it.
+- **Skip Langfuse scores until evals are mature**: rejected. Simple user-thumbs
+  and handoff-triggered scores are cheap and immediately useful for filtering
+  traces and building the first annotation queues.

docs/adr/0011-python-structured-logging-with-structlog.md

@@ -0,0 +1,425 @@
+# 0011. Python structured logging with structlog and request context
+
+## Status
+
+Proposed
+
+## Context
+
+The service needs application-level Python logging in addition to the durable
+Postgres records from ADR-0009 and the LLM/agent traces from ADR-0010.
+
+Postgres audit tables answer durable business questions such as which tenant API
+key called an endpoint, which point mutation was requested, which ingestion job
+ran, and what usage ledger rows were produced. Langfuse answers LLM observability
+questions such as which graph node or prompt version produced an answer.
+Structured Python logs answer operational questions while the service is running:
+
+- Which request failed and where?
+- Which logs belong to one FastAPI request or LangGraph run?
+- Which tenant, API key, thread, run, file, point, or ingestion job was involved?
+- Which dependency was slow or unavailable?
+- Which fallback path or retry was used?
+
+FastAPI, LangGraph, SQLAlchemy, Qdrant, Langfuse, and HTTP clients can all emit
+logs from asynchronous code. Since async tasks can interleave on the same event
+loop, relying on process-global mutable variables is unsafe. The logging context
+must be request-scoped and safe across async task switching. Python
+`contextvars`, exposed through `structlog.contextvars`, provide this behavior.
+
+The user has used a previous `log.py` based on `logging`, `structlog`,
+`logging.config.dictConfig`, `ProcessorFormatter`, JSON rendering, stdlib log
+capture, and manual `ContextVar` fields such as `session_id` and `property_id`.
+This service should keep the same core idea but adapt the field names to the
+current architecture:
+
+- `thread_id` instead of `session_id` for LangGraph conversations, matching
+  ADR-0007 and ADR-0008;
+- `tenant_id`, `tenant_slug`, `api_key_id`, and `actor_type` from `AuthContext`;
+- `request_id` from FastAPI middleware;
+- `run_id` from `graph_runs` for chat runs;
+- `file_id`, `point_id`, and `ingestion_job_id` for ingestion and point work;
+- `langfuse_trace_id` when available for cross-navigation to ADR-0010 traces.
+
+## Decision
+
+### Use structlog as the application logging interface
+
+Use `structlog` for application logs and integrate it with Python stdlib logging
+so framework/library logs are formatted consistently.
+
+Application code imports loggers with:
+
+```python
+import structlog
+
+logger = structlog.get_logger(__name__)
+```
+
+Log events use stable event names and structured fields:
+
+```python
+logger.info(
+    "graph.run.completed",
+    status="answered",
+    duration_ms=duration_ms,
+    retrieved_chunk_count=len(retrieved_chunks),
+)
+```
+
+Do not build log messages by interpolating operational metadata into prose.
+Prefer fields over long strings because fields are queryable.
+
+### Emit JSON logs by default in production
+
+Production logs are JSON on stdout so process managers, container runtimes, and
+log collectors can ingest them directly. Local development may use a colored
+console renderer controlled by configuration.
+
+File logging is optional and mainly for local development. If enabled, it must
+use explicit rotation settings such as `maxBytes` and `backupCount`. Do not rely
+on a default `RotatingFileHandler` with no rotation parameters. In containerized
+production, stdout/stderr collection is preferred over writing `logs/app.log`
+inside the application container.
+
+### Configure stdlib and structlog together
+
+The logging setup should happen once during process startup, before the FastAPI
+app begins serving requests.
+
+Indicative configuration shape:
+
+```python
+import logging
+import logging.config
+import sys
+
+import structlog
+
+
+def configure_logging(*, log_level: str, json_logs: bool) -> None:
+    shared_processors = [
+        structlog.contextvars.merge_contextvars,
+        structlog.stdlib.add_log_level,
+        structlog.stdlib.add_logger_name,
+        structlog.processors.TimeStamper(fmt="iso", utc=True),
+        structlog.processors.StackInfoRenderer(),
+    ]
+
+    structlog.configure(
+        processors=[
+            *shared_processors,
+            structlog.processors.format_exc_info,
+            structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
+        ],
+        logger_factory=structlog.stdlib.LoggerFactory(),
+        wrapper_class=structlog.stdlib.BoundLogger,
+        cache_logger_on_first_use=True,
+    )
+
+    renderer = (
+        structlog.processors.JSONRenderer()
+        if json_logs
+        else structlog.dev.ConsoleRenderer(colors=True)
+    )
+
+    logging.config.dictConfig(
+        {
+            "version": 1,
+            "disable_existing_loggers": False,
+            "formatters": {
+                "default": {
+                    "()": structlog.stdlib.ProcessorFormatter,
+                    "processors": [
+                        structlog.stdlib.ProcessorFormatter.remove_processors_meta,
+                        renderer,
+                    ],
+                    "foreign_pre_chain": [
+                        structlog.stdlib.ExtraAdder(),
+                        *shared_processors,
+                    ],
+                },
+            },
+            "handlers": {
+                "console": {
+                    "class": "logging.StreamHandler",
+                    "level": log_level,
+                    "formatter": "default",
+                    "stream": sys.stdout,
+                },
+            },
+            "loggers": {
+                "": {
+                    "handlers": ["console"],
+                    "level": log_level,
+                    "propagate": False,
+                },
+                "uvicorn": {
+                    "handlers": ["console"],
+                    "level": log_level,
+                    "propagate": False,
+                },
+                "uvicorn.access": {
+                    "handlers": ["console"],
+                    "level": log_level,
+                    "propagate": False,
+                },
+                "sqlalchemy.engine": {
+                    "handlers": ["console"],
+                    "level": "WARNING",
+                    "propagate": False,
+                },
+                "watchfiles": {
+                    "handlers": ["console"],
+                    "level": "INFO",
+                    "propagate": False,
+                },
+            },
+        }
+    )
+```
+
+Notes:
+
+- Use the logger name `sqlalchemy.engine`, not `sqlalchemy.engin`.
+- SQL statement logging is too noisy and can leak values; keep it `WARNING` by
+  default in production and enable `INFO`/`DEBUG` only in controlled debugging.
+- `structlog.stdlib.ExtraAdder()` keeps useful fields from stdlib log records.
+- `structlog.contextvars.merge_contextvars` ensures request-bound fields appear
+  on both structlog and stdlib logs processed through the formatter.
+
+### Bind request context with contextvars
+
+At FastAPI ingress, clear stale context, bind request identifiers, and return the
+request id to callers. This makes it possible to select all logs from one request
+or one graph run even when async tasks interleave.
+
+Indicative middleware:
+
+```python
+from time import perf_counter
+from uuid import uuid4
+
+import structlog
+from fastapi import Request
+from starlette.types import ASGIApp
+
+REQUEST_ID_HEADER = "X-Request-ID"
+
+
+async def logging_context_middleware(request: Request, call_next: ASGIApp):
+    structlog.contextvars.clear_contextvars()
+
+    request_id = request.headers.get(REQUEST_ID_HEADER) or str(uuid4())
+    route = request.scope.get("route")
+    path_template = getattr(route, "path", request.url.path)
+
+    structlog.contextvars.bind_contextvars(
+        request_id=request_id,
+        method=request.method,
+        path_template=path_template,
+    )
+
+    logger = structlog.get_logger("app.http")
+    started = perf_counter()
+    logger.info("request.started")
+
+    try:
+        response = await call_next(request)
+    except Exception:
+        logger.exception(
+            "request.failed",
+            duration_ms=round((perf_counter() - started) * 1000, 2),
+        )
+        raise
+
+    response.headers[REQUEST_ID_HEADER] = request_id
+    logger.info(
+        "request.completed",
+        status_code=response.status_code,
+        duration_ms=round((perf_counter() - started) * 1000, 2),
+    )
+    return response
+```
+
+After API-key authentication succeeds, the auth dependency or route handler binds
+trusted tenant/auth fields:
+
+```python
+structlog.contextvars.bind_contextvars(
+    tenant_id=str(auth.tenant_id),
+    tenant_slug=auth.tenant_slug,
+    api_key_id=str(auth.api_key_id),
+    actor_type=auth.actor_type,
+)
+```
+
+Route handlers bind route-specific fields when they become known:
+
+```python
+structlog.contextvars.bind_contextvars(
+    external_user_id=body.user_id,
+    thread_id=thread_id,
+    run_id=str(run_id),
+)
+```
+
+Use these canonical context keys:
+
+| Field | Source | Notes |
+|---|---|---|
+| `request_id` | FastAPI middleware | Primary log correlation id; also appears in ADR-0008/0009 records. |
+| `tenant_id` | `AuthContext` | Trusted server-side tenant id; never request body/query. |
+| `tenant_slug` | `AuthContext` | Useful for filtering; avoid if contractual policy treats it as sensitive. |
+| `api_key_id` | `AuthContext` | Non-secret id only. Never log raw API keys or auth headers. |
+| `actor_type` | `AuthContext` | `backend`, `admin`, or `worker`. |
+| `external_user_id` | Main backend | May be high-cardinality; acceptable in logs, not metrics labels. |
+| `thread_id` | REST path | LangGraph thread id. |
+| `run_id` | `graph_runs.id` | One chat run. |
+| `ingestion_job_id` | `ingestion_jobs.id` | File ingestion correlation. |
+| `file_id` | `source_files.id` | Source-file correlation. |
+| `point_id` | Qdrant point id | Point mutation/read correlation. |
+| `langfuse_trace_id` | Langfuse | Cross-link to ADR-0010 trace when available. |
+
+Use `structlog.contextvars.clear_contextvars()` at request/task ingress to avoid
+leaking a previous request's context into reused workers.
+
+### Bind context explicitly for jobs and background work
+
+Context variables work across normal async task switching, but background jobs,
+worker processes, scheduled jobs, and threadpool work should bind context at
+their own entry point from durable identifiers.
+
+Examples:
+
+- ingestion worker binds `tenant_id`, `ingestion_job_id`, `file_id`, and
+  `request_id` if inherited from the upload request;
+- LangGraph run execution binds `thread_id`, `run_id`, `tenant_id`, and
+  `external_user_id` before invoking the graph;
+- point batch workers bind `tenant_id`, `api_request_log_id`, and operation
+  metadata before processing each batch.
+
+If code crosses a boundary where contextvars may not propagate automatically,
+pass the identifiers explicitly and bind them again at the boundary.
+
+### Log levels and event naming
+
+Use log levels consistently:
+
+| Level | Use |
+|---|---|
+| `DEBUG` | Local diagnostics, disabled by default in production. |
+| `INFO` | Normal lifecycle events: request started/completed, graph run completed, ingestion job completed. |
+| `WARNING` | Recoverable anomalies: fallback prompt used, retry scheduled, insufficient retrieval before clarification/escalation. |
+| `ERROR` | Failed operations requiring attention: unhandled exception, dependency outage, ingestion failure. |
+
+Do not log expected user behavior at `ERROR`. For example, a user asking an
+ambiguous question that leads to clarification is an `INFO` event; a retriever
+being unavailable is an `ERROR` event.
+
+Use stable dot-separated event names:
+
+- `request.started`
+- `request.completed`
+- `request.failed`
+- `auth.succeeded`
+- `auth.failed`
+- `graph.run.started`
+- `graph.run.completed`
+- `graph.run.escalated`
+- `retrieval.completed`
+- `retrieval.insufficient`
+- `llm.call.completed`
+- `llm.call.failed`
+- `ingestion.job.started`
+- `ingestion.job.completed`
+- `point.mutation.completed`
+
+Do not include dynamic values in logger names or event names. Put dynamic values
+in structured fields.
+
+### Security and privacy rules
+
+Logs must not contain secrets or raw sensitive payloads.
+
+Never log:
+
+- plaintext API keys;
+- `Authorization` headers;
+- database URLs or provider credentials;
+- raw uploaded file contents;
+- full retrieved chunks by default;
+- raw user messages, raw prompts, or raw model outputs by default;
+- embeddings or vectors.
+
+Prefer:
+
+- ids (`request_id`, `thread_id`, `run_id`, `file_id`, `point_id`);
+- hashes (`input_message_hash`, `output_message_hash`, `content_sha256`);
+- counts, sizes, durations, and status codes;
+- short redacted summaries only when useful and allowed by tenant policy.
+
+The same redaction policy used for ADR-0009 `llm_call_payloads` and ADR-0010
+Langfuse tracing should guide log redaction. Logging should be safe even when
+log aggregation has broader access than the application database.
+
+### Relationship to Postgres and Langfuse
+
+Structured logs complement but do not replace ADR-0009 and ADR-0010.
+
+| Question | System of record |
+|---|---|
+| What happened operationally inside this process? | Structured logs. |
+| Which API key called which endpoint and what durable side effect occurred? | Postgres audit tables from ADR-0009. |
+| Which graph node, prompt version, retrieved chunks, and model calls produced an answer? | Langfuse traces from ADR-0010. |
+| What should be used for tenant billing and compliance reports? | Postgres `llm_calls`, `llm_pricing`, `api_request_logs`, and audit tables. |
+| What should be used for interactive debugging of one LLM answer? | Langfuse trace, linked from logs/Postgres by ids. |
+
+Logs may contain `request_id`, `run_id`, and `langfuse_trace_id` so engineers can
+navigate across all three systems.
+
+## Consequences
+
+### Positive
+- Logs become queryable by `request_id`, `thread_id`, `run_id`, `tenant_id`,
+  `file_id`, and `ingestion_job_id`.
+- Contextvars prevent async task interleaving from mixing request context.
+- Stdlib/framework logs and application logs share one JSON structure.
+- Production logs are compatible with common log collectors and container
+  runtimes.
+- Logs, Postgres audit rows, and Langfuse traces can be correlated without
+  duplicating each system's purpose.
+
+### Negative
+- Logging setup is more complex than plain `logging.basicConfig()`.
+- Developers must learn to use structured fields instead of prose-only log
+  messages.
+- Context must be rebound at worker/background-task boundaries.
+- Too much logging can increase cost and leak sensitive data if redaction rules
+  are not followed.
+- JSON logs are less pleasant locally unless a console renderer is enabled for
+  development.
+
+## Alternatives Considered
+
+- **Use Python stdlib logging only**: rejected. Stdlib logging can work, but
+  `structlog` gives cleaner structured context, contextvars integration, and
+  consistent event dictionaries across application and framework logs.
+- **Use manual `ContextVar` fields only**: rejected as the default. Manual
+  context variables work, but `structlog.contextvars.bind_contextvars()` and
+  `merge_contextvars` provide a standard way to bind arbitrary request fields
+  without maintaining one `ContextVar` per field. Manual `ContextVar`s may still
+  be used for special cases.
+- **Use `session_id` as the primary chat correlation field**: rejected for this
+  service. ADR-0007 standardizes on `thread_id`, which maps to LangGraph threads
+  and Langfuse sessions. If the main backend calls the same concept a session,
+  it is translated to `thread_id` at this service boundary.
+- **Write only to `logs/app.log`**: rejected for production. File logging is
+  useful locally, but stdout JSON is the better default for deployed services.
+- **Use Langfuse for all observability**: rejected. Langfuse is excellent for
+  LLM/agent traces, prompt versions, scores, and evals, but it is not a
+  replacement for process logs covering FastAPI middleware, auth, SQLAlchemy,
+  Qdrant calls, worker lifecycle, and non-LLM failures.
+- **Use Postgres audit tables as logs**: rejected. ADR-0009 tables are durable
+  business/audit records. They should not receive high-volume operational debug
+  logs.