feat(ingestion): add DOCX/CSV/XLSX parsing and fixed-size chunking (ADR-0018)

Adds src/application/ingestion/ -- Persian normalization, DOCX body
walk with structural data/layout table classification, CSV/XLSX row
rendering, and fixed-size token chunking (cl100k_base, 400/60/512) --
as pure functions per ADR-0015, tested against real production
documents (asia_data_sample, kept out of the repo). ADR-0018 records
where this diverges from ADR-0004 (fixed-size default, no invented
headings/tree, structural table classification, header-provable
labeling only). Plan 001's scope line is corrected from CSV-only to
DOCX/XLSX/CSV, and CLAUDE.md's stale project-status paragraph is
updated to match current implementation state.

docs/plans/001-ingestion-vertical-slice.md

@@ -3,7 +3,7 @@
 ## Purpose
 
 This plan turns the accepted architectural direction in the ADRs into the first
-working product slice: a tenant-scoped CSV upload is stored in MinIO, represented
+working product slice: a tenant-scoped DOCX/XLSX/CSV upload is stored in MinIO, represented
 by durable Postgres records, parsed/chunked/embedded inline in the request
 (ADR-0017), and indexed as Qdrant points before the response returns.
 
@@ -47,14 +47,19 @@ them.
 
 ### In scope
 
-- `POST /v1/files` for authenticated tenant-scoped **CSV** upload.
+- `POST /v1/files` for authenticated tenant-scoped **DOCX, XLSX, and CSV** upload.
+  `.doc` is rejected with `415` pending an out-of-process conversion service
+  (ADR-0018). An earlier revision of this plan scoped the slice to CSV only and
+  placed DOCX out of scope; the real corpus is DOCX and XLSX, so ADR-0018
+  corrects that.
 - File validation, size limits, content hashing, and streaming upload to MinIO.
 - Alembic-managed Postgres schema for the minimal tenant/auth, source file,
   ingestion job, and job event records needed by this slice.
 - Inline ingestion in `POST /v1/files`, with batched/bounded-concurrent
   embedding, thread-offloaded parsing, and enforced size/timeout/capacity
   bounds.
-- CSV parsing and deterministic chunk creation.
+- DOCX/XLSX/CSV parsing into structural units and deterministic chunk creation
+  (ADR-0018, implemented in `src/application/ingestion/`).
 - Tenant-filtered Qdrant point upserts using deterministic point identifiers.
 - Job status/progress persistence and `GET /v1/files/{file_id}` status lookup.
 - Structured correlation logging at HTTP and ingestion-stage boundaries.
@@ -63,7 +68,11 @@ them.
 
 ### Explicitly out of scope
 
-- XLSX, DOCX, and legacy DOC ingestion.
+- Legacy `.doc` ingestion — rejected with `415` until an out-of-process
+  conversion service exists (ADR-0018). DOCX and XLSX are **in** scope; they were
+  listed here before ADR-0018 corrected the scope line.
+- `qa_pair` structural detection and embedded-image captioning (ADR-0004),
+  deferred by ADR-0018.
 - The conversational LangGraph API and SSE streaming.
 - Final reranker selection, GPU deployment, or unresolved model licensing from
   ADR-0005.
@@ -196,7 +205,7 @@ reads/writes and valid job transitions.
 ### Phase 3: MinIO upload and durable job creation
 
 1. Implement API-key authentication and `AuthContext` tenant derivation.
-2. Implement `POST /v1/files` for CSV only, including streaming-size controls,
+2. Implement `POST /v1/files` for DOCX, XLSX, and CSV, including streaming-size controls,
    file-type validation, SHA-256 calculation, and a private MinIO upload using
    an internal object key.
 3. In one short Postgres transaction, persist `source_files` and create
@@ -208,11 +217,11 @@ reads/writes and valid job transitions.
    response that does not expose raw storage credentials or internal artifacts.
 6. Add cleanup/compensation handling for a MinIO upload that succeeds while the
    database transaction fails.
-7. Add unit/API tests for trusted tenant derivation, CSV validation, idempotency,
+7. Add unit/API tests for trusted tenant derivation, upload validation, idempotency,
    the terminal `201 Created` response, and tenant-scoped status. Add MinIO adapter integration tests
    for server-derived private object paths and compensation behavior.
 
-**Exit criteria:** an authenticated CSV upload creates a private object and a
+**Exit criteria:** an authenticated upload creates a private object and a
 `running` job row committed before any ingestion work; a tenant cannot retrieve
 another tenant's file status.
 
@@ -241,14 +250,14 @@ code and a terminal job row.
 1. Implement the ingestion service called by the route, using the
    application-lifetime database, MinIO, Qdrant, model, and logging clients.
 2. Validate the persisted records before fetching the MinIO object.
-3. Append progress events, parse CSV, create deterministic chunks, embed them,
+3. Append progress events, parse the document, create deterministic chunks, embed them,
    and upsert tenant-scoped Qdrant points — without holding a Postgres session
    open across the work.
 4. In a second short transaction, mark the job `succeeded` with counters or
    `failed` with a safe error summary, then return the terminal response.
 5. Make a retried upload safe: no duplicate logical chunks, no incorrect
    counters, and no transition from a terminal state back to `running`.
-6. Add unit tests for deterministic CSV chunks, point IDs, and terminal job
+6. Add unit tests for deterministic chunks, point IDs, and terminal job
    transitions. Add Testcontainers Qdrant and Postgres integration tests for
    tenant-filtered upserts, terminal state persistence, retrying an upload, and
    parser/Qdrant failure handling.
@@ -274,7 +283,7 @@ retrying the upload produces a correct final state without duplicate chunks.
    and the operations runbook.
 
 **Exit criteria:** a new developer can start the stack, apply migrations, upload a
-CSV, observe the job through completion, and understand how to investigate or
+a document, observe the job through completion, and understand how to investigate or
 retry a failure.
 
 ## Definition of done for the vertical slice
@@ -283,7 +292,7 @@ The first slice is done when the following path works in local Compose and is
 covered by automated tests:
 
 ```text
-POST /v1/files (authenticated CSV upload)
+POST /v1/files (authenticated DOCX/XLSX/CSV upload)
   -> raw bytes stored privately in MinIO
   -> source file and running job committed in Postgres, connection released
   -> parse/chunk on threads, embed in bounded concurrent batches