feat(ingestion): add DOCX/CSV/XLSX parsing and fixed-size chunking (ADR-0018)

Adds src/application/ingestion/ -- Persian normalization, DOCX body
walk with structural data/layout table classification, CSV/XLSX row
rendering, and fixed-size token chunking (cl100k_base, 400/60/512) --
as pure functions per ADR-0015, tested against real production
documents (asia_data_sample, kept out of the repo). ADR-0018 records
where this diverges from ADR-0004 (fixed-size default, no invented
headings/tree, structural table classification, header-provable
labeling only). Plan 001's scope line is corrected from CSV-only to
DOCX/XLSX/CSV, and CLAUDE.md's stale project-status paragraph is
updated to match current implementation state.

docs/adr/0004-docx-csv-chunking-strategy.md

@@ -4,6 +4,23 @@
 
 Accepted
 
+> Amended by
+> [ADR-0018](0018-docx-and-spreadsheet-parsing-with-fixed-size-chunking.md):
+> v1 ships **fixed-size** chunking rather than the semantic-aware default
+> below, and defers `qa_pair` detection and image captioning. Docx tables do
+> become `table_row` units as this ADR specifies, but only when they are data:
+> a table holding a cell larger than one chunk, or a cell containing nested
+> tables, is treated as page layout and its cells are chunked as prose.
+> Row labels are applied only when row 0 is provably a header, and cells are
+> joined unlabeled otherwise. No document tree is built, and no heading is
+> inferred from text. `.doc` is rejected with `415` pending an out-of-process
+> conversion service rather than shelling out to LibreOffice. ADR-0018 also
+> adds a Persian normalization step at parse time and fixes the chunk-size
+> numbers this ADR left open. The spreadsheet row-to-chunk rules, the
+> embedding model, the task-prefix invariant, and the `content_type` value set
+> below all apply unchanged — except that `.csv` is read with the standard
+> library `csv` module rather than `pandas`.
+
 ## Context
 
 ADR-0001 deferred two things to "when we start the docx/csv chunking work":

docs/adr/0018-docx-and-spreadsheet-parsing-with-fixed-size-chunking.md

@@ -0,0 +1,306 @@
+# 0018. DOCX and spreadsheet parsing with fixed-size chunking
+
+## Status
+
+Proposed
+
+## Context
+
+ADR-0004 specified the full parsing and chunking design: structural extraction
+before chunking (table rows and Q&A pairs kept atomic, prose chunked
+separately), semantic-aware chunking as the default, image captioning via a
+vision API, and legacy `.doc` conversion through headless LibreOffice. Plan
+001's first vertical slice needs a working parser now, and that full design is
+substantially more work than the slice can absorb. This ADR records what v1
+actually ships and why it differs, so the code does not silently contradict an
+Accepted ADR.
+
+Four forces shaped the decision:
+
+**A working extractor already exists.** The `chunking_strategies_evaluation`
+repository — the harness the user built to compare chunking strategies against
+this same Farsi corpus — contains a DOCX extractor that walks the document body
+in reading order and handles the "prose lives inside table cells" pattern
+common in Farsi documents exported from older Word versions. Roughly 200 lines
+of it are production-quality; the rest is evaluation scaffolding (five
+competing strategies, an LLM-as-judge benchmark, a dashboard, an HTML report
+generator). Porting it is cheaper and better-tested against real documents than
+writing a parser from scratch.
+
+**Semantic chunking does not fit the inline request.** ADR-0004 chose
+semantic-aware chunking as the default on the strength of the user's own
+offline accuracy comparison, in which fixed-size was the "viable, simpler
+runner-up". That comparison measured retrieval accuracy, not ingestion cost.
+Semantic boundary detection requires embedding every sentence *before* chunk
+boundaries can be decided — a second network round-trip pass inside the request
+budget ADR-0017 bounds with `INGESTION_TIMEOUT_SECONDS`. ADR-0017's own cost
+table already assumes the cheaper strategy, listing "Chunk (fixed-size,
+ADR-0004) | Blocking CPU, pure Python | Negligible". ADR-0004 and ADR-0017 are
+therefore already in tension, and this ADR resolves it toward ADR-0017 for v1.
+Fixed-size is acceptable specifically because ADR-0001 gives every chunk
+`previous_chunk_id`/`next_chunk_id` pointers: a chunk boundary that cuts a
+thought in half is recoverable by expanding to neighbors at retrieval time
+(ADR-0003).
+
+**Nothing normalizes the text the dense embedders see.** ADR-0005 resolved the
+sparse analyzer as `bm25-fa-norm-stop` — "normalization + stopword removal",
+computed in our own BM25 pipeline outside Qdrant. That covers the sparse vector
+only. Persian text authored on mixed Arabic/Persian keyboards contains both
+`ک` (U+06A9) and `ك` (U+0643), both `ی` (U+06CC) and `ي` (U+064A); these are
+distinct codepoints and therefore distinct tokens to `nomic-embed-text-v2-moe`
+and `text-embedding-3-large` alike, so the same Persian word can embed two
+different ways depending on which key the author pressed. Word documents in
+this corpus reliably contain both forms.
+
+**The corpus is spreadsheets more than it is CSVs.** ADR-0004 inspected the
+real sample and found the tabular files are entirely `.xlsx`; no `.csv` exists
+in practice. Plan 001 and the `source_files.source_type` CHECK constraint both
+name `csv`. The row-to-chunk mapping is identical either way — only the reader
+differs — so v1 reads both rather than forcing a manual export step that would
+silently drop the merged-cell values ADR-0004 warns about.
+
+## Decision
+
+### 1. Formats
+
+v1 ingests `.docx`, `.csv`, and `.xlsx`.
+
+`.doc` is rejected with `415 Unsupported Media Type`. ADR-0004 specified
+conversion via `soffice --headless --convert-to docx`; that is a subprocess
+with a multi-second startup cost running inside the inline request ADR-0017
+defines, and it adds a system binary to the container image. Conversion is
+deferred to an out-of-process HTTP conversion service, tracked in the backlog.
+`source_files.source_type` continues to allow `doc` so the row can be recorded
+once conversion lands.
+
+### 2. Structural units before chunking
+
+Every source file is decomposed into ordered **structural units** before any
+chunking runs, as ADR-0004 requires. There are two kinds:
+
+- `PARAGRAPH` — a run of flowing prose. Consecutive paragraphs accumulate into
+  one unit rather than one unit each, so the splitter sees flowing text instead
+  of a series of one-sentence fragments ("the whole remaining run of
+  paragraphs", per ADR-0004).
+- `TABLE_ROW` — one row of a data table. Atomic; split only when a single row
+  exceeds the model's sequence length.
+
+Walk `doc.element.body` children in document order — not `doc.paragraphs` and
+`doc.tables` separately — so tables interleaved with paragraphs keep their
+position. This is ADR-0004's rule, unchanged.
+
+**No headings are invented.** A real `Heading N` Word style becomes a `#`
+prefix on its paragraph; where a document declares none, none appear. The
+evaluation repository upgrades paragraphs to headings by text pattern (`^بخش`,
+`^\d+[-.]\d+`, leading `*`); those patterns are tuned to a Farsi regulatory
+corpus, not an insurance one, and a wrongly-detected heading silently reshapes
+the document in a way that is hard to notice downstream. They are not adopted.
+
+There is also **no document tree**. An earlier draft built a `Document >
+Section > Article > Paragraph` hierarchy from heading styles. Not one document
+in the sample corpus carries a single `Heading` style, so that tree was flat in
+every real case, and nothing consumed it — chunking works from the unit list,
+and ADR-0001's payload has no tree field. It is not built.
+
+### 3. Data tables against layout tables
+
+A docx table is either data or page furniture, and the two need opposite
+treatment. The classification is **structural, never a reading of content**: a
+data cell is by definition small enough to be a chunk, so a table containing a
+cell that alone exceeds `chunk_size`, or a cell containing nested tables, is a
+layout container. In the sample corpus this separates by two orders of
+magnitude — 24 to 171 tokens for the largest cell of each data table, against
+54,007 tokens for a cell holding an entire sub-document across 738 paragraphs
+and 5 nested tables.
+
+- **Data table** → one `TABLE_ROW` unit per row, rendered by the same code that
+  renders spreadsheet rows.
+- **Layout table** → its cells are prose, recursed into and folded into the
+  surrounding prose block.
+
+### 4. Table rows are labeled only when a header is provable
+
+Applied to docx tables and spreadsheets alike:
+
+- A header is **row 0 or nothing**. Never scan further down for a
+  header-shaped row. Scanning discarded every row above the match and then
+  labeled the rest from a data row, turning a 30-row compensation table into
+  chunks reading `80: 70`.
+- Leading rows that are structurally a merged banner — fewer than two populated
+  cells, or one value repeated across the row — are skipped first. That is a
+  fact about the merge, not a guess about meaning.
+- Row 0 is accepted as a header only when it is **inconsistent with the column
+  beneath it**: a text label above a numeric column, or a short label above much
+  longer values. This is the test `csv.Sniffer.has_header` uses; it is a
+  property of the table rather than a pattern borrowed from one document.
+- When no header is provable, cells are joined with `" | "` — unlabeled, but
+  never mislabeled. Losing a label is recoverable at retrieval time; labeling
+  every row from a data row is not.
+- A header merged vertically across two rows resolves to the same text in the
+  row below it; that duplicate is skipped rather than emitted as data.
+- A cell merged across columns is reported once per grid position it spans;
+  those repeats are collapsed.
+
+
+ is a pipeline invariant
+
+### 5. Persian normalization is a pipeline invariant
+
+Every extracted text block is normalized before chunking, for all formats:
+
+- Arabic to Persian letter folding: `ك`→`ک`, `ي`→`ی`, `ى`→`ی`, `أ`/`إ`→`ا`
+- `unicodedata.normalize("NFKC")`
+- Removal of harakat (diacritics) and tatweel
+- `¬` → space, then collapse runs of whitespace
+
+Digits and punctuation are **not** rewritten. Persian digits (`۱۲۳`) and
+Persian punctuation (`؛`, `٬`) are left as authored, because chunk `content` is
+what citations render back to the user and Western digits inside Persian prose
+read as wrong.
+
+Normalization runs **per text block, before the markdown is assembled** — the
+whitespace-collapse step maps `\n` to a space, so applying it to an assembled
+document would flatten every heading and paragraph onto a single line.
+
+This complements rather than replaces ADR-0005's sparse-side normalization,
+which additionally removes stopwords and is specific to the BM25 vector. It
+also stabilizes the text that feeds `content_hash`.
+
+### 6. Spreadsheet handling
+
+ADR-0004's rules stand, under the header discipline of section 4: each cell is
+rendered as `"{column_header}: {cell_value}"`, merged cell ranges are
+forward-filled before rendering, and sheets with no non-empty data rows are
+skipped.
+
+Forward-filling merges is not cosmetic. openpyxl stores a merged range's value
+only in its top-left cell, so in the branch directory the province is present
+on the first branch of each province and absent from every other one. Filling
+the range makes each row-chunk self-contained — a branch carries its province
+even though the source cell is blank.
+
+One correction: `.csv` is read with the standard library's `csv` module, not
+`pandas` as ADR-0004 states. Adding pandas for delimiter handling and row
+iteration is not warranted. `.xlsx` uses `openpyxl`, as ADR-0004 assumed.
+
+There is no sheet-shape sniffing. A two-column Q&A sheet and a branch-directory
+sheet go through the same generic renderer; a Q&A row renders as
+`question: …\nanswer: …` and a branch row as `branch_name: …\ncity: …`, both
+self-describing without a schema heuristic that could misfire.
+
+### 7. Chunking
+
+The `fixed_size` strategy, over tokens counted with tiktoken `cl100k_base`:
+
+| Setting | Value |
+|---|---|
+| `chunk_size` | 400 tokens |
+| `chunk_overlap` | 60 tokens |
+| `max_chunk_tokens` | 512 (hard cap) |
+
+These numbers are recorded here because they exist in no ADR today — ADR-0001
+explicitly left "size/overlap are tunable config, not fixed by this ADR" open,
+and ADR-0004 gives only the 512 ceiling.
+
+`cl100k_base` is a deliberate proxy. `text-embedding-3-large` has an 8191-token
+window and never binds; `nomic-embed-text-v2-moe`'s 512-token sequence length
+is the only real constraint. cl100k tokenizes Persian inefficiently while
+nomic's multilingual tokenizer does not, so a cl100k count reliably
+*over-estimates* the nomic count — measuring with cl100k and capping at 512 is
+safe in the conservative direction, without shipping a second tokenizer and its
+model download into the ingestion path. The 400/512 gap leaves headroom for the
+mandatory `search_document: ` task prefix (ADR-0004) and any heading text
+carried into a chunk.
+
+Spreadsheet rows are atomic and bypass the splitter. A row that exceeds
+`max_chunk_tokens` falls through the fixed-size splitter in place, emitting
+several ordered chunks, rather than being silently truncated by the embedding
+model.
+
+### 8. `content_type` values emitted
+
+ADR-0004's four-value set is unchanged. v1 emits `paragraph` (DOCX prose and
+flattened tables) and `table_row` (spreadsheet rows). `qa_pair` and
+`image_caption` remain defined but are not produced.
+
+### 9. Deferred, not rejected
+
+Semantic-aware chunking; DOCX `table_row` and `qa_pair` structural detection;
+embedded-image captioning; `.doc` conversion; Matryoshka-256 truncation. Each
+remains ADR-0004's stated intent; this ADR only records that v1 does not ship
+them.
+
+## Consequences
+
+### Positive
+
+- Plan 001's ingestion slice is unblocked with a parser already proven against
+  this specific Farsi corpus, rather than one written speculatively.
+- Chunking stays pure, synchronous, and cheap — it fits ADR-0017's inline
+  request budget with no network round-trip, and runs safely under
+  `anyio.to_thread.run_sync`.
+- Persian normalization closes a real defect that would otherwise degrade both
+  dense vectors silently, with no error and no obvious symptom.
+- Concrete chunk-size numbers and their rationale are now recorded somewhere
+  other than a config default, so a later change is a visible decision.
+- Q&A sheets, branch directories, and docx contact tables are all served by one
+  renderer with no schema sniffing, so a new sheet shape needs no new code.
+- Refusing to label a table whose header is unprovable means the parser degrades
+  to unlabeled rows instead of producing confidently wrong ones, which is the
+  failure mode that is hard to notice downstream.
+
+### Negative
+
+- **v1 ships the strategy the user's own comparison ranked second.** Retrieval
+  accuracy is expected to be measurably lower than semantic chunking would
+  give. Neighbor expansion via ADR-0001's pointers is the mitigation, and it is
+  unproven at this scale.
+- A table whose header cannot be proven — short text over short text, which is
+  genuinely ambiguous — produces unlabeled `" | "` rows. A reader or model can
+  still see the values but not which column each belongs to.
+- The layout-table rule keys on `chunk_size`, so changing that setting silently
+  changes which tables are treated as data. The observed margin is two orders of
+  magnitude, so this is unlikely to flip in practice, but it is a coupling.
+- A DOCX that alternates literal `سوال:`/`پاسخ:` paragraphs loses question/answer
+  atomicity; a boundary can fall between a question and its answer, because
+  `qa_pair` detection is deferred.
+- cl100k is a proxy for nomic's tokenizer. The relationship is safe in the
+  conservative direction for Persian, but a document in another language could
+  in principle tokenize the other way; the 512 assertion is what catches it.
+- Normalizing stored `content` means the text served in citations is not
+  byte-identical to the source document. Letter folding was chosen over full
+  normalization specifically to keep this difference invisible to a reader.
+- `.doc` files are rejected outright rather than converted, so any legacy
+  document must be re-saved by hand until the conversion service lands.
+
+## Alternatives Considered
+
+- **Implement ADR-0004 in full now** (DOCX table-row detection with header
+  inference, Q&A pair heuristics, image captioning, semantic boundary
+  detection): rejected for v1 as roughly triple the work, none of which exists
+  in the evaluation repository to port, and which would block the first
+  ingestion slice on parser research.
+- **Semantic chunking inside the inline request**: rejected — it adds a
+  per-sentence embedding pass to a request already bounded by
+  `INGESTION_TIMEOUT_SECONDS`, and ADR-0017 chose inline ingestion on the
+  assumption that chunking is negligible. Revisit when ingestion moves back off
+  the request path.
+- **The `nomic-embed-text-v2-moe` tokenizer** for exact chunk sizing: rejected
+  — it requires `transformers`/`tokenizers` and a model file download in the
+  ingestion path to buy precision that the conservative cl100k over-estimate
+  already provides.
+- **Character-based splitting** (no tokenizer at all): rejected — Persian
+  characters-per-token varies enough that a character budget cannot guarantee
+  the 512-token ceiling that actually matters.
+- **Full Persian normalization** including digit unification (`۱۲۳`→`123`) and
+  punctuation mapping: rejected — it would improve lexical matching slightly
+  when a query uses the other digit form, at the cost of rendering Persian
+  citations with Western digits.
+- **Storing raw and normalized text as separate payload fields**: rejected —
+  ADR-0001 fixes the payload field list, and doubling the stored text per point
+  is not justified when letter folding alone is visually lossless.
+- **Row-chunking DOCX tables like spreadsheets**: rejected for v1 — the Farsi
+  `.doc` exports in this corpus use tables as page layout, with ordinary prose
+  inside cells, so treating every row as an atomic unit would shred paragraphs
+  mid-sentence. Revisit once real chunk output from the table-heavy documents
+  has been inspected.

docs/backlog.md

@@ -0,0 +1,109 @@
+# Backlog
+
+Ideas and open questions not yet ready to be an ADR decision or a plan phase.
+Each entry is short: what the idea is, which ADR/plan it would eventually
+touch, and what's still unresolved. When an entry is picked up, turn it into
+an ADR amendment (or a new ADR) and delete it from here — this file is not a
+permanent record, `docs/adr/` is.
+
+## Legacy `.doc` conversion
+
+Relates to: [ADR-0018](adr/0018-docx-and-spreadsheet-parsing-with-fixed-size-chunking.md).
+
+`.doc` is rejected with `415`. ADR-0004 specified `soffice --headless
+--convert-to docx`, which ADR-0018 rejected as a multi-second subprocess inside
+an inline request. Gotenberg is the obvious candidate since it is already in
+use elsewhere — **but verify before committing to it**: Gotenberg's LibreOffice
+route is built for converting *to PDF*, and `.doc` → `.docx` output may not be
+supported on that endpoint. If it is not, the options are a dedicated
+LibreOffice sidecar or asking uploaders to re-save.
+
+## Structural units ADR-0004 specifies but v1 does not emit
+
+Relates to: [ADR-0004](adr/0004-docx-csv-chunking-strategy.md),
+[ADR-0018](adr/0018-docx-and-spreadsheet-parsing-with-fixed-size-chunking.md).
+
+- **`qa_pair`**: one sample document alternates literal `سوال:`/`پاسخ:`
+  paragraphs. v1 chunks it as prose, so a chunk boundary can fall between a
+  question and its answer.
+- **`image_caption`**: two sample documents embed images with no alt text.
+  ADR-0004 routes these through a vision API at ingest; v1 drops them silently.
+
+Both need a decision on whether heuristic detection is worth the misfire risk —
+the header-detection work showed that guessing structure is expensive when wrong.
+
+## Tables whose header cannot be proven
+
+Relates to: [ADR-0018](adr/0018-docx-and-spreadsheet-parsing-with-fixed-size-chunking.md).
+
+A table of short text over short text (`branch,city` with no numeric or long
+column) is genuinely ambiguous, so v1 emits unlabeled `" | "` rows rather than
+risk labeling every row from a data row. No file in the current corpus hits
+this, but a future one will.
+
+The honest fix is not a better heuristic — it is to stop guessing: let the
+upload declare whether a sheet has a header, since the uploader knows. That is
+a `POST /v1/files` contract change, so it belongs with plan 001 Phase 3 rather
+than in the parser.
+
+## Recalibrate chunk size against nomic's tokenizer
+
+Relates to: [ADR-0018](adr/0018-docx-and-spreadsheet-parsing-with-fixed-size-chunking.md).
+**Revisit after retrieval quality is measurable** — deliberately deferred, not
+forgotten.
+
+ADR-0018 counts tokens with tiktoken `cl100k_base` and caps chunks at 512. But
+512 is `nomic-embed-text-v2-moe`'s limit, measured in *nomic's* tokenizer, not
+OpenAI's. Those are different units, and on Persian they differ by a lot.
+
+Measured against a real production document (`bimeh_havades.docx`, 5,911 chars
+of Farsi) via the Ollama server that already hosts the model:
+
+| Sample | cl100k tokens | nomic tokens | ratio |
+|---|---|---|---|
+| 300 chars | 217 | 80 | 2.71 |
+| 600 chars | 426 | 165 | 2.58 |
+| 1,200 chars | 846 | 303 | 2.79 |
+
+So **~2.7 cl100k tokens per nomic token** on Persian. The current
+`chunk_size=400` is therefore about **148 nomic tokens — roughly 29% of the
+512-token window**. Chunks land near 570 characters where ~1,500 would fit.
+
+Two things this measurement also established:
+
+- **Silent truncation is real, and now demonstrated.** Feeding 2,400 and 4,800
+  characters both returned `prompt_eval_count` of exactly 512, with no error
+  and no warning. This is what ADR-0004 meant by "silently truncated by the
+  model, not an error", confirmed on our own hardware.
+- **Measuring nomic tokens needs no new dependency.** Ollama's `/api/embed`
+  returns `prompt_eval_count`, so the real count is obtainable from the
+  embedding call we already have to make. Note the value saturates at 512, so
+  it cannot measure anything longer than the window — calibration samples must
+  stay under it.
+
+When picking this up, decide between: raising `chunk_size`/`max_chunk_tokens`
+in cl100k terms using a calibration ratio (cheap, drifts if the corpus language
+mix changes); counting with nomic's own tokenizer offline via HuggingFace
+`tokenizers` and its `tokenizer.json` (exact, and lighter than ADR-0018
+assumed — the tokenizer file only, not the 475M-param model weights); or
+keeping small chunks because neighbor expansion recovers the context anyway.
+
+Do not change this on the ratio alone. The reason to keep 400/60/512 for now is
+that smaller chunks are not automatically worse for retrieval — measure
+retrieval quality first, then decide.
+
+Also note `nomic-embed-text:latest` (v1.5) is on the same Ollama server with a
+2,048-token context, but it is the English-focused model; v2-moe is the
+multilingual one and the reason ADR-0004 chose it for Farsi. Do not switch to
+v1.5 just to get a bigger window.
+
+## Get LLM usage/price from the OpenAI API
+
+Relates to: [ADR-0009](adr/0009-postgres-sqlalchemy-alembic-schema.md)'s
+`llm_calls`/`llm_pricing` tables.
+
+Get token usage and price from the OpenAI API's response metadata, instead of
+computing/tracking them ourselves. Need to check whether OpenAI actually
+returns price, or only token counts — if only counts, we still need
+`llm_pricing` for price and this only changes how `llm_calls` gets its
+usage numbers.

docs/plans/001-ingestion-vertical-slice.md

@@ -3,7 +3,7 @@
 ## Purpose
 
 This plan turns the accepted architectural direction in the ADRs into the first
-working product slice: a tenant-scoped CSV upload is stored in MinIO, represented
+working product slice: a tenant-scoped DOCX/XLSX/CSV upload is stored in MinIO, represented
 by durable Postgres records, parsed/chunked/embedded inline in the request
 (ADR-0017), and indexed as Qdrant points before the response returns.
 
@@ -47,14 +47,19 @@ them.
 
 ### In scope
 
-- `POST /v1/files` for authenticated tenant-scoped **CSV** upload.
+- `POST /v1/files` for authenticated tenant-scoped **DOCX, XLSX, and CSV** upload.
+  `.doc` is rejected with `415` pending an out-of-process conversion service
+  (ADR-0018). An earlier revision of this plan scoped the slice to CSV only and
+  placed DOCX out of scope; the real corpus is DOCX and XLSX, so ADR-0018
+  corrects that.
 - File validation, size limits, content hashing, and streaming upload to MinIO.
 - Alembic-managed Postgres schema for the minimal tenant/auth, source file,
   ingestion job, and job event records needed by this slice.
 - Inline ingestion in `POST /v1/files`, with batched/bounded-concurrent
   embedding, thread-offloaded parsing, and enforced size/timeout/capacity
   bounds.
-- CSV parsing and deterministic chunk creation.
+- DOCX/XLSX/CSV parsing into structural units and deterministic chunk creation
+  (ADR-0018, implemented in `src/application/ingestion/`).
 - Tenant-filtered Qdrant point upserts using deterministic point identifiers.
 - Job status/progress persistence and `GET /v1/files/{file_id}` status lookup.
 - Structured correlation logging at HTTP and ingestion-stage boundaries.
@@ -63,7 +68,11 @@ them.
 
 ### Explicitly out of scope
 
-- XLSX, DOCX, and legacy DOC ingestion.
+- Legacy `.doc` ingestion — rejected with `415` until an out-of-process
+  conversion service exists (ADR-0018). DOCX and XLSX are **in** scope; they were
+  listed here before ADR-0018 corrected the scope line.
+- `qa_pair` structural detection and embedded-image captioning (ADR-0004),
+  deferred by ADR-0018.
 - The conversational LangGraph API and SSE streaming.
 - Final reranker selection, GPU deployment, or unresolved model licensing from
   ADR-0005.
@@ -196,7 +205,7 @@ reads/writes and valid job transitions.
 ### Phase 3: MinIO upload and durable job creation
 
 1. Implement API-key authentication and `AuthContext` tenant derivation.
-2. Implement `POST /v1/files` for CSV only, including streaming-size controls,
+2. Implement `POST /v1/files` for DOCX, XLSX, and CSV, including streaming-size controls,
    file-type validation, SHA-256 calculation, and a private MinIO upload using
    an internal object key.
 3. In one short Postgres transaction, persist `source_files` and create
@@ -208,11 +217,11 @@ reads/writes and valid job transitions.
    response that does not expose raw storage credentials or internal artifacts.
 6. Add cleanup/compensation handling for a MinIO upload that succeeds while the
    database transaction fails.
-7. Add unit/API tests for trusted tenant derivation, CSV validation, idempotency,
+7. Add unit/API tests for trusted tenant derivation, upload validation, idempotency,
    the terminal `201 Created` response, and tenant-scoped status. Add MinIO adapter integration tests
    for server-derived private object paths and compensation behavior.
 
-**Exit criteria:** an authenticated CSV upload creates a private object and a
+**Exit criteria:** an authenticated upload creates a private object and a
 `running` job row committed before any ingestion work; a tenant cannot retrieve
 another tenant's file status.
 
@@ -241,14 +250,14 @@ code and a terminal job row.
 1. Implement the ingestion service called by the route, using the
    application-lifetime database, MinIO, Qdrant, model, and logging clients.
 2. Validate the persisted records before fetching the MinIO object.
-3. Append progress events, parse CSV, create deterministic chunks, embed them,
+3. Append progress events, parse the document, create deterministic chunks, embed them,
    and upsert tenant-scoped Qdrant points — without holding a Postgres session
    open across the work.
 4. In a second short transaction, mark the job `succeeded` with counters or
    `failed` with a safe error summary, then return the terminal response.
 5. Make a retried upload safe: no duplicate logical chunks, no incorrect
    counters, and no transition from a terminal state back to `running`.
-6. Add unit tests for deterministic CSV chunks, point IDs, and terminal job
+6. Add unit tests for deterministic chunks, point IDs, and terminal job
    transitions. Add Testcontainers Qdrant and Postgres integration tests for
    tenant-filtered upserts, terminal state persistence, retrying an upload, and
    parser/Qdrant failure handling.
@@ -274,7 +283,7 @@ retrying the upload produces a correct final state without duplicate chunks.
    and the operations runbook.
 
 **Exit criteria:** a new developer can start the stack, apply migrations, upload a
-CSV, observe the job through completion, and understand how to investigate or
+a document, observe the job through completion, and understand how to investigate or
 retry a failure.
 
 ## Definition of done for the vertical slice
@@ -283,7 +292,7 @@ The first slice is done when the following path works in local Compose and is
 covered by automated tests:
 
 ```text
-POST /v1/files (authenticated CSV upload)
+POST /v1/files (authenticated DOCX/XLSX/CSV upload)
   -> raw bytes stored privately in MinIO
   -> source file and running job committed in Postgres, connection released
   -> parse/chunk on threads, embed in bounded concurrent batches