feat(documents): add text pdf ingestion with heading reconstruction

Why:
- Support Text PDFs in the same DocumentTree/markdown contract as DOCX.

Changes:
- PyMuPDF parser, text-layer gate, shared heading heuristics, upload dispatch for .pdf.

Impact:
- Scanned/image PDFs are rejected at upload; requires pymupdf installed.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
2026-08-02 16:59:42 +03:30
parent afc20200f7
commit 55907a8dec
8 changed files with 418 additions and 131 deletions

View File

@@ -0,0 +1,16 @@
# PyMuPDF for Text PDF extraction
Text PDF ingestion uses PyMuPDF (`fitz`) so Heading Reconstruction can read font size/weight, text blocks, and outline bookmarks — the same structural signals `python-docx` gives us for DOCX. We rejected flat string extractors (`pypdf`), table-first libraries as the primary path (`pdfplumber`), and LibreOffice PDF→DOCX conversion (lossy styles, slow, fights ADR 0006’s “prefer native structure”).
## Considered Options
- **PyMuPDF** — chosen; best fit for Heading Reconstruction; AGPL acceptable while this stays an internal benchmarker
- **pdfplumber** — strong tables, weaker hierarchy; deferred to backlog if tables are a measured failure
- **pypdf** — too little layout/font signal
- **LibreOffice PDF→DOCX → existing parser** — reuses DOCX path but conversion quality is unreliable
## Consequences
- Add `pymupdf` dependency; keep a single PDF code path in the documents parser seam
- If we later ship the parser as a distributed service, revisit AGPL vs a permissive alternative
- Table-heavy and Scanned PDF work stays out of this ADR (see `backlog/`)