feat(documents): add text pdf ingestion with heading reconstruction
Why: - Support Text PDFs in the same DocumentTree/markdown contract as DOCX. Changes: - PyMuPDF parser, text-layer gate, shared heading heuristics, upload dispatch for .pdf. Impact: - Scanned/image PDFs are rejected at upload; requires pymupdf installed. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
16
docs/adr/0016-pymupdf-for-text-pdf-extraction.md
Normal file
16
docs/adr/0016-pymupdf-for-text-pdf-extraction.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# PyMuPDF for Text PDF extraction
|
||||
|
||||
Text PDF ingestion uses PyMuPDF (`fitz`) so Heading Reconstruction can read font size/weight, text blocks, and outline bookmarks — the same structural signals `python-docx` gives us for DOCX. We rejected flat string extractors (`pypdf`), table-first libraries as the primary path (`pdfplumber`), and LibreOffice PDF→DOCX conversion (lossy styles, slow, fights ADR 0006’s “prefer native structure”).
|
||||
|
||||
## Considered Options
|
||||
|
||||
- **PyMuPDF** — chosen; best fit for Heading Reconstruction; AGPL acceptable while this stays an internal benchmarker
|
||||
- **pdfplumber** — strong tables, weaker hierarchy; deferred to backlog if tables are a measured failure
|
||||
- **pypdf** — too little layout/font signal
|
||||
- **LibreOffice PDF→DOCX → existing parser** — reuses DOCX path but conversion quality is unreliable
|
||||
|
||||
## Consequences
|
||||
|
||||
- Add `pymupdf` dependency; keep a single PDF code path in the documents parser seam
|
||||
- If we later ship the parser as a distributed service, revisit AGPL vs a permissive alternative
|
||||
- Table-heavy and Scanned PDF work stays out of this ADR (see `backlog/`)
|
||||
Reference in New Issue
Block a user