Mahdi Bazrafshan
55907a8dec
Why: - Support Text PDFs in the same DocumentTree/markdown contract as DOCX. Changes: - PyMuPDF parser, text-layer gate, shared heading heuristics, upload dispatch for .pdf. Impact: - Scanned/image PDFs are rejected at upload; requires pymupdf installed. Co-authored-by: Cursor <cursoragent@cursor.com>
17 lines
1.1 KiB
Markdown
17 lines
1.1 KiB
Markdown
# PyMuPDF for Text PDF extraction
|
||
|
||
Text PDF ingestion uses PyMuPDF (`fitz`) so Heading Reconstruction can read font size/weight, text blocks, and outline bookmarks — the same structural signals `python-docx` gives us for DOCX. We rejected flat string extractors (`pypdf`), table-first libraries as the primary path (`pdfplumber`), and LibreOffice PDF→DOCX conversion (lossy styles, slow, fights ADR 0006’s “prefer native structure”).
|
||
|
||
## Considered Options
|
||
|
||
- **PyMuPDF** — chosen; best fit for Heading Reconstruction; AGPL acceptable while this stays an internal benchmarker
|
||
- **pdfplumber** — strong tables, weaker hierarchy; deferred to backlog if tables are a measured failure
|
||
- **pypdf** — too little layout/font signal
|
||
- **LibreOffice PDF→DOCX → existing parser** — reuses DOCX path but conversion quality is unreliable
|
||
|
||
## Consequences
|
||
|
||
- Add `pymupdf` dependency; keep a single PDF code path in the documents parser seam
|
||
- If we later ship the parser as a distributed service, revisit AGPL vs a permissive alternative
|
||
- Table-heavy and Scanned PDF work stays out of this ADR (see `backlog/`)
|