docs: capture pdf domain language and deferred ingestion backlog

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
2026-08-02 16:59:42 +03:30
parent 987b0493ac
commit 118255acdc
8 changed files with 166 additions and 1 deletions

View File

@@ -0,0 +1,20 @@
# LibreOffice PDF→DOCX fallback
## Why deferred
We already convert `.doc` → `.docx` via LibreOffice. PDF→DOCX is tempting for “reuse the DOCX parser,” but styles/fonts often come out wrong — fighting the reason we chose PyMuPDF.
## Trigger
Heading Reconstruction on PyMuPDF has an unacceptable false-positive/false-negative rate on a representative sample, and bookmarks/fonts are absent.
## Approach to try
1. Optional fallback: `soffice --headless --convert-to docx` then existing `parse_docx`
2. Compare markdown side-by-side (PyMuPDF vs converted DOCX) in Chunk Preview before making it default
3. Keep as opt-in or auto-fallback only when PDF outline + font variance is below a threshold
## Success criteria
- Converted DOCX headings beat PyMuPDF heuristics on the failing sample set
- Conversion time acceptable for upload UX (or async later — currently sync REST)