chunking_strategies_evaluation/backlog/pdfplumber-tables.md

pdfplumber for tables

Why deferred

v1 uses PyMuPDF only. Dual extractors add complexity before we know tables are the failure mode.

Trigger

Chunk Preview / Experiments show table cells concatenated in wrong order, or key cells missing, on table-heavy insurance PDFs — after Heading Reconstruction is otherwise fine.

Approach to try

1. Keep PyMuPDF for body text + Heading Reconstruction
2. Detect table regions (PyMuPDF or pdfplumber) and extract those via pdfplumber
3. Flatten tables the same way DOCX does (sequential text blocks), not HTML tables in markdown (unless Strategies learn to use them)

Success criteria

• Questions that answer from table cells improve without regressing prose headings
• No second full-document parse path unless necessary