mba/chunking_strategies_evaluation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: capture pdf domain language and deferred ingestion backlog

Browse Source 
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Mahdi Bazrafshan
2026-08-02 16:59:42 +03:30
parent 987b0493ac
commit 118255acdc
8 changed files with 166 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
backlog/multi-column-layout.md Normal file
View File

@@ -0,0 +1,20 @@
# Multi-column layout reading order
## Why deferred
Many Text PDFs are single-column policies. Multi-column (or sidebar) layouts can make naive top-to-bottom extraction interleave columns.
## Trigger
Chunk Preview shows sentences from column A mixed with column B; Experiment Faithfulness drops on those docs only.
## Approach to try
1. Use PyMuPDF block/bbox clustering to detect columns
2. Read column-by-column (right-to-left for Farsi multi-column if applicable)
3. Add a fixture PDF with known two-column layout to regression tests
## Success criteria
- Reading order matches human reading on the fixture
- No regression on single-column Farsi docs