Ali Zarinkolah
5cdfb70085
Adds src/application/ingestion/ -- Persian normalization, DOCX body walk with structural data/layout table classification, CSV/XLSX row rendering, and fixed-size token chunking (cl100k_base, 400/60/512) -- as pure functions per ADR-0015, tested against real production documents (asia_data_sample, kept out of the repo). ADR-0018 records where this diverges from ADR-0004 (fixed-size default, no invented headings/tree, structural table classification, header-provable labeling only). Plan 001's scope line is corrected from CSV-only to DOCX/XLSX/CSV, and CLAUDE.md's stale project-status paragraph is updated to match current implementation state.
31 lines
1.4 KiB
JSON
31 lines
1.4 KiB
JSON
{
|
||
"prose_docx": {
|
||
"filename": "bimeh_havades.docx",
|
||
"why": "38 paragraphs, all Normal, no tables: the plain-prose path, and the proof that no heading is invented where the document declares none."
|
||
},
|
||
"list_docx": {
|
||
"filename": "طرح ها-عمروحوادث-14050220.docx",
|
||
"why": "109 paragraphs mixing Normal and List Paragraph: prose carrying list styling."
|
||
},
|
||
"table_docx": {
|
||
"filename": "شماره تماس معاونت و مدیریتها14050309.docx",
|
||
"why": "One 35x5 contact table, no body prose. A data table with a real header row and a vertically merged first column."
|
||
},
|
||
"mixed_docx": {
|
||
"filename": "شرایط عمومی بیمه حوادث اشخاص14041205.docx",
|
||
"why": "Prose interleaved with two tables: a 30x3 list of injury compensations with NO header row, and an 11x2 table that has one."
|
||
},
|
||
"layout_docx": {
|
||
"filename": "چت بات-مهندسی.docx",
|
||
"why": "A 3x2 layout table whose single merged cell holds an entire 77k-character sub-document across 738 paragraphs and 5 nested tables."
|
||
},
|
||
"qa_xlsx": {
|
||
"filename": "fire14050319.xlsx",
|
||
"why": "Two-column Q&A sheet (q/a) plus a dead second sheet."
|
||
},
|
||
"branches_xlsx": {
|
||
"filename": "مشخصات شعب 28 اردیبهشت 1405.xlsx",
|
||
"why": "Branch directory: a merged title row above a two-row-tall merged header, and a vertically merged province column spanning each province's branches."
|
||
}
|
||
}
|