Files
chunking_strategies_evaluation/backlog/page-metadata-in-chunks.md

728 B
Raw Blame History

Page metadata in chunks

Why deferred

v1 Strategies and Qdrant payloads don’t need page numbers. Page-as-section markdown was rejected (arbitrary cuts). Page info is citation UX, not chunking semantics.

Trigger

Users (or the chatbot) need “see page N” citations, or Chunk Preview should show page ranges per chunk.

Approach to try

  1. During PDF parse, map character/block offsets → page numbers
  2. Attach optional page_start / page_end on Chunk / ChunkMetadata (nullable for DOCX)
  3. Do not inject ## Page N into markdown unless a Strategy specifically needs it

Success criteria

  • DOCX path unchanged (null pages)
  • Query/answer path can cite pages without changing Strategy ranking logic