chunking_strategies_evaluation/docs/LLD.md

Low-Level Design (LLD)

RAG Chunking Benchmarker — module interfaces, data schemas, algorithms, and API contracts.

Audience: implementers
Companion: HLD · API Reference · CONTEXT.md

---

1. Package Map

src/
├── main.py                      # App factory, middleware, router mount
├── static/index.html            # Dashboard (React + Babel CDN)
├── core/
│   ├── config.py                # Settings (.env)
│   ├── models.py                # Chunk, DocumentTree, StrategyName
│   ├── exceptions.py            # Domain errors + handlers
│   └── dependencies.py          # OpenAI / Ollama / Qdrant singletons
├── storage/
│   ├── sqlite.py                # documents, queries, experiments, app_settings
│   └── qdrant.py                # Model Corpus CRUD + search
├── documents/
│   ├── parser.py                # DOCX / .doc (+ LibreOffice)
│   ├── pdf_parser.py            # Text PDF + Text-layer Gate
│   ├── heading_heuristics.py    # PDF Heading Reconstruction helpers
│   ├── service.py               # Upload / process / delete
│   ├── routes.py
│   └── models.py                # ProcessRequest, DocumentResponse, …
├── chunking/
│   ├── base.py                  # ChunkingStrategy ABC, tokens, sentences
│   ├── embedding.py             # Boundary/Corpus resolve + embed_texts
│   ├── embedding_models.py      # Registry, thresholds, task prefixes
│   ├── service.py               # run_strategies orchestration
│   └── strategies/
│       ├── fixed_size.py
│       ├── recursive.py
│       ├── semantic.py
│       ├── contextual_retrieval.py
│       └── semantic_parent_child.py
├── benchmarking/
│   ├── query_service.py         # Query + Neighbor Expansion
│   ├── benchmark_service.py     # Experiment runner + cost estimate
│   ├── evaluation.py            # LLM-as-Judge
│   ├── report.py                # HTML reports
│   ├── routes.py
│   └── models.py
└── admin/
    ├── routes.py                # /admin/*
    └── service.py

---

2. Application Bootstrap

Entry: create_app() in src/main.py

1. init_db() — create/migrate SQLite schema
2. Register CORS + RequestLoggingMiddleware
3. Exception handlers: ChunkingError, BenchmarkError, QueryError
4. Routers: documents, benchmarking, admin
5. Mount StaticFiles at /app → src/static/

# Singletons (lru_cache)
get_openai_client()   # Cloud LLM + cloud embeds
get_ollama_client()   # OpenAI-compatible client → Ollama base URL
get_qdrant_client()

---

3. Configuration

Settings (src/core/config.py) — selected fields:

Field	Default	Use
openai_api_key	required	Cloud Provider
embedding_model	text-embedding-3-large	Config legacy; runtime uses Registry + Admin
llm_model	gpt-4o-mini	Answer + judge
ollama_base_url	host URL	Local embeds
qdrant_url	http://localhost:6333	Vectors
top_k	5	Retrieval
neighbor_prev / neighbor_next	0 / 0	Expansion defaults
chunk_size / chunk_overlap	512 / 50	fixed_size / recursive targets
semantic_threshold	0.3	Fallback if model default missing
semantic_min_chunk_size	3	Min units per semantic chunk
database_url	sqlite:///./data/chunking_benchmark.db	Metadata
pdf_min_total_chars / pdf_min_median_chars_per_page	100 / 40	Text-layer Gate

---

4. Core Domain Models

4.1 StrategyName

class StrategyName(str, Enum):
    RECURSIVE = "recursive"
    FIXED_SIZE = "fixed_size"
    SEMANTIC = "semantic"
    CONTEXTUAL_RETRIEVAL = "contextual_retrieval"
    SEMANTIC_PARENT_CHILD = "semantic_parent_child"

4.2 Chunk (unified)

Field	Type	Notes
document_name	str	Source filename
chunk_id	str	{strategy}_{safe_doc}_{index:06d}
strategy_name	StrategyName
chunk_index	int	Document order (used by Neighbor Expansion)
text	str	Stored in Qdrant; used for LLM context
token_count / character_count	int	tiktoken cl100k_base
parent_id	str | None	Parent-child Strategy
enriched_content	str | None	Contextual retrieval embed text (not always in payload)

chunk_to_metadata() drops enriched_content for Qdrant payload (ChunkMetadata).

4.3 DocumentTree

DocumentTree
  └── root: DocumentTreeNode
        node_type: document | section | article | paragraph
        text, heading, heading_level, children[]

Serialized as JSON in SQLite documents.document_tree. Chunking primarily uses parsed_text (markdown); tree supports structure-aware Strategies and preview.

---

5. Storage LLD

5.1 SQLite schema

documents

Column	Type	Purpose
id	TEXT PK	UUID hex
filename	TEXT	Original name
parsed_text	TEXT	Markdown for chunking
document_tree	TEXT	JSON tree
chunk_counts	TEXT	JSON {strategy: count}
last_corpus_embedding_model_id	TEXT	Provenance
last_boundary_embedding_model_id	TEXT	Provenance
created_at	TEXT	ISO UTC

queries

Column	Purpose
id, document_id, strategy_name	Identity
question, answer	Content
retrieved_chunks	Flat LLM/eval list (JSON)
expansion_tree	Per-hit neighbors (JSON)
latency_breakdown, token_usage	Observability JSON
created_at

experiments

Column	Purpose
id, document_id	Identity
benchmark_config	top_k, neighbor_prev/next, etc.
questions, per_question	Inputs + per-cell results
aggregate_metrics	Per-Strategy averages
strategies_used	JSON list
embedding_model_id, embedding_provider	Corpus snapshot
boundary_embedding_model_id	Boundary snapshot (nullable)
created_at

app_settings — key/value for:

• corpus_embedding_model_id, boundary_embedding_model_id
• Legacy active_embedding_model_id (migrated into both roles)
• semantic_threshold:{model_id} overrides

Connection: WAL mode, foreign keys ON, one connection per call (no pool).

5.2 Qdrant

Collection name

def collection_name(strategy, model_id) -> str:
    return f"{strategy}__{sanitize(model_id)}_collection"
# sanitize: replace : and / with -

Point

Field	Value
id	uuid5(NAMESPACE_URL, chunk_id)
vector	Corpus embedding (dim from model: 1536 / 3072 / 768)
payload	ChunkMetadata fields

Search

• Cosine distance
• Optional filter: document_name == filename
• Returns payload + score

Neighbor fetch: get_chunks_by_indices(strategy, document_name, indices, model_id) for Expansion.

Admin: list/create/delete collections; wipe points.

---

6. Embedding Subsystem

6.1 Registry (embedding_models.py)

id	Provider	Dimension	Default semantic_threshold	Task prefixes
text-embedding-3-small	cloud	1536	0.3	no
text-embedding-3-large	cloud	3072	0.3	no (default Admin)
nomic-embed-text-v2-moe	local	768	0.6	yes (search_document: / search_query:)

EmbeddingModelSpec: id, provider, model_name, dimension, display_name, task_prefixes, default_semantic_threshold.

6.2 Role resolution (embedding.py)

resolve_corpus_model(optional_id)   → snapshot for process/query/experiment
resolve_boundary_model(optional_id) → snapshot for semantic cuts

get_corpus_embedding_model() / get_boundary_embedding_model()
  → app_settings → registry → DEFAULT_CLOUD_MODEL_ID

Admin switches persist to SQLite; mid-flight Admin changes do not affect an in-progress operation that already resolved snapshots.

6.3 embed_texts / embed_single

1. apply_task_prefixes(texts, model, purpose) if needed
2. Batch: OpenAI 2048 / Ollama 64
3. Route to get_openai_client() or get_ollama_client() by Provider
4. Raise EmbeddingError on failure

Threshold: get_semantic_threshold(model_id) = Admin override → registry default. Boundary Strategy cuts always use Boundary model's threshold (never Corpus).

---

7. Documents LLD

7.1 Upload

upload_document(filename, bytes)
  → temp file → parse_document(path)
  → db.save_document(parsed_text=markdown, document_tree=JSON)

Supported suffixes: .docx, .doc, .pdf (SUPPORTED_SUFFIXES).

PDF path (pdf_parser.py):

1. Text-layer Gate (pdf_min_total_chars, pdf_min_median_chars_per_page) — reject Scanned PDF
2. Extract text + Heading Reconstruction (outline → font → Farsi/English heuristics)
3. Table Flattening to sequential plain text
4. Emit markdown # / ## + DocumentTree

7.2 Process

ProcessRequest:
  strategies: list[StrategyName]  # default all five
  boundary_model_id: str | None
  corpus_model_id: str | None

process_document → chunking.service.run_strategies(...)
  → ProcessResponse(completed[], failed[], corpus_*, boundary_*)

7.3 Delete

Deletes SQLite row (cascade queries/experiments) and Qdrant points for that document across known collections (implementation in documents/service.py + qdrant).

---

8. Chunking LLD

8.1 Interface

class ChunkingStrategy(ABC):
    name: StrategyName

    @abstractmethod
    def chunk(*, doc_name: str, tree: DocumentTree, markdown: str) -> list[Chunk]:
        ...

Semantic Strategies accept extra kwargs from orchestrator (sentence_embeddings / paragraph_embeddings, semantic_threshold).

8.2 Orchestrator (run_strategies)

corpus = resolve_corpus_model(corpus_model_id)
boundary = resolve_boundary_model(...) if any(semantic*) else unused

for strategy in strategies:
  try:
    ensure_collection(strategy, corpus.id, corpus.dimension)
    chunks = _chunk_document(...)   # inject Boundary embeds if needed
    texts = enriched_content or text
    embeddings = embed_texts(texts, model=corpus, purpose="document")
    upsert_chunks(chunks, embeddings, model_id=corpus.id)
  except → failed[] (others continue)

update_chunk_counts + update_process_embedding_provenance

8.3 Strategy algorithms (summary)

Strategy	Input units	Boundary	Output
fixed_size	markdown tokens	none	sliding window chunk_size / chunk_overlap
recursive	markdown	cascade # → \n\n → \n → punct → space	merge up to target size
semantic	sentences (split_sentences, Farsi-aware)	adjacent cosine vs Boundary threshold; semantic_min_chunk_size	joined sentence groups
contextual_retrieval	base chunks + LLM context prefix	none	text = original; enriched_content = prefix+text for embed
semantic_parent_child	paragraphs	paragraph cosine vs Boundary threshold	parents + children with parent_id

Semantic Strategies fail hard if Boundary embeddings missing/mismatched (ADR-0020) — no fixed-count fallback.

Sentence split fallback: punctuation → non-empty lines → blank-line paragraphs → single unit.

Deep dive: strategy-technical-details.md.

---

9. Query LLD

9.1 run_query pipeline

1. Load document (filename for filter)
2. Resolve Corpus model
3. embed_single(question, purpose="query")
4. qdr.search(strategy, vector, top_k, document_filter=filename, model_id)
5. apply_neighbor_expansion(...) → retrieved_chunks, expansion_tree
6. _build_context (parent scroll for semantic_parent_child)
7. _generate_answer (settings.llm_model, temp 0)
8. db.save_query(...)

9.2 Neighbor Expansion (apply_neighbor_expansion)

Applies only when strategy == fixed_size and (neighbor_prev > 0 or neighbor_next > 0).

1. Build Expansion Tree nodes for each top-k hit (score order)
2. Collect needed chunk_index values: hit±prev/next (skip < 0)
3. Fetch missing indices via Qdrant
4. Fill neighbors_prev / neighbors_next per hit
5. Flat list: hits + neighbors, dedupe by chunk_id (prefer hit), sort by chunk_index
6. Neighbors have role="neighbor", score=None

Knobs ignored for non-fixed_size Strategies (tree still returns hits with empty neighbor arrays).

9.3 Parent-child context

For semantic_parent_child, scroll Qdrant by chunk_id == parent_id and append parent text under each child in the prompt context.

---

10. Benchmark LLD

10.1 Questions format

{
  "questions": [
    {
      "id": "q1",
      "question": "...",
      "expected_answer": "...",
      "category": "...",
      "difficulty": "..."
    }
  ]
}

Load from questions_file path or inline questions list.

10.2 run_benchmark

embedding_model = resolve_corpus_model(corpus_model_id)
boundary_id = doc.last_boundary_embedding_model_id  # if semantic in strategies

for question in questions:
  for strategy in strategies:
    result = run_query(..., neighbor_prev, neighbor_next, embedding_model=...)
    scores = evaluate_single(question, retrieved_chunks, expected, generated)
    append per_question row

aggregate_metrics[strategy] = averages + hallucination_rate
best_strategy = argmax (e.g. answer_similarity / composite — see service)
save experiment with provenance + benchmark_config

10.3 Evaluation (evaluate_single)

LLM returns JSON:

{
  "context_relevance": 1-10,
  "answer_similarity": 1-10,
  "faithfulness": 1-10,
  "hallucination": true|false,
  "reasoning": "..."
}

response_format=json_object, temperature 0. Metrics definitions: evaluation-metrics.md.

10.4 Reports

report.py renders managerial or technical HTML from an Experiment (rankings, KPIs, Expansion Tree samples, token usage).

10.5 Cost estimate

estimate_cost(num_questions, num_strategies) — heuristic USD; Local Corpus → embedding cost 0.

---

11. Admin LLD

Endpoint	Behavior
GET /admin/health	App + Qdrant + SQLite ping
GET /admin/embedding-models	Registry + Boundary/Corpus defaults + thresholds
PUT .../corpus · .../boundary	Set role defaults
PUT .../{id}/semantic-threshold	Persist override (0, 1]
GET/POST/DELETE /admin/qdrant/collections*	Collection CRUD + wipe points
GET /admin/chunks/{doc_id}?strategy=	Chunk Preview from Qdrant
GET/POST/DELETE /admin/questions*	Manage files/*.json
POST /admin/cost-estimate	Same heuristic as dry-run

Admin does not duplicate document/query/benchmark domain endpoints (ADR-0006).

---

12. Dashboard LLD (behavioral)

Concern	Design
Delivery	One index.html; React + ReactDOM + Babel from CDN
Navigation	Top Tabs; components stay mounted (useState on App)
Theme	Dark #111113… + amber accent (ADR-0008)
Word vs PDF	Documents/Query/Benchmarks filter Word; PDF Tab = same sections + format filter
Retrieval Inspect	Full-page mode in Benchmarks: question rail, Strategy picker, Generated | Expected, Expansion Tree
Decision Board	Client-side Candidate discovery from /experiments; Corpus filter; exclude bad Experiment ids; two-stage heat comparison
Neighbor badge	±P/N on Experiment list/Compare; mismatch warning across Boundary/Corpus/Neighbor

No client router or global store — props from root state.

---

13. API Surface (concise)

Method	Path	Service
GET/POST	/documents	list / upload
POST	/documents/{id}/process	Strategies
DELETE	/documents/{id}	delete
GET	/strategies	catalog
POST	/queries	query
GET	/queries/{id}	history
POST	/benchmarks	Experiment or dry_run
GET	/benchmarks/{id}	Experiment detail
GET	/benchmarks/{id}/report	HTML
GET	/experiments	list (+ filters used by Decision Board)
*	/admin/*	ops
GET	/app/	Dashboard

Full schemas: api-reference.md, Pydantic models in documents/models.py and benchmarking/models.py.

Request highlights

# Process
boundary_model_id: Optional[str]
corpus_model_id: Optional[str]

# Query / Benchmark
top_k: int = 5
neighbor_prev / neighbor_next: int = 0..5
corpus_model_id: Optional[str]

---

14. Error Model

Exception	Typical cause	HTTP
DocumentProcessingError	Bad file, Text-layer Gate, missing doc	400
ChunkingError	Unknown Strategy, semantic embed failure	400
EmbeddingError	Provider/API failure	400
QdrantError	Collection/upsert/search failure	400
QueryError	Missing doc, LLM answer failure	400
BenchmarkError	Bad questions file, eval failure	400

Per-Strategy process failures are returned in strategies_failed without aborting the whole request.

---

15. Sequence Diagrams

Process (one Strategy)

Client → DocumentsAPI → DocumentService → ChunkingService
  ChunkingService → resolve Boundary/Corpus
  ChunkingService → SemanticStrategy.chunk (w/ Boundary embeds)
  ChunkingService → embed_texts (Corpus)
  ChunkingService → Qdrant.upsert
  ChunkingService → SQLite.update counts + provenance
Client ← ProcessResponse

Query with Neighbor Expansion

Client → QueryAPI → run_query
  → embed_single (Corpus)
  → Qdrant.search top-k
  → apply_neighbor_expansion → get_chunks_by_indices
  → _build_context → OpenAI chat
  → SQLite.save_query
Client ← QueryResponse (retrieved_chunks + expansion_tree)

---

16. Extension Points

Extension	Steps
New Strategy	Subclass ChunkingStrategy; register in _STRATEGIES; add StrategyName; update Dashboard labels
New Embedding Model	Add EmbeddingModelSpec to registry; ensure dimension matches Qdrant collections
New eval metric	Extend judge prompt JSON + aggregate in benchmark_service + report templates
New Admin op	Prefer /admin only when domain routers lack the capability

---

17. Scripts

scripts/run_neighbor_sweep.py — Benchmark Sweep: fixed Strategy + Corpus, steps Neighbor Expansion (0,0)…(3,3) across documents (each level = separate Experiment).

---

18. Related Documents

Doc	Role
HLD.md	System context and component view
data-flow.md	Narrative pipelines
configuration.md	Env knobs
adr/	Decision records
CONTEXT.md	Language and Tab semantics