chunking_strategies_evaluation/docs/chunking_strategies.md

Chunking Strategies

This document explains the 5 chunking strategies implemented in the RAG Chunking Strategy Benchmarking Framework.

---

1. FIXED_SIZE (Baseline)

How it works: Splits text into chunks of N tokens with M token overlap.

Algorithm

1. Encode full markdown into tokens
2. Take first chunk_size tokens as chunk 1
3. Slide forward by chunk_size - overlap tokens
4. Repeat until end of text

Pros & Cons

• Pros: Simple, predictable, fast, no dependencies
• Cons: Ignores meaning — can split mid-sentence, mid-word, or across topics

Configuration

Uses chunk_size and chunk_overlap from settings.

---

2. RECURSIVE (Cascade Splitting)

How it works: Tries to split on meaningful boundaries first, falling back to less meaningful ones.

Separator Cascade (in order)

1. Markdown headers (#, ##, ###)
2. Double newline (\n\n) — paragraph breaks
3. Single newline (\n) — line breaks
4. Sentence endings (. ! ? + space)
5. Space (word-level, last resort)

Algorithm

1. Try splitting by highest-priority separator
2. If parts are still too big, recurse with next separator
3. Merge small parts back up to target size

Pros & Cons

• Pros: Respects document structure, produces natural chunks
• Cons: Still rule-based, no semantic understanding

---

3. SEMANTIC (Similarity-Based)

How it works: Groups sentences by meaning — when similarity drops, it starts a new chunk.

Algorithm

1. Split markdown into sentences
2. Embed each sentence via OpenAI (text-embedding-3-small)
3. Compute cosine similarity between adjacent sentences
4. When similarity < semantic_threshold, create chunk boundary
5. Enforce minimum chunk size (semantic_min_chunk_size sentences)

Pros & Cons

• Pros: Respects topic changes, produces coherent chunks
• Cons: Requires embeddings at chunk-time (API calls), slower, costs money

Configuration

• semantic_threshold (default 0.5)
• semantic_min_chunk_size (default 5)

---

4. CONTEXTUAL_RETRIEVAL (LLM-Enriched)

How it works: Based on Anthropic's research — prepends a short context summary to each chunk before embedding.

Algorithm

1. Split text using fixed-size token splitting (same as #1)
2. For each chunk, send surrounding text + chunk to LLM
3. LLM generates a 1-2 sentence context prefix
4. Enriched chunk = context prefix + original text

Example Output

This section discusses insurance claim deadlines for property damage...

[Original chunk text about specific deadlines...]

Pros & Cons

• Pros: Improved retrieval by 49% in Anthropic's benchmarks
• Cons: Expensive (1 LLM call per chunk), slowest strategy

Configuration

Uses llm_model (gpt-4o-mini) for context generation.

---

5. SEMANTIC_PARENT_CHILD (Hierarchical)

How it works: Groups paragraphs into semantic clusters. Each cluster is a parent; each paragraph is a child.

Algorithm

1. Split markdown into paragraphs
2. Embed each paragraph
3. Cluster consecutive paragraphs by similarity (threshold-based)
4. Each cluster = parent chunk (full cluster text)
5. Each paragraph = child chunk (linked to parent)

Query-time Behavior

• Search finds child paragraph via vector match
• Return full parent cluster as context to LLM

Pros & Cons

• Pros: Rich context, no headings needed, works on flat documents
• Cons: More storage (both parent + child vectors), complex retrieval

Configuration

Uses semantic_threshold for clustering.

---

Summary Table

Strategy	Split Method	Requires LLM	Requires Embeddings	Speed	Cost
fixed_size	Token count	No	No	Fast	Free
recursive	Separator cascade	No	No	Fast	Free
semantic	Similarity threshold	No	Yes (at chunk time)	Medium	Low
contextual_retrieval	Fixed-size + LLM context	Yes (per chunk)	No	Slow	High
semantic_parent_child	Similarity clustering	No	Yes (at chunk time)	Medium	Low