chunking_strategies_evaluation/docs/architecture.md

Architecture Overview

System structure and design decisions for the RAG Chunking Benchmarker.

---

System Overview

┌─────────────────────────────────────────────────────────────────┐
│                    RAG Chunking Benchmarker                     │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐         │
│  │  Documents   │    │  Chunking   │    │ Benchmarking│         │
│  │   Module     │    │   Module    │    │   Module    │         │
│  └──────┬──────┘    └──────┬──────┘    └──────┬──────┘         │
│         │                  │                  │                 │
│         ▼                  ▼                  ▼                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                    Core Layer                           │   │
│  │  (config, models, exceptions, dependencies)            │   │
│  └─────────────────────────────────────────────────────────┘   │
│         │                  │                  │                 │
│         ▼                  ▼                  ▼                 │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐         │
│  │   SQLite    │    │   Qdrant    │    │   OpenAI    │         │
│  │  (metadata) │    │  (vectors)  │    │  (LLM/Emb)  │         │
│  └─────────────┘    └─────────────┘    └─────────────┘         │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

---

Module Structure

src/
├── core/                    # Foundation layer
│   ├── config.py           # Settings from .env
│   ├── models.py           # Shared Pydantic models
│   ├── exceptions.py       # Custom exceptions
│   └── dependencies.py     # Client singletons
│
├── storage/                 # Data persistence
│   ├── sqlite.py           # Structured data (docs, queries, experiments)
│   └── qdrant.py           # Vector embeddings
│
├── documents/               # Document handling
│   ├── parser.py           # .docx/.doc → DocumentTree + markdown
│   ├── service.py          # Upload, process, delete orchestration
│   ├── routes.py           # REST API endpoints
│   └── models.py           # Request/response schemas
│
├── chunking/                # Chunking strategies
│   ├── base.py             # Abstract base class + utilities
│   ├── embedding.py        # OpenAI embedding service
│   ├── service.py          # Orchestration (chunk → embed → store)
│   └── strategies/         # Strategy implementations
│       ├── fixed_size.py
│       ├── recursive.py
│       ├── semantic.py
│       ├── contextual_retrieval.py
│       └── semantic_parent_child.py
│
├── benchmarking/            # Evaluation system
│   ├── evaluation.py       # LLM-as-Judge scoring
│   ├── benchmark_service.py # Orchestration
│   ├── query_service.py    # Single query pipeline
│   ├── routes.py           # REST API endpoints
│   ├── models.py           # Request/response schemas
│   └── report.py           # HTML report generation
│
└── main.py                  # FastAPI app factory

---

Data Flow

Document Processing

Upload .docx → Parser → DocumentTree + Markdown → SQLite
                                                    ↓
Process → Strategy.chunk() → Chunks → Embed → Qdrant

Query Pipeline

Question → Embed → Qdrant Search → Top-K Chunks → LLM → Answer

Benchmark Pipeline

Questions JSON → For each question × strategy:
                    ├── Query Pipeline
                    └── LLM-as-Judge Evaluation
                         ↓
                    Aggregate Metrics → SQLite → HTML Report

---

Key Design Decisions

Decision	Rationale
One collection per strategy	Enables independent comparison
SQLite for metadata	Simple, no setup, sufficient scale
Qdrant for vectors	Purpose-built for vector search
OpenAI for embeddings	Industry standard, consistent results
LLM-as-Judge for evaluation	Automated, consistent scoring
Per-strategy failure isolation	One strategy failure doesn't break others
Text stored in Qdrant payload	Enables retrieval without extra DB lookups

---

Technology Stack

Component	Technology	Version
Framework	FastAPI	0.100+
ORM/DB	SQLite	built-in
Vector DB	Qdrant	1.18+
Embeddings	OpenAI text-embedding-3-small	-
LLM	OpenAI gpt-4o-mini	-
Tokenizer	tiktoken (cl100k_base)	-
PDF/DOCX	python-docx, LibreOffice	-

---

Configuration

All configuration via environment variables (.env):

# Database
DATABASE_URL=sqlite:///./data/chunking_benchmark.db
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=

# OpenAI
OPENAI_API_KEY=sk-...

# Chunking
CHUNK_SIZE=512
CHUNK_OVERLAP=50
SEMANTIC_THRESHOLD=0.5
SEMANTIC_MIN_CHUNK_SIZE=5

# Models
EMBEDDING_MODEL=text-embedding-3-small
LLM_MODEL=gpt-4o-mini

---

Error Handling

Exception	Handler	HTTP Status
ChunkingError	chunking_exception_handler	400
BenchmarkError	benchmark_exception_handler	400
QueryError	benchmark_exception_handler	400
DocumentProcessingError	chunking_exception_handler	400
QdrantError	chunking_exception_handler	400
EmbeddingError	chunking_exception_handler	400

---

Scaling Considerations

Component	Current	Production Recommendation
SQLite	Single file	PostgreSQL for concurrent access
Qdrant	Single instance	Qdrant Cloud or cluster
OpenAI	Direct API	Connection pooling, rate limiting
Workers	Single process	Celery for async processing
Caching	None	Redis for repeated queries