# High-Level Design (HLD)

RAG Chunking Benchmarker — system purpose, boundaries, components, and major flows.

**Audience:** architects, tech leads, new engineers  
**Companion:** [LLD](LLD.md) · [ADRs](adr/) · [CONTEXT.md](../CONTEXT.md)

---

## 1. Purpose

Compare five chunking Strategies on regulatory (and similar) documents under a controlled RAG pipeline: ingest → chunk → embed → retrieve → answer → LLM-as-Judge evaluation.

Operators use a Dashboard to upload documents, process Strategies, run Queries and Experiments, inspect retrieval, and choose a final Strategy Candidate on the Decision Board.

---

## 2. Goals and Non-Goals

### Goals

| Goal | Notes |
|------|--------|
| Fair Strategy comparison | Same document, questions, Corpus Embedding Model, and Neighbor Expansion knobs per Experiment |
| Auditable provenance | Experiments record Corpus (+ Boundary when semantic Strategies were processed) |
| Operator UX | Dashboard at `/app` replaces Swagger as primary UI |
| Embedding flexibility | Cloud (OpenAI) and Local (Ollama) via Embedding Model Registry |
| Failure isolation | One Strategy failing during process does not discard others |

### Non-Goals (v1)

| Out of scope | See |
|--------------|-----|
| Scanned PDF / OCR | [out-of-scope-v1.md](out-of-scope-v1.md), backlog |
| Multi-tenant auth | Single-operator local tool |
| Production-scale concurrency | SQLite + single process |
| Comparing Experiments across different Boundary/Corpus/Neighbor settings without warning | Compare UI warns; Decision Board filters by Corpus |

---

## 3. System Context

```
┌──────────────┐     HTTP/REST      ┌─────────────────────────────────────┐
│  Operator    │◄──────────────────►│  FastAPI App (RAG Chunking Benchmarker)│
│  (Browser)   │   Dashboard /app   │                                     │
└──────────────┘                    │  Documents · Chunking · Benchmarking │
                                    │  Admin · Static Dashboard           │
                                    └───────────┬─────────────────────────┘
                                                │
              ┌─────────────────────────────────┼─────────────────────────────────┐
              ▼                                 ▼                                 ▼
     ┌────────────────┐              ┌────────────────┐              ┌────────────────┐
     │     SQLite     │              │     Qdrant     │              │  LLM / Embed   │
     │  metadata,     │              │  Model Corpus  │              │  OpenAI Cloud  │
     │  Experiments,  │              │  vectors +     │              │  + Ollama Local│
     │  app_settings  │              │  chunk payload │              └────────────────┘
     └────────────────┘              └────────────────┘
```

**External dependencies**

| System | Role |
|--------|------|
| OpenAI API | Corpus/Boundary embeddings (cloud models); answer generation; LLM-as-Judge |
| Ollama | Local Embedding Model Provider (`nomic-embed-text-v2-moe`) |
| Qdrant | Vector search per Strategy × Corpus Embedding Model |
| LibreOffice (optional) | Legacy `.doc` → `.docx` conversion path in parser |

---

## 4. Logical Architecture

```
┌──────────────────────────────────────────────────────────────────────────┐
│                         Presentation Layer                               │
│  src/static/index.html — single-file React Dashboard (CDN, no build)     │
│  Tabs: Home · Documents · Query · Benchmarks · Decision · PDF · Admin    │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │ REST
┌────────────────────────────────▼─────────────────────────────────────────┐
│                         API Layer (FastAPI)                              │
│  /documents*   /queries*   /benchmarks*   /experiments*   /admin/*       │
│  /strategies   /app (StaticFiles)                                        │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │
┌────────────────────────────────▼─────────────────────────────────────────┐
│                       Domain Services                                    │
│  documents/service   chunking/service   query_service   benchmark_service│
│  evaluation          admin/service      report                           │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │
┌────────────────────────────────▼─────────────────────────────────────────┐
│                       Foundation                                         │
│  core/config · models · exceptions · dependencies                        │
│  chunking/base · embedding · embedding_models · strategies/*             │
│  documents/parser · pdf_parser                                           │
│  storage/sqlite · storage/qdrant                                         │
└──────────────────────────────────────────────────────────────────────────┘
```

---

## 5. Major Components

| Component | Responsibility | Key modules |
|-----------|----------------|-------------|
| **Dashboard** | Operator UI; persistent Tabs; Decision Board; Retrieval Inspect; PDF Workspace | `src/static/index.html` |
| **Documents** | Upload/parse Word & Text PDF; process Strategies; delete | `documents/*` |
| **Chunking** | Strategy registry; Boundary vs Corpus embeds; chunk → embed → Qdrant | `chunking/*` |
| **Query** | Embed question → search → Neighbor Expansion → LLM answer | `benchmarking/query_service.py` |
| **Benchmark** | Question × Strategy Experiment; aggregate metrics; HTML reports | `benchmarking/benchmark_service.py`, `report.py` |
| **Evaluation** | LLM-as-Judge (context relevance, answer similarity, faithfulness, hallucination) | `benchmarking/evaluation.py` |
| **Admin** | Health, Embedding Model defaults/thresholds, Qdrant CRUD, Chunk Preview, questions files, cost estimate | `admin/*` |
| **Storage** | SQLite metadata; Qdrant Model Corpus | `storage/*` |

---

## 6. Domain Concepts (summary)

Full glossary: [CONTEXT.md](../CONTEXT.md).

| Term | Meaning |
|------|---------|
| **Strategy** | One of: `fixed_size`, `recursive`, `semantic`, `contextual_retrieval`, `semantic_parent_child` |
| **Experiment** | Completed benchmark run for one document × N Strategies × M questions, with provenance |
| **Boundary Embedding Model** | Used only for Semantic Boundary Detection |
| **Corpus Embedding Model** | Embeds finished chunks and queries; scopes Qdrant collections |
| **Model Corpus** | Collections `{strategy}__{model_id}_collection` for one Corpus model |
| **Neighbor Expansion** | Query-time prev/next chunks for `fixed_size` only (`±P/N`) |
| **Expansion Tree** | Per-hit neighbor grouping for operator audit (vs flat LLM context) |
| **Strategy Candidate** | Comparable config for Decision Board (`fixed_size±N` or `semantic@Boundary`) |
| **Decision Board** | Two-stage selection over the fixed evaluation document set |

---

## 7. End-to-End Flows

### 7.1 Ingest and process

```
Upload (.docx/.doc/.pdf)
  → Text-layer Gate (PDF) / parse DOCX
  → Heading Reconstruction (PDF) → DocumentTree + markdown
  → SQLite documents

Process(strategies, boundary?, corpus?)
  → For each Strategy (isolated failures):
       Boundary embeds (semantic*) → Strategy.chunk()
       → Corpus embeds → Qdrant Model Corpus upsert
  → Update chunk_counts + last_* embedding provenance on document
```

### 7.2 Query

```
POST /queries
  → Resolve Corpus Embedding Model
  → Embed question (Corpus)
  → Qdrant top-k (document filter)
  → Neighbor Expansion if fixed_size (±P/N)
  → Build context (parent fetch for semantic_parent_child)
  → LLM answer → SQLite queries (+ expansion_tree)
```

### 7.3 Experiment

```
POST /benchmarks
  → Load questions JSON
  → For each question × Strategy: Query + evaluate_single
  → Aggregate metrics → SQLite experiments
  → Optional HTML report (managerial / technical)
```

### 7.4 Decision

```
Dashboard Decision Tab
  → Load Experiments filtered by Corpus Embedding Model
  → Discover Strategy Candidates (newest per doc × Candidate)
  → Stage 1: best fixed_size ±N vs best semantic@Boundary
  → Stage 2: compare winners + per-document breakdown
```

---

## 8. Data Architecture

### 8.1 Responsibility split

| Store | Owns |
|-------|------|
| **SQLite** | Documents (tree + markdown), queries, Experiments, `app_settings` (Embedding defaults, thresholds) |
| **Qdrant** | Vectors + chunk text payload (no vectors in SQLite) |
| **Filesystem** | Question JSON under `files/`; uploaded bytes are not retained after parse |

### 8.2 Model Corpus naming

```
{strategy}__{sanitized_model_id}_collection
```

Example: `fixed_size__text-embedding-3-large_collection`

Process, Query, and Experiment only read/write the Corpus Embedding Model in force. Other corpora remain untouched.

### 8.3 Provenance

Experiments store:

- `embedding_model_id` / `embedding_provider` — Corpus
- `boundary_embedding_model_id` — when semantic Strategies were on the process path for that document
- Neighbor Expansion levels in `benchmark_config`

Documents store `last_corpus_embedding_model_id` and `last_boundary_embedding_model_id` after process.

---

## 9. Deployment View

```
┌─────────────────────────────────────────┐
│  Host process                           │
│  uvicorn src.main:app                   │
│  └── FastAPI + Dashboard static         │
│                                         │
│  data/chunking_benchmark.db             │
│  .env (secrets + knobs)                 │
└───────────────┬─────────────────────────┘
                │
    ┌───────────┴───────────┐
    ▼                       ▼
 Qdrant (:6333)      OpenAI / Ollama
```

Typical local stack: app + Qdrant (Docker or native) + optional Ollama host. No build step for the Dashboard.

---

## 10. Cross-Cutting Concerns

| Concern | Approach |
|---------|----------|
| Config | `pydantic-settings` from `.env` ([configuration.md](configuration.md)) |
| Errors | Domain exceptions → HTTP 400 handlers (`ChunkingError`, `BenchmarkError`, `QueryError`, …) |
| Logging | Request middleware + structured stage logs in query/benchmark |
| CORS | Permissive (`*`) for local Dashboard development |
| Cost | Dry-run `/benchmarks` and `/admin/cost-estimate`; Local embeds = $0 embedding side |
| Extensibility | New Strategy = implement `ChunkingStrategy` + registry; new Embedding Model = registry entry |

---

## 11. Key Architectural Decisions

| ADR | Decision |
|-----|----------|
| 0001–0009 | CDN React Dashboard; `/admin` for ops gaps; top Tabs; dark amber theme |
| 0016–0017 | Text PDF via PyMuPDF; PDF Workspace via format-filtered shared sections |
| 0020 | Semantic Boundary Detection required (no fixed-count fallback) |
| 0021 | Always scope Qdrant collections by Embedding Model id |
| 0022 | Per-model `semantic_threshold` |
| 0023 | Neighbor Expansion + Expansion Tree for `fixed_size` |
| 0024 | Boundary vs Corpus Embedding Model roles |
| 0025 | Retrieval Inspect in Benchmarks Tab |
| 0026 | Decision Board two-stage Candidate selection |

---

## 12. Related Documents

| Doc | Role |
|-----|------|
| [LLD.md](LLD.md) | Module interfaces, schemas, algorithms |
| [architecture.md](architecture.md) | Older overview (prefer HLD for current shape) |
| [data-flow.md](data-flow.md) | Pipeline detail |
| [api-reference.md](api-reference.md) | HTTP contracts |
| [strategy-technical-details.md](strategy-technical-details.md) | Per-Strategy algorithms |
| [evaluation-metrics.md](evaluation-metrics.md) | Scoring definitions |
| [CONTEXT.md](../CONTEXT.md) | Ubiquitous language |