5.8 KiB
Context — RAG Chunking Benchmarker Admin Dashboard
Language
Dashboard:
A single-file React app (CDN-loaded, no build step) served by FastAPI at /app. Replaces Swagger as the primary UI for operating the RAG benchmarking platform.
Avoid: Admin panel, web UI, frontend
Strategy: One of the five chunking algorithms: fixed_size, recursive, semantic, contextual_retrieval, semantic_parent_child. Avoid: Method, approach, technique
Experiment: A completed benchmark run — one document, N strategies, M questions, with per-question and aggregate metrics. Avoid: Run, trial, benchmark
Chunk Preview: The ability to visualize what a specific strategy produces for a given document — the raw text blocks, their sizes, and hierarchy. Avoid: Chunk inspection, chunk view
Tab: A persistent top-level navigation section of the Dashboard (Home, Documents, Query, Benchmarks, Admin, PDF). Tabs stay mounted when switching — state survives. Avoid: Page, route, view
PDF Workspace:
The PDF Tab’s end-to-end surface for Text PDF work: upload, process Strategies, Chunk Preview, query, run Experiments, and open reports — scoped to PDF documents only. Documents / Query / Benchmarks Tabs list Word documents only (.doc/.docx). Inside the Tab, features are collapsible accordion sections (Upload & Process open by default); Admin-only ops (health, Qdrant CRUD) stay on Admin. Built from shared Dashboard section components with a format filter — not a forked UI.
Avoid: PDF page, PDF mode, PDF dashboard
Text PDF: A PDF with a real, selectable text layer that can be extracted without OCR. In scope for PDF ingestion v1. Avoid: Digital PDF, native PDF, searchable PDF (ambiguous in ops talk)
Scanned PDF: A PDF whose pages are images (or have a useless/empty text layer) and need OCR before chunking. Out of scope for PDF ingestion v1. Avoid: Image PDF, photo PDF, OCR PDF
Heading Reconstruction:
Inferring section/article boundaries from PDF signals in priority order — outline bookmarks, then font size/weight, then Farsi/English text heuristics — and emitting markdown # / ## for Strategies. Later signals fill gaps; they do not override outline titles when an outline is present.
Avoid: Heading detection, structure recovery, outline parsing (too narrow)
Text-layer Gate: Upload-time check that a PDF has enough extractable text to count as a Text PDF; failure rejects the upload. Scanned PDFs never enter the benchmark corpus. Avoid: OCR check, PDF validation, empty-page filter
Table Flattening: Turning table cells into sequential plain-text blocks in reading order (same contract for DOCX and Text PDF). Strategies never receive grid/markdown-table structure in v1. Avoid: Table extraction, table parsing, structured tables
Architecture Decisions
| # | Decision | Status |
|---|---|---|
| ADR-0001 | Single-file React via CDN (no build step), served by FastAPI | Approved |
| ADR-0002 | New /admin/ router for dashboard-specific backend ops (health, Qdrant CRUD, chunk preview, questions, cost) |
Approved |
| ADR-0003 | Top-tab navigation (Home, Documents, Query, Benchmarks, Admin) — not sidebar | Approved |
| ADR-0004 | Babel-in-browser JSX: single index.html with inline <script type="text/babel">, React+ReactDOM+Babel from CDN. Zero build tooling, one file to edit. | Approved |
| ADR-0005 | Persistent tabs with useState on root App. No routing library, no state library — 5-7 shared state values passed as props. Tab components stay mounted, state survives tab switches. | Approved |
| ADR-0006 | Dashboard calls existing REST endpoints for documents/queries/benchmarks. New /admin/* router ONLY fills gaps: Qdrant CRUD, health, chunk preview, questions dataset, cost estimation. No endpoint duplication. | Approved |
| ADR-0007 | Admin router v1 endpoints: /admin/health, /admin/qdrant/collections (CRUD + wipe), /admin/chunks/{doc_id}, /admin/questions (CRUD + upload), /admin/cost-estimate. Config/SQLite/request-log deferred to v1.1. | Approved |
| ADR-0008 | Custom dark theme: bg #111113/#1a1a1e/#222228, border #2a2a30, text #e4e4e7/#fafafa/#71717a, accent #eab308 (amber). Inter font from Google Fonts CDN. Continuity with existing amber/green report palette. | Approved |
| ADR-0009 | File structure: dashboard HTML in src/static/index.html, admin backend in src/admin/ (routes.py, service.py). FastAPI mounts /app -> src/static via StaticFiles(html=True). | Approved |
| ADR-0010 | Cant-do-yet implementation order: Admin panels first (chunk preview, questions mgmt, cost estimator), then cross-tab UX (experiment detail, query history, doc preview, progress), then cross-cutting (error handling). | Approved |
| ADR-0011 | Admin tab uses collapsible sections (accordion pattern). Each panel is a card with a clickable header toggling display. Health starts expanded, others collapsed by default. | Approved |
| ADR-0012 | Chunk Preview: table with expandable rows. Select document + strategy, click Load. Rows show index + 80-char text preview + token/char counts. Click to expand full text. Parent column hidden by default. | Approved |
| ADR-0013 | Questions Management: file list table + upload button + expandable row detail (id, question, category, difficulty, expected answer) + delete + "Use This File" shortcut to set Benchmarks tab path and switch tabs. Auto-refresh after mutations. | Approved |
| ADR-0014 | Cost Estimator: two number inputs (questions, strategies), Estimate button, result card with total cost in amber, token estimate, 3 breakdown cards (embedding/queries/evaluation). Simple numbers, no tables. | Approved |
| ADR-0016 | Text PDF extraction via PyMuPDF + Heading Reconstruction; Scanned PDFs hard-rejected (Text-layer Gate). See docs/adr/0016-*.md | Approved |
| ADR-0017 | PDF Workspace via shared Dashboard sections + format filter (not a forked UI). See docs/adr/0017-*.md | Approved |