Mahdi Bazrafshan
e1ca9f4127
Why: - Manager deliverable for human evaluation across 10 benchmark documents. Changes: - Report markdown, scored CSV, chart generator, and SVG assets - Linked from docs/README.md Co-authored-by: Cursor <cursoragent@cursor.com>
109 lines
4.7 KiB
Markdown
109 lines
4.7 KiB
Markdown
# Documentation Index
|
|
|
|
Complete documentation for the RAG Chunking Benchmarker.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
1. **New to the project?** Start with [HLD](HLD.md) (system design) then [LLD](LLD.md) (module detail)
|
|
2. **Domain language?** Read [CONTEXT.md](../CONTEXT.md)
|
|
3. **Want to understand strategies?** Read [Strategy Technical Details](strategy-technical-details.md)
|
|
4. **Need to use the API?** Check [API Reference](api-reference.md)
|
|
5. **Configuring the system?** See [Configuration Guide](configuration.md)
|
|
6. **Understanding results?** Read [Evaluation Metrics](evaluation-metrics.md)
|
|
|
|
---
|
|
|
|
## Documentation Files
|
|
|
|
| File | Purpose | Audience |
|
|
|------|---------|----------|
|
|
| [HLD.md](HLD.md) | High-level design: context, components, flows | Architects, new team members |
|
|
| [LLD.md](LLD.md) | Low-level design: schemas, algorithms, APIs | Implementers |
|
|
| [architecture.md](architecture.md) | Legacy overview (prefer HLD) | New team members |
|
|
| [strategy-technical-details.md](strategy-technical-details.md) | Deep dive into each strategy | Engineers |
|
|
| [api-reference.md](api-reference.md) | All endpoints documented | Developers |
|
|
| [configuration.md](configuration.md) | Settings and environment variables | DevOps |
|
|
| [evaluation-metrics.md](evaluation-metrics.md) | How scoring works | Data scientists |
|
|
| [data-flow.md](data-flow.md) | How data moves through the system | Engineers |
|
|
| [chunking_strategies.md](chunking_strategies.md) | High-level strategy overview | Everyone |
|
|
| [final-chunking-strategy-decision.md](final-chunking-strategy-decision.md) | Final Strategy family decision (`fixed_size`) + charts | Managers, operators |
|
|
| [human-eval-fixed-size-plus3-report.md](human-eval-fixed-size-plus3-report.md) | Human evaluation of fixed_size ±3 (sample + charts) | Managers, operators |
|
|
| [adr/](adr/) | Architectural Decision Records | Everyone |
|
|
| [phases.md](phases.md) | Implementation phases | Project managers |
|
|
| [tasks.md](tasks.md) | Task tracking | Developers |
|
|
|
|
---
|
|
|
|
## Architecture at a Glance
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ RAG Chunking Benchmarker │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ Documents → Parser → Chunking → Embedding → Qdrant │
|
|
│ │
|
|
│ Questions → Query Pipeline → LLM → Answers │
|
|
│ │
|
|
│ Answers → Evaluation → Metrics → Reports │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Key Concepts
|
|
|
|
### Chunking Strategies
|
|
|
|
| Strategy | Splitting | Best For |
|
|
|----------|-----------|----------|
|
|
| fixed_size | Token count | Baseline comparison |
|
|
| recursive | Structural boundaries | Structured documents |
|
|
| semantic | Similarity thresholds | Topic shifts |
|
|
| contextual_retrieval | LLM-enriched tokens | Retrieval quality |
|
|
| semantic_parent_child | Paragraph clusters | Context needed |
|
|
|
|
### Evaluation Metrics
|
|
|
|
| Metric | What It Measures |
|
|
|--------|------------------|
|
|
| Context Relevance | Did we find the right information? |
|
|
| Answer Similarity | Did we produce the right answer? |
|
|
| Faithfulness | Is the answer trustworthy? |
|
|
| Hallucination | Did we invent information? |
|
|
|
|
---
|
|
|
|
## API Endpoints
|
|
|
|
| Endpoint | Method | Purpose |
|
|
|----------|--------|---------|
|
|
| `/documents` | GET/POST | Manage documents |
|
|
| `/documents/{id}/process` | POST | Run chunking strategies |
|
|
| `/queries` | POST | Ask questions |
|
|
| `/benchmarks` | POST | Run comparisons |
|
|
| `/benchmarks/{id}/report` | GET | View results |
|
|
| `/experiments` | GET | List experiments |
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
All settings via `.env` file. See [Configuration Guide](configuration.md) for details.
|
|
|
|
Key settings:
|
|
- `OPENAI_API_KEY` - Required for embeddings and LLM
|
|
- `CHUNK_SIZE` - Target tokens per chunk (default: 512)
|
|
- `SEMANTIC_THRESHOLD` - Similarity threshold (default: 0.5)
|
|
|
|
---
|
|
|
|
## Further Reading
|
|
|
|
- [Phases](phases.md) - Implementation roadmap
|
|
- [Tasks](tasks.md) - Task tracking
|
|
- [ADRs](adr/) - Architectural Decision Records
|