chunking_strategies_evaluation

mba/chunking_strategies_evaluation

Fork 0

Commit Graph

Author	SHA1	Message	Date
Mahdi Bazrafshan	ef8bab5962	feat(benchmarking): add evaluation, benchmark service, and report Why: - Need LLM-as-Judge evaluation for automated scoring - Need benchmark orchestration to run questions × strategies - Need HTML report generation with two views (managerial/technical) Changes: - evaluation.py: LLM-as-Judge scoring on 4 metrics (context, similarity, faithfulness, hallucination) - benchmark_service.py: Orchestration with per-strategy failure isolation - report.py: Dual-view HTML reports with dark mode, charts, and strategy cards	2026-07-27 14:14:40 +03:30
Mahdi Bazrafshan	496a58c62a	feat(benchmarking): add query pipeline with logging and tracing Why: - Need query pipeline to ask questions against chunked documents - Need comprehensive logging for debugging Changes: - Query service: embed question → vector search → LLM answer - Query routes: POST /queries, GET /queries/{id} - Query models: QueryRequest, QueryResponse, RetrievedChunk - Added QueryError exception class - Added STEP 1-6 logging for full traceability	2026-07-26 12:06:04 +03:30

Author

SHA1

Message

Date

Mahdi Bazrafshan

ef8bab5962

feat(benchmarking): add evaluation, benchmark service, and report

Why:
- Need LLM-as-Judge evaluation for automated scoring
- Need benchmark orchestration to run questions × strategies
- Need HTML report generation with two views (managerial/technical)

Changes:
- evaluation.py: LLM-as-Judge scoring on 4 metrics (context, similarity, faithfulness, hallucination)
- benchmark_service.py: Orchestration with per-strategy failure isolation
- report.py: Dual-view HTML reports with dark mode, charts, and strategy cards

2026-07-27 14:14:40 +03:30

Mahdi Bazrafshan

496a58c62a

feat(benchmarking): add query pipeline with logging and tracing

Why:
- Need query pipeline to ask questions against chunked documents
- Need comprehensive logging for debugging

Changes:
- Query service: embed question → vector search → LLM answer
- Query routes: POST /queries, GET /queries/{id}
- Query models: QueryRequest, QueryResponse, RetrievedChunk
- Added QueryError exception class
- Added STEP 1-6 logging for full traceability

2026-07-26 12:06:04 +03:30

2 Commits