mba/chunking_strategies_evaluation

Go to file

Mahdi Bazrafshan 16c918538b fix(chunking): require semantic boundary detection

Why:
- Semantic strategies were silently falling back to fixed-count grouping when boundary embeds were missing.

Changes:
- Fail hard without aligned unit embeddings; orchestrator supplies Boundary embeds; Farsi-aware sentence split with line/paragraph fallback.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-08-10 14:12:45 +03:30

docs: capture pdf domain language and deferred ingestion backlog

2026-08-02 16:59:42 +03:30

feat(dashboard): add pdf workspace via shared format filter

2026-08-02 16:59:42 +03:30

docs(data): add 21-question evaluation dataset for benchmarking

2026-07-26 13:48:36 +03:30

fix(chunking): require semantic boundary detection

2026-08-10 14:12:45 +03:30

.env.example

feat(chunking): add embedding model registry and ollama config

2026-08-10 14:12:36 +03:30

.gitignore

Initial commit

2026-07-21 11:27:49 +03:30

CONTEXT.md

docs: capture pdf domain language and deferred ingestion backlog

2026-08-02 16:59:42 +03:30

LICENSE

Initial commit

2026-07-21 11:27:49 +03:30

pyproject.toml

build(deps): add pymupdf for text pdf extraction

2026-08-02 16:59:42 +03:30

README.md

Initial commit

2026-07-21 11:27:49 +03:30

README.md

chunking_strategies_evaluation

This repo is in order to reach multiple chunking strategies and evaulate them