mba/chunking_strategies_evaluation

Go to file

Mahdi Bazrafshan 143351b96b feat(chunking): add base strategy interface, embedding service, and orchestration

Why:
- Need abstract base class for all chunking strategies
- Need OpenAI embedding service (text-embedding-3-small)
- Need orchestration to run chunk → embed → store pipeline

Changes:
- Base: ChunkingStrategy ABC, token counting, chunk ID generation, sentence splitting
- Embedding: batch embedding with 2048 text limit per call
- Service: strategy registry, single/multi-strategy runners with per-strategy failure isolation

2026-07-26 09:37:46 +03:30

docs: add domain glossary and architectural decision records

2026-07-21 18:51:46 +03:30

feat(chunking): add base strategy interface, embedding service, and orchestration

2026-07-26 09:37:46 +03:30

.env.example

build: add project configuration and environment template

2026-07-21 18:51:51 +03:30

.gitignore

Initial commit

2026-07-21 11:27:49 +03:30

CONTEXT.md

docs: add domain glossary and architectural decision records

2026-07-21 18:51:46 +03:30

LICENSE

Initial commit

2026-07-21 11:27:49 +03:30

pyproject.toml

build: add project configuration and environment template

2026-07-21 18:51:51 +03:30

README.md

Initial commit

2026-07-21 11:27:49 +03:30

README.md

chunking_strategies_evaluation

This repo is in order to reach multiple chunking strategies and evaulate them