Why:
- Semantic strategies were silently falling back to fixed-count grouping when boundary embeds were missing.
Changes:
- Fail hard without aligned unit embeddings; orchestrator supplies Boundary embeds; Farsi-aware sentence split with line/paragraph fallback.
Co-authored-by: Cursor <cursoragent@cursor.com>
Why:
- Need abstract base class for all chunking strategies
- Need OpenAI embedding service (text-embedding-3-small)
- Need orchestration to run chunk → embed → store pipeline
Changes:
- Base: ChunkingStrategy ABC, token counting, chunk ID generation, sentence splitting
- Embedding: batch embedding with 2048 text limit per call
- Service: strategy registry, single/multi-strategy runners with per-strategy failure isolation
