docs(rag): add domain model and original research input
This commit is contained in:
66
chunking/docs/adr/0003-dual-embedding-strategy.md
Normal file
66
chunking/docs/adr/0003-dual-embedding-strategy.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# ADR 0003: Dual Embedding Model Strategy
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The system operates in an environment with intermittent internet connectivity, including government-imposed blackouts lasting up to 90 days. The retrieval system must function fully offline with zero external API dependencies during these periods.
|
||||
|
||||
Documents and queries are bilingual (Persian/Farsi and English). Embedding models must handle both languages effectively.
|
||||
|
||||
## Decision
|
||||
|
||||
Store **two embeddings per chunk** in Qdrant using named vectors:
|
||||
|
||||
| Model | Use Case | Type |
|
||||
|-------|----------|------|
|
||||
| `text-embedding-3-large` (OpenAI) | Primary retrieval when internet available | Cloud |
|
||||
| `nomic-embed-text` | Fallback during blackouts, fully local | On-prem |
|
||||
|
||||
### Qdrant Configuration
|
||||
```python
|
||||
# Named vectors in Qdrant collection
|
||||
collection_config = {
|
||||
"vectors": {
|
||||
"openai": {"size": 3072, "distance": "Cosine"},
|
||||
"nomic": {"size": 768, "distance": "Cosine"}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Query-Time Behavior
|
||||
- **Online:** Query with OpenAI embeddings, retrieve using `openai` vector
|
||||
- **Offline:** Query with nomic embeddings, retrieve using `nomic` vector
|
||||
- **A/B testing:** Can compare retrieval quality between models in real-time
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- **Resilience** — system works identically online and offline
|
||||
- **No re-indexing** — switch models at query time, not ingestion time
|
||||
- **A/B testing** — can measure retrieval quality difference between models
|
||||
- **Graceful degradation** — fallback is automatic when cloud APIs unavailable
|
||||
|
||||
### Negative
|
||||
- **2x storage** — two vectors per chunk
|
||||
- **2x ingestion time** — must embed with both models during indexing
|
||||
- **Complexity** — query logic must select correct vector at runtime
|
||||
|
||||
### Neutral
|
||||
- Requires monitoring to detect when cloud APIs become unavailable
|
||||
- May need to tune retrieval parameters (top_k, score threshold) per model
|
||||
|
||||
## Generation Layer
|
||||
|
||||
During offline periods, answer generation uses **Qwen 2.5** running locally. This provides:
|
||||
- Bilingual support (Persian/English)
|
||||
- Fully on-prem inference
|
||||
- No external API dependencies
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
1. **Single model (nomic only)** — simpler but loses quality of `text-embedding-3-large` when online
|
||||
2. **Single model (multilingual-e5)** — good bilingual support but not tested in production
|
||||
3. **Re-embed on switch** — would require re-indexing entire corpus during blackout (impractical)
|
||||
Reference in New Issue
Block a user