ADR 0003: Dual Embedding Model Strategy

Status

Accepted

Context

The system operates in an environment with intermittent internet connectivity, including government-imposed blackouts lasting up to 90 days. The retrieval system must function fully offline with zero external API dependencies during these periods.

Documents and queries are bilingual (Persian/Farsi and English). Embedding models must handle both languages effectively.

Decision

Store two embeddings per chunk in Qdrant using named vectors:

Model	Use Case	Type
`text-embedding-3-large` (OpenAI)	Primary retrieval when internet available	Cloud
`nomic-embed-text`	Fallback during blackouts, fully local	On-prem

Qdrant Configuration

# Named vectors in Qdrant collection
collection_config = {
    "vectors": {
        "openai": {"size": 3072, "distance": "Cosine"},
        "nomic": {"size": 768, "distance": "Cosine"}
    }
}

Query-Time Behavior

Online: Query with OpenAI embeddings, retrieve using openai vector
Offline: Query with nomic embeddings, retrieve using nomic vector
A/B testing: Can compare retrieval quality between models in real-time

Consequences

Positive

Resilience — system works identically online and offline
No re-indexing — switch models at query time, not ingestion time
A/B testing — can measure retrieval quality difference between models
Graceful degradation — fallback is automatic when cloud APIs unavailable

Negative

2x storage — two vectors per chunk
2x ingestion time — must embed with both models during indexing
Complexity — query logic must select correct vector at runtime

Neutral

Requires monitoring to detect when cloud APIs become unavailable
May need to tune retrieval parameters (top_k, score threshold) per model

Generation Layer

During offline periods, answer generation uses Qwen 2.5 running locally. This provides:

Bilingual support (Persian/English)
Fully on-prem inference
No external API dependencies

Alternatives Considered

Single model (nomic only) — simpler but loses quality of text-embedding-3-large when online
Single model (multilingual-e5) — good bilingual support but not tested in production
Re-embed on switch — would require re-indexing entire corpus during blackout (impractical)

2.4 KiB Raw Blame History