Mahdi Bazrafshan
6c9cebaaa0
Body:
Why:
- Insurance RAG pipeline needs Farsi-optimized embeddings
- Current local baseline (Nomic Embed) is English-only
Changes:
- 8 cloud model profiles with curl + python test commands
- 5 local model profiles with VRAM, serving platform notes
- Persian benchmark landscape (MIRACL-Fa, mMARCO, FaMTE, MTEB)
- Decision report with ranked options and recommendations
- Full comparison table (15 models × 10 columns)
Impact:
- Knowledge base for embedding model selection
- Top picks: BGE-M3 (local), Cohere embed-v4.0 (cloud)
- Critical: Nomic Embed must be replaced (English-only)
78 lines
2.8 KiB
Markdown
78 lines
2.8 KiB
Markdown
---
|
|
last_updated: 2026-07-20
|
|
tags: [embedding, research, farsi, insurance, rag, index]
|
|
source: import-knowledge
|
|
---
|
|
|
|
# Embedding Models Research — Farsi Insurance RAG
|
|
|
|
**Date**: July 20, 2026
|
|
**Context**: LangGraph + Qdrant RAG pipeline for insurance domain. Only language is Farsi (Persian).
|
|
**Goal**: Find Farsi-strong embedding models that outperform OpenAI on top-K retrieval.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
1. **Read the big picture**: [overview/big-picture.md](overview/big-picture.md) — full comparison table + recommendations
|
|
2. **Read the decision report**: [decision-report/README.md](decision-report/README.md) — ranked options with pros/cons
|
|
3. **Test cloud models**: Each model file has curl + Python test commands
|
|
4. **Replace Nomic**: Deploy BGE-M3 on Ollama (see [bge-m3.md](bge-m3.md))
|
|
|
|
---
|
|
|
|
## Files
|
|
|
|
### Cloud Models
|
|
|
|
| Model | Price | File |
|
|
|-------|-------|------|
|
|
| OpenAI text-embedding-3-large | $0.13/1M | [openai.md](openai.md) |
|
|
| OpenAI text-embedding-3-small | $0.02/1M | [openai.md](openai.md) |
|
|
| Cohere embed-v4.0 | Enterprise | [cohere.md](cohere.md) |
|
|
| Voyage voyage-4-large | $0.12/1M | [voyage.md](voyage.md) |
|
|
| Voyage voyage-4 | $0.06/1M | [voyage.md](voyage.md) |
|
|
| Google Gemini Embedding | $0.15/1M | [google.md](google.md) |
|
|
| Google text-embedding-004 | ~$0.10/1M | [google.md](google.md) |
|
|
| Jina jina-embeddings-v3 | ~$0.02/1M | [jina.md](jina.md) |
|
|
|
|
### Local Models
|
|
|
|
| Model | VRAM | Farsi | File |
|
|
|-------|------|-------|------|
|
|
| **BGE-M3** ⭐ | 1.1 GB | ✅ | [bge-m3.md](bge-m3.md) |
|
|
| **gte-multilingual-base** | 610 MB | ✅ | [gte-multilingual.md](gte-multilingual.md) |
|
|
| multilingual-E5-large | 1.1 GB | ✅ | [mE5-large.md](mE5-large.md) |
|
|
| **Nomic Embed** ⚠️ | 274 MB | ❌ EN only | [nomic.md](nomic.md) |
|
|
|
|
### Cross-Cutting
|
|
|
|
| Topic | File |
|
|
|-------|------|
|
|
| Comparison table | [overview/big-picture.md](overview/big-picture.md) |
|
|
| Decision report | [decision-report/README.md](decision-report/README.md) |
|
|
| Persian benchmarks | [benchmarks/README.md](benchmarks/README.md) |
|
|
|
|
---
|
|
|
|
## Critical Finding
|
|
|
|
**Nomic Embed v1.5 (current local baseline) is English-only.** It does not support Farsi. Replace with BGE-M3 immediately.
|
|
|
|
## Top Recommendations
|
|
|
|
| Decision | Winner | Why |
|
|
|----------|--------|-----|
|
|
| Local model | BGE-M3 | Triple-mode (dense+sparse+ColBERT), Farsi, 1.1 GB VRAM, MIT |
|
|
| Cloud model | Cohere embed-v4.0 | Strongest Farsi, 512K context, enterprise pricing |
|
|
| Budget cloud | Voyage voyage-4-large | $0.12/1M, 200M free tokens |
|
|
| Serving platform | Ollama | Already deployed, BGE-M3 available |
|
|
|
|
## TODO
|
|
|
|
- [ ] Deploy BGE-M3 on Ollama
|
|
- [ ] Run 100-question eval set against all candidates
|
|
- [ ] Benchmark on MIRACL-Farsi
|
|
- [ ] Calculate monthly cloud cost at production volume
|
|
- [ ] Verify Cohere enterprise pricing feasibility
|