Mahdi Bazrafshan
6c9cebaaa0
Body:
Why:
- Insurance RAG pipeline needs Farsi-optimized embeddings
- Current local baseline (Nomic Embed) is English-only
Changes:
- 8 cloud model profiles with curl + python test commands
- 5 local model profiles with VRAM, serving platform notes
- Persian benchmark landscape (MIRACL-Fa, mMARCO, FaMTE, MTEB)
- Decision report with ranked options and recommendations
- Full comparison table (15 models × 10 columns)
Impact:
- Knowledge base for embedding model selection
- Top picks: BGE-M3 (local), Cohere embed-v4.0 (cloud)
- Critical: Nomic Embed must be replaced (English-only)
94 lines
4.1 KiB
Markdown
94 lines
4.1 KiB
Markdown
---
|
||
last_updated: 2026-07-20
|
||
tags: [embedding, comparison, overview, farsi, insurance, rag, big-picture]
|
||
source: import-knowledge
|
||
---
|
||
|
||
# Big Picture — All Embedding Models Comparison
|
||
|
||
## Context
|
||
|
||
- **Domain**: Insurance, Farsi-only
|
||
- **Pipeline**: LangGraph + Qdrant (migrating from LlamaIndex + ChromaDB)
|
||
- **Current**: Cloud = OpenAI text-embedding-3-large | Local = Nomic Embed (English-only ⚠️)
|
||
- **Hardware**: 1× RTX 3090 (24 GB VRAM)
|
||
|
||
## Executive Summary
|
||
|
||
After reviewing 15 embedding models across 5 cloud providers and 7 local options, the most promising candidates are:
|
||
1. **BGE-M3** (local) — triple-mode retrieval, Farsi, 1.1 GB VRAM
|
||
2. **Cohere embed-v4.0** (cloud) — strongest Farsi, 512K context, enterprise pricing
|
||
3. **gte-multilingual-base** (local) — lightest with Farsi, 610 MB VRAM
|
||
|
||
**Critical**: Nomic Embed (current local) is English-only and must be replaced.
|
||
|
||
---
|
||
|
||
## Cloud Models
|
||
|
||
| Model | Price/1M tokens | Context | Dims | Farsi | Source |
|
||
|-------|----------------|---------|------|-------|--------|
|
||
| text-embedding-3-large ⭐ | $0.13 | 8,191 | 3072 | Multilingual | [openai.md](../openai.md) |
|
||
| text-embedding-3-small | $0.02 | 8,191 | 1536 | Multilingual | [openai.md](../openai.md) |
|
||
| embed-v4.0 | Enterprise | 512,000 | 1536 | 100+ langs ✅ | [cohere.md](../cohere.md) |
|
||
| voyage-4-large | $0.12 | 32,000 | 1024 | Multilingual | [voyage.md](../voyage.md) |
|
||
| voyage-4 | $0.06 | 32,000 | 1024 | Multilingual | [voyage.md](../voyage.md) |
|
||
| Gemini Embedding | $0.15 | 8,192 | 768 | Multilingual | [google.md](../google.md) |
|
||
| text-embedding-004 | ~$0.10 | 8,192 | 768 | Multilingual | [google.md](../google.md) |
|
||
| text-multilingual-embedding-002 | ~$0.10 | 8,192 | 768 | Multilingual | [google.md](../google.md) |
|
||
| jina-embeddings-v3 | ~$0.02 | 8,192 | 1024 | 89 langs ✅ | [jina.md](../jina.md) |
|
||
|
||
⭐ = current cloud baseline
|
||
|
||
## Local Models
|
||
|
||
| Model | VRAM | Dims | Context | Farsi | ColBERT | Sparse | Serving | Source |
|
||
|-------|------|------|---------|-------|---------|--------|---------|--------|
|
||
| **BGE-M3** ⭐ | 1.1 GB | 1024 | 8,192 | ✅ 100+ | ✅ | ✅ | Ollama/TEI | [bge-m3.md](../bge-m3.md) |
|
||
| **gte-multilingual-base** | 610 MB | 768 | 8,192 | ✅ 70+ | ❌ | ❌ | TEI | [gte-multilingual.md](../gte-multilingual.md) |
|
||
| multilingual-E5-large | 1.1 GB | 1024 | 512 | ✅ 100 | ❌ | ❌ | Ollama/TEI | [mE5-large.md](../mE5-large.md) |
|
||
| gte-Qwen2-1.5B | 3 GB | 1536 | 8,192 | ⚠️ | ❌ | ❌ | TEI | — |
|
||
| e5-mistral-7B | 14.2 GB | 4096 | 512 | ⚠️ | ❌ | ❌ | vLLM/TEI | — |
|
||
| MiniLM-L12-v2 | 235 MB | 384 | 128 | ✅ 50+ | ❌ | ❌ | Ollama | — |
|
||
| **Nomic Embed** ⚠️ | 274 MB | 768 | 8,192 | ❌ EN only | ❌ | ❌ | Ollama | [nomic.md](../nomic.md) |
|
||
| jina-embeddings-v3 (local) | 1.1 GB | 1024 | 8,192 | ✅ 89 | ✅ | ❌ | TEI | [jina.md](../jina.md) |
|
||
|
||
⚠️ = English-only (must replace)
|
||
|
||
## Cost Comparison
|
||
|
||
### Cloud Models (monthly estimate at 10M tokens/month)
|
||
|
||
| Model | Cost/month | Notes |
|
||
|-------|-----------|-------|
|
||
| OpenAI text-embedding-3-large | $1.30 | Current baseline |
|
||
| OpenAI text-embedding-3-small | $0.20 | 6.5× cheaper, lower quality |
|
||
| Cohere embed-v4.0 | ~$4-5/hr | Enterprise only |
|
||
| Voyage voyage-4-large | $1.20 | 200M free tokens first |
|
||
| Google Gemini Embedding | $1.50 | Most expensive |
|
||
| Jina jina-embeddings-v3 | $0.20 | Cheapest |
|
||
|
||
### Local Models (monthly estimate)
|
||
|
||
All local models are free to run. The only cost is:
|
||
- **Electricity**: ~$5-10/month for RTX 3090 running 24/7
|
||
- **Hardware**: Already purchased (RTX 3090)
|
||
- **Ops time**: One-time setup, then minimal maintenance
|
||
|
||
## Recommendations
|
||
|
||
| Decision | Recommendation | Fallback |
|
||
|----------|---------------|----------|
|
||
| **Local model** | BGE-M3 | gte-multilingual-base |
|
||
| **Cloud model** | Cohere embed-v4.0 | Voyage voyage-4-large |
|
||
| **Serving platform** | Ollama | TEI |
|
||
| **Immediate action** | Replace Nomic with BGE-M3 | — |
|
||
|
||
## Pricing Sources
|
||
|
||
- OpenAI: https://openai.com/api/pricing/
|
||
- Cohere: https://cohere.com/pricing
|
||
- Voyage AI: https://docs.voyageai.com/docs/pricing
|
||
- Google Vertex: https://cloud.google.com/vertex-ai/generative-ai/pricing
|
||
- Jina AI: https://jina.ai/pricing/
|