docs(research): add farsi embedding model research knowledge base

Body:
        Why:
        - Insurance RAG pipeline needs Farsi-optimized embeddings
        - Current local baseline (Nomic Embed) is English-only

        Changes:
        - 8 cloud model profiles with curl + python test commands
        - 5 local model profiles with VRAM, serving platform notes
        - Persian benchmark landscape (MIRACL-Fa, mMARCO, FaMTE, MTEB)
        - Decision report with ranked options and recommendations
        - Full comparison table (15 models × 10 columns)

        Impact:
        - Knowledge base for embedding model selection
        - Top picks: BGE-M3 (local), Cohere embed-v4.0 (cloud)
        - Critical: Nomic Embed must be replaced (English-only)
This commit is contained in:
2026-07-20 15:00:11 +03:30
parent 72d8a3ebe4
commit 6c9cebaaa0
13 changed files with 1296 additions and 0 deletions

79
embedding_models/nomic.md Normal file
View File

@@ -0,0 +1,79 @@
---
last_updated: 2026-07-20
tags: [embedding, nomic, local, english-only, farsi, critical-finding]
source: import-knowledge
---
# Nomic Embed v1.5 — Current Local Baseline
## ⚠️ CRITICAL: This model is English-only
**Nomic Embed v1.5 does NOT support Farsi.** Your current local baseline does not support the only language in your RAG pipeline. This means local retrieval quality for Farsi queries is effectively random or based on tokenization spillover, not genuine semantic understanding.
**Action required**: Replace with BGE-M3 or gte-multilingual-base immediately.
## Overview
- **Type**: Local
- **Provider**: Nomic AI
- **Parameters**: 137M
- **VRAM (fp16)**: ~274 MB
- **Embedding dimensions**: 768 (Matryoshka: 768, 512, 256, 128, 64)
- **Max input tokens**: 8,192
- **Language coverage**: **English only**
- **Serving platform**: sentence-transformers, Ollama, TEI
- **License**: Apache-2.0
- **Persian benchmark**: N/A — not applicable (English-only model).
- **Model card**: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5
## What It Has Going for It (If English Were the Language)
- Matryoshka Representation Learning — resizable embeddings (64 to 768 dims)
- 8K context window — long for a 137M parameter model
- Lightweight — only 274 MB VRAM
- Open-source, reproducible training
- Task prefixes required: `"search_query:"`, `"search_document:"`, `"clustering:"`
## What It Doesn't Have
-**No Farsi support** — English only
- ❌ No ColBERT/late interaction
- ❌ No sparse vectors
- ❌ No hybrid search capability
## Why We Used It
- Default/recommended Ollama embedding model
- Very lightweight (274 MB VRAM)
- Good English retrieval quality
- Easy to deploy on Ollama
## Replacement Candidates
| Model | VRAM | Farsi | Why Better |
|-------|------|-------|------------|
| **BGE-M3** ⭐ | 1.1 GB | ✅ 100+ langs | Triple-mode, 8K context |
| **gte-multilingual-base** | 610 MB | ✅ 70+ langs | Lightest with Farsi support |
| **multilingual-E5-large** | 1.1 GB | ✅ 100 langs | Proven baseline, but 512 context |
## Ollama Test (Current Setup)
```bash
# Pull and run (current)
ollama pull nomic-embed-text
ollama run nomic-embed-text
# Test with Farsi query (will produce poor results — model is English-only)
curl -s http://localhost:11434/api/embeddings \
-d '{
"model": "nomic-embed-text",
"prompt": "بیمه نامه شخص ثالث چیست؟"
}' | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'dims={len(d[\"embedding\"])}')"
```
## TODO
- [ ] Deploy BGE-M3 on Ollama as replacement
- [ ] Compare Nomic vs BGE-M3 on Farsi queries (expect dramatic improvement)
- [ ] Remove Nomic from production after validation
- [ ] Consider keeping Nomic for any future English-only tasks