docs(research): add farsi embedding model research knowledge base
Body:
Why:
- Insurance RAG pipeline needs Farsi-optimized embeddings
- Current local baseline (Nomic Embed) is English-only
Changes:
- 8 cloud model profiles with curl + python test commands
- 5 local model profiles with VRAM, serving platform notes
- Persian benchmark landscape (MIRACL-Fa, mMARCO, FaMTE, MTEB)
- Decision report with ranked options and recommendations
- Full comparison table (15 models × 10 columns)
Impact:
- Knowledge base for embedding model selection
- Top picks: BGE-M3 (local), Cohere embed-v4.0 (cloud)
- Critical: Nomic Embed must be replaced (English-only)
This commit is contained in:
127
embedding_models/bge-m3.md
Normal file
127
embedding_models/bge-m3.md
Normal file
@@ -0,0 +1,127 @@
|
||||
---
|
||||
last_updated: 2026-07-20
|
||||
tags: [embedding, bge-m3, local, farsi, insurance, rag, triple-mode, recommended]
|
||||
source: import-knowledge
|
||||
---
|
||||
|
||||
# BAAI/bge-m3 — Top Local Recommendation ⭐
|
||||
|
||||
## Overview
|
||||
|
||||
- **Type**: Local
|
||||
- **Provider**: BAAI (Beijing Academy of AI)
|
||||
- **Parameters**: ~568M
|
||||
- **VRAM (fp16)**: ~1.1 GB
|
||||
- **Embedding dimensions**: 1024
|
||||
- **Max input tokens**: 8,192
|
||||
- **Language coverage**: 100+ languages including Farsi
|
||||
- **ColBERT/late interaction**: YES (native multi-vector retrieval)
|
||||
- **Sparse vectors**: YES (native lexical weight retrieval)
|
||||
- **Serving platform**: FlagEmbedding library, sentence-transformers, TEI, or Ollama (available)
|
||||
- **License**: MIT
|
||||
- **Model card**: https://huggingface.co/BAAI/bge-m3
|
||||
|
||||
## Why It's #1
|
||||
|
||||
**Triple-mode retrieval** (dense + sparse + ColBERT) from a single model. This is uniquely suited to LangGraph + Qdrant hybrid search:
|
||||
- **Dense**: Standard cosine similarity embeddings
|
||||
- **Sparse**: BM25-like lexical retrieval without a separate sparse encoder
|
||||
- **ColBERT**: Token-level late interaction for finer-grained matching
|
||||
|
||||
Only 1.1 GB VRAM on a 24 GB RTX 3090 — leaves room for other workloads. 8K context handles long insurance policy chunks.
|
||||
|
||||
## Pros
|
||||
|
||||
- Supports Farsi (100+ languages)
|
||||
- Triple-mode: dense + sparse + ColBERT multi-vector
|
||||
- Hybrid search ready — no separate sparse encoder needed
|
||||
- 8,192 token context (can handle long insurance policy chunks)
|
||||
- Only 1.1 GB VRAM — leaves headroom for other workloads
|
||||
- MIT license — no commercial restrictions
|
||||
- Available on Ollama, TEI, sentence-transformers
|
||||
- Active development by BAAI (Beijing Academy of AI)
|
||||
|
||||
## Cons
|
||||
|
||||
- No published Farsi-specific retrieval benchmark scores
|
||||
- Slightly less community adoption than multilingual-E5
|
||||
- Triple-mode adds architectural complexity if only dense mode is used
|
||||
|
||||
## API Test — Ollama
|
||||
|
||||
```bash
|
||||
# Pull model
|
||||
ollama pull bge-m3
|
||||
|
||||
# Test with Farsi insurance query
|
||||
curl -s http://localhost:11434/api/embeddings -d '{
|
||||
"model": "bge-m3",
|
||||
"prompt": "بیمه نامه شخص ثالث چیست؟"
|
||||
}' | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'dims={len(d["embedding"])}')"
|
||||
```
|
||||
|
||||
## API Test — Python (sentence-transformers)
|
||||
|
||||
```python
|
||||
from sentence_transformers import SentenceTransformer
|
||||
|
||||
model = SentenceTransformer("BAAI/bge-m3")
|
||||
|
||||
# Farsi insurance query
|
||||
query = "شرایط لغو بیمه نامه چیست؟"
|
||||
embedding = model.encode(query)
|
||||
print(f"Dimensions: {len(embedding)}, Type: {type(embedding)}")
|
||||
# Expected: Dimensions: 1024
|
||||
```
|
||||
|
||||
## API Test — Python (FlagEmbedding for triple-mode)
|
||||
|
||||
```python
|
||||
from FlagEmbedding import BGEM3FlagModel
|
||||
|
||||
model = BGEM3FlagModel("BAAI/bge-m3", use_fp16=True)
|
||||
|
||||
# Encode with all three modes
|
||||
sentences = ["بیمه نامه شخص ثالث چیست؟", "شرایط بیمه آتشسوزی"]
|
||||
output = model.encode(sentences, return_dense=True, return_sparse=True, return_colbert_vecs=True)
|
||||
|
||||
print(f"Dense dims: {output['dense_vecs'].shape}")
|
||||
print(f"Sparse tokens: {len(output['lexical_weights'][0])}")
|
||||
print(f"ColBERT vecs: {len(output['colbert_vecs'][0])}")
|
||||
```
|
||||
|
||||
## Serving Platform Notes
|
||||
|
||||
| Platform | Status | Notes |
|
||||
|----------|--------|-------|
|
||||
| **Ollama** ✅ | Available | Easiest. `ollama pull bge-m3` |
|
||||
| **TEI** ✅ | Available | Best throughput for production |
|
||||
| **sentence-transformers** ✅ | Available | Direct Python integration |
|
||||
| **vLLM** | Not natively | vLLM is for LLM serving, not embeddings |
|
||||
|
||||
## Qdrant Integration
|
||||
|
||||
BGE-M3's sparse vectors work natively with Qdrant's sparse vector support:
|
||||
|
||||
```python
|
||||
from qdrant_client import QdrantClient
|
||||
from qdrant_client.models import VectorParams, Distance, SparseVectorParams
|
||||
|
||||
client = QdrantClient("localhost", port=6333)
|
||||
|
||||
# Create collection with both dense and sparse vectors
|
||||
client.create_collection(
|
||||
collection_name="insurance_docs",
|
||||
vectors_config=VectorParams(size=1024, distance=Distance.COSINE),
|
||||
sparse_vectors_config={
|
||||
"sparse": SparseVectorParams()
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
- [ ] Benchmark on MIRACL-Farsi subset
|
||||
- [ ] Compare dense-only vs triple-mode retrieval quality
|
||||
- [ ] Test with Qdrant sparse vector indexing
|
||||
- [ ] Measure inference latency on RTX 3090
|
||||
Reference in New Issue
Block a user