Research/embedding_models/bge-m3.md

last_updated, tags, source

last_updated	tags	source
2026-07-20	embedding bge-m3 local farsi insurance rag triple-mode recommended	import-knowledge

BAAI/bge-m3 — Top Local Recommendation ⭐

Overview

• Type: Local
• Provider: BAAI (Beijing Academy of AI)
• Parameters: ~568M
• VRAM (fp16): ~1.1 GB
• Embedding dimensions: 1024
• Max input tokens: 8,192
• Language coverage: 100+ languages including Farsi
• ColBERT/late interaction: YES (native multi-vector retrieval)
• Sparse vectors: YES (native lexical weight retrieval)
• Serving platform: FlagEmbedding library, sentence-transformers, TEI, or Ollama (available)
• License: MIT
• Model card: https://huggingface.co/BAAI/bge-m3

Why It's #1

Triple-mode retrieval (dense + sparse + ColBERT) from a single model. This is uniquely suited to LangGraph + Qdrant hybrid search:

• Dense: Standard cosine similarity embeddings
• Sparse: BM25-like lexical retrieval without a separate sparse encoder
• ColBERT: Token-level late interaction for finer-grained matching

Only 1.1 GB VRAM on a 24 GB RTX 3090 — leaves room for other workloads. 8K context handles long insurance policy chunks.

Pros

• Supports Farsi (100+ languages)
• Triple-mode: dense + sparse + ColBERT multi-vector
• Hybrid search ready — no separate sparse encoder needed
• 8,192 token context (can handle long insurance policy chunks)
• Only 1.1 GB VRAM — leaves headroom for other workloads
• MIT license — no commercial restrictions
• Available on Ollama, TEI, sentence-transformers
• Active development by BAAI (Beijing Academy of AI)

Cons

• No published Farsi-specific retrieval benchmark scores
• Slightly less community adoption than multilingual-E5
• Triple-mode adds architectural complexity if only dense mode is used

API Test — Ollama

# Pull model
ollama pull bge-m3

# Test with Farsi insurance query
curl -s http://localhost:11434/api/embeddings   -d '{
    "model": "bge-m3",
    "prompt": "بیمه نامه شخص ثالث چیست؟"
  }' | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'dims={len(d["embedding"])}')"

API Test — Python (sentence-transformers)

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-m3")

# Farsi insurance query
query = "شرایط لغو بیمه نامه چیست؟"
embedding = model.encode(query)
print(f"Dimensions: {len(embedding)}, Type: {type(embedding)}")
# Expected: Dimensions: 1024

API Test — Python (FlagEmbedding for triple-mode)

from FlagEmbedding import BGEM3FlagModel

model = BGEM3FlagModel("BAAI/bge-m3", use_fp16=True)

# Encode with all three modes
sentences = ["بیمه نامه شخص ثالث چیست؟", "شرایط بیمه آتش‌سوزی"]
output = model.encode(sentences, return_dense=True, return_sparse=True, return_colbert_vecs=True)

print(f"Dense dims: {output['dense_vecs'].shape}")
print(f"Sparse tokens: {len(output['lexical_weights'][0])}")
print(f"ColBERT vecs: {len(output['colbert_vecs'][0])}")

Serving Platform Notes

Platform	Status	Notes
Ollama ✅	Available	Easiest. ollama pull bge-m3
TEI ✅	Available	Best throughput for production
sentence-transformers ✅	Available	Direct Python integration
vLLM	Not natively	vLLM is for LLM serving, not embeddings

Qdrant Integration

BGE-M3's sparse vectors work natively with Qdrant's sparse vector support:

from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance, SparseVectorParams

client = QdrantClient("localhost", port=6333)

# Create collection with both dense and sparse vectors
client.create_collection(
    collection_name="insurance_docs",
    vectors_config=VectorParams(size=1024, distance=Distance.COSINE),
    sparse_vectors_config={
        "sparse": SparseVectorParams()
    }
)

TODO

• Benchmark on MIRACL-Farsi subset
• Compare dense-only vs triple-mode retrieval quality
• Test with Qdrant sparse vector indexing
• Measure inference latency on RTX 3090