Files

Mahdi Bazrafshan 6c9cebaaa0 docs(research): add farsi embedding model research knowledge base

Body:
        Why:
        - Insurance RAG pipeline needs Farsi-optimized embeddings
        - Current local baseline (Nomic Embed) is English-only

        Changes:
        - 8 cloud model profiles with curl + python test commands
        - 5 local model profiles with VRAM, serving platform notes
        - Persian benchmark landscape (MIRACL-Fa, mMARCO, FaMTE, MTEB)
        - Decision report with ranked options and recommendations
        - Full comparison table (15 models × 10 columns)

        Impact:
        - Knowledge base for embedding model selection
        - Top picks: BGE-M3 (local), Cohere embed-v4.0 (cloud)
        - Critical: Nomic Embed must be replaced (English-only)

2026-07-20 15:00:11 +03:30

2.4 KiB

Raw Blame History

last_updated, tags, source

last_updated

intfloat/multilingual-e5-large

Overview

Type: Local
Provider: intfloat
Parameters: ~560M (24-layer, xlm-roberta-large init)
VRAM (fp16): ~1.1 GB
Embedding dimensions: 1024
Max input tokens: 512
Language coverage: 100 languages including Farsi
Serving platform: sentence-transformers, TEI, Ollama (available)
License: MIT
Model card: https://huggingface.co/intfloat/multilingual-e5-large

Pros

Most widely used multilingual embedding model
Strong community support and documentation
100 languages including Farsi
MIT license
Available on Ollama

Cons

512 token max context — requires aggressive chunking for insurance documents
No ColBERT/sparse support
Requires "query: " / "passage: " prefixes for optimal retrieval
Benchmarked on Mr. TyDi but Farsi is NOT one of the 11 Mr. TyDi languages

⚠️ 512 Token Limit

Insurance policy clauses and FAQ answers can easily exceed 512 tokens. You will need to chunk aggressively. Consider BGE-M3 (8K context) or gte-multilingual-base (8K context) instead.

API Test — Ollama

ollama pull multilingual-e5-large

curl -s http://localhost:11434/api/embeddings   -d '{
    "model": "multilingual-e5-large",
    "prompt": "query: بیمه نامه شخص ثالث چیست؟"
  }' | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'dims={len(d["embedding"])}')"

⚠️ Important: Ollama may not auto-add the "query: " prefix. You must add it manually in the prompt for optimal retrieval quality.

API Test — Python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("intfloat/multilingual-e5-large")

# ⚠️ Must add "query: " prefix for queries, "passage: " for documents
query_embedding = model.encode("query: بیمه نامه شخص ثالث چیست؟")
doc_embedding = model.encode("passage: بیمه نامه شخص ثالث شامل پوشش خسارات مالی و جانی است.")

print(f"Query dims: {len(query_embedding)}, Doc dims: {len(doc_embedding)}")
# Expected: Query dims: 1024, Doc dims: 1024

TODO

Benchmark on MIRACL-Farsi subset
Test with aggressive chunking (max 500 tokens)
Compare with BGE-M3 on Farsi retrieval quality

2.4 KiB Raw Blame History