docs(research): add farsi embedding model research knowledge base

Body:
        Why:
        - Insurance RAG pipeline needs Farsi-optimized embeddings
        - Current local baseline (Nomic Embed) is English-only

        Changes:
        - 8 cloud model profiles with curl + python test commands
        - 5 local model profiles with VRAM, serving platform notes
        - Persian benchmark landscape (MIRACL-Fa, mMARCO, FaMTE, MTEB)
        - Decision report with ranked options and recommendations
        - Full comparison table (15 models × 10 columns)

        Impact:
        - Knowledge base for embedding model selection
        - Top picks: BGE-M3 (local), Cohere embed-v4.0 (cloud)
        - Critical: Nomic Embed must be replaced (English-only)
This commit is contained in:
2026-07-20 15:00:11 +03:30
parent 72d8a3ebe4
commit 6c9cebaaa0
13 changed files with 1296 additions and 0 deletions

122
embedding_models/openai.md Normal file
View File

@@ -0,0 +1,122 @@
---
last_updated: 2026-07-20
tags: [embedding, openai, cloud, farsi, insurance, rag]
source: import-knowledge
---
# OpenAI — text-embedding-3-large & text-embedding-3-small
## Overview
OpenAI's embedding API is the current baseline for our RAG pipeline. Two models available: text-embedding-3-large (higher quality) and text-embedding-3-small (cheaper).
## text-embedding-3-large *(current baseline)*
- **Type**: Cloud
- **Provider**: OpenAI
- **Price**: $0.13 per 1M input tokens ([source](https://openai.com/api/pricing/))
- **Max input tokens**: 8,191
- **Embedding dimensions**: 256, 1024, or 3072 (configurable via `dimensions` param)
- **Language coverage**: Multilingual (trained on multilingual corpus including Farsi)
- **Persian benchmark**: No public Persian-specific retrieval benchmark score published by OpenAI. MTEB aggregate available; MIRACL-Farsi subtask scores exist on the MTEB leaderboard but not officially published by OpenAI.
- **Notes**: Good general multilingual performance but not specifically optimized for Farsi. Pricing is mid-range among cloud options. Already integrated — zero migration cost.
## text-embedding-3-small *(cheaper alternative)*
- **Type**: Cloud
- **Provider**: OpenAI
- **Price**: $0.02 per 1M input tokens ([source](https://openai.com/api/pricing/))
- **Max input tokens**: 8,191
- **Embedding dimensions**: 512 or 1536 (configurable)
- **Language coverage**: Multilingual
- **Persian benchmark**: No public Persian-specific scores.
- **Notes**: 6.5× cheaper than text-embedding-3-large but generally lower retrieval quality. Useful as a cost-optimized fallback, not as a quality upgrade.
## API Test — Curl
**Endpoint**: `POST https://api.openai.com/v1/embeddings`
**Auth**: `Authorization: Bearer $OPENAI_API_KEY`
**Get key**: https://platform.openai.com/api-keys
### text-embedding-3-large
```bash
curl -s https://api.openai.com/v1/embeddings \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "text-embedding-3-large",
"input": ["بیمه نامه شخص ثالث چیست؟"]
}' | python3 -m json.tool
```
### text-embedding-3-small
```bash
curl -s https://api.openai.com/v1/embeddings \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "text-embedding-3-small",
"input": ["بیمه نامه شخص ثالث چیست؟"]
}' | python3 -m json.tool
```
**Response shape**: `data[0].embedding` = float array, `usage.total_tokens` = token count
## API Test — Python
```python
from openai import OpenAI
client = OpenAI() # uses OPENAI_API_KEY env var
# Test with a Farsi insurance query
response = client.embeddings.create(
model="text-embedding-3-large",
input=["بیمه نامه شخص ثالث چیست؟"]
)
embedding = response.data[0].embedding
tokens = response.usage.total_tokens
print(f"Dimensions: {len(embedding)}, Tokens: {tokens}")
# Expected: Dimensions: 3072, Tokens: 6
```
## Python Test — Compare Both Models
```python
from openai import OpenAI
import time
client = OpenAI()
query = "شرایط لغو بیمه نامه چیست؟"
for model in ["text-embedding-3-large", "text-embedding-3-small"]:
start = time.time()
resp = client.embeddings.create(model=model, input=[query])
elapsed = time.time() - start
dims = len(resp.data[0].embedding)
tokens = resp.usage.total_tokens
print(f"{model}: dims={dims}, tokens={tokens}, time={elapsed:.2f}s")
```
## Response Format
| Field | Path |
|-------|------|
| Embedding vector | `data[0].embedding` |
| Token count | `usage.total_tokens` |
## Key Differences from Other Providers
- Uses standard `input` field (string or array)
- Configurable dimensions — can reduce from 3072 to 1024 or 256 to save storage/latency
- 8K context limit — may require aggressive chunking for long insurance documents
- Data sent to US servers — not on-premises
## TODO
- [ ] Benchmark on MIRACL-Farsi subset
- [ ] Test dimension reduction (3072 → 1024) impact on retrieval quality
- [ ] Calculate monthly cost at production volume