docs(rag): add domain model and original research input

chunking/docs/adr/0011-inference-infrastructure.md

@@ -0,0 +1,97 @@
+# ADR 0011: Inference Infrastructure
+
+## Status
+
+Accepted
+
+## Context
+
+The system needs local inference for:
+- **LLM generation:** Qwen 2.5 (bilingual Persian/English)
+- **Local embeddings:** Nomic-embed-text
+- **Local reranking:** BGE Reranker
+
+Hardware available: NVIDIA RTX 3090 (24GB VRAM)
+
+## Decision
+
+### Local Inference Stack
+
+| Component | Tool | Model | VRAM Usage |
+|-----------|------|-------|------------|
+| LLM | Ollama | Qwen 2.5 | ~12-16GB (depends on quantization) |
+| Embeddings | Ollama or direct | Nomic-embed-text | ~1-2GB |
+| Reranker | Python + PyTorch | BGE Reranker | ~2-4GB |
+
+### Ollama Configuration
+
+```bash
+# Pull Qwen 2.5
+ollama pull qwen2.5:14b  # or 7b for lower VRAM
+
+# API endpoint
+OLLAMA_HOST=0.0.0.0:11434 ollama serve
+```
+
+**Qwen 2.5 model selection:**
+- `qwen2.5:7b` — fits comfortably, fast inference
+- `qwen2.5:14b` — better quality, still fits in 24GB
+- `qwen2.5:32b` — would require quantization or exceed VRAM
+
+Recommendation: Start with `qwen2.5:14b` for balance of quality and speed.
+
+### Nomic Embedding
+
+Option 1: Via Ollama
+```bash
+ollama pull nomic-embed-text
+```
+
+Option 2: Direct with sentence-transformers (faster for batch)
+```python
+from sentence_transformers import SentenceTransformer
+model = SentenceTransformer('nomic-ai/nomic-embed-text-v1.5')
+```
+
+### BGE Reranker
+
+Run directly with PyTorch:
+```python
+from sentence_transformers import CrossEncoder
+reranker = CrossEncoder('BAAI/bge-reranker-v2-m3')
+```
+
+### VRAM Allocation (24GB Total)
+
+| Component | VRAM | Notes |
+|-----------|------|-------|
+| Qwen 2.5 14b | ~14GB | Loaded once, kept warm |
+| BGE Reranker | ~3GB | Loaded on demand or kept warm |
+| Nomic Embedding | ~1GB | Can share with reranker |
+| Overhead | ~2GB | CUDA, framework |
+| **Total** | ~20GB | Fits within 24GB with headroom |
+
+## Consequences
+
+### Positive
+- **Single GPU** — all local inference fits on one RTX 3090
+- **Ollama simplicity** — easy model management and API
+- **Low latency** — local inference, no network calls during blackouts
+- **Bilingual support** — Qwen 2.5 and BGE-m3 handle Persian well
+
+### Negative
+- **Model limits** — can't run largest models (70b+) without quantization
+- **Concurrent load** — heavy concurrent requests may queue
+- **Power/heat** — GPU runs continuously under load
+
+### Neutral
+- May need to tune Ollama's `num_gpu` and `num_ctx` parameters
+- Consider quantized models (GGUF) for efficiency
+
+## Implementation Notes
+
+- Run Ollama as systemd service for auto-restart
+- Monitor GPU memory with `nvidia-smi`
+- Consider GPU temperature monitoring and throttling
+- Implement request queuing if concurrent load is high
+- Test with expected load to validate VRAM allocation