101 lines
3.0 KiB
Markdown
101 lines
3.0 KiB
Markdown
# ADR 0007: Hybrid Search (Dense + Sparse)
|
|
|
|
## Status
|
|
|
|
Accepted
|
|
|
|
## Context
|
|
|
|
Insurance queries often contain specific terms that require exact matching:
|
|
- Policy numbers and IDs
|
|
- Coverage names (e.g., "comprehensive", "third-party")
|
|
- Legal phrases and terminology
|
|
- Product codes and identifiers
|
|
|
|
Pure dense vector search excels at semantic similarity but can dilute exact term matches. A query for "policy AC-12345" should match documents containing that exact string, not just semantically similar policy-related content.
|
|
|
|
## Decision
|
|
|
|
Implement **hybrid search** combining dense and sparse vectors:
|
|
|
|
### Dense Vectors (Semantic)
|
|
- `text-embedding-3-large` (online) or `nomic-embed-text` (offline)
|
|
- Captures semantic meaning and intent
|
|
|
|
### Sparse Vectors (Lexical)
|
|
- **BM25** — classic term frequency scoring, fast, no model needed
|
|
- Alternative: **SPLADE** — learned sparse embeddings with term expansion
|
|
|
|
### Qdrant Configuration
|
|
|
|
```python
|
|
from qdrant_client.models import VectorParams, SparseVectorParams
|
|
|
|
client.create_collection(
|
|
collection_name="insurance_docs",
|
|
vectors_config={
|
|
"openai": VectorParams(size=3072, distance=Distance.COSINE),
|
|
"nomic": VectorParams(size=768, distance=Distance.COSINE),
|
|
},
|
|
sparse_vectors_config={
|
|
"bm25": SparseVectorParams(
|
|
modifier=models.Modifier.IDF, # Use IDF weighting
|
|
)
|
|
}
|
|
)
|
|
```
|
|
|
|
### Query Fusion
|
|
|
|
Use **Reciprocal Rank Fusion (RRF)** to combine dense and sparse results:
|
|
|
|
```python
|
|
def hybrid_search(query, top_k=20):
|
|
# Dense search
|
|
dense_results = client.search(
|
|
collection_name="insurance_docs",
|
|
query_vector=("openai", dense_embedding),
|
|
limit=top_k
|
|
)
|
|
|
|
# Sparse search
|
|
sparse_results = client.search(
|
|
collection_name="insurance_docs",
|
|
query_vector=("bm25", sparse_vector),
|
|
limit=top_k
|
|
)
|
|
|
|
# RRF fusion
|
|
return reciprocal_rank_fusion(dense_results, sparse_results)
|
|
```
|
|
|
|
### Why BM25 over SPLADE
|
|
- No model required — simpler deployment
|
|
- Faster indexing and query
|
|
- Works offline without additional dependencies
|
|
- SPLADE could be added later if term expansion proves valuable
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
- **Exact term matching** — policy numbers, IDs, specific terminology
|
|
- **Semantic + lexical** — best of both worlds
|
|
- **Improved precision** — especially for queries with specific terms
|
|
- **No additional model** — BM25 is built into Qdrant
|
|
|
|
### Negative
|
|
- **Added complexity** — must maintain and query two vector types
|
|
- **Fusion tuning** — RRF parameters may need adjustment
|
|
- **Storage overhead** — sparse vectors add storage (typically smaller than dense)
|
|
|
|
### Neutral
|
|
- Need to implement RRF or similar fusion logic
|
|
- May need different fusion weights for different query types
|
|
|
|
## Implementation Notes
|
|
|
|
- Use Qdrant's built-in BM25 sparse indexing
|
|
- Implement RRF with tunable `k` parameter (default k=60)
|
|
- Log dense vs sparse contributions to final ranking for debugging
|
|
- Consider query routing: queries with numbers/IDs → boost sparse weight
|