ADR 0007: Hybrid Search (Dense + Sparse)

Status

Accepted

Context

Insurance queries often contain specific terms that require exact matching:

Policy numbers and IDs
Coverage names (e.g., "comprehensive", "third-party")
Legal phrases and terminology
Product codes and identifiers

Pure dense vector search excels at semantic similarity but can dilute exact term matches. A query for "policy AC-12345" should match documents containing that exact string, not just semantically similar policy-related content.

Decision

Implement hybrid search combining dense and sparse vectors:

Dense Vectors (Semantic)

text-embedding-3-large (online) or nomic-embed-text (offline)
Captures semantic meaning and intent

Sparse Vectors (Lexical)

BM25 — classic term frequency scoring, fast, no model needed
Alternative: SPLADE — learned sparse embeddings with term expansion

Qdrant Configuration

from qdrant_client.models import VectorParams, SparseVectorParams

client.create_collection(
    collection_name="insurance_docs",
    vectors_config={
        "openai": VectorParams(size=3072, distance=Distance.COSINE),
        "nomic": VectorParams(size=768, distance=Distance.COSINE),
    },
    sparse_vectors_config={
        "bm25": SparseVectorParams(
            modifier=models.Modifier.IDF,  # Use IDF weighting
        )
    }
)

Query Fusion

Use Reciprocal Rank Fusion (RRF) to combine dense and sparse results:

def hybrid_search(query, top_k=20):
    # Dense search
    dense_results = client.search(
        collection_name="insurance_docs",
        query_vector=("openai", dense_embedding),
        limit=top_k
    )

    # Sparse search
    sparse_results = client.search(
        collection_name="insurance_docs",
        query_vector=("bm25", sparse_vector),
        limit=top_k
    )

    # RRF fusion
    return reciprocal_rank_fusion(dense_results, sparse_results)

Why BM25 over SPLADE

No model required — simpler deployment
Faster indexing and query
Works offline without additional dependencies
SPLADE could be added later if term expansion proves valuable

Consequences

Positive

Exact term matching — policy numbers, IDs, specific terminology
Semantic + lexical — best of both worlds
Improved precision — especially for queries with specific terms
No additional model — BM25 is built into Qdrant

Negative

Added complexity — must maintain and query two vector types
Fusion tuning — RRF parameters may need adjustment
Storage overhead — sparse vectors add storage (typically smaller than dense)

Neutral

Need to implement RRF or similar fusion logic
May need different fusion weights for different query types

Implementation Notes

Use Qdrant's built-in BM25 sparse indexing
Implement RRF with tunable k parameter (default k=60)
Log dense vs sparse contributions to final ranking for debugging
Consider query routing: queries with numbers/IDs → boost sparse weight

3.0 KiB Raw Permalink Blame History