58ca6109d1
Why: - ADR-0001's Qdrant payload records embedding_model_version so a future model swap can identify which chunks need re-embedding. The embedder is what knows which model produced its vectors, so it reports this rather than the call site reconstructing it from settings. Changes: - DenseEmbedder/SparseEmbedder protocols gain a model_version: str attribute. - OpenAICompatibleEmbedder reports its configured model; Bm25SparseEmbedder reports its analyzer (bm25-<analyzer>).
64 lines
2.3 KiB
Python
64 lines
2.3 KiB
Python
"""Embedding ports (ADR-0001, ADR-0017).
|
|
|
|
`src/infrastructure/embedding/` holds the production adapters; tests use
|
|
scripted fakes (ADR-0016). Application code depends on these Protocols, not
|
|
on `httpx`/provider SDKs directly.
|
|
"""
|
|
|
|
from collections.abc import Sequence
|
|
from typing import Protocol
|
|
|
|
from src.application.ingestion.models import SparseVector
|
|
|
|
|
|
class DenseEmbedder(Protocol):
|
|
"""One named dense vector's embedding client (`dense_nomic`/`dense_openai`).
|
|
|
|
`embed_batch` is a single batched network call — callers own concurrency
|
|
bounding (ADR-0017's `embed_concurrency` semaphore), not this Protocol.
|
|
"""
|
|
|
|
name: str
|
|
model_version: str
|
|
"""Identifies the model that produced these vectors (ADR-0001).
|
|
|
|
Written into every point's `embedding_model_version` payload field, which
|
|
exists so a future model swap can tell which chunks need re-embedding. The
|
|
embedder is what knows this, so it is reported here rather than
|
|
reconstructed from configuration at the call site.
|
|
"""
|
|
|
|
async def embed_batch(self, texts: Sequence[str]) -> list[list[float]]:
|
|
"""Return one vector per input text, same order. Raises `EmbedderError`
|
|
(see `src/application/ingestion/errors.py`) on transport/response
|
|
failure.
|
|
"""
|
|
...
|
|
|
|
|
|
class SparseEmbedder(Protocol):
|
|
"""The `sparse` (BM25) vector's embedding client.
|
|
|
|
Blocking/CPU-bound (ADR-0017): callers offload it via
|
|
`anyio.to_thread.run_sync` with the ingestion `CapacityLimiter`, not call
|
|
it directly from an `async def`.
|
|
"""
|
|
|
|
name: str
|
|
model_version: str
|
|
"""Identifies the analyzer/parameters that produced these vectors.
|
|
|
|
Same purpose as `DenseEmbedder.model_version`; for BM25 the "model" is the
|
|
analyzer choice (ADR-0005), which is equally a re-embedding trigger.
|
|
"""
|
|
|
|
def embed_batch(self, texts: Sequence[str], *, query: bool = False) -> list[SparseVector]:
|
|
"""Return one sparse vector per input text, same order.
|
|
|
|
`query=True` selects the query-side weighting, which omits document
|
|
length normalization. Ingestion always passes `False`; the flag exists
|
|
so retrieval (ADR-0003) encodes queries through this same port rather
|
|
than growing a second, silently divergent implementation.
|
|
"""
|
|
...
|