docs(architecture): adopt inline synchronous ingestion (ADR-0017)

Why:
- ingestion is inline in the request instead of dispatched through RabbitMQ/outbox/worker; ADR-0014 is superseded (not deleted) and named as the design to adopt once ingestion needs to move off the request path.

Changes:
- new ADR-0017 plus amendments to every ADR/plan that referenced the job-shaped/broker contract, so none silently contradict it.

Impact:
- no broker, outbox, or worker code; rabbitmq test marker removed.
This commit is contained in:
2026-08-16 11:53:59 +03:30
parent e2322a2909
commit fd70ad01af
9 changed files with 542 additions and 154 deletions

View File

@@ -105,6 +105,62 @@ Qdrant's `update_filter`, giving an optimistic-concurrency-style guard
against races between a concurrent ingestion re-run (ADR-0001) and a manual
edit through this API.
### Re-embedding on content edit
`PUT /points/{point_id}` can change `content`, which leaves the stored
vectors stale unless they're recomputed. When `content` changes, the point
is **re-embedded inline**, reusing the same async embedding ports and
batch/semaphore bounds ingestion uses ([0017](0017-synchronous-ingestion-in-the-request-path.md)),
for parity between the two write paths. When `content` is unchanged, the
edit applies only the supplied vector/payload fields and skips re-embedding
entirely. Failure modes on this path reuse ingestion's status codes: `502`
on embedder failure, `504` if the edit's embedding step exceeds the same
timeout budget class as ingestion. The `version` guard (`update_filter`)
still applies to the write — re-embedding happens before the guarded write,
not instead of it, so a stale-version edit still fails with `409` rather
than re-embedding for nothing.
Rejected alternatives: requiring the caller to supply vectors when content
changes (pushes model knowledge onto the client, and is easy to get subtly
wrong); marking the point stale for background re-embedding later (needs
background work, which ADR-0017 currently rules out for this slice).
### `order_id` gap exhaustion
Repeatedly inserting into the same gap between two neighbors eventually
exhausts float precision (ADR-0001's known limitation). This slice does
**not** ship a renormalize endpoint. Instead, any operation that assigns a
new fractional `order_id` between two neighbors (insert, reorder) computes
the resulting gap and:
- logs a structured warning (`points.order_id.gap_low`) with `file_id` and
the two neighbor point IDs once the gap falls under a defined safety
threshold, so the condition is observable before it becomes uninsertable;
- **rejects** the write with `409` and a distinct error code if the
computed gap is no longer representable (would collapse to one of the two
neighbor values), instead of silently applying an imprecise value.
Recovering from an exhausted gap is a manual data-fix operation covered by
the operator runbook, not an endpoint this slice builds — deferring the
renormalize primitive is acceptable, silently producing an unrepresentable
gap is not.
### `POST /points/batch` semantics
Batch requests are **all-or-nothing**, capped at **100 operations per
request**. The service layer validates every operation's `version`
precondition before applying any of them; if any operation's precondition
fails, the whole request is rejected with `409` and nothing is applied — no
partially-applied batch ever reaches Qdrant. This follows directly from the
`version`-guard rule above applied at the batch level, and from the
pointer-relinking rule (a reorder/insert/delete's neighbor updates must land
in the same `points/batch` call, and a partial relink is a defect): partial
application of a batch is exactly the failure mode that would produce a
stale pointer chain. The 100-operation cap is independent of ADR-0001's
64–256-point bulk-ingestion batch sizing — that number is about upload
throughput; this one bounds an admin/manual edit request to something that
comfortably finishes inside a normal request timeout.
## Consequences
### Positive
@@ -122,6 +178,15 @@ edit through this API.
- Optimistic concurrency via `version` requires every writer (ingestion and
this API) to consistently read-check-write; a writer that skips this can
silently clobber concurrent edits.
- Inline re-embedding puts embedder latency and `502`/`504` failure modes on
an admin content edit, not just on ingestion — an edit that only intended
to fix a typo pays the same embedding cost as a fresh chunk.
- Deferring the `order_id` renormalize endpoint means a file whose gaps are
genuinely exhausted has no automated recovery in this slice; an operator
must intervene by hand until that endpoint exists.
- All-or-nothing batch semantics mean one stale operation in a 100-operation
batch fails the entire request, even when the other 99 operations are
independent and would have succeeded on their own.
## Alternatives Considered
@@ -132,3 +197,16 @@ edit through this API.
- **Client-supplied `tenant_id` in request body**: rejected — trusting
client input for the isolation boundary is a direct multitenancy security
risk; it must come from server-side auth context.
- **Caller-supplied vectors on content edit**: rejected — pushes embedding
model knowledge onto the client and makes it easy to silently desync
vectors from content.
- **Mark-stale-and-re-embed-later on content edit**: rejected for this
slice — needs background work, which ADR-0017 currently rules out.
- **Renormalize `order_id` automatically within this slice**: rejected —
nothing in current scope has hit gap exhaustion; building the primitive
now is speculative. Revisit if the logged warning starts firing in
practice.
- **Partial-success batch semantics (per-operation status)**: rejected —
a partially-applied batch is exactly the failure mode that leaves the
pointer chain (`previous_chunk_id`/`next_chunk_id`) inconsistent, which
this ADR treats as a defect, not a degraded-but-acceptable outcome.