docs(adr): record the re-ingestion rule and close plan 002's open decisions
Why: - ADR-0002 already answered three of the four questions plan 002 listed as "decisions needed"; the fourth -- what happens to a manually edited point when its file is re-uploaded -- was left for a Phase 6 test to force. Deciding it in code rather than in the ADR would invert this repo's rule. Changes: - ADR-0002 gains "Re-ingestion versus manual edits": the new file wins, surviving points are overwritten with an incremented version, absent points are flagged inactive rather than removed, and manually created points sit past the ingested chunk_index range so the existing sweep covers them. - Plan 002's stale decisions section becomes a pointer table; its audit scope is pinned to both ADR-0009 tables. Impact: - Clobbered edits are recoverable from point_audit_events, not from Qdrant: the deterministic point ID cannot hold both versions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -105,6 +105,41 @@ Qdrant's `update_filter`, giving an optimistic-concurrency-style guard
|
||||
against races between a concurrent ingestion re-run (ADR-0001) and a manual
|
||||
edit through this API.
|
||||
|
||||
### Re-ingestion versus manual edits
|
||||
|
||||
A file can be re-uploaded after someone has hand-edited one of its points
|
||||
through this API. **The newly ingested file wins.** Ingestion is authoritative
|
||||
for the content of the file it ingested; a manual edit is a correction that
|
||||
survives only until the source document is replaced.
|
||||
|
||||
Concretely:
|
||||
|
||||
- A point that still exists in the new version (same `file_id` +
|
||||
`chunk_index`, hence the same deterministic point ID) is **overwritten in
|
||||
place**. Ingestion performs a read-check-write so `version` is incremented
|
||||
from whatever the manual edit left it at, rather than reset to `1`.
|
||||
- A point from the previous ingestion that is **absent** from the new version
|
||||
is flagged `is_active: false` with `deleted_at` set. It is never removed
|
||||
from Qdrant — the soft-delete rule above applies to re-ingestion exactly as
|
||||
it applies to `DELETE`.
|
||||
- A manually created point (`POST /points`) is assigned a `chunk_index` past
|
||||
the ingested range, so the same sweep deactivates it on the next upload of
|
||||
its file. This is the intended consequence of "the new file wins", not an
|
||||
accident of the sweep's bounds.
|
||||
|
||||
Because the point ID is derived from the immutable `chunk_index`, an
|
||||
overwritten point cannot hold both the manual edit and the new file's content.
|
||||
The clobbered content is therefore recorded in `point_audit_events`
|
||||
(ADR-0009) as a `reingest_overwrite` operation carrying `before_version`, so
|
||||
the edit is recoverable from the audit trail even though it is no longer a
|
||||
live point.
|
||||
|
||||
Rejected alternative: preserving manual edits by having ingestion skip points
|
||||
with `version > 1`. It breaks the guarantee that a successful upload leaves
|
||||
Qdrant matching the uploaded document, and it needs a second, separate rule
|
||||
for edited points that no longer exist in the new version — two divergent
|
||||
notions of authority over one file.
|
||||
|
||||
### Re-embedding on content edit
|
||||
|
||||
`PUT /points/{point_id}` can change `content`, which leaves the stored
|
||||
|
||||
Reference in New Issue
Block a user