docs(architecture): adopt inline synchronous ingestion (ADR-0017)
Why: - ingestion is inline in the request instead of dispatched through RabbitMQ/outbox/worker; ADR-0014 is superseded (not deleted) and named as the design to adopt once ingestion needs to move off the request path. Changes: - new ADR-0017 plus amendments to every ADR/plan that referenced the job-shaped/broker contract, so none silently contradict it. Impact: - no broker, outbox, or worker code; rabbitmq test marker removed.
This commit is contained in:
@@ -4,6 +4,13 @@
|
||||
|
||||
Proposed
|
||||
|
||||
> Amended by [ADR-0017](0017-synchronous-ingestion-in-the-request-path.md):
|
||||
> there is no broker, outbox publisher, queue, or separate worker process for
|
||||
> now. `messaging/`, `workers/`, and `infrastructure/rabbitmq/` are part of the
|
||||
> target shape but are **not created yet**; ingestion runs inline in the request,
|
||||
> so the FastAPI route is the only entry adapter and it calls
|
||||
> `application/ingestion/` directly. Every layering rule below applies unchanged.
|
||||
|
||||
## Context
|
||||
|
||||
The repository currently contains only a small FastAPI-oriented scaffold under
|
||||
@@ -69,17 +76,18 @@ src/
|
||||
│ │ ├── models/
|
||||
│ │ ├── repositories/
|
||||
│ │ ├── database.py
|
||||
│ │ └── outbox.py
|
||||
│ │ └── outbox.py # ADR-0017: not yet; no outbox today
|
||||
│ ├── qdrant/
|
||||
│ ├── minio/
|
||||
│ ├── rabbitmq/
|
||||
│ ├── embedding/ # async embedding adapters (batched, bounded)
|
||||
│ ├── rabbitmq/ # ADR-0017: not yet; ADR-0014 target only
|
||||
│ ├── langgraph/
|
||||
│ └── observability/
|
||||
├── messaging/
|
||||
├── messaging/ # ADR-0017: not yet; ADR-0014 target only
|
||||
│ ├── events.py
|
||||
│ ├── subjects.py
|
||||
│ └── outbox_publisher.py
|
||||
└── workers/
|
||||
└── workers/ # ADR-0017: not yet; ADR-0014 target only
|
||||
├── ingestion.py
|
||||
└── maintenance.py
|
||||
```
|
||||
@@ -103,7 +111,6 @@ tests/
|
||||
├── integration/
|
||||
│ ├── postgres/
|
||||
│ ├── minio/
|
||||
│ ├── rabbitmq/
|
||||
│ └── qdrant/
|
||||
└── e2e/
|
||||
```
|
||||
@@ -193,8 +200,10 @@ bootstrap. Graph nodes call application services, particularly
|
||||
- `qdrant/` owns Qdrant client lifecycle, collection/bootstrap helpers, low-level
|
||||
point operations, and hybrid retrieval adapter mechanics.
|
||||
- `minio/` implements object-storage operations against MinIO/S3-compatible APIs.
|
||||
- `embedding/` implements the async dense/sparse embedding adapters, including
|
||||
provider batching and the concurrency semaphore from ADR-0017.
|
||||
- `rabbitmq/` owns the RabbitMQ connection/channel lifecycle plus low-level
|
||||
publish and consumer adapters (aio-pika).
|
||||
publish and consumer adapters (aio-pika). Not created while ADR-0017 stands.
|
||||
- `langgraph/` configures the concrete Postgres-backed LangGraph persistence
|
||||
adapters.
|
||||
- `observability/` configures structlog and Langfuse integrations.
|
||||
@@ -204,6 +213,12 @@ must not create mutable external clients at import time.
|
||||
|
||||
### Messaging and workers
|
||||
|
||||
Under ADR-0017 neither package exists yet: ingestion runs inline in the request,
|
||||
so the FastAPI route plays the entry-adapter role described here and obeys the
|
||||
same rule — it binds logging context and invokes an application service, and
|
||||
holds no parsing/chunking/Qdrant business logic itself. The rest of this section
|
||||
describes the shape both packages take when ADR-0014 is adopted.
|
||||
|
||||
`messaging/` contains versioned event schemas, stable routing-key names, and the
|
||||
outbox-publisher orchestration. The outbox publisher coordinates Postgres outbox
|
||||
records with the RabbitMQ adapter; it does not become a second source of job
|
||||
|
||||
Reference in New Issue
Block a user