Commit Graph

6 Commits

Author SHA1 Message Date
5c0a5938f8 feat(ingestion): add bounded, benchmark-aligned embedding execution
Why:
- Plan 001 Phase 4 needs batched, concurrency-bounded embedding wired into
  the inline upload path, with process-wide capacity/timeout/chunk-limit
  guards (ADR-0017).
- The BM25 analyzer and dense-model config are ported from the `emet`
  evaluation lab, which benchmarked them against the real Farsi corpus
  (bm25-fa-norm-stop; nomic-embed-text-v2-moe at 768-dim; text-embedding-3-large
  at native 3072-dim), closing open items in ADR-0001/ADR-0005.

Changes:
- New: embedding ports, orchestration (embed_chunks), request-bounds
  helpers, and dense/sparse adapters (analyzers.py, bm25.py,
  openai_compatible.py).
- upload.py now parses/chunks/embeds inline behind INGESTION_MAX_CONCURRENCY
  (503), INGESTION_TIMEOUT_SECONDS (504), and the chunk-count ceiling (413);
  every failure path still writes a terminal job row.
- Lifespan builds and warms both dense embedders at startup (fail-soft) and
  creates the sparse embedder and concurrency semaphore.
- httpx moves from dev to main dependencies (adapters use it directly).

Impact:
- Qdrant point upserts are still Phase 5 -- chunks_indexed stays 0.
- New EMBEDDING_* env vars documented in .env.example; safe defaults.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-19 17:13:32 +03:30
9858e27c2d test(integration): add coverage for repositories, auth resolution, and the two-phase upload 2026-08-19 15:01:00 +03:30
e70ad13b10 test(postgres): support multi-session integration tests
Why:
- Code under test (auth resolution, the two-phase upload) opens more than
  one session per operation; the existing fixture only exposed one
  rolled-back session.

Changes:
- Add a db_sessionmaker fixture sharing one outer transaction.
- Pin loop_scope="session" -- without it, a second async test against the
  session-scoped Postgres container fails with "Event loop is closed."
2026-08-19 15:00:50 +03:30
7753651dd6 fix(tests): pin the Postgres container URL to IPv4 so asyncpg can connect
Testcontainers reports the container host as `localhost`, which resolves
to ::1 before 127.0.0.1 on this machine. Docker publishes the mapped port
on IPv4 only, and the IPv6 SYN is dropped rather than refused, so asyncpg
blocked on the first address until its connect timeout instead of falling
back to the second -- the migration fixture hung rather than failing.

127.0.0.1 connects in 0.07s where localhost timed out at 15s; the
integration suite now runs in 3.6s, inside the 10s pytest-timeout.
2026-08-18 10:40:32 +03:30
80ed5b1577 test(postgres): allow Testcontainers to override the migration sqlalchemy.url
alembic/env.py previously always overwrote sqlalchemy.url from
Settings().postgres.dsn, which made it impossible for a test fixture to
point Alembic at a Testcontainers-managed database. Now env.py only
sets it when unset, and a new integration suite runs `alembic upgrade
head` against a real Postgres container per ADR-0016 (no create_all()).
2026-08-18 10:21:56 +03:30
d71dd1bd0c test: add unit test suite for config, health checks, and lifespan wiring 2026-08-16 11:54:14 +03:30