9e8987968c
Why:
- Wanted human-readable console output while developing locally, without
losing a machine-parseable log for later grepping/parsing. A single
renderer chosen by a flag can't do both at once.
- ADR-0011 had no way to correlate an issue with a specific deployment
(build/region/instance) independent of any one request.
Changes:
- configure_logging() now builds two independent handlers: console (always
on, colored unless LOG_JSON_FORMAT=true) and an optional rotating JSON file
(LOG_FILE_PATH, unset by default) -- the same structlog event fans out to
both, so call sites are unaffected.
- A static structlog processor binds env/service_version onto every event.
Deliberately not a contextvar: RequestIdMiddleware's clear_contextvars()
would wipe a value bound there before the first request.
- New settings: APP_SERVICE_VERSION, LOG_FILE_PATH/LOG_FILE_MAX_BYTES/
LOG_FILE_BACKUP_COUNT.
- ADR-0011 amended with both decisions ("console and file are independent
sinks locally"; "bind process-level environment context once at startup").
Impact:
- configure_logging() signature changed to (logging_settings, app_settings);
both call sites (lifespan, qdrant_bootstrap CLI) updated.
111 lines
4.1 KiB
Plaintext
111 lines
4.1 KiB
Plaintext
# Application local development environment.
|
|
#
|
|
# Usage:
|
|
# cp .env.example .env
|
|
# docker compose -f docker-compose.yml up -d
|
|
#
|
|
# All values below are non-secret local-development defaults matching
|
|
# docker-compose.yml. Do not commit .env.
|
|
|
|
# Application
|
|
APP_ENV=local
|
|
APP_READINESS_CHECK_TIMEOUT_SECONDS=2.0
|
|
# Set by CI/CD at build/deploy time; never computed at runtime.
|
|
APP_SERVICE_VERSION=dev
|
|
|
|
# Logging
|
|
LOG_LEVEL=INFO
|
|
LOG_JSON_FORMAT=false
|
|
# Optional second sink, always JSON regardless of LOG_JSON_FORMAT. Local dev
|
|
# only -- leave unset in production, where stdout/stderr collection is
|
|
# preferred over an in-container log file.
|
|
# LOG_FILE_PATH=logs/app.log
|
|
LOG_FILE_MAX_BYTES=10485760
|
|
LOG_FILE_BACKUP_COUNT=5
|
|
|
|
# Postgres (application database, separate from Langfuse's Postgres)
|
|
# Use 127.0.0.1 rather than localhost: some environments resolve localhost to
|
|
# the IPv6 loopback first, and Docker only publishes these ports on IPv4.
|
|
POSTGRES_HOST=127.0.0.1
|
|
POSTGRES_PORT=5433
|
|
POSTGRES_USER=chatbot
|
|
POSTGRES_PASSWORD=chatbot
|
|
POSTGRES_DB=chatbot
|
|
|
|
# MinIO (application bucket, separate from Langfuse's MinIO)
|
|
MINIO_ENDPOINT=127.0.0.1:9100
|
|
MINIO_ACCESS_KEY=chatbot
|
|
MINIO_SECRET_KEY=chatbot-secret
|
|
MINIO_SECURE=false
|
|
MINIO_BUCKET=chatbot-source-files
|
|
|
|
# Inline ingestion bounds (ADR-0017; no broker, no queue).
|
|
# INGESTION_TIMEOUT_SECONDS must stay below any proxy/client read timeout.
|
|
INGESTION_MAX_CONCURRENCY=4
|
|
INGESTION_THREAD_POOL_SIZE=8
|
|
INGESTION_TIMEOUT_SECONDS=120.0
|
|
INGESTION_MAX_UPLOAD_SIZE_MB=25
|
|
INGESTION_MAX_CHUNKS_PER_FILE=5000
|
|
INGESTION_EMBED_BATCH_SIZE=128
|
|
INGESTION_EMBED_CONCURRENCY=4
|
|
|
|
# Qdrant
|
|
QDRANT_URL=http://127.0.0.1:6343
|
|
QDRANT_API_KEY=
|
|
QDRANT_COLLECTION=chunks
|
|
QDRANT_UPSERT_BATCH_SIZE=128
|
|
QDRANT_UPSERT_CONCURRENCY=4
|
|
|
|
# Dense embedders (ADR-0001). Both speak an OpenAI-compatible /embeddings
|
|
# endpoint, so one adapter serves both. Models and endpoints are the ones the
|
|
# `emet` evaluation lab benchmarked as winners on the Farsi corpus.
|
|
#
|
|
# dense_nomic runs behind Ollama's OpenAI-compat shim, which accepts any
|
|
# non-empty API key. KEEP_ALIVE holds the model resident: a cold load of
|
|
# nomic-embed-text-v2-moe takes >150s, well past INGESTION_TIMEOUT_SECONDS,
|
|
# so an idle-then-upload would otherwise 504.
|
|
EMBEDDING_NOMIC_BASE_URL=http://192.168.10.10:11435/v1
|
|
EMBEDDING_NOMIC_MODEL=nomic-embed-text-v2-moe
|
|
EMBEDDING_NOMIC_API_KEY=sk-not-set
|
|
EMBEDDING_NOMIC_KEEP_ALIVE=30m
|
|
EMBEDDING_NOMIC_TIMEOUT_SECONDS=30.0
|
|
# Empty = emet parity. The model card specifies `search_document: ` (ADR-0004),
|
|
# but the benchmark ran without it and the prefix shifts the vector a lot
|
|
# (cosine 0.57 on identical text) — so if you set this, the query side must
|
|
# send `search_query: ` to match, or retrieval gets worse rather than better.
|
|
EMBEDDING_NOMIC_DOCUMENT_PREFIX=
|
|
|
|
# Leave DIMENSIONS empty for text-embedding-3-large's native 3072, which is
|
|
# what was benchmarked. Setting it truncates via Matryoshka and is a
|
|
# re-embedding migration, not a config tweak.
|
|
EMBEDDING_OPENAI_BASE_URL=https://api.openai.com/v1
|
|
EMBEDDING_OPENAI_MODEL=text-embedding-3-large
|
|
EMBEDDING_OPENAI_API_KEY=
|
|
EMBEDDING_OPENAI_DIMENSIONS=
|
|
EMBEDDING_OPENAI_DOCUMENT_PREFIX=
|
|
EMBEDDING_OPENAI_TIMEOUT_SECONDS=30.0
|
|
|
|
# Sparse BM25 (ADR-0001, ADR-0005): the benchmarked `bm25-fa-norm-stop`.
|
|
# k/b saturation is applied client-side; IDF comes from Qdrant's
|
|
# modifier="idf" on the sparse vector field. AVG_LEN is the average document
|
|
# length in analyzer tokens — emet's placeholder, worth recalibrating from
|
|
# real corpus statistics.
|
|
EMBEDDING_SPARSE_ANALYZER=fa_norm_stop
|
|
EMBEDDING_SPARSE_K=1.2
|
|
EMBEDDING_SPARSE_B=0.75
|
|
EMBEDDING_SPARSE_AVG_LEN=256.0
|
|
|
|
# Parsing and chunking (ADR-0018).
|
|
# max_chunk_tokens is nomic-embed-text-v2-moe's sequence length; text past it
|
|
# is silently truncated by the model, so the cap is enforced before embedding.
|
|
# chunk_size sits under it to leave room for the `search_document: ` prefix.
|
|
CHUNKING_STRATEGY=fixed_size
|
|
CHUNKING_CHUNK_SIZE=400
|
|
CHUNKING_CHUNK_OVERLAP=60
|
|
CHUNKING_MAX_CHUNK_TOKENS=512
|
|
CHUNKING_ENCODING_NAME=cl100k_base
|
|
|
|
# tiktoken downloads its vocabulary on first use; point this at a
|
|
# pre-populated directory for offline/air-gapped deployments.
|
|
# TIKTOKEN_CACHE_DIR=
|