67 docs indexed

Environment Variables

Complete reference for all obleth environment variables with types, defaults, and purpose.

All obleth configuration is done via environment variables. There are no config files.

Core listeners

VariableTypeDefaultPurpose
OBLETH_PROXY_LISTENstring0.0.0.0:8080Data plane bind address
OBLETH_ADMIN_LISTENstring0.0.0.0:9180Management API bind address
OBLETH_METRICS_LISTENstring0.0.0.0:9091Prometheus metrics bind address

Upstream

VariableTypeDefaultPurpose
OBLETH_UPSTREAM_BASE_URLURLhttp://127.0.0.1:8081Default upstream for all models without a per-model api_base
OBLETH_UPSTREAM_TIMEOUT_SECSinteger300Default per-request upstream timeout in seconds (covers streaming). Overridable per model via request_timeout_secs; see Reliability & Failover.
OBLETH_UPSTREAM_POOL_IDLE_SECSinteger15How long an idle pooled keep-alive connection to an upstream is kept before it is dropped. Lower values reduce the chance of reusing a socket the inference server has already closed (a common source of 502s); a connection-level send failure is also retried once on a fresh connection.
OBLETH_UPSTREAM_TCP_KEEPALIVE_SECSinteger30TCP keep-alive interval on upstream connections, so dead pooled sockets are detected sooner.

Datastores

VariableTypeDefaultPurpose
OBLETH_REDIS_URLURLredis://127.0.0.1:6379Redis connection string
OBLETH_DATABASE_URLURLpostgres://obleth:obleth@127.0.0.1:5432/oblethPostgres connection string
OBLETH_CLICKHOUSE_URLURLhttp://127.0.0.1:8123ClickHouse HTTP endpoint
OBLETH_CLICKHOUSE_DBstringoblethClickHouse database name
OBLETH_CLICKHOUSE_USERstringdefaultClickHouse user
OBLETH_CLICKHOUSE_PASSWORDstring""ClickHouse password

Admission and fairshare

VariableTypeDefaultPurpose
OBLETH_GLOBAL_MAX_IN_FLIGHTinteger256Maximum concurrent in-flight requests per pod
OBLETH_FAIRSHARE_ALGORITHMenumhierarchicalFairshare algorithm: weighted or hierarchical

Auto model routing

These seed the model: "auto" classifier on first boot. After that, the persisted auto_router setting (managed via PUT /api/v1/settings/auto-router) is authoritative. See Auto Model Routing.

VariableTypeDefaultPurpose
OBLETH_AUTO_CLASSIFIER_ENABLEDbooleanfalseBootstrap only. Enables the small-model intent classifier for auto routing.
OBLETH_AUTO_CLASSIFIER_MODELstringunsetBootstrap only. Registered model name used to classify request intent into routing tags.
OBLETH_AUTO_CLASSIFIER_TIMEOUT_MSinteger250Bootstrap only. Per-request classifier timeout; on timeout obleth falls back to keyword heuristics.

Reliability

VariableTypeDefaultPurpose
OBLETH_FAIL_OPENbooleantrueServe requests when Redis is unavailable (budget checks skipped)
OBLETH_WAL_PATHpath./obleth-telemetry.walPath to the telemetry write-ahead log file

Model health

VariableTypeDefaultPurpose
OBLETH_MODEL_HEALTH_ENABLEDbooleantrueEnables the scheduled model health worker
OBLETH_MODEL_HEALTH_INTERVAL_SECSinteger900Default interval for scheduled checks
OBLETH_MODEL_HEALTH_TIMEOUT_SECSinteger30Timeout for each health probe request
OBLETH_MODEL_HEALTH_RETENTION_DAYSinteger30Health check history retention window

Scheduled probes are sent directly to each model's configured upstream and are matched to the model's modality: chat and embedding models get a minimal inference call on their own endpoint, text-to-speech and transcription models a one-character / 0.1-second probe, and image models a catalog check against the upstream's model list. Results are stored in model-health history, and the probes' tokens are recorded under the internal health_probe identity rather than a tenant. A model with recent successful traffic in the usage ledger is settled from that instead of being probed. See Model Health.

Usage retention

VariableTypeDefaultPurpose
OBLETH_USAGE_RETENTION_DAYSinteger180Rolling window of raw per-request usage history kept in ClickHouse before old day-partitions are pruned. The permanent usage_daily rollup is never pruned. Tunable live via PUT /api/v1/settings/usage-retention (which overrides this default); floor of 1 day.

Security

VariableTypeDefaultPurpose
OBLETH_ADMIN_TOKENstringrequiredBearer token for all Management API requests. No default — the gateway/control plane will not start without it. Generate with openssl rand -hex 32.
OBLETH_ENCRYPTION_KEYstringunsetBase64 of 32 random bytes (openssl rand -base64 32). Encrypts upstream secrets at rest (model api_key, MCP auth_header) with AES-256-GCM. If unset, those columns are stored in plaintext and a warning is logged.
OBLETH_API_KEY_PEPPERstringunsetOptional server-side pepper mixed into API-key hashes (defense-in-depth if the config DB leaks). Keep it secret and stable; changing it invalidates all issued keys. Generate with openssl rand -hex 32.
OBLETH_BLOCK_PRIVATE_NETWORKSbooleanunset (false)SSRF policy for admin-registered upstream URLs (model api_base, MCP upstream_url). Default (local-first): private/LAN/loopback/CGNAT targets are allowed; only link-local and cloud metadata (169.254.0.0/16, fe80::/10) are blocked. Set truthy to switch to strict mode, which rejects all private/internal targets unless allow-listed in OBLETH_ALLOWED_PRIVATE_CIDRS.
OBLETH_ALLOWED_PRIVATE_CIDRSstringunsetComma-separated CIDR allowlist of private ranges to permit under strict mode (OBLETH_BLOCK_PRIVATE_NETWORKS=1). Ignored in the default permissive mode. Example: 10.0.0.0/8,192.168.0.0/16.

Compression sidecar

The optional neural prose compression sidecar is configured only through the environment — it has no Management API setting. Read its live status with GET /api/v1/settings/compressor.

VariableTypeDefaultPurpose
OBLETH_COMPRESSOR_URLURLunsetBase URL of the compression sidecar (e.g. http://compressor:8080). Empty or unset disables the neural stage of the compression boon entirely; the lossless structural passes still run.
OBLETH_COMPRESSOR_TIMEOUT_MSinteger800Per-request scoring timeout on the hot path. The sidecar fails open: on timeout or any error the text is left uncompressed. The Management API status probe uses this value with a floor of 2000 ms, since a cold sidecar answers its first /health slowly.

Observability

VariableTypeDefaultPurpose
OBLETH_OTEL_ENDPOINTURLunsetOpenTelemetry OTLP endpoint. If unset, tracing is disabled.
OBLETH_SESSION_ID_DERIVATIONbooleanonWhen a request carries no explicit session id, derive a stable conversation id from it (tenant + leading system/developer text + first user message). Set to 0/false/no/off to record only client-supplied ids. See Conversations & Sessions.
OBLETH_SLACK_WEBHOOK_URLURLunsetBootstrap only. Seeds the Slack alert webhook on first boot when no alert settings have been saved. Once you save settings from the dashboard or Management API, the persisted config is authoritative and this is ignored. See Alerting.
OBLETH_SLACK_ALERT_MIN_INTERVAL_SECSinteger300Bootstrap only. Initial per-alert cooldown used until alert settings are saved.
RUST_LOGstringinfo,obleth=debugRust log filter. The built-in default applies only when RUST_LOG is unset or unparseable.

Control plane

These variables apply to the obleth-control-plane Next.js service, not the gateway.

VariableTypeDefaultPurpose
OBLETH_ADMIN_BASE_URLURLhttp://localhost:9180Management API URL
OBLETH_ADMIN_TOKENstringrequiredAdmin token for Management API calls
DASHBOARD_ADMIN_EMAILstringadmin@example.comEmail for the break-glass admin, seeded on first boot. Must be a valid email address (a bare username is rejected).
DASHBOARD_PASSWORDstringrequiredBreak-glass admin password, ≥8 chars. Seeded on first boot and stored hashed by the auth layer.
DASHBOARD_SESSION_SECRETstringrequiredSession cookie signing secret. Must be ≥32 characters; the dashboard fails closed if missing or too short. Generate with openssl rand -hex 32.
DATABASE_URLURLrequiredPostgres URL for the auth tables (users, sessions, accounts). Points at the same database the gateway uses.
BETTER_AUTH_SECRETstringDASHBOARD_SESSION_SECRETSecret signing auth tokens; falls back to DASHBOARD_SESSION_SECRET when unset.
BETTER_AUTH_URLURLhttp://localhost:3000External, browser-facing dashboard URL; used to build OIDC redirect URIs. Docker Compose defaults it to http://localhost:3002, the published host port.
TRUSTED_ORIGINSstringunsetComma-separated extra origins better-auth accepts logins from, on top of BETTER_AUTH_URL. Set this to reach the dashboard by LAN IP or an alternate hostname without changing the base URL. * trusts every origin — use it only on a trusted private network.
OBLETH_PROXY_BASE_URLURLhttp://localhost:8080Data plane base URL. The dashboard's model-test, MCP-probe, and Playground features call the gateway through it.
OBLETH_RECIPES_DIRpath./recipes relative to the process working directoryDirectory of curated read-only sbatch recipe files offered by the Slurm model launcher.
OIDC_PROVIDERSJSONunsetJSON array of OIDC providers for SSO. Empty/unset = SSO off. See the Dashboard SSO guide.

Compose stack

These apply only to the Docker Compose stack in deploy/docker/, not to the gateway binary itself. See Docker Compose.

VariableTypeDefaultPurpose
COMPOSE_PROFILESstringbenchmark,edge,observability in .env.exampleComma-separated profiles to start. Listing them here means a single docker compose up -d --build brings up the extras, with no --profile flags.
OBLETH_VERSIONstringlatestImage tag for the first-party services. Pin a release (e.g. v0.9.6) for reproducible deploys; main tracks unreleased edge builds.
POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DBstringobleth / required / oblethBundled Postgres credentials, also composed into OBLETH_DATABASE_URL.
CLICKHOUSE_USER / CLICKHOUSE_PASSWORD / CLICKHOUSE_DBstringobleth / required / oblethBundled ClickHouse credentials, also passed to the gateway.
COMPRESSOR_CPUSinteger4CPU quota for the compression sidecar, and the matching onnxruntime thread count (compressor profile).
COMPRESSOR_SOURCE / COMPRESSOR_MODEL_ID / COMPRESSOR_ONNX_FILEstringonnx / chopratejas/kompress-v2-base / onnx/kompress-fp32.onnxBuild-time arguments selecting which scoring model the sidecar image bakes.
GRAFANA_ADMIN_USERstringadminGrafana admin username (observability profile).
GRAFANA_ADMIN_PASSWORDstringrequiredGrafana admin password. Anonymous access is disabled, so Compose errors out if this is unset when the observability profile runs.
HAPROXY_STATS_USERstringadminBasic-auth username for the HAProxy /stats page (edge profile).
HAPROXY_STATS_PASSWORDstringadminBasic-auth password for the HAProxy /stats page. Change from the dev example.

Slurm provisioner (obleth-provisioner service)

These apply only to the obleth-provisioner container — not the gateway itself. Slurm connection details (URL, version, user, JWT) are configured in the dashboard under Settings → Slurm and are stored encrypted in Postgres; the provisioner fetches them from the Management API on each tick.

VariableRequired?DefaultMeaning
OBLETH_ADMIN_TOKENrequired—Bearer token for the obleth Management API (same token the core stack uses)
OBLETH_ADMIN_BASE_URLoptionalhttp://localhost:9180Management API base URL
OBLETH_PROVISIONER_INTERVAL_SECSoptional15Reconcile tick interval in seconds
OBLETH_PROVISIONER_HEALTH_TIMEOUT_SECSoptional5Per-replica health probe timeout in seconds
OBLETH_PROVISIONER_WARMUP_TIMEOUT_SECSoptional600Timeout for the one throwaway warmup request fired at each replica right after it is promoted, so the cold first-token cost (graph capture, cache warmup) is paid by the gateway instead of the first real user. 0 disables warmup.
OBLETH_PROVISIONER_LOST_RETENTION_SECSoptional900How long lost replica rows are kept before GC (seconds)
OBLETH_PROVISIONER_RESTART_AFTER_FAILURESoptional20Self-heal threshold: restart a healthy replica after this many net failing health-probe ticks while its Slurm job still reports RUNNING (a zombie job). The counter decays on every passing probe, so brief flaps on a busy server never accumulate; 20 ticks at the default 15-second interval is roughly five minutes of sustained failure. Restarts are capped at one per model per tick. 0 disables self-heal.
OBLETH_PORT_SPANoptional8Width of the disjoint port window assigned to each replica. The job binds the first free port in [port_base, port_base + span) and the provisioner probes that window to discover it.
OBLETH_PROVISIONER_JOB_PREFIXoptionalobleth-Job-name prefix used to identify jobs owned by this gateway instance

Note:

If you pinned OBLETH_PROVISIONER_RESTART_AFTER_FAILURES=3 from an older deployment file, remove or raise it. At the default tick interval that is about 45 seconds of probe flaps before a healthy replica is cancelled.

See Slurm Provisioning for the full setup guide.