64 docs indexed

Environment Variables

Complete reference for all obleth environment variables with types, defaults, and purpose.

All obleth configuration is done via environment variables. There are no config files.

Core listeners

VariableTypeDefaultPurpose
OBLETH_PROXY_LISTENstring0.0.0.0:8080Data plane bind address
OBLETH_ADMIN_LISTENstring0.0.0.0:9180Management API bind address
OBLETH_METRICS_LISTENstring0.0.0.0:9091Prometheus metrics bind address

Upstream

VariableTypeDefaultPurpose
OBLETH_UPSTREAM_BASE_URLURLhttp://127.0.0.1:8081Default upstream for all models without a per-model api_base
OBLETH_UPSTREAM_TIMEOUT_SECSinteger300Default per-request upstream timeout in seconds (covers streaming). Overridable per model via request_timeout_secs; see Reliability & Failover.
OBLETH_UPSTREAM_POOL_IDLE_SECSinteger15How long an idle pooled keep-alive connection to an upstream is kept before it is dropped. Lower values reduce the chance of reusing a socket the inference server has already closed (a common source of 502s); a connection-level send failure is also retried once on a fresh connection.
OBLETH_UPSTREAM_TCP_KEEPALIVE_SECSinteger30TCP keep-alive interval on upstream connections, so dead pooled sockets are detected sooner.

Datastores

VariableTypeDefaultPurpose
OBLETH_REDIS_URLURLredis://127.0.0.1:6379Redis connection string
OBLETH_DATABASE_URLURLpostgres://obleth:obleth@localhost/oblethPostgres connection string
OBLETH_CLICKHOUSE_URLURLhttp://127.0.0.1:8123ClickHouse HTTP endpoint
OBLETH_CLICKHOUSE_DBstringoblethClickHouse database name
OBLETH_CLICKHOUSE_USERstringdefaultClickHouse user
OBLETH_CLICKHOUSE_PASSWORDstring""ClickHouse password

Admission and fairshare

VariableTypeDefaultPurpose
OBLETH_GLOBAL_MAX_IN_FLIGHTinteger256Maximum concurrent in-flight requests per pod
OBLETH_FAIRSHARE_ALGORITHMenumhierarchicalFairshare algorithm: weighted or hierarchical

Auto model routing

These seed the model: "auto" classifier on first boot. After that, the persisted auto_router setting (managed via PUT /api/v1/settings/auto-router) is authoritative. See Auto Model Routing.

VariableTypeDefaultPurpose
OBLETH_AUTO_CLASSIFIER_ENABLEDbooleanfalseBootstrap only. Enables the small-model intent classifier for auto routing.
OBLETH_AUTO_CLASSIFIER_MODELstringunsetBootstrap only. Registered model name used to classify request intent into routing tags.
OBLETH_AUTO_CLASSIFIER_TIMEOUT_MSinteger250Bootstrap only. Per-request classifier timeout; on timeout obleth falls back to keyword heuristics.

Reliability

VariableTypeDefaultPurpose
OBLETH_FAIL_OPENbooleantrueServe requests when Redis is unavailable (budget checks skipped)
OBLETH_WAL_PATHpath./obleth-telemetry.walPath to the telemetry write-ahead log file

Model health

VariableTypeDefaultPurpose
OBLETH_MODEL_HEALTH_ENABLEDbooleantrueEnables the scheduled model health worker
OBLETH_MODEL_HEALTH_INTERVAL_SECSinteger900Default interval for scheduled checks
OBLETH_MODEL_HEALTH_TIMEOUT_SECSinteger30Timeout for each health probe request
OBLETH_MODEL_HEALTH_RETENTION_DAYSinteger30Health check history retention window

Health probes are token-free liveness checks (GET {api_base}/models) sent directly to each model's configured upstream. They are stored in model-health history, not normal usage analytics. See Model Health.

Usage retention

VariableTypeDefaultPurpose
OBLETH_USAGE_RETENTION_DAYSinteger180Rolling window of raw per-request usage history kept in ClickHouse before old day-partitions are pruned. The permanent usage_daily rollup is never pruned. Tunable live via PUT /api/v1/settings/usage-retention (which overrides this default); floor of 1 day.

Security

VariableTypeDefaultPurpose
OBLETH_ADMIN_TOKENstringrequiredBearer token for all Management API requests. No default — the gateway/control plane will not start without it. Generate with openssl rand -hex 32.
OBLETH_ENCRYPTION_KEYstringunsetBase64 of 32 random bytes (openssl rand -base64 32). Encrypts upstream secrets at rest (model api_key, MCP auth_header) with AES-256-GCM. If unset, those columns are stored in plaintext and a warning is logged.
OBLETH_API_KEY_PEPPERstringunsetOptional server-side pepper mixed into API-key hashes (defense-in-depth if the config DB leaks). Keep it secret and stable; changing it invalidates all issued keys. Generate with openssl rand -hex 32.
OBLETH_BLOCK_PRIVATE_NETWORKSbooleanunset (false)SSRF policy for admin-registered upstream URLs (model api_base, MCP upstream_url). Default (local-first): private/LAN/loopback/CGNAT targets are allowed; only link-local and cloud metadata (169.254.0.0/16, fe80::/10) are blocked. Set truthy to switch to strict mode, which rejects all private/internal targets unless allow-listed in OBLETH_ALLOWED_PRIVATE_CIDRS.
OBLETH_ALLOWED_PRIVATE_CIDRSstringunsetComma-separated CIDR allowlist of private ranges to permit under strict mode (OBLETH_BLOCK_PRIVATE_NETWORKS=1). Ignored in the default permissive mode. Example: 10.0.0.0/8,192.168.0.0/16.

Observability

VariableTypeDefaultPurpose
OBLETH_OTEL_ENDPOINTURLunsetOpenTelemetry OTLP endpoint. If unset, tracing is disabled.
OBLETH_SESSION_ID_DERIVATIONbooleanonWhen a request carries no explicit session id, derive a stable conversation id from it (tenant + leading system/developer text + first user message). Set to 0/false/no/off to record only client-supplied ids. See Conversations & Sessions.
OBLETH_SLACK_WEBHOOK_URLURLunsetBootstrap only. Seeds the Slack alert webhook on first boot when no alert settings have been saved. Once you save settings from the dashboard or Management API, the persisted config is authoritative and this is ignored. See Alerting.
OBLETH_SLACK_ALERT_MIN_INTERVAL_SECSinteger300Bootstrap only. Initial per-alert cooldown used until alert settings are saved.
RUST_LOGstringobleth=infoRust log filter. Use obleth=debug for verbose output.

Control plane

These variables apply to the obleth-control-plane Next.js service, not the gateway.

VariableTypeDefaultPurpose
OBLETH_ADMIN_BASE_URLURLhttp://localhost:9180Management API URL
OBLETH_ADMIN_TOKENstringrequiredAdmin token for Management API calls
DASHBOARD_ADMIN_EMAILstringadmin@example.comEmail for the break-glass admin, seeded on first boot. Must be a valid email address (a bare username is rejected).
DASHBOARD_PASSWORDstringrequiredBreak-glass admin password (plaintext, ≥8 chars). Provide this or DASHBOARD_PASSWORD_HASH.
DASHBOARD_PASSWORD_HASHstringunsetbcrypt hash of the admin password. Preferred over DASHBOARD_PASSWORD. Generate with npx bcrypt-cli hash 'your-password' 12.
DASHBOARD_SESSION_SECRETstringrequiredSession cookie signing secret. Must be ≥32 characters; the dashboard fails closed if missing or too short. Generate with openssl rand -hex 32.
DATABASE_URLURLrequiredPostgres URL for the auth tables (users, sessions, accounts). Points at the same database the gateway uses.
BETTER_AUTH_SECRETstringDASHBOARD_SESSION_SECRETSecret signing auth tokens; falls back to DASHBOARD_SESSION_SECRET when unset.
BETTER_AUTH_URLURLhttp://localhost:3000External, browser-facing dashboard URL; used to build OIDC redirect URIs.
OIDC_PROVIDERSJSONunsetJSON array of OIDC providers for SSO. Empty/unset = SSO off. See the Dashboard SSO guide.

Compose stack (observability & edge profiles)

These apply only to the Docker Compose stack in deploy/docker/, not to the gateway binary itself.

VariableTypeDefaultPurpose
GRAFANA_ADMIN_USERstringadminGrafana admin username (observability profile).
GRAFANA_ADMIN_PASSWORDstringrequiredGrafana admin password. Anonymous access is disabled, so Compose errors out if this is unset when the observability profile runs.
HAPROXY_STATS_USERstringadminBasic-auth username for the HAProxy /stats page (edge profile).
HAPROXY_STATS_PASSWORDstringadminBasic-auth password for the HAProxy /stats page. Change from the dev example.

Slurm provisioner (obleth-provisioner service)

These apply only to the obleth-provisioner container — not the gateway itself. Slurm connection details (URL, version, user, JWT) are configured in the dashboard under Settings → Slurm and are stored encrypted in Postgres; the provisioner fetches them from the Management API on each tick.

VariableRequired?DefaultMeaning
OBLETH_ADMIN_TOKENrequired—Bearer token for the obleth Management API (same token the core stack uses)
OBLETH_ADMIN_BASE_URLoptionalhttp://localhost:9180Management API base URL
OBLETH_PROVISIONER_INTERVAL_SECSoptional15Reconcile tick interval in seconds
OBLETH_PROVISIONER_HEALTH_TIMEOUT_SECSoptional5Per-replica health probe timeout in seconds
OBLETH_PROVISIONER_WARMUP_TIMEOUT_SECSoptional600Timeout for the one throwaway warmup request fired at each replica right after it is promoted, so the cold first-token cost (graph capture, cache warmup) is paid by the gateway instead of the first real user. 0 disables warmup.
OBLETH_PROVISIONER_LOST_RETENTION_SECSoptional900How long lost replica rows are kept before GC (seconds)
OBLETH_PROVISIONER_JOB_PREFIXoptionalobleth-Job-name prefix used to identify jobs owned by this gateway instance

See Slurm Provisioning for the full setup guide.