Docker Compose

The stack runs obleth on one host for development, evaluation or staging. Run every command from deploy/docker/ so Compose loads deploy/docker/.env. What each variable does, and how Compose sets it, is in Environment variables.

Start the stack

cd obleth-gateway/deploy/docker
cp .env.example .env          # then change every password and secret
docker compose up -d --build

docker compose pull && docker compose up -d runs the published images from ghcr.io/thediymaker/obleth-gateway/ instead of building. Pin a release in .env:

OBLETH_VERSION=v2.4.0
OBLETH_COMPRESSOR_VERSION=v2.4.0   # only with the compressor profile

OBLETH_VERSION defaults to latest; dev tracks edge builds. The compression sidecar has its own tag because edge builds skip it.

Profiles

Core services always start. .env.example sets COMPOSE_PROFILES=benchmark,edge,observability; leave it empty for the core services only, or add one for a single command with docker compose --profile slurm up -d.

Services

First-party images are under ghcr.io/thediymaker/obleth-gateway/. The benchmark backend runs with a 20 ms time to first token, 3 ms per token and 256 concurrent requests. postgres, redis and clickhouse have healthchecks every 3 s; obleth, control-plane and compressor use the healthcheck built into their image (/health, /login and /health). Ports are listed in Endpoints and ports.

Required variables

Compose stops with an error when any of these is unset or empty. .env.example has development values for all of them; replace each before a shared deployment.

openssl rand -hex 16       # POSTGRES_PASSWORD, REDIS_PASSWORD, CLICKHOUSE_PASSWORD
openssl rand -hex 32       # OBLETH_ADMIN_TOKEN, DASHBOARD_SESSION_SECRET
openssl rand -base64 32    # OBLETH_ENCRYPTION_KEY (optional, recommended)

Paste the values into .env; Compose does not run commands there. Keep REDIS_PASSWORD URL-safe, since it is placed in redis://:PASSWORD@redis:6379. After changing .env, recreate the service, for example docker compose up -d obleth.

Variables Compose does not forward

The obleth service hardcodes these, so a value in .env has no effect:

OBLETH_UPSTREAM_POOL_IDLE_SECS, OBLETH_UPSTREAM_TCP_KEEPALIVE_SECS, OBLETH_PORT_SPAN and the provisioner's OBLETH_PROVISIONER_* variables other than RESTART_AFTER_FAILURES are not passed, so their defaults apply.

To change any of them, override the service's environment: in a second file:

# deploy/docker/compose.tuning.yml
services:
  obleth:
    environment:
      OBLETH_DEFAULT_MODEL_MAX_IN_FLIGHT: "64"
      OBLETH_FAIRSHARE_HISTORY_SECS: "7200"

docker compose -f docker-compose.yml -f compose.tuning.yml up -d obleth

Or set COMPOSE_FILE=docker-compose.yml:compose.tuning.yml in .env (; on Windows) so every command includes it.

The obleth service has stop_grace_period: 345s: the 300 s OBLETH_SHUTDOWN_GRACE_SECS drain, 20 s for the telemetry flush, 10 s for the trace exporter, and a margin. If you raise the grace period, raise stop_grace_period by the same amount in the override file. See shutdown and timeouts.

Volumes

Read-only bind mounts supply clickhouse/prometheus.xml and clickhouse/system-logs.xml (ClickHouse's own logs at information level, with a week of its system log tables kept), haproxy.cfg, dex/config.docker.yaml, prometheus.yml and the Grafana provisioning and dashboards directories.

Standalone datastores

datastores.compose.yml runs Postgres, Redis and ClickHouse on their own, as a durable tier for a gateway that runs elsewhere (a Helm install with the bundled datastores off, or a binary):

cd deploy/docker
cp datastores.env.example datastores.env   # change every password
docker compose --env-file datastores.env -f datastores.compose.yml up -d

The project is named obleth-datastores and uses the same images and volume names. Ports publish on BIND_ADDR (default 127.0.0.1): 5432, 6379, 8123 and 9000. Set it to a private interface address to reach them from another host, behind a firewall. Redis runs with --appendfsync everysec and a password. Point the gateway at it:

OBLETH_DATABASE_URL=postgres://obleth:POSTGRES_PASSWORD@DB_HOST:5432/obleth
OBLETH_REDIS_URL=redis://:REDIS_PASSWORD@DB_HOST:6379
OBLETH_CLICKHOUSE_URL=http://DB_HOST:8123
OBLETH_CLICKHOUSE_USER=obleth
OBLETH_CLICKHOUSE_PASSWORD=CLICKHOUSE_PASSWORD
OBLETH_CLICKHOUSE_DB=obleth

Stop the stack

docker compose down