Modular Deploy

obleth ships as a small core plus optional components. Both the Docker Compose stack and the Helm chart let you turn pieces on and off so the same artifacts serve a one-command demo and a lean production install.

Core vs optional

Core (always on): Postgres (config source of truth + audit), Redis (hot cache + token budgets + response cache), ClickHouse (usage ledger), the obleth data plane, and the control-plane dashboard.

Optional:

HAProxy is a normal part of a gateway of this class. Keep it for self-contained demos; in production you'll usually replace it with a managed load balancer (Compose edge profile) or a Kubernetes Ingress (Helm ingress.enabled).

Docker Compose profiles

Profiles gate the optional services. Core services have no profile and always start.

# Lean core only (point obleth at a real upstream, see below)
docker compose -f deploy/docker/docker-compose.yml up

# Full dev/demo stack
docker compose -f deploy/docker/docker-compose.yml \
  --profile benchmark --profile edge --profile observability up

Or default the profiles on for dev by setting COMPOSE_PROFILES in deploy/docker/.env (copy from .env.example):

COMPOSE_PROFILES=benchmark,edge,observability

The compressor profile builds an image that bakes a ~600 MB ONNX model, so its first build is slow and needs network access. It stays off unless you list it in COMPOSE_PROFILES. Enabling it also requires pointing obleth at the sidecar with OBLETH_COMPRESSOR_URL=http://compressor:8080; without that URL the feature stays off even when the container is running. See Model boons → Neural prose scoring.

Pointing at a real upstream

obleth no longer hard-depends on the benchmark fixture backend. Set the upstream in .env:

OBLETH_UPSTREAM_BASE_URL=https://my-vllm-or-aibrix-endpoint/v1

Run without the benchmark profile and obleth routes straight to your real inference endpoint. Per-model overrides still take precedence via the model registry.

Shared settings

.env.example documents the keys that must agree across services:

• OBLETH_ADMIN_TOKEN — shared by obleth and control-plane.
• OBLETH_UPSTREAM_BASE_URL — default upstream for the data plane.
• OBLETH_OTEL_ENDPOINT — OTLP/HTTP collector base URL (empty = tracing off).
• DASHBOARD_ADMIN_EMAIL / DASHBOARD_PASSWORD / DASHBOARD_SESSION_SECRET — break-glass admin login. Add OIDC_PROVIDERS (and BETTER_AUTH_URL) to enable SSO.

Helm

The chart bundles dependencies for a self-contained install and lets you swap any of them for an external/operator-managed endpoint.

Dependency toggles

postgres:
  enabled: false                 # use an external/operator Postgres
  external:
    url: postgres://user:pass@pg-host:5432/obleth
redis:
  enabled: false
  external:
    url: redis://redis-host:6379
clickhouse:
  enabled: false
  external:
    url: http://clickhouse-host:8123
benchmarkBackend:
  enabled: false                 # production: no bundled benchmark fixture backend
obleth:
  upstreamBaseUrl: https://my-vllm-endpoint/v1

When a dependency is enabled: false, its external.url is required — the chart wires obleth at that URL instead of an in-chart service. Leaving it blank fails the render with a clear message rather than booting a misconfigured pod.

After helm install, register models and mint tenant keys — the chart does not do this for you. See Installation — post-install steps.

Ingress (HAProxy's k8s equivalent)

ingress:
  enabled: true
  className: nginx
  host: obleth.example.com
  servicePort: 8080
  tls:
    - secretName: obleth-tls
      hosts: [obleth.example.com]

Tracing

obleth:
  otelEndpoint: http://my-collector:4318

Dashboard auth

controlPlane:
  dashboardAdminEmail: admin@example.com
  dashboardPassword: change-me
  dashboardSessionSecret: a-long-random-secret

Neural prose compression sidecar

Off by default. Enable it to run the compressor scoring service and wire obleth at it automatically (OBLETH_COMPRESSOR_URL is injected for you). The sidecar is stateless, so it scales horizontally.

Scale with small pods, not fat ones. The scorer is CPU-only (no GPU needed) but CPU-hungry, and its throughput does not grow with pod size: a single inference saturates at about 4 cores (measured on the default model — 4 cores ≈ 105 sentences/s, 8/16/24 cores barely more). Giving a pod 32 cores wastes 28 of them. Because each pod also holds the ~900 MB model resident in RAM, very small pods multiply memory. So the sweet spot is ~4 cores per pod, then add replicas for aggregate throughput. Keep numThreads equal to the CPU limit so each replica cleanly owns its cores, and turn on autoscaling under variable load.

compressor:
  enabled: true
  # Aggregate throughput comes from replicas, not big pods. On a CPU-rich cluster,
  # e.g. replicas: 24 with 4 cores each (~2500 sentences/s over ~96 cores) beats a
  # handful of 32-core pods using the same or more cores.
  replicas: 4
  numThreads: 4          # keep small (~4); == limits.cpu
  resources:
    requests: { cpu: "4", memory: 1536Mi }
    limits:   { cpu: "4", memory: 2Gi }
  autoscaling:
    enabled: true
    minReplicas: 4
    maxReplicas: 24
    targetCPUUtilizationPercentage: 70

CPU-based autoscaling is only meaningful because resources.limits.cpu is set — leave resources empty and the scorer grabs node cores uncapped, starving neighbours and making the HPA meaningless.

The image bakes a ~600 MB ONNX model, so plan for slow pulls on first deploy. If the sidecar is unreachable, obleth falls back to the built-in heuristic prose pass — the request never fails. See Model boons → Neural prose scoring.

Values matrix (quick reference)