67 docs indexed

Helm Values

Full reference for deploy/k8s/obleth/values.yaml with defaults and descriptions.

The Helm chart lives in deploy/k8s/obleth/. Install with:

helm install obleth ./deploy/k8s/obleth \
  --namespace obleth \
  --create-namespace \
  -f my-values.yaml

Images

image:
  obleth: ghcr.io/thediymaker/obleth-gateway/obleth
  benchmarkBackend: ghcr.io/thediymaker/obleth-gateway/benchmark-backend
  controlPlane: ghcr.io/thediymaker/obleth-gateway/control-plane
  provisioner: ghcr.io/thediymaker/obleth-gateway/obleth-provisioner
  tag: ""                     # empty -> v{appVersion} from Chart.yaml
  pullPolicy: IfNotPresent

Leaving tag empty pins every image to the chart's own appVersion, so a chart release and the images it runs stay in lockstep. Override it only to test a specific tag (main tracks unreleased edge builds). The compression sidecar carries its own compressor.image block.

Gateway

obleth:
  replicas: 1
  globalMaxInFlight: 256
  failOpen: true
  adminToken: ""              # REQUIRED — chart errors if empty (openssl rand -hex 32)
  encryptionKey: ""           # recommended — base64 of 32 bytes (openssl rand -base64 32)
  apiKeyPepper: ""            # optional — openssl rand -hex 32
  existingSecret: ""          # PRODUCTION — name of a pre-created Secret (see below)
  allowedPrivateCidrs: ""     # only needed with OBLETH_BLOCK_PRIVATE_NETWORKS=1 (strict SSRF)
  upstreamBaseUrl: ""         # empty -> defaults to the in-chart benchmark fixture backend
  otelEndpoint: ""            # set to enable tracing, e.g. "http://jaeger:4317"
  slackWebhookUrl: ""         # optional Slack incoming-webhook alerts (keep in a Secret)
  slackAlertMinIntervalSecs: 300
  modelHealthEnabled: true
  modelHealthIntervalSecs: 900
  modelHealthTimeoutSecs: 30
  modelHealthRetentionDays: 30
  usageRetentionDays: 180
  resources:
    requests: { cpu: "250m", memory: "256Mi" }
    limits: { cpu: "2", memory: "1Gi" }

Note:

replicas defaults to 1 on purpose. Fairshare admission state is per-process: each replica runs its own scheduler and enforces globalMaxInFlight independently, so N replicas admit up to N x globalMaxInFlight concurrently. For redundancy, set replicas: N and divide globalMaxInFlight by N — and keep hpa.enabled: false, since replicas cannot lend each other idle capacity.

adminToken, the datastore passwords, and the dashboard secrets have no default values. The chart uses Helm's required function, so helm install/ template fails fast with a clear message if any are missing. Supply them via --set, a local untracked values file, or a secrets manager — do not commit production credentials to git.

Pre-created Secrets (existingSecret)

For production, point the chart at a Secret you created out-of-band so real credentials never enter values files or --set/CLI history. When obleth.existingSecret is set, the chart does not render its own Secret and references the named one instead — it must carry every key the chart would have generated:

OBLETH_ADMIN_TOKEN, OBLETH_DATABASE_URL, OBLETH_CLICKHOUSE_PASSWORD,
OBLETH_ENCRYPTION_KEY, OBLETH_API_KEY_PEPPER, OBLETH_SLACK_WEBHOOK_URL

controlPlane.existingSecret works the same way and must carry DASHBOARD_PASSWORD, DASHBOARD_SESSION_SECRET, BETTER_AUTH_SECRET, DATABASE_URL, DASHBOARD_ADMIN_EMAIL, BETTER_AUTH_URL, TRUSTED_ORIGINS, and OIDC_PROVIDERS. See the Production scenario and values-production.yaml.

SSRF and in-cluster upstreams

The default SSRF policy permits private/LAN addresses, so api_base values pointing at in-cluster Services (e.g. *.svc.cluster.local) usually work without setting allowedPrivateCidrs. If you enable strict mode (OBLETH_BLOCK_PRIVATE_NETWORKS=1), list trusted CIDRs here (e.g. 10.0.0.0/8). See Security.

Post-install

Helm does not register models or create tenant API keys. After install, follow Installation — post-install steps.

Autoscaling (HPA)

Off by default, for the reason above: an autoscaled replica count moves the aggregate concurrency limit that reaches your upstream with no config change — scaling 2 to 8 pods raises it fourfold. If you enable it, divide globalMaxInFlight by maxReplicas so the ceiling still holds at full scale-out, and accept that the upstream is under-subscribed below that.

hpa:
  enabled: false
  minReplicas: 1
  maxReplicas: 20
  targetCPUUtilizationPercentage: 70

Security contexts (Pod Security Standard)

Pod- and container-level hardening applied to the stateless workloads the chart builds (obleth data plane, control-plane, benchmark-backend). The defaults satisfy the Kubernetes restricted Pod Security Standard and are on out of the box.

podSecurityContext:
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault
securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop: [ALL]
  runAsNonRoot: true
  # readOnlyRootFilesystem: true   # opt-in; confirm nothing writes the rootfs first

UID is intentionally not pinned — the images already ship distinct non-root users (obleth 10001, control-plane 1000), and runAsNonRoot enforces non-root without hardcoding a UID. The bundled datastores are excluded on purpose: their official images manage their own users (the Postgres entrypoint, for example, must start as root).

Spread and disruption protection

# Spread obleth replicas across nodes. "soft" prefers distinct nodes but still
# schedules on a single-node cluster; "hard" requires distinct nodes (won't
# schedule more replicas than nodes); "" disables.
affinity:
  antiAffinity: soft

# Keep a minimum number of obleth pods available during voluntary disruptions
# (node drains, rolling upgrades). Rendered only when replicas > 1.
podDisruptionBudget:
  enabled: true
  minAvailable: 1

NetworkPolicy

# Restrict the BUNDLED datastore ports (Postgres 5432, Redis 6379, ClickHouse
# 8123/9000) to the obleth data plane only. Off by default. Requires a CNI that
# enforces NetworkPolicy (Calico, Cilium, …); inert otherwise. No effect on
# external datastores, which run outside the chart.
networkPolicy:
  enabled: false

A NetworkPolicy object is rendered per bundled datastore (only for the ones with enabled: true).

Monitoring (Prometheus Operator)

enabled is the only key. The rendered ServiceMonitor scrapes the metrics port on a fixed 5-second interval.

serviceMonitor:
  enabled: false   # set to true if the Prometheus Operator CRDs are installed

Ingress

The chart does not deploy HAProxy. On Kubernetes the edge role is filled by the obleth Service plus this optional Ingress.

ingress:
  enabled: false
  className: ""
  host: obleth.example.com
  servicePort: 8080     # the data plane port on the obleth Service
  annotations: {}
  tls: []               # e.g. [{ secretName: obleth-tls, hosts: [obleth.example.com] }]

The Service publishes three ports: proxy (8080), admin (9180), and metrics (9091). Route the Ingress at 8080 only — never expose 9180.

Postgres

postgres:
  enabled: true           # set false to use an external Postgres
  image: postgres:16
  user: obleth
  password: ""            # REQUIRED — chart errors if empty
  db: obleth
  persistence:
    enabled: true         # PVC-backed storage; false = emptyDir (data lost on restart)
    size: 10Gi
    storageClass: ""      # "" = cluster default StorageClass
    accessMode: ReadWriteOnce
  external:
    url: ""               # set when enabled=false

Redis

redis:
  enabled: true           # set false to use an external Redis
  image: redis:7
  persistence:
    enabled: true         # AOF on a PVC; false = emptyDir (rebuildable cache)
    size: 1Gi
    storageClass: ""
    accessMode: ReadWriteOnce
  external:
    url: ""               # e.g. "redis://my-redis:6379"

ClickHouse

clickhouse:
  enabled: true           # set false to use an external ClickHouse
  image: clickhouse/clickhouse-server:24.8
  db: obleth
  user: obleth
  password: ""            # REQUIRED — chart errors if empty
  persistence:
    enabled: true         # PVC-backed usage ledger; false = emptyDir (history lost)
    size: 20Gi
    storageClass: ""
    accessMode: ReadWriteOnce
  external:
    url: ""               # e.g. "http://my-clickhouse:8123"

Storage scenarios

The bundled datastores support three storage models. See Self-Hosting for the full walkthrough and ready-made values files in deploy/k8s/obleth/examples/.

ScenarioSettingData on restart
Persistent (PVC)persistence.enabled: true (default)survives
Ephemeral (test)persistence.enabled: falselost
Externalenabled: false + external.urlmanaged by you
ProductionExternal datastores + existingSecret, PDB, anti-affinity, NetworkPolicymanaged by you

Datastore Deployments use the Recreate rollout strategy so a new pod never attaches a ReadWriteOnce PVC still held by the old pod.

Benchmark fixture backend (dev only)

benchmarkBackend:
  enabled: true           # disable in production
  ttftMs: 20
  tokenMs: 3
  concurrency: 256

Reaching Slurm compute nodes

Applied to the obleth and provisioner pods only — the control plane never talks to compute nodes. All empty by default, which leaves cluster DNS unchanged. Use these when the pods cannot resolve your cluster's node hostnames, for example when the gateway runs in Kubernetes and the Slurm nodes live on a separate site DNS.

nodeResolution:
  # Static /etc/hosts entries: a node -> IP map.
  hostAliases: []
  #  - ip: "10.0.0.25"
  #    hostnames: ["scgh001", "scgh001.example.edu"]

  # Custom resolver injected into the pods.
  dnsConfig: {}
  #  nameservers: ["10.0.0.10"]
  #  searches: ["sol.rc.example.edu"]
  #  options: [{ name: ndots, value: "2" }]

  # Override dnsPolicy (e.g. "None" when you fully specify nameservers above).
  # Empty keeps Kubernetes' default, which still honors the extra nameservers.
  dnsPolicy: ""

A per-model node hostname → IP override list is also configurable at runtime in the dashboard under Settings → Slurm; see Slurm Provisioning.

Slurm provisioner

Off by default. This only controls whether the reconciler process is deployed — the slurmrestd connection details and the master enable switch are configured at runtime in the dashboard, not here. It runs as a singleton (replicas: 1, Recreate); do not scale it.

provisioner:
  enabled: false
  intervalSecs: 15            # reconcile cadence
  restartAfterFailures: 20    # self-heal threshold, in net failing probe ticks
  logLevel: "info,obleth_provisioner=debug"
  resources:
    requests: { cpu: "50m", memory: "64Mi" }
    limits: { cpu: "500m", memory: "256Mi" }

restartAfterFailures restarts a healthy replica whose Slurm job still reports RUNNING but whose health probe has been failing. The counter decays on every passing probe, so 20 ticks at the default 15-second interval is roughly five minutes of sustained failure, not 20 consecutive misses. Restarts are capped at one per model per tick; 0 disables self-heal. If you pinned this to 3 on an older chart, raise it — that is about 45 seconds of probe flaps before a healthy replica is cancelled.

Compression sidecar

Off by default. The image bakes a roughly 600 MB ONNX model, so plan for slow first pulls. When enabled the chart wires OBLETH_COMPRESSOR_URL on the gateway automatically; the data plane fails open if the sidecar is unreachable.

compressor:
  enabled: false
  replicas: 1
  numThreads: 4               # onnxruntime intra-op threads; keep == limits.cpu
  image:
    repository: ghcr.io/thediymaker/obleth-gateway/obleth-compressor
    tag: ""                   # empty -> v{appVersion}
    pullPolicy: IfNotPresent
  resources:
    requests: { cpu: "4", memory: 1536Mi }
    limits:   { cpu: "4", memory: 2Gi }
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 5
    targetCPUUtilizationPercentage: 70

Scale this horizontally with small pods. A single inference saturates near four cores, so a fat pod buys almost no extra throughput while each replica holds the model resident in memory. Keep numThreads equal to resources.limits.cpu and raise replicas for aggregate throughput. Leaving resources empty in production lets the scorer take node cores uncapped and makes CPU-based autoscaling meaningless.

Control plane

controlPlane:
  enabled: true
  replicas: 1
  dashboardAdminEmail: admin@example.com  # break-glass admin login (email-based)
  dashboardPassword: ""             # REQUIRED — >= 8 chars; chart errors if empty
  dashboardSessionSecret: ""        # REQUIRED — >= 32 chars (openssl rand -hex 32)
  databaseUrl: ""                   # auth tables; defaults to the bundled Postgres
  betterAuthUrl: ""                 # external dashboard URL; used for OIDC redirect URIs
  trustedOrigins: ""                # extra origins better-auth accepts logins from
  oidcProviders: ""                 # JSON array to enable SSO; empty = break-glass only
  existingSecret: ""                # PRODUCTION — pre-created Secret with all control-plane keys
  resources:
    requests: { cpu: "100m", memory: "256Mi" }
    limits: { cpu: "1", memory: "512Mi" }

The dashboard credentials and auth settings are rendered into a dedicated Kubernetes Secret and injected via envFrom, not as plaintext Deployment env values. Set existingSecret to skip that rendered Secret and reference your own (see Pre-created Secrets). The control-plane Deployment also carries /login readiness/liveness probes and the resource requests/limits above.

trustedOrigins is a comma-separated list of extra origins better-auth accepts logins from, on top of betterAuthUrl. Set it when the dashboard is reached by an alternate hostname or a LAN IP; without it, better-auth rejects those logins as invalid-origin. * trusts every origin — private networks only.

Beyond the standard providerId, displayName, discoveryUrl, clientId, clientSecret, and scopes, an oidcProviders entry accepts three optional keys: claims maps a user field to a non-standard claim (e.g. {"email":"preferred_username"}; email, name, and image are mappable), overrideUserInfo re-applies the profile to existing users on each sign-in rather than only at sign-up, and authentication: "basic" is for IdPs that only accept HTTP Basic at the token endpoint. See Dashboard SSO.

Minimal production values.yaml

Note:

This inlines secrets for brevity. For a fully hardened install — pre-created Secrets (existingSecret), anti-affinity, a PodDisruptionBudget, and the restricted security contexts — start from values-production.yaml and the Production scenario.

obleth:
  adminToken: "my-random-32-char-token"
  encryptionKey: "base64-of-32-random-bytes"
  upstreamBaseUrl: "http://aibrix-gateway.aibrix.svc.cluster.local:8080/v1"
  globalMaxInFlight: 64
  # Two pods for redundancy; each enforces globalMaxInFlight on its own, so the
  # aggregate ceiling is 2 x 64. Halve the value if 64 is the real ceiling.
  replicas: 2

postgres:
  enabled: false
  external:
    url: "postgres://obleth:strongpassword@my-rds-endpoint/obleth"

redis:
  enabled: false
  external:
    url: "redis://my-redis-sentinel:6379"

clickhouse:
  enabled: false
  external:
    url: "http://my-clickhouse-cloud:8123"

benchmarkBackend:
  enabled: false

controlPlane:
  dashboardAdminEmail: "admin@example.com"
  dashboardPassword: "my-dashboard-password"
  dashboardSessionSecret: "my-random-session-secret-64-chars"

serviceMonitor:
  enabled: true

ingress:
  enabled: true
  className: nginx
  host: obleth.my-company.com
  tls:
    - secretName: obleth-tls
      hosts:
        - obleth.my-company.com