Install on Kubernetes

The chart runs the gateway and the dashboard, and either runs Postgres, Redis and ClickHouse itself or connects to ones you run. Image tags default to v plus the chart's appVersion, so chart 2.4.0 runs the v2.4.0 images.

Install

1. 1

   Pick a datastore scenario

   Choose one of the scenarios below and download its values file from deploy/k8s/obleth/examples/ for the release you install. Edit obleth.upstreamBaseUrl and the Ingress host.

2. 2

   Install with the required secrets

   helm install obleth oci://ghcr.io/thediymaker/charts/obleth --version 2.4.0 \
     -n obleth --create-namespace -f values-persistent.yaml \
     --set obleth.adminToken="$(openssl rand -hex 32)" \
     --set obleth.encryptionKey="$(openssl rand -base64 32)" \
     --set postgres.password="$(openssl rand -hex 16)" \
     --set redis.password="$(openssl rand -hex 16)" \
     --set clickhouse.password="$(openssl rand -hex 16)" \
     --set controlPlane.dashboardPassword="$(openssl rand -hex 16)" \
     --set controlPlane.dashboardSessionSecret="$(openssl rand -hex 32)"
   

   Without these values the chart refuses to render. redis.password goes into the Redis URL unescaped, so it must not contain @ : / ? # % or spaces; hex is always safe. To install from a checkout, use ./deploy/k8s/obleth instead of the OCI reference. Keep the generated values: the next helm upgrade needs them, or use pre-created Secrets.

3. 3

   Check it

   kubectl -n obleth port-forward svc/obleth-obleth 9180:9180
   curl -s http://localhost:9180/api/v1/health    # ok
   

   Services are named after the release: <release>-obleth (ports 8080, 9180, 9091) and <release>-control-plane (port 3000).

4. 4

   Add tenants, keys and models

   The chart creates none. Until you do, requests return 401 (no key) or 404 (no model). See After install.

Choose where the datastores live

Bundled datastores each run as one Deployment with the Recreate strategy. Their PVCs use the cluster's default StorageClass unless you set <store>.persistence.storageClass; if a datastore pod stays Pending, its PVC didn't bind. They have no backups and no failover. The bundled Redis runs with a 512Mi limit, maxmemory 400mb and volatile-lru, so only entries with a TTL are evicted. The bundled ClickHouse keeps 7 days of its own system logs (clickhouse.systemLogRetentionDays); give an external ClickHouse a retention of its own.

External datastores are set with <store>.enabled: false and <store>.external.url; the chart refuses to render without the URL. ClickHouse still needs clickhouse.user and clickhouse.password. Redis must present a single primary (a standalone server or a managed failover endpoint): the gateway runs multi-key scripts, so clustered Redis is not supported, and it must have a password. For Postgres and ClickHouse, operators such as CloudNativePG and the Altinity ClickHouse operator give you backups and failover. See Datastores for what each one holds.

Production

Production keeps real credentials out of values files and shell history by pointing the chart at Secrets you create, with obleth.existingSecret and controlPlane.existingSecret. The chart then renders no Secret of its own, so the Secret must hold every key the gateway reads:

kubectl -n obleth create secret generic obleth-secrets \
  --from-literal=OBLETH_ADMIN_TOKEN="$(openssl rand -hex 32)" \
  --from-literal=OBLETH_DATABASE_URL="postgres://obleth:PASS@pg-rw.db.svc:5432/obleth" \
  --from-literal=OBLETH_REDIS_URL="redis://:PASS@redis.db.svc:6379" \
  --from-literal=OBLETH_CLICKHOUSE_PASSWORD="PASS" \
  --from-literal=OBLETH_ENCRYPTION_KEY="$(openssl rand -base64 32)" \
  --from-literal=OBLETH_API_KEY_PEPPER="$(openssl rand -hex 32)" \
  --from-literal=OBLETH_SLACK_WEBHOOK_URL="" \
  --from-literal=OBLETH_JWT_ISSUERS=""

SESSION="$(openssl rand -hex 32)"
kubectl -n obleth create secret generic obleth-control-plane \
  --from-literal=DASHBOARD_ADMIN_EMAIL="admin@example.org" \
  --from-literal=DASHBOARD_PASSWORD="$(openssl rand -hex 16)" \
  --from-literal=DASHBOARD_SESSION_SECRET="$SESSION" \
  --from-literal=BETTER_AUTH_SECRET="$SESSION" \
  --from-literal=DATABASE_URL="postgres://obleth:PASS@pg-rw.db.svc:5432/obleth" \
  --from-literal=BETTER_AUTH_URL="https://dashboard.example.org" \
  --from-literal=TRUSTED_ORIGINS="" \
  --from-literal=OIDC_PROVIDERS=""

helm install obleth oci://ghcr.io/thediymaker/charts/obleth --version 2.4.0 \
  -n obleth -f values-production.yaml

If a bundled datastore stays enabled next to a pre-created Secret, add its password too: POSTGRES_PASSWORD for Postgres, and REDIS_PASSWORD for Redis with OBLETH_REDIS_URL set to redis://:<password>@<release>-redis:6379. A missing key stops the new pods from starting. In values-production.yaml the external URLs only satisfy rendering; the live ones come from the Secret, so keep them the same. OBLETH_API_KEY_PEPPER can't change once keys are issued. Every key is described in Helm values.

The chart runs the gateway and dashboard pods as non-root with the restricted Pod Security settings, gives the gateway a startup probe, a writable volume for the telemetry spill file and 345 seconds to drain on shutdown, and renders the PodDisruptionBudget only when more than one replica is guaranteed.

Replicas and capacity

Pool sizes, obleth.globalMaxInFlight (default 4096) and per-tenant and per-key caps hold across all gateway replicas together: the replicas share slots in Redis. Adding replicas or turning on the HPA (hpa.enabled) adds gateway capacity, not backend concurrency. See Fairshare engine and Scaling.

Optional components

After install

1. 1

   Reach the management API

   Port-forward svc/<release>-obleth port 9180, or call it from inside the cluster.

2. 2

   Create a tenant and a key

   Follow First tenant and key. Each client needs its own sk_… key or an identity token.

3. 3

   Register your models

   In the dashboard's Models page, the API or a manifest; see Model registry. In-cluster addresses such as *.svc.cluster.local are allowed by default. With OBLETH_BLOCK_PRIVATE_NETWORKS=1, list your pod network in obleth.allowedPrivateCidrs; see Security.

4. 4

   Send traffic

   Clients call port 8080 of <release>-obleth, or your Ingress, with Authorization: Bearer <key> or x-api-key.

Reach the dashboard

The chart's Ingress covers only the data plane, so give the dashboard one of its own, pointing at <release>-control-plane port 3000:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: obleth-dashboard
  namespace: obleth
spec:
  ingressClassName: nginx        # your Ingress class
  rules:
    - host: dashboard.example.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: obleth-control-plane
                port: { number: 3000 }

Set controlPlane.betterAuthUrl to the address people open, such as https://dashboard.example.org, and list every other address they use (a LAN IP, the other scheme) in controlPlane.trustedOrigins, comma-separated. Sign-in from an address in neither is refused with Invalid origin. Sign in with controlPlane.dashboardAdminEmail (default admin@example.com) and controlPlane.dashboardPassword.

Without an Ingress, port-forward the dashboard's Service:

kubectl -n obleth port-forward svc/obleth-control-plane 3000:3000

Then open http://localhost:3000, after adding it to controlPlane.trustedOrigins (or making it controlPlane.betterAuthUrl). Port-forwarding to the dashboard works from chart 2.1.0, which sets the dashboard to listen on all interfaces; in earlier charts it listened only on its pod's IP and the forward didn't connect.

Upgrade

Use helm upgrade --install against the same chart source you installed from, with your own -f values file. Values in the chart's own values.yaml belong to that source. When a value in the chart-rendered gateway Secret changes, the gateway, Redis and ClickHouse pods roll on their own. After changing a pre-created Secret or the dashboard's Secret, run kubectl rollout restart on the affected Deployments yourself. Release-specific steps are in Upgrades, and the Secret keys 2.0.0 added in Upgrading from v1.x.

compressor:
  enabled: true
  replicas: 4
  numThreads: 4
  resources:
    requests: { cpu: "4", memory: 1536Mi }
    limits:   { cpu: "4", memory: 2Gi }
  autoscaling:
    enabled: true
    minReplicas: 4
    maxReplicas: 24
    targetCPUUtilizationPercentage: 70