64 docs indexed

Upgrades

How to upgrade obleth on Kubernetes with Helm: chart version bumps, schema migrations, verification, and rollback.

obleth releases are published in lockstep to GitHub Container Registry (GHCR). A version-matched upgrade is normally just a chart version bump — the chart pins its own matching images, so you do not set image tags by hand.

Published artifacts

ArtifactReferenceVersion form
Helm chart (OCI)oci://ghcr.io/thediymaker/charts/obleth0.3.7 (no v)
Container imagesghcr.io/thediymaker/obleth-gateway/{obleth,control-plane,benchmark-backend}v0.3.7 (with v)

Tag convention. The chart version has no prefix (0.3.7); the images are v-prefixed (v0.3.7). Same release, two forms — don't mix them up.

Pull the published chart and carry your existing release values (secrets, datastore URLs, registry overrides, etc.) forward so only the version changes:

helm upgrade obleth \
  oci://ghcr.io/thediymaker/charts/obleth \
  --version 0.3.7 \
  --namespace obleth-system \
  --reset-then-reuse-values \
  --wait
  • --reset-then-reuse-values resets to the new chart's defaults, then re-applies only the overrides you explicitly set on the previous release. This is what you want for a version bump: chart-managed defaults like the image tag advance to the new release automatically, while your secrets and overrides are preserved.
  • --wait blocks until the new pods pass their health checks and reports failure (without serving traffic from a broken pod) if they don't.

Why not plain --reuse-values? --reuse-values reuses the fully computed values from the last release and ignores the new chart's defaults. Because this chart pins image.tag to its own appVersion, a plain --reuse-values upgrade leaves the old image tag in place — you'd be running the previous binary against the new chart templates. Use --reset-then-reuse-values (Helm 3.14+) for version bumps; reserve plain --reuse-values for re-applying the same chart version. If you're on older Helm, force the tag explicitly with --reuse-values --set image.tag=v0.3.7.

Kubernetes cluster unreachable on a non-standard distro. If Helm fails with Error: UPGRADE FAILED: Kubernetes cluster unreachable: Get "http://localhost:8080/version": dial tcp [::1]:8080: connect: connection refused, it could not find a kubeconfig and fell back to the default localhost:8080 API address. Distros like k0s/k3s keep their admin kubeconfig outside the default path — prefix the same command with KUBECONFIG pointing at it:

KUBECONFIG=/var/lib/k0s/pki/admin.conf helm upgrade obleth \
  oci://ghcr.io/thediymaker/charts/obleth \
  --version 0.3.7 \
  --namespace obleth-system \
  --reset-then-reuse-values \
  --wait

(k3s commonly uses /etc/rancher/k3s/k3s.yaml.)

--set + value reuse caveat. This chart marks obleth.adminToken, datastore passwords, and dashboard secrets as required. A bare helm upgrade ... --set foo=bar (without a reuse flag or -f) resets to chart defaults and fails with obleth.adminToken is required. Always pass --reset-then-reuse-values (or your -f values-prod.yaml) when using --set on an existing release.

Preview first

helm upgrade obleth oci://ghcr.io/thediymaker/charts/obleth --version 0.3.7 \
  -n obleth-system --reset-then-reuse-values --dry-run | grep 'image:'

Confirm the rendered images are the expected …:v0.3.7 before applying.

Schema migrations

Migrations are automatic and idempotent — the first pod on the new version applies schema/postgres/0001_init.sql (create … if not exists, add column … if not exists) on startup. Changes are additive (new nullable columns / tables only), so old and new pods coexist safely during the rolling update. Nothing to run by hand.

Verify

helm history obleth -n obleth-system | tail -2

kubectl -n obleth-system get deploy -o \
  'jsonpath={range .items[*]}{.metadata.name}{"\t"}{.spec.template.spec.containers[0].image}{"\n"}{end}'

kubectl -n obleth-system get pods

The gateway log should end with postgres connected + schema applied, redis connected, clickhouse connected + schema applied, and listening: proxy=…:

kubectl -n obleth-system logs deploy/obleth-obleth -c obleth --tail=25

Rollback

helm rollback obleth <previous-revision> -n obleth-system   # e.g. 18

Schema migrations are not reverted, but because they are additive the older binary simply ignores columns it does not read.

Docker Compose upgrades

# Pull the new images (OBLETH_VERSION pinned in .env or passed inline)
OBLETH_VERSION=v0.3.7 docker compose -f deploy/docker/docker-compose.yml pull

# Recreate containers (brief downtime)
OBLETH_VERSION=v0.3.7 docker compose -f deploy/docker/docker-compose.yml up -d --force-recreate

Docker Compose does not support zero-downtime rolling upgrades natively. For production, use Kubernetes.

Edge / unreleased builds

Every push to main publishes amd64 :main images. To track edge on top of the current chart:

helm upgrade obleth oci://ghcr.io/thediymaker/charts/obleth --version 0.3.7 \
  -n obleth-system --reset-then-reuse-values --set image.tag=main --wait

Switching an existing release to GHCR

If a release was originally installed with a private registry override (image.obleth, image.tag, …), value reuse keeps that registry. To move it back onto the public GHCR images, override the image block explicitly once:

helm upgrade obleth oci://ghcr.io/thediymaker/charts/obleth --version 0.3.7 \
  -n obleth-system --reset-then-reuse-values \
  --set image.obleth=ghcr.io/thediymaker/obleth-gateway/obleth \
  --set image.controlPlane=ghcr.io/thediymaker/obleth-gateway/control-plane \
  --set image.benchmarkBackend=ghcr.io/thediymaker/obleth-gateway/benchmark-backend \
  --set image.tag=v0.3.7 \
  --wait

Subsequent --reuse-values upgrades then stay on GHCR automatically.