Redis Layout

Redis is obleth's hot cache and the real-time token budget store. All Redis data is derivable from Postgres and can be rebuilt on cache miss.

Key patterns

The three config-cache entries (obleth:key:, obleth:model:, obleth:mcp:) carry no Redis expiry. They are written when the gateway warms its cache from Postgres at startup and on every Management API mutation, and are deleted explicitly on invalidation. The 300-second TTL applies to the in-process moka layer in front of them, which bounds how long a pod can serve a stale value if it misses an invalidation message.

The obleth:provisioner:* keys are shared state rather than per-pod, so the dashboard reads the same heartbeat no matter which gateway replica served the provisioner's poll. They expire together when the provisioner stops, which is how the dashboard distinguishes a running provisioner from a stopped one. See Slurm Provisioning.

ResolvedKey structure

{
  "key_id": "661f9500-...",
  "tenant_id": "550e8400-...",
  "tenant_name": "chatbot",
  "fairshare_group": "default",
  "group_weight": 100,
  "weight": 100,
  "tokens_per_minute": 50000,
  "max_in_flight": null,
  "disabled": false,
  "status": "active",
  "timezone": "UTC",
  "active_from": null,
  "active_until": null,
  "weekly_windows": null,
  "budget_tokens": null,
  "budget_cost_usd": null,
  "budget_period": null,
  "budget_started_at": null,
  "key_budget_tokens": null,
  "key_budget_cost_usd": null,
  "key_budget_period": null,
  "key_budget_started_at": null,
  "allowed_models": null,
  "internal": false,
  "tracing_enabled": false,
  "guardrails_policy": null,
  "compression_policy": null,
  "synthetic": false
}

The hot path reads only this blob; it carries everything admission, budgeting, access windows, the model allowlist, and the guardrails/compression policies need without a relational lookup. internal marks gateway-owned traffic (health probes); synthetic marks a tenant whose requests are recorded as benchmark traffic. Both are excluded from usage and cost stats by default.

Token budget hash

The obleth:budget:{tenant_uuid} key is a Redis hash holding a continuously refilling token bucket, not a fixed window:

On each request, a Lua script atomically:

1. Adds (now - ts) * tokens_per_minute / 60000 tokens, clamped to the burst ceiling.
2. Subtracts the estimated token count if the balance covers it.
3. Writes back tokens and ts, sets a 600-second expiry, and returns (allowed, remaining).

A tenant with a term budget takes a combined script instead. It checks the cumulative term cap first, so an over-budget request never reserves bucket tokens it would have no way to refund, then reserves. It returns 1 (reserved), 0 (per-minute bucket exhausted, 429), or -1 (term budget exhausted, 403).

After the response completes, the true cost is reconciled against the reservation and the difference is refunded or charged.

Term usage

For tenants with a term budget, obleth:term_usage:{tenant_uuid} tracks cumulative tokens and cost for the current period (lifetime, monthly, or term). The hash also stores the period it was accumulated under; when that value changes the script clears the counters, so starting a new term resets usage. obleth reads it before admission and adds the reconciled true cost after each stream; when either configured cap is met the data plane returns 403.

Response cache

obleth:cache:{sha256} stores the cached response body. The key is SHA-256(model || 0x00 || raw_request_body), computed on the client's body before any boon rewrites it, so two requests match only when both the model and the exact body match. TTL is set per model via cache_ttl_secs; responses over 512 KiB are not cached. See Response Cache.

Pub/sub invalidation

Channel: obleth:invalidate

When a config change is made via the Management API, obleth publishes a plain string target to this channel. Every subscribed pod drops the matching entry from its in-process moka cache, so the next request re-reads the fresh value from Redis.

Cache miss behavior

Lookups check the in-process moka cache first (300-second TTL), then Redis. The data plane does not read Postgres on the hot path: Redis is populated by the Management API on every mutation and re-warmed from Postgres when a gateway process starts, so a Redis restart is repaired by restarting a gateway pod (or by the next config write) rather than by a per-request fallback.