Redis layout

Redis holds the data plane's hot state: resolver entries for keys, models, and MCP servers, budget counters, shared fairshare slots, and a few caches. Postgres stays the source of truth for configuration; see Postgres schema.

Requirements

• A password. Redis holds API keys, so an open instance lets anyone who reaches it mint keys. Connect with OBLETH_REDIS_URL=redis://:PASSWORD@host:6379. The bundled Compose and Helm Redis always run with --requirepass (REDIS_PASSWORD, Helm redis.password).
• One primary. Redis Cluster and sharded keyspaces are not supported: the budget and slot scripts touch several keys in one call. A managed endpoint that fails over behind one address is fine.
• Timeouts. Each command waits at most OBLETH_REDIS_RESPONSE_TIMEOUT_MS (250) and each connect OBLETH_REDIS_CONNECT_TIMEOUT_MS (2000). A failed budget call follows OBLETH_FAIL_OPEN; a failed key lookup returns 503.

redis-cli -a "$REDIS_PASSWORD" --no-auth-warning --scan --pattern 'obleth:model:*'

Key prefixes

<scope> is a tenant id, or a key id for a key-level budget. A TTL of 0 for the response cache, compression originals, or query cache writes nothing.

• Response cache. <sha256> covers the tenant id, model name, and raw request body, so no entry reaches another tenant (the compression key carries the tenant id too). Bodies over 512 KiB are not cached. See Response cache.
• Query cache. The hash covers the embedding model and the query text. A read error, or a length not a multiple of 4 bytes, counts as a miss.
• Daily counters. MCP calls through /mcp/<name> count as direct, calls from the gateway tool loop as tool_calls. GET /api/v1/stats/daily (kind=mcp or knowledge, days 1–40, default 7) reads them.
• Provisioner keys. The gateway writes them each time the provisioner fetches its Slurm settings, so every replica reports the same status. See Slurm provisioning.
• Video jobs live in Postgres (video_jobs), not Redis, because a key with a TTL can be evicted mid-render.

ResolvedKey

Everything admission needs is in one JSON value, so the request path does no relational lookup:

{
  "key_id": "661f9500-...",
  "tenant_id": "550e8400-...",
  "tenant_name": "chatbot",
  "fairshare_group": "default",
  "group_weight": 100,
  "weight": 100,
  "tokens_per_minute": 50000,
  "max_in_flight": null,
  "disabled": false,
  "status": "active",
  "timezone": "UTC",
  "active_from": null,
  "active_until": null,
  "weekly_windows": null,
  "budget_tokens": null,
  "budget_cost_usd": null,
  "budget_period": null,
  "budget_started_at": null,
  "key_budget_tokens": null,
  "key_budget_cost_usd": null,
  "key_budget_period": null,
  "key_budget_started_at": null,
  "key_weight": 100,
  "key_max_in_flight": null,
  "end_user_fairshare": false,
  "allowed_models": null,
  "internal": false,
  "tracing_enabled": false,
  "guardrails_policy": null,
  "compression_policy": null,
  "synthetic": false
}

weight and max_in_flight are the tenant's; key_weight and key_max_in_flight are the key's share and cap inside each model's pool. end_user_fairshare gives each end user the key names their own place in those pools. internal marks gateway-owned traffic such as health probes; synthetic marks benchmark traffic.

Budget scripts

Budget changes run as Lua scripts, so they are atomic across replicas.

The bucket refills at tokens_per_minute / 60000 tokens per millisecond. An estimate larger than the whole bucket is admitted when the bucket is full and runs into debt down to -capacity. The term check runs first, so a request refused for its term budget holds no bucket tokens.

A reservation that is never settled, for example after a crash, goes when its 10-minute key expires. All of a scope's reservations expire together, so brief over-admission is possible, bounded by concurrency times the estimate. Commits are always exact, and a failed upstream call is settled unbilled. See Quotas and rate limits.

Shared fairshare slots

With OBLETH_FAIRSHARE_SHARED_SLOTS on (the default) and more than one replica live, every fairshare limit is a set of cluster-wide slots. See Multiple gateway replicas.

• Replica set (obleth:gateway:replicas): one member per process, its host name plus a random suffix, scored by heartbeat expiry on the Redis clock. Each replica heartbeats every OBLETH_FAIRSHARE_REPLICA_HEARTBEAT_SECS (5); a member expires after OBLETH_FAIRSHARE_REPLICA_TTL_SECS (15, at least twice the interval).
• Totals (obleth:fairshare:slots): g for every pool together, p|<pool>, t|<pool>|<tenant>, and k|<pool>|<key>, plus wait markers w|<pool> and w (global) holding the Redis time until which a replica waits, and wa|<pool>, until which a request there has waited too long for slots to be held (session affinity).
• Holdings (obleth:fairshare:held:<instance>): the same count fields for one replica. Every script applies a holdings change to the totals too, so the totals are always their sum. Counts at zero are deleted.
• Held slots (session affinity): each obleth:fairshare:hold:<id> holds the count fields of one slot kept for a session's next step, plus h|<pool>, which the totals also count, so a held slot still counts against every limit. <id> is the pool id, |, the key id, / and the session id. Their ids are in obleth:fairshare:holds, scored by expiry on the Redis clock, and the totals are the sum of the replicas' holdings and the held slots.

When a release, reconcile, or reclaim frees a slot in a pool with a current wait marker, the script publishes the pool id on obleth:fairshare:released and waiting replicas retry once. An uncontended fleet publishes nothing. After a resubscribe a replica retries every pool, and a random 100–250 ms retry covers other lost messages.

The replica set and slot hashes carry no TTL, so volatile-lru never evicts them: an evicted set would drop every replica's count to 1.

Invalidation channel

Management API writes publish a target on obleth:invalidate. Each replica drops the matching entry from its in-process cache (300-second TTL in front of Redis).

After a resubscribe, a replica drops its whole in-process cache and re-warms Redis, since messages sent while it was disconnected are lost.

Rebuilding from Postgres

Resolver entries have no TTL, so the gateway keeps them in step with Postgres four ways:

1. Startup. Each gateway pushes every key, model, and MCP server.
2. Every write. A Management API change updates or deletes the entry and publishes an invalidation. A delete whose eviction fails returns 502.
3. Re-warm. Every OBLETH_REDIS_REWARM_SECS (300, at most 86400; 0 turns the timer off) and after every Redis reconnect, each gateway pushes a fresh snapshot and prunes stale entries.
4. Resync. POST /api/v1/resync, or Reconcile cache under Settings › Data and backups, runs the same push and prune on demand and returns counts per namespace.

The prune SCANs obleth:key:*, obleth:model:*, and obleth:mcp:* and deletes what the snapshot doesn't list, so a revoked key stops resolving. It then re-reads Postgres and restores anything created during the pass. The request path never reads Postgres: a key missing from Redis returns 401 until the entry is written back.

Eviction

The bundled Helm Redis runs with --maxmemory (redis.maxmemory, default 400mb, under a 512Mi limit) and --maxmemory-policy volatile-lru, so only keys with a TTL are evicted. Resolver entries, committed term usage, the replica set, and the slot hashes are never evicted. When memory is full and nothing is left to evict, Redis rejects writes, which the gateway treats as a Redis failure under OBLETH_FAIL_OPEN. The bundled Compose Redis sets no memory limit. On your own Redis, use a volatile-* policy or noeviction: an allkeys-* policy can evict resolver entries, which then return 401 until the next re-warm.