OBLETHdocs

One key, many people

Let an app that serves many people through one key, such as a chatbot, have each of those people wait in line on their own.

2 min readUpdated for v2.4.0Edit on GitHub

ForA chatbot or other app with one key and many users
Turn it onThe key's panel › Per-user fairshare
The app sendsx-obleth-end-user, or the request's user
Set byAn administrator, one key at a time

A key's requests wait in the order they arrive. When an app sends everyone's requests through one key, a person with a long backlog holds up everyone else on that key. With per-user fairshare on, each person the app names waits in line on their own, as if they held their own key.

Turn it on

  1. 1

    Give the app its own tenant

    On Tenants, add a tenant for the app and set its weight: its share of each busy model against the other tenants. See How fairshare works.

  2. 2

    Turn on per-user fairshare on its key

    Open the key on API keys, turn on Per-user fairshare, and save.

  3. 3

    Have the app name each person

    Send a stable id for the person on every request, in the x-obleth-end-user header or the request's user field. On /v1/messages, Anthropic's metadata.user_id works too. The header wins when both are sent.

curl https://gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer $OBLETH_KEY" -H "Content-Type: application/json" \
  -H "x-obleth-end-user: user-4821" \
  -d '{"model": "qwen3-8b", "messages": [{"role": "user", "content": "Hello"}]}'

What changes

  • Each person gets their own place in the queue inside the key's tenant. The key's Weight and Max in flight apply to each person: a max in flight of 2 lets each person run two requests at a time in each model's pool.
  • The tenant's weight, limits and budget still cover the whole app, and the key's budget, usage and cost stay on the key.
  • A request that names no one shares the key's own place, as before.
  • The header is not sent to the model's backend. The user field is passed on unchanged.

Ids are compared exactly after surrounding spaces are trimmed, and only the first 128 characters count.

Turn it on only for an app you trust to name people honestly. A caller that sends a new name on each request would otherwise jump ahead of the tenant's other keys.

See it working

On Fairshare, each person appears in the key rows as the key's name and the person, such as chatbot · user-4821. The request ledger records the person in the end_user column of the ClickHouse usage table.

Through the Management API

The setting has its own route, because PUT /api/v1/keys/{id} replaces every field and would otherwise turn it off whenever a key is edited without it.

curl -X PUT http://localhost:9180/api/v1/keys/$KEY_ID/end-user-fairshare \
  -H "Authorization: Bearer $OBLETH_ADMIN_TOKEN" -H "Content-Type: application/json" \
  -d '{"end_user_fairshare": true}'

It returns 204, and the key's end_user_fairshare field shows the current value. Replace… keeps it, and backups carry it.

How a person's place is kept

The gateway derives a queue id from the key and the person, so a person keeps their place across requests and gateway replicas, and people behind two different keys never share one. Like a key, a person's place is forgotten once they have nothing running or waiting. For people who need their own caps and budgets, use identity tokens instead: each person then gets a key of their own.