Conversations and sessions
Group the requests of one conversation under a session id, so you can follow it in Request logs, sum its cost and keep it on one backend replica.
2 min readUpdated for v2.4.0Edit on GitHub
x-obleth-session-id: <your id> on each requestsession:<id>A chat conversation is many requests, one per turn. The gateway gives each request a session id so the turns can be grouped afterwards. You don't have to change your client for this, but a client that sends its own id controls the grouping exactly.
Send a session id
Send the same id on every request of a conversation:
curl http://localhost:8088/v1/chat/completions \
-H "Authorization: Bearer $OBLETH_KEY" \
-H "Content-Type: application/json" \
-H "x-obleth-session-id: conv-abc123" \
-d '{"model": "qwen3-235b", "messages": [{"role": "user", "content": "continue our chat"}]}'
The gateway takes the first of these that is present and not empty:
| Order | Source |
|---|---|
| 1 | x-obleth-session-id header |
| 2 | x-session-id header |
| 3 | session_id in the JSON body |
| 4 | metadata.session_id in the JSON body |
| 5 | prompt_cache_key in the JSON body (OpenAI's prompt cache key) |
| 6 | Derived by the gateway (see below) |
Values are trimmed and cut to 200 characters. The OpenAI user field is not a session source; it names an end user, not a conversation. Session headers are forwarded to the backend like any other header, and a body session_id stays in the body.
On /v1/messages the Anthropic metadata object is not carried over, so use a header there. Claude Code's own x-claude-code-session-id header is not read as a session id.
Find a session in Request logs
Open a request in Request logs. Its panel shows Session and Session came from: the client or derived by the gateway. Choose This session to list every request with that id, or type session:<id> in the search box.
The id is also on each usage row in ClickHouse (session_id and session_id_source), so you can sum a conversation's tokens and cost; see ClickHouse usage. GET /api/v1/usage/logs filters on session_id. On traces the root span carries session.id and session.id.source; see Observability.
Derived ids
With no id from the client, the gateway hashes the tenant, the leading system or developer messages and the first user message. That opening is resent unchanged on every turn, so the id stays the same for the whole conversation. Up to 8 KiB of the system and developer text is hashed, and up to 8 KiB of the first user message on top, so agents that share one long system prompt are still told apart by their first message. Requests with no messages, such as embeddings, get no derived id.
Derivation is on by default. Set OBLETH_SESSION_ID_DERIVATION=false to record only ids clients send.
Two separate conversations that open with the same system prompt and first message get the same derived id. When that matters, as with agents or shared front ends, send your own id.
Keep a session on one replica
Set a model's endpoint selection mode to session_hash and each session goes to the same backend endpoint, chosen by a consistent hash of its id, so it keeps hitting the replica whose prompt cache is already warm. A request with no session id falls back to load balancing. See Reliability and failover.
When agent sessions compete for too few slots, session affinity also lets a warm session's next step go first, so its prompt cache is still there when it runs.