Conversations & Sessions

A single chat "conversation" is many HTTP requests — one per turn. obleth gives each request a conversation id so those turns can be grouped after the fact: for routing (keep a session on the replica whose cache is already warm), for cost (sum a whole conversation's spend), and for tracing (see every turn of a conversation as one story). None of this requires the caller to change anything — but a caller that does supply an id gets to control the grouping exactly.

Where the id comes from

For every request obleth resolves a conversation id, in priority order — the first match wins:

Client-supplied ids are trimmed and capped at 200 characters. The dashboard Request Logs view shows the id with a client or derived tag, so you can tell a caller-supplied grouping from an inferred one at a glance.

Derived ids

When no explicit id is present, obleth hashes a seed built from the request: the tenant, the leading system/developer message, and the first user message — the part of a conversation that stays the same on every turn. Because the same leading bytes are replayed each turn, the derived id is stable across the whole conversation without the client sending anything.

Derivation is on by default. Turn it off with OBLETH_SESSION_ID_DERIVATION=0 (or false/no/off) to record only client-supplied ids and leave everything else ungrouped. An explicit client id always takes precedence over derivation.

Derivation is best-effort grouping, not an identity guarantee. Two genuinely separate conversations that open with identical system + first-user text will hash to the same id. When you need exact control — or to keep two such conversations apart — send your own x-session-id.

What the id is used for

Cache-warm routing (session_hash)

Set a model's endpoint_selection_mode to session_hash and obleth pins each conversation to the same upstream replica by a consistent hash of its id, so the session keeps hitting the replica that already holds its KV cache. With a conversation id now available for (almost) every request, that pinning actually sticks instead of silently falling back to load-balancing. A request with no session key at all degrades safely to the weighted load_balance order. See Reliability & Failover.

Grouped spend

The conversation id is written to each ClickHouse usage row (session_id), so a whole conversation's cost and token totals can be summed — not just a single request. See ClickHouse Usage.

Tied-together traces

The id is recorded on spans and exported as the session.id attribute (with session.id.source = client/derived) on OpenTelemetry / Jaeger traces, so every turn of a conversation can be retrieved as one trace group. See Observability.

Using it from a client

You don't have to do anything — derivation groups conversations on its own. To control the grouping explicitly (recommended for agentic or multi-user clients where the leading text may collide), send a stable id per conversation:

curl http://localhost/v1/chat/completions \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -H "x-session-id: conv-abc123" \
  -d '{
    "model": "qwen3-235b",
    "messages": [{"role": "user", "content": "continue our chat"}]
  }'

obleth records the id and does not forward x-session-id to the upstream unless you also put it in the body yourself.

Related

• OpenAI-compatible API — the full list of accepted session-id sources on the request surface.
• Reliability & Failover — session_hash endpoint selection.
• Observability — the session.id trace attribute.