Data-plane routes

The data plane listens on OBLETH_PROXY_LISTEN (default 0.0.0.0:8080). Every route except /health needs a credential.

Routes

Path handling

• POST spellings. Repeated slashes collapse, a trailing slash is dropped, and the bare /responses, /messages, /messages/count_tokens and /verdicts map to their /v1 forms. The request is then served like the canonical path, with translation, boons and guardrails.
• Path traversal. A .. segment, or %2e, %2f or %5c anywhere in the path, returns 400 before authentication.
• Unknown endpoints. A path that is not a known endpoint and whose body names no registered model returns 404 unknown endpoint. It never reaches an upstream or the usage ledger.
• Registered-model paths. Chat, completions, embeddings, responses, the three audio paths, the three image paths and POST /v1/videos must name a registered model in model (the form field for multipart). An alias is replaced by the canonical name before admission, budgets, caching, the allowlist and the ledger. A variant is too, with its boons added, its own response-cache entries and its name kept in the ledger's model_variant. Only chat takes part in auto routing.
• Upstream URL. The path is appended to the endpoint's (or model's) api_base. Set api_base to the base ending in /v1; a doubled /v1 is dropped, and a base that already ends with the request path is used as is.

Authentication

The credential comes from Authorization: Bearer <credential>, or from x-api-key when there is no bearer token. It is an API key or, with identity tokens configured, a JWT. See Authentication.

Route behaviour

• Model listing. GET /v1/models answers from the registry without calling an upstream: every enabled model once, sorted by id, with id, object: "model", owned_by: "obleth", model_type, mode, quantization, tags, aliases, the model's own supports_* flags, boons and boon_details (whether each boon runs, and the helper model that does its work), and each variant as its own entry with variant_of and description. Staged and retired models and search models are left out; a deprecated entry adds status and its lifecycle fields. A request body is ignored. GET /v1/models/{id} returns the same entry for a name, alias or variant, including a retired model, and 404 for a staged model or one outside the tenant's allowlist. /model/info never returns api_base or api_key. See Model registry and Model lifecycle.
• Multipart uploads. Audio, image edits and variations, and a video create with a reference frame are parsed: model is swapped for the upstream name, text fields are scanned and billed like JSON (image n), and file parts are forwarded unscanned. A multipart request gets one attempt, never a retry. See Images, audio and speech.
• Video jobs. A create is billed cost_per_video once the job is recorded. Follow-ups go to the endpoint that holds the job, take no fairshare slot or budget, write no ledger row, and pass the upstream status through. By default a key sees only its own jobs. See Video generation.
• Mesh jobs. The same as video jobs on /v1/meshes, billed the model's per-job price as request type mesh. A job is found only through the endpoint that created it. See 3D mesh generation.
• Responses API. Body up to 32 MiB. store: true, background: true or a previous_response_id returns 400. See OpenAI-compatible API.
• Messages API. Body up to 32 MiB; errors use the Anthropic envelope. An unknown model is served by the default model for Anthropic clients, else 404. count_tokens returns {"input_tokens": <n>} and is not admitted, budgeted or recorded. See Claude Code and Anthropic SDKs.
• Verdicts. Needs a registered chat model or auto; one fairshare permit and one ledger row cover every question. See Verdicts.
• Search. Body up to 64 KiB. Takes a slot in the search tool's pool with no token estimate, writes one free search ledger row, and answers in the Perplexity Search API shape. Any other route that names a search tool gets 400. See Search API.
• Response cache. Checked before admission for a model with caching on and cache_ttl_secs above 0; never for video creates, tool loops or output guardrails. See Response cache.

Request pipeline

1. Authenticate and gate the key and tenant.
2. Read the body (up to 64 MiB), resolve the model, and check its lifecycle: a retired model is refused with 410 or answered by its replacement.
3. Refuse endpoints the tenant's guardrails cannot enforce, then run boons and the input guardrails. Guardrails also run under x-obleth-boons: off.
4. Check the response cache.
5. Take a slot in the model's pool, waiting up to OBLETH_ADMISSION_TIMEOUT_SECS (default 60).
6. Reserve the per-minute and term budgets in Redis.
7. Forward with the model's upstream headers and the endpoint's api_key, retrying and failing over as configured.
8. Stream the response back, reconcile the budgets and write the usage row.

A request that fails on every attempt is not billed. An upstream 4xx or 5xx without a usage block records zero tokens. Images bill n × cost_per_image, speech bills input characters × cost_per_character, and embeddings bill prompt tokens.

Error responses

Gateway errors on the OpenAI-shaped routes use this body:

{ "error": { "message": "token budget exceeded", "type": "obleth_gateway_error" } }

Upstream errors pass through with the upstream's status and body. For a stream that stops early, see When a stream is cut short.

On /v1/messages, errors use {"type": "error", "error": {"type", "message"}} with this error.type:

When a stream is cut short

A stream is cut when, before its last event, the upstream's connection breaks, the upstream goes silent past OBLETH_UPSTREAM_READ_TIMEOUT_SECS, or a chat or text completion stream closes with neither [DONE] nor a finish reason. The client then gets one error event, and the response is aborted, with no finish chunk, usage chunk or [DONE]:

data: {"error":{"message":"upstream stream ended before the response was complete (after 47 s)","type":"obleth_gateway_error","code":502}}

The seconds count from the start of the upstream call. OpenAI's SDKs raise the event as an APIError. Every stream ends a cut this way: one passed through from the upstream, each turn of the tool loop (including a follow-up or final turn that can't be dispatched), and the speculation boon's continuation. The translated Responses stream ends with response.failed instead, and the Messages stream with an error event, each carrying the message.

A cut is recorded as 502 and billed for what reached the client: the usage the upstream reported, else the streamed text at about 4 characters per token, else nothing. It is never billed the max_tokens estimate. A connection the upstream drops after [DONE] is not a cut.

An error event the upstream sends itself is forwarded once, and the gateway adds no event of its own. The request is recorded under the error's status (its code when that is an HTTP status, else 502) and billed the same way. When the tool loop or the speculation boon runs the stream, [DONE] follows the error event; otherwise the client gets the rest of the stream as the upstream sends it.

Headers

Client headers are forwarded upstream except host, content-length, connection, accept-encoding, authorization, x-api-key, x-obleth-boons and x-obleth-end-user. The model's upstream headers are added after them, so the operator's value wins. On /v1/messages, anthropic-beta and the beta query parameter are dropped.

MCP gateway

/mcp/{server} and its sub-paths take the data-plane credential and tenant gate, replace Authorization with the server's stored auth_header, and reverse-proxy the bodies unchanged, so streamable HTTP and SSE both work. Outcomes are counted in obleth_mcp_requests_total. See MCP gateway.