Data Plane Routes

The data plane listens on port 8080 and proxies OpenAI-compatible requests to the configured upstream. All routes require a valid API key in the Authorization header.

Route table

All other paths are forwarded to the upstream as-is (passthrough). This means vendor-specific extensions like /v1/batch or /v1/files work without any obleth configuration.

Routes that require a registered model resolve by the request model field against the model registry: /v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/audio/transcriptions, /v1/audio/translations, /v1/audio/speech, and the /v1/images/* routes. Only chat routes participate in auto model routing. See Multi-modal Models.

Multipart (speech-to-text)

Audio transcription/translation clients send multipart/form-data with the audio file. obleth parses the multipart body, rewrites the model field to the route's upstream_model, rebuilds the form, and forwards the file part unchanged.

Upstream URL construction

obleth appends the client's request path to the model's api_base. Configure api_base as the provider base ending in /v1 (not a full endpoint URL) to avoid a doubled path such as .../v1/embeddings/v1/embeddings. When a model defines multiple endpoints, the selected endpoint's api_base is used the same way. See Reliability & Failover.

Authentication

Every request (except /health) must include:

Authorization: Bearer sk_<48 hex chars>

The key is resolved as described in the Authentication reference.

Request handling

1. Auth key is resolved → tenant, weight, group, TPM quota looked up
2. Model name extracted from JSON body → matched against model registry
3. Token estimate computed from the request body
4. Response cache checked if enabled for the model
5. Admission: in-flight check → queue if at capacity (admitted as soon as a slot opens)
6. Per-minute token budget checked and reserved atomically via Redis Lua (429 if exceeded)
7. Term budget checked for tenants with a cumulative cap (403 if exhausted)
8. Request forwarded to upstream with headers:
   • Authorization replaced with the selected endpoint's (or model's) api_key (if set)
   • bounded by a per-request timeout, retried with backoff on transient failures, and failed over to the next endpoint if configured — see Reliability & Failover
9. Streaming response streamed back to client
10. Actual token counts extracted from the response tail when the upstream reports usage
11. Budget reconciled (reserve → actual delta refunded)
12. Usage record written to ClickHouse (or WAL)

For non-token modalities, cost is computed from the response shape instead of token usage: image routes bill n × cost_per_image, and text-to-speech bills input characters × cost_per_character. Embeddings bill prompt tokens via input_cost_per_token. See Multi-modal Models.

Header handling

obleth forwards client headers except hop-by-hop, auth, and encoding headers:

• host
• content-length
• authorization
• x-api-key
• accept-encoding
• connection

Response cache

When cache is enabled for a model, the data plane:

1. Computes SHA-256(model + request_body) before forwarding
2. Checks obleth:cache:{hash} in Redis
3. On hit: returns cached response immediately (no upstream call, no fairshare permit, no budget reserve)
4. On miss: forwards request, then stores successful 200 responses up to 512 KiB with the model's cache_ttl_secs

Cache hits include X-Obleth-Cache: hit. Misses continue through the normal streaming path and are recorded in the usage ledger as cache_status = "miss".