Images, audio and speech

Register the model

Give the model the model_type for what it serves. The type decides which routes it answers, how its health is checked and how it is priced. It defaults to chat.

curl -X POST http://localhost:9180/api/v1/models \
  -H "Authorization: Bearer $OBLETH_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "sdxl",
    "model_type": "image",
    "upstream_model": "stabilityai/stable-diffusion-xl-base-1.0",
    "api_base": "https://images.example.internal/v1",
    "cost_per_image": 0.02,
    "enabled": true
  }'

Set api_base to the backend's base URL ending in /v1, not a full endpoint URL: the gateway appends the client's path to it. Every other model field is in Model registry, and you can also use Add model on the Models page.

Only chat models take part in model: "auto". Every other type is called by name.

Embeddings

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8088/v1", api_key="sk_...")

resp = client.embeddings.create(model="qwen3-embedding-8b", input="The quick brown fox")
print(len(resp.data[0].embedding))

Embedding inputs go through the tenant's input guardrails like chat prompts do.

Images

Any backend that serves the OpenAI images API works, including Stable Diffusion servers that expose it. Backends with their own graph APIs, such as ComfyUI workflows, do not.

img = client.images.generate(model="sdxl", prompt="a watercolor fox", n=2, size="1024x1024")
print(img.data[0].url)

Edits and variations send the source image as multipart/form-data. The gateway reads the form's model field, swaps in the model's upstream_model and forwards the file unchanged. n is read from the form (default 1). The form's prompt is scanned by input guardrails; the image is not.

with open("photo.png", "rb") as image:
    result = client.images.edit(model="sdxl", image=image, prompt="make it a watercolor")

An image model can also answer chat users mid-conversation through the image generation boon, and you can try it in the Playground's Image mode.

Speech-to-text

Transcription and translation clients upload audio as multipart/form-data. Both routes are served by an audio_transcription model.

with open("speech.wav", "rb") as f:
    result = client.audio.transcriptions.create(model="qwen3-asr", file=f)
print(result.text)

A model can carry cost_per_audio_second, which is stored and reported, but no per-second charge is applied to transcription requests in this release.

Text-to-speech

/v1/audio/speech returns audio bytes and is charged cost_per_character for each character of input.

audio = client.audio.speech.create(model="qwen3-tts", voice="alloy", input="Hello from obleth.")
audio.write_to_file("out.mp3")

Streaming

The gateway passes audio through as the backend sends it, so a backend that streams speech can be heard while the rest is still being made. Ask for a format that plays in pieces: pcm or wav with stream_format: "audio" for raw bytes, or stream_format: "sse" (or stream: true) for OpenAI's speech.audio.delta and speech.audio.done events. A backend that does not stream sends the whole file when it is done, so the same request works with either.

with client.audio.speech.with_streaming_response.create(
    model="my-tts", voice="tara", input=long_text,
    response_format="pcm", stream_format="audio",
) as response:
    for chunk in response.iter_bytes():
        player.write(chunk)  # 16-bit mono PCM at the backend's sample rate

Voices

GET /v1/audio/voices?model=<name> returns the voices the model's backend offers, in the shape vLLM-Omni uses:

curl https://gateway.example.com/v1/audio/voices?model=my-tts \
  -H "Authorization: Bearer $OBLETH_API_KEY"
# {"voices": ["tara", "leo", ...], "uploaded_voices": []}

The gateway checks the key and the tenant's allowlist as it would for a speech request, then forwards the call. It costs nothing and writes no usage row. A backend without this route answers 404; any voice name it accepts still works on /v1/audio/speech.

In the Playground

The Playground's Speech mode reads a pasted block of text aloud with any audio_speech model. It plays the audio as it arrives when the backend streams, and shows how long the first audio took, how fast the audio was made against real time, and what it cost. Get code gives the same call in curl, Python and JavaScript.

Discovery

GET /v1/models reports each model's model_type and mode (where image is image_generation and video is video_generation), so a client can group models by what they do. See Model discovery.