Structured output

You needNothing extra; a separate fixer model is optional

Turn it onSettings › Boons › Structured output, then the model's page › Boons

It costsA repair call only when a reply fails, billed as structured_output_boon

Turn it on

1
Turn Structured output on
Open Settings › Boons, switch Structured output on and Open its row. Under Fixer, pick the model that repairs broken replies, or leave "The same model, asked again". Set Retries (at most 3) and the Repair timeout. Save with the save bar.
2
Grant it to the model
On the model's page, go to Boons, switch Structured output on and save. Leave Response schema under Native off: a model that supports response_format itself doesn't need the boon, and the boon steps aside for it.

Or through the Management API:

curl -X PUT http://localhost:9180/api/v1/settings/boons \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "structured_output_enabled": true,
    "structured_output_fixer_model": "qwen3-235b",
    "structured_output_max_repair_attempts": 1,
    "structured_output_timeout_ms": 30000
  }'

Send structured_output_fixer_model as "" to repair with the request's own model.

Ask for JSON

Use response_format as you would with any OpenAI-compatible API:

curl http://localhost/v1/chat/completions \
  -H "Authorization: Bearer $OBLETH_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "minimax-m2-7-fast",
    "messages": [{"role": "user", "content": "Extract the person: Ada Lovelace, born 1815 in London."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "schema": {
          "type": "object",
          "properties": {
            "birth_year": {"type": "integer"},
            "city": {"type": "string"},
            "name": {"type": "string"}
          },
          "required": ["birth_year", "name"]
        }
      }
    }
  }'

The reply's content is the JSON document alone, with no prose or markdown fences around it:

{"birth_year":1815,"city":"London","name":"Ada Lovelace"}

"type": "json_object" works too; the reply is then checked for valid JSON with no schema.

What the model sees

The gateway removes response_format from the request and adds a system message after any existing ones:

# Response format

Respond with a single JSON document that conforms to the JSON Schema below. Output only the JSON document — no prose, no markdown fences, nothing before or after it.

JSON Schema: {"type":"object","properties":{...},"required":["birth_year","name"]}

A model flagged as not supporting system messages gets the section at the start of the first user message instead.

When the model replies, the gateway finds the JSON document in the text (tolerating fences and stray prose) and validates it. A valid document replaces the reply's content. An invalid one goes to the fixer with the validation errors, up to Retries times.

Settings

Settings label	Field	Default
Structured output on	`structured_output_enabled`	`false`
Fixer model	`structured_output_fixer_model`	none: the request's own model
Retries	`structured_output_max_repair_attempts`	`1`, at most `3`; `0` turns repair off
Repair timeout	`structured_output_timeout_ms`	`30000` ms, per repair call

When it applies

The boon runs when Structured output is on in Settings, the model has structured_output in its boons list and is not flagged supports_response_schema, and the request is a chat completion whose response_format.type is json_schema or json_object.

If the fixer model is missing or disabled, the request's own model does the repair.

Streaming, limits and failures

While the boon is active the gateway calls the model without streaming, checks the whole reply, and then sends it. A client that asked for stream: true still gets a stream, but it starts only once the reply is complete, so allow a longer client timeout. The gateway's own upstream timeout for these requests is at least 120 seconds.

Schemas larger than 64 KB are put in the prompt but not validated; the reply is only checked for valid JSON.

If every repair fails, the model's original reply passes through unchanged and the response carries x-obleth-boons-warning: structured_output_validation_failed.

With the image generation boon on the same request, the validated JSON wins: generated images are not attached, and the response carries x-obleth-boons-warning: image_suppressed_by_response_format.

Billing

The first answer is billed against the model you asked for, as usual. Each repair call is a usage row with request_type = structured_output_boon under the fixer's name, priced at the fixer's rates, and counts toward term budgets. A reply that validates first time costs nothing extra.

PreviousVision

NextTool loop