Variants

A boon on a model acts for everyone who calls it. Some boons change what a caller gets: with speculation, a fast drafter may write the answer. A variant is a second name for the model with extra boons turned on, such as my-model-spec for my-model plus speculation. Callers who want the boon ask for the variant; callers who name the model get its own answer.

Add a variant

1. 1

   Open the model's General section

   On a chat model's page, find Variants under General and choose Add a variant.

2. 2

   Name it and pick its boons

   Enter the name clients will send, a short description of what callers get, and tick the boons it adds. The line under the row says what the name reaches, for example "my-model-spec reaches my-model with Speculation turned on as well."

3. 3

   Save

   Save with the save bar. The boons must also be on in Settings › Boons, as for any model.

Or through the Management API, on POST /api/v1/models or PUT /api/v1/models/{id}:

{
  "variants": [
    {
      "name": "my-model-spec",
      "description": "Faster answers, drafted by a smaller model and checked by this one",
      "boons": ["speculation"]
    }
  ]
}

On PUT, variants replaces the list, [] removes every variant, and leaving it out keeps the list.

What a variant shares

A request to a variant is served by its model's route, with the variant's boons added to the model's own. Everything else is the model's: backend and endpoints, pool and capacity, prices, health and lifecycle. Two names never make two pools for one backend.

• Discovery. GET /v1/models lists each variant as its own entry, after its model, with variant_of, its description and the boons a request through it runs with. /model/info lists them under the model's variants.
• Routing. auto never picks a variant.
• Allowlists. A tenant allowed the model may call its variants, and a tenant can be allowed a variant alone by listing the variant's name.
• Response cache. A variant's cached answers are kept apart from its model's.
• Verdicts. /v1/verdicts treats a variant as another name for its model and runs no boons.

Responses and usage

A response served through a variant carries x-obleth-model-variant: <name>. When speculation shipped a draft, it also carries x-obleth-drafted-by: <drafter>.

The usage row records the variant in model_variant, while model stays the model's name, so per-model reports and the daily rollup are unchanged. The request's panel in Request logs shows Variant. Calls a boon makes for the request are their own rows, each linked to it by parent_request_id; see Helper calls and billing.