Recipes

Write a recipe

1. 1

   Start from a script

   On Deployments › Recipes, choose New recipe and paste a script you already run on the cluster. To change a library recipe, choose Copy to edit.

2. 2

   Add the header

   Put the YAML header between --- fences above the script: at least name, engine, model_type, api_model_name and port. Declare what someone launching it may change as inputs, and write {{name}} where each goes in the script.

3. 3

   Save and launch

   Save it, then choose Launch on its row. See Launch on Slurm.

---
name: Qwen3 8B
description: "Qwen3 8B on one GPU with vLLM."
engine: vllm
kind: model
model: Qwen/Qwen3-8B
weights_gb: 17
model_type: chat
api_model_name: qwen3-8b
port: 8000
target_replicas: 1
max_job_failures: 2
env:
  VLLM_ENGINE_READY_TIMEOUT_S: "3600"
inputs:
  - name: context
    label: Context length
    type: choice
    options: [16384, 32768]
    default: 32768
  - name: tools
    label: Tool calling
    type: flag
    default: true
    adds: --enable-auto-tool-choice --tool-call-parser hermes
    capabilities: [function_calling, tool_choice]
  - name: image
    label: Container image
    type: path
    default: "{{cluster.image.vllm}}"
    required: true
  - name: cache
    label: Weight cache
    type: path
    default: "{{cluster.cache}}"
    required: true
---
#!/bin/bash -l
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=8
#SBATCH --mem=48G
set -euo pipefail
{{cluster.setup}}
export HF_HOME={{cache}}
apptainer exec --nv --bind {{cache}} {{image}} \
  vllm serve Qwen/Qwen3-8B \
    --served-model-name {{api_model_name}} \
    --host 0.0.0.0 --port "$OBLETH_SERVING_PORT" \
    --max-model-len {{context}} \
    {{tools}}

obleth submits the script after a short block it adds to bind the replica's port, with env exported at the top and Unix line endings. The server must listen on $OBLETH_SERVING_PORT, not a fixed port, and serve the model under the API model name it is launched as, which {{api_model_name}} fills in. slurmrestd ignores #SBATCH lines in a submitted script, so obleth reads them into the job's settings and leaves them out of the script it stores.

Header fields

Placement fields are optional too: partition, gres, cpus_per_task, mem, time_limit, nodes, account, qos, constraints, exclude. Leave them out to keep a recipe usable on any cluster and choose them at launch. What you choose at launch wins over the header, and the header wins over the script's #SBATCH lines. mem: 0 asks for all of each node's memory.

Inputs

Each input is a setting someone launching the recipe may change. It has a name (a letter or underscore, then letters, digits or underscores), and optionally a label, help text, a default and required: true.

A flag can also declare capabilities, from function_calling, tool_choice, response_schema and vision. When the flag is on at launch, the model obleth creates gets those capabilities, so a recipe whose tool flag adds a tool parser launches a model flagged for function calling, and image generation and web search work on it.

by_nodes gives an input a default per node count, such as {"1": "270", "4": "20"}, for a value that follows Runs on.

At launch, each declared {{name}} gets its value, or its default. An optional input left empty becomes empty; a required one without a value stops the launch. Undeclared {{…}} and shell ${…} and $(…) pass through untouched.

Cluster values

A default can name a cluster value, so no recipe hard-codes a path. Each comes from Cluster defaults on the Slurm connection:

A launch is refused while a value the script needs is unset, and says which one to set.

The older variables

variables: (name, label, default, required) still works: each is read as a text input. When a name is in both lists, the typed input wins.

Record where it ran

A recipe can list the runs that served it under tested, newest first:

tested:
  - hardware: "1× 80 GB GPU"
    engine: "vLLM <version>"
    date: 2026-10-01
    result: "What you saw: time to ready, speed, tool calls."

hardware is required. New deployment shows each run as a sentence, and the Library and Engines lists say where a recipe last served.

The Recipes tab

The tab has three lists. Each row shows the recipe's engine, the values it asks for and the deployments launched from it; a library row also says where it last served. A recipe that can't be read is still listed, with the reason.

• Saved recipes are your team's, stored by the gateway: a library recipe or engine with the values that worked. Save one at launch with Also save these settings as a recipe, or from a serving deployment's page with Save as recipe, which writes its values into the source recipe's header and keeps cluster defaults as {{cluster.*}} references. You can Edit and Delete them; deployments launched from a deleted recipe keep running.
• Library recipes are the *.recipe files with kind: model in OBLETH_RECIPES_DIR (default: recipes/ in the dashboard's working directory), named by their file stem. They can't be changed from the dashboard. The repository ships a set in control-plane/recipes/, each with where it was tested.
• Engines are the files with kind: engine: one per engine (vLLM, SGLang, llama.cpp and Ollama) in the shipped set. They serve a Hugging Face model that has no recipe yet, and are offered after you pick one under From Hugging Face.

The old /recipes page redirects here.