Deployment Recipes

A deployment recipe pairs a YAML metadata header with the raw sbatch script you have already tested on your cluster. The header carries the routing metadata obleth needs — engine, model name, port — and the body is submitted to Slurm verbatim. Recipes are the practical way to create a Slurm-managed model: you keep the script you know works, and obleth handles registration, replicas, and health.

Recipes are admin-authored. Anyone who can write one can run arbitrary commands on your cluster as the Slurm user, so treat the recipe directory as trusted operator territory.

File layout

Recipes are *.recipe files. The file stem is the recipe id, and the loader only reads immediate children of the configured directory — a path or a symlink pointing outside it is rejected.

Header fields

The header is YAML between --- fences.

Placement fields are all optional and can be left out so a recipe stays cluster-agnostic: partition, gres, cpus_per_task, mem, time_limit, nodes, account, qos, constraints, exclude. Anything omitted here can be supplied at deploy time.

The script body

Everything after the closing --- is submitted verbatim as the job's script_body. Its #SBATCH directives are parsed and lifted into the JSON fields of the submission, because slurmrestd ignores #SBATCH comments in a submitted script.

Bind the server to $OBLETH_SERVING_PORT, not to a hardcoded port. The provisioner injects a probe block after the shebang that walks the port window and exports the first free port, so several replicas can share a node without colliding.

---
name: Deploy Ollama
description: "Serve any Ollama model from an Apptainer image."
engine: ollama
model_type: chat
api_model_name: qwen2.5:0.5b
port: 8000
health_path: /api/tags
target_replicas: 1
max_job_failures: 0
---
#!/bin/bash -l
#SBATCH --nodes=1
#SBATCH --output=logs/qwen2.5-0.5b-%j.out
apptainer exec --nv /path/to/ollama.sif \
  sh -c "OLLAMA_HOST=0.0.0.0:${OBLETH_SERVING_PORT} ollama serve & \
         sleep 5 && ollama pull qwen2.5:0.5b && wait"

Keep the engine's own model alias equal to api_model_name. Deploying a recipe sets upstream_model from api_model_name, so a mismatch means the gateway asks the server for a model id it does not serve.

Variables

A recipe can declare substitution variables so one file covers several models or paths. Declare them in the header and reference them as {{name}} in the body.

variables:
  - name: model_path
    label: "Path to the GGUF file"
    required: true
  - name: ctx_size
    default: "8192"

Variable names must start with a letter or underscore. At deploy time each declared variable takes the value you supply, falling back to its default. A required variable with no value fails the deploy; an optional one that is left unset keeps its {{token}} in the script rather than expanding to nothing.

Deploy overrides

Deploying a recipe opens a form pre-filled from the header. Placement fields left out of the recipe are filled in there, and the script body itself stays editable before submission, so a cluster-specific tweak does not require editing the file on disk.

Managing recipes

Recipes appear on the dashboard's Recipes page, where they can be created, edited, and deleted with an in-browser editor. The same operations are available on the Management API:

A recipe that fails validation is still listed, marked invalid with the reason, rather than being hidden.