Management API

The Management API runs on port 9180. All routes require:

Authorization: Bearer <OBLETH_ADMIN_TOKEN>
Content-Type: application/json  (for POST/PUT/PATCH)

An OpenAPI JSON schema is available at GET /api/v1/openapi.json.

Tenants

# Create tenant
curl -X POST http://localhost:9180/api/v1/tenants \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "acme", "weight": 200, "tokens_per_minute": 100000}'

# Update weight
curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/weight \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"weight": 500}'

Lifecycle, schedules, and budgets

A tenant's lifecycle status gates whether its keys admit traffic at all (active serves; suspended and archived are blocked with 403).

# Suspend a tenant
curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/status \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"status": "suspended"}'

Access windows restrict when an active tenant may send traffic, evaluated in its own IANA timezone. weekly_windows are recurring; day is 0–6 with 0 = Sunday, and start_min/end_min are minutes from local midnight.

curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/schedule \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "active_from": "2026-01-01T00:00:00Z",
    "active_until": null,
    "weekly_windows": [
      {"day": 1, "start_min": 540, "end_min": 1080}
    ]
  }'

A term budget caps cumulative usage over a period. budget_period is one of lifetime, monthly, or term. Either or both of budget_tokens and budget_cost_usd may be set; exhausting either returns 403 tenant term budget exhausted on the data plane.

curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/budget \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "budget_tokens": 50000000,
    "budget_cost_usd": 250.0,
    "budget_period": "monthly"
  }'

The per-tenant model allowlist restricts which registered models the tenant may call. An empty or null list permits every model.

curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/allowlist \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"allowed_models": ["qwen3-8b", "auto"]}'

The per-tenant guardrails policy scans request (and optionally response) content and blocks, redacts, or logs matches. Guardrails are on whenever a policy is set; send {"policy": null} to clear it.

curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/guardrails \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"policy": {"action": "redact", "input_scanners": ["pii", "prompt_injection"], "output_scanners": ["pii"], "guard_model": null, "ban_keywords": [], "fail_open": true}}'

The per-tenant compression policy chooses which pieces of the compression boon apply for this tenant — each is an independent toggle. Send {"policy": null} to clear it and fall back to the global defaults (lossless JSON compaction only).

curl -X PATCH http://localhost:9180/api/v1/tenants/$ID/compression \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"policy": {"enabled": true, "code_compaction": true, "dedup": true, "allow_lossy": true}}'

API Keys

# Create key (secret returned once)
curl -X POST http://localhost:9180/api/v1/tenants/$TENANT_ID/keys \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"name": "prod-key"}'
# -> {"key": {"id": "...", "key_prefix": "sk_a1b2c3d4e5f6a1b"}, "secret": "sk_a1b2..."}

Per-key usage summary

GET /api/v1/keys/{id}/usage answers "when was this key last used, what did it last call, and how much has it used recently?" — without scraping the request log. It is a single ClickHouse query: max(ts_ms) plus argMax for the last model/status, and windowed count/sum for the rolling totals.

last_used_ms is not bounded by since_ms — it reflects the true last use within the ledger's retention window. The rolling totals are bounded by it.

curl "http://localhost:9180/api/v1/keys/$KEY_ID/usage" \
  -H "Authorization: Bearer $TOKEN"

{
  "key_id": "22222222-2222-2222-2222-222222222222",
  "tenant_id": "11111111-1111-1111-1111-111111111111",
  "last_used_ms": 1749427821000,
  "last_model": "gemma4-31b-it",
  "last_status_code": 200,
  "requests": 142,
  "input_tokens": 84200,
  "output_tokens": 15600,
  "total_tokens": 99800,
  "cost_usd": 1.24,
  "energy_wh": 96.4,
  "energy_cost_usd": 0.011,
  "co2_g": 38.5
}

A known key with no traffic in retention returns the same shape with last_used_ms: 0, an empty last_model, and zeroed totals. An unknown key id returns 404.

Models

Model endpoints

A model can front several upstream clusters that all serve the same upstream_model. The data plane routes across them with failover or weighted load-balancing. See Reliability & Failover.

Example model registration:

curl -X POST http://localhost:9180/api/v1/models \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "qwen3-235b-a22b-instruct-2507",
    "description": "Qwen3 235B A22B instruct model routed through Aibrix least-request",
    "model_type": "chat",
    "upstream_model": "asuair/qwen3-235b-a22b-instruct-2507",
    "api_base": "http://envoy-aibrix-system-aibrix-eg-903790dc.envoy-gateway-system.svc.cluster.local/v1",
    "api_key": "sk_1234",
    "input_cost_per_token": 0.000000071,
    "output_cost_per_token": 0.0000001,
    "context_window": 262144,
    "admission_weight": 100,
    "supports_function_calling": true,
    "supports_system_messages": true,
    "supports_response_schema": true,
    "supports_tool_choice": true,
    "supports_vision": false,
    "tags": ["general", "reasoning"],
    "boons": [],
    "tool_servers": []
  }'

Model type and modality fields

model_type selects the modality a route serves and defaults to chat, so existing routes need no changes. Non-chat routes carry per-modality cost fields.

Set api_base to the provider base URL ending in /v1 for every model type. See Multi-modal Models for per-type registration examples.

Chat routes also carry capability flags used by auto routing and model boons. supports_vision defaults to false; flag a chat model true when it natively accepts image input. In the dashboard this flag is derived from the vision routing tag. A model left without a native capability (supports_vision, supports_function_calling, supports_response_schema) can be granted the matching boon by adding it to boons.

boons is a per-model, fixed-vocabulary list (vision, structured_output, compression) of gateway capabilities granted to a model that lacks them natively (or, for compression, that should reduce its input tokens). It defaults to [] (no boons). A boon only takes effect when both the model opts in via boons and the matching global boon setting is enabled.

tool_servers is a separate per-model list of registered MCP-server names whose tools the gateway injects into the model's chat requests and executes itself (the gateway tool loop). Unlike boons it has no fixed vocabulary — names are the MCP servers you registered — and it requires the model to have native supports_function_calling. It defaults to []. See Model Boons and the MCP Gateway guide.

energy_slots_per_node (integer, default 0) declares how many of this model's requests one node can serve at once when fully loaded, for energy & carbon accounting. 0 opts the model out — its requests record zero energy.

Model capacity and auto-tune

Each model can optionally cap concurrent in-flight requests with max_in_flight (null = no per-model cap; only the global scheduler and fairshare limits apply). capacity_mode records how that cap was chosen:

Set mode without changing slots:

curl -X PUT http://localhost:9180/api/v1/models/$MODEL_ID/capacity-mode \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"capacity_mode": "static"}'

Run the ramp probe (chat and embedding only; does not write config):

curl -X POST http://localhost:9180/api/v1/models/$MODEL_ID/autotune \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"target_p99_ms": 2000, "max_concurrency": 64}'

The response is an AutotuneReport: recommended_max_in_flight, knee_reason (slo_breach, plateau, max_concurrency, or no_data), per-step throughput and latency, and probe duration. Apply explicitly:

curl -X POST http://localhost:9180/api/v1/models/$MODEL_ID/autotune/apply \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"max_in_flight": 32}'

The probe drives real load directly at the model upstream (bypassing gateway admission), consumes tokens, and is bounded to ≤60s and ≤20k requests. See Capacity Auto-tune for behavior, safety caps, and when to use static vs tuned mode.

Reliability and endpoints

PUT /api/v1/models/{id}/reliability sets the per-request timeout, retry policy, and endpoint selection mode in one call:

curl -X PUT http://localhost:9180/api/v1/models/$MODEL_ID/reliability \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "request_timeout_secs": 60,
    "max_retries": 2,
    "retry_backoff_ms": 200,
    "endpoint_selection_mode": "failover"
  }'

Each model endpoint is a separate upstream cluster serving the same upstream_model:

# Add a standby cluster
curl -X POST http://localhost:9180/api/v1/models/$MODEL_ID/endpoints \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "cluster-b",
    "api_base": "http://cluster-b.internal/v1",
    "api_key": "sk_cluster_b",
    "priority": 200,
    "weight": 100,
    "enabled": true
  }'

The response is a ModelEndpoint carrying the fields above plus per-endpoint health (health_status, consecutive_failures, last_checked_at, last_latency_ms, last_http_status, last_message). See Reliability & Failover.

Model health

Model health checks are persisted probes that judge each route without spending tokens. obleth first looks for a passive signal in the ClickHouse usage ledger; when a model has no recent traffic, it falls back to a token-free GET {api_base}/models liveness probe at the upstream. See Model Health.

Example config update:

curl -X PUT http://localhost:9180/api/v1/models/$MODEL_ID/health/config \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "checks_enabled": true,
    "alerts_enabled": true,
    "check_interval_secs": 900,
    "failure_threshold": 2,
    "maintenance_until": null,
    "maintenance_note": null
  }'

Manual check:

curl -X POST http://localhost:9180/api/v1/models/$MODEL_ID/health/check \
  -H "Authorization: Bearer $TOKEN"

Fairshare

MCP Servers

Usage and cost attribution

Per-key activity summary (bulk)

GET /api/v1/usage/keys/summary is the bulk form of /keys/{id}/usage: one row per key that saw traffic in the window, each with last-used metadata and rolling totals. It powers the Last used column on the dashboard Keys table, so the UI needs a single call instead of one request-log fetch per key.

Unlike the single-key endpoint, last_used_ms here is bounded by since_ms (the whole scan is windowed for efficiency), and only keys with activity in the window appear. Keys idle for the whole window are simply absent — the dashboard renders those as "Never". Rows are ordered by token volume, descending.

# Per-key summary for one tenant over the last 30 days
curl "http://localhost:9180/api/v1/usage/keys/summary?tenant_id=$TENANT&since_ms=$(($(date +%s)*1000-2592000000))&limit=5000" \
  -H "Authorization: Bearer $TOKEN"

Each row has the same shape as the single-key summary.

Request logs

GET /api/v1/usage/logs returns individual rows from the raw usage ledger, newest first. It is the only usage endpoint that surfaces per-request detail rather than an aggregate, and it powers the dashboard Request Logs page.

ClickHouse stores tenant and key as UUIDs; the handler enriches each row with tenant_name, key_name, and key_prefix from Postgres before returning it.

# Last hour of chat completions for one tenant, newest first
curl "http://localhost:9180/api/v1/usage/logs?since_ms=$(($(date +%s)*1000-3600000))&tenant_id=$TENANT&request_type=chat&limit=50" \
  -H "Authorization: Bearer $TOKEN"

Each row is a UsageLogEntry (flattened in JSON):

[
  {
    "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "ts_ms": 1749427821000,
    "tenant_id": "11111111-1111-1111-1111-111111111111",
    "key_id": "22222222-2222-2222-2222-222222222222",
    "model": "gemma4-31b-it",
    "request_type": "chat",
    "session_id": "conv-abc123",
    "admission": "fast",
    "status_code": 200,
    "input_tokens": 842,
    "output_tokens": 156,
    "total_tokens": 998,
    "queue_wait_ms": 0,
    "ttft_ms": 1180,
    "total_ms": 1240,
    "cache_status": "miss",
    "cost_usd": 0.00008,
    "energy_wh": 0.07,
    "energy_cost_usd": 0.0000084,
    "co2_g": 0.028,
    "tenant_name": "RCUsers",
    "key_name": "akuma",
    "key_prefix": "sk_a1b2...",
    "has_trace": true
  }
]

has_trace is true when at least one span was recorded for the request (i.e. tracing_enabled was set on the key or tenant at request time). Use GET /api/v1/usage/logs/{request_id}/spans to fetch the full span set.

Pagination uses a stable (ts_ms, request_id) keyset cursor. Pass the last row's ts_ms and request_id as before_ms and before_request_id to fetch the next (older) page. Rows are ordered by ts_ms desc, toString(request_id) desc.

request_id is an internal obleth UUID assigned at completion time, not the upstream provider's id (e.g. chatcmpl-…).

Request spans

GET /api/v1/usage/logs/{request_id}/spans returns all recorded spans for a single request, ordered by start_ms. The array is empty for requests that were not traced (i.e. has_trace is false).

curl http://localhost:9180/api/v1/usage/logs/$REQUEST_ID/spans \
  -H "Authorization: Bearer $TOKEN"

[
  {
    "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "span_name": "proxy_request",
    "parent_span": "",
    "start_ms": 1749427821000,
    "duration_ms": 1243,
    "status": "ok",
    "attributes": "{}"
  },
  {
    "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "span_name": "auth_resolve",
    "parent_span": "proxy_request",
    "start_ms": 1749427821001,
    "duration_ms": 2,
    "status": "ok",
    "attributes": "{}"
  },
  {
    "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "span_name": "upstream",
    "parent_span": "proxy_request",
    "start_ms": 1749427821010,
    "duration_ms": 1180,
    "status": "ok",
    "attributes": "{}"
  }
]

See Per-request span tracing for the full span vocabulary.

Daily usage rollup

GET /api/v1/usage/daily reads the permanent usage_daily rollup rather than the raw per-request ledger, so it returns history for the full date range even after the raw usage table has been pruned to its retention window. It is the endpoint behind the dashboard Reports page and its CSV export.

# Daily totals for the last 30 days, broken down per model
curl "http://localhost:9180/api/v1/usage/daily?start_day=2025-05-01&end_day=2025-05-31&group_by=model" \
  -H "Authorization: Bearer $TOKEN"

Each row is a UsageDailyRow. Identity columns that aren't part of the requested group_by come back empty (day) or zero UUID:

[
  {
    "day": "",
    "tenant_id": "00000000-0000-0000-0000-000000000000",
    "key_id": "00000000-0000-0000-0000-000000000000",
    "model": "gemma4-31b-it",
    "requests": 12840,
    "success_requests": 12790,
    "error_requests": 50,
    "input_tokens": 4210000,
    "output_tokens": 1980000,
    "total_tokens": 6190000,
    "estimated_tokens": 4250000,
    "cache_hits": 3120,
    "cache_misses": 9720,
    "avg_ttft_ms": 184.2,
    "avg_total_ms": 5120.5,
    "cost_usd": 742.18,
    "energy_wh": 61240.5,
    "energy_cost_usd": 7.35,
    "co2_g": 24496.2
  }
]

Averages (avg_ttft_ms / avg_total_ms) are computed over successful (2xx/3xx) requests only, so upstream timeouts and errors don't distort them.

cost_usd is the summed USD spend for the grouped rows. It is summed from each request's cost frozen at completion time (using the model price in effect then), not recomputed from current prices — so historical spend never shifts when a model's pricing is edited. Use group_by=key_model for per-key, per-model spend. Rows recorded before cost tracking existed report 0.

energy_wh / energy_cost_usd / co2_g follow the same frozen-at-completion rule when energy accounting is enabled; rows recorded while it was off (or for opted-out models) report 0.

Compacting the raw ledger

POST /api/v1/usage/compact prunes the raw usage ledger to the configured retention window immediately, instead of waiting for the hourly worker. It drops whole day-partitions and never touches the permanent usage_daily rollup. The response reports the window applied and how many partitions were dropped:

curl -X POST http://localhost:9180/api/v1/usage/compact \
  -H "Authorization: Bearer $TOKEN"
# -> {"retention_days": 180, "partitions_dropped": 3}

Capacity and stats

Settings & alerting

Runtime-configurable operational alerting over Slack and email (SMTP). Changes apply immediately, with no restart. See Alerting for the full guide.

Auto router

The auto model-routing classifier is configured here. The persisted setting is authoritative once saved; the OBLETH_AUTO_CLASSIFIER_* env vars only seed it on first boot. See Auto Model Routing.

curl -X PUT http://localhost:9180/api/v1/settings/auto-router \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "classifier_enabled": true,
    "classifier_model": "qwen3-0.6b",
    "classifier_timeout_ms": 250
  }'

Send classifier_model as "" to clear it (disables the brain; routing falls back to capacity/cost scoring and keyword heuristics).

# Configure Slack + email alerting
curl -X PUT http://localhost:9180/api/v1/settings/alerts \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "slack_webhook_url": "https://hooks.slack.com/services/T000/B000/XXXX",
    "min_interval_secs": 300,
    "email": {
      "smtp_host": "smtp.example.com",
      "smtp_port": 587,
      "from_address": "alerts@example.com",
      "recipients": ["oncall@example.com"],
      "starttls": true
    }
  }'

The GET response masks secrets, reporting slack_webhook_set and email.password_set booleans instead of values. On PUT, omit slack_webhook_url / smtp_password to keep the stored secret, or send clear_slack_webhook / clear_smtp_password set to true to remove it.

Model boons

Model boons grant a capability at the gateway to models that lack it natively. There are three boons — vision (relay images to a describer model and rewrite them as text), structured output (enforce response_format JSON schemas, repairing invalid JSON with a fixer model), and compression (reduce input tokens via structural JSON/code compaction, cross-turn dedup, and deterministic lossy text compaction) — plus the gateway tool loop, which is configured from the same endpoint. The settings below are the global switches; each model also opts in individually via its boons list (vision, structured output, compression) or its tool_servers list (tool loop) — see model registration. Settings are hot-reloadable. See Model Boons.

curl -X PUT http://localhost:9180/api/v1/settings/boons \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "vision_enabled": true,
    "vision_fallback_model": "glm-4-5v",
    "vision_describe_prompt": "Describe this image in thorough, faithful detail.",
    "vision_max_images": 6,
    "vision_timeout_ms": 30000,
    "structured_output_enabled": true,
    "structured_output_fixer_model": "qwen3-235b",
    "structured_output_max_repair_attempts": 1,
    "structured_output_timeout_ms": 30000,
    "tool_loop_enabled": true,
    "tool_loop_max_turns": 4,
    "tool_loop_tool_timeout_ms": 30000,
    "tool_loop_nudge": "You have tools available in this conversation and can call them...",
    "compression_enabled": true,
    "compression_min_tokens": 512,
    "compression_max_segments": 64,
    "compression_max_lossy_segments": 4,
    "compression_code_compaction": false,
    "compression_original_ttl_secs": 3600
  }'

Compression is gated per piece and partly per tenant — see the compression boon for how the global switches above combine with each tenant's compression policy.

A boon is active for a request only when its master switch is true, any helper model it needs is set, and the target model has the boon in its boons list and lacks the matching native capability (supports_vision, supports_response_schema). The structured-output boon applies to chat completions and buffers streaming responses (obleth re-emits the transformed result as synthesized SSE). Vision describe calls and structured-output repair calls are billed to the tenant as vision_boon / structured_output_boon records against the helper model.

The gateway tool loop is active when tool_loop_enabled is true, the model has one or more tool_servers granted, and the model has native supports_function_calling. Each model round trip inside the loop is billed to the tenant as a tool_loop record. Unlike the structured-output boon, the tool loop can stream live to the client token-by-token (only tool execution between turns pauses the stream). See Model Boons.

Usage retention

The raw per-request usage ledger is pruned to a rolling window so storage stays bounded; the permanent usage_daily rollup is kept forever. The window defaults to OBLETH_USAGE_RETENTION_DAYS (180) and can be overridden live from the control plane. GET reports whether the value is a saved setting or the environment default; PUT accepts a minimum of 1 day and records an audit entry.

# Read the current window
curl http://localhost:9180/api/v1/settings/usage-retention \
  -H "Authorization: Bearer $TOKEN"
# -> {"days": 180, "configured": false}

# Keep 90 days of raw history
curl -X PUT http://localhost:9180/api/v1/settings/usage-retention \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"days": 90}'
# -> {"days": 90, "configured": true}

Slurm provisioner

System-wide Slurm connection settings consumed by the obleth-provisioner plugin service. The JWT is encrypted at rest; GET returns a masked view (jwt_set, jwt_last4). The resolved route returns the decrypted JWT for the provisioner process only — it is still gated by the admin token. See Slurm Provisioning.

# Enable Slurm and set the connection details
curl -X PUT http://localhost:9180/api/v1/settings/slurm \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "slurmrestd_url": "http://slurm-head.cluster.local:6820",
    "slurmrestd_api_version": "v0.0.40",
    "slurm_user": "hpcuser",
    "slurm_jwt": "<raw-jwt>"
  }'

# Test the connection (JWT expiry + slurmrestd ping)
curl -X POST http://localhost:9180/api/v1/settings/slurm/test \
  -H "Authorization: Bearer $TOKEN"

POST /settings/slurm/test returns a SlurmHealthView:

{
  "jwt": {
    "set": true,
    "expired": false,
    "expires_at": "2026-12-01T00:00:00Z",
    "expires_in_secs": 15552000
  },
  "ping": {
    "ok": true,
    "status_code": 200,
    "latency_ms": 12,
    "error": null
  }
}

Omit slurm_jwt (or send it empty) on PUT to keep the stored JWT; send a new value to replace it.

Energy accounting

Per-request energy/carbon attribution from your own Prometheus power metrics. See Energy & Carbon Accounting for the full guide, the slot-share math, and query examples.

# Configure and enable
curl -X PUT http://localhost:9180/api/v1/settings/energy \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "prometheus_url": "http://prometheus:9090",
    "power_query": "habana_device_power_watts",
    "poll_interval_secs": 60,
    "energy_cost_per_kwh": 0.12,
    "carbon_g_per_kwh": 400.0,
    "pue": 1.2
  }'

# Dry-run a query without saving anything
curl -X POST http://localhost:9180/api/v1/settings/energy/test \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"prometheus_url": "http://prometheus:9090", "power_query": "habana_device_power_watts"}'

POST /settings/energy/test returns the live reading, or the raw Prometheus error string on failure:

{ "cluster_watts": 44800.0, "node_count": 64 }

Managed models (Slurm provisioner)

Per-model spec for the obleth-provisioner plugin. Each spec tells the provisioner which Slurm resources to request, which Apptainer image to run, and how many replicas to keep alive. See Slurm Provisioning.

# Set up a Slurm-provisioned model
curl -X PUT http://localhost:9180/api/v1/models/$MODEL_ID/managed \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "partition": "gpu",
    "gres": "gpu:a100:1",
    "nodes": 1,
    "image": "/shared/images/vllm.sif",
    "launch_command": "python -m vllm.entrypoints.openai.api_server --model /data/models/llama3 --port 8080",
    "serving_port": 8080,
    "health_path": "/health",
    "target_replicas": 2,
    "account": "mylab",
    "qos": "gpu-normal",
    "time_limit": "8:00:00"
  }'

Backup & Restore

# Export all configuration
curl http://localhost:9180/api/v1/backup/export \
  -H "Authorization: Bearer $TOKEN" \
  -o backup.json

# Restore from backup (on a fresh gateway)
curl -X POST http://localhost:9180/api/v1/backup/restore \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @backup.json

See Backup & Restore for import/export workflows and safe restore procedures.

Audit

Each row carries id, ts, actor, action, entity_type, entity_id, and a structured detail object. The actor is taken from the x-obleth-audit-actor request header — the dashboard and self-service portal set it to the signed-in user's email, the provisioner sets system, and it falls back to admin when absent. See Audit Log.

Utility