Management API

Conventions

The gateway serves the Management API on the admin listener (OBLETH_ADMIN_LISTEN, default 0.0.0.0:9180). The dashboard uses the same routes. Keep the port on a private network; see Security.

Public and system routes:

POST /api/v1/resync returns counts: keys, keys_pruned, models, model_names_pruned, mcp_servers, mcp_servers_pruned. It is the retry for any 502, and Settings › Data and backups › Reconcile cache runs it. See Redis layout.

curl -X POST http://localhost:9180/api/v1/resync -H "Authorization: Bearer $TOKEN"

Tenants and keys

Request bodies. Task pages: Tenants and keys, Quotas and rate limits, Guardrails.

Guardrails action is block, redact or log_only; input scanners are prompt_injection, pii, ban_keywords, harm; output scanners pii, ban_keywords, harm; fail_open defaults to true.

API keys

POST /api/v1/tenants/{id}/keys and PUT /api/v1/keys/{id} take the same body. PUT replaces every field, so send the current weight and max_in_flight to keep them.

Create returns {"key": {…}, "secret": "sk_…"}. The secret is sk_ plus 48 hex characters; only its SHA-256 hash and an 18-character prefix are stored. Keys with kind: "identity" are created by the gateway for a verified identity token (Identity tokens) and managed through the same routes.

GET /api/v1/budgets/usage returns one row per capped budget: scope (tenant or key), id, tenant_id, period, budget_tokens, budget_cost_usd, used_tokens, used_cost_usd, period_start, and resets_at (monthly only).

Per-key usage summary

GET /api/v1/keys/{id}/usage returns last_used_ms, last_model, last_status_code, and requests, token, cost, energy and CO2 totals over the window. GET /api/v1/usage/keys/summary returns the same row for every key with traffic, busiest first; idle keys are absent.

A key with no traffic returns zeros and last_used_ms: 0.

Usage and request logs

Usage reads exclude internal traffic (health probes and synthetic tenants' benchmark traffic) unless you pass include_internal=true.

/usage/speed rows carry bucket_ms (0 for a whole-window row), model, requests (every request, failed and cached included), rated (the requests a speed was measured on: successful, uncached and producing tokens), gen_tokens_per_sec (median, the same decode rate as /usage/models), slow_tokens_per_sec (the slowest tenth), prompt_tokens_per_sec and p50_ttft_ms.

Request logs

/usage/logs, /usage/logs/histogram and /usage/logs/facets share these filters. Task page: Observability.

Spans come back ordered by start_ms, kept for 14 days. request_id is obleth's own id, also sent to clients as x-obleth-request-id.

Daily usage rollup

GET /api/v1/usage/daily reads the permanent rollup, so it covers dates the raw ledger has dropped. It excludes benchmark traffic.

Models

Model fields, for POST and PUT /api/v1/models/{id} and the manifest. Task page: Model registry.

PUT leaves omitted optional fields unchanged. Responses add id, capacity_tuned_at, cache and reliability fields, lifecycle, effective_status (the status in force now) and timestamps. Deleting a model or dropping an alias evicts it from Redis. Rename a model by creating it under the new name with the old one as an alias.

The manifest ("format": "obleth-models", "version": 1) is keyed by model_name, never exports secrets (has_api_key, upstream_header_names), and every other field is optional. Import creates or updates listed models, deletes nothing, reports all errors together with 400, and returns created, updated, unchanged and per-model action, changed_fields and warnings. See Import models.

Endpoint fields (POST/PUT …/endpoints):

Endpoint responses add health_status, consecutive_failures, alert_state, and the last check's time, latency, HTTP status and message.

Model capacity and auto-tune

PUT …/capacity-mode takes capacity_mode plus the optional capacity_source, capacity_namespace, capacity_service, per_replica_max_in_flight and capacity_headroom, with the same rules as PUT /api/v1/models/{id}. A discovered write the gateway cannot discover is refused with 400. See Pool sizes.

POST …/autotune probes the upstream directly (bypassing admission) for chat and embedding models, within 60 s and 20000 requests, and writes nothing. See Capacity auto-tune.

The report has recommended_max_in_flight, knee_reason (latency_degraded, plateau, max_concurrency, no_data), baseline_p99_ms, latency_ceiling_ms, recommended_throughput_rps, and per-step results. …/autotune/apply takes {"max_in_flight": N}.

Other per-model bodies:

POST /api/v1/models/health returns checked and skipped. …/health/activate returns enabled, activated and the health detail; an already-enabled model gets a manual check.

Fairshare and capacity

Task pages: Fairshare engine, Pool sizes.

mode, in /fairshare/live, /stats and /capacity/discovery, is how the answering replica enforces limits:

cluster_in_flight is the count across every replica in shared mode and null otherwise; every other count is the answering replica's own. See Multiple gateway replicas.

/fairshare/history takes since_ms (default: start of retention) and model (absent sums all pools). It returns interval_ms (2000), retention_ms (OBLETH_FAIRSHARE_HISTORY_SECS), oldest_ts_ms, and points of ts_ms, in_flight, queued, groups. /capacity/services takes ?model= (name or id) to add default_service and default_match; an unknown model is 404, and the listing is cached for 15 s.

Settings & alerting

Settings live in Postgres; the data plane picks up a change within 15 s.

Every field in the table below is optional on PUT.

Task pages: Alerting, Auto model routing, Boons, Speculation, Knowledge base, Energy accounting, the Playground assistant.

GET /api/v1/settings/auto-router also returns available_tags. /router/simulate takes prompt or messages, and optionally max_tokens, tenant_id, effort, needs_function_calling, needs_tool_choice, needs_response_schema, weight overrides, busyness, uniform and classify; it returns the scored candidates and rejections without sending traffic. /router/readiness returns findings (severity, code, title, detail, models), pool_size, classifier_active, difficulty_enabled. /settings/energy/test takes prometheus_url and any of power_query, model_power_query, model_power_label and model_requests_query, and returns cluster_watts and node_count, plus models (total_watts, series, matched, unmatched_series, requests_in_progress) when a per-model query is given. /settings/compressor returns configured, url, reachable, model, revision, error.

Slurm provisioner

Task page: Slurm provisioning. Replica state is pending, starting, healthy, draining or lost.

PUT /api/v1/settings/slurm takes enabled, slurmrestd_url, slurmrestd_api_version (default v0.0.40), slurm_user, slurm_jwt (omit or "" keeps it), node_aliases ([{host, ip}], the full list), cluster_defaults (cache_dir, images_dir, log_dir, setup and an images map of engine to image; omit to keep) and hf_token (omit or null keeps it, "" clears it). GET returns jwt_set, jwt_last4, cluster_defaults, hf_token_set and hf_token_last4, plus the provisioner's provisioner_running, provisioner_last_seen_secs, version, git SHA, build time, provisioner_tick_status (ok, idle, error), provisioner_tick_detail, provisioner_last_ok_secs and provisioner_held_secs. The test returns jwt (set, expired, expires_at, expires_in_secs) and ping (ok, status_code, latency_ms, error). /settings/slurm/resolved returns the JWT and the Hugging Face token in full, for the provisioner.

GET /api/v1/deployments/launches takes model_id, recipe_id and limit (50, at most 500). Each row is one replica's launch, kept after the replica is gone: a snapshot of the spec, the allocated nodes, submitted_at, started_at, healthy_at, ended_at, and end_state (Slurm's own, such as TIMEOUT or OUT_OF_MEMORY, gone, or cancelled:*) with end_reason, plus the computed queued_secs, load_secs and ran_secs.

Launch spec (PUT …/managed):

Knowledge, MCP, audit, and backup

Knowledge

PUT also takes chunk_tokens (400) and chunk_overlap_tokens (50, less than chunk_tokens). A collection reports indexed_embedding_model, embedding_dim, needs_reindex, chunk_count, estimated_bytes. Uploads take title, filename and content_base64 for a UTF-8 .md, .txt or .csv file within max_upload_bytes; a document's status goes pending, indexing, ready or failed. Search takes query and top_k and returns hits with score, text and would_inject; it is 400 until the collection is indexed. See Knowledge base.

MCP servers

A name is 1–64 letters, digits, ., _ or -, starting with a letter or digit. PUT requires upstream_url and takes auth_header (omit or "" keeps it), clear_auth: true to remove it, enabled, and name (a rename carries into every model's grant). Responses show auth_header_set. See MCP gateway.

Audit

Rows have id, ts, actor, action, entity_type, entity_id, detail. See Audit log.

Backup

A backup holds fairshare groups, tenants, keys (hashes only), models, endpoints, MCP servers and every settings row, not knowledge collections. Secrets are carried as stored; restore refuses a backup whose key_check does not match the target's OBLETH_ENCRYPTION_KEY, warns on an API-key pepper mismatch, and returns inserted and updated counts plus warnings. See Backup and restore.