OBLETHdocs

Quick start

Run obleth on your machine with Docker Compose, make a tenant and key, send a first request and open the dashboard.

2 min readUpdated for v2.4.0Edit on GitHub

You needDocker with Compose v2, git, curl and jq
BackendThe bundled benchmark backend, a simulated model server
TakesAbout five minutes

The example .env holds development passwords. Use this stack on your own machine only; for anything others can reach, follow Install with Docker Compose.

  1. 1

    Start the stack

    git clone https://github.com/thediymaker/obleth-gateway.git
cd obleth-gateway/deploy/docker
cp .env.example .env
docker compose pull
docker compose up -d

    The example file turns on the benchmark, edge and observability profiles, so this starts the gateway, the dashboard, the three datastores, the benchmark backend, HAProxy, Prometheus, Grafana and Jaeger. To build from your checkout instead of pulling images, run docker compose up -d --build.

    When docker compose ps shows the services healthy, check the gateway:

    curl -s http://localhost/health               # ok
curl -s http://localhost:9180/api/v1/health   # ok
  2. 2

    Create a tenant and a key

    The management API takes the admin token from .env (dev-admin-token in the example). A tenant is who gets a share of capacity; a key is what a client sends.

    TOKEN=dev-admin-token

TID=$(curl -s -X POST http://localhost:9180/api/v1/tenants \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"my-team"}' | jq -r .id)

SECRET=$(curl -s -X POST "http://localhost:9180/api/v1/tenants/$TID/keys" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"laptop"}' | jq -r .secret)

echo "$SECRET"

    The secret (sk_ and 48 hex characters) is returned once. Only its hash is stored.

  3. 3

    Register a model

    Point a model at the benchmark backend:

    curl -s -X POST http://localhost:9180/api/v1/models \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{
    "model_name": "benchmark-endpoint",
    "upstream_model": "benchmark-endpoint",
    "api_base": "http://benchmark-backend:8081"
  }'

    It gets its own fairshare pool of 32 slots. Every field is in Model registry.

  4. 4

    Send a request

    Use the tenant key, not the admin token. HAProxy on port 80 forwards to the gateway; port 8088 reaches it directly.

    curl -s http://localhost/v1/chat/completions \
  -H "Authorization: Bearer $SECRET" -H "Content-Type: application/json" \
  -d '{"model":"benchmark-endpoint","messages":[{"role":"user","content":"Hello"}],"max_tokens":32}'
  5. 5

    Open the dashboard

    Go to http://localhost:3002 and sign in with DASHBOARD_ADMIN_EMAIL and DASHBOARD_PASSWORD from .env (admin@example.com and obleth-admin in the example). Request Logs shows the requests you just sent; Fairshare shows the model's pool.

Ports

ServiceURL
HAProxy (client entry)http://localhost
Data plane, directhttp://localhost:8088
Management APIhttp://localhost:9180 (this host only)
Dashboardhttp://localhost:3002
Grafanahttp://localhost:3001

The full list is in Endpoints and ports.

If something fails

401, 403, 404 or 503 from the data plane
  • 401 invalid api key: you sent the admin token, or a key that doesn't exist. Send the sk_… secret.
  • 503 authentication backend unavailable: the gateway couldn't reach Redis. Check the redis container, and that REDIS_PASSWORD hasn't changed since the volume was created.
  • 404 model '…' is not registered: register the model (step 3) or fix the model field. A disabled model returns 403.
  • Connection refused on 8080: Compose publishes the data plane on 8088. Use http://localhost or http://localhost:8088.

More cases are in Troubleshooting.

Sign-in fails when you browse from another machine

Sign-in checks the browser's origin against BETTER_AUTH_URL (default http://localhost:3002). Set it to the URL you browse to, or list extra origins in TRUSTED_ORIGINS, then run docker compose up -d again. See Sign-in and SSO.

Stop the stack

docker compose down        # keep the data
docker compose down -v     # also delete every tenant, key, model and usage row

Next steps