Introduction

obleth is a lightweight, high-performance, fairshare-first AI gateway for self-hosted LLM environments. It sits between your front door (HAProxy) and your inference backend (Aibrix, vLLM, or any OpenAI-compatible API) and owns multi-tenant identity, contention-based weighted fair queuing, token-accurate cost accounting, and reliability.

The project targets operators running shared GPU clusters on their own hardware — research labs, internal platform teams, and anyone migrating off managed APIs to self-hosted models.

Start here

• Quick Start — Docker Compose, first API key, first chat completion (~5 min)
• Why obleth? — how it differs from LiteLLM and raw vLLM routing
• Fairshare engine — weighted queuing under saturation
• Saturation behavior — queue vs hard reject (429, 403, 503)

Popular guides

• Auto model routing — model: "auto" with capacity and cost signals
• OpenAI-compatible API
• MCP Gateway — front MCP servers behind obleth auth
• Control plane dashboard
• Modular deploy — Compose profiles and Helm toggles

Reference

• Management API
• Environment variables
• Prometheus metrics

The gateway source code lives in obleth-gateway. The repo includes Docker Compose, Helm manifests, the Rust data plane, the Management API, dashboard, benchmark fixture backend, and observability assets. See the License page for source-availability terms.