About
obleth is a community project for teams that share model servers with a fixed capacity: research labs, platform teams, and anyone running their own models instead of a managed API. When several groups share one pool, who gets the next slot matters more than another routing layer.
Fairness is an admission decision
When a model's pool is full, obleth decides who gets the next slot before work reaches the backend. A rate limit rejects; fairshare queues and weighs.
Identity on every request
Every request resolves to a tenant and key. Raw key secrets are never stored, and every usage row carries its tenant, key, admission and token counts.
Next to your serving stack, not instead of it
obleth handles tenants, fair share, routing and accounting. vLLM, SGLang, an inference gateway, or any OpenAI-compatible server keeps running the models.
Queue before reject
Under load, requests wait in a weighted queue rather than failing at the door. Budgets are the hard limit, not contention for slots.
Get involved
Run the quick start, then open the dashboard to watch pools, queues and usage under load. If something doesn't match these docs, open an issue.