Energy & Carbon Accounting

When you self-host, you own the power bill — and no cloud gateway can tell you what a request actually cost in electricity. obleth can, because it runs next to your cluster: it reads real power draw from your Prometheus, attributes each completed request its share, and freezes three values into the usage ledger alongside cost_usd:

The numbers roll up everywhere cost does — per request, per tenant, per model, per day — so questions like "this course consumed 4.2 kWh this term" are answered with the same fidelity as spend reporting.

Everything is off by default and fails open: until you configure it, and for any model that doesn't opt in, the three fields are simply 0.

How attribution works

obleth uses amortized slot-share with declared saturation. You state, per model, how many concurrent requests saturate one node (energy_slots_per_node). A request is then charged its wall-time share of one slot at the cluster's average per-node power:

serving_ms     = total_ms − queue_wait_ms          (queued time holds no slot)
watts_per_slot = cluster_watts / node_count / energy_slots_per_node × PUE
energy_wh      = watts_per_slot × serving_ms / 3,600,000
energy_cost    = energy_wh / 1000 × energy_cost_per_kwh
co2_g          = energy_wh / 1000 × carbon_g_per_kwh

cluster_watts and node_count come from your Prometheus power query (below), refreshed on a poll interval. PUE (Power Usage Effectiveness) is an optional facility-overhead multiplier for cooling and power delivery — 1.0 (default) charges bare device power; typical data centers run 1.1–1.6.

A worked example: a query reporting 44,800 W across 64 nodes is 700 W per node. A model declared at 8 slots per node gives 87.5 W per slot, or 105 W after a PUE of 1.2. A request that spends 2.4 s being served is charged 105 × 2400 / 3,600,000 = 0.07 Wh — at $0.12/kWh and 400 gCO₂/kWh, that's $0.0000084 and 0.028 g of CO₂.

What the numbers deliberately leave out

The model is designed to never over-attribute past physical draw, at the cost of understating:

• Idle power is unattributed. Nodes burning watts while serving nothing are nobody's requests. Totals therefore understate the utility bill — never overstate it.
• Cluster averaging smears busy and idle nodes. A request on a hot node and one on a cold node are charged the same average rate.
• Queue time is free. A request waiting in admission holds no serving slot, so only its serving time is charged. (serving_ms includes a small amount of gateway overhead.)

Treat the results as a defensible lower bound suitable for showback and sustainability reporting, not a metered invoice.

Enable it

Open Settings → Energy in the dashboard (or use the Management API):

Use the Test query button before enabling: it runs the query once against Prometheus and shows a live "X kW across N nodes". On failure it surfaces the raw Prometheus error string verbatim — that's your debugging tool for a bad URL or expression.

Settings apply live — the poller picks up changes without a restart, including turning the feature on for the first time.

Choosing a power query

obleth only ever computes sum() and count() over your expression, so any exporter that exposes a per-device or per-node power gauge works:

If the metric is per-accelerator rather than per-node, either aggregate it yourself (sum by (node) (...)) or declare slots per device consistently — the math only needs cluster_watts / node_count to mean "average power of the unit you declared slots against." Filter with label selectors (habana_device_power_watts{cluster="prod"}) to scope mixed clusters.

Per-model energy slots

Each model carries an energy_slots_per_node count (model form in the dashboard, or the models API): how many of this model's requests one node can serve at once when fully loaded — replicas per node × concurrent sequences per replica. For example, 2 vLLM instances per node each serving 4 concurrent sequences declares 8.

0 (the default) opts the model out entirely — its requests record zeros. Leave external API-backed models at 0: their energy isn't yours to meter.

Frozen at completion

Energy follows the same immutability rule as cost_usd: computed once when the request completes, from the settings, power reading, and slot count in effect at that moment, then stored. Editing rates, PUE, slots, or the query later changes only future requests — history is never rewritten, and there is no backfill. Rows written before the feature existed read as 0.

Where the numbers appear

• Request Logs — an Energy column (Wh) per request; the detail view adds Energy cost and CO₂.
• Reports — an Energy KPI (kWh) and CO₂ KPI over the selected range, plus optional Energy (kWh) / CO₂ (g) / Energy cost (USD) columns in the breakdown table by day, tenant, key, or model. Export CSV includes them.
• Management API — energy/carbon sums ride beside cost_usd in /usage, breakdowns, series, and /usage/daily.
• ClickHouse — raw columns on usage and summed columns on the permanent usage_daily rollup; see ClickHouse Usage.

Failure behavior

The module's contract is fail-open: energy accounting can never delay or fail a request. Requests record all-zero energy whenever any input is missing — the feature is disabled or incompletely configured, the model's energy_slots_per_node is 0, no power reading has arrived yet, or the query returned no nodes or non-positive watts.

If a poll fails, the gateway keeps using the last good reading and raises a deduplicated alert (key energy_power_poll_failed) instead of zeroing live traffic. The poller runs continuously — while the feature is inactive it just rechecks settings every 15 seconds, which is why enabling it needs no restart.

Management API

# Configure and enable
curl -X PUT http://localhost:9180/api/v1/settings/energy \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "prometheus_url": "http://prometheus:9090",
    "power_query": "habana_device_power_watts",
    "poll_interval_secs": 60,
    "energy_cost_per_kwh": 0.12,
    "carbon_g_per_kwh": 400.0,
    "pue": 1.2
  }'

# Dry-run a query without saving anything
curl -X POST http://localhost:9180/api/v1/settings/energy/test \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"prometheus_url": "http://prometheus:9090", "power_query": "habana_device_power_watts"}'

POST /settings/energy/test returns the reading or the raw Prometheus error:

{ "cluster_watts": 44800.0, "node_count": 64 }

Set a model's slot count on create or update alongside its other fields:

curl -X PUT http://localhost:9180/api/v1/models/$MODEL_ID \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "energy_slots_per_node": 8 }'

Troubleshooting zeros

Work down this list — every zero is one of these by design:

1. Settings incomplete — active means enabled and a Prometheus URL and a non-empty power query. An empty query silently disables the whole feature.
2. Model opted out — energy_slots_per_node is 0 for the model that served the request.
3. No reading yet — the first poll hasn't completed; wait one poll interval.
4. Prometheus unreachable from the gateway — check for the energy_power_poll_failed alert and remember the container-reachability note above. The Test query button reproduces the poller's view exactly.
5. Query returns nothing — sum()/count() over your expression came back empty or non-positive.

Old rows from before the feature (or before a model opted in) are permanently 0 — frozen history is never backfilled.