obleth

You needNothing extra; the neural sidecar is optional

Turn it onSettings › Boons › Compression, then the model's page › Boons

It costsNo helper calls; it lowers the tokens the model is billed for

Compression runs before the request goes to the model's backend, so the savings cut the input tokens billed and the time to first token. It makes no model calls of its own.

Turn it on

1
Turn Compression on
Open Settings › Boons and switch Compression on. With nothing else set, it only compacts JSON, which is lossless. Open the row to add more under What it does: Code compaction, Log compaction, Cross-turn dedup and Lossy. These are the defaults for every tenant that has no settings of its own. Save with the save bar.
2
Grant it to the model
On the model's page, switch Compression on under Boons and save.
3
Adjust it per tenant (optional)
On a tenant's page, under Compression, switch on Its own compression settings. Then set Compression on or off for that tenant, and each of Code compaction, Cross-turn dedup, Log compaction and Allow lossy. Switch it off again to follow the gateway's defaults.

Or through the Management API:

curl -X PUT http://localhost:9180/api/v1/settings/boons \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "compression_enabled": true, "compression_dedup": true, "compression_compact_logs": true }'

curl -X PATCH http://localhost:9180/api/v1/tenants/$TENANT_ID/compression \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "policy": { "enabled": true, "code_compaction": true, "dedup": true, "compact_logs": true, "allow_lossy": false } }'

Send { "policy": null } to clear a tenant's settings.

What each pass does

Pass	What changes	Lossless?	On by default
JSON	An array of similar objects becomes an `OBLETH_TABLE` (a header plus CSV rows); other JSON is minified. Each rewrite is checked to reconstruct the original.	Yes	Yes, whenever the boon runs
Code compaction	Trailing whitespace stripped and runs of blank lines collapsed in fenced code	Whitespace only	No
Cross-turn dedup	A large block repeated across messages becomes a `[ref:HASH]` marker	The original is kept to retrieve	No
Log compaction	Repeated log lines collapse to one line with a count; errors and warnings stay	Near-lossless	No
Lossy	Long prose keeps its most important sentences, with a note of how many were left out	No; extractive, never rewritten	No

Every pass skips segments under Smallest input tokens, skips a segment that wouldn't get smaller, and never touches a trailing assistant message. The latest user message, older history and tool output are all eligible.

Compare with and without

Two request headers let you measure it on one request:

# Baseline: every boon off
curl -si http://localhost/v1/chat/completions -H "x-obleth-boons: off" ...

# Lossy forced on for this request
curl -si http://localhost/v1/chat/completions -H "x-obleth-boons: lossy" ...

lossy turns the lossy pass on even where the tenant hasn't; the model must still hold the boon. Whenever compression ran, the response says what it did:

x-obleth-compression: before=18423;after=11278;saved=7145

obench compression runs this comparison across sample payloads.

What the model sees

Compacted segments in place of the originals: OBLETH_TABLE blocks, [ref:HASH] markers, collapsed log lines and shortened prose. When dedup, log compaction or lossy replaced something, the original is kept in Redis for Originals kept, per tenant. If the model has function calling and the tool loop is on, the gateway also adds a retrieve_original tool and a short note, so the model can fetch the full text by its hash. The gateway answers that call itself from Redis; it never reaches the backend or the client. An expired or unknown hash returns a "no longer available" message.

Without function calling or the tool loop, compaction still happens; only the tool is left out.

Settings

Settings label	Field	Default
Compression on	`compression_enabled`	`false`
Code compaction	`compression_code_compaction`	`false`
Cross-turn dedup	`compression_dedup`	`false`
Log compaction	`compression_compact_logs`	`false`
Lossy	`compression_allow_lossy`	`false`
Smallest input	`compression_min_tokens`	`128` tokens
Pieces	`compression_max_segments`	`64` segments per request for JSON and code
Lossy pieces	`compression_max_lossy_segments`	`4` segments per request for lossy prose
Originals kept	`compression_original_ttl_secs`	`3600` s
Prose kept	`compression_neural_keep_ratio`	`0.5` of sentences, above 0 up to 1

A 0, or a Prose kept outside its range, keeps the stored value.

The optional neural sidecar

By default the lossy pass ranks sentences with a built-in heuristic (length, density and overlap with the question). The compressor sidecar ranks them with a trained extractive model instead, served on CPU in a container you run, so no text leaves your infrastructure. The gateway still splits sentences, picks what to keep and stores the original; the sidecar only scores. It makes one batched call per request, and on a timeout or any error the heuristic is used.

Docker Compose: add compressor to COMPOSE_PROFILES and set OBLETH_COMPRESSOR_URL=http://compressor:8080 in deploy/docker/.env. OBLETH_COMPRESSOR_TIMEOUT_MS defaults to 800. The first build bakes a model of about 600 MB into the image.
Helm: set compressor.enabled: true. The chart deploys the sidecar and sets OBLETH_COMPRESSOR_URL for you.

See Install with Docker Compose for the sizing values. Settings › Needs you says when a configured sidecar doesn't answer.

Who gets which pass

JSON compaction runs on chat requests when the boon is on, the model holds it, and the tenant hasn't turned compression off. Code compaction, dedup, log compaction and lossy each follow the tenant's own settings when it has them, and the gateway defaults otherwise. They run on any model; none needs function calling. Internal probe keys are never compressed.

Metrics and traces

obleth_compression_tokens_saved_total counts tokens saved. With tracing on, one boon:compression span per request records json_compacted, dedup_refs, log_segments, lossy_segments and the token totals before and after.

PreviousKnowledge

NextAuto routing