67 docs indexed
The in-dashboard assistant that benchmarks models, probes capabilities, verifies MCP servers, and answers questions from the documentation.
Charo is the assistant built into the control plane. It is not a chatbot bolted onto the sidebar: every answer it gives about your deployment comes from running real work against the gateway — a benchmark, a capability probe, an MCP handshake — and reporting what actually happened, including the request trace.
Open it from the launcher in any dashboard page. The panel is titled Gateway Chat and opens as a centered pop-out over the page you are on, so you never lose your place.
Charo needs a model to think with. Pick one under Settings → Assistant; any model registered in the gateway can serve as the brain, and the traffic it generates is ordinary gateway traffic subject to the usual routing, quotas, and accounting.
Until a brain model is set, Charo runs in a reduced mode: it relays your prompt to the model you name and returns the reply with its timings, but it does not run the agent loop and cannot use tools.
With a brain model configured, Charo runs an agent loop over four tools. The loop is capped at four iterations per turn, so a confused model cannot spin.
| Tool | What it does |
|---|---|
run_benchmark | Runs a concurrency-ramp capacity benchmark against a model served by this gateway. Returns per-step latency and throughput, a detected capacity knee, and a 0–100 score with an A–F grade, rendered inline as a capacity-curve card. |
test_capabilities | Probes a model's configured capabilities — quick ping, tool/web-search loop, JSON mode, vision — by sending each as a real request through the gateway, and reports which boons actually fired. |
test_mcp | Runs the real MCP handshake (initialize + tools/list) against /mcp/{name} and reports reachability, protocol info, and each server's tools. Omit the server list to sweep every registered server. See MCP Gateway. |
search_docs | Searches this documentation and answers grounded in it, citing the pages it used. It says so plainly when the docs do not cover something rather than guessing. |
The vision capability probe requires a real image. Attach one with the paperclip or drag it onto the panel; there is no bundled placeholder.
Tool runs are gated on the admin role. A non-admin signed into the dashboard gets a plain brain chat with no tools — the tool-run and agent endpoints refuse the call rather than downgrading silently.
Benchmarking, capability testing, and chatting with a specific model are also available as step-by-step workflow cards in the chat thread, where you pick the model and options inline. Asking in plain language opens the matching workflow — "test gemma4" starts the capability workflow against that model.
Chatting with a named model gives you a raw, persona-free line to it. A banner shows which model you are talking to; exiting returns you to Charo.
Settings → Assistant, or the Management API at
GET/PUT /api/v1/settings/charo.
| Field | Default | Meaning |
|---|---|---|
enabled | true | Master switch for the assistant. |
brain_model | unset | Model Charo thinks with. Unset leaves it in the reduced mode described above. |
tools_enabled | all enabled | Per-tool enable/disable, keyed by tool name. A missing key means enabled. |
bench_max_concurrency | 40 | Ceiling on concurrency run_benchmark will ramp to. |
bench_max_duration_s | 120 | Ceiling on total benchmark duration. |
bench_max_requests | 500 | Ceiling on total requests issued by a benchmark. |
The three benchmark caps are enforced on the server, not just in the UI, so a prompt cannot talk Charo into a larger load test than you allow.
Benchmarks and capability probes reach the gateway as real requests. Run them from a tenant flagged synthetic and their traffic is recorded as benchmark traffic, excluded from usage and cost stats by default, and never written to the permanent daily rollup. See Multi-tenancy.
search_docs runs against an index of this documentation site that is built
into the dashboard image, so answers work without outbound network access. The
index is a point-in-time snapshot: it is regenerated when the documentation is
rebuilt, and a dashboard image will answer from the documentation as it stood
when that image was built.