Research
Run a tree of measured experiments from a chat, each idea a git branch of the workspace code, with every run charted on the project's Research tab.
3 min readUpdated for Chat 0.5.2Edit on GitHub
Research follows the playbook of OpenResearch (MIT), adapted for Chat: agree on one metric and one command, then try ideas as branches and keep what measures better.
Start research
- 1
Open a chat in the project
The chat needs a workspace with the code you want to improve.
- 2
Describe the goal
For example: "make the solver converge in fewer iterations". The model agrees with you on the metric (a
name=valueline the code prints), whether higher or lower is better, and the one command that runs it. - 3
Let it set up
The model calls
research_setup. The code as it is becomes the baseline, on the branchresearch/baseline.
A chat started from the project's Research tab gets a workspace and starts in Auto mode, so a long run doesn't stop to wait for Allow and has no step or time limit. It works until the goal is met or the ideas run out, and Stop ends it.
When the model suggests it
You don't have to start in a project. In any chat that isn't already running research, the model can suggest it when you want something measurably better and getting there takes several tries. Its reply ends with a Worth researching card showing the goal and the metric. Start research:
- Moves the chat into a new project named after the goal, or keeps its project when that project has no research yet
- Turns on a workspace for the chat
- Switches the chat to Auto mode
- Asks the model to set the research up
Not now leaves the chat as it is. The suggestion appears only for people who may use workspaces.
From there the model works in rounds. Each round tries a few alternatives to one decision as sibling branches, runs them, and the next round branches from the winner.
The rules it follows
- A measured node is frozen. Once a run measures a node's metric, later ideas branch a child from it.
- The command never changes. Only committed code differs between nodes, so results compare.
- A broken run can be fixed. A crash, a missing metric or a lost run leaves the node open to repair.
The Research tab
The project's Research tab fills once the model has agreed the metric and command with you and called research_setup. Until then, follow the setup in the chat.
The tab shows:
- The goal, and the best result against the baseline
- The tree, with each node's status
- A chart of every run, with the best so far
- For the selected node: its runs, the values they printed, the diff from its parent and the output
The owner can stop a running run from the tab. Members of a group the project is shared with see the research read-only.
Tools
| Tool | What it does |
|---|---|
research_setup | Records the goal, metric, direction and command, and makes the baseline |
experiment_create | Branches a child from a node and checks it out; uncommitted edits come along |
experiment_checkout | Switches to another node |
experiment_run | Commits the node's work and starts a run in the background |
experiment_wait | Returns when the first run finishes, with its result against its parent and the best so far |
research_status | Prints the tree |
run_log | Reads a run's output |
How runs work
Each run is a detached git worktree of its commit under /workspace/.obleth-research/<project>/runs/<n>/, outside the project folder. Untracked and ignored files (.venv, node_modules, data) aren't there, so the command must set up what it needs or use absolute paths.
A supervisor on every Chat replica checks running runs every 10 seconds. It keeps the workspace awake while they go, and records the last 32 KB of output, every name=value the run printed (up to 40), and the diff from the parent at launch. A run whose workspace stopped is marked lost.