Browser
Let the model search the web and read pages in a real Chrome tab that you watch beside the chat, and take over when a page needs you.
3 min readUpdated for Chat 0.5.2Edit on GitHub
Turn it on
Press Browser in the composer. If you ask for something that needs a real page in a chat without it, the model asks first: its reply ends with Turn on browser, which turns it on and carries on with your request.
Ask for something to look up: "what changed in the latest vLLM release?" The model finds pages, opens the promising ones and reads them before it answers, and says which pages it used. When your installation gives models web search (an Obleth gateway with search on), the model searches with that, which is faster, and uses the browser to read and use the pages. Without it, or when a search finds nothing useful, it searches in the browser. The tab appears in the Browser tab of the side pane, live, so you can watch it work. In a chat with a workspace, Browser sits next to Preview, Files and Terminal.
Each step of the reply shows what the model did (searched, opened, clicked, typed) with a small picture of the page at that moment. They group under Browse in the reply's work.
Take over
Press Take over to drive the tab yourself: your clicks, scrolling, typing and pasting go to the page, and the address bar opens any address. The model is paused while you have control. It gets nothing from the tab, no page text and none of your typing, and its browser tools refuse to run.
The model hands you the tab itself when a page needs you: a sign-in, a password, a captcha or bot check, a payment or a choice only you can make. Its reply ends with Your turn in the browser and what to do. When you're done, Hand back returns the tab and the chat carries on.
The model never types passwords. Sites you sign in to keep you signed in, so it can read pages behind your sign-in, such as a journal your library pays for.
What asks first
Reading, scrolling, searching and following links never ask. Before the model submits a form, or presses a button that buys, sends, posts, publishes or deletes something, the step waits with the question, such as "Press “Submit order” on shop.example?":
- Allow once runs this step.
- Always on this site runs it and doesn't ask again for that site.
- Deny skips it, and the model is told not to try another way.
A chat in Auto mode doesn't ask.
Text on web pages is information for the model, never instructions: it's told to ignore anything on a page that asks it to do something you didn't ask for.
Your browser
Settings › Personal › Browser shows whether your browser is running and its tabs, one per chat. There you can:
- Search with DuckDuckGo, Google or Bing, for the searches the model runs in your browser.
- Remove sites you allowed with Always on this site.
- Sleep now stops the browser. Your sign-ins are kept.
- Forget all browser data deletes sign-ins, cookies and history, closes every chat's tab and removes the sites you allowed.
Your browser sleeps after 15 minutes unused and starts again, in a few seconds, when the model browses or you press Open the browser. Watching the tab keeps it awake. Its data is deleted 7 days after you last used it.
Downloads aren't kept yet: a link to a file such as a PDF doesn't open. The model gives you the link instead.
How the browser is isolated
Your browser is a headless Chromium in its own pod, beside your workspace, under the same rules: it reaches only public web addresses (not Chat's network, the cluster or your LAN), and only Chat can connect to it. Each person's browser and sign-ins are separate. The live view accepts connections only from Chat's own origin, and only the chat's owner can watch or drive the tab.
What the model sees
Each page comes back as text: headings, paragraphs, list items and table rows, with every link, button and field numbered, such as [12] link "Release notes" → github.com/vllm-project/vllm/releases. The model clicks and types by number. Long pages come in windows of about 7,000 characters, which it reads on with browser_read. Password fields show only whether they hold something.
Administrators turn the browser on alongside workspaces; see Enable workspaces.
