A desktop and browser for your AI agent

Set up the desktop#

Deploy does this for you. By hand, it's one script that installs the desktop, Chrome, noVNC, Node.js 24, and Playwright MCP, and starts them as systemd services.

ssh exe.dev new --name=my-computer-use
ssh my-computer-use.exe.xyz

curl -fsSL https://exe.dev/deploy/computer-use/setup.sh | bash

curl -s localhost:9222/json/version

Watch the desktop#

Open https://my-computer-use.exe.xyz:6080/ in your browser and sign in to exe.dev if asked. noVNC connects straight away and shows the desktop, scaled to the tab, with Chrome open. It says Connected (encrypted): the connection is HTTPS from your browser to exe.dev, and VNC from there stays inside the VM.

The screen is shared: you see the agent's clicks as they happen, and you can click and type too. The tab on the left edge opens noVNC's controls, including a clipboard box for pasting text into the desktop and full screen. To let someone else watch, share the VM with them: ssh exe.dev share add my-computer-use teammate@example.com on your computer. They sign in to exe.dev and open the same address.

Warning

The desktop has no password of its own; exe.dev login is the lock. Don't forward port 6080 elsewhere or point port 8000 at it with share port and share set-public, or anyone could use it.

Let a coding agent drive the browser#

Playwright MCP gives an agent tools such as browser_navigate, browser_click, browser_type and browser_snapshot, which reads the page's accessibility tree rather than pixels. It drives the desktop's Chrome, so you watch each step on the desktop, and pages you have signed in to are signed in for the agent too.

On the VM, connect Claude Code to the LLM integration (if you haven't) and to Playwright MCP:

exeuntu configure claude
claude mcp add --transport http --scope user playwright http://localhost:8931/mcp
claude mcp list

The list shows playwright: http://localhost:8931/mcp (HTTP) - ✔ Connected. Try it while you watch the desktop:

claude -p "Use the playwright tools to open https://en.wikipedia.org/wiki/Alan_Turing and tell me his date of birth." --allowedTools=mcp__playwright

Chrome goes to the page, and Claude answers 23 June 1912. In an interactive claude session, it asks before using each tool unless you allow them.

For Codex:

exeuntu configure codex
codex mcp add playwright --url http://localhost:8931/mcp

Codex asks you to approve each browser action. To let it browse without asking, for codex exec for example, add default_tools_approval_mode = "approve" under [mcp_servers.playwright] in ~/.codex/config.toml.

Any other agent that speaks MCP over HTTP works the same way with http://localhost:8931/mcp. Programs can skip MCP and attach to Chrome themselves: Playwright's chromium.connect_over_cdp("http://localhost:9222") and Puppeteer's puppeteer.connect({ browserURL: "http://localhost:9222" }) both do.

Run a computer-use agent#

For anything that isn't a web page, or when you want the agent to see what a person sees, use computer use: the model gets screenshots of the whole desktop and answers with clicks and keystrokes. agent.py is a short script that runs Claude's computer use tool on this desktop. It uses the Anthropic Python SDK, with xdotool doing the clicks and typing. It asks before every action, prints each step as it happens, and records the screen. On the VM:

curl -fsSLO https://exe.dev/deploy/computer-use/agent.py
uv run agent.py "Open the file manager from the dock, go into the agent-runs folder, and tell me how many folders are in it."

uv installs the SDK the first time. The model comes through the LLM integration, so there is no key to set; the default is claude-sonnet-5-5, and --model or MODEL picks another. With the desktop open in a tab beside it, you watch Claude open the file manager while the terminal shows each step:

Task: Open the file manager from the dock, go into the agent-runs folder, and tell me how many folders are in it.
Model: claude-sonnet-5-5. Run kept in /home/exedev/agent-runs/2026-10-29-010838
  > screenshot
  > left_click (615, 776)   run it? [y/n/a] y
  > wait duration=2
  > screenshot
  > double_click (723, 166)   run it? [y/n/a] y
  > wait duration=2
  > screenshot
Claude: The agent-runs folder contains 1 folder, named "2026-10-29-010838". ...
Tokens: 26,250 in, 337 out.
Recording: /home/exedev/agent-runs/2026-10-29-010838/screen.mp4

Screenshots, zooms and waits run without asking. For anything else, answer y to run it, n to refuse it (Claude is told, and usually stops to ask you), or a to run it and everything after. --yes approves everything from the start. Asking is the right default for an agent on the open web: a page can contain instructions aimed at it. Ctrl-C stops a run, and --max-steps (default 50) caps how many turns Claude gets.

Review a run#

Each run keeps a folder in ~/agent-runs/: log.txt has the task, every action, and Claude's replies, with times, and screen.mp4 is a recording of the desktop at 5 frames a second, from start to finish. A minute of recording is a few hundred kilobytes. Copy one to your computer to watch it:

scp my-computer-use.exe.xyz:agent-runs/2026-10-29-010838/screen.mp4 .

--no-record skips the recording. To record anything else on the desktop, such as a Claude Code session driving Chrome, run this on the VM and press q to stop:

ffmpeg -f x11grab -framerate 5 -video_size 1280x800 -i :1 -c:v libx264 -preset veryfast -crf 30 -pix_fmt yuv420p ~/session.mp4

Build your own#

agent.py is a starting point: the SDK's computer toolset does the model side, and the Desktop class, about a hundred lines, turns its calls into xdotool commands and screenshots of display :1. Its SYSTEM prompt tells Claude about the desktop; tips that save Claude steps on your tasks belong there. Anthropic's computer toolset quickstart has a driver that talks VNC instead, so it also works from another computer (see below), and the SDK guide covers running the toolset safely.

OpenAI's computer use tool works through the same integration: send Responses API requests with "tools": [{"type": "computer"}] to https://llm.int.exe.xyz/v1/responses with a model that supports it, such as gpt-5.5. OpenAI leaves the driver to you, and the Desktop class's xdotool commands and screenshots carry over.

Connect an agent from elsewhere#

The agent doesn't have to run on the VM. SSH forwards the desktop's ports to your computer, without opening them to anyone else. On your computer:

ssh -N -L 9222:localhost:9222 -L 5901:localhost:5900 -L 8931:localhost:8931 my-computer-use.exe.xyz

While that runs, http://localhost:9222 is the VM's Chrome for Playwright or Puppeteer on your computer, http://localhost:8931/mcp is its Playwright MCP for a local agent, and localhost:5901 is the VNC server, for a VNC client or a computer-use driver such as Anthropic's quickstart (VNC_PORT=5901). Port 5901 rather than 5900 keeps clear of a VNC server on your own computer, such as macOS Screen Sharing. The quickstart's driver sends every screenshot uncompressed, about 4 MB at 1280×800, so it is noticeably slower over the internet than on the VM. agent.py needs the display itself, so it runs on the VM.

Often simpler is to leave the agent on the VM and start it remotely: ssh my-computer-use.exe.xyz 'claude -p "..."' or ssh -t my-computer-use.exe.xyz 'uv run agent.py "..."' (-t, so it can ask you before each action), or from a program with exe.dev's HTTPS API. To run many tasks side by side, give each its own copy of the VM; see the FAQ.

Back up and restore#

The only state worth keeping is Chrome's profile in ~/.config/agent-chrome (cookies, signed-in sessions, history) and any files the agent made. To copy the whole VM, on your computer:

ssh exe.dev cp my-computer-use my-computer-use-backup

The copy starts with the desktop, Chrome, and its profile, ready to use at https://my-computer-use-backup.exe.xyz:6080/. To carry just the profile to another desktop, stop Chrome, copy the folder, and start it again:

sudo systemctl stop desktop-chrome
tar -C ~/.config -czf ~/agent-chrome.tgz agent-chrome
sudo systemctl start desktop-chrome

Restore by stopping Chrome there, unpacking it into ~/.config, and starting Chrome again.

Update#

On the VM:

sudo apt-get update && sudo apt-get upgrade -y          # Chrome, Xfce, noVNC, Node.js
sudo npm install -g @playwright/mcp@0                    # Playwright MCP
sudo systemctl restart desktop-chrome desktop-playwright-mcp
curl -fsSLO https://exe.dev/deploy/computer-use/agent.py  # the computer-use script

Troubleshooting#

• The tab says Failed to connect to server. systemctl status desktop-novnc desktop-vnc shows which half is down; journalctl -u desktop-vnc -n 50 says why.
• A grey screen with no panel. The Xfce session stopped: sudo systemctl restart desktop-xfce, and see journalctl -u desktop-xfce -n 50.
• Chrome doesn't come back, or curl localhost:9222 fails. journalctl -u desktop-chrome -n 50. Chrome's own D-Bus errors in that log are harmless.
• claude mcp list says Failed to connect. Check systemctl status desktop-playwright-mcp; it needs Chrome on 9222. If a browser tool fails after Chrome restarted, ask the agent to try again: Playwright MCP reattaches on the next call.
• agent.py says error: X connection failed. The desktop isn't running: sudo systemctl restart desktop-vnc desktop-xfce. If it says authentication_error or 401, the LLM integration isn't attached to this VM, or ANTHROPIC_BASE_URL or ANTHROPIC_API_KEY is set to something else in your shell; see the LLM integration.
• Claude types a web address onto the end of the old one. When Chrome's address bar already has the focus, a click there only moves the cursor rather than selecting the address. Claude notices on the next screenshot and fixes it, at the cost of a few steps; agent.py's SYSTEM prompt tells it to press Ctrl+L first, which avoids it. Tips like that for your own apps go there too.
• The agent clicks in the wrong places. Keep the screen at most 1920×1200, which agent.py sends unscaled, and let the agent take a fresh screenshot after every page load.