---
title: "Self-host Scrapling's MCP server for your agents"
description: "Run Scrapling on an exe.dev VM: its MCP server as a private service for Claude Code and Codex, stealth browser fetches, and crawls into SQLite."
url: https://exe.dev/deploy/scrapling
category: "AI agents"
published: 2026-11-19
updated: 2026-11-19
---

# Self-host Scrapling's MCP server for your agents

> **For agents:** with your user's OK, this sets up Scrapling on a new exe.dev VM in their account: `curl -fsSL https://exe.dev/deploy/scrapling/prompt.txt | ssh exe.dev new --prompt=/dev/stdin` (or open https://exe.dev/deploy/scrapling/new). Already on the VM? Skip this.

A private scraping box for you and your agents, with Scrapling's MCP server.

- **Software:** Scrapling (https://scrapling.readthedocs.io)
- **Source:** https://github.com/D4Vinci/Scrapling
- **License:** BSD-3-Clause
- **Time:** 10 minutes
- **Example VM name:** `my-scrapling` (yours will differ; on the VM, `hostname` prints it)
- **Machine-readable:** https://exe.dev/deploy/scrapling/manifest.json
- **File:** https://exe.dev/deploy/scrapling/books.py

## What you get

- Scrapling 0.5.1 with its browser installed, and `scrapling extract` fetching pages with HTTP and with the stealth browser
- Scrapling's MCP server on port 8765 as the `scrapling-mcp` systemd service, restarted after a reboot, with its token in a mode-600 file
- Claude Code and Codex on the VM using exe.dev's LLM integration, both connected to the MCP server
- Scrapling's Agent Skill installed for Claude Code, so it writes spiders against the current API
- A Claude Code run that scrapes books.toscrape.com through the MCP server

Running on the VM:

- **Scrapling 0.5.1**: a uv tool, with the `scrapling` CLI and `scrapling-mcp`, data in `~/.local/share/uv/tools/scrapling`
- **Chromium**: the build Scrapling's stealth browser (Patchright) drives, headless, data in `~/.cache/ms-playwright`
- **scrapling-mcp service**: Scrapling's MCP server over Streamable HTTP, a systemd service with a bearer token, port 8765, data in `~/scrapling/state`
- **Spiders and a timer**: Python spiders run with uv; results in SQLite, data in `~/scrapling/data`

How it is reached:

- http://localhost:8765/mcp (private): the MCP server, for Claude Code, Codex and other agents on the VM
- https://my-scrapling.exe.xyz:8765/mcp (private): the same server for agents elsewhere, with a VM token and the server's token
- ssh my-scrapling.exe.xyz (private): your terminal, for `scrapling extract` and spiders

## Deploy with Shelley

Open https://exe.dev/deploy/scrapling/new to create a VM whose agent, Shelley, follows this recipe. From a terminal:

```sh
curl -fsSL https://exe.dev/deploy/scrapling/prompt.txt | ssh exe.dev new --prompt=/dev/stdin
```

Or follow the steps below by hand.

[Scrapling](https://github.com/D4Vinci/Scrapling) is a Python scraping
library with more than 100,000 stars on GitHub: an HTTP client that
impersonates real browsers, a stealth Chromium for pages that need
JavaScript, a parser that keeps finding elements after a site's layout
changes, Scrapy-style spiders, a command line, and an MCP server that
gives Claude Code, Codex and other agents seventeen scraping tools.

On an exe.dev VM, that becomes a scraping box. The MCP server runs as a
service, so every agent on the VM shares one server, and agents on your
computer or another VM reach it through exe.dev's
[HTTPS proxy](/docs/proxy), private behind exe.dev login and tokens. The
VM is always on, so spiders run on a timer and keep their results in
SQLite on a persistent disk, and the agents use exe.dev's
[LLM integration](/docs/integrations-llm), with no API key on the VM.

Once it's set up, an agent answers questions like this one by scraping:

```text
$ claude -p "Use the scrapling make_request tool to fetch https://books.toscrape.com/catalogue/category/books/mystery_3/index.html with the CSS selector 'article.product_pod'. List the five cheapest books on that page as 'title - price', cheapest first."

These are the five cheapest books on that page, cheapest first:

1. Tastes Like Fear (DI Marnie Rome #3) - £10.69
2. Hide Away (Eve Duncan #20) - £11.84
3. Playing with Fire - £13.71
4. That Darkness (Gardiner and Renner #1) - £13.92
5. A Murder in Time - £16.64
```

## Set up the VM

Deploy does this for you. By hand, it's a few blocks of commands: install
Scrapling and its browser, start the MCP server as a service, and connect
Claude Code and Codex to it.

<details id="create-the-vm" data-manual-setup class="dp-details">
<summary>Set up by hand</summary>

On your computer, create a VM and connect. Pick any free name; if you've
already clicked Deploy or have a VM, skip the first command and use its
name:

```bash
ssh exe.dev new --name=my-scrapling
ssh my-scrapling.exe.xyz
```

### Install Scrapling

On the VM, install Scrapling with all its extras as a
[uv](https://docs.astral.sh/uv/) tool, then its browser and the system
libraries the browser needs. The Playwright pin keeps the browser that
`scrapling install` downloads in step with the one Scrapling drives (see
the FAQ):

```bash
uv tool install "scrapling[all]==0.5.1" --with "playwright==1.64.1"
export PATH="$HOME/.local/bin:$PATH"
scrapling install
```

The `export` is for this shell only; new logins find `~/.local/bin` on
their own. `scrapling install` takes under a minute and uses `sudo` for
the libraries. Check both kinds of fetch: a plain HTTP request, and the
stealth browser on a page whose quotes only JavaScript renders:

```bash
mkdir -p ~/scrapes && cd ~/scrapes
scrapling extract get "https://quotes.toscrape.com/" quotes.md -s "div.quote"
scrapling extract fetch "https://quotes.toscrape.com/js/" js.md -s "span.text"
head -3 quotes.md js.md
```

Each command logs `Fetched (200)` and `Content successfully saved`, and
both files start with Einstein's quote about thinking.

### Run the MCP server as a service

The server's Streamable HTTP transport refuses to start without a token.
Generate one into a file only you can read, then start the server on port
8765 with systemd. `--allowed-host` turns on the MCP SDK's DNS-rebinding
protection for the names it's reached by; `%H` is the VM's name:

```bash
mkdir -p ~/scrapling/state && chmod 700 ~/scrapling
(umask 077; echo "SCRAPLING_MCP_AUTH_TOKEN=$(openssl rand -hex 32)" > ~/scrapling/mcp.env)
sudo tee /etc/systemd/system/scrapling-mcp.service >/dev/null <<'EOF'
[Unit]
Description=Scrapling MCP server
After=network-online.target
Wants=network-online.target

[Service]
User=exedev
WorkingDirectory=/home/exedev/scrapling
EnvironmentFile=/home/exedev/scrapling/mcp.env
ExecStart=/home/exedev/.local/bin/scrapling-mcp --http --host 0.0.0.0 --port 8765 --allowed-host %H.exe.xyz:8765 --allowed-host localhost:8765 --allowed-host 127.0.0.1:8765
Restart=on-failure
UMask=0077

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now scrapling-mcp
timeout 60 sh -c 'until curl -so /dev/null localhost:8765/; do sleep 1; done'
```

The last line waits the few seconds the server takes to start. Check it:
`systemctl is-active scrapling-mcp` prints `active`, and a request
without the token gets `401`:

```bash
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8765/mcp -d '{}'
```

The server logs a `MCPDeprecationWarning` about
`validate_token_resource` at startup; it's harmless.

### Connect Claude Code and Codex

Connect both agents to the LLM integration, then add the server to each,
with its token:

```bash
exeuntu configure claude
exeuntu configure codex
TOKEN=$(cut -d= -f2 ~/scrapling/mcp.env)
claude mcp add --scope user --transport http scrapling http://localhost:8765/mcp \
  --header "Authorization: Bearer $TOKEN"
cat >> ~/.codex/config.toml <<EOF

[mcp_servers.scrapling]
url = "http://localhost:8765/mcp"
http_headers = { Authorization = "Bearer $TOKEN" }
EOF
```

Then install Scrapling's [Agent Skill](https://scrapling.readthedocs.io/en/latest/ai/agent-skill.html)
for Claude Code. It teaches the agent Scrapling's current API, so the
spiders it writes work with 0.5:

```bash
mkdir -p ~/.claude/skills /tmp/skill
curl -fsSLo /tmp/skill.zip https://github.com/D4Vinci/Scrapling/raw/refs/tags/v0.5.1/agent-skill/Scrapling-Skill.zip
unzip -q /tmp/skill.zip -d /tmp/skill
rm -rf ~/.claude/skills/scrapling-official
mv /tmp/skill/Scrapling-Skill ~/.claude/skills/scrapling-official
rm -rf /tmp/skill /tmp/skill.zip
```

Check it end to end. `claude mcp list` shows
`scrapling: http://localhost:8765/mcp (HTTP) - ✔ Connected`, and Claude
Code answers with the five cheapest mystery books, starting with
*Tastes Like Fear* at £10.69:

```bash
mkdir -p ~/project && cd ~/project
claude mcp list
claude -p "Use the scrapling make_request tool to fetch https://books.toscrape.com/catalogue/category/books/mystery_3/index.html with the CSS selector 'article.product_pod'. List the five cheapest books on that page as 'title - price', cheapest first." --allowedTools "mcp__scrapling__make_request"
```

</details>

## Ask an agent to scrape

On the VM, start `claude` or `codex` in a project folder and ask in plain
words. Naming the tool saves the agent a guess, and a CSS selector keeps
the page small, which saves tokens:

> Use the scrapling browser_fetch_once tool to open
> https://quotes.toscrape.com/js/ with the CSS selector 'div.quote', then
> tell me how many quotes are on the page and who wrote the first one.

The answer is ten quotes, the first by Albert Einstein; that page builds
its quotes with JavaScript, so it needs the browser. Codex works the same
way: `codex exec --skip-git-repo-check "Use the scrapling MCP server's make_request tool to ..."`.

Which tools to ask for:

| Tool | Use it for |
| --- | --- |
| `make_request` | one page or API call over HTTP, with a real browser's TLS fingerprint and headers; fast, no browser |
| `browser_fetch_once` | one page that needs JavaScript |
| `open_request_session`, `session_make_request` | several HTTP requests that share cookies |
| `browser_open`, `browser_fetch`, `browser_actions` | a browser session the agent clicks, types and scrolls in, page by page |
| `browser_screenshot`, `browser_extract`, `browser_network_requests` | look at the open page, read part of it, or find the API calls behind it |
| `browser_save_state`, `browser_load_state` | keep a login (see below) |

Tell the agent to close sessions when it's done; open browser sessions
hold memory until it does (`list_sessions` shows them). Scrapling strips
hidden text, HTML comments and zero-width characters that pages use to
smuggle instructions to agents, as long as `main_content_only` stays on,
which is the default. That's a filter, not a guarantee: don't give an
agent that reads arbitrary pages more power than you'd give the pages.

## Scrape from the terminal

`scrapling extract` saves a page, or the parts matching a CSS selector, as
Markdown, HTML or text, chosen by the file's extension. `get` is a plain
HTTP request; `fetch` uses the stealth browser:

```bash
cd ~/scrapes
scrapling extract get "https://books.toscrape.com/" titles.txt -s "article.product_pod h3 a"
scrapling extract fetch "https://quotes.toscrape.com/js/" page.md --network-idle
scrapling extract get "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" book.md --ai-targeted
```

`--ai-targeted` keeps the main content and strips the same hidden text as
the MCP server: use it for anything a model will read. `scrapling shell`
opens an IPython console with Scrapling loaded, for trying selectors, and
`scrapling extract fetch --help` lists the browser's options.

## Use it from your computer or another VM

The server is also at `https://my-scrapling.exe.xyz:8765/mcp`, behind
exe.dev's proxy. A program reaching it needs two tokens: a
[token for the VM](/docs/https-tokens-for-vms), which gets it through the
proxy, and the server's own token, which Scrapling checks. On your
computer, create the VM token and copy the server's token, each into a
file only you can read:

```bash
(umask 077
 ssh exe.dev ssh-key generate-api-key --vm=my-scrapling --label=scrapling-mcp > ~/.scrapling-vm-token
 ssh my-scrapling.exe.xyz 'cut -d= -f2 ~/scrapling/mcp.env' > ~/.scrapling-mcp-token)
```

The first file should hold a token starting with `exe1.`. Then add the
server to Claude Code on your computer:

```bash
claude mcp add --scope user --transport http scrapling https://my-scrapling.exe.xyz:8765/mcp \
  --header "X-Exedev-Authorization: Bearer $(cat ~/.scrapling-vm-token)" \
  --header "Authorization: Bearer $(cat ~/.scrapling-mcp-token)"
claude mcp list
```

`claude mcp list` shows the server as `✔ Connected`. Ask Claude Code to
scrape something, and the request runs on the VM: its IP address, its
browser, its saved logins. exe.dev's proxy consumes the
`X-Exedev-Authorization` header and passes `Authorization` on to
Scrapling. Without the VM token, the proxy answers `401` and nothing
reaches the VM. Other MCP clients take the same URL and two headers.

For an agent on another of your exe.dev VMs, you can keep both tokens off
that VM with a [VM-to-VM integration](/docs/integrations-vm-to-vm): exe.dev
adds the VM token at the edge, and `--bearer` adds Scrapling's. On your
computer (`my-agent-vm` is the VM that runs the agent):

```bash
ssh exe.dev integrations add http-proxy --name scrapling \
  --target https://my-scrapling.exe.xyz:8765/ --peer \
  --bearer "$(cat ~/.scrapling-mcp-token)" --attach vm:my-agent-vm
```

On `my-agent-vm`, the server is then
`claude mcp add --transport http scrapling https://scrapling.int.exe.xyz/mcp`,
with no headers.

> [!WARNING]
> Whoever can call the server can make this VM fetch any URL, including
> services on the VM itself and the exe.dev integrations attached to it,
> such as the LLM integration. Hand out the tokens like you'd hand out a
> shell, give the VM only the integrations it needs, and rotate the
> server's token (below) if one leaks.

## Write spiders and crawl on a schedule

For more than a few pages, write a spider: a Python class that says where
to start, what to pull out of each page, and which links to follow.
Scrapling runs it with concurrency limits, delays, robots.txt checks,
retries, and pause and resume. [books.py](https://exe.dev/deploy/scrapling/books.py)
is an example: it walks all 50 pages of books.toscrape.com, two requests
at a time with half a second between them, and adds every book's title,
price, rating and stock to a SQLite table, stamped with the time of the
crawl. On the VM:

```bash
mkdir -p ~/scrapling/crawls && cd ~/scrapling/crawls
curl -fsSLO https://exe.dev/deploy/scrapling/books.py
uv run books.py
```

The script pins `scrapling[fetchers]==0.5.1` in its header, so `uv run`
builds its environment the first time. After about 35 seconds:

```text
1000 books saved to /home/exedev/scrapling/data/books.db (50 requests, 34 s)
```

Query it with `sqlite3`:

```bash
sqlite3 -header -column ~/scrapling/data/books.db "
  SELECT title, price, rating FROM prices
  WHERE crawled_at = (SELECT max(crawled_at) FROM prices) AND in_stock
  ORDER BY price LIMIT 5"
```

Run it every morning with a systemd timer. It runs whether or not you're
connected, and catches up after a reboot:

```bash
sudo tee /etc/systemd/system/scrapling-books.service >/dev/null <<'EOF'
[Unit]
Description=Crawl books.toscrape.com with Scrapling

[Service]
Type=oneshot
User=exedev
WorkingDirectory=/home/exedev/scrapling/crawls
ExecStart=/usr/local/bin/uv run books.py
EOF
sudo tee /etc/systemd/system/scrapling-books.timer >/dev/null <<'EOF'
[Unit]
Description=Crawl books.toscrape.com every day

[Timer]
OnCalendar=*-*-* 06:00:00 UTC
RandomizedDelaySec=15m
Persistent=true

[Install]
WantedBy=timers.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now scrapling-books.timer
```

`sudo systemctl start scrapling-books` runs it now,
`journalctl -u scrapling-books -n 5` shows the summary line, and
`systemctl list-timers` shows the next run. Each run adds a new set of
rows, so you can compare crawls, for example prices that changed since
the one before:

```bash
sqlite3 -header -column ~/scrapling/data/books.db "
  WITH runs AS (SELECT DISTINCT crawled_at AS t FROM prices ORDER BY t DESC LIMIT 2)
  SELECT new.title, old.price AS was, new.price AS now
  FROM prices new JOIN prices old USING (url)
  WHERE new.crawled_at = (SELECT max(t) FROM runs)
    AND old.crawled_at = (SELECT min(t) FROM runs)
    AND new.price <> old.price"
```

(The practice site's prices never change, so this one prints nothing.)
For a database that other programs read and write, see
[SQLite in production](/deploy/sqlite).

To write your own spider, ask Claude Code; the Agent Skill gives it
Scrapling's current API:

> Use the scrapling-official skill. Write a Scrapling spider, quotes.py,
> with uv inline script metadata pinning scrapling[fetchers]==0.5.1, that
> crawls every page of https://quotes.toscrape.com/ politely (obey
> robots.txt, a delay of 1 second) and writes each quote's text, author
> and tags to quotes.jsonl. Run it with `uv run quotes.py`.

In testing it wrote a working spider on the first try and saved all 100
quotes. While you change a spider's parsing, set
`development_mode = True` (the commented line in `books.py`): the first
run saves every response under `.scrapling_cache/<spider name>/`, and
later runs replay them from disk without touching the site. A rerun of
`books.py` drops from 35 seconds to 3. Turn it off before the timer uses
the spider: cached pages never expire.

## Extract fields with a model

Selectors are exact and free, so use them when a page has structure. When
it doesn't, or you want a summary, hand the cleaned page to a model
through the LLM integration. Scrapling has no model calls of its own; this
is `jq` and `curl` with a JSON schema, so the answer is always valid JSON:

```bash
cd ~/scrapes
scrapling extract get "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" book.md --ai-targeted
jq -n --rawfile page book.md '{
  model: "gpt-6-luna",
  instructions: "Extract the book from this page.",
  input: $page,
  text: {format: {type: "json_schema", name: "book", strict: true, schema: {
    type: "object", additionalProperties: false,
    required: ["title", "price_gbp", "upc", "copies_in_stock", "summary"],
    properties: {
      title: {type: "string"}, price_gbp: {type: "number"}, upc: {type: "string"},
      copies_in_stock: {type: "integer"}, summary: {type: "string"}}}}}
}' | curl -s https://llm.int.exe.xyz/v1/responses -H 'Content-Type: application/json' -d @- \
  | jq -r '.output[] | select(.type == "message") | .content[0].text' | jq .
```

```json
{
  "title": "A Light in the Attic",
  "price_gbp": 51.77,
  "upc": "a897fe39b1053632",
  "copies_in_stock": 22,
  "summary": "A special 20th-anniversary edition of Shel Silverstein’s classic collection of humorous poetry and drawings."
}
```

No API key: the integration supplies it and bills your exe.dev account.
In a spider, do the same per item with any OpenAI-compatible client
pointed at `https://llm.int.exe.xyz/v1`.

## Sites that need a login

The MCP server can save a browser session's cookies and local storage to
a file on the VM and load them into a later session, so an agent signs in
once. Paths are relative to `~/scrapling`, the service's folder. The
practice site accepts any name and password:

> With the scrapling tools: open a browser session, fetch
> https://quotes.toscrape.com/login, fill the username and password fields
> with 'demo', submit the form, save the browser state to
> state/quotes.json and close the session. Then open a new session, load
> state/quotes.json, fetch https://quotes.toscrape.com/ and tell me
> whether the page shows a Logout link.

It does, and `~/scrapling/state/quotes.json` holds the session cookie. In
your own spiders, `StealthySession` takes the same files.

> [!WARNING]
> A state file is a logged-in session: anyone who can read it, or call the
> MCP server, can act as that account until the site logs it out. Keep
> them in `~/scrapling/state` (the service creates files only you can
> read), use accounts made for scraping with as few rights as possible,
> never your email, bank or admin accounts, and delete files you no
> longer need. Passwords for scripts belong in files with mode 600 or in
> [agent secrets](/deploy/agent-secrets), not in prompts.

## When you need a visible browser

Scrapling's browser runs headless, which is what scraping wants: no
desktop, little memory (the idle server uses about 170 MB). Use a browser
you can see when a person has to step in: to sign in with two-factor
authentication, to solve a check by hand, or to watch an agent work out a
site. Two ways:

- [Cloud browser](/deploy/cloud-browser) runs Chrome you open in a tab
  and exposes it over the DevTools protocol at `http://localhost:9222`.
  Ask the agent to *open a browser session connected over CDP to
  http://localhost:9222*, and Scrapling drives that Chrome, with the
  logins you made in it by hand.
- [Browser Use](/deploy/browser-use) puts an AI agent in a Chrome on a
  [Linux desktop](/deploy/linux-desktop), for tasks described in words
  rather than scraped.

## Scrape responsibly

Follow each site's terms and robots.txt (`robots_txt_obey = True`), crawl
slowly, leave personal data alone, and don't use stealth or
challenge-solving where a site has said no to bots: what the VM fetches is
on you, under exe.dev's [terms of service](/docs/terms-of-service). Test
against books.toscrape.com and quotes.toscrape.com, which exist for scraping.

<details id="back-up-and-restore" class="dp-details">
<summary>Back up and restore</summary>

What's worth keeping is in `~/scrapling`: `crawls/` (your spiders),
`data/` (results), `state/` (saved logins) and `mcp.env` (the server's
token). Copying the VM keeps everything, services included. On your
computer:

```bash
ssh exe.dev cp my-scrapling my-scrapling-backup
```

For a copy of just the data, back up the database while nothing writes
to it, and copy the folder to your computer:

```bash
sqlite3 ~/scrapling/data/books.db ".backup '/home/exedev/scrapling/data/books-backup.db'"
```

```bash
scp -r my-scrapling.exe.xyz:scrapling ./scrapling-backup
```

To restore on a VM set up the same way, copy the folder back with
`scp -r ./scrapling-backup my-scrapling.exe.xyz:scrapling` and run
`sudo systemctl restart scrapling-mcp`. The copy holds saved logins and
the token: keep it private.

</details>

<details id="update" class="dp-details">
<summary>Update</summary>

Scrapling releases often, sometimes with breaking changes (0.5 renamed
MCP tools and removed the bulk ones). Read the [changelog](https://scrapling.readthedocs.io/en/latest/changelog.html)
first, then on the VM install the version you want. Keep Playwright on
the minor version of the Patchright that Scrapling installs (Patchright
1.64.4 goes with Playwright 1.64.1); this shows the installed pair:
`uv pip list --python ~/.local/share/uv/tools/scrapling/bin/python | grep wright`.

```bash
uv tool install --force "scrapling[all]==0.5.1" --with "playwright==1.64.1"
scrapling install --force
sudo systemctl restart scrapling-mcp
```

Change the pin in your spiders' headers too, and update the Agent Skill
by repeating its commands with the new tag in the URL. To rotate the
server's token, write a new `mcp.env` as in setup, restart the service,
and add the server to your agents again (`claude mcp remove scrapling -s user`
first, and replace the line in `~/.codex/config.toml`).

</details>

<details id="troubleshooting" class="dp-details">
<summary>Troubleshooting</summary>

- **`Executable doesn't exist at ~/.cache/ms-playwright/chromium-…`.**
  Playwright and Patchright want different Chromium builds. Reinstall
  with the `--with "playwright==…"` pin from setup, then
  `scrapling install --force`.
- **`scrapling: command not found`.** `~/.local/bin` isn't on this
  shell's `PATH` yet: log in again, or
  `export PATH="$HOME/.local/bin:$PATH"`.
- **The service won't start.** `journalctl -u scrapling-mcp -n 50` says
  why. "Refusing to serve the MCP server over HTTP without
  authentication" means `~/scrapling/mcp.env` is missing or empty.
- **`401` from the server.** Without the VM token, it's exe.dev's proxy:
  the token is missing, expired, or for another VM (copies need their
  own). With it, the server's token is wrong or missing.
- **`421 Misdirected Request`.** The host name isn't one of the
  `--allowed-host` values, for example after renaming the VM. Restart the
  service so `%H` picks up the new name.
- **`claude mcp list` says Failed to connect.** Check
  `systemctl status scrapling-mcp`, then that the header carries the
  current token.
- **A site answers `403`, `429` or a challenge page.** It's telling you
  to stop or slow down; see "Scrape responsibly". Some sites block cloud
  addresses outright.
- **Memory grows.** Open browser sessions stay open until closed. Ask the
  agent to `list_sessions` and close them, or
  `sudo systemctl restart scrapling-mcp`.

</details>

## Frequently asked questions

### Is Scrapling free? What does this cost?

Scrapling is free and open source (BSD-3-Clause). You pay for the VM as part of your exe.dev plan. Agents and the extraction example use exe.dev's [LLM integration](/docs/integrations-llm), billed to your exe.dev account; exe.dev keeps a [list of models and prices](/llm-gateway-models). Scrapling itself calls no model.

### Does the Scrapling MCP server use stdio or HTTP?

Both. `scrapling-mcp` speaks stdio by default, which is enough for one agent on the same machine (`claude mcp add scrapling -- scrapling-mcp`). With `--http` it serves Streamable HTTP at `/mcp` and requires a bearer token. This page runs the HTTP server as a service, so every agent on the VM, and agents elsewhere with a VM token, share one server, its browser sessions and its saved logins.

### Who can reach the MCP server?

Programs on the VM that have the server's token, and, through exe.dev's [HTTPS proxy](/docs/proxy), only callers that are signed in to exe.dev with access to the VM or hold a [token for the VM](/docs/https-tokens-for-vms), and also send the server's token. Ports other than the VM's main one can't be made public. Anyone who gets through can make the VM fetch any URL, so treat both tokens like a shell on the VM.

### Why the pinned Playwright version?

`scrapling install` downloads Chromium with Playwright, but Scrapling's browser runs on Patchright, a patched Playwright that trails it by a release. Since Playwright 1.65 (November 2026), a plain install gets a Chromium that Patchright can't find, and every browser fetch fails with "Executable doesn't exist". Pinning Playwright 1.64.1, the version Patchright 1.64.4 matches, makes both use the same Chromium.

### Can I use the Docker image instead?

Yes: `pyd4vinci/scrapling` (and `ghcr.io/d4vinci/scrapling`) has the CLI, the browser and the MCP server. The uv install here is smaller to update, puts saved logins and spiders in plain folders, and lets spiders on the VM import Scrapling directly.

### Can Scrapling get past Cloudflare and CAPTCHAs?

Its stealth browser has options to solve some Cloudflare challenges (`--solve-cloudflare`, `solve_cloudflare`), and the MCP server can work through some interactive checks. A challenge is a site telling you it doesn't want automated traffic: check its terms first, and don't use these options where you aren't allowed to scrape.


## Related

- [Self-host browser-use AI browser agents](https://exe.dev/deploy/browser-use.md): AI agents that browse in a Chrome you can watch, on your own VM.
- [A private Chrome in the cloud, for you and your agents](https://exe.dev/deploy/cloud-browser.md): A remote Chrome you open in a tab and your scripts drive over CDP.
- [Run Claude Code on a remote server](https://exe.dev/deploy/claude-code.md): A computer for Claude. A breather for your laptop.
- [Run Codex on a remote server](https://exe.dev/deploy/codex.md): A computer for Codex. An early night for your laptop.
- [SQLite in production](https://exe.dev/deploy/sqlite.md): One file on a real disk, streamed to your own bucket every second.
- [A Linux desktop in your browser](https://exe.dev/deploy/linux-desktop.md): Xfce, Chrome, and coding agents on a VM you open in a browser tab.
