Skip to content

Self-host PR-Agent, the AI pull request reviewer, on exe.dev

Your own AI pull request reviewer, with no model key on the VM.

  • Developer tools
  • 20 minutes, most of it creating the GitHub App
  • Tested Nov 16, 2026
Deploy on exe.dev

Shelley handles the setup. About 20 minutes, most of it creating the GitHub App.

Prefer a terminal?
curl -fsSL https://exe.dev/deploy/pr-agent/prompt.txt | ssh exe.dev new --prompt=/dev/stdin

PR-Agent is an open-source AI reviewer for pull requests (MIT). When a pull request opens it writes the description and posts a review with the effort to review it, the security concerns, and the problems it sees; anyone on the pull request can ask for more by commenting /review, /improve for code suggestions, or /ask "how is this tested?". Version 0.50.0 came out on November 7, 2026.

This page runs it as your own GitHub App on an exe.dev VM. The model comes from exe.dev's LLM integration, so no model key is on the VM, and GitHub's webhooks reach it through exe.dev's HTTPS proxy, with nginx in front so that nothing but the webhook is public.

What you get#

  • Docker on every boot and the PR-Agent 0.50.0 and nginx images pulled
  • PR-Agent using exe.dev's LLM integration, with no model key on the VM
  • A webhook secret in ~/pr-agent/webhook-secret
  • A review of a sample diff, which flags its SQL and command injection
  • No GitHub App yet; you create it on GitHub
What's running, and where
Public https://my-pr-agent.exe.xyz/api/v1/github_webhooks GitHub's signed webhooks, once you make the VM public; every other path is a 404
Private ssh my-pr-agent.exe.xyz configuration and logs
my-pr-agent.exe.xyz exe.dev VM
PR-Agent pragent/pr-agent:0.50.0-github_app in Docker :3000~/pr-agent/.secrets.toml
Webhook gate nginx in Docker, passing only GitHub's webhook POSTs to PR-Agent :8000

Deploy with Shelley#

Click Deploy and Shelley will set up PR-Agent on a new VM, then check that it works. No config-file wrangling required.

Prefer a terminal?
curl -fsSL \
  https://exe.dev/deploy/pr-agent/prompt.txt \
  | ssh exe.dev new --prompt=/dev/stdin
What Shelley is told
Set up PR-Agent on this VM by following the exe.dev recipe at https://exe.dev/deploy/pr-agent.md from top to bottom.

The recipe's example VM name is my-pr-agent. This VM's name is the output of `hostname`: use it wherever the recipe writes my-pr-agent. Skip the step that creates the VM; you are on it.

You are done when all of these are true. Check each one yourself:
- Docker on every boot and the PR-Agent 0.50.0 and nginx images pulled
- PR-Agent using exe.dev's LLM integration, with no model key on the VM
- A webhook secret in `~/pr-agent/webhook-secret`
- A review of a sample diff, which flags its SQL and command injection
- No GitHub App yet; you create it on GitHub

Run the commands in "Install by hand", from "Get PR-Agent" on, and the
check at its end. Show the owner the first lines of the review. Don't
start the server (`docker compose up`) or make the VM public: that needs
the GitHub App, which the owner creates. Finish by telling the owner to
read "What's exposed" and follow "Create the GitHub App".

Finish with the URLs I should open.

Also at /deploy/pr-agent/prompt.txt. The recipe it follows is /deploy/pr-agent.md.

Install#

On exe.dev, click Deploy to have Shelley set it up on a new VM. Or create it yourself:

Install by hand

On your computer:

ssh exe.dev new --name=my-pr-agent
ssh my-pr-agent.exe.xyz

Get PR-Agent#

On the VM, start Docker, now and on every boot, and download the Compose file, the nginx config and a sample diff to try it on:

sudo systemctl enable --now docker.service
mkdir -p ~/pr-agent && cd ~/pr-agent
curl -fsSLO https://exe.dev/deploy/pr-agent/compose.yaml
curl -fsSLO https://exe.dev/deploy/pr-agent/nginx.conf
curl -fsSLO https://exe.dev/deploy/pr-agent/sample.diff
docker compose pull

The Compose file sends PR-Agent's model calls to the LLM integration, with gpt-5.5 and a placeholder key, and puts the nginx gate on port 8000, the port exe.dev's proxy serves.

Make the webhook secret#

On the VM, generate the secret GitHub will sign its webhooks with, and start PR-Agent's secrets file with it. PR-Agent runs as user 10001 in its container, so that group may read the file:

cd ~/pr-agent
(umask 077; openssl rand -hex 32 > webhook-secret)
printf '[github]\ndeployment_type = "app"\nwebhook_secret = "%s"\n' "$(cat webhook-secret)" > .secrets.toml
sudo chgrp 10001 .secrets.toml && chmod 640 .secrets.toml

Check the setup#

On the VM, have PR-Agent review the sample diff, which adds a Flask app with a SQL injection and a command injection:

cd ~/pr-agent
docker compose run --rm --no-deps -v ./sample.diff:/tmp/sample.diff:ro \
  --entrypoint python pr-agent pr_agent/cli.py --diff-file /tmp/sample.diff review

Within a minute it prints a PR Reviewer Guide whose security section names both injections.

What's exposed#

GitHub sends pull request events as webhooks, so the VM needs a public address. exe.dev's proxy makes a VM's port 8000 public as a whole, not one path, so nginx holds that port and lets through only POST /api/v1/github_webhooks; everything else is a 404. PR-Agent checks GitHub's signature on every webhook and answers 403 without a valid one. PR-Agent's own port, 3000, stays private, and the VM's other ports are never public.

Two things remain. PR-Agent's webhook handler is reachable from the internet, so keep it updated. And on a public repository, anyone who can open a pull request or comment on one can start a model call billed to you; install the App only on repositories whose contributors you trust.

Create the GitHub App#

On the VM, print the webhook secret:

cat ~/pr-agent/webhook-secret

On GitHub, under Settings → Developer settings → GitHub Apps → New GitHub App (for an organization: the organization's settings), fill in:

  • GitHub App name: anything; it's the name on the comments.
  • Homepage URL: https://my-pr-agent.exe.xyz
  • Webhook URL: https://my-pr-agent.exe.xyz/api/v1/github_webhooks
  • Webhook secret: the line you printed
  • Repository permissions: Contents, read-only; Issues, read and write; Pull requests, read and write. Metadata is read-only by default.
  • Subscribe to events: Issue comment, Pull request, Pull request review, Pull request review comment.
  • Where can this GitHub App be installed?: Only on this account.

Create it, note the App ID at the top of its page, and under Private keys click Generate a private key. Your browser downloads a .pem file. On your computer, copy it to the VM:

scp ~/Downloads/*.private-key.pem my-pr-agent.exe.xyz:pr-agent/app.pem

Start PR-Agent#

On the VM, add the App ID and the key to the secrets file, replacing 123456 with your App ID, and start PR-Agent and the gate:

cd ~/pr-agent
APP_ID=123456
{ echo "app_id = $APP_ID"; echo 'private_key = """'; cat app.pem; echo '"""'; } >> .secrets.toml
rm app.pem
docker compose up -d

On your computer, make port 8000 public:

ssh exe.dev share set-public my-pr-agent

On the VM, curl -s http://localhost:3000/ prints {"status":"ok"}. Then check the public side from your computer, since a VM can't reach its own public address:

curl -s -o /dev/null -w '%{http_code}\n' https://my-pr-agent.exe.xyz/
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://my-pr-agent.exe.xyz/api/v1/github_webhooks

The first prints 404, the gate; the second 403, PR-Agent refusing a request GitHub didn't sign.

Install it on a repository#

On the App's page on GitHub, click Install App and pick the repositories. Open a pull request in one of them: within a minute or two PR-Agent fills in the description and posts a PR Reviewer Guide comment. Comment /improve for code suggestions, or /help for every command.

The App's Advanced tab lists each webhook delivery and the VM's answer; docker compose logs -f pr-agent on the VM shows the other side.

Configure it#

A .pr_agent.toml at the root of a repository changes what PR-Agent does there, for example what its reviews look for:

[pr_reviewer]
extra_instructions = "Focus on authorization checks and SQL."
num_max_findings = 10

The configuration guide lists the options. Settings for every repository go in compose.yaml as environment variables, section and key joined by two underscores, such as PR_REVIEWER__EXTRA_INSTRUCTIONS.

Change the model#

To use another OpenAI model the LLM integration serves, change CONFIG__MODEL in ~/pr-agent/compose.yaml to openai/ plus its name, for example openai/gpt-5.6-terra, and run docker compose up -d. The model list shows what's available.

For a Claude model, replace the OPENAI__API_BASE, OPENAI__KEY and CONFIG__MODEL lines with these, and run docker compose up -d:

      ANTHROPIC_BASE_URL: https://llm.int.exe.xyz
      ANTHROPIC__KEY: exe-gateway
      CONFIG__MODEL: anthropic/claude-sonnet-5-5

Update#

Check the release notes, change the image tag in ~/pr-agent/compose.yaml, then:

cd ~/pr-agent
docker compose pull && docker compose up -d

Clean up#

On your computer, close the public port and delete the VM:

ssh exe.dev share set-private my-pr-agent
ssh exe.dev rm my-pr-agent

Then delete the GitHub App under Settings → Developer settings → GitHub Apps.

Troubleshooting#

On the App's Advanced tab:

  • A delivery got 403: the App's webhook secret differs from ~/pr-agent/webhook-secret. Paste it again.
  • A delivery got 404: the webhook URL's path is wrong; it ends in /api/v1/github_webhooks.
  • A delivery failed to connect, or got a login page: port 8000 isn't public. ssh exe.dev share show my-pr-agent shows it.
  • Deliveries got 200 but no comment appears: docker compose logs pr-agent --tail 100 on the VM. A wrong App ID or key shows up as a GitHub authentication error; a model problem as an error from litellm.
  • The review check fails: curl -s https://llm.int.exe.xyz/v1/models on the VM should list models; if it does, the model name in compose.yaml is likely one the integration doesn't serve.

Frequently asked questions#

How much does it cost?

The VM is part of your exe.dev plan. The model is billed to your exe.dev account through the LLM integration: a few cents for a review of a small pull request with gpt-5.5, more for large ones. Every opened pull request gets a description and a review, and every command someone types in a comment is another call.

Is this the same as Qodo Merge?

PR-Agent started at Qodo, which still sells a hosted product built on it. The open-source project now lives at The-PR-Agent/pr-agent under the MIT license; this page runs that.

Why does it need a public URL?

GitHub delivers pull request events as webhooks, so the VM has to be reachable from the internet. The page exposes only the webhook path, and PR-Agent rejects anything GitHub didn't sign. See What's exposed.

Can it review pull requests in public repositories?

It can, but anyone who opens a pull request or comments /review there starts a model call billed to you. Install the App only on repositories whose contributors you trust.

Can I use Claude models?

Yes. The LLM integration also serves Anthropic's API; set ANTHROPIC_BASE_URL and the model to anthropic/ plus a Claude model. See Change the model.

Updated Nov 16, 2026 · tested end to end Nov 16, 2026 · Markdown · manifest.json · Something wrong? Tell us