Deploy a Docker Compose app from GitHub

Create the VM#

On your computer (pick another name if my-docker-compose is taken):

ssh exe.dev new --name=my-docker-compose
ssh my-docker-compose.exe.xyz

Everything else runs on the VM, as exedev.

Set up Docker#

exe.dev starts Docker on demand. Enable it at boot, so your containers come back after a reboot, and switch container logs to Docker's local driver, which rotates them:

sudo systemctl enable --now docker.service
if [ ! -e /etc/docker/daemon.json ]; then
  echo '{"log-driver": "local"}' | sudo tee /etc/docker/daemon.json
  sudo systemctl restart docker.service
fi
systemctl is-enabled docker.service
docker info --format '{{.LoggingDriver}}'

Expect enabled and local.

Make the deploy repository#

Create the repository you'll push to, the settings file, and the directories for data and backups:

mkdir -p ~/data ~/backups
touch ~/app.env && chmod 600 ~/app.env
grep -q '^COMPOSE_PROJECT_NAME=' ~/app.env || echo 'COMPOSE_PROJECT_NAME=app' >> ~/app.env
git init -q --bare -b main ~/app.git

COMPOSE_PROJECT_NAME names the Compose project app whatever your compose file says, so containers are app-web-1 and named volumes app_<name>, the same on every deploy.

Then the hook that deploys each push. It runs before git accepts the push, so a deploy that fails rejects it:

cat > ~/app.git/hooks/pre-receive <<'EOF'
#!/usr/bin/env bash
# git push deploys: build the pushed commit in ~/app and start it with
# Docker Compose, before git accepts the push. If the build or the start
# fails, put back what was running and reject the push, so main in
# ~/app.git is always the commit that's running.
set -euo pipefail
zero=0000000000000000000000000000000000000000
branch=$(git symbolic-ref --short HEAD)
rev=
while read -r old new ref; do
  if [ "$ref" != "refs/heads/$branch" ]; then echo "Not deploying $ref: only $branch deploys."; continue; fi
  if [ "$new" = "$zero" ]; then echo "-----> $branch is what's running; it can't be deleted."; exit 1; fi
  prev=$old rev=$new
done
[ -n "$rev" ] || exit 0

exec 9>"$HOME/.deploy.lock" && flock 9    # one deploy at a time
# Put a commit's files in ~/app (without touching refs: the push isn't accepted yet).
checkout() {
  mkdir -p "$HOME/app"
  GIT_WORK_TREE=$HOME/app GIT_INDEX_FILE=$HOME/.deploy-index git --git-dir="$HOME/app.git" read-tree -u --reset "$1"
  ln -sfn "$HOME/app.env" "$HOME/app/.env"    # settings live outside the code
}
echo "-----> Building $(git log -1 --format='%h %s' "$rev")"
checkout "$rev"
cd "$HOME/app" && unset GIT_DIR
# Without build attestations, unchanged code builds the same image ID,
# so Compose leaves services that didn't change running.
export BUILDX_NO_DEFAULT_ATTESTATIONS=1
started=$(date +%s)
if ! { docker compose build && docker compose pull -q --ignore-buildable --policy missing; }; then
  if [ "$prev" != "$zero" ]; then checkout "$prev"; fi
  echo "-----> Build failed. Push rejected; nothing changed."
  exit 1
fi

echo "-----> Starting"
up() { docker compose up -d --remove-orphans --quiet-pull --wait --wait-timeout 180 "$@"; }
if ! up; then
  docker compose ps -a
  docker compose logs --since "$started" --tail=30
  if [ "$prev" = "$zero" ]; then
    docker compose down
  else
    echo "-----> Rolling back to ${prev:0:7}"
    checkout "$prev"
    up --build || echo "-----> The rollback failed too. See: cd ~/app && docker compose logs"
  fi
  echo "-----> ${rev:0:7} didn't start. Push rejected."
  exit 1
fi

docker image prune -f >/dev/null                      # old images
docker builder prune -f --max-used-space 3gb >/dev/null  # build cache
docker compose ps
norestart=$(docker compose config --format json |
  jq -r '.services | to_entries[] | select((.value.restart // "no") == "no") | .key')
if [ -n "$norestart" ]; then
  echo "-----> No restart policy, so these won't come back after a reboot:" $norestart
  echo "       Fine for one-off jobs; give the others restart: unless-stopped."
fi
echo "-----> Deployed ${rev:0:7} to https://$(hostname).exe.xyz/"
EOF
chmod +x ~/app.git/hooks/pre-receive

Now follow one of the next two steps: start a new app, or bring yours.

Start fresh#

This is a tiny but complete Compose app: a Python web app, built from its Dockerfile, and PostgreSQL, with healthchecks, restart policies, the database in ~/data, and its password in ~/app.env. Replace the app with yours later; keep the shape.

mkdir -p /tmp/new-app && cd /tmp/new-app

cat > compose.yaml <<'EOF'
# A web app built from this repo, and PostgreSQL.
# Settings come from .env (on exe.dev, a link to ~/app.env).
services:
  web:
    build: .
    restart: unless-stopped
    ports:
      - "127.0.0.1:8000:8000"   # exe.dev serves port 8000 at https://<vm>.exe.xyz/
    environment:
      DATABASE_URL: postgresql://app:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@db:5432/app
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/healthz')"]
      interval: 30s
      timeout: 5s
      start_period: 30s
      start_interval: 1s

  db:
    image: postgres:18
    restart: unless-stopped
    environment:
      POSTGRES_USER: app
      POSTGRES_DB: app
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
    volumes:
      - ${DATA_DIR:-./data}/postgres:/var/lib/postgresql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d app"]
      interval: 10s
      timeout: 5s
      start_period: 30s
      start_interval: 1s
EOF

cat > Dockerfile <<'EOF'
FROM python:3.14-slim
ENV PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
RUN useradd --create-home app
USER app
EXPOSE 8000
CMD ["gunicorn", "--bind=0.0.0.0:8000", "--workers=2", "--preload", "--access-logfile=-", "app:app"]
EOF

cat > requirements.txt <<'EOF'
flask~=3.1
gunicorn~=26.2
psycopg[binary]~=3.3
EOF

cat > app.py <<'EOF'
"""A tiny web app that counts visits in PostgreSQL. Replace it with yours."""
import os

import psycopg
from flask import Flask, request
from markupsafe import escape

app = Flask(__name__)
DATABASE_URL = os.environ["DATABASE_URL"]

with psycopg.connect(DATABASE_URL) as conn:
    conn.execute("CREATE TABLE IF NOT EXISTS visits "
                 "(id serial PRIMARY KEY, at timestamptz NOT NULL DEFAULT now(), email text)")


@app.get("/")
def index():
    # exe.dev's proxy sends X-ExeDev-Email for visitors signed in to exe.dev.
    email = request.headers.get("X-ExeDev-Email")
    with psycopg.connect(DATABASE_URL) as conn:
        conn.execute("INSERT INTO visits (email) VALUES (%s)", (email,))
        (count,) = conn.execute("SELECT count(*) FROM visits").fetchone()
    # Once the VM is public, visitors sign in at /__exe.dev/login.
    sign_in = "" if email else '<p><a href="/__exe.dev/login?redirect=/">Sign in</a></p>\n'
    return f"<h1>Hello, {escape(email or 'stranger')}!</h1>\n<p>Visit number {count}, counted in PostgreSQL.</p>\n{sign_in}"


@app.get("/healthz")
def healthz():
    with psycopg.connect(DATABASE_URL) as conn:
        conn.execute("SELECT 1")
    return "ok\n"
EOF

cat > .env.example <<'EOF'
# Copy to .env and fill in. On exe.dev, these live in ~/app.env.
POSTGRES_PASSWORD=change-me
DATA_DIR=./data
EOF
printf '.env\ndata/\n__pycache__/\n' > .gitignore
printf '.env\ndata/\n.git\n__pycache__/\n' > .dockerignore

Generate the database password, once: PostgreSQL keeps the one it started with.

grep -q '^POSTGRES_PASSWORD=' ~/app.env || cat >> ~/app.env <<EOF
POSTGRES_PASSWORD=$(openssl rand -hex 24)
DATA_DIR=$HOME/data
EOF

Commit it and push it to the deploy repository. This first push deploys, exactly as your own pushes will:

cd /tmp/new-app
git init -q -b main && git add -A
git -c user.name="$(hostname)" -c user.email="exedev@$(hostname).exe.xyz" commit -qm "New Compose app: web and PostgreSQL"
git push -q ~/app.git main
curl -s localhost:8000/

The push ends with -----> Deployed, and the page says Hello, stranger! with a visit number: through exe.dev it will use your email. Then remove the temporary copy; the code lives in ~/app.git:

cd ~ && rm -rf /tmp/new-app

Skip to Set up backups.

Bring your repo#

Set REPO to the repository, as owner/name, and clone it. Public repositories clone from GitHub; private ones through exe.dev's GitHub integration, with no token on the VM:

REPO=owner/name
repo=${REPO#https://github.com/}; repo=${repo%.git}
rm -rf /tmp/seed
export GIT_TERMINAL_PROMPT=0
git clone -q "https://github.com/$repo.git" /tmp/seed ||
  git clone -q "https://github.int.exe.xyz/$repo.git" /tmp/seed

If both fail, the repository is private and the VM can't reach it yet. On your computer, link GitHub on exe.dev's Integrations page if you haven't, then run this with your repository, and try the clone again:

ssh exe.dev integrations add github --name app-repo --repository owner/name --attach vm:my-docker-compose

Deploy the repository's default branch:

branch=$(git -C /tmp/seed symbolic-ref --short HEAD)
git -C ~/app.git symbolic-ref HEAD "refs/heads/$branch"
echo "$branch"

Now fit it to the VM. Look at what the compose file asks for:

cd /tmp/seed
ls -a
ln -sfn ~/app.env .env
docker compose config --format json | jq '.services | map_values({ports, restart, volumes: [.volumes[]? | .source]})'

Work through this list, then run docker compose config --quiet until it prints nothing:

1. Compose file. Not at the top of the repo, or more than one? Add COMPOSE_FILE= to ~/app.env, as in Change things for this VM only.
2. Settings. Copy each variable the app needs from its .env.example (or README) into ~/app.env. Generate passwords and keys with openssl rand -hex 32. If the repo commits a .env, copy its values too (git show HEAD:.env): deploys replace it with a link to ~/app.env.
3. Port 8000. If no service publishes host port 8000, add one in ~/app.override.yaml: the service people should visit, as 127.0.0.1:8000:<its port>. Move other web ports to 3000-9999 on 127.0.0.1, and drop published database ports.
4. Restart policies and healthchecks. Give every long-running service restart: unless-stopped, in the override if the repo lacks it. Leave one-off jobs, like migrations, without one. Deploys wait for healthchecks, and a service without one counts as up as soon as it starts, even if it crashes a second later. So give the web service a healthcheck: that fetches a page, as in the fresh app's compose.yaml, with a tool its image has (curl, wget, python or node).
5. Development settings. Many compose files are written for a laptop: they mount the source into the container (./src:/app), run a dev server with reloading, or build a dev target. Override those for the app's services: volumes: !reset [] (only where the volumes are source code), and the production target: (look for stages in the Dockerfile), command: or entrypoint:. Switch off dev-only services, like a database admin UI, or a proxy that only routes localhost between dev servers, with profiles: [dev]: Compose skips services whose profile isn't enabled. Replace hard-coded passwords, like MYSQL_ROOT_PASSWORD: secret, with ${...} from ~/app.env. A compose.prod.yaml in the repo may already do all this.
6. Data. Note each database service and each named volume or ~/data directory with files worth keeping, for the backup script.

For example, Docker's example voting app publishes ports 8080 and 8081, has no restart policies, and runs its two web apps in development mode. Its ~/app.override.yaml:

# This VM's changes to the repo's compose file. See COMPOSE_FILE in ~/app.env.
services:
  vote:
    build:
      target: final            # gunicorn, not Flask's dev server
    volumes: !reset []         # run the built code, not the checkout
    ports: !override
      - "127.0.0.1:8000:80"
    restart: unless-stopped
  result:
    entrypoint: !reset null    # the image's node, not nodemon
    volumes: !reset []
    ports: !override
      - "127.0.0.1:8081:80"
    restart: unless-stopped
  worker:
    restart: unless-stopped
  redis:
    restart: unless-stopped
  db:
    restart: unless-stopped

and ~/app.env holds COMPOSE_FILE=docker-compose.yml:/home/exedev/app.override.yaml.

Push it to the deploy repository; this first push deploys it:

cd /tmp/seed
git push -q ~/app.git "HEAD:$(git symbolic-ref --short HEAD)"
curl -s -o /dev/null -w '%{http_code}\n' -H "Host: $(hostname).exe.xyz" localhost:8000/

The push ends with -----> Deployed, and the status is 200 (or a redirect, 302, for an app that starts at a login page). The Host header is the one visitors' requests carry, so this also catches apps and proxies that only answer to localhost. If the push is rejected, see Something not working?, fix ~/app.env or the override, and push again. If it deployed but the status is wrong, apply your fix with cd ~/app && docker compose up -d --wait. When it works, remove the temporary clone:

cd ~ && rm -rf /tmp/seed

Set up backups#

The backup script dumps each database, then archives the dumps with ~/data and the settings. This one is for the fresh app's PostgreSQL; for your own stack, write one dump line per database service (see Other databases) and exclude their directories in ~/data from the tar:

cat > ~/app-backup.sh <<'EOF'
#!/usr/bin/env bash
# Back up the app to ~/backups/app-<time>.tar.gz: a dump of each
# database, the files in ~/data, and the settings. Keeps the newest 14.
set -euo pipefail
umask 077
exec 9>~/.deploy.lock && flock 9   # wait for any deploy to finish
cd ~/app
work=$(mktemp -d ~/backups/.work-XXXXXX)
trap 'rm -rf "$work"' EXIT

# Databases: dump them; a copy of a running database's files isn't
# consistent. One line per database service.
docker compose exec -T db pg_dump -U app -d app --format=custom > "$work/db.dump"

# Files in ~/data, except the databases' own (dumped above).
sudo tar -C ~ --exclude=data/postgres -cf - data > "$work/data.tar"

# Settings, and the commit that was running.
cp ~/app.env "$work/"
[ ! -e ~/app.override.yaml ] || cp ~/app.override.yaml "$work/"
git -C ~/app.git rev-parse HEAD > "$work/commit"

out=~/backups/app-$(date -u +%Y%m%d-%H%M%S).tar.gz
tar -C "$work" -czf "$out.part" .
mv "$out.part" "$out"
ls -1t ~/backups/app-*.tar.gz | tail -n +15 | xargs -r rm --
echo "$out"
EOF
chmod +x ~/app-backup.sh

Run it every night with a systemd timer, and take the first backup now:

sudo tee /etc/systemd/system/app-backup.service >/dev/null <<'EOF'
[Unit]
Description=Back up the Compose app to ~/backups
After=docker.service

[Service]
Type=oneshot
User=exedev
ExecStart=/home/exedev/app-backup.sh
EOF
sudo tee /etc/systemd/system/app-backup.timer >/dev/null <<'EOF'
[Unit]
Description=Back up the Compose app every night

[Timer]
OnCalendar=*-*-* 03:00:00
RandomizedDelaySec=30m
Persistent=true

[Install]
WantedBy=timers.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now app-backup.timer
sudo systemctl start app-backup.service
systemctl list-timers app-backup.timer
ls -lh ~/backups

The timer shows its next run, and ~/backups has one archive. If not: journalctl -u app-backup.service.

Test a restore#

Restore the newest dump into a scratch database next to the real one, and count what came back:

latest=$(ls -1t ~/backups/app-*.tar.gz | head -n 1)
rm -rf /tmp/restore-test && mkdir /tmp/restore-test
tar -xzf "$latest" -C /tmp/restore-test
cd ~/app
docker compose exec -T db dropdb -U app --if-exists restore_test
docker compose exec -T db createdb -U app restore_test
docker compose exec -T db pg_restore -U app -d restore_test --no-owner --exit-on-error < /tmp/restore-test/db.dump
docker compose exec -T db psql -U app -d restore_test -c 'SELECT count(*) FROM visits;'
docker compose exec -T db dropdb -U app restore_test
rm -rf /tmp/restore-test

Expect a count of at least 1. For your own stack, use your database service, user and one of your tables. Installation done: go back to Open your app.

To take a backup now, on the VM: ~/app-backup.sh. It prints the archive's path. Each archive holds db.dump, data.tar, app.env, app.override.yaml if you have one, and commit.

Restore on the same VM#

This replaces the database and the files in ~/data with the backup's. Pick the archive and use its name; these commands are for the fresh app's PostgreSQL:

backup=~/backups/app-20261017-030512.tar.gz
rm -rf /tmp/restore && mkdir /tmp/restore
tar -xzf "$backup" -C /tmp/restore
cd ~/app
docker compose stop web
docker compose exec -T db dropdb -U app --force app
docker compose exec -T db createdb -U app app
docker compose exec -T db pg_restore -U app -d app --no-owner --exit-on-error < /tmp/restore/db.dump
sudo tar -C ~ -xf /tmp/restore/data.tar
docker compose start web
rm -rf /tmp/restore

Open the app and check your data is back.

Restore on a new VM#

On the new VM, follow Install by hand up to and including Make the deploy repository. Copy the backup over from your computer, and put its settings in place before the first deploy, so the database starts with the same password:

scp app-20261017-030512.tar.gz my-docker-compose.exe.xyz:backups/

On the VM:

mkdir -p /tmp/restore
tar -xzf ~/backups/app-20261017-030512.tar.gz -C /tmp/restore
install -m 600 /tmp/restore/app.env ~/app.env
[ ! -e /tmp/restore/app.override.yaml ] || cp /tmp/restore/app.override.yaml ~/
cat /tmp/restore/commit

If the VM has a new name, change it wherever ~/app.env mentions the old one. Then deploy your code with a push from your computer, or by cloning it on the VM as in Bring your repo, and run the restore above. Set up backups again, as in Set up backups.

Other databases#

One line per database service in ~/app-backup.sh, with the service's name in place of db. These read the official images' own settings:

# PostgreSQL
docker compose exec -T db sh -c 'pg_dump -U "$POSTGRES_USER" -d "${POSTGRES_DB:-$POSTGRES_USER}" -Fc' > "$work/db.dump"
# MySQL
docker compose exec -T db sh -c 'MYSQL_PWD="$MYSQL_ROOT_PASSWORD" exec mysqldump -uroot --single-transaction --all-databases' > "$work/db.sql"
# MariaDB
docker compose exec -T db sh -c 'exec mariadb-dump -uroot -p"$MARIADB_ROOT_PASSWORD" --single-transaction --all-databases' > "$work/db.sql"
# MongoDB
docker compose exec -T db sh -c 'exec mongodump --archive --gzip -u "$MONGO_INITDB_ROOT_USERNAME" -p "$MONGO_INITDB_ROOT_PASSWORD" --authenticationDatabase admin' > "$work/db.archive"
# SQLite, in a file under ~/data
sqlite3 ~/data/app.db ".backup '$work/app.db'"

Restore them with pg_restore, mysql or mariadb reading the .sql file, mongorestore --archive --gzip --drop, or by copying the SQLite file back while the app is stopped. Files in a named volume rather than ~/data: stream them out with docker run --rm -v app_uploads:/v:ro alpine tar -C /v -cf - . > "$work/uploads.tar" (the volume is named app_ and the name in your compose file). A cache like Redis usually needs no backup.

Your app updates with every git push. The rest, on the VM:

• Images you pull, like postgres:18: docker compose pull fetches the newest image for each tag, and docker compose up -d --wait starts them. Back up first. A new major version of a database, like PostgreSQL 18 to 19, can't read the old one's files: dump, upgrade, restore.
• Base images in your Dockerfiles: deploys reuse the cached ones. For security fixes, rebuild with fresh ones now and then: cd ~/app && docker compose build --pull && docker compose up -d --wait.
• Docker comes from Ubuntu's packages (docker.io, docker-compose-v2): sudo apt-get update && sudo apt-get upgrade. Upgrading restarts Docker and your containers with it.

Start on the VM:

cd ~/app
docker compose ps -a
docker compose logs --tail=100

• 502 or an error page at https://my-docker-compose.exe.xyz/: nothing answers on port 8000. Check curl -sI localhost:8000/ and docker compose ps: the web service must show 127.0.0.1:8000->.
• required variable POSTGRES_PASSWORD is missing (or any other): add it to ~/app.env, then docker compose up -d --wait.
• port is already allocated: two services, or another program, want the same host port. sudo ss -ltnp shows who has it.
• Redirect loops, http:// links, CSRF errors: your app doesn't trust exe.dev's X-Forwarded-Proto: https. See Port 8000, and the other ports.
• Traefik or Caddy can't get a certificate: they can't, here. exe.dev handles TLS; serve plain HTTP on port 8000.
• 404 at the real URL, but curl localhost:8000 works: a proxy's routing rules, or the app's allowed hosts, only match localhost. Add my-docker-compose.exe.xyz and test with curl -H "Host: my-docker-compose.exe.xyz" localhost:8000/.
• Permission denied writing to ~/data/...: Docker created the directory as root. Give it to the container's user, for example sudo chown -R 1000:1000 ~/data/uploads.
• no space left on device: see Keep the disk from filling up.
• The push is rejected (pre-receive hook declined): the deploy failed and the old version is still running. The remote: lines above it show the build error, or the logs of the service that didn't start.
• The push says Permission denied (publickey): add your SSH key to exe.dev, or use the remote vm+my-docker-compose@vm.exe.xyz:app.git (SSH destinations).
• Down after a reboot: run sudo systemctl enable --now docker.service, and check every service has a restart policy; the last deploy's output names any that don't.
• A service you changed in ~/app.env didn't change: use docker compose up -d, not restart.