Deploy a Go web app

Create the VM#

On your computer (pick another name if my-go is taken):

ssh exe.dev new --name=my-go
ssh my-go.exe.xyz

Everything else runs on the VM, as exedev.

Check Go#

Go is preinstalled in /usr/local/go, with go on the PATH that systemd, the git hook, and your shell all use:

go version
go env GOTOOLCHAIN

Expect go1.27 or later and auto. With auto, a build whose go.mod asks for a newer Go (go 1.28, or a toolchain line) downloads that version by itself, so each app's go.mod decides which Go builds it.

Settings, data, and the service#

Create the data and release directories and a private settings file. This leaves an existing ~/app.env alone:

mkdir -p ~/data ~/releases ~/backups
if [ ! -e ~/app.env ]; then
  install -m 600 /dev/null ~/app.env
  cat >> ~/app.env <<'EOF'
# Settings and secrets for the app: KEY=value, one per line. Quote values
# with spaces or $ in 'single quotes'. Read by the build and by app.service.
# After changing it: sudo systemctl restart app
# The package the deploy hook builds, such as ./cmd/server:
MAIN_PACKAGE=.
EOF
fi
stat -c '%a %n' ~/app.env

Expect 600 /home/exedev/app.env. Now the service. It runs the binary the hook builds, ~/app/bin/app, in ~/app, which will point at the current release. PORT=8000 is where exe.dev's proxy sends https://my-go.exe.xyz/:

sudo tee /etc/systemd/system/app.service >/dev/null <<'EOF'
[Unit]
Description=Go app (~/app/bin/app)
After=network-online.target
Wants=network-online.target

[Service]
User=exedev
WorkingDirectory=/home/exedev/app
EnvironmentFile=/home/exedev/app.env
Environment=PORT=8000
ExecStart=/home/exedev/app/bin/app
Restart=always
RestartSec=3
# systemctl stop/restart sends SIGTERM, then SIGKILL after this long.
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable app.service

It starts with the first deploy. If your app takes its port from a flag rather than PORT, add it to ExecStart, such as /home/exedev/app/bin/app -addr :8000, and run sudo systemctl daemon-reload.

Back up every night#

~/bin/app-backup saves everything that isn't in a release: ~/data (each SQLite database through .backup, which is safe while the app writes, and then checked), ~/app.env, and the repository in ~/app.git as a git bundle. A new app's only copy lives there until you clone it. It keeps the 14 newest backups of each kind: nightly, deploy (the hook takes one before each switch), and manual.

mkdir -p ~/bin
cat > ~/bin/app-backup <<'EOF'
#!/usr/bin/env bash
# Back up ~/data, ~/app.env and ~/app.git to ~/backups/app-<kind>-<time>.tar.gz.
# SQLite databases are copied with .backup, safe while the app runs, and
# checked. Keeps the 14 newest backups of each kind.
# Usage: app-backup [nightly|deploy|manual]
set -euo pipefail
umask 077
kind=${1:-manual}
work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT
mkdir -p "$HOME/backups" "$work/data"
cd "$HOME/data"
rsync -a --exclude='*.db' --exclude='*.db-*' --exclude='*.sqlite*' ./ "$work/data/"
find . -type f \( -name '*.db' -o -name '*.sqlite' -o -name '*.sqlite3' \) | while read -r db; do
  mkdir -p "$work/data/$(dirname "$db")"
  sqlite3 "$db" ".backup '$work/data/$db'"
  [ "$(sqlite3 "$work/data/$db" 'PRAGMA quick_check')" = ok ]
done
cp "$HOME/app.env" "$work/"
if git -C "$HOME/app.git" rev-parse -q --verify HEAD >/dev/null; then
  git -C "$HOME/app.git" bundle create -q "$work/app.bundle" --all
fi
out="$HOME/backups/app-$kind-$(date -u +%Y%m%d-%H%M%S).tar.gz"
tar -czf "$out.part" -C "$work" .
mv "$out.part" "$out"
ls -1t "$HOME/backups/app-$kind-"*.tar.gz | tail -n +15 | xargs -r rm --
echo "$out"
EOF
chmod +x ~/bin/app-backup

Run it every night with a systemd timer:

sudo tee /etc/systemd/system/app-backup.service >/dev/null <<'EOF'
[Unit]
Description=Back up the app's data, settings and repository to ~/backups

[Service]
Type=oneshot
User=exedev
ExecStart=/home/exedev/bin/app-backup nightly
EOF
sudo tee /etc/systemd/system/app-backup.timer >/dev/null <<'EOF'
[Unit]
Description=Back up the app every night

[Timer]
OnCalendar=*-*-* 03:00:00 UTC
RandomizedDelaySec=30m
Persistent=true

[Install]
WantedBy=timers.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now app-backup.timer
systemctl list-timers app-backup.timer

Expect the next run, tonight. You'll take the first backup once the app is deployed.

Get the code#

The VM deploys from a bare git repository, ~/app.git. Create it one of two ways.

Start a new app (REPO=new): an empty repository whose branch is main.

git init --bare -b main ~/app.git

Bring your GitHub repository: a copy of it. A public repository clones directly; replace OWNER/NAME:

git clone --bare https://github.com/OWNER/NAME.git ~/app.git
git -C ~/app.git symbolic-ref --short HEAD

The last line prints the branch that will deploy: your default branch.

For a private repository, connect GitHub to exe.dev once (on the Integrations page) and attach the repository to this VM. On your computer:

ssh exe.dev integrations add github --name go-repo --repository OWNER/NAME --readonly --attach vm:my-go

Then clone through the integration on the VM. No token lands on the disk (GitHub integration):

git clone --bare https://github.int.exe.xyz/OWNER/NAME.git ~/app.git

If that clone fails with "not found" or a 403, the integration isn't attached to this VM yet. If your app needs other private modules, the VM's build can't download them: run go mod vendor on your computer and commit vendor/, which go build then uses instead.

Add the deploy hook#

Git runs hooks/pre-receive before it accepts a push, as Heroku does. This one builds the pushed commit in its own directory under ~/releases, backs up ~/data, switches ~/app to it, restarts the service, and checks that it answers. If any step fails, it switches back and rejects the push, so main on the VM is always what's running.

cat > ~/app.git/hooks/pre-receive <<'EOF'
#!/usr/bin/env bash
# Deploys the app. Git runs this before accepting a push to ~/app.git;
# if the deploy fails, the push is rejected and the running version stays.
# To redeploy main by hand:  ~/app.git/hooks/pre-receive HEAD
# To deploy a prebuilt linux binary:  ~/app.git/hooks/pre-receive --binary FILE (- for stdin)
set -euo pipefail
export GIT_DIR=$HOME/app.git
branch=$(git symbolic-ref --short HEAD)  # the branch that deploys
rev= binary=
if [ "${1:-}" = --binary ]; then
  binary=${2:?usage: pre-receive --binary FILE}
elif [ $# -gt 0 ]; then
  rev=$(git rev-parse --verify "$1^{commit}")
else
  while read -r _ new ref; do
    [ "$ref" = "refs/heads/$branch" ] || continue
    case $new in
      *[!0]*) rev=$new ;;
      *) echo "==> Refusing to delete $branch: it's what deploys."; exit 1 ;;
    esac
  done
  [ -n "$rev" ] || exit 0  # other branches are kept, not deployed
fi

exec 9>"$HOME/.deploy.lock" && flock 9  # one deploy at a time
release=$HOME/releases/$(date -u +%Y%m%d-%H%M%S)-${rev:0:7}${binary:+binary}
previous=$(readlink "$HOME/app" || true)
fail() {  # put the previous release back and reject the push
  trap - ERR
  echo "==> Deploy FAILED${1:+: $1}. Still running: ${previous:-nothing}"
  if [ -n "$previous" ] && [ "$(readlink "$HOME/app")" != "$previous" ]; then
    ln -sfn "$previous" "$HOME/app.new" && mv -T "$HOME/app.new" "$HOME/app"
    sudo systemctl restart app
  fi
  rm -rf "$release"
  exit 1
}
trap fail ERR

mkdir -p "$release/bin"
if [ -n "$binary" ]; then
  echo "==> Installing a prebuilt binary in $release"
  if [ "$binary" = - ]; then cat > "$release/bin/app"; else cp "$binary" "$release/bin/app"; fi
  chmod +x "$release/bin/app"
else
  echo "==> Building ${rev:0:7} in $release"
  git archive "$rev" | tar -x -C "$release"
fi
unset GIT_DIR GIT_OBJECT_DIRECTORY GIT_ALTERNATE_OBJECT_DIRECTORIES GIT_QUARANTINE_PATH
cd "$release"
set -a; . "$HOME/app.env"; set +a
[ -n "$binary" ] || go build -trimpath -o bin/app "${MAIN_PACKAGE:-.}"
"$HOME/bin/app-backup" deploy >/dev/null  # the new version may migrate when it starts
if [ -n "${MIGRATE_COMMAND:-}" ]; then
  bash -c "$MIGRATE_COMMAND"
fi

echo "==> Starting the new version"
ln -sfn "$release" "$HOME/app.new" && mv -T "$HOME/app.new" "$HOME/app"
sudo systemctl restart app
deadline=$((SECONDS + 60))  # wait up to a minute for an answer
while [ $SECONDS -lt $deadline ]; do
  code=$(curl -s -o /dev/null -m 5 -w '%{http_code}' http://localhost:8000/ || true)
  case $code in 000|5??) sleep 1 ;; *) break ;; esac
done
case $code in
  000|5??) fail "the new version answers HTTP $code (journalctl -u app -n 50)" ;;
esac
trap - ERR
ls -1d "$HOME"/releases/*/ | head -n -3 | xargs -r rm -rf  # keep 3
echo "==> Deployed ${rev:0:7}${binary:+a prebuilt binary}: HTTP $code on port 8000"
EOF
chmod +x ~/app.git/hooks/pre-receive

The hook builds MAIN_PACKAGE from ~/app.env into the release's bin/app, the first time downloading modules into ~/go/pkg/mod, where later builds find them. It takes a deploy backup, runs MIGRATE_COMMAND if you set one, and counts any answer on port 8000 below 500 as up.

Start a new app#

Skip to Deploy if you brought a repository. Otherwise, make a module with the SQLite driver:

rm -rf /tmp/new-app && mkdir -p /tmp/new-app/migrations /tmp/new-app/templates /tmp/new-app/static
cd /tmp/new-app
git init -q -b main
go mod init "$(hostname)"
go get modernc.org/sqlite@latest
printf '# SQLite databases (on the VM, data lives in ~/data)\n*.db\n*.db-*\n# Builds\n/app-linux\n' > .gitignore
echo 'DATABASE_URL=file:/home/exedev/data/app.db' >> ~/app.env

The server, in main.go. Its templates, static files, and migrations are embedded in the binary; it migrates the database on start, signs messages with the email exe.dev's proxy sends, and shuts down gracefully on SIGTERM:

cat > /tmp/new-app/main.go <<'EOF'
// A small Go web app: a guestbook in SQLite, signed with exe.dev.
package main

import (
	"context"
	"database/sql"
	"embed"
	"errors"
	"fmt"
	"html/template"
	"io/fs"
	"log/slog"
	"net/http"
	"os"
	"os/signal"
	"path"
	"strconv"
	"strings"
	"syscall"
	"time"

	_ "modernc.org/sqlite" // SQLite in pure Go: no cgo, so it cross-compiles
)

// Compiled into the binary, so a release needs nothing else.
//
//go:embed templates static migrations
var files embed.FS

var page = template.Must(template.ParseFS(files, "templates/*.html"))

type message struct{ Author, Body string }

func main() {
	db, err := openDB(env("DATABASE_URL", "file:app.db"))
	if err != nil {
		fatal("database", err)
	}
	defer db.Close()

	static, _ := fs.Sub(files, "static")
	mux := http.NewServeMux()
	mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServerFS(static)))
	mux.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) {
		rows, err := db.QueryContext(r.Context(), "SELECT author, body FROM messages ORDER BY id DESC LIMIT 50")
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
		defer rows.Close()
		var msgs []message
		for rows.Next() {
			var m message
			if err := rows.Scan(&m.Author, &m.Body); err != nil {
				http.Error(w, err.Error(), http.StatusInternalServerError)
				return
			}
			msgs = append(msgs, m)
		}
		page.ExecuteTemplate(w, "index.html", msgs)
	})
	mux.HandleFunc("POST /sign", func(w http.ResponseWriter, r *http.Request) {
		body := strings.TrimSpace(r.FormValue("body"))
		if body == "" || len(body) > 280 {
			http.Error(w, "say something, in 280 bytes or fewer", http.StatusBadRequest)
			return
		}
		// exe.dev's proxy tells the app who is signed in.
		author := "a visitor"
		if email := r.Header.Get("X-ExeDev-Email"); email != "" {
			author, _, _ = strings.Cut(email, "@")
		}
		if _, err := db.ExecContext(r.Context(), "INSERT INTO messages (author, body) VALUES (?, ?)", author, body); err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
		http.Redirect(w, r, "/", http.StatusSeeOther)
	})

	srv := &http.Server{
		Addr: ":" + env("PORT", "8000"),
		// Rejects cross-site form posts (CSRF), using the browser's Sec-Fetch-Site and Origin headers.
		Handler:           http.NewCrossOriginProtection().Handler(mux),
		ReadHeaderTimeout: 10 * time.Second,
	}
	go func() {
		if err := srv.ListenAndServe(); !errors.Is(err, http.ErrServerClosed) {
			fatal("serve", err)
		}
	}()
	slog.Info("listening", "addr", srv.Addr)

	// systemctl stop and restart send SIGTERM: stop accepting connections,
	// let requests in flight finish (up to 20 seconds), then exit.
	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
	defer stop()
	<-ctx.Done()
	slog.Info("shutting down")
	ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
	defer cancel()
	if err := srv.Shutdown(ctx); err != nil {
		slog.Error("shutdown", "err", err)
	}
}

// openDB opens SQLite and applies new migrations. WAL lets requests read
// while another writes; busy_timeout makes writers wait their turn.
func openDB(url string) (*sql.DB, error) {
	sep := "?"
	if strings.Contains(url, "?") {
		sep = "&"
	}
	db, err := sql.Open("sqlite", url+sep+"_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)")
	if err != nil {
		return nil, err
	}
	return db, migrate(db)
}

// migrate runs migrations/NNNN_name.sql files newer than the database's
// user_version, in order, each in a transaction that records its number.
func migrate(db *sql.DB) error {
	var version int
	if err := db.QueryRow("PRAGMA user_version").Scan(&version); err != nil {
		return err
	}
	names, _ := fs.Glob(files, "migrations/*.sql")
	for _, name := range names {
		n, err := strconv.Atoi(strings.SplitN(path.Base(name), "_", 2)[0])
		if err != nil {
			return fmt.Errorf("%s: name migrations NNNN_name.sql", name)
		}
		if n <= version {
			continue
		}
		stmts, _ := files.ReadFile(name)
		tx, err := db.Begin()
		if err != nil {
			return err
		}
		if _, err := tx.Exec(string(stmts)); err != nil {
			tx.Rollback()
			return fmt.Errorf("%s: %w", name, err)
		}
		if _, err := tx.Exec(fmt.Sprintf("PRAGMA user_version = %d", n)); err != nil {
			tx.Rollback()
			return err
		}
		if err := tx.Commit(); err != nil {
			return err
		}
		slog.Info("migrated", "file", name)
	}
	return nil
}

func env(key, fallback string) string {
	if v := os.Getenv(key); v != "" {
		return v
	}
	return fallback
}

func fatal(what string, err error) {
	slog.Error(what, "err", err)
	os.Exit(1)
}
EOF

The first migration, the page, and its stylesheet:

cd /tmp/new-app
cat > migrations/0001_messages.sql <<'EOF'
CREATE TABLE messages (
  id INTEGER PRIMARY KEY,
  author TEXT NOT NULL,
  body TEXT NOT NULL,
  created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
EOF
cat > templates/index.html <<'EOF'
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Guestbook</title>
<link rel="stylesheet" href="/static/style.css">
</head>
<body>
<main>
<h1>Guestbook</h1>
<form method="post" action="/sign">
  <input name="body" required maxlength="280" placeholder="Say hello">
  <button>Sign</button>
</form>
<ul>
{{range .}}  <li><b>{{.Author}}</b>: {{.Body}}</li>
{{else}}  <li>No messages yet. Be the first.</li>
{{end}}</ul>
</main>
</body>
</html>
EOF
cat > static/style.css <<'EOF'
body { font: 16px/1.5 system-ui, sans-serif; margin: 0; }
main { max-width: 36rem; margin: 3rem auto; padding: 0 1rem; }
form { display: flex; gap: .5rem; }
input { flex: 1; padding: .5rem; }
button { padding: .5rem 1rem; }
ul { padding: 0; list-style: none; }
EOF

Check that it builds and vets cleanly, then commit and push to the VM's repository. The push deploys:

cd /tmp/new-app
go mod tidy
go vet ./...
git add .gitignore go.mod go.sum main.go migrations templates static
git -c user.name="$(hostname)" -c user.email="exedev@$(hostname).exe.xyz" commit -qm "A guestbook: net/http and SQLite"
git push ~/app.git HEAD:main

The output ends with ==> Deployed and a commit. The code now lives in ~/app.git (the owner clones it from there), so remove the scratch copy:

cd ~ && rm -rf /tmp/new-app

Deploy and check#

If you brought a repository, set MAIN_PACKAGE in ~/app.env if its main package isn't at the top (such as ./cmd/server), and add the settings it reads (see its README or .env.example). Make it listen on port 8000 and keep SQLite files in ~/data. Then deploy its default branch:

~/app.git/hooks/pre-receive HEAD

(A new app deployed when you pushed it.) Check, for either kind:

systemctl is-enabled app; systemctl is-active app
readlink ~/app
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8000/

Expect enabled, active, the release directory, and 200 (or your home page's usual status, such as a 302 to a login page). For a new app, check that the guestbook reads the database and serves its stylesheet:

curl -s http://localhost:8000/ | grep -o 'No messages yet'
curl -s -o /dev/null -w '%{http_code} %{content_type}\n' http://localhost:8000/static/style.css
sqlite3 ~/data/app.db 'PRAGMA user_version' .tables

Expect No messages yet, 200 text/css; charset=utf-8, 1, and messages.

Check the backup#

Take the first backup by hand, then open it in a scratch directory, never over ~/data, to check that it holds what a restore needs:

latest=$(~/bin/app-backup manual)
rm -rf /tmp/restore-check && mkdir /tmp/restore-check
tar -xzf "$latest" -C /tmp/restore-check
ls -A /tmp/restore-check /tmp/restore-check/data
git -C ~/app.git bundle verify /tmp/restore-check/app.bundle
find /tmp/restore-check/data -type f \( -name '*.db' -o -name '*.sqlite' -o -name '*.sqlite3' \) |
  while read -r db; do echo "$db: $(sqlite3 "$db" 'PRAGMA integrity_check')"; done
rm -rf /tmp/restore-check

Expect app.bundle, app.env, and data; The bundle records a complete history; and ok for each database (a new app has app.db).

Setup is done. Continue with Open your app.

On the same VM#

On the VM, pick the archive (ls -lt ~/backups), unpack it, and swap its data in while the app is stopped. The current data is kept aside:

backup=~/backups/app-nightly-20261031-030512.tar.gz   # the one you want
rm -rf /tmp/restore && mkdir /tmp/restore && tar -xzf "$backup" -C /tmp/restore
sudo systemctl stop app
mv ~/data ~/data.before-restore-$(date +%s)
cp -a /tmp/restore/data ~/data
sudo systemctl start app

Open the app and check your data. ~/app.env is in /tmp/restore too, if you need an old setting back. Delete ~/data.before-restore-* when you're sure. If the backup is older than the code's migrations, the app applies the missing ones when it starts.

On a new VM#

Create the VM and follow Set up by hand up to, but not including, "Get the code". Copy the archive over from your computer (scp app-nightly-20261031-030512.tar.gz my-go.exe.xyz:), then on the VM:

rm -rf /tmp/restore && mkdir /tmp/restore && tar -xzf ~/app-nightly-20261031-030512.tar.gz -C /tmp/restore
git clone --bare /tmp/restore/app.bundle ~/app.git && git -C ~/app.git remote remove origin
install -m 600 /tmp/restore/app.env ~/app.env
rm -rf ~/data && cp -a /tmp/restore/data ~/data

Then follow Add the deploy hook and deploy with ~/app.git/hooks/pre-receive HEAD. On your computer, point the exe remote at the new VM: git remote set-url exe <new-vm>.exe.xyz:app.git.

Both travel through Git, like any other change. On your computer:

• Dependencies: go get -u ./... (or go get modernc.org/sqlite@latest for one), then go mod tidy, run your tests, commit go.mod and go.sum, and push.
• Go: go get go@latest raises the go line in go.mod to the newest release (or name one: go get go@1.27.3). Commit and push: if it's newer than the VM's Go, the build downloads that version first, into ~/go/pkg/mod.

The Go in /usr/local/go comes with the VM's image and only matters for builds whose go.mod asks for something older. sudo apt-get update && sudo apt-get upgrade -y keeps Ubuntu current. Old toolchains and modules pile up in ~/go/pkg/mod; go clean -modcache empties it, and the next build downloads what it needs.

On the VM, the app's logs:

journalctl -u app -n 100 --no-pager

• git push says Deploy FAILED and pre-receive hook declined. The lines above it are the error, usually the same one go build gives on your computer. The running version didn't change; fix it and push again.
• missing go.sum entry or updates to go.mod needed. Run go mod tidy on your computer and commit go.mod and go.sum.
• no Go files in /home/exedev/releases/.... The main package isn't at the top of the repository: set MAIN_PACKAGE in ~/app.env.
• HTTP 000 in the failed deploy. The new version exited or didn't listen on port 8000; journalctl -u app -n 50 says why, often a missing setting.
• cross-origin request detected (403). http.CrossOriginProtection refused a post from another site; trust one with AddTrustedOrigin.
• Disk filling up. du -sh ~/releases/* ~/backups ~/go ~/.cache/go-build; the hook keeps three releases, and go clean -cache -modcache empties Go's caches.