fix: harden deploys and close top security holes after /events outage
Isolate next dev from production .next, add build-guard/atomic deploy/health watchdog, error boundaries, and fix JWT startup, meetup leaks, media path traversal, SVG/memory uploads, and JSON-LD escaping. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
bbe
@@ -0,0 +1,85 @@
|
||||
# Ops: keeping the site up
|
||||
|
||||
These files harden the deploy/run path so a stray `next dev` or a half-finished
|
||||
build can no longer take the public site down (root cause of the `/events` 500).
|
||||
|
||||
## What's here
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `frontend/scripts/verify-prod-build.sh` | Fails fast if `.next` is missing or dev-contaminated. Wired as `ExecStartPre`. |
|
||||
| `scripts/deploy-frontend.sh` | Atomic build-verify-swap-restart deploy. Use this to deploy the frontend. |
|
||||
| `scripts/healthcheck.sh` | Watchdog: probes `/`, `/events`, `/blog`, `/api/health`; restarts frontend after repeated failures. |
|
||||
| `ops/systemd/bbe-frontend.service.d/verify-build.conf` | Drop-in adding the build guard to `bbe-frontend.service`. |
|
||||
| `ops/systemd/bbe-site-healthcheck.{service,timer}` | Runs the watchdog every 5 minutes. |
|
||||
|
||||
## Layered defenses
|
||||
|
||||
1. **Isolation** — `next dev` now writes to `.next-dev` (`NEXT_DIST_DIR` in
|
||||
`frontend/package.json` + `distDir` in `frontend/next.config.js`), so it can
|
||||
never overwrite the production `.next`.
|
||||
2. **Fail-fast boot** — the `ExecStartPre` guard refuses to start on a bad build.
|
||||
3. **Atomic deploys** — `deploy-frontend.sh` builds into `.next-build`, verifies,
|
||||
then renames into place; the running server never sees a partial `.next`.
|
||||
4. **Self-healing** — the watchdog restarts a wedged frontend within minutes.
|
||||
5. **Graceful UI** — `app/error.tsx` / `app/global-error.tsx` render a retry page
|
||||
instead of a bare 500 if a request-time route ever throws.
|
||||
|
||||
## Install (requires sudo)
|
||||
|
||||
One-shot installer (preferred):
|
||||
|
||||
```bash
|
||||
cd /home/bbe/BelgianBitcoinEmbassy
|
||||
sudo ops/install-systemd.sh
|
||||
sudo systemctl restart bbe-backend # after backend code changes
|
||||
```
|
||||
|
||||
Or manually:
|
||||
|
||||
```bash
|
||||
cd /home/bbe/BelgianBitcoinEmbassy
|
||||
|
||||
# 1. Build guard (ExecStartPre)
|
||||
sudo mkdir -p /etc/systemd/system/bbe-frontend.service.d
|
||||
sudo cp ops/systemd/bbe-frontend.service.d/verify-build.conf \
|
||||
/etc/systemd/system/bbe-frontend.service.d/verify-build.conf
|
||||
|
||||
# 2. Health watchdog
|
||||
sudo cp ops/systemd/bbe-site-healthcheck.service /etc/systemd/system/
|
||||
sudo cp ops/systemd/bbe-site-healthcheck.timer /etc/systemd/system/
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart bbe-frontend
|
||||
sudo systemctl enable --now bbe-site-healthcheck.timer
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
systemctl status bbe-frontend --no-pager
|
||||
systemctl list-timers bbe-site-healthcheck --no-pager
|
||||
journalctl -u bbe-healthcheck -n 20 --no-pager
|
||||
```
|
||||
|
||||
## Deploying the frontend from now on
|
||||
|
||||
Do **not** run `npm run build` directly in the live tree and then restart. Use:
|
||||
|
||||
```bash
|
||||
scripts/deploy-frontend.sh
|
||||
```
|
||||
|
||||
It refuses to build when free memory is low (`MIN_FREE_MB`, default 600), builds
|
||||
into `.next-build`, verifies, atomically swaps to `.next`, and restarts the
|
||||
service. The previous build is kept at `.next-prev` for a quick manual rollback.
|
||||
|
||||
## Note on the watchdog restarting the service
|
||||
|
||||
`healthcheck.sh` runs `systemctl restart bbe-frontend`. The systemd service above
|
||||
runs as root, so this works out of the box. If you ever run the script as the
|
||||
`bbe` user instead, grant a narrow sudoers rule:
|
||||
|
||||
```
|
||||
bbe ALL=(root) NOPASSWD: /usr/bin/systemctl restart bbe-frontend
|
||||
```
|
||||
Executable
+30
@@ -0,0 +1,30 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Install the production hardening units (build guard + health watchdog).
|
||||
# Requires sudo. Safe to re-run (idempotent).
|
||||
#
|
||||
# sudo ops/install-systemd.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
echo "install-systemd: re-run as root, e.g.:" >&2
|
||||
echo " sudo $0" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p /etc/systemd/system/bbe-frontend.service.d
|
||||
cp "${REPO_DIR}/ops/systemd/bbe-frontend.service.d/verify-build.conf" \
|
||||
/etc/systemd/system/bbe-frontend.service.d/verify-build.conf
|
||||
cp "${REPO_DIR}/ops/systemd/bbe-site-healthcheck.service" /etc/systemd/system/
|
||||
cp "${REPO_DIR}/ops/systemd/bbe-site-healthcheck.timer" /etc/systemd/system/
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl restart bbe-frontend
|
||||
systemctl enable --now bbe-site-healthcheck.timer
|
||||
|
||||
echo "install-systemd: OK"
|
||||
systemctl status bbe-frontend --no-pager -n 5 || true
|
||||
systemctl list-timers bbe-site-healthcheck --no-pager || true
|
||||
@@ -0,0 +1,15 @@
|
||||
# Drop-in override for bbe-frontend.service.
|
||||
#
|
||||
# Refuses to (re)start the frontend when the .next production build is missing or
|
||||
# has been contaminated by a `next dev` run, so we fail loudly at boot instead of
|
||||
# silently serving 500s on every request-time route (e.g. /events).
|
||||
#
|
||||
# Install:
|
||||
# sudo mkdir -p /etc/systemd/system/bbe-frontend.service.d
|
||||
# sudo cp ops/systemd/bbe-frontend.service.d/verify-build.conf \
|
||||
# /etc/systemd/system/bbe-frontend.service.d/verify-build.conf
|
||||
# sudo systemctl daemon-reload
|
||||
# sudo systemctl restart bbe-frontend
|
||||
|
||||
[Service]
|
||||
ExecStartPre=/home/bbe/BelgianBitcoinEmbassy/frontend/scripts/verify-prod-build.sh
|
||||
@@ -0,0 +1,18 @@
|
||||
# Oneshot health probe for the BBE site, driven by bbe-site-healthcheck.timer.
|
||||
#
|
||||
# Runs as root so it can `systemctl restart bbe-frontend` when the site is down.
|
||||
#
|
||||
# Install:
|
||||
# sudo cp ops/systemd/bbe-site-healthcheck.service /etc/systemd/system/
|
||||
# sudo cp ops/systemd/bbe-site-healthcheck.timer /etc/systemd/system/
|
||||
# sudo systemctl daemon-reload
|
||||
# sudo systemctl enable --now bbe-site-healthcheck.timer
|
||||
|
||||
[Unit]
|
||||
Description=Belgian Bitcoin Embassy — site health watchdog
|
||||
After=network.target bbe-frontend.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/home/bbe/BelgianBitcoinEmbassy/scripts/healthcheck.sh
|
||||
SyslogIdentifier=bbe-healthcheck
|
||||
@@ -0,0 +1,13 @@
|
||||
# Fires the BBE site health watchdog every 5 minutes.
|
||||
|
||||
[Unit]
|
||||
Description=Run BBE site health watchdog every 5 minutes
|
||||
|
||||
[Timer]
|
||||
OnBootSec=3min
|
||||
OnUnitActiveSec=5min
|
||||
AccuracySec=30s
|
||||
Unit=bbe-site-healthcheck.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
Reference in New Issue
Block a user