bbeandCursor
ede8817583
Isolate next dev from production .next, add build-guard/atomic deploy/health watchdog, error boundaries, and fix JWT startup, meetup leaks, media path traversal, SVG/memory uploads, and JSON-LD escaping. Co-authored-by: Cursor <cursoragent@cursor.com>
86 lines
3.1 KiB
Markdown
86 lines
3.1 KiB
Markdown
# Ops: keeping the site up
|
|
|
|
These files harden the deploy/run path so a stray `next dev` or a half-finished
|
|
build can no longer take the public site down (root cause of the `/events` 500).
|
|
|
|
## What's here
|
|
|
|
| Path | Purpose |
|
|
|------|---------|
|
|
| `frontend/scripts/verify-prod-build.sh` | Fails fast if `.next` is missing or dev-contaminated. Wired as `ExecStartPre`. |
|
|
| `scripts/deploy-frontend.sh` | Atomic build-verify-swap-restart deploy. Use this to deploy the frontend. |
|
|
| `scripts/healthcheck.sh` | Watchdog: probes `/`, `/events`, `/blog`, `/api/health`; restarts frontend after repeated failures. |
|
|
| `ops/systemd/bbe-frontend.service.d/verify-build.conf` | Drop-in adding the build guard to `bbe-frontend.service`. |
|
|
| `ops/systemd/bbe-site-healthcheck.{service,timer}` | Runs the watchdog every 5 minutes. |
|
|
|
|
## Layered defenses
|
|
|
|
1. **Isolation** — `next dev` now writes to `.next-dev` (`NEXT_DIST_DIR` in
|
|
`frontend/package.json` + `distDir` in `frontend/next.config.js`), so it can
|
|
never overwrite the production `.next`.
|
|
2. **Fail-fast boot** — the `ExecStartPre` guard refuses to start on a bad build.
|
|
3. **Atomic deploys** — `deploy-frontend.sh` builds into `.next-build`, verifies,
|
|
then renames into place; the running server never sees a partial `.next`.
|
|
4. **Self-healing** — the watchdog restarts a wedged frontend within minutes.
|
|
5. **Graceful UI** — `app/error.tsx` / `app/global-error.tsx` render a retry page
|
|
instead of a bare 500 if a request-time route ever throws.
|
|
|
|
## Install (requires sudo)
|
|
|
|
One-shot installer (preferred):
|
|
|
|
```bash
|
|
cd /home/bbe/BelgianBitcoinEmbassy
|
|
sudo ops/install-systemd.sh
|
|
sudo systemctl restart bbe-backend # after backend code changes
|
|
```
|
|
|
|
Or manually:
|
|
|
|
```bash
|
|
cd /home/bbe/BelgianBitcoinEmbassy
|
|
|
|
# 1. Build guard (ExecStartPre)
|
|
sudo mkdir -p /etc/systemd/system/bbe-frontend.service.d
|
|
sudo cp ops/systemd/bbe-frontend.service.d/verify-build.conf \
|
|
/etc/systemd/system/bbe-frontend.service.d/verify-build.conf
|
|
|
|
# 2. Health watchdog
|
|
sudo cp ops/systemd/bbe-site-healthcheck.service /etc/systemd/system/
|
|
sudo cp ops/systemd/bbe-site-healthcheck.timer /etc/systemd/system/
|
|
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart bbe-frontend
|
|
sudo systemctl enable --now bbe-site-healthcheck.timer
|
|
```
|
|
|
|
Verify:
|
|
|
|
```bash
|
|
systemctl status bbe-frontend --no-pager
|
|
systemctl list-timers bbe-site-healthcheck --no-pager
|
|
journalctl -u bbe-healthcheck -n 20 --no-pager
|
|
```
|
|
|
|
## Deploying the frontend from now on
|
|
|
|
Do **not** run `npm run build` directly in the live tree and then restart. Use:
|
|
|
|
```bash
|
|
scripts/deploy-frontend.sh
|
|
```
|
|
|
|
It refuses to build when free memory is low (`MIN_FREE_MB`, default 600), builds
|
|
into `.next-build`, verifies, atomically swaps to `.next`, and restarts the
|
|
service. The previous build is kept at `.next-prev` for a quick manual rollback.
|
|
|
|
## Note on the watchdog restarting the service
|
|
|
|
`healthcheck.sh` runs `systemctl restart bbe-frontend`. The systemd service above
|
|
runs as root, so this works out of the box. If you ever run the script as the
|
|
`bbe` user instead, grant a narrow sudoers rule:
|
|
|
|
```
|
|
bbe ALL=(root) NOPASSWD: /usr/bin/systemctl restart bbe-frontend
|
|
```
|