Files
Spanglish/photo-api/README.md
T
Michilis 498d7d8a7d Add photo content-hash dedup and local↔S3 library sync.
Uploads skip per-gallery duplicates, checksums can be backfilled, and STORAGE_BACKEND plus sync tooling make switching storage backends safe.
2026-08-05 05:41:12 +00:00

170 lines
7.8 KiB
Markdown

# photo-api
Standalone Go service for event photo galleries: admins upload photos from
past events, group them into galleries, and share them with the community.
Lives in this monorepo but is its own module, binary, and deployable unit.
Design/decisions: [PLAN.md](./PLAN.md).
## What it does
- Galleries with four visibility modes: `public` (listed on /photos),
`private` (admins only), `link` (share-token URL), `ticket` (logged-in
users with a confirmed ticket for the linked event — any confirmed ticket
when the event is free).
- Originals are stored byte-identical for download; a worker generates JPEG
variants (thumb 512px q78, preview 2048px q85, EXIF stripped/orientation
applied) queued in the DB with retries.
- Uploads are deduplicated per gallery: see
[Duplicate detection](#duplicate-detection).
- Storage: local disk (`STORAGE_PATH`) or S3/Garage/MinIO (set `S3_ENDPOINT`
+ `S3_BUCKET`), same selection convention as the backend, overridable with
`STORAGE_BACKEND`. S3 downloads use short-lived presigned URLs; nothing in
the bucket is public. Switching backends later: see
[Move the library between backends](#move-the-library-between-backends).
- Auth: validates the backend's HS256 JWTs with the shared `JWT_SECRET`
(issuer `spanglish`, audience `spanglish-app`) including the DB-backed
tokenVersion/account-status revocation check. Admin surface is
`admin`/`organizer` only.
- Database: the same Postgres or SQLite database as the backend (`DB_TYPE`,
`DATABASE_URL`). This service owns only the `photos_*` tables via its own
embedded migrations (`photo-api migrate`); drizzle-kit never sees them.
Reads of backend tables are confined to `internal/store/access.go` and
sanity-checked at startup.
## Develop
```bash
cp .env.example .env # set JWT_SECRET + DATABASE_URL to match backend/.env
go run ./cmd/photo-api migrate
go run ./cmd/photo-api # serves on :3003 (dev)
go test ./... # SQLite; add PHOTO_TEST_PG=<url> to also run on Postgres
```
From the repo root: `npm run dev:photos`, `npm run build:photos`,
`npm run test:photos`, `npm run migrate:photos`, `npm run sync:photos`,
`npm run backfill:photos:checksums`. The Next dev server rewrites `/api/photos/*` to
`PHOTO_API_URL` (default `http://localhost:3003`), so the frontend needs no
extra config in dev.
HEIC uploads require a converter CLI on the host: `apt install libvips-tools`
(or `libheif-examples`). Without one, HEIC uploads are rejected with a clear
message and a startup warning is logged.
## Move the library between backends
`photo-api sync` copies the whole photo library one way between local disk and
S3, so local↔S3 is a config switch rather than a migration project. **Both
backends must be configured in `photo-api/.env`** (`STORAGE_PATH` *and* the
`S3_*` values); `STORAGE_BACKEND` decides which one actually serves requests,
so filling in S3 does not switch anything by itself.
Local disk → S3:
```bash
npm run sync:photos -- to-s3 --dry-run # see what would be copied
npm run sync:photos:to-s3 # copy it
# then set STORAGE_BACKEND=s3 in photo-api/.env and restart the service
```
S3 → local disk is the same with `to-local` / `STORAGE_BACKEND=local`
(`npm run sync:photos:to-local`). From `photo-api/`: `make sync-to-s3`,
`make sync-to-local`.
Flags (`npm run sync:photos -- to-s3 --overwrite`, or after the direction on
the direct scripts): `--dry-run`, `--overwrite` (re-copy objects already
present with the same size), `--concurrency=N` (default 4), `--gallery=<id>`.
How it behaves:
- The `photos_photos` rows are the inventory — for each photo the original
plus, once processed, the thumb and preview. Objects with no row (worker
scratch files, leftovers of deleted galleries) are not copied.
- Only the destination is written. The source stays as a fallback; delete it
by hand once the switch is verified.
- Reruns are cheap and safe: objects already on the destination with the same
size are skipped, so an interrupted or partly failed sync just needs
rerunning. A failed object is logged and the exit code is non-zero.
- Keys are identical on both backends, so nothing in the database changes and
no re-processing is triggered.
- Photos uploaded *after* the copy but *before* the restart land on the old
backend. For a clean cutover, stop the service, sync, flip
`STORAGE_BACKEND`, start again — or sync a second time after the switch to
pick up stragglers.
## Duplicate detection
Every upload is hashed (sha256 of the original bytes) into
`photos_photos.checksum`, unique per `(gallery_id, checksum)`. If a gallery
already holds those exact bytes, the incoming copy is **discarded**: no object
is stored, no row is inserted, and the upload response echoes the existing
photo with `"duplicate": true`. The rest of the batch continues normally — a
duplicate is not an error and does not consume a position.
Scope is one gallery. The same image can still live in several galleries, each
with its own row and its own stored object, so deleting a gallery never orphans
another one's photos.
The admin uploader panel shows those rows as *"Already in this gallery"*; no
second tile appears in the grid.
Photos uploaded before this existed have no checksum, so they are not matched
until hashed once:
```bash
npm run backfill:photos:checksums -- --dry-run # what would be hashed
npm run backfill:photos:checksums # hash it
```
From `photo-api/`: `make backfill-checksums`. Flags: `--dry-run`,
`--concurrency=N` (default 4), `--gallery=<id>`. It reads from the active
`STORAGE_BACKEND`, only ever writes the checksum column, and is idempotent —
rerun it after a sync or a restore. Photos whose content already matches an
earlier one in the same gallery are **reported and left unhashed**; the command
never deletes anything, so removing the extras is an admin's call.
## API
Everything under `/api/photos`. Errors are `{"error": string}`.
Admin (Bearer token, role admin/organizer):
| Method | Path | Purpose |
|---|---|---|
| POST | `/api/photos/galleries` | create gallery |
| GET | `/api/photos/galleries` | list all (`?eventId=`) |
| GET | `/api/photos/galleries/:id` | gallery + photos (all statuses) |
| PATCH | `/api/photos/galleries/:id` | update title/visibility/event/cover |
| DELETE | `/api/photos/galleries/:id` | delete gallery + objects |
| POST | `/api/photos/galleries/:id/photos` | multipart upload (`files`), deduplicated |
| PATCH | `/api/photos/galleries/:id/order` | reorder (`{photoIds}`) |
| POST | `/api/photos/galleries/:id/share-token` | rotate share token |
| DELETE | `/api/photos/photos/:photoId` | delete photo |
| POST | `/api/photos/photos/:photoId/retry` | requeue failed photo |
Viewer (anonymous or member token; `?token=` carries the share token):
| Method | Path | Purpose |
|---|---|---|
| GET | `/api/photos/public/galleries` | public index |
| GET | `/api/photos/public/galleries/:slug` | gallery view (access-checked) |
| GET | `/api/photos/public/events/:eventSlug/gallery` | newest gallery of an event (access-checked) |
| GET | `/api/photos/files/:photoId/:variant` | bytes; `thumb\|preview\|original` |
| GET | `/api/photos/health` | liveness |
## Deploy (systemd, primary)
```bash
cd photo-api && go build -o bin/photo-api ./cmd/photo-api
sudo apt install libvips-tools
sudo cp ../deploy/spanglish-photos.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now spanglish-photos
sudo nginx -t && sudo systemctl reload nginx # after installing the updated confs
```
nginx routes `location ^~ /api/photos/``127.0.0.1:3020` (see
`deploy/front-end_nginx.conf`, `deploy/back-end_nginx.conf`,
`deploy/spanglish_upstreams.conf`). The frontend's production `.env` should
set `PHOTO_API_URL=http://127.0.0.1:3020` for server-side rendering of the
public gallery pages. A `Dockerfile` is included as a secondary path for the
compose-scale setup.