Add photo content-hash dedup and local↔S3 library sync.

Uploads skip per-gallery duplicates, checksums can be backfilled, and STORAGE_BACKEND plus sync tooling make switching storage backends safe.
This commit is contained in:
Michilis
2026-08-05 05:41:12 +00:00
parent dafa3711f8
commit 498d7d8a7d
31 changed files with 2216 additions and 64 deletions
+16
View File
@@ -86,8 +86,23 @@ npm run dev:photos # go run ./cmd/photo-api
npm run build:photos # go build -o bin/photo-api
npm run migrate:photos # apply photos_* migrations
npm run test:photos # go test ./...
# Move the photo library between storage backends (both must be configured
# in photo-api/.env; the source is left untouched, reruns skip what is there)
npm run sync:photos:to-s3 # local disk -> S3
npm run sync:photos:to-local # S3 -> local disk
npm run sync:photos -- to-s3 --dry-run # flags: --dry-run --overwrite
# --concurrency=N --gallery=<id>
# Uploads are deduplicated per gallery by content hash. Photos uploaded before
# that existed need hashing once (idempotent, deletes nothing):
npm run backfill:photos:checksums
```
After a sync, set `STORAGE_BACKEND=s3` (or `local`) in `photo-api/.env` and
restart the service to serve from the new backend. See
[`photo-api/README.md`](photo-api/README.md#move-the-library-between-backends).
You can also run per workspace:
```bash
@@ -117,6 +132,7 @@ Key settings (see `photo-api/.env.example`):
- **DB**: `DB_TYPE` and `DATABASE_URL` — point at the **same** database as the backend
- **Auth**: none needed for user auth — the service validates Better Auth session cookies against the shared database. `PHOTO_VIEW_SECRET` signs gallery image view tokens (falls back to `JWT_SECRET` during migration).
- **Storage**: `STORAGE_PATH` (local disk) or `S3_ENDPOINT` + `S3_BUCKET` (S3/Garage/MinIO); S3 downloads use short-lived presigned URLs
- **Storage switch**: `STORAGE_BACKEND=auto|local|s3` (`auto` = S3 when it is configured). Keep both sides configured and flip this one line to move between them; `npm run sync:photos:to-s3` / `:to-local` copies the existing library first
- **Uploads/worker**: `MAX_UPLOAD_MB`, `WORKER_CONCURRENCY` (a worker generates thumb/preview JPEG variants with EXIF stripped)
### Frontend (`frontend/.env`)