From 716ae14b0f682decbe0516077ace50231a6a1075 Mon Sep 17 00:00:00 2001 From: Ralksta <16941050+ralksta@users.noreply.github.com> Date: Fri, 7 Aug 2026 11:07:34 +0200 Subject: [PATCH] Close the gaps left after v0.9.2's outage handling: route tests, stale cache, loading states (#397) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs: brainstorm resilience/adoption features and plan follow-up work Brainstorm and deep-dive on where Immich Folio breaks under Immich outages and what blocks self-hosted adoption, plus an implementation plan for the remaining gaps after v0.9.2 (route-handler tests, stale-while-error caching, loading skeletons). * test: enable route-handler tests and cover admin guards and image content-type vitest.config.ts only ran lib/__tests__/**, so route handlers were structurally untestable. Two mechanisms live only at the route level and cannot be extracted the way lib/imageSize.ts was: the admin auth guards and the image proxy's Content-Type sanitisation (the stored-XSS fix from c2fa8e7). Both were correct but unverified by any test. Adds a table-driven test over all ten guarded /api/admin/* handlers, including a filesystem walk that fails if a future route is added without a row in the table. Adds a route-level test for /api/image/[id] covering token validation, the six Content-Type rewrite cases, ETag/304, and the 404 vs 503 (ImmichUnavailableError) distinction. Verified both catch real regressions by temporarily removing the admin guards and the Content-Type rewrite and confirming the expected tests fail, then restoring and confirming green again. 134 new tests (244 total, up from 210), tsc 0 errors, build succeeds. * feat: serve stale cache entries while Immich is unavailable v0.9.2 (#394) correctly split "gone" (404/410) from "unavailable" (ImmichUnavailableError), and pages now render a 5xx instead of a false 404 during an outage — but that 5xx still replaced a gallery that had loaded fine seconds earlier. lib/cache.ts deleted expired entries on access, so nothing survived long enough to fall back to. Cache entries now carry two deadlines: staleAt (the existing CACHE_TTL, after which get() still reports a miss) and hardExpiresAt (the new STALE_MAX_AGE, default 24h, after which the entry is dropped entirely). Between the two, getStale() can still return the data. getAlbums(), getAlbum() and getAssetInfo() catch ImmichUnavailableError and fall back to getStale() before propagating. The two MISSING-sentinel writes (definitive 404s) are deliberately left on the short cacheTtl window — a corrected asset ID should take effect after the normal TTL, not sit in the 24h stale window meant for outages. STALE_MAX_AGE=0 disables the fallback. Verified the fallback only engages on a real outage, not on a cache hit at the TTL boundary, and that a definitive 404 still reports missing rather than falling back. 256 tests (up from 244), tsc 0 errors, build succeeds. * feat: add loading skeletons for dynamic gallery routes Every gallery route is force-dynamic and waits on a live Immich round-trip — up to IMMICH_TIMEOUT_MS when the server is struggling — with nothing rendered until it resolves. loading.tsx renders instantly while the page component awaits its data. Not browser-verified against live data: this worktree has no .env.local and only .example content files, so the app boots into setup mode (confirmed the dev server starts cleanly and the setup screen renders, HTTP 200). tsc 0 errors, build succeeds, no new lint errors. * docs: document stale cache window, image token revocation, route-test convention - README: STALE_MAX_AGE env var and a "Behaviour when Immich is unreachable" section (stale-serves-until-STALE_MAX_AGE, 503 not 404, outages never cached). - docs/gallery-config.md: new "Image Token Revocation" subsection under System & Security — un-publishing an album does not revoke already issued image URLs; rotating AUTH_SECRET does. - CLAUDE.md: updated cache description to cover the stale window and MISSING sentinel, and a new code-convention note that route-level logic (guards, header sanitisation) should get a route-level test rather than being extracted into lib/ just to reach the test runner. --------- Co-authored-by: ralksta --- .env.local.example | 6 + CLAUDE.md | 30 +- README.md | 11 + app/[...path]/loading.tsx | 20 + app/api/admin/__tests__/admin-guards.test.ts | 158 +++ app/api/image/__tests__/image-route.test.ts | 114 ++ app/loading.module.css | 47 + app/loading.tsx | 15 + docs/brainstorm-2026-08-05.md | 336 +++++ docs/deep-dive-2026-08-05.md | 364 ++++++ docs/gallery-config.md | 20 +- .../plans/2026-08-05-resilience-boundaries.md | 1154 +++++++++++++++++ lib/__tests__/cache-stale.test.ts | 66 + lib/__tests__/immich.test.ts | 66 + lib/cache.ts | 56 +- lib/config/index.ts | 2 + lib/config/schema.ts | 2 + lib/env.ts | 9 + lib/immich.ts | 34 +- vitest.config.ts | 2 +- 20 files changed, 2478 insertions(+), 34 deletions(-) create mode 100644 app/[...path]/loading.tsx create mode 100644 app/api/admin/__tests__/admin-guards.test.ts create mode 100644 app/api/image/__tests__/image-route.test.ts create mode 100644 app/loading.module.css create mode 100644 app/loading.tsx create mode 100644 docs/brainstorm-2026-08-05.md create mode 100644 docs/deep-dive-2026-08-05.md create mode 100644 docs/superpowers/plans/2026-08-05-resilience-boundaries.md create mode 100644 lib/__tests__/cache-stale.test.ts diff --git a/.env.local.example b/.env.local.example index d2cc2ea..6178a1d 100644 --- a/.env.local.example +++ b/.env.local.example @@ -19,6 +19,12 @@ SITE_SUBTITLE=A visual journal # Cache TTL in seconds (integer >= 0, default: 300 = 5 minutes) CACHE_TTL=300 +# How long (in seconds) an expired cache entry may still be served while the +# Immich server is unreachable. Keeps the public gallery online across an +# Immich restart instead of showing an error page. Set to 0 to disable. +# Default: 86400 (24 hours) +STALE_MAX_AGE=86400 + # Cache-buster for image and video URLs (any string, default: empty = off). # Images are served with a one-year immutable cache, so browsers never # revalidate them. If you regenerate thumbnails in Immich, or rotate/edit a diff --git a/CLAUDE.md b/CLAUDE.md index 59be0de..2cc3453 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,7 +52,7 @@ Singleton `ImmichClient` class exported as `immich`. All Immich API calls are se - **Album allowlist** — `getAlbums()` fetches `?shared=true` albums but only returns IDs listed in `config.albums`. Requests for unlisted albums are silently rejected. - **Request coalescing** — pending promises are stored in `Map` fields to deduplicate concurrent requests for the same album/asset (important for `Promise.all` calls in grids). -- **In-memory LRU cache** (`lib/cache.ts`) — 200-entry LRU, configurable TTL via `CACHE_TTL`. Cache keys: `albums-list`, `album-`, `asset-`. +- **In-memory LRU cache** (`lib/cache.ts`) — 200-entry LRU. Entries carry two deadlines: `staleAt` (the `CACHE_TTL` window, after which `get()` reports a miss) and `hardExpiresAt` (`STALE_MAX_AGE`, after which the entry is dropped). Between the two, `getStale()` still returns the data — `lib/immich.ts` falls back to it when a request fails with `ImmichUnavailableError`, so the gallery keeps serving the last known albums during an Immich outage instead of erroring. Definitive 404s are cached as a `MISSING` sentinel under the normal TTL and are deliberately excluded from the stale window. Cache keys: `albums-list`, `album-`, `asset-`. - **Image streaming** — `streamAsset()` proxies binary responses; never loads the full image into memory. ### Asset token security (`lib/tokens.ts`) @@ -61,19 +61,19 @@ Singleton `ImmichClient` class exported as `immich`. All Immich API calls are se ### API routes (`app/api/`) -| Route | Purpose | -|---|---| -| `GET /api/image/[id]` | Image proxy — decodes token, rate-limits, streams from Immich | -| `GET /api/exif/[id]` | EXIF data for lightbox panel | -| `POST /api/auth` | Password submission → sets `HttpOnly` cookie | -| `GET /api/og` | Dynamic OG image generation (rate-limited) | -| `GET /api/map` | Aggregated GPS coordinates for map view | -| `GET /api/health` | Health check | -| `GET/POST/DELETE /api/admin/auth` | Admin login, session check, logout | -| `GET/PUT /api/admin/gallery` | Read/write `gallery.yaml` | -| `GET/PUT /api/admin/settings` | Read/write `settings.yaml` | -| `GET /api/admin/albums` | Browse all shared Immich albums (admin-only) | -| `POST /api/admin/reload` | Invalidate config + Immich cache | +| Route | Purpose | +| --------------------------------- | ------------------------------------------------------------- | +| `GET /api/image/[id]` | Image proxy — decodes token, rate-limits, streams from Immich | +| `GET /api/exif/[id]` | EXIF data for lightbox panel | +| `POST /api/auth` | Password submission → sets `HttpOnly` cookie | +| `GET /api/og` | Dynamic OG image generation (rate-limited) | +| `GET /api/map` | Aggregated GPS coordinates for map view | +| `GET /api/health` | Health check | +| `GET/POST/DELETE /api/admin/auth` | Admin login, session check, logout | +| `GET/PUT /api/admin/gallery` | Read/write `gallery.yaml` | +| `GET/PUT /api/admin/settings` | Read/write `settings.yaml` | +| `GET /api/admin/albums` | Browse all shared Immich albums (admin-only) | +| `POST /api/admin/reload` | Invalidate config + Immich cache | The image proxy maps requested pixel widths (`?w=`) to Immich size tiers: `≤250px→thumbnail`, `≤1440px→preview`, `>1440px→original` (`lib/imageSize.ts`). @@ -82,6 +82,7 @@ The image proxy maps requested pixel widths (`?w=`) to Immich size tiers: `≤25 ### Routing (`app/[...path]/page.tsx`) Single catch-all route handles three cases: + 1. `/[subpage-slug]/[album-slug]` — renders album detail with back-link to subpage 2. `/[subpage-slug]` — if subpage has >1 album, renders `SubpageGridView`; if exactly 1 album, renders `AlbumDetailView` directly 3. `/[album-slug]` — standalone album detail @@ -134,3 +135,4 @@ Next.js 16 renamed the `middleware` file convention to `proxy`: the file is `pro - **All Immich data flows server-side** — raw asset UUIDs must never appear in client-rendered HTML or JS. Always use `encodeAssetId()` / `imageUrl()` / `exifUrl()` from `lib/urls.ts` before passing IDs to components. - **Rate-limit all expensive endpoints** — apply `checkRateLimit` from `lib/rate-limit.ts` to any route doing heavy computation or upstream API calls. - **Commit style** — Conventional Commits: `feat:`, `fix:`, `security:`, `docs:`, `chore:`. +- **Route handlers are testable** — `vitest.config.ts` includes `app/**/__tests__/**/*.test.ts`. Logic that lives in a route (auth guards, header sanitisation) belongs in a route-level test; do not extract it into `lib/` purely to make it reachable by the test runner. diff --git a/README.md b/README.md index 61b301e..2a93690 100644 --- a/README.md +++ b/README.md @@ -96,6 +96,7 @@ IMMICH_API_KEY=your-api-key SITE_TITLE=My Photography # default: "Gallery" SITE_SUBTITLE=A visual journal # default: empty CACHE_TTL=300 # seconds, default: 300 +STALE_MAX_AGE=86400 # seconds an expired cache entry survives during an outage, default: 86400 (24h), 0 disables IMMICH_TIMEOUT_MS=15000 # Immich response wait, default: 15000 IMAGE_CACHE_VERSION=1 # bump to bust browser image caches, default: off RATE_LIMIT_RPM=1500 # requests/min/IP, default: 1500 @@ -191,6 +192,16 @@ The container includes a built-in health check at `/api/health`: curl http://localhost:7211/api/health ``` +### Behaviour when Immich is unreachable + +Immich Folio buffers your gallery rather than merely proxying it: + +- Album and asset pages keep serving the last known good data for up to `STALE_MAX_AGE`, so a restarting or briefly unreachable Immich does not take the public site down with it. +- Once nothing cached is left, they return `503`, never `404` — a `404` would tell search engines to drop a URL for content that still exists. +- Outages are never cached, so the gallery recovers as soon as Immich does. + +The cache lives in the process, so it is empty right after a container restart. + ### Reverse Proxy Put Immich Folio behind nginx / Caddy / Traefik with TLS. Example Caddy config: diff --git a/app/[...path]/loading.tsx b/app/[...path]/loading.tsx new file mode 100644 index 0000000..f604de3 --- /dev/null +++ b/app/[...path]/loading.tsx @@ -0,0 +1,20 @@ +/** + * Instant shell for album and subpage routes. These are force-dynamic and wait + * on a live Immich round-trip — up to IMMICH_TIMEOUT_MS when the server is + * struggling — so without this the viewer stares at a blank page. + */ + +import styles from '../loading.module.css'; + +export default function Loading() { + return ( + <> +