Users see stale data

Symptom

Diagnostic scenario: an event is saved but missing from the next /events view. The API may log “web cache revalidate failed: /events on the web server may be stale for up to 30 s”. That quoted warning exists in the code; it is not a guarantee of maximum end-to-end age.

Read the error first

The API reports a failed call to the web cache’s invalidation endpoint. The database write may already have succeeded. Reach means the web hook cannot be contacted; Password means its shared secret was rejected with 401. IAM Allowed is not what authorizes this HTTP hook. Old data without a warning may simply come from another cache layer.

Decide

flowchart TB
  A[Saved event absent] --> B{API list also stale?}
  B -->|Yes| C[Inspect generation and refresh errors]
  B -->|No| D{Server HTML stale?}
  D -->|Yes| E[Inspect Next cache invalidation]
  D -->|No| F[Inspect browser query cache]
  C --> G[Compare filters with saved data]

Likely causes

Ordered from the symptom’s nearest cache outward; no incident-frequency claim:

1. Browser query data remains fresh for 30 seconds, or Next.js still holds its own list.
2. The web revalidate call failed, was unconfigured, or reached a different web instance.
3. Redis invalidation failed, refresh failed, or a direct database edit bypassed normal invalidation.

How to check

Console

Browser Network: compare the actual list response with server HTML and client display. CloudWatch → /minitickets/dev/api: inspect cache status and revalidate warnings. /minitickets/dev/web: look for next data cache miss: events.list. Match canonical filters before comparing lists.

CLI

export AWS_PROFILE=ticketlite
curl -fsS https://minitickets.theversehub.in/trpc/events.list
aws logs tail /minitickets/dev/api --since 10m
aws logs tail /minitickets/dev/web --since 10m

Compare one controlled request at each layer. Avoid a refresh loop that obscures background work.

CloudTrail

These application reads and cache invalidations are not CloudTrail management events. If the problem started with deployment, correlate ECS UpdateService with app logs. CloudTrail alone cannot tell whether a cached list was stale.

Fix

Repair the failing layer. Normal event creation and booking invalidate Redis’s generation first, then request Next.js invalidation. Check Service Connect and matching secret configuration without printing its value. Refresh browser data when its server copy changes.

Prevent

Keep invalidation tests around apps/api/src/services/events.ts → invalidateEventLists and the web route. Monitor revalidate failures. Treat browsing as eventually fresh; booking checks seats in Postgres.

Say it in 30 seconds

Say it like this (20-30 seconds)

I compare the API response, server HTML, and browser data to find the layer keeping the old copy. MiniTickets has Redis, Next.js and browser caches, so one successful invalidation is not enough. Writes invalidate Redis before notifying the web server. I check hook failures and filters, repair the responsible layer, and test freshness after a write. Booking correctness still comes from the database transaction.

Advanced: Why thirty seconds is not an end-to-end bound

Redis uses a 30–33 second soft lifetime and five-minute hard lifetime. Next.js and the browser each use 30 seconds. Refreshing an upper cache can copy already stale lower data. apps/web/src/features/events/server/cached-events.ts records a local production-build measurement: a direct database edit appeared after 90.6 seconds. Event creation with its hook enabled appeared in server HTML after 8–68 ms. These are local observations, not service promises. Each web instance has its own cache; the current hook invalidates only the instance it reaches.