Our incident: the web-vitals fetch bug

Symptom

After P10, four end-to-end tests on /events timed out at waitForLoadState("networkidle") against the dev server. A 15-second network trace showed 33 requests: zero browser events.search calls, but four telemetry.report calls with status 200 that stayed pending. These are the owner’s incident measurements, not a new run.

Read the error first

Playwright’s wait timed out; the API had accepted the telemetry. A 200 response means headers arrived, not that the response body finished. Reach succeeded; neither an IAM Allowed failure nor a Password failure fits this trace. Inspect which requests stay open before blaming event search or hydration, the step that makes server HTML interactive.

Decide

flowchart TB
  A["networkidle wait times out"] --> B{"Which traffic remains?"}
  B -->|"events.search repeats"| C["inspect list query lifecycle"]
  B -->|"dev tooling only"| D["use a page readiness signal"]
  B -->|"telemetry 200, pending"| E{"Does the hook read the body?"}
  E -->|"no"| F["consume or cancel the response"]
  E -->|"yes"| G["inspect response completion"]

Likely causes

Ranked for this trace:

1. Unread telemetry replies: the hook ignored each fetch response body. This was the cause.
2. Dev-server background traffic: another reason network idle can be a poor readiness signal.
3. A list request loop: ruled out here by zero browser search calls.

How to check

Console

Browser DevTools → Network: group by endpoint and compare status with completion. The four telemetry replies remained pending despite their 200 headers. Check the Search events region’s data-hydrated="true" marker separately.

CLI

Read the response handling and regression assertions:

rg -n 'response.text|pending|requestfinished|15_000|data-hydrated' apps/web/src/features/telemetry/hooks/use-web-vitals.ts apps/web/e2e/events.spec.ts

CloudTrail

Not applicable: these are browser-to-app requests, not AWS management API calls. Use browser request events and API logs; CloudTrail cannot show an unread browser response body.

Fix

Consume or cancel every fetch response. The hook now drains its small acknowledgement with response.text(), with a catch for failures. Replace network-idle waits with specific assertions: the hydration marker, and the expected auth response where needed.

apps/web/src/features/telemetry/hooks/use-web-vitals.ts → report

Prevent

The regression in apps/web/e2e/events.spec.ts waits for hydration and a telemetry 200, then requires all tracked app requests to finish or fail. It observes 15 seconds without interaction and rejects new app requests or pending ones. It failed before the fix and passed after, according to the owner’s incident report. This bounded observation checks a quiet-page property; it is not the page’s readiness signal.

Find the web app on the map →

Say it in 30 seconds

Say it like this (20-30 seconds)

Four events tests timed out waiting for network idle. A trace showed no browser event-search calls, but four telemetry requests returned 200 and stayed pending because their response bodies were unread. We consumed every response, waited for an explicit hydration marker, and added a regression that checks requests finish and no new app traffic appears after load. Always consume or cancel fetch responses; network idle is a fragile test signal.

Advanced: Why the two checks are separate

The fix in this incident reads the body; cancellation is an alternative when the body is unwanted. Response.text() reads a response stream to completion. This incident does not establish that every unread browser response always hangs. Playwright discourages networkidle for readiness and recommends assertions about the page instead. The regression ignores hot module replacement (HMR) and static assets, tracking only paths beginning /trpc/.