Casebook

Start with the symptom, then identify which service produced it. Reach asks whether the request can get there; Allowed asks whether its identity has permission; Password asks whether the supplied secret is valid. Some cases concern application behavior rather than any of these three.

The first twenty pages are troubleshooting scenarios tied to MiniTickets. Some include historical observations, clearly labelled. The last six recount our incidents. Each page gives a decision diagram, checks, a fix, prevention and a short interview answer. Start with the runbook index during an incident; use these cases to understand why the checks matter. The scheduled-Lambda scenario is an interview exercise, not a deployed job.

Real symptoms, read like a detective: the error first, then the likely causes, checks and fixes.

Reference: open a page when you need it. No need to read it in order.

1. A scheduled Lambda gets AccessDenied after weeks
2. ECS cannot pull the image
3. A task cannot read its secret
4. The load balancer returns 502 or 503
5. A target stays unhealthy
6. The API cannot reach Postgres
7. Lambda times out
8. The dead-letter queue keeps growing
9. Users get the same email twice
10. SNS delivers nothing
11. A presigned URL returns 403
12. A browser upload to S3 fails with CORS
13. GitHub cannot assume the AWS role
14. CD deployed old code
15. A migration failed in the middle of a deploy
16. Users see stale data
17. Real users get 429 Too Many Requests
18. Slow first requests: cold starts
19. Lambda is throttled at the account limit
20. The bill suddenly jumps
21. Our incident: the OIDC subject that would not match
22. Our incident: a stale approval deleted resources
23. AccessDenied right after a policy change: propagation delay, or did the policy really change?
24. Our incident: CloudFront needs account verification
25. Our incident: the web-vitals fetch bug
26. Our incident: deleting a Lambda trigger needs read permission too