Rate limits and the sale-day gate

In 1 minute

On a sale day, thousands of clicks arrive for a few hundred seats. Two checks in Valkey protect Postgres. A rate limiter gives each user a small bucket of booking tokens, so floods get 429. A seat counter, the inventory gate, rejects bookings for a sold-out event with 409 before they touch the database. Postgres still makes the final decision under the row lock, so nobody is oversold.

Picture it

flowchart TB
  R["booking request"] --> L{"token left?"}
  L -->|"no"| E429["429, Retry-After"]
  L -->|"yes"| G{"seats reserved in Valkey?"}
  G -->|"no"| E409["409 sold out"]
  G -->|"yes"| PG["Postgres transaction<br/>final authority"]

How it works

1. Token bucket. Each caller has a bucket. Booking: up to 10 tokens, one new token per second. Log-in: up to 20 per IP, one new token every 3 s. No token: 429 with a Retry-After time.
2. Atomic. One Lua script in Valkey reads and takes the token in one step, so all api tasks share one count.
3. The gate. Another Lua script tries to reserve the seats in Valkey. Denied means sold out: 409, without touching Postgres.
4. Postgres decides. A reserved request still runs the locked transaction. If Valkey fails, the gate is skipped and Postgres decides alone; a loop repairs the counter later.

In MiniTickets

The limiter is apps/api/src/services/rate-limits.ts → enforceRateLimit, the gate apps/api/src/services/booking-inventory.ts → createGatedBooking (on by default, INVENTORY_GATE_ENABLED).

The two fail in opposite directions on purpose. The limiter fails closed: without Valkey it answers 503, so expensive password checks never run unbounded. The gate fails open: without Valkey it steps aside, because Postgres can still decide correctly.

Measured locally (128 requests, 32 users, 16 seats): with the gate on, 16 booking transactions reached Postgres instead of 128 (87.5% fewer), throughput rose from 784 to 1107 requests per second, and 0 seats were oversold in both runs.

From → to	What happens	What allows it
API app → Inventory gate	a booking asks the gate before the database transaction	code services/bookings.ts bookTicket → createGatedBookingcreateGatedBooking
Inventory gate → ElastiCache Valkeyonly when dataEnabled = true	seat counters and leases per event	code queries/inventory.ts runs the RESERVE and SETTLE Lua scriptsreserve

Play the sale-day flow on the map →

See it in AWS

Console

The buckets and counters are keys in Valkey. 429s show in the API log; the ALB counts all responses by status.

CLI

aws logs tail /minitickets/dev/api --since 10m --filter-pattern 429

Code

apps/api/src/queries/rate-limits.ts → takeToken

Break it

Symptom	Cause	Where to look
Real users get 429	Many users share one office IP for log-in	429s per client IP
Log-in answers 503 for everyone	Valkey unreachable; the limiter fails closed	redis-sg; Valkey status
Sold out, but Postgres shows seats	The Valkey counter drifted	“inventory reconciliation” log lines

Say it in an interview

Say it like this (20-30 seconds)

Two Valkey checks protect Postgres. Atomic Lua token buckets limit log-in per IP and booking per user, returning 429 with Retry-After, and they fail closed. An inventory gate reserves seats in Valkey and rejects sold-out requests with 409 before any row lock; it fails open, because the locked Postgres transaction remains the final authority. Locally that cut booking transactions by 87.5 percent with zero oversells.

Follow-ups

Why a token bucket and not a fixed window?

A fixed window allows a double burst at the window edge. A bucket allows a short burst, then a steady rate, which matches how real people click.

Why is the client IP tricky behind a load balancer?

Every request arrives from the ALB. The real IP is in X-Forwarded-For. TRUST_PROXY_HOPS says how many proxies to trust; too many, and a client can write its own IP into the header.

What would you add for a really big sale?

A waiting room in front, seat holds that expire, and more api tasks with autoscaling. They are explained in the decisions, not built.

Advanced: Reconciliation

Each api task runs a repair loop every 5 seconds over at most 50 events per batch. It compares the Valkey counter with Postgres and fixes drift, for example after a crash between reserving and settling. The 5 s is a scheduling interval, not a repair promise: a large backlog takes longer.

Checkpoint

Answer these without scrolling up. If one feels hard, reread that section.

1. What are the booking and log-in bucket sizes and refill rates?
2. Why must taking a token be atomic, and how is it?
3. What does the gate save Postgres from?
4. Which check fails closed and which fails open, and why?
5. Who makes the final booking decision, and why?

Redraw it from memory: request → token left? → seats reserved? → Postgres, with the 429 and 409 exits

Try this: on the map, play the sale-day flow and open the failure branch of the inventory gate step

Mark this page as read