How do you stop queue processing from overwhelming downstream services?

Say it in 30 seconds

Say it like this (20-30 seconds)

I limit work where it enters the consumer and size the limit against downstream capacity. MiniTickets caps the email event source mapping at five concurrent batches and tickets at two. These are concurrency caps, not requests-per-second guarantees. I also inspect account concurrency, function reservations and throttles. Raising concurrency can drain a backlog faster, but can worsen SES throttling or database pressure when the downstream system is already saturated.

The deeper answer

Concurrency is the number of invocations running at once. Throughput depends on how much work each invocation completes and how long it runs. Five sequential email batches can still send at a rate that exceeds a provider quota; a mapping cap is not a distributed rate limiter.

The email mapping takes up to ten records; the ticket mapping takes one booking so it can render up to four ticket images. Different batch sizes and work costs explain their different caps. The worker reservation defaults to no explicit reservation unless configured. Never repeat a historical account quota as the account’s current limit.

AWS distinguishes mapping concurrency from function reserved concurrency. If reservations are used, they must accommodate the mappings sharing that function. Otherwise pollers can encounter throttling instead of useful work. I would watch oldest-message age, throttles, duration and downstream rejections together, and load-test before increasing capacity.

Counter-questions

What is the first check when Throttles rises?

Compare actual account usage, current quotas, function reservations and mapping settings. A configured cap alone does not reveal which limit rejected work.

Can more concurrency lower throughput?

Yes. Downstream overload can cause retries and longer durations, consuming capacity without completing more jobs.

Would you put both consumers in one queue?

Independent queues permit separate pacing and retries. A single consumer pool couples the fast and expensive side effects.

Where in MiniTickets

Read the path chapter, select the relevant map piece →, then inspect infra/src/jobs/queue-trigger.ts → createQueueTrigger. Work through the related case for a diagnostic example.