Lambda times out

Symptom

Troubleshooting scenario: the email worker ends with “Task timed out after 10.00 seconds”. Tickets have a separate 30-second limit. Messages return, sometimes after earlier emails were sent.

Read the error first

Lambda stopped an invocation that exceeded its time budget. This is an execution failure, not an IAM “no identity-based policy allows” or “explicit deny” response. For Reach / Allowed / Password, inspect the last dependency call: no response can mean Reach; AccessDenied means Allowed. This worker has no database password and runs outside the VPC.

Decide

flowchart TB
  A["Lambda timeout"] --> B{"Where was time spent?"}
  B -->|"dependency call"| C["Check latency, retries and reachability"]
  B -->|"many messages"| D["Compare batch work with budget"]
  B -->|"initialization"| E["Compare cold and warm runs"]
  C --> F["Bound work, then align queue visibility"]
  D --> F
  E --> F

Likely causes

Ranked for the email worker’s sequential network work; logs must decide.

1. A slow DynamoDB or SES call, including SDK retries, consumes the remaining time.
2. Up to ten messages, followed by failed-message visibility updates, exceed the shared budget.
3. Initialization or CPU work dominates; compare the tickets worker’s QR rendering separately.

How to check

Console

Lambda → minitickets-dev-worker → Monitor: compare Duration and Errors. In /minitickets/dev/worker, match awsRequestId to REPORT and the last message log. Check the trigger’s batch size and the source queue’s visibility timeout.

CLI

This prints selected settings, not environment variables:

export AWS_PROFILE=ticketlite
aws lambda get-function-configuration --function-name minitickets-dev-worker --query '{Timeout:Timeout,Memory:MemorySize,Vpc:VpcConfig.VpcId}'
aws lambda list-event-source-mappings --function-name minitickets-dev-worker

CloudTrail

Search management events UpdateFunctionConfiguration and UpdateEventSourceMapping near the first failure; inspect the changed timeout, memory or batch size. CloudTrail does not explain time spent inside an invocation; use its logs for that.

Fix

Bound the slow work or reduce batch size. If the timeout must grow, change queue visibility with it: email uses 10/60 seconds; tickets 30/180. A timeout fails the entire invocation; a partial failure list cannot help when the handler never returns.

apps/worker/src/lambda.ts → handler; infra/src/jobs/queues.ts → createJobQueues.

Prevent

Keep the worker-errors alarm, compare duration to its budget, and test dependency failures and partial batches. Preserve duplicate protection because completed side effects can be retried.

Inspect the worker’s limits →

Say it in 30 seconds

Say it like this (20-30 seconds)

A Lambda timeout means the invocation ran out of time, so I find the last dependency call and compare cold and warm durations before changing limits. Our email worker processes up to ten messages sequentially within ten seconds. A timeout fails the whole batch, even with partial failure reporting. I bound slow calls or reduce batch work, align queue visibility with the timeout, and keep duplicate protection for emails already sent.

Advanced: Timeouts, retries and visibility

The email SDK clients configure 1-second connection and 3-second request timeouts. Those are not a total batch deadline: multiple calls and SDK retries can accumulate. The tickets client also sets maxAttempts: 1 and throwOnRequestTimeout: true. Email backoff may already have changed individual visibility delays before an invocation times out; do not assume every message returns exactly 60 seconds later. AWS recommends visibility at least six times the function timeout, plus any batch window and documents whole-invocation versus partial failures. apps/worker/src/ticket-lambda.ts → handler; infra/src/jobs/worker-lambda.ts → WORKER_TIMEOUT_SECONDS.