SNS delivers nothing

Symptom

Troubleshooting scenario: the relay logs “outbox message accepted by SNS”, yet no email arrives. The jobs queue stays empty, or only the tickets consumer works. A successful publish is not a delivery receipt.

Read the error first

SNS accepted the publish. Under Reach / Allowed / Password, the relay reached SNS and its publish was allowed. Delivery into SQS is another Allowed check: the queue policy must permit sns.amazonaws.com. A missing queue allow differs from an explicit deny. The Lambda role’s receive permission is separate; none of these paths uses a password.

Decide

flowchart TB
  A["SNS accepted, no email"] --> B{"Queue has messages?"}
  B -->|"yes"| C["Check consumer, schema and DLQ"]
  B -->|"no"| D{"Correct subscription exists?"}
  D -->|"no"| E["Restore route to queue ARN"]
  D -->|"yes"| F["Check queue policy, filters and delivery metrics"]

Likely causes

Ranked for an empty destination queue after accepted publication; confirm metrics over time.

1. The queue policy lost its SNS allow, or its topic/account condition names the wrong source.
2. The SQS subscription is missing or names another queue ARN, account or Region.
3. A newly added filter discards events. If messages reached the queue, investigate consumption instead: disabling raw delivery wraps the JSON and makes our schema reject it.

How to check

Console

SNS → Topics → minitickets-dev-booking-events → Subscriptions: expect jobs and tickets queues. CloudWatch → Metrics → All metrics → SNS → Topic Metrics: compare published, delivered, failed and filtered counts for this topic. SQS → Queues → each queue → Access policy: compare principal, action and source conditions. An empty queue may also mean fast consumption.

CLI

Set TOPIC_ARN and JOBS_QUEUE_URL from the console. Set JOBS_SUBSCRIPTION_ARN to the jobs subscription returned by the first command. These addresses are not credentials.

export AWS_PROFILE=ticketlite
aws sns list-subscriptions-by-topic --topic-arn "$TOPIC_ARN"
aws sns get-subscription-attributes --subscription-arn "$JOBS_SUBSCRIPTION_ARN" --query 'Attributes.{Raw:RawMessageDelivery,Filter:FilterPolicy,Scope:FilterPolicyScope}'
aws sqs get-queue-attributes --queue-url "$JOBS_QUEUE_URL" --attribute-names Policy ApproximateNumberOfMessages

CloudTrail

Search Subscribe, Unsubscribe, SetSubscriptionAttributes and SetQueueAttributes for route changes. Publish is an SNS data event and needs explicit data-event logging; ordinary Event history does not prove its absence. CloudTrail is not a consumer delivery log.

Fix

Restore the two SQS subscriptions and queue policies in createBookingEvents: SNS principal, sqs:SendMessage, one queue ARN, matching aws:SourceArn and aws:SourceAccount. Keep raw delivery on. SNS accepts messages with zero subscribers; events already published then are not automatically replayed to a later subscription.

Prevent

Keep the publisher dependent on both subscriptions. Check both consumers independently; retain the outbox and define a reviewed recovery for missed events, preserving consumer idempotency.

infra/src/jobs/booking-events.ts → createBookingEvents; Inspect the fan-out routes →.

Say it in 30 seconds

Say it like this (20-30 seconds)

SNS accepting a publish proves the producer reached the topic and could publish. It does not prove the queue received anything. I compare delivery metrics, verify the subscription’s queue ARN, then check that queue’s resource policy allows SNS from our topic and account. If messages arrived, I switch to the consumer and schema. Our publisher waits for both subscriptions, and raw delivery preserves the worker’s JSON contract.

Advanced: Publication and delivery have separate evidence

Current subscriptions have no filter policy. Adding one changes which events each consumer receives and must be treated as a contract change. Raw delivery off produces valid JSON with the wrong outer shape, rather than necessarily a JSON parse error. Consumer DLQs catch jobs that reached SQS and failed processing; they cannot catch an event SNS never delivered there. AWS describes SNS CloudTrail management and data events. SNS-to-SQS delivery needs a subscription and queue policy.