Serverless Architecture Patterns for Event-Driven Apps

The serverless patterns that hold up in production are few: a thin API on API Gateway and Lambda, queues between producers and consumers, EventBridge events for choreography between services, Step Functions for multi-step processes that need retries and compensation, and the transactional outbox for publishing events reliably. What makes them work is not the pattern itself but the failure handling around it: idempotent handlers, dead-letter queues, partial batch failures, concurrency limits that protect downstream systems, and tracing across every hop.

Key takeaways

• Every asynchronous path in AWS delivers at least once. Make every handler idempotent, or you will eventually charge a card twice.
• Put a queue between anything that can burst and anything that cannot. SQS plus a concurrency cap on the consumer protects databases and third-party APIs.
• Use choreography (EventBridge) between services and orchestration (Step Functions) inside a business process. Do not choreograph a saga.
• Every retry path needs an end: a dead-letter queue or failure destination, and an alarm on it.
• Timeouts must shrink as you go deeper into the call chain, or upstream callers give up while downstream work keeps running.
• Serverless is cheapest for spiky load. For steady high throughput, run the numbers against containers.

The limits that shape the design

Serverless designs are shaped by a handful of documented limits. Know them before you draw boxes.

Check current quotas in Service Quotas for your account and region before a launch; several of these are adjustable.

Designing events

An event is an immutable fact about something that happened: OrderPlaced, not PlaceOrder. Give every event the same envelope so consumers, logs and traces can rely on it:

{
  "source": "com.example.orders",
  "detail-type": "OrderPlaced",
  "detail": {
    "event_id": "01J9Z6K6QF3V3T6W0G9X8B7C2D",
    "event_version": 2,
    "occurred_at": "2026-10-04T09:12:44Z",
    "correlation_id": "req-7f3c1a",
    "order_id": "ord_48213",
    "customer_id": "cus_9921",
    "total": 129.5,
    "currency": "EUR",
    "items_ref": "s3://orders-events/ord_48213/items.json"
  }
}

Three rules. Carry enough data that most consumers do not need to call back to the producer, but put large or sensitive payloads in S3 and pass a reference. Version the schema and only add fields within a version; a breaking change is a new version that runs alongside the old one until consumers move. Include an event ID consumers can use for deduplication, and a correlation ID that travels through every hop.

Pattern 1: a thin API

API Gateway (or a Lambda function URL for simple cases) handles routing, authentication, throttling and request validation; Lambda holds the business logic. Group routes by bounded context, one function per resource or small group of related routes, rather than one function per route or one function for everything.

Skip Lambda where it adds nothing. API Gateway can integrate directly with SQS, DynamoDB, EventBridge and Step Functions. A webhook receiver that only needs to accept and enqueue a payload is better as an API Gateway to SQS integration: fewer moving parts, no cold start, and the queue absorbs bursts.

Choose the API type on purpose. REST APIs offer request validation, usage plans and API keys, caching and direct WAF association. HTTP APIs are simpler and cheaper per request with built-in JWT authorizers. If you need none of the REST features, HTTP APIs are the default.

Anything that might take longer than the integration timeout should not run inside the request. Accept it, return 202 Accepted with a job ID, process it asynchronously, and let the client poll or receive a callback.

Pattern 2: queue-based load leveling

When a producer can burst and a consumer cannot (a relational database, a payment provider, a partner API with rate limits), put an SQS queue between them and cap how fast the consumer drains it.

resource "aws_sqs_queue" "orders_dlq" {
  name                      = "orders-dlq"
  message_retention_seconds = 1209600
}
 
resource "aws_sqs_queue" "orders" {
  name                       = "orders"
  visibility_timeout_seconds = 185
 
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.orders_dlq.arn
    maxReceiveCount     = 5
  })
}
 
resource "aws_lambda_event_source_mapping" "orders" {
  event_source_arn                   = aws_sqs_queue.orders.arn
  function_name                      = aws_lambda_function.process_order.arn
  batch_size                         = 10
  maximum_batching_window_in_seconds = 5
  function_response_types            = ["ReportBatchItemFailures"]
 
  scaling_config {
    maximum_concurrency = 20
  }
}

The numbers that matter, and why:

• Visibility timeout is six times the function's 30-second timeout plus the 5-second batching window, which is AWS's guidance for SQS event sources. Shorter, and a message can become visible again while it is still being processed, producing duplicates.
• Maximum concurrency on the event source mapping caps how many function instances this queue can drive. That is your protection for the database behind it. It is better than reserved concurrency for this purpose because it stops pulling messages instead of pulling them and getting throttled.
• Partial batch responses (ReportBatchItemFailures) return only the failed messages to the queue. Without them, one bad message makes the whole batch retry, and the good messages get processed again.
• The dead-letter queue is configured on the source queue, after five receives. Messages that keep failing land there for inspection instead of blocking the queue.

The handler reports failures per message and is idempotent, because SQS delivers at least once:

import json
 
from aws_lambda_powertools import Logger
from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer,
    IdempotencyConfig,
    idempotent_function,
)
 
logger = Logger()
persistence = DynamoDBPersistenceLayer(table_name="idempotency")
config = IdempotencyConfig(event_key_jmespath="order_id", expires_after_seconds=86400)
 
 
@idempotent_function(data_keyword_argument="order", persistence_store=persistence, config=config)
def process_order(order: dict) -> dict:
    charge_payment(order)
    reserve_stock(order)
    return {"order_id": order["order_id"], "status": "processed"}
 
 
def handler(event, context):
    config.register_lambda_context(context)
    failures = []
    for record in event["Records"]:
        try:
            process_order(order=json.loads(record["body"]))
        except Exception:
            logger.exception("order processing failed", extra={"message_id": record["messageId"]})
            failures.append({"itemIdentifier": record["messageId"]})
    return {"batchItemFailures": failures}

The idempotency key is the business identity (order_id), not the SQS message ID, because the same order can arrive in two different messages if the producer retries.

Pattern 3: choreography with EventBridge

In choreography, a service publishes facts to an event bus and other services subscribe through rules. The publisher does not know its consumers, so adding a new reaction (fraud scoring, loyalty points, analytics) means adding a rule, not changing the producer.

Rules filter on content, so consumers receive only what they care about:

{
  "source": ["com.example.orders"],
  "detail-type": ["OrderPlaced"],
  "detail": {
    "currency": ["EUR", "USD"],
    "total": [{ "numeric": [">=", 500] }]
  }
}

Production settings worth having on every bus: an archive so you can replay events after fixing a consumer bug, a dead-letter queue on each target so events that exhaust their retries are kept, and a schema registry or contract tests so producers cannot silently break consumers.

Have consumers put an SQS queue in front of their Lambda function rather than invoking it directly from the rule. The queue gives each consumer its own buffer, its own concurrency cap and its own dead-letter queue, so one slow consumer does not affect the others. For point-to-point flows with filtering and enrichment (a queue feeding a state machine, a stream feeding a bus), EventBridge Pipes removes the glue function.

Choreography has a limit. When understanding one business process requires reading a dozen rules across five repositories, or when the process needs compensation on failure, it has become an orchestration problem.

Pattern 4: orchestration and sagas with Step Functions

A saga is a sequence of local transactions where each step has a compensating action that undoes it if a later step fails. Step Functions makes the sequence, retries and compensation explicit and visible, with the state of every execution inspectable.

{
  "Comment": "Order saga with compensation",
  "StartAt": "ReserveStock",
  "States": {
    "ReserveStock": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": { "FunctionName": "reserve-stock", "Payload.$": "$" },
      "ResultSelector": { "reservation.$": "$.Payload" },
      "ResultPath": "$.stock",
      "Retry": [
        {
          "ErrorEquals": ["Lambda.TooManyRequestsException", "States.Timeout"],
          "IntervalSeconds": 2,
          "MaxAttempts": 3,
          "BackoffRate": 2.0,
          "JitterStrategy": "FULL"
        }
      ],
      "Catch": [{ "ErrorEquals": ["States.ALL"], "ResultPath": "$.error", "Next": "OrderFailed" }],
      "Next": "ChargePayment"
    },
    "ChargePayment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": { "FunctionName": "charge-payment", "Payload.$": "$" },
      "ResultSelector": { "charge.$": "$.Payload" },
      "ResultPath": "$.payment",
      "Retry": [
        {
          "ErrorEquals": ["PaymentProviderUnavailable"],
          "IntervalSeconds": 5,
          "MaxAttempts": 4,
          "BackoffRate": 2.0,
          "JitterStrategy": "FULL"
        }
      ],
      "Catch": [{ "ErrorEquals": ["States.ALL"], "ResultPath": "$.error", "Next": "ReleaseStock" }],
      "Next": "PublishConfirmed"
    },
    "ReleaseStock": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": { "FunctionName": "release-stock", "Payload.$": "$" },
      "Retry": [
        { "ErrorEquals": ["States.ALL"], "IntervalSeconds": 5, "MaxAttempts": 10, "BackoffRate": 2.0 }
      ],
      "Next": "OrderFailed"
    },
    "PublishConfirmed": {
      "Type": "Task",
      "Resource": "arn:aws:states:::events:putEvents",
      "Parameters": {
        "Entries": [
          {
            "EventBusName": "orders",
            "Source": "com.example.orders",
            "DetailType": "OrderConfirmed",
            "Detail.$": "$"
          }
        ]
      },
      "End": true
    },
    "OrderFailed": {
      "Type": "Fail",
      "Error": "OrderFailed",
      "Cause": "Order could not be completed"
    }
  }
}

Details that matter in production. Retries use backoff with jitter so a recovering dependency is not hit by synchronized waves. Compensation steps retry far more than forward steps, because a failed refund or release is worse than a failed charge, and every compensating action must be idempotent. Payment calls pass an idempotency key to the provider so a retried step cannot charge twice.

Pattern 5: fan-out and fan-in

Fan-out to many consumers: publish once to SNS or EventBridge and give each consumer its own SQS queue. Each consumer fails, retries and scales independently.

Fan-out of work items: for large batch jobs, the Step Functions Distributed Map state iterates over data in S3 (a set of objects, or the rows of a CSV file or items of a JSON file) and runs a child workflow per item or batch, with a configurable concurrency limit and a failure tolerance threshold. It replaces hand-built splitter functions and completion tracking, and fan-in is built in: the map state completes when its children do, and it can write their results to S3.

Pattern 6: the transactional outbox

The classic bug in event-driven systems is the dual write: save the order to the database, then publish OrderPlaced. If the function dies between the two, the order exists and nobody hears about it. Retrying the whole function risks the opposite.

The outbox pattern makes the event part of the same transaction as the state change. With DynamoDB, write the entity (or the entity plus an outbox item, with TransactWriteItems) and let DynamoDB Streams carry the change out. An EventBridge Pipe reads the stream, filters to the relevant records, transforms them into events and puts them on the bus, with no custom glue code. With a relational database, write an outbox row in the same transaction and publish from it with change data capture or a polling relay.

Either way, the event is published if and only if the state change committed, and consumers deduplicate on the event ID.

Pattern 7: strangling a legacy system

You can move an existing application to serverless one route at a time. Put API Gateway (or the load balancer) in front of the legacy application, move a well-bounded capability such as a webhook handler or a report endpoint to a function, and route only that path to the new code. Each route can be rolled back on its own. This is the same strangler fig approach we describe for cloud migrations, and it pairs well with breaking out services along the boundaries in our microservices patterns guide.

Observability: what to alarm on

Distributed, short-lived functions cannot be debugged by logging into a server. Instrument from the first function:

• Structured JSON logs with the correlation ID, request ID and business identifiers in every line. Powertools Logger does this with one decorator.
• Tracing with AWS X-Ray or OpenTelemetry across API Gateway, Lambda, SQS, EventBridge and Step Functions, so one request is one trace.
• Business metrics through CloudWatch Embedded Metric Format, written as log lines instead of API calls.

Alarm on rates and on the signals that mean work is stuck, not on individual errors:

Our guide to observability in production covers how to turn these into service level objectives.

Cost and performance

Lambda allocates CPU in proportion to memory, so a function with more memory often finishes faster and can cost the same or less. Measure it with the open-source Lambda Power Tuning tool rather than guessing. Arm64 (Graviton) functions are usually cheaper per unit of compute for code that runs on arm64.

Cold starts come from the initialization phase: downloading code, starting the runtime and running your init code. Keep packages small, create SDK clients once outside the handler, and lazy-load what only some paths need. Since August 2025, AWS bills the initialization phase for all on-demand functions, so heavy init code costs money as well as latency. SnapStart (for Java, Python and .NET) and provisioned concurrency remove cold starts on paths that cannot tolerate them.

Serverless is cheapest for spiky, low or unpredictable load. For steady, high-throughput services that keep functions busy all day, per-request pricing can cost more than containers running at high utilization. Our comparison of Kubernetes and serverless walks through that decision.

What goes wrong in production

Duplicate side effects. At-least-once delivery meets a non-idempotent handler, and a customer is charged twice or emailed five times. Idempotency is not optional in event-driven systems.

Poison messages and retry storms. A message that always fails is retried until it blocks the batch or burns concurrency. Partial batch responses, a dead-letter queue with a bounded receive count, and an alarm on that queue fix it.

A dead-letter queue nobody watches. Failed orders accumulate silently for weeks. Alarm on any message in a DLQ and build a redrive procedure; SQS supports redriving messages from a DLQ back to the source queue once the bug is fixed.

Lambda overwhelms the database. A burst scales the function to hundreds of concurrent instances, each opening database connections. Cap concurrency at the event source, use RDS Proxy for relational databases, or move hot paths to DynamoDB.

Visibility timeout shorter than the work. Messages reappear while still being processed and are handled twice. Follow the six-times rule and keep function timeouts realistic.

Timeouts that do not nest. API Gateway gives up while the function keeps running and the client retries, starting the work again. Every downstream timeout should be shorter than its caller's.

Recursive loops. A function triggered by S3 writes its output to the same bucket, or a consumer republishes to its own queue. Lambda detects and stops some recursive loops, but design them out with separate buckets or prefixes and loop-safe event filters.

One function starves the account. All functions in a region share the concurrency quota. Reserve concurrency for critical functions, and cap noisy ones.

Frequently Asked Questions

What are the most common serverless architecture patterns?

The core set is an API on API Gateway and Lambda, queue-based load leveling with SQS, event choreography with EventBridge, orchestration with Step Functions, fan-out and fan-in, and the transactional outbox for reliable event publishing. Most production systems combine several of them. The failure handling around each pattern matters more than the choice of pattern.

When should I use Step Functions instead of EventBridge?

Use Step Functions when a business process has a defined order, needs retries, timeouts or compensating actions, and someone must be able to see where each instance is. Use EventBridge when independent services react to facts and the publisher should not know who is listening. Many systems use both: events between services, a state machine inside one.

How do you handle failures in an SQS-triggered Lambda function?

Enable partial batch responses so only the failed messages return to the queue, and configure a dead-letter queue on the source queue with a sensible maximum receive count. Set the queue's visibility timeout to at least six times the function timeout. Alarm on the dead-letter queue depth and on the age of the oldest message.

How do you make Lambda functions idempotent?

Derive a key from the business identity of the request, such as an order ID, and record it with the result in a store like DynamoDB before returning. When a duplicate arrives, return the stored result instead of repeating the side effect. Powertools for AWS Lambda provides an idempotency utility that implements this pattern.

When is serverless more expensive than containers?

Serverless tends to cost more for steady, high-throughput workloads that keep functions busy around the clock, where you pay per request and per millisecond for capacity a container would use more efficiently. It is usually cheaper for spiky, low or unpredictable traffic. Compare using your own request volume and duration, and include the operations work each option needs.

How do you reduce Lambda cold starts?

Keep deployment packages and initialization code small, initialize SDK clients once outside the handler, and avoid heavy frameworks in latency-sensitive functions. For Java, Python and .NET, SnapStart restores from a snapshot of an initialized environment. Provisioned concurrency keeps environments initialized for paths that cannot tolerate any cold start.

Need help building event-driven systems?

We build serverless systems with the failure handling in from the first commit: idempotent handlers, bounded retries and alarms on every place work can get stuck. If we build it, we are on the pager for it, so we design for the duplicate message and the retry storm at three in the morning, not just the happy path.

• Software development
• Cloud architecture and infrastructure
• Talk to an engineer