You start with one Lambda function that processes uploads to an S3 bucket. Then someone asks for a thumbnail, a database record and a Slack alert for every upload, and that one function slowly turns into a 400-line monster. This post walks you through six serverless design patterns for AWS Lambda that keep things clean, and when I’d reach for each one.
Quick definitions first. AWS Lambda is a serverless service that runs your code in response to events, without you provisioning or managing servers. A design pattern is a proven, reusable answer to a common problem, so you don’t have to reinvent the structure every time.
Process a single event
Simple event processing is the most basic serverless pattern: one event triggers one Lambda function, which does one job. The trigger could be an object uploaded to Amazon S3 or a message published to Amazon SNS, and the function might process the uploaded file or forward the message to another service.
I’d suggest starting here for almost everything. If a single function does a single job well, you don’t need anything fancier yet.
Fan out one event to many functions
The fan-out pattern sends one event to several Lambda functions that run in parallel. Use it when you have independent tasks, like sending emails, updating a database and generating reports, that don’t need to wait on each other.
A common setup is one SNS topic with an SQS queue per task, and a Lambda function reading from each queue. To subscribe a queue to a topic, run:
aws sns subscribe \
--topic-arn arn:aws:sns:us-east-1:123456789012:TopicName \
--protocol sqs \
--notification-endpoint arn:aws:sqs:us-east-1:123456789012:QueueName
Repeat it for each queue, then check that every queue’s access policy allows the topic to send messages to it. If a task fails, only its own queue retries, and the others keep going.
Fan in results from parallel tasks
The fan-in pattern collects the results of several independent Lambda functions and combines them in one place. It’s the natural partner of fan-out: the parallel functions push their results to Amazon SNS, Amazon SQS or another AWS service, and a final Lambda function aggregates and processes them.
The tricky part of fan-in is knowing when every task has finished, so plan how you’ll track that before you build it.
Run long tasks with asynchronous workflows
The asynchronous workflow pattern runs long or multi-step tasks in the background while your app keeps responding to new events. AWS Step Functions is the main tool here: it lets you build, visualize and run workflows that coordinate several Lambda functions step by step.
You can also use Amazon SNS or Amazon SQS to queue up Lambda executions and track progress. I’d pick Step Functions as soon as a workflow has retries, branches or more than two or three steps, because chaining functions by hand gets hard to debug fast.
Keep a full history with event sourcing
Event sourcing is a pattern where you store every event that changed your application’s state, instead of only the latest state. On AWS, you can write those events to an Amazon DynamoDB table and use Lambda functions to process them and update the current state of your app.
The payoff is a complete, re-playable history, which is handy for audits and for rebuilding data after a bug.
Split reads and writes with CQRS
Command Query Responsibility Segregation (CQRS) is a pattern that separates the code that writes data from the code that reads it. With Lambda, you can send commands (requests that change data) to one set of functions and queries (requests that read data) to another.
CQRS lets you tune reads and writes separately and keeps each part of your app focused on one concern. It also adds moving parts, so I’d save it for apps where reads and writes really behave differently.
How to pick the right serverless pattern
The right serverless pattern depends on what your application needs to do with each event. Here’s a quick way to choose:
| Pattern | Use it when | Main AWS services |
|---|---|---|
| Simple event processing | One event needs one action | Lambda, S3, SNS |
| Fan-out | One event triggers several independent tasks | Lambda, SNS, SQS |
| Fan-in | You need to combine results from parallel tasks | Lambda, SNS, SQS |
| Asynchronous workflows | Tasks are long or have several steps | Lambda, Step Functions, SQS |
| Event sourcing | You need a full history of changes | Lambda, DynamoDB |
| CQRS | Reads and writes have very different needs | Lambda, DynamoDB |
Every pattern has trade-offs, and you’ll often mix two or three in one app. Start simple, and add a pattern only when the problem it solves actually shows up.
If you’re comparing serverless platforms, my post on serverless containers with Google Cloud Run is a good next read. And if this kind of work is your thing, here’s why I think DevOps is one of the best careers in tech.
Hope this saves you from the 400-line Lambda. 🚀
Leave a Reply