Objects are uploaded into Amazon Simple Storage Service (S3), a highly available and persistent data store. The S3 bucket published an event notification to an Amazon SNS topic.
Amazon SNS is a fully managed push messaging system that can fan-out messages to multiple subscribers.
AWS Lambda is used to create a layer of processing for a wide variety of data sets and subsequently sends the results to a post-processing storage layer. For example, Lambda Func. 1 can process objects put into the S3 bucket and creates a data derivative.
Lambda Func. 2 can, in parallel, process and create another data derivative from the same objects in S3.
N number of Lambda functions can be created to process data, all without the need to provision or manage servers.
A post-processing storage layer such as S3 can be added to store the results in a cost-effective and durable fashion.
Alternatively, results can be sent to Amazon DynamoDB for further processing and/or querying with low-latency.
Design Better and Scale Forever with Serverless Architectures!