Amazon Web Services provides multiple messaging options that you can use to create scalable, distributed systems, implement event sourcing to unlock hidden context and utilise CQRS for efficient data access. In this session we will look at various messaging patterns and discuss techniques and use cases for Amazon SQS, Amazon SNS, Amazon Kinesis, Amazon DynamoDB and Amazon Web Services IoT in your application.
Speaker: Stephen Liedig, Solutions Architect, Amazon Web Services
2. “Loosely coupled systems”
The looser they are coupled,
the bigger they will scale,
the more fault tolerant they will be,
the less dependencies they will have,
the faster you will innovate.
4. Amazon SQS
• Message queue service that store
messages waiting to be processed
(60 sec -14 days)
• Separate parts of an application
• Perform tasks asynchronously
• Batch and burst processing
5. Amazon SQS
Customer Order
Queue
Order
message
Amazon
CloudWatch
Scale independently
• Queue length
• ApproximateAgeOfOldestMessage (New!)
Priority Order
Queue
Client
Extend your business requirements
without impacting your application logic
Dead letter
queue
Priority Order
message
Amazon
RDS
Amazon
EC2
14. • Need to provision topic
• Only publishes to subscribed
endpoints
• Multiple Protocols / Mobile
• 64-256 KB
Pub / Sub Messaging Options Compared
• Ephemeral topics, no
provisioning necessary
• Wildcard subscription
• MQTT over Websockets
• NodeJs, iOS, Android SDK’s
for non-IoT use cases
• 128 KB
15. Pub / Sub Messaging Options Compared
Wildcard Description
# Matching the current tree and all subtrees.
Subscribe: Sensor/#
Publish: Sensor/
Sensor/temp
Sensor/temp/room1
+ Matches exactly one item in the topic hierarchy.
Subscribe: Sensor/+/room1
Publish: Sensor/temp/room1
Sensor/moisture/room1
16. • Need to provision topic
• Only publishes to subscribed
endpoints
• Multiple Protocols / Mobile
• 64-256 KB
Pub / Sub Messaging Options Compared
• Ephemeral topics, no
provisioning necessary
• Wildcard subscription
• MQTT over Websockets
• NodeJS, iOS, Android SDK’s
for non-IoT use cases
• 128 KB
18. CQRS
• Based on CQS (Command Query Separation) pattern
[Myer]
• CQRS [Young, Dahan] extends CQS pattern by providing
physical separation of Command and Queries.
• CQRS is a pattern where you use one model to update
information and a separate model to read information
19. CQRS
• Benefits
• Distribute and scale reads and writes
• Take advantage of eventual consistency
• Flexibility in read and write models (polyglot persistence)
• Not a general purpose architecture, will add complexity!
• Specific bounded context
• Designed for multi-user collaborative applications.
20. CQRS
Amazon
RDS
(3NF DB)
Amazon
DynamoDB
(1NF View Model)
Client
Commands Queries
Domain Model
Command Handlers
ORM < Event >
Separate read and write models
(Eventually Consistent)
View models:
• one table = one page view
• Select * from table
• Does not require layered
architecture and multiple data
transformations.
Commands:
• Asynchronous
• Designed not to fail
De-normaliser
AWS
Lambda
Amazon
SQS
Thin Read Layer
< DTO >
22. Event Sourcing
• Rebuild domain state by re-playing event sequence.
• Allows for projections and what-if scenarios.
• CQRS pattern lays the foundations for an event sourced application.
• Capture all changes to an application domain state as a sequence of
events.
• As events come in, they are appended to an immutable, append-only
log and can trigger automation workflows, much like a accounting
ledger.