Presentation for ITCamp 2013. www.itcamp.ro
Talking about different patterns that can be used when we need to deal with messages. Sample code is based on Windows Azure Service Bus
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Messaging patterns in the cloud
1. itcampro@ itcamp13# Premium conference on Microsoft technologies
Messaging Patterns in the
Cloud
Radu Vunvulea
iQuest Group
@RaduVunvulea
http://vunvulearadu.blogspot.com
2. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudHuge thanks to our sponsors!
3. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• World of Messages
• On-Premise and Cloud solutions
• Messaging Patterns
• Costs, Benefits and Limitations
Agenda
9. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudWorld of Messages
Message
Message
Message
10. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
11. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Interaction and communication between
software applications
12. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Topic
Subscriptions
13. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Topic
Subscriptions
14. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Topic
Subscriptions
15. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
17. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Pro
– Open Source
– Flexible
– Good price
– Transactional queue read/write
– Flexible load distribution
• Cons
– No queue management
– Document not up to date
– Only one queue technology supported (MSMQ)
NService Bus
18. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Pro
– Free for commercial use
– Supports MSMQ and RabittMQ
– Open Source
– Good administration console
• Cons
– No commercial support
– Not as fast as NServiceBus
MassTransit
19. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Pro
– Scalable
– Lots of features
– Good administration console
– Support
– Cross platform and multiple environments
• Cons
– Is not free
– Latency
Azure Service Bus
25. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Splitter
Topic
Subscriptions
26. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Group messages based on session
Message Splitter
MessageSession msgSession =
msgReceiver.AcceptMessageSession();
BrokeredMessage message;
while ((message = msgSession.Receive()) != null)
{
// Process message
}
msgSession.Complete()
BrokeredMessage message = new BrokereMessage(content);
message.SessionId = "sessionId";
• Consume messages based on session id
27. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Group messages based on session
Message Splitter
MessageSession msgSession =
msgReceiver.AcceptMessageSession();
BrokeredMessage message;
while ((message = msgSession.Receive()) != null)
{
// Process message
}
msgSession.Complete()
BrokeredMessage message = new BrokereMessage(content);
message.SessionId = "sessionId";
• Consume messages based on session id
29. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Aggregator
Topic
Subscriptions
30. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Group messages based on session
Message Aggregator
BrokeredMessage message = new BrokereMessage(content);
message.SessionId = "sessionId";
• Use correlation id and filters
BrokereMessage message = new BrokereMessage(content);
message.CorrelationId = “CJ"
topic.Send(message);
namespaceManager.CreateSubscription(
“itcamp”,
“iquestsubscriptioncar”,
new CorrelationFilterExpression(“CJ"));
33. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer
Topic
Subscriptions
1
2
3
4
1
2
3
4
1234
34. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer
Topic
Subscriptions
1
2
3
4
1
2
3
4
3412
35. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Session, message index and message count
Message Resequencer
• Mark message as dead letter when in
incorrect order
• Iterate dead letter queue to access a
message with a lower index
BrokeredMessage message = new BrokereMessage(content);
message.Properties["index"] = 2;
message.Properties["count"] = 10
message.SessionId = “CJ-15-IQU";
37. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Recipient
Topic
Subscriptions
38. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Use a property to specific the list of groups
Message Recipient
• Define custom filters using LIKE operator
BrokeredMessage message = new BrokeredMessage(content);
message.Properties[“groups”] = “orange magenta”;
SqlFilter filter = new SqlFilter(
“groups LIKE „%orange%‟”);
topic.AddSubscription(“subscription3”, filter);
• OR different properties for each group type
SqlFilter filter = new SqlFilter(
“EXISTS orange”);
topic.AddSubscription(“subscription3”, filter);
39. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Use a property to specific the list of groups
Message Recipient
• Define custom filters using LIKE operator
BrokeredMessage message = new BrokeredMessage(content);
message.Properties[“groups”] = “orange magenta”;
SqlFilter filter = new SqlFilter(
“groups LIKE „%orange%‟”);
topic.AddSubscription(“subscription3”, filter);
• OR different properties for each group type
• Use only one property and a prime number
for each group
40. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudContent-Based Router
Topic
Subscriptions
41. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudContent-Based Router
Topic
Subscriptions
42. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudScatter-Gather
Topic
Subscriptions
Queue
43. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudScatter-Gather
Topic
Subscriptions
Queue
44. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudDynamic Router
Topic
Subscriptions
45. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudDynamic Router
Topic
Subscriptions
46. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudDynamic Router
Topic
Subscriptions
47. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• [key, values] properties
• Store properties
• Decorate properties
Dynamic Router
49. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• What are the costs of processing:
– 24kB message size
– 10M messages
– 1 Topic
- 8 hours
Costs
50. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
Costs
51. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
• 10 $ Receiving
• 0 $ Bandwidth (receiving)
Costs
52. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
• 10 $ Receiving
• 0 $ Bandwidth (receiving)
• 47.46$ Costs related to Service Bus
Costs
53. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
• 10 $ Receiving
• 0 $ Bandwidth (receiving)
• 47.46$ Costs related to Service Bus
• 8.64$ 4 Medium Worker Roles used to
consume messages
• 56.1$ Total cost
Costs
54. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Cheap
• 99.9% Uptime
• Extremely scalable and flexible
• No message is lost
• Filters and actions support
• REST API
• Death-letter and transaction support
• Same API for on-premise and cloud
Benefits
55. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Not for real-time application
• When processing more than 1M messages
on the same topic in a 30 minutes time
interval latency increases
• You need to pay for each send/receive
command
• Batch maximum size – 100 messages
Limitations