5. Messaging Use Cases
Things a Message Broker Can Do
● Work distribution
● Pipelining
● Store-and-forward of messages
● Absorbing spikes of activity (again, decoupling)
● Routing to different consumers based on message properties
6. Messaging Use Cases
Patterns:
● spatial decoupling (aka communication)
● temporal decoupling (aka buffering)
● logical decoupling (aka routing)
Promote: flexibility, resilience, performance, and scale
Conclusion: There is an enormous range of applications and
problems to which messaging is an effective solution.
8. Why AMQP?
AMQP is Especially Nice Because
● All resources are dynamically created and destroyed by clients as
they need them – no static pre-configuration
● A clean and simple model: just three key nouns to learn
● Open standard, developed by the AMQP Working Group (VMware is
a member)
● Lots of client libraries available in many languages, for free
● An excellent, freely available, open source broker implementation,
called RabbitMQ.
9. AMQP 101
Advanced != Complicated
Create an exchange...
X
“my_exchange”
type = fanout
10. AMQP 101
Advanced != Complicated
...create a queue...
X
“my_exchange” “my_queue”
type = fanout
11. AMQP 101
Advanced != Complicated
...add a binding...
X
“my_exchange” “my_queue”
type = fanout
12. AMQP 101
Advanced != Complicated
...all inside a broker...
X
“my_exchange” “my_queue”
type = fanout
13. AMQP 101
Advanced != Complicated
...publish a message...
X
“my_exchange” “my_queue”
type = fanout
14. AMQP 101
Advanced != Complicated
...message sits in queue.
X
“my_exchange” “my_queue”
type = fanout
15. AMQP 101
Advanced != Complicated
Publish another message...
X
“my_exchange” “my_queue”
type = fanout
16. AMQP 101
Advanced != Complicated
...it also sits in queue.
X
“my_exchange” “my_queue”
type = fanout
17. AMQP 101
Advanced != Complicated
Consumer retrieves messages in order...
X
“my_exchange” “my_queue”
type = fanout
18. AMQP 101
Advanced != Complicated
Consumer retrieves messages in order...
X
“my_exchange” “my_queue”
type = fanout
19. AMQP 101
Advanced != Complicated
With no binding, the message is discarded
(producer can ask to be notified)
X
“my_exchange” “my_queue”
type = fanout
20. AMQP 101
Advanced != Complicated
Publish several messages...
E
D
C
B
A
X
“my_exchange” “my_queue”
type = fanout
21. AMQP 101
Advanced != Complicated
and they are distributed amongst consumers
on the same queue...
D A
X E B
C
“my_exchange” “my_queue”
type = fanout
22. AMQP 101
Advanced != Complicated
Let's create another queue and bind it to
the same exchange...
X
“my_queue”
“my_exchange”
type = fanout “your_queue”
23. AMQP 101
Advanced != Complicated
..with a fanout exchange...
B
A
X
“my_queue”
“my_exchange”
type = fanout “your_queue”
24. AMQP 101
Advanced != Complicated
..all messages go to every queue...
X B A
“my_queue”
“my_exchange” B A
type = fanout “your_queue”
25. AMQP 101
Advanced != Complicated
...and can be consumed...
A
X B
“my_queue”
“my_exchange” B A
type = fanout “your_queue”
26. AMQP 101
Advanced != Complicated
...at different rates...
A
X B
“my_queue”
“my_exchange”
type = fanout “your_queue” B A
27. AMQP 101
Advanced != Complicated
...depending on the consumer, network and broker
B A
X
“my_queue”
“my_exchange”
type = fanout “your_queue” B A
28. AMQP 101
Advanced != Complicated
Let's change the exchange to be a direct type...
X
“my_queue”
“my_exchange” “your_queue”
type = direct
29. AMQP 101
Advanced != Complicated
...then the bindings have to have a key...
bk = “vodka”
X
bk “my_queue”
= “be
er”
“my_exchange” “your_queue”
type = direct
bk = Binding Key
30. AMQP 101
Advanced != Complicated
...and the messages have to have a routing key...
rk = “vodka”
bk = “vodka”
X
bk “my_queue”
= “be
er”
“my_exchange” “your_queue”
type = direct
rk = Routing Key
bk = Binding Key
31. AMQP 101
Advanced != Complicated
...a direct exchange matches the
routing key to a binding key...
rk = “beer”
bk = “vodka”
X
bk “my_queue”
= “be
er”
“my_exchange” “your_queue”
type = direct
rk = Routing Key
bk = Binding Key
32. AMQP 101
Advanced != Complicated
...a direct exchange matches the
routing key to a binding key...
bk = “vodka”
X
bk “my_queue”
= “be
er”
“my_exchange” “your_queue”
type = direct
rk = Routing Key
bk = Binding Key
33. AMQP 101
Advanced != Complicated
...messages with no matching binding...
rk = “tequila”
bk = “vodka”
X
bk “my_queue”
= “be
er”
“my_exchange” “your_queue”
type = direct
rk = Routing Key
bk = Binding Key
34. AMQP 101
Advanced != Complicated
...are discarded
bk = “vodka”
X
bk “my_queue”
= “be
er”
“my_exchange” “your_queue”
type = direct
rk = Routing Key
bk = Binding Key
35. AMQP 101
Advanced != Complicated
Let's change the exchange to be a topic type...
X
“my_queue”
“my_exchange” “your_queue”
type = topic
36. AMQP 101
Advanced != Complicated
...permits wildcard in bindings...
bk = “a.*.c”
X
bk “my_queue”
= “#.
e”
“my_exchange” “your_queue”
type = topic
bk = Binding Key
37. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
rk = “a” A bk = “a.*.c”
rk = “a.b” X
B bk
= “my_queue”
rk = “a.b.c” C “#.
e”
rk = “a.e” D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
38. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
rk = “a” X
A bk = “a.*.c”
rk = “a.b” X
B bk
= “my_queue”
rk = “a.b.c” C “#.
e”
rk = “a.e” D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
39. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
bk = “a.*.c”
rk = “a.b” X
B bk
= “my_queue”
rk = “a.b.c” C “#.
e”
rk = “a.e” D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
40. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
bk = “a.*.c”
X
rk = “a.b” X
B bk “my_queue”
rk = “a.b.c” = “#.
C e”
rk = “a.e” D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
41. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
bk = “a.*.c”
X
bk “my_queue”
rk = “a.b.c” = “#.
C e”
rk = “a.e” D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
42. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
bk = “a.*.c”
X C
bk “my_queue”
= “#.
e”
rk = “a.e” D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
43. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
bk = “a.*.c”
X C
bk “my_queue”
= “#.
e”
D
“my_exchange” “your_queue”
rk = “a.b.e” E
type = topic
rk = Routing Key
bk = Binding Key
44. AMQP 101
Advanced != Complicated
Binding keys are period(.)-separated:
* matches 1 element
# matches 0 or more elements
bk = “a.*.c”
X C
bk “my_queue”
= “#.
e”
E D
“my_exchange” “your_queue”
type = topic
rk = Routing Key
bk = Binding Key
45. Other Stuff the AMQP Broker Does
Yes, all the standard things
● Errors can be raised for messages that do not get routed to any
queues
● Messages can be consumed so that the broker does not forget about
the message until the client explicitly acknowledges the message
● Messages can be published with a property indicating whether the
message should be written to disk
● Transactions: making publication and acknowledgement of several
messages atomic
● Flow control: e.g. used to stop publishers from overwhelming the
broker in extreme situations
46. Conclusion: Why AMQP?
An Open Protocol has some advantages...
● Makes it easier to have multiple implementations that interoperate at
the wire-level
● Avoids vendor lock-in: easy to rip out and replace individual
components with alternative implementations
● Allows third-parties to write client libraries for other languages
● Decouples flag-day upgrades for both client libraries and broker
● Allows third-party traffic analysis tools to inspect and decode
interactions between the clients and brokers
● Promotes similarities in APIs presented by different client libraries
47. Introducing RabbitMQ
RabbitMQ, AMQP and Messaging
● RabbitMQ consists of the broker (server) and several clients
● (Java, .Net, plus others)
● They speak the AMQP protocol to each other
● Anyone can write (and many have) other clients which also speak
AMQP and work with RabbitMQ
● RabbitMQ is freely available and open source, licensed under the
Mozilla Public License v1.1
● It’s already in many popular Linux distributions (Ubuntu, Debian,
Fedora, Gentoo) and can be easily installed on OS X both through
MacPorts and Homebrew
48. AMQP and JMS
How does AMQP compare to JMS?
When would I use one or the other?
49. AMQP and JMS
AMQP JMS
Protocol API
No language Java
Industry committee, mainly finance JEE
Interoperability from client libraries Interoperability from proprietary
features
Resources can be managed by clients Resources often centrally managed
Not ubiquitous in enterprise Ubiquitous through JEE
Flexible usage patterns, emphasise Two basic producer patterns (Queue
routing and Topic)
Designed for large numbers of Hard to manage large numbers of
queues and exchanges Destinations
50. AMQP and JMS
JMS
• standard in the Java space
• lots of open source options
• provided with Java EE application servers
• Spring JMS
Java producers Message broker Java consumers
C
P queue
C
P queue
C
51. JMS
Sending JMS Messages
• Inject an instance of Spring's JmsTemplate.
• Provide the JMS ConnectionFactory in the JmsTemplate bean definition.
@Component
public class MessageSender {
@Autowired
private volatile JmsTemplate jmsTemplate;
public void send(String message) {
this.jmsTemplate.convertAndSend("example.queue", message);
}
@Bean public ConnnectionFactory cf (){
} return new CachingConnectionFactory(...);
}
@Bean public JmsTemplate template(){
return new JmsTemplate(this.cf()) ;
}
52. AMQP and JMS
●
AMQP
• real standard
• a product of the the companies with the most mission critical requirements
• Spring AMQP
AMQP producers exchanges Message broker AMQP consumers
C
P X queue
C
P X queue
C
53. Spring AMQP
● Java and .NET
● AMQP core abstraction plus RabbitMQ implementation (built on
rabbit client libraries)
● Higher level patterns for clients: admin, producers and consumers
● ConnectionFactory – extra abstraction on top of Rabbit client (e.g. for re-
connection)
● RabbitAdmin – declare exchanges, queues, bindings
● RabbitTemplate – convenience methods for send and receive
● MessageListenerContainer – POJO message handler, asynchronous
● Spring Integration support
http://www.springsource.org/spring-amqp
54. Spring AMQP: RabbitTemplate
public class MyComponent {
private RabbitTemplate rabbitTemplate;
public MyComponent(ConnectionFactory connectionFactory) {
this.rabbitTemplate = new RabbitTemplate(connectionFactory);
}
public void read() throws Exception {
...
String value = rabbitTemplate.receiveAndConvert("myQueueName");
...
}
}
Convenience methods