Introduction to RabbitMQ, Amqp and some messaging patterns. Easy to follow code examples step by step provided. Sample code and escenarios can be found at: https://gist.github.com/javierarilos/9348168

1. intro and messaging patterns
javier arias losada @javier_arilos
amqp and rabbitmq

2. introduction to messaging
messaging
● provides asynchronous communications
● loosely coupled modules / processes
● MoM => Message Oriented Middleware

3. amqp concepts (i)
amqp is a message oriented wire-level protocol
● message broker: receives and dispatches msgs
using AMQP
● connection physical connection (eg: tcp/ip)
● channel: allows n clients over one connection
consumerproducer
producer consumer

4. amqp concepts (ii)
consumerproducer
producer consumer
Clients produce and
consume messages.
Exchanges Route and filter
messages to queues: binding
rules, direct, one-to-one,
fanout, topic, headers
Queues buffer messages between
producers and consumers
Messages are always:
>> sent to exchanges
>> consumed from queues

5. escenario 1
● producing and consuming, simplest routing
examples using Python, pika and rabbitmq, everything should apply to other
languages, libraries and amqp brokers
‘important’ messages must be sent to queue ‘important-jobs’
consumerproducer
source code for the example: https://gist.github.com/javierarilos/9348168

6. step 1- connect & channel setup
from pika import BlockingConnection, ConnectionParameters
conn = BlockingConnection(ConnectionParameters('localhost'))
ch = conn.channel()
consumerproducer
Same code for consumer and
producer

7. step 2- declare exchange
Declare an exchange, important parameters:
● name: if exists, nothing is done
● type: direct, fanout, pub-sub, headers
● durable: must exchange be kept between server restarts?
● autodelete: must exchange be deleted when not in use?
● internal: is it for internal routing use, or public to clients?
ch.exchange_declare(exchange='important', type='direct')
consumerproducer
exchange: important
type: direct

8. step 3- declare queue
Declare a queue, important parameters:
● name: if exists, nothing is done
● durable: must queue declaration be kept between server restarts?
● exclusive: Is this the only connection that can consume from the queue?
● autodelete: must queue be deleted when not in use?
ch.queue_declare(queue='important-jobs')
consumerproducer
exchange: important
type: direct
queue: important-jobs

9. step 4- bind queue and exchange
ch.queue_bind(exchange='important', queue='important-jobs', routing_key='important')
consumerproducer
exchange: important
type: direct
queue: important-jobs
routing_key:
important
Binding a queue and exchange:
● establishes a route (exchange => queue)
● based on a criteria (exchange type + routing key)
Here: when producer sends a message to ‘important’ exchange with routing key
‘important’, message will be forwarded to queue ‘important-jobs’.

10. step 5- send the message
ch.basic_publish(exchange='important', routing_key='important', body='new important task')
consumerproducer
exchange: important
type: direct
queue: important-jobs
routing_key:
important
Bonus track: default exchange, forwards to a queue, without any exchange declaration:
default exchange’s name is empty string = ‘’
routing_key is the queue name ‘important-jobs’
ch.basic_publish(exchange=’’, routing_key='important-jobs', body=’def exch important task')

11. step 6- consume the message
method_frame, header_frame, body = ch.basic_get('important-jobs')
print body
ch.basic_ack(method_frame.delivery_tag)
consumerproducer
exchange: important
type: direct
queue: important-jobs
routing_key:
important
Messages must be acknowledged

12. escenario 2
● default exchange in more detail
● message to two queues, depending on routing key
‘important’ messages must be sent to queues ‘important-jobs’ and ‘traces’
consumerproducer
consumer

13. step 1- create & bind new queue
ch.queue_declare(queue='traces')
ch.queue_bind(exchange='important', queue='traces', routing_key='important')
ch.basic_publish(exchange='important', routing_key='important', body='[another task to be handled]')
consumerproducer
exchange: important
type: direct
queue: important-jobs
routing_key:
important
consumer
routing_key:
important
queue: traces

14. step 2- consuming from both
consumerproducer
exchange: important
type: direct
queue: important-jobs
consumer
routing_key:
important
queue: traces
routing_key:
important
● Binding the new ‘traces’ queue to existing ‘important’ exchange does
not affect publishing code.
● Consumers and queues may be added dynamically without affecting the
producer.

15. escenario 3
● 2 x (binding + routing-key) => to 1 queue
● exchange to exchange binding
● headers exchange
‘customer’ messages to ‘important’ exchange must be sent to different queues
depending on the operation to perform (‘signup’, ‘update’) and to ‘traces’
traces
consumer
producer
update
consumer
signup
consumer

16. step 1- bind traces, declare q’s
routing_key:
customer traces
consumer
producer
update
consumer
signup
consumer
exchange: important
type: direct
queue: traces
queue: signup
queue: update
‘traces’ queue is bound to
‘important’ exch with two routing
keys: ‘customer’ and ‘important’
ch.queue_bind(exchange='important', queue='traces', routing_key='customer')
ch.queue_declare(queue='signup')
ch.queue_declare(queue='update')

17. exchange: important
type: direct
traces
consumer
producer
update
consumer
signup
consumer
queue: traces
queue: signup
queue: update
exchange: customer
type: headers
routing_key:
customer
step 2- customer exchange & bind
ch.exchange_declare(exchange='customer', type='headers')
ch.exchange_bind(source='important', destination='customer', routing_key='customer')
ch.queue_bind(exchange='customer', queue='signup',
arguments={'operation': 'signup', 'x-match':'any'})
ch.queue_bind(exchange='customer', queue='update',
arguments={'operation': 'update', 'x-match':'any'})

18. step 3- customer signup message
ch.basic_publish(exchange='important', routing_key='customer',
body='cust num=25', properties=BasicProperties(headers=s{'operation': 'signup'}))
method_frame, header_frame, msg = ch.basic_get('signup')
print "msg received from queue 'signup' : ", msg
ch.basic_ack(method_frame.delivery_tag)
routing_key:
customer traces
consumer
producer
update
consumer
signup
consumer
exchange: important
type: direct
queue: traces
queue: signup
queue: update
exchange: customer
type: headers
msg also routed to traces queue

19. bonus: topic exchanges
traces
consumer
producer
customer
consumer
signup
consumerexchange: operations
type: topic
queue: traces
queue: signup
queue: update
r_k: #
r_k: *.signup
r_k: customer.*
topic exchanges allow to route messages based on topics.
Examples from escenario 3:
○ Producer sends with routing keys: ‘customer.signup’, ‘customer.update’
○ ‘traces’ consumer subscribes to ‘#’ that means all routing keys
○ ‘signup’ customer consumer subscribes to ‘customer.signup’
○ Consumer wanting all customer operations: ‘customer.*’
○ Consumer wanting all signup operations: *.signup’

20. escenario 4
● DeadLetter exchanges in RabbitMQ
What will we do with a message RabbitMQ cannot deliver?
(on client rejection, timeout, queue length exceeded)
By default those messages are dropped, we want not to lose them
consumerproducer

21. step 1- create rejected-jobs
ch.exchange_declare(exchange='rejected-jobs', type='direct')
ch.queue_declare(queue='rejected-jobs')
ch.queue_bind(exchange='rejected-jobs', queue='rejected-jobs', routing_key='important')
consumerproducer
queue: rejected-jobs
exchange: rejected-jobs
type: direct

22. step 2- deadlettr important-jobs
ch.queue_delete('important-jobs')
ch.queue_declare(queue='important-jobs', arguments={'x-dead-letter-exchange': 'rejected-jobs'})
ch.queue_bind(exchange='important', queue='important-jobs', routing_key='important')
consumerproducer
queue: rejected-jobs
exchange: rejected-jobs
type: direct
exchange: important
type: direct
routing_key:
important
x-dead-letter-exchange:
rejected-jobs

23. step 3- consumer rejects job
ch.basic_publish(exchange='important', routing_key='important', body='[unparseable message]')
method_frame, header_frame, important_job = ch.basic_get('important-jobs')
ch.basic_reject(method_frame.delivery_tag, requeue=False)
method_frame, header_frame, rejected_job = ch.basic_get('rejected-jobs')
ch.basic_ack(method_frame.delivery_tag)
consumerproducer
queue: rejected-jobs
exchange: rejected-jobs
type: direct
exchange: important
type: direct
routing_key:
important
x-dead-letter-exchange:
rejected-jobs

24. What are you waiting to start working with RabbitMQ?
It’s fun!

25. Questions?
Thank you for attending.