RabbitMQ is a message broker – an application that allows communication between applications by way of a message queuing system. In this talk, we’ll set up a RabbitMQ instance, take an intermediate-level look into the technical features it provides and also how you can apply RabbitMQ in your applications to scale them efficiently.

1. Practical Message
Queueing Using
RabbitMQ
James Titcumb
Nomad PHP EU December 2014

2. James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim
Who is this guy?

3. What is message
queueing?

4. Separation of Concerns

5. Scaling with Rabbit
RabbitMQApplication
Background processing

6. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing

7. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing

8. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing

9. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing
Background processing

10. Real world uses?

11. Why RabbitMQ?

12. (on precise64, other OSs may vary)
Installing RabbitMQ

13. ● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
● apt-get update
● apt-get install rabbitmq-server
● rabbitmq-plugins enable rabbitmq_management
● sudo service rabbitmq-server restart
Using Apt

14. http://localhost:15672/

15. http://localhost:15672/

16. http://localhost:15672/

17. http://localhost:15672/

18. http://localhost:15672/

19. Basic Message Queuing

20. Objective: Basic Queuing
Producer Consumer
test_queue
1 2 3 4 5

21. composer.json
1 {
2 "require": {
3 "videlalvaro/php-amqplib": "2.*"
4 }
5 }
then composer install

22. Please wait, connecting...
1 use PhpAmqpLibConnectionAMQPConnection;
2
3 $connection = new AMQPConnection(
4 'localhost',
5 5672,
6 'guest',
7 'guest',
8 '/'
9 );
10 $channel = $connection->channel();

23. basic/producer.php
1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->queue_declare(
4 'test_queue',
5 false,
6 true,
7 false,
8 false);
9
10 $message = new AMQPMessage('my test message');
11 $channel->basic_publish($message, '', 'test_queue');

24. basic/consumer.php
1 $channel->basic_consume(
2 'test_queue', // Queue to consume
3 '', // Consumer identifier
4 false,
5 true, // No-ack means messages are "auto acknowledged"
6 false, // Exclusive - no other consumers can use the queue
7 false,
8 function(AMQPMessage $message) {
9 echo $message->body . "n";
10 }
11 );
12
13 while (count($channel->callbacks)) {
14 $channel->wait();
15 }

25. What to expect...

26. Exchanges: Fanout

27. Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5

28. 1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->exchange_declare(
4 'test_exchange',
5 'fanout',
6 false,
7 false,
8 false);
9
10 $message = new AMQPMessage('my test message #' . $id);
11 $channel->basic_publish($message, 'test_exchange');
fanout/producer.php

29. fanout/consumer.php
1 $q = $channel->queue_declare(
2 '', // Lets RabbitMQ pick a name for queue
3 false, false, false,
4 true // Delete this queue
5 );
6 $queue_name = $q[0];
7
8 $channel->exchange_declare(
9 'test_exchange', 'fanout', false, false, false);
10 $channel->queue_bind($queue_name, 'test_exchange');

30. What to expect...

31. A word on Temporary Queues
test_exchangeProducer
Messages
go nowhere

32. Exchanges: Direct

33. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
BK = orange, banana,
apple
Consumer

34. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= ORANGE
BK = orange, banana,
apple
Consumer

35. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= BANANA
BK = orange, banana,
apple
Consumer

36. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= APPLE
BK = orange, banana,
apple
Consumer

37. direct/producer.php
1 $channel->exchange_declare(
2 'test_direct', 'fanout', false, false, false);
3
4 $messageContent = 'my test message, key=' . $routingKey;
5 $message = new AMQPMessage($messageContent);
6 $channel->basic_publish(
7 $message,
8 'test_direct',
9 $routingKey
10 );

38. direct/consumer.php
1 $q = $channel->queue_declare('', false, false, false, true);
2 $queue_name = $q[0];
3 $channel->exchange_declare(
4 'test_direct', 'direct', false, false, false);
5
6 // Bind for each routing key we want (BINDING KEY)
7 $channel->queue_bind($queue_name, 'test_direct', 'apple');
8 $channel->queue_bind($queue_name, 'test_direct', 'orange');
9 $channel->queue_bind($queue_name, 'test_direct', 'banana');

39. What to expect...

40. Exchanges: Topic

41. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

42. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

43. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

44. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

45. Real World Example

47. Fetch message
Logging Sequence
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

48. Flexibility!

49. Parallel Processing

50. Message
Acknowledgement

51. RPC

52. TTL

53. DLX

54. http://tryrabbitmq.com/

55. Infrastructure

56. Problem: SPOF

57. Solution 1: Clustering

58. Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 5
RabbitMQ
Node 6
Load Balance / Floating IP / Low TTL DNS etc.

59. Everything Replicates
(except queues…)

60. RAM / Disk

61. Configuration...

62. /etc/rabbitmq/rabbitmq.config
[
{rabbit, [
{loopback_users, []},
{vm_memory_high_watermark, 0.8}
]}
].

63. Creating a cluster
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}]
...done.
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}]
...done.

64. node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.
Creating a cluster

65. Starting/Stopping Nodes

66. node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.
Removing Nodes

67. Solution 2: HA

68. HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.

69. https://github.com/asgrim/rmq-slides
Have a go yourself!

70. Questions?

71. https://joind.in/13093
James Titcumb
@asgrim
Thanks for watching!