Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
@asgrim
this slide is intentionally left blank
@asgrim
Before we begin...
Copy folder from USB to your local machine.
$ cd /path/to/the/folder/you/copied/from/usb/
$ vag...
@asgrim
Introducing
Practical RabbitMQ
James Titcumb
James Titcumb
www.jamestitcumb.com
www.roave.com
www.phphants.co.uk
www.phpsouthcoast.co.uk
@asgrim
Who is this guy?
@asgrim Photo: http://spitalfieldslife.com/2011/08/27/a-fox-in-hoxton-2/
@asgrim
What is a message?
@asgrim
What is message queueing?
@asgrim
Separation of Concerns
@asgrim
Scaling with Rabbit
RabbitMQApplication
Background processing
@asgrim
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
@asgrim
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
@asgrim
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Backgrou...
@asgrim
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Backgrou...
@asgrim
AMQP
@asgrim
AMQP Low Level Frame
1 1 67 <frame payload> 0xCE
@asgrim
AMQP Frame: Basic.Publish
<frame payload>
Publish <exchange> <routing key> <flag>
1 1 67 0xCE
Basic
@asgrim
AMQP Message: Multiple Frames
1 1 67 Basic.Publish 0xCE
2 1 102 Content Header 0xCE
3 1 1024 Body 0xCE
3 1 1024 Bo...
@asgrim
AMQP Headers
● content-type
@asgrim
AMQP Headers
● content-type
● content-encoding
@asgrim
AMQP Headers
● content-type
● content-encoding
● message-id
@asgrim
AMQP Headers
● content-type
● content-encoding
● message-id
● correlation-id
● reply-to
@asgrim
AMQP Headers
● content-type
● content-encoding
● message-id
● correlation-id
● reply-to
● expiration
@asgrim
AMQP Headers
● content-type
● content-encoding
● message-id
● correlation-id
● reply-to
● expiration
● priority
@asgrim
AMQP Headers
● content-type
● content-encoding
● message-id
● correlation-id
● reply-to
● expiration
● priority
● ...
@asgrim
Real world uses?
@asgrim
Who is using it?
@asgrim
SOA
@asgrim
Why RabbitMQ?
@asgrim
(you need Vagrant+VirtualBox already)
Installing RabbitMQ
@asgrim
vagrant up
Copy folder from USB to your local machine.
$ cd /path/to/the/folder/you/copied/from/usb/
$ vagrant box...
@asgrim
Library with Composer
composer require php-amqplib/php-amqplib
@asgrim
The Management Console
http://192.168.33.99:15672/
@asgrim
Practical
@asgrim
https://github.com/asgrim/rmq-tutorial
@asgrim
Objective: Basic Queuing
Producer Consumer
test_queue
1 2 3 4 5
@asgrim
Get vs Consume
@asgrim
Exchanges: Fanout
@asgrim
Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5
@asgrim
A word on Temporary Queues
test_exchangeProducer
Messages
go nowhere
@asgrim
Exchanges: Direct
@asgrim
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
BK = orange, banan...
@asgrim
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KE...
@asgrim
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KE...
@asgrim
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KE...
@asgrim
Exchanges: Topic
@asgrim
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK =...
@asgrim
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#...
@asgrim
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green...
@asgrim
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = gree...
@asgrim
Message Acknowledgement
@asgrim
Another Example
@asgrim
@asgrim
Fetch message
Logging Sequence
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
Rabbi...
@asgrim
Flexibility!
@asgrim
Parallel Processing
@asgrim
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Producer
1
1
2
2
3
3
4
4
5
5
Consumer
Consumer
Consumer
Consumer
...
@asgrim
RPC
@asgrim
RPC example
Producer Consumer
test_queue
1
reply_to: amq.gen-Xa2
1’
@asgrim
TTL
@asgrim
TTL - per queue message
Producer Consumer
test_queue
1 2 3 4 5
}
10s
@asgrim
TTL - per message
Producer Consumer
test_queue
1 2 3 4 5
5s 3s 7s 1s 9s
@asgrim
TTL - queue
Producer
test_queue
}
5s (if no consumers)
@asgrim Photo: © Rob Allen (akrabat) https://flic.kr/p/6wp9iz
@asgrim
DLX
@asgrim
DLX
test_exchange
amq.KfgPZ3PE
Producer 1
dlx_exchange
dlx_queue
1
@asgrim
Scheduling / Delayed messages
@asgrim
Shovel
@asgrim
Federated Queues
@asgrim
Federated Queues
1 Consumer
WAN
Rabbit Node #1 Rabbit Node #2
my_queue my_queue (federated)
@asgrim
Priority
@asgrim
Management HTTP API
@asgrim
Infrastructure
@asgrim
Problem: SPOF
@asgrim
Solution 1: Clustering
@asgrim
Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 5
RabbitMQ
Node 6
Load Ba...
@asgrim
Everything Replicates
(except queues…)
@asgrim
Disk / RAM
@asgrim
Configuration...
@asgrim
/etc/rabbitmq/rabbitmq.config
[
{rabbit, [
{loopback_users, []},
{vm_memory_high_watermark, 0.8}
]}
].
@asgrim
/etc/rabbitmq/rabbitmq.config
[{{{[{{[{{}}{][[[{[{{}[[}{[[{}[][}{}}}{}}{{,},]{
[[{rabbit, [{{}[[}{,,{}[][}{[][][{}...
@asgrim
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{runnin...
@asgrim
node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl...
@asgrim
Starting/Stopping Nodes
@asgrim
node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ ra...
@asgrim
Solution 2: HA
@asgrim
HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.
@asgrim
BunnyPHP
@asgrim
composer require bunny/bunny
@asgrim
Challenge!
Any questions?
https://joind.in/talk/be4e5
James Titcumb @asgrim
Prochain SlideShare
Chargement dans…5
×

Adding 1.21 Gigawatts to Applications with RabbitMQ (Bulgaria PHP 2016 - Tutorial)

312 vues

Publié le

As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ. In this tutorial, I will introduce RabbitMQ as a solution to scalable, interoperable and flexible applications.

This tutorial is perfect for those who would like a deep dive into RabbitMQ with little or no pre-existing knowledge about message queuing systems. Once you’ve finished the tutorial, you will have learnt how to set up basic publish/subscribe message queues, control the flow of messages using various exchanges, and understand various features of RabbitMQ such as RPC, TTL, and DLX.

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Adding 1.21 Gigawatts to Applications with RabbitMQ (Bulgaria PHP 2016 - Tutorial)

  1. 1. @asgrim this slide is intentionally left blank
  2. 2. @asgrim Before we begin... Copy folder from USB to your local machine. $ cd /path/to/the/folder/you/copied/from/usb/ $ vagrant box add asgrim/rmq-vm rmq-vm.box $ vagrant up --no-provision Test it in your browser... http://192.168.33.99:15672/
  3. 3. @asgrim Introducing Practical RabbitMQ James Titcumb
  4. 4. James Titcumb www.jamestitcumb.com www.roave.com www.phphants.co.uk www.phpsouthcoast.co.uk @asgrim Who is this guy?
  5. 5. @asgrim Photo: http://spitalfieldslife.com/2011/08/27/a-fox-in-hoxton-2/
  6. 6. @asgrim What is a message?
  7. 7. @asgrim What is message queueing?
  8. 8. @asgrim Separation of Concerns
  9. 9. @asgrim Scaling with Rabbit RabbitMQApplication Background processing
  10. 10. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing
  11. 11. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing
  12. 12. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing
  13. 13. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing Background processing
  14. 14. @asgrim AMQP
  15. 15. @asgrim AMQP Low Level Frame 1 1 67 <frame payload> 0xCE
  16. 16. @asgrim AMQP Frame: Basic.Publish <frame payload> Publish <exchange> <routing key> <flag> 1 1 67 0xCE Basic
  17. 17. @asgrim AMQP Message: Multiple Frames 1 1 67 Basic.Publish 0xCE 2 1 102 Content Header 0xCE 3 1 1024 Body 0xCE 3 1 1024 Body 0xCE 3 1 1024 Body 0xCE 3 1 1024 Body 0xCE
  18. 18. @asgrim AMQP Headers ● content-type
  19. 19. @asgrim AMQP Headers ● content-type ● content-encoding
  20. 20. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id
  21. 21. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to
  22. 22. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to ● expiration
  23. 23. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to ● expiration ● priority
  24. 24. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to ● expiration ● priority ● headers
  25. 25. @asgrim Real world uses?
  26. 26. @asgrim Who is using it?
  27. 27. @asgrim SOA
  28. 28. @asgrim Why RabbitMQ?
  29. 29. @asgrim (you need Vagrant+VirtualBox already) Installing RabbitMQ
  30. 30. @asgrim vagrant up Copy folder from USB to your local machine. $ cd /path/to/the/folder/you/copied/from/usb/ $ vagrant box add asgrim/rmq-vm rmq-vm.box $ vagrant up --no-provision Test it in your browser... http://192.168.33.99:15672/
  31. 31. @asgrim Library with Composer composer require php-amqplib/php-amqplib
  32. 32. @asgrim The Management Console http://192.168.33.99:15672/
  33. 33. @asgrim Practical
  34. 34. @asgrim https://github.com/asgrim/rmq-tutorial
  35. 35. @asgrim Objective: Basic Queuing Producer Consumer test_queue 1 2 3 4 5
  36. 36. @asgrim Get vs Consume
  37. 37. @asgrim Exchanges: Fanout
  38. 38. @asgrim Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  39. 39. @asgrim A word on Temporary Queues test_exchangeProducer Messages go nowhere
  40. 40. @asgrim Exchanges: Direct
  41. 41. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  42. 42. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  43. 43. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  44. 44. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  45. 45. @asgrim Exchanges: Topic
  46. 46. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  47. 47. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  48. 48. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  49. 49. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  50. 50. @asgrim Message Acknowledgement
  51. 51. @asgrim Another Example
  52. 52. @asgrim
  53. 53. @asgrim Fetch message Logging Sequence ApplicationBrowser Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  54. 54. @asgrim Flexibility!
  55. 55. @asgrim Parallel Processing
  56. 56. @asgrim test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Producer 1 1 2 2 3 3 4 4 5 5 Consumer Consumer Consumer Consumer Consumer Consumer Consumer Parallel Processing
  57. 57. @asgrim RPC
  58. 58. @asgrim RPC example Producer Consumer test_queue 1 reply_to: amq.gen-Xa2 1’
  59. 59. @asgrim TTL
  60. 60. @asgrim TTL - per queue message Producer Consumer test_queue 1 2 3 4 5 } 10s
  61. 61. @asgrim TTL - per message Producer Consumer test_queue 1 2 3 4 5 5s 3s 7s 1s 9s
  62. 62. @asgrim TTL - queue Producer test_queue } 5s (if no consumers)
  63. 63. @asgrim Photo: © Rob Allen (akrabat) https://flic.kr/p/6wp9iz
  64. 64. @asgrim DLX
  65. 65. @asgrim DLX test_exchange amq.KfgPZ3PE Producer 1 dlx_exchange dlx_queue 1
  66. 66. @asgrim Scheduling / Delayed messages
  67. 67. @asgrim Shovel
  68. 68. @asgrim Federated Queues
  69. 69. @asgrim Federated Queues 1 Consumer WAN Rabbit Node #1 Rabbit Node #2 my_queue my_queue (federated)
  70. 70. @asgrim Priority
  71. 71. @asgrim Management HTTP API
  72. 72. @asgrim Infrastructure
  73. 73. @asgrim Problem: SPOF
  74. 74. @asgrim Solution 1: Clustering
  75. 75. @asgrim Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 5 RabbitMQ Node 6 Load Balance / Floating IP / Low TTL DNS etc.
  76. 76. @asgrim Everything Replicates (except queues…)
  77. 77. @asgrim Disk / RAM
  78. 78. @asgrim Configuration...
  79. 79. @asgrim /etc/rabbitmq/rabbitmq.config [ {rabbit, [ {loopback_users, []}, {vm_memory_high_watermark, 0.8} ]} ].
  80. 80. @asgrim /etc/rabbitmq/rabbitmq.config [{{{[{{[{{}}{][[[{[{{}[[}{[[{}[][}{}}}{}}{{,},]{ [[{rabbit, [{{}[[}{,,{}[][}{[][][{}{{{{}}}}[[}{{ {{}}{loopback_users, []},[][][]{}{}{}<}{[}[][][} [{{[{vm_memory_high_watermark, 0.8}]]{}{[[[]]{}] {{]}[{[{{}[[}{]]{}[][,{}[][}{[][][{}.[]}{]][][]} ]...{}[][,]{.}[][}{}[[[{}{][]}{}{}[}{}{}{]{}{}}[
  81. 81. @asgrim node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. Creating a cluster
  82. 82. @asgrim node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done. Creating a cluster
  83. 83. @asgrim Starting/Stopping Nodes
  84. 84. @asgrim node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done. Removing Nodes
  85. 85. @asgrim Solution 2: HA
  86. 86. @asgrim HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  87. 87. @asgrim BunnyPHP
  88. 88. @asgrim composer require bunny/bunny
  89. 89. @asgrim Challenge!
  90. 90. Any questions? https://joind.in/talk/be4e5 James Titcumb @asgrim

×