StormMQ Introduction to AMQP, Dublin
•
1 j'aime•1,678 vues
Message queuing connects systems and components loosely by allowing them to exchange data and information asynchronously without being directly connected or aware of each other. It is ideal for cloud applications as it provides loose coupling between systems, making individual systems less likely to suffer outages when another system changes or has issues. The document discusses the history and benefits of message queuing including AMQP, which provides an open standard to enable interoperability between messaging systems.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à StormMQ Introduction to AMQP, Dublin
Similaire à StormMQ Introduction to AMQP, Dublin (20)
Dernier
Dernier (20)
StormMQ Introduction to AMQP, Dublin
- 1. Introduction to Message Queuing raphael.cohn@stormmq.com
- 2. Agenda A Real Application ! 20 min Nibbles and Hands–On ! 60 min Introduction to AMQP ! 20 min Why Message Queuing? ! 25 min Classic Architecture ! 10 min
- 3. History of Message Queuing Manual Telegraphy Machine Assisted Telegraphy 1920s1911 - 192019th Century 1900s Telegrams sent using “Store and Forward” 1900 1930s
- 4. History of Message Queuing Telcos UseElectronic Telegraphy 1950s1940s 1960s IBM System/360 with BTAM & QTAM Message Switching 1964 First Electronic Mail Solutions 1965 Banking Users 1970s IBM TCAM which is the first true solution Retired 1990! 1971
- 5. History of Message Queuing l FMCG & Utilities 1990s MQSeries phereMQ) 92 Corporates Large Websites YOU Noughties Today Sun Release Java JMS, Reinvigorating Enterprise Messaging 2001 AMQP Working Group Formed by Investment Banks 2006 Cloud Enables and Drives StormMQ Adoption 2009
- 6. The Integration Tag Soup WS –File Transfer !"#$%&'(#)#*#+%+ Message Queuing SOAP EMail SMTP FTP RSYNC NTFS !','- !).$%&'/$.0%&1$%+ 234%5)6!%$7%$ SOA Message Orientated Middleware JMS MQ MQSeries AMQP Amazon SQS Tibco StormMQ RabbitMQ HTTP TCP/IP DCE / RPC CORBA DCOM REST RMI XML-RPC .NET Remoting Remote Procedure Call
- 7. The Integration Tag Soup WS –File Transfer !"#$%&'(#)#*#+%+ Message Queuing SOAP EMail SMTP FTP RSYNC NTFS !','- !).$%&'/$.0%&1$%+ 234%5)6!%$7%$ SOA Message Orientated Middleware JMS MQ MQSeries AMQP Amazon SQS Tibco StormMQ RabbitMQ HTTP TCP/IP DCE / RPC CORBA DCOM REST RMI XML-RPC .NET Remoting Remote Procedure Call
- 8. The Integration Tag Soup Message Queuing
- 9. Under Examination, though Deployment Intimate System Knowledge Configuration Admin File Transfer !"#$%&'(#)#*#+%+ Remote Procedure Call Don’tWork in the Cloud Scaling
- 10. Ideal: Message Queuing Systems are “loosely-coupled” Message Queuing Systems don’t know each other
- 11. Why Use it: Loose Coupling Billing Catalogue Shipping S-a-a-S Inventory How do we connect them, without one outage or system change taking everything down like a pack of cards?
- 12. Why Use it: Loose Coupling Billing Catalogue Shipping S-a-a-S Inventory How do we connect them, without one outage or system change taking everything down like a pack of cards? Message Queuing lets Systems and Components exchange data, events, commands and actions with one another with no explicit knowledge or need for them to be online
- 13. It should be ideal for the cloud
- 14. It should be ideal for the cloud
- 15. ! ! ! Until today, messaging is … (ZeroMQ) (SQS uses HTTP) (MSMQ uses COM) (memcached) (Java JMS uses Source) (MQSeries / WebsphereMQ) (Talarian, Rendezvous, etc) Platform Restricted BespokeProprietary
- 16. ! ! ! Until today, messaging is … (ZeroMQ) (SQS uses HTTP) (MSMQ uses COM) (memcached) (Java JMS uses Source) (MQSeries / WebsphereMQ) (Talarian, Rendezvous, etc)
- 17. Why is it Hell? You need more staff And moneyYou need bridge technology
- 18. Why is it Hell? You need more staff And moneyYou need bridge technology
- 19. However, there’s a solution
- 20. A-MQPAdvanced Message Queue Protocol However, there’s a solution
- 21. However, AMQP fixes this A common wire-level binary format and protocol An explicit definition of a server (aka broker)’s semantics Open Means Interoperable
- 22. That is good … “AMQP will be to Messaging what HTTP was to theWeb” MRG Clients run on any Platform Vendors are Interoperable
- 23. That is good … “AMQP will be to Messaging what HTTP was to theWeb” MRG Clients run on any Platform Vendors are Interoperable 64K
- 24. From Anywhere to Anywhere
- 25. From Anywhere to Anywhere
- 26. EddieVelez Quick Recap ✓ The fifth male member of the A-Team, Frankie, was played by Question Message Queuing connects systems and components. Is it ideal for the cloud? Does Loose-Coupling make individual systems more likely to suffer outages? Yes No AMQP is Open. This makes it suitable for programming in C and Javascript? ! ✓
- 27. Agenda Introduction to AMQP ! 20 min Why Message Queuing? ! 25 min Classic Architecture ! 10 min A Real Application ! 20 min Pizza and Hands–On ! 60 min
- 28. Store and Forward Billing “Could you send me a duplicate of my bill please” REST StormMQ Messaging Billing !
- 29. Fire and Forget Shipping “Too many orders.The website is running like a dog.” SQL Push SQL Pull StormMQ Messaging Shipping
- 30. One-To-Many “There’s a new widget in the Catalogue: Tell all the systems.” Catalogue Shipping S-a-a-S Inventory File Transfer ETL
- 31. One-To-Many “There’s a new widget in the Catalogue: Tell all the systems.” Catalogue Shipping S-a-a-S Inventory StormMQ Messaging
- 32. StormMQ Messaging Round-Robin Billing “How do we easily scale a massive batch job like Billing?” EMail BillGenerate
- 33. StormMQ Messaging Publish-Subscribe (“Topics”) “Shipments Sent, Delivered and Received” Shipping S-a-a-S Inventory Billing RIASent Sent or Returned
- 34. 22 Web Sockets Architecture Copyright © 2010, Kaazing Corporation,. All rights reserved.
- 35. Agenda Introduction to AMQP ! 20 min Why Message Queuing? ! 25 min Classic Architecture ! 10 min A Real Application ! 20 min Nibbles and Hands–On ! 60 min
- 36. Messaging: Which Jargon? EMail (SMTP, POP3, IMAP) VoIP (VoiceMail, XMPP) Texting (SMS) Instant Messaging (ICQ, MSN, Jabber) Twitter Enterprise Service Bus (ESB) Dynamic OO Languages (eg Ruby) Message Queuing (MQ) Message Queuing (MQ) What do we mean by Messaging?
- 37. Essential Terms Message Queue What’s a Message? Envelope Payload Header
- 38. Essential Terms *Strictly speaking a receiver polls for messages a consumer has messages pushed ≣ ≊Send Publish Enqueue ≣ ≊ DequeueConsume*Receive
- 39. Connection AMQP Client AMQP ServerTCP / IP Network Connection Virtual Host Connections and Channels TLS “Shielding” Channels Each Channel is Independent: Effectively, aVirtual Connection
- 40. Basic AMQP Basic AMQP: Connections Open a Connection to aVirtual Host Open a Channel Send a Message Receive a Message Close Channel Close Connection You only need one channel! " # $ % & '
- 41. Exchanges Route Messages Exchanges route Messages to Message Queues You send messages to an exchange, not a message queue Exchange MQ A MQ B
- 42. How do Exchanges Route? Exchange MQ A The Exchange finds a Binding matching the Routing Key # A Binding connects a Routing Key to one or more Message Queues $ Every sent message has a Routing Key " The Exchange delivers the Message to the Message Queue % A message queue can be bound more than once to one or more exchanges ( “string”
- 43. Types of Exchange More rarely used exchange types include amq.headers and extensions Message Queue Name direct Like a ‘Map’:All MQs bound with the routing key receive copies of the message “” (blank) amq.direct fanout Empty String All MQs bound to the Exchange receive copies of the message amq.fanout topic Dotted Bindings use globbing expressions (wildcards) to route messages to MQs amq.topic A message queue can be bound more than once to an exchange; A message queue can be bound to more than one exchange But a message queue will only receive a message sent once Routing Key Routing Behaviour Default Definitions Point – to – Point One – to – Many Publish – SubscribeTypical Use
- 44. And there is more! Transactions More Message Properties Custom Message Properties Immediate Delivery Additional Exchange Types Auto Deletion Fine-Grained User Permissions Queue Purging QoS
- 45. Agenda Introduction to AMQP ! 20 min Why Message Queuing? ! 25 min Classic Architecture ! 10 min A Real Application ! 20 min Nibbles and Hands–On ! 60 min
- 46. The Problem Intrusion Detection Lots of Logs Firewall Monitoring Performance & Failure
- 47. What’s needed Live Debugging Live Logging Live Statistics Audit ComplianceStatistics Mining SMS Alerts EMail Alerts
- 48. It must not get in the way Add Hardware Config MaintenanceNo Extra Admins Change Software
- 49. Design of the Clients intrusion api.daemon.info.charon api.memory api.daemon.sshd api.warn checksstatisticslog Content-Type: application/json; charset=utf-8 Timestamp: 2010-11-03T16:45:56.981Z Priority: Low / High Type: statistics_2011.10.01.1203 Expiry: 5 mins api.firewall.warn.in.portscan.tcp
- 50. checks Handling Log Messages api.*.*.warn Queue # Queue Timestamp No Acks QoS Prefetch WebSockets log MessageId Routing Key *.warn Queue checks Statistics A rchitecture is the Sam e
- 51. From Checks to Alerts # Queue emergency Queue # Queue checks alerts alerts Contact Details not in message Message Aggregation / Suppression by alerts Uses Message Priority
- 52. And debugging ?
- 53. A-MQPAdvanced Message Queue Protocol And debugging ?