Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
@berndruecker
The Big Picture
Monitoring and Orchestration
of Your Microservices
Landscape with
Kafka and Zeebe
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Microservices…
Data, data, data, …
Source: Microservices orchestration survey,
July 2018, 354 responses
92 %
The microservice community favours an
alternative approach: smart endpoints and
dumb pipes.
[… Microservices …] are choreo...
De-coupled services with event backbone
Checkout Inventory
Payment Shipment
Kafka
Event-driven
Checkout
Payment
Inventory
Shipment
Order
Placed
Payment
Received
Goods
Fetched
Peer-to-peer event chain
Checkout
Payment
Inventory
Shipment
The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
...
The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
...
The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
...
Challenges?
Microservice pioneers
have become aware
What we wanted
Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0
Finding the right balance
16
Observe Microservice Paradigms
● Loose Coupling
● Autonomy
● Event Collaboration
Ensure Flow ...
Workflow automation | orchestration
is an essential building block
in microservice architectures
Of course I am opinioated...
Workflow automation | orchestration
is an essential building block
in microservice architectures
Of course I am opinioated...
Warning:
Contains Opinion
HQ Germany (Berlin) + Offices in US
bernd.ruecker@camunda.com
@berndruecker
Bernd Ruecker
Co-founder and
Developer Advocat...
Different levels of leveraging workflow automation
Manage the flowTrack the flow
Non-invasive, read-only
Additionally to c...
A possible journey from pogo to beauty…
Manage the flowTrack the flow
Non-invasive, read-only
Additionally to choreography...
Tracking
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Tracking + BI
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Camunda
Optimize
Other reporting
/ BIO solutions
Tracking + BI
Checkout Inventory
Payment Shipment
Kafka
Workflow
Engine
Camunda
Optimize
Other reporting
/ BIO solutions
Tracking + BI
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Camunda
Optimize
Other reporting
/ BIO solutions
This is tot...
But you can start acting upon certain events
Live hacking
https://github.com/berndruecker/flowing-retail/
Demo architecture
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Spring
Boot
Services
Spring Cloud Streams
Kafka Connect
...
Why so shy?
Photo by LuidmilaKot, available under Creative Commons CC0 1.0 License.
A possible journey away from pogo…
Manage the flowTrack the flow
Non-invasive, read-only
Additionally to choreography
Limi...
Changing the flow
Checkout
Payment
Inventory
Shipment
Fetch the goods
before the
payment
Changing the flow
Checkout
Payment
Inventory
Shipment
Fetch the goods
before the
payment
Photo by born1945, available under Creative Commons BY 2.0 license.
Order
Checkout
Payment
Inventory
Shipment
Important responsibilities need a home
Distributed systems
Distributed systems
Distributed systems
Order
Checkout
Payment
Inventory
Shipment
Stateful orchestration
This orchestration
requires state
Order
Checkout
Payment
Inventory
Shipment
Now it is easy to change the flow
Manage the flow
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Order
Owned heredeployed here
Live hacking
https://github.com/berndruecker/flowing-retail/
Demo architecture
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Spring
Boot
Services
Spring Cloud Streams
Order
Alternat...
BPMN
Business Process
Model and Notation
ISO Standard
BPMN supports integration use cases
Aggregator
Re-Sequencer
Combined with timeouts:
Zeebe supports Message TTL
BPMN supports integration use cases
Compensation
Biz Dev Ops
Hybrid architectures
Order
Credit
Card
Customer
Credit
Payment
Pub/Sub
e.g. Kafka
Pub/Sub
Work Distribution
e.g. External ...
Order
Checkout
Payment
Inventory
Shipment
Every service owns its workflow!
Central Zeebe or …
Checkout Inventory
Payment Shipment
Kafka
Zeebe
Order
Owned heredeployed here
Owned here
…decentral Zeebe
Checkout Inventory
Payment Shipment
Kafka
ZeebeOrderZeebe
Workflow automation at scale!
low latency,
high-throughput
low frequency,
latency doesn‘t matter
What people think workflo...
Vodafone, Liongate & WDW
Presented at CamundaCOn Berlin 2018
Recap
• In event-driven-systems it is easy to loose sight of the flow
• Visibility is essential to survive
• You need to b...
Thank you!
mail@berndruecker.io
@berndruecker
https://berndruecker.io
https://medium.com/berndruecker
https://github.com/berndruecker...
Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe
Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe
Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe
Prochain SlideShare
Chargement dans…5
×

Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe

1 456 vues

Publié le

Slides from my talk at the Kafka Summit San Francisco 2018 about orchestration vs. choreography as well as track vs. manage event flows. Source code for live demos available on https://github.com/berndruecker/flowing-retail

Publié dans : Technologie

Kafka Summit 2018: Monitoring and Orchestration of Your Microservices Landscape with Zeebe

  1. 1. @berndruecker The Big Picture Monitoring and Orchestration of Your Microservices Landscape with Kafka and Zeebe
  2. 2. Some Service Some Service Some Service Some Service Some Service Some Service Some Service Microservices…
  3. 3. Data, data, data, … Source: Microservices orchestration survey, July 2018, 354 responses
  4. 4. 92 %
  5. 5. The microservice community favours an alternative approach: smart endpoints and dumb pipes. [… Microservices …] are choreographed using simple RESTish protocols rather than complex protocols such as WS-Choreography or BPEL or orchestration by a central tool. https://www.martinfowler.com/articles/microservices.html
  6. 6. De-coupled services with event backbone Checkout Inventory Payment Shipment Kafka
  7. 7. Event-driven Checkout Payment Inventory Shipment Order Placed Payment Received Goods Fetched
  8. 8. Peer-to-peer event chain Checkout Payment Inventory Shipment
  9. 9. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  10. 10. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  11. 11. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  12. 12. Challenges?
  13. 13. Microservice pioneers have become aware
  14. 14. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0
  15. 15. Finding the right balance 16 Observe Microservice Paradigms ● Loose Coupling ● Autonomy ● Event Collaboration Ensure Flow Manageability • End-To-End Visibility • Meet Business Process SLA • Support Continuous Process Improvement
  16. 16. Workflow automation | orchestration is an essential building block in microservice architectures Of course I am opinioated – but I truly believe this!
  17. 17. Workflow automation | orchestration is an essential building block in microservice architectures Of course I am opinioated – but I truly believe this! Orchestration is not a central thing! „It is not BPEL“
  18. 18. Warning: Contains Opinion
  19. 19. HQ Germany (Berlin) + Offices in US bernd.ruecker@camunda.com @berndruecker Bernd Ruecker Co-founder and Developer Advocate of Camunda
  20. 20. Different levels of leveraging workflow automation Manage the flowTrack the flow Non-invasive, read-only Additionally to choreography Limited in power Needs to be incorporated Mixture of choreography and orchestration Leverage full potential
  21. 21. A possible journey from pogo to beauty… Manage the flowTrack the flow Non-invasive, read-only Additionally to choreography Limited in power Needs to be incorporated Mixture of choreography and orchestration Leverage full potential
  22. 22. Tracking Checkout Inventory Payment Shipment Kafka Zeebe
  23. 23. Tracking + BI Checkout Inventory Payment Shipment Kafka Zeebe Camunda Optimize Other reporting / BIO solutions
  24. 24. Tracking + BI Checkout Inventory Payment Shipment Kafka Workflow Engine Camunda Optimize Other reporting / BIO solutions
  25. 25. Tracking + BI Checkout Inventory Payment Shipment Kafka Zeebe Camunda Optimize Other reporting / BIO solutions This is totally non-invasive But also limited in power
  26. 26. But you can start acting upon certain events
  27. 27. Live hacking https://github.com/berndruecker/flowing-retail/
  28. 28. Demo architecture Checkout Inventory Payment Shipment Kafka Zeebe Spring Boot Services Spring Cloud Streams Kafka Connect Zeebe Simple Monitor
  29. 29. Why so shy? Photo by LuidmilaKot, available under Creative Commons CC0 1.0 License.
  30. 30. A possible journey away from pogo… Manage the flowTrack the flow Non-invasive, read-only Additionally to choreography Limited in power Needs to be incorporated Mixture of choreography and orchestration Leverage full potential
  31. 31. Changing the flow Checkout Payment Inventory Shipment Fetch the goods before the payment
  32. 32. Changing the flow Checkout Payment Inventory Shipment Fetch the goods before the payment
  33. 33. Photo by born1945, available under Creative Commons BY 2.0 license.
  34. 34. Order Checkout Payment Inventory Shipment Important responsibilities need a home
  35. 35. Distributed systems
  36. 36. Distributed systems
  37. 37. Distributed systems
  38. 38. Order Checkout Payment Inventory Shipment Stateful orchestration This orchestration requires state
  39. 39. Order Checkout Payment Inventory Shipment
  40. 40. Now it is easy to change the flow
  41. 41. Manage the flow Checkout Inventory Payment Shipment Kafka Zeebe Order Owned heredeployed here
  42. 42. Live hacking https://github.com/berndruecker/flowing-retail/
  43. 43. Demo architecture Checkout Inventory Payment Shipment Kafka Zeebe Spring Boot Services Spring Cloud Streams Order Alternative: Kafka Connect
  44. 44. BPMN Business Process Model and Notation ISO Standard
  45. 45. BPMN supports integration use cases Aggregator Re-Sequencer Combined with timeouts: Zeebe supports Message TTL
  46. 46. BPMN supports integration use cases Compensation
  47. 47. Biz Dev Ops
  48. 48. Hybrid architectures Order Credit Card Customer Credit Payment Pub/Sub e.g. Kafka Pub/Sub Work Distribution e.g. External Task Request/Response e.g. REST
  49. 49. Order Checkout Payment Inventory Shipment Every service owns its workflow!
  50. 50. Central Zeebe or … Checkout Inventory Payment Shipment Kafka Zeebe Order Owned heredeployed here Owned here
  51. 51. …decentral Zeebe Checkout Inventory Payment Shipment Kafka ZeebeOrderZeebe
  52. 52. Workflow automation at scale! low latency, high-throughput low frequency, latency doesn‘t matter What people think workflow automation can do What we currently teach workflow automation to be able to do What workflow automation can already do today
  53. 53. Vodafone, Liongate & WDW Presented at CamundaCOn Berlin 2018
  54. 54. Recap • In event-driven-systems it is easy to loose sight of the flow • Visibility is essential to survive • You need to balance orchestration and chorography • Distributed systems need stateful orchestration • Track, monitor or manage the flow – it might be a journey • Workflow automation is an essential building block • BPMN is very powerful and provides BizDevOps
  55. 55. Thank you!
  56. 56. mail@berndruecker.io @berndruecker https://berndruecker.io https://medium.com/berndruecker https://github.com/berndruecker https://www.infoq.com/articles/events- workflow-automation Contact: Slides: Blog: Code: https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html https://thenewstack.io/5-workflow-automation- use-cases-you-might-not-have-considered/

×