Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

DDD Europe 2019: Lost in transaction

619 vues

Publié le

Slides from my talk at DDD Europe about how to avoid getting lost in transaction and manage consistency across boundaries. Speaking about saga pattern, distributed transactions, choreography, orchestration and more...

Publié dans : Technologie
  • Soyez le premier à commenter

DDD Europe 2019: Lost in transaction

  1. 1. @berndruecker Lost in transaction? Strategies to manage consistency across boundaries
  2. 2. Order Order Line Item Customer Customer Address orderId customerId chargeId … customerId … productId price … Charge Charge Order chargeId amount paymentProvider transactionId … Aggregate Entity Value Object Reference by identity
  3. 3. Order Order Line Item orderId customerId chargeId … productId price … Order Charge Charge chargeId amount paymentProvider transactionId … Create Charge Mark order as paid All or nothing +
  4. 4. A C I D Atomicity Consistency Isolation Durability
  5. 5. Aggregates = Consistency Boundaries Order Order Line Item orderId customerId chargeId … productId price … Order Charge Charge chargeId amount paymentProvider transactionId … You can do ACID within here Or here But not a joined ACID transaction!
  6. 6. Aggregates = Consistency Boundaries Order Order Line Item orderId customerId chargeId … productId price … Order Charge Charge chargeId amount paymentProvider transactionId … Own DB / autonomous technology decisions Scalability Microservices …
  7. 7. Boundaries need to be designed carefully Charge chargeId amount paymentProvider transactionId … Order Order Line Item orderId customerId chargeId … productId price …Order
  8. 8. But no implicit constraints! Order Order Line Item orderId customerId chargeId … productId price … Order Charge Charge chargeId amount paymentProvider transactionId … Joined DB transaciton
  9. 9. There is two-phase commit (XA)!! TX Coordinator Resource Managers Prepare Phase Commit Phase
  10. 10. Pat Helland “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce
  11. 11. Pat Helland Grown-Ups Don’t Use Distributed Transactions “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce
  12. 12. Starbucks does not use two phase commit https://www.enterpriseintegrationpatterns.com/ramblings/18_starbucks.html Photo by John Ingle
  13. 13. Eric Brewer Atomicity Consistency Isolation Durability http://pld.cs.luc.edu/courses/353/spr11/notes/brewer_keynote.pdf
  14. 14. That means Do A Do B Temporarily inconsistent Eventually consistent again t Consistent Local ACID Local ACID 1 aggregate 1 (micro-)service 1 program 1 resource Violates „I“ of ACID
  15. 15. Youmightknowthisfrom: Do A Do B Temporarily inconsistent Eventually consistent again t Consistent Photo by Gerhard51, available under Creative Commons CC0 1.0 license.
  16. 16. „Building on Quicksand“ Paper A C I D 2.0Pat Helland
  17. 17. Associative Commutative Idempotent Distributed 2.0 (a + b) + c = a + (b + c) a + b = b + a f(x) = f( f(x) ) „Building on Quicksand“ Paper Pat Helland
  18. 18. Distributed
  19. 19. Distributed systems
  20. 20. Distributed systems
  21. 21. It is impossible to differentiate certain failure scenarios: Independant of communication style! Service Provider Client
  22. 22. Network problems Credit Card Payment charge
  23. 23. Strategy: retry Credit Card Payment Charge Credit Card cardNumber amount Charge Credit Card cardNumber amount transactionId Not idempotent Idempotent has to be idempotent charge
  24. 24. Photo by pixabay, available under Creative Commons CC0 1.0 license.
  25. 25. Requirement: Idempotency of services! Photo by pixabay, available under Creative Commons CC0 1.0 license.
  26. 26. Requirement: Idempotency of services! Photo by Chr.Späth, available under Public Domain.
  27. 27. Strategy: Cleanup Credit Card Payment charge Make sure it is not charged! Cancel charge cardNumber amount transactionId Raise payment failed
  28. 28. Distributed systems
  29. 29. Some communication challenges require state.
  30. 30. Strategy: Stateful retry Credit Card Payment charge
  31. 31. Strategy: Stateful retry Credit Card Payment charge Make sure it is not charged!
  32. 32. Warning: Contains Opinion
  33. 33. Berlin, Germany bernd.ruecker@camunda.com @berndruecker Bernd Ruecker Co-founder and Technologist of Camunda
  34. 34. Let‘s use a lightweight OSS workflow engine for this:
  35. 35. Payment Stateful retry Credit Card REST
  36. 36. Stateful retry & cleanup Credit Card Payment REST Cancel charge
  37. 37. Live hacking https://github.com/flowing/flowing-retail/tree/master/rest
  38. 38. Architecture Infrastructure Application Domain Workflow Engine
  39. 39. Embedded Engine Example (Java) https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  40. 40. Remote Engine Example (Polyglot) https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
  41. 41. A relatively common pattern Service (e.g. Go) Kafka / Rabbit RDMS 1. Receive 4. Send additional events 2. Business Logic 3. Send response? ACK
  42. 42. Compensation – the classical example Saga book hotel book car book flight cancel hotel cancel car 1. 2. 3. 5.6. In case of failure trigger compensations book trip
  43. 43. 2 alterntive approaches: choreography & orchestration
  44. 44. Event-driven choreography Hotel Flight Car Trip Trip booked Flight booked Trip requested Hotel booked Car booked Request trip
  45. 45. Event-driven choreography Hotel Flight Car Trip Trip failed Trip requested Hotel booked Car booked Request trip Flight failed Car canceled Hotel canceled Perform undo (cancel car booking) Perform undo (cancel hotel)
  46. 46. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  47. 47. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  48. 48. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html
  49. 49. Classical example Saga book hotel book car book flight cancel hotel cancel car 1. 2. 3. 5.6. In case of failure trigger compensations book trip
  50. 50. If your transaction involves 2 to 4 steps, choreography might be a very good fit. However, this approach can rapidly become confusing if you keep adding extra steps in your transaction as it is difficult to track which services listen to which events. Moreover, it also might add a cyclic dependency between services as they have to subscribe to one another’s events. Denis Rosa Couchbase https://blog.couchbase.com/saga-pattern-implement-business-transactions-using-microservices-part/
  51. 51. Implementing changes in the process Hotel Flight Car Trip Trip failed Trip requested Hotel booked Car booked Request trip Flight failed Car canceled Hotel canceled We have a new basic agreement with the car rental agency and can cancel for free within 1 hour – do that first!
  52. 52. Implementing changes in the process Hotel Flight Car Trip Trip failed Trip requested Hotel booked Car booked Request trip Flight failed Car canceled Hotel canceled You have to adjust all services and redeploy at the same time! We have a new basic agreement with the car rental agency and can cancel for free within 1 hour – do that first!
  53. 53. Photo by born1945, available under Creative Commons BY 2.0 license.
  54. 54. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0
  55. 55. Orchestration Hotel Flight Car Trip Trip booked Request trip Book hotel Hotel booked Car booked Flight booked Book car Book flight
  56. 56. Orchestration Hotel Flight Car Trip Trip booked Request trip Book hotel Hotel booked Car booked Flight booked Book car Book flight We have a new basic agreement with the car rental agency and can cancel for free within 1 hour – do that first! You have to adjust one service and redeploy only this one!
  57. 57. Describe orchestration with BPMN Trip Trip booked Request trip
  58. 58. The workflow is domain logic as part of the service Trip
  59. 59. The workflow is domain logic as part of the service Trip Payment Payment could be one step in the Trip Saga
  60. 60. Graphical models?
  61. 61. Clemens Vasters Architect at Microsoft http://vasters.com/archive/Sagas.html
  62. 62. Clemens Vasters Architect at Microsoft http://vasters.com/archive/Sagas.html
  63. 63. Clemens Vasters Architect at Microsoft http://vasters.com/archive/Sagas.html
  64. 64. BPMN Business Process Model and Notation ISO Standard
  65. 65. Living documentation for long-running behaviour
  66. 66. Visual HTML reports for test cases
  67. 67. BizDevOps
  68. 68. Remember… Ubiquitous Language Software Experts Domain Experts
  69. 69. Fancy a DSL? Just do it! https://github.com/berndruecker/flowing-trip-booking-saga The visual get auto-generated…
  70. 70. Thoughts on the state machine | workflow engine market
  71. 71. Thoughts on the state machine | workflow engine market OSS Workflow or Orchestration Engines Stack Vendors, Pure Play BPMS Low Code Platforms Homegrown frameworks to scratch an itch Integration Frameworks Cloud Offerings Uber, Netflix, AirBnb, ING, … AWS Step Functions, Azure Durable Functions, … Camunda, Zeebe, jBPM, Activiti, Mistral, … PEGA, IBM, SAG, … Apache Airflow, Spring Data Flow, … Apache Camel, Balerina, … Data Pipelines
  72. 72. Does it support stateful operations? Does it support the necessary flow logic? Does it support BizDevOps? Does it scale?
  73. 73. My personal pro-tip for a shortlist ;-) OSS Workflow or Orchestration Engines Stack Vendors, Pure Play BPMS Low Code Platforms Homegrown frameworks to scratch an itch Integration Frameworks Cloud Offerings Data Pipelines Camunda & Zeebe
  74. 74. Recap • Aggregates = Consistency boundaries • No distributed transactions but eventual consistency • Idempotency is super important • Some consistency challenges require state • Stateful retry & cleanup • Saga / Compensation
  75. 75. Thank you!
  76. 76. mail@berndruecker.io @berndruecker https://berndruecker.io https://medium.com/berndruecker https://github.com/berndruecker https://www.infoq.com/articles/events- workflow-automation Contact: Slides: Blog: Code: https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html https://thenewstack.io/5-workflow-automation- use-cases-you-might-not-have-considered/

×