Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Distributed transactions in SOA and Microservices

857 vues

Publié le

We are familiar with transactions in databases, messaging systems and caches. When messaging system works fine, but database transaction fails - we are in big trouble! How to make all this guys work as a team? - What need to be used is distributed transactions or compensation mechanisms. On this talk we see both approaches and discuss how to take advantage of them when we are building Microservices architecture.

See code examples here: https://github.com/kslisenko/distributed-transactions

Publié dans : Technologie
  • Soyez le premier à commenter

Distributed transactions in SOA and Microservices

  1. 1. 1CONFIDENTIAL DISTRIBUTED TRANSACTIONS IN SOA AND MICROSERVICES KANSTANTSIN SLISENKA LEAD SOFTWARE ENGINEER NOV 9, 2017
  2. 2. 2CONFIDENTIAL kanstantsin_slisenka@epam.com Kanstantsin Slisenka Java Team Lead Financial services, trading solutions Speaker at: Minsk Java Tech Talks, IT Week, SEC Online, Java Professionals github.com/kslisenko
  3. 3. 3CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
  4. 4. 4CONFIDENTIAL WHY DO WE NEED TRANSACTIONS?
  5. 5. 5CONFIDENTIAL WHY DO WE NEED TRANSACTIONS? New order created Payment received Product shipped Data must be in known state anytime
  6. 6. 6CONFIDENTIAL Application Business workflows Infrastructure Transactions in file systems, databases, message brokers, caches, application servers
  7. 7. 7CONFIDENTIAL Transaction processing is based on logging of states and transitions HOW IT WORKS
  8. 8. 8CONFIDENTIAL Transaction log • Always in known state • States and transitions are logged • Log can be used for recovery DATABASE TRANSACTION AS STATE MACHINE begin in progress commit rollback INSERT UPDATE DELETE commitbegin
  9. 9. 9CONFIDENTIAL ACID TRANSACTION GUARANTEES Atomicity All or nothing Write-ahead log
  10. 10. 10CONFIDENTIAL ACID TRANSACTION GUARANTEES Atomicity All or nothing Write-ahead log Consistency Data always in valid state Constraints
  11. 11. 11CONFIDENTIAL ACID TRANSACTION GUARANTEES Atomicity All or nothing Write-ahead log Consistency Data always in valid state Constraints Isolation Visibility of concurrent actions Locking
  12. 12. 12CONFIDENTIAL ACID TRANSACTION GUARANTEES Consistency Data always in valid state Constraints Isolation Visibility of concurrent actions Locking Durability Changes become permanent Write-ahead log Atomicity All or nothing Write-ahead log
  13. 13. 13CONFIDENTIAL Transactions guarantee the data to always be in valid state
  14. 14. 14CONFIDENTIAL HOW IT WORKS?
  15. 15. 15CONFIDENTIAL TRAVEL BOOKING EXAMPLE
  16. 16. 16CONFIDENTIAL Book trip 1. Book hotel 2. Book flight 3. Hire rental car 4. Record booking
  17. 17. 17CONFIDENTIAL TRAVEL BOOKING SYSTEM DBBackend browser hotels flights cars bookings
  18. 18. 18CONFIDENTIAL TRAVEL BOOKING TRANSACTION hotels flights cars bookings DB Action Rollback action
  19. 19. 19CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin
  20. 20. 20CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null LOCK UPDATE
  21. 21. 21CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null LOCK UPDATE LOCK UPDATE
  22. 22. 22CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null LOCK UPDATE LOCK UPDATE LOCK UPDATE
  23. 23. 23CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null INSERT traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 DELETE traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 LOCK UPDATE LOCK UPDATE LOCK UPDATE LOCK UPDATE
  24. 24. 24CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null INSERT traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 DELETE traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 commit LOCK UPDATE LOCK UPDATE LOCK UPDATE LOCK UPDATE Commit
  25. 25. 25CONFIDENTIAL TRAVEL BOOKING TRANSACTION Begin hotels flights cars bookings DB UPDATE UPDATE UPDATE UPDATE Rollback Action Rollback action begin room_id=15 status=booked, traveler=Kostia room_id=15 status=free, traveler=null seat_id=17c status=booked, traveler=Kostia seat_id=17c status=free, traveler=null car_num=1234AA7 status=booked, traveler=Kostia car_num=1234AA7 status=free, traveler=null INSERT traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 DELETE traveler=Kostia, room_id=15, seat_id=17c, car_num=1234AA7 rollback LOCK LOCK LOCK LOCK
  26. 26. 26CONFIDENTIAL NOT ONLY IN DATABASES TRANSACTIONS ARE
  27. 27. 27CONFIDENTIAL • Isolation levels • Locking • Commit logs • Transactions – ONE and TWO phase
  28. 28. 28CONFIDENTIAL Cache Messages Transactions Subscriptions ACKs, … • Transacted sessions • JMS attributes – JMSXConsumerTXID – JMSXProducerTXID
  29. 29. 29CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
  30. 30. 30CONFIDENTIAL Application HTTP REST SOAP WE WANT THIS commit
  31. 31. 31CONFIDENTIAL Application HTTP REST SOAP OR THIS rollback
  32. 32. 32CONFIDENTIAL Application HTTP REST SOAP WHAT ACTUALLY HAPPENS commit fail commit fail
  33. 33. 33CONFIDENTIAL
  34. 34. 34CONFIDENTIAL
  35. 35. 35CONFIDENTIAL Application HTTP REST SOAP Coordinator
  36. 36. 36CONFIDENTIAL Application HTTP REST SOAP Coordinator
  37. 37. 37CONFIDENTIAL WEB-SERVICE AS STATE MACHINE /reserve /confirm /cancel reserved confirmed cancelled /reserve /confirm /cancel timeout 2 1 Idempotent operations
  38. 38. 38CONFIDENTIAL 2-PHASE COMMIT JTA/XA
  39. 39. 39CONFIDENTIAL 2-PHASE COMMIT ALGORITHM begin in progress commit rollback prepare All changes saved to log Changes copied to storage Changes cancelled 2 1
  40. 40. 40CONFIDENTIAL JTA AND XA SPECIFICATION Java application
  41. 41. 41CONFIDENTIAL Application server JTA AND XA SPECIFICATION Transaction Manager Java application
  42. 42. 42CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB JMS ? Java application JDBC CUSTOM
  43. 43. 43CONFIDENTIAL JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Application server Transaction Manager DB ? Java application begin begin begin
  44. 44. 44CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB ? Java application update update update
  45. 45. 45CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB ? Java application prepare prepare prepare
  46. 46. 46CONFIDENTIAL Application server JTA AND XA SPECIFICATION Resource manager Resource manager Resource manager Transaction Manager DB ? Java application commit commit commit
  47. 47. 47CONFIDENTIAL TM does all the job for us, but: • Needs too many messages • Doesn’t scale well • Not all vendors support XA
  48. 48. 48CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
  49. 49. 49CONFIDENTIAL WHAT IS SAGA?
  50. 50. 50CONFIDENTIAL SAGA EXAMPLE Book hotel Book flight Book car Cancel hotel Cancel flight Confirm booking Fail! Retry Retry failed Start Finish Skip
  51. 51. 51CONFIDENTIAL SAGA TYPES Central coordinator Routing slip (peer-to-peer)
  52. 52. 52CONFIDENTIAL SAGA TYPES Forward Backward
  53. 53. 53CONFIDENTIAL ACID Availability Consistency Isolation Durability BASE Basic Availability Soft state Eventual consistency (Trading consistency for availability) 2-PHASE COMMIT SAGA
  54. 54. 54CONFIDENTIAL System is partitioned CAP THEOREM CONSISTENCY AVAILABILITY Consistency and Availability System is single node
  55. 55. 55CONFIDENTIAL AGENDA Local transactions1 Distributed transactions2 Compensations: SAGA pattern3 Live demo4
  56. 56. 56CONFIDENTIAL Car service Frontend Tier 1 Backend Tier 2 Agency DB Hotels DB Flight service requests responsescache Distributed transaction JMS endpoint Remote call Local call JMS queue github.com/kslisenko/distributed-transactions
  57. 57. 57CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight service Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
  58. 58. 58CONFIDENTIAL Car service Frontend Tier 1 Backend cache Tier 2 /bookTrip Agency DB Hotels DB Flight service JMS requests responses Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
  59. 59. 59CONFIDENTIAL Car service Frontend Tier 1 Backend cache Tier 2 /bookTrip Agency DB Hotels DB Flight service requests responses JMS JMS Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
  60. 60. 60CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight service JMS JMS Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
  61. 61. 61CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight service JMS JMS JMS Distributed transaction JMS endpoint Remote call Local call JMS queue /getTrips github.com/kslisenko/distributed-transactions
  62. 62. 62CONFIDENTIAL Frontend Tier 1 Backend cache Tier 2 /bookTrip requests responses Car service Agency DB Hotels DB Flight serviceJMS JMS JMS JMS /getTrips Distributed transaction JMS endpoint Remote call Local call JMS queue github.com/kslisenko/distributed-transactions
  63. 63. 63CONFIDENTIAL LET’S GO!
  64. 64. 64CONFIDENTIAL CONCLUSION 1. State machines everywhere 2. Transactions everywhere 3. Design for failure 4. Rely on tools or handle by own
  65. 65. 65CONFIDENTIAL QUESTIONS? THANK YOU! KANSTANTSIN_SLISENKA@EPAM.COM

×