Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Two way data sync between legacy and your brand new micro-service architecture

Two way data sync between legacy and your brand new micro-service architecture

  • Identifiez-vous pour voir les commentaires

Two way data sync between legacy and your brand new micro-service architecture

  1. 1. Two way data sync between legacy and your brand new micro-service architecture
  2. 2. Brice LEPORINI 43 YO CTO PALO IT France Senior Developer @blep
  3. 3. SMALL ENOUGH TO CARE, BIG ENOUGH TO DELIVER Key figures & offices 350+ Experts Nationalities 30+ Turnover 30M Organic Growth +15% Debt 0 Held by Managers 100% Hong Kong Singapore Thailand Bangkok Mexico Mexico City Australia Sydney France Paris, Nancy, Nantes, Toulouse @blep
  4. 4. Replace a teenager monolith (15 YO) Micro-service oriented architecture Progressive migration The mission @blep
  5. 5. @blep
  6. 6. Microservices ? Why? Distinct life cycle for each service Ability to scale out specific service @blep
  7. 7. Gradual migration No big bang Strangler pattern: features / bounded contexts gradually replaced Canary releases @blep
  8. 8. The legacy and the micro-services have to run side by side. They can modify concurrently same data @blep
  9. 9. Schéma sync ms <-> legacy @blep
  10. 10. Conflictless ! @blep
  11. 11. Periods in the migration of each context ● The new context is not fully effective for everyone → the legacy system is the master of data ● The new context is battle tested → the corresponding features are no longer available in the legacy but it may need to be notified of changes for other features @blep
  12. 12. Remember how work databases @blep
  13. 13. You can’t change what happened. But you can add an event that cancels a previous one. Events are immutable What happens if a cancelation event is consumed before the event it cancels! Events are ordered Strong coupling with the snapshot will imply complex migration during application’s life. Beware of tight coupling your event model @blep
  14. 14. Entity state in MS environment = Last entity found in the snapshot + Events without validation @blep
  15. 15. Yes it is a kind of CQRS + ES usage CQRS = Command Query Responsability Segregation ES = Event Sourcing @blep
  16. 16. Legacy —> Microservices snapshot synchronization No interrogation allowed Minimal latency @blep
  17. 17. Legacy —> Microservices snapshot synchronization Business events , however : ● Huge legacy code base, missing events may occur ● High costs Database triggers : ● Run with each DML, high impact on performance and scalability @blep
  18. 18. CDC : Change Data Capture Logs every change that occurs in the database Nothing to change in the legacy application Technical options: Debezium, built in in MSSQL Technical events @blep
  19. 19. Consideration for CDC Usage in SQL Server One update on one row ⇒ two CDC rows with previous state and new state 1000 on one row ⇒ 2000 CDC rows… One bulk update … Note: ● Beware on the tables you activate CDC: high traffic tables need your attention … ○ Performance ○ File system filling ● CDC tables should be stored in other disks than business tables @blep
  20. 20. Retail example: order article count select count(*) from tbl_item where fkid_order=? Item count may be inferred from the sum of article prices compared to total order price @blep
  21. 21. What if one or more items come with 0 as price? @blep
  22. 22. This design is based on the eventual consistency principle which means that you may be inconsistent for a short moment (a few milliseconds?) but you will finally be consistent @blep
  23. 23. Keystone : no message can be lost Apache Kafka: ● Data streaming platform ● Messages are persisted and replicated ● Delivery order is kept by partition ● Distributed, fault tolerant and horizontally scalable @blep
  24. 24. @blep
  25. 25. Data path: changes coming from legacy @blep
  26. 26. Data path: changes coming from microservices @blep
  27. 27. Rollout procedure @blep

×