Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
@crichardson
Developing Microservices
with Aggregates
Chris Richardson
Founder of Eventuate.io
Founder of the original Clo...
@crichardson
Presentation goal
Show how
Domain Driven Design Aggregates
and Microservices
are a perfect match
@crichardson
About Chris
@crichardson
About Chris
Consultant and trainer
focusing on modern
application architectures
including microservices
(http...
@crichardson
About Chris
Founder of a startup that is creating
an open-source/SaaS platform
that simplifies the development...
@crichardson
For more information
http://learnmicroservices.io
@crichardson
Agenda
The problem with Domain Models and microservices
Overview of aggregates
Maintaining consistency betwee...
@crichardson
The Microservice architecture
tackles complexity through
modularization
@crichardson
Microservice
=
Business capability
@crichardson
Microservice architecture
Browser
Mobile
Device
Store
Front UI
API
Gateway
Catalog
Service
Review
Service
Ord...
@crichardson
But there are challenges…
@crichardson
Product service
Customer serviceOrder service
Domain model = tangled web of
classes
Order
OrderLine
Item
quan...
@crichardson
Reliance on ACID transactions
to enforce invariants
BEGIN TRANSACTION
…
SELECT ORDER_TOTAL
FROM ORDERS WHERE ...
@crichardson
But it violates encapsulation…
BEGIN TRANSACTION
…
SELECT ORDER_TOTAL
FROM ORDERS WHERE CUSTOMER_ID = ?
…
SEL...
@crichardson
.. and requires 2PC
BEGIN TRANSACTION
…
SELECT ORDER_TOTAL
FROM ORDERS WHERE CUSTOMER_ID = ?
…
SELECT CREDIT_...
@crichardson
2PC is not an option
Guarantees consistency
BUT
2PC coordinator is a single point of failure
Chatty: at least...
@crichardson
Doesn’t fit with the
NoSQL DB “transaction” model
@crichardson
Agenda
The problem with Domain Models and microservices
Overview of aggregates
Maintaining consistency betwee...
@crichardson
Domain Driven Design -
building blocks
Entity
Value object
Services
Repositories
Aggregates
Adopted
Ignored
(...
About Aggregates
Cluster of objects that can
be treated as a unit
Graph consisting of a root
entity and one or more
other ...
Aggregate: rule #1
Reference other
aggregate roots via
identity
(primary key)
Order
OrderLine
Item
quantity
productId
prod...
@crichardson
Foreign keys in a
Domain Model?!?
@crichardson
Domain model = collection of
loosely connected aggregates
Order
OrderLine
Item
quantity
…
Address
street
city...
@crichardson
Product service
Customer serviceOrder service
Easily partition into microservices
Order
OrderLine
Item
quanti...
@crichardson
Aggregate rule #2
Transaction
=
processing one command by
one aggregate
@crichardson
Product service
Customer serviceOrder service
Transaction scope = service
Order
OrderLine
Item
quantity
…
Add...
@crichardson
Transaction scope
=
NoSQL database
“transaction”
@crichardson
Agenda
The problem with Domain Models and microservices
Overview of aggregates
Maintaining consistency betwee...
@crichardson
Customer management
How to maintain data consistency
between aggregates?
Order management
Order Service
place...
@crichardson
Saga
Using event-driven Sagas instead of
2PC
Distributed transaction
Order Customer
Local transaction
Order
L...
@crichardson
Order Service
Saga-based, eventually consistent
order processing
Customer Service
Order created
Credit Reserv...
@crichardson
Order Service
Order
id : 4567
total: 343
state = CREATED
Customer Service
Customer
creditLimit : 12000
credit...
@crichardson
Complexity of compensating
transactions
ACID transactions can simply rollback
BUT
Developer must write applic...
@crichardson
How atomically update
database and publish an event
Order Service
Order
Database
Message Broker
insert Order
...
@crichardson
Failure = inconsistent system
Order Service
Order
Database
Message Broker
insert Order
publish
OrderCreatedEv...
@crichardson
2PC is not an option
@crichardson
Reliably publish events when
state changes
@crichardson
Agenda
The problem with Domain Models and microservices
Overview of aggregates
Maintaining consistency betwee...
@crichardson
Event sourcing = event-centric
persistence
Application
Database
Event store
update
publish
X
@crichardson
Event sourcing
For each DDD aggregate:
Identify (state changing) domain events
e.g. use Event Storming
Define ...
@crichardson
Persists events
NOT current state
Order
status
….
101 ACCEPTED
Order table
X
@crichardson
Event-centric schema
@crichardson
Persists events
NOT current state
Event table
Entity type
Event
id
Entity
id
Event
data
Order 902101 …OrderAp...
@crichardson
Replay events to recreate
state
Order
state
OrderCreated(…)
OrderAccepted(…)
OrderShipped(…)
Events
Periodica...
@crichardson
The present is a fold over
history
currentState = foldl(applyEvent, initialState, events)
@crichardson
Event Store
Application architecture
Order 123 Customer 456
OrderCreated
OrderApproved
…
CustomerCreated
Cust...
@crichardson
Request handling in an event sourced application
HTTP
Handler
Event
Store
pastEvents = findEvents(orderId)
Ord...
@crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
pu...
@crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
pu...
Event store = database + message
broker
Home grown/DIY
.NET geteventstore.com
https://
www.lightbend.com/
lagom
http://eve...
@crichardson
Benefits of event sourcing
Solves data consistency issues in a Microservice/NoSQL based
architecture
Reliable ...
@crichardson
Drawbacks of event
sourcing…
Requires application rewrite
Weird and unfamiliar style of programming
Events = ...
@crichardson
… Drawbacks of event
sourcing
Querying the event store can be challenging
Some queries might be complex/ineffi...
@crichardson
Agenda
The problem with Domain Models and microservices
Overview of aggregates
Maintaining consistency betwee...
@crichardson
Orders and Customers
example
Kafka
Rest API
Event Store
Event
Store
Adapter
Customer
Service
Rest API
Order S...
@crichardson
Customer microservice
Customer
Service
Customer
Controller
Spring
Cloud
Stream
Kafka
Customer
Aggregate
Rest ...
@crichardson
The Customer aggregate
Money creditLimit
Map<OrderId, Money> creditReservations
Customer
List<Event> process(...
@crichardson
Familiar concepts restructured
class Customer {
public void reserveCredit(
orderId : String,
amount : Money) ...
@crichardson
Customer command processing
@crichardson
Customer applying events
@crichardson
Customer Controller
@crichardson
Creating a Customer
save() concisely specifies:
1.Creates Customer aggregate
2.Processes command
3.Applies eve...
@crichardson
Event handling in Customers
Durable subscription name
1. Load Customer aggregate
2. Processes command
3. Appl...
@crichardson
Event handler in Orders
@crichardson
Agenda
The problem with Domain Models and microservices
Overview of aggregates
Maintaining consistency betwee...
@crichardson
Order Service
Order Service ⬄ Customer Service:
tightly coupled, cyclic dependency 😢
Customer Service
Order c...
@crichardson
Order Service
Decouple by maintaining available
credit in OrderService
Customer Service
Available Credit Chan...
Use an explicit saga
Saga object that orchestrates the
flow
Created in response to an initial
event, e.g. OrderCreated
A st...
@crichardson
Customer
Service
Order Service
Using an OrderCreationSaga
Order
state
total
…
AvailableCredit
Tracker
creditL...
@crichardson
Order saga state machine
NEW
SUCCEEDED
FAILED
OrderCreatedEvent
/ reserveCredit()
CustomerCreditReservedEvent...
@crichardson
Saga as a singleton state
machine A state machine state
e.g. enum { NEW, …}
A data object
@crichardson
Event processing
(State + Data, Event)
(State’+ Data’, Commands)
@crichardson
(OrderSagaState, OrderSagaData, Event)
(Commands, OrderSagaState’, OrderSagaData’)
@crichardson
OrderCreationSaga state machine
definition
event saga id
starting + state transitions
@crichardson
Initial action Triggers saga
creation
1. Send command
2. Transition to new state
3. Create data
@crichardson
Event handler
Triggering
event
1. Send command
2. Transition to new state
@crichardson
Kafka-based command bus
Order
Saga
Credit
Tracking
Aggregate
CreditTrackingCommand Topic
OrderSagaCommandFail...
@crichardson
Summary
Aggregates are the building blocks of microservices
Use event-driven sagas to maintain consistency be...
@crichardson
@crichardson chris@chrisrichardson.net
http://learnmicroservices.io
Questions?
Prochain SlideShare
Chargement dans…5
×

Developing microservices with aggregates (devnexus2017)

2 266 vues

Publié le

he Domain Model pattern is a great way to develop complex business logic. Unfortunately, a typical domain model is a tangled, birds nest of classes. It can’t be decomposed into microservices. Moreover, business logic often relies on ACID transactions to maintain consistency. Fortunately, there is a solution to this problem: aggregates.

An aggregate is an often overlooked modeling concept from the must read book Domain Driven Design. In this talk you will learn how aggregates enable you to develop business logic for the modern world of microservices and NoSQL. We will describe how to use aggregates to design modular business logic that can be partitioned into microservices. You will learn how aggregates enable you to use eventual consistency instead of ACID. We will describe the design of a microservice that is built using aggregates, and Spring Cloud.

Publié dans : Logiciels
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Awesome lonely girl looking for fun on webcam with the you now - www.xslideshare.usa.cc
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Developing microservices with aggregates (devnexus2017)

  1. 1. @crichardson Developing Microservices with Aggregates Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://eventuate.io
  2. 2. @crichardson Presentation goal Show how Domain Driven Design Aggregates and Microservices are a perfect match
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Consultant and trainer focusing on modern application architectures including microservices (http://www.chrisrichardson.net/)
  5. 5. @crichardson About Chris Founder of a startup that is creating an open-source/SaaS platform that simplifies the development of transactional microservices (http://eventuate.io)
  6. 6. @crichardson For more information http://learnmicroservices.io
  7. 7. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
  8. 8. @crichardson The Microservice architecture tackles complexity through modularization
  9. 9. @crichardson Microservice = Business capability
  10. 10. @crichardson Microservice architecture Browser Mobile Device Store Front UI API Gateway Catalog Service Review Service Order Service … Service Catalog Database Review Database Order Database … Database HTML REST REST Database per service
  11. 11. @crichardson But there are challenges…
  12. 12. @crichardson Product service Customer serviceOrder service Domain model = tangled web of classes Order OrderLine Item quantity … Address street city … Customer Product name price ? ? creditLimit
  13. 13. @crichardson Reliance on ACID transactions to enforce invariants BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Simple and ACID
  14. 14. @crichardson But it violates encapsulation… BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION Private to the Order Service Private to the Customer Service
  15. 15. @crichardson .. and requires 2PC BEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION
  16. 16. @crichardson 2PC is not an option Guarantees consistency BUT 2PC coordinator is a single point of failure Chatty: at least O(4n) messages, with retries O(n^2) Reduced throughput due to locks Not supported by many NoSQL databases (or message brokers) CAP theorem 2PC impacts availability ….
  17. 17. @crichardson Doesn’t fit with the NoSQL DB “transaction” model
  18. 18. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
  19. 19. @crichardson Domain Driven Design - building blocks Entity Value object Services Repositories Aggregates Adopted Ignored (at least by me)
  20. 20. About Aggregates Cluster of objects that can be treated as a unit Graph consisting of a root entity and one or more other entities and value objects Typically business entities are Aggregates, e.g. customer, Account, Order, Product, …. Order OrderLine Item quantity productId productName productPrice customerId Address street city …
  21. 21. Aggregate: rule #1 Reference other aggregate roots via identity (primary key) Order OrderLine Item quantity productId productName productPrice customerId Address street city …
  22. 22. @crichardson Foreign keys in a Domain Model?!?
  23. 23. @crichardson Domain model = collection of loosely connected aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
  24. 24. @crichardson Product service Customer serviceOrder service Easily partition into microservices Order OrderLine Item quantity … Address street city … Customer Product name price
  25. 25. @crichardson Aggregate rule #2 Transaction = processing one command by one aggregate
  26. 26. @crichardson Product service Customer serviceOrder service Transaction scope = service Order OrderLine Item quantity … Address street city … Customer Product name price
  27. 27. @crichardson Transaction scope = NoSQL database “transaction”
  28. 28. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
  29. 29. @crichardson Customer management How to maintain data consistency between aggregates? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
  30. 30. @crichardson Saga Using event-driven Sagas instead of 2PC Distributed transaction Order Customer Local transaction Order Local transaction Customer Local transaction Order Event Event
  31. 31. @crichardson Order Service Saga-based, eventually consistent order processing Customer Service Order created Credit Reserved Credit Check Failed Create Order OR Customer creditLimit creditReservations ... Order state total … approve()/reject() Event Handler Event Handler reserveCredit()
  32. 32. @crichardson Order Service Order id : 4567 total: 343 state = CREATED Customer Service Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = APPROVED Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
  33. 33. @crichardson Complexity of compensating transactions ACID transactions can simply rollback BUT Developer must write application logic to “rollback” eventually consistent transactions For example: CreditCheckFailed => cancel Order Money transfer destination account closed => credit source account Careful design required!
  34. 34. @crichardson How atomically update database and publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
  35. 35. @crichardson Failure = inconsistent system Order Service Order Database Message Broker insert Order publish OrderCreatedEvent X
  36. 36. @crichardson 2PC is not an option
  37. 37. @crichardson Reliably publish events when state changes
  38. 38. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
  39. 39. @crichardson Event sourcing = event-centric persistence Application Database Event store update publish X
  40. 40. @crichardson Event sourcing For each DDD aggregate: Identify (state changing) domain events e.g. use Event Storming Define Event classes For example, Order events: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped
  41. 41. @crichardson Persists events NOT current state Order status …. 101 ACCEPTED Order table X
  42. 42. @crichardson Event-centric schema
  43. 43. @crichardson Persists events NOT current state Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated
  44. 44. @crichardson Replay events to recreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
  45. 45. @crichardson The present is a fold over history currentState = foldl(applyEvent, initialState, events)
  46. 46. @crichardson Event Store Application architecture Order 123 Customer 456 OrderCreated OrderApproved … CustomerCreated CustomerCreditReserved … CreateOrder UpdateOrder GetOrder Subscribe Order Service CreateCustomer UpdateCustomer GetCustomer Subscribe Customer Service
  47. 47. @crichardson Request handling in an event sourced application HTTP Handler Event Store pastEvents = findEvents(orderId) Order new() applyEvents(pastEvents) newEvents = processCmd(someCmd) saveEvents(newEvents) (optimistic locking) Order Service applyEvents(newEvents)
  48. 48. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Customer update() Customer Service
  49. 49. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) CQRS View update() Service Xyz send notifications …
  50. 50. Event store = database + message broker Home grown/DIY .NET geteventstore.com https:// www.lightbend.com/ lagom http://eventuate.io (mine) … Event Store Save aggregate events Get aggregate events Subscribe to events
  51. 51. @crichardson Benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built in, reliable audit log temporal queries Preserved history More easily implement future requirements
  52. 52. @crichardson Drawbacks of event sourcing… Requires application rewrite Weird and unfamiliar style of programming Events = a historical record of your bad design decisions Must detect and ignore duplicate events Idempotent event handlers Track most recent event and ignore older ones …
  53. 53. @crichardson … Drawbacks of event sourcing Querying the event store can be challenging Some queries might be complex/inefficient, e.g. accounts with a balance > X Event store might only support lookup of events by entity id Must use Command Query Responsibility Segregation (CQRS) to handle queries application must handle eventually consistent data
  54. 54. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
  55. 55. @crichardson Orders and Customers example Kafka Rest API Event Store Event Store Adapter Customer Service Rest API Order Service Customer Aggregate Order Aggregate Rest API Order History Service MongoDB CQRS View Event Store Adapter Event Store Adapter Used to notify Order Service of invalid customer ids!
  56. 56. @crichardson Customer microservice Customer Service Customer Controller Spring Cloud Stream Kafka Customer Aggregate Rest API Customer Event Handlers Event Store Event Store Adapter Order*Event POST /customers
  57. 57. @crichardson The Customer aggregate Money creditLimit Map<OrderId, Money> creditReservations Customer List<Event> process(CreateCustomerCommand cmd) { … } List<Event> process(ReserveCreditCommand cmd) { … } … void apply(CustomerCreatedEvent anEvent) { … } void apply(CreditReservedEvent anEvent) { … } … State Behavior
  58. 58. @crichardson Familiar concepts restructured class Customer { public void reserveCredit( orderId : String, amount : Money) { // verify // update state this.xyz = … } public List<Event> process( ReserveCreditCommand cmd) { // verify … return singletonList( new CreditReservedEvent(…); ) } public void apply( CreditReservedEvent event) { // update state this.xyz = event.xyz }
  59. 59. @crichardson Customer command processing
  60. 60. @crichardson Customer applying events
  61. 61. @crichardson Customer Controller
  62. 62. @crichardson Creating a Customer save() concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events
  63. 63. @crichardson Event handling in Customers Durable subscription name 1. Load Customer aggregate 2. Processes command 3. Applies events 4. Persists events Triggers BeanPostProcessor Publish message to Spring Cloud Stream (Kafka) when Customer not found
  64. 64. @crichardson Event handler in Orders
  65. 65. @crichardson Agenda The problem with Domain Models and microservices Overview of aggregates Maintaining consistency between aggregates Using event sourcing with Aggregates Customers and Orders example Using explicit saga orchestration
  66. 66. @crichardson Order Service Order Service ⬄ Customer Service: tightly coupled, cyclic dependency 😢 Customer Service Order created Credit Reserved Credit Check Failed OR Must subscribe to all Order events that affect available credit! Order …
  67. 67. @crichardson Order Service Decouple by maintaining available credit in OrderService Customer Service Available Credit Changed Customer Created Customer creditLimit availableCredit ... Order state total … AvailableCredit Tracker creditLimit availableCredit ... Simpler 😄 Still cyclic 😢
  68. 68. Use an explicit saga Saga object that orchestrates the flow Created in response to an initial event, e.g. OrderCreated A state machine Receives events from aggregates Sends commands to aggregates Implemented as an event-sourced aggregate Saga Event Command
  69. 69. @crichardson Customer Service Order Service Using an OrderCreationSaga Order state total … AvailableCredit Tracker creditLimit availableCredit ... Order Creation Saga create() create() Reserve Credit Credit Reserved Approve Order Customer Update Available Credit OrderCreated Acyclic 😄
  70. 70. @crichardson Order saga state machine NEW SUCCEEDED FAILED OrderCreatedEvent / reserveCredit() CustomerCreditReservedEvent / approveOrder() EntityNotFoundCommandExecutionError CustomerCreditLimitExceededEvent / rejectOrder()
  71. 71. @crichardson Saga as a singleton state machine A state machine state e.g. enum { NEW, …} A data object
  72. 72. @crichardson Event processing (State + Data, Event) (State’+ Data’, Commands)
  73. 73. @crichardson (OrderSagaState, OrderSagaData, Event) (Commands, OrderSagaState’, OrderSagaData’)
  74. 74. @crichardson OrderCreationSaga state machine definition event saga id starting + state transitions
  75. 75. @crichardson Initial action Triggers saga creation 1. Send command 2. Transition to new state 3. Create data
  76. 76. @crichardson Event handler Triggering event 1. Send command 2. Transition to new state
  77. 77. @crichardson Kafka-based command bus Order Saga Credit Tracking Aggregate CreditTrackingCommand Topic OrderSagaCommandFailure Topic Reserve Credit CustomerNotFound Per aggregate type Per saga type
  78. 78. @crichardson Summary Aggregates are the building blocks of microservices Use event-driven sagas to maintain consistency between aggregates Event sourcing is a good way to implement a event-driven architecture Use explicit sagas to orchestrate transactions
  79. 79. @crichardson @crichardson chris@chrisrichardson.net http://learnmicroservices.io Questions?

×