Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
@crichardson
Events on the outside, on
the inside and at the core
Chris Richardson
Founder of Eventuate.io
Founder of the ...
@crichardson
Presentation goal
Events play a key role in
applications
Event sourcing enables
the event-driven enterprise
@crichardson
About Chris
@crichardson
About Chris
Consultant and
trainer focusing on
microservices
http://www.chrisrichardson.net/
@crichardson
About Chris
Founder of a startup that is
creating
a platform that makes it easy for
application developers to...
@crichardson
For more information
https://github.com/cer/event-sourcing-examples
http://microservices.io
http://plainoldob...
@crichardson
Agenda
Events on the outside
Events on the inside
Events at the core with event sourcing
Designing event-cent...
@crichardson
Events on the outside
Application A
Application B
Application C
Event X
Event Y
Event Z
@crichardson
What’s an event?
http://www.merriam-webster.com/dictionary/event
@crichardson
Examples of events
@crichardson
Instacart:
event-
driven
grocery
shopping
@crichardson
An event is produced when…
Creation or update of a business object
Attempt to violate a business rule
@crichardson
Who consumes an event?
Order
Application
Human
Notification
Service
Dashboard/
Monitoring
Shipping
Application...
@crichardson
How to deliver events?
@crichardson
Polling for events
HTTP
Periodically poll for events
Atom Publishing Protocol (AtomPub)
Based on HTTP
Head is...
@crichardson
Using WebSockets
Browser
Web
Socket
STOMP
Client
application
Service
Web
Socket
STOMP
Application
SUBSCRIBE
M...
@crichardson
Webhooks = user-defined
HTTP callback
Client Service
register(events, callbackUrl)
POST callbackUrl
POST callb...
@crichardson
Github webhooks
https://developer.github.com/webhooks/
Installed on an organization or repository
e.g. POST /...
@crichardson
Twilio - Telephony and SMS as
a service
Twilio Your
Application
TwiML doc
HTTP GET/
POST
REST API
Manage reso...
@crichardson
Integration hubs - Zapier,
IFTTT
Application abstraction:
Triggers - events published by application: polling...
@crichardson
WebHooks
=
web friendly publish/subscribe
@crichardson
Enterprise integration patterns
http://www.enterpriseintegrationpatterns.com/patterns/messaging/
@crichardson
messaging system
Messaging-based IPC
Sender RecipientChannel
message
@crichardson
Example messaging systems
@crichardson
The event-driven enterprise
App A App BApp X
App Y
Firewall
Message Broker
webhooks
WebSockets
AtomPub
App Z ...
@crichardson
Agenda
Events on the outside
Events on the inside
Events at the core with event sourcing
Designing event-cent...
@crichardson
Events on the inside
Application
Service A
Service B
Service C
Event X
Event YEvent Z
Event X
Event Y
@crichardson
Traditional monolithic architecture
Simple to
develop
test
deploy
scale
Shopping
Cart
Customers
Orders
Mobile...
@crichardson
But that leads* to
monolithic hell
For large and/or complex applications…
@crichardson
Today: use a microservice, polyglot
architecture
Orders
Customers
…
Shopping UI
Mobile Client
Browser
API Gat...
@crichardson
But now we have
distributed data management
problems
@crichardson
Example: placing an order
Order Service Customer Service
Order
Database
Customer Database
Order #1
Customer #...
@crichardson
Customer management
How to maintain invariants?
Order management
Order Service
placeOrder()
Customer Service
...
@crichardson
Use an event-driven
architecture
Services publish events when something important happens,
e.g. state changes...
@crichardson
Event-driven application architecture
Application
Service A Service B
Message Broker
Service C
Msg
Broker
Cli...
@crichardson
Order Management
Order
id : 4567
total: 343
state = CREATED
Customer Management
Customer
creditLimit : 12000
...
@crichardson
Now there are two problems
to solve….
@crichardson
Problem #1: How to design
eventually consistent business logic?
More on that later….
@crichardson
Problem #2: How to atomically update
database and publish an event
Order Service
Order
Database
Message Broke...
@crichardson
Failure = inconsistent system
Order Service
Order
Database
Message Broker
insert Order
publish
OrderCreatedEv...
@crichardson
Two-phase commit
X
@crichardson
Agenda
Events on the outside
Events on the inside
Events at the core with event sourcing
Designing event-cent...
@crichardson
Just publish events
Application
Database
Message Broker
update
publish
X
@crichardson
Event sourcing
For each aggregate (business entity):
Identify (state-changing) domain events
Define Event clas...
@crichardson
Persists events
NOT current state
Order
status
….
101 ACCEPTED
Order table
X
@crichardson
Persists events
NOT current state
Event table
Entity type
Event
id
Entity
id
Event
data
Order 902101 …OrderAp...
@crichardson
Replay events to recreate
state
Order
state
OrderCreated(…)
OrderAccepted(…)
OrderShipped(…)
Events
Periodica...
@crichardson
The present is a fold over
history
currentState = foldl(applyEvent, initialState, events)
@crichardson
Events at the core
Service
Aggregate A
Aggregate B
Aggregate C
Event X
Event YEvent Z
Event Z
Event Y
@crichardson
Domain logic = event-driven
aggregates
Customer
AggregateOrder
Aggregate
Order Created
Event Check Credit
Com...
@crichardson
Request handling in an event sourced application
HTTP
Handler
Event
Store
pastEvents = findEvents(entityId)
Or...
@crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
pu...
@crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
pu...
Event store = database + message
broker
Hybrid database and
message broker
Implementations:
Home-grown/DIY
geteventstore.c...
@crichardson
Benefits of event sourcing
Solves data consistency issues in a Microservice/NoSQL based
architecture
Reliable ...
@crichardson
Drawbacks of event
sourcing…
Requires application rewrite
Weird and unfamiliar style of programming
Events = ...
@crichardson
… Drawbacks of event
sourcing
Querying the event store can be challenging
Some queries might be complex/ineffi...
@crichardson
Agenda
Events on the outside
Events on the inside
Events at the core with event sourcing
Designing event-cent...
@crichardson
Use the familiar building
blocks of DDD
Entity
Value object
Services
Repositories
Aggregates ⟸ essential
About Aggregates
Graph consisting of a root
entity and one or more other
entities and value objects
Each core business ent...
@crichardson
Domain model = collection of
loosely connected aggregates
Order
OrderLine
Item
quantity
…
Address
street
city...
@crichardson
Product service
Customer serviceOrder service
Easily partition into microservices
Order
OrderLine
Item
quanti...
@crichardson
Transaction = processing one
command by one aggregate
No opportunity to update multiple aggregates within a
t...
@crichardson
Product service
Customer serviceOrder service
Transaction scope = service
Order
OrderLine
Item
quantity
…
Add...
@crichardson
Aggregate granularity
Consistency
Scalability/
User experience
Customer
Order
Product
Customer
Order
Product
...
Designing domain events
Record state changes for an
aggregate
Part of the public API of the domain
model ProductAddedToCar...
@crichardson
Example event
@crichardson
Designing commands
Created by a service from incoming request
Processed by an aggregate
Immutable
Contains va...
@crichardson
Example command
@crichardson
Various programming models
“Traditional Java” mutable object-oriented domain objects
https://github.com/cer/e...
@crichardson
OO = State + Behavior
creditLimit
creditReservations : Map<OrderId, Money>
Customer
List<Event> process(Creat...
@crichardson
Familiar concepts restructured
class Customer {
public void reserveCredit(
orderId : String,
amount : Money) ...
@crichardson
Kanban board example
@crichardson
Architecture
Create/update boards
and tasks
Change notifications
Materialized
views
Event Store
@crichardson
Demo
@crichardson
Summary
Events are a central to modern applications
Events integrate applications
Events maintain data consis...
@crichardson
@crichardson chris@chrisrichardson.net
http://eventuate.io
http://plainoldobjects.com http://microservices.io...
Prochain SlideShare
Chargement dans…5
×

Events on the outside, on the inside and at the core (jfokus jfokus2016)

7 246 vues

Publié le

This is the talk I gave at JFokus 2016 on event-driven microservices.

This presentation looks at the importance of events and the role that they play in applications. We describe how events are a key application integration mechanism and how they are used by applications to communicate with the outside world. You will learn how the microservices inside a system can use events to maintain data consistency. We discuss how easy it is to implement both of these mechanisms by developing your core business logic using an event-centric approach known as event sourcing.

Publié dans : Logiciels

Events on the outside, on the inside and at the core (jfokus jfokus2016)

  1. 1. @crichardson Events on the outside, on the inside and at the core Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://microservices.io http://eventuate.io http://plainoldobjects.com
  2. 2. @crichardson Presentation goal Events play a key role in applications Event sourcing enables the event-driven enterprise
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Consultant and trainer focusing on microservices http://www.chrisrichardson.net/
  5. 5. @crichardson About Chris Founder of a startup that is creating a platform that makes it easy for application developers to write microservices http://eventuate.io
  6. 6. @crichardson For more information https://github.com/cer/event-sourcing-examples http://microservices.io http://plainoldobjects.com/ https://twitter.com/crichardson http://eventuate.io/
  7. 7. @crichardson Agenda Events on the outside Events on the inside Events at the core with event sourcing Designing event-centric domain model
  8. 8. @crichardson Events on the outside Application A Application B Application C Event X Event Y Event Z
  9. 9. @crichardson What’s an event? http://www.merriam-webster.com/dictionary/event
  10. 10. @crichardson Examples of events
  11. 11. @crichardson Instacart: event- driven grocery shopping
  12. 12. @crichardson An event is produced when… Creation or update of a business object Attempt to violate a business rule
  13. 13. @crichardson Who consumes an event? Order Application Human Notification Service Dashboard/ Monitoring Shipping Application OrderCreated Email SMS …
  14. 14. @crichardson How to deliver events?
  15. 15. @crichardson Polling for events HTTP Periodically poll for events Atom Publishing Protocol (AtomPub) Based on HTTP Head is constantly changing Tail is immutable and can be efficiently cached High-latency, inefficient
  16. 16. @crichardson Using WebSockets Browser Web Socket STOMP Client application Service Web Socket STOMP Application SUBSCRIBE MESSAGE MESSAGE Low latency, more efficient, but what about past events?
  17. 17. @crichardson Webhooks = user-defined HTTP callback Client Service register(events, callbackUrl) POST callbackUrl POST callbackUrl … https://en.wikipedia.org/wiki/Webhook Low latency, more efficient, but what about past events?
  18. 18. @crichardson Github webhooks https://developer.github.com/webhooks/ Installed on an organization or repository e.g. POST /repos/:owner/:repo/hooks Available events: push - push to a repository fork - repository is forked pull_request - assigned, unassigned, … push - push to a repository …
  19. 19. @crichardson Twilio - Telephony and SMS as a service Twilio Your Application TwiML doc HTTP GET/ POST REST API Manage resources Send SMS Initiate voice calls Webhooks handle incoming SMS and voice calls Voice SMS Phone number SMS URL +VOICE URL
  20. 20. @crichardson Integration hubs - Zapier, IFTTT Application abstraction: Triggers - events published by application: polling or Webhooks Action - operation supported by application, e.g. REST API end points
  21. 21. @crichardson WebHooks = web friendly publish/subscribe
  22. 22. @crichardson Enterprise integration patterns http://www.enterpriseintegrationpatterns.com/patterns/messaging/
  23. 23. @crichardson messaging system Messaging-based IPC Sender RecipientChannel message
  24. 24. @crichardson Example messaging systems
  25. 25. @crichardson The event-driven enterprise App A App BApp X App Y Firewall Message Broker webhooks WebSockets AtomPub App Z App C Msg Broker Client Msg Broker Client Msg Broker Client App D Msg Broker Client Inside the firewallOutside the firewall
  26. 26. @crichardson Agenda Events on the outside Events on the inside Events at the core with event sourcing Designing event-centric domain model
  27. 27. @crichardson Events on the inside Application Service A Service B Service C Event X Event YEvent Z Event X Event Y
  28. 28. @crichardson Traditional monolithic architecture Simple to develop test deploy scale Shopping Cart Customers Orders Mobile client REST API Browser Web UI Message Broker Adapter MySQL Adapter RDBMS … Message Broker
  29. 29. @crichardson But that leads* to monolithic hell For large and/or complex applications…
  30. 30. @crichardson Today: use a microservice, polyglot architecture Orders Customers … Shopping UI Mobile Client Browser API Gateway Order management REST API Customer Management REST API …. REST API Order Database (MongoDB) MongoDB Adapter Customer Database (Sharded MySQL) MySQL Adapter
  31. 31. @crichardson But now we have distributed data management problems
  32. 32. @crichardson Example: placing an order Order Service Customer Service Order Database Customer Database Order #1 Customer #1 No 2PC No ACID NoSQL SQL
  33. 33. @crichardson Customer management How to maintain invariants? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
  34. 34. @crichardson Use an event-driven architecture Services publish events when something important happens, e.g. state changes Services subscribe to events and update their state Maintain eventual consistency across multiple aggregates (in multiple datastores) Synchronize replicated data
  35. 35. @crichardson Event-driven application architecture Application Service A Service B Message Broker Service C Msg Broker Client Msg Broker Client Msg Broker Client Service D Msg Broker Client
  36. 36. @crichardson Order Management Order id : 4567 total: 343 state = CREATED Customer Management Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = OPEN Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
  37. 37. @crichardson Now there are two problems to solve….
  38. 38. @crichardson Problem #1: How to design eventually consistent business logic? More on that later….
  39. 39. @crichardson Problem #2: How to atomically update database and publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
  40. 40. @crichardson Failure = inconsistent system Order Service Order Database Message Broker insert Order publish OrderCreatedEvent X
  41. 41. @crichardson Two-phase commit X
  42. 42. @crichardson Agenda Events on the outside Events on the inside Events at the core with event sourcing Designing event-centric domain model
  43. 43. @crichardson Just publish events Application Database Message Broker update publish X
  44. 44. @crichardson Event sourcing For each aggregate (business entity): Identify (state-changing) domain events Define Event classes For example, ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent Order: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped
  45. 45. @crichardson Persists events NOT current state Order status …. 101 ACCEPTED Order table X
  46. 46. @crichardson Persists events NOT current state Event table Entity type Event id Entity id Event data Order 902101 …OrderApproved Order 903101 …OrderShipped Event type Order 901101 …OrderCreated
  47. 47. @crichardson Replay events to recreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
  48. 48. @crichardson The present is a fold over history currentState = foldl(applyEvent, initialState, events)
  49. 49. @crichardson Events at the core Service Aggregate A Aggregate B Aggregate C Event X Event YEvent Z Event Z Event Y
  50. 50. @crichardson Domain logic = event-driven aggregates Customer AggregateOrder Aggregate Order Created Event Check Credit Command Credit Reserved Event Approve Order Command Create Order Command External request Emits events Event Command
  51. 51. @crichardson Request handling in an event sourced application HTTP Handler Event Store pastEvents = findEvents(entityId) Order new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) (optimistic locking) Order Service apply(newEvents)
  52. 52. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Customer update() Customer Service
  53. 53. @crichardson Event Store publishes events consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) CQRS View update() Service Xyz send notifications …
  54. 54. Event store = database + message broker Hybrid database and message broker Implementations: Home-grown/DIY geteventstore.com by Greg Young http://eventuate.io (mine) Event Store Save aggregate events Get aggregate events Subscribe to events
  55. 55. @crichardson Benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built in, reliable audit log, temporal queries Preserved history More easily implement future requirements
  56. 56. @crichardson Drawbacks of event sourcing… Requires application rewrite Weird and unfamiliar style of programming Events = a historical record of your bad design decisions Must detect and ignore duplicate events Idempotent event handlers Track most recent event and ignore older ones …
  57. 57. @crichardson … Drawbacks of event sourcing Querying the event store can be challenging Some queries might be complex/inefficient, e.g. accounts with a balance > X Event store might only support lookup of events by entity id Must use Command Query Responsibility Segregation (CQRS) to handle queries application must handle eventually consistent data
  58. 58. @crichardson Agenda Events on the outside Events on the inside Events at the core with event sourcing Designing event-centric domain model
  59. 59. @crichardson Use the familiar building blocks of DDD Entity Value object Services Repositories Aggregates ⟸ essential
  60. 60. About Aggregates Graph consisting of a root entity and one or more other entities and value objects Each core business entity = Aggregate: e.g. customer, Account, Order, Product, …. Reference other aggregate roots via primary key Often contains partial copy of other aggregates’ data Order OrderLine Item quantity productId productName productPrice customerId Address street city …
  61. 61. @crichardson Domain model = collection of loosely connected aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
  62. 62. @crichardson Product service Customer serviceOrder service Easily partition into microservices Order OrderLine Item quantity … Address street city … Customer Product name price
  63. 63. @crichardson Transaction = processing one command by one aggregate No opportunity to update multiple aggregates within a transaction event driven eventual consistency between aggregates If an update must be atomic (i.e. no compensating transaction) then it must be handled by a single aggregate Therefore, aggregate granularity is important
  64. 64. @crichardson Product service Customer serviceOrder service Transaction scope = service Order OrderLine Item quantity … Address street city … Customer Product name price
  65. 65. @crichardson Aggregate granularity Consistency Scalability/ User experience Customer Order Product Customer Order Product Customer Order Product
  66. 66. Designing domain events Record state changes for an aggregate Part of the public API of the domain model ProductAddedToCart id : TimeUUID senderId: UUID productId productName productPrice … Required by aggregate Enrichment: Useful for consumers Event metadata
  67. 67. @crichardson Example event
  68. 68. @crichardson Designing commands Created by a service from incoming request Processed by an aggregate Immutable Contains value objects for Validating request Creating event Auditing user activity
  69. 69. @crichardson Example command
  70. 70. @crichardson Various programming models “Traditional Java” mutable object-oriented domain objects https://github.com/cer/event-sourcing-examples/tree/master/ java-spring Functional Scala with immutable domain objects https://github.com/cer/event-sourcing-using-scala-typeclasses Hybrid OO/Functional Scala with immutable domain objects https://github.com/cer/event-sourcing-examples/tree/master/ scala-spring
  71. 71. @crichardson OO = State + Behavior creditLimit creditReservations : Map<OrderId, Money> Customer List<Event> process(CreateCustomerCommand cmd) { … } List<Event> process(ReserveCreditCommand cmd) { … } … void apply(CustomerCreatedEvent anEvent) { … } void apply(CreditServedEvent anEvent) { … } … State Behavior
  72. 72. @crichardson Familiar concepts restructured class Customer { public void reserveCredit( orderId : String, amount : Money) { // verify // update state this.xyz = … } public List<Event> process( ReserveCreditCommand cmd) { // verify … return … new CreditReservedCredit(); } public void apply( CreditReservedCredit event) { // update state this.xyz = event.xyz }
  73. 73. @crichardson Kanban board example
  74. 74. @crichardson Architecture Create/update boards and tasks Change notifications Materialized views Event Store
  75. 75. @crichardson Demo
  76. 76. @crichardson Summary Events are a central to modern applications Events integrate applications Events maintain data consistency in a microservices architecture Build events into the core of your application using event sourcing
  77. 77. @crichardson @crichardson chris@chrisrichardson.net http://eventuate.io http://plainoldobjects.com http://microservices.io Questions?

×