This is the talk that the Kenny Bastani and I gave at JavaOne 2016. When you’re building JVM applications in a microservice architecture, managing state becomes a distributed systems problem. Instead of being able to manage state as transactions inside the boundaries of a single monolithic application, a microservice must be able to manage consistency by using transactions that are distributed across a network of many different applications and databases. This session explores the problems of data consistency and high availability in JVM-based microservices and how to use event sourcing to solve these problems.

1. @crichardson
Handling Eventual Consistency in
JVM Microservices with Event
Sourcing
Chris Richardson
Founder of Eventuate.io
@crichardson
http://eventuate.io
Copyright © 2015. Chris Richardson Consulting, Inc. All rights reserved
Kenny Bastani
Spring Developer Advocate
@kennybastani
http://pivotal.io

2. @crichardson
Presentation goal
Show how event sourcing is a great
foundation for a microservice
architecture

3. @crichardson
About Chris
http://learnmicroservices.io

4. @crichardson
About Kenny
@kennybastani

5. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS

6. @crichardson
Traditional persistence
Order
id
state
….
101 ACCEPTED
Order table
…
ID STATE …
102 … …

7. @crichardson
But how did we get here?
Who did what and when?
What was the state of the
Order last Monday at 3:01pm?

8. @crichardson
Order example
History

9. @crichardson
Task example State
History
Audit

10. @crichardson
Usually
auditing, history and temporal
queries
is additional code and/or
an after-thought

11. @crichardson
Event sourcing
For each domain object (i.e. DDD aggregate):
Identify (state changing) domain events, e.g. use Event
Storming
Define Event classes
For example, Order events: OrderCreated, OrderCancelled,
OrderApproved, OrderRejected, OrderShipped

12. @crichardson
Persists events NOT current state
Event table
Entity type
Event
id
Entity
id
Event
data
Order 902101 …OrderApproved
Order 903101 …OrderShipped
Event
type
Order 901101 …OrderCreated

13. @crichardson
Replay events to recreate
state
Order
state
OrderCreated(…)
OrderAccepted(…)
OrderShipped(…)
Events
Periodically snapshot to avoid loading all events

14. @crichardson
Benefits of event sourcing
Reifies state changes:
Built in, reliable audit log
temporal queries
Preserved history More easily implement future requirements
Eliminates O/R mapping problem (mostly)
Reliable event publishing: publishes events needed by
predictive analytics etc, user notifications,…

15. @crichardson
Drawbacks of event sourcing
Requires application rewrite
Weird and unfamiliar style of programming
Events live forever carefully evolve schema
Querying the event store can be challenging
Current state is no longer directly available
Often need to maintain views for efficiency

16. @crichardson
Demo

17. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS

18. @crichardson
The Microservice architecture
tackles complexity through
modularization

19. @crichardson
Microservice architecture
Browser
Mobile
Device
Store
Front UI
API
Gateway
Catalog
Service
Review
Service
Order
Service
…
Service
Catalog
Database
Review
Database
Order
Database
…
Database
HTML
REST
REST
Apply X-axis and Z-axis
scaling to each service
independently

20. @crichardson
But there are challenges
implementing
transactions and queries

21. @crichardson
ACID transactions cannot be
used
BEGIN TRANSACTION
…
SELECT ORDER_TOTAL
FROM ORDERS WHERE CUSTOMER_ID = ?
…
SELECT CREDIT_LIMIT
FROM CUSTOMERS WHERE CUSTOMER_ID = ?
…
INSERT INTO ORDERS …
…
COMMIT TRANSACTION
Private to the
Order Service
Private to the
Customer Service
Requires 2PC

22. @crichardson
2PC is not a viable option
Guarantees consistency
BUT
2PC is best avoided
Not supported by many NoSQL databases etc.
CAP theorem 2PC impacts availability
….

23. @crichardson
Queries can’t use joins
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND o.ORDER_TOTAL > 100000
AND o.STATE = 'SHIPPED'
AND c.CREATION_DATE > ?
Customer
Service
Order Service

24. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS

25. @crichardson
Event-driven architecture

26. @crichardson
Order Management
Order
id : 4567
total: 343
state = CREATED
Customer Management
Customer
creditLimit : 12000
creditReservations: {}
Customer
creditLimit : 12000
creditReservations: { 4567 -> 343}
Order
id : 4567
total: 343
state = APPROVED
Eventually consistent credit checking
Message Bus
createOrder()
Publishes:
Subscribes to:
Subscribes to:
publishes:
OrderCreatedEvent
CreditReservedEvent
OrderCreatedEvent CreditReservedEvent

27. @crichardson
How atomically update
database and publish an event
Order Service
Order
Database
Message Broker
insert Order
publish
OrderCreatedEvent
dual write problem
?

28. @crichardson
Failure = inconsistent system
Order Service
Order
Database
Message Broker
insert Order
publish
OrderCreatedEvent
X

29. @crichardson
2PC is not an option

30. @crichardson
How to reliably publish
events when state changes?

31. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS

32. @crichardson
Event sourcing = event-centric
persistence
Application
Database
Event store
update
publish
X

33. @crichardson
Event Store
Application architecture
Order 123 Customer 456
OrderCreated
OrderApproved
…
CustomerCreated
CustomerCreditReserved
…
CreateOrder
UpdateOrder
GetOrder
Subscribe
Order
Service
CreateCustomer
UpdateCustomer
GetCustomer
Subscribe
Customer
Service

34. @crichardson
Request handling in an event sourced application
HTTP
Handler
Event
Store
pastEvents = findEvents(entityId)
Order
new()
applyEvents(pastEvents)
newEvents = processCmd(someCmd)
saveEvents(newEvents) (optimistic locking)
Order Service
applyEvents(newEvents)

35. @crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
Customer
update()
Customer Service

36. @crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
CQRS View
update()
Service Xyz
send notifications
…

37. Event store = database + message
broker
Hybrid database and
message broker
Implementations:
Home grown/DIY
geteventstore.com by
Greg Young
http://eventuate.io
(mine)
Event Store
Save
aggregate
events
Get
aggregate
events
Subscribe
to events

38. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS

39. @crichardson
Find recent, valuable
customers
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND o.ORDER_TOTAL > 100000
AND o.STATE = 'SHIPPED'
AND c.CREATION_DATE > ?
Customer
Service
Order Service
What if event
sourcing is
used?…. is no longer easy

40. @crichardson
Use Command Query
Responsibility Segregation
(CQRS)

41. @crichardson
Command Query Responsibility
Segregation (CQRS)
Application logic
Commands Queries
X
POST
PUT
DELETE
GET

42. @crichardson
Command Query Responsibility
Segregation (CQRS)
Command side
Commands
Event Sourcing
domain
objects
Event Store
Events
Query side
Queries
(Materialized)
View
Events
POST
PUT
DELETE
GET

43. @crichardson
Query side design
Event Store
Updater
View Updater
Service
Events
Reader
HTTP GET
Request
View Query
Service
View
Store
e.g.
MongoDB
ElasticSearch
Neo4J
update query

44. @crichardson
Persisting a customer and
order history in MongoDB
{
"_id" : "0000014f9a45004b 0a00270000000000",
"_class" : "net.chrisrichardson…..views.orderhistory.CustomerView",
"version" : NumberLong(5),
"orders" : {
"0000014f9a450063 0a00270000000000" : {
"state" : "APPROVED",
"orderId" : "0000014f9a450063 0a00270000000000",
"orderTotal" : {
"amount" : "1234"
}
},
"0000014f9a450063 0a00270000000001" : {
"state" : "REJECTED",
"orderId" : "0000014f9a450063 0a00270000000001",
"orderTotal" : {
"amount" : "3000"
}
}
},
"name" : "Fred",
"creditLimit" : {
"amount" : "2000"
}
}
Denormalized = efficient lookup

45. @crichardson
Persisting customers and order info
using Spring Data for MongoDB...

46. @crichardson
Persisting customers and order
using Spring Data for MongoDB...

47. Benefits and drawbacks of
CQRS
Benefits
Necessary in an event sourced
architecture
Separation of concerns =
simpler command and query
models
Supports multiple denormalized
views
Improved scalability and
performance
Drawbacks
Complexity
Potential code duplication
Replication lag/eventually
consistent views

48. @crichardson
Demo

49. @crichardson
Summary
Microservice architecture functionally decomposes an
application into services
Transactions and queries resist decomposition
Use an event-driven architecture based on event sourcing to
maintain data consistency
Implement queries using CQRS

50. @crichardson
chris@chrisrichardson.net kbastani@pivotal.io
http://learnmicroservices.io http://pivotal.io
Questions?