Event-sourced architectures with Akka

1. Event-sourced architectures with Akka @Sander_Mak Luminis Technologies

2. Today's journey Event-sourcing Actors Akka Persistence Design for ES

3. Event-sourcing Is all about getting the facts straight

4. Typical 3 layer architecture UI/Client Service layer Database fetch ↝ modify ↝ store

5. Typical 3 layer architecture UI/Client Service layer Database fetch ↝ modify ↝ store Databases are shared mutable state

6. Typical entity modelling Concert ! artist: String date: Date availableTickets: int price: int ... TicketOrder ! noOfTickets: int userId: String 1 *

7. Typical entity modelling Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1

8. Typical entity modelling Changing the price Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1

9. Typical entity modelling Changing the price Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1 ✘100

10. Typical entity modelling Canceling an order Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1

11. Typical entity modelling Canceling an order Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 ! noOfTickets = 3 userId = 1

12. Update or delete statements in your app? Congratulations, you are ! LOSING DATA EVERY DAY

13. Event-sourced modelling ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketsOrdered TicketsOrdered ! noOfTickets = 3 userId = 1 TicketsOrdered ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 time

14. Event-sourced modelling TicketsOrdered TicketsOrdered Changing the price ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketsOrdered ! noOfTickets = 3 userId = 1 ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 time

15. Event-sourced modelling TicketsOrdered TicketsOrdered Changing the price ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! PriceChanged ! price = 100 TicketsOrdered ! noOfTickets = 3 userId = 1 ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 time

16. Event-sourced modelling ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! PriceChanged ! price = 100 TicketsOrdered TicketsOrdered ! noOfTickets = 3 userId = 1 TicketsOrdered ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 Canceling an order time

17. Event-sourced modelling ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! PriceChanged ! price = 100 OrderCancelled ! userId = 1 TicketsOrdered TicketsOrdered ! noOfTickets = 3 userId = 1 TicketsOrdered ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 Canceling an order time

18. Event-sourced modelling ‣ Immutable events ‣ Append-only storage (scalable) ‣ Replay events: reconstruct historic state ‣ Events as integration mechanism ‣ Events as audit mechanism

19. Event-sourcing: capture all changes to application state as a sequence of events Events: where from?

20. Commands & Events Do something (active) It happened. Deal with it. (facts) Can be rejected (validation) Can be responded to

21. Querying & event-sourcing How do you query a log?

22. Querying & event-sourcing How do you query a log? Command Query Responsibility Segregation

23. CQRS without ES Command Query UI/Client Service layer Database

24. CQRS without ES Command Query UI/Client Service layer Database Command Query UI/Client Command Model Datastore Query Model(s) DaDtaastatosrtoere Datastore ?

25. Event-sourced CQRS Command Query UI/Client Command Model Journal Query Model(s) DaDtaastatosrtoere Datastore Events

26. Actors ‣ Mature and open source ‣ Scala & Java API ‣ Akka Cluster

27. Actors "an island of consistency in a sea of concurrency" Actor ! ! ! mailbox state behavior async message send Process message: ‣ update state ‣ send messages ‣ change behavior Don't worry about concurrency

28. Actors A good fit for event-sourcing? Actor ! ! ! mailbox state behavior mailbox is non-durable (lost messages) state is transient

29. Actors Just store all incoming messages? Actor ! ! ! mailbox state behavior async message send store in journal Problems with command-sourcing: ‣ side-effects ‣ poisonous (failing) messages

30. Persistence ‣ Experimental Akka module ‣ Scala & Java API ‣ Actor state persistence based on event-sourcing

31. Persistent Actor PersistentActor actor-id ! ! state ! event async message send (command) ‣ Derive events from commands ‣ Store events ‣ Update state ‣ Perform side-effects event journal (actor-id)

32. Persistent Actor PersistentActor actor-id ! ! state ! event ! Recover by replaying events, that update the state (no side-effects) ! event journal (actor-id)

33. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } }

35. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } } async callback (but safe to close over state)

37. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } } Isn't recovery with lots of events slow?

38. Snapshots class SnapshottingCounterActor extends PersistentActor { def persistenceId = "snapshotting-counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } case "takesnapshot" => saveSnapshot(state) } ! val receiveRecover: Receive = { case Incremented => state += 1 case SnapshotOffer(_, snapshotState: Int) => state = snapshotState } }

39. Snapshots class SnapshottingCounterActor extends PersistentActor { def persistenceId = "snapshotting-counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } case "takesnapshot" => saveSnapshot(state) } ! val receiveRecover: Receive = { case Incremented => state += 1 case SnapshotOffer(_, snapshotState: Int) => state = snapshotState } }

40. Journal & Snapshot Cassandra Cassandra Kafka Kafka MongoDB HBase DynamoDB MongoDB HBase MapDB JDBC JDBC Plugins:

41. Plugins: Serialization Default: Java serialization Pluggable through Akka: ‣ Protobuf ‣ Kryo ‣ Avro ‣ Your own

42. Persistent View Persistent Actor journal Persistent View Persistent View Views poll the journal ‣ Eventually consistent ‣ Polling configurable ‣ Actor may be inactive ‣ Views track single persistence-id ‣ Views can have own snapshots snapshot store other datastore

43. Persistent View ! "The Database is a cache of a subset of the log" - Pat Helland Persistent Actor journal Persistent View Persistent View snapshot store other datastore

44. Persistent View ! case object ComplexQuery class CounterView extends PersistentView { override def persistenceId: String = "counter" override def viewId: String = "counter-view" var queryState = 0 def receive: Receive = { case Incremented if isPersistent => { queryState = someVeryComplicatedCalculation(queryState) // Or update a document/graph/relational database } case ComplexQuery => { sender() ! queryState; // Or perform specialized query on datastore } } }

47. Sell concert tickets ConcertActor ! ! price availableTickets startTime salesRecords ! Commands: CreateConcert BuyTickets ChangePrice AddCapacity journal ConcertHistoryView ! ! ! ! 60 45 30 15 0 100 50 0 $50 $75 $100 code @ bit.ly/akka-es

48. Scaling out: Akka Cluster Single writer: persistent actor must be singleton, views may be anywhere Cluster node Cluster node Cluster node Persistent Persistent Actor id = "1" Actor id = "1" Persistent Actor id = "1" Persistent Actor id = "2" Persistent Actor id = "1" Persistent Actor id = "3" distributed journal

49. Scaling out: Akka Cluster Single writer: persistent actor must be singleton, views may be anywhere How are persistent actors distributed over cluster? Cluster node Cluster node Cluster node Persistent Persistent Actor id = "1" Actor id = "1" Persistent Actor id = "1" Persistent Actor id = "2" Persistent Actor id = "1" Persistent Actor id = "3" distributed journal

50. Scaling out: Akka Cluster Sharding: Coordinator assigns ShardRegions to nodes (consistent hashing) Actors in shard can be activated/passivated, rebalanced Cluster node Cluster node Cluster node Persistent Persistent Actor id = "1" Actor id = "1" Persistent Actor id = "1" Persistent Actor id = "2" Persistent Actor id = "1" Persistent Actor id = "3" Shard Region Shard Region Shard Region Coordinator distributed journal

51. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors

52. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => hash(cmd.concert) } ShardResolver assigns new actors to shards

53. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => hash(cmd.concert) } ShardResolver assigns new actors to shards ClusterSharding(system).start( typeName = "Concert", entryProps = Some(ConcertActor.props()), idExtractor = ConcertActor.idExtractor, shardResolver = ConcertActor.shardResolver) Initialize ClusterSharding extension

54. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => hash(cmd.concert) } ShardResolver assigns new actors to shards ClusterSharding(system).start( typeName = "Concert", entryProps = Some(ConcertActor.props()), idExtractor = ConcertActor.idExtractor, shardResolver = ConcertActor.shardResolver) Initialize ClusterSharding extension val concertRegion: ActorRef = ClusterSharding(system).shardRegion("Concert") concertRegion ! BuyTickets(concertId = 123, user = "Sander", quantity = 1)

55. Design for event-sourcing

56. DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate ! Aggregate ! Eventual Consistency Root entity enteitnytity entity Root entity entity entity

57. DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate ! Aggregate ! Eventual Consistency Root entity enteitnytity entity Root entity entity entity Persistent Actor Persistent Actor Message passing

58. DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate ! Aggregate ! Eventual Consistency Akka Persistence is not a DDD/CQRS framework But it comes awfully close Root entity enteitnytity entity Root entity entity entity Persistent Actor Persistent Actor Message passing

59. Designing aggregates Focus on events Structural representation(s) follow ERD

60. Designing aggregates Focus on events Structural representation(s) follow ERD Size matters. Faster replay, less write contention Don't store derived info (use views)

61. Designing aggregates Focus on events Structural representation(s) follow ERD Size matters. Faster replay, less write contention Don't store derived info (use views) With CQRS read-your-writes is not the default When you need this, model it in a single Aggregate

62. Between aggregates ‣ Send commands to other aggregates by id ! ‣ No distributed transactions ‣ Use business level ack/nacks ‣ SendInvoice -> InvoiceSent/InvoiceCouldNotBeSent ‣ Compensating actions for failure ‣ No guaranteed delivery ‣ Alternative: AtLeastOnceDelivery

63. Between systems System integration through event-sourcing topic 1 topic 2 derived Kafka Application Akka Persistence Spark Streaming External Application

64. Designing commands ‣ Self-contained ‣ Unit of atomic change ‣ Granularity and intent

65. Designing commands ‣ Self-contained ‣ Unit of atomic change ‣ Granularity and intent UpdateAddress street = ... city = ... vs ChangeStreet street = ... ChangeCity street = ...

66. Designing commands ‣ Self-contained ‣ Unit of atomic change ‣ Granularity and intent UpdateAddress street = ... city = ... vs ChangeStreet street = ... ChangeCity street = ... Move street = ... city = ... vs

67. Designing events CreateConcert

68. Designing events CreateConcert ConcertCreated Past tense Irrefutable

69. Designing events CreateConcert ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99

70. Designing events CreateConcert ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99 TicketsBought quantity = 1 Delta-based No derived info

71. Designing events CreateConcert ConcertCreated ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99 TicketsBought quantity = 1 Delta-based No derived info

72. Designing events CreateConcert ConcertCreated ConcertCreated who/when/.. Add metadata to all events ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99 TicketsBought quantity = 1 Delta-based No derived info

73. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1

74. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1 Actor logic: v2 event v2 event v2 event v2 event v1 event v2 event v1

75. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1 Actor logic: v2 event v2 event v2 event v2 event v1 event v2 event v1 Actor logic: v2 event v2 event v2 snapshot event v1 event v1

76. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1 Actor logic: v2 event v2 event v2 event v2 event v1 event v2 event v1 Actor logic: v2 event v2 event v2 snapshot event v1 event v1 ‣ Be backwards compatible ‣ Avro/Protobuf ‣ Serializer can do translation ! ‣ Snapshot versioning: harder

77. In conclusion Event-sourcing is... Powerful but unfamiliar

78. In conclusion Event-sourcing is... Powerful but unfamiliar Combines well with UI/Client Command Model Journal Query Model DaDtaastatosrtoere Datastore Events DDD/CQRS

79. In conclusion Event-sourcing is... Powerful but unfamiliar Combines well with UI/Client Command Model Journal Query Model DaDtaastatosrtoere Datastore Events DDD/CQRS Not a

80. In conclusion Akka Actors & Akka Persistence A good fit for event-sourcing PersistentActor actor-id ! ! state ! async message send (command) event event journal (actor-id) Experimental, view improvements needed

81. Thank you. ! code @ bit.ly/akka-es @Sander_Mak Luminis Technologies

Event-sourced architectures with Akka

Sander Mak (@Sander_Mak)

Event-sourced architectures with Akka