Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Event-sourced architectures with Akka
Similaire à Event-sourced architectures with Akka (20)
Plus de Sander Mak (@Sander_Mak)
Plus de Sander Mak (@Sander_Mak) (20)
Dernier
Dernier (20)
Event-sourced architectures with Akka
- 1. Event-sourced architectures with Akka @Sander_Mak Luminis Technologies
- 2. Today's journey Event-sourcing Actors Akka Persistence Design for ES
- 3. Event-sourcing Is all about getting the facts straight
- 4. Typical 3 layer architecture UI/Client Service layer Database fetch ↝ modify ↝ store
- 5. Typical 3 layer architecture UI/Client Service layer Database fetch ↝ modify ↝ store Databases are shared mutable state
- 6. Typical entity modelling Concert ! artist: String date: Date availableTickets: int price: int ... TicketOrder ! noOfTickets: int userId: String 1 *
- 7. Typical entity modelling Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1
- 8. Typical entity modelling Changing the price Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1
- 9. Typical entity modelling Changing the price Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1 ✘100
- 10. Typical entity modelling Canceling an order Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 TicketOrder ! noOfTickets = 3 userId = 1 noOfTickets = 3 userId = 1
- 11. Typical entity modelling Canceling an order Concert ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketOrder TicketOrder ! noOfTickets = 3 userId = 1 ! noOfTickets = 3 userId = 1
- 12. Update or delete statements in your app? Congratulations, you are ! LOSING DATA EVERY DAY
- 13. Event-sourced modelling ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketsOrdered TicketsOrdered ! noOfTickets = 3 userId = 1 TicketsOrdered ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 time
- 14. Event-sourced modelling TicketsOrdered TicketsOrdered Changing the price ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! TicketsOrdered ! noOfTickets = 3 userId = 1 ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 time
- 15. Event-sourced modelling TicketsOrdered TicketsOrdered Changing the price ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! PriceChanged ! price = 100 TicketsOrdered ! noOfTickets = 3 userId = 1 ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 time
- 16. Event-sourced modelling ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! PriceChanged ! price = 100 TicketsOrdered TicketsOrdered ! noOfTickets = 3 userId = 1 TicketsOrdered ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 Canceling an order time
- 17. Event-sourced modelling ConcertCreated ! artist = Aerosmith availableTickets = 100 price = 10 ... ! PriceChanged ! price = 100 OrderCancelled ! userId = 1 TicketsOrdered TicketsOrdered ! noOfTickets = 3 userId = 1 TicketsOrdered ! noOfTickets = 3 ! userId = 1 noOfTickets = 3 userId = 1 Canceling an order time
- 18. Event-sourced modelling ‣ Immutable events ‣ Append-only storage (scalable) ‣ Replay events: reconstruct historic state ‣ Events as integration mechanism ‣ Events as audit mechanism
- 19. Event-sourcing: capture all changes to application state as a sequence of events Events: where from?
- 20. Commands & Events Do something (active) It happened. Deal with it. (facts) Can be rejected (validation) Can be responded to
- 21. Querying & event-sourcing How do you query a log?
- 22. Querying & event-sourcing How do you query a log? Command Query Responsibility Segregation
- 23. CQRS without ES Command Query UI/Client Service layer Database
- 24. CQRS without ES Command Query UI/Client Service layer Database Command Query UI/Client Command Model Datastore Query Model(s) DaDtaastatosrtoere Datastore ?
- 25. Event-sourced CQRS Command Query UI/Client Command Model Journal Query Model(s) DaDtaastatosrtoere Datastore Events
- 26. Actors ‣ Mature and open source ‣ Scala & Java API ‣ Akka Cluster
- 27. Actors "an island of consistency in a sea of concurrency" Actor ! ! ! mailbox state behavior async message send Process message: ‣ update state ‣ send messages ‣ change behavior Don't worry about concurrency
- 28. Actors A good fit for event-sourcing? Actor ! ! ! mailbox state behavior mailbox is non-durable (lost messages) state is transient
- 29. Actors Just store all incoming messages? Actor ! ! ! mailbox state behavior async message send store in journal Problems with command-sourcing: ‣ side-effects ‣ poisonous (failing) messages
- 30. Persistence ‣ Experimental Akka module ‣ Scala & Java API ‣ Actor state persistence based on event-sourcing
- 31. Persistent Actor PersistentActor actor-id ! ! state ! event async message send (command) ‣ Derive events from commands ‣ Store events ‣ Update state ‣ Perform side-effects event journal (actor-id)
- 32. Persistent Actor PersistentActor actor-id ! ! state ! event ! Recover by replaying events, that update the state (no side-effects) ! event journal (actor-id)
- 33. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } }
- 34. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } }
- 35. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } } async callback (but safe to close over state)
- 36. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } }
- 37. Persistent Actor ! case object Increment // command case object Incremented // event ! class CounterActor extends PersistentActor { def persistenceId = "counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } } ! val receiveRecover: Receive = { case Incremented => state += 1 } } Isn't recovery with lots of events slow?
- 38. Snapshots class SnapshottingCounterActor extends PersistentActor { def persistenceId = "snapshotting-counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } case "takesnapshot" => saveSnapshot(state) } ! val receiveRecover: Receive = { case Incremented => state += 1 case SnapshotOffer(_, snapshotState: Int) => state = snapshotState } }
- 39. Snapshots class SnapshottingCounterActor extends PersistentActor { def persistenceId = "snapshotting-counter" ! var state = 0 ! val receiveCommand: Receive = { case Increment => persist(Incremented) { evt => state += 1 println("incremented") } case "takesnapshot" => saveSnapshot(state) } ! val receiveRecover: Receive = { case Incremented => state += 1 case SnapshotOffer(_, snapshotState: Int) => state = snapshotState } }
- 40. Journal & Snapshot Cassandra Cassandra Kafka Kafka MongoDB HBase DynamoDB MongoDB HBase MapDB JDBC JDBC Plugins:
- 41. Plugins: Serialization Default: Java serialization Pluggable through Akka: ‣ Protobuf ‣ Kryo ‣ Avro ‣ Your own
- 42. Persistent View Persistent Actor journal Persistent View Persistent View Views poll the journal ‣ Eventually consistent ‣ Polling configurable ‣ Actor may be inactive ‣ Views track single persistence-id ‣ Views can have own snapshots snapshot store other datastore
- 43. Persistent View ! "The Database is a cache of a subset of the log" - Pat Helland Persistent Actor journal Persistent View Persistent View snapshot store other datastore
- 44. Persistent View ! case object ComplexQuery class CounterView extends PersistentView { override def persistenceId: String = "counter" override def viewId: String = "counter-view" var queryState = 0 def receive: Receive = { case Incremented if isPersistent => { queryState = someVeryComplicatedCalculation(queryState) // Or update a document/graph/relational database } case ComplexQuery => { sender() ! queryState; // Or perform specialized query on datastore } } }
- 45. Persistent View ! case object ComplexQuery class CounterView extends PersistentView { override def persistenceId: String = "counter" override def viewId: String = "counter-view" var queryState = 0 def receive: Receive = { case Incremented if isPersistent => { queryState = someVeryComplicatedCalculation(queryState) // Or update a document/graph/relational database } case ComplexQuery => { sender() ! queryState; // Or perform specialized query on datastore } } }
- 46. Persistent View ! case object ComplexQuery class CounterView extends PersistentView { override def persistenceId: String = "counter" override def viewId: String = "counter-view" var queryState = 0 def receive: Receive = { case Incremented if isPersistent => { queryState = someVeryComplicatedCalculation(queryState) // Or update a document/graph/relational database } case ComplexQuery => { sender() ! queryState; // Or perform specialized query on datastore } } }
- 47. Sell concert tickets ConcertActor ! ! price availableTickets startTime salesRecords ! Commands: CreateConcert BuyTickets ChangePrice AddCapacity journal ConcertHistoryView ! ! ! ! 60 45 30 15 0 100 50 0 $50 $75 $100 code @ bit.ly/akka-es
- 48. Scaling out: Akka Cluster Single writer: persistent actor must be singleton, views may be anywhere Cluster node Cluster node Cluster node Persistent Persistent Actor id = "1" Actor id = "1" Persistent Actor id = "1" Persistent Actor id = "2" Persistent Actor id = "1" Persistent Actor id = "3" distributed journal
- 49. Scaling out: Akka Cluster Single writer: persistent actor must be singleton, views may be anywhere How are persistent actors distributed over cluster? Cluster node Cluster node Cluster node Persistent Persistent Actor id = "1" Actor id = "1" Persistent Actor id = "1" Persistent Actor id = "2" Persistent Actor id = "1" Persistent Actor id = "3" distributed journal
- 50. Scaling out: Akka Cluster Sharding: Coordinator assigns ShardRegions to nodes (consistent hashing) Actors in shard can be activated/passivated, rebalanced Cluster node Cluster node Cluster node Persistent Persistent Actor id = "1" Actor id = "1" Persistent Actor id = "1" Persistent Actor id = "2" Persistent Actor id = "1" Persistent Actor id = "3" Shard Region Shard Region Shard Region Coordinator distributed journal
- 51. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors
- 52. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => hash(cmd.concert) } ShardResolver assigns new actors to shards
- 53. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => hash(cmd.concert) } ShardResolver assigns new actors to shards ClusterSharding(system).start( typeName = "Concert", entryProps = Some(ConcertActor.props()), idExtractor = ConcertActor.idExtractor, shardResolver = ConcertActor.shardResolver) Initialize ClusterSharding extension
- 54. Scaling out: Sharding val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.concertId, cmd) } IdExtractor allows ShardRegion to route commands to actors val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => hash(cmd.concert) } ShardResolver assigns new actors to shards ClusterSharding(system).start( typeName = "Concert", entryProps = Some(ConcertActor.props()), idExtractor = ConcertActor.idExtractor, shardResolver = ConcertActor.shardResolver) Initialize ClusterSharding extension val concertRegion: ActorRef = ClusterSharding(system).shardRegion("Concert") concertRegion ! BuyTickets(concertId = 123, user = "Sander", quantity = 1)
- 56. DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate ! Aggregate ! Eventual Consistency Root entity enteitnytity entity Root entity entity entity
- 57. DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate ! Aggregate ! Eventual Consistency Root entity enteitnytity entity Root entity entity entity Persistent Actor Persistent Actor Message passing
- 58. DDD+CQRS+ES DDD: Domain Driven Design Fully consistent Fully consistent Aggregate ! Aggregate ! Eventual Consistency Akka Persistence is not a DDD/CQRS framework But it comes awfully close Root entity enteitnytity entity Root entity entity entity Persistent Actor Persistent Actor Message passing
- 59. Designing aggregates Focus on events Structural representation(s) follow ERD
- 60. Designing aggregates Focus on events Structural representation(s) follow ERD Size matters. Faster replay, less write contention Don't store derived info (use views)
- 61. Designing aggregates Focus on events Structural representation(s) follow ERD Size matters. Faster replay, less write contention Don't store derived info (use views) With CQRS read-your-writes is not the default When you need this, model it in a single Aggregate
- 62. Between aggregates ‣ Send commands to other aggregates by id ! ‣ No distributed transactions ‣ Use business level ack/nacks ‣ SendInvoice -> InvoiceSent/InvoiceCouldNotBeSent ‣ Compensating actions for failure ‣ No guaranteed delivery ‣ Alternative: AtLeastOnceDelivery
- 63. Between systems System integration through event-sourcing topic 1 topic 2 derived Kafka Application Akka Persistence Spark Streaming External Application
- 64. Designing commands ‣ Self-contained ‣ Unit of atomic change ‣ Granularity and intent
- 65. Designing commands ‣ Self-contained ‣ Unit of atomic change ‣ Granularity and intent UpdateAddress street = ... city = ... vs ChangeStreet street = ... ChangeCity street = ...
- 66. Designing commands ‣ Self-contained ‣ Unit of atomic change ‣ Granularity and intent UpdateAddress street = ... city = ... vs ChangeStreet street = ... ChangeCity street = ... Move street = ... city = ... vs
- 68. Designing events CreateConcert ConcertCreated Past tense Irrefutable
- 69. Designing events CreateConcert ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99
- 70. Designing events CreateConcert ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99 TicketsBought quantity = 1 Delta-based No derived info
- 71. Designing events CreateConcert ConcertCreated ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99 TicketsBought quantity = 1 Delta-based No derived info
- 72. Designing events CreateConcert ConcertCreated ConcertCreated who/when/.. Add metadata to all events ConcertCreated Past tense Irrefutable TicketsBought newCapacity = 99 TicketsBought quantity = 1 Delta-based No derived info
- 73. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1
- 74. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1 Actor logic: v2 event v2 event v2 event v2 event v1 event v2 event v1
- 75. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1 Actor logic: v2 event v2 event v2 event v2 event v1 event v2 event v1 Actor logic: v2 event v2 event v2 snapshot event v1 event v1
- 76. Versioning Actor logic: v1 and v2 event v2 event v2 event v1 event v1 Actor logic: v2 event v2 event v2 event v2 event v1 event v2 event v1 Actor logic: v2 event v2 event v2 snapshot event v1 event v1 ‣ Be backwards compatible ‣ Avro/Protobuf ‣ Serializer can do translation ! ‣ Snapshot versioning: harder
- 77. In conclusion Event-sourcing is... Powerful but unfamiliar
- 78. In conclusion Event-sourcing is... Powerful but unfamiliar Combines well with UI/Client Command Model Journal Query Model DaDtaastatosrtoere Datastore Events DDD/CQRS
- 79. In conclusion Event-sourcing is... Powerful but unfamiliar Combines well with UI/Client Command Model Journal Query Model DaDtaastatosrtoere Datastore Events DDD/CQRS Not a
- 80. In conclusion Akka Actors & Akka Persistence A good fit for event-sourcing PersistentActor actor-id ! ! state ! async message send (command) event event journal (actor-id) Experimental, view improvements needed
- 81. Thank you. ! code @ bit.ly/akka-es @Sander_Mak Luminis Technologies