Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

#JaxLondon: Building microservices with Scala, functional domain models and Spring Boot

1 451 vues

Publié le

In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.

Publié dans : Logiciels
  • Soyez le premier à commenter

#JaxLondon: Building microservices with Scala, functional domain models and Spring Boot

  1. 1. @crichardson Building microservices with Scala, functional domain models and Spring Boot Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson chris@chrisrichardson.net http://plainoldobjects.com
  2. 2. @crichardson Presentation goal Share my experiences with building an application using Scala, functional domain models, microservices, event sourcing, CQRS, and Spring Boot
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Founder of a buzzword compliant (stealthy, social, mobile, big data, machine learning, ...) startup Consultant helping organizations improve how they architect and deploy applications using cloud, micro services, polyglot applications, NoSQL, ...
  5. 5. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  6. 6. @crichardson Let’s imagine that you are building a banking app...
  7. 7. @crichardson Domain model Account balance open(initialBalance) debit(amount) credit(amount) MoneyTransfer fromAccountId toAccountId amount
  8. 8. @crichardson Traditional application architecture BanBaknikningg UI Accounts Transfers Tomcat Browser/ Client WAR/EAR RDBMS Customers Simple to develop test deploy Load balancer scale Spring MVC Spring Hibernate ... HTML REST/JSON
  9. 9. @crichardson Problem #1: monolithic architecture Intimidates developers Obstacle to frequent deployments Overloads your IDE and container Obstacle to scaling development Requires long-term commitment to a technology stack
  10. 10. Standalone services @crichardson Solution #1: use a microservice architecture Banking UI Account Management Service Transaction Management Service Account Database Transaction Database
  11. 11. @crichardson Problem #2: relational databases Scalability Distribution Schema updates O/R impedance mismatch Handling semi-structured data
  12. 12. @crichardson Solution #2: use NoSQL databases Avoids the limitations of RDBMS For example, text search ⇒ Solr/Cloud Search social (graph) data ⇒ Neo4J highly distributed/available database ⇒ Cassandra ...
  13. 13. @crichardson Different modules use different databases IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
  14. 14. But now we have problems with data consistency! @crichardson
  15. 15. Problem #3: Microservices = distributed data management @crichardson Each microservice has it’s own database Some data is replicated and must be kept in sync Business transactions must update data owned by multiple services Tricky to implement reliably without 2PC
  16. 16. Problem #4: NoSQL = ACID-free, denormalized databases Limited transactions, i.e. no ACID transactions Tricky to implement business transactions that update multiple rows, e.g. http://bit.ly/mongo2pc Limited querying capabilities Requires denormalized/materialized views that must be synchronized Multiple datastores (e.g. DynamoDB + Cloud Search ) that need to be kept in sync @crichardson
  17. 17. Solution to #3/#4: Event-based architecture to the rescue @crichardson Microservices publish events when state changes Microservices subscribe to events Synchronize replicated data Maintains eventual consistency across multiple aggregates (in multiple datastores)
  18. 18. Eventually consistent money transfer MoneyTransferService AccountService @crichardson Message Bus transferMoney() Subscribes to: Publishes: Subscribes to: publishes: AccountDebitedEvent AccountCreditedEvent MoneyTransferCreatedEvent DebitRecordedEvent MoneyTransferCreatedEvent DebitRecordedEvent AccountDebitedEvent AccountCreditedEvent
  19. 19. To maintain consistency a service @crichardson must atomically publish an event whenever a domain object changes
  20. 20. @crichardson How to generate events reliably? Database triggers, Hibernate event listener, ... Reliable BUT Not with NoSQL Disconnected from the business level event Limited applicability Ad hoc event publishing code mixed into business logic Messy code, poor separation of concerns Unreliable, e.g. too easy to forget to publish an event
  21. 21. How to atomically update datastore and publish event(s) Use 2PC @crichardson Database and message broker must support 2PC 2PC is best avoided Use datastore as a message queue 1. Update database: new entity state & event to publish 2. Publish event & mark event as published Eventually consistent mechanism See BASE: An Acid Alternative, http://bit.ly/ebaybase • Difficult to implement when using a NoSQL database :-(
  22. 22. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  23. 23. @crichardson Event sourcing For each aggregate: Identify (state-changing) domain events Define Event classes For example, Account: AccountOpenedEvent, AccountDebitedEvent, AccountCreditedEvent ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent
  24. 24. @crichardson Persists events NOT current state Account balance open(initial) debit(amount) credit(amount) Account X table 101 450 Event table AccountOpened AccountCredited AccountDebited 101 101 101 901 902 903 500 250 300
  25. 25. @crichardson Replay events to recreate state Account balance Events AccountOpenedEvent(balance) AccountDebitedEvent(amount) AccountCreditedEvent(amount)
  26. 26. @crichardson Aggregate traits Apply event returning updated Aggregate Map Command to Events
  27. 27. Account - command processing Prevent overdraft @crichardson
  28. 28. @crichardson Account - applying events Immutable
  29. 29. Request handling in an event-sourced application @crichardson HTTP Handler Event Store pastEvents = findEvents(entityId) Account new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) Microservice A
  30. 30. @crichardson Event Store publishes events - consumed by other services Event Store Microservice B Event Subscriber subscribe(EventTypes) publish(event) publish(event) Aggregate NoSQL materialized view update() update()
  31. 31. @crichardson Persisting events Ideally use a cross platform format Use weak serialization: enables event evolution, eg. add memo field to transfer missing field ⇒ provide default value unknown field ⇒ ignore JSON is a good choice
  32. 32. Optimizing using snapshots Most aggregates have relatively few events BUT consider a 10-year old Account ⇒ many transactions Therefore, use snapshots: Periodically save snapshot of aggregate state Typically serialize memento of the aggregate Load latest snapshot + subsequent events @crichardson
  33. 33. @crichardson Event Store API trait EventStore { def save[T <: Aggregate[T]](entity: T, events: Seq[Event], assignedId : Option[EntityId] = None): Future[EntityWithIdAndVersion[T]] def update[T <: Aggregate[T]](entityIdAndVersion : EntityIdAndVersion, entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]] def find[T <: Aggregate[T] : ClassTag](entityId: EntityId) : Future[EntityWithIdAndVersion[T]] def findOptional[T <: Aggregate[T] : ClassTag](entityId: EntityId) Future[Option[EntityWithIdAndVersion[T]]] def subscribe(subscriptionId: SubscriptionId): Future[AcknowledgableEventStream] }
  34. 34. @crichardson Business benefits of event sourcing Built-in, reliable audit log Enables temporal queries Publishes events needed by big data/predictive analytics etc. Preserved history ⇒ More easily implement future requirements
  35. 35. Technical benefits of event sourcing Solves data consistency issues in a Microservice/NoSQL-based @crichardson architecture: Atomically save and publish events Event subscribers update other aggregates ensuring eventual consistency Event subscribers update materialized views in SQL and NoSQL databases (more on that later) Eliminates O/R mapping problem
  36. 36. Drawbacks of event sourcing Weird and unfamiliar Events = a historical record of your bad design decisions Handling duplicate events can be tricky Application must handle eventually consistent data Event store only directly supports PK-based lookup (more on that later) @crichardson
  37. 37. Example of an eventual consistency problem Scenario: 1.Create the user 2.Create shopping cart 3.Update the user with the shopping cart’s id The user temporarily does not have a shopping cart id! Client might need to retry their request at a later point Server should return status code 418?? @crichardson
  38. 38. Handling duplicate messages Idempotent operations e.g. add item to shopping cart Duplicate detection: e.g. track most recently seen event and discard earlier ones @crichardson
  39. 39. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  40. 40. The anatomy of a microservice @crichardson HTTP Request HTTP Adapter Cmd Xyz Request Aggregate Xyz Adapter Event Adapter Event Store Cmd Events Events microservice
  41. 41. @crichardson Asynchronous Spring MVC controller
  42. 42. @crichardson MoneyTransferService DSL concisely specifies: 1.Creates MoneyTransfer aggregate 2.Processes command 3.Applies events 4.Persists events
  43. 43. @crichardson MoneyTransfer Aggregate
  44. 44. @crichardson Handling events published by Accounts 1.Load MoneyTransfer aggregate 2.Processes command 3.Applies events 4.Persists events
  45. 45. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  46. 46. Let’s imagine that you want to display an account and it’s recent transactions... @crichardson
  47. 47. Displaying balance + recent credits and debits We need to do a “join: between the Account and the corresponding MoneyTransfers (Assuming Debit/Credit events don’t include other account, ...) @crichardson BUT Event Store = primary key lookup of individual aggregates, ... ⇒ Use Command Query Responsibility Separation Define separate “materialized” query-side views that implement those queries
  48. 48. HTTP GET Request View Query Service @crichardson Query-side microservices Updater - microservice View Updater Service Events Event Store Reader - microservice View Store e.g. MongoDB Neo4J CloudSearch update query
  49. 49. Persisting account balance and recent transactions in MongoDB Transfers that update The sequence of debits and credits @crichardson { id: "298993498", balance: 100000, transfers : [ {"transferId" : "4552840948484", "fromAccountId" : 298993498, "toAccountId" : 3483948934, "amount" : 5000}, ... ], changes: [ the account {"changeId" : "93843948934", "transferId" : "4552840948484", "transactionType" : "AccountDebited", "amount" : 5000}, ... ] } Denormalized = efficient lookup
  50. 50. duplicate event detection @crichardson Updating MongoDB using Spring Data class AccountInfoUpdateService (mongoTemplate : MongoTemplate, ...) extends CompoundEventHandler { @EventHandler def recordDebit(de: DispatchedEvent[AccountDebitedEvent]) = { ... val ci = AccountChangeInfo(...) mongoTemplate.updateMulti( new Query(where("id").is(de.entityId.id).and("version").lt(changeId)), new Update(). dec("balance", amount). push("changes", ci). set("version", changeId), classOf[AccountInfo]) } @EventHandler def recordTransfer(de: DispatchedEvent[MoneyTransferCreatedEvent]) = ... } insert/In-place update updates to and from accounts
  51. 51. Retrieving account info from MongoDB using Spring Data class AccountInfoQueryService(accountInfoRepository : AccountInfoRepository) { @crichardson def findByAccountId(accountId : EntityId) : AccountInfo = accountInfoRepository.findOne(accountId.id) } case class AccountInfo(id : String, balance : Long, transactions : List[AccountTransactionInfo], changes : List[ChangeInfo], version : String) case class AccountTransactionInfo(changeId : String, transactionId : String, transactionType : String, amount : Long, balanceDelta : Long) trait AccountInfoRepository extends MongoRepository[AccountInfo, String] Implementation generated by Spring Data
  52. 52. @crichardson Other kinds of views AWS Cloud Search Text search as-a-Service View updater batches aggregates to index View query does text search AWS DynamoDB NoSQL as-a-Service On-demand scalable - specify desired read/write capacity Document and key-value data models Useful for denormalized, UI oriented views
  53. 53. @crichardson Dealing with eventually consistent views Scenario: Client creates/updates aggregate Client requests view of aggregate But the view might not yet have been updated Solution: Create/Update response contains aggregate version Query request contains desired version Alternatively: “Fake it” in the UI until the view is updated
  54. 54. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  55. 55. Building microservices with Spring Boot Makes it easy to create stand-alone, production ready Spring Applications Automatically configures Spring using Convention over Configuration Externalizes configuration Generates standalone executable JARs with embedded web server @crichardson
  56. 56. Spring Boot simplifies configuration Spring You write less Boot @crichardson Application components Spring Container Fully configured application Configuration Metadata •Typesafe JavaConfig •Annotations •Legacy XML Default Configuration Metadata of this Inferred from CLASSPATH
  57. 57. Tiny Spring configuration for Account microservice Scan for controllers @crichardson @Configuration @EnableAutoConfiguration @Import(classOf[JdbcEventStoreConfiguration])) @ComponentScan class AccountConfiguration { @Bean def accountService(eventStore : EventStore) = new AccountService(eventStore) @Bean def accountEventHandlers(eventStore : EventStore) = EventHandlerRegistrar.makeFromCompoundEventHandler( eventStore, "accountEventHandlers", new TransferWorkflowAccountHandlers(eventStore)) @Bean @Primary def scalaObjectMapper() = ScalaObjectMapper } Service Event handlers Customize JSON serialization
  58. 58. @crichardson The Main program object BankingMain extends App { SpringApplication.run(classOf[AccountConfiguration], args :_ *) }
  59. 59. @crichardson Building with Gradle buildscript { repositories { mavenCentral() } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:1.0.0.RELEASE") } } apply plugin: 'scala' apply plugin: 'spring-boot' ...
  60. 60. Command line arg processing @crichardson Running the microservice $ java -jar build/libs/banking-main-1.0-SNAPSHOT.jar --server.port=8081 ... 11:38:04.633 INFO n.c.e.e.bank.web.main.BankingMain$ - Started BankingMain. in 8.811 seconds (JVM running for 9.884) $ curl localhost:8081/health {"status":"UP", "mongo":{"status":"UP","version":"2.4.10"}, "db":{"status":"UP","database":"H2","hello":1} } Built in health checks
  61. 61. NodeJS-based API gateway Motivations Small footprint Efficient, event-driven I/O Availability of modules: express, request, googleapis, ... Routes requests to the appropriate backend service based on URL prefix Handles Google/OAuth-based authentication @crichardson
  62. 62. @crichardson Jenkins-based deployment pipeline Build & Test micro-service Build & Test Docker image Deploy Docker image to registry One pipeline per micro-service
  63. 63. @crichardson Production environment EC2 instance running Docker Deployment tool: 1.Compares running containers with what’s been built by Jenkins 2.Pulls latest images from Docker registry 3.Stops old versions 4.Launches new versions
  64. 64. @crichardson Summary Event Sourcing solves key data consistency issues with: Microservices Partitioned/NoSQL databases Spring and Scala play nicely together Spring Boot makes it very easily to build production ready microservices NodeJS is useful for building network-centric services
  65. 65. @crichardson chris@chrisrichardson.net @crichardson Questions? http://plainoldobjects.com http://microservices.io