Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
EVENT BUS AS
BACKBONE FOR
DECOUPLED
MICROSERVICE
CHOREOGRAPHY
LECTURE & WORKSHOP
Lucas Jellema (CTO AMIS)
May 2018 – Fonty...
AGENDA
• Introduction of microservices - objectives, traits, implementation
• The making of a microservice (demo)
• The mi...
SOURCE CODE – BIT.LY/FONTYS-AMIS
• https://github.com/lucasjellema/workshop-event-bus-microservice-choreography-may-2018
•...
WHAT IS IT ALL ABOUT?
Application
Production Runtime
WHAT IS IT ALL ABOUT?
Application
Production Runtime
Platform
WHAT IS IT ALL ABOUT?
Application
Platform
Production Runtime
Operations
Monitoring &
Management
ONE TEAM HAS AGILE RESPONSIBILITY
THROUGH FULL LIFECYLE
Application
Platform
Production Runtime
Operations
Monitoring &
Ma...
ONE TEAM HAS AGILE RESPONSIBILITY
THROUGH FULL LIFECYLE
Application
Platform
Production Runtime
Operations
Monitoring &
Ma...
ONE TEAM HAS AGILE RESPONSIBILITY
THROUGH FULL LIFECYLE
Application
Platform
Application
Platform
DEVOPS TEAM OWNS AND RUNS
ONE (OR MORE) PRODUCTS
Application
Platform
Generic Infrastructure Platform for running DevOps P...
MULTIPLE PRODUCTS FROM MULTIPLE TEAMS
RUN ON A SHARED GENERIC INFRASTRUCTURE
Generic Infrastructure Platform for running D...
APP PLUS PLATFORM UNDER DEVOPS
== MICROSERVICE
Generic Infrastructure Platform for running DevOps Products
µ µ µ µ µ
APP PLUS PLATFORM UNDER DEVOPS
== MICROSERVICE
Generic Infrastructure Platform for running DevOps Products
µ µ µ µ µ
MICROSERVICES
MICROSERVICE OBJECTIVES
(BECAUSE OF ENTERPRISE OBJECTIVES)
• Flexible, agile (Dev)
• Functionality evolves rapidly with li...
STANDING ON SHOULDERS OF GIANTS
• Monolith++
• API
• Scale out
• Automated CI/CD
• SOA++
• Stateless
• HTTP native (REST)
...
MICROSERVICES HOW
• Public APIs in standardized protocols
• Deployable on enterprise standardized microservices platform
•...
MICROSERVICES PLATFORM
• Receives microservice deployment
• Handles scale out & fail over
• Start/stop microservice instan...
THE MICROSERVICE
API
HTTP REST/JSON
THE MICROSERVICE: VALIDATE TWEET
API
HTTP REST/JSON
MICROSERVICE BUSINESS FUNCTIONALITY:
TWEET VALIDATION
MICROSERVICE BUSINESS FUNCTIONALITY:
TWEET VALIDATION
THE MAKING OF A MICROSERVICE
Dockerfile Pod.yaml
Service.yaml
Volume
(Storage)
Config&Depency
Injection
CREATE
AND COMMIT A NODE.JS APPLICATION TO GITHUB
Commit Node
Application
RUN A NODE.JS APPLICATION FROM GITHUB
USING GENERIC CONTAINER IMAGE
1. pull from Docker Hub registry
run container
3. clon...
VALIDATETWEETPOD.YAML K8S FILE
VALIDATETWEETSERVICE.YAML K8S FILE
SHARED PLATFORM CAPABILITY
• Microservices are isolated
• Not aware of each other (except through public APIs)
• Not shari...
MICROSERVICES CROSS PLATFORM
CAPABILITIES
• Authentication
• Persistent Storage
• Cache
• Load balancing/API Gateway
• Ser...
EXAMPLE SYSTEM ARCHITECTURE
Microservices Platform
API
API
Logging
Cache
API API
UI
HTML 5
Web
Component
REST/
JSON
Authen...
TRENDS, CHALLENGES, COMMON
• Use of containers and container management
• Docker Containers – layered, packaged & shippabl...
EVENTS
Producers
Consumers
MICROSERVICES AND EVENTS
• Report business events [without knowing to whom and without expecting a response]
• Allowing in...
REQUIREMENTS FOR EVENT CAPABILITY
IN MICROSERVICES PLATFORM
• Provide decoupling between publisher and consumer
• Generall...
INTRODUCING APACHE KAFKA
• ..- 2010 – creation at Linkedin
• Message Bus | Event Broker | Streaming Data Platform
• High v...
KAFKA TERMINOLOGY
• Topic
• partition
• Message
• == ByteArray
• Broker
• replicated
• Producer
• Consumer
• Working toget...
Producers
Consumers
Topic
Broker
tcp
tcp
Node + kafka-node
Kafka Namespace
KAFKA ON KUBERNETES
Kubernetes Dashboard
Kafka Manager
EXTENDED API OF MICROSERVICE
• Deployment API
• Injectable dependencies – reference to cache, logging, storage URL, …
• Co...
HANDSON – PART ONE
• Install Kubernetes (kubectl plus minikube on Virtual Box or <…>)
• Roll out a simple microservice
• A...
Event Bus
MICROSERVICES PLATFORM
Cache Cache
Inspector
LogMonitor
PART TWO
MICROSERVICE
CHOREOGRAPHY
REQUIREMENTS ON WORKFLOW
• Management information on running instances and their state
• Recover or fully rollback functio...
CROSS MICROSERVICES WORKFLOW
Choreography vs Orchestration – Which one is which?
MICROSERVICE WORKFLOW
CHOREOGRAPHY
• Multi step process
• Long running, with state, happy & non-happy
• Changes in workflo...
DESIRED WORKFLOW
Tweet about
<some topic>
Validate
Tweet
No simple retweet, no
black listed words used,
no known robot twe...
MICROSERVICES TO MAP THE WORKFLOW TO
Microservices Platform
API
Event Bus
REST/
JSON
APIUI
On premises
Tweet
Board
Validat...
CHOREOGRAPHED WORKFLOW
Microservices Platform
APIAPI
Event Bus
APIUI
On premises
TweetBoard
Validate
Tweet
Tweet
Receiver
...
ROUTING SLIP PUBLISHED BY WORKFLOW LAUNCHER
Validate
Tweet
Enrich Tweet
Add Tweet to
TweetBoard
Publish
TweetBoard
If NOK,...
ROUTING SLIP FOR COMPLETED WORKFLOW
CHOREOGRAPHED WORKFLOW
Microservices Platform
APIAPI
Event Bus
APIUI
On premises
TweetBoard
Validate
Tweet
Tweet
Receiver
...
Event Bus
MICROSERVICE CHOREOGRAPHY TOPOLOGY
– STAGE ONE
Workflow
Launcher
Tweet
Validator
Cache Cache
Inspector
LogMonito...
Event Bus
MICROSERVICE CHOREOGRAPHY TOPOLOGY
Workflow
Launcher
Tweet
Validator
Tweet
Enricher
Tweet
Board
Cache Cache
Insp...
EXTENDING THE WORKFLOW CHOREOGRAPHY –
WITH TRANSLATION OF THE TWEET
Validate
Tweet
Enrich Tweet
Add Tweet to
TweetBoard
Pu...
EXTENDING THE WORKFLOW CHOREOGRAPHY –
WITH TRANSLATION OF THE TWEET
• Implement microservice
for
• Translating tweets
• Pa...
CHOREOGRAPHED WORKFLOW
Microservices Platform
APIAPI
Event Bus
APIUI
On premises
TweetBoard
Validate
Tweet
Tweet
Receiver
...
MODIFIED WORKFLOW TEMPLATE
API
Translate
Tweet
API
APIUI
TweetBoard
RESULT ON TWEET BOARD INCLUDING
TRANSLATION
MAIN TECHNICAL CHALLENGES
• Network
• Dockerize NodeJS applications
• Run Docker container images in Kubernetes cluster
• ...
SUMMARY
• Microservices are about rapid rollout of scalable, available functionality
• (session) Stateless, Continuous dep...
HANDSON – PART TWO
• Workflow Choreography based on Event Bus
with microservices on Kubernetes
• Roll out microservices Tw...
• Blog: technology.amis.nl
• Email: lucas.jellema@amis.nl
• : lucasjellema
• : lucas-jellema
• : www.amis.nl, info@amis.nl...
Kafka Namespace
ADDING KSQL
KSQL
Prochain SlideShare
Chargement dans…5
×

Event Bus as Backbone for Decoupled Microservice Choreography - Lecture and Workshop (Fontys Hogeschool, Eindhoven, The Netherlands)

104 vues

Publié le

Microservices are independent, encapsulated entities that produce meaningful results and business functionality in tentative collaboration. Events and pub/sub are great for allowing such decoupled interaction. Using Apache Kafka as robust, distributed, real-time, high volume event bus, this session demonstrates how microservices packaged with Docker and implemented in Java, Node, Python and SQL collaborate unknowingly. The microservices respond to social (media) events - courtesy of IFTTT - and publish results to multiple channels. The event bus operates across cloud services and on premises platforms such as Kubernetes: both the bus and the microservices can run anywhere. A microservices platform is discussed with generic capabilities.

The resources for the accompanying workshop are in GitHub: https://github.com/lucasjellema/workshop-event-bus-microservice-choreography-may-2018

Publié dans : Logiciels
  • Soyez le premier à commenter

Event Bus as Backbone for Decoupled Microservice Choreography - Lecture and Workshop (Fontys Hogeschool, Eindhoven, The Netherlands)

  1. 1. EVENT BUS AS BACKBONE FOR DECOUPLED MICROSERVICE CHOREOGRAPHY LECTURE & WORKSHOP Lucas Jellema (CTO AMIS) May 2018 – Fontys Hogeschool, Eindhoven, The Netherlands
  2. 2. AGENDA • Introduction of microservices - objectives, traits, implementation • The making of a microservice (demo) • The microservices platform - generic capabilities • Introduction of Apache Kafka for implementing the Event Bus • Microservices and Event Bus in a hybrid world – cross on-premises and clouds • Handson Part One – The microservices platform and some microservices • Using events for decoupled interaction and workflow choreography • Implementing a multi-microservice workflow with event based choreography • Design, architecture, implementation and live demo • Music maestro – demonstrating event based workflow choreography by microservices • Handson Part Two – Implementing microservice choreography
  3. 3. SOURCE CODE – BIT.LY/FONTYS-AMIS • https://github.com/lucasjellema/workshop-event-bus-microservice-choreography-may-2018 • Node & Java -sources • Kafka Producer & Consumer • Kubernetes YAML • Workshop Labs • Presentation Slide Deck
  4. 4. WHAT IS IT ALL ABOUT? Application Production Runtime
  5. 5. WHAT IS IT ALL ABOUT? Application Production Runtime Platform
  6. 6. WHAT IS IT ALL ABOUT? Application Platform Production Runtime Operations Monitoring & Management
  7. 7. ONE TEAM HAS AGILE RESPONSIBILITY THROUGH FULL LIFECYLE Application Platform Production Runtime Operations Monitoring & ManagementApplication Preparation Runtime Platform Development CD Agile Design, Build, Test
  8. 8. ONE TEAM HAS AGILE RESPONSIBILITY THROUGH FULL LIFECYLE Application Platform Production Runtime Operations Monitoring & ManagementApplication Preparation Runtime Platform Development CD Agile Design, Build, Test
  9. 9. ONE TEAM HAS AGILE RESPONSIBILITY THROUGH FULL LIFECYLE Application Platform Application Platform
  10. 10. DEVOPS TEAM OWNS AND RUNS ONE (OR MORE) PRODUCTS Application Platform Generic Infrastructure Platform for running DevOps Products Floorspace, Power, Cooling, Storage, Compute Monitoring, Management, Cache, Authentication, RDBMS, Event Hub
  11. 11. MULTIPLE PRODUCTS FROM MULTIPLE TEAMS RUN ON A SHARED GENERIC INFRASTRUCTURE Generic Infrastructure Platform for running DevOps Products Floorspace, Power, Cooling, Storage, Compute Monitoring, Management, Cache, Authentication, RDBMS, Event Hub Application Platform Application Platform Application Platform Application Platform Application Platform
  12. 12. APP PLUS PLATFORM UNDER DEVOPS == MICROSERVICE Generic Infrastructure Platform for running DevOps Products µ µ µ µ µ
  13. 13. APP PLUS PLATFORM UNDER DEVOPS == MICROSERVICE Generic Infrastructure Platform for running DevOps Products µ µ µ µ µ
  14. 14. MICROSERVICES
  15. 15. MICROSERVICE OBJECTIVES (BECAUSE OF ENTERPRISE OBJECTIVES) • Flexible, agile (Dev) • Functionality evolves rapidly with little effort • Easy quick rollout • Low impact • Manageable Non Functionally (Ops) • Scalable – handle flexible workload (horizontal scaleout) • Available – deal with failing nodes • Comprehendable • Dependencies, Impact, Implementation, deployment, operations • Ownership (culture, organization, process) • One team can do functional and technical evolution and deployment continuously and independently How do we get from a Monolith to Microservices?
  16. 16. STANDING ON SHOULDERS OF GIANTS • Monolith++ • API • Scale out • Automated CI/CD • SOA++ • Stateless • HTTP native (REST) • Multiple tiers & platform components included • Deployable
  17. 17. MICROSERVICES HOW • Public APIs in standardized protocols • Deployable on enterprise standardized microservices platform • Omnia mea porto mecum - no external dependencies… • …except: • Calls to public APIs (exposed for example by microservices) • Usage of platform facilities • Generically available via contract • Injected via parameters • No sharing of data or other private resources across microservices • Stateless and Horizontally scalable • No session state, no client stickyness • Potentially: micro-silo with multiple tiers (including UI) • Any implementation technology • that can run on the platform
  18. 18. MICROSERVICES PLATFORM • Receives microservice deployment • Handles scale out & fail over • Start/stop microservice instances based on non functional requirements and live observed behavior • Supports automated DevOps • CD, monitoring, throttling, circuit breaker, auditing, … • Provides Capabilities – generic facilities available to microservices from the run time platform • Provided through public APIs whenever possible • Injected meta-data at run time • implemented by generic/platform level microservices Microservices Platform API deploy, inject dependencies, start, watch, restart, stop, scale API API API Gateway Authenticate Logging Cache
  19. 19. THE MICROSERVICE API HTTP REST/JSON
  20. 20. THE MICROSERVICE: VALIDATE TWEET API HTTP REST/JSON
  21. 21. MICROSERVICE BUSINESS FUNCTIONALITY: TWEET VALIDATION
  22. 22. MICROSERVICE BUSINESS FUNCTIONALITY: TWEET VALIDATION
  23. 23. THE MAKING OF A MICROSERVICE Dockerfile Pod.yaml Service.yaml Volume (Storage) Config&Depency Injection
  24. 24. CREATE AND COMMIT A NODE.JS APPLICATION TO GITHUB Commit Node Application
  25. 25. RUN A NODE.JS APPLICATION FROM GITHUB USING GENERIC CONTAINER IMAGE 1. pull from Docker Hub registry run container 3. clone from GitHub Specify: - GITHUB_URL - APP_HOME - APP_STARTUP - APP_PORT - NODE_VERSION 2. docker run: - port mapping - volume 4. npm install 5. npm start or node app.js 6. access application
  26. 26. VALIDATETWEETPOD.YAML K8S FILE
  27. 27. VALIDATETWEETSERVICE.YAML K8S FILE
  28. 28. SHARED PLATFORM CAPABILITY • Microservices are isolated • Not aware of each other (except through public APIs) • Not sharing private resources • Ideally each microservice brings its own platform • To prevent run time environment from being out of synch and creating dependency/impact between multiple platform users • However: At some level, sharing is inevitable: Storage, Compute, power supply, building • In practice: having full blown RDBMS or Java EE server or Kafka cluster as part of a microservice may be unfeasible • Even if Docker images are light weight from layering – the run time resource usage is probably not – and there could be an issue with software licenses • Provide generic ‘heavy duty’ platform capabilities, available for use in any microservice in a standardized way
  29. 29. MICROSERVICES CROSS PLATFORM CAPABILITIES • Authentication • Persistent Storage • Cache • Load balancing/API Gateway • Service Discovery/Lookup • Monitoring • Functional/Business KPIs • Non Functional Platform/Container & Infra • Audit, Usage tracking, Billing • Notifications and alerting • Logging • Relational Database Capability Microservices Platform API API UI API UI Logging Cache Authentication Notification Usage Tracking
  30. 30. EXAMPLE SYSTEM ARCHITECTURE Microservices Platform API API Logging Cache API API UI HTML 5 Web Component REST/ JSON Authentication API UI Java / Spring Boot NodeJS & Express & MongoDB Redis Widgets REST/ JSON Storage Python & MySQL REST/ JSON WebLogic & Oracle Database Legacy Application API UI Strangler NodeJS & Express Notification Usage Tracking
  31. 31. TRENDS, CHALLENGES, COMMON • Use of containers and container management • Docker Containers – layered, packaged & shippable, registry • Docker Container Management: Composer, Mesos, Swarm or Kubernetes • Application Container platform such as Google App Engine, Azure App Service, Oracle Application Container Cloud, AWS Beanstalk • Serverless computing – AWS Lambda, Oracle Fn, Azure Functions • Use of cache for [state of] longer running conversations • Transaction, session, workflow, business process • New ways to consider data • Every microservice owns the data it requires – data denormalization and duplication of data across microservices is a logical consequence • Command Query Responsibility Segregation (CQRS) and Event Sourcing • Orchestration | Choreography across microservices
  32. 32. EVENTS Producers Consumers
  33. 33. MICROSERVICES AND EVENTS • Report business events [without knowing to whom and without expecting a response] • Allowing interested microservices to respond – for example trigger serverless functions • Provide response to stateless caller – with conversation key • Choreograph cross-microservice workflow | process • Inform workflow | process orchestrator | job scheduler about activity status • Enable distributed transaction – commit and rollback/compensate • Make data events available for event sourcing • Allowing microservices to maintain their own [derived] data set • Synchronize cache refresh • Informing any microservice caching data about the need to refresh specific records • Hand systems events & metrics to monitoring service • Extreme decoupling – microservice choreography • Microservice never call each other, not even through public API; all interactions are through events
  34. 34. REQUIREMENTS FOR EVENT CAPABILITY IN MICROSERVICES PLATFORM • Provide decoupling between publisher and consumer • Generally accessible for all microservices • Across the platform • Using standardized protocols and formats for communications and event payload (http, JSON) • Scalable (handle high loads) • Available (allow speedy event publication) • Reliable (do not lose events, at least once delivery) • Event Ordering (deliver events in the order of publication) • Allow multiple, parallel consumers (each event to exactly only one of them) • Retain Event History • Event Catalog – which events are published, what do they mean and what is their payload • Harvested from microservices
  35. 35. INTRODUCING APACHE KAFKA • ..- 2010 – creation at Linkedin • Message Bus | Event Broker | Streaming Data Platform • High volume, low latency, highly reliable, cross technology Commit Log (ledger) • Scalable (#messages & #consumers), distributed, durable, strict message ordering, …. • 2011/2012 – open source under the Apache Incubator/ Top Project • Backed by Confluent – Confluent Open Source & Confluent Enterprise • Kafka is used by most many [large] corporations: • And embraced by [almost] all many software vendors & cloud providers • Client libraries available for NodeJS, Java, C/C++, Python, Ruby, PHP, Scala, Clojure, Rust, .NET, go (aka golang) and many more • Apache Kafka includes Connect, Mirror Maker, Streams • KSQL is Open Source, part of the Confluent Platform
  36. 36. KAFKA TERMINOLOGY • Topic • partition • Message • == ByteArray • Broker • replicated • Producer • Consumer • Working together in Consumer Groups: each message goes to one group member Producer Consumer Topic Broker Key Value Time Message
  37. 37. Producers Consumers Topic Broker tcp tcp
  38. 38. Node + kafka-node Kafka Namespace KAFKA ON KUBERNETES Kubernetes Dashboard Kafka Manager
  39. 39. EXTENDED API OF MICROSERVICE • Deployment API • Injectable dependencies – reference to cache, logging, storage URL, … • Configurable meta-data – run time parameters, log level, credential (key) • Interaction API • REST Resources & Operations – query and URL parameters, message formats • Events Consumed – alternative way to call | activate a microservice • Reference to entry in Event Catalog • May include reference to shared Cache Resource • Events Produced – alternative output from microservice • Event can be an asynchronous response to a stateless consumer API
  40. 40. HANDSON – PART ONE • Install Kubernetes (kubectl plus minikube on Virtual Box or <…>) • Roll out a simple microservice • And test its API • Extend the Microservices Platform with • Caching Capability (Redis) • Event Bus (Kafka) • Generic (utility) microservices (LogMonitor & Cache Inspector) • Leverage Microservices Platform capabilities from a couple of additional microservices
  41. 41. Event Bus MICROSERVICES PLATFORM Cache Cache Inspector LogMonitor
  42. 42. PART TWO MICROSERVICE CHOREOGRAPHY
  43. 43. REQUIREMENTS ON WORKFLOW • Management information on running instances and their state • Recover or fully rollback functionally failed process instances • No stuck process instances: an instance that is no longer being processed • either because of a failed or a lacking actor • No race conditions – multiple action working on the same instance should not interfere when updating state • New workflow definitions can be created without impacting running instances or changing existing microservices • New or updated actors – a microservice that can execute a step in a workflow – can be introduced without impacting other components • In general: ideally, the workflow and its constituent actors are decoupled – communication is not synchronous nor direct
  44. 44. CROSS MICROSERVICES WORKFLOW Choreography vs Orchestration – Which one is which?
  45. 45. MICROSERVICE WORKFLOW CHOREOGRAPHY • Multi step process • Long running, with state, happy & non-happy • Changes in workflow definition & state structure • Each step (potentially) in different microservice • Microservices are unaware of each other • Multiple approaches • Orchestrator – running the process by invoking the required microservices subsequently, responding either to synch response, asynch callback or event • Choreography – allow the required microservices to react to relevant events • Act when it is your turn (as determined by routing slip?) • Share state through cache with claim check in routing slip • When done, publish updated routing slip • Possibly implement compensation handler
  46. 46. DESIRED WORKFLOW Tweet about <some topic> Validate Tweet No simple retweet, no black listed words used, no known robot tweeter or otherwise excluded authors, no undesirable location Enrich Tweet Details about author, location, hashtags, acronyms and abbreviations used in tweet Add Tweet to TweetBoard Add the tweet to the top of the TweetBoard – a list of recent, relevant tweets Publish TweetBoard Publish the TweetBoard through API and UI (HTML web document) done
  47. 47. MICROSERVICES TO MAP THE WORKFLOW TO Microservices Platform API Event Bus REST/ JSON APIUI On premises Tweet Board Validate Tweet API Java SE REST/ JSON Enrich Tweet Java SE/ Node.js CacheAPI Tweet Receiver
  48. 48. CHOREOGRAPHED WORKFLOW Microservices Platform APIAPI Event Bus APIUI On premises TweetBoard Validate Tweet Tweet Receiver API Enrich Tweet Workflow Launcher Cache
  49. 49. ROUTING SLIP PUBLISHED BY WORKFLOW LAUNCHER Validate Tweet Enrich Tweet Add Tweet to TweetBoard Publish TweetBoard If NOK, then done
  50. 50. ROUTING SLIP FOR COMPLETED WORKFLOW
  51. 51. CHOREOGRAPHED WORKFLOW Microservices Platform APIAPI Event Bus APIUI On premises TweetBoard Validate Tweet Tweet Receiver API Enrich Tweet Workflow Launcher Cache
  52. 52. Event Bus MICROSERVICE CHOREOGRAPHY TOPOLOGY – STAGE ONE Workflow Launcher Tweet Validator Cache Cache Inspector LogMonitorTweet Receiver
  53. 53. Event Bus MICROSERVICE CHOREOGRAPHY TOPOLOGY Workflow Launcher Tweet Validator Tweet Enricher Tweet Board Cache Cache Inspector LogMonitorTweet Receiver
  54. 54. EXTENDING THE WORKFLOW CHOREOGRAPHY – WITH TRANSLATION OF THE TWEET Validate Tweet Enrich Tweet Add Tweet to TweetBoard Publish TweetBoard If NOK, then done Translate Tweet
  55. 55. EXTENDING THE WORKFLOW CHOREOGRAPHY – WITH TRANSLATION OF THE TWEET • Implement microservice for • Translating tweets • Participating in Workflow • Deploy microservice - with access to Event Bus & Cache • Extend Workflow Template • Add new ‘translation’ step – dependency on Enrich Tweet • Change current ‘tweetboard’ step – modify dependency to Translation Validate Tweet Enrich Tweet Add Tweet to TweetBoard Publish TweetBoard If NOK, then done Translate Tweet
  56. 56. CHOREOGRAPHED WORKFLOW Microservices Platform APIAPI Event Bus APIUI On premises TweetBoard Validate Tweet Tweet Receiver API Enrich Tweet Workflow Launcher Cache API Translate Tweet API
  57. 57. MODIFIED WORKFLOW TEMPLATE API Translate Tweet API APIUI TweetBoard
  58. 58. RESULT ON TWEET BOARD INCLUDING TRANSLATION
  59. 59. MAIN TECHNICAL CHALLENGES • Network • Dockerize NodeJS applications • Run Docker container images in Kubernetes cluster • Pass environment variables and disk volumes • Expose services through ports • Create Replica Sets to manage availability & scaling • Link NodeJS applications in Kubernetes to Kafka Cluster in VirtualBox • Producing to and consuming from Topic • Leverage in-cloud cache from Kubernetes Pod members • Share workflow instance state across microservices • Instead: use local cache (Redis) in Kubernetes • Create two-way (n-way) EventBridge between local microservices and cloud(s)
  60. 60. SUMMARY • Microservices are about rapid rollout of scalable, available functionality • (session) Stateless, Continuous deployment, horizontal scale out • One team is owner of a microservice and can evolve and deploy independently • Microservice is understandable and manageable • Microservices contain everything that is special for their implementation • External dependencies only on generic platform capabilities and public APIs • Microservices expose a public API • REST resources & operations and events (consumed and produced) • Decoupled, Event-based interaction is crucial for microservices • Cache synchronization, Monitoring, CQRS, Event Sourcing, Choreographed workflows • The microservices platform should provide an Event Bus capability • Apache Kafka is a proven, popular Event Bus implementation – available on premises and in the cloud (for example through Oracle Event Hub)
  61. 61. HANDSON – PART TWO • Workflow Choreography based on Event Bus with microservices on Kubernetes • Roll out microservices TweetReceiver & WorkflowLauncher • Invoke TweetReceiver AP – and inspect logging and cache • Rollout microservices ValidateTweet, EnrichTweet, TweetBoard • Invoke TweetReceiver AP – and inspect logging and cache • Update workflow definition in WorkflowLauncher with new activity • Invoke TweetReceiver AP – and inspect logging and cache • Hanging instances – no one can execute the new activity • Rollout microservice TranslateTweet • Inspect Logging and Cache
  62. 62. • Blog: technology.amis.nl • Email: lucas.jellema@amis.nl • : lucasjellema • : lucas-jellema • : www.amis.nl, info@amis.nl +31 306016000 Edisonbaan 15, Nieuwegein Source Code: https://github.com/lucasjellema/workshop-event-bus-microservice-choreography-may-2018
  63. 63. Kafka Namespace ADDING KSQL KSQL

×