Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis


Consultez-les par la suite

1 sur 41 Publicité

Plus De Contenu Connexe

Diaporamas pour vous (20)


Similaire à The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis (20)

Plus par confluent (20)


The Enterprise Service Bus is Dead! Long live the Enterprise Service Bus, Rimas Silkaitis

  1. 1. The Enterprise Kafka Summit 2016 Rimas Silkaitis
  2. 2. how Heroku leveraged Kafka to realize the original vision of the enterprise service bus (esb)
  3. 3. Rimas Silkaitis Product
  4. 4. Heroku Postgres Heroku Redis
  5. 5. What is an ESB?
  6. 6. Analyst Definitions
  7. 7. ESBs receive service consumption requests, route the requests to the correct service provider, transform the requests to a format compatible with the service provider, wait for the results, and deliver them back to the service consumer. IDC http://www.idc.com/2010st/esbcm.html
  8. 8. Uses a bus architecture for scalability and reliability, establishes a unified communications channel, provides integration, mediation and communication services and is standards based. http://redmonk.com/public/OpenSourceESBs.pdf Redmonk
  9. 9. Vendor Definitions
  10. 10. Oracle Service Bus transforms complex and brittle architectures into agile integration networks by connecting, virtualizing, and managing interactions between services and applications. http://www.oracle.com/technetwork/middleware/service-bus/overview/index.html Oracle
  11. 11. Mule ESB is a lightweight Java-based enterprise service bus (ESB) and integration platform that allows developers to connect applications together quickly and easily, enabling them to exchange data. https://www.mulesoft.com/resources/esb/what-mule-esb Mulesoft
  12. 12. How should we define it?
  13. 13. ¯_(ツ)_/¯
  14. 14. App App App App Finance HR BizOps Inventory
  15. 15. ESB Routing Mediation Messaging Event Processing Service Orchestration
  16. 16. ESB Push Centralized
  17. 17. Architecture
  18. 18. API CLI GIT Dashboard Data Runtime BizOps
  19. 19. $ heroku addons:create heroku-postgresql:standard-0 -a sushi Creating postgresql-reticulated-74861... done, ($50.00/month) Adding postgresql-reticulated-74861 to neovintage... done Setting HEROKU_POSTGRESQL_GRAY_URL and restarting sushi... done, v249 The database should be available in 3-5 minutes. ! The database will be empty. If upgrading, you can transfer ! data from another database with pg:copy. Use `heroku pg:wait` to track status Use `heroku addons:docs heroku-postgresql` to view documentation.
  20. 20. API CLI GIT Dashboard Data Runtime BizOps
  21. 21. API CLI GIT Web Data Runtime BizOps THE API IS OUR ESB!
  22. 22. Problems Emerge • Bottleneck in getting features released that touch API • Difficult to get Analytics across the entire Org • Hiring has become difficult • API is down. So is the entire platform.
  23. 23. PUSH vs PULL http://kafka.apache.org/documentation.html#design_pull
  24. 24. PUSH
  25. 25. PULL Producers Consumers
  26. 26. ESB Redefinition • Pull over Push • Consumers dictate their own consumption rate • Independent development, scaling and scheduling • Stop the world processing doesn’t exist
  27. 27. CLI GIT Dashboard API Data Runtime BizOps
  28. 28. Other Use Cases Around Heroku
  29. 29. Operational Experience
  30. 30. aggregation router aggregation router metrics-ingest route r metrics-ingest route r ZK postgres metrics-ingest router aggregation router aggregation router aggregation router sink router aggregation router aggregation router metrics-api router
  31. 31. $ heroku kafka:list -a metaas-addons === Kafka Topics on KAFKA_URL Name Messages Traffic ─────────────────────────────────── ────────────── ───────────── ROUTER_REQUEST.v1 132,823.12/sec 14.6 mb/sec DYNO_MEMORY.v1 1,478.97/sec 187 kb/sec DYNO_LOAD.v1 1,468.01/sec 153 kb/sec ROUTER_STATUS_SUMMARY.v1 987.4/sec 28.3 kb/sec SYSLOG.v1 691.9/sec 76.8 kb/sec ROUTER_LATENCY_SUMMARY.v1 678.84/sec 37.5 kb/sec ROUTER_ERROR.v1 476.12/sec 93.9 kb/sec DYNO_ERROR.v1 354.06/sec 38.5 kb/sec DYNO_LOAD_SUMMARY.v1 241.33/sec 9.73 kb/sec DYNO_MEMORY_SUMMARY.v1 219.26/sec 25.1 kb/sec DYNO_ERROR_SUMMARY.v1 73.49/sec 1.83 kb/sec ROUTER_ERROR_SUMMARY.v1 12.35/sec 276 bytes/sec __consumer_offsets 4.1/sec 405 bytes/sec ROUTER_STATUS_PARTIAL_SUMMARY.v1 0.0/sec 0 bytes/sec ROUTER_LATENCY_PARTIAL_HISTOGRAM.v1 0.0/sec 0 bytes/sec ROUTER_REQUESTS.v1 0.0/sec 0 bytes/sec
  32. 32. What do we need to do next?
  33. 33. Schema Registry
  34. 34. Security
  35. 35. as a Service
  36. 36. Creating kafka-lively-27896... done, (free) Adding kafka-lively-27896 to sushi... done Kafka cluster has been created and will be available shortly Use `heroku addons:docs heroku-kafka` to view documentation. === KAFKA_URL Name: kafka-silhouetted-79119 Created: 2016-03-24 21:21 UTC Plan: Beta 3 Status: available Version: Topics: 0 topics (see heroku kafka:list) Connections: 0 consumers (0 applications) Messages: 0.0 messages/sec Traffic: 0 Bytes/sec in / 0 Bytes/s out $ heroku kafka:info -a sushi $ heroku addons:create heroku-kafka:beta-3 -a sushi
  37. 37. Public Beta https://heroku.com/kafka
  38. 38. Come See Us At Our Booth We’re Hiring!