At OOP 2018 in Munich, I presented an updated version of my talk about building scalable, mission-critical microservices with the Apache Kafka ecosystem and Deep Learning frameworks like TensorFlow, DeepLearning4J or H2O. I want to share the updated slide deck and mention a few changes. The main story is the same as in my Confluent blog post about Kafka and Machine Learning: How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka. But I also discuss a few innovations in the ecosystem of Apache Kafka and trends in ML in the last months: KSQL, ONNX, Auto ML, ML platforms from Uber and Netflix. Let's take a look into these interesting topics and how this is related to each other.

1. 1Confidential
Highly scalable machine learning and deep learning
in real time with open source frameworks
Kai Waehner
Technology Evangelist
kontakt@kai-waehner.de
LinkedIn
@KaiWaehner
www.confluent.io
www.kai-waehner.de

2. 2Apache Kafka and Machine Learning – Kai Waehner
Agenda
1) Machine Learning and Real World Applications
2) Machine Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

3. 3Apache Kafka and Machine Learning – Kai Waehner
Agenda
1) Machine Learning and Real World Applications
2) Machine Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

4. 4Apache Kafka and Machine Learning – Kai Waehner
Machine Learning
... allows computers to find hidden insights without being
explicitly programmed where to look.
Machine Learning
• Decision Trees
• Naïve Bayes
• Clustering
• Neural Networks
• etc.
Deep Learning
• CNN
• RNN
• Autoencoder
• etc.

5. 5Apache Kafka and Machine Learning – Kai Waehner
Neural Network compared to other algorithms

6. 6Apache Kafka and Machine Learning – Kai Waehner
Real World Examples of Machine Learning
Spam Detection
Search Results +
Product Recommendation
Picture Detection
(Friends, Locations, Products)
Your Company
The Next Disruption:
Google Beats Go Champion

7. 7Apache Kafka and Machine Learning – Kai Waehner
Leverage Machine Learning to Analyze and Act on Critical Business Moments
Seconds Minutes Hours
Price
Optimization
Predictive
Maintenance
Fraud
Detection
Cross Selling
Transportation
Rerouting
Customer
Service
Inventory
Management
Windows of Opportunity

8. 8Apache Kafka and Machine Learning – Kai Waehner
Live Demo – Building an Analytic Model
Neural Networks in Action
http://playground.tensorflow.org/

9. 9Apache Kafka and Machine Learning – Kai Waehner
Agenda
1) Machine Learning and Real World Applications
2) Machine Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

10. 10Apache Kafka and Machine Learning – Kai Waehner
Hidden Technical Debt in Machine Learning Systems
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf

11. 11Apache Kafka and Machine Learning – Kai Waehner
The Log ConnectorsConnectors
Producer Consumer
Streaming Engine
Apache Kafka – A Streaming Platform

12. 12Apache Kafka and Machine Learning – Kai Waehner
Apache Kafka’s Open Source Ecosystem and Machine Learning
Kafka
Streams
Kafka
Connect
Rest Proxy
Schema Registry
Go / .NET / Python
Kafka Producer
KSQL
Kafka
Streams

13. 13Apache Kafka and Machine Learning – Kai Waehner
Uber’s internal ML-as-a-Service Platform
https://eng.uber.com/michelangelo/
• Cover the end-to-end ML
workflow: manage data, train,
evaluate, and deploy models,
make predictions, and monitor
predictions
• Supports various AI
technologies: Traditional ML
models, time series forecasting,
and deep learning

14. 14Apache Kafka and Machine Learning – Kai Waehner
Netflix’ Meson: Automation Engine for ML Pipelines
https://www.infoq.com/presentations/netflix-ml-meson

15. 15Apache Kafka and Machine Learning – Kai Waehner
Agenda
1) Machine Learning and Real World Applications
2) Machine Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

16. 16Apache Kafka and Machine Learning – Kai Waehner
Languages, Frameworks and Tools for Machine Learning
There is no Allrounder!
Portable Format
for Analytics (PFA)

17. 17Confidential
Machine Learning with H2O.ai
H2O Engine
R / Python /
Scala / Flow UI
Java Code

18. 18Apache Kafka and Machine Learning – Kai Waehner
Live Demo
Use Case:
Airline Flight Delay Prediction
Machine Learning Algorithm:
Deep Learning
using Neural Networks
Technology:
H2O.ai, TensorFlow

19. 19Apache Kafka and Machine Learning – Kai Waehner
H2O Deep Water (TensorFlow, MXNet, …)
https://h2o-release.s3.amazonaws.com/h2o/rel-vapnik/1/
docs-website/h2o-docs/booklets/DeepWaterBooklet.pdf
Deep Water
(H2O + TensorFlow)
Pre-Defined Networks
+
User-Defined Networks

20. 20Apache Kafka and Machine Learning – Kai Waehner
Agenda
1) Machine Learning and Real World Applications
2) Machine Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

21. 21Apache Kafka and Machine Learning – Kai Waehner
Stream Processing
Data at Rest Data in Motion

22. 22Apache Kafka and Machine Learning – Kai Waehner
Stream Processing Pipeline
APIs
Adapters /
Channels
Integration
Messaging
Stream
Ingest
Transformation
Aggregation
Enrichment
Filtering
Stream
Preprocessing
Process
Management
Analytics
(Real Time)
Applications
& APIs
Analytics /
DW Reporting
Stream
Outcomes
• Contextual Rules
• Windowing
• Patterns
• Analytics
• Machine Learning
• …
Stream
Analytics
Index / SearchNormalization
Applying an Analytic Model
is just a piece of the puzzle!

23. 23Apache Kafka and Machine Learning – Kai Waehner
Kafka Streams (shipped with Apache Kafka)
Map, filter, aggregate,
apply analytic model,
„any business logic“
Input Stream
(Kafka Topic)
Kafka Cluster
Output Stream
(Kafka Topic)
Kafka Cluster
Stream Processing
Microservice
(Kafka Streams)
Deployed Anywhere
Java App, Docker,
Kubernetes, Mesos,
“you-name-it”

24. 24Apache Kafka and Machine Learning – Kai Waehner
When to use Kafka Streams for Stream Processing?

25. 25Apache Kafka and Machine Learning – Kai Waehner
A complete streaming microservices, ready for production at large-scale
Word
Count
App configuration
Define processing
(here: WordCount)
Start processing

26. 26Apache Kafka and Machine Learning – Kai Waehner
Use Case:
Airline Flight Delay Prediction
Machine Learning Algorithm:
Neural Network
built with H2O and TensorFlow
Streaming Platform:
Apache Kafka and Kafka Streams
Live Demo

27. 27Apache Kafka and Machine Learning – Kai Waehner
H2O.ai Model + Kafka Streams
Filter
Map
1) Create H2O DL model
2) Configure Kafka Streams Application
3) Apply H2O DL model to Streaming Data
4) Start Kafka Streams App

28. 28Apache Kafka and Machine Learning – Kai Waehner
Github Examples: Kafka + Machine Learning
https://github.com/kaiwaehner/kafka-streams-machine-learning-examples
1) git clone … 2) mvn clean package … 3) look at implementations and unit tests

29. 29Apache Kafka and Machine Learning – Kai Waehner
Online Model Training with Apache Kafka and Kafka Streams
How to improve models?
1. Manual Update
2. Automated Batch
3. Real Time
Your choice… All possible with Kafka!

30. 30Apache Kafka and Machine Learning – Kai Waehner
Caveats for Online Model Training
• Processes and infrastructure not ready
• Validation needed before production
• Slows down the system
• Only a few ML implementations à Build your own!
• Only possible for unsupervised ML (e.g. clustering)
• Many use cases do not need it
à Do it only when feasible!

31. 31Apache Kafka and Machine Learning – Kai Waehner
Key Take-Aways
à Data Scientist and Developers have to work together continuously (org + tech!)
à Mission critical, scalable production deployment is key for success of Machine Learning projects
à Apache Kafka Ecosystem for Batch and Real Time Machine Learning (Training, Inference, Monitoring)

32. 32Apache Kafka and Machine Learning – Kai Waehner
Kai Waehner
Technology Evangelist
kontakt@kai-waehner.de
@KaiWaehner
www.kai-waehner.de
www.confluent.io
LinkedIn
Questions? Feedback?
Please contact me!