Talk from JavaOne 2017: Apache Kafka + Kafka Streams for Scalable, Mission Critical Deep Learning. Intelligent real time applications are a game changer in any industry. Deep Learning is one of the hottest buzzwords in this area. New technologies like GPUs combined with elastic cloud infrastructure enable the sophisticated usage of artificial neural networks to add business value in real world scenarios. Tech giants use it e.g. for image recognition and speech translation. This session discusses some real-world scenarios from different industries to explain when and how traditional companies can leverage deep learning in real time applications. This session shows how to deploy Deep Learning models into real time applications to do predictions on new events. Apache Kafka will be used to execute analytic models in a highly scalable and performant way. The first part introduces the use cases and concepts behind Deep Learning. It discusses how to build Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Autoencoders leveraging open source frameworks like TensorFlow, DeepLearning4J or H2O. The second part shows how to deploy the built analytic models to real time applications leveraging Apache Kafka as streaming platform and Apache Kafka’s Streams API to embed the intelligent business logic into any external application or microservice. Some further material around Apache Kafka and Machine Learning: - Blog Post: How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka: https://www.confluent.io/blog/build-deploy-scalable-machine-learning-production-apache-kafka/ - Video: Build and Deploy Analytic Models with H2O.ai and Apache Kafka: https://www.youtube.com/watch?v=-q7CyIExBKM&feature=youtu.be - Code: Github Examples using Apache Kafka, TensorFlow, H2O, DeepLearning4J: https://github.com/kaiwaehner/kafka-streams-machine-learning-examples

1. 1Confidential
Kafka Streams + Deep Learning
TensorFlow and H2O Applied to Real Time Streaming Applications
Kai Waehner
Technology Evangelist
kontakt@kai-waehner.de
LinkedIn
@KaiWaehner
www.kai-waehner.de

2. 2Apache Kafka and Deep Learning – Kai Waehner
Agenda
1) Deep Learning and Real Time World
2) Deep Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

3. 3Apache Kafka and Deep Learning – Kai Waehner
Agenda
1) Deep Learning and Real World Applications
2) Deep Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

4. 4Apache Kafka and Deep Learning – Kai Waehner
Machine Learning
... allows computers to find hidden insights without being
explicitly programmed where to look.
Machine Learning
• Decision Trees
• Naïve Bayes
• Clustering
• Neural Networks
• etc.
Deep Learning
• CNN
• RNN
• Autoencoder
• etc.

5. 5Apache Kafka and Deep Learning – Kai Waehner
Real World Examples of Deep Learning
Language Translation
https://recombu.com/mobile/article/google-instant-voice-translation-the-end-of-franglais_M11397.html
http://wellroundedfashion.com/2015/01/13/trendy-tech-camouflage-from-facial-detection/
http://www.washington.edu/news/2017/07/11/lip-syncing-obama-new-tools-turn-audio-clips-into-realistic-video/
https://www.technologyreview.com/s/602796/starcraft-will-become-the-next-big-playground-for-ai/
Speech and Video Generation
Picture Analysis (Face, Location, Items, …)
Intelligent Decision Making

6. 6Apache Kafka and Deep Learning – Kai Waehner
Deep Learning to Analyze and Act on Critical Business Moments
Seconds Minutes Hours
Price
Optimization
Predictive
Maintenance
Fraud
Detection
Cross Selling
Transportation
Rerouting
Customer
Service
Inventory
Management
Windows of Opportunity

7. 7Apache Kafka and Deep Learning – Kai Waehner
Live Demo
Deep Learning in Action
http://playground.tensorflow.org/

8. 8Apache Kafka and Deep Learning – Kai Waehner
Agenda
1) Deep Learning and Real World Applications
2) Deep Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

9. 9Apache Kafka and Deep Learning – Kai Waehner
Apache Kafka Ecosystem and Deep Learning
Kafka
Streams
Kafka
Connect
Rest Proxy
Schema Registry
Go / .NET / Python
Kafka Producer
KSQL
Kafka
Streams

10. STREAMING PLATFORM
BIG DATA ANALYTICS
Oracle DB
CoaP IoT
Kafka
Java Client
…..
HP Vertica
K
A
F
K
A
H2O.ai,
TensorFlow
Batch
Real
Time
Confluent
REST Proxy
MQTT IoT
iPhone
App
Kafka
Go Client
C
K O
A N
F N
K E
A C
T
H
I
V
E
Grafana
Kafka
Java EE
Web App
Spark
C
K O
A N
F N
K E
A C
T
Confluent
Schema Registry
Kafka Streams
H2O.ai
Mesos
Kafka Streams
TensorFlow
Kubernetes
Avro
Avro
1) Data Producer
2) Analytics Platform
3) Streaming Platform
4) Data Consumer

11. 11Apache Kafka and Deep Learning – Kai Waehner
Agenda
1) Deep Learning and Real World Applications
2) Deep Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

12. 12Apache Kafka and Deep Learning – Kai Waehner
Languages, Frameworks and Tools for Deep Learning
There is no Allrounder!
Portable Format
for Analytics (PFA)

13. 13Confidential
Deep Learning with H2O.ai
H2O Engine
R / Python /
Scala / Flow UI
Java Code

14. 14Apache Kafka and Deep Learning – Kai Waehner
Live Demo
Use Case:
Airline Flight Delay Prediction
Machine Learning Algorithm:
Deep Learning
using Neural Networks
Technology:
H2O.ai, TensorFlow

15. 15Apache Kafka and Deep Learning – Kai Waehner
H2O Deep Water (TensorFlow, MXNet, …)
https://h2o-release.s3.amazonaws.com/h2o/rel-vapnik/1/
docs-website/h2o-docs/booklets/DeepWaterBooklet.pdf
Deep Water
(H2O + TensorFlow)
Pre-Defined Networks
+
User-Defined Networks

16. 16Apache Kafka and Deep Learning – Kai Waehner
Agenda
1) Deep Learning and Real World Applications
2) Deep Learning and the Apache Kafka Ecosystem
3) Building Neural Networks with TensorFlow and H2O
4) Deployment of Neural Networks with Kafka Streams

17. 17Apache Kafka and Deep Learning – Kai Waehner
Stream Processing
Data at Rest Data in Motion

18. 18Apache Kafka and Deep Learning – Kai Waehner
Stream Processing Pipeline
APIs
Adapters /
Channels
Integration
Messaging
Stream
Ingest
Transformation
Aggregation
Enrichment
Filtering
Stream
Preprocessing
Process
Management
Analytics
(Real Time)
Applications
& APIs
Analytics /
DW Reporting
Stream
Outcomes
• Contextual Rules
• Windowing
• Patterns
• Analytics
• Machine Learning
• …
Stream
Analytics
Index / SearchNormalization
Applying an Analytic Model
is just a piece of the puzzle!

19. 19Apache Kafka and Deep Learning – Kai Waehner
Kafka Streams (shipped with Apache Kafka)
Map, filter,
aggregate,
apply analytic
model,
„any business logic“
Input Stream
(Kafka Topic)
Kafka Cluster
Output Stream
(Kafka Topic)
Kafka Cluster
Stream Processing
Microservice
(Kafka Streams)
Deployed Anywhere
Java App, Docker,
Kubernetes, Mesos,
“you-name-it”

20. 20Apache Kafka and Deep Learning – Kai Waehner
When to use Kafka Streams for Stream Processing?

21. 21Apache Kafka and Deep Learning – Kai Waehner
A complete streaming microservices, ready for production at large-scale
Word
Count
App configuration
Define processing
(here: WordCount)
Start processing

22. 22Apache Kafka and Deep Learning – Kai Waehner
Use Case:
Airline Flight Delay Prediction
Machine Learning Algorithm:
Neural Network
built with H2O and TensorFlow
Streaming Platform:
Apache Kafka and Kafka Streams
Live Demo

23. 23Apache Kafka and Deep Learning – Kai Waehner
H2O.ai Model + Kafka Streams
Filter
Map
1) Create H2O DL model
2) Configure Kafka Streams Application
3) Apply H2O DL model to Streaming Data
4) Start Kafka Streams App

24. 24Apache Kafka and Deep Learning – Kai Waehner
Github Examples: Kafka + Deep Learning
https://github.com/kaiwaehner/kafka-streams-machine-learning-examples
1) git clone … 2) mvn clean package … 3) look at implementations and unit tests

25. 25Apache Kafka and Deep Learning – Kai Waehner
Online Model Training with Apache Kafka and Kafka Streams
How to improve models?
1. Manual Update
2. Automated Batch
3. Real Time
Your choice… All possible with Kafka!

26. 26Apache Kafka and Deep Learning – Kai Waehner
Caveats for Online Model Training
• Processes and infrastructure not ready
• Validation needed before production
• Slows down the system
• Only a few ML implementations à Build your own!
• Many use cases do not need it
à Do it only when really needed!

27. 27Apache Kafka and Deep Learning – Kai Waehner
Key Take-Aways
à Data Scientist and Developers have to work together continuously (org + tech!)
à Mission critical, scalable production deployment is key for success of Machine Learning projects
à Apache Kafka Ecosystem for Batch and Real Time Machine Learning (Training, Inference, Monitoring)

28. 28Apache Kafka and Deep Learning – Kai Waehner
Confluent KSQL à www.confluent.io/product/ksql
Use SQL-like queries to write stream processing applications with Kafka Streams
Leverage Kafka and Kafka Streams from anywhere without coding
• Streaming ETL
• Anomaly Detection
• Real Time Monitoring
• etc.
SELECT STREAM
CEIL(timestamp TO HOUR) AS timeWindow, productId, COUNT(*) AS hourlyOrders, SUM(units) AS units
FROM Orders GROUP BY CEIL(timestamp TO HOUR), productId;
timeWindow | productId | hourlyOrders | units
------------+-----------+--------------+-------
08:00:00 | 10 | 2 | 5
08:00:00 | 20 | 1 | 8
09:00:00 | 10 | 4 | 22
09:00:00 | 40 | 1 | 45
... | ... | ... | ...
à … maybe also use it to apply analytic models via user defined functions? J

29. 29Apache Kafka and Deep Learning – Kai Waehner
Kai Waehner
Technology Evangelist
kontakt@kai-waehner.de
@KaiWaehner
www.kai-waehner.de
LinkedIn
Questions? Feedback?
Please contact me!