4. Hands Up
Raise your hand if…
● ...you know what Stream Processing is?
● ...you have ever used Stream Processing?
5. Hands Up
Raise your hand if…
● ...you know what Stream Processing is?
● ...you have ever used Stream Processing?
● ...you have ever used Hazelcast Jet?
6. Agenda
● Part 1: Stream Processing Basics
○ What is Stream Processing and Hazelcast Jet?
○ Example: Word Count
● Part 2: Jet Under the Hood
○ How does it work?
○ Infinite Streams
○ Example: Twitter Cryptocurrency Analysis
● Part 3: Jet in the Cloud
○ Cloud (Kubernetes) integration
○ Example: Stock Trade Aggregator
● Part 4: Jet Features & Use Cases
○ Why would I need it?
○ Example: Web Crawler
15. Example 1: Word Count
Problem:
Count the number of occurrences of each word in the given text.
Sample Input:
Lorem ipsum dolor, dolor.
Sample Output:
lorem=1
ipsum=1
dolor=2
16. Example 1: Word Count
Pure Java
Pattern delimiter = Pattern.compile("W+");
return lines.entrySet().stream()
.map(e -> e.getValue().toLowerCase())
.flatMap(t -> Arrays.stream(delimiter.split(t)))
.filter(word -> !word.isEmpty())
.collect(
groupingBy(
identity(),
counting()));
33. How does it work?
Under the Hood:
● Generate DAG representation from Pipeline
● Serialize DAG
● Send DAG to every Node
● Deserialize DAG
● Executes DAG on each Node
38. Example 2: Twitter Cryptocurrency Analysis
Problem:
Present in real-time the sentiments about cryptocurrencies
Input:
Tweets are streamed from Twitter and categorized by coin type
(BTC, ETC, XRP, etc)
Output:
Tweets sentiments (last 30 sec, last minute, last 5 minutes)
39. Example 2: Twitter Cryptocurrency Analysis
Demo:
https://jet.hazelcast.org/demos/
48. Jet in the Cloud: deploying on k8
$ helm install stable/hazelcast-jet
49. Jet in the Cloud: deploying on k8
$ kubectl scale <name> --replicas=6
50. Example 3: Stock Trade Aggregator
Problem:
Present in real-time the aggregated trade price of stocks
Input:
Stock trades with name and price
Output:
Sum of prices per stock name
51. Example 3: Stock Trade Aggregator
Demo:
https://github.com/hazelcast/hazelcast-jet-code-samples/t
ree/master/integration/kubernetes