Introduction to Real time event processing, Storm and Trident

1. 1
Introduction to Storm
Vinoth Kumar Kannan
Vinoth.kannan@widas.de
Introduction & Concepts

2. Storm is a free and open source distributed realtime
computation system
2
What is Storm
Hadoop : Batch Processing :: Storm : Real-Time Processing
Provides general primitive to do real time computation
Scalable and Fault-tolerant
Guaranteed Message Processing atleast once
Can be used with any programming language

3. 3
Comparison of Hadoop vs Storm

4. 4
Concepts
What does it do?
Stream Processing Continuous Computation DRPC

5. 5
Concepts
What is a Stream ?
Tuple Tuple Tuple Tuple Tuple
Streams
Unbounded sequence of tuples

6. 6
Concepts
What is a spout?
Source of Stream is called Spout

7. 7
Concepts
What are Bolts?
Bolts Processes input Stream and produces a new Stream
Tuple Tuple
Bolts

8. 8
Concepts
What are Bolts?
Bolts
• Functions
• Filters
• Aggregation
• Joins
• Database connections

9. 9
Concepts
What is a topology ?
Network of Spouts and Bolts
Topology

10. 10
Concepts
What are Grouping
Used to decide which task in the subscribing bolt, the tuple is sent to

11. 11
DRPC
Distributed Remote Procedure Call
DRPC parallelizes the computation of really intense
functions on the fly using Storm
The Storm topology takes in as input a stream of
function arguments, and it emits an output stream of
the results for each of those function calls.
DRPC is not so much a feature of Storm as it is a
pattern expressed from Storm's primitives of streams,
spouts, bolts, and topologies.

12. 12
DRPC
Distributed Remote Procedure Call

13. 13
Trident
What is Trident?
Trident is a high-level abstraction for doing realtime
computing on top of Storm.
Similar to Cascading or Pig in Hadoop
Easier to build topologies
Trident has joins, aggregations, grouping, functions,
and filters
Trident lets you express realtime computations in a
natural way while still getting maximal performance.

14. 14Vinoth.kannan@widas.de
Thank You