Today, a growing number of commodity devices, like mobile phones or smart meters, is equipped with rich sensors and capable of producing continuous data streams. The sheer amount of these devices and the resulting overall data volumes of the streams raise new challenges with respect to the scalability of existing stream processing systems. At the same time, massively-parallel data processing systems like MapReduce have proven that they scale to large numbers of nodes and efficiently organize data transfers between them. Many of these systems also provide streaming capabilities. However, unlike traditional stream processors, these systems have disregarded QoS requirements of prospective stream processing applications so far. We intend to address this gap. First, we analyze common design principles of today's parallel data processing frameworks and identify those principles that provide degrees of freedom in trading off the QoS goals latency and throughput. Second, we propose a scheme which allows these frameworks to detect violations of user-defined latency constraints and optimize the job execution without manual interaction in order to meet these constraints while keeping the throughput as high as possible. As a proof of concept, we implemented our approach for our parallel data processing framework Nephele and evaluated its effectiveness through a comparison with Hadoop Online. For a multimedia streaming application we can demonstrate an improved processing latency by factor of at least 15 while preserving high data throughput when needed. The Stratosphere Streaming Distribution, that implements the researched techniques, is available as open-source via github.com: https://github.com/bjoernlohrmann/stratosphere More about me: http://www.cit.tu-berlin.de/menue/personen/lohrmann_bjoern/parameter/en/

1. Massively-Parallel
Stream Processing
Under QoS Constraints
with Nephele
Björn Lohrmann, Daniel Warneke,
and Odej Kao
Technische Universität Berlin

2. Background
Nephele is part of the Stratosphere platform for
massively-parallel data processing
in
in
map
red
match
out
Cloud
PACTs
Compiler
Nephele
Runtime
Cluster
Open Source, downloadable at stratosphere.eu
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
2

3. Background
Nephele and PACTs currently focus on batch-job
workloads
-to-
What about streaming workloads?
Possible with Nephele, but (as of now) not PACTs
May have different goals
Meet pipeline latency and throughput requirements
Max/Min other custom metrics
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
3

4. Motivation
Live Processing of streamed data is an important issue
Proliferation of mobile devices capable of producing
streamed data (video, audio, other sensors)
Large Scale Deployments of Sensors in Science and
Industry
Examples: Smart Grids, Traffic Monitoring, Astronomy
Why not adapt todays mass.-parallel frameworks?
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
4

5. Goals
Identify major aspects of massively-parallel
frameworks that affect QoS goals
Find general strategies to deal with QoS goals
Implement & Evaluate them using the Nephele
Execution Engine
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
5

6. Agenda
1. Highlight common mass.-parallel framework design
principles
2. Explain implications for streamed workloads
3. Meeting latency requirements in Nephele
4. Experimental Results
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
6

7. Framework Design
Principles
Compute Node X
Task
n
Task
n+1
Compute Node Y
Task
n
Task
n+1
Task
n
Compute Node Z
Task
n
Input Buffer
Queue
22.06.2012
Task
n+1
Output
Buffer
Data
Item
Thread/Process
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
7

8. Implications for Streaming
Applications
Large buffer = high tp, high latency
Small buffer = low tp, low latency
Trade-off needs to be found to meet latency goals
Thread/Process Model
1 Task= 1 Thread model is flexible, but has overhead
Thread scheduling, synchronization, communication
Serialization may be necessary (bad for TP & latency)
N Tasks = 1 Thread model can sometimes provide
better better tp and latency
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
8

9. Meeting Latency
Requirements
QoS goal:
Meet latency constraint X, then maximize throughput
Based on observations we designed two strategies:
1. Adaptive Output Buffer Sizing
2. Dynamic Task Chaining
Both strategies
work autonomously (only latency constraint is required)
are applied on-demand at runtime
are applicable in systems with similar design principles
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
9

10. Adaptive Output Buffer
Sizing
Only applied when latency constraint violated
For each channel
Determine output buffer latency (obl)
If obl > threshold, decrease buffer size:
size : max( , size r obl )
r 0.98,
200
If obl < threshold, increase buffer size again
size : min( , size r obl )
r 1.1,
22.06.2012
500 103
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
10

11. Task Chaining
Again, only applied when overall latency constraint is
violated
Compute Node
Conditions:
Task
n
Task
n+1
Compute Node
Task
n
18.11.2013
Task
n+1
Autor - Vortragstitel
Pipeline of unchained
tasks
Sum of CPU utilizations
is < 90% of capacity of
one core
Only apply to longest
chainable pipeline of
tasks
11

12. Complete System Overview
300ms
JM
Periodical measurements
(latency, throughput)
TM
22.06.2012
TM
TM
Buffer Size Updates,
Chain Commands
TM
TM
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
TM
TM
12

13. Sample Application: Video
Livestreaming
Node 1
Node 2
RTP
Server
Node n-1
Node n
Encoder
Overlay
Merger
Decoder
Partitioner
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
13

14. Latency w/o Optimizations
Setup:
10 nodes, 80 cores
32 KB output buffer
size
320 video streams
Results:
Latency oscillates
around 4s
Large buffers cause
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
14

15. Latency w/ Adaptive Buffer
Sizing
Final Latency:
improvement)
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
15

16. Latency /w ABS+TC
Final Latency:
improvement)
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
16

17. Conclusion and Future Work
Massively-parallel frameworks can be adapted to do
latency constrained stream processing
Prototype implementation on Nephele showed up to
94% latency improvement on video livestreaming job
Future Work
Distribute latency monitoring (better scalability)
Adapt PACT layer of Stratosphere to provide streaming
capabilities and latency awareness
22.06.2012
Massively-Parallel Stream Processing Under QoS Constraints with Nephele
17