This tutorial gives out an brief and interesting introduction to modern stream computing technologies. The participants can learn the essential concepts and methodologies for designing and building a advanced stream processing system. The tutorial unveils the key fundamentals behind various kinds of design choices. Some forecast of technology developments in this domain is also introduced at the last section of this tutorial.

1. Play With Streams
chentianjian@baidu.com
Jun, 2014

2. About Me
• Principal Architect
@Baidu.com
• Contract Programmer
– C/C++/Python
• Post Engineering
Disorder Therapist

3. Post Engineering Disorder

4. Murphy’s Law
• Everything that Can
Go Wrong, Goes
Wrong
– Unstable Servers
– Unstable Network
– Unstable Data Source
– Unstable Managers
– Unstable…

6. Agenda
• Section I: A Scratch
– Brief Intro to Streams
• Section II: Build From Ground Up
– Modern Stream Processing Architecture
• Section III: Could Be Much Sexier
– Stream Evolution in Progress

7. Section I: A Scratch
Brief Intro to Streams

8. Highlights
• Natural Beauty of Streams
• Stream Processing Design Language(SDL)
• Scratching Stream Applications

9. Streams are Everywhere

10. Definition of Stream
• A Series Of Data Packs
• Data is Structured or Semi-Structured
• With Internal Topologies
– DAG in most cases

11. Traffic Cam Network

12. Wireless Sensor Network

13. Recommender System

14. Stream Description Language
• Normalize
• Accumulate
• Join
• Partition
• Filtering

15. Normalize
NORMMessage
Type A
Message
Type B
Standard
Message

16. Accumulate
ACCU
BUFFER
Message Accumulated
Buffers

17. Common Accumulator Types
• Time Window
• Volume
• Condition Trigger

18. Join
JOIN
Indexed Info.
Message
With a key
Joined Message

19. Common Join Rules
• Internal Mode
– Indexing Only Input Messages
– Limited Time Window
• External Mode
– Indexing External Data
– Extreme Useful In Integrating Asynchronous
Components

20. Partition
PARTMessage
With a key

21. Common Partition Rules
• Feature Based
– Application Related
• Random
– Balancing Load
• Hash
– Aggregating Data by User Defined Key
• Replication
– For Improving Availability

22. Filtering
FILT
Drop
Pass ThroughInput Messages

23. Pit Stop
• Normalize
• Accumulate
• Join
• Partition
• Filtering

24. Distinct Count

25. A Simple Hello World:
Word Count
Text Parser ACCU

26. An Improved Version
Text Parser PART ACCU
ACCU
ACCU
NORM

27. Sum Up!
• Streams are everywhere
• We have a modeling tool to describe stream
processing flows(SDL)
• Proceed to build real systems

28. See U In Next Hour

29. Section II: Build From Ground Up
Modern Stream Processing
Architecture

30. Highlights
• Flow Means Non-Stopping
• Reliability Matters
• Workload Fluctuation Handling

31. Message Relay System Basics
• Message Queuing
• Message Passing
• Hybrid Queuing

32. With Message Queuing
Operator 1
Queue A Queue B Queue C
Operator 2

33. Message Passing
Operator 1
Operator 2
Operator 3

34. Hybrid Queuing
Operator 1
Queue A Queue B
Operator 2
Operator 3

35. Pit Stop
• Message relay is the foundation of stream
processing
• 3 basic methods of message relay

36. Why Reliability Matters?
• We got mission critical applications
– Real time stock exchange analysis
– Ads network click monitoring & billing

37. Reliability Solutions
• Upstream Backup & Replay
• Source Backup & Replay
• Processing Status Backup & Replay

38. Upstream Backup & Replay
Upstream Downstream Upstream Downstream
ACK ACK

39. Source Backup & Replay
Upstream Downstream Upstream Downstream
ACK
ACK

40. Processing Status Backup & Replay
Upstream
Downstream Upstream
Downstream
Stream Operator
Downstream Upstream
Stream Operator Shadow
Status
Synchronize
Messages
Redirect
Redirect

41. While Reliability Hurts Performance
• All Reliability Solutions are Based on
– Indexing
– Snapshot
– Replay
• A Club Of Performance Penalties

42. Tuning Strategies
• State Operators VS. Stateless Operators
– Independent State Storage
– White Board Programming Model
– Lazy State Synchronization
• Micro Batching Snapshot

43. Micro Batching
Batch Size Time Window Throughput Snapshot Cost Restore Cost
1 1ms 1x very high very low
10 10ms 10x high low
100 100ms 100x medium low
1000 100ms 1000x medium low
10000 1s 5000x low high
Most Systems Are Here
May Constrained By
Network Configuration

44. Pit Stop
• Reliability is based on
– Message Backup & Replay
– Status Snapshot
• When tuning, think of
– How to handle operator status
– Micro Batching

45. Workload Fluctuation

46. How Does Fluctuation Happen?
• Data Source Fluctuation
• Fault Tolerance Operations

47. Fluctuation Handling
• Technologies To Obtain
– High Performance RPC Framework
– Auto Partitioning
– Dynamic Resource Allocation
– Global Flow Control

48. High Performance RPC Framework
• Indication of High Performance
– Over 20k QPS/sec, with 1byte payload
– On commodity server with 2 6-core CPU and Giga
Ethernet
• See Also
– SOFA Framework from Baidu.com
– https://github.com/BaiduPS/sofa-pbrpc

49. Auto Partitioning
• Adjust Sub-Stream Num.
• Tricks:
– Consistent Hashing
– Exponential Allocation

50. Dynamic Resource Allocation
• Tree Model
• Evaluation
Function
• Network
Constrains
IDC
A
IDC
B
PHY
1
PHY
2
PHY
3
PHY
4
VM 1
VM 2
VM 3

51. Global Flow Control
Op1
Op2
Op3
Op4
Flow Control
Service

52. Pit Stop
• Workload Fluctuation Handling
– Very Fast RPC Framework
– Auto Partitioning
– Dynamic Resource Allocation
– Global Flow Control

53. Sum Up!
• Basic Architecture of Stream Processing
System
• Reliability of Stream Processing
• Ways to Handle Workload Fluctuations

54. See U in the Afternoon

55. Section III:
Could Be Much Sexier
Stream Evolution in Progress

56. Highlights
• Challenges in Real World Applications
• High Level Stream Programming
• Optimization Inside Hardware

57. Applications
• Stream Web Crawler
• Co Serving with Hadoop M/R

58. Stream Web Crawling
OP1
OP2
Redis Cluster
Web Page Cache
OP3 OP4
Log Filter
Data Join
Feature
Extraction
Logging API
Crawling Job
Updater
OP5
User-Model
Updater
OP6HBase Cluster
Web DataBase
Crawling Job
Generator
OP7 Crawling Bot
OP8
Image
Crawling Job
Generator
OP9
Cache
Synchronizer
OP
10
Image Crawling Cluster
Status
DB

59. Co-Serving
• Distribute RPC
• Dynamic
Routing
OP1
OP2
OP5
OP6
Online Web Services
Query
Preprocess
Query
Transform
Result
Merge
Intent
Extraction
Online
Query Log
OP3 OP4
User
Intent
Mining
Hadoop
Initiating DRPC from Mapper
Mapper Receiving DRPC result

60. Pit Stop
• Programming Complexity
• Network Management Complexity
• Scale-out Difficulty

61. High Level Programming
• Stream DataBase & StreamSQL
• Stream Computing Description Language
• Stream Programming Framework

62. StreamSQL

63. Stream DML Stack(Naiad)

64. Stream Framework

65. Pit Stop
Programming Interface Complexity Flexibility
Stream SQL Low Low
Stream DML Medium High
Stream Framework High Very High

66. Network Management Challenges
• Things Are App. Dependent
– QoS Request
– Relay Priority
– Security Strategy
– Resource Allocation

67. Network Management Solution
SDN
Integration
Stream App Flow Control
Network Configuration

68. SDN Integration
Stream Flow
Control Service
Monitors on
Streams
Open Flow
Supported
Network
Devices

69. Pit Stop
• Network Management is Crucial
– Stream Processing is Bandwidth Consuming
– Stream Processing may be Latency Sensitive
• Solution is Simple
– Software Defined Network Integration (SDN)

70. Scale-Out Difficulty
• Application Constrains
• Algorithm Constrains
• Network Hardware Constrains

71. So Scale-Up !
• Co-Process Device
• Re-Programmable Co-Processors

72. FPGA Computing Card
• Cheaper than GPGPU
• Flexible than GPGPU
• Higher POE than GPGPU

73. New Re-Configurable Xeon CPU

74. See Also
• A Reconfigurable Fabric for Accelerating Large-
Scale Datacenter Services
– 20x Performance
– ISCA 2014 by Microsoft Research
– http://research.microsoft.com/pubs/212001/Cata
pult_ISCA_2014.pdf

75. Pit Stop
• Scale-out is difficult, think of scale-up
• Reconfigurable CPU has got significant
performance improvements

76. Conclusion
• Stream Processing System can be Well
Modeled by SDL
• Trade Off between Reliability & Performance
• High level programming & Scale-Up are Future
Trends

77. References
• Stonebraker, Michael, Uǧur Çetintemel, and Stan Zdonik. "The 8 requirements of real-time
stream processing." ACM SIGMOD Record 34, no. 4 (2005): 42-47.
• Zaharia, Matei, Tathagata Das, Haoyuan Li, Timothy Hunter, Scott Shenker, and Ion Stoica.
"Discretized streams: Fault-tolerant streaming computation at scale." In Proceedings of the
Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 423-438. ACM, 2013.
• Murray, Derek G., Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Martin
Abadi. "Naiad: a timely dataflow system." In Proceedings of the Twenty-Fourth ACM
Symposium on Operating Systems Principles, pp. 439-455. ACM, 2013.
• Castro Fernandez, Raul, Matteo Migliavacca, Evangelia Kalyvianaki, and Peter Pietzuch.
"Integrating scale out and fault tolerance in stream processing using operator state
management." In Proceedings of the 2013 international conference on Management of data,
pp. 725-736. ACM, 2013.

78. Thank U!
• G+: chentianjian@gmail.com
• Skype: tianjian_chen
• Linkedin: http://lnkd.in/bRN6xsh