Lightning talk given at the Big Data Camp in Santa Clara on June 28th, 2011.

1. Big Data and Dataflow:Made for each other

2. The Need for Other Compute Models “… in addition, these data stores often expose a proprietary interface for application programming (e.g. PL/SQL or TSQL), but not the full power of procedural programming. More programmer-friendly parallel dataflow languages await discovery, I think. MapReduce is one (small) step in that direction.” Engineer-to-Engineer Lectures Jeff Hammerbacher June 2010 2

3. Support for Other Programming Paradigms “MapReduceNextGen provides a completely generic computation framework to support MapReduceand other paradigms.” The Next Generation of Apache Hadoop MapReduce Arun C Murthy February 2011 3

4. What is dataflow Based on operators that provide a specific function (nodes) Data queues (edges) connecting operators High Productivity Message Passing Architecture Natural Fit for Big Data 4 find grep awk sort

5. Where it’s been applied Bioinformatics Next Generation Sequencing Nearly 1 TCUP throughput using Smith Waterman Scalable BFAST implementation Telecom Analyzing Call Data Records (network logs) Operational intelligence Fraud and waste detection Public Sector State income tax revenue recovery Cyber security Financial Services Mortgage analysis Healthcare Claims processing and analysis Fraud detection Network Analyzing network log data Cyber security 5

6. Lends itself to graphical programming 6

7. Coming Soon … Community Edition Write mapper/reducer using dataflow constructs Simple and efficient Handles details of formats, data types, record parsing, serialize/deserialize, partition/sort FREE! FREE! FREE! 7 Hadoop Distributed File System Mapper Mapper Mapper Mapper DataRush DataRush DataRush DataRush Reducer Reducer DataRush DataRush

8. Integration with Hive Integrates with Hive Distributed DataRush for query execution Increases execution efficiency, lowers latency Looking for early adopters 8

9. Come to Austin! 9 We are hiring!