2. It takes two to tango! Is SQL-on-Hadoop the next big step?

3. Big Data Crunching A Retrospective

4. Three Phases

5. What was it like before Hadoop?
ThePhylogeneticTreeofElephants

6. Partitioned or Sharded RDBMSs
Data Warehouses
Massively Parallel Databases
Tech before Hadoop

7. Massively Parallel Databases
Shared Nothing Architecture

8. Hadoop - Early days

9. Acceptance Life Cycle
Acceptance
Exploration
Resistance

10. Complementary over Competitive

11. Split by Structure

12. What’s the best way to answer questions that span these
two worlds?
Can we interface SQL atop Hadoop?
Can we combine the strengths of parallel databases with
those of Hadoop?

13. SQL-on-Hadoop : Technology

14. Distributed Query Processing
Cloudera’s Impala
MapR supported Apache Drill and more..
Split Query Processing
Microsoft Polybase
Hadapt
SQL-on-Hadoop : Technical Approaches
Faster Hive
Hortonworks’ Stinger initiative
Qubole’s Hive-on-the-Cloud

15. Distributed Query Processing

16. Cloudera Impala : Architecture
Clients
Impala Shell JDBC/ODBC Client SQL Tools
Data Node Data Node
Impala Daemon Impala Daemon Impala Daemon
Data Node
Query Execution
Query Planning
Query Coordination
Query Execution
Query Planning
Query Coordination
Query Execution
Query Planning
Query Coordination
State StoreMetadata Catalog HDFS Name Node
Unified Metadata Store

17. Life Cycle of an Impala Query
Clients
Impala Shell JDBC/ODBC Client SQL Tools
Impala Daemon
Data Node
State StoreMetadata Catalog HDFS Name Node
Impala Daemon
Data Node
Impala Daemon
Data Node
Impala Daemon
Data Node
Coordinate Execution
Plan and Optimize
Parse Query

18. Split Query Processing

19. Polybase + PDW : Architecture
Clients
ADO.NET JDBC/ODBC Client OLEDB
PDW Engine Service DMS Controller Loader Manager SQL Server
HDFS Bridge
Compute Node
Data Move Service
SQL Server
Job Tracker
Hadoop Cluster
Name Node
Data Node
Task Tracker
Data Node
Task Tracker
Data Node
Task Tracker
PDW Cluster
SQL Server
Compute Node
Data Move Service
HDFS Bridge
Compute Node
Data Move Service
SQL Server
SQL Server
Compute Node
Data Move Service
SQL Server PDW : Architecture
Control Node

20. CREATE HADOOP_CLUSTER GSL_CLUSTER WITH
(namenode=‘hadoop-head’,namenode_port=9000,
jobtracker=‘hadoop-head’,jobtracker_port=9010);
Register the Hadoop Cluster with PDW

21. Map HDFS File to External Tables in PDW
CREATE EXTERNAL TABLE hdfsCustomer
( c_custkey!! bigint not null,
c_name!! varchar(25) not null,
c_address!! varchar(40) not null,
c_nationkey! integer not null,
c_phone! ! char(15) not null,
c_acctbal!! decimal(15,2) not null,
c_mktsegment! char(10) not null,
c_comment!! varchar(117) not null)
WITH (LOCATION='/tpch1gb/customer.tbl',
FORMAT_OPTIONS (EXTERNAL_CLUSTER = GSL_CLUSTER,
EXTERNAL_FILEFORMAT = TEXT_FORMAT));

22. Life Cycle of a Split Query
Clients
ADO.NET JDBC/ODBC Client OLEDB
Loader Manager
Control Node
DMS Controller
Engine Service SQL Server
HDFS Bridge
Compute Node
Data Move Service
SQL Server
Hadoop Cluster
Data Node
Task Tracker
Data Node
Task Tracker
Data Node
Task Tracker
PDW Cluster
HDFS Bridge
Compute Node
Data Move Service
SQL Server
Plan
Job Tracker
Name Node
Data Node
Task Tracker

23. SQL-on-Hadoop : The Technology
Faster Hive
Distributed Query Processors
Split Query Processors

24. SQL-on-Hadoop or Map Reduce?

25. </presentation>
More on
www.systemswemake.com
Follow : @systems_we_make