Feature engineering- writing code to map raw input data into a set of signals that will be fed into a machine learning algorithm- is the dark art of data science. Although the process of crafting new features is tedious and failure-prone, the key to a successful model is a diverse set of high-quality features that are informed by domain experts. Recently, academic researchers have begun to focus on the problem of feature engineering, and have started to publish research that addresses the relative lack of tools that are designed to support the feature engineering process. In this talk, I will review some of my favorite papers and present some efforts to convert these ideas into tools that leverage the principles of reactive application design in order to make feature engineering (dare I say it) fun.

1. 1© Cloudera, Inc. All rights reserved.
Engines, Models, and Algorithms for Building Data Products
Josh Wills | Senior Director of Data Science
Data Science in Action

2. 2© Cloudera, Inc. All rights reserved.
About Me

3. 3© Cloudera, Inc. All rights reserved.
Think Like A Data Scientist:
SQL, MapReduce, and Spark

4. 4© Cloudera, Inc. All rights reserved.
1. Read and deserialize input data.
2. Project/filter input records.
3. Shuffle: serialize it, send over the
network, deserialize it.
4. Apply aggregation logic.
5. Serialize output data.
The Life of a Data Processing Job

5. 5© Cloudera, Inc. All rights reserved.
• Most of the time in any data
processing job is spent
serializing/deserializing data
• Two ways we can pay this cost
• CPU (Compressed data)
• I/O (Uncompressed data)
• The different engines have different
strategies for handling this cost
Handling the Cost of Serialization

6. 6© Cloudera, Inc. All rights reserved.
The Traditional RDBMS Approach

7. 7© Cloudera, Inc. All rights reserved.
The Cost of The Traditional RDBMS Approach

8. 8© Cloudera, Inc. All rights reserved.
Query Scheduling and Exploratory Data Analysis

9. 9© Cloudera, Inc. All rights reserved.
The Spark Approach

10. 10© Cloudera, Inc. All rights reserved.
The Cost of the Spark Approach

11. 11© Cloudera, Inc. All rights reserved.
(Shameless Plug)

12. 12© Cloudera, Inc. All rights reserved.
The MapReduce Approach

13. 13© Cloudera, Inc. All rights reserved.
MapReduce In The Hands of a Data Scientist

14. 14© Cloudera, Inc. All rights reserved.
Example: Hive Multi-Insert

15. 15© Cloudera, Inc. All rights reserved.
Some Speculative Thoughts on the Future

16. 16© Cloudera, Inc. All rights reserved.
Data Modeling for Data Scientists

17. 17© Cloudera, Inc. All rights reserved.
• Data modeling is the process by which
we organize our data in order to meet
our business requirements
• Historically, this has been a relational-
centric design exercise
• Given the properties of these
different engines and our goals, how
should we organize our data?
Data Engines and Data Modeling

18. 18© Cloudera, Inc. All rights reserved.
SQL Engines and the Dimensional Data Model

19. 19© Cloudera, Inc. All rights reserved.
NoSQL Data Models: Normalization and Aggregation

20. 20© Cloudera, Inc. All rights reserved.
Motivating Example: Spelling Correction

21. 21© Cloudera, Inc. All rights reserved.
Event Series Analytics

22. 22© Cloudera, Inc. All rights reserved.
A Simple Star Schema for Spell Correction

23. 23© Cloudera, Inc. All rights reserved.
• What parameters does this model
need…
• during the analysis phase?
• during deployment?
• Some Candidates
• Lag time between events
• Similarity of queries
• What else?
Designing the Spell Correction Data Product

24. 24© Cloudera, Inc. All rights reserved.
A Supernova Schema for Search

25. 25© Cloudera, Inc. All rights reserved.
Spell Correction in SQL

26. 26© Cloudera, Inc. All rights reserved.
Exhibit: http://github.com/jwills/exhibit

27. 27© Cloudera, Inc. All rights reserved.
An Illustrative Example

28. 28© Cloudera, Inc. All rights reserved.
Build A Supernova Table

29. 29© Cloudera, Inc. All rights reserved.
The Resulting Table Schema

30. 30© Cloudera, Inc. All rights reserved.
A Simple Aggregation

31. 31© Cloudera, Inc. All rights reserved.
A More Interesting Aggregation

32. 32© Cloudera, Inc. All rights reserved.
• Crunch-based MapReduce job that
uses a YAML file to execute a series of
computations over a given Hive table
• Each run may generate multiple
output tables
• Each output table is composed of
one or more aggregations
• SQL/Javascript Code
• Key Fields
• Types: SUM, QUANTILE, etc.
Exhibit ETL Engine

33. 33© Cloudera, Inc. All rights reserved.
• Your first supernova should be used
to support batch reporting and model
building operations
• Apache HBase is a supernova data
model with real-time update
capabilities
• BigTable was originally used to
handle real-time updates to
DocJoins for web indexing
FAQ: How Do You Handle Updates?

34. 34© Cloudera, Inc. All rights reserved.
FAQ: What Data Belongs Inside of the Supernova?

35. 35© Cloudera, Inc. All rights reserved.
Pushing The Limits: The TPC-H Benchmark

36. 36© Cloudera, Inc. All rights reserved.
Query Hive MR (Seconds, w/# of MR Jobs) Impala (Seconds)
1 228 (1 MR) 24
3 462 (3 MR) 115
4 232 (2 MR) 147
6 85 (1 MR) 6
10 502 (3 MR) 50
12 241 (2 MR) 40
13 228 (2 MR) 127
18 859 (4 MR) 189
TOTAL 2837 (18 MR) 698
TPC-H: Hive vs. Impala (Floratou et al., 2014)

37. 37© Cloudera, Inc. All rights reserved.
Data Modeling on Spark

38. 38© Cloudera, Inc. All rights reserved.
• Graph algorithms on Spark
• Developed at AMPLab, now part of
core project
• Created by Joseph Gonzalez, one of
the founders of Graphlab/Dato
• Uses custom partitioned VertexRDD
and EdgeRDD structures to
represent graph data and apply
graph algorithms like PageRank,
connected components, etc.
GraphX

39. 39© Cloudera, Inc. All rights reserved.
• Spark Time Series Project
• Utilities for munging time series data
and applying statistical tests
• Similar to MATLAB/Pandas
• EWMA
• Autoregressive models
• Missing data imputation
• https://github.com/cloudera/spark-
timeseries
Spark for Time Series Data

40. 40© Cloudera, Inc. All rights reserved.
Executing A Data Product Strategy

41. 41© Cloudera, Inc. All rights reserved.
Aligning Data Strategy with Product Strategy

42. 42© Cloudera, Inc. All rights reserved.
• Google has built a factory for creating
data products
• Spell correction engine started as a
one-off, resource-intensive data
analysis job
• But it was generalized to power
multiple data products
• Oneboxes
• Search recommendations
• Personalized search
The Data Products Factory

43. 43© Cloudera, Inc. All rights reserved.
Public Transit for Questions

44. 44© Cloudera, Inc. All rights reserved.
Building Public Transit Is Expensive…

45. 45© Cloudera, Inc. All rights reserved.
• Our data infrastructure is evolving
into a major metropolitan area
• More data producers and
consumers
• Diverse needs
• Major metropolitan areas without
significant mass transit systems:
• Los Angeles
• Atlanta
• Jakarta
…but it beats the alternative

46. 46© Cloudera, Inc. All rights reserved.
Going To Real Time

47. 47© Cloudera, Inc. All rights reserved.
• Lots of approaches to building
systems that create real-time data
cubes
• Summingbird from Twitter
• Kylin from eBay
• Pinot from LinkedIn
• Lots of volume and velocity, but not
so much variety
• Excellent for reporting, less useful for
machine learning
Real-Time Systems and Counting Stuff

48. 48© Cloudera, Inc. All rights reserved.
The Operational/Analytical Impedance Mismatch

49. 49© Cloudera, Inc. All rights reserved.
NoSQL Data Models Revisited

50. 50© Cloudera, Inc. All rights reserved.
• The challenge in taking machine
learning models to production is the
feature engineering, not the model
itself
• Ensuring all data is available
• Verifying that features are
calculated the same way in both
environments
• One data model with multiple
backend implementations
• Offline vs. Online
Exhibit-Style Machine Learning

51. 51© Cloudera, Inc. All rights reserved.
Thanks!
jwills@cloudera.com
@josh_wills