H2O World 2015 - Viral Bajria @ 6sense

1. Viral Bajaria, CTO & Co-Founder
@viralbajaria
@6senseInc
#h2oworld #6sense
#bestpresentationyet

2. Let’s start with a prediction
BEST TALK @ H2O WORLD!!BEST TALK @ H2O WORLD!!

3. ONLY 1 FEATURE!
BEST TALK @ H2O WORLD
TOP FEATURE: SAYS MY MOM

4. PREDICTION ?
BEST TALK @ H2O WORLD
LOW PRECISION, LOW RECALL

5. WE FIND PROSPECTS THAT ARE IN MARKET TO BUY
WE ARE THE CENTRAL NERVOUS SYSTEM
EMPOWERING ALL MARKETING, SALES AND BIZ
6sense
EMPOWERING ALL MARKETING, SALES AND BIZ
OPERATIONS TEAMS
AS A TEAM, WE LIVE ON: DATA, STATISTICS AND
BEER

6. CTO & CO-FOUNDER @ 6SENSE
EARLY HADOOP ADOPTER (LATE 2008)
about.me
3B+ EVENTS PER DAY
FUN FACT: Used a sledgehammer to unrack my first
hadoop cluster

7. Predict who is in-market to buy!!
eg: Company XYZ is 90% going to buy routers in next
90 days.
Problem
90 days.
What kind of data do we need…. A lot!

8. 1st Party:
- Web (eg. apache logs)
- Marketing Automation (eg. Eloqua)
- CRM (eg. Salesforce)
Data Needs
- CRM (eg. Salesforce)
6sense Data Network:
- Publishers
- Ads
- Blogs

9. Research patterns are different for different products
- Expensive routers
Insights
- Expensive routers
- Freemium cloud services
- Open source tools (think H2O)

10. Need to build different models for each product
Data Science Needs
Plus, we don’t like to make our life’s easy :)
- Where’s the fun in easy ?
- Need to build 4 models per product

11. Need to build different models for each product
Data Science Needs
Plus, we don’t like to make our life’s easy :)
- Where’s the fun in easy ?
- Need to build 4 models per product
100’S OF MODELS IN PROD

12. Data Sync Pipeline

13. Data Sync Pipeline

14. Pre Processing Pipeline
MOST IMPORTANT

15. Processing Pipeline
Web
Identify
Companies
Identify
Contacts
Customer
Contacts
Sales
Normalize
Companies
Custom
Data Set
Make
Consistent

16. Modeling
Baseilne
Model
Model
Stats
Modeling
Predictive
Model

17. Scikit-Learn or H2O
Output Types: pickle files or pojo
Modeling
Output Types: pickle files or pojo
Script to promote model to production
Puts all artifacts used in s3
eg: data, stats, queries

18. Modeling
Model Info
• Name
• Type• Type
• Binary Location
• Active
• ……..

19. Multiple Models for same prediction
Model 1 Model Stats
Continue
Prod Pipeline
Model 2 Model Stats
Model 3 Model Stats

20. Same pipeline as before…
Output written to temporary tables
use templating to switch settings at runtime
Experimental Modeling
use templating to switch settings at runtime
Stats compared to production runs
top decile
raw data for top-100 items

21. Platform : AWS
Backend: Hadoop, Hive, Presto, Redshift… and a lot more
Tech Stack
ML: H2O, Scikit-Learn
Ops: Fabric, Mesos, Docker, Marathon and home-grown
tools

22. Questions ??
JOBS@6SENSE.COM

23. THANK YOU!
VIRAL BAJARIA, CTO & CO-FOUNDER
viral@6sense.com
@viralbajaria
@6senseInc