SlideShare une entreprise Scribd logo
1  sur  33
Télécharger pour lire hors ligne
Daniel




How to Interview a Data Scientist
Daniel Tunkelang
Director of Data Science, LinkedIn
     Recruiting Solutions                     1
Drew Conway’s Venn Diagram




                             2
GOAL




       3
Specification for a Data Scientist



                        implements
                         algorithms

      analyzes data
                      thinks product



                                       4
What about




C                  ulture
             ommunication
                 uriosity


     Hold that thought…
                            ?
                                5
What can you learn from an interview?




                                        6
Interviewing is a last resort.




               Alternatives?

                                 7
Only hire people you’ve worked with.




                                       8
Hire interns. Convert to full-time. Profit!




                                              9
Try before you buy: short-term contracts.




                                            10
Alternatives are at best a partial solution.

§  Only hiring people you’ve worked with doesn’t scale.
   –  And traps you in a locally optimal monoculture.


§  Interns are great! But they are a significant investment.
   –  Managing interns well is a productivity gamble.
   –  Most interns have at least a year of school left.
   –  Not all interns will make your bar. You won’t always make theirs.


§  Try before you buy: nice in theory.
   –  Adverse selection bias when other offers are permanent roles.
   –  Creates bureaucracy.


                                                                          11
Can we at least make interviews natural?




                                           12
Spend a day working together.




                                13
Take-home assignment.




                        14
Review candidate’s previous work.




                                    15
High-fructose corn syrup is 100% natural.
§  Working sessions are difficult to set up.
   –  No more natural than a final exam.
   –  High variance, and very difficult to calibrate performance.


§  Take-home assignments are great for the employer.
   –  But they are a significant investment for the candidate.
   –  Adverse selection bias if other companies don’t require them.
   –  Creates incentive to cheat if significant part of hiring process.


§  Previous work is like natural experiments.
   –  Always good to review a candidate’s previous work.
   –  But not always possible to find work with high predictive value.



                                                                          16
So you gotta do interviews. But how?




                                       17
Three Principles

1.  Keep it real.

2.  No gotchas.

3.  Maybe = no.




                    18
Keeping It Real




                  19
Test basic coding with FizzBuzz questions.

        multiple of 3 -> Fizz
        multiple of 5 -> Buzz
        multiple of 15 -> FizzBuzz

   1, 2, Fizz, 4, Buzz, Fizz,
   7, 8, Fizz, Buzz, 11, Fizz,
   13, 14, FizzBuzz, 16, …
                                        20
Whiteboards suck for coding.




      http://ericleads.com/2012/10/how-to-conduct-a-better-coding-interview/


                                                                               21
Don’t ask pointless algorithm questions.




             implement




                                           22
Use real-world algorithms questions.



        bigdatascientist


         Did you mean:
         big data scientist
                                       23
Ask candidates to design your products.




                                          24
Keeping it real is also a great sell.
                                Similar Profiles




                               People You May Know




                                                     25
But no gotchas.




                  26
Gotchas reduce the signal-to-noise ratio.

§  Avoid problems where success hinges on a single insight.
   –  Good interview problems offer lots of room for partial credit.
   –  Making a key insight often reflects experience, not intelligence.


§  Don’t test a candidate’s knowledge of a niche technique.
   –  Unless that niche technique is critical to job performance.
   –  And can’t be learned on the job as part of on-boarding.


§  Be a hard interviewer, but don’t be an asshole.
   –  An interview is not a stress-test to see where candidates break.
   –  Interviews communicate your values to the candidate.


                                                                          27
Maybe = no.




              28
Commit to binary interview outcomes.

§  Forced choice so interviewers don’t take easy way out.
   –  Just like having 4 choices instead of 5 on a rating scale.
   –  Encourages interviewers to take their role seriously.


§  Each team member is a critical filter.
   –  Two no’s or one strong no is a no.
   –  All weak yes’s is a no.


§  Short-circuit candidates early in the process.
   –  Resume and phone screening should be aggressive.
   –  Onsite interviews should have ~50% chance of leading to offers.


                                                                        29
But what about




C                 ulture
            ommunication
                uriosity

         All are must-haves.
                                ?
 Every interview evaluates all three.
                                        30
Remember Your Goal




                     31
Three Principles

1.  Keep it real.
  –  Avoid whiteboard coding. Filter with FizzBuzz.
  –  Use real-world algorithms questions.
  –  Ask candidates to design your products.
2.  No gotchas.
  –  Gotchas reduce the signal-to-noise ratio.
3.  Maybe = no.
  –  Bad hires suck. Be conservative.
  –  Trust your team.
                                                  32
Thank you!




             33

Contenu connexe

Tendances

Introduction To Data Science
Introduction To Data ScienceIntroduction To Data Science
Introduction To Data ScienceSpotle.ai
 
(BDT311) Deep Learning: Going Beyond Machine Learning
(BDT311) Deep Learning: Going Beyond Machine Learning(BDT311) Deep Learning: Going Beyond Machine Learning
(BDT311) Deep Learning: Going Beyond Machine LearningAmazon Web Services
 
Data science presentation
Data science presentationData science presentation
Data science presentationMSDEVMTL
 
Lessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsLessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsXavier Amatriain
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data scienceSampath Kumar
 
How to win data science competitions with Deep Learning
How to win data science competitions with Deep LearningHow to win data science competitions with Deep Learning
How to win data science competitions with Deep LearningSri Ambati
 
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...Edureka!
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data ScienceEdureka!
 
Graph attention network - deep learning paper review
Graph attention network -  deep learning paper reviewGraph attention network -  deep learning paper review
Graph attention network - deep learning paper reviewtaeseon ryu
 
The Evolution of Data Science
The Evolution of Data ScienceThe Evolution of Data Science
The Evolution of Data ScienceKenny Daniel
 
Continual Learning with Deep Architectures - Tutorial ICML 2021
Continual Learning with Deep Architectures - Tutorial ICML 2021Continual Learning with Deep Architectures - Tutorial ICML 2021
Continual Learning with Deep Architectures - Tutorial ICML 2021Vincenzo Lomonaco
 
Data Science Project Lifecycle
Data Science Project LifecycleData Science Project Lifecycle
Data Science Project LifecycleJason Geng
 
Synthetic Data Generation for Statistical Testing
Synthetic Data Generation for Statistical TestingSynthetic Data Generation for Statistical Testing
Synthetic Data Generation for Statistical TestingLionel Briand
 
Introduction to continual learning
Introduction to continual learningIntroduction to continual learning
Introduction to continual learningNguyen Giang
 
Data science presentation 2nd CI day
Data science presentation 2nd CI dayData science presentation 2nd CI day
Data science presentation 2nd CI dayMohammed Barakat
 
What Is Data Science? | Introduction to Data Science | Data Science For Begin...
What Is Data Science? | Introduction to Data Science | Data Science For Begin...What Is Data Science? | Introduction to Data Science | Data Science For Begin...
What Is Data Science? | Introduction to Data Science | Data Science For Begin...Simplilearn
 

Tendances (20)

Data science
Data scienceData science
Data science
 
Introduction To Data Science
Introduction To Data ScienceIntroduction To Data Science
Introduction To Data Science
 
(BDT311) Deep Learning: Going Beyond Machine Learning
(BDT311) Deep Learning: Going Beyond Machine Learning(BDT311) Deep Learning: Going Beyond Machine Learning
(BDT311) Deep Learning: Going Beyond Machine Learning
 
Data science presentation
Data science presentationData science presentation
Data science presentation
 
Lessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systemsLessons learned from building practical deep learning systems
Lessons learned from building practical deep learning systems
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data science
 
Data science
Data scienceData science
Data science
 
Introduction to data science
Introduction to data scienceIntroduction to data science
Introduction to data science
 
How to win data science competitions with Deep Learning
How to win data science competitions with Deep LearningHow to win data science competitions with Deep Learning
How to win data science competitions with Deep Learning
 
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
Data Science Training | Data Science Tutorial for Beginners | Data Science wi...
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Graph attention network - deep learning paper review
Graph attention network -  deep learning paper reviewGraph attention network -  deep learning paper review
Graph attention network - deep learning paper review
 
The Evolution of Data Science
The Evolution of Data ScienceThe Evolution of Data Science
The Evolution of Data Science
 
Continual Learning with Deep Architectures - Tutorial ICML 2021
Continual Learning with Deep Architectures - Tutorial ICML 2021Continual Learning with Deep Architectures - Tutorial ICML 2021
Continual Learning with Deep Architectures - Tutorial ICML 2021
 
Data Science Project Lifecycle
Data Science Project LifecycleData Science Project Lifecycle
Data Science Project Lifecycle
 
Synthetic Data Generation for Statistical Testing
Synthetic Data Generation for Statistical TestingSynthetic Data Generation for Statistical Testing
Synthetic Data Generation for Statistical Testing
 
Introduction to continual learning
Introduction to continual learningIntroduction to continual learning
Introduction to continual learning
 
Data lake ppt
Data lake pptData lake ppt
Data lake ppt
 
Data science presentation 2nd CI day
Data science presentation 2nd CI dayData science presentation 2nd CI day
Data science presentation 2nd CI day
 
What Is Data Science? | Introduction to Data Science | Data Science For Begin...
What Is Data Science? | Introduction to Data Science | Data Science For Begin...What Is Data Science? | Introduction to Data Science | Data Science For Begin...
What Is Data Science? | Introduction to Data Science | Data Science For Begin...
 

En vedette

Hadoop and Machine Learning
Hadoop and Machine LearningHadoop and Machine Learning
Hadoop and Machine Learningjoshwills
 
A Statistician's View on Big Data and Data Science (Version 1)
A Statistician's View on Big Data and Data Science (Version 1)A Statistician's View on Big Data and Data Science (Version 1)
A Statistician's View on Big Data and Data Science (Version 1)Prof. Dr. Diego Kuonen
 
Data By The People, For The People
Data By The People, For The PeopleData By The People, For The People
Data By The People, For The PeopleDaniel Tunkelang
 
Hands-on Deep Learning in Python
Hands-on Deep Learning in PythonHands-on Deep Learning in Python
Hands-on Deep Learning in PythonImry Kissos
 
10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning SystemsXavier Amatriain
 
A tutorial on deep learning at icml 2013
A tutorial on deep learning at icml 2013A tutorial on deep learning at icml 2013
A tutorial on deep learning at icml 2013Philip Zheng
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDevashish Shanker
 
Introduction to Mahout and Machine Learning
Introduction to Mahout and Machine LearningIntroduction to Mahout and Machine Learning
Introduction to Mahout and Machine LearningVarad Meru
 
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...Sebastian Raschka
 
Machine Learning and Data Mining: 12 Classification Rules
Machine Learning and Data Mining: 12 Classification RulesMachine Learning and Data Mining: 12 Classification Rules
Machine Learning and Data Mining: 12 Classification RulesPier Luca Lanzi
 
Myths and Mathemagical Superpowers of Data Scientists
Myths and Mathemagical Superpowers of Data ScientistsMyths and Mathemagical Superpowers of Data Scientists
Myths and Mathemagical Superpowers of Data ScientistsDavid Pittman
 
Tutorial on Deep learning and Applications
Tutorial on Deep learning and ApplicationsTutorial on Deep learning and Applications
Tutorial on Deep learning and ApplicationsNhatHai Phan
 
Tips for data science competitions
Tips for data science competitionsTips for data science competitions
Tips for data science competitionsOwen Zhang
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networksSi Haem
 
Introduction to Big Data/Machine Learning
Introduction to Big Data/Machine LearningIntroduction to Big Data/Machine Learning
Introduction to Big Data/Machine LearningLars Marius Garshol
 
Artificial neural network
Artificial neural networkArtificial neural network
Artificial neural networkDEEPASHRI HK
 
10 R Packages to Win Kaggle Competitions
10 R Packages to Win Kaggle Competitions10 R Packages to Win Kaggle Competitions
10 R Packages to Win Kaggle CompetitionsDataRobot
 
Artificial Intelligence Presentation
Artificial Intelligence PresentationArtificial Intelligence Presentation
Artificial Intelligence Presentationlpaviglianiti
 
The Business Analytics Value Proposition
The Business Analytics Value PropositionThe Business Analytics Value Proposition
The Business Analytics Value PropositionEric Stephens
 

En vedette (20)

Hadoop and Machine Learning
Hadoop and Machine LearningHadoop and Machine Learning
Hadoop and Machine Learning
 
A Statistician's View on Big Data and Data Science (Version 1)
A Statistician's View on Big Data and Data Science (Version 1)A Statistician's View on Big Data and Data Science (Version 1)
A Statistician's View on Big Data and Data Science (Version 1)
 
Data By The People, For The People
Data By The People, For The PeopleData By The People, For The People
Data By The People, For The People
 
Hands-on Deep Learning in Python
Hands-on Deep Learning in PythonHands-on Deep Learning in Python
Hands-on Deep Learning in Python
 
10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems10 Lessons Learned from Building Machine Learning Systems
10 Lessons Learned from Building Machine Learning Systems
 
A tutorial on deep learning at icml 2013
A tutorial on deep learning at icml 2013A tutorial on deep learning at icml 2013
A tutorial on deep learning at icml 2013
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language Processing
 
Introduction to Mahout and Machine Learning
Introduction to Mahout and Machine LearningIntroduction to Mahout and Machine Learning
Introduction to Mahout and Machine Learning
 
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
 
Machine Learning and Data Mining: 12 Classification Rules
Machine Learning and Data Mining: 12 Classification RulesMachine Learning and Data Mining: 12 Classification Rules
Machine Learning and Data Mining: 12 Classification Rules
 
Myths and Mathemagical Superpowers of Data Scientists
Myths and Mathemagical Superpowers of Data ScientistsMyths and Mathemagical Superpowers of Data Scientists
Myths and Mathemagical Superpowers of Data Scientists
 
Tutorial on Deep learning and Applications
Tutorial on Deep learning and ApplicationsTutorial on Deep learning and Applications
Tutorial on Deep learning and Applications
 
Tips for data science competitions
Tips for data science competitionsTips for data science competitions
Tips for data science competitions
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
Introduction to Big Data/Machine Learning
Introduction to Big Data/Machine LearningIntroduction to Big Data/Machine Learning
Introduction to Big Data/Machine Learning
 
Artificial neural network
Artificial neural networkArtificial neural network
Artificial neural network
 
10 R Packages to Win Kaggle Competitions
10 R Packages to Win Kaggle Competitions10 R Packages to Win Kaggle Competitions
10 R Packages to Win Kaggle Competitions
 
Artificial Intelligence Presentation
Artificial Intelligence PresentationArtificial Intelligence Presentation
Artificial Intelligence Presentation
 
The Business Analytics Value Proposition
The Business Analytics Value PropositionThe Business Analytics Value Proposition
The Business Analytics Value Proposition
 
Business intelligence
Business intelligenceBusiness intelligence
Business intelligence
 

Similaire à How to Interview a Data Scientist

Avoiding the Heuristic Solution: Moving past functional and correct to joyful...
Avoiding the Heuristic Solution: Moving past functional and correct to joyful...Avoiding the Heuristic Solution: Moving past functional and correct to joyful...
Avoiding the Heuristic Solution: Moving past functional and correct to joyful...Steven Hoober
 
10 Observations from 10+ years in the Corporate UX Trenches
10 Observations from 10+ years in the Corporate UX Trenches10 Observations from 10+ years in the Corporate UX Trenches
10 Observations from 10+ years in the Corporate UX TrenchesArio Jafarzadeh
 
Strata 2012: Humans, Machines, and the Dimensions of Microwork
Strata 2012: Humans, Machines, and the Dimensions of MicroworkStrata 2012: Humans, Machines, and the Dimensions of Microwork
Strata 2012: Humans, Machines, and the Dimensions of MicroworkDaniel Tunkelang
 
The Top Ten Execution Missteps
The Top Ten Execution MisstepsThe Top Ten Execution Missteps
The Top Ten Execution MisstepsBill Nussey
 
How to change your career to consulting
How to change your career to consultingHow to change your career to consulting
How to change your career to consultingPurnima Thakre
 
La potenza di Linkedin: i candidati passivi
La potenza di Linkedin: i candidati passiviLa potenza di Linkedin: i candidati passivi
La potenza di Linkedin: i candidati passiviAndrea Attana
 
Inside the world of passive talent research & tips
Inside the world of passive talent   research & tipsInside the world of passive talent   research & tips
Inside the world of passive talent research & tipsLynne Rooney
 
Inside the World of Passive Talent
Inside the World of Passive TalentInside the World of Passive Talent
Inside the World of Passive Talentharrydhebar
 
Inside the world of passive talent research & tips
Inside the world of passive talent research & tipsInside the world of passive talent research & tips
Inside the world of passive talent research & tipsDonna Graham
 
Inside The World Of Passive Talent
Inside The World Of Passive TalentInside The World Of Passive Talent
Inside The World Of Passive TalentDaniel Sanchez-Grant
 
Inside the world of passive talent - Research tips
Inside the world of passive talent - Research tipsInside the world of passive talent - Research tips
Inside the world of passive talent - Research tipsHarry Dhebar
 
Inside the world of passive talent
Inside the world of passive talentInside the world of passive talent
Inside the world of passive talentLeonardo Intriago
 
Inside The World Of Passive Talent Research & Tips
Inside The World Of Passive Talent   Research & TipsInside The World Of Passive Talent   Research & Tips
Inside The World Of Passive Talent Research & Tipshaimeecode
 
Inside the world of passive talent research & tips
Inside the world of passive talent  research & tipsInside the world of passive talent  research & tips
Inside the world of passive talent research & tipsCarly Rodger
 
Valtech - Innovation Needs Waste
Valtech - Innovation Needs WasteValtech - Innovation Needs Waste
Valtech - Innovation Needs WasteValtech
 
Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013
Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013
Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013Dan Arra
 
Design thinking in everyday life
Design thinking in everyday lifeDesign thinking in everyday life
Design thinking in everyday lifeMadhumita Gupta
 
Learnings from startups
Learnings from startupsLearnings from startups
Learnings from startupsTopi Järvinen
 
UCF Sales Club Presentation
UCF Sales Club PresentationUCF Sales Club Presentation
UCF Sales Club PresentationSteve Urquhart
 
It Takes A Village To Create A Great Candidate Experience
It Takes A Village To Create A Great Candidate ExperienceIt Takes A Village To Create A Great Candidate Experience
It Takes A Village To Create A Great Candidate ExperienceGreg Gerber (PHR in progress)
 

Similaire à How to Interview a Data Scientist (20)

Avoiding the Heuristic Solution: Moving past functional and correct to joyful...
Avoiding the Heuristic Solution: Moving past functional and correct to joyful...Avoiding the Heuristic Solution: Moving past functional and correct to joyful...
Avoiding the Heuristic Solution: Moving past functional and correct to joyful...
 
10 Observations from 10+ years in the Corporate UX Trenches
10 Observations from 10+ years in the Corporate UX Trenches10 Observations from 10+ years in the Corporate UX Trenches
10 Observations from 10+ years in the Corporate UX Trenches
 
Strata 2012: Humans, Machines, and the Dimensions of Microwork
Strata 2012: Humans, Machines, and the Dimensions of MicroworkStrata 2012: Humans, Machines, and the Dimensions of Microwork
Strata 2012: Humans, Machines, and the Dimensions of Microwork
 
The Top Ten Execution Missteps
The Top Ten Execution MisstepsThe Top Ten Execution Missteps
The Top Ten Execution Missteps
 
How to change your career to consulting
How to change your career to consultingHow to change your career to consulting
How to change your career to consulting
 
La potenza di Linkedin: i candidati passivi
La potenza di Linkedin: i candidati passiviLa potenza di Linkedin: i candidati passivi
La potenza di Linkedin: i candidati passivi
 
Inside the world of passive talent research & tips
Inside the world of passive talent   research & tipsInside the world of passive talent   research & tips
Inside the world of passive talent research & tips
 
Inside the World of Passive Talent
Inside the World of Passive TalentInside the World of Passive Talent
Inside the World of Passive Talent
 
Inside the world of passive talent research & tips
Inside the world of passive talent research & tipsInside the world of passive talent research & tips
Inside the world of passive talent research & tips
 
Inside The World Of Passive Talent
Inside The World Of Passive TalentInside The World Of Passive Talent
Inside The World Of Passive Talent
 
Inside the world of passive talent - Research tips
Inside the world of passive talent - Research tipsInside the world of passive talent - Research tips
Inside the world of passive talent - Research tips
 
Inside the world of passive talent
Inside the world of passive talentInside the world of passive talent
Inside the world of passive talent
 
Inside The World Of Passive Talent Research & Tips
Inside The World Of Passive Talent   Research & TipsInside The World Of Passive Talent   Research & Tips
Inside The World Of Passive Talent Research & Tips
 
Inside the world of passive talent research & tips
Inside the world of passive talent  research & tipsInside the world of passive talent  research & tips
Inside the world of passive talent research & tips
 
Valtech - Innovation Needs Waste
Valtech - Innovation Needs WasteValtech - Innovation Needs Waste
Valtech - Innovation Needs Waste
 
Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013
Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013
Macadamian - Product Camp - Top10 researchsecretswebinar dan_arra_march,2013
 
Design thinking in everyday life
Design thinking in everyday lifeDesign thinking in everyday life
Design thinking in everyday life
 
Learnings from startups
Learnings from startupsLearnings from startups
Learnings from startups
 
UCF Sales Club Presentation
UCF Sales Club PresentationUCF Sales Club Presentation
UCF Sales Club Presentation
 
It Takes A Village To Create A Great Candidate Experience
It Takes A Village To Create A Great Candidate ExperienceIt Takes A Village To Create A Great Candidate Experience
It Takes A Village To Create A Great Candidate Experience
 

Plus de Daniel Tunkelang

Query Understanding and Ecommerce
Query Understanding and EcommerceQuery Understanding and Ecommerce
Query Understanding and EcommerceDaniel Tunkelang
 
Semantic Equivalence of e-Commerce Queries
Semantic Equivalence of e-Commerce QueriesSemantic Equivalence of e-Commerce Queries
Semantic Equivalence of e-Commerce QueriesDaniel Tunkelang
 
Helping Searchers Satisfice through Query Understanding
Helping Searchers Satisfice through Query UnderstandingHelping Searchers Satisfice through Query Understanding
Helping Searchers Satisfice through Query UnderstandingDaniel Tunkelang
 
Query Understanding: A Manifesto
Query Understanding: A ManifestoQuery Understanding: A Manifesto
Query Understanding: A ManifestoDaniel Tunkelang
 
Where should you put your data scientists?
Where should you put your data scientists?Where should you put your data scientists?
Where should you put your data scientists?Daniel Tunkelang
 
Data Science: A Mindset for Productivity
Data Science: A Mindset for ProductivityData Science: A Mindset for Productivity
Data Science: A Mindset for ProductivityDaniel Tunkelang
 
My Three Ex’s: A Data Science Approach for Applied Machine Learning
My Three Ex’s: A Data Science Approach for Applied Machine LearningMy Three Ex’s: A Data Science Approach for Applied Machine Learning
My Three Ex’s: A Data Science Approach for Applied Machine LearningDaniel Tunkelang
 
Web science - How is it different?
Web science - How is it different?Web science - How is it different?
Web science - How is it different?Daniel Tunkelang
 
Better Search Through Query Understanding
Better Search Through Query UnderstandingBetter Search Through Query Understanding
Better Search Through Query UnderstandingDaniel Tunkelang
 
Social Search in a Professional Context
Social Search in a Professional ContextSocial Search in a Professional Context
Social Search in a Professional ContextDaniel Tunkelang
 
Find and be Found: Information Retrieval at LinkedIn
Find and be Found: Information Retrieval at LinkedInFind and be Found: Information Retrieval at LinkedIn
Find and be Found: Information Retrieval at LinkedInDaniel Tunkelang
 
Search as Communication: Lessons from a Personal Journey
Search as Communication: Lessons from a Personal JourneySearch as Communication: Lessons from a Personal Journey
Search as Communication: Lessons from a Personal JourneyDaniel Tunkelang
 
Enterprise Search: How do we get there from here?
Enterprise Search: How do we get there from here?Enterprise Search: How do we get there from here?
Enterprise Search: How do we get there from here?Daniel Tunkelang
 
Big Data, We Have a Communication Problem
Big Data, We Have a Communication Problem Big Data, We Have a Communication Problem
Big Data, We Have a Communication Problem Daniel Tunkelang
 
Information, Attention, and Trust: A Hierarchy of Needs
Information, Attention, and Trust: A Hierarchy of NeedsInformation, Attention, and Trust: A Hierarchy of Needs
Information, Attention, and Trust: A Hierarchy of NeedsDaniel Tunkelang
 
Content, Connections, and Context
Content, Connections, and ContextContent, Connections, and Context
Content, Connections, and ContextDaniel Tunkelang
 
Scale, Structure, and Semantics
Scale, Structure, and SemanticsScale, Structure, and Semantics
Scale, Structure, and SemanticsDaniel Tunkelang
 
Recommendations as a Conversation with the User
Recommendations as a Conversation with the UserRecommendations as a Conversation with the User
Recommendations as a Conversation with the UserDaniel Tunkelang
 

Plus de Daniel Tunkelang (20)

Query Understanding and Ecommerce
Query Understanding and EcommerceQuery Understanding and Ecommerce
Query Understanding and Ecommerce
 
Semantic Equivalence of e-Commerce Queries
Semantic Equivalence of e-Commerce QueriesSemantic Equivalence of e-Commerce Queries
Semantic Equivalence of e-Commerce Queries
 
Helping Searchers Satisfice through Query Understanding
Helping Searchers Satisfice through Query UnderstandingHelping Searchers Satisfice through Query Understanding
Helping Searchers Satisfice through Query Understanding
 
MMM, Search!
MMM, Search!MMM, Search!
MMM, Search!
 
Enterprise Intelligence
Enterprise IntelligenceEnterprise Intelligence
Enterprise Intelligence
 
Query Understanding: A Manifesto
Query Understanding: A ManifestoQuery Understanding: A Manifesto
Query Understanding: A Manifesto
 
Where should you put your data scientists?
Where should you put your data scientists?Where should you put your data scientists?
Where should you put your data scientists?
 
Data Science: A Mindset for Productivity
Data Science: A Mindset for ProductivityData Science: A Mindset for Productivity
Data Science: A Mindset for Productivity
 
My Three Ex’s: A Data Science Approach for Applied Machine Learning
My Three Ex’s: A Data Science Approach for Applied Machine LearningMy Three Ex’s: A Data Science Approach for Applied Machine Learning
My Three Ex’s: A Data Science Approach for Applied Machine Learning
 
Web science - How is it different?
Web science - How is it different?Web science - How is it different?
Web science - How is it different?
 
Better Search Through Query Understanding
Better Search Through Query UnderstandingBetter Search Through Query Understanding
Better Search Through Query Understanding
 
Social Search in a Professional Context
Social Search in a Professional ContextSocial Search in a Professional Context
Social Search in a Professional Context
 
Find and be Found: Information Retrieval at LinkedIn
Find and be Found: Information Retrieval at LinkedInFind and be Found: Information Retrieval at LinkedIn
Find and be Found: Information Retrieval at LinkedIn
 
Search as Communication: Lessons from a Personal Journey
Search as Communication: Lessons from a Personal JourneySearch as Communication: Lessons from a Personal Journey
Search as Communication: Lessons from a Personal Journey
 
Enterprise Search: How do we get there from here?
Enterprise Search: How do we get there from here?Enterprise Search: How do we get there from here?
Enterprise Search: How do we get there from here?
 
Big Data, We Have a Communication Problem
Big Data, We Have a Communication Problem Big Data, We Have a Communication Problem
Big Data, We Have a Communication Problem
 
Information, Attention, and Trust: A Hierarchy of Needs
Information, Attention, and Trust: A Hierarchy of NeedsInformation, Attention, and Trust: A Hierarchy of Needs
Information, Attention, and Trust: A Hierarchy of Needs
 
Content, Connections, and Context
Content, Connections, and ContextContent, Connections, and Context
Content, Connections, and Context
 
Scale, Structure, and Semantics
Scale, Structure, and SemanticsScale, Structure, and Semantics
Scale, Structure, and Semantics
 
Recommendations as a Conversation with the User
Recommendations as a Conversation with the UserRecommendations as a Conversation with the User
Recommendations as a Conversation with the User
 

Dernier

Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 

Dernier (20)

Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 

How to Interview a Data Scientist

  • 1. Daniel How to Interview a Data Scientist Daniel Tunkelang Director of Data Science, LinkedIn Recruiting Solutions 1
  • 3. GOAL 3
  • 4. Specification for a Data Scientist implements algorithms analyzes data thinks product 4
  • 5. What about C ulture ommunication uriosity Hold that thought… ? 5
  • 6. What can you learn from an interview? 6
  • 7. Interviewing is a last resort. Alternatives? 7
  • 8. Only hire people you’ve worked with. 8
  • 9. Hire interns. Convert to full-time. Profit! 9
  • 10. Try before you buy: short-term contracts. 10
  • 11. Alternatives are at best a partial solution. §  Only hiring people you’ve worked with doesn’t scale. –  And traps you in a locally optimal monoculture. §  Interns are great! But they are a significant investment. –  Managing interns well is a productivity gamble. –  Most interns have at least a year of school left. –  Not all interns will make your bar. You won’t always make theirs. §  Try before you buy: nice in theory. –  Adverse selection bias when other offers are permanent roles. –  Creates bureaucracy. 11
  • 12. Can we at least make interviews natural? 12
  • 13. Spend a day working together. 13
  • 16. High-fructose corn syrup is 100% natural. §  Working sessions are difficult to set up. –  No more natural than a final exam. –  High variance, and very difficult to calibrate performance. §  Take-home assignments are great for the employer. –  But they are a significant investment for the candidate. –  Adverse selection bias if other companies don’t require them. –  Creates incentive to cheat if significant part of hiring process. §  Previous work is like natural experiments. –  Always good to review a candidate’s previous work. –  But not always possible to find work with high predictive value. 16
  • 17. So you gotta do interviews. But how? 17
  • 18. Three Principles 1.  Keep it real. 2.  No gotchas. 3.  Maybe = no. 18
  • 20. Test basic coding with FizzBuzz questions. multiple of 3 -> Fizz multiple of 5 -> Buzz multiple of 15 -> FizzBuzz 1, 2, Fizz, 4, Buzz, Fizz, 7, 8, Fizz, Buzz, 11, Fizz, 13, 14, FizzBuzz, 16, … 20
  • 21. Whiteboards suck for coding. http://ericleads.com/2012/10/how-to-conduct-a-better-coding-interview/ 21
  • 22. Don’t ask pointless algorithm questions. implement 22
  • 23. Use real-world algorithms questions. bigdatascientist Did you mean: big data scientist 23
  • 24. Ask candidates to design your products. 24
  • 25. Keeping it real is also a great sell. Similar Profiles People You May Know 25
  • 27. Gotchas reduce the signal-to-noise ratio. §  Avoid problems where success hinges on a single insight. –  Good interview problems offer lots of room for partial credit. –  Making a key insight often reflects experience, not intelligence. §  Don’t test a candidate’s knowledge of a niche technique. –  Unless that niche technique is critical to job performance. –  And can’t be learned on the job as part of on-boarding. §  Be a hard interviewer, but don’t be an asshole. –  An interview is not a stress-test to see where candidates break. –  Interviews communicate your values to the candidate. 27
  • 29. Commit to binary interview outcomes. §  Forced choice so interviewers don’t take easy way out. –  Just like having 4 choices instead of 5 on a rating scale. –  Encourages interviewers to take their role seriously. §  Each team member is a critical filter. –  Two no’s or one strong no is a no. –  All weak yes’s is a no. §  Short-circuit candidates early in the process. –  Resume and phone screening should be aggressive. –  Onsite interviews should have ~50% chance of leading to offers. 29
  • 30. But what about C ulture ommunication uriosity All are must-haves. ? Every interview evaluates all three. 30
  • 32. Three Principles 1.  Keep it real. –  Avoid whiteboard coding. Filter with FizzBuzz. –  Use real-world algorithms questions. –  Ask candidates to design your products. 2.  No gotchas. –  Gotchas reduce the signal-to-noise ratio. 3.  Maybe = no. –  Bad hires suck. Be conservative. –  Trust your team. 32