Strata New York 2014

1. Computing Professional Identity for the Economic Graph

2. Agenda
1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

3. 1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

4. About me
©2013 LinkedIn Corporation. All Rights Reserved.
4
Vitaly Gordon

5. About me
©2013 LinkedIn Corporation. All Rights Reserved.
5

6. About me
©2013 LinkedIn Corporation. All Rights Reserved.
6

7. About me
©2013 LinkedIn Corporation. All Rights Reserved.
7

8. About me
©2013 LinkedIn Corporation. All Rights Reserved.
8

9. About me
©2013 LinkedIn Corporation. All Rights Reserved.
9

10. About me
@bigdatasc /in/vitalygordon
©2013 LinkedIn Corporation. All Rights Reserved.
10

11. What’s in it for you?
©2013 LinkedIn Corporation. All Rights Reserved.
11

12. What’s in it for you?
©2013 LinkedIn Corporation. All Rights Reserved.
12
1. You will learn about how LinkedIn
takes a massive vision and breaks it
down to small data problems

13. What’s in it for you?
©2013 LinkedIn Corporation. All Rights Reserved.
13
1. You will learn about how LinkedIn
takes a massive vision and breaks it
down to small data problems
2. You will learn about how hard
cleaning data can be

14. What’s in it for you?
©2013 LinkedIn Corporation. All Rights Reserved.
14
1. You will learn about how LinkedIn
takes a massive vision and breaks it
down to small data problems
2. You will learn about how hard
cleaning data can be
3. You will learn why LinkedIn needs
endorsements

15. What’s in it for me?
©2013 LinkedIn Corporation. All Rights Reserved.
15

16. What’s in it for me?
©2013 LinkedIn Corporation. All Rights Reserved.
16

17. What’s in it for me?
©2013 LinkedIn Corporation. All Rights Reserved.
17
@bigdatasc

18. 1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

19. ©2013 LinkedIn Corporation. All Rights Reserved.
19
Create economic opportunity for
every professional in the world

20. ©2013 LinkedIn Corporation. All Rights Reserved.
20

21. ©2013 LinkedIn Corporation. All Rights Reserved.
21
• CEO
• Chief Executive Officer
• CEO and Founder
• CEO & Co-founder
• President and CEO
• Owner

22. ©2013 LinkedIn Corporation. All Rights Reserved.
22
• IBM
• International Business Machines
• International Bus. Machines
• IBM Research
• IBM T.J. Watson Research Center
• IBM Canada
• IBM India

23. ©2013 LinkedIn Corporation. All Rights Reserved.
23
• UCLA
• University of California, Los Angeles
• UC Los Angeles
• The Anderson School of Management

24. ©2013 LinkedIn Corporation. All Rights Reserved.
24

25. 1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

26. Why Do We Need Identity Standardization?
©2013 LinkedIn Corporation. All Rights Reserved.
26

27. Why Do We Need Identity Standardization?
©2013 LinkedIn Corporation. All Rights Reserved.
27

28. Why Do We Need Identity Standardization?
©2013 LinkedIn Corporation. All Rights Reserved.
28

29. ©2013 LinkedIn Corporation. All Rights Reserved.
29

30. Text Based Solution
 Applies acronym expansion (e.g. vp -> vice president)
 Applies abbreviation expansion (e.g. sr. -> senior)
 Select the most common standard titles
 Selects standard sub strings (e.g. software engineer and tech lead
in search -> [software engineer, tech lead])
©2013 LinkedIn Corporation. All Rights Reserved.
30

31. Problems with a Text Based Approach
©2013 LinkedIn Corporation. All Rights Reserved.
31
Senior Software
Engineer
Software Engineer

32. Problems with a Text Based Approach
©2013 LinkedIn Corporation. All Rights Reserved.
32
Software Engineer Software Developer Programmer

33. Problems with a Text Based Approach
©2013 LinkedIn Corporation. All Rights Reserved.
33
Architect

34. ©2013 LinkedIn Corporation. All Rights Reserved.
34

35. ©2013 LinkedIn Corporation. All Rights Reserved.
35

36. ©2013 LinkedIn Corporation. All Rights Reserved.
36

37. ©2013 LinkedIn Corporation. All Rights Reserved.
37

38. 38
Profile Inferred Skills Endorsements Skill Vectors

39. 39
Profile Inferred Skills Endorsements Skill Vectors
30%
25%
20%
15%
10%
5%
0%
Data Mining Hadoop Machine
Learning
Java Algorithms Python MapReduce Data Science

40. 40
Profile Inferred Skills Endorsements Skill Vectors
30%
25%
20%
15%
10%
5%
0%
Data Mining Hadoop Machine
Learning
Java Algorithms Python MapReduce Data Science
http://www.slideshare.net/s_shah/strata-endorsements

41. Ontology Creation
41

42. Ontology Creation
42

43. Classification
43

44. 1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

45. Normalization
©2013 LinkedIn Corporation. All Rights Reserved.
45

46. Normalization
46

47. Clustering
©2013 LinkedIn Corporation. All Rights Reserved.
47

48. Clustering
48
• Each topic is a distribution over words
• Each document is a mixture of corpus-wide topics
• Each word is drawn from one of those topics

49. Anomaly Detection
49

50. Anomaly Detection
50

51. Anomaly Detection
51
http://www.slideshare.net/tdunning/strata-2014-anomaly-detection

52. 1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

53. Summary
©2013 LinkedIn Corporation. All Rights Reserved.
60
1. User generated content from 300M
members, creates 300M problems

54. Summary
©2013 LinkedIn Corporation. All Rights Reserved.
61
1. User generated content from 300M
members, creates 300M problems
2. Data cleaning is so much more than
filtering out empty values

55. Summary
©2013 LinkedIn Corporation. All Rights Reserved.
62
1. User generated content from 300M
members, creates 300M problems
2. Data cleaning is so much more than
filtering out empty values
3. Try to be creative and work around
difficult language problems

56. 1 Introduction
2 LinkedIn’s Vision
3 Computing Professional Identity
4 Selected Topics
5 Summary
6 Final Words

57. We’re building the next big thing

58. We’re building the next big thing
Join Us

59. We’re building the next big thing
Join Us!

60. We’re building the next big thing
Join Us!
DJ Patil Gary Flake Beau Cronin

61. @bigdatasc /in/vitalygordon