This is a talk I gave about all the unsexy things you have to deal with when putting AI in production.

1. AI in Production
Crunch, Budapest
October 20, 2017

2. Hello

3. Hello

4. Hello

5. Hello

6. I am a fool

7. We’re here to talk
about reality.

8. Reality Bytes
Everything Else
Models
What you see What you get

9. 1. Infrastructure
Reality Bytes

10. 1. Infrastructure
2. Product
Reality Bytes

11. WORK

12. Infrastructure

13. Infrastructure
“Fanny pack”
0.8
Document
Model

14. Training Data
Infrastructure

15. Training Data
“Fanny pack”
Document
Yes / No

16. Training Data

17. Is this item a fanny pack?
Is this item relevant to the
query “fanny pack”?
On a scale of 1-10 how
“fanny pack” is this thing?
Which of these is the most
“fanny pack”?
Crowd Sourcing

18. Fanny Que?

20. Crowd Sourcing
So...umm...Is this data
any good?

21. Traffic to the
Rescue!
Training Data

22. Search results for “fanny pack”

23. SKIN WINS

24. Traffic FTW

25. Transfer
Learning!
Training Data

26. Wolf or Husky?

27. Transfer Learning
Does it work for
our data?

28. Transfer Learning
When it works
1. Embeddings
2. Super General Tasks

29. Training Data REGEX

30. Serving
Inferences
Infrastructure

31. Serving
Search Engine Ranking Model

32. Model
Search
Engine
Serving
1. RPC

33. Serving
1. RPC
a. Service Wrapper?
b. Monitoring/Deployment?
c. New Models?
d. Expertise?

34. Serving
1. RPC
40 Results
800ms/inference
32 seconds

35. Serving
1. RPC
2. Rewrite Search Engine
Model

36. Modern ML applications are comprised of an
increasingly diverse mix of libraries and systems...Even if
each of these libraries is optimized in isolation, real
pipelines combine multiple libraries, so production use at
scale usually requires a software engineering team to
rewrite the whole application in low-level code”

37. words = []
for word in title.split():
words.append(stripNonAlphaNumeric(
word.strip()
))
val words = title
.trim
.split(" ")
.map(w => w.trim)
.map(w => stripNonAlphaNumeric(w))
.map(w => stem(w))
Training
Inference

38. Serving
1. RPC
2. Rewrite
3. Cache
Search
Engine
Cache Model

39. Serving
A new hope?

40. Product

41. Top 3 Reasons AI
Products Fail
Product

42. Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.

44. Nightly Retraining
Model

45. Day 1
Top Result for “shorts”
Control Experiment

46. Clicks for “shorts”
Control
Control
Model
Experiment
Model
Experiment

47. Top Result for “shorts”
Day 1
Day 20
ExperimentControl

48. Clicks for “shorts”
Control
Control
Model
Experiment
Model
Experiment

49. 1. Think about your objective.
2. Isolate.
3. Change one thing: the model.
4. Check your instrumentation.
Measurement is hard

50. Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
2. Wrong Venue/Treatment/Approach.

51. The Taste Test

52. Choose an item you like

53. Items matching your taste

54. Venue
Home Page
Objective
↑ Registrations

58. Product
Top 3 Reasons AI Products Fail
1. Measuring stuff is hard.
2. Wrong Venue/Treatment/Approach.
3. Models First. Users Last.

59. Models First
Cool Model: word2vec!
bluenavy

60. Models First
Cool Model → Query Expansion!
(navy or blue) totenavy tote

61. Search Results for “indian sea”

62. Precision Problems
Garbage results
Recall Problems
Not enough results

63. f you wanted to make this one statement
as well.
Or another one.
Search results for “wedding dress”

64. “1930s”“1920s”

65. UsersModel

66. Search results for “jewellery”

67. UsersModel

68. Everything Else
Models
What you see What you get

69. AI Product✨

70. Denoument

71. Denoument
Things you should care about
1. Humans
2. Measurement
3. Training Data
4. Serving Inferences
5. Models

72. gio@relatedworks.io