This document provides an introduction and overview of machine learning. It discusses use cases for machine learning like real estate pricing and spam filtering. It covers the two phases of machine learning as training a model and then predicting with the model. It also discusses limitations of machine learning like needing enough high quality training data. The document recommends using an ML canvas to plan machine learning projects by defining the problem, data, metrics, and model development process. It provides an example case study of using machine learning for churn prediction and analysis.
35. –Katherine Barr, Partner at VC-firm MDV
"Pairing human workers with
machine learning and automation
will transform knowledge work
and unleash new levels of human
productivity and creativity."
51. –McKinsey & Co. (2011)
“A significant constraint on
realizing value from big data will
be a shortage of talent,
particularly of people with deep
expertise in statistics and machine
learning.”
59. The two phases of machine learning:
• TRAIN a model
• PREDICT with a model
60. The two methods of predictive APIs:
• TRAIN a model
• PREDICT with a model
61. The two methods of predictive APIs:
• model = create_model(dataset)
• predicted_output =
create_prediction(model, new_input)
62. The two methods of predictive APIs:
• model = create_model(‘training.csv’)
• predicted_output =
create_prediction(model, new_input)
66. From Large to Small & Medium Enterprises
• recommendations in e-commerce
• => 71% increase in revenue
• churn detection
• => 11% increase in retention
72. • Who: SaaS company selling monthly subscription
• Question asked:“Is this customer going to leave
within 1 month?”
• Input: customer
• Output: no-churn or churn
• Data collection: history up until 1 month ago
• Baseline: if no usage for more than 15 days then
churn
74. Customer representation:
• basic info (age, income, etc.)
• usage of service (# times used app, avg time spent,
features used, etc.)
• interactions with customer support (how many,
topics of questions, satisfaction ratings)
75. Taking action to prevent churn:
• contact customers (in which order?)
• switch to different plan
• give special offer
• no action?
76. Measuring accuracy:
• #TP (we predict customer churns and he does)
• #FP (we predict customer churns but he doesn’t)
• #FN (we predict customer doesn’t churn but he does)
• Compare to heuristic/baseline
77. Return On Investment:
• Taking action for each TP (and FP) has a cost
• For each TP we“gain”:
(success rate of action) * (revenue /cust. /month)
• Imagine…
• perfect predictions
• revenue /cust. /month = 10€
• success rate of action = 20%
• cost of action = 2€
• Which ROI?
80. PREDICTIONS OBJECTIVES DATA
Context
Who will use the predictive system / who will be
affected by it? Provide some background.
Value Proposition
What are we trying to do? E.g. spend less time on
X, increase Y...
Data Sources
Where do/can we get data from? (internal
database, 3rd party API, etc.)
Problem
Question to predict answers to (in plain English)
Input (i.e. question "parameter")
Possible outputs (i.e. "answers")
Type of problem (e.g. classification, regression,
recommendation...)
Baseline
What is an alternative way of making predictions
(e.g. manual rules based on feature values)?
Performance evaluation
Domain-specific / bottom-line metrics for
monitoring performance in production
Prediction accuracy metrics (e.g. MSE if
regression; % accuracy, #FP for classification)
Offline performance evaluation method (e.g.
cross-validation or simple training/test split)
Dataset
How do we collect data (inputs and outputs)?
How many data points?
Features
Used to represent inputs and extracted from
data sources above. Group by types and
mention key features if too many to list all.
Using predictions
When do we make predictions and how many?
What is the time constraint for making those predictions?
How do we use predictions and confidence values?
Learning predictive models
When do we create/update models? With which data / how much?
What is the time constraint for creating a model?
Criteria for deploying model (e.g. minimum performance value — absolute,
relative to baseline or to previous model)
IDEASPECSDEPLOYMENT
83. PREDICTIONS OBJECTIVES DATA
BACKGROUND End-user Value prop Sources
ENGINE SPECS ML problem Perf eval Preparation
INTEGRATION Using pred Learning modelINTEGRATION Using pred Learning model
84. Why fill in ML canvas?
• Target the right problem for your company
• Choose right algorithm, infrastructure, or ML
solution
• Guide project management
• Improve team communication
87. • ML to create value from data
• 2 phases: TRAIN and PREDICT
• MLaaS & Predictive APIs make it more accessible
• Good data is essential
• What do we do with predictions?
• Accuracy is not the objective! A/B test?
• Start with the ML Canvas
• Later: deploy, maintain, improve…