The 9th May Incident in Pakistan A Turning Point in History.pptx
915 keynote stern_using our laptop
1. Jim Sterne
eMetrics Summit
Digital Analytics Association
Artificial Intelligence for Marketing
Getting Started
Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
3. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
As Seen on TV
“Strong AI” – thinks and acts human
Sentience
“Weak AI” – task specific
Functional
AI: Anything computers can’t
SciFi: Anything AI can’t
4. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Why Machine Learning Now?
50 years of study
Huge amount of data
Specialize chipsets
5. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Software Grows Up
Specific Logic Mathematical Model
Do this, then this, then this Describe numerical relationships
If this happens, do that Calculate alternatives
If confused, report error Human compares results & iterates
Statistical Model Artificial Intelligence
Calculate probabilities Uses examples to figure it out
Project likelihoods and changes it's mind
Human compares & iterates
7. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Artificial Intelligence
Speech to text
This means that
Repeated correction
Taught over time
Contractions
Accents
Patois
Wreck a nice beach
Recognize speech
Natural Language Processing
8. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Natural Language Processing
Can I help you?
Yes, I have a problem
Oh?
– one of the keys is broken
Current Customer
with my keyboard
Hardware
1. Take it to a local store
2. Send it in for repair
3. Send it in for replacement
FAQ
How long?
Loaner?
Data back up?
Warranty?
Incoming: 805-403-4075
Customer service
9. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Natural Language Processing
10. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Artificial Intelligence
Natural Language
Speech to text
This means that
Repeated correction
Taught over time
Conversation Bots
Text to meaning
Concept & emotion imitation
Repeated correction
Taught over time
12. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Artificial Intelligence
Natural Language
Speech to text
This means that
Repeated correction
Taught over time
Conversation Bots
Text to meaning
Concept & emotion imitation
Repeated correction
Taught over time
13. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Artificial Intelligence
Natural language
Speech to text
This means that
Repeated correction
Taught over time
Conversation Bots
Text to meaning
Concept & emotion imitation
Repeated correction
Taught over time
Vision
Pattern discovery
This means that
Repeated correction
Taught over time
19. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Rules based
This means that
Repeated correction
Is taught over time
Pattern discovery
This means that
Repeated correction
Learns over time
Complex concept imitation
Emotional intelligence imitation
Repeated correction
Is taught and learns over time
Robots
21. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Artificial Intelligence
Natural Language Processing - Call center
Conversation Bots - Customer service
Computer Vision - Social media
Robots - In store
Machine Learning - Everything else
22. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Revise
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Revise
Alter opinions
about attributes
and their
weightings
03
3 Needs for 3 Deeds
of Machine Learning
23. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03Wind speed
3 Needs for 3 Deeds
of Machine Learning
Barometric pressure
Temperature
Hours of daylight
Sunrise, sunset
Rain?
UV Index
24. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
Wind speed
3 Needs for 3 Deeds
of Machine Learning
Barometric pressure
Hours of daylight
Sunrise, sunset
Rain:
UV Index
Temperature
25. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03Day Part
Gender
Age
Income
Location
Education
Behavior
Weather
3 Needs for 3 Deeds
of Machine Learning
26. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03Day Part
Gender
Age
Income
Location
Education
Behavior
Weather
3 Needs for 3 Deeds
of Machine Learning
27. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Develop
Alter opinions
about attributes
and their
weightings
03
GOAL: Conversion
DATA:
previous purchase
search term
pageviews
time-on-item
email opt in
post code
Y
>1.5
*
>5
Y
Then
send 15% off email
3 Needs for 3 Deeds
of Machine Learning
28. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Revise
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Revise
Alter opinions
about attributes
and their
weightings
03
GOAL: Conversion
DATA:
previous purchase
search term
pageviews
time-on-item
email opt in
post code
Y
>1.5
*
>5
Y
Then
send 15% off email
>2
>4
3 Needs for 3 Deeds
of Machine Learning
Control:
29. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Revise
Alter opinions
about attributes
and their
weightings
03
Detect
Discover the most
predictive
attributes for a
given outcome
01
Decide
Infer rules from
the data, weigh
the attributes, and
suggest a course of
action
02
Revise
Alter opinions
about attributes
and their
weightings
03
Data Goal Control
3 Needs for 3 Deeds
of Machine Learning
30. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
More data than a human can wrangle
More attributes than a human can manage
More permutations than a human can comprehend
31. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Classification
Clustering
Segmentation
Gender
M F
Age A Age B Age C
Man and Machinevs.
32. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Classification
Clustering
Segmentation
Motorcycle Insurance Targeting
Man and Machinevs.
33. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Testing
A. Buy one get one free
B. Two for the price of one
A2.1.1
A2.1.2 15%
Lift
A2.1
A2.2
A
B
A1
A2
B1
B2
Man and Machinevs.
34. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Is there a pattern?
Is there an anomaly?
What can be omitted?
What if we did it backwards?
What if we changed the time scale?
What if we look at it sideways?
What additional data would be revealing?
What if it had wheels?
What would Chuck Norris do?
What if this is the wrong problem?
Man and Machine
35. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Testing
A. Buy one get one free
B. Two for the price of one
A2.1.1
A2.1.2 15%
Lift
A2.1
A2.2
A
B
A1
A2
B1
B2
Man and Machine
36. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Learning Machine Learning
What's it good at?
How is it classified?
How does it work?
37. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Learning Machine Learning
High Dimensionality
High Cardinality
What's it good at?
38. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Dimensionality = Elements per Object
dog
bird
grumpy keyboardLOL can haz cheeseburger?
fish
cool
hip
crazy
aloofsnootytrying to kill you
tail
fur
claws
Low Customers: 10 columns
name, address, phone, DoB, interests, orders, CLTV, etc.
Medium Web analytics: 100 columns
High Language: > 1,000 dimensions
cat
39. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Cardinality = Options per Element
High: Phone # 7.442 billion
Medium: ZIP Code 43,000
Low: Alive or Dead 2.5
Age 122
40. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Learning Machine Learning
High Dimensionality
High Cardinality
What's it good at?
41. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Learning Machine Learning
How is it classified?
Supervised
Unsupervised
Reinforcement
42. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Supervised: You know the right answer
Correcting Autocorrect
Dog: Yes Cat: No
Tag a friend?
43. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Unsupervised
Segment my customers
Find look-alike prospects
Create customer personas
Good for unlabeled data
Tell me something I don't know
45. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Reinforcement Learning
Given:
data
goal
action
feedback
Respond to the environment
46. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Learning Machine Learning
How does it work?
Decision Trees / Random Forest
Support Vector Machines
Neural Nets / Deep Learning
Ensemble
47. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Decision Trees Random Forest
Message A
Message B
48. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Random data samples
Random variables
Decision Trees Random Forest
49. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Random data samples
Random variables
Decision Trees Random Forest
50. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Random data samples
Random variables
Solution
Decision Trees Random Forest
51. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Support Vector Machines
52. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Support Vector Machines
53. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Support Vector Machines
54. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Support Vector Machines
55. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Neural Network Deep Learning
Go to the movies?
56. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Neural Network Deep Learning
Go to the movies?
57. Wisdom of Machines
57
Validation
Each model’s predictive
accuracy is tested on the
hold out data set.
Wisdom of the crowd
The combination of models that
delivers the best accuracy is
selected and deployed.
58. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Learning Machine Learning Language
High Dimensionality Lots of Elements per Object
High Cardinality Lots of Options per Element
Supervised / Unsupervised Examples vs. Exploration
Decision Trees / Random Forest Random data & variables
Support Vector Machines Looking at it from a different angle
Neural Nets / Deep Learning Sort of how we think the mind works
Ensemble Diversity Rules
59. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Bringing AI Into Your Organization
Look what followed me home!
Can we keep him?
60. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
61. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Valid
Credible
Reliable
Consistent
Clean
Unbiased
Defined
Relevant
Correlate-able
Understandable
Complete
Timely
62. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Would I advise my uncle?
Would I stake my reputation?
Would I risk my own money?
Would I bet my job?
63. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Start with repetitive, taxing tasks machines can do better
Ranking
Sorting big data
Finding patterns
Finding look-alikes
Counting, measuring
Finding a needle in a haystack
One of these things is not like the other
64. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
What Can ML Do Better?
Testing
Lead scoring
Meeting scheduling
Personalizing content
Inbound e-mail sorting
Social media monitoring
Programmatic advertising
Creating social media messages & ad copy
65. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Start with repetitive, taxing tasks machines can do better
Buy vs. Build?
67. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Start with repetitive, taxing tasks machines can do better
Buy vs. Build?
Buy!
68. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Start with repetitive, taxing tasks machines can do better
Buy vs. Build
Determine which data sets are useful
69. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Start with repetitive, taxing tasks machines can do better
Buy vs. Build
Determine which data sets are useful
Become proficient at the Smell Test
70. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
The Smell Test
71. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Know Your Data
Start with repetitive, taxing tasks machines can do better
Buy vs Build
Determine which data sets are useful
Become proficient at the Smell Test
Be the change you want to see
72. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Find other enthusiasts (meet-ups)
Find internal enthusiasts (host a meet-up)
Lunch and Learn (buy them lunch)
Combine resources to make every decision lead to creating an
AI Center of Excellence
Be The Change You Want to See
73. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
-1 Ignorance or ennui
0 Aware and Learning
1 Ad-Hoc Experimentation
2 Organized Experimentation
3 Goal Setting
4 System Training
5 System Testing
6 System Deployed
7 Continuous Learning
Everything
Maturity
Model
74. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
AI Onboarding Tips
Clearly identified goals
Start with repetitive, taxing tasks
Buy vs. build
Know your data
Determine which data sets are useful
Be the change you want to see
Become proficient at the Smell Test
Hone your domain knowledge
75. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Man and Machine
76. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
Jim Sterne
eMetrics Summit
Digital Analytics Association
Artificial Intelligence for Marketing
Getting Started