This document discusses building smart AI and the potential problems with deep learning. It notes that while machine learning and deep learning have advanced significantly, it is important not to lose sight of causality and transparency. Deep learning models can ignore causal relationships and reinforce biases if not developed properly. The document provides examples of using predictive analytics and machine learning responsibly in areas like recruiting, customer service chatbots, and summarizing key insights from chat data to improve agent performance. It emphasizes the need to formalize why certain approaches are taken and ensure models are designed to avoid potential harms.

1. Building Smart AI:
How Deep Learning Can Get You Into Deep
Trouble
December 14
2017
Michael Housman, A.M., Ph.D.
Co-Founder, Chief Data Science
Officer
RapportBoost.AI

2. Why should
we care about
artificial intelligence?

3. A brief history of data science
1990 2020STATISTICS &
ECONOMETRICS
MACHINE LEARNING DEEP LEARNING
CAUSAL INFERENCE
WHY?
TRANSPARENT MODELS
HOW?BLACK
BOXES
2000 2010
PREDICTION
W
YX

4. 60-second primer on deep learning

5. Turning pixels into lables

6. Caveat: An economist’s rant
BACK IN MY DAY, WE
DID ECONOMETRICS!
NONE OF THE NEURAL
NETS THE KIDS THESE
DAYS ARE USING.
• First, do no harm.
• AI must be
designed so that it
does not:
Ignore
causal
relationship
s
(the why)
Reinforce
existing
biases
(the how)

7. What was the problem with recruiting?

8. What was the solution?
Predictive
Analytics
Big Data
Machine
Learning
Workforce
Science Data-driven
Candidate
Scoring and
Prioritization
Psychometric
Assessments
Core HR System

9. Job applicants who use Chrome and Firefox exhibit 15% longer tenure, 19% less absenteeism, and
significantly higher productivity and customer satisfaction.
INSIGHT:

10. What if we analyzed video to screen applicants?

11. What is the problem with bots?
Don’t reflect the
brand identity
Unable to sell
effectively
Aren’t
personalized to
adapt to the user

12. What is the solution?
S T E P 1
Feed the Engine Chat
Data & KPI Outcomes
S T E P 2
Process in the
Machine Learning Engine
S T E P 3
Spits Out Actionable
Insights and Best Practices
CHAT DATA
KPI OUTCOMES
DATA WAREHOUSE
MACHINE LEARNING
NATURAL LANGUAGE
PROCESSING

13. ACTION
1
Formal
Language
5%
Action: A 5% decrease in formal language
increases customer upgrades by 12.11%
Definition: Ratio of formal language to non-
formal language within a conversation
Examples: Hello, can not, it is, of course, thank
you (vs. hi, can't, it’s, sure, thanks)
Rationale: Overly-formal language feels robotic
and cold; customers prefer to feel like they’re
speaking with a human.
Decrease Formal Language
Upgrades
Increase
12%
Revenue
Increase
$198k
Action Customer Upgrades Revenue
1% 2.17% $38,400
5% 12.11% $197,760
10% 24.22% $382,080
Confidence
90.79%
Relationship:

14. What is the problem with chat?

15. D I G I TA
L
I N P E RS O N + P H O NE
Copy Editors / Web Designers
Google / Mobile Analytics
Website Optimization
A/B Testing
Online or on-the-job
training
One-on-one teaching
Random QA Checks

16. +
D I G I TA
L
I N P E RS O N + P H O NE
Copy Editors / Web Designers
Google / Mobile Analytics
Website Optimization
A/B Testing
Online or on-the-job
training
One-on-one teaching
Random QA Checks

17. M E S SAG I NG
+
C H AT
+
D I G I TA
L
I N P E RS O N + P H O NE
Copy Editors / Web Designers
Google / Mobile Analytics
Website Optimization
A/B Testing
Online or on-the-job
training
One-on-one teaching
Random QA Checks

18. M E S SAG I NG
+
C H AT
+
D I G I TA
L
I N P E RS O N + P H O NE
Copy Editors / Web Designers
Google / Mobile Analytics
Website Optimization
A/B Testing
Online or on-the-job
training
One-on-one teaching
Random QA Checks What are the
KPIs that matter?
What is your
brand voice?
Who are
your brand
ambassadors?
Brilliant Smart
Stylish Spontaneous Fresh
Personal Professional Loud

19. What is the solution?

20. ST E P 1
Feed the Engine Chat
Data & KPI Outcomes
CHAT DATA
KPI OUTCOMES

21. ST E P 2
Process in the
Machine Learning Engine
CHAT DATA
KPI OUTCOMES
DATA WAREHOUSE
MACHINE LEARNING
NATURAL LANGUAGE
PROCESSING
ST E P 1
Feed the Engine Chat
Data & KPI Outcomes

22. ST E P 2
Process in the
Machine Learning Engine
ST E P 1
Feed the Engine Chat
Data & KPI Outcomes
ST E P 3
Spits Out Data Telling
What Gains to Expect
CHAT DATA
KPI OUTCOMES
DATA WAREHOUSE
MACHINE LEARNING
NATURAL LANGUAGE
PROCESSING

23. How much does the conversation matter?
What Drives
Sales Conversions?
Agent chat behavior
drives
61.8%of sales conversions
18%
27%
6%
6%
7%
8%
9%
18%
Message Topic
Friendliness
Effort
Visit/Visitor Stats
Visit/Visitor
Demographics
Emotion
Agent Demographics
Responsiveness

25. Worst keywords from agents
Cluster ID Keywords Coefficient P-Value
27
inconvenience, apologize, sorry, delay, confusion, apologies, difficulties, hassle,
caused
-2.6718 0.0152
41 mailing, mail, mailed, mailbox, unsubscribe, postcards, envelope, postage -1.8169 0.0183
47 overlap, shuffle, overlapping, overlaps, select, inpage, itseld, available -1.3677 0.0298
13 pics, pix, pic, upload, uploaded, shoot, take, nude, photograher -1.3360 0.0000
15 hey, nope, yup, yeah, huh, nevermind, ugh, bummer, oops, ditto -1.2768 0.0000
49 ipad, iphone, tablet, laptop, macbook, microsoft, samsung, firefox, canada -1.2666 0.0000
29 thoughts, suggestions, ideas, wo, quests, chat, let, tryyour -1.2111 0.1190
1 located, command, km, meeeeee, i-25, dicolored, kph, angroupon -1.1757 0.1710
34 logged, yet, in, registered, counted, both..i, un-highlighted, faca -1.1302 0.0070
12 credit, card, debit, visa, check, refund, charging, checking, prepaid -1.0747 0.0170

26. How much damage can a bot do? A lot.

27. Data + Scientist = Data Scientist
1. Hypotheses and Testable
Predictions
2. Gather data to test predictions
3. Refine, alter, expand, or reject
hypotheses
HOW?
Recent advances in Deep Learning have unleashed the
potential to solve countless problems, but we can’t lose sight of:
WHY?