2. +
Objectives
n Understand what data analytics (and automated tools) can –
and cannot – do successfully
n Identify the benefits of decision-centric organization, and
how to achieve them in your organization
n J.P. Eggers, Professor of Management & Organizations
n Teach strategic & analytical decision-making (MBA, MS Analytics,
Online, Executive)
n Research on innovation and organizational decision making
n I’m an organizations researcher who is also a data scientist
How to build a machine learning tool!
N
O
T
FO
R
D
ISTR
IBU
TIO
N
3. +
Classifier Exercise
n A classifier is a tool that separates
observations into positive and negative
types
n “tool” is typically logit regression or machine
learning, but can be manually done as well
n “observations” can be locations, pictures,
songs, etc
n “types” means two sets, and each observation
belongs completely to one set or the other
n Is there or isn’t there a Starbucks on this
corner?
n Is this song slow or fast?
n Is this a picture of a hipster or not?
N
O
T
FO
R
D
ISTR
IBU
TIO
N
4. +
Classifier Exercise
n Organize into groups
n I’ll provide you with a set of pictures
n ½ country bands
n ½ heavy metal bands
n Your job is to build a classifier (pen &
paper only)
n Set of statements that categorize the pictures
you have into country vs. heavy metal
n Objective enough that anyone could use your
classifier to tell whether an additional picture
(not in your folder) is country or heavy metal
Adapted from https://datascopeanalytics.com/blog/hipster-classifier-icebreaker/
N
O
T
FO
R
D
ISTR
IBU
TIO
N
5. +
Classifier Exercise
n Examples of statements:
n If red shoes, then country band
n If holding piano, then heavy metal
n Can weight statements, making some
count more than others (optional)
n Take 30 minutes to “build” your
classifier
n Then we will test with different
pictures
Adapted from https://datascopeanalytics.com/blog/hipster-classifier-icebreaker/
N
O
T
FO
R
D
ISTR
IBU
TIO
N
7. +
Classifier Exercise Takeaways
n What is a classifier?
n How can we build statements that classify observations? What
does weighting do?
n What is a training & a test set? Why do we care?
n Can “overfit” models if we just use our data, so test set preserves
honesty
n Concept of error and probabilities
n While pictures fit into perfect categories, our models tended to
produce probabilities
n Given that we don’t know contents of test set beforehand, likely to
have some errors in classification
N
O
T
FO
R
D
ISTR
IBU
TIO
N
11. +
Data should be relevant
Price of
milk
($/gal)
Red Sox
batting avg.
Blood
alcohol
content (%)
3.79 .304 .03
3.45 .320 .09
4.06 .259 .01
3.89 .298 .05
4.12 .332 .13
3.92 .270 .06
3.23 .294 .10
Body mass
(kg)
Margaritas Blood
alcohol
content (%)
103 3 .03
67 5 .09
87 1 .01
52 2 .05
73 5 .13
79 3 .06
110 7 .10
Irrelevant Relevant
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
12. +
Data should be accurate
Inaccurate Accurate
(Precise)
X X
X X
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
13. +
Data can be numbers or names
Numbers
n Amount: 38.3 degrees
n Count: 39 pizzas
n Money: $1,387
n Pixel brightness: 232/255
n Sound intensity: 0.64
Names
n Type: Shih Tzu
n Variety: Caramel latte
n ID: Air Force One
n Model number: R2-D2
n Category: Chocolate
n Text:“Best. Show. Ever. <3”
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
14. +
Data can be numbers or names
Names the look like Numbers
n Phone number: 847-5609
n Zip code: 90210
n ID number: 007
n Serial number: 100000184573
n Credit card number:
5738-7539-9898-0023
n Social security number:
627-42-0932
Numbers that look like Names
n Place: first, second, third
n Size: small, medium, large
n Side: left, middle, right
n Time zone: Pacific, Mountain,
Central, Eastern
n Subway stops: Coney Island,
NY Aquarium, Ocean Parkway,
Brighton Beach
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
16. +
Vague vs. Sharp questions
Cannot be answered with a
name or a number
n What can my data tell me
about my business?
n What should I do?
n How can I increase my
profits?
Can be answered with a
name or a number.
n How many Model Q Gizmos
will I sell in Montreal
during the third quarter?
n Which car in my fleet is
going to fail first?
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
17. +
Data science can only
answer five questions:
1. How much / how many?
2.Which category?
3.Which groups?
4. Is it weird?
5.Which action?
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
18. +
How much / how many?
n What will the temperature be next Tuesday?
n What will my fourth quarter sales in Portugal be?
n How many new followers will I get next week?
[regression]
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
19. +
Which category?
n Is this an image of a cat or a
dog?
n Which aircraft is causing this
radar signature?
n What is the topic of this news
article?
[classification]
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
20. +
Which groups?
n Which shoppers have similar
tastes in produce?
n Which viewers like the same
kind of movies?
n What is a natural way to break
these documents into five topic
groups?
[clustering, recommendation]
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
21. +
Is this weird?
n Is this pressure reading unusual?
n Is this internet message typical?
n Is this combination of purchases
very different from what this
customer has made in the past?
[anomaly detection]
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
22. +
Which action?
n Should I raise or lower the
temperature?
n Should I vacuum the living room
again or stay plugged in to my
charging station?
n Should I brake or accelerate in
response to that yellow light?
[reinforcement learning]
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
24. +
One target per row
Stock
price
Date Day
of
week
Dow
Jones
Last
month
sales
Last
quarter
sales
Market
share
New
users
last
month
New
users
last
quarter
Days
since
press
release
Days
since
product
release
Total
users
57.3 5/21 Tue 17,245 68.8M 211.2M 23.1% 63,522 195,322 3 96 2.49M
58.8 5/22 Wed 17,289 68.8M 211.2M 23.1% 63,522 195,322 4 97 2.49M
56.9 5/23 Thu 17,115 68.8M 211.2M 23.1% 63,522 195,322 5 98 2.49M
57.4 5/24 Fri 17,278 68.8M 211.2M 23.1% 63,522 195,322 6 99 2.49M
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
25. +
One target per row
n Aggregate
User name Date joined
little_lil Jan 27, 2014
popoverGuy Jan 27, 2014
Red_Red Jan 28, 2014
David_G_53 Jan 30, 2014
randll Jan 30, 2014
… …
Stock
price
Date Day
of
week
Dow
Jones
Last
month
sales
Last
quarter
sales
Market
share
New
users
last
month
New
users
last
quarter
Days
since
press
release
Days
since
product
release
Total
users
57.3 5/21 Tue 17,245 68.8M 211.2M 23.1% 63,522 195,322 3 96 2.49M
58.8 5/22 Wed 17,289 68.8M 211.2M 23.1% 63,522 195,322 4 97 2.49M
56.9 5/23 Thu 17,115 68.8M 211.2M 23.1% 63,522 195,322 5 98 2.49M
57.4 5/24 Fri 17,278 68.8M 211.2M 23.1% 63,522 195,322 6 99 2.49M
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
26. +
One target per row
n Aggregate
n Distribute
Stock
price
Date Day
of
week
Dow
Jones
Last
month
sales
Last
quarter
sales
Market
share
New
users
last
month
New
users
last
quarter
Days
since
press
release
Days
since
product
release
Total
users
57.3 5/21 Tue 17,245 68.8M 211.2M 23.1% 63,522 195,322 3 96 2.49M
58.8 5/22 Wed 17,289 68.8M 211.2M 23.1% 63,522 195,322 4 97 2.49M
56.9 5/23 Thu 17,115 68.8M 211.2M 23.1% 63,522 195,322 5 98 2.49M
57.4 5/24 Fri 17,278 68.8M 211.2M 23.1% 63,522 195,322 6 99 2.49M
Quarter Total
sales
2015Q4 119.2M
2016Q1 221.0M
2016Q2 215.9M
2016Q3 189.3M
2016Q4 211.2M
… …
Month Total
sales
2016/01 43.0M
2016/02 60.1M
2016/03 55.5M
2016/04 41.7M
2016/05 68.8M
… …
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
27. +
One target per row
n Aggregate
n Distribute
n Compute
Stock
price
Date Day
of
week
Dow
Jones
Last
month
sales
Last
quarter
sales
Market
share
New
users
last
month
New
users
last
quarter
Days
since
press
release
Days
since
product
release
Total
users
57.3 5/21 Tue 17,245 68.8M 211.2M 23.1% 63,522 195,322 3 96 2.49M
58.8 5/22 Wed 17,289 68.8M 211.2M 23.1% 63,522 195,322 4 97 2.49M
56.9 5/23 Thu 17,115 68.8M 211.2M 23.1% 63,522 195,322 5 98 2.49M
57.4 5/24 Fri 17,278 68.8M 211.2M 23.1% 63,522 195,322 6 99 2.49M
Press release
date
Subject
2016/03/24 Mega amazing
whizbang
2016/05/03 Super widget
upgrade
2016/05/18 New gizmos on
the flimflam
… …
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
28. +
One target per row
n Aggregate
n Distribute
n Compute
n Measure
Stock
price
Date Day
of
week
Dow
Jones
Last
month
sales
Last
quarter
sales
Market
share
New
users
last
month
New
users
last
quarter
Days
since
press
release
Days
since
product
release
Total
users
57.3 5/21 Tue 17,245 68.8M 211.2M 23.1% 63,522 195,322 3 96 2.49M
58.8 5/22 Wed 17,289 68.8M 211.2M 23.1% 63,522 195,322 4 97 2.49M
56.9 5/23 Thu 17,115 68.8M 211.2M 23.1% 63,522 195,322 5 98 2.49M
57.4 5/24 Fri 17,278 68.8M 211.2M 23.1% 63,522 195,322 6 99 2.49M
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
29. +
One target per row
n Aggregate
n Distribute
n Compute
n Measure
n Estimate
Stock
price
Date Day
of
week
Dow
Jones
Last
month
sales
Last
quarter
sales
Market
share
New
users
last
month
New
users
last
quarter
Days
since
press
release
Days
since
product
release
Total
users
57.3 5/21 Tue 17,245 68.8M 211.2M 23.1% 63,522 195,322 3 96 2.49M
58.8 5/22 Wed 17,289 68.8M 211.2M 23.1% 63,522 195,322 4 97 2.49M
56.9 5/23 Thu 17,115 68.8M 211.2M 23.1% 63,522 195,322 5 98 2.49M
57.4 5/24 Fri 17,278 68.8M 211.2M 23.1% 63,522 195,322 6 99 2.49M
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
31. +
ID First
name
Last name Birth
year
Height Birthplace Identity
is secret
Can
fly
Alignment Wears cape
7435 Bruce Wayne 1969* 6’ 2” Gotham Y 3 anti-villain black
0958 Ororo Munroe --1979-- 5’ 11” Manhattan 9 good long
9471 Diana Trevor 1618 5’ 8” Paradise Island Y Jet truth rarely
9483 Janet Van Dyne 19.42 5’ 4” Cresskill tiny Good Not really
0696 Peter Parker 1111983 5’ 10” Queens Y Fall right never
5531 Harleen Quinzell 1981 5’ 2” Gotham Y - evil no
4734 Erik Lehnsherr 1-9-3-2 6’ 0” Hamburg Lev. mutants Absolutely
7757 Natasha Romanova 1983 5’ 7” St. Petersburg jet depends No way
0323 Jean Grey “1977” 5’ 6” Annandale No good Mostly not
3980 Clark Kent “1954” 6’ 4” Krypton Y 12 Truth always
3057 Victor Von Doom “1943” 6’ 2” Latveria 1 Bad yes
0573 Stephen Strange 1968 6’ 2” Philidelphia not light Y
7452 Thor Odinson 2287 BC 6’ 6” Norway 10 Good Of course
1437 Selina Kyle 1998 5’ 7” Gotham Y NA Neutral It clashes
1883 Raven Darkholme ..1911.. 5’ 10” unknown Y no mostly bad Not really
5830 Kara Zor-el 1961 5’ 7” Krypton Y fast G Yes
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
32. +
ID First
name
Last name Birth
year
Height Birthplace Identity
is secret
Can
fly
Alignment Wears cape
7435 Bruce Wayne 1969* 6’ 2” Gotham Y 3 anti-villain black
0958 Ororo Munroe --1979-- 5’ 11” Manhattan 9 good long
9471 Diana Trevor 1618 5’ 8” Paradise Island Y Jet truth rarely
9483 Janet Van Dyne 19.42 5’ 4” Cresskill tiny Good Not really
0696 Peter Parker 1111983 5’ 10” Queens Y Fall right never
5531 Harleen Quinzell 1981 5’ 2” Gotham Y - evil no
4734 Erik Lehnsherr 1-9-3-2 6’ 0” Hamburg Lev. mutants Absolutely
7757 Natasha Romanova 1983 5’ 7” St. Petersburg jet depends No way
0323 Jean Grey “1977” 5’ 6” Annandale No good Mostly not
3980 Clark Kent “1954” 6’ 4” Krypton Y 12 Truth always
3057 Victor Von Doom “1943” 6’ 2” Latveria 1 Bad yes
0573 Stephen Strange 1968 6’ 2” Philidelphia not light Y
7452 Thor Odinson 2287 BC 6’ 6” Norway 10 Good Of course
1437 Selina Kyle 1998 5’ 7” Gotham Y NA Neutral It clashes
1883 Raven Darkholme ..1911.. 5’ 10” unknown Y no mostly bad Not really
5830 Kara Zor-el 1961 5’ 7” Krypton Y fast G Yes
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
33. +
ID First
name
Last name Birth
year
Height Birthplace Identity
is secret
Can
fly
Alignment Wears cape
7435 Bruce Wayne 1969 6’ 2” Gotham Y 3 anti-villain black
0958 Ororo Munroe 1979 5’ 11” Manhattan 9 good long
9471 Diana Trevor 1618 5’ 8” Paradise Island Y Jet truth rarely
9483 Janet Van Dyne 1942 5’ 4” Cresskill tiny Good Not really
0696 Peter Parker 1983 5’ 10” Queens Y Fall right never
5531 Harleen Quinzell 1981 5’ 2” Gotham Y - evil no
4734 Erik Lehnsherr 1932 6’ 0” Hamburg Lev. mutants Absolutely
7757 Natasha Romanova 1983 5’ 7” St. Petersburg jet depends No way
0323 Jean Grey 1977 5’ 6” Annandale No good Mostly not
3980 Clark Kent 1954 6’ 4” Krypton Y 12 Truth always
3057 Victor Von Doom 1943 6’ 2” Latveria 1 Bad yes
0573 Stephen Strange 1968 6’ 2” Philidelphia not light Y
7452 Thor Odinson -2287 6’ 6” Norway 10 Good Of course
1437 Selina Kyle 1998 5’ 7” Gotham Y NA Neutral It clashes
1883 Raven Darkholme 1911 5’ 10” unknown Y no mostly bad Not really
5830 Kara Zor-el 1961 5’ 7” Krypton Y fast G Yes
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
34. +
ID First
name
Last name Birth
year
Height Birthplace Identity
is secret
Can
fly
Alignment Wears cape
7435 Bruce Wayne 1969 74 Gotham Y N Good Y
0958 Ororo Munroe 1979 71 Manhattan N Y Good Y
9471 Diana Trevor 1618 68 Paradise Island Y N Good N
9483 Janet Van Dyne 1942 64 Cresskill N Y Good N
0696 Peter Parker 1983 70 Queens Y N Good N
5531 Harleen Quinzell 1981 62 Gotham Y N Bad N
4734 Erik Lehnsherr 1932 72 Hamburg N N Bad Y
7757 Natasha Romanova 1983 67 St. Petersburg N N Good N
0323 Jean Grey 1977 66 Annandale N N Good N
3980 Clark Kent 1954 76 Krypton Y Y Good Y
3057 Victor Von Doom 1943 74 Latvia N N Bad Y
0573 Stephen Strange 1968 74 Philadelphia N N Good Y
7452 Thor Odinson -2287 78 Norway N Y Good Y
1437 Selina Kyle 1998 67 Gotham Y N Neutral N
1883 Raven Darkholme 1911 70 unknown Y N Bad N
5830 Kara Zor-el 1961 67 Krypton Y Y Good Y
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
36. +
Sometimes the data needs to be
transformed
n Reasons:
n Violates assumptions of regression (e.g., non-normal
distributions)
n Change between two periods (e.g., growth in income from 2000 to
2010)
n Really want differences between variables (e.g., may care about
“net steals”, which is [stolen bases] – [caught stealing])
N
O
T
FO
R
D
ISTR
IBU
TIO
N
38. +
Taking action generates new data
n Have to think through whether answer for what to do next is
clear or not
n If not, go back to data and/or consider different empirical
approaches to address question
n If yes, need to consider factors not accounted for in data analytics
approach that may affect choice
n Data science is not perfect and is as much are as science
n Deploying a new marketing campaign, entering a new
market, launching a new product each provides access to
new data
N
O
T
FO
R
D
ISTR
IBU
TIO
N
45. +
Gap 1
n Nearly all machine learning algorithms (and data
science approaches in general) assume that the
world does not change
Borrowed from
Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
46. +
Gap 2
n Most machine
learning algorithms
take a lot of
examples to learn
n Manual data
science can often
deal with fewer
examples, but
difficult
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
47. +
Gap 3
n Machine learning can’t tell what caused what
http://www.tylervigen.com/spurious-correlationsBorrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
48. +
Human insight and judgment close
the gap
n We’re good at making reasonable guesses without enough
information
Borrowed from
Brandon
Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
49. +
Summary
n Data science can be useful to address a limited set of sharp
questions, provided the appropriate data are available
n All predictions will have error, which depends on data
available and underlying nature of the phenomenon
n Assessing model reliability may involve using training and
testing sets to build veracity
n But there are important limitations to data science and
predictive modeling that are important for organizations
N
O
T
FO
R
D
ISTR
IBU
TIO
N
52. +
Getting value from analytics:
Preview & agenda
n Two major organizational impediments to getting value from
analytics investments:
n Not asking the right
questions
n Not designing the
organization properly
→ What can (and can’t) analytics
do, how to ask good questions
→ Value of decision-centric
organizations
N
O
T
FO
R
D
ISTR
IBU
TIO
N
53. +
Linking contextual and analytical
skills
n Contextual (business) knowledge is vital for getting value
from analytics
n Figuring out which questions have value
n Understanding what to do – and NOT to do – with analytical output
n So how do we better link analytical and contextual skill sets?
N
O
T
FO
R
D
ISTR
IBU
TIO
N
54. +
A Tale of Two Cities
(and their baseball analytics staff)
n In 1999, the Oakland A’s hired
27 year old Paul DePodesta
n Harvard alum, had been with
Cleveland Indians
n Working with GM Billy Beane,
DePodesta helped reshape the
way the A’s evaluated players
and productivity
n Led to analytics revolution
within the A’s, and in baseball
more generally
N
O
T
FO
R
D
ISTR
IBU
TIO
N
55. +
A Tale of Two Cities
(and their baseball analytics staff)
n In 2013, the Philadelphia
Phillies hired 29 year old Scott
Freedman
n Indiana U business school
alum, had been working for
MLB on labor issues
n In 2013 and 2014, no real sign
that analytics had led to any
real decisions within Phillies
organization
n And they had the worst record
in baseball…
GM Ruben Amaro
N
O
T
FO
R
D
ISTR
IBU
TIO
N
56. +
A Tale of Two Cities
(and their baseball analytics staff)
n Is DePodesta more talented than Freedman?
n Maybe, maybe not
n Different structure?
n Not really. Both “Special Assistant to the GM”, reporting directly to
GM
n So what is different?
n Intuitively, it is about whether the organization listens and is willing to
change the way that it does business
N
O
T
FO
R
D
ISTR
IBU
TIO
N
57. +
A Tale of Two Cities
(and their baseball analytics staff)
n “We don’t have an in-house stats guy, and I kind of feel we
never will.We’re not a statistics-driven organization by any
means.”
n GM Ruben Amaro, 2010
n “I don’t know if it [hiring Freedman] is going to change the
way we do business, necessarily.”
n Amaro, 2013
N
O
T
FO
R
D
ISTR
IBU
TIO
N
58. +
Signs your org may not be taking
analytics seriously
n People problems
n The organization is only hiring people with “core” analytics
skillsets, and no one with “bridging” skillsets
n The champion for analytics in the organization doesn’t understand
how business decisions are made in the firm
n Politics problems
n The champion for analytics in the organization cannot talk to
division heads as a political peer (not necessarily equal)
n Responsibility is completely located in divisions, where
preserving the status quo becomes to normal response
N
O
T
FO
R
D
ISTR
IBU
TIO
N
59. +
Signs your org may not be taking
analytics seriously
n Process problems
n The champion for analytics in the organization does not report to
someone with authority over decisions in multiple aspects of the
organization, but only focuses on one
n Transforming decision making starts (and ends) with offering
useful data to existing decision makers, as opposed to revisioning
the decision making process
N
O
T
FO
R
D
ISTR
IBU
TIO
N
60. +
How to solve it? Start at the
beginning
DATA ANALYTICS
What is data analytics about?
N
O
T
FO
R
D
ISTR
IBU
TIO
N
61. +
What is data analytics about?:
A forwards process
Data
Inputs
Analytic
Models
Support
Tools
Decisions?
N
O
T
FO
R
D
ISTR
IBU
TIO
N
62. +
What is data analytics about?:
A focus on decision making
n The essence of organizations is decision making
n Particularly about resources and their allocation to
improve performance
n Data analytics is an advanced technique for
improving decision making effectiveness
n Improving existing decision making
n Simplifying existing decision making
n Reducing knowledge required for decision making
n Allowing for new decisions to be made
N
O
T
FO
R
D
ISTR
IBU
TIO
N
63. +
What is data analytics about?:
A backwards process
Data
Inputs
Analytic
Models
Support
Tools
Decisions
N
O
T
FO
R
D
ISTR
IBU
TIO
N
64. +
Building a decision-centric
organization
n Three factors – people, process, and
incentives – are at the heart of
corporate governance
n Decision-centricity means starting
with key decisions and answering
questions:
n What skills does the person need to
make the right decision?
n What structures and support does the
person need to make the right
decision?
n What incentives and information need
to be present to make the right
decision?
People
& Skills
Process &
Structure
Incentives
& Info
N
O
T
FO
R
D
ISTR
IBU
TIO
N
65. +
Steps for thinking about decision-
centricity:
n Identify the key repeated decisions in the life of the company (at
any level)
n Focus on a series of factors:
n What tacit, contextual knowledge is required to make good decisions?
n What data is needed to inform decision-making?
n What political pressures and biases may get in the way of making
good decisions?
n Design around these constraints:
n Identify people with right skills
n Get them contextual knowledge
n Collect, organize, and present the right data
n Consider limitations to making good decisions
N
O
T
FO
R
D
ISTR
IBU
TIO
N
66. +
Example: Roller coaster operator
n Decision: Roller coaster manager, seating
people
n Current:“Grab and place”; some people make
requests (front, back); sometimes a singles line;
need party of a certain size
n Data-driven system could suggest seating
arrangements
n Knowing party size in advance helps from
queuing theory perspective
n Benefits: Never empty seats = shorter
lines, faster seating = quicker line
movement
n Knowing height could allow you to avoid
seating a child behind a tall adult
n Benefit: No complaining children about
inability to see
n Knowing preferences for seating (e.g., front,
back)
n Benefit: Delight customer with favorite
opportunity; make sure not always same
people getting front/back
N
O
T
FO
R
D
ISTR
IBU
TIO
N
67. +
Example: Roller coaster operator
n Data:
n Requires an RFID tagging system,
customer group information, customer
preferences, historical information
n People:
n Similar to existing, but should reduce
skill level for good job performance
n Needs to be good with blending
system suggestion and real-world
observation
n Organization needs to be about
logging irregularities to “learn” as
people want different things, trade
tags
N
O
T
FO
R
D
ISTR
IBU
TIO
N
68. +
Decision-Centric Organization
Takeaways
n Philosophy of considering how analytics and machine
learning can transform the organization
n Core idea is to:
n Identify key decisions in the organization that could be
transformed via analytics
n Consider how data can transform the process of decision making
and what knowledge and skills are required to make the decision
n Build processes to facilitate decision making
n Collecting and presenting information
n Monitoring, metrics and incentives
N
O
T
FO
R
D
ISTR
IBU
TIO
N
69. +
Decision-Centric Organization
Takeaways
n When thinking about building an organization around
analytics, there are a few key concepts:
n Evidence-based decision making (vs. instinct-based)
n Proactive strategy creation (vs. reactive)
n Decision-centric organization (vs. traditional, power-centric)
n Requires developing a plan
n Don’t simply append analytics to the organization
n Rethink organization with analytics – and the key decisions that
analytics are meant to support – at its core
n Recognize that the goal of analytics is to facilitate decision
making
N
O
T
FO
R
D
ISTR
IBU
TIO
N
70. +
Applying Decision-Centric
Principles inYour Organization
n Identify a specific, single decision within your organization
n It should be repeated, important
n Articulate:
n How could data & analytics make it better than the current
arrangement? Where is the new value?
n What data do you need to make the decision? Where could the
data come from? What are limits of data, analytics here?
n What skills and traits do people in these roles need? How can the
organization build around this decision to support it?
n What metrics are available to know if you have improved
outcomes?
N
O
T
FO
R
D
ISTR
IBU
TIO
N
71. +
Applying Decision-Centric
Principles inYour Organization
n Start:Work individually for 15 minutes to develop a plan to improve one
aspect of business via decision-centric approach
n Link clearly to how it creates value for the organization
n Discuss: Pair with another participant to discuss each other’s ideas for
10 minutes
n Brainstorm question, problems, solutions for each other
n Finalize: Spend 5 more minutes finalizing your thoughts and how to
present them
n Present:We will have volunteers share a 2-3 minute summary of your
decision with the group
n Feel free to approach me to discuss as needed
n I will move among participants to talk and participate in group discussions
N
O
T
FO
R
D
ISTR
IBU
TIO
N
74. +
Getting value from analytics:
Roadblocks are organizational
n Analytics skills clearly add value
n But thorniest impediments to
creating value are organizational:
n Asking the right questions
n Knowing how to interpret the output
n Linking business with analytics
N
O
T
FO
R
D
ISTR
IBU
TIO
N
75. +
Getting value from analytics:
Program objectives
n Two major organizational impediments to getting value from
analytics investments:
n Not asking the right
questions
n Not designing the
organization properly
→ What can (and can’t) analytics
do, how to ask good questions
→ Value of decision-centric
organizations
N
O
T
FO
R
D
ISTR
IBU
TIO
N
76. +
Classifier Exercise
n What is a classifier? How does that
tell us what regression is and can do?
n Key factors:
n Combinations of statements
n Role of weighting
n Training & test set
n Error & probabilities
n How can we build statements that
classify observations? What does
weighting do?
N
O
T
FO
R
D
ISTR
IBU
TIO
N
78. +
Data science can only
answer five questions:
1. How much / how many?
2.Which category?
3.Which groups?
4. Is it weird?
5.Which action?
Borrowed from Brandon Rohrer
N
O
T
FO
R
D
ISTR
IBU
TIO
N
79. +
Signs your org may not be taking
analytics seriously
n People problems
n The organization is only hiring people with “core” analytics
skillsets, and no one with “bridging” skillsets
n The champion for analytics in the organization doesn’t understand
how business decisions are made in the firm
n Politics problems
n The champion for analytics in the organization cannot talk to
division heads as a political peer (not necessarily equal)
n Responsibility is completely located in divisions, where
preserving the status quo becomes to normal response
N
O
T
FO
R
D
ISTR
IBU
TIO
N
80. +
Signs your org may not be taking
analytics seriously
n Process problems
n The champion for analytics in the organization does not report to
someone with authority over decisions in multiple aspects of the
organization, but only focuses on one
n Transforming decision making starts (and ends) with offering
useful data to existing decision makers, as opposed to revisioning
the decision making process
N
O
T
FO
R
D
ISTR
IBU
TIO
N
81. +
Steps for thinking about decision-
centricity:
n Identify the key repeated decisions in the life of the company (at
any level)
n Focus on a series of factors:
n What tacit, contextual knowledge is required to make good decisions?
n What data is needed to inform decision-making?
n What political pressures and biases may get in the way of making
good decisions?
n Design around these constraints:
n Identify people with right skills
n Get them contextual knowledge
n Collect, organize, and present the right data
n Consider limitations to making good decisions
N
O
T
FO
R
D
ISTR
IBU
TIO
N
82. +
Getting value from analytics:
Concluding perspective
n We can help to improve the value of our investments in
analytics:
n Asking better questions, understanding benefit and limit of
analytical tools
n Designing decision-centric organizations to make full use of
analytics investments
n Improving link between analytical and organizational
perspectives on decision making will improve value of
analytics investments
N
O
T
FO
R
D
ISTR
IBU
TIO
N
