Jon Puleston, Lightspeed GMI

1. HOW GOOD ARE BUSINESSES AT
MAKING DECISIONS?
Jon Puleston, VP Innovation, Lightspeed GMI

2. BUSINESSES ARE CONSTANTLY MAKING DECISIONS
WHICH ARE IN EFFECT PREDICTIONS…
• Will Ad A be more effective than Ad B?
• Should we recruit someone or not?
• What will our sales be next year?
• Should we invest in buying this piece of technology?
• What products should we sell?
• Should we put up our prices?

3. How good are the prediction protocols
businesses use to make these decisions?

4. When you step back and look at how some
business decisions are made it’s a little
scary…

5. CASE STUDY: INTERNATIONAL FMCG BUSINESS
PREDICTION CHALLENGE:
CHOOSING WHAT AD TO MAKE
BACKGROUND:
Agency had developed 7 advertising ideas
The 3 person marketing team had to evaluate each idea
and decide which ad the agency should produce
THE ECONOMICS:
Cost to produce the ad = $500k
Media spend = c$2m+

6. EXPERIMENT TO TEST THE DECISION MAKING SKILLS OF
THE AGENCY AND MARKETING TEAM
• We asked groups of people in the agency and marketing team
to predict which idea would make the best ad campaign?
– 15 people in the creative team
– 17 people in the planning team
– 3 members of the marketing team
• In parallel the same ideas were tested out on consumers:
– 500 members of the target audience in Brazil
– A control group of 300 consumers in the UK

7. Clear consensus amongst the Brazilian target audience, the UK
control, the agency creative and the planners

8. THIS IS NOT TO DISCREDIT THE 3 PEOPLE IN
THE MARKETING TEAM
• They only had a 13% chance of making the correct choice - It
takes 200 people to get a really reliable answer
13%
36%
48%
65%
86% 99%
0%
50%
100%
3 people 15 people 30 people 50 people 100 people 200 people

9. RECRUITMENT
• Imagine choosing between 7 candidates
– Same scale of problem
– Same maths
• A task often done by 1 person!
• Emotions play such an over riding part

10. HOW EMOTIONS CLOUD OUR JUDGEMENT
• We tend to select people we like the look of over their raw
ability
• “Humans make up their minds in a few seconds then spend
the rest of the interview asking questions aimed at confirming
their instinctive point of view”
• There are network errors with groups of interviewers too
caused by the overall “affinity” with individual candidates e.g.
being likable/attractive
• Algorithmic selection can be far more effective than interview
based selection
http://www.bloomberg.com/news/articles/2015-11-17/machines-are-better-
than-humans-at-hiring-top-employees

11. SALES FORECASTING
• Monthly projections based on last 2 years of data
• OMG…Where do you start!!!!

12. Sales forecasting science
• Sales go up  All the result of the great
sales team
• Sales go down  operation issues and
trading conditions!

13. Sales forecasting science
• Did sales go up this year  sales will go up next year!
• Did sales drop last year  that was an anomaly! 
Sales will definitely go up next year!
• What did we do in that month last year? …add 10%!
• Wait last year was a bad month, what happened the
year before?...add 10% anyway

14. BOARD ROOM DECISION MAKING
• 12 people ~15 ideal – not bad
• Non-execs adds some independent unbiased opinions - good
• Discussion forum - good
• Issues:
– No dark trading
– Voting process is not independent or unbiased
– Subject to herding effects
– The mix of people on a board not always made up of the best
predictors

15. ARE LEADERS GOOD PREDICTORS
• What makes good leader does not
necessarily make a good predictor
Good leader = firm opinions
Patch record when it comes to
buying players

16. FACTORS DETERMINING BEING A GOOD
PREDICTOR
• Actively open minded thinking* = -0.07
• Knowledge = -0.16
• Deliberation effort -0.18
• General Intelligence (numeracy) = -0.24
• Belief updating -0.36
*They avoid the “myside bias”—the tendency to
bolster one’s own views and dismiss contradictory evidence
(Baron, 2000). Greater tolerance for ambiguity and weaker need for closure
Dogmatism the enemy of good prediction

17. POLITICAL EXPERTS MAKE POOR
PREDICTIONS

18. A GOOD PREDICTOR
GOOD
PREDICTOR
LEADER

19. THE FUTURE = MORE DEMOCRATIC CROWD
DRIVEN DECISION MAKING IN BUSINESS
• Integrating crowd prediction protocols
• Isolating the people in organisations that are good at
making predictions
• Allowing the informed members of an organisation to
steer the boat
• An interesting and important role for research

20. Prediction market: Idea evaluation