A terrific piece of work delivered at AMA ServSig Doctoral Consortium Taipei 2013 National Taiwan University, Taipei, Taiwan — July 3-4, 2013 by Lerzan Aksoy & Timothy Keiningham

1. The linked image cannot be displayed. The file may have been moved, renamed, or deleted. Verify that the link points to the correct file and location.
Research Journey
Our Investigation of the
Net Promoter Score
AMA ServSig Doctoral Consortium Taipei 2013
National Taiwan University, Taipei, Taiwan — July 3-4, 2013
Lerzan Aksoy & Timothy Keiningham
Lerzan Aksoy
Associate Professor of Marketing
FORDHAM UNIVERSITY
Timothy Keiningham
Global Chief Strategy Officer
IPSOS LOYALTY

2. The Team
Tor Wallin Andreassen
Professor of
Marketing
Norwegian School of Management
Lerzan Aksoy
Associate Professor of
Marketing
Fordham University
Timothy Keiningham
Global Chief Strategy
Officer
Ipsos Loyalty
Bruce Cooil
The Dean Samuel B. and Evelyn R.
Richmond Professor of Management
Vanderbilt University

3. “A Longitudinal Examination of Net Promoter and Firm
Revenue Growth,” Journal of Marketing, July 2007
2007 MSI H. Paul Root Award

4. “The Value of Different Customer Satisfaction and Loyalty Metrics in
Predicting Customer Retention, Recommendation, and Share-of-Wallet,”
Managing Service Quality, 17 (4), 2007
2007 Outstanding Paper (Best Paper) Award

5. “Linking Customer Loyalty to Growth,”
MIT Sloan Management Review, Summer, 2008

6. REALLY
Know the Topic
Timothy Keiningham

7. Net Promoter Score Featured in the
Harvard Business Review, MIT Sloan Management
Review, and the book The Ultimate Question

8. How to Calculate a Net Promoter Score
Ø Net Promoter asks one question, “Would you
recommend us to a friend or colleague?” that
customers answer on a scale of 0 to 10.
Ø Responses are grouped into three categories:
Detractors, Passives and Promoters.
Ø Subtract Detractors from Promoters and you have a
Net Promoter Score.
0 1 2 3 4 5 6 7 8 9 10
Likelihood to Recommend
Not At All Extremely
Likely Neutral Likely
Detractors Passives Promoters
Promoters Detractors Net Promoter

9. Net Promoter Is Sold As “The Single Most Reliable
Indicator” Of Firm Growth
http://www.netpromoter.com/

10. Reichheld’s Analysis
Reichheld, in partnership with
Satmetrix and Bain & Company,
conducted both a micro (customer-
level) analysis and a macro (firm-
level) analysis as part of his Net
Promoter research.

11. Key Findings from Reichheld’s Macro-level (Firm-Level)
Analysis
“Net Promoter is the “single most reliable indicator of a
company's ability to grow.”
• Net Promoter links to firms’ relative growth rates
within their respective industries
• Net Promoter leaders outgrow their competitors in most
industries by an average of 2.5 times.
• A twelve-point increase in Net Promoter leads to a
doubling in a company’s rate of growth on average.
• The key finding of this research is that firms need
only track Net Promoter if the goal is firm growth.
Reichheld, Frederick F. (2003), “The One Number You Need to Grow,” Harvard Business Review, vol. 81, no. 12
(December), 46-54.
Satmetrix (2004), “The Power Behind a Single Number: Growing Your Business with Net Promoter,” Satmetrix Systems
white paper <http://www.satmetrix.com/pdfs/netpromoterWPfinal.pdf>

12. Two Key Findings from Reichheld’s Micro-level
(Customer-Level) Analysis
“The likelihood to recommend question proved to be
the top correlate to actual customer behavior 80% of
the time” (Satmetrix 2004).
“More explicitly, if customers reported that they were likely to
recommend a particular company to a friend or colleague, then
these same customers were also likely to actually repurchase from
the company, as well as generate new business by referring the
company via word-of-mouth.”
Recommend intention is the only variable needed to
predict customers’ loyalty behaviors.
Reichheld states, “Interestingly, creating a weighted index – based
on the responses to multiple questions and taking into account the
relative effectiveness of those questions – provided insignificant
predictive advantage.” (Reichheld 2003).
Reichheld, Frederick F. (2003), “The One Number You Need to Grow,” Harvard Business Review, vol. 81, no. 12
(December), 46-54.
Satmetrix (2004), “The Power Behind a Single Number: Growing Your Business with Net Promoter,” Satmetrix Systems
white paper <http://www.satmetrix.com/pdfs/netpromoterWPfinal.pdf>

13. Data Is King!!!

14. We Obtained Data to Replicate the Micro-Level Analysis
from a Large Market Research Firm
• We examined data from a two-year study of
over 8,000 customers of firms in one of three
industries: retail banking, mass-merchant retail,
and Internet service providers (ISPs).
• Individual customer ratings of common
satisfaction and loyalty metrics were monitored
over two years.
• In the second year of the study, customers’
purchasing (retention and share-of-wallet) and
recommendation behaviors were also tracked.

15. Our Findings from the Micro-Level Analysis
Customers’ future loyalty behaviors are
distinct (retention, share of wallet, and
word of mouth). As a result,
no single measure adequately
predicted customers' future loyalty
behaviors

16. Finding Valid Firm-Level Multi-Industry Longitudinal Data
for the Macro-Level Analysis Proved Difficult
• The American Customer Satisfaction Index (ACSI)
does not ask a recommend intention question
• Fortunately, we were able to obtain equivalent
data to that used by Reichheld from the
Norwegian Customer Satisfaction Barometer
(NCSB) .
• We used 15,500+ interviews from the NCSB to
replicate the macro-level analysis

17. Our Findings from the Macro-Level Analysis
No single measure did an
adequate job of predicting firm
growth.
For example, Net Promoter was
only best 2 out of 19 times!

18. Possible Research Bias?
We examined potential factors that
could have caused our results to
differ, however, none appear
plausible.
Based upon our analysis, it is
difficult to imagine a scenario
whereby Net Promoter would be
classified as the superior metric.

19. Go the Extra Mile
Timothy Keiningham

20. How Can We Make and Apples-to-Apples Comparison?
• Without access to the raw data used by
Reichheld, it is impossible to know with
certainty if the variables under
consideration were examined fairly.
• There is, however, one opportunity to
compare Net Promoter to one metric that
Reichheld examined and criticizes: the
American Customer Satisfaction Index
(ACSI).

21. Reichheld Specifically Targets the American Customer
Satisfaction Index as Inferior in His HBR Article
Our research indicates that
satisfaction lacks a consistently
demonstrable connection to
actual customer behavior and
growth. This finding is borne out
by the short shrift that investors
give to such reports as the
American Customer Satisfaction
Index. The ACSI, published
quarterly in the Wall Street
Journal, reflects customer
satisfaction ratings of some 200
U.S. companies. In general, it is
difficult to discern a strong
correlation between high
customer satisfaction scores and
outstanding sales growth.

22. Reichheld Specifically Targets the American Customer
Satisfaction Index as Inferior in His Book
“Some recent evidence that there is little connection between satisfaction
scores and economic results comes from the ACSI itself, whose data used to
be published each quarter in the Wall Street Journal (under the heading of
marketing, not investing)” (The Ultimate Question, p. 85).

23. Reichheld States that Bain Research Shows a ZERO
Correlation between the ACSI and Growth
Frederick F. Reichheld (2004), “Net Promoters,” Bain Audio Presentation, (February 24), <http://resultsbrief.bain.com/videos/0402/main.html>
“A Bain team looked at the correlation between growth and
satisfaction, and found there is none.” Reichheld 2004
An R-square of 0.00
indicates absolutely no
correlation whatsoever!
The ACSI

24. An Opportunity to Compare Net Promoter and the
American Customer Satisfaction Index
§ Reichheld presents charts for three industries
that are also tracked by the ACSI: airlines, life
insurance, and computers.
§ Data showing the relationship between Net
Promoter scores and growth were
reconstructed based upon scatter-plots of the
data featured in the appendix to The Ultimate
Question (Reichheld 2006, pp. 192-194).
• To insure accuracy, the tables were scanned, and the
corresponding graphics imported into a charting software
package where they were used as background images.
• As a final check of the data, the R-square of the recreated
data was compared to the R-square reported by Reichheld --
all were identical.
§ As these data sets were specifically selected
by Reichheld to demonstrate the linkage
between Net Promoter and growth, they
clearly should reveal relationships where Net
Promoter is a superior predictor of growth to
other metrics.

25. Comparison of Net Promoter and ACSI:
Personal Computers
R2: .68
R2 (ACSI only companies*): .70 R2 (ACSI only companies*): .76
Net Promoter American Customer Satisfaction Index (ACSI)
Threeyearshipmentgrowth
670% 10% 20% 30% 40% 50% 60% 69 71 73 75 77 79
0%
-20%
-10%
10%
20%
30%
40%
0%
-20%
-10%
10%
20%
30%
40%
IBM
Dell
H-P
GatewayCompaq
Dell
H-P
GatewayCompaq
Threeyearshipmentgrowth
Net Promoters Q1 2001 - Q4 2002 ACSI 2001 - 2002
* The ACSI does not track IBM

26. Comparison of Net Promoter and ACSI:
Life Insurance
R2: .86
R2 (ACSI only companies): .83 R2 (ACSI only companies): .58
Net Promoter American Customer Satisfaction Index (ACSI)
PremiumGrowth(1999-2003)
-20% -10% 0% 10% 20% 30% 40%
5%
-5%
0%
10%
15%
NY Life
Dell
Met Life
Prudential
Net Promoters Q1 2001 - Q4 2002
Northwestern
State Farm
Primerica
73 83
5%
-5%
0%
10%
15%
NY Life
Dell
Met Life
Prudential
ACSI 2001 - 2002
Northwestern
75 77 79 81
PremiumGrowth(1999-2003)
* The ACSI does not track Primerica and State Farm

27. Comparison of Net Promoter and ACSI:
Airlines
62
R2: .68
R2 (ACSI only companies): .57 R2 (ACSI only companies): .70
Net Promoter American Customer Satisfaction Index (ACSI)
Net Promoters Q1 2000 - Q4 2002 ACSI 2000 - 2002
Threeyeargrowth
-10% 0% 10% 20% 30% 40% 50%
-5%
-10%
0%
5%
10%
60%
Southwest
Alaska
Continental
United
Delta
AMR
US Air
TWA
Northwest
Am. West
Threeyeargrowth
58 64 66 68 70 72
-5%
-10%
0%
5%
10%
74
Southwest
Continental
United
Delta
AMR
Northwest
60
US Air
* The ACSI does not track Alaska, Am. West, and TWA

28. Slow = Dead
Someone Has to Drive
the Process

29. Keeping the Process Moving Quickly
Will Not Make You Popular

30. Getting There Is Not Half the Fun?
Why do you
keep this job?
It’s killing you.
I thought
“publish or
perish” was
rhetorical.

31. Actual Quote from Co-Author
Regarding the Relentless Pressure to
Complete This Paper Quickly
“If our friendship survives
this paper, we will be
friends forever.”

32. The Reality:
If Your Topic Is Important, Someone Else Is Working On It Too!
§ Leading scientific
researchers teamed up
with J.D. Power to gather
the necessary data to
investigate the ability of
Net Promoter to predict
financial performance.
§ The results of this
research were published
in Marketing Research in
the Summer of 2007.
• Our Journal of Marketing
paper was published in
July 2007.

33. You Cannot Please
Everyone
Important Works Often
Have Difficult Reviews

34. Reviewer Reaction Is Split!
The reviews of 06-263-IR, “A Longitudinal
Examination of ‘Net Promoter’ … ,” are now
in. This version was reviewed by a panel of
four knowledgeable reviewers. In addition I
read the paper carefully myself.
The reviewer reaction is split, with
Reviewers 1 and 4 recommending
acceptance, Reviewer 3 recommending
revision, and Reviewer 2 recommending
rejection.

35. The Entire Review from One
Reviewer
Reviewer Number: 2
This paper is interesting and clearly written.
Unfortunately it is not particularly scholarly.
Almost no literature is cited, no theories drawn upon, or
built upon. The introduction is high in fluff quotient.
Difficult to assess whether data differences (i.e.,
whether the Norwegian data are comparable to the
findings in Sweden and the U.S.) are attributable to
cultural differences or mere measurement issues.
Methodological techniques are far too narrow and
simple.

36. The Importance of a Strong
Editor Who Knows the Topic
I know enough about this industry to know the potential
importance of this paper in the business world, given
the surprising popularity of Reichheld’s measure
among business leaders. This paper gives proper
scrutiny to a current management fad.
…
Overall I believe that the positives of the paper, noted
clearly by the reviewers, far outweigh the negatives. …
Because of this, based on the reviewers’ evaluations
and my own reading, I am pleased to conditionally
accept your paper for publication in JM.

37. PROMOTE,
PROMOTE,
PROMOTE!

38. Getting the Word Out
We researched every article and
blog post we could find that had
written on Net Promoter.
We personally contacted every
one of these authors to let them
know the findings of our research.

39. Net Promoter Score Under Attack,
Research Magazine, July 2007

40. The Net Promoter Debate, Ad Map, May 2007

41. Reichheld Under Attack,
Colloquy, Summer 2007

42. The Only Number You Need to Know Does Not
Add Up to Much, Marketing Week, March 6 2008

43. One Question and Plenty of Debate,
Wall Street Journal, December 4, 2007

44. If Your Research
Challenges Something
Popular, Expect a
Backlash

45. Nobody Likes to Be Told the Emperor Has No Clothes,
Particularly Those Who Have Chosen the Emperor’s Tailor

46. No Regrets
Despite the backlash
(and career threats),
we would do it again…
in a heartbeat!

47. The Keiningham-Aksoy
Guiding Principles for
Writing & Publishing

48. The Keiningham-Aksoy Guiding
Principles for Writing & Publishing
1. You never know what will get accepted, so work on lots of
things simultaneously.
2. You cannot be an expert at everything, so
a) build great teams, and
b) do everything you can to make your teams successful.
§ Work with people you like, and who are committed to working as
a team.
3. Faster, faster, faster! A slow paper is an idea that will be
published first by someone else.
4. Get your paper published! Our personal addition is “even if
it is on a cereal box!”
§ Everything you write cannot be a Tier 1 paper, but if your
research was important enough for you to conduct and report,
then it needs to be in print.

49. Discussion & Questions
Timothy Keiningham