Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
#RelateLive
Lori Gauthier, Ph.D.
Zendesk
Director of Marketing Research
@datadocgauthier
#RelateLive
What Your Customers Really Think About You
Part 1: Review the Do’s and Don'ts of Survey Design
Let’s Start with the Don’ts!
Question source: The Effortless Experience
How much effort did you personally have to put forth to get your issue resolved...
What’s wrong with this question?
Measuring Customer Satisfaction
How much do you agree with the following statement?
I am ...
Measurement Error
The survey itself impact responses
specification error
random error
systematic error
largest
source of
e...
“I know you think you understand
what you thought I said but I'm
not sure you realize that what you
heard is not what I me...
Oops Data
Wait, I thought you meant…
Specification Error
Even well designed surveys can yield
bad data when the wrong cons...
What’s wrong with this question?
Measuring Customer Effort
Question source: The Effortless Experience
How much effort did ...
What’s wrong with this question?
Measuring Customer Satisfaction
How much do you agree with the following statement?
I am ...
Stewie Data
Look at him go!
Random Error
Bad survey design can introduce data-
destroying random error, making your
data —...
Question source: The Effortless Experience
How much effort did you personally have to put forth to get your issue resolved...
What’s wrong with this question?
Measuring Customer Satisfaction
How much do you agree with the following statement?
I am ...
Rooting Out Random Error
So long, Stewie!
no!
nooo!
noo!
double barreled question
unexpected scale direction
insensitive s...
Tower of Pisa Data
One way or another, it’s gonna getcha!
Systematic Error
Bad survey design can introduce
data-destroying...
Question source: The Effortless Experience
How much effort did you personally have to put forth to get your issue resolved...
What’s wrong with this question?
Measuring Customer Satisfaction
How much do you agree with the following statement?
I am ...
Banishing Bias
Arrivederci, Pisa!
worst
ever!!!
thingunbalanced scale
leading question
true|false, yes|no, agree|disagree ...
Done with the Don’ts. Let’s Review the Do’s!
Know What You Need from Your Data
answer construct question scale
Start with your destination.
What Are You Measuring?
Are You Sure?
Define What You Need to Measure
Words Mean Things
Search definitions, synonyms,
antonyms.
Use the language and tone
approp...
Source: snappywords.com
What Questions and Scales Should You Use?
Understanding Construct Polarity and Scale Sensitivity
Which Way Do We Go?
Construct polarity
Unipolar Construct Bipolar Construct
Very common; typically specific; often descrip...
Ideal scale sensitivity (example 1)
How Many Scale Points Should You Use?
unipolar
notatall
extrem
ely
m
oderately
slightl...
Ideal scale sensitivity (example 2)
How Many Scale Points Should You Use?
unipolar
notatall
a
greatdeal
a
m
oderate
am
oun...
How Many Scale Points Should You Use?
Sensitivity reduced as scale points removed
unipolar
not at all
likely
extremely
lik...
How Many Scale Points Should You Use?
Sensitivity reduced as scale points removed
bipolar
1000 5025 75-25-75-100 -50
neith...
How Many Scale Points Should You Use?
Sensitivity reduced as scale points removed
neither
like nor
dislike
like
a great
de...
How Many Scale Points Should You Use?
Sensitivity reduced as scale points removed
bipolar
1000 5025 75-25-75-100 -50
neith...
How Many Scale Points Should You Use?
Sensitivity reduced as scale points removed
bipolar
1000 5025 75-25-75-100 -50
neith...
That’s A Lot of Stuff to Remember. Let’s Recap.
Phew!
A step-by-step approach to designing sound surveys
What Have We Learned?
start at your destination
define your construct
c...
Let Me Know What YOU Think!
Share your thoughts about
Part 1 of today’s workshop.
Two minutes, a few taps in your
Relate L...
#RelateLive
#RelateLive
What Your Customers Really
Think About You
Part 2: Critique and Create Survey Questions
Critique One Question
Time to Design!
How satisfied are you withAcme’s customer support?
1 3 42
Customer Satisfaction
What’s wrong with these questions?
To what...
Rooting Out Random Error
So long, Stewie!
no!
nooo!
noo!
double barreled question
unexpected scale direction
insensitive s...
Banishing Bias
Arrivederci, Pisa!
worst
ever!!!
thingunbalanced scale
leading question
true|false, yes|no, agree|disagree ...
Create One Question
Time to Design!
What questions and scales will YOU use?
Customer Satisfaction + Customer Effort
define construct
determine polarity
design...
Which Way Do We Go?
Construct polarity
Unipolar Construct Bipolar Construct
Very common; typically specific; often descrip...
Part 1 - Review Critiques
Reconvene
What’s wrong with this question?
Measuring Customer Satisfaction
How satisfied are you withAcme’s customer support?
1 3 42...
To what extent do you agree or disagree with the following statement?
The company made it easy for me to handle my issue.
...
Part 2 - Review New Questions
Reconvene
A methodologically sound question
Measuring Customer Satisfaction
Overall, how satisfied or dissatisfied are you withAcme’...
Measuring Customer Effort
A methodologically sound question
How easy was it to get the help you needed from us today?
not ...
Measuring Drivers of Customer Effort
Design One, Get One!
Measuring Customer Effort
What is driving customer effort?
Content source for drivers of effort: The Effortless Experience...
Workshop Recap
What Your Customers Really Think About You
Sound design. Accurate data. Better relationships.
A Step-by-Step Approach to Survey Design
start at your destination
defi...
Thank You!
Questions? Contact me at lgauthier@zendesk.com or @datadocgauthier.
Let Me Know What YOU Think!
Your finger here!
Share your thoughts about
Parts 1 + 2 of today’s workshop.
Two minutes, a fe...
#RelateLive
What Your Customers Really Think About You (Relate Live London)
What Your Customers Really Think About You (Relate Live London)
What Your Customers Really Think About You (Relate Live London)
What Your Customers Really Think About You (Relate Live London)
What Your Customers Really Think About You (Relate Live London)
Prochain SlideShare
Chargement dans…5
×

What Your Customers Really Think About You (Relate Live London)

Lori Gauthier, Ph.D., Director of Marketing Research, Zendesk
In this two-session workshop, you’ll learn how to create survey questions that deliver insightful responses and inspire measurable actions. Each attendee will leave the workshop with two surveys that quickly and accurately measure customer satisfaction (CSAT) and customer effort (CES) -- two surveys that can be used by any organization whether they serve customers, employees, students, volunteers, vendors, or the general public. During the first session, Lori will review the key do's and don'ts of designing methodologically sound surveys. You'll learn how to accurately define and measure what you really want to measure while avoiding data-destroying random error and bias.

  • Soyez le premier à commenter

What Your Customers Really Think About You (Relate Live London)

  1. 1. #RelateLive
  2. 2. Lori Gauthier, Ph.D. Zendesk Director of Marketing Research @datadocgauthier
  3. 3. #RelateLive What Your Customers Really Think About You Part 1: Review the Do’s and Don'ts of Survey Design
  4. 4. Let’s Start with the Don’ts!
  5. 5. Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort missing correct midpointmissing “no effort” end point confusing scale incorrectly defined construct awkward question Measuring Customer Effort What’s wrong with this question?
  6. 6. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree construct not specified in scale unbalanced question non-modified response options agree/disagree scale question as a statement missing ambivalent midpoint incorrectly defined construct
  7. 7. Measurement Error The survey itself impact responses specification error random error systematic error largest source of error controlled by surveyor
  8. 8. “I know you think you understand what you thought I said but I'm not sure you realize that what you heard is not what I meant” - Unknown
  9. 9. Oops Data Wait, I thought you meant… Specification Error Even well designed surveys can yield bad data when the wrong constructs are measured or the right constructs aren’t measured completely.
  10. 10. What’s wrong with this question? Measuring Customer Effort Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort incorrectly defined construct (specification error)
  11. 11. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree incorrectly defined construct (specification error)
  12. 12. Stewie Data Look at him go! Random Error Bad survey design can introduce data- destroying random error, making your data — and decisions — bounce all over the place.
  13. 13. Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort awkward question (random error) confusing scale (random error) missing correct midpoint (random error) Measuring Customer Effort What’s wrong with this question?
  14. 14. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree construct not specified in scale (random error) non-modified response options (random error) agree/disagree scale (random and systematic error)
  15. 15. Rooting Out Random Error So long, Stewie! no! nooo! noo! double barreled question unexpected scale direction insensitive scale overly sensitive scale scale without midpoint scale without verbal labels overlapping scale labels non construct-specific scale confusing question or scale true|false, yes|no, agree|disagree scale
  16. 16. Tower of Pisa Data One way or another, it’s gonna getcha! Systematic Error Bad survey design can introduce data-destroying systematic error, leading you to make biased decisions.
  17. 17. Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort missing “no effort” end point (systematic error) Measuring Customer Effort What’s wrong with this question?
  18. 18. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree unbalanced question (systematic error) agree/disagree scale (random and systematic error) question as a statement (systematic error) missing ambivalent midpoint (systematic error)
  19. 19. Banishing Bias Arrivederci, Pisa! worst ever!!! thingunbalanced scale leading question true|false, yes|no, agree|disagree scale missing extreme endpoints bipolar scale without neither/nor midpoint order effects context effects unbalanced question question formatted as statement
  20. 20. Done with the Don’ts. Let’s Review the Do’s!
  21. 21. Know What You Need from Your Data answer construct question scale Start with your destination.
  22. 22. What Are You Measuring? Are You Sure?
  23. 23. Define What You Need to Measure Words Mean Things Search definitions, synonyms, antonyms. Use the language and tone appropriate for your population. Result: Respondents answer the question you think you’re asking.
  24. 24. Source: snappywords.com
  25. 25. What Questions and Scales Should You Use? Understanding Construct Polarity and Scale Sensitivity
  26. 26. Which Way Do We Go? Construct polarity Unipolar Construct Bipolar Construct Very common; typically specific; often descriptive Very rare; typically global; occasionally comparative Measures absence to maximum: not at all likely to extremely likely Measures maximum negative to maximum positive: disapprove a great deal to approve a great deal Midpoint represents half of construct Midpoint represents ambiguity or no opinion 5-point scale is ideal 7- or 9-point scale is ideal How likely are you to vote in a primary this year? Do you approve or disapprove of negative campaigning? Examples: likelihood, frequency, duration, intensity Examples: bad/good, dis/satisfied, dis/like, worse/better common labels: not at all, slightly, moderately, very, extremely none, a little, a moderate amount, a lot, a great deal common labels (mirrored sides): extremely, very, moderately, slightly, neither/nor … a great deal, a lot, a moderate amount, a little, neither/nor … zero ????
  27. 27. Ideal scale sensitivity (example 1) How Many Scale Points Should You Use? unipolar notatall extrem ely m oderately slightly very 1000 5025 75 bipolar neither/nor extrem ely m oderately slightly very 1000 5025 75 slightly very extrem ely m oderately -25-75-100 -50
  28. 28. Ideal scale sensitivity (example 2) How Many Scale Points Should You Use? unipolar notatall a greatdeal a m oderate am ount a little a lot 1000 5025 75 bipolar neither/nor a greatdeal a m oderate am ount a little a lot 1000 5025 75 a little a lot a greatdeal a m oderate am ount -25-75-100 -50
  29. 29. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed unipolar not at all likely extremely likely moderately likely slightly likely very likely 1000 5025 75 ???? not likely likely
  30. 30. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount
  31. 31. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed neither like nor dislike like a great deal like a moderate amount like a little dislike a little dislike a great deal dislike a moderate amount 1000 33 67-33-67-100 bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount
  32. 32. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount 1000 50-100 -50 neither like nor dislike like a great deal like a moderate amount dislike a great deal dislike a moderate amount
  33. 33. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount 1000-100 neither like nor dislike like a great deal dislike a great deal
  34. 34. That’s A Lot of Stuff to Remember. Let’s Recap. Phew!
  35. 35. A step-by-step approach to designing sound surveys What Have We Learned? start at your destination define your construct check for random error check for systematic error collect good data bing! bing! bing! draft question + scale determine polarity
  36. 36. Let Me Know What YOU Think! Share your thoughts about Part 1 of today’s workshop. Two minutes, a few taps in your Relate Live app, and I’ll know what you think. Thank you! Your finger here!
  37. 37. #RelateLive
  38. 38. #RelateLive What Your Customers Really Think About You Part 2: Critique and Create Survey Questions
  39. 39. Critique One Question Time to Design!
  40. 40. How satisfied are you withAcme’s customer support? 1 3 42 Customer Satisfaction What’s wrong with these questions? To what extent do you agree or disagree with the following statement? The company made it easy for me to handle my issue. Strongly disagree Strongly agree Neither agree nor disagree Disagree Agree Somewhat disagree Somewhat agree Customer Effort* *Question source: The Effortless Experience
  41. 41. Rooting Out Random Error So long, Stewie! no! nooo! noo! double barreled question unexpected scale direction insensitive scale overly sensitive scale scale without midpoint scale without verbal labels overlapping scale labels non construct-specific scale confusing question or scale true|false, yes|no, agree|disagree scale
  42. 42. Banishing Bias Arrivederci, Pisa! worst ever!!! thingunbalanced scale leading question true|false, yes|no, agree|disagree scale missing extreme endpoints bipolar scale without neither/nor midpoint order effects context effects unbalanced question question formatted as statement
  43. 43. Create One Question Time to Design!
  44. 44. What questions and scales will YOU use? Customer Satisfaction + Customer Effort define construct determine polarity design question design scale check for errors 1 2 3 4 5
  45. 45. Which Way Do We Go? Construct polarity Unipolar Construct Bipolar Construct Very common; typically specific; often descriptive Very rare; typically global; occasionally comparative Measures absence to maximum: not at all likely to extremely likely Measures maximum negative to maximum positive: disapprove a great deal to approve a great deal Midpoint represents half of construct Midpoint represents ambiguity or no opinion 5-point scale is ideal 7- or 9-point scale is ideal How likely are you to vote in a primary this year? Do you approve or disapprove of negative campaigning? Examples: likelihood, frequency, duration, intensity Examples: bad/good, dis/satisfied, dis/like, worse/better common labels: not at all, slightly, moderately, very, extremely none, a little, a moderate amount, a lot, a great deal common labels (mirrored sides): extremely, very, moderately, slightly, neither/nor … a great deal, a lot, a moderate amount, a little, neither/nor … zero ????
  46. 46. Part 1 - Review Critiques Reconvene
  47. 47. What’s wrong with this question? Measuring Customer Satisfaction How satisfied are you withAcme’s customer support? 1 3 42 unbalanced question (systematic error) missing construct- specific verbal labels (random error)missing negative half of scale (systematic error) missing midpoint on positive half of scale (random error) missing zero scale point (systematic error)
  48. 48. To what extent do you agree or disagree with the following statement? The company made it easy for me to handle my issue. Strongly disagree Strongly agree Neither agree nor disagree Disagree Agree Somewhat disagree Somewhat agree Question source: The Effortless Experience construct not specified in scale (random error) non-modified response options (random error) agree/disagree scale (random and systematic error) question as a statement (systematic error) What’s wrong with this question? Measuring Customer Effort
  49. 49. Part 2 - Review New Questions Reconvene
  50. 50. A methodologically sound question Measuring Customer Satisfaction Overall, how satisfied or dissatisfied are you withAcme’s customer support? moderately dissatisfied slightly dissatisfied neither satisfied nor dissatisfied slightly satisfied moderately satisfied extremely dissatisfied extremely satisfied 7-point, fully labeled, construct-specific, bipolar scale measures what we want to measure: satisfaction with customer support “overall” appropriate for global-level measure balanced question ambivalent midpoint
  51. 51. Measuring Customer Effort A methodologically sound question How easy was it to get the help you needed from us today? not at all easy extremely easy moderately easy very easy slightly easy measures what we want to measure: effort needed to get company’s help “today” appropriate for transaction-level measure 5-point, fully labeled, construct-specific, unipolar scale
  52. 52. Measuring Drivers of Customer Effort Design One, Get One!
  53. 53. Measuring Customer Effort What is driving customer effort? Content source for drivers of effort: The Effortless Experience How did we make it difficult? (Check all that apply) You didn’t solve the problem I had to contact the company multiple times I felt like I was talking to a robot I had to repeat myself I had to use a channel I don’t like (phone, web form, chat, email, FAQ) I was transferred from person to person Some other reason (Please specify) don’t assume resolution pick list Q measures frequency of known drivers open-ended option captures unknown drivers limit list to 7-9 options random rotate pick list
  54. 54. Workshop Recap What Your Customers Really Think About You
  55. 55. Sound design. Accurate data. Better relationships. A Step-by-Step Approach to Survey Design start at your destination define your construct draft question + scale check for random error check for systematic error collect accurate data bing! bing! bing! determine polarity
  56. 56. Thank You! Questions? Contact me at lgauthier@zendesk.com or @datadocgauthier.
  57. 57. Let Me Know What YOU Think! Your finger here! Share your thoughts about Parts 1 + 2 of today’s workshop. Two minutes, a few taps in your Relate Live app, and I’ll know what you think. Thank you!
  58. 58. #RelateLive

×