How to combine quantitative data with qualitative insights to improve UX

1. How to Combine Quantitative Data with
Qualitative Insights for CRO
Robert Mulder & Sven Cune

2. “The greatest enemy of knowledge is not ignorance,
it is the illusion of knowledge.”
Stephen Hawking

3. User
Feedback
shows
you
why.
Analytics
shows
you
what is
happening
User
Feedback
shows
you
why.

5. PERFECT FOR:
• Randomized site
experience surveys
• Content based surveys
• Shorter surveys with
fewer pages
TYPES OF FEEDBACK

6. PASSIVE FEEDBACK

8. ACTIVE FEEDBACK

9. PERFECT FOR:
• Randomized site
experience surveys
• Content based surveys
• Shorter surveys with
fewer pages
The humble feedback button

10. @usabilla
Avoid expensive assumptions
Key Objective:
Improve booking process to increase conversion
Results:
• Quickly uncovered what was really happening
• reduced drop-off rates by 10%
UK AIRLINE

11. @usabilla
See what users want and detect
issues early
Key Objective:
Prioritise changes based on user input
Results:
• Validated new design & reprioritised UX changes
• Uncovered a serious bug quickly to save over £50K
UK RETAILER

12. @usabilla
Improve your reputation
Key Objective:
Provide a continues channel for customer feedback
Results:
• Increased App store rating
HUMAN
“An App store review should
never be the easiest way to
give feedback”
Paul Veugen, founder @ Human

13. Targeted Surveys

14. FRICTION
DEFINITION
The psychological resistance that your
visitors experience when trying to
complete an action. Friction is a
conversion killer usually caused by
unclear messaging, lack of information

15. FRICTION
DEFINITION
The psychological resistance that your
visitors experience when trying to
complete an action. Friction is a
conversion killer usually caused by
unclear messaging, lack of information

16. FRICTION
DEFINITION
The psychological resistance that your
visitors experience when trying to
complete an action. Friction is a
conversion killer usually caused by
unclear messaging, lack of information

17. 17

18. 18

19. 19

20. 20

21. 21

23. While at Home
While at the Office
While commu0ng
While shopping (in store)
While being outside
other

24. VoC enhanced testing

25. @usabilla
Feedback
Segment feedback based on variations
Make UX informed changes
Surveys
A/B testing
Target surveys at variations

26. @usabilla
Fake door tests

27. Quick Guide to Effective Surveys
Simple steps to getting actionable insight from your users

28. WHERE ARE YOUR USERS GETTING STUCK?
FRICTION
DEFINITION
The psychological resistance that your
visitors experience when trying to
complete an action. Friction is a
conversion killer usually caused by
unclear messaging, lack of information

29. SURVEY PLAN CHECKLIST
DEFINE
• Why you ask — Define your objectives
• Who you ask — Target very specific segments
• What to ask — Decide on friction areas and survey questions
• When to ask — Decide on behaviours that trigger the survey
• Where to ask — Target certain urls and pages of the website

30. Qualitative
Surveys
KEEPING IT SIMPLE
• Keep it short - 3 questions per page maximum
• Keep it real - brand & humanize your language
• Keep it relevant - Conditional questions

31. 31

32. 32

33. WINNER!!
CONTROL VARIANT VARIANT B
–
No Introduction
15 seconds
–
With Introduction
15 seconds
WHICH VARIANT WON?

34. PHILIPS - SURVEY A/B TEST
6"
TEST$NUMBER$ B$VARIANT$
CHANGE$IN$
RESPONDENTS$
COUNTRY$
1" A%er"30"seconds"" 49.2%$ China"
6a" No"Intro,"15"seconds" +131%$ US"
6b" No"Intro,"15"seconds" +194%$ China"
6c" No"Intro,"15"seconds" +113%$ Brazil"
2" A%er"4"Page"Views" 449.7%$ Netherlands"
3" Upon"Exit" 450.2%$ US"
4" A%er"5"Page"Views" 439%$ Netherlands"
5" No"Intro,"15"seconds" +60%$ France"

35. Industry Conversion
Travel 7-15%
Electronics 1-2%
Telecom 3-24%
Healthcare 5-11%
Transport 1-3%
Travel 2-3%
Publishing 6-14%
SURVEY CONVERSION PER INDUSTRY

36. • Why did you visit our website today?
• Did you find what you were looking for?
• What was your goal today and were you able to achieve it?
• Do you have any questions that you could not find the answer to?
• What was the biggest challenge in finding what you were looking for?
• How can we improve our X page?
• What made you not complete your purchase today?
• What almost prevented you form purchasing today?
MAGIC QUESTIONS

37. Thank you!
Questions?
PS: Drop by the Usabilla stand to play our balls of fire game!