A key tool in the design process for UX professionals is concept testing. Traditionally, concepts for new products or design approaches are tested qualitatively by showing potential users visual sketches or wireframes during an interview session. While this approach can generate rich information about individual reactions to the concept, it is difficult to extrapolate to understand reactions from a larger potential user base. This poster will outline the methods we have used to enhance concept evaluation by including quantitative user reaction surveys that combine UX and market research techniques, as well as describe how this methodology can motivate further concept development.

1. Quantify
your
concept:
Combining market research and UX
survey methods to evaluate new ideas
How do you test out a new idea?
Ryan
Kasper
Idea Talk to People
Qualitative
Survey the Market
Quantitative
Part 1
Show Concept
Part 2
Ask UX Questions
User tests of new ideas can provide rich qualitative information about how the idea can impact
users, but it takes additional research to understand how strong that impact could be in the larger
market. To gauge the impact on a larger scale, you can use a concept evaluation survey, which
allows you to show and describe concepts while gathering quantitative data.
Concept Evaluation Survey
Part 3
Market Research
Reaction ratings
Multiple choice
Open-ended
A/B testing
System Usability Scale
Net Promoter Score
Jobs-To-Be-Done
Likehood to purchase
Pricing sensitivity
Demographics
@Dr_RyanKasper
ryan.kasper@citrix.com

2. Survey Example
Part 1.
Show Concept
Part 2.
UX Questions
Part 3.
Market Research
“This is your phone. You open the app
and arrive on this screen...”
Reaction Ratings
1 2 3 4 5 6 7
Would you like to use this feature?
Multiple Choice How would you share with a friend?
Other
Multiple Choice
I think I would........... ...... .......
“How would you use this saving option?”
1 2 3 4 5
a) I would like to use this tool frequently
...
k) I needed to learn a lot before using the tool
System Usability Scale
Would you recommend this to a friend?Net Promoter Score
1 2 3 4 5 6 7 8 9 10
How likely would you be to purchase this tool?Likehood to Purchase
1 2 3 4 5 6 7
Please enter the price point at which this tool:
a) is too expensive:
b) is so inexpensive you question the quality:
c) is getting expensive, but you still consider it:
d) is a bargain:
Pricing Sensitivity
Minimize my effort to find a photograph
a) How important is this need to you?
b) How well is this need currently satisified?
Jobs-To-Be-Done
1 2 3 4 5 6 7 8 9 10
Please select your industry
Healthcare Law Other
Demographics

3. Survey Method Details
A/B Testing
Show a group of people one version, and the
other group a different version. Then, compare
their answers to certain questions.
Version A Q1 avg = 3.7 Version B Q1 avg = 5.9
Part 1. Show Concept
This part of the survey focuses on showing and describing
the concept. You can use the same images as in 1:1 sessions.
One interesting approach to comparing alternative concept
versions is to do an A/B test.
Part 2. UX Questions System Usability Scale (SUS)¹
If you want to gauge how usable your concept seems,
consider using the SUS. The SUS is a 10-question
tool that gives you a perceived usability measure.
SUS Analysis
SUS
Score
100
0
Industry
Average
Concept
Ask UX questions along the way to understand how
people react to the concept. Some standard survey
questions include reaction ratings, multiple choice,
and open-ended questions.
References
1. Brooke, J. (1996) Usability Evaluation in Industry.
2. Reichheld, F. (2003) Harvard Business Review
3. Van Westendorp, P. (1976) Proceedings of the ESOMAR Congress.
Pricing Sensitivity3
To determine the potential monetary value of the
concept, use the Van Westendorp Pricing Sensitivity
Model, which uses 4 questions to determine
acceptable price points.
a) too expensive
b) so inexpensive you would question the quality
c) getting expensive, but would still consider it
d) a bargain
$$ Amount
%Responses
Highest acceptable
Price Points:
Lowest acceptable
Part 3. Market Research
Jobs-To-Be-Done2
This method can help identify the market opportunity for a
customer need, as well as the potential value of the concept.
Current Satisfaction
How well is this need
currently satisfied?
Importance
How important is
this need to you?
High
Opportunity
Gauge market opportunity
Satisfaction w/ Tool
How well would this tool
satisfy this need?
Importance
How important is
this need to you?
High
Value
Gauge value of concept
To understand more about the potential market for the
concept, include market research questions. For
example, you may want to know the demographics of
those that respond well to the concept, or whether
people would purchase the tool. Below are two
methods that can give great market insights.
Ryan
Kasper
ryan.kasper@citrix.com
@Dr_RyanKasper