This talk is an interactive workshop with live demonstrations and simulations. Participants will practice their interview skills as well as on-line survey design skills and learn from each other in the workshop. Novices and veterans alike will gain new skills in a group setting.
2. What, How, Who
• Define the WHAT:
– Clearly articulate hypotheses to be tested – these form
the core of your goals and objectives
• Define the HOW:
– Pick a methodology
– Develop content for the methodology (e.g. discussion
guide; survey design; etc)
– Develop a project plan (including who-does-what-when)
• Define the WHO:
– Define the profile of the subject
– Develop a subject recruitment questionnaire
6. Business Model Hypotheses
6
Emails sent
Click through
Signed up for trial
Began using product
Converted to paid
subscriber
Free Beta
500
40%
80%
100%
N/A
30 day trial to paid subscription
5000
14%
1 subscriber
7. Example: Zeo Mobile
• Product: The Zeo Sleep Manager (Direct to
Consumer)
• Setting: 1st product is in market; contemplating
strategy for second product
• What hypotheses do you think we should test here?
8. Example: Baxter Robot
• Product: Collaborative manufacturing robot (B2B)
• Setting: Product architecture is defined; user persona
hypothesis validated; working on decision making unit
• What hypotheses do you think we should test here?
10. Common methodologies
• Contextural interview
• Observation / shadowing
• Immersion
• Longitudinal diary study
• Photo essay
• Usability benchmark
• Focus groups
• … etc
• Qualitative (<30 samples) • Quantitative (>1000 samples)
• Surveys
– General interest
– Conjoint analysis
– Pricing studies
• Monadic
• Multiple monadic
• Van Westendorp
• … etc
– Customer satisfaction: NPS, P/M
fit
• Web testing
– A/B split, Multivariate
– Web analytics
– … etc
11. Qualitative versus quantitative
• Open ended: you are having a
conversation
• Anecdotal: you derive meaning
from each conversation
• Requires excellent listening and
observation skills
• Best done face to face (or at
least by video Skype or by
voice call)
• Best used for exploratory
research where you are trying
to get to know a problem or a
persona
• Qualitative (<50 samples) • Quantitative (>1000 samples)
• Close ended: you are
administering a questionnaire
• Statistical: you derive meaning
from looking at aggregate results
• Requires systematic analysis skills
• Best done via on-line survey (or,
in the olden days, via phone
survey)
• Best used for confirmation
research where you are trying to
verify / quantify what you think is
important
12. Qualitative, then quantitative
The qualitative pass helps you develop your quantitative pass and interpret the result with
context and meaning
14. Example recruitment questionnaire
• What is your age, gender and profession? (terminate once age/gender
quota has been filled for the matching segment)
• What is your household income? (terminate if <$xxk)
• Are you interested in learning more about your sleep? (terminate if not)
• Are you currently encountering sleep problems? (terminate if not)
• Are you currently under the care of a doctor for your sleep issues?
(terminate if true)
• How many nights in an average week are you encountering sleep issues?
(select between 2-5)
• Please describe the sleep issues you are currently encountering (check all
that apply). (cannot fall asleep; multiple awakenings; cannot go back to
sleep after awakening; snoring spouse; etc)
• Do you currently share your bed? (partner, child, pet) If so: how many
nights in an average week is your sleep affected by your bed partner(s)?
23. Technique cheat sheet
• Clearly state hypotheses to test
• Establish rapport before you begin
• Ask short, open ended questions. “Tell me the story
of…”, “say more…”, “Why?” “Why not?”
• Let the subject lead the conversation
• Use active listening techniques: “I think I heard you
say xxx. Is that right?”
• Talk very little. Success = they talk 95% of the time,
you talk 5% of the time (most of which is during set up
and wrap up)
23
28. Technique cheat sheet
• Clearly define goals and objectives
• Ask effective questions to meet research goals
• Decide on length of survey (how many minutes?)
• Use skip logic judiciously to shorten completion time
• Be smart about answer formats
– Closed ended: radio buttons, checkboxes,
– Open ended: text boxes
• Carefully consider which q’s should be required
• Demographics and contact info at the end, please
• Carrots!
28