Chuck Liu Design Research Lead KISSmetrics @chuckjliu cliu@kissmetrics.com Market research helps you make decisions. 3 Essential Mantras of Market Research 1 Goal: Make better decisions, faster. Get things done in days, not weeks or months. Market research priorities are different depending on what stage your business is at There are many FREE resources out there. Google Trends: Measure market potential and interest AVINASH KAUSHIK HTTP://KISS.LY/LEANAC Talking to experts: Get the detailed scoop of workflows and processes 1. Hypothesis- driven Have an idea to prove or disprove 2. Short and targeted 5 days, 2 weeks max 3 Ways for Early Stage Businesses to do Lean Market Research 2 1. Survey + Social Distribution Cheap (or free), but requires more work on your part 1. Make a screener or survey 2. Tweet/share it out 3. Analyze Try the good ole’ “asking for a friend” (except it’s you really asking) Whether you’re actually asking for a friend or not, this actually works be er, especially if you tag a potential competitor 1. Ask a question on Quora 2. Revitalize an old relevant thread with a new comment 3. Ask people to answer an existing question 2. AdWords Easy setup, variable expenses You Pay for Clicks, Which Is Pre y Realistic AdWords Keyword Planner does the work for you in volume and interest 1. Practice your pitch 2. Limited character count = concise messaging 3. Bad ideas = no problem 3. Amazon Mechanical Turk Disclaimer: I haven’t tried yet, but I want to Mechanical Turk Plan: Simple • Design a test • Distribute a test • Analyze the data 3 Strategies for Existing Businesses/ Enterprises to Getting Faster Research Done 3 1. In-App Surveys Contextual, relevant, and dismissible Existing workflow and pain points • Nudge your customers with in-app surveys • Open-ended In-App Survey Pros and Cons Pros • Low cost • Low effort • Can be turned on/off as you please to measure activities over time • Quick responses based on targeting technique Cons • Limits demographic to your existing users • Can potentially annoy your users 2. Experience Sampling Uncover user needs and behaviors Pros • Highlights behaviors, moods, stress levels • Gives context to these behaviors depending on how it was administered (same time every day, multiple times a day, etc.) • Measures differences over time Cons • Risk of participants dropping off or stopping participation • Incentive needed to lure in • Not good for checking if someone is doing a task repetitively Experience Sampling Pros and Cons 3. Persona Advisory Board Quick Review: Personas (thanks to Buffer for these images! Pros • Highly contextual information about day in the life, workflow, and process • Visibility into which tools are used for tasks • Deeper relationship and trust built with customer Cons • High amount of effort on your part • Recruiting can be hit or miss depending on your relationship with customers / th

1. Chuck Liu
How to Do Be er Market
Research
January 22nd, 2015

2. Chuck Liu
Design Research Lead
KISSmetrics
@chuckjliu
cliu@kissmetrics.com
Thue L Madsen
Marketing Operations Specialist
KISSmetrics
@thuelmadsen
tmadsen@kissmetrics.com

3. @thuelmadsen #KISSwebinar
Join the conversation on Twi er

6. Market research helps you
make decisions.

7. 1 3 Essential Mantras of Market Research
Email
In-App/Site Opt-In
In-App/Site Pop-Up
2
3 Ways for Early Stage Businesses to Do Lean Market
Research
3
3 Strategies for Existing Businesses/Enterprises to
Ge ing Faster Research Done
Table of Contents
4 Bonus: The KISSmetrics Market Research Flow

8. WATCH WEBINAR RECORDING NOW

9. @chuckjliu #KISSwebinar
Join the conversation on Twi er

10. 3 Essential Mantras
of Market Research
1

12. Goal:
Make be er
decisions, faster.

14. Get things done
in days, not
weeks or
months.

16. Market research
priorities are
different depending
on what stage your
business is at

17. There are many
FREE resources out
there.

18. Google Trends: Measure market potential and interest
AVINASH KAUSHIK HTTP://KISS.LY/LEANAC

19. Twi er, LinkedIn, Quora: Get the lay of the land on problems

20. Talking to experts: Get the detailed scoop of workflows and processes

21. LOG IN WITH GOOGLE
Start Your Free KISSmetrics Trial

22. 1. Hypothesis-
driven
Have an idea to prove or
disprove

23. 2. Short and
targeted
5 days, 2 weeks max

24. 3. Focus on
outcomes
Know what you want to know

25. “A er this research is
completed, we will be able to
decide on…”
https://www.flickr.com/photos/mikebehnken/

26. “A er this research is
completed, we will be able to
move forward with…”
https://www.flickr.com/photos/angeloangelo

27. “A er this research is
completed, we know whether
we need to do …. or do ….”
https://www.flickr.com/photos/mikebehnken/

28. 3 Ways for Early
Stage Businesses to
do Lean Market
Research
2

29. What problems, needs, and motivations do
people have?
Do people understand your product’s value
proposition?
Which messages are most effective at explaining
your product?
Can people figure out how to use your product?
Why do people stop using your product?

34. What do people
need?
Measuring how people behave

35. What do people
want?
Measuring what they say

36. Can people use
it?
Measuring tasks

37. 1. Survey + Social
Distribution
Cheap (or free), but requires
more work on your part

39. The super lazy version: ask #lazyweb on Twi er

40. 1. Make a screener
or survey
2. Tweet/share it out
3. Analyze

41. Tweet a screener or survey out

42. Tweet a screener or survey out

43. Try the good ole’
“asking for a friend”
(except it’s you really
asking)

44. Whether you’re actually asking for a friend or not, this actually
works be er, especially if you tag a potential competitor

45. 1. Ask a question on
Quora
2. Revitalize an old
relevant thread with a
new comment
3. Ask people to answer
an existing question

46. Pros
• Li le setup other than survey itself
• Piggyback off existing product’s customers and users
• Network with strangers you would have not recruited otherwise
• Advanced targeting to help you limit your audience
Cons
• Targeting limited to own follower count or others’ followers
• Meaningful conversations need to be taken offline
Social Distribution Pros and Cons

47. 2. AdWords
Easy setup, variable expenses

49. You Pay for Clicks, Which Is Pre y Realistic

50. AdWords Keyword Planner does the work for you in
volume and interest

51. 1. Practice your pitch
2. Limited character
count = concise
messaging
3. Bad ideas = no
problem

52. Additional AdWords Gotchas
• Your keywords determine who sees what (mind your demographics)
• Use targeting area to limit your ads be er to industries you’re focused on
• Potentially slow depending on your keyword/bidding level
• If your keyword is near $1, DON’T DO IT! Sweet spot: $0.25-0.75
• Set a daily maximum spend so your costs don’t go out of control

53. Pros
• Pay for clicks
• Keyword tool does market sizing and competition
• Advanced targeting for segmenting demographics
Cons
• Keywords that you want could be expensive
• Keywords you think you want are non-existent and no one searches for
them
• Can take long if your ads themselves are poorly wri en
AdWords Pros and Cons

54. 3. Amazon
Mechanical Turk
Disclaimer: I haven’t tried yet,
but I want to

56. Mechanical Turk Plan: Simple
• Design a test
• Distribute a test
• Analyze the data

57. Mechanical Turk Plan: Expanded
• Design a test
• Two designs
• Two messaging concepts
• Two branding concepts
• Distribute a test
• Write simple instructions: “Imagine you are…”
• Build a survey (within MTurk itself or TypeForm/Google Forms)
• Analyze the data

58. Mechanical Turk Survey Link Example

59. Marketing Skills Time! Write a great title and description

60. Set costs, number of participants, time to complete, how
long you want the task to be posted for

61. Mechanical Turk Caveats
• Professional survey takers
• You can limit your demographic somewhat, but some people live off
Mechanical Turk as income ($0.05 per 30-second HIT is a $6.00 hourly wage)
• Spam
• Surveys should have codes at the end to verify that they were completed by
an actual person, not a robot
• Surveys should take 2-4 minutes for tasks that pay less than $0.25. Anything
longer will get skipped over and your responses will come in slower.
• Interested in a longer read? See Rapid User Testing with Mechanical Turk

62. Mechanical Turk Pros and Cons
Pros
• Pay for only good answers
• Flexible price points based on your budget
• Potentially cheaper than paying survey tools for their “audience”
Cons
• Longer setup
• Potential spam / low quality answers to filter through

63. 3 Strategies for
Existing
Businesses/
Enterprises to
Ge ing Faster
Research Done
3

65. Can new customers understand and figure out how to use the
product?
What are customers’ existing workflow/process and pain points?
What are pros/cons of competitive products?
How satisfied are existing customers with the product?
How does customers’ usage change over time?

66. 1. In-App Surveys
Contextual, relevant, and
dismissible

68. Existing workflow and pain points
• Nudge your customers with in-app surveys
• Open-ended “What best describes why you logged in to use this app/service
today?”
• Open-ended “What frustrated you recently about this app/service?”
• Target customers at least a er 60 seconds have elapsed, otherwise it’s annoying
• Leave feedback forms accessible from any part of the app

69. Example question and choices for finding value and use
cases

70. Targeting - don’t make people annoyed at you

71. In-App Survey Pros and Cons
Pros
• Low cost
• Low effort
• Can be turned on/off as you please to measure activities over time
• Quick responses based on targeting technique
Cons
• Limits demographic to your existing users
• Can potentially annoy your users

72. 2. Experience
Sampling
Uncover user needs and behaviors

74. A psychology technique, now also used for market research
• Measure ongoing experiences and events by asking people the same question(s)
one or more times per day
• On demand via notification (in-app, or email)
• When certain conditions are met (used a feature at least 5 times in one day)
• At specific periods of time
• Recruitment requires incentive because of the commitment
• I like to do “a chance to win” incentives
• Make it large, so the chance is worth it

75. Jawbone Experience Sampling Recruitment Email

76. Jawbone Studied My Sleep Pa erns

77. Questions To Ask For Your Own Experience Sampling
• Open-ended: What was the reason you recently did X?
• Likert-scale: How did you feel when you used this app today?
• Open-ended: What is the reason you are using this app today?
• Likert-scale: How much do you agree or disagree with the following statement?

78. On Likert Scales (pronounced like “lick”)
• 5 -point scale with symmetrical answers
• Not at all interested, not that interested, neutral, somewhat interested, very
interested
• Not at all, not really, undecided, somewhat, very much
• Never, rarely, once in a while, sometimes, always
• Strongly disagree, disagree, neutral, agree, strongly agree

79. On Likert Scales (pronounced like “lick”)

80. Statements you could use with Likert scales as answers
• YOUR APP is easy to use.
• YOUR APP is easy to setup.
• YOUR APP has a pleasing color scheme.
• YOUR APP has useful FEATURE.
• YOUR BUSINESS looks trustworthy.
• YOUR BUSINESS looks professional.
• YOUR BUSINESS has a clear message on what it does.
• Does this design look too bright or too plain?
• Does this page look too busy or too minimal?
• Does this screen look too gimmicky or practical?

81. Pros
• Highlights behaviors, moods, stress levels
• Gives context to these behaviors depending on how it was administered
(same time every day, multiple times a day, etc.)
• Measures differences over time
Cons
• Risk of participants dropping off or stopping participation
• Incentive needed to lure in
• Not good for checking if someone is doing a task repetitively
Experience Sampling Pros and Cons

82. 3. Persona Advisory
Board

84. Quick Review: Personas (thanks to Buffer for these
images!

85. Quick Review: Personas (thanks to Buffer for these
images!

86. Recruiting Script
I wanted to extend an invitation to you to be part of theYOUR_COMPANY Customer Advisory
Board. I'd like to have your on the advisory board in order to shape and influence the types of
tools we build in order to help you do your job better.
The engagement on your part is pretty low. We'd love to hang out with you initially on-site to
get a sense of your workflow and process. After that, we ask for just 15-30 minutes of your time,
twice a month.
We'll also hold optional get-togethers among advisory members to share strategies and best
practices.
Let me know what you think. I think you'd be a great fit with your experience.
Cheers,
YOUR NAME

87. Guaranteed market research sessions every 2 weeks
• Recruit at least 2 people from each persona you have developed to cover the
spectrum of customers you have
• 15-30 min phone call, screenshare, or on-site visit every 2 weeks
• Ask them about the tools they use now or are considering using
• Are there any opportunities or threats?
• Ask them about any annoying but essential tasks they do right now
• Are there any opportunities for you to explore that problem?

88. Pros
• Highly contextual information about day in the life, workflow, and process
• Visibility into which tools are used for tasks
• Deeper relationship and trust built with customer
Cons
• High amount of effort on your part
• Recruiting can be hit or miss depending on your relationship with
customers / their availability
Advisory Board Pros and Cons

89. The KISSmetrics
Market Research
Process
4

91. Our Market Research Process
This works for us, but may not work for you.
Hypothesis
A guiding point for what your market
research is about
Informs questions you ask in screeners,
surveys, interviews, or onsite visits
Screener
In-app pop up survey tool with
targeting capabilities
Segmenting and targeting specific
customer types.
Survey / 1:1 Remote
Interview / Onsite
visit
Methods for finding out broad /
specific / contextual information
Manage opt-in lists and screeners for
demographic data
Report or Write Up
Summary of your findings, the
methodology you used, demographic
information, results of the research, and
recommendations
Informing your team or management to
make a decision
What It Is How It’s Used
P.S. Please don’t do focus groups. Ever. They are a waste of time.

92. Research Tools of the Trade
Our toolkit for ge ing survey’s done.
Typeform
Beautiful survey forms that display well
on web and mobile.
Surveys for market and user research
Qualaroo
In-app pop up survey tool with
targeting capabilities
Nudge or opt-in for a larger survey or
getting permission to follow up for a
another research study
Google Forms
Simple screeners and opt-in forms for
collecting people’s information.
Manage opt-in lists and screeners for
demographic data
Salesforce Pardot /
Gmail + Boomerang
Marketing automation and email
marketing
Schedule and create emails for
distributing survey links. Manage opt-in
lists and research campaigns.
Calendly Calendar scheduling app
People sign themselves up on an open
slot that works for them. No email back
and forths!
What It Is How It’s Used

93. Book Resource: Just Enough Research

94. Questions?
Chuck Liu
Design Research Lead
KISSmetrics
@chuckjliu
cliu@kissmetrics.com
Thue L Madsen
Marketing Operations Specialist
KISSmetrics
@thuelmadsen
tmadsen@kissmetrics.com

95. THANK YOU
Chuck Liu
@chuckjliu