Mobile versus online: modality considerations for data quality Elaine B. Coleman - Co-Founder & Chief Research Officer - Resolve Market Research Dean Wiltse - Founder & President – Thumbspeak Examining how technology-driven data applications have become viable options for ascertaining consumer feedback. Determining the implications for mobile applications for delivering data quality to brands. How to motivate consumers to respond quickly to branded information through channel novelty.

1. Towards an
Understanding of
Smartphones as a
Survey Platform
Elaine B. Coleman, Ph. D, Resolve Market Research & Dean Wiltse, Thumbspeak

2. Lead Biographies
Elaine Is Chief Research Officer and Co-Founder of Resolve Market Research, a
leading global research consultancy with unique insight into how technologies and
digital media transform consumers and audiences alike. Elaine holds a Ph.D. in
Cognitive Science from the University of Toronto and was awarded a McDonnell
Post-Doctoral Fellowship which was held at UC Berkeley. Coleman has been an
executive of consumer research and product design for over 15 years. Her expertise
in designing actionable research has attracted clients from wireless, games,
technology, sports marketing and home entertainment.
Dean Wiltse is a market visionary who has pioneered multiple game-changing
products and services including Mobile Relationship Marketing and Online
Communities. Dean is the founder and President of Thumbspeak LLC and was the
former CEO of Greenfield Online. Thumbspeak LLC focuses exclusively on leveraging
smart phone technology to create a disruptive approach to gathering data for
customer feedback and marketing research while encouraging advocacy and building
loyalty. After being hired as CEO of Greenfield Online in 2001, he immediately
recognized an opportunity to transform the business model. Under Dean’s
leadership, Greenfield completed a successful IPO in 2004, a secondary offering later
that same year and the purchase of Ciao in 2005, which led to the eventual sale of
the company to Microsoft in 2008. In 2005, he was nominated for Ernst & Young’s
‘Entrepreneur of the Year’ award and Research Magazine’s ‘Marketing Industry CEO
of the Year’ honor. Dean is the author of – ‘Weapons of Mass Collaboration: How to
Use Online Communities to Drive Competitive Advantage’.

3. How does mobile
compare?

4. Sample Sample
N=1480 N=1000
Panel 1 Panel 2
Online n=366 iPhone Survey Topic Mobile n=500 iPhone
and Tablet Owners and Tablet Owners
Tablets

5. The study was executed via an online and mobile survey from
(April 13 – April 28, 2011) among a national US sample of 866
iPhone and Tablet Owners
Group Base Description
Those who completed the survey from an online,
Online platform N=366
computer platform
Those who completed the survey using their mobile
Mobile platform N=500
device via the Thumbspeak Application
Total Sample N=866 US Tablet owners and users
Controlled for: Identical questions and stubs, fieldwork period, gender
breakout, median age and age groups, business and personal tablet usage,
base sizes with the mobile sample were randomly selected.
The margin of error for the full sample is +/- 4%, higher for subgroups.

6. Comparing the Experiences
Mobile Online

7. Mode Comparisons…

8. Are you downloading more or fewer apps on
your smartphone now that you are using your
tablet device?
Online Platform Mobile Platform
3%
6%
17% More 19%
Fewer
46% No change
34% 48% 27%
Not sure
N=366 N=500
Q9: Are you downloading more or fewer apps on your smartphone now that you are using your tablet device?
BASE: N=866

9. For the next tablet device you purchase, what
size tablet would you most prefer?
The largest size possible (i.e., 16 inches or 5%
more) 7%
Mobile
9% Platform
The same size as my laptop (i.e., 15 inches)
7% 348
Online
Small enough to fit in my purse or briefcase 58% Platform
(i.e., 10 inches) 67% 500
Small enough to fit in my back pocket (i.e., 7 13%
inches) 10%
15%
Not sure
9%
Q5: For the next tablet device you purchase, what size tablet would you most prefer?
BASE: N=866

10. Content is now accessible across multiple devices
(smartphone, tablet, PC, laptop, TV set, etc.). Which
devices do you use to watch your favorite TV and movie
programming?
80%
70%
60%
50%
40% Online
30% Platform
20% 348
10%
Mobile
0%
Platform
500
Q16: Content is now accessible across multiple devices (smartphone, tablet, PC, laptop, TV set, etc.).
Which devices do you use to watch your favorite TV and movie programming? Base: N=848

11. Do you pay attention to Have you noticed any
advertisements while using advertising while you are
your tablet device? using applications?
73%
68% 67%
55%
45%
32% 33%
27%
Yes No Yes No
Mobile Platform 366
Online Platform 500
Q14: Do you pay attention to advertisements while using your tablet device? Base: N=866

12. Where do you use your tablet device at work?
70%
60%
50%
40%
30%
20%
10%
0%
At my desk In meetings During Business On Other I don't use
presentations lunches conference my tablet at
calls the
workplace
Online Platform Mobile Platform
348 500
Q7: Where do you use your tablet device at work?
Base: N=866

13. Research Operations
Comparisons

14. Qualified Completes
1000
100
200
300
400
500
600
800
900
0
700
4/13/2011
4/14/2011
4/15/2011
Full
4/16/2011
4/17/2011 Point
4/18/2011
Mobile
4/19/2011
4/20/2011
Date
Online
4/21/2011
4/22/2011
4/23/2011
Completion Rates
4/24/2011
4/25/2011
4/26/2011
4/27/2011
Full
Point

15. Response Rates
Mobile Online
50%
45% 43%
40%
35%
30%
24%
25%
20%
15%
10%
5%
0%
Response Rate

16. How can we make the mobile
survey platform experience
actionable for our clients?

17. Mobile Surveys or Polls
But who wants to answer a 40
Polls work Great! question survey on their phone?
How can we extract valid and reliable samples
from mobile?

18. Innovate on the Sampling
Structures
That require “Big Data” Managing
and analyzing large volumes of data in real-time

19. Close Approximate Match
This is John…. This is Jon…. This is Johnny….
Age: 21
Ethnicity: Caucasian
Environment: Lives at home
Occupation: University Student on a scholarship
Online Interests: Avid Pandora User
Activity: Sends 50 texts per day Commonalities across all of our Johns
Hobbies: Plays video games 2 hours per day
Entertainment: Cord Cutter who watches HULU
plus on a laptop at least 8+ hours per week

20. I have a survey that is 40 questions
John 1
long that we plan to distribute via
mobile.
John 2 -Next Approximate Match
John 3 – Next Approximate
Match
Answers the first module of
10 questions then stops.
Answers the second 10
question module then
Matching Variables stops.
City, State Media Consumption
And so forth until the John
Group has completed 4
Age, Gender, Ethnicity Marital Status
modules of 10 questions
Online Behaviors Occupation

21. Expect all Johns to answer in the
same approximate way
Aggregated as a cohort
group –with little
variance.
Each individual’s
responses are
aggregated with
others in their cohort
and represent a
more robust set of
opinions of one type
respondent.

22. Aggregate Responses
John
This enables
clients to
distribute a 25
minute mobile
Kari
questionnaire
with the
contextual
benefits of a
mobile
Victor experience.

23. Historically, with every new
technology or research method
comes an “effect” that needs to
be understood separately from
its novelty

24. Historical Effects
Face-to-Face:
The “Hawthorne Effect” “Demand or Pleasing Effect”
“Respondents take on the goal of pleasing the experimenter.”

25. Historical Effects
Online:
The “Anonymity Effect” “The Power of Invisibility”
“There is a tendency to be more disparaging, inflammatory or honest.”

26. Historical Effects
Mobile:
The “________ Effect”
…

27. 4 Calls to Action
• Conduct more studies to “cognitively unpack” the Mobile
Effect
• Develop new types of sampling methodologies make mobile
actionable for business and product
• Develop elegant, easy mobile UI’s for end user experiences
• Think about “Big Data” and develop real-time data
processing applications for research

28. Questions
Elaine.Coleman@Resolvemr.com
Dean@Thumbspeak.com

29. Presented at:
Market Research in the
Mobile World
2nd International Conference | July 19 & 20, 2011 Atlanta
Organized by: Thank you to sponsors:
LinkedIn Group: Mobile MR
Upcoming Merlien Events: Merlien.org GreenBook Directory: GreenBook.org
Market Research Blog: GreenBookBlog.org New Qual Blog & Directory: NewQualitative.org
Get Inspired. Stay Informed. Cut Through the Clutter. Sign up for our free email newsletter