1. May 27-30, 2014
Congress Plaza Hotel & Convention Center
#MRMW
Organized by

2. Title Sponsor Gold Sponsors
Silver Sponsors
MRMW Chicago May 27-30, 2014 #MRMW

3. Workshop Sponsors
Association & Education Partners
Networking Reception
Sponsor
Event App
Partner
Media Partners
MRMW Chicago May 27-30, 2014 #MRMW

4. Outline
Introduction
Results
Summary
A US View of Smartphone Research
with Global Benchmarks
Brian Fine 1 Con Menictas 2
1Chairman
QOR, Quality Online Research
2Senior Statistician
Qantas Loyalty, Qantas Airways Ltd.
May 28, 2014
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

5. Outline
Introduction
Results
Summary
1 Introduction
Initial motivation
Why the case study?
What we tested
Visuals
Sample
2 Results
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
3 Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

6. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
What inspired us
We looked at myths about mobile research: too many not to
care about.
This compelled us to carry out a case study.
Previously, we examined Asia i.e., Japan, Korea, China.
Today we examine the US, Australia, UK.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

7. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
We put a number of myths to the test
Myth 1
Long surveys can’t be
done on mobiles!
Myth 2
PC results differ to
Mobile.
Myth 3
Mobile data not
representative or valid.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

8. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
Because of the importance of mobile devices to market
research and today’s data collection
Smartphone penetration growing, particularly in developed
markets.
40-70% penetration, with USA sitting about average.
Penetration levels as high as 80% among younger
demographic segments.
2013 Penetration USA Australia UK Japan Korea China
Smartphones 56% 65% 49% 42% 68% 55%
Table 1 : USA: Pew Research, Jun 2013; Australia: Google, June 2013; UK:
eMarketer, April 2013; eMarketer, Japan: May 2013; Korea: Strategy
Analytics, June 2013; China: China Internet Watch, 2013; and Nielsen Sep
2013
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

9. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
The Platforms
Smartphones versus PC
1 SMoSM: Smartphone users that were invited to conduct the
study on their Smartphone.
2 SMoPC: Smartphone users that were invited to conduct the
study on their PC.
3 PC: PC users.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

10. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
Mobile Phone Screens
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

11. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

12. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
PC Screen
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

13. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
Question types and visual response options
Mix of Question Types Mix of Response Options
Penetration Single selection
Satisfaction Multiple selection
Recommendation Drop down box
Attitudes Grid selection
Demographics Open ended
Rating scales
Table 2 : Case Study Design
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

14. Outline
Introduction
Results
Summary
Initial motivation
Why the case study?
What we tested
Visuals
Sample
Sample sizes across modes and countries
Category USA Australia UK Japan Korea China Total
SMoSM 546 493 407 543 642 397 3,028
SMoPC 941 498 1,213 155 510 689 4,006
PC 607 503 734 413 356 171 2,784
TOTAL 2,094 1,494 2,354 1,111 1,508 1,257 9,818
Table 3 : Case Study Sample Sizes (SoS: Smartphone Users on Smartphone;
SoPC: Smartphone Users on PC; PC: PC Users; TPC: Total on PC which was
General population that includes smartphone owners responding on PC)
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

15. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Enjoyment as an indicator of validity
Figure 1 : Assessing Validity via Enjoyment
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

16. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Repeatability as an indicator of validity
Figure 2 : Assessing Validity via Likelihood of Repeatability
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

17. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Drop-outs as an indicator of validity
Survey Section Smartpone (%) PC (%)
Start of survey 0.08 0.19
Demographics 0.02 0.01
1st category 0.00 0.00
2nd category 0.04 0.01
3rd category 0.02 0.01
Final section 0.05 0.01
Total drop outs 0.21 0.22
Total sample 157 664
Table 4 : Assessing Validity via Drop-outs as % of Total Completes
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

18. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Assessment for difference in results by mode
We carried out extensive statistical analysis to assess
differences in the quality of data obtained via SMoSM,
SMoPC or PC.
Very few differences if any!
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

19. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Comparability across NPS type questions and Mode as an
indicator of validity
Satisfaction USA Australia UK
Mobile
Banks
Insurance
Table 5 : Satisfaction across Country, Category and Mode.
Recommendation USA Australia UK
Mobile
Banks
Insurance
Table 6 : Recommendation across Country, Category and Mode.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

20. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Mode MobileProv FinInst’n MVIns
Sample Block 1 Block 4 Block 7
SMoSM 1.95 4.24 4.51
SMoPC 1.94 3.91 4.33
PC 1.97 3.89 4.19
Sample Block 2 Block 5 Block 8
SMoSM 1.96 4.03 3.83
SMoPC 1.96 3.81 4.08
PC 1.97 3.94 4.19
Sample Block 3 Block 6 Block 9
SMoSM 1.98 4.15 4.06
SMoPC 2.00 4.02 3.99
PC 2.00 3.87 4.16
Table 7 : Satisfaction Means by Block Rotation
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

21. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Attitudes Towards Technology
Table 8 : Estimation Results : Logit
Variable zAUS zUK zUS
DiffKpUpEmail -0.20 1.68 -0.12
DontLkeTwitter -4.36 -5.26 1.97
OnlineIsFuture 3.28 3.81 4.55
WebLimited -0.84 1.23 1.16
PersRelBetThanConn 0.13 2.67 1.20
PrivConc -5.40 -0.76 -1.25
PersIntIT 2.53 1.99 2.30
TechSavv 5.30 4.65 -0.24
LikeSMSabbrev 3.07 1.47 -0.40
ReadOnTechDev 3.91 4.48 2.68
SeekInfOnBlogs 1.02 -0.12 0.97
Intercept -2.66 -10.02 -6.34
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

22. Outline
Introduction
Results
Summary
Enjoyment
Likelihood to Do Again
Wear Out
Mode Variance
Logit Models
Logits Summary
Figure 3 : Z-values to Demonstrate Country Ratios
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

23. Outline
Introduction
Results
Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
Myths we aimed to disprove
Myths Long held views Our results suggest
1 Long surveys can’t be done on mobiles
2 PC results differ to mobiles
3 Mobile data are not representative or valid
Table 9 : Negativing 3 Myths of Mobile Research
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

24. Outline
Introduction
Results
Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
1 Can a longer survey (eg. 15 minutes) be done on a mobile?
Yes! Enjoyment is high.
Response rates and drop out rates do not differ.
2 What kind of surveys can be done on mobile?
All question types proved viable use of software which adjusts
to screen size allows for consistency.
3 Are the data valid?
YES!
Similar results were achieved.
4 Are the data representative of population?
YES!
As long as weighting by age and gender is applied.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

25. Outline
Introduction
Results
Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
Device Independence.
Convenience for respondents.
More immediate response.
Speed of survey results.
Accessing younger and harder-to-reach demographics.
Increased reach as Smartphone penetration increases.
Centralised communication social medium of the future.
Cost effectiveness - fast and non-invasive medium.
Smartphone features lend to more capabilities.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

26. Outline
Introduction
Results
Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
Limitations of smaller screen. (Although 146m Phablets to be
sold by 2016 (IHS, 2013))
Some evidence that higher drop out with more complex
questions (e.g. grids, multi-selection).
Need to adapt questions and software to smaller screen.
Smaller pool of potential respondents.
May need to calibrate results at mobile brand level.
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

27. Outline
Introduction
Results
Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
No need to limit to short 5-minute or SMS surveys!
Myths of limitations may be based on historical software
limitations.
Can reach anyone at any time.
Can be fun and likely to retain panellists.
Provides valid data, across all sections of a 15-minute
questionnaire!
Overall Conclusion
Smartphone research is a viable alternative to PC research!
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

28. Outline
Introduction
Results
Summary
Myths
Benefits of Smartphone Research
Caveats of Smartphone Research
Final Comments
Contact Details
Brian Fine
brian.fine@australiaonlineresearch.com
61 408 672 754
Con Menictas
conmenictas@qantas.com.au
cmenictas@gmail.com
61 425 339 790
Brian Fine and Con Menictas A US View of Smartphone Research with Global Benchmarks

29. Title Sponsor Gold Sponsors
Silver Sponsors
MRMW Chicago May 27-30, 2014 #MRMW

30. Workshop Sponsors
Association Education Partners
Networking Reception
Sponsor
Event App
Partner
Media Partners
MRMW Chicago May 27-30, 2014 #MRMW

31. May 27-30, 2014
Congress Plaza Hotel Convention Center
#MRMW
Organized by