Presentation at the ASC International conference, Bristol, 2011. A full written paper is also available from the meaning website

1. Making it fit
How survey technology providers are responding
to the challenges of handling web surveys on
mobile devices
Tim Macer, meaning ltd, London, UK
ASC Sixth International Conference, Bristol, 22-23 September 2011

2. Agenda
1. Emerging best practices
2. Requirements identified
3. How are the vendors responding
4. Conclusions and recommendations
2

3. Making it fit: web surveys on mobile devices. ASC 2011
1. Emerging best
practices
Specific considerations
Operating systems
App or browser?
3

4. Making it fit: web surveys on mobile devices. ASC 2011
Multi-mode parallel surveys on web/mobile
Design the survey around single-choice, multiple-choice
and open-ended questions alone.
Adopt more concise wording
Limit the survey length to fifteen questions.
Zahariev et al (2010)
4

5. Making it fit: web surveys on mobile devices. ASC 2011
Specific technical requirements identified…
Identifying which specific smartphones are supported, as
only specific makes or models may be supported in
some packages.
Verifying that error messages and other components are
not dependent on the use of JavaScript.
Having a specific „skin‟ or page template to format the
survey for delivery on a mobile device, which maximised
the limited space available for questions.
Zahariev et al (2010)
5

6. Making it fit: web surveys on mobile devices. ASC 2011
Recommended optimiziations for mobile surveys
Use a concise, well-worded SMS invitation with an
integrated link to survey.
To ensure any costs a participant may incur are
reimbursed
Avoid complex matrix questions: replace with questions
presented one by one.
Encourage active engagement, e.g. taking photographs.
Pick time to send invites in order to optimize response
(e.g. between 17.00 and 18.00).
Pferdekaemper (2010)
6

7. Making it fit: web surveys on mobile devices. ASC 2011
Further recommendations for mobile
Reduce and simplify –
 avoid horizontal scrolling,
 reduce clutter from logos, branding,
 consider using drop-down lists rather than radio buttons
Luck (2011) “How to make mobile surveys work for everyone”
Adjust the survey to fit limited the screen of each device
 Understand the dimension of the device
 Capture the browser agent string
Callegaro (2010) “Do You Know Which Device Your Respondent Has Used to Take
Your Online Survey?”
7

8. Making it fit: web surveys on mobile devices. ASC 2011
Current trends in Mobile operating systems in use
OS 2010 2014 trend
Android (Google) 17.7 29.6 
iOS (Apple) 15.4 14.9 
Research In Motion (BlackBerry) 17.5 11.7 
Symbian 40.1 30.2 
Windows Phone 4.7 3.9 
Other Operating Systems 4.7 9.6 
Mobile Communications devices: sales to end users by OS
Gartner (2010)
8

9. Making it fit: web surveys on mobile devices. ASC 2011
What defines an app or a browser survey?
browser app
A web survey which has been A survey delivered through a
formatted so that it will work dedicated application or app
effectively in the browser of a which the participant must first
range of mobile devices within download onto his/her device
the constraints of these and then accesses via an icon
devices. on the device‟s main menu.
9

10. Making it fit: web surveys on mobile devices. ASC 2011
Browser pros and cons
 UsesHTML: achieve high ✘ Lowest common denominator
coverage; not restricted to a approach to question types
subset of devices
✘ Not all devices support flash
 Participant
can engage
✘ Integrating media, e.g.
immediately without having to
uploading photos, can be too
download anything
difficult for participants to do
 Mix formats within one survey
✘ Needs a stable internet
instrument: laptop/desktop,
connection each time “next” is
tablets, handhelds
pressed
10

11. Making it fit: web surveys on mobile devices. ASC 2011
App pros and cons
 Does not need a permanent ✘ Download requires high level
stable internet connection at of trust and engagement from
all times participant
 Can exploit all the ✘ Apps rarely support the entire
programmable capabilities of range of devices out there –
the device, e.g. diaries with reach less of the population
reminders, integrated capture
and upload of pictures and
video
11

12. Making it fit: web surveys on mobile devices. ASC 2011
2. Identifying the key
requirements
Essential needs for browser tools
Essential needs for apps
The survey
12

13. Making it fit: web surveys on mobile devices. ASC 2011
Essential requirements examined: browser-based
tools
1. Automatically detect the device 6. Allow respondent to save and
resolution and reformat survey resume later in „desktop‟ or
to fit mobile mode
2. Optimize the presentation of the 7. No dependency on mouse
survey for small-format devices controls (shift+click etc)
3. Gracefully unwind grids into a 8. No dependency on using Flash
series of individual questions controls
4. Record device type used in the 9. Can create specific routing and
data validation dependent on device
type
5. Integrated survey invitations and
reminders by email and SMS
13

14. Making it fit: web surveys on mobile devices. ASC 2011
Essential requirements: app-based tools
1. Simple, easy download for 5. Diary surveys: repeated
respondent capture of same questions
2. Web server-based 6. Diary reminder alerts with
authoring and deployment do not disturb option
environment
7. Online/offline capability
3. Capture location data, with
8. Bar code scanning with
respondent permission
respondent confirmation
4. Capture and upload photos
and video
14

15. Making it fit: web surveys on mobile devices. ASC 2011
…plus, requirements common to both modes
10. Wide range of devices supported
11. Device-specific preview in authoring tool or via simulators
15

16. Making it fit: web surveys on mobile devices. ASC 2011
The survey
Contacted 10 different Contacts made in
technology producers April 2011 with a simple
active in the field electronic questionnaire
7 firms Eight companies
with browser-based solutions participated
6 firms with apps One refused
5 with both One failed to respond

17. Making it fit: web surveys on mobile devices. ASC 2011
3. How are the vendors
responding
Platforms supported
Features provided
17

18. Making it fit: web surveys on mobile devices. ASC 2011
Who is providing what?
Supplier Browser App
Askia ✓ ✓
CfMC WebSurvent ✓
Confirmit ✓
Globalpark EFS ✓ ✓
iPinion Surveys ✓ ✓
Kinesis ✓ ✓
Surveyswipe & Surveypocket ✓ ✓
Techneos ✓
18

19. Making it fit: web surveys on mobile devices. ASC 2011
Browser tools – devices supported
Survey- Con- Global
Device iPinion Kinesis CfMC Askia
swipe firmit -park
Android ✓ ✓ ✓ ✓ ✓ ✓ ✓
BlackBerry ✓ ✓ ✓ ✓ ✓ ✓ ✓
Symbian ✓ ✓ ✓ ✓ ✓
iOS (Apple) ✓ ✓ ✓ ✓ ✓ ✓ ✓
Windows mobile ✓ ✓ ✓ ✓ ✓ ✓ ✓
WML  ✓ ✓
XHTML  ✓ ✓ ✓ ✓ ✓
✓ Capability currently offered  Planned for a Future release
19

20. Making it fit: web surveys on mobile devices. ASC 2011
Mobile Apps – devices supported
Survey- Techneos Askia Global-
iPinion Kinesis pocket SODA mobile park
Android ✓  ✓ ✓ ✓ 
BlackBerry ✓  ✓ ✓ 
Symbian  ✓ 
iOS (Apple) ✓ ✓ ✓ ✓ ✓ ✓
Windows
mobile
  ✓ ✓
✓ Capability currently offered  Planned for a Future release
20

21. Making it fit: web surveys on mobile devices. ASC 2011
Web browser solutions
Analyzing the capabilities offered in
the market
21

22. Making it fit: web surveys on mobile devices. ASC 2011
Browser tools – capabilities offered by all vendors
Capability
Mixed mode – full screen and mobile ✓
Detect and reformat to fit device ✓
Optimize for mobile ✓
No dependency on mouse ✓
No dependency on Flash ✓
22

23. Making it fit: web surveys on mobile devices. ASC 2011
Browser tools – capabilities where support varies
Survey- Con- Global-
Capability iPinion Kinesis CfMC Askia
swipe firmit park
Unwind grids ✓ ✓  ✓ ✓ ✓ ✓
Store device data
• At survey level ✓ ✓ ✓ ✓ ✓ ✓
• At question level ✓ ✓ ✓ ✓
Invitations, reminders by
Email and SMS
✓ ✓  ✓ ✓ ✓ ✓
Preview via simulator  ✓   ✓
Save and resume ✓  ✓ ✓ ✓ ✓
Routing by device  ✓ ✓ ✓ ✓
23

24. Making it fit: web surveys on mobile devices. ASC 2011
Essential requirements fulfilled: browser tools
All offered device auto-detection and optimization for
mobile, plus unwinding of grids
Only 2/7 products met all the key capabilities
6/7 supported email and SMS invitations/reminders
5/7 supported save and resume
Capturing of the agent string
 Most will record device data at a survey level;
 Only 4/7 record the device at question level or allow
conditional routing on the device type
24

25. Making it fit: web surveys on mobile devices. ASC 2011
Mobile app solutions
Analyzing the capabilities offered in
the market
25

26. Making it fit: web surveys on mobile devices. ASC 2011
Mobile apps – capabilities offered by all vendors
Capability
Capture location data, with respondent
✓
permission
Capture and upload photos and video ✓
Support for Android, BlackBerry and
✓
Apple iOS
26

27. Making it fit: web surveys on mobile devices. ASC 2011
Mobile Apps – download method supported
Survey- Techneos Askia Global-
iPinion Kinesis pocket SODA mobile park
Simple 1-click from a ✓ ✓ ✓ ✓ ✓
web link
From an integrated ✓ ✓ ✓  ✓ ✓
panel portal
From app site or ✓ ✓ ✓ ✓ ✓
App Store
27

28. Making it fit: web surveys on mobile devices. ASC 2011
Mobile Apps – capabilities where support varies
Survey- Techneos Askia Global-
iPinion Kinesis pocket SODA mobile park
Web browser authoring
and deployment
1 ✓ ✓ ✓  ✓
Device-specific preview   ✓2 ✓
Diary survey mode ✓ ✓  ✓  ✓
Diary alerts (regulated)  ✓ ✓  ✓
Online/offline ✓ ✓ ✓ ✓ 
QR/Bar code scanning  ✓  ✓ 
1Authoring via either Confirmit or iBM SPSS Dimensions design module 28
2via 3rd party simulator

29. Making it fit: web surveys on mobile devices. ASC 2011
Essential requirements fulfilled: Apps
6/6 will capture location data, photos or video, all under
respondent control
6/6 offered a range of one-click download options
5/6 would operate without a constant internet connection
4/6 offer diary support and the other 2 are developing it
 Diary reminders are only fully supported (with the option for
participants to disable them) by 4 of the 5.
QR/bar codes supported by 4 but with 2 more developing
29

30. Making it fit: web surveys on mobile devices. ASC 2011
Discussion
Conclusions
Recommendations
30

31. Making it fit: web surveys on mobile devices. ASC 2011
Observations
Some vendors have specialized in serving the browser or
the app-based solution, but many cater for both
All the browser solutions we examined will integrate
seamlessly with standard online surveys (auto-detecting
the device)
Most but not all vendors support a wide range of devices
 iOS, Android and Blackberry universally supported
 Symbian only offered by 5/7 browser and 5/6 app products
 XHTML supported by just 5/7 browser tools and the old (and
very restrictive) WML/WAP by only 2.
31

32. Making it fit: web surveys on mobile devices. ASC 2011
Observations
Models of best practice are still developing
Previewing and simulating how the survey will appear on
different small-format devices is really important
 Only 2/7 browser tools and 2/6 apps did this,
 However, two more in each category plan to support this
soon
Software developers are not necessarily taking
advantage of the capabilities that mobile devices offer
32

33. Making it fit: web surveys on mobile devices. ASC 2011
Conclusions 1 of 2
Mobile research is still at an Support still rudimentary
early stage of development Only two of the solutions
met all of the key
capabilities
Wide variation in what is But developing fast
offered by different vendors Most of the vendors
platforms supported, key recorded key capabilities
features etc that they were planning to
introduce soon

34. Making it fit: web surveys on mobile devices. ASC 2011
Conclusions 2 of 2
Browser-based tools may All browser-based solutions
offer better coverage but lack examined are competent at
the more advanced mobile- rending surveys on full-
specific capabilities screen and mobile devices
Developers seem not to be The more advanced and
aware of the importance of distinctive features of Mobile
testing for survey designers Research are not well
supported in mobile apps
yet

35. Making it fit: web surveys on mobile devices. ASC 2011
Further work
Future work needs to focus on the effect of mobile
devices on conventional online research
Survey takers increasingly using mobile devices to take
online
 Typically around 4% and growing
 For them, the survey experience frequently degenerates into
chaos
Web survey providers in general not addressing mobile
surveys – desktop/laptop participation is assumed
35

36. Questions and comments welcome
Tim Macer tim@meaning.uk.com
36