1.
Putting Users in UX
Episode 3
Planning, Executing and Analyzing
User Research

2.
Putting Users in UX
Episode 1
Research Methods for Strategy
Now Available at Usability Matters Blog
Episode 2
Research Methods for Design
Now Available at Usability Matters Blog
Episode 3
Planning, Executing and Analyzing User Research
Today

3.
Meet Usability Matters
• Passionate about making technology work
for people
• Thrive in solving complex design problems
• Dedication to finding the right solution

4.
Terry Costantino
+ Steven LeMay

5.
What we’ll be talking about today
Background
• UX Process
• Research Process
User Research
1. Planning.
2. Recruiting
3. Conducting
4. Analyzing
Wrap-up

6.
UX Process
Usability Testing
A/B Testing
Eye Tracking
Heuristic Evaluation
Contextual Inquiry
Interviews
Surveys
Focus Groups
World Café
Usability Testing
Card Sorting
Usability Testing
Collaborative Sketching
Online Discussion

7.
DETERMINE
OBJECTIVES
PLAN RECRUIT CONDUCT
ANALYZE
& REPORT
Research Process

8.
1Planning
Determining the what, why, and how

9.
Planning
Test Plan
• Clarify the objectives
• Select method(s) and outline approach
• Logistics: location, device(s), timeline, schedule, etc.
• Determine participant criteria and recruiting method

10.
Research Objectives – Overall

11.
Assess the site’s ability to inform and drive users to conversion:
• Do participants understand the offer?
• Does the content drive them to subscribe or convert?
• Is there enough of the right kinds of information for participants to make the
decision to subscribe or convert?
• Is the site meeting their expectations? Is anything missing?
Assess the ease of use of the subscription flow?
• Are participants able to understand requirements for subscribing?
• Is anything confusing or unclear?
Research Objectives - Examples

12.
Planning
Logistics
• Location – office, research facility, remote via web conference, online via web
survey tool or social media
• Device(s) – desktop, mobile, other?
• Timeline and schedule
• Participant criteria and recruiting method

13.
2Recruiting
Determining who will participate in the research and finding them

14.
Image
bins
courtesy
of
slimmer_jimmer,
CC-­‐BY
2.0

15.
Recruit when you are …
• Trying to understand your audience
• Designing for specific:
Recruiting
Contexts Roles Demographics Abilities

16.
Participant Criteria – Baseline Must-Have’s
• Example: all participants must
• Have access to a laptop or desktop computer with Internet access
• Be able to speak English fluently and confidently enough to be well
understood
• Not work for a market research company, telco or broadcaster
• Be 18 years of age or older
Recruiting

17.
Participant Criteria – Project Specific Variables
• Example: the study must include
• Customers who work in a small business
• Members of our loyalty program
• Small business customer who have called the customer support line
within the last 6 months
Recruiting

18.
Participant Criteria – Demographics and Technographics
Recruiting
• Example: the study must include people who
• Are older than 65 years of age
• Who speak a language other than English at home
• Who use Facebook, Twitter, Pinterest or Instagram
at least twice a week

19.
Recruiting process
• Invite people to complete an online qualification survey
• Informal friends and family email
• Email to a customer or member list
Recruiting

20.
Example screener question
Recruiting
Have you used
public transit with a
wheelchair, walker,
or other mobility
assistive device?
☐ Within the past week
RECRUIT MIN 1, MAX 2
☐ Within the past month
☐ Within the past 6 months
RECRUIT MIN 1, MAX 2
☐ Longer ago than 6 months
☐ Never RECRUIT MAX 10

21.
Incentives
• Usually cash (or gift card)
• Very specific criteria may require more $$$
• Baked goods are great for guerrilla research!
• May not be necessary in some cases.
• May be prohibited in some cases.
Recruiting

22.
What you need
• Screener or survey
• Schedule
• Contact list
• Overview of what to expect
• Instructions for participants
• NDA and recording waiver
• Summary of participant answers
Recruiting

23.
3Conducting
Using your preparation to have a useful dialogue with participants

24.
Write research guide, script or survey
• A few easy questions to get the participant warmed up
• Instructions that guide the tasks and activities
• Probe only as needed
• Do at least one pilot session or survey response
Conducting

25.
Set-up
• Choose tools for conducting, recording, and
monitoring
• Clarify roles – facilitator, analyst, notetaker,
observers
• Test your entire set-up
Conducting

26.
Facilitating
• Getting warmed up
• Guiding the tasks
• Getting the participant(s) to talk
• Listening to the participant(s)
• Using probes to get more information
• Letting the objectives guide you
Conducting

27.
Image
observa6on
room
courtesy
of
Dave,
CC-­‐BY
2.0

28.
Monitoring
• Watching responses
• Planned and unplanned interventions
• Probing for additional information
Conducting

29.
Recording
• Flipcharts
• Sticky notes
• Note taking – manual and/or electronic
• Audio
• Video
Conducting

30.
4Analyzing
Finding and communicating useful, actionable insights

31.
Pear Note

32.
Communication
• Keep the team involved
• Debrief immediately after the research is done
• Discuss and prioritize key findings together
Analysis and Reporting

33.
Reporting
• Framework
• Smashing Elements of Mobile User
Experience
• Global issues / Page level issues
• Formal report
• Summary presentation
Analysis and Reporting

34.
Questions?

35.
Thank You

36.
3 Easy Ways to Improve the
Accessibility of Your Website

37.
Stay in touch
Terry Costantino and Steven LeMay
Usability Matters
www.usabilitymatters.com
Follow us on Twitter: @umatters
+1 416 598 7770
terry@usabilitymatters.com
steven@usabilitymatters.com