Quantitative and Qualitative Research Methods for UX Design

Quantitative Meets Qualitative

Quantitative Meets
Qualitative
Dr. Nichole Carelock
Manager User Insights, PenFed Credit
Union

The Case of the Disappearing
Data
 The Problem: A blended learning math platform
for the State of Ohio was losing an alarming
amount of student data everyday.
 Quantitative data told us we had 97%
(compulsory) engagement but only a 63% data
transfer from day to day.
 Qualitative data told us our Log in page was
enticing ESL and non readers to sign in as Guest.
• Which dumped their data at the end of each session.

UX Methods are Categorized in
Two Ways:
 Quantitative methodology is defined as producing
numerical data which can be expressed as stats,
percentages, etc.
 Qualitative methodology on the other hand, is
concerned with collecting textual (word) data
which is used to identify themes and patterns.

Compare
QUANTITATIVE QUALITATIVE
Measuring things Describing things
Numbers Natural description
Fixed state Negotiable state
Language of statistical analysis Natural language of participants
Formulation of facts Helps develop ideas
Closed questions in surveys Open questions in surveys
A/B Tests Interviews
Web analytics Voice of customers tools

Benefits and Drawbacks
QUANTITATIVE QUALITATIVE
PRO PRO
Make data-driven business decisions Brings numerical data to life
Great at convincing to Stakeholders Great to plan or develop design strategy
CON CON
Numbers require interpretation Time consuming- expensive
Easy to draw wrong conclusion Non-Automated
Free or Inexpensive tools Free or Inexpensive Tools
Facebook insights, Clicktale 4Q by iPerceptions
Remote, asynchronous focus grouping (google
Facebook)
Kiss insights, Optimizly Usertesting.com, GuerillaTesting (friends and family)

Quantitative Research → Qualitative
Research
 Helps explain why certain things happen after you
discover them through statistical analysis.

Qualitative Research → Quantitative
Research
 This order allows you to uncover trends and/or
preferences from individuals and then support
these claims on larger samples.

Quantitative and Qualitative
Research Simultaneously
 One simple way to obtain both
types of data is to send a
survey with closed and open
questions.
 The closed surveys will give
you statistical results, while
open ended surveys provide
you with descriptive answers
from users.
 The two methods work together
 Sky and Water MC Escher

Using Mixed Methods to get
Results
 There are several steps involved in identifying the
right research method or combination of methods.
• Understand the problem.
• Define the research questions.
• Identify the data that will answer those questions.
• Select the research methods that will provide the
necessary data.

Understand The Real Problem
 Fake Problem: “We
get too many calls to
the call center that
people should be
using the website to
fix”
 Real Problem:
Information is
organized poorly or
the site and the FAQ’s
are anything but.
 How would we find
that out?

Define the Research Goals Correctly
 Poor Research
Question: “Do Users
like the message
boards”
 Good research
question: “Are
message boards the
best way of meeting
the organization’s
objectives”
 How would we find
that out?

Identify the Data that Will Answer
Those Questions.
 Do you need to understand what people actually
do or what their opinions are?

Select the Research Methods that will
Provide the Necessary Data.

Definitions: UX Methods
 Usability-Lab Studies: participants are brought into a lab,
one-on-one with a researcher, and given a set of scenarios
that lead to tasks and usage of specific interest within a
product or service.
 Ethnographic Field Studies: researchers meet with and
study participants in their natural environment, where they
would most likely encounter the product or service in
question.
 Participatory Design: participants are given design
elements or creative materials in order to construct their
ideal experience in a concrete way that expresses what
matters to them most and why.
 Focus Groups: groups of 3-12 participants are lead
through a discussion about a set of topics, giving verbal
and written feedback through discussion and exercises.
 A/B Testing (also known as “multivariate testing,” “live
testing,” or “bucket testing”): a method of scientifically
testing different designs on a site by randomly assigning
groups of users to interact with each of the different
designs and measuring the effect of these assignments on
user behavior.
 True-Intent Studies: a method that asks random site
visitors what their goal or intention is upon entering the
site, measures their subsequent behavior, and asks
whether they were successful in achieving their goal upon
exiting the site.
 Interviews: a researcher meets with participants one-on-
one to discuss in depth what the participant thinks about
the topic in question.
 Eyetracking: an eyetracking device is configured to
precisely measure where participants look as they perform
tasks or interact naturally with websites, applications,
physical products, or environments.
 Usability Benchmarking: tightly scripted usability studies
are performed with several participants, using precise and
predetermined measures of performance.
 Unmoderated Remote Panel Studies: a panel of trained
participants who have video recording and data collection
software installed on their own personal devices uses a
website or product while thinking aloud, having their
experience recorded for immediate playback and analysis
by the researcher or company.
 Clickstream Analysis: analyzing the record of screens or
pages that users clicks on and sees, as they use a site or
software product; it requires the site to be instrumented
properly or the application to have telemetry data
collection enabled.
 Intercept Surveys: a survey that is triggered during the
use of a site or application.
 Email Surveys: a survey in which participants are
recruited from an email message.

Activity
 Scenario:
 You are an
entrepreneur,
developer and
designer looking to
make a breakthrough
medical app for the
Cystic Fibrosis
Community.
 In groups of 4 or 5, take 7
minutes to read the
information from Packet 1
 What type of app do you have
in mind?
 Take an additional 7 minutes to
read the information from
packet 2.
 What type of App do you have
in mind now? Did it change and
why?

Conclusion
 Using the right data in the right combination is crucial to
creating the experience you want your user to have.

A Website For
Everyone
Vencer Cotton
Senor Director of Technology, Columbia
Lighthouse for the Blind
Nina Amato
Senior UX Designer, Brightfind

It’s cheaper to build a
ramp into the design
than to rip out the
stairs and add the
ramp later.

Color Contrast for Text
 Text color must have sufficient contrast compared
to it’s background.
 Text requires a contrast ratio of 4.5:1 for normal
text (<18px) and 3:1 for large text (>24px).
 You can check the color contrast of text to its
background by entering the color codes in a
contrast checker such as the one at:
• http://webaim.org/resources/contrastchecker/

Color Blindness
8% of men and 0.5% of women with Northern
European ancestry are red-green color blind1
1NIH National Eye Institute, https://nei.nih.gov/health/color_blindness/facts_about

Color Blindness
 View graphics/websites as a color blind user:
• Adobe Photoshop or Illustrator [View > Proof Setup >
Color Blindness (Deuteranopia for red/green color
blindness)]
• Chrome plugin: Spectrum
https://chrome.google.com/webstore/detail/spectrum/of
clemegkcmilinpcimpjkfhjfgmhieb?hl=en

Skip Navigation
 WebAIM’s 2015 survey of screen reader users1 asked
participants if they utilized “skip navigation” links.
Respondents answered:
18.3% - Whenever they're available
19.5% - Often
30.2% - Sometimes
17.7% - Seldom
14.3% - Never
68% use skip nav functionality at least sometimes
1http://webaim.org/projects/screenreadersurvey6/

Buttons on Mobile
82% of blind users reported using a screen reader
on their mobile device in 2014.1
1 http://webaim.org/projects/screenreadersurvey5/#mobileusage

Buttons on Mobile
“When some developers design their apps, they
don’t label all of their buttons and controls, so the
screen reader just says, ‘This is a button,’ but it
doesn’t say what the button actually does,” Mr.
Perez said. “That’s an area where we need a lot of
improvement.”
http://bits.blogs.nytimes.com/2013/09/29/disruptions-guided-by-touch-
screens-blind-turn-to-smartphones-for-sight/?_r=0

Testing for Accessibility
 W3C Web Accessibility Evaluation Tools List:
https://www.w3.org/WAI/ER/tools/
 NVDA is a free screen reader for Windows (they
request a donation)
• http://www.nvaccess.org/

AInspector Sidebar [for Firefox]
https://addons.mozilla.org/en-US/firefox/addon/ainspector-sidebar/

Accessibility Developer Tools [for Chrome]
https://chrome.google.com/webstore/detail/accessibility-developer-
t/fpkknkljclfencbdbgkenhalefipecmb

ARIA Validator [for Chrome]
chrome-extension://oigghlanfjgnkcndchmnlnmaojahnjoc/summary.html

 WAVE browser plugin
• https://wave.webaim.org/toolbar/
• https://chrome.google.com/webstore/detail/wa
ve-evaluation-
tool/jbbplnpkjmmeebjpijfedlgcdilocofh?hl=en-
US

 Navigate every web page using the keyboard only.
• Navigate with these keys:
o Tab (forwards)
o Shift + Tab (backwards)
o Space bar (sometimes used to select and unselect
checkboxes)
o Arrow keys
o Enter key
 Test that all links get visual focus (an outline) when they
are tabbed to, so that users who do not use a mouse
know they successfully landed on a link.

 Check that all data tables have proper structure:
• THEAD
• TBODY
• TFOOT
• Scope=“col”
• Scope=“row”
 Check color contrast of text against background
color
• http://webaim.org/resources/contrastchecker/

 Check for ARIA tags
 View websites with styles disabled
• Web Developer Toolbar for Chrome and Firefox
o https://chrome.google.com/webstore/detail/web-
developer/bfbameneiokkgbdmiekhjnmfkcnldhhm?hl=en-US

Open Discussion &
Questions

30 Ideas in 30 Minutes
Nichole Carelock, Manager User Insights,
PenFed Credit Union
Nina Amato, Senior UX Designer,
Brightfind
Megan McKelvy, VP Customer Engagement
Brightfind, Brightfind
Elizabeth Greenstein, UX Designer,
Brightfind

Tip 1
Push for research first, or
you’ll be chasing finished
products that no one is
interested in changing.
NC

Tip 2
Don’t use the same text in
both alt attributes and title
attributes.
NA

Tip 3
Challenge your personas.
MM

Tip 4
Design for fingers and
thumbs.
EG

Tip 5
Hone in on key users that
can give you the most for
your time.
NC

Tip 6
Data tables will be read aloud by
screen readers differently
depending on how they are coded.
•Add the attributes scope=“col”
and scope=“row” to your header
cells.
NA

Tip 7
UAT testing is not usability
testing.
MM

Tip 8
Make a persona on a
Post-It in 5 minutes or
less.
EG

Tip 9
Don’t wait until something
is broken to address a
potential problem.
NC

Tip 10
Add ARIA-Live and ARIA-
Atomic to areas of a page
that update without the
browser reloading.
NA

Tip 11
Respect your users’
moods.
MM

Tip 12
It’s impossible to imagine
all the edge cases, so be
a good listener when your
users give you feedback.
EG

Tip 13
Incentivize your research
participation wherever
possible.
NC

Tip 14
Sitemaps are helpful for
users with cognitive
delays.
NA

Tip 15
Start small; go free.
MM

Tip 16
Avoid putting important
content or calls-to-action
in the right rail.
EG

Tip 17
Make sure you’re fixing a
root not a symptom.
NC

Tip 18
Navigate every web
product you produce with
the keyboard only.
NA

Tip 20
Make mega-menus clearly
narrower than the page
width.
EG

Tip 21
Be ready to fight for time
to do adequate research.
Get your elevator speech
ready!
NC

Tip 22
PDFs need to be made
accessible too. This often
takes longer than HTML.
NA

Tip 24
Avoid using carousels or
sliders.
EG

Tip 25
Create an email list of
beta testers; send them
surveys and new features.
NC

Tip 26
Add visual focus around
links.
NA

Tip 27
Refine, don’t redesign.
MM

Tip 28
Embrace scrolling on
websites; use visual cues
to indicate that more
content is available.
EG

Tip 29
Don’t be restricted by time
and space to find
participants. Google groups
and Facebook groups are
focus groups.
NC

Tip 30
Use the label tag for form
labels.
NA

Tip 31
Use contextual links.
MM

Tip 32
Optimize your website to
improve your users’
perception of its speed.
EG

Tip 33
Think of your assumptions
about the site as
hypotheses. This will
make them easier to part
with in testing.
NC

Tip 34
Required form fields need
aria-required="true".
NA

Tip 35
Understand that some people
search, some people
navigate.
MM

Tip 36
Avoid using hamburger
menus on desktop views.
EG

Tip 37
The order of data
collection matters.
EG

Tip 38
Use ARIA landmark roles
to mark page elements
such as navigation and
main content.
NA

Tip 39
Take a class in web
writing.
MM

Tip 40
Align form labels and
fields in a single vertical
line so users may easily
scan them.
EG

Quantitative and Qualitative Research Methods for UX Design

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Quantitative and Qualitative Research Methods for UX Design

Similaire à Quantitative and Qualitative Research Methods for UX Design (20)

Dernier

Dernier (20)

Quantitative and Qualitative Research Methods for UX Design

Notes de l'éditeur