User Testing Tactics

User Testing:
The Gory Tactical Details

Cindy Alvarez
The Experience is the Product -
http://www.cindyalvarez.com

Copyright 2009 Cindy Alvarez

What We’ll Cover
• Why In-Person User Testing?
• Methodology
• Setting Up
• Running the Test
• Followup


My Background
• 10 years running product design and
product management for startups of
various sizes
• Designed/managed products used by
millions on sites like Bank of America,
American Express, AOL, PCWorld, US
News, Sega.com
• Average yearly user testing budget:
$3,000


What You Can Learn For $700

• How do people complete this task?
• How well do they understand what
they’re doing?
• What issues are hidden by successful
completion rates?
• What would they not even dream of
asking for?


Why Not Online Testing?
Body Language

Tangents

Errors


When Should I Test? Now.
• Test as soon as you’ve identified your
2-3 core use cases
• Users don’t know your product isn’t
complete
• You only need the minimum pages to
get through workflow
- Paper, BalsamIQ, HTML
- Think static, minimal, not all links
need to work

2 Core Use Cases + ...
New product concept Splash screen
Registration > 1 screen
Entire first user
Registration requires signup
personal/financial info experience

Distribution relies on
social/viral Invite/share

Validating minimum Navigation
viable feature set menus

Minimize Effort
• Existing market? Use a competitor site
• Borrow HTML forms from other sites
• Use screenshots for non-interactive
parts of page


Methodology

• 2-3 task-based use cases
• 7-8 users
• 45 minutes
• Moderator + note-taker (or video)
• $50-75 (Amazon GCs are easy)


Finding People
• Craig’s List “gigs” section
• 1-2 weekdays in advance
• Provide a non-anon email address and
voicemail
• Ask for qualifiers (profession and city
are good)
• Plan for 10-15% “flake rate”
• Allow extra time after 1st tester


Advance Prep
• “Standard” computer setup in private
room or isolated area
• Test any recording equipment/software
• Provide dummy “personal” info, credit
card #s
• Make templates for note-taking so it’s
easier to line up feedback later


Moderator Intro
• Act like a third party
• Encourage “thinking out loud”
• Explain upfront that this is NOT a
working product, some things may not
work


Example Test


QUESTIONS:
• What do you think
this product will do
for you?
• Would you be likely
to continue on?
• Why?
TIP: Include dummy footer
to pass basic “credibility
gut-check”


“I Just Wouldn’t Use This”
• Not understood or not useful?
• Substitute?
• Specific concern?
• Do they use any sites/products in this
category?
• “Well, let’s pretend that you
were curious about it and
finish the tasks.”

TIP: Don’t skip any steps,
even if they seem
obvious.
You don’t need to ask
questions if none seem
appropriate.


QUESTIONS:
• What do you think
“Connect Your
Networks” means?
• Have you seen
something like this
before?
• What would you do
next?


“I Have No Idea What to Do Next”

• Does anything stand out?
• What would you do in “real life”?
• Acknowledge frustration
• 3 Strikes, then “Actually,
you’d click here.” How could
that be clearer?


TIP: When there is hard-
coded “personal”
information, ask them to
pretend it is theirs.
QUESTIONS:
• Has anything so far
surprised you?


QUESTIONS:
• Is this the type of
information you
were expecting to
see?
• What would you do
next?
TIP: Don’t stop users from
clicking on “dead” links -
find out why they were
more interesting, then hit
the Back button.

QUESTIONS:
• What can you do on
this page?
• Why do you think the
blue box is asking
you to enter more
information?
• What do you expect
to happen if you
were to click one of
the links in “In the
News?”

“I Don’t Like this Color”

• Legible?
• Credible?
• “I don’t think these are the
final colors and graphics.
Let’s focus on the task you’re
trying to complete.”


TIP: It’s OK to hack things
like links to other sites,
applications, etc.


QUESTIONS:
• Would having this
information make it
easier for you to
write an email to
this contact?


QUESTIONS:
• Is this what you
expected to see
when you clicked
“Compose Email”?
• If you had been on
your own, how likely
would you have been
to go through this
whole process?


Moderator Wrap-up
• Ask for any additional feedback/
comments
• Thank the user for their time
• Ask for future participation


Followup
• Share results internally ASAP
• Prioritize issues that need fixing
• Look for patterns in freeform user
comments
• Share results with testers who agreed to
participate in future


Questions?

Send them to: cindy@cindyalvarez.com

Sample templates for Moderator Script,
Notetaking, and Craig’s List Ad available at
http://www.cindyalvarez.com/resources/
UserTestingTactics.zip


User Testing Tactics

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à User Testing Tactics

Similaire à User Testing Tactics (20)

Plus de Cindy Alvarez

Plus de Cindy Alvarez (20)

Dernier

Dernier (20)

User Testing Tactics

Notes de l'éditeur