Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Easy & affordable

user testing
Ida Aalen / @idaaa
CPO, Confrere
ida@confrere.com
Workshop
confrere.com/usertest
Task 1
Read the title and the intro.
Write down 1-5 questions you
expect will be answered on this
page.
Task 2
Read the text.
• Were your questions answered?
• Is there content that could be cut
or should be expanded upon?
#1
Micro-testing content.
• Time: 30 min+
• Price: Mostly free
Question 1
Read the title and description. Write down 3-4 questions you expect
will be answered on this page.
Don’t peek!
Feedback on
needs, not
grammar and
typos
Feedback on
needs, not
grammar and
typos
• What equipment is supported?
• Can this service be used on
mobile devices?
• How can I know that it is safe?
• What is t...
User testing can be

easy & affordable.
Why do

user testing?
User testing is

real users solving
real tasks.
Do you recognise this icon?




Survey with 4273 respondents, 2014
Mobile:
53%

said no
Desktop:
73%

said no
…but I can’t see the
football…
Colleagues

are not your users.
Designers

are not your users.
Developers

are not your users.
1 in 3

problems found

by experts are

false alarms
1 in 2

user problems

are overlooked

by experts
Only users can let you know
whether it’s intuitive.
But how many users?
5
N (1-(1-L)n)
0%
25%
50%
75%
100%
Number of test subjects
0 3 6 9 12 15
Shareobservedproblems
Small and frequent
tests are better than
b...
User testing
• …it’s the only way to know if the design truly is “intuitive”
• …you’re taking big risks when you’re not tes...
#2
Information architecture
• Time: 30 min+
• Price: Free / $99-199 per month
#3
First click.
• Time: 30 min+
• Price: Free / $99-199 per month
Tasks.
Finding possible issues
Customer service and receptionists
What are people calling and asking about?
Website analytics
Whe...
You should test things that…
• are done frequently or by many users
• has serious consequences if users fail with their tas...
What we want to find out:
Is it clear to physicians
that they can use Confrere
for video appointments?
Our question
≠

User task
Open & explorative
What would you do to
find a video calling
service that’s suitable
for physicians?
Specific & measurable
C...
Open & explorative
• If you want to find
more problems
• It’s ok to give
different tasks to
different people
• More realist...
Phrasing the tasks
• Don’t use the exact same phrases as on the website
• Give the users a problem to solve, rather than a...
What do we want to know?
“Is it clear to physicians that they can use Confrere for video
appointments?”

How do we phrase ...
Intro
• User’s rights and informed consent
• Introductory questions / warm-up

Tasks
• The user tries to solve this, while...
Time schedule
• You’ll always underestimate how long the tasks take
• Begin with 2-5 tasks, and do a trial test
• Prioriti...
Moderating.
Help the
person to
relax
Make it
easier for
observers to
understand
Make sure we
follow the test
plan
Help the person to relax.
“Would you like something
to drink?”
Checklist before beginning
• Welcome the person, help them feel comfortable
• Go through rights and consent form, make it c...
Help the person
think out loud
What are you thinking about?
I’m looking for the menu…
Help the user
think out loud
and ask open
questions
What happens if I click here?
What do you think might
happen if you click there?
Make
sure you’re
not helping
the user
How to help someone think
out loud
• What are you looking for now?
• What are you thinking about now?
• What did you expec...
How do you lead a test,
without leading the user?
I’ve made the design for the
new website and I’d love to
hear what you think!
The team that designed the
site needs help fi...
You’re really good at this!
You’re solving all the tasks!
Thank you! Let’s move on

to the next task.
It would have gone so much
faster if you had just clicked
the menu-button.
There's really big room for
improvement here. T...
Sigh…
The user

is there to help
you
#3
Get out.
• Time: 1-4 hours
• Price: Snacks
It’s this easy:
(Translated
by Google)
What did you find?
Analysis in 1-2-3
Is there anything you observed…
…across several users on the same task??
…across several tasks?
You star...
Observation

and notes.
Don’t listen.
Observe.
Rationalization


Michael S. Gazzaniga:
“The Interpreter”
http://youtu.be/mJKloz2vwlc
Differentiate between…
Observation
What the person did and said during the task
Interpretation
Why you believe the person d...
Example
Observation
• The user did not click the “Buy”-button
Interpretations
• The button should be placed elsewhere
• Th...
Quantitative
Qualitative
Self-reported Observed
Surveys
A/B-testing
User testing
Interviews
Focus groups Field studies
Goo...
Read this!
How to take notes?
• Remember the purpose of the task - don’t write a novel on
what each person does
• Note which words th...
Paper sketch
Paper prototype
Wireframes Sketch / Photoshop
InvisionApp
HTML prototype
without styling
HTML&CSS&JS
prototyp...
#5
Scrolling, clicking and
hovering.
• Time: Done in 30 min
• Price: Free (e.g. Hotjar + Google Analytics)
#5
Was this helpful?
• Time: 15 min+
• Price: Often free
#4
Lab test on a budget.
• Time: 1 day for planning and recruiting, 1 day for the actual
test
• Price: Rewarding users
(Translated
by Google)
“Hugging”
If you need observers:
Test machine Observation
+ Tool for

screen sharing+
Informed
consent
• They can opt-out any time
• What data are you gathering and
why
• For what and when will you use it
• W...
20-30 min break
between each test
When do you need observers?
• If you need to align stakeholders
• In general, the more observers, the better, because diff...
Analysis and
prioritisation.
The analysis must be done as
soon as possible, by the
people who did the user test.
Which of our findings or
observations should be
turned into to-dos?
What’s relevant for our
further work?
Documenting and sharing
your findings
As a tool for your team
Put screenshots
in InvisionApp in
advance
Add
observations
between (or
during) each
test
Connect
InvisionApp to
Slack, and your
teammates can
follow along
Vise alvoretWake-up calls
Innhold
A highlights video usually
does the trick.
Prioritizing
number of users consequences other sources cost
Make sure something
happens with your findings
• Each (prioritised) usability issue needs a to-do
• Each to-do must be assi...
Do it together!
Summarizing
and prioritizing
in Google Sheets
or AirTable
Recruiting.
Selection criteria
• Diversity
• Language skills
• ICT skills
• Target group criteria
• Background knowledge or
experience...
Recruiting tips
• Friends of friends, social media, newsletters
• Go to where they are
• Run it outside of regular work ho...
Avoiding drop-outs
• Send e-mails or texts with very clear instructions for how to
get there, sign them personally and add...
7 methods:
1. Micro testing content
2. Information architecture
3. First click
4. Get out
5. Lab test on a budget
6. Scrol...
7 methods:
1.Micro testing content
2.Information architecture
3. First click
4. Get out
5.Lab test on a budget
6. Scrollin...
7 methods:
1. Micro testing content
2. Information architecture
3. First click
4.Get out
5. Lab test on a budget
6.Scrolli...
7 methods:
1. Micro testing content
2. Information architecture
3. First click
4.Get out
5.Lab test on a budget
6. Scrolli...
You learn so
much more by
meeting people
face-to-face!
Thank you!
confrere.com/usertest
Ida Aalen
Chief Product Officer
confrere.com
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Easy and affordable user testing - Workshop
Prochain SlideShare
Chargement dans…5
×

Easy and affordable user testing - Workshop

2 881 vues

Publié le

Read more at confrere.com/usertest

Everyone wants to make intuitive products. But you can’t really know they’re intuitive, if you haven’t tested them with actual users. So why don’t we user test more often? It might be because we think we’re lacking the time, budget, equipment or simply the knowledge to do it. But that doesn’t have to be true.

CONTENTS
Tools and methods: Get to know 7 easy and affordable techniques to do user testing
Planning and prioritising: When and what should you test?
Tasks and moderating: How do you make sure you’re not leading users during the test?
Recruiting: How many people do you need to test with, and where can you find them?
Observation: How should you interpret what you see and take notes during a user test?
Analysis: How can make sure your usability findings lead to actual improvements?

Publié dans : Design
  • Penis Enlargement and Enhancement Techniques: What REALLY Works?!? ▲▲▲ https://bit.ly/30G1ZO1
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Easy and affordable user testing - Workshop

  1. 1. Easy & affordable
 user testing Ida Aalen / @idaaa CPO, Confrere ida@confrere.com Workshop
  2. 2. confrere.com/usertest
  3. 3. Task 1 Read the title and the intro. Write down 1-5 questions you expect will be answered on this page.
  4. 4. Task 2 Read the text. • Were your questions answered? • Is there content that could be cut or should be expanded upon?
  5. 5. #1 Micro-testing content. • Time: 30 min+ • Price: Mostly free
  6. 6. Question 1 Read the title and description. Write down 3-4 questions you expect will be answered on this page. Don’t peek!
  7. 7. Feedback on needs, not grammar and typos
  8. 8. Feedback on needs, not grammar and typos
  9. 9. • What equipment is supported? • Can this service be used on mobile devices? • How can I know that it is safe? • What is the flow like, step-by- step, when setting up a call? • How much does it cost? • How do you embed/link this on your own site?
  10. 10. User testing can be
 easy & affordable.
  11. 11. Why do
 user testing?
  12. 12. User testing is
 real users solving real tasks.
  13. 13. Do you recognise this icon? 
 
 Survey with 4273 respondents, 2014 Mobile: 53%
 said no Desktop: 73%
 said no
  14. 14. …but I can’t see the football…
  15. 15. Colleagues
 are not your users.
  16. 16. Designers
 are not your users.
  17. 17. Developers
 are not your users.
  18. 18. 1 in 3
 problems found
 by experts are
 false alarms 1 in 2
 user problems
 are overlooked
 by experts
  19. 19. Only users can let you know whether it’s intuitive. But how many users?
  20. 20. 5
  21. 21. N (1-(1-L)n)
  22. 22. 0% 25% 50% 75% 100% Number of test subjects 0 3 6 9 12 15 Shareobservedproblems Small and frequent tests are better than bigger and infrequent tests
  23. 23. User testing • …it’s the only way to know if the design truly is “intuitive” • …you’re taking big risks when you’re not testing • …it improves communication with stakeholders
 And it’s cheaper than you think :)
  24. 24. #2 Information architecture • Time: 30 min+ • Price: Free / $99-199 per month
  25. 25. #3 First click. • Time: 30 min+ • Price: Free / $99-199 per month
  26. 26. Tasks.
  27. 27. Finding possible issues Customer service and receptionists What are people calling and asking about? Website analytics Where do users drop off? Where do they spend more time than expected? Search logs Are there things that people aren't able to find? Which words do they use?
  28. 28. You should test things that… • are done frequently or by many users • has serious consequences if users fail with their task • are hangups within the organization, but need to be determined if they're actual issues
  29. 29. What we want to find out: Is it clear to physicians that they can use Confrere for video appointments?
  30. 30. Our question ≠
 User task
  31. 31. Open & explorative What would you do to find a video calling service that’s suitable for physicians? Specific & measurable Could you find the page which explains how Confrere is HIPAA compliant? Partially open Go to confrere.com. What would you do to figure out whether this is suitable for physicians?
  32. 32. Open & explorative • If you want to find more problems • It’s ok to give different tasks to different people • More realistic, but also more time- consuming Specific & measurable • When you know there’s a problem, but you want to figure out why • Easier to measure and compare • Can be leading Partially open
  33. 33. Phrasing the tasks • Don’t use the exact same phrases as on the website • Give the users a problem to solve, rather than asking them to “find” something • Adapt the tasks to each person, to make them more realistic
  34. 34. What do we want to know? “Is it clear to physicians that they can use Confrere for video appointments?”
 How do we phrase it? “Go to confrere.com. What would you do to figure out whether this is suitable for physicians?”
  35. 35. Intro • User’s rights and informed consent • Introductory questions / warm-up
 Tasks • The user tries to solve this, while the moderator sits next to them
 Closing • General feedback • Thanking the user Typical duration
 15-60 min
  36. 36. Time schedule • You’ll always underestimate how long the tasks take • Begin with 2-5 tasks, and do a trial test • Prioritize the tasks, in case you don't have time to get to all of them • Make sure you have time to test the equipment • Breaks between each test: At least 15 minutes of break time per hour!
  37. 37. Moderating.
  38. 38. Help the person to relax Make it easier for observers to understand Make sure we follow the test plan
  39. 39. Help the person to relax.
  40. 40. “Would you like something to drink?”
  41. 41. Checklist before beginning • Welcome the person, help them feel comfortable • Go through rights and consent form, make it clear that we’re testing the system, not you • Explain the “think out loud”-method • Ask if the person has any questions before you start • Remember to turn on the recording!
  42. 42. Help the person think out loud
  43. 43. What are you thinking about?
  44. 44. I’m looking for the menu… Help the user think out loud and ask open questions
  45. 45. What happens if I click here? What do you think might happen if you click there? Make sure you’re not helping the user
  46. 46. How to help someone think out loud • What are you looking for now? • What are you thinking about now? • What did you expect would happen just now? • …but don’t interrupt
  47. 47. How do you lead a test, without leading the user?
  48. 48. I’ve made the design for the new website and I’d love to hear what you think! The team that designed the site needs help figuring out if it's really working.
  49. 49. You’re really good at this! You’re solving all the tasks! Thank you! Let’s move on
 to the next task.
  50. 50. It would have gone so much faster if you had just clicked the menu-button. There's really big room for improvement here. Thank you for helping us identify this!
  51. 51. Sigh…
  52. 52. The user
 is there to help you
  53. 53. #3 Get out. • Time: 1-4 hours • Price: Snacks
  54. 54. It’s this easy:
  55. 55. (Translated by Google)
  56. 56. What did you find?
  57. 57. Analysis in 1-2-3 Is there anything you observed… …across several users on the same task?? …across several tasks? You start doing the analysis between each individual test.
  58. 58. Observation
 and notes.
  59. 59. Don’t listen. Observe.
  60. 60. Rationalization 
 Michael S. Gazzaniga: “The Interpreter” http://youtu.be/mJKloz2vwlc
  61. 61. Differentiate between… Observation What the person did and said during the task Interpretation Why you believe the person did what they did Improvements What you believe could be a solution to the problem
  62. 62. Example Observation • The user did not click the “Buy”-button Interpretations • The button should be placed elsewhere • The button has the wrong color or shape • The microcopy “buy” sounds too committing • The user didn’t think they really were supposed to “buy” since it’s just a test
  63. 63. Quantitative Qualitative Self-reported Observed Surveys A/B-testing User testing Interviews Focus groups Field studies Google Analytics Key word analysis “Was this helpful?” Crawling
 social media
  64. 64. Read this!
  65. 65. How to take notes? • Remember the purpose of the task - don’t write a novel on what each person does • Note which words the user is using • Remember, it's mostly about what they're actually doing and saying while they're doing it
  66. 66. Paper sketch Paper prototype Wireframes Sketch / Photoshop InvisionApp HTML prototype without styling HTML&CSS&JS prototype Interactive Flat Rough Detailed Functional Implemented Text
  67. 67. #5 Scrolling, clicking and hovering. • Time: Done in 30 min • Price: Free (e.g. Hotjar + Google Analytics)
  68. 68. #5 Was this helpful? • Time: 15 min+ • Price: Often free
  69. 69. #4 Lab test on a budget. • Time: 1 day for planning and recruiting, 1 day for the actual test • Price: Rewarding users
  70. 70. (Translated by Google)
  71. 71. “Hugging”
  72. 72. If you need observers: Test machine Observation + Tool for
 screen sharing+
  73. 73. Informed consent • They can opt-out any time • What data are you gathering and why • For what and when will you use it • When will you delete it • Check local rules/laws KONTAKT OSS +47 22 42 46 42 kontakt@netliferesearch.com BESØK OSS Stenersgaten 8, Oslo http://netliferesearch.com/ Dine rettigheter som testperson 1. Det er ikke du som testes, men løsningen. Det er derfor ingen krav til prestasjoner eller ferdigheter. 2. Det er frivillig å delta og du kan avbryte testen når som helst 3. Det blir sendt bilde av skjermen og testsituasjonen til et annet rom 4. Din anonymitet sikres ved at det ikke brukes navn eller annen informasjon som kan koble personen til notatene 5. Opptak av skjermbildet vil kunne bli vist i møter eller lignende etter testen er ferdig, men det vil ikke være fokus på hvem du er. Videoene vil kun være til internt bruk i prosjektet og vil slettes etter tre måneder 6. Du skal behandles høflig og med respekt Jeg bekrefter å ha fått den informasjon som er angitt ovenfor og at jeg gir tillatelse til at opptak av skjermbildet kan bli vist til andre. _________________________ _________________________ UNDERSKRIFT DATO KUNDE: PROSJEKT:
  74. 74. 20-30 min break between each test
  75. 75. When do you need observers? • If you need to align stakeholders • In general, the more observers, the better, because different people notice different things • Bring your teammates from different disciplines (e.g. developers) • Observers should stick to observing, and not introduce themselves to the users
  76. 76. Analysis and prioritisation.
  77. 77. The analysis must be done as soon as possible, by the people who did the user test.
  78. 78. Which of our findings or observations should be turned into to-dos? What’s relevant for our further work?
  79. 79. Documenting and sharing your findings
  80. 80. As a tool for your team
  81. 81. Put screenshots in InvisionApp in advance Add observations between (or during) each test
  82. 82. Connect InvisionApp to Slack, and your teammates can follow along
  83. 83. Vise alvoretWake-up calls
  84. 84. Innhold
  85. 85. A highlights video usually does the trick.
  86. 86. Prioritizing number of users consequences other sources cost
  87. 87. Make sure something happens with your findings • Each (prioritised) usability issue needs a to-do • Each to-do must be assigned to a specific person
  88. 88. Do it together!
  89. 89. Summarizing and prioritizing in Google Sheets or AirTable
  90. 90. Recruiting.
  91. 91. Selection criteria • Diversity • Language skills • ICT skills • Target group criteria • Background knowledge or experience with your service
 Exclusion criteria • People with too much knowledge about your service • People working with development, design, web etc Stricter criteria
 =
 Harder to recruit
  92. 92. Recruiting tips • Friends of friends, social media, newsletters • Go to where they are • Run it outside of regular work hours • Be creative about rewards • Lower your standards
  93. 93. Avoiding drop-outs • Send e-mails or texts with very clear instructions for how to get there, sign them personally and add your phone number • Send a text the day before or the same day to remind them • Recruit one more person than you think you actually need
  94. 94. 7 methods: 1. Micro testing content 2. Information architecture 3. First click 4. Get out 5. Lab test on a budget 6. Scrolling, clicking and hovering 7. “Was this helpful?” confrere.com/ usertest
  95. 95. 7 methods: 1.Micro testing content 2.Information architecture 3. First click 4. Get out 5.Lab test on a budget 6. Scrolling, clicking and hovering 7. “Was this helpful?” confrere.com/ usertest
  96. 96. 7 methods: 1. Micro testing content 2. Information architecture 3. First click 4.Get out 5. Lab test on a budget 6.Scrolling, clicking and hovering 7.“Was this helpful?” confrere.com/ usertest
  97. 97. 7 methods: 1. Micro testing content 2. Information architecture 3. First click 4.Get out 5.Lab test on a budget 6. Scrolling, clicking and hovering 7. “Was this helpful?” confrere.com/ usertest
  98. 98. You learn so much more by meeting people face-to-face!
  99. 99. Thank you! confrere.com/usertest Ida Aalen Chief Product Officer confrere.com

×