Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Usability testing / Nearly everything you need to know to get started

924 vues

Publié le

Intro to usability testing with emphasis on getting stakeholder / client buy-in and efficiency or testing and reporting.

Publié dans : Technologie
  • Soyez le premier à commenter

Usability testing / Nearly everything you need to know to get started

  1. 1. Usability testing Nearly everything you need to know to get started Rebecca Destello
  2. 2. Rebecca Destello •  Restaurant manager •  Front-end web developer •  User researcher •  Affiliate lecturer, UW HCDE •  Chair of Puget Sound PSSIGCHI •  Manager, Research & Insights, Atlas Informatics R. Destello | Usability Testing
  3. 3. Atlas Recall is a searchable photographic memory for your digital life — if you see it, you can search it and share with anyone, from anywhere. atlas.co R. Destello | Usability Testing
  4. 4. Are you experienced? R. Destello | Usability Testing
  5. 5. R. Destello | Usability Testing Tonight’s lecture What you’ll need to know to do this as a professional
  6. 6. R. Destello | Usability Testing Why do we do usability testing?
  7. 7. Because there is a science to how people use things. R. Destello | Usability Testing
  8. 8. usability testing: the flow R. Destello | Usability Testing Identify problems in the system Create a study Recruit and schedule target users Test with participants Analyze + Synthesize Report out Evangelize findings! Repeat
  9. 9. The one book you need: Handbook of Usability Testing by Jeffery Rubin and Dana Chisnell R. Destello | Usability Testing
  10. 10. Planning R. Destello | Usability Testing
  11. 11. •  Identify key stakeholders •  Scope and priorities •  Identify & recruit target users •  Establish a timeline •  Identify usability concerns planning a usability study R. Destello | Usability Testing
  12. 12. It is critical to get key players involved up-front •  Project sponsors •  Product owners •  People who will have to act on the findings identify key stakeholders R. Destello | Usability Testing
  13. 13. R. Destello | Usability Testing How do we generate support for usability testing?
  14. 14. 1. Start running studies! 2. Get people to observations 3. Always be prepared with talking points; include findings from other studies that made a critical difference 4. Be a good team player - be willing to compromise and negotiate 5. Map usability planning to the company and product(s) 5 steps to getting buy-in R. Destello | Usability Testing
  15. 15. Think about: •  What is the political landscape? How much buy-in do you have from the stakeholders? •  Has usability been done before? •  What questions are you trying to answer? •  What is your budget? •  What time constraints are you working with? scope and priorities R. Destello | Usability Testing
  16. 16. Tips for identifying areas of concern: •  Ask your stakeholders •  What do you want to learn from this test? •  What are your greatest fears about this product? •  Are there different points of view in the group about this issue? •  What are you willing to change as a result of this test? •  Mine customer feedback •  Perform a task analysis •  Evaluate the product (heuristic evaluation or cog. walkthrough) identifying usability concerns R. Destello | Usability Testing
  17. 17. ! Bad recruiting is bad testing R. Destello | Usability Testing
  18. 18. •  Find your target user by asking your client to add a survey to their website •  Use Craigslist (with caution) •  Ping your social networks •  Go where your users might congregate And, write a good screener… tips to find your target user R. Destello | Usability Testing
  19. 19. The screening questionnaire needs to be •  Specific, inclusive of all necessary criteria •  Contain boundary conditions •  Contain exclusion criteria May need to stand alone •  May be passed off to others, firm, colleague Can be lengthy •  10 pages •  28 questions screeners R. Destello | Usability Testing
  20. 20. Tips for writing a good screener •  Ask the elimination questions first •  Eliminate conflicts of interest •  Screen for experience •  Recruit based on behavior and attitudes •  Screen out “professional” users •  Write a good screener (bit.ly/uxmattersrecruiting) screeners R. Destello | Usability Testing
  21. 21. usability test participant incentives Why should you give an incentive? •  Participants are helping you by giving up their time •  Participants will leave with a positive attitude •  Participants will tell their friends (larger pool of participants for future studies) •  Best way to get qualified, representative participants R. Destello | Usability Testing
  22. 22. usability test participant incentives What should the incentive be? •  Incentives are determined based on the client / budget •  Example: –  Microsoft gives out free software –  Boeing gives out movie tickets –  Chocolate (at the very least) How much should an incentive be? •  Based on budget and target audience R. Destello | Usability Testing
  23. 23. Your timeline will be determined by •  The complexity of the product •  The broadness of the scope of the test •  What set up will be needed (facilities, travel, special equip, etc.) •  Level of difficulty in recruiting the target users •  # of participants needed •  How formal the report out will need to be •  Related deadlines (design, dev, and even holidays!) establish a timeline R. Destello | Usability Testing
  24. 24. Conducting R. Destello | Usability Testing
  25. 25. what’s in the usability study? ü  Product or assets to test ü  Participants (5 works if doing iterative testing, if only 1 user group) ü  Moderator script with scenarios and tasks ü  Pre/Post task questionnaires ü  Checklists for study setup ü  Participant documents (Informed Consent, NDAs, Questionnaires) ü  Data collection tools ü  Participant incentives R. Destello | Usability Testing
  26. 26. •  Greet and make participant comfortable •  Explain study, setup, and complete forms •  Explain Thinking Aloud Protocol •  Remind them they are not being tested •  Begin tasks •  Wrap-up •  Questionnaire, if applicable •  Final Comments/Questions •  Thank them profusely •  Provide gratuity •  Parking validation, if applicable testing framework R. Destello | Usability Testing
  27. 27. what about scheduling? Allow time during the study: •  Time for test (30 minutes - 1 hour) •  Time in between (1 hour between participants) –  Setup for the next study, check in with observers, clean up notes, start examining data while it’s fresh R. Destello | Usability Testing E X A M P L E
  28. 28. stay organized with a checklist Why use a checklist? •  Checklists are good, reusable tools for any usability study •  Checklists outline all things that must be completed before, during, and after study •  Helps ensure study consistency and increases internal validity of the study from day 1! Key Takeaway: Even if you don’t think you’ll need it, create a checklist of all the things you need to do for study (you’ll be surprised what you forget!) R. Destello | Usability Testing
  29. 29. participant documents Before Testing •  Pre-Test Questionnaire (optional) •  Informed Consent/NDA (required) During Testing •  Post-task Questionnaires After Testing •  Post-Test Questionnaire R. Destello | Usability Testing
  30. 30. R. Destello | Usability Testing What’s the role of the moderator?
  31. 31. so what does the “moderator” do? ü  Interacts with the participant ü  Ensures the participant’s comfort ü  Sets the pace and tone of the study ü  Administers scenarios, tasks, and questionnaires ü  Handles prompting and working with the participant to understand the reasons behind problems. ü  Provides clarifications so note taker can capture data ü  Takes care of the participant! R. Destello | Usability Testing
  32. 32. why is moderating challenging? •  Goal is unbiased feedback •  Participants need to be comfortable, but the test situation is often unsettling •  You need to know what the participant is thinking, but you can’t read their minds R. Destello | Usability Testing
  33. 33. intervention: what are they doing? When and how to intervene during a test: •  In general, it’s preferable to let participants decide how much time to work on a task •  Some exceptions –  If they are getting very frustrated –  If you feel that you’ve got data on the task at hand and you want to keep going and get to other tasks –  If they’ve wandered so far down a bunny hole they are unable to proceed with the remaining tasks R. Destello | Usability Testing
  34. 34. prompting: guidelines When prompting, you want to… •  Get the user to elaborate on something that is unclear Without… •  Introducing bias •  Influencing the user’s behavior •  Taking over as the expert or primary speaker •  Unnerving the user through implication that she did something wrong R. Destello | Usability Testing
  35. 35. prompting: techniques Provide Acknowledgment Tokens at natural intervals •  Mm hmm, I see….. Focus on Questions •  How do you feel about that? •  Tell me what you are thinking… •  Tell me more about that… Repeat their word or phrase back to them as a question •  They say: That was confusing / You say: “confusing…?” Make the alternative explicit •  Was this task easy or difficult? R. Destello | Usability Testing
  36. 36. Keep in mind… •  People cannot reliably reflect on their own cognition •  Without video to playback questions like “what were you thinking when…” or “why do you think you…” you will find you often have missing / bad data •  Participants will struggle to come up with a believable rationale while being unaware of the true cause for their actions prompting: words of warning R. Destello | Usability Testing
  37. 37. participant questions Participants sometimes ask the moderator for help. Turn their question into a question. How would you respond to the following? •  Is that right? Did I do it right? •  Am I supposed to keep going? •  What should I do now? •  Should I click here? •  What is this? R. Destello | Usability Testing
  38. 38. The most important thing to remember about moderating is this: Being in a study is stressful for the participant – their comfort level and state of mind are vital. When people are stressed, their behavior/memory are impaired. R. Destello | Usability Testing
  39. 39. Things to keep in mind while moderating: •  Be patient •  Be excited about the process of discovery •  Be “small” in relationship with the participant •  Be as neutral as possible •  Be professional •  Be caring to the participant’s needs Important: The newer you are, the less you should talk. moderate with care R. Destello | Usability Testing
  40. 40. q  Does the moderator use relaxed body language? q  Does the moderator encourages/discourages responses? q  Does the moderator follow-up (when allowed) and dig into ambiguous statements or promising topics or ideas q  Does the moderator have any verbal or behavioral tics that would be distracting? q  Does the moderator provide excessive reinforcements? q  Is the moderator aware of expressions of support, sympathy, joy, anger, and other emotions? q  Does the moderator recover from miss-steps well. q  Does the moderator make the interviewee comfortable during the initial briefing and questions. q  Does the moderator clearly explain the limits of his/her ability to help the participant? q  Does the moderator follow protocols for prompting? q  Was the moderator sensitive to cultural/org. issues? CHECKYOURSELF beforeyouwreckyourself R. Destello | Usability Testing – Moderator Checklist credit: Chauncey Wilson
  41. 41. moderator script: the essentials Prepares participants for the session •  Describes what to expect (how long, what they’ll be doing, if they’ll be observed, if they’ll be videotaped) •  Gives think-aloud instructions •  Lets them know their rights Why is the script important? •  Ensures that the facilitator gives the same information to all the participants •  Stresses that the study is a test of the product, not of the participant R. Destello | Usability Testing
  42. 42. moderator script: the essentials How to write unbiased, non-leading usability tasks (at the 10,000 ft. level): First think: What is the user naturally trying to do at this step? Then – •  Make them short (hard to remember long tasks) •  Write tasks in the user’s natural language •  Avoid jargon •  Don’t use words found in the system R. Destello | Usability Testing
  43. 43. For a successful study… •  Make sure to know what data you are looking for (what questions are you trying to answer?) •  Define success criteria for each task •  Pilot the study at least once •  Do a quick “clean up” of notes after each session •  Have a plan for effective note-taking so you can spend less time analyzing data other considerations R. Destello | Usability Testing
  44. 44. Analyzing + Synthesizing R. Destello | Usability Testing
  45. 45. Data you can capture during the study: •  Pathways •  Verbal commentary •  Think Aloud Protocol (TAP) •  Time on task (only when not using TAP) •  Task completion •  And more data collection: available data Data can be gathered using: •  Handwriting •  Document •  Spreadsheet •  Usability software R. Destello | Usability Testing
  46. 46. data collection: DOs Do capture: •  Observations; what you see the participant do •  Quotes; what you hear the participant say •  Details; what specifically caused issues or prevented task completion R. Destello | Usability Testing
  47. 47. data collection: DON’Ts Don’t capture: •  Why you think they are doing what they are doing (if you aren’t sure, ask the participant) •  Broad generalizations that don’t contain a description - “She had a lot of trouble finding the link” won’t be helpful when you’re analyzing findings R. Destello | Usability Testing
  48. 48. Use codes to note instances of issues that you’re tracking Example: F = failure E = error message H = goes to help Keep an issues matrix and add to it after each participant. data collection: note-taking tools R. Destello | Usability Testing
  49. 49. data collection: note-taking tools R. Destello | Usability Testing
  50. 50. usability software: Morae R. Destello | Usability Testing
  51. 51. usability software: Silverback R. Destello | Usability Testing
  52. 52. Reporting + Evangelizing R. Destello | Usability Testing
  53. 53. •  Word of mouth and email •  Executive summary •  Formal report •  Presentation How you choose to communicate results can depend upon your audience, the culture, timeline, and the potential “Freak Out Factor” communicating results R. Destello | Usability Testing
  54. 54. •  Know your user (reader) – what’s important to them? •  Accentuate the positive, but deliver the news •  Report the facts (don’t overreach) •  Be thoughtful about word choice (opportunity vs. issue) •  When in doubt, recommend further testing •  Check grammar and spelling communicating results: tips R. Destello | Usability Testing
  55. 55. R. Destello | Usability Testing
  56. 56. In groups of 2, using a phone 1 person act as the participant, the other as the note taker and go to Nordstrom.com and complete the following tasks: 1.  You’d like to see the hot new items for this season. Where on the site would you browse what’s trending this fall for yourself? 2.  You want to buy a new shirt for your 5 year old niece. Where would you find a shirt that might work for her? Focus on problems in navigation and filter use. in-class exercise: Navigation R. Destello | Usability Testing
  57. 57. Don’t forget! •  Take notes (Where did the participant have trouble? What did they say? What worked?) •  Use prompts to keep them talking: “What are you thinking?” “Say more.” “Is that what you expected?” •  Don’t say much and listen closely in-class exercise: Navigation R. Destello | Usability Testing
  58. 58. in-class exercise: Navigation R. Destello | Usability Testing In groups of 2, using a phone 1 person act as the participant, the other as the note taker and go to Nordstrom.com and complete the following tasks: 1.  You’d like to see the hot new items for this season. Where on the site would you browse what’s trending this fall for yourself? SWITCH ROLES 2.  You want to buy a new shirt for your 5 year old niece. Where would you find a shirt that might work for her?
  59. 59. What was the experience like? in-class exercise: Navigation R. Destello | Usability Testing
  60. 60. Questions? RebeccaDestello@gmail.com