Contenu connexe Similaire à Exploring Usability Testing (20) Exploring Usability Testing1. TI
AM Tutorial
10/1/2013 8:30:00 AM
"Exploring Usability Testing"
Presented by:
Rob Sabourin
AmiBug.com
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Rob Sabourin
AmiBug.com
Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of
software development professionals. A well-respected member of the software engineering
community, Rob has managed, trained, mentored, and coached hundreds of top professionals
in the field. He frequently speaks at conferences and writes on software engineering, SQA,
testing, management, and internationalization.
3. Exploring Usability Testing
Robert Sabourin
President
AmiBug.Com, Inc.
Montreal, Canada
rsabourin@amibug.com
© 2013 Robert Sabourin EUT v1.1
1
Exploring Usability Testing
WELCOME
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
2
1
4. Exploring Usability Testing
• Robert Sabourin ,
Software Evangelist
• President
• AmiBug.Com Inc.
• Montreal, Quebec,
Canada
• rsabourin@amibug.com
• www.amibugshare.com
3
© 2013 Robert Sabourin EUT v1.1
Exploring Usability
Testing
• Pain points?
– What hurts?
– How Much?
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
4
2
5. Exploring Usability Testing
SOME PHILOSOPHY
5
© 2013 Robert Sabourin EUT v1.1
Conforming to
requirements
Suiting purpose
Delivering value
to stakeholders
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
6
3
6. Edsger W. Dijkstra
• “Program testing can be used to show the
presence of bugs, but never to show their
absence”
© 2013 Robert Sabourin EUT v1.1
7
Exploring Usability Testing
NON-FUNCTIONAL
TESTING
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
8
4
7. Non-Functional Testing
Quality Factors
Attributes
Characteristics
Reliability
Other “-ilities”
© 2013 Robert Sabourin EUT v1.1
9
Non-Functional Testing
Challenges
Goals
• Elicit
• Articulate
• Quantify
Subjective
• Needs or wants?
• Relative or absolute?
• What is good enough?
Tests
• Tricky to orchestrate
• Difficult to interpret results
• Challenging to baseline or regress
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
10
5
9. Quality Factors
13
© 2013 Robert Sabourin EUT v1.1
Product Backlog
Constraints
Tom Gilb, Competitive Engineering, suggests defining
Scale: "What is measured"
Meter: "How to measure (method)"
Target: "Level we're aiming for. Success"
Constraint: "Level we're seeking to avoid.
Failure"
Benchmark: "Where we are today"
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
14
7
10. Non-Functional Testing
© 2013 Robert Sabourin EUT v1.1
15
Exploring Usability Testing
NET PROMOTER SCORE
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
16
8
11. Net Promoter Score
17
© 2013 Robert Sabourin EUT v1.1
Net Promoter Score
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
18
9
12. Net Promoter Score
Response Scale
•11 values
•0 not at all likely
•10 extremely likely
19
© 2013 Robert Sabourin EUT v1.1
Net Promoter Score
Promoters
• respondents giving a 9 or 10 score
Passives
• respondents giving a 7 or 8 score
Detractors
• respondents giving a 0 to 6 score
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
20
10
13. Net Promoter Score
NPS
• The difference between the percentage
of Promoters and Detractors
• Is not expressed as a percentage
• Is an absolute number lying between
-100 and +100
• uTest suggests “any positive number is
good” (ref: TK Maxx Mobile Usability
Testing Results Report)
21
© 2013 Robert Sabourin EUT v1.1
Net Promoter Score
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
22
11
14. © 2013 Robert Sabourin EUT v1.1
23
Exploring Usability Testing
SYSTEM USABILITY SCALE
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
24
12
15. System Usability Scale
SUS - System Usability Scale
• Measures usability in terms of
effectiveness, efficiency and satisfaction
• SUS can be read as a percentage
• SUS can be used as a benchmarking
mechanism
• SUS can be used to compare the current
system to competitors’ systems
• SUS can be used to compare the current
system to future versions
© 2013 Robert Sabourin EUT v1.1
25
System Usability Scale
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
26
13
16. System Usability Scale
SUS Questions
1.
2.
3.
4.
I think that I would like to use this system frequently
I found the system unnecessarily complex
I thought the system was easy to use
I think that I would need the support of a technical
person to be able to use this system
5. I found the various functions in this system were well
integrated
6. I thought there was too much inconsistency in this
system
7. I would imagine that most people would learn to use
this system very quickly
8. I found the system very cumbersome to use
9. I felt very confident using the system
10. I needed to learn a lot of things before I could get
going with this system
27
© 2013 Robert Sabourin EUT v1.1
System Usability Scale
SUS Responses
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
28
14
17. System Usability Scale
Scoring SUS
• For odd items: subtract one from the user
response.
• For even-numbered items: subtract the
user responses from 5
• This scales all values from 0 to 4 (with
four being the most positive response).
• Add up the converted responses for each
user and multiply that total by 2.5. This
converts the range of possible values from
0 to 100 instead of from 0 to 40.
29
© 2013 Robert Sabourin EUT v1.1
System Usability Scale
Scoring SUS
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
30
15
19. Charter Statement
•
•
•
•
•
•
Statement of mission
Ties to purpose
Focuses work
Confirms understanding
Delineates scope
Analogy to test story
33
© 2013 Robert Sabourin EUT v1.1
Charter Statement
•
•
•
•
•
Short, terse
To the point
Inclusions …
Exclusions …
Limits …
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
34
17
20. eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1
35
eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
36
18
21. eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1
37
eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
38
19
22. Pairing Testers With …
Non Functional Testing
•Usability Gurus
•SMEs
•Human Factors
Experts
39
© 2013 Robert Sabourin EUT v1.1
Heuristics
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
40
20
24. Prototype Styles
•
•
•
•
•
•
•
•
•
•
•
•
Paper and pencil
Comic books style story boards
Wire frame models
Mock ups
Prototypes
Site maps
User flow
Wire flow
Story boards
Wire frame
Page description diagram
Functional specification
43
© 2013 Robert Sabourin EUT v1.1
Whiteboarding
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
44
22
27. Buying a Book
Usage Scenarios
49
© 2013 Robert Sabourin EUT v1.1
Story Board
Whiteboarding
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
50
25
29. Buying a Book
Usage Scenarios
© 2013 Robert Sabourin EUT v1.1
53
Slide 54
© 2013 Robert Sabourin
EUT v1.1
© 2013 Robert Sabourin
27
30. Slide 55
© 2013 Robert Sabourin
EUT v1.1
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
56
28
32. Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
59
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
60
30
33. Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
61
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
62
31
34. Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
63
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
64
32
35. Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
65
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
66
33
36. TOFT Testing
• Task Oriented
Functional Testing
– Can the user
accomplish useful
tasks correctly?
© 2013 Robert Sabourin EUT v1.1
Slide 67
Exploring Usability Testing
USABILITY HEURISTICS
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
68
34
37. Jakob Nielsen
• Usability heuristics
• Rules of thumb
• General principles
of user interface
design
• www.nngroup.com
69
© 2013 Robert Sabourin EUT v1.1
Usability Heuristics
• Visibility of system status
– Always keep users informed
– Provide appropriate feedback
– Respond in a reasonable timeframe
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
70
35
38. Usability Heuristics
• Match between system and the real
world
– Speak the users' language
– Use familiar concepts
– Follow a natural order
71
© 2013 Robert Sabourin EUT v1.1
Usability Heuristics
• User control and freedom
– Provide means to exit from unwanted states
– Provide undo and redo capabilities
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
72
36
39. Usability Heuristics
• Consistency and standards
– Same words mean same thing in different
contexts
– Follow environment conventions
73
© 2013 Robert Sabourin EUT v1.1
Usability Heuristics
• Error prevention
– Prevents problems from occurring
– Eliminate error-prone conditions
– Have a confirmation option
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
74
37
40. Usability Heuristics
• Recognition rather than recall
– Making objects visible
– Make instructions available
75
© 2013 Robert Sabourin EUT v1.1
Usability Heuristics
• Flexibility and efficiency of use
– Accelerators for the expert user
– Make common options adaptable
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
76
38
41. Usability Heuristics
• Aesthetic and minimalist design
– Dialogues should contain
relevant information
77
© 2013 Robert Sabourin EUT v1.1
Usability Heuristics
• Help users recognize, diagnose, and
recover from errors
– Use plain language for error messages
– Constructively suggest solutions
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
78
39
42. Usability Heuristics
• Help and documentation
–
–
–
–
Easy to search
Focused on the user's task
Offer concrete steps to be carried out
Short and to the point
© 2013 Robert Sabourin EUT v1.1
79
Exploring Usability Testing
USABILITY SEVERITY
LEVELS
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
80
40
43. Usability Severity
© 2013 Robert Sabourin EUT v1.1
81
Exploring Usability Testing
ISO 9421
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
82
41
44. Usability ISO 9421
Effectiveness
• the extent to which the intended goals of use are achieved
Efficiency
• the resources that have been expended to achieve the intended goals
Satisfaction
• the extent to which the user finds the use of the product acceptable
Safety
• can user be harmed
83
© 2013 Robert Sabourin EUT v1.1
Usability ISO 9421
Completeness
• the extent to which the system is complete in design and meets its desired
goals
Learn-ability
• Is the capability of a software product to enable the user to learn how to
use it
Convenience
• the extent to which the product fits into normal course of daily life
Ambiguity
• Confusing or vague usability. Misleading information that might also result
in providing no usability at all
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
84
42
45. Exploring Usability Testing
USABILITY TEST
FRAMEWORK
© 2013 Robert Sabourin EUT v1.1
85
Usability Test Framework
Basic Usability Test Steps
• Select representative users
• Complete defined set of tasks
• Facilitator observes user behaviour
• Interview user after task is completed
• User is asked to think aloud (recorded)
• Review findings to identify concerns
• Generate actionable recommendations
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
86
43
46. Usability Test Framework
User groups
• Select 5 or 6 users for each group
• All should attempt the same tasks
• Complex applications may need many groups
• Groups can be based
• Domain (subject matter) expertise
• Experience level (task or professional)
• Solution familiarity
© 2013 Robert Sabourin EUT v1.1
87
Usability Test Framework
Where to perform usability tests?
• Control lab
• Conference room with recording
equipment
• Work area with
• Local
• Distributed with local observer
• Distributed without local observer
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
88
44
47. Usability Test Framework
Prepare Usability Scenario
• Identify project usability test objectives
• Select relevant user tasks
• Keep user instructions short and to the point
• Do not over complicate instructions
• Use the language of the user not the
language of the system
• Make task parameters specific
• Use a logical flow
89
© 2013 Robert Sabourin EUT v1.1
Usability Test Framework
Running a test
•Let participants make mistakes
•Take note of questions
•Take note of decisions points
•Take note of alternative
selections
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
90
45
48. Usability Test Framework
Think Aloud
Approach
Retrospective
Think Aloud
• the user
verbalizes actions
and decisions
• user verbalizes
questions
concerns or
ambiguities
• records and eye
tracking software
record event
• review with user
and ask them to
explain their
actions
91
© 2013 Robert Sabourin EUT v1.1
Usability Test Framework
Reporting Findings
•
•
•
•
List of usability concerns
Summary of user questions
Description of users behaviour
Tasks of concern
• Too long
• Awkward
• Unsuccessful
• Time to complete
• Feedback
• Expected
• Unexpected
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
92
46
49. Usability Test Framework
Measures and metrics
•Effectiveness
•Efficiency
•Satisfaction
•Error frequency
•Memorability
93
© 2013 Robert Sabourin EUT v1.1
Usability Test Framework
Measures and metrics
• Successful task completion
• Error rates
• Time on task
• Subjective comments
• Likes
• Dislikes
• Recommendations
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
94
47
50. Usability Test Framework
Facilitating a test
• Make participant comfortable with environment and
technology
• Give participants a change to ask questions
• Explain process
• Ask participants about the amount of experience they
have with the technology
• Ask participants about the amount of experience they
have with the domain
• Review task description with participant
• Explain facilitators involvement
© 2013 Robert Sabourin EUT v1.1
95
Exploring Usability Testing
FORMS AND WORKFLOWS
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
96
48
51. Form Design Principles
•
•
•
•
Minimize pain
Illuminate path
Consider context
Ensure consistent communication
Luke Wroblewski
Web Form Design
97
© 2013 Robert Sabourin EUT v1.1
Eye Tracking
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
98
49
52. Form Usability
•
•
•
•
•
•
Number and location of errors
Severity of errors
Completion rates
Time to complete forms
Satisfaction scores
Subjective comments
99
© 2013 Robert Sabourin EUT v1.1
Awkward Form Path
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
100
50
53. Clear Form Path
101
© 2013 Robert Sabourin EUT v1.1
Form Usability
• Eye tracking
– What people looked at
– Number of eye fixations
– Length of eye fixations
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
102
51
54. Form Usability Checklist
Any unnecessary questions
Succinct questions
Sincere voice
Natural language
Logical groups of information
Structure form as a conversation
Ask optional questions separately
© 2013 Robert Sabourin EUT v1.1
103
Form Usability Checklist
Name the form
Name sections of the form
Clear scan line
Spacing between sections
Minimal distractions
Start page guidance for long forms
Indicate progress
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
104
52
55. Progress Indication
105
© 2013 Robert Sabourin EUT v1.1
Input Validation
Suggest valid answers
Illustrate valid format
Validate syntax after user enters data
Indicate limits or ranges of inputs
Provide smart defaults
Hide unneeded controls
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
106
53
56. Error Messages
Provide clear error messages
Indicate if error blocks completion
Guide users to resolve error
Provide visual emphasis of error
Use red text and icons for errors
Indicate successful completion
© 2013 Robert Sabourin EUT v1.1
107
Exploring Usability Testing
DO IT YOURSELF
USABILITY TESTING
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
108
54
57. Do it yourself
109
© 2013 Robert Sabourin EUT v1.1
Do it yourself
A morning a month
Get insights from users
Watch real users
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
110
55
58. Do it yourself
Recruit diversely
Find important problems
Improve design
111
© 2013 Robert Sabourin EUT v1.1
Do it yourself
Three testers
Test on site
Observe shared screens
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
112
56
59. Do it yourself
Observers take notes
User questions recorded
Action decided at lunch
© 2013 Robert Sabourin EUT v1.1
113
Exploring Usability Testing
SOME MOBILE USABILITY
CONCERNS
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
114
57
60. Mobile Usability
Difficulties
•Small Screens
•Limited bandwidth
•Fingers
•No Flash Like GUI
115
© 2013 Robert Sabourin EUT v1.1
Mobile Usability
Risks
•Bad first impression
•Operator errors
•Negative feedback / reviews
•Competitor fills gap
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
116
58
61. Mobile Usability
Guidelines
•
•
•
•
•
Important stuff on top
Easy to find stuff
Single column layout
Minimize navigation
Light data
117
© 2013 Robert Sabourin EUT v1.1
Mobile Usability
Guidelines
•
•
•
•
•
•
Finger taps
Avoid fancy web gui technologies
Minimize forms
Minimize feature sets
Separate mobile web resources
Fast
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
118
59
62. Mobile Usability
Mobile Usability Test
•In the lab
•Beta
•Crowd source
119
© 2013 Robert Sabourin EUT v1.1
Mobile Usability
Mobile Usability Test
• Vary and control
• Technologically
• OS
• Browser
• Device
• Carrier
• Bandwidth
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
120
60
63. Mobile Usability
Vary and control
• Geographically
• Continent
• Country
• City
• Language
• Locale
121
© 2013 Robert Sabourin EUT v1.1
Mobile Usability
Vary and control
• Demographically
• Age
• Gender
• Education
• Employment
• Industry
• Computer savvy
• Domain expertise
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
122
61
65. Lean UX
Lean Principles
• Cross functional teams
• Small teams
• Dedicated teams
• Collocated teams
• Progress equals Outcomes
• Problem focused teams
125
© 2013 Robert Sabourin EUT v1.1
Lean UX
Lean Principles
• Remove waste
• Small batch size
• Continuous discovery
• GOOB Get out of the building
• Shared understanding
• No Rock Stars Gurus or Ninjas
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
126
63
66. Lean UX
Lean Principles
• Externalize Your Work
• Making over Analysis
• Learning over Growth
• Permission to Fail
• Getting out of the Deliverables
Business
127
© 2013 Robert Sabourin EUT v1.1
Lean UX
Assumptions
Hypotheses
Outcomes
Personas
Features
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
128
64
67. Lean UX
Assumptions
•Business
•User
129
© 2013 Robert Sabourin EUT v1.1
Lean UX
Hypotheses
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
We believe that
creating this
experience for these
personas will achieve
this outcome. We will
know this to be true
when we see this
feedback or measure.
130
65
68. Lean UX
Outcomes
More business
More referrals
More fun
131
© 2013 Robert Sabourin EUT v1.1
Lean UX
Personas
• Sketch and Name
• Demographic
• Pain points & needs
• Potential solutions
© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin
132
66