Innovative techniques to make Heuristic Evaluations and Expert Reviews more valuable for business and technical stakeholders

1. Expert Reviews
Using the VIMM Model to Communicate to Stakeholders
Michael Rawlins, Certified Usability Analyst
Presented to the Connecticut Chapter of Usability Professionals
August, 2009

2. About me...
Michael Rawlins
Director, User Experience, Open Solutions Inc.
•Certified Usability Analyst since 2003
•Led UCD teams at major Fortune 100 companies
•Performed 100’s of Expert Reviews
2

3. Power of the VIMM Model
3 Credited to Human Factors International

4. Definition of VIMM
Reducing:
Visual Clutter
Intellectual Complexity
Memory load (short & long term)
Motor load (and inconsistency)
Credited to Human Factors International
4

5. Key takeaways...
VIMM is as an organizing principle
VIMM to communicate findings
VIMM helps you establish a common language with a
stakeholder (leverage their mental model - their mindset)
5

6. Stakeholder Mindset...
6

7. Stakeholder Need...
Data that communicates the level of risk and exposure to
business failure
Easy to understand metrics and frameworks that can be
trended over time
Their agendas satisfied - often times they are not as
interested in learning a discipline like Usability
7

8. Definitions...
An Expert Review (ER) is an evaluation of the status of
an existing device, application or website. A single reviewer
(or team of reviewers) examines the user interface in a very
specific context or construct (example: apply a checklist of
generally accepted *heuristics)
*Heuristics are “rules of thumb” 8

9. ER’s Generate Quick Wins...
Planning tool - evaluate existing web site or application
Establish a common language - when preparing for a major
redesign
Support Usability Testing Cycle - identify critical issues
Post-Mortem Tracking Mechanism - post release assessment
tool to identify design additional issues
9

10. The typical ER
Identify heuristic violations (severity and impact)
Feature annotated screens illustrating issues
Outline of short & long term recommendations
Action plan, highlighting critical changes for improving
application or website
10

11. Simple ER
Checklist
Rating Scale
Comments area
11

12. More Structured ER
Categories
Applicability
Comments
12 Credited to Xerox

13. ER with a set of Heuristics
•Structure
•Visual Layout
•Text
•User Interface Metaphor
•Information Content
•User Assistance
•Navigation
•Consistency & Standards
•Mental Workload
•Graphics & Animation
•Errors
•Feedback
•Flexibility & Efficiency
•Initial Experience
•Accessibility
13 Credited to IBM

14. Communication is Key...
Position ER’s with your stakeholder as a Preventive
Measure - an assessment tool that helps mitigate risk of
costly usability issues
Provide positive and negative usability issues
(always lead with the positive)
Only a diagnostic tool - does not represent actual
behaviors of target users
14

15. Innovations...
15

16. Organize with VIMM Model
Credited to Human Factors International
16

17. Add Scorecards and Graphs
Scorecards
illustrate trends
30
22.5
15
7.5
2007 0
2008
2009
Structure
Design
Layout
Interaction
17 Scorecard Credited to Human Factors International

18. Add Impact Matrixes
Communicate the risk of not fixing
18

19. Create Action Plans
19

20. Add Risk Assessment
20

21. Rationalizing Findings...
http://www.measuringusability.com/problem_discovery.php#estp
ER’s cannot
be measured unless you
collaborate with multiple
reviewers.
Multiple reviewers often
results in multiple
findings.
This tool helps rationalize.
Credited to Jeff Sauro - Measuring Usability
Planning tool “How many reviewer will it take to find 85% of the problems”
21

22. Justifying Results...
Credited to Jeff Sauro - Measuring Usability
Increase Predictability: communicate to stakeholders that science is involved
22

23. Summary
Stakeholders need evidence that usability issues exist
ER’s capture evidence quickly & effectively
ER’s inspire collaboration with your peers
Innovate ER’s using VIMM model
Present to stakeholders using charts, graphs, to stakeholder
support decisions
23

24. Reading...
Jacob Nielson and Robert L. Mack (editors), Usability
Inspections Methods, John Wiley & Sons, 1994
Wendy Yee, (whitepaper) The Value of an Expert Review and
5-Step Review Checklist, Human Factors International, 2002
Usability toolkit
http://www.stcsig.org/usability/resources/toolkit/toolkit.html
24

25. questions...
(michael.rawlins@gmail.com)
25

26. thank you...
26