During this webinar, Managing Director Robert Schumacher discussed the recently released Electronic Health Record (EHR) Usability Protocol from the National Institute of Standards and Technology. User Centric has been heavily involved in ensuring EHRs are evaluated for usability. This webinar provides insights and tactics to help you: • Position usability next to other methods of evaluating electronic health records • Show the application of some key human performance principles to EHR design • Discuss the emerging methods of expert review and validation testing for evaluating and measuring user performance of EHRs (With audio: http://vimeo.com/36793907) http://www.usercentric.com/healthcare

1. Presenter:
Robert Schumacher, PhD.
Managing Director
User Centric, Inc
Moderator:
Pamela Stoffregen-Gay
Sr. Marketing Manager
User Centric, Inc
#uxlunch

2. UAT
Aesthetic Functional
Attitude & Opinion
(These are not behavior)

3. SAY
≠
DO

4. A couple of caveats…
4

5. User Experience Encompasses Usability
User
Experience
(UX)
Usability
5

6. Utility (Usefulness)
Useful Does Not Mean Usable
Usable Does Not Mean Useful
Usability

7. yü-zə-ˈbí lə-tē
-
from O.Fr. usable (1311), from user (see use). Not a common word before c.1840.
The extent to which a product can be used
by specified users to achieve specified
goals with effectiveness, efficiency, and
satisfaction in a specified context of use
ISO definition

11. Psychology
Cognitive
Human Factors
Engineering::Physics
as
Human Factors::Psychology
Usability

12. Methods of measuring Principles of behavior that
behavior inform design
 Reaction time  Performance  Memory
 Errors  Cognitive workload  Motor capabilities
 Eye tracking  Attention  Signal detection
 …  Psycholinguistics
 Choice theory
 …
There is a rich history of Knowledge of human
constructing studies to capabilities embodied in user
understand and explain interface design can optimize
human performance user performance

13. Methods of measuring Principles of behavior that
behavior inform design
 Reaction time  Performance  Memory
 Errors  Cognitive workload  Motor capabilities
 Eye tracking  Attention  Signal detection
 …  Psycholinguistics
 Choice theory
 …
There is a rich history of Knowledge of human
constructing studies to capabilities embodied in user
understand and explain interface design can optimize
human performance user performance

14. A Negative A Positive statement,
statement, with No with Yes (affirmative)
(negative) response response
Yes or No?
The circle isis above the star
The circle not above the star

15. Negative statement, Positive statement,
with No (negative) with Yes (affirmative)
response response
The circle is not The circle is
above the star above the star
• Positive statements evoke faster responses than negative statements
• Yes responses are much faster than No responses (~ 500 msec)

16. Positive
Logically
Negative
Equivalent
Affirmative Negative
Response Response
Different phrasings are used when you want different responses
- Large variability in cognitive processing times
- Significant impact on error

17. Statement Type Use When
 True/Affirmative Fast, easy, low-cost-to-user outcome,
confirmation only
 False/Affirmative Need user to think about the response, high
cost to use
 True/Negative Should almost never use
 False/Negative Never use, unless trying to deceive user
e.g., Opt out response

18. Methods of measuring Principles of behavior that
behavior inform design
 Reaction time  Performance  Memory
 Errors  Cognitive workload  Motor capabilities
 Signal detection
 Eye tracking  Attention
 Psycholinguistics
 …  Choice theory
 …
There is a rich history of Knowledge of human
constructing studies to capabilities embodied in user
understand and explain human interface design can optimize
performance user performance

20. Ready…

22. When was the patient’s
highest platelet count?

23. Human performance depends on
how well tools are designed
Change interface
=
Change performance

24. How do we measure that
change?
How do we understand
users, tasks and
environments?

25. Usability / UX Methods
Access
Testing
Card sorting
User Experience Participatory design
(UX) Research
Methods Focus groups
Eye tracking
Ethnographic
Cognitive
research
walkthrough
Usability Validation testing
Evaluation
Methods
Formative usability
Heuristic evaluation testing
KLM-
GOMS
Surveys
Do not involve users Involve users
25

26. Eye Tracking
Cognitive Formative Usability
Modeling Testing
Validation Testing
Focus Group
Expert
Review
The trick is to know when to use each tool
;
; ;

27. Describe Improve
FieldLab Environment/Tasks
Users
DevelopmentDeployed
Users
QualitativeQuantitative
AbsoluteRelative
Tasks/Users
Users
NovicesExperts
Walk Up And UseRequire Training
Tasks/Environment Attitudinal Behavioral Users

28. Fully
Deployed
Eye tracking
Validation
testing
Expert Evaluation
Describe
Improve
Cognitive Walkthrough
Formative
usability
Heuristic Evaluation testing
Focus groups
Observational Participatory
research design Card sorting
Early
Development

29. Behavior
Eye tracking Validation
Validation
testing
testing
Formative
Formative Heuristic Evaluation
usability
usability
testing
testing Expert Evaluation
Expert Evaluation
Quantitative
Cognitive Walkthrough
Qualitative
Ethnographic
research
Participatory
design
Focus groups
Card sorting
Surveys
Attitude

30. Collaborators: Supported & Improved by:
- Svetlana Lowry, NIST - SHARPC - Jiajie Zhang’s Lab
- Emily Patterson, OSU and others…
- M. Chris Gibbons, Johns Hopkins
- Robert North, Human Centered Technologies

31.  Apply existing established human
factors methods in the technical
evaluation (i.e., measurement) of EHR
Usability
 Focus on critical aspects of usability
(related to patient safety)
 Provide detailed systematic steps for
conducting measurement usability
validation studies
 Provide a framework for reporting
usability findings
 Not a policy document or framework

34. Who are the users?
What is the user’s environment?
What are their tasks?
How well does the application do the
tasks in the environment for the users?
What mistakes might users make?
How has the application been designed to mitigate mistakes and
optimize usability?

35. An evaluation of the critical components of the user interface
by executing various use case scenarios.
Clinical subject matter experts together with usability / human
factors experts conduct a structured evaluation of the user
interface.

36. 

 image
• In order to assure independence in the review process, each expert reviews the
user interface on his/her own and produces a set of ratings and findings.
• Once all experts have completed the evaluations, a lead expert consolidates all
the findings into a single report.


37. • •
•
•
•
•
•
•

38. •
•
•
•
•
•
•

39. •
•
•
•
•
•

40. •
•
•

41. •
•
•

42. •
•
•
•

43. •
•

44. •
•
•
•
•
•
•
•

45. •
o
o
o
•

46. Common
Industry Format (CIF)
• Specifies format for reporting
results of a summative
usability evaluation
• CIF approved as ISO
standard ISO/IEC 25062:2006
• CIF format was agreed by the
IUSR working group:
IBM, Microsoft, HP, Boeing, U
S West, Kodak
• See NIST IR 7742

47. •
o
o
•
•

48. Presenter:
Robert Schumacher, PhD.
Managing Director
User Centric, Inc
rschumacher@usercentric.com
User Centric is a global user experience
research and design firm.
We believe experiences matter.TM
www.usercentric.com
Oakbrook Terrace  Chicago  Atlanta
630-320-3900
Connect with us: @UserCentricInc
Webinar Schedule (www.usercentric.com/webinars):
Behaviors First, Colors Second: The Importance of Interaction
Design
Thursday, March 8
Medical Devices: Catching up to the FDA
Tuesday, March 27

49. Presenter:
Robert Schumacher, PhD.
Managing Director
User Centric, Inc
rschumacher@usercentric.com
User Centric is a global user experience
research and design firm.
We believe experiences matter.TM
www.usercentric.com
Oakbrook Terrace  Chicago  Atlanta
630-320-3900
Connect with us: @UserCentricInc
Webinar Schedule (www.usercentric.com/webinars):
Behaviors First, Colors Second: The Importance of Interaction
Design
Thursday, March 8
Medical Devices: Catching up to the FDA
Tuesday, March 27