This document discusses user errors with medical devices and how to conduct a Failure Modes and Effects Analysis (FMEA) focused on user errors. It recommends expanding the traditional FMEA process to include a task analysis to understand user behaviors and identify potential use errors. A 12-step process is outlined for conducting a use error-focused FMEA, including forming a team, conducting a task analysis, brainstorming potential use errors, analyzing effects and assigning severity and probability ratings to derive risk levels and priorities. The goal is to identify design changes that can eliminate or reduce risks from use errors.

1. There is no such thing as User Error
PRESENTER:
Gavin Lew Korey Johnson
Managing Director Associate Director
User Centric, Inc. User Centric, Inc.
MODERATOR: #uxlunch
Pamela Stoffregen-Gay
Sr. Marketing Mgr. Join the discussion!
User Centric, Inc. @UserCentricInc

2. Company Overview: Snapshot Profile
• Founded in 1999; privately held firm with over 50 full time UX
research and design consultants
• Most have graduate degrees in psychology, human factors /
ergonomics, ethnography, design and human computer
interaction
• In 2007, 2008, 2009, 2010, 2011 User Centric was included on the
Inc. 5000 List of the fastest growing companies in the U.S.
• UC is arguably the strongest user experience firm in the US
• Healthcare (medical devices and HIT: EHRs)
© 2011 – Proprietary and Confidential 2

3. Introduction: In Search of Use Errors
© July 31, 2012 – Proprietary and Confidential 3

4. Introduction
No One Wants to Make a Device that Causes Errors
© July 31, 2012 – Proprietary and Confidential 4

5. Introduction
Usual Suspects When Use Errors Are Committed
 Designers cannot ever think that being “highly
trained”
solves a medical device that is poorly designed
© July 31, 2012 – Proprietary and Confidential 5

6. Introduction
Increased Vulnerability
 While it is true that HCPs are highly educated, often the tasks
performed puts them into an “automatic mode” which makes
them more vulnerable to use errors*
* Guenter, et al (2008). Enteral Feeding Misconnections: A Consortium Position
Statement, The Joint Commission Journal on Quality and Patient Safety, 34(5):
282-292.
© July 31, 2012 – Proprietary and Confidential 6

7. Introduction
Don’t Forget Other Users…
 Originally, pharmacists were not considered to be a relevant
user group for use errors, but Flynn, et al. (2003) estimated
51.5 million errors occur per year (4 per day per pharmacy)
 Moreover, a human factors-related recall with pharmacists as
the user as recent as September 23, 2011
© July 31, 2012 – Proprietary and Confidential 7

8. Introduction
What About Patients?
 Especially when a medical device used by HCPs is
“repurposed” for use with patients…!
© July 31, 2012 – Proprietary and Confidential 8

9. Introduction
Patients…
 Are incredibly diverse
– Literacy, education, experience
 Receive a wide range of training
– Skewed towards no training
 Have different perspective and context
 Live in varied environments
– i.e., uncontrolled environments
– Consider: dogs, younger children
 Do research long enough and patients resemble…
© July 31, 2012 – Proprietary and Confidential 9

10. Introduction
A Truly Diverse User Group
© July 31, 2012 – Proprietary and Confidential 10

11. Introduction
Must Users Really Shoulder the Blame?
© July 31, 2012 – Proprietary and Confidential 11

12. Introduction
Definition of a Use Error
 Use errors are made by humans and can often be
attributed to the design itself
 “Use” as opposed to “human error” or “user error”
removes blame from the user as the cause of the failure
 Targeting design deficiencies of the design itself links
remedy to design changes (earlier the better)
 Often, we focus on the device itself, but
– Interaction with other devices, connections (things) and
– Instructions also contribute to use errors
© July 31, 2012 – Proprietary and Confidential 12

13. Introduction
Focus Towards Improved Risk Management
 In an effort to improve
patient safety…
– The healthcare industry
has focused more
attention on risk
management tools
– Awareness of the
important role that
human factors plays in
the design process
– By driving more human
factors into the design
process sooner, rather
than later, results in more
positive change
© July 31, 2012 – Proprietary and Confidential 13

14. What is FMEA?
© July 31, 2012 – Proprietary and Confidential 14

15. What is FMEA?
FMEA: Failure Modes and Effects Analysis
 Answers the simple question:
“If a system component fails, what is the effect on
system performance or safety?”
 Usage dates back to the 1940’s to evaluate military M&P
to identify risks using a bottom-up assessment
 As technology advanced into areas, such as space
exploration and nuclear power, the need to identify,
prioritize and manage risk amplified
 At that time, assessment of risk in the design of
systems tended to involve engineering and material
failure. © July 31, 2012 – Proprietary and Confidential 15

16. What is FMEA?
Engineers: Two types of FMEA Risk Analyses
1. DESIGN FMEA. A design FMEA is used to examine the
components of a product to identify potential failures.
– For example, in the automotive industry, a design FMEA is
conducted on all components and subsystems of a new
car during the design and manufacturing phases.
– The FMEA tool is used to evaluate the correctness of the
materials, accuracy of specifications, and all other
elements of design required to make a safe automobile.
1. PROCESS FMEA. A process FMEA is used to analyze
the processes used to make a product.
© July 31, 2012 – Proprietary and Confidential 16

17. What is FMEA?
Wait — Where is the User in the FMEA?
© July 31, 2012 – Proprietary and Confidential 17

18. How to…
Need to Focus on the User in the FMEA
“If a system component fails, what is the effect
on system performance or safety?”
Severity of Harm
(S)
“If a user commits an
error, what is the effect
Probability (P) on system performance or
safety?”
(S x P)
© July 31, 2012 – Proprietary and Confidential 18

19. How to Conduct an FMEA
© July 31, 2012 – Proprietary and Confidential 19

20. How to…
Towards a More “Use–Error Focused-–FMEA”
 Expanding upon a chapter entitled “Risk management
in medical products” by Ed Israelski and Bill Muto
 In Pascale Carayon’s recently published textbook, Handbook of
Human Factors and Ergonomics in Health Care and Patient
Safety, Second Edition
 Ed and Bill describe a step-by-step process to identify
hazards and integrate it into the medical product
development process
 They advocate one major addition to the FMEA
procedure which adds the user into the process
– Perform a Task Analysis as this can predict user behaviors
that relate to design interaction
© July 31, 2012 – Proprietary and Confidential 20

21. How to…
But First, Here is an Example FMEA
 FMEA appears in many different styles
 Ultimately, it is really a worksheet (table)
 Used by the team to document and work through issues
 Often has color to pull attention to specific rows
© July 31, 2012 – Proprietary and Confidential 21

22. How to…
12-Step Process (Israelski & Muto, 2012)
1. Form a Team
2. Perform a Task Analysis
3. Start a Worksheet
4. Brainstorm Potential Use Errors (failure modes)
5. List potential Effects of each Failure Mode / operator error
6. Assign Severity ratings (S)
7. Estimate the Probability of occurrence ratings (P)
8. Derive a Risk Index (S x P)
9. Prioritize Risks by risk index values
10. Take Actions to eliminate or reduce the high priority failure modes
11. Estimate Effectiveness of action ratings
12. Revise the Risk Priorities
Israelski, E., & Muto. W. (2012). Risk management in medical products. In P. Carayon (Ed.), Handbook
of Human Factors and Ergonomics in Health Care and Patient Safety, Second Edition. CRC Press.
© July 31, 2012 – Proprietary and Confidential 22

23. How to…
Step 1: Form a Team
 Team composition includes representatives from:
– Product team
– Regulatory
– Human Factors
– Engineering (Development)
– Medical Affairs
– Quality Assurance
– Training
– User Group representatives
 Agree upon goals and procedures to conduct the FMEA
© July 31, 2012 – Proprietary and Confidential 23

24. How to…
Step 2: Conduct a Task Analysis
Back to
Entry Just-browsing...
Point Seeing what's available... Just Browsing
for Info by:
- Act
- Venue
- Date
- Genre
User may be first-time or repeat visitor - Geographic Area
User is just browsing for information on upcoming events - Upcoming Events
"I want to see what's coming soon"
"What's "What would "What is "What deals
playing at the be fun to do available this are available?"
Metro?" with the weekend?"
family?"
Try to limit browse
to 3-4 steps
Browse by Name Browsing by Genre Browsing by Date Browse by Deals
(Event / Venue)
Show me what's coming up
Show Genre Show me latest deals
this weekend
Search by Name
Promo on Home
Select Genre
Show search results Search by Date
List of Acts: Metro
List of Venues: Metro
Link to Promos
e.g., Amazon.com Show sub-category lisitng
Show Genre
AlphaNumeric Jum Search by full listing? Select Promo
Select Act or Select sub-
Venue categories Show
 Conduct a formal task analysis
Events Show
Events
Show List of Acts
Show
Show List of Events
Events
Sort by State Sort by Venue
AlphaNumeric Jump
Jump to Date Closest to Zip
 Don’t get lured into “we’ll do the
Search by full listing?
Search by Price Sort by State
TM.com should help
Show Anything No
- Try another venue?
Events interesting?
- List headliners
task analysis AND brainstorm”
Yes
TM.com should offer help
- Email new dates
Tix No
Depression - Avail tickets
Avail? - Next Concert
- etc.
Yes
Select
 Provide team with tasks, user
Performance
Go to Exit
Buy Ticketmaster.com
profiles, use environment, and
examples of use errors
 Save “brainstorming” for identifying use errors
© July 31, 2012 – Proprietary and Confidential 24

25. How to…
Step 3: Start a Worksheet
 Settle on the format and columns
 Consider using large sheets of paper (11” x 17” or larger)
with gridlines
 Documenting electronically via MS Excel and projecting
screen to a wall can facilitate group interactions and
discussions
© July 31, 2012 – Proprietary and Confidential 25

26. How to…
Step 4: Brainstorm Potential Use Errors (Failure Modes)
 Follow typical brainstorming rules (i.e., facilitate
ideation over debate over merit discussions)
 Recognize efficiencies of breaking into sub-
groups to reduce “tangent takeover”
 Actually helps to have the device (sounds silly…)
 Use data from customer complaints or MDRs
© July 31, 2012 – Proprietary and Confidential 26

27. How to…
Step 5: List Potential Effects of Each Failure Mode
 For each use error, describe the potential harm
resulting from the Potential Effect
 When multiple harms can occur
– List harms and elevate worst-case scenarios to the top, so
they can be addressed first
© July 31, 2012 – Proprietary and Confidential 27

28. How to…
Step 6: Assign Severity Ratings (S)
 Assign a quantitative level of the Severity of the Harm
 Various levels of (S) are often used
© July 31, 2012 – Proprietary and Confidential 28

29. Methodology
Example of a Five-level Severity Scale (S)
 Three-level Severity Scale
– “Severe, Moderate, Negligible”
© July 31, 2012 – Proprietary and Confidential 29

30. How to…
Step 7: Estimate the Probability of Occurrence Ratings (P)
 Assign a quantitative level of the Probability of the use
error occurring
 Various levels of (P) are often used
© July 31, 2012 – Proprietary and Confidential 30

31. Methodology
Step 7: Estimate the Probability of Occurrence Ratings (P)
 (P) levels can be 3 or 10 point scales derived from:
– Historical data, prediction of probabilities using analytical
or simulation techniques, experimental data, and even
expert judgment.
© July 31, 2012 – Proprietary and Confidential 31

32. Methodology
Step 7: Estimate the Probability of Occurrence Ratings (P)
 Consider that this (P) encompass situations and
circumstances, as well as, an entire sequence of
events from the occurrence of the initiating cause
through to the occurrence of harm
– Does the hazardous situation occur in the absence of a
failure?
– Does the hazardous situation occur in a fault condition?
– Does the hazardous situation occur only in a multiple-fault
condition?
– How likely is it that a hazardous situation will lead to harm?
– The likelihood that a hazardous situation will lead to harm
is influenced by the life-cycle of the medical device and
estimated number of devices in the market
© July 31, 2012 – Proprietary and Confidential 32

33. How to…
Step 8: Derive a Risk Index (S x P)
 Risk Level = Severity x Probability
 Forms a Risk Priority Number (RPN)
 Compare to a Risk Evaluation Matrix
© July 31, 2012 – Proprietary and Confidential 33

34. Methodology
Risk Evaluation Matrix
 Regulators require special attention be paid to all
catastrophic harm severities regardless of (P)
– Historically, too many manufacturers have low balled (P)
© July 31, 2012 – Proprietary and Confidential 34

35. How to…
Step 9: Prioritize the Risks by Risk Index Values
 Sort by Risk Index Level
 Consider Risk Control Measures
– Failure modes that can be addressed immediately
– Failure modes that require
no action
© July 31, 2012 – Proprietary and Confidential 35

36. How to…
Step 10: Take Action to Eliminate or Reduce Risk
 Talking point:
© July 31, 2012 – Proprietary and Confidential 36

37. Introduction
Really Cool Opening Scene in a Nuclear Missile Silo
 West Wing star was more skeptical when a message
came through and was hesitant to turn his key…
© July 31, 2012 – Proprietary and Confidential 37

38. Introduction
Forcing the Reservoir Dogs Star to Suggest He Comply
 “Turn your key, sir!”
© July 31, 2012 – Proprietary and Confidential 38

39. Introduction
Potentially Dangerous Actions Should Require More Effort
 This is where design can play a role to mitigate use
errors and improve patient safety
 Many medical device manufacturers mitigate risk based
on the ALARP (As Low As Reasonably Practicable)
– Accepts that there is a practical limit to the time, effort, and
money that can be spent trying to drive a risk to zero
 Still, manufacturers must justify
– Burden is to show that cost required to reduce the tolerable
risk further would be grossly disproportionate to the benefit
gained and risks are those that are improbable and
marginally severe thereby requiring no further mitigation
Make it hard for users to do something wrong
and easier for them to do it right
© July 31, 2012 – Proprietary and Confidential 39

40. How to…
Step 11: Estimate the Effectiveness of Actions
 Did the Risk Control Measure make the device safer by:
– Eliminating the particular hazard?
– Reducing the Severity (S) and/or Probability (P)?
– Adding protective measures, such as safeties or alarms?
– Providing warning labels and instructions?
 Step 12: Revise Risk Priority (Risk Acceptability)
ISO 14971:2007 Annex Updates
© July 31, 2012 – Proprietary and Confidential 40

41. Taking FMEA a Step Further…
© July 31, 2012 – Proprietary and Confidential 41

42. A Step Further…
Addendum to the 12 Steps…
 This paper seeks to evolve the process and take it to the
next level
 The critical outcome of the FMEA with a Task Analysis is
a list of potential use errors
 However, these potential use errors are limited to
what can be ideated in the brainstorming activity
 All too often, these brainstorming activities occur in a
closed office environment or even worse, sometimes
conducted without the device!
© July 31, 2012 – Proprietary and Confidential 42

43. A Step Further…
Where Does Brainstorming Occur?
© July 31, 2012 – Proprietary and Confidential 43

44. A Step Further…
Review FMEA in a Simulated Environment
© July 31, 2012 – Proprietary and Confidential 44

45. A Step Further…
Review FMEA in a Simulated Environment
© July 31, 2012 – Proprietary and Confidential 45

46. A Step Further…
Break Out When Necessary
 Should a set of use errors be uncovered…
– Break off sub-team to explore the new line of errors while
the main team will continue down the list
© July 31, 2012 – Proprietary and Confidential 46

47. A Step Further…
Takeaways
 An FMEA process combined with Task Analysis improves
risk management
 Addition of simulated environments can increase the
number of identified use errors and precision
 The goal is not to box the answer correctly the first time,
but to get smarter all through the research process
– FMEA should continue to evolve from exploratory to
validation testing
© July 31, 2012 – Proprietary and Confidential 47

48. References
 ISO 14971:2007 (with special emphasis on Annex D and E)
 Flynn, E.A., Barker, K.N., & Carnahan, N.J. (2003). National Observational
Study of Prescription Dispensing: Methods, J Am Pharm Assoc. 43(2): 191-
200.
 Guenter, P., Hicks, R.W., Simmons, D., Crowley, J., Joseph, S., Croteau, R.,
Gosnell, C., Pratt, N.G., & Vanderveen, T.W. (2008). Enteral Feeding
Misconnections: A Consortium Position Statement, The Joint Commission
Journal on Quality and Patient Safety, 34(5): 282-292.
 Israelski, E., & Muto. W. (2012). Risk management in medical products. In P.
Carayon (Ed.), Handbook of Human Factors and Ergonomics in Health Care
and Patient Safety, Second Edition. CRC Press.
 Kaye, R,, & Crowley, J. (2000). Medical device use-safety: Incorporating
human factors engineering into risk management. Food and Drug
Administration
 Logan, M.K. (2010). A roundtable discussion: Increased focus on human
factors drives device safety improvements. Human Factors Horizons. 8-15
© July 31, 2012 – Proprietary and Confidential 48

49. Questions?
PRESENTER:
Gavin Lew Korey Johnson
Managing Director Associate Director
User Centric, Inc. User Centric, Inc.
glew@usercentric.com kjohnson@usercentric.com
#uxlunch
Up next:
Conducting Longitudinal Mobile Studies
Tuesday, Aug 21, 12:00 PM CDT Join the discussion!
www.usercentric.com/webinars @UserCentricInc
49