2. WHO’S THIS GUY?
Bryan Smith
Airborne Law Enforcement
Association
Safety Program Manager
Lee County Sheriff’s Office (FL)
IHST SMS Committee Chair
safety@alea.org
239-938-6144
3. • Many of these slides have only speaker’s notes I use during class.
• I have also removed many of the videos in order to make the file sizes
more manageable.
• In these online versions I have added some additional information to
the bottom of many slides. This additional info should help to explain
the main points of the slide.
• If you still have questions or would like to see the videos that have
been removed, please contact me.
• Many of the charts can be found in the 2011 JSHAT report:
http://www.ihst.org/portals/54/US_JSHAT_Compendium_Report1.pdf
• Page numbers at the bottom of some slides refer to the FREE ALEA
SMS Toolkit (2nd edition), which can be downloaded here…
https://www.alea.org/assets/cms/files/safety/SMS-Toolkit.pdf
• If you are still looking at this in the ‘edit’ mode – hit [F5] or go to
‘Slide Show’ on the menu bar and click ‘From Current
Slide’ second from the left.
-Bryan
4. SETTING THE GAUGES
• Who is with us today?
• Who currently works with
an established SMS?
• Who is working on
establishing an SMS?
5. Another Safety Class?
This is howreality… seen at
Is this the SMS is
your operation?
Safety classes and programs have a bad reputation of being boring and limiting to
operations, especially those operations that are regarded as necessary ones to ‘get the
job done’ or, frankly, the ‘fun stuff’. We need to start with an understanding that safety
programs can actually increase productivity, profit and ensure a long career in a fun job
6. 1. Brief Review of SMS
FLIGHT PLAN…
2. In depth look at key components
3. SMS and Decision Making
4. Open Workshop Discussion
7. 1. BRIEF REVIEW OF SMS
“Insanity is doing the same
thing over and over again,
and expecting different
results.”
~Albert Einstein
8. WHY DO WE NEED SMS?
• Industry-wide Helicopter Stats:
• 41% Loss of Control
• 32% Autorotation
• 3% CFIT
• Average total time 4000 hours
• 237 less than 500hrs in make and model (45%)
It is little surprise for most of us to see what is causing accidents. The usual suspects. A
couple surprising points come out of the data, such as the high rate of accidents during
repositioning and RTB phases of flight. Also the high average total time for accident pilots
• *August 2011 JHSAT report
was striking. The low number of hours in make/model in those same pilots is also
important to note. We will revisit these points a little later on. But what we are left with is a
general plateau in the accident rate. So we have a choice, write off the remaining rate as
an unavoidable cost of doing business, or do something else.
9. SHIFT IN DEFINITION OF WHAT ‘RISK’ IS
• In the 1970’s Occupational Risk Management was implemented to shift safety
"Onemanagement from governmentaccidentistothatififsafety is not the highest
"Onething we learned from this accident isthat safety is not the highest
thing we learned from this oversight individual professions.
organizational priority, an organization may accomplish could missions, but there
“They were convinced, without study, that nothing more be done about
organizational priority, an organization may accomplish more missions, but there
“They were convinced, without study, that nothing could be done about
Recent Landmark Cases in Aviation Risk Management:
can be aahigh price to pay for that public sector in late 1980’s – legal, injury
can be Management brought into success,"
• Risk high price to pay for that success,"
such“TheHelicopters The intellectual curiosity and by military, EPA, etc. aid
suchbased. program S-61 ‘Ironchanges. curiosity andskepticism that to aid
•Carsonemergency. does not employ any policy guidance a
an emergency. The intellectual Spearheaded skepticism that to
“Thealso identified apolicynot employ any policy guidance NMSP's
program does 44’ of safety-related deficiencies in the a
an Mostly reactive
The Board also identified anumber of safety-related deficiencies in the NMSP's
The Board number
aviation Mexico inSomeof these deficiencies included the lackof respect to for
aviationpolicies.in making riskdeficiencies included the with report
•New policies.Some of these managed decisions with report
the pilot making risk almost entirely absent” the requirement
solidassessmentStaterequiresduringmanagedthesystemoflackthe aarespect to for aa
•
the pilot
risk Gradual culture requires was mission;
risk flight scheduling decision making..”
at
decisions of
safety culturemorepoint wasmanagement the lack an risk in requirement
Police
solidassessmentat any point duringaamission; of lack ofwideeffective fatigue
safety shift to any ‘complete’ almost entirely absent” effective fatigue
an 1990’s
flight scheduling decision making..”
management program forManagement Training (little emphasis on employee
management program forpilots
(Swiss Cheese, etc.). pilots
further stated that such a culture was, “incompatible with an
further stated that such a culture was, “incompatible with an
inclusion)
As aaresult of this accident investigation, the NTSB issued recommendations
As result of this accident investigation, the NTSB issued recommendations
organization that dealsofofawithlaw enforcementtechnology” system programs and risk
addressingfrom aaNTSBdealswith high-risk management system programs and risk
organization that report afatal law enforcementIIMC/CFIT accident
addressingpilot NTSB report fatal high-risk technology”
~Excerpt from
~Excerpt
decision-making, safety IIMC/CFIT accident
pilotdecision-making, safety management
• Sept 11,
assessments,2001 – no more-SpaceShuttle Columbiaanything offReviewBoard
assessments, excuses. Cannot write as
-Space Shuttle ColumbiaAccident Review Board
Accident
unmanageable because of the ‘nature’ of the business. Complete cultural
The recommendations implemented. All the Governor of New Mexico, the Airborne
The changes still being were aissuedanymore. There mitigated…shift in the definition of
recommendations wereissued to risk can be has of New we are the Airborne
to the Governor been a Mexico,
Actually, we don’t really have choice
Law Enforcement Association, the International Association of Chiefs of Police, and
Law accountable forAssociation, the International Association of Chiefs of Police, and
Enforcement
risk. Risk is definedeverything. not us - as an acceptable probability of an unfavorable
- by society,
the National Association of State Aviation Officials.
the National Association of State Aviation Officials.
• outcome. What used to implementation 2006 is. What we used to write of as ‘the cost
FAA SMS Program be acceptable, no longer
of doing business’ is no longer acceptable, as seen in accident responses and litigation
over the last ten years.
Sources: Gander et al, 2009; O’Hara, 2005; Archbold, 2005
10. “If you had one superpower, what would it be?”
“Luck.”
Since we do not have this superpower either…we need something better than the
traditional safety program.
11. LIMITS OF TRADITIONAL SAFETY PROGRAMS…
• Limited understanding exactly what the threats are
• No analysis of the nature (prioritization) of the risks that create
accidents
• System of ‘educated guesses’ based on personal experiences
• No method of tracking safety implementations (for ROI and
Effectiveness)
We
Fa at h er
LT E tig
ue
Historically the biggest challenge to safety was simply a failure to get a handle on the
endless number of possible e to our business. Typically we would deal with eachgone
risks
Maintenanc Trainin
as they came up. The problem was they first needed to ‘come up’ which was often too
late. It also led to a lot of wasted time and effort as we guessed at which threats needed
to be dealt with. There area t
Pilo million threats out there. We either deal with it by being the
ir
d -a
‘Chicken Little’ pointing out every possible danger we can think up or the “Maverick’ and
Error
Mi
just ignore them all expecting our personal ‘awesome’ness to pull us through.
12. THE TRADITIONAL SAFETY PROGRAM…
• The limits of a traditional Safety Program:
• Reactionary
• Focus on last couple links in the chain of errors direct or only those factors
directly related factors
• ‘What’ not ‘Why’
• Often uses only information from external sources
• No prioritization
• Covers for unknown factors by limiting operations and applying across the
board caution
• No method of tracking results of safety efforts
13. LEAD VS. LAG
LEAD LAG
What is the aim of risk management? It is not to prevent accidents…that is a byproduct. It
is not simply identify all possible risks either. It is to identify the main ingredients in the
witches brew that allows an accident to happen, understanding how they interact, and find
a way of removing as many components as possible, even if it is just one. In this video (
http://www.youtube.com/watch?v=-eKsDwU7kdo ) we see there was no lag information
generated – accident or incident. But is the witches brew complete? Yes. SMS can deal
with this before his luck runs out. Look at the video. The challenging aspect of this from a
safety point of view is the unsafe act did not generate any lag info. The fact that it did not
also fueled the unsafe mentality for the pilot and anyone who saw it. Inexperienced pilots
may mistake the lack of anWhat is the pilot skillrisk management?low risk maneuver. If
• accident for aim of and perceive it as a
we rely only on lag info, we will not keep this pilot, or others from having an accident.
14. WORKING TOWARDS A SOLUTION…
Safety Management Systems
We
Full Spectrum
Full Spectrum Fa at h er
Risk E
LT Analysis
RiskAnalysis t ig
u e
Intervention
Intervention
Recommendations
SMS
Recommendations
ilot
PPrioritized
Prioritized Training
Error
Implementation
Implementation
planning
planning
ir
d -a
Mi
Maintenance
15. COMMON GROUND…
• The pillarsof SMS: TheManagement System:
Definition of a Safety formal, top-down approach to
managing safety risk. It includes systematic procedures,
•Policy
practices, and policies for the management of safety.
•Risk Management
•Assurance
•Promotion
“Incomprehensible jargon is the hallmark of a profession.”
~Kingman Brewster Jr.
IHST SMS Toolkit p.6, 96
16. FOUR PILLARS OF SMS
Safety: Policy
• “What” is to be done, as opposed to ‘How”
objectives, safety commitment, etc.
• “Who” Authority, Responsibility, Roles
• Set by management
• Documentation and Records
• Emergency Preparedness
IHST SMS Toolkit p. 6, 9, 15 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
17. FOUR PILLARS OF SMS
Safety: Policy
All operations conducted at Bob’s Helicopter Service will
be done in the safest manner possible. Notwo separate or
Safety Policy and Operations Policy should be the same document, not
mission
customer is so important as tosafety statement. That statementfrom
ones. The organization’s policy should start with a require deviation
should be more specific than ‘be safe’ or ‘safety first’. It should include a commitment to a
safety policies,also be signed by the chief administrator every year. or the
Just Culture. It should procedures, industry standards,
prudent judgment of our employees. Safe operations
are always the priority in every task we undertake.
IHST SMS Toolkit p. 14-16 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
18. FOUR PILLARS OF SMS
Safety: Risk Management
• Risk Assessment and Control (Mitigation)
1. Context (scope of inquiry, limits of risk, POLICY)
2. ID Hazards (reports, under the RM pillar, observation)
Risk Assessment and Control is mainlyaudits, lag data, but it requires input from the
other pillars to get the job done. Info used to ID hazards can also come from Assurance
and Promotion Pillars.Risk (likelihood vs.the RM process are trained for in the
3. Analyze Interventions deigned in consequence)
Promotion Pillar and documented in the Policy Pillar. Don’t get hung up on the idea that
particularEvaluate Risk (Prioritize, compareThey all work together. limits)
4. functions are only conducted under one pillar. to accepted risk
5. Treat the Risks (policy/procedure, training, equipment, also
The limits of what risks are acceptable are outlined in policy. This is the first step in setting
under PROMOTION)
your context. Then break the operations down into sections: training, normal ops,
maintenance, scheduling, etc. This will allow you to focus your efforts instead of taking on
every possible risk atand Review (Safety ASSURANCE)
6. Monitor once. Once context is defined…start looking for hazards…
IHST SMS Toolkit p. 7, 27 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
19. FOUR PILLARS OF SMS
Bob County Sheriff’s Office Aviation Unit
Safety Survey
1. What are yourmany three safety concerns?
There are biggest methods of identifying hazards. Here are a couple examples. The Hazard
ID form is in the toolkit (p.52). I also recommend using Lead Indicator Identification
_____________________________________________________________________________________ identified, one
techniques (look for my presentation on that topic). Once the hazards are
_____________________________________________________________________________________
of the great strengths of an SMS is to then prioritize those risk using measurable labels.
_____________________________________________________________________________________
This chart is a easy to use method of doing just that (p.37 of toolkit). Another method will
be discussed later.
2. What suggestions do you have for addressing these safety concerns?
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
3. How safe do you feel reporting safety hazards to the Safety Officer?
Very safe Neutral Not Safe
1 2 3 4 5 6 7
IHST SMS Toolkit p. 32, 37, 52 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
20. FOUR PILLARS OF SMS
IHST SMS Toolkit p. 37, 87, 93
IHST SMS Toolkit p. POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
21. FOUR PILLARS OF SMS
Safety: Assurance
• Policy and procedure (Intervention) performance
monitoring.
Safety Assurance is a component completely missing from most
traditional Safety Programs.of is key to making sure efforts are including
• Management It change (impact of new factors,
being directed to the right places, policy and procedures are
safety interventions)
effective and that the benefits of the program are being tracked in
• Return on Investment (ROI) tracking
order to keep employees invested and management supportive.
• Requires use of metrics (quantification) to be
successful.
IHST SMS Toolkit p. 7, 28, 54, 61 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
22. FOUR PILLARS OF SMS
Safety: Assurance
80
Let’s say you decide to use a preflight risk assessment in order to
70
mitigate risk you’ve identified. Assurance can be obtained by Normal
60 Ops
tracking the assessments so you can see if they are havingWaiver, Mitigate
50 a
positive impact on safety, failing to mitigate the targeted risk, or just
40 STOP WORK
wasting time.
30
20
10
0
Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09
Source: Dave Huntzinger POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
23. FOUR PILLARS OF SMS
Safety: Promotion
•Training and Education
Initial, recurrent, general and specific
Establish proficiency and currency requirements
•Communications
SMS program performance, status
Management’s commitment to the program
Safety related information
IHST SMS Toolkit p. 68 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
24. 1. Brief Review of SMS
FLIGHT PLAN… 2. In depth look at key components
3. How SMS and Decision Making are
connected
4. Open Workshop Discussion
The quote describes the same rule that applies to having a Safety
Program on the shelf that is either not used, or is ineffective.
25. FOUR PILLARS OF SMS – A CLOSER LOOK
• Safety Climate - The support
and emphasis given to a safety
program by administrators.
• Safety Knowledge – Actual
safety information an employee
has on how they should
perform their work, and why
• Safety Culture - Actual safety
practices and attitudes
generally covering operations.
These three components must be strong in each of the four pillars
of an SMS, or one will fall and bring the others with it.
POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
Source: Vinodkumar & Bhasi, 2010
26. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Policy
• It is likely that your program already has this component
• Make this Safety Policy part of your operation’s SOP, not a
separate document
• Is Safety ‘First’?? No, it is the product of doing business a
certain way
• Set by management, but must include input from line level
staff
• Scheduled updates with big-picture vision statements and
MEASURABLE intermediate objectives to pave the way.
IHST SMS Toolkit p.9, 15 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
27. FOUR PILLARS OF SMS
Safety: Policy
1. the intentional understatement of the helicopter's empty weight
1. the intentional understatement of the helicopter's empty weight
2. the alteration of the power available chart to exaggerate the helicopter's lift
2. the alteration of the power available chart to exaggerate the helicopter's lift
capability
capability
3. the practice of using unapproved above-minimum specification torque in
3. the practice of using unapproved above-minimum specification torque in
performance calculations that, collectively, resulted in the pilots relying on
performance calculations that, collectively, resulted in the pilots relying on
performance calculations that significantly overestimated the helicopter's load-
performance calculations that significantly overestimated the helicopter's load-
carrying capacity and did not provide an adequate performance margin for aa
carrying capacity and did not provide an adequate performance margin for
successful takeoff
successful takeoff
Look over these items from a landmark case. How many of them
could have been addressed with a simple policy statement guiding
all operations? Do you think one was written in a book
somewhere? Probably. POLICY – RISKitMANAGEMENT – ASSURANCE - PROMOTION
Why didn’t work?
28. FOUR PILLARS OF SMS – A CLOSER LOOK
Policy and Risk Management
-Hazard Identification requires input from everyone
-That input depends on Just Culture being written into policy
This picture shows blade damage that occurred after the pilot did
his preflight. Fortunately the crewmember who caused the
damage, while nobody was looking, trusted the just culture at the
operation and reported the incident. If he had not, the pilot would
have flown without seeing it. It was a case of normalized deviation
that was occurring throughout the entire operation so it could have
happened to anyone.
IHST SMS Toolkit p. 56, 89 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
29. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Risk Management
• Risk can also be defined vs.
Risk was earlier shown on a consequence as: likelihood chart. Risk
can also be defined this way…this V [ I,T,V value 1-4 ]
R = I x T x formula gives you the
opportunity to address either the environmental factor (T) or the
Impact – Level of damage and/or cost
human factor (V). This formula is used by the FBI to deal with
Threat – Capability of risk to inflict estimatedinfo is
security threats that have never happened, thus no lag
impact
Vulnerability – Of the operations that have not recently
available. This would be useful inperson or resource to risk
had an incident, to deal with management of change (avionics,
IIMC/CFIT
Bird Strike
mission, etc.) or a newly identified hazard. This formula could also
I=4
be used to show the impact of an SMS driven Intervention
T = 1-4 (depends on bird sizeon wx often encountered in your area)
2-4 (Can very with policy most minimums, avionics, flight area)
(Control) or other variables. For example, the threat level (T) could
V= 1-4 (depends on altitudes,culture, experience) equipment)
training, flight paths, safety
change with a change in seasons, mission parameters, or
equipment. The (V) Vulnerability factor could be changed with
Source – FBI;improved safety culture, etc.
training, Lee, 2005 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
30. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Risk Management
Failing to provide a suitable procedure and training to support a
•Need to develop policy AND procedures AND recommend
new policy can lead to normalized deviation. This is when a policy
training – normalized deviance
says one thing, but its understood that everybody does something
against that policy as a general rule. Fatigue rules are a prime
example of this. For example, a policy may say that crews get 8
hours of sleep. But if you have a 12 hour shift with a 45 minute
drive each way and family at home it is unlikely that you will often
get a full 8 hours. If you do not, or if you are ill, is there a procedure
to allow crews to adhere to the policy (i.e. ability to have someone
cover the shift, leave the shift open, etc.)? If not, the policy is just
there as an administrative checkmark to cover liability for the
organization, the policy does not improve safety.
IHST SMS Toolkit p. 64, 87 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
31. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Risk Management – Hazard Identification
•Don’t limit yourself to just looking at the direct factors in
identified hazards or lag data
•Search for Latent Factors as well
•These can be used to develop LEAD INDICATORS
•Swiss Cheese, 5-Why’s, etc
IHST SMS Toolkit p. 7, 27, 32 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
32. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Risk Management – Latent Factors
1. “Why did Thunder Pig hit the side of the hangar with the tailboom?”
“He lost control during a landing.”
2. “Why did he lose control?”
“He put the tail in the wind (downwind hover) when heavy and got into LTE.”
3. “Why did he not put in enough control input more quickly or hover into the wind?”
“He had not flown in those conditions for several months and was ‘rusty’.”
Tell me where you think a traditional accident investigation would
end. Be honest.
4. “Why had he not flown in unit SOP approved wind conditions in several months?”
“He set personal minimums that were below the conditions on the day of the accident and
turned down flights if the winds this process and LEAD INDICATORS,
For more information on exceeded those.”
please look for my presentation on this topic.
5. “Why did he take a flight in conditions that exceeded those personal limits on the day of the
accident?”
“The call was for a missing 2 year-old and he felt compelled to go.”
33. CHECKLISTS
• Use SMS generated lead indicators (interventions) in your
checklists
• Develop preflight (post-preflight) and mission checklists
• Stop Checklist at major objective and start new one
• Consider the ‘flow’ of the checklist
• Alternating colors
• Larger print at bottom of list
34. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Assurance
• Feedback – Anything
without feedback is a
guess…at best an
educated guess
• Traditionally, safety
implementations were
unquestionable once
made into policy
• Love the results, not the
policy or procedure
IHST SMS Toolkit p. 28, 39, 44 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
35. FOUR PILLARS OF SMS – A CLOSER LOOK
Safety: Promotion
•Training and Testing must be separated by
definitive lines. i.e. If every flight with an
Instructor seems like a test, the pilot will never
be comfortable asking for instruction on
something they are not 100% sure about.
•Safety Management and Training cannot
operate independently of each other.
IHST SMS Toolkit p. 66, 68 POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
36. Training and Aviation Safety
Often the suggested answer to dealing with shrinking budgets and
the high number of training accidents is to simply cut training. As
we can see here, the number one method of stopping accidents is
through training! We cannot improve safety by cutting training. All
SMS efforts end in a control or intervention that cannot be put into
place without some sort of training. Training is vital to safety,
without it SMS collects information, but does not have an avenue
for actually affecting safety.
37. 1. Brief Review of SMS
FLIGHT PLAN…
2. In depth look at key components
3. How SMS and Decision Making are connected
4. Open Workshop Discussion
38. 4. SMS AND DECISION MAKING
“MAN – A creature that was created at the end of the
week when God was very tired.”
~Mark Twain
39. DECISION MAKING THEORY
Analytical Decision Making
Ideal for the following conditions:
• Clear goal or outcome
• Plenty of time
• All conditions, factors are known
From this, the decision maker can:
• Develop wide range of options
• Evaluate and compare options
• Choose the optimal path
Source: Dave Huntzinger & Fred Brisbois - IHST 2012 Heli-Expo SMS Presentation
40. DECISION MAKING THEORY
Safety Management
Analytical Method Characteristics System They
The analytical decision making processes are structured, deliberate and thoughtful.
• Structured
are ideal for planning stages and lend themselves to flight planning, aircraft purchasing or
Implementations, Policies, Procedures,
• Time the best in
design. These workconsuminga group environment with access to loads of information.
• Process breaks down with stress, limited time
Training, Communications,
Can you see where this is going? What we have come to learn is that these methods are
Analytical Methods
Education….
not well suited for decision making while flying. Up there, we have exactly the opposite
situation; all factors are not known, there are very likely competing goals (safety,
customer satisfaction, contract requirements, financials, etc.) and time is extremely tight.
Deliberate & thoughtful; best suited for:
• Aircraft these
We don’t need to castdesigntheories out because they don’t work well in the aircraft. Use
analytical •
methods to develop good procedures and policy while on the ground. Use this
Flight planning
method to understand the issues as best as possible and develop safety tools that can be
• Aircraft purchasing
used in the aircraft with the following decision making theories in mind.
• And………
Source: Dave Huntzinger & Fred Brisbois - IHST 2012 Heli-Expo SMS Presentation
41. DECISION MAKING THEORY
Intuitive Methods
Fast
• There is a name for the decision making processes we use while
• flying. They are called intuitive decision making processes. These
Simple
• are fast, simple and memory based. They work reasonably well
Memory based
with limited information and can expect to produce a solution that
Work with limited information
• has a chance of being successful (or not). This process is better
• suited to fast paced, dynamic situations such as car driving, sports
Option chosen probably OK, but not optimal
and combat.
Better suited see, SMS plugs into this nicely. Memory baseddynamic,
As you can to real time decision making (flying) and other items
are developed through SMSdriving, sports, combat
fast paced situations: car influenced training materials and
methods. When working with limited information – use SMS to
understand problem and help prepare pilot for what information
they need to seek out
Source: Dave Huntzinger & Fred Brisbois - IHST 2012 Heli-Expo SMS Presentation
42. DECISION MAKING THEORY
Naturalistic Decision Making (Intuitive DM process)
One intuitive method in particular is called Naturalistic Decision
Used in complex, fast paced situations
making. It takes this name from its dependence on environmental
• Based on environmental this
cues, clues and feedback. Ininput case, the decisions are
sequential and interdependent. That is,both decision affects the as result
• Conditions constantly changing, one independently and
of your actions
next one. And other things could be changing in the middle of
everything (such as weather, time, system status, people, etc.).
• Real time decision making (not planning)
• Goals not well defined
• Could be competing goals (safety vs …)
• Decision maker is: knowledgeable, experienced & professional
(Peter Simpson)
Source: Dave Huntzinger & Fred Brisbois - IHST 2012 Heli-Expo SMS Presentation
43. DECISION MAKING THEORY
Naturalistic Decision Making
Not so much a method as the way we actually do things…
Naturalistic decision making has two important parts. The first is
Step 1: Situation Assessment (SA)
Situation Assessment. You identify the problem and resources
needed to get the job Identify - how much time you have. Then run
1. Problem definition:
done and
• Problem
a risk assessment. What is the worst credible outcome and the
• Goal(s)
likelihood this will work or not?
• Information
SMS can drive thesources needed to training needed to help aircrews
knowledge and succeed
• Prioritize incoming information
seek out the info needed and prioritize the info coming in. It can
2. Risk assessment
allow them to regain Situational Awareness faster. It can also allow
• severity
for faster severity vs. probability decisions.
• probability
3. Time available
44. DECISION MAKING THEORY
Naturalistic DecisionofMaking have three basic
The second half is Course Action. We
programs we can use. One is rule based; if this, then that. These
are memoryStep 2: come from experience and(CoA)
based and Course of Action training.
Emergency procedures fall into this category. The second option is
1. a choice. I can go either Consideredfor fuel. The last one is
Potential Solutions here or there
• Rule based – single, memory based solution
creativity. This is where you have to respond to a situation where
neither the first nor training, EP drills, mentioned above apply.
(experience, the second choices etc.)
• can only try to draw parallels from some
You Choice based – Multiple Options other experience. An
• Creative – is good example. There are no procedures
airframe vibrationNoaobvious choice, must use substitute and
whatexperiences have? To understand or solve the problem
choices do you
2. you may have to experiment.
Simulation
• Mental test of potential solutions
From that set of potential solutions you create a course of action
3. Act
and act.
Source: Dave Huntzinger & Fred Brisbois - IHST 2012 Heli-Expo SMS Presentation
45. DECISION MAKING THEORY
Common Errors - Two basic areas
Situation Assessment errors
• Poor understanding of situation
• Poor risk assessment
• Misjudge time available
Course of Action errors
• Right rule, wrong time
• Right rule, poor application
• Choose wrong procedure or option
Source: Dave Huntzinger & Fred Brisbois - IHST 2012 Heli-Expo SMS Presentation
46. DIRECTING FIRE
OnceINTERVENTIONSID hazards, analyze them, and
you use your SMS to AND DECISION MAKING
• prioritize them,Decision Making factors when developingto control them
Consider these you need to start looking at ways Interventions
through Interventions (Controls). Consider the decision making
• Checklists – Consider flow and critical tasks
process that the people you are trying to help will be using when
• facing – Combine with SA are a few areas specific to your profession
ADM each risk. Here information. Make where you can use decision
• making Teach crew toSMS data‘triggers’ based on lead indicators will be
CRM – theory and recognize to create an Intervention that
• useful in – Notcockpit. thing every time. Direct training accordingly
Training the the same
• Environment - Cannot program out all human error. Minimize error and build in
Remember, human error cannot be programmed completely out.
protective environmental layers
When you can, put in a non-human control for the risk. In the
picture at the end, I could ‘train’ my daughter not to draw on the
wall…or I could move the markers away from the wall so the
temptation is removed.
47. DECISION MAKING THEORY
The top chart shows the mental state of a fatigued person. The
bars indicate the speed the person needs to respond to a certain
task. You can see that fatigue is not uniform, it goes up and down.
The bottom chart is made up tasking for a flight – again, not
uniform because some tasks require more work from the pilot than
others. We often evaluate our own level of fatigue during those
phases when our brains are not running as slow, and we do not
recognize the high peaks.state of a fatigued person. The bars the peaksspeed
The top chart shows the mental During a flight luck keeps indicate the
apart, not needs to respond to a runs out high can seetasking occursuniform, it
the person skill. When luck certain task. You flight that fatigue is not during
goes up and down. The bottom chart is made up tasking for a flight – again, not uniform
a because some tasks requirefatigued personpilot than others. We often evaluate our
high fatigue peak. A more work from the not able to evaluate
themself of fatigue during those phases when our brains are not running as slow, and we
own level any more than a drunk person can. Environmental
intervention is the high peaks. During arisk (policy inthe peaks apart, not skill.
do not recognize needed to control flight luck keeps this case).
When luck runs out high flight tasking occurs during a high fatigue peak. A fatigued
person not able to evaluate themself any more than a drunk person can. Environmental
intervention is needed to control risk (policy in this case).
48. DECISION MAKING THEORY
ADM AND CRM
• Once your most significant risks are identified (prioritized), develop ADM type
triggers and responses.
• Aeronautical Decision Making – Hazardous Attitudes
• Invulnerability “It won’t happen to me”
• “The best crews have fallen victim to the simplest of errors”
• Two different sources of mission information are conflicting
• Hold on, attempt to verify both
• “If the ceiling drops another 100 feet, we’re out of here”
• If I (you) are saying that, it is already time to go home.
49. DECISION MAKING THEORY
“The pilot advised the SAR personnel to load quick, as he
“The pilot advised the SAR personnel to load quick, as he
had no intentions of spending the night there...they lost
had no intentions of spending the night there...they lost
1) Contributing to the accident was the failure of the flight crewmembers to
1) Contributing to the accident was the failure of the flight crewmembers to
address the fact that the helicopter had approached itscontinuedto
sight of the fact that the helicopter had approached itsmaximum to
address the helicopter about 50 feet agl. They continued
sight of the helicopter about 50 feet agl. They maximum
performance capabilityto the time of a collision sound, accident
hear the helicopter to the time prior departuressound, accident
performance capabilityon their two of a collision from the
hear the helicopter on their two prior departures from the
followed by the sound of an avalanche.” at the limit of the
followed by the sound of an avalanche.”
site because they were accustomed to operating at the limit of the
site because they were accustomed to operating
helicopter’s performance.
helicopter’s performance.
~Excerpt from aaNTSB report of aalaw enforcement IIMC/CFIT accident with multiple fatalities
~Excerpt from NTSB report of law enforcement IIMC/CFIT accident with multiple fatalities
POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
50. 1. Brief Review of SMS
FLIGHT PLAN…
2. In depth look at key components
3. How SMS and Decision Making are connected
4. Open Workshop Discussion
51. 5. WORKSHOP DISCUSSION
• Who is with us today
• Who currently works with an established SMS?
• What were your biggest challenges?
• How did you overcome them?
• Who is working on establishing an SMS?
• What is your biggest challenge?
• What would you ask the SMS genie to create out
of this air in order to help facilitate your effort?
POLICY – RISK MANAGEMENT – ASSURANCE - PROMOTION
52. There are no new ways to crash an aircraft…
…but there are new ways to keep people from crashing them…
Bryan Smith
safety@alea.org
239-938-6144
www.ihst.org
www.alea.org
Notes de l'éditeur
I realize that today’s class is made up of folks from all areas of our industry, but this is how I market SMS to the law enforcement guys. And while you all may be doing different missions, the core of the message is the same. Approach SMS as a means of facilitating business and maximizing profit instead of the other way around.
It is little surprise for most of us to see what is causing accidents. The usual suspects. A couple surprising points come out of the data, such as the high rate of accidents during repositioning and RTB phases of flight. Also the high average total time for accident pilots was striking. The low number of hours in make/model in those same pilots is also important to note. We will revisit these points a little later on. But what we are left with is a general plateau in the accident rate. So we have a choice, write off the remaining rate as an unavoidable cost of doing business, or do something else.
That is what we are counting on without an SMS. We have a lot of data, that is where a traditional safety program stops.
Historically the biggest challenge to safety was simply a failure to get a handle on the endless number of possible risks to our business. Typically we would deal with each one as they came up. The problem was they first needed to ‘come up’ which was often too late. It also led to a lot of wasted time and effort as we guessed at which threats needed to be dealt with. There area million threats out there. We either deal with it by being the ‘chicken Little’ pointing out every possible danger we can think up or the “maverick’ and just ignore them all expecting our personal ‘awesome’ness to pull us through.
What is the aim of risk management? It is not to prevent accidents…that is a byproduct. It is not simply identify all possible risks either. It is to identify the main ingredients in the witches brew that allows an accident to happen, understanding how they interact, and find a way of removing as many components as possible, even if it is just one. In this video (http://www.youtube.com/watch?v=-eKsDwU7kdo) we see there was no lag information generated – accident or incident. But is the witches brew complete? Yes. SMS can deal with this before his luck runs out. Look at the video. The challenging aspect of this from a safety point of view is the unsafe act did not generate any lag info. The fact that it did not also fueled the unsafe mentality for the pilot and anyone who saw it. Inexperienced pilots may mistake the lack of an accident for pilot skill and perceive it as a low risk manuver. If we rely only on lag info, we will not keep this pilot, or others from having an accident.
Eventually a desire to create a more defined and coordinated attack on these risks led to ‘safety programs’. Safety programs still relied ‘lag’ information, meaning something had to happen first (be it your aircraft, or someone else’s) and the analysis rarely went beyond the immediate factors in the crash. i.e. Don’t let your rotor RPM get too low during a practice hover auto, etc. Finally, the SMS program had been developed. It is a machine that can take all of this data, process it and show you how to mitigate risk at various levels. It can also prioritize your risks so effort is spent on the most important items.
There are many terms and definitions for SMS and its components. Lets find some common ground before we move on…
Policy is ‘what’ we want to accomplish, or what the rules are. Procedures define ‘how’ they should be done. Set your safety policy first.
Hazard Identification is large under the RM pillar, but it requires input from the other pillars to get the job done. Info used to ID hazards can also come from Assurance and Promotion Pillars
Safety Assurance is a component completely missing from most traditional Safety Programs. It is key to making sure efforts are being directed to the right places, policy and procedures are effective and that the benefits of the program are being tracked in order to keep employees invested and management supportive.
Let’s say you decide to use a preflight risk assessment in order to mitigate risk you’ve identified. Assurance can be obtained by tracking the assessments so you can see if they are having a positive impact on safety, failing to mitigate the targeted risk, or just wasting time.
Look over these items from a landmark case. How many of them could have been addressed with a simple policy statement guiding all operations? Do you think one was written in a book somewhere? Probably. Why didn’t it work?
Risk was earlier shown on a consequence vs likelihood chart. Risk can also be defined this way…this formula gives you the opportunity to address either the environmental factor (T) or the human factor (V). This formula is used by the FBI to deal with security threats that have never happened, thus no lag info is available. This would be useful in operations that have not recently had an incident, to deal with management of change (avionics, mission, etc.) or a newly identified hazard. For example, the threat level (T) could change with a change in seasons, mission parameters, or equipment. The (V) Vulnerability factor could be changed with training, improved safety culture, etc.
Failing to provide a suitable procedure and training to support a new policy can lead to normalized deviation. This is when a policy says one thing, but its understood that everybody does something against that policy as a general rule. Fatigue rules are a prime example of this. For example, a policy may say that crews get 8 hours of sleep. But if you have a 12 hour shift with a 45 minute drive each way and family at home it is unlikely that you will often get a full 8 hours. If you do not, or if you are ill, is there a procedure to allow crews to adhere to the policy (eg. Ability to have someone cover the shift, leave the shift open, etc.)? If not, the policy is just there as an administrative checkmark to cover liability of the organization, the policy does not improve safety.
Give example -
Tell me where you think a traditional accident investigation would end. Be honest.
Checklists limit human error and program behavior that will be needed when time does not permit analytical decision making
Policy is ‘what’ we want to accomplish, or what the rules are. Procedures define ‘how’ they should be done. Set your safety policy first.
Policy is ‘what’ we want to accomplish, or what the rules are. Procedures define ‘how’ they should be done. Set your safety policy first.
Think of the acronym based decision making tools. What do you feel about these tools and their usefulness in the cockpit?
The analytical decision making processes are structured, deliberate and thoughtful. They are ideal for planning stages and lend themselves to flight planning, aircraft purchasing or design. These work the best in a group environment with access to loads of information. Can you see where this is going? What we have come to learn is that these methods are not well suited for decision making while flying. Up there, we have exactly the opposite situation; all factors are not known, there are very likely competing goals (safety, customer satisfaction, contract requirements, financials, etc.) and time is extremely tight. We don’t need to cast these theories out because they don’t work well in the aircraft. Use analytical methods to develop good procedures and policy while on the ground. Use this method to understand the issues as best as possible and develop safety tools that can be used with the following decision making theories in mind.
There is a name for the decision making processes we use while flying. They are called intuitive decision making processes. These are fast, simple and memory based. They work reasonably well with limited information and can expect to produce a solution that has a chance of being successful (or not). This process is better suited to fast paced, dynamic situations such as car driving, sports and combat. As you can see, SMS plugs into this nicely. Memory based items are developed through SMS influenced training materials and methods. When working with limited information – use SMS to understand problem and help prepare pilot for what information they need to seek out
One intuitive method in particular is called Naturalistic Decision making. It takes this name from its dependence on environmental cues, clues and feedback. In this case, the decisions are sequential and interdependent. That is, one decision affects the next one. And other things could be changing in the middle of everything (such as weather, time, system status, people, etc.).
Naturalistic decision making has two important parts. The first is Situation Assessment. You identify the problem and resources needed to get the job done and how much time you have. Then run a risk assessment. What is the worst credible outcome and the likelihood this will work or not? SMS can drive the knowledge and training needed to help aircrews seek out the info needed and prioritize the info coming in. It can allow them to regain Situational Awareness faster. It can also allow for faster severity vs. probability decisions.
The second half is Course of Action. We have three basic programs we can use. One is rule based; if this, then that. These are memory based and come from experience and training. Emergency procedures fall into this category. The second option is a choice. I can go either here or there for fuel. The last one is creativity. This is where you have to respond to a situation where neither the first nor the second choices mentioned above apply. You can only try to draw parallels from some other experience. An airframe vibration is a good example. There are no procedures and what choices do you have? To understand or solve the problem you may have to experiment. From that set of potential solutions you create a course of action and act.
There are many opportunities for error in this process. You can mess up the situational assessment. You may not have all the necessary information that you need. Or you could misjudge the time available. Under course of action you could choose the wrong rule to apply or misapply the correct one. Right rule, wrong time…in an auto you flare at 100’ agl… Right rule poor application…engine failure in twin and shut down the good engine In general, more experience has shown to make a significant difference when it comes to good outcomes. More experience helps the decision maker identify the problem quicker and more accurately. It also allows the decision maker to choose the best course of action. Need more examples of SMS interaction here.
Top chart is response speed in a fatigued person – not uniform. Bottom chart is made up tasking for a flight – not uniform. Luck keeps the peaks apart, not skill. Fatigued person not able to evaluate self any more than a drunk person can. Environmental intervention needed to control risk.