10 Reasons Why Simulations are the Ideal Learning Tool for Your Business
Readiness is a Predictive Measure
1. Slide 1
Readiness is a predictive measure
James Rollins
Takouba
Anyone remember the Nisqually Earthquake? Do you remember where you were? What you
were doing?
I was in a conference room on the second floor of a factory building in Tukwila. I remember the
room sort of shaking. It wasn’t that dramatic as to be shocking, but it stopped the conversation.
We looked at each other wondering, “What the heck is that.” Then slowly it sunk in that we
were experiencing an earthquake.
The thought was, “What do we do?” “Oh yeah! Duck and cover.” So we scrambled under the
table and a few of us stood in the door frame watching the building shake. It was over in about
a minute. Just enough time to realize what was going on, and then it was over.
From there we headed downstairs to check and see if anyone was hurt. The power went out, so
the factory was shut down. Everyone was standing around looking at each other. Then people
began to evacuate, not because it was planned, but because that is what we do in case of fire –
it’s the same right?
We had no idea when the power would come on, a few of us tried to make phone calls, but the
cell phone lines were down. The bottom line was – we weren’t ready. We were lucky, but we
weren’t ready.
So how should we get ready? And if we think we are ready, how do we really know? How do
we measure it?
2. Slide 2
What is readiness?
A predictive measure of . . .
The state of being ready or prepared, as for use or action . . .
For the purpose of reducing risk . . .
by mitigating consequences.
So let’s start by discussing what readiness is
Readiness is the state of being ready or prepared, as for use or action – so readiness is being
CAPABLE of action.
For the purpose of reducing risk . . .
By mitigating consequences. Since readiness is about a capability for a future event, it must
mean that the risk event has occurred – so we are left with mitigating the consequences.
3. Slide 3
Risk
ResidualRisk
Readiness
How does readiness relate to risk?
= Risk
+ Probability of Occurrence
+ Severity of Hazard
+ Vulnerabilities
= Readiness
+ Reduce vulnerabilities
+ Manage consequences
Risk
The danger that
loss or injury will
occur or that a
process could
fail
To build a solid definition of READINESS, it is important to understand the relationship between
READINESS and RISK. Risk is the danger that loss or injury will occur or that a process could fail.
From this definition, we learn that not only are sources of risk found outside of our
organizations – but also inside our organizations if our processes and procedures are not robust.
Risk is normally expressed as a function of its probability of occurrence, plus the SEVERITY of
the consequences, plus your VULNERABILITIES. This is key, because VULNERABILITIES represent
the gaps in your armor. Ethically speaking, you should be at least aware of them, and
objectively looking to remove them. There are two ways to deal with VULNERABILITIES. One
way is to remove them through some sort of improvement. The other way is to mitigate the
consequences – which brings us to . . .
READINESS. READINESS is your organization’s capability to reduce the consequences of the risk
event, so in a way reduces your vulnerabilities.
This is KEY – sometimes it is not feasible to fix vulnerabilities before they occur as they may be
too expensive when compared to the value of the asset or the probability of occurrence. A
strategy, then, to deal with these vulnerabilities is to reduce injury by reducing the
consequences of the risk event, thereby leaving a much smaller residual risk.
4. Slide 4
What factors influence readiness?
Time
People
Money
Intellect
Maint.
Org
Leaders
Readiness
Stuff
Shelf
life
KM
Standards
Move
So what factors influence READINESS? Factors are entities that improve, take-away or
otherwise affect READINESS.
There is TIME, MONEY (usually not enough), ORGANIZATION and STUFF. I say STUFF because it
is a shorter word than “equipment” or “technology.”
ORGANIZATIONS are made up of PEOPLE, with priorities set by LEADERS and hopefully, the
organization has some from of KNOWLEDGE MANAGEMENT so it isn’t constantly “Reinventing
the wheel.”
So lets start connecting these entities together and describing their relationship to each other.
Let’s start with TIME. TIME is an enemy of READINESS. What I mean is, the more time that
goes by, the more READINESS deteriorates. People forget, organizations change, policies
change. The inevitable march of time erodes READINESS.
MONEY is definitely a supporter of readiness, but it usually has to be managed by an
ORGANIZATION or used to buy STUFF.
PEOPLE have knowledge and experience – they have intellect, or what some companies call
“intellectual capital.” PEOPLE are mobile, they have a tendency to move around. They get
promoted, or move to other jobs to expand their experience, which is good for the organization.
5. But sometimes they get fired or just move off to find other opportunities, usually taking some
part of the intellectual capital in the organization. This affects READINESS negatively.
All-in-all though, PEOPLE are definitely a positive influence on ORGANIZATION. And the quality
of the ORGANIZATION positively influences READINESS.
Finally there is STUFF, which is influenced by NEED FOR MAINTENANCE and SHELF-LIFE. Both of
these factors influence STUFF negatively, because like TIME, when things sit, they get out-of-
date, they expire or they fall into disrepair.
PEOPLE also have a shelf-life too. There is forgetting. Skills must be regularly reinforced,
otherwise PEOPLE forget and make mistakes.
Overall, fresh, well maintained STUFF positively influences READINESS.
That is quite a plate of spaghetti, huh?! With meatballs! The bottom line, is that assessing
READINESS is complicated because it is influenced by so many different factors. The purpose of
this brief seminar is to discuss how to structure your approach to assessing READINESS.
6. Slide 5
What are the steps to measure readiness?
Audit organization to test for compliance2
Conduct a test4
Evaluate organization KPI3
Evaluate your risk1
What are the steps to measure READINESS? In my estimation there are 4 steps that should be
followed:
First you need to evaluate your RISK. This is normally done by reviewing local All-HAZARD
assessments that will give you an indication of what you are facing, how probable and how
severe the threat is. However, there is more . . . Understanding what can hit you is one thing,
but understanding its effect on your organization is another. In my experience, I review risk
worksheets of various kinds, but it rarely describes the vulnerabilities of the business. Nor is
there any cross-walk between these vulnerabilities and the countermeasures the business has
in place to mitigate the effect of this vulnerability.
Second you need to AUDIT your organization. Auditing is a way to see if your organization is
complying with its internal policies and procedures. Auditing can and should be based on some
STANDARD – and we’ll talk more about that later, but it is not absolutely necessary. The key
point is that an organization’s COMPLIANCE with its own policies and procedures is a basic
organizational COMPETENCY.
Third is how well is your organization doing against its KEY PERFORMANCE INDICATORS? Many
organizations don’t even have KPI – but should. If your organization provides any service or
product, it likely has some numbers it uses to measure its performance. The key point is that an
INTERNAL MEASUREMENT is a basic organizational COMPETENCY.
7. Lastly, a great way to see if you are ready is to conduct a TEST. By simulating the scenario you
are most likely to face, you can devise a method to test your organization to see how well it will
respond to a risk event. So let’s start on the top of our list . . .
8. Slide 6
Risk
Review past AARs2
Focus effort on critical capabilities4
Prioritize critical capabilities3
Analyze hazards1
How do assess our risk?
First we analyze our hazards. As I mentioned earlier, there are various documented sources of
risk information. Everything from county risk assessments, to specific geological and climactic
studies made by various agencies. But what ever you do, don’t forget to just walk around and
look. Sometimes the risks we face are staring right in the face. Identify the your vulnerabilities
based on these hazards and use them to determine what CAPABILITIES you will need to
develop.
Review past After Action Reviews. You are likely not the only one who has spent some time
assessing your organization’s risk. AAR’s are a rich source of vulnerability information, and
some even recommend solutions (that nobody instituted). I can tell you that I have personally
reviewed pages and pages of AARs with great information that no one has acted upon.
PRIORITIZE. Once you understand your vulnerabilities, you will be able to decide what
capabilities you need to develop in order to respond effectively to the risk event. You likely will
not have enough resources to address all your needed responsibilities, so you will need to
PRIORITIZE.
And of course, once you have a priority it will allow you to FOCUS on what is important.
9. Slide 7
Audit - What does right look like?
Standards1 Agreements2
Contracts EMAC
Before we audit, we first need to define “What right looks like.”
World-class organizations adhere to industry standards such as ISO or ANSI. As a result of the
quality revolution in the 80’s, TOTAL QUALITY MANAGEMENT practices were codified into
standards. These standards make good business sense, because everyone evaluates their basic
business competencies the same way. The definitions are all the same, and it is easier to
compare apples-to-apples. Because business now have standards, their overall quality improves
and typically results in more efficient, cost-effective operations.
AGREEMENTS are another source of “what right looks like.” For example, the Emergency
Management Assistance Compact (EMAC) was hammered out between states that covers
thorny issues like how to request resources, how they will be paid for, how licenses and other
authorities translate across state lines and such all which establish a standard by which all can
do business.
11. Slide 9
Audit – What do we look at?
Policies1 WHY we do it
Procedures2 HOW we do it
Resources3
WHAT we have
WHO we have
STANDARDS tell us what policies we should have in our operation in order to fulfill that function.
So for example, if I am looking at an Emergency Operations Center, I would likely have a policy
that describes the need for the organization to have and maintain an overarching Emergency
Management Plan. The policy would likely describe who is responsible for making the plan and
how often the EMP would be reviewed. A procedure associated with the EMP, might describe
how a advisory committee is formed, how the chair is selected and what processes they will use
to advise on the EMP. Finally, we would review what resources in terms of organization and
stuff (you remember stuff from earlier?), the organization has its disposal and determine its
adequacy with respect to need.
12. Slide 10
Audit – What we are looking for
Does organization have a culture of fixing problems?2
Does the organization self-audit?4
Do leaders conduct management reviews?3
Do people follow policies and procedures?1
So you are looking at your organization – what do you look for?
Are people aware of and do people actually follow policies and procedures? Do they know
where procedural documents are kept? Do they refer to them? Does the organization have a
method for controlling the document versions?
Does the organization have a method they use to deliberately define, highlight and elevate
problems so they can be fixed?
Do leaders conduct management reviews? Do they pull information about institutional issues
or problems up to the management level so they can allocate resources to fix the problems?
Do leaders ask for internal audits to evaluate the effectiveness of their programs? Is audit a bad
word?
13. Slide 11
KPI – Key performance indicators
Measurable outcome (number of minutes between call and response)2
Statistical process control (is process consistent and repeatable?)3
Specific to a function (call response time)1
Another way to tell if an organization is READY is by looking see if they have set up and are using
KEY PERFORMANCE INDICATORS.
KPI are measurements of a specific function, such as a call response time. If we are running a
dispatch center, for example, we might be interested in knowing how long it takes for a resource
to be dispatched to a location from the point of the call.
KPIs measure a specific outcome in some repeatable way, with a standard quantity. So using
the call response time example, we would track that time in minutes from the time the call was
received by a dispatcher to the time a unit arrives on scene. This definition will provide us with
a standard by which to measure this function.
Typically KPIs can be analyzed using a statistical process control technique in order to determine
if the process is consistent and repeatable. This can give an organization the ability to
determine the causes of variation to an otherwise consistent process.
14. Slide 12
Test – how can we test?
Individual skills requirements2
Exercise!4
Collective skills requirements3
Job task analysis1
Finally – once we determine that an organization has all of the basics elements of competency
in place, we can run a test. A test is a great way to provide a simulated future event, that
provides context to observe the performance of the organization. Testing is a form of auditing,
but differs in that it provides a future context.
To build out a test, we have to develop a list of observable behaviors that an individual would
perform. This first step in this process is conducting a job task analysis. The job task analysis
starts by reviewing a process or procedure and determining and listing out the observable
behaviors that would be associated with that role in the future context. So for example, if we
are going to evaluate a logistics role an Emergency Operations Center (EOC), we would likely
look through the job action sheet and associated procedures to see what actions that person
would take in the performance of their job.
The individual actions that an individual is responsible are called individual skills. These are the
skills an individual must successfully perform in order to be individually competent for the job.
Rarely does an individual operate completely independent of the organization. For most tasks,
we work together. So our individual skills come together in team environment to accomplish
and organizational task – such as “Write a Incident Action Plan.” We call these “collective skills”
and use these to form an exercise evaluation plan. Using the evaluation plan, we can
specifically evaluate each function, and each function in relation to each other.
15. Finally, we exercise. The exercise is a learning event were we test our processes and procedures
and attempt to build the confidence of the exercise participants. It is critically important to
maintain a positive outlook in an exercise, because during an exercise, we are there to fail.
Today’s failures are tomorrow’s strengths.
16. Slide 13
Training Design Features
COMPETENCY TASK SUBTASK TRAINING DESIGN FEATURES
1.0 Estimate the effect of the
hazard on the hospital
1.1 Activate a Hospital Incident
Command Post
Provide a scenario briefing PowerPoint
Organize HICS staff representatives:
Command
PIO
Operations Section
Planning Section
Medical Care Branch
Infrastructure Branch
Logistics Branch
EMS
1.2 Conduct a facilities vulnerability
assessment
1.2.1 Estimate the effect of hazard on critical
infrastructure
1.2.2 Estimate effect of consequences on
hospital facilities, staff and resources
1.2.3 Develop and consider measures to
mitigate hazard consequences
Staff provide an estimate process based on the
following documents:
Hospital vulnerability assessment.
Weather report or other consequences
information (county risk register).
Facilities manager judgment.
Use Hospital Facilities Hazard Impact
Assessment form to estimate post-mitigation
status of systems.
1.3 Determine the degree to which
the hospital can operate in post
hazard environment*
Staff provides a briefing for the IC.
1.4 Determine how the hospital can
safely evacuate patients before
disaster strikes.
1.4a Determine how the hospital
can safely evacuate patients
1.4.1 Estimate the time required to evacuate the
hospital
Staff provides a rough-cut estimate using a form
or calculation to estimate time to evacuate.
Formula based on complexity of facility, patient
mix, transport type and distance to gaining
hospitals.
17. Slide 14
How do we develop our team?
Individual training - training that an individual requires to fulfill their
primary job. Normally provided on-the-job or
through employer provided training.
So how do we develop our team? Individual training key because in ensures that an individual
can fulfill the tasks in their primary job. Normally provided on-the-job or through employer
provided training.
18. Slide 15
Collective training -training that an organization performs together
in context to learn and test processes such
as planning and reporting.
And we provide CONTEXT for collective training to learn and test processes – like planning and
reporting.
19. Slide 16
Leaders –
experienced leaders who
are conditioned to the
nuances of the context
and confident in the
people and
capabilities of the
organization
The final dimension of team-building are leaders. Exercises provide leaders (provided they
participate) with an exception way to build confidence in their team, to provide a venue to test
out processes and procedures and to ultimately validate that the organization is ready.
20. Slide 17
Time
Long hard climb
Fast decline
Readiness
Typically, organizations don’t arrive at a high degree of readiness unless the spend time ramping
up.
Indeed, it can be a long hard climb to a high state of readiness. And the unfortunate thing is
that the decline from readiness is even faster! As I explained in the beginning, time is the main
detractor from readiness, because people move about, they forget, technology changes and
readiness just starts to fall. So what are our challenges to maintaining READINESS?
21. Slide 18
Ways to make readiness last
Turn over2
Knowledge management4
Scale and capacity3
Frequency1 Documentation5
Follow up6
Train frequently enough to reinforce organizational knowledge. If an organization doesn’t
regularly exercise, then readiness can fall so low that it requires substantial investment in time
and resources to get it to an acceptable level again.
Manage turn-over. Turn over is inevitable – but do you have a strategy to transfer knowledge?
Do you train regularly enough that the new person can have an opportunity to exercise in their
job?
Find novel ways to exercise that don’t put a drain on the organization. One reason why full-
scale and functional exercises are not done regularly, is because they take a lot of time, money
and coordination. Computer aided simulations could provide a training venue that can be easily
repeated, not cost as much as a full-scale exercise, and actually do a better job of simulating
capacities and identifying bottlenecks. Another idea is to incorporate exercises into the regular
work day – that is what happens when the urgency is over and the recovery begins.
Manage your institutional knowledge. Have a way to capture processes and procedures, control
versions and keep these up to date. This provides you with good idea of “what right looks like.”
Without knowledge management, you will likely find yourself reinventing the wheel instead of
incrementally developing your organization’s ability.
Document your lessons learned. The outcome of an exercise should be a corrective action plan
and updated procedures.
22. And for the good of the order FOLLOW UP!! Assign people complete recommendations
highlighted in the corrective action plan. Give them a deadline and empower them to complete
the task.
23. Slide 19
Time
Readiness
Risk
Standards1 Agreements2
ResidualRisk
So let’s summarize.
READINESS IS A PREDICTIVE MEASURE of how well your organization would perform in context
to a future event, as measured against a set of standards and agreements, reducing exposure to
the consequences of a given risk event, leaving a smaller residual risk.
25. Anyone remember the Nisqually Earthquake? Do you remember where you were? What
you were doing?
I was in a conference room on the second floor of a factory building in Tukwila. I remember
the room sort of shaking. It wasn’t that dramatic as to be shocking, but it stopped the
conversation. We looked at each other wondering, “What the heck is that.” Then slowly it
sunk in that we were experiencing an earthquake.
The thought was, “What do we do?” “Oh yeah! Duck and cover.” So we scrambled under
the table and a few of us stood in the door frame watching the building shake. It was over
in about a minute. Just enough time to realize what was going on, and then it was over.
From there we headed downstairs to check and see if anyone was hurt. The power went
out, so the factory was shut down. Everyone was standing around looking at each other.
Then people began to evacuate, not because it was planned, but because that is what we
do in case of fire – it’s the same right?
We had no idea when the power would come on, a few of us tried to make phone calls, but
the cell phone lines were down. The bottom line was – we weren’t ready. We were lucky,
but we weren’t ready.
1
26. So how should we get ready? And if we think we are ready, how do we really know? How do
we measure it?
1
27. So let’s start by discussing what readiness is:
1. Readiness is the state of being ready or prepared, as for use or action – so readiness is
being CAPABLE of action.
2. For the purpose of reducing risk . . .
3. By mitigating consequences. Since readiness is about a capability for a future event, it
must mean that the risk event has occurred – so we are left with mitigating the
consequences.
2
28. To build a solid definition of READINESS, it is important to understand the relationship
between READINESS and RISK. Risk is the danger that loss or injury will occur or that a
process could fail. From this definition, we learn that not only are sources of risk found
outside of our organizations – but also inside our organizations if our processes and
procedures are not robust.
Risk is normally expressed as a function of its probability of occurrence, plus the SEVERITY
of the consequences, plus your VULNERABILITIES. This is key, because VULNERABILITIES
represent the gaps in your armor. Ethically speaking, you should be at least aware of them,
and objectively looking to remove them. There are two ways to deal with
VULNERABILITIES. One way is to remove them through some sort of improvement. The
other way is to mitigate the consequences – which brings us to . . .
READINESS is your organization’s capability to reduce the consequences of the risk event,
so in a way reduces your vulnerabilities.
This is KEY – sometimes it is not feasible to fix vulnerabilities before they occur, as they may
be too expensive when compared to the value of the asset or the probability of occurrence.
A strategy, then, to deal with these vulnerabilities is to reduce injury by reducing the
3
29. consequences of the risk event, thereby leaving a much smaller residual risk.
3
30. So what factors influence READINESS? Factors are entities that improve, take-away or
otherwise affect READINESS.
First - there is TIME, MONEY (usually not enough), ORGANIZATION and STUFF. I say STUFF
because it is a shorter word than “equipment” or “technology.”
ORGANIZATIONS are made up of PEOPLE, with priorities set by LEADERS and hopefully, the
organization has some from of KNOWLEDGE MANAGEMENT so it isn’t constantly
“Reinventing the wheel.”
So lets start connecting these entities together and describing their relationship to each
other. Let’s start with TIME. TIME is an enemy of READINESS. What I mean is, the more
time that goes by, the more READINESS deteriorates. People forget, organizations change,
policies change. The inevitable march of time erodes READINESS.
MONEY is definitely a supporter of readiness, but it usually has to be managed by an
ORGANIZATION or used to buy STUFF.
PEOPLE have knowledge and experience – they have intellect, or what some companies call
“intellectual capital.” PEOPLE are mobile, they have a tendency to move around. They get
4
31. promoted, or move to other jobs to expand their experience, which is good for the
organization. But sometimes they get fired or just move off to find other opportunities,
usually taking some part of the intellectual capital in the organization. This affects
READINESS negatively. All-in-all though, PEOPLE are definitely a positive influence on
ORGANIZATION. And the quality of the ORGANIZATION positively influences READINESS.
Finally there is STUFF, which is influenced by NEED FOR MAINTENANCE and SHELF-LIFE. Both
of these factors influence STUFF negatively, because like TIME, when things sit, they get out-
of-date, they expire or they fall into disrepair.
PEOPLE also have a shelf-life too. There is forgetting. Skills must be regularly reinforced,
otherwise PEOPLE forget and make mistakes. Overall, fresh, well maintained STUFF
positively influences READINESS.
That is quite a plate of spaghetti, huh?! With meatballs! The bottom line, is that assessing
READINESS is complicated because it is influenced by so many different factors. The purpose
of this brief seminar is to discuss how to structure your approach to assessing READINESS.
4
32. What are the steps to measure READINESS? In my estimation there are 4 steps that should
be followed:
First you need to evaluate your RISK. This is normally done by reviewing local All-HAZARD
assessments that will give you an indication of what you are facing, how probable and how
severe the threat is. However, there is more . . . Understanding what can hit you is one
thing, but understanding its effect on your organization is another. In my experience, I
review risk worksheets of various kinds, but it rarely describes the vulnerabilities of the
business. Nor is there any cross-walk between these vulnerabilities and the
countermeasures the business has in place to mitigate the effect of this vulnerability.
Second you need to AUDIT your organization. Auditing is a way to see if your organization
is complying with its internal policies and procedures. Auditing can and should be based on
some STANDARD – and we’ll talk more about that later, but it is not absolutely necessary.
The key point is that an organization’s COMPLIANCE with its own policies and procedures is
a basic organizational COMPETENCY.
Third is how well is your organization doing against its KEY PERFORMANCE INDICATORS?
Many organizations don’t even have KPI – but should. If your organization provides any
service or product, it likely has some numbers it uses to measure its performance. The key
5
33. point is that an INTERNAL MEASUREMENT is a basic organizational COMPETENCY.
Lastly, a great way to see if you are ready is to conduct a TEST. By simulating the scenario
you are most likely to face, you can devise a method to test your organization to see how
well it will respond to a risk event. So let’s start on the top of our list . . .
5
34. How do assess our risk?
Start by analyzing your hazards. As I mentioned earlier, there are various documented
sources of risk information. Everything from county risk assessments, to specific geological
and climactic studies made by various agencies. But what ever you do, don’t forget to just
walk around and look. Sometimes the risks we face are staring right in the face. Identify
the your vulnerabilities based on these hazards and use them to determine what
CAPABILITIES you will need to develop.
Review past After Action Reviews. You are likely not the only one who has spent some time
assessing your organization’s risk. AAR’s are a rich source of vulnerability information, and
some even recommend solutions (that nobody instituted). I can tell you that I have
personally reviewed pages and pages of AARs with great information that no one has acted
upon.
PRIORITIZE. Once you understand your vulnerabilities, you will be able to decide what
capabilities you need to develop in order to respond effectively to the risk event. You likely
will not have enough resources to address all your needed responsibilities, so you will need
to PRIORITIZE.
6
35. And of course, once you have a priority it will allow you to FOCUS on what is important.
6
36. Before we audit, we first need to define “What right looks like.”
World-class organizations adhere to industry standards such as ISO or ANSI. As a result of
the quality revolution in the 80’s, TOTAL QUALITY MANAGEMENT practices were codified
into standards. These standards make good business sense, because everyone evaluates
their basic business competencies the same way. The definitions are all the same, and it is
easier to compare apples-to-apples. Because business now have standards, their overall
quality improves and typically results in more efficient, cost-effective operations.
AGREEMENTS are another source of “what right looks like.” For example, the Emergency
Management Assistance Compact (EMAC) was hammered out between states that covers
thorny issues like how to request resources, how they will be paid for, how licenses and
other authorities translate across state lines and such all which establish a standard by
which all can do business.
7
38. STANDARDS tell us what policies we should have in our operation in order to fulfill that
function. So for example, if I am looking at an Emergency Operations Center, I would likely
have a policy that describes the need for the organization to have and maintain an
overarching Emergency Management Plan. The policy would likely describe who is
responsible for making the plan and how often the EMP would be reviewed. A procedure
associated with the EMP, might describe how a advisory committee is formed, how the
chair is selected and what processes they will use to advise on the EMP. Finally, we would
review what resources in terms of organization and stuff (you remember stuff from
earlier?), the organization has its disposal and determine its adequacy with respect to
need.
9
39. So you are looking at your organization – what do you look for?
Are people aware of and do people actually follow policies and procedures? Do they know
where procedural documents are kept? Do they refer to them? Does the organization
have a method for controlling the document versions?
Does the organization have a method they use to deliberately define, highlight and elevate
problems so they can be fixed?
Do leaders conduct management reviews? Do they pull information about institutional
issues or problems up to the management level so they can allocate resources to fix the
problems?
Do leaders ask for internal audits to evaluate the effectiveness of their programs? Is audit a
bad word?
10
40. Another way to tell if an organization is READY is by looking see if they have set up and are
using KEY PERFORMANCE INDICATORS.
KPI are measurements of a specific function, such as a call response time. If we are running
a dispatch center, for example, we might be interested in knowing how long it takes for a
resource to be dispatched to a location from the point of the call.
KPIs measure a specific outcome in some repeatable way, with a standard quantity. So
using the call response time example, we would track that time in minutes from the time
the call was received by a dispatcher to the time a unit arrives on scene. This definition will
provide us with a standard by which to measure this function.
Typically KPIs can be analyzed using a statistical process control technique in order to
determine if the process is consistent and repeatable. This can give an organization the
ability to determine the causes of variation to an otherwise consistent process.
11
41. Finally – once we determine that an organization has all of the basics elements of
competency in place, we can run a test. A test is a great way to provide a simulated future
event, that provides context to observe the performance of the organization. Testing is a
form of auditing, but differs in that it provides a future context.
To build out a test, we have to develop a list of observable behaviors that an individual
would perform. This first step in this process is conducting a job task analysis. The job task
analysis starts by reviewing a process or procedure and determining and listing out the
observable behaviors that would be associated with that role in the future context. So for
example, if we are going to evaluate a logistics role an Emergency Operations Center (EOC),
we would likely look through the job action sheet and associated procedures to see what
actions that person would take in the performance of their job.
The individual actions that an individual is responsible are called individual skills. These are
the skills an individual must successfully perform in order to be individually competent for
the job.
Rarely does an individual operate completely independent of the organization. For most
tasks, we work together. So our individual skills come together in team environment to
accomplish and organizational task – such as “Write a Incident Action Plan.” We call these
12
42. “collective skills” and use these to form an exercise evaluation plan. Using the evaluation
plan, we can specifically evaluate each function, and each function in relation to each other.
Finally, we exercise. The exercise is a learning event were we test our processes and
procedures and attempt to build the confidence of the exercise participants. It is critically
important to maintain a positive outlook in an exercise, because during an exercise, we are
there to fail. Today’s failures are tomorrow’s strengths.
12
43. When creating a test – we use a competency to training design feature matrix. By
identifying the actions we want to observe in behavioral terms, we are able to create an
evaluation scheme. In this example, we are observing the competency of “Estimating the
effect of the hazard on the hospital.” This is a requirement for hospitals that have been
affected by a natural hazard. Tasks that must be performed to enable this competency are
identified in the Task column. For example, “Activate an incident Command Post” is a task
that is necessary to enable the first competency. “Conduct a facility vulnerability
assessment” is another supporting task. Tasks can be broken down further into supporting
elements such as “Estimate effect of hazard on critical infrastructure” and “Estimate effect
of consequences on hospital facilities, staff and resources.” From here the matrix identifies
necessary training design features that would support the evaluation of those tasks.
Documents, processes, scenario injects and related items are all examples of design
features that would make the evaluation possible.
13
44. So how do we develop our team? Individual training key because in ensures that an
individual can fulfill the tasks in their primary job. Normally provided on-the-job or through
employer provided training.
14
45. And we provide CONTEXT for collective training to learn and test processes – like planning
and reporting.
15
46. The final dimension of team-building are leaders. Exercises provide leaders (provided they
participate) with an exception way to build confidence in their team, to provide a venue to
test out processes and procedures and to ultimately validate that the organization is ready.
16
47. Typically, organizations don’t arrive at a high degree of readiness unless the spend time
ramping up.
Indeed, it can be a long hard climb to a high state of readiness. And the unfortunate thing
is that the decline from readiness is even faster! As I explained in the beginning, time is the
main detractor from readiness, because people move about, they forget, technology
changes and readiness just starts to fall. So what are our challenges to maintaining
READINESS?
17
48. 1. Train frequently enough to reinforce organizational knowledge. If an organization
doesn’t regularly exercise, then readiness can fall so low that it requires substantial
investment in time and resources to get it to an acceptable level again.
2. Manage turn-over. Turn over is inevitable – but do you have a strategy to transfer
knowledge? Do you train regularly enough that the new person can have an opportunity to
exercise in their job?
3. Find novel ways to exercise that don’t put a drain on the organization. One reason why
full-scale and functional exercises are not done regularly, is because they take a lot of time,
money and coordination. Computer aided simulations could provide a training venue that
can be easily repeated, not cost as much as a full-scale exercise, and actually do a better
job of simulating capacities and identifying bottlenecks. Another idea is to incorporate
exercises into the regular work day – that is what happens when the urgency is over and
the recovery begins.
4. Manage your institutional knowledge. Have a way to capture processes and procedures,
control versions and keep these up to date. This provides you with good idea of “what right
looks like.” Without knowledge management, you will likely find yourself reinventing the
wheel instead of incrementally developing your organization’s ability.
18
49. 5. Document your lessons learned. The outcome of an exercise should be a corrective action
plan and updated procedures.
6. And for the good of the order FOLLOW UP!! Assign people complete recommendations
highlighted in the corrective action plan. Give them a deadline and empower them to
complete the task.
18
50. So let’s summarize.
READINESS IS A PREDICTIVE MEASURE of how well your organization would perform in
context to a future event, as measured against a set of standards and agreements, reducing
exposure to the consequences of a given risk event, leaving a smaller residual risk.
19
