Krzysztof pulkiewicz kpi

How to measure your
business resiliency
Define the KPI’s/KRI’s and scorecards to
control your security and business
continuity capabilities?

Krzysztof Pulkiewicz | BCMLogic

Abstract

Business Continuity Management is the process, not just a one-time
project activity. In order to control the alignment between the BC plans
and business as usual as well as synchronize the changes, it is required to
setup the scorecard based measurement process.
The set of KPI's and KRI's is aimed to visualize the maturity of BCM, risk vs.
lost metrics and level of protection mechanisms against the business
requirement.
I will present the business resiliency scorecard framework with special
focus on the methods of data gathering and integration with IT
infrastructure landscape.

Platforma Zarządzania Ciągłością Działania BCMLogic | www.bcmlogic.com

Agenda

Why to measure?
What to measure?
How to gather data?
How to present results?
How to do it in practical way?
Key takeaways


KPI primer
KPI/KRI fundamentals
Key performance indicator(KPI) is a measure of performance, commonly used to help an organization
define and evaluate how successful it is, typically in terms of making progress towards its long-term
organizational goals.

Key Risk Indicators Measures are used to indicate how risky an activity is. KRI give us an early warning
to identify potential event that may harm continuity of the process

From row data to metrics A Key Performance Indicator
• Must be something that can be measured and
• Details raw information
continued to be measured
• Metrics are refined data
• Must be precise, meaningful and understandable
• KPIs are metrics with business-
• Must be relevant to the business
context
• May be required by legislation and/or Regulations
• Business context makes security
relevant. • Must have a measurement index that has meaning
• Should be tied to the organization’s vision and
strategy


Why to measure?
• You can control only the things you really cannot measure
• To understand the overall readiness level of your company
• To justify investment
• To plan and assess the risk based on the statistic and past experience
• Executives love metrics and dashboards. Always time-constrained, they ask for
metrics that can be reviewed at a glance
• Money talks - especially when you speak with your CFO
• C-level managers are used to percept from KPI’s- give it to them
• Justify your security investments based on the measurable objectives
• The KPIs can be used to help comply with legislative or regulatory
requirements


What to measure?
Recoverability Planning Compliance Technology BC project

Can our organization be Status and results of Regulatory and audit Project based reporting
planning activities point of view
recovered within our
tolerance for downtime?

• Incidents statistic • BIA overview • BCM Maturity • IT infrastructure • Milestones
• How did we react? • Risk assessment • Compliancy level • IT services SLA • Financial spent
• FTE effort
results • Risk assessment • Service Desk
• Processes covered results • Business RTO/RPO vs.
technology
by BCP capabilities
• Minimum
operational teams


BIA overview
Tactical view on the BIA requirements
• # of Department- Business Impact Analyses vs. Total Expected
• # of Department- Table Top Exercises Completed vs. Total Expected
• # of Supplier Business Continuity Assessments Completed vs. Total Number of “Critical” Suppliers
• Ten top processes (based on criticality score)
• Most critical assets
• RTO/RPO distribution

Example BIA dashboard


Tłumaczenie radar chart
BIA per business unit
Tactical view on the BIA requirements defined at the level of each business unit
• Financial impact over time
• Time wise
• One day stop
• Reputation impact
• Formal and Legal impact
• Number of scenarios affecting the business unit
• Required assets (MAC)
• Minimum operational team vs. total unit
headcount
Radar charts allows to visualize benchmarking
• Critical processes vs. all processes handled by
unit

Business Unit X Business Unit X
Criticality Business Process BIA Updated Plan Updated Tested Criticality Business Process Recovery Objective Recovery Capability Gap
(based on most recent test)
1 Call Center customer support Yes Yes Yes 1 Call Center customer support
4 hours 8 hours 4 hours
2 Accounts Payable Yes No No 2 Accounts Payable 2 hours 1 hour 1 hour
1 Liquidity managment No No No 1 Liquidity managment 2 days 2 days 0


BIA per business process
Criticality level defined for each process
• Prioritize the BC process list based on:
– Business impact when interrupted
– Vulnerability of underlying assets
– Risk level
• Benchmark criticality among different business units/ entities


Tłumaczenie + oryginalny wykres
Risk assessment
Defined scenarios are depicted based on the probability and impact
Przykładowe scenariusze: Przykładowe scenariusze/rozwiązania:
• •
High

Niedostępność budynku Centrali relokacja Awaria zasilania/agregaty
• Awaria IT- Disaster Recovery • Awaria łączy/redundantne linie telekomunikacyjne

PREVENTION ELIMINATION
TOLERANCE MONITORING
Impact

Przykładowe scenariusze: Przykładowe scenariusze
• Chwilowa przerwa w zasilaniu • Przeciążenia zasobów IT/ monitorujemy
• Okresowa absencja pracowników elementy systemów

High
Low Probability

Readiness level
No. BCM management objective Related example KPI
Proper crisis situation management (incident Number of reported incidents
1 management, start-up and implementation of
the recovery tasks) The ratio of the risk response plans for scenarios of potential threats

Number of incidents that were not closed before crossing the processes RTO that are related
to. Incidents can be divided into:
Eliminating the potential effects of process - incidents that concerned the processes associated with financial impact
2
interruption
- incidents that concerned the processes associated with reputation impact
- incidents that concerned the processes associated with law impact
Providing processes resume after the crisis
3 The ratio of recovery tasks completed successfully for all recovery tasks
situation
Number of performed BCM tests
The ratio of the number of BCM plan tests completed successfully for all BCM plan tests at this
Continuous development and improvement of
4 time
BCM
Number of risk which probability or potential impact was reduced after implementation BCM
corrective tasks

To report the progress of BCP project:
• How many process have contingency measures
• How many scenarios are planned
• How many solutions tested


IT services management
Monitor and visualize critical service conditions


Example KPIs
Typical examples of BC KPIs that can be included in a BC Policy Document:

• Level of disruption response/recovery time

• Time to detect disruptions

• Time to trigger action to disruption events

• Time to complete recovery action

• Time to declare `business as usual'

• Level of business continuity testing/exercising/audit

• Level of service delivery and quality acknowledgement by clients?

• Level of knowledge of business continuity awareness/acceptance/culture

• Level of availability and/or knowledge of alternative fall-back to critical resources
(human/ technical/ location)

• Level of effectiveness of Service level agreements


Effectiveness of Investment
• KPIs can be used to measure the Effectiveness of Investment (EOI)
• A Return on Investment (ROI) for business continuity is difficult to measure since risk, and
especially risk reduction, is challenging to quantify in terms of money
• The Effectiveness of Investment (EOI) could be the comparison of the effectiveness of the
resiliency measures with the value of the investment
• Proper KPI/KRI reporting may also provide a financial institution the ability to reduce the
percentage of reserve required to offset operational risk defined by the Basel II, Solvency

High Cost

Equilibrium
Loss

Risk

Low Protection High


How to gather data?
Methods
• Retrieve information from IT systems
– Data base interface (direct or ETL)
– API
• Gather information from people
– Automated forms workflow (reporting)

Sources
• Service Desk system (incidents, time to resolve)
• IT infrastructure monitoring (alerts, up/downtime,
service level)
• BPM (process effectiveness)
• PMO (project reporting)
• Call Center
• People (line managers)


How to visualize?
The reporting mechanism must support three purposes

• Highlight or alert whether business expected targets are being not met
• Provide trending and an overview of performance indicators
• Provide details that pinpoint which areas within each performance indicator require
actions


KPI reporting audience
Each audience may require different information and different presentation

Value at Risk
• The level of aggregation and
C-level Complikancy level or abstraction required may
BC scope and cost vary considerably

• You may not want to talk
Business continuity events about number of backup site
Managers workstations to the Chairman
of the Board!
IT service availability
Business E2E process SLA • Don’t assume that the higher
units Customer service level the level, the simpler the
Customer service downtime presentation
IT infrastructure failures
IT MTTR RTO /RPO
DR testing
Critical services incidents


How to do it in practical way?
• Define 10-15 (smart) KPI
• Identify the KPI stakeholders (RACI)
• Understand where information resist
• Leverage the available data to link the KPIs to other
• key operational metrics that include both technology
• and process metrics
• Integrate with other systems and applications
• Use existing reporting capabilities to establish periodical reporting
• You can use MS Excel or one of the specialized tools
• Share the information across the organization
• Make the KPIs actionable


KPI reporting mistakes
• Lack of management commitment
• Measuring too much, too soon
• Measuring too little, too late
• Measuring the wrong things
• Imprecise KPI definitions
• Using KPI data to evaluate individuals
• Using KPI to motivate, rather than to understand
• Collecting data that is not used
• Lack of communication and training


Key takeaways
1. Define measurable objectives of the security process
2. Utilize existing data sources
3. Learn from the past
4. Align the results presentation to the audience
5. KPIs can be used to help comply with legislative or regulatory requirements


Questions


Krzysztof pulkiewicz kpi

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Krzysztof pulkiewicz kpi

Similaire à Krzysztof pulkiewicz kpi (20)

Krzysztof pulkiewicz kpi