Stephen Cresswell presentation for APM Yorkshire and North Lincolnshire branch, 25 June 2019, Sheffield

1. Risk & Opportunity –
are we missing a trick?
Stephen Cresswell
25th June, 2019
stephen.cresswell@intorisk.com
07810 544073
Risk Management and project forecasting as commonly practices is missing a trick
(or many tricks) by ignoring the interdependencies between the identified risks. This
has potentially serious consequences. I will explain why I think this is the case – the
intention is to illustrate / open your eyes to these effects. There are other reasons
why projects overrun – this focuses on the calculation of ‘cost of risk’.
1

2. Simple Additive Approach for Cost
Base Plan
• WP1 - £xxxx
• WP2 - £xxxx
• WP3 - £xxxx
• WP4 - £xxxx
Risk Register
• Risk 1 – % x £
• Risk 2 - % x £
• Risk 3 - % x £
Base Cost £xxxx
+ Cost of Risk £xxxx
Forecast £xxxx
The most common approach for providing a risk adjusted forecast of project costs
involves simple addition.
An average or mean cost for each item in the risk register / list is calculated by
multiplying the Probability x Impact
The risk cost is calculated based on the assumption that it is the only risk that occurs,
and uses the base/planned cost as a reference.
The mean cost of the project is found by summing the mean value for each risk item
then summing all these and adding to the planned cost.
Monte carlo analysis might be used instead of an arithmetic approach – however the
limitations are the same.
This simple additive approach is used by most corporate processes and commercial
off the shelf data base systems
2

3. Common RM Frameworks - View
Cause Risk Event Consequence
Cause Risk Event Consequence
Cause Risk Event Consequence
This is the basic premise of a risk register, corporate data base, ERM system etc
Implicitly it assumes (management system not necessarily the PM) that all risks items
are independent of each other.
It also views the sources of uncertainty as risks, specific events that may or may not
happen, to which a probability can be assigned, This is a narrow definition.
3

4. Beyond Risk Events – Chris Chapman
• ‘Scheduling/Planning Risk’ – (Ambiguity-uncertainty)
- Top down from the ‘bubble chart’ challenged by ongoing ‘bottom up’ detailed execution planning
• Systemic-uncertainty
- Simple Mechanisms e.g. known damages for delay
- Projects as complex Systems – difficult to understand, potentially unpredictable
• Capability-culture uncertainty
- Cultural & behavioural factors e.g. deliberate misrepresentation, inherent optimism,
- Reliability of management & reporting systems
- Expertise, knowhow, productivity
• Variability-uncertainty
- Price, quantities, exchange rates, unnamed weather,
• Event-uncertainty – classic Risk Management
- named weather
www.enlightenedplanning.uk
This slide wasn’t shown, but there was an observation about risk as events only.
This is alternative portrayals of uncertainty from Chris new book ‘ Enlightened
Planning’
4

5. ‘Cause Association’
Cause Risk Event Consequence
Cause Risk Event Consequence
Cause Risk Event Consequence
- Systematic
optimism
- Team
Productivity and
Capability
- Material Costs
- Weather
Sticking with the Cause > Risk > Event despite its limitations. There could be
interdependencies on the cause side or the consequence side.
I call general cause side dependencies ‘Cause Association’ – you may also hear risk
experts talking about correlation.
Where the risks have common root causes they are more likely to happen / not
happen together – if they do happen then the are more likely to be higher or lower in
value together.
For instance risks will have been quantified by the same people – any systematic
optimism (or less likely pessimism) will affect all risks simultaneously. Similarly the
works may be undertaken by the same resources; any issues with productivity will
manifest in the likelihood and impact of the risks as a group.
5

6. Multiplying Effects
‘Qty of Units’
variability
+50%
‘Price per unit’
variability
+50%
Contractors OHP +10%
+
+
• Sub-component of project (£100), 3 risks occur:
Risk Register/Model: + £110
+50%
+50%
+10%
x
x
Final Account: + £147.50
Addition Multiplication
Here are some consequence side tricks. For instance if impacts are modelled as
additive when multiplicative would have been more appropriate.
In the example above if any risk occurs it affects the potential impact value of the
other items (in reality not in the simple additive risk model).
Consider that all the identified risks occur (this is more likely to happen than might be
assumed by the model if there is causal association). The actual impact will be
greater than that calculated in the model.
This is a very simple example – would see bigger effects if large numbers of variables
Real story – s/w development with offshore arrangement 1/3 cost of UK. SW was
more complexed than planned with 3x more effort and it had to be brought back to
the UK.
6

7. Prolongation
Cause Risk Event Consequence
Cause Risk Event Consequence
Cause Prolongation Consequence
Another common consequence side impact on projects.
Prolongation is a derivative of the other risks – not independent.
If other risk happen then prolongation is more likely – cannot happen spontaneously
on its own.
7

8. Why it matters:
• Real Project Data
• -40% to 164% overrun, 16% average
• Typical Simple Monte Carlo Cost Forecast
-5% 0% 30%
Two probability density functions showing cost overruns,
Left shows real data for early stage infrastructure projects, distinctive asymmetrical
shape with long tail. Project probably won’t overrun by much, but in the worst case it
could be bad’.
One on the right shows the output typical from a simple additive forecast ‘project will
definitely overrun but not by much’.
In my view the lack of realism is because the interdependencies are not being
appreciated:
Cause association increases the range of possibilities. Multiplicative effects do also –
probably there will be fewer virtuous cycles than you would like so an element of
asymmetry appears. Derivative impacts and complex knock ons like prolongation will
further accentuate the tail.
Conclusion:
Simple Addition can be a dangerous assumption. Projects can be thought of as a
system, the elements are work packages, resources, deliverables, risks etc. The
elements of the system may be ambiguous or poorly understood. There are
mechanisms between the elements – these may also be ambiguous. Assuming
8

9. simplicity may be very dangerous.
[Note to self: In future present this slide at the beginning, draw to scale and
superimpose on top of each other. ]
8

10. Identifying Interdependencies
• Ask and listen
• ‘Pairwise’ Review of key Risks in the Register
• A Visual Map or Influence Diagram
• Scenarios (or Stress Test)
9

11. Innovation
Wisdom of small (but well trained) crowds
This is a brief introduction to a project myself and colleague are currently working on.
Training component aims to make people better estimators of risk and uncertainty. In
conjunction with the wisdom of the crowd may provide an additional method gather
intelligence and insights on projects and the ecosystem.
This is an invitation to take part in the pilot.
10

12. Calibration Training
• Structured question sets with feedback
• Students become better at capturing uncertainty
Calibration training is scientifically validated vocationally (medics, weather
forecasters) but also works with general knowledge.
Performance improves over the course of the tests
App so can play on the train etc instead of candy saga
Takes about 1 hour (CPD credit)
11

13. Wisdom of the Crowd
• Sir Francis Galton famous analysis of
weight of prize winning bull
• Mean estimate from crowd of c800
visitors was accurate
• Relax precision requirements to a
90% ‘range’:
- 5 responses – Take minimum and
maximum
- 8 responses – Take the 2nd minimum and
2nd maximum
- 11 responses – Take the 3rd minimum and
3rd maximum, etc
Wisdom of the crowd correctly predicted weight of bull – to galtons surprise
(intellectual snob).
If we are interested in a range then much smaller numbers needed
If the crowd is trained (calibrated) can be a smaller crowd or better quality forecast
12

14. Star Performers
• How much did the Space Shuttle weigh at take off in metric tons?
Initial cohort of 40
Training showed who were the best performers (green line) x3 persons
Consistently got the measure of the crowd
13

15. Star Performers
• How long ago was the Boston (USA) marathon first run?
Average of the group was much lower estimate than the reality
Theory, Socially jogging is modern (70/80s ) phenomena
Star reasoned that boston was founded around 300 years ago - no justification or
data to go younger than this
Stars provided a better estimate
14

16. Pilot Invitation
• Android (at the moment)
• Calibration Training via the app
• All confidential
• Intelligence questions
- How long do you think it will be before the first truly self-driving car will be
available for public sale in the UK?
- How long until Notice to Proceed (permission to start works) will be given
to HS2 Main Works Civil Contractors?
• Contact
- Stephen.cresswell@intorisk.com / 07810 544073
After training complete will be asked intelligence questions
Answers will be shared with the piloteers.
Contact me if interested in taking part 0 and if you have an intelligence question!
15

17. Thank you
16

18. A Real Project - ‘System’
Penetration Rate
Planned
Maintenance
Unplanned
Maintenance
Unplanned
Stoppages
Drives Duration of
Tunnelling Works
Chalk Strength
Flint Presence & Profile
Water Pressure
Fissures & Caverns
Treatment Process 1
Treatment Process 2
Uncertainty - Ground Conditions
?Secondary Risk
Object
Mechanism
Key:
17

19. A Business
Value
System
18