The document discusses integrating risk management into the acquisition process to provide a more comprehensive evaluation of supplier proposals, identifying potential risks from supplier performance that could impact cost, schedule, and program success. It recommends using a maturity model to assess supplier capabilities and evaluate proposals, identifying risks, impacts, and mitigation strategies to select the supplier best able to meet program requirements and minimize risk exposure. Integrating risk management in this way aims to avoid financial losses from issues originating from supplier performance throughout the program lifecycle.

1. Getting What you Paid for:
Acquisition Risk Analysis
James Taylor
Omnia Paratus
Corporation
Used with Permission
0

2. Lamborghini Reventon
$1.6M
Kia Rio
$12,145
Omnia Paratus
Corporation 1

3. A recent poll showed that 68% of companies experienced financial
losses directly related to Supply Chain disruptions
 Most of the financial impacts were related to supplier performance that did not meet demand
requirements, and delayed, damaged or misdirected shipments.
 The majority of companies polled are in the early stages or have yet to think about integrating Risk
Management into their Supply Chain
 Supplier related risks are most often identified after contract award is a program issue, not a risk:
– Supplier shipment delay
– Supplier capacity exceeded
– Unable to meet technical requirements
– 1st Article or Flight Test failures
– Parts damaged during shipment or rejected during quality inspection
 Schedule delays caused by supplier performance can have an equal or greater financial impact to
a program’s bottom line, but are often overshadowed budget impacts due to supplier cost overruns
Omnia Paratus
Corporation 2

4. Procurement analysis exists in varying degrees, although the most
commonly used practice is Best Value Analysis (BVA)
Supplier w/ Existing New
Incumbent
Capabilities Supplier
Score based on
savings to program
~$300K
Technical Score based on:
- New Capability to Supplier
- Technical deviations req’d
based on proposal
- 1st Article & Flight Req’d
based on Customer reqt’s
 BVA considers not only cost, but other quantifiable and non-quantifiable factors supporting an
investment decision
– Utilizes weighting scales for analyzing “True” program value of supplier bids
– Can include, but is not limited to, performance, producibility, reliability, maintainability, and
supportability enhancements
– Intended to select the source offering the greatest overall benefit in response to the requirement
Omnia Paratus
Corporation 3

5. While a BVA attempts to provide insight to the least risky procurement,
it does not specifically address or integrate impacts of risk
Design changes resulting from
Technical deviations resulted in Technical Deviations
cost growth to original required by Supplier
purchase order
Impact from production
delays of 4 weeks
Program Award fee lost
resulting in Supplier
delays cause IMS Orders placed were
milestones to be missed against original design
and fulfilled by
Incumbent suppler
Fees required to get
Incumbent supplier
operations back up and $390K overrun was a direct
minimize delay of loss of company profit
deliveries to Customer
 Performing a sample analysis of a new supplier shows the potential risks that may be incurred when
basing decisions on costs alone
– Technical requirements outside current capabilities
– Schedule impact due from potential delay of 1st Article Testing or Flight Test Requirements
– Cost growth due to technical deviation’s required
– Impact to Operations due to late supplier deliveries
Omnia Paratus
Corporation 4

6. Despite the value-add it offers to improving the acquisition process,
risk management is seldom considered or implemented
 Risk Management Is Not Difficult, Is Well Documented, and Training
Materials Are Readily Available. So Why Is It “Hard”?
– Seldom seems urgent
 It’s a “lower right quadrant” activity
High
Urgency
 It’s often overcome by events
 It’s someone else’s job
– Requires careful thought
Low
 People think it’s “easy” because it’s not difficult
 Fail to distinguish between perception and reality Low High
 Skip the analysis and solve the wrong problem Importance
 Determining what can be controlled, influenced, or changed
– Team participation Risk
Management
 Part of the culture
 Common understanding
 Training, support, and reinforcement
Omnia Paratus
Corporation 5

7. Risks that impact large scale programs often directly originate from
supplier performance throughout all phases of a program
 The main challenges that confront these programs include:
– Program Requirements – Technical requirements can have a tendency to increase unknown
potential for inhibiting program success or realization of full award fee are often over looked
while developing scope of work and supplier selection process.
– Source Selection Analysis - A review of industry standard Best Value Analysis or Lowest
Cost Alternative approach reveals gaps in several areas of Procurement Analysis skewing the
perception of results derived, inherently selecting a supplier who may not adequately meet
program requirements.
– Risk Identification – Most risks identified often have root causes stemming from supplier
performance and/or capabilities, these risks tend to be identified after procurement award.
– Risk Impact Analysis – Procurement analysis does not provide insight into the potential cost
and schedule impacts associated with selecting a supplier, impacts that could affect the
success or ability of program operations, award fee and sustaining customer contracts.
– Mitigation Analysis – Developing a mitigation plan after a problem has occurred limits a
program’s mitigation options resulting in exuberant mitigation costs that extend beyond
program office and/or client budget.
Omnia Paratus
Corporation 6

8. Integrating Risk Management directly into the acquisition analysis
process is seamless and beneficial
Request for Proposal
Proposal Evaluation Contract Award
Development
Description
 Develop and issue RFPs based  Evaluate proposals using a  Analyze potential risk impact
on a standard format; collect clear and structured evaluation and mitigation posture relating
vendor proposals mechanism and methodology to supplier proposal’s
 Assess program requirements  RFP responses collection  Supplier proposal risk analysis
 Assess supplier capabilities
Activities
 Technical and contractual /  Identify potential mitigation plans
 Assessment methodology & procedural evaluation  Assess mitigation posture
criteria development  Risk identification  Integration of risk impact into
 Define risk parameters  Risk ratings defined program budget and schedule
 Detailed functional and technical  Technical, contractual /  Pre-Mitigation Analysis
Outcome
requirements procedural and commercial  Post-Mitigation Analysis and
 Evaluation methodology evaluation of proposals Effectiveness
 Scoring model for RFP evaluation  List of potential risks and  Program level risk adjusted
 Vendor bidders list risk ratings within individual cost and schedule analysis
proposals  Supplier selection
 RFP document
 Risk rating scales & categories  Contract Award
Compare supplier risk profile at
Assess program requirements, Provide insight into where supplier
proposal evaluation completion to
supplier capabilities and evaluation risks affect the program and uncover
determine outstanding risk
criteria to establish RFP their true impacts.
exposure.
Omnia Paratus
Corporation 7

9. Acquisition Risk Analysis follows a typical Risk Management process
and can be tailored to program specific needs
1 Risk 2 Risk Risk 4 Risk Applying
3 5
Planning Identification Analysis Mitigation Risk Results
Risk
Identificat
• Creating standard risk • Collecting SME input, • Conduct probabilistic • Identifying, evaluating, • Using cumulative risk
ion
terms and definitions best practices, assessments of risk and selecting scores as a vehicle for
metrics, etc., to impacts strategies to reduce ranking alternatives
• Developing processes, identify possible risks risk
criteria, and scoring
approach for risk • Documenting risk • Developing an
data into a central actionable risk
repository mitigation plan, with
sound rationale (if
applicable)
Identifies risks that
Sets risk parameters for Examines risks to Develops measures for Uses results of risk
may impact cost,
effective risk analysis determine impacts by keeping risk at analysis in assessing
schedule, or
and across alternatives acceptable levels alternatives
performance
Omnia Paratus
Corporation 8

10. Building qualitative definitions for risk ratings enables an objective, not
subjective quantification of proposal risks
 The first step in Risk Planning involves defining standard risk terminology using many different resources, but
should be robust and accurate enough to reflect the program’s overall risk tolerance
 Developing a robust, well-defined risk analysis process will produce results that reflect program needs; effectively
communicates these needs across stakeholders; reduce personal bias in determining relative risk importance; and
uncover potential impacts of great importance to the program
 Risk criteria is not normative, and are based on available resources, time constraints, and amount/type of
information solicited in generating the criteria
Considerations for Defining Risk Criteria
Performance The analyst needs to ensure that the units of measure for risk impact reflect program needs. Critical program drivers might include life-cycle
Metrics costs or metrics reliability (e.g., Mean Time Between Failure).
If impact thresholds are expressed in percentages (for example) the analyst needs to review those figures within the context of the overall
Order of Magnitude anticipated program cost—e.g., 10% of $5 million program is substantially different than 10% of a $5 billion program.
The analyst needs to ensure that there is an adequate number of thresholds for evaluating the risk impact. The more levels of consequence
Level of Impact that are utilized, the greater the insight into the need for increased data fidelity/quantity of data the analyst would need to consider.
Qualitative data is represented by probability values ranging from “Very Unlikely” to “Very Likely” , using stakeholder input to assess and weigh
the importance of benefits, cost, and risk in relationship to the total analysis or evaluation of benefit, risk, and cost factors that cannot be
quantified.
Qualitative vs.
Quantitative Quantitative data is represented by consequence values using linear numerical probabilities (e.g., 0.1, 0.3, 0.5, 0.7, and 0.9), nonlinear
numerical probabilities (e.g., 0.05, 0.1, 0.2, 0.4, 0.8), cost estimates, metrics, ranges, or percentages.
Omnia Paratus
Corporation 9

11. A Supplier Management Maturity module is used to determine if
supplier capabilities comply with program requirements
1 – Minimal Capability 2 – Functional 3 – High Performance 4 – Best in Class
 Framework used to assess each IPTs risk
 Minimal understanding or  Dedicated resources do not  Applied use of specific Regular training conducted to enhance
experience in applying basic exist processes/tools skills and capabilities management maturity in terms of People, Process,
People
concepts and principles  Limited to individuals who may  Dedicated team resources  Dedicated organizational resources
 Minimal understanding of
principles or language
have had little or no formal
training
 In-house core experts, formally
trained in basic skills
All staff is informed and capable of
applying mid- to advanced concepts
Technology, and Governance
 Inconsistent application of  Common processes defined  Formal processes integrated into Qualitative/quantitative analysis
 The optimal goal of the risk enhancement effort is
concepts or principles
 Formal risk decision-making
and formally documented
 Process effectiveness limited to
different areas of the program methodologies employed with emphasis on
valid and reliable data sources
to select supplier cable of bidding based on their
Process
body does not exist
 Risk reporting and/or metrics is
a dedicated team
 Qualitative analysis based on
 Active allocation and
management of budgets
 Metrics used and reported, with
consistent feedback for improvement
“Best in Class” maturity
minimal or does not exist ill-defined rating system  Metrics collected  External stakeholders actively
 Formal, documented process  Established decision-body  Key internal stakeholders actively participate in process
does not exist forum participate in process Integration into organizational processes
and decision-making
NOTIONAL
• No structured application • Customizable solutions tailored • Integrated set of tools and  State-of-the-art tools and
Technology
 Management and tracking tools to program methodologies methodologies
not in use • Few repeatable technological  Centralized data environment  Distributed data environment that
 Analysis not performed solutions in place managed by dedicated team provides access to all program
resources resources
 Standardized and automated reporting
capabilities
• Minimal awareness  Upper management • Accepted as a program  Integral to informed decision-making by
 Minimal upper management encourages, but does not management function upper management
Governance
involvement require use  Benefits recognized and expected  Active use encouraged and rewarded
 Tendency to continue with  Application varied through out  Upper management requires  Part of the organizational philosophy to
existing processes even in the program tracking and reporting achieving program success
face of potential failure • Process may be viewed as  Focus on mitigation effectiveness  Top-down commitment by leadership
 Dedicated resources do not exist additional overhead with versus reporting and status
variable benefits tracking
Average
People Process Tech Gov’t Metrics Quality Capacity
Rating
Supplier A 3 4 4 1 2 2 1 2.42
 Supplier’s will be scored based on the Supplier B 2 3 - 3
NOTIONAL 3 1 1 1.85
maturity model to establish a list of Supplier C 3 3 4 2 3 3 2 2.85
qualified suppliers for proposal
solicitation Supplier D 3 3 4 4 3 3 2 3.14
Omnia Paratus
Corporation 10

12. Once proposals have been received a technical evaluation should be
conducted for clarifications prior to identifying supplier risks
Proposal Clarification Items
Ref.
Issue
Topic Area Questions Answer Given
Resolved
 Is the team available as indicated in the Proposal? P. 60
Staffing  Additionally, a further elaboration on local staff/ qualifications is
required  All questions to a
 A further elaboration on the timeline is required, especially on the 9 P. 46 Bidder should be
days timeframe proposed for defining the architecture and standards compiled and sent
General  A show-case of the deliverables must be provided in a formal email /
Approach
 How realistic are the assumptions on available data and fax
Organization staff?
 All Bidders should
Assumptions  What is the impact on costs and timeline of a provision of a P. 18 be given ample and
documentation in other languages? the same amount of
Phase 0: Kick-Off Alignment  How will Bidder ensure the alignment of the PM tools to the P. 34 time to respond to
Organization’s PMO?
clarifications
 How will the automatic “systems” feed work (XML). Is it a P. 36- requested
requirement? 38
 How will any incompatibilities of import functions affect the
Phase 1: Baseline Tools suggested timeline?
 What if the Organization wants to use other EA tools, like ARIS?
 Further elaboration on “GEAS” layers required. A mapping to the
Organization framework layers is needed as well
 Explanation of the rationale for selecting Singapore, Australia and P. 43
Phase 2: Best Canada
Approach
Practice Review  Is the access to those countries only via databases/benchmarks or
“real” contact?
Phase 3: Definition  No reference given, on how the Organization’s imperatives will be P. 45
of Architecture & Imperatives ensure – therefore, a further elaboration on the approach is required
Standards
Omnia Paratus
Corporation 11

13. The next step to the risk assessment process is to identify risks that
may impact a program’s cost, schedule, or technical performance……
 The risk analysis focuses on risks that affect each supplier’s ability to ensure program success (i.e.,
ongoing, uninterrupted support to the operational forces).
Risk
 Although identification of risk relies on the skill, experience, and insight of subject matter experts and
Identificat
risk personnel, the methods and tools for initiating the identification of risk may vary
ion
 As such utilizing risk categories such as these bolsters risk identification…
Budget
Suppler Metrics Schedule Scope of Work
Constraints
Supplier Performance “Common” Risks Lessons Learned
Capabilities Metrics
Material Testing & Qualification
Resources
Availability Integration Requirements
Sensitivity Program
Components Management
Analysis Requirements
Omnia Paratus
Corporation 12

14. …each risk is then reviewed against a set of Program-specific impact
definitions, with the highest rating used to determine overall risk severity
Cost ( % over of cost
Level Technical Supplier Delivery Schedule
target)
Disastrous Existing technology does not exists >5 month slip in MRP requirements > 6.00%
Severe Existing technology exists, but has not been proven 4-5 month slip in MRP requirements 5.01 – 6.00%
Critical Supplier has never built component before, Flight Test required 3-4 month slip in MRP requirements 4.01 – 5.00%
Substantial Supplier has never built component before, 1st article required 2-3 month slip in MRP requirements 3.01 – 4.00%
Significant Supplier has never built component before 1-2 month slip in MRP requirements 2.01 – 3.00%
High Supplier built similar component, Flight Test required < 1 month slip in MRP requirements 1.01 – 2.00%
Moderate Supplier has built a component similar in nature, 1st article required Risk erodes 100% of schedule margin .76 – 1.00%
Medium Supplier has built identical component for other Programs Risk erodes 51 – 75% of schedule margin .51 - .75%
Low Supplier has previously built component Risk erodes 26 – 50% of schedule margin .26 - .50%
Minimal Incumbent supplier Risk erodes < 25% of schedule margin < .25%
Technical = High 3 Schedule = Critical 1 Cost = High 2
Consequence Rating = Critical
Omnia Paratus
Corporation 13

15. Risks are then integrated into the suppliers proposal to determine
impacts of risks based on the risk ratings identified
 The risk database transposes risk rating into cost and schedule impacts in quantifiable dollar and
days
 WBS and IMS task are later used to integrate supplier risk adjusted proposal into the Program’s
Budget and IMS
Schedule
Risk WBS IMS Risk Cost Impact
Risk Description Prob. Conseq. Impact
# Impact Task ID Rating ($K's)
(Days)
Lack of supplier staff to support delivery
1 1.2.1.3 3 8 31.00
53 requirements $7,500 88
2 2.3.1.6 71 CCB is a development item 5 9 71.25 $8,500 118
Supplier X's On Time Delivery rating
3 4.3.5.1.1 2 10 28.50
101 subpar $9,500 150
Requirements of flux capacitor increase
4 6.3.3.7 the need for Supplier X System Test & 4 6 32.25
88,97 Eval staff and resources $5,500 52
Lack of technical capability in power
5 1.3.1.5 generation / storage hardware may cost / 1 3 3.30
70 schedule over runs $2,500 18
1.3.1.2 Tooling re-use approach may not be
6 4 10 71.25
1.3.3.15 67,97 compatible with new technology $9,500 150
Omnia Paratus
Corporation 14

16. Monte Carlo Analysis is used in calculating risks associated with
supplier cost impacts to program budget….
 This chart example utilized Crystal
Ball to perform Monte Carlo, multiple
types of software exists and can be
used for this analysis
 Pick a Confidence Level based on
program maturity and requirements
 This example highlights the 75th
Percentile
– 75% of costs are below the line,
25% of costs are above
– The 75% CL is $49,448
Omnia Paratus
Corporation 15

17. ….as well as supplier risks that have the potential for disrupting a
program’s Operations schedule.
 The analysis example utilized
SCRAM to run the schedule
risks analysis
 Confidence Level (CL) picked
for cost is also used for
schedule to determine how
risk may impact a supplier’s
delivery schedule
Omnia Paratus
Corporation 16

18. Schedule & Cost Risk Assessment Module (SCRAM) is a MS
Project add-in and FREE for use on NASA projects.
 SCRAM’s capabilities
compare with that of
Pertmaster, @Risk and
Risk++
 Extremely user friendly and
reliable
 Customizable aspects not
available with other tools
 Compatible will all MS Office
Products
Omnia Paratus
Corporation 17

19. Once Monte Carlo simulations are complete and confidence levels
selected, supplier bids can be compared based on potential risk impacts
Supplier A Supplier B Supplier C
Proposed $ 494,958 $ 465,135 $ 514,453
10% $ 507,645 $ 512,622 $ 547,565
20% $ 509,542 $ 523,002 $ 555,345
30% $ 510,834 $ 530,892 $ 561,676
40% $ 512,016 $ 537,855 $ 567,689
50% $ 513,090 $ 544,29 $ 573,385
60% $ 514,241 $ 551,640 $ 579,471
70% $ 517,449 $ 552,673 $ 585,912
80% $ 520,923 $ 565,114 $ 594,321
$500,000,000 $600,000,000 $700,000,000
90% $ 523,007 $ 582,310 $ 607,304
100% $ 629,484 $ 589,685 $ 711,107
Supplier B
Confidence Level Chosen Supplier A
Supplier C
Omnia Paratus
Corporation 18

20. Side-by-side risk exposure calculations provides leadership with
comparative insights into supplier potential costs impacts
Analysis will lead to a cumulative
assessment of the total risk exposure All costs reported at the
and the potential impact to program 90% confidence interval.
budget.
$39,100
Potential Risk
$650,000
Potential Risk Impact $179K
Potential Risk Impact $99K
Program Cost (in thousands)
$600,000 Impact $25K
$550,000
$500,000
$450,000
$400,000
Component
$350,000
budget $500K
$300,000
$250,000 $497,042 $482,331
$200,000
$427,436
Supplier Initial Cost
$150,000
Risk Exposure
$100,000
$50,000
$0
Supplier A Risk Adjusted Bid Supplier B Risk Adjusted Bid Supplier C Risk Adjusted Bid
Omnia Paratus
Corporation 19

21. Risk adjusted delivery schedules are then compared to determine
the potential risk impact to program’s operations
Supplier A Supplier B Supplier C
9 8 4.5
8 4
7
7 .5
6
6 3
5
5 .5
4
4 Illustrative Illustrative 2
Illustrative
3
3 .5
2
2 1
1
1
0.5
0 0 0
5/3/10 5/7/10 5/11/10 5/15/10 6/2/10 6/6/10 6/10/10 6/14/10 6/18/10 6/22/10 7/7/10 7/11/10 7/15/10 7/19/10 7/23/10 7/27/10 7/31/10 8/4/10 8/8/10
Supplier’s Initial Proposal
Supplier A B C
Proposed Schedule 04/18/2010 04/04/2010 04/21/2010
Risk Adjusted Schedule 05/13/2010 07/18/2012 08/21/2012
Material Resource Planning requirement for this component is 04/25/2010
Omnia Paratus
Corporation 20

22. The risk-adjusted cost and schedule results are then compared for
awarding contracts on a risk averse path
Supplier’s Initial Proposal Supplier’s Risk Adjusted Proposal
Supplier A B C Supplier A B C
Risk Adjusted Cost $ 523,007 $ 582,310 $ 607,304
Proposed Cost $ 497,042 $ 482,331 $ 427,436
Risk Adjusted
Proposed Schedule 04/18/2010 04/04/2010 04/21/2010 05/13/2010 07/18/2012 08/21/2012
Schedule
 Initial review of Supplier bids would indicate “C” as the supplier of choice
 Based on Supplier B & C delivery metrics and potential risk, schedule impact could result in
more than a 3 months past MRP requirements
 Based on supplier C’s lack of technical capabilities and schedule risk to operations, risk impact
could equate to ~$100K over component budget
 Based on risk adjusted Cost & Schedule proposal analysis supplier “A” should receive program
consensus for contract award based on least amount of risk exposure to the program
Omnia Paratus
Corporation 21

23. Given these potential benefits, a few key considerations are worth
noting
 In order to have a successful portfolio risk management process, it’s important that the
constituent components of the program have sufficiently mature supply chain management
and risk management processes.
 Integration of acquisition risk analysis into a program’s budget and schedule is necessary to
capture the magnitude of potential program risk impact by a single supplier
 Identifying risks within a proposal enables forward looking program management that can be
streamlined into existing risk database’s for future risk management planning and mitigation.
 Qualitative risk analysis provides enhanced proposal evidentiary support and solid justification
for awarding contracts
 The success of a supply chain risk management program requires the consistent and active
support of program leadership in order to be successful.
Omnia Paratus
Corporation 22

24. For more information on how acquisition risk analysis can be
applied to your specific challenges, please:
 Contact:
James Taylor
Huntsville, Alabama
310-462-6878
James.Taylor@omniaparatus.com
Omnia Paratus
Corporation 23