Project success starts with a credible plan for delivering the needed capabilities as planned. This starts with knowing what Done looks like in units of measure meaningful to the decision makers.

1. Creating Credible Plans,
Integrated Reporting,
and Control Systems
Part 1
Glen B. Alleman
Niwot Ridge L.L.C.
glen.alleman@niwotridge.com
+1 303 241 9633
IPM Workshop 2015
Thomas Coonce
Cost Analysis and Research Division
Institute for Defense Analyses
tcoonce@ida.org
+1 703 575 6634
Rick Price
Project Management & Planning
Operations Principal
Lockheed Martin Space Systems Company
Rick.a.price@lmco.com
+1 303 971 1826

2. 2
Learning Objectives
• TLO #1 – the student will understand how to apply the 5 core
processes needed to produce a credible Performance
Measurement Baseline.
• TLO #2 – the student will understand how to objectively
measure progress and use the performance information to
control the program.
GA

3. 3
Building a credible PMB for Wilbur and
Orville’s flying machine.
• Our talk today uses the Wright Brothers
Flyer as an example for developing a
Credible PMB
• The Urban Legend of Orville and
Wilbur's efforts to invent manned flight
has been revealed in McCullough’s
book.
• Their work has all the attributes of the
Five Core Processes.
GA

4. 4
There is a Method to our Madness
• If you’re over 60 you’ll recognize
this chart template for HIPO, used
to describe Inputs, Processes,
Outputs we’re using to create a
credible Performance
Measurement Baseline, measure
progress, and control the program
• These are the minimum HIPO
elements, there can be and should
be more.
GA

5. 5
The data and processes needed to create a
Credible Performance Measurement Baseline
GA

6. 6
A Readable Version
The Five processes needed to build a credible PMB
❶ Define Done in Units of Measure Meaningful to the
Decision Makers.
❷ Develop the Plan and Schedule to reach Done as planned.
❸ Define the Resources Needed to Reach Done as Planned.
❹ Adjust Plan for Reducible and Irreducible Risks.
❺ Finalize the Risk Adjusted Measures of Progress Toward
Done.
GA

7. 7
Inputs and Outputs for Building the Credible
Performance Measurement Baseline
• Customer Capabilities
• Time phased budget
• Desired completion date
• Available technologies
• Time phase resources
• Reference Class Data
• Programmatic and technical risks with
probabilities and impacts
• Historic uncertainties
• Measures of Effectiveness
• Measures of Performance
• Work Breakdown Structure
• Integrated Master Plan
• Technical Performance Management
Plan
• Initial Integrated Master Schedule
• Time Phased Staffing Plan
• Schedule Reserve
• Management Reserve
• Risk Register and Mitigation Plans
• Adjusted Integrated Master Schedule
• Risk Management Plan
Inputs Outputs
GA

8. 8
❶ DEFINE DONE
Define Done in units of measure meaningful to the decision makers
for each deliverable in the WBS.
Measures of Effectiveness (MOE), Measures of Performance
(MOP), Technical Performance Measures (TPM), and …illities are
defined before the work starts, for each needed capability that
accomplishes the mission or fulfills the Business Case.
TC

9. 9
Systems Engineering Concepts Used By
The Wright Brothers [1]
SE Concept Wright BrothersActivities
Operational
Concept
Vision: clear concept of the system they were developing
MissionRequirements: powered flight for substantial timeand distance
Scenarios: recognitionof a key scenarioinvolvinglateral control during flight
Define the System
Boundary
Understoodthe need for the pilot to be part of the system
Objectives Definitionof distanceand time objectives
Derived
Requirements
Horsepower and weight of the engine
Thrust of the propellers
Requirements
Management
Managed weight requirement of the engine carefully
Functional
Analysis
Understood the functions of the airplane, which drove their design process
Further understood the functions of the propulsionsubsystem
Physical
Architecture&
Interfaces
Matched the physical architecture tothe functions of the airplane
Interfaces between the pilot and the control surfaces
Interfacebetween the engine and propellers TC

10. 10
Systems Engineering Concepts Used By
The Wright Brothers [1]
SE Concept Wright BrothersActivities
Prototypes and
Testing
Built a successfullymorecomplex set of gliders before building the airplane
First airplane was successful,due to the series of gliders and kites
Employed sophisticatedtestingwitheach prototype, atypical of others at the
time
Trade-Off
Decisions
Design favoredcontrol of the airplane over stability
Test System
Extensiveevenduring glider tests
Allowed problem identificationand solvingduring all test phases
Created wind tunnel to address discrepancy between their test data and the
literature
Included the careful selectionof the test site
Included in the first airplane
Verification
Best illustratedby testingof the engine to determine whether it met its
requirements
Validation The final test that proved flight was possible
Team Activities Two brothers formed a design-build team TC

11. 11
Measure of Effectiveness (MoE)
Measures of Effectiveness …
• Are stated in units meaningful to the buyer,
• Focus on capabilities independent of any technical
implementation,
• Are connected to the mission success.
Operational measures of success that are closely related to
the achievements of the mission or operational objectives
evaluated in the operational environment, under a specific set
of conditions.
MoE’s Belong to the End User
“Technical Measurement,” INCOSE–TP–2003–020–01
11 TC

12. 12
Measure of Performance (MoP)
Measures of Performance are …
• Attributes that assure the system has the capability and capacity to
perform,
• Assessment of the system to assure it meets design requirements to
satisfy the MoE.
Measures that characterize physical or functional attributes
relating to the system operation, measured or estimated
under specific conditions.
“Technical Measurement,” INCOSE–TP–2003–020–01
MoP’s belong to the Program – Developed by the Systems
Engineer, Measured By CAMs, and Analyzed by PP&C
12 TC

13. 13
Technical Performance Measures (TPM)
Technical Performance Measures …
• Assess design progress,
• Define compliance to performance requirements,
• Identify technical risk,
• Are limited to critical thresholds,
• Include projected performance.
Attributes that determine how well a system or system
element is satisfying or expected to satisfy a technical
requirement or goal
“Technical Measurement,” INCOSE–TP–2003–020–01
13 TC

14. 14
What Done Looks Like for the Wright Brothers’
Military Flyer
• Propose $25,000 to build and
deliver a flyer by 1st week in
August 1908
• Conduct Acceptance testing for
30 days thereafter
Contract
Specification
Value/Goal
Passenger # Two persons
Payload 350 lbs
Range 125 miles
Air Speed 40 mph
Duration Aloft One hour
Maneuverability
Be steered in all directions
w/o difficulty and in perfect
control at all times
Ease of Use
An intelligentman become
proficientwithin a
reasonable length oftime
Logistics
Assembled and
disassembled within one
hour and packed into an
Army wagon
TC

15. 15
❷ DEVELOP A PLAN FOR
TECHNICAL SUCCESS
The Technical Plan shows the sequence of work needed to
produce products and services that move the program toward
Done.
The Technical Plan can take many forms. From a formal Integrated
Master Plan / Integrated Master Schedule to a Kanban work plan or
Scrum Release/Sprint/Task Plan in Agile Software Development
TC

16. 16
Develop the Plan and Schedule
• Define the technical approach to building the product that will
allow it to meet the performance and effectiveness goals
– Identify all the variables that will impact the solution to the
technical and operational requirements
– Orville and Wilbur used the spiral development approach for
the Wright Flyer
• Sequence the work to assure the increasing maturity of the
deliverables supports the plan to reach Done.
TC

17. 17
The Wright Brothers’ Technical Plan
• Modify aircraft to reduce weight from 845 to 800 pounds
• Modify aircraft and controls for two persons
• Increase HP from 21 to 31 horsepower
• Apply a Modify – Test – Modify approach
• Spend lots of time practicing (and not kill yourself)
• Dry run assembly and disassembly procedures
TC

18. 18
❸ DEFINE THE
PROGRAMMATIC PLAN
The Programmatic Plan describes all the activities that enable the
Technical Plan to function properly. What work do we need to do in
what order and what’s the cost of that planned work.
What facilities, resources, equipment, and materials are needed to
reach done as planned.
RP

19. 19
Define the Programmatic Plan
• Assure the programmatic plan aligns with the technical plan
• Qualified Staffing Plan to produce the needed outcomes at the
needed time
• Facilities and equipment needed to produce the products
• Material availability and readiness Plan
• Phased funding available to sustain the planned progress of the
Technical Plan
RP

20. 20
Wright Brothers’ Flyer Resources Needed
• Facilities and equipment for modifications
– Fabrication hangar @ Kitty Hawk
– Launching track
– Catapult
• Materials (propellers, wood, fabric, sewing supplies, chain, etc.)
• Test areas
– Kitty Hawk at Kill Devil Hills
• Requisite labor skills
– Orville & Wilbur Wright
– Charlie Furnas (Dayton mechanic)
– Charlie Taylor (engine design/builder)
– Carpenters at Kitty Hawk (to rebuild the hangar)
• Money to support the development/build effort
• GFE
– Site availability at Ft Myer, VA
– Hangar shed at Ft Myer, VA
RP

21. 21
Initial Wright Brothers’ Summary Level IMS
RP

22. 22
❹ ADJUST FOR REDUCIBLE
AND IRREDUCIBLE RISK
Any impediment reduces the probability of success. Each
impediment must be identified, a plan to address the impediment
developed, each handling strategy applied, monitored, and
measured to assure there is an increasing probability of success.
GA

23. 2323
Uncertainties are
things we can not
be certain about.
Uncertainty is
created by our
incomplete
knowledge; not
by our ignorance
GA

24. 24
2 Types of Uncertainty – Both Create
Programmatic and Technical Risk
Uncertainty
Irreducible
(Aleatory)
Reducible
(Epistemic)
Natural Variability
Ambiguity
Ontological
Uncertainty
Probabilistic Events
Probabilistic
Impacts
Periods of Exposure
GA

25. 25
Assemble a credible WBS and Integrated Master Plan / Integrated
Master Schedule (IMP/IMS)
–WBS Dictionary says what will be built
–IMP/IMS says how, where, when, and what will be built and the
processes to build it.
Identify Reducible – Event Based – uncertainties and the
resulting risks from WBS Dictionary and IMP Narratives in the
Risk Register
Put these risks in the Risk Register, with probability and impacts
Develop risk reduction plans and place them in the IMS using
CBB funding.
How to Build a Risk Adjusted IMS in 8
Steps
0
1
2
3
RP

26. 26
Assess the Irreducible – naturally occurring –
uncertainties and the resulting schedule and cost
risks in the WBS and IMS.
Use Monte Carlo Simulation to determine schedule
margin and budget impacts from work activity
durations to handle these irreducible uncertainties.
Formulate schedule and cost margin to protect key
item deliverables in the IMS.
How to Build a Risk Adjusted IMS in 8
Steps
4
5
6
RP

27. 27
Determine cost and schedule impacts of unmitigated
risks in the risk register with their exposure and impact
and place them in Management Reserve and Schedule
Margin
Assemble mitigated Irreducible (Aleatory) and
Reducible (Epistemic) risks with the unmitigated event–
based risks into the Total Allocated Budget
– Risk handling plans included in baseline for reducible
and irreducible in the PMB.
– Risk Handling in Management Reserve.
Building a Risk Adjusted IMS in 8 Steps
(Concluded)
8
7
Adjust the IMS with rearranging parallel or serial work, changing scope, or add
resources.
RP

28. 28
Wright Brothers’ Risk Register and Risk
Strategy
1. Control. Buy $1.6K of key spare parts and send to Kitty Hawk
2. Accept. Add schedule margin in schedule
3. Accept. Add schedule margin in schedule
4. Accept. Add schedule margin in schedule
TC

29. 29
Wright Brothers’ 1908 Flyer Management
Reserve Calculation
Element Cost CL Comments
P80 Cost $19,696 80%
This is the cost at the
80% CL assuming
only non-reducible
risks in the RR
Deterministic
BACf
$16,355 < 1%
Final BAC based on
the resource-loaded
schedule
MR (P80
Cost - BACf)
3,341 Calculated MR
MR as a
percent of
BACf
20.4%
Caculated MR % of
BACf
TC

30. 30
Wright Brothers’ 1908 Flyer Schedule
Margin Calculation
Element
Finish
Date CL
Months
Work
Days
IMS
Deterministic Pf
Duration and
Date:= 7.25 152 9/15/1908 5%
Need Duration
and Date:= 7.75 163 9/30/1908 80%
Schedule Margin:= 1 11
Duration
TC

31. 31
Schedule Margin in Final WB’s Flyer IMS
TC

32. 32
❺ DEVELOP OBJECTIVE
MEASURES OF PROGRESS
For each deliverable develop an objective measure of progress to
plan using physical percent complete supported by Quantifiable
Backup Data (Objective Criteria)
Use these measures to assess progress and identify variances
requiring corrective actions to Keep the Program Green.
GA

33. 33
Objective Measures of Progress to Plan
• Measures of Effectiveness (MOE)
• Measures of Performance (MOP)
• Technical Performance Measures (TPM)
– Weight TPM Plan
– Risk Burn Down Plan
– Schedule Margin Burn Down Plan
• All the other …ilities
– Availability (assemble within one hour)
– Usability (simple in construction and operation)
GA

34. 34
Weight TPM for Wright Brothers 1908 Flyer
770
780
790
800
810
820
830
840
850
860
870
15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08
Pounds
15-
Feb-08
28-
Feb-08
15-
Mar-08
31-
Mar-08
15-
Apr-08
30-
Apr-08
15-
May-
08
31-
May-
08
15-Jun-
08
30-Jun-
08
15-Jul-
08
Upper Limit 860 855 845 835 825 815 800 800 800 800 800
Planned 845 840 830 820 810 805 800 800 800 800 800
Target 800 800 800 800 800 800 800 800 800 800 800
GA

35. 35
What’s Next
• TLO #1 – the student will learn how to produce an integrated report
of the Technicalprogress consistentwith the Programmatic
progress, detect variances,and suggestcorrective actions.
• TLO #2 – the student will understand how to objectively measure
progress and use the performance to manage the program.
• Earned Value Managementis part of a Technicaland
Programmatic control system
– Integrated risk, performance, effectiveness,technicaland other
measures provide leading indicators of program performance.

36. 36
References
1. “The Concepts of Systems Engineering as Practiced by the
Wright Brothers,” Dennis Buede, Stevens Institute of
Technology School of Engineering, Hoboken, NJ 07030

37. Executing the Credible Plan,
Using an Integrated Reporting
and Control System
Part 2
Glen B. Alleman
Niwot Ridge L.L.C.
glen.alleman@niwotridge.com
+1 303 241 9633
IPM Workshop 2015
Thomas Coonce
Cost Analysis and Research Division
Institute for Defense Analyses
tcoonce@ida.org
+1 703 575 6634
Rick Price
Project Management & Planning
Operations Principal
Lockheed Martin Space Systems Company
Rick.a.proce@lmco.com
+1 303 971 1826

38. 38
Learning Objectives
• TLO #1 – the student will learn how to produce an integrated
report of the Technical progress consistent with the
Programmatic progress, detect variances, and suggest
corrective actions.
• TLO #2 – the student will understand how to objectively
measure progress and use the performance to manage the
program.
• Earned Value Management is part of a Technical and
Programmatic control system
– Integrated risk, performance, effectiveness, technical and
other measures provide leading indicators of program
performance.
GA

39. 39
In Part 1, we talked about …
• Addressed the five core processes needed to produce a
credible Performance Measurement Baseline (PMB).
• Showed how to use objective measures of progress and how to
inform program performance with these measures
• In Part 2, you’ll see how to execute the program with this
information.
GA

40. 40
Executing the Performance Measurement
Baseline
GA

41. 41
A Readable Version
The Four processes needed to execute the credible PMB
❶ Compare Technical and Programmatic Actuals to Plan
❷ Compare BCWP and Cost and Schedule Performance
Data with Management Reserve (MR) and Schedule
Margin (SM) balances
❸ Revise Integrated Master Schedule (IMS) and Risk
Register
❹ Run or Rerun Monte Carlo Simulation
GA

42. 42
Inputs and Outputs for Building the Credible
Performance Measurement Baseline
• Customer Capabilities
• Time phased budget
• Desired completion date
• Available technologies
• Time phase resources
• Reference Class Data
• Programmatic and technical risks with
probabilities and impacts
• Historic uncertainties
• Measures of Effectiveness
• Measures of Performance
• Work Breakdown Structure
• Integrated Master Plan
• Technical Performance Management
Plan
• Initial Integrated Master Schedule
• Time Phased Staffing Plan
• Schedule Reserve
• Management Reserve
• Risk Register and Mitigation Plans
• Adjusted Integrated Master Schedule
• Risk Management Plan
Inputs Outputs
GA

43. 43
❶ COMPARE PLANNED TO
ACTUAL PERFORMANCE
Compare technical and programmatic performance to the planned
performance. Identify variances needed to make decisions about
corrective actions to Keep the Program Green.
TC

44. 44
Compare Planned to Actual Performance
• Physical Percent Complete
– Planned
– Actual
• Do this at the Work Package level
– Quantifiable Backup Data
– Avoid averaging out the variances at the Control Account
level
Objectively assess accomplishments at the Work Performance Level
EIA-748-C, page 1
TC

45. 45
Wright Brothers’ Technical, Cost and
Schedule Progress
0.80
0.85
0.90
0.95
1.00
1.05
1.10
770
780
790
800
810
820
830
840
850
860
870
15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08
CPI/SPI
Pounds
WBs' 1908 Flyer Integrated Progress
(2/28/1908)
Upper Limit Planned Target Achieved SPI CPI
Compare side-by-side the Technical Performance (Weight) with
the Programmatic performance CPI and SPI
TC

46. 46
❷ COMPARE MR AND SM
BALANCES
Using the performance data and the variances, the computed
BCWP from measures of Physical Percent Complete.
Assess planned Management Reserve and Schedule Margin with
the current MR and SM and inform EAC and ETC from past
performance.
TC

47. 47
Compare BCWP with MR and SM
• With actual BCWP assess impact on Estimate at Completion
and Estimate To Complete.
• Assess planned Management Reserve against actual
Management Reserve
• Assess planned Schedule Margin against actual Schedule
Margin
• Assess planned risk burn down against actual Risk Burn Down
TC

48. 48
Wright Brothers’ MR Planned Use vs Actuals
$0
$500
$1,000
$1,500
$2,000
$2,500
$3,000
$3,500
$4,000
WB 1908 Flyer Planned vs Actual MR Use
(15 May 1908)
MR Planned Balance MR Actual Balance
TC

49. 49
Wright Brothers’ Schedule Margin Burn
Planned Use vs Actuals
In May, when the schedule margin went below plan, immediate
corrective action is needed to get back on Schedule Margin Plan
0
2
4
6
8
10
12
15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08 15-Aug-08 15-Sep-08
Days
WB 1908 Flyer Planned vs Actual Schedule Margin Use
(15 May 1908)
Schedule Margin Plan Balance Schedule Margin Actual Balance
This dip
characterized as
negative slack
RP

50. 50
❸ REVISE IMS AND RISK
REGISTER
With this data, update the Integrated Master Schedule and Risk
levels in the Risk Register for future periods of performance.

51. 51
Revise IMS and Risk Register
• Every reporting period update the Risk Register with actual
assessment of
– Reduced risks
– Actual margin needed for unmitigated and irreducible risks
• Adjust Risk Register contents for current assessments
– New risks
– Retired risks
– Changes in risk attributes
• Update ETC, EAC, and ECD with the statused IMS and Risk
Register
TC

52. 52
❹ RUN AND RERUN THE
SCHEDULE RISK ANALYSIS
Schedule Risk Analysis (per DI-MGMT-81861) at periodic intervals
to confirm the Estimate to Complete and the Estimate At
Completion are still within acceptable values to ensure the
probability of program success.
TC

53. 53
Run and Rerun Schedule Risk Analysis
• With updated Risk Register – rerun the Schedule Risk Analysis
• Reassess EAC, ECD, and ETC
• Determine sensitivity indices (which elements are most likely to
cause the forward plan to go awry)
• If needed
– Replan IMS, possibly with more schedule margin
– Add resources
– Adjust technical solution
TC

54. 54
Summary
• Cost and Schedule performance informed by Technical
Performance
– At IBR TechnicalPerformance plan defines the work activities
in the IMS
• Build a risk adjusted PMB
– Reducible and irreducible risks in the risk register, traceable to
the IMS.
• Budget active mitigation activities
• Protect unmitigated risks with ManagementReserve and
Schedule Margin
• Make sure programmatic data is aligned with Technical
Performance Data
• Perform these activities continually during execution to Keep the
Program Green
TC

55. 55
Questions???
55