2. 2
Today’s Briefing
• How can Agile Software Development methods increase the
Probability of Program Success (PoPS) on Earned Value
programs?
• How can Agile development integrate with the FAR / DFAR and
OMB mandates for program performance measures using
Earned Value?
• What are the “touch” points (or possible collision points)
between Agile and EIA-748-C?
• What are the measures of success for Agile methods in the
context of EIA-748-C?
3. 3
3
Let’s Start With A Critical
Assumption:
Project Controls are needed
for Project Success
4. 4
In case we’ve forgotten what EIA-748-C says
• EIA-748-C, page 1, defines the top level activities for
a successful EV based project.
• We need to “connect the dots” between these and
agile development.
§ Plan all work scope for the program from inception to completion.
§ Break down the program work into finite pieces that can be assigned to a
responsible person or organization for control of technical, schedule, and cost
objectives.
§ Integrate program work scope, schedule, cost objectives into a performance
measurement baseline plan against with accomplishments may be measured.
Control changes to the baseline.
§ Use actual costs incurred and recorded in accomplishing the work performed.
§ Objectively assess accomplishments at the work performance level.
§ Analyze significant variances from the plan, forecast impacts, and prepare and
estimate at completion based on performance to date and work to be performed.
§ Use this information in the organization's management processes
5. 5
When we Hear Agile, What Does That Mean?
† Dr. Ashton Carter, Under Secretary of Defense for Acquisition, Technology and Logistics,
Sep/Oct, 2010 Defense AT&L
“Being able to turn inside the loop of unfolding events.”†
7. 7
IMP Captures End User Requirements in
Terms MOEs, MOPs, KPPs, and TPMs
SOW
ConOps
WBS
CWBS
Technical Performance
Measures
(TPM)
Messages of Performance
(MOP)
Measures of Effectiveness
(MOE)
Key Performance
Parameters
(KPP)
Integrated Master Plan
(IMP)
Integrated Master
Schedule
(IMS)
Performance
Measurement Baseline
(PMB)
Technical Requirements
8. 8
Some Critical Concepts in Agile
Product Time
Capabilities
A high level system
function completed
across multiple
releases
Features
Stories
A well define system
function to be
completed within a
release
A small well defined
system function that
can be developed
within one sprint
Release
Sprint
† Earned Value Managementfor Scrum Programs:Industry Best Practices, National Defense Industry Association
Integrated Program ManagementDivision (NDIAIPMD) Scrum EVM Working Group
9. 9
Putting Earned Value Management and Agile
Software Development Together
Feature 1,2,3
Feature 4,5,6
Feature 7,8,9
Release 1 Release 2
Release 2 PP’s
WP
PP
SLPP
in IMS
CA
EVM Reporting from
Work Package
Performance
EVM Performance
• BCWS – fromWBS
Basis of Estimate
• ACWP – fromtime
cards
• BCWP – from
completed Story
Points
Sprints
Time Now
n Starting with the IMP,defined Capabilitiesare flowed to the PMB
n Capabilitiesproduce the value fromeach Release
n Control Accounts and Work Packages are on baseline in the PMB
n Work Packages contain Features produced in each Release by Sprints
n Release Planning baseline for period of performance and BCWS
Vertical and horizontal integration
of all work planned fromthe
IMP/IMSdown to Releases, Sprints,
and Tasks in the Scrum
Development Management system.
Performance Measurement Baseline
Scrum Software Development Lifecycle
Feature n
The Bright Line
Milestones
Data Items
IMP ProgramEvents
IMP Accomplishments
and Criteria
Scrum Development Control Account
Task
Task
Task
Task
Task
Task
Task
Task
Task
…
10. 10
Ordinal Story Point cannot be basis of
BCWS higher than a Feature
10
Feature 1
Story
Story
Story
Story
Story
Story
Team 1’s
Uncalibrated
Ordinal SP
estimates
Feature n
Story
Story
Story
Story
Story
Story
Team 2’s
Uncalibrated
Ordinal SP
estimates
∑ F1(SP) ∑ Fn(SP)
Release 1 ∑ of SP’s
• • •
§ At the Story level,
relative effort defines
individual estimates.
§ At the Feature level,
lower level SP’s don’t
have the same unit of
measure in the way
Dollars do.
§ When Features
summed to the
Release, relative
measures do not
provide basis of
Physical Percent
Complete.
Not same units of measure between
Features – Uncalibrated SP’s
11. 11
Building the PMB in a Agile Paradigm
WBS
Iteration 1 Iteration 2 Iteration 3 … Iteration n Close Out
§ Deliverables
§ Tasks
§ Tasks
§ Deliverables
§ Deliverables
§ Deliverables
§ Tasks
§ Tasks
CA CoDR
…
PDR
WBS basis of deliverables Backlog, per MIL-STD-881C,
decomposed into Release Backlog, then into Iteration
Backlog for delivery by Stories and Tasks.
13. 13
CONNECTING THE MOVING PARTS
INTO AN INTEGRATED WHOLE
With both Earned Value and Agile parts, let’s connect them into a
Performance Measurement Baseline ready for execution and
reporting in the IPMR
14. 14
Three Systems are Needed for an Integrated
Solution
• Shared data between
each system with a
single system of
record.
• Planning flowed from
IMS to Agile
development system
• Physical Percent
Complete flowed
back to IMS then to
EVMS
17. 17
STEPS TO SETTING UP THE
PMB FOR AGILE + EVM
Here’s the step-by-step activities to define the Performance
Measurement Baseline (PMB) and get it on Baseline in Team
Foundation Server, the IMS, and COBRA.
This includes Baseline Change Requests and statusing of the
baseline to report EV in the IPMR.
This is the secret sauce of the principles and theory
18. 18
Simple Rule for Earning Value in Agile
Each Iteration of each Release is a
“value earning” opportunity
The next step is to connect Agile’s
definition of Value with Earned Value’s
definition of Value
Business Value ≠ BCWP
19. 19
Starting to “Connect the Dots” † ‡
Agile Point of View DoD Program Point of View
Requirements evolve Scope agreed to and maintained
Simple designs are best Architecture thought out and maintained
Teams are self organizing
Organizational structure establishes
boundaries
Delivery teams establish
best prescriptive processes
High level guidance organizes work
Development teams know
what to do
Process professionals define the
boundaries
Agile teams work in an
iterative manner
Product Development Lifecycle is serial
over broader periods of time
† Abstracted from “Reality over Rhetoric,” Scott Ambler IBM Developer Works
‡ John Goodpastuer, Project Management the Agile Way
20. 20
Let’s Start With 3 Simple Connections
Earned Value Management Agile
Both EV and Agile Development Measure Progress as
Physical Percent Complete
+
1
Measures of progress in units of
“physical percent complete.”
Each sprint produces 100%
working product.
2
Forecast of future performance
provided by past performance.
Forecast performance in units of
product(s) produced.
3
A systems approach to the
development of products and
connecting Cost, Schedule, and
Technical Performance.
Increasing fidelity of product and
problem understanding takes
place after each sprint and
release.
21. 21
EVM + Agile Process Flow
WBS
Story
Feature
Place Work
on Backlog
Estimate
Story Points
Work Package
BCWS
Execute
Sprint
Record SPs
Complete
Calculate P%C
with SPs
Load P%C for
WP in IMS
Send P%C to
Cobra
Calculate
BCWP
1. With the WBS decomposed to
Features, Stories, and Story
Point estimates
2. Place Features and Stories on
Scrum backlog.
3. Perform work in Sprints.
4. Record Stories earned and
calculate Physical Percent
Complete (P%C)
5. Send to the IMS and to Cobra
6. Calculate BCWP = BCWS ×
P%C in Cobra
The Bright Line
SOW
ConOps
22. 22
Assess Performance On A Weekly Basis
Feature
Story
Story
Story
Story
Planned
24 SP
% Complete
100%
100%
0%
0%
Remaining
4 SP
Earned
20 SP
Week 1 Week 2 Week 3 Week 4
20 Day Sprints
Every Thursday Status
§ Physical Percent Complete
§ SP’s remaining to 100%
8 SP’s
8 SP’s
4 SP’s
4 SP’s
23. 23
Here’s Those Connections
GL EVM Criteria Agile Approach
1 Define WBS Features and Stories define tasks
2 Identify Organization Self organizing teams
5 Integrate WBS and OBS Self organized teams with a customer
6 Schedule Work Iterations and Releases
7 Identify Products & Milestones Working software at the end of iterations
8 Set time phased budget Fixed length iterations and releases
16 Record direct costs Fixed staff = Level of Effort
23 Determine variances EV + Velocity measures missed features
25 Sum data and variance Missed features moved to next iteration
26 Manage action plans Replan missed features, adjust velocity
28 Incorporate changes Replan missed features, adjust velocity
24. 24
Provide managers
with information at a
practical level of
summarization
Relate time
phased budgets to
specific contract
tasks
Enable statistical
estimation of
completion costs
Track and
monitor
discrete project
metrics
Communicate
project status
Provide
quantitative data
for decision
making
Provide a
documented
project
performance trail
Alert project
managers to
potential schedule
and cost risk
Program
Controls
Practice
But First, Let’s Not Forget Business Management Practices …
…That Must Be Recognized Before Connecting Agile and EVM
25. 25
11 EVM GL’s and 12 Agile Principles
Connecting Agile with Earned Value
Management is actually obvious
once we get over the social aspects
and focus on the Program Planning
and Controls aspects.
WBS
OBS
WBS+OBS
IMS
Productsand
Milestones
PMB
ACWP
Variances
Sumof
Variances
Corrective
Action
Record
Changes
Early and Continuous Delivery ✔ ✔ ✔
Welcome Change ✔ ✔ ✔ ✔
Deliver working Software ✔ ✔ ✔
Business and DEV work together ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Motivated individuals ✔
Sustainable development ✔ ✔ ✔ ✔ ✔
Working SW measure of progress ✔ ✔
Face-to-Face communications ✔ ✔ ✔ ✔
Continuous technical excellence ✔ ✔
Maximize work not done ✔
Self-organizing teams ✔ ✔
Regular reflections ✔ ✔ ✔ ✔ ✔
26. 26
Putting Earned Value Management and Agile
Software Development Together one more time
Feature 1,2,3
Feature 4,5,6
Feature 7,8,9
Release 1 Release 2
Release 2 PP’s
WP
PP
SLPP
in IMS
CA
EVM Reporting from
Work Package
Performance
EVM Performance
• BCWS – fromWBS
Basis of Estimate
• ACWP – fromtime
cards
• BCWP – from
completed Story
Points
Sprints
Time Now
n Starting with the IMP,defined Capabilitiesare flowed to the PMB
n Capabilitiesproduce the value fromeach Release
n Control Accounts and Work Packages are on baseline in the PMB
n Work Packages contain Features produced in each Release by Sprints
n Release Planning baseline for period of performance and BCWS
Vertical and horizontal integration
of all work planned fromthe
IMP/IMSdown to Releases, Sprints,
and Tasks in the Scrum
Development Management system.
Performance Measurement Baseline
Scrum Software Development Lifecycle
Feature n
The Bright Line
Milestones
Data Items
IMP ProgramEvents
IMP Accomplishments
and Criteria
Scrum Development Control Account
Task
Task
Task
Task
Task
Task
Task
Task
Task
…
27. 27
The Reason EV + Agile is a
Match Made in Heaven
The faster you obtain your intelligence
The more intelligent your decisions
From an Aviation Week & Space Technology Ad†
Agile forces the measures of physical percent complete
at the end of every Sprint.
By adopting this principle alone, EVM will have
measures of Physical Percent Complete – Working
Software – assessment every two weeks
Otherwise you’re not doing Agile
† L3 ad for UAV, AW&ST, September 14-27, 2014, pg. 37
29. 29
PUTTING THESE IDEAS TO
WORK
Using the Earned Value Management Intent Guide (EVMIG), here’s
how to connect the dots at the next level down.
The 11 criteria of Earned Value connected with the 12 principles of
Agile.
30. 30
Assemble a credible WBS and Integrated Master Plan / Integrated
Master Schedule (IMP/IMS)
–WBS Dictionary says what will be built
–IMP/IMS says how, where, when, and what will be built and the
processes to build it.
Identify Reducible – Event Based – uncertainties and the
resulting risks from WBS Dictionary and IMP Narratives in the
Risk Register
Put these these risks in the Risk Register, with probability and
impacts
Develop risk retirement plans and place them in the IMS using
CBB funding for risk buy down.
30
Building a Risk Adjusted PMB in Steps
0
1
2
3
8
31. 31
Assess the Irreducible – naturally occurring –
uncertainties and the resulting schedule and cost
risks in the WBS and IMS.
Use Monte Carlo Simulation to determine schedule
margin and budget impacts from work activity
durations to handle these irreducible uncertainties.
Assign schedule and cost margin to protect key
item deliverables in the IMS.
31
Building a Risk Adjusted PMB in Steps
4
5
6
8
32. 32
Determine cost and schedule impacts of unmitigated
risks in the risk register and place them in
Management Reserve, with their exposure and impact
Assemble mitigated Irreducible (Aleatory) and
Reducible (Epistemic) risks with the unmitigated
event–based risks into the Total Allocated Budget
– Risk handling an baseline for reducible and
irreducible in the PMB.
– Risk Handling in Management Reserve.
32
Building a Risk Adjusted PMB in Steps
(Concluded)
8
7
8
33. 33
Steps to connecting the Agile Release Cycle
with the Performance Measurement Baseline
1. Establish Releases
2. Establish Iterations within each release
3. Establish Stories from the WBS and allocate them to each
release
4. Assign resources to each Story
5. Estimate work effort – in hours – to each story
6. Assess if capacity for work ≥ demand for work
7. Repeat Steps 4, 5, and 6 until demand equals capacity
34. 34
GL 1: Define Authorized Work Elements
Define the authorized work elements for the program. A work breakdown structure
(WBS), tailored for effective internal management control, is commonly used in this
process.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Work Breakdown Structure (WBS).
§ Road Map & Release Plan consisting of
Capabilities, Product Backlog & Iteration
Backlog.
§ WBS dictionary (may or may not be used,
but a method to reconcile the statement
of work to the WBS structure must be
demonstrated).
§ WBS dictionary: agile user stories are
deliverables that you can measure “done”
for, therefore user stories satisfy wbs
dictionary.
35. 35
GL 2: Identify Program Organizational Structure
Identify the program organizational structure, including the major subcontractors
responsible for accomplishing the authorized work, and define the organizational
elements in which work will be planned and controlled.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Organization Breakdown Structure
(OBS).
§ CAM just builds a team as usual, but the
team needs to be persistent, and not
interchangeable parts.
§ OBS intersections with the WBS.
§ Team hierarchy definition with resources
associated with their sub–teams.
§ Done at the level of granularity to
support the basis of estimate (BOE).
§ Persistent teams are needed to apply
throughput benchmarks to product
backlogs to validate plans.
36. 36
GL 5: Integrate WBS and OBS
Provide for integration of the program work breakdown structure and the program
organizational structure in a manner that permits cost and schedule performance
measurement by elements of either or both structures as needed.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Control accounts.
§ Evidence that the CA meets the 90%
discrete work rule.
§ Defend schedule & cost performance at
the CA level?
§ Agile CA = one release.
§ Actuals captured at the story level.
§ Responsibility Assignment Matrix (RAM).
§ Done at too high a level for the SW
development approach to make a
difference.
§ Contract Performance Reports (CPRs), if
applicable.
§ Given an objective of X stories in
iteration Y, completed stories are earned;
all unearned return to backlog and a new
ETC is developed from the benchmarks
& backlog.
37. 37
GL 6: Schedule the Work
Schedule the authorized work in a manner which describes the sequence of work and
identifies significant task interdependencies required to meet the requirements of the
program.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Integrated network schedules including
master, intermediate (if any), and detailed
schedules.
§ CAM’s agile roadmap becomes the
auditable intermediate schedule
demonstrating significant
accomplishments (SA).
§ MRP or ERP schedules, or planned order
reports.
§ Each task in IMS has associated
resources.
§ Control account plans (may be separate
plans or detail schedules).
§ CAM creates schedules compliant to
DCMA 14 point assessment.
§ Work authorization documents. § Nothing different.
38. 38
GL 7: Identify Products and Milestones
Identify physical products, milestones, technical performance goals, or other indicators
that will be used to measure progress.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Integrated schedules including master,
intermediate (if any), and detailed
schedules that identify contract
milestones and key events.
§ Agile dev performance reporting follows
the approved program system description
§ Apportioned technical performance
milestones to reduce risk & roll up
intermediate technical performance.
§ MRP or ERP production planned order
reports.
§ Not relevant to sw development.
§ Control account plans (may be separate
plans or detail schedules)
§ Not relevant to Software Development
because we are reporting tasks as
physical % complete, which will
automatically roll up.
39. 39
GL 8: Set Time Phased Budget
Establish and maintain a time–phased budget baseline, at the control account level,
against which program performance can be measured. Initial budgets established for
performance measurement will be based on either internal management goals or the
external customer negotiated target cost including estimates for authorized but
undefinitized work.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Control account plans.
§ Time phased budget created for the current iteration(s)
and future work.
§ Summary level planning
packages.
§ Agile summary level planning documented in road
map. Comprises capabilities, features and stories
§ Agile planning packages driven by persistent teams
with proven benchmarks.
§ Performance Measurement
baseline.
§ Is there a target threshold for future work as described
in a PMB? Within 10% OTB?
§ Undistributed budget logs. § Does this have anything to do with SW dev approach?
§ Notification to the customer
of an over–target baseline.
§ Does this have anything to do with SW dev approach?
§ Work authorization
document.
§ Does this have anything to do with sw dev approach?
40. 40
GL 16: Record Direct Costs
Record direct costs in a manner consistent with the budgets in a formal system
controlled by the general books of account.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Reconciliation of project costs with the
accounting system.
§ CAM would follow program direction on
these.
§ These are not impacted by sw dev
approach
§ Actual costs are reported at the control
account level at a minimum.
§ Not impacted by SW development
approach.
§ Reconciliation of subcontract reported
actual costs to subcontract payments.
§ Not impacted by SW development
approach.
§ Internal and external performance reports
for subcontractors.
§ Not impacted by SW development
approach.
§ Subcontractor control account plans,
when utilized.
§ Not impacted by SW development
approach.
41. 41
GL 23: Determine Variances
Identify, at least monthly, the significant differences between both planned and actual
schedule performance and planned and actual cost performance, and provide the
reasons for the variances in the detail needed by program management.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Variance analyses (budget based
schedule variances and cost variances).
§ Can track & report variances per the
approved program system description
§ Management action plans. § Actionable recovery plans per issue.
§ Updated schedule task completion and
cost–at–completion forecasts.
§ Scrum Agile has a POD and Plan for
Iteration.
§ CAM’s monthly EAC reporting follows
the approved program system
description
§ Project schedules and schedule analysis
outputs.
§ PM tracks the dynamic backlog, which
will go up and down based on sponsor
feedback
42. 42
GL 25: Summarize Variances
Summarize the data elements and associated variances through the program
organization and/or work breakdown structure to support management needs and any
customer reporting specified in the project.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Variance analyses. § There is nothing in Agile’s approach to SW
development that precludes reporting variances at
the WP level.
§ Agile is more dynamic than EVM so variances are
less the issue than the evolving baseline, as
approved in governance. The sponsor will want to
track accumulating business value and variances
to total product needs.
§ Schedule and cost performance
reports.
§ Similar – but measures of performance not
usually in dollars
§ Management action plans. § Similar – but less formal. Collaborative discussion
of what actions to take include the customer.
§ Updated schedule and cost
forecasts.
§ Similar – but less formal. Planning processes
include the customer.
43. 43
GL 26: Implement Management Plan
Implement managerial action taken as the result of earned value information.
EVMIG Objective
Evidence
Agile Objective Evidence for EV
§ To–Complete
Performance Index
(TCPI).
§ TCPI = Work Remaining / Cost Remaining ((BAC –
BCWPcum) / (EAC – ACWPcum)). In Agile, work remaining
is the product backlog. Backlog is BAC – BCWP.
§ Independent
completion estimates.
§ No longer used in 2010
§ Risk management data
and similar metrics.
§ Qualitative Risk Burn–down Chart (risk rating)
§ Management action
plans and review
briefings.
§ Agile approach called Commitment Based Planning –
where the SCRUM team makes and meets its time phase
BCWS commitments.
§ Any team, when behind, gives voice to the customer when
evaluating/reweighting the triple constraint.
§ Variance analyses.
§ This is an issue of cost mgmt and system description
would define when and where team members would bill
44. 44
GL 28: Incorporate Changes (1)
Incorporate authorized changes in a timely manner, recording the effects of such
changes in the budgets and schedules.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Contractual change documents.
§ Bug reports, new user stories, but not
necessarily cost sized.
§ User stories above baseline are tracked as
new scope (with a valid BOE) and require
BCWS
§ Change control logs (management
reserve, undistributed budget,
performance measurement baseline,
and contract budget base).
§ New or materially altered features or stories
are changes.
§ Control account/work
package/planning package plans.
§ Product and iteration backlogs are frozen
during the development period
45. 45
GL 28: Incorporate Changes (2)
Incorporate authorized changes in a timely manner, recording the effects of such
changes in the budgets and schedules.
EVMIG Objective Evidence Agile Objective Evidence for EV
§ Master schedules, intermediate
schedules (if any), and detailed
schedules.
§ Iterations and evolutionary planning at the
detailed levels merges with the end to end
planning for agile.
§ Statement of work, WBS, and WBS
dictionary.
§ Customer owner and Planning processes
identify requires work and its description.
§ Work authorization documents.
§ Planning sessions, authorize a set of
Stories to be developed during the iteration.
§ Management reports (contract
performance reports or other
applicable management reports).
§ Big Visible Charts, “sticky notes” display
progress to plan for the agile team.