Rovinsky final

Federal Aviation
FAA’s Successful Administration

Implementation of
Earned Value
Management

Presented to :NASA PM Challenge
By: Robert Rovinsky
Date: February, 2010
Improving Program Performance at FAA Federal Aviation 1 1
Administration
Used with permission

Contents of Talk
• Why Talk about the FAA at the PM Challenge
• Quick FAA Facts
• History/Drivers for Earned Value Management
• What our Project Managers say about EVM
• What Products and Processes we follow
• Agency Accomplishments and GAO Audits
• Lessons Learned and the Way Forward
• Summary and Q&A

Improving Program Performance at FAA FAA's Successful Implementation of Federal Aviation 2 2
Administration
EVM

Why Talk about the FAA to Present at the PM Challenge?
• “NASA is currently on the GAO High Risk List and FAA just
came off that list – why and how did FAA make this happen?”
• “NASA faces challenges for EVM buy-in both internally (in-
house projects) and externally (contractors) – is it a culture
thing?”
• “There is no Agency mandated EVM system – how did the
FAA create one”
• “How did FAA make the changes and get to its leadership
position – was it an internal thing, was it a cultural thing, was
it a mandate?
• “Why are some folks intimidated or afraid of EVM? What is in
it for them? What benefits did the FAA get? How much did it
cost?”
Administration

FAA – Quick Facts
• 12/17/1903 –1st sustained & controlled flight (Orville &
Wilbur Wright)
• 07/01/1934 –Bureau of Air Commerce formed
• 08/23/1958 –Federal Aviation Act of ’58 signed

• TODAY:
• Safest System ever!
• ~ $16B budget(FY10)
• 45,757 FAA Employees(FY10)
• 19,930 U.S. Airports
• 811,593,000,000 –Passenger Miles Flown(‘08)
• 590,349 Active Pilots

Administration

FAA – Mission/Vision/Values
Our Mission Our Values
To provide the safest, most
SAFETY IS OUR PASSION.
efficient aerospace system in We are the worlds leaders in aerospace safety.
the world.
QUALITY IS OUR TRADEMARK.
We serve our country, our stakeholders, our
customers, and each other.

INTREGITY IS OUR CHARACTER.
Our Vision We do the right thing, even when no one is
We continue to improve the looking.

safety and efficiency of flight. PEOPLE ARE OUR STRENGTH.
We are responsive to our We treat people as we want to be treated.
customers and are
accountable to the taxpayer
and the flying public.

Administration

Background – History and Drivers for EVM
FY 03-04 FAA Business Case Failure and Recovery: The original
driver for the EVM initiative

2005:
• We assessed our major investments (80% of FAA capital
expenditures) against the industry EVM Standard and
produced our “EVM Flag”.
• We committed to OMB and GAO that FAA would implement
full EVM on our major acquisitions by the end of 2007.
• We established FAA’s EVM policy.

Administration

Administration

History and Drivers for EVM
2006:
• We formed a multi-functional EVM Council
• We met with our major prime and support contractors on our plan and
solicited industry best practices
• We conducted a “cost and benefits of EVM” study
• We revised FAAs Acquisition Policy to:
– Establish Standard Program Milestones
– Require Implementation Strategy and Planning (ISP) requirement for
EVM plan
• Result: Significant traction toward FAA compliance targets
• We produced a Lay Guide to EVM
• All FAA Acquisition Executives received EVM Executive Briefing
• We developed standard program milestones for EVM

Administration

FAA’s EVM Council

• The EVM Council, composed of representatives across the agency and all
disciplines, leads the timely development, implementation and operation of FAA
EVMS procedures, policies, training and tools to fully realize efficiencies and
savings arising from the adoption and continuous improvement of a common
EVMS employing best practices.
• It provides support to Agency wide program management with respect to
implementation of EVM on FAA programs as required by Acquisition Policy and
the FAA EVM Guide.
• The council reviews and evaluates draft EVM policy documents, including
pertinent background information and special instructions or requirements to
ensure all internal stakeholders are involved in the development and
implementation of the FAA EVMS.
• The EVM Council oversees the implementation and application of common EVM
methods throughout FAA.

Administration

2007:
• Achieved Green EVM assessments for the 2005 legacy
programs
• Established Portfolio/Program Performance Metrics (P3M)
• Established requirement for product oriented WBS
• Conducted EVM Data and Tools Studies
• Conducted ANSI Accounting Guidelines EVM assessment
• Conducted more EVM industry days
• FAA Decides to try to get off GAO High Risk List; GAO
sees our EVM effort as a step in right direction.

Administration

Administration

2008:
• Completed EVM Guides for Project Managers and
Contracting Officers
– Integrated Baseline Reviews
– Surveillance Guides
– EVM System Acceptance Guide
• Revised EVM assessment criteria – consistent with NDIA
Intent Guide
• Formulated product oriented WBS using standard
program milestones
• EVM progress reported to GAO as part of High Risk
Initiative

Administration

2009:
• FAA Gets off the High Risk List!
• Acquisition Executive Board formed to institutionalize
best practices including EVM
• Developed methodology/criteria for “fee for service”
projects
• FAA one of only two federal agencies to certify
contractors not already DCMA certified
• FAA EVM approach adopted by Dept of
Transportation
Administration

Today’s FAA EVM “Flag”

Today we are: 82% Green, 17% Yellow and 1% Red
Administration

Industry Days have been conducted since 2005
to obtain our suppliers EVM Best Practices
Description / Contractor A B C D E
EVM Surveillance
Joint Contractor / Government EVM surveillance

Contractor Internal EVM Surveillance
Use of EVM surveillance template or checklist

IBRs
Internal IBRs

Joint Contractor / Government Reviews
Perf Meas B/L implemented prior to Definitization

Training
Executive level EVM orientation briefings
EVM Training Program

Other
EVM for Service contracts
Program Start Up process P

EVM User Guide
Use EVM on LOE.

Use EVM on FFP contracts.
Program performance metrics plan
Eliminating cultural organizational EVM barriers P

Executive EVM metrics reporting
= Current Practice
Contracting for EVM “Tips & Traps”

Tailored EVM
P = Planned

Administration

We are integrating EVM with our PM processes

• Many of the Program Management Metrics reported to senior executive
board use EVM data.
• Standard Program Milestones improve completion criteria and map to FAA
standard WBS.
• EVM defined Program Baseline WBS used for Final Investment Decisions
– We are piloting product oriented WBS using standard program
milestones
– Our OMB Exhibit 300 format uses the same Program Baseline WBS
• We use EVM for DOT program performance “Bubble Chart” reports to OMB
• We use Primavera for the Corporate Workplan – it is the enterprise
schedule management solution for EVM
• Our Implementation Strategy & Planning document is used to document the
program EVM plan.

Administration

From our program managers we are
getting affirmation of EVM
• NEXCOM program manager, Dieter Thigpen,
“earned value data clearly identified schedule performance variances and got
management attention quicker as compared to the traditional approach not
using EVM”.

• Kevin Sharrett, ASR-11 Special Projects,
“Implementing EVM provides early visibility into future schedule risks that we
did not have before. As a result of the schedule risk assessment the ASR-11
EVM plan, we accelerated 5 system deliveries from 2007 to 2006 to mitigate
our schedule risk for the last site Operational Readiness Demonstration which
is baselined for 9/30/2009. We also had the schedule analysis to justify
construction and systems acquisition funds in the first quarters of 2007 under
the CR that had become critical for Last ORD if delayed. Implementing EVM
has provided the ASR-11 team members more insight and involvement into the
overall program cost and schedules.”

Administration

From our program managers we are
getting affirmation of EVM

• Dan Watts, the ERAM program manager,
“Earned value data is the only true indicator I have to obtain insight for cost and
schedule performance for a large scale complex program like ERAM”.

• Former ECG program manager, Bill Boyer,
“EVM is one of the more powerful tools in my PM toolbox. EVM allows me to
see the variations in program execution that enable me to take action to
resolve issues that are often beyond the capability of the prime contractor to
handle. With EVM, I am better able to manage planning and execution of
program funds, better able to track program progress against a benchmark,
and better able to work with the prime contractor and the support contractors to
apply resources when needed...and where needed.”

Administration

EVM training is customized
• We make extensive use of FAA standard program milestones
– Program Baseline WBS – Program phases / useful segments
– Schedule completion criteria – Control account definitions
• We include FAA enterprise tools (SPIRE, WorkLenz, and Primavera) in EVM
training material
• EVM training is consistent with FAA Accounting Order 2500.8B
– FAA funding types tightly linked to EVM
– Training includes accounting system (Delphi) examples
• Acquisition program baseline definitions used
– EVM tightly linked to program life cycle
– Knowledge-Based Product Development Phases used
• EVM is consistent with and supports our OMB Exhibit 300 reporting requirements
• FAA risk management process used
• FAA program examples & best practices are used

Administration

FAA EVM Program examples and best practices
are being used for EVM training
Drop 2 ESI Progress
ERAM Successes:
• Able to load all processor types and roles.
ELOM functionality working well

• Can display Target, track, & strobe data
on glass
• Can display an active flight plan in the
Aircraft List. Can amend it and remove
strip.

• SAR data already being used for
debugging defects. Initial set of EOPD
reports working.
• ELOM onboard cache working
• Can load/unload and configure a single
OpSim session

Still Working
• Multiple OpSims & Adaptation variations
• Continue testing Surveillance & Wx
threads
• Release Management Threads
• Continue with EOPD report testing
• Continue Dynamic Library testing

EVM reporting integrated with monthly PMRs
Administration

FAA EVM Program examples and best practices
are being used for EVM training
Overall RoadRoad ECG A GA Site
Overall ECG to to Site Last
Government Acceptance
100.0

90.0

80.0

70.0
Per Cent Complete Plan
60.0
% Complete

50.0
GA outlook
40.0 5/31/05 vs
baseline
30.0 I&I 07/06/05
com plete
Fitup and 4/26/05 vs.
20.0 SAT
Install baseline
05/23/05 outlook
10.0 Com plete
5/23/05 vs
04/05/05
baseline
0.0 06/28/05
2/4/2005

2/11/2005

2/18/2005

2/25/2005

3/4/2005

3/11/2005

3/18/2005

3/25/2005

4/1/2005

4/8/2005

4/15/2005

4/22/2005

4/29/2005

5/6/2005

5/13/2005

5/20/2005

5/27/2005

6/3/2005

6/10/2005

6/17/2005

6/24/2005

7/1/2005

7/8/2005

7/15/2005

7/22/2005

7/29/2005

8/5/2005
= Plan
= Actual

FFP EVM reporting using performance based payments

Administration

FAA P3M is included in EVM training
Program Summary Metrics
Cost Schedule Performance Resources External Program
Interest
G G Y Y Y G

Supporting Metrics
Portfolio Metric Cost Schedule Portfolio Metric Performance Portfolio Metric Resources External
Definition Measure of Measure of Measure of Interest
cost schedule Definition Measure of Definition
performance performance technical metrics current Assessment
of work of work and milestones of funding and of External
performed performed work performed staffing Reviews
EVM Metrics Level 1 AMS Milestones Resource Metrics
- Cost Performance Index G - Prime Contractor Y
(CPI) - Negative Deviations G
- Program Cost Reserve G Technical Metrics - Support Contractors G
- FAA G
- To-Complete Performance G - Requirements Stability G
Index (TCPI) - Funding G
- Cost Variance- G
- System Defects Y
At-completion (CVAC) % - Obligations G
- Test Results n/a
- Schedule Performance G Flight Plan Acquisition
Index (SPI) - Deployment G
Program Goals
- Program Schedule G - Value of remaining High G**
Reserve - Cost G
Risks
- Schedule Variance-At- G External Reviews
Completion (SVAC) % - Technical Variance At- G
Completion (TVAC) - IG Y
Level 1 AMS Milestones
- Negative Deviations G
-GAO G
Flight Plan Acquisition - OMB Y
Program Goals
- Schedule G

Administration

FAA Programs that implemented higher quality EVM
have lower PM costs as a percent of total program cost
Cost of PM Using EVM (FY06)
FAA Cost of Program Management Using EVM
34.5%

27.5%
PM % Total Cost
PM % of of Total Cost

▪
22.2% Avg

20.5%

▪
16.6% Avg

▪
13.0% Avg
13.5%

6.5%
0.7 RED 1.2 1.7 YELLOW 2.2 2.7 GREEN 3.2
Quality of EVM Implementation
Quality of EVM Implementation ▪ = Wg’t Avg PM %

- Quality of EVM implementation based on EVM assessments (FAA EVM Flag)
- PM% of total cost based on FY06 Resource Planning Document (RPD)

Administration

FAA Program Performance Reporting
(DOT Monthly – JRC / OMB Quarterly)
Over Spent Under Spent
Ahead of Schedule Ahead of Schedule
10%

5%
ASR-9/1B
Schedule Variance %

ERAM ATOP
ITWS
STARS
ASR-11
SASO
ASDE-X
ASWON
SWIM
ATCBI-6 WAAS
IFPA ASR-11 TFM-M
TAMR

SBS
-5% ASR-11NEXCOM

-10%

Program Name
Color Code -
>-15% EVMS Quality:

Green: Meets
Yellow: Partially
Meets
Over Spent Red: Does Not Meet
Under Spent
Behind Schedule Black: Not Assessed
Behind Schedule
Not updated

-10% -5% 5% 10% >15%
Cost Variance %

Administration

FAA Program Performance Reporting
(JRC Quarterly)

April 2008 EVM Data Analysis (Trends) -

• ASR-11 Program: 88% Complete
ASR-11 Cost and Schedule Variance Percentages PPPM
Financial
2.00% Y
Schedule
1.00%
Y
0.00% Technical
CV%/SV%

-1.00% Y
CV%
Resources
-2.00% SV%
G
-3.00% External
Interest
-4.00% Y

-5.00% Program
Manager
7 7 7 7 7 7 07 v- 07 7 8 8 8 8
-0 -0 -0 l- 0 -0 -0 ct- -0 -0 -0 -0 -0
A pr ay J un Ju ug S ep O N o ec J an F eb Mar A pr Y
M A D

Administration

GAO EVM Review – July 2008 Report
“FAA has taken important steps to oversee program compliance with EVM
policies, but its oversight process lacks sufficient rigor.”
Seven Key Components of an Effective EVM Policy

Policy component Assessment of
FAA policy
Establish clear criteria for which programs are to use EVM Fully met

Require programs to comply with national standards Fully met

Require programs to use a standard structure for defining the work products that Partially met
enables managers to track cost and schedule by defined deliverables (e.g., hardware
or software component)
Require programs to conduct detailed reviews of expected costs, schedules, and Fully met
deliverables (called an integrated baseline review)
Require and enforce EVM training Partially met

Define when programs may revise cost and schedule baselines (called rebaselining) Partially met
Require system surveillance—routine validation checks to ensure that major Fully met
acquisitions continue to comply with agency policies and standards

Administration

GAO EVM Review – July 2008 Report
Recommendations for Executive Action
GAO Recommendation FAA Response
Modify acquisition policies governing EVM to:
- require the use of a product-oriented standard - FAA milestone based WBS implementation 8/08
work breakdown structure - EVM Council approval 9/08
- Acquisition Executive Board approval 10/08
- Update current acquisition management system 10/09

- enforce existing EVM training requirements and - Develop EVM training course for understanding FAA
expand these requirements to include senior EVM reports – complete by 9/08
executives responsible for investment oversight - Conduct executive EVM training 8/09
and program staff responsible for program
oversight, and
- define acceptable reasons for rebaselining and - Develop program rebaseline procedure:
require programs seeking to rebaseline to (1) - Rebaseline criteria 6/09
perform a root cause analysis to determine why - Root cause analysis methodology by 10/08
significant cost and schedule variances occurred - Lessons learned and mitigation process and Acquisition
and (2) establish mitigation plans to address the Executive Board approval by 2/09
root cause.

Administration

EVM challenges
• Programs EVM remaining “Green”
– Some programs are slipping
– Acceptance across agency continues to be “lumpy”
– July 2008 GAO finding highlighted lack of program performance
surveillance and data analysis - ”FAA assessments are not thorough
enough to identify anomalies in contractor data”
• Efficient EVM implementation
– Need more Standards
• 1) Updating our program baseline management
• 2) Need clear responsibility assignment matrix
• 3) Need program control log procedures
– Need to Optimize and standardize on toolkit
– Continue to integrate processes, procedures, systems, and reporting

Administration

EVM challenges

• Maintain “High Risk Initiative” commitments to GAO
– Program best practices
– Apply to smaller “IT” programs
– Program baseline management
– Original baseline performance reporting

• Fee for Service EVM
– Limited FAA/industry experience
– Industry has some service EVM experience
– Innovative solutions will be required

Administration

EVM Transformation
Concrete Accomplishments
• All major programs assessed in FY05 still in development have
implemented full EVM or are close to implementing full EVM.
• Financial systems have been assessed so all 32 ANSI elements
have been assessed for our major programs.
• Tools and data studies have been completed and we are getting
ready to procure some standard tools.
• All new major programs entering development have been
assessed against the ANSI standard and make progress towards
green
• The EVM Council has created and will measure and enforce
compliance with our EVM policies and standards via an internal
surveillance process

Administration

EVM Transformation
Concrete Accomplishments
• More than half of our program managers are using EVM as
an integral part of the way they manage their programs
• FAA Executives use EVM to support their critical investment
decisions including rebaselining, Air Traffic quarterly
reviews, Integrated Baseline and Service Level reviews, etc.
• OMB, other stakeholders continue to recognize FAAs use of
EVM and increase their confidence in our Program
management.
• FAA is starting to baseline and measure its progress
towards its new objective.
• We are making the required continuous process
improvement.

Administration

EVM Accomplishments - Implied
• EVM was part of moving the FAA ATC modernization
program off the GAO High Risk List – only program
removed during 2009-11 cycle
• OMB Budget Passback and President’s Budget have
supported FAA programs since 2006
• Program Managers using EVM have faced fewer audits
and none have been “painful”
• FAA’s leadership position has helped attract positive
attention and good people have come our way.
• Industry sees us as their partner and shares best
practices with us more and more.

Administration

Keys to FAA EVM Success
• We obtained and have kept Executive Management Support
• EVM protects FAA’s Capital Investment funding
• EVM helped remove FAA from the GAO high risk list
• Our EVM Leadership position helps convince our vendors, aviation
partners and stakeholders that FAA is a good steward of the taxpayers
and airline user fees money
• We obtained and have Project Management Support
• We tailored EVM implementation based on type of contract and scope
and avoided legacy EVM Problem areas
• We provided clear, and concise ANSI/EIA 748 criteria
• We adapted the NDIA EVM Intent Guide using FAA terms, processes,
and documents
• We reached out to our major PMs and our prime contractors
Administration

Keys to FAA EVM Success
We kept our stakeholders and the industry fully engaged.
• Contractors and Program Managers see this as a win-win.
• We did constant socialization with GAO, OMB, the Department of
Transportation and other agencies.
• We provided our materials – policies, guidelines, best practices,
training materials, guidebooks and analyses – to everyone.
We focused on improving project performance rather than EVM. It
was the transformation caused by EVM, not EVM itself.
We focused on fixing the problems with our projects, not fixing the
blame, and on continuous improvement.
We involved everyone, both inside and outside the agency.
We never declared victory, but we always demonstrated concrete
progress.
We were not afraid to show our costs and to be transparent about
what our problems and challenges were and are.

Administration

EVM Transformation – Lessons
Learned and the Way Forward
• The proper balance between oversight and engagement is critical
• The involvement of staff and contractors with program management
experience and empathy is key
• Constant pressure is needed to make change.
• EVM must evolve to broader program management excellence
• Constant education for and use by senior managers is required
• Program teams must be encouraged or forced to employ the right folks
• Standard tools/data/methods are critical and usually more acceptable to
program teams.
• Deciding when to do it for them vs teaching them to do it for themselves is an
art form.
• Metrics and measurement and transparency is critical
• The external reviewers can be your best friends
• You are never done and the “elephant walk” may always return.

We will share any and all of our documents and lessons learned.

Administration

Thank You for Inviting me!

Are there Any Questions?

Administration

Rovinsky final

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Rovinsky final

Similaire à Rovinsky final (20)

Plus de NASAPMC

Plus de NASAPMC (20)

Dernier

Dernier (20)

Rovinsky final