SlideShare a Scribd company logo
1 of 56
Draft - January 23 2012




Program Cancellation Failure
Modes & Lessons Learned
               J. Steven Newman D.Sc.
        NASA Human Exploration & Operations
     Risk & Knowledge Management Support Team
                   ARES Corporation

                    David Lengyel
   Manager – NASA Human Exploration & Operations
           Risk & Knowledge Management

  PM Challenge 2012 – Orlando Fl. – February 22-23, 2012
                                                                            1
Outline
• Part I: Introduction – The Need for Program
  Cancellation Reviews (PCRs)
• Part II: Program Cancellation Overview
• Part III: Program Cancellation Factors/Model
• Part IV: Program Cancellation Scenarios &
  Examples
• Part V: Proposed PCR Process


                                                 2
Part I – Background / Call for Action
• Catastrophic operational failures are documented and analyzed in-detail
  identifying their proximate and contributing or underlying causes
    • $2 Million trigger for Class A mishap formal reviews
    • Formal mishap review boards established in accordance with NASA Procedural
      Requirements document NPR 8621, Mishap and Close Call Reporting,
      Investigating, and Recordkeeping

• No comparable process for program cancellation
    • Program termination announcements invariably cite cost overruns and schedule
      slippage as causal factors with little investigation / documentation of root causes
    • No formal process parallel to NPR 8621 to address program/project cancellations.
            Note: the term “program” is used throughout this presentation to discuss any
            cancelled activity – program, project, technology demonstrator
    • Program cancellations far exceed a mere $2 million “trigger value” by one ($20
      million), two ($200 million), or even three ($2 billion) orders of magnitude

• Action: Establish a formal Program Cancellation Review (PCR) process



                                                                                        3
Part I - The Need & Benefits of Program
       Cancellation Reviews (PCRs)
 •   Preparation for External Scrutiny / Reviews
      • Government Accountability Office (GAO), Office of Management & Budget
          (OMB), Inspector General (IG), Aerospace Safety Review Panel (ASAP)
 •   Avoid Repeating Past Mistakes
 •   Knowledge Capture and Transfer
      • Documenting and transferring Lessons Learned
 •   Promote Increased Risk / Vulnerability Awareness within NASA PM community
 •   Promote Greater Cultural Willingness to Look in the Mirror
      • Get beyond embedded NASA “Stopping Rules”
 •   More Effective PM Training
      • Incorporated within APPEL curriculum
 •   See Future Cancellations Coming In Time to Remedy
 •   More Effective Reviews and Internal Controls
      • Program Management Council (PMC), Directorate Program Management
          Council (DPMC), Baseline Performance Review (BPR), Standing Review
          Board (SRB)                                                          4
Part I: Goals
• Understand the roles of Agency
  leadership, Congress, and the White House
  in the cancellation process
• Identify key factors in cancellation decisions
  – Analyze selected NASA and DoD Program
    Cancellations
• Develop a model to assess cancellation
  vulnerability
• Propose a cancellation review process

                                                   5
Part I: Program Analysis Data-Set
NASA                              DoD
  • X-33                             •   F-35
  • X-34                             •   A-12
  • X-37                             •   Crusader
  • X-38 Crew Rescue Vehicle         •   Comanche
    (CRV)                            •   FCS
  • James Webb Space Telescope       •   F-22
    (JWST)                           •   Sea Wolf
  • Constellation (CxP)
                                     This data-set is considered sufficient to
  • Space Station Freedom (SSF)      begin the process of identifying trends
                                     related to program cancellation. It is
  • International Space Station      anticipated that future efforts will
    (ISS)                            expand the data set and analysis.


                                                                             6
PART II
Cancellation
 Overview

               7
Program Cancellation: Overview
         Cancellation or Not – They Decide
US House and Senate           White House / OSTP / Agency
                                      Leadership




 Program Cancellation Vulnerability Domains




                                                            8
Congressional Leadership
           Considerations
•   Budget / budget deficit / tradeoffs
•   Alignment with space policy objectives
•   National importance or need
•   Affordability
•   Jobs in home district or home Center
•   Industry views

                                             9
Agency Leadership Considerations
•   Alignment with White House Space Policy Agenda
•   Alignment with Agency Strategic Plan
•   Does program addresses primary agency objectives?
•   Is program/project on critical path to other programs?
•   Does the program fulfill unique functional
    requirements?
•   Is the mission operational scenario consistent with
    Agency risk posture?
•   Are there overriding Agency-level or Program-level
    budget constraints?
•   Are there external events & constraints to consider?

                                                         10
Program In Trouble
         LTA Req
       Management         LTA Cost
                         Estimation
                                         Cost Overrun

LTA Pgm       Schedule
Planning



                         LTA Funding /   Cancellation
        Unknown             Phasing
       Technology
       Challenges
                                           … or not
                                           LTA = Less Than Adequate



                                                                      11
Program in Trouble
• A program “… in trouble” has started down the path toward cancellation
    – In Trouble usually means
        • Failure to deliver functional capability (and/or)
        • Cost Overruns
              – Cost Overrun is tightly coupled to Schedule Slippage
              – Schedule Slippage is driven by many factors, Leading drivers include:
                   » Requirements Creep
                   » Technology Challenges
                   » Inadequate funding or phasing
    – A program “in trouble” may also have been “helped” get in trouble by external
      factors altering the baseline in such a way that the program may not be
      executable
        • Altered funding profile
        • Altered phasing of resources
        • Addition of requirements without allocation of funding or schedule adjustments
    – Mitigating factors may forestall cancellation or even prevent cancellation
    – One way or the other, a program in-trouble will be “in the conversation” (…
      on the chopping block)



                                                                                           12
Program Out-of-Step
                       New
                   Administration



 Game
                                              New
Changing
                                           Administrator
Event(s)




     Shift in Senate                 Economic
       or House                     Imperatives

                                                           13
Program Out-of-Step
• A program “out-of-step” is no longer aligned with
  Agency priorities or Agency risk posture
• Loss of alignment can result from:
   – Change in Agency leadership
   – Change in National Space Policy
   – Change in Congressional support
   – Public Pressure (e.g., potential adverse environmental
     impact)
   – External Factors (e.g., end of Cold War, Financial Crisis)
• A private sector metaphor: The business case is no
  longer viable
                                                                  14
Part III: Cancellation
   Factors / Model


                         15
Cancellation Vulnerability Scorecard (CVS)
• The CVS is proposed as a notional framework
  to assist in the discussion and analysis of
  program cancellations
  – Derived from preliminary analysis of data set on
    previous slide
  – Mini-case studies evaluating individual programs
    are provide in Appendices A and B




                                                       16
Cancellation Vulnerability Scorecard
                                  Factor                 High              Relative Strength / Weakness                         Low
Project Health




                         Tech Performance                   G        (+20) -------------------------------------(-20)               R


                            Cost/Schedule                   G        (+20) -------------------------------------(-20)               R


                         Agency Alignment                   A        (+15) -------------------------------------(-25)               a
Policy Alignment




                             Senate
                                                            S        (+10) -------------------------------------(-10)               s
                        Support/Opposition

                              House
                                                            H        (+10) -------------------------------------(-10)               h
                        Support/Opposition

                          External Factor(s)                G        (+20) -------------------------------------(-20)               R

                   Note: The case of Agency (OSTP/White House) misalignment with Congress would be represented by High scores for
                                                                                                                                        17
                   Agency alignment and low scores for Senate and/or House alignment – guaranteed churn and flail
Cancellation Vulnerability Scorecard
• CVS can give a general / relative sense of vulnerability
• Attributes or Factors are key
• Scoring values represent relative weights and are tailorable
  to an organization or program management environment
   – Technical Performance and cost given greatest weight (+/- 20)
   – Agency Alignment given asymmetric weighting where out of alignment
     can be very bad (-25) while fully aligned is +15 an expectation
   – Senate and House weights are bounded by +/- 10 reflecting the diffuse
     nature of Congressional influence
   – External factors (Wild Card) +/- 20 reflects potential powerful
     influence by an individual (e.g., support from a Congressional
     Committee Chairman) or an event (e.g., cancellation of a parent
     program)
• Individual cases will play out uniquely
• Cancellation outcomes also differ as discussed on next chart
                                                                       18
Cancellation “Outcomes”
• A range of outcomes                  Options          Examples
  from cold shutdown to
                                                       X-38, X-33,
  delay, de-scoping,                    Cold
  storage, or transfer       Done                    Comanche, A-12,
                                      Shutdown
  may allow a                                           Crusader
  “cancelled” program to                              Future Combat
  re-emerge down-                      De-Scope
                                                          System
  stream or continue
  under a different set of               Delay         SSF, ISS, F-35
  expectations and
  constraints                Lives    Partial Buy     Sea Wolf, F-22
                              On
• Knowledge Capture /                   Storage      X-34 (USAF/EAFB)
  Technology Capture is                 Transfer      X-37 to DARPA
  critical – NASA and DoD
  both have program                   Re-Frame/
                                                     CxP Orion / Ares
  termination record                   Re-Name
  retention processes          On      In Trouble
                             Bubble
                                                           JWST
                                      Hill Concern                      19
Part IV: Cancellation
Scenarios & Examples


                        20
Scenario

Program In Trouble – Cancelled
                   • Fails To Overcome
                     Technical, Schedule,
                     Budget Challenges –
                     Cannot Deliver

                   • Alternatives: re-
                     base-lining, re-
                     scoping not viable
                     options

                   • Weak Political
                     Support

                   • Not Agency Critical    21
Scenario

Program In Trouble – Lives-On
              • “Fixes” identified and
                implemented
                  • De-scoped / Re-planned
                  • Overcomes technical
                    challenges
              • Strong political support
              • Agency-critical
                 “Zombie      Danger
                Paradigm”
               Mega-overrun
                Programs
                Can “Eat”
                   other
                Program/
                  Project
                 Budgets
                                             22
Scenario

Program “On-the-Bubble”
               • Ongoing schedule /
                 budget challenges

               • Discover of costly “known
                 unknowns” (technical )
                 problems

               • One or more re-plan / re-
                 baseline

               • Agency Critical

               • Mixed Political Support



                                         23
Scenario

Program Out-of-Step/Cancelled
                    • Change Driven
                      • Policy
                      • People
                      • Political
                        Environment
                      • Economic
                        Environment
                      • World
                        Environment

                                   24
Scenario

Program Out-of-Step / Lives On
                    Survival Scenarios
                    • Misaligned With
                      Agency Direction / Re-
                      direction But Enjoying
                      Strong Political
                      Support

                    • Realignment
                        • Re-definition Of
                          Goals & Objectives

                    • Metamorphous
                       • Rename
                       • Storage
                       • Transfer To
                         Another Agency
                                          25
Examples – Cancellation Causation
         Summary Matrix
• Mini Case Studies (see
  adjacent X-33
  example)are presented
  in Appendix A and B.
  Summary results are
  provided in the matrix
  on the following page.



                                      26
C= cancelled                    Op=operating
                                                                                                           RB=re-baselined
                                                                           D= de-scoped
Key Factors
                                                                                                           M=morphed
                                                                           T = transferred




                                                                                                            Comanche
              Factor




                                                                                                Crusader




                                                                                                                                    Seawolf
                                                        JWST




                                                                                         A-12
                            X-33
                                   X-34
                                          X-37
                                                 X-38




                                                                                  F-35




                                                                                                                             F-22


                                                                                                                                              EFV
                                                               CxP




                                                                                                                       FCS
                                                                     SSF
                                                                            ISS
                Tech
                            X             X             X      X     X       X    X      X      X           X          X                      X
Project
Heath




            Performance

            Cost/Schedule   X      X                    X      X     X       X    X      X      X           X          X     X      X         X
              Agency
             Alignment
                            X      X      X      X             X                                X           X                X      X         X
Alignment
  Policy




               Senate
            Support/Opp
                                                        X      X     X       X
               House
            Support/Opp
                                                        X      X     X       X
Wild          External
              Factor(s)
                                                 X                                X      X      X           X                X      X         X
Card
                                   C/     C/            tb     C     C                                                       D/     D/
             Disposition    C
                                   T      T
                                                 C
                                                        d      M     M
                                                                            Op    RB     c       C           C         D
                                                                                                                             Op     Op
                                                                                                                                              C


                                                                                                                                              27
Part V: Proposed
Program Cancellation
   Review Process

                       28
PCR Process Outline
• Use NPR 8621.1B Mishap and Close Call
  Reporting as a Guide:
  – Selection of board members / advisors /
    consultants
  – Gather evidence and facts
  – Review and analysis of data
  – Generate recommendations
  – Develop report
  – Develop lessons learned


                                              29
Typical Steps in a PCR
                      Prepare for the PCR


                     Gather Evidence and
                            Facts

Interview Program                             Interview External
    Personnel                                     Personnel

                    Review and Analyze Data


                     Draw Conclusions and
                      Document Findings

    Generate                                   Generate Lessons
Recommendations                                    Learned

                       Develop PCR Report
                                                                   30
Mishap Report and PCR Similarities and Differences
          Elements              Mishap Report               PCR
HQ and Center Contingency           YES                     NO
Plans
Determine Appointing Official       YES                     YES
and Appoint Members
Secure the Site & Preserve          YES         Preserve NASA and Contractor
Evidence. Impound Data.                            Records and Databases
Witness Statements                  YES            YES. Inside NASA, the
                                                Contractor and Also External
                                                       Organizations
Contract Clause                     YES               May Be Required
Gather Physical Evidence and        YES                     YES
Facts
Review and analyze Data             YES                     YES
Draw Conclusions and                YES                     YES
Recommend Findings
Generate Recommendations            YES                     YES
Develop Report and Lessons          YES                     YES
Learned                                                                   31
Review & Analysis: Expanded
PCR Framework          PCR- Areas of
                       Investigation
                      A1. Executability
Tech Performance         Analysis
                         Elements
                      A2. U.S. Navy PoPS,
                         Probability of
                         Program Success
  Cost/Schedule          framework

                      B. Better Buying
                         Power Elements
                      C. Alignment
Agency Alignment
                      D. Circuit Breakers
     Senate
Support/Opposition
      House
Support/Opposition

 External Factor(s)
                                            32
Investigation Area A-1: Executability Analysis
                    Elements
• Executability analysis
  – Acquisition strategy
  – Planning and scheduling
  – Requirements
  – Software/hardware development,
    integration, & test
  – Risk management
  – Metrics & performance
    monitoring
  – Integrated product team (IPT)    (Bruckman 2012)
    effectiveness
  – Contractor integration &
    management

                                                       33
Investigation Area A-2: U.S. Navy PoPS,
Probability of Program Success framework
   Program Planning & Execution
   •   Acquisition Management
   •   Industry/Company Assessment
   •   Total Ownership Cost Estimating
   •   Test and Evaluation
   •   Technology Maturity
   •   Sustainment
   •   Software
   •   Contract Planning and
       Execution
   •   Government Program Office
       Performance
   •   Technology Protection


   Program Requirements
                                         Program Resources   External Influencers
   •   Parameter Status
                                         • Budget Planning   •   Fit in Vision
   •   Scope Evolution
                                                             •   Program Advocacy.
   •   CONOPS                            • Manning
                                                             •   Interdependencies.
                                                                                      34
Investigation Area B: DoD Better
 Buying Power Elements Related to
   Requirements / Cost / Schedule
Were programs
wired for success
of failure? Did
cancelled program
conduct:

• Affordability
  Analysis

• Will Cost /
  Should Cost
  Analysis

• Schedule Risk
  Management
                                     35
Investigation Area C: Alignment
• Lost Alignment
  – Weak political support / why?
    • Options / lessons learned / span of control / future
      guidance
    • Role of legislative affairs
    • Role of external affairs
  – Weak agency support / why?
    • Options / lessons learned / span of control / future
      guidance
    • Changes in agency strategic plan?
    • Changes in agency policies?
                                                         36
Investigation Area D: Circuit Breakers
• Role of “Circuit-breakers - Internal and
  External Review Processes
  – Internal
      • Program Management Council (PMC), Directorate
        Program Management Council (DPMC), Baseline
        Performance Review (BPR), Standing Review Board
        (SRB)
  – External
      • Government Accountability Office (GAO), Office of
        Management & Budget (OMB), Inspector General (IG),
        Aerospace Safety Review Panel (ASAP)
    How Effective was Review?   / Were we Listening?
                                                             37
Conclusion / Action
• Implement PCR Process - It will establish and enable “the
  discussion” that will help avoid future cancellations

   – Preparation for External Scrutiny / Reviews
      • GAO, OMB, Inspector General
   – Avoid repeating past mistakes
   – Knowledge capture and transfer
      • Documenting lessons learned
   – Promote greater program manager (PM) self awareness
      • More proactive pm’s
      • See future cancellation coming in time to remedy
   – Promote greater cultural willingness to look in the mirror
      • Get beyond embedded NASA “stopping rules”
   – More effective PM training
      • APPEL training
   – More effective reviews and internal controls
      • DPMC, PMC, BPR, SRB
                                                                  38
Appendix A
NASA Mini-Case Studies



                         39
Mini-Case
     X-33 Technology Demonstrator Test-bed
                                                                               The 1999 test failure
 X-33, SLI Program                                                             of the multi-lobed
                                                                               composite LH2 tank
 Start                    1993 / Goldin                                        doomed the program.
                                                                               Two options one a
 Cancelled                March 2001 / Goldin                                  redesign of the
                                                                               composite structure     inconsistent with test
 Sunk Cost                $940M                                                introduced additional   demonstration
                                                                               weight and created cg   objectives.
 Phase at                 Manufacturing underway (Phase D)                     issues. The other
 Cancellation                                                                  alternative, an Al-Li
                                                                               metal tank was
 Prime                    Lockheed Martin Skunk Works                          considered to be
 Contractor
 Acq. Vehicle             Cooperative Agreement
 Official Cause           technical
 Underlying               Composite, complex geometry LH2 fuel
 Cause(s)                 tank problems

NASA S&MA Review of X-33, March 1998
http://www.nasaspaceflight.com/2006/01/x-33venturestar-what-really-happened/
http://forum.nasaspaceflight.com/forums/thread-view.asp?tid=347&start=1
http://en.wikipedia.org/wiki/Lockheed_Martin_X-33

                                                                                                                           40
Mini-Case
        X-34 Technology Demonstrator Test-bed
X-34, SLI Program                                                                        X-34 Cancellation: In 2001 “goal
                                                                                         posts moved” after consecutive
Start                   1996 / Goldin                                                    Mars failures. Introduction of new
                                                                                         “safety requirements” (man-in-the-
Cancelled               March 2001 / Goldin                                              loop control) after the vehicle was
                                                                                         already in operational test created
Sunk Cost               $378 M                                                           an untenable cost/schedule
                                                                                         scenario.

Phase at                Integrated Test (Phase D), FASTRAC engine
                                                                                         X-34 Re-emergence?
Cancellation            integrated , tow testing completed, captive                      November 2010: “The two X-34s were hauled to a
                        carry testing underway                                           National Test Pilot School hangar at the Mojave
                                                                                         facility for inspections by Orbital Sciences personnel
Prime Contractor        Orbital Sciences                                                 to determine their viability for potential future flight
                                                                                         programs. “
Acq. Vehicle            Cooperative Agreement

Official Cause          “technical and schedule” reasons / (mainly
                        involving GFE, FASTRAC engine)
Underlying              Change in NASA senior management risk
Cause(s)                posture after back-to-back Mars failures
NASA Office of Safety & Mission Assurance, S&MA Review of X-33, June, 1998
http://www.nasa.gov/centers/dryden/news/FactSheets/FS-060-DFRC_prt.htm
http://history.nasa.gov/monograph31.pdf
http://en.wikipedia.org/wiki/Orbital_Sciences_X-34
http://forum.nasaspaceflight.com/index.php?topic=5806.30
http://www.flightglobal.com/blogs/graham-warwick/2008/02/whatever-happened-tonasas-x34-1.html
http://www.wired.com/dangerroom/2010/11/grounded-nasa-space-plane-poised-for-comeback/
http://www.sciencedaily.com/releases/2000/08/000801075234.htm
http://nasawatch.com/archives/2010/11/x-34-to-fly.html                                                                                              41
http://www.nasa.gov/centers/dryden/status_reports/X-34s_moved_11_17_10.html
Mini-Case
   X-37 Technology Demonstrator Test-bed
                                                                            The X-37 project
                                                                            progressed in a
X-37                                                                        satisfactory manner,
                                                                            overcoming a variety of
Start                 1998 as Cooperative Agreement                         technical challenges.
                                                                            Change in policy /
Cancelled             2003                                                  priorities ended NASA
                                                                            sponsorship in 2003.      The X-37B was
Sunk Cost             $173M cost sharing with Boeing                        The X-37 was              launched into orbit in
                                                                            transferred to DARPA      April 2010 and made
                                                                            in 2004 and continued     an autonomous landing
Phase at              Sub-scale X-40 flights completed. NASA                development.
                                                    http://wl                                         at VAFB in Dec. 2010.
Cancellation          transferred X-37 to DARPA in 2003.
Prime Contractor      Boeing

Official Cause(s)     Change in Agency policy direction

Notes:                X-37B emerging as Air Force Orbital Test
                      Vehicle OTV (AIAA 2011)


 http://www.spacenews.com/launch/100423-af-launches-space-plane.html
 http://www.spacenews.com/commentaries/100823-civilian-role-x37b.html
 http://www.msfc.nasa.gov/news/x37news/index.html
 http://www.nasa.gov/centers/marshall/news/news/releases/2003/03-126.html
 http://www.msfc.nasa.gov/news/news/releases/1998/98-141.html


                                                                                                                          42
Mini-Case
      X-38 Technology Demonstrator Test-bed
   X-38, International Space Station                                  The X-38 project
                                                                      was developed to
   Crew Return Vehicle (CRV)                                          serve as a potential
                                                                      ISS return vehicle.
   Start                1994 / Goldin                                 Overcoming a
                                                                      variety of technical
   Cancelled            April 2002 /OKeefe                            issues X-38 was on
                                                                      the path to succeed.
   Sunk Cost            $1.5 B                                        Severe cost overruns   primary factor in the X-38
                                                                      within the ISS         cancellation. Lack of mission
   Phase at             Three drop-test flight programs completed –   program during the     critical alignment was also a
                                                                      2000/2001 time-        factor in the decision.
   Cancellation         par foil descent to landing on skids          frame were a
   Prime Contractor     In-House JSC / Scaled Composites (airframe)

   Official Cause       ISS budget problems eventually led to
                        cancellation
   Underlying           Linkage to ISS / New Administrator / no
   Cause(s)             policy driver for X-38



X-38/CRV All Hands, EA3/John F. Muratore, 3 Dec 2001
http://en.wikipedia.org/wiki/NASA_X-38
http://en.wikipedia.org/wiki/File:ISS_Crew_Return_Vehicle.jpg
http://www.fas.org/programs/ssp/man/uswpns/air/xplanes/x38.html
http://www.nasa.gov/centers/dryden/news/FactSheets/FS-038-DFRC.html                                                  43
Mini-Case
    Constellation Program
 Constellation Program                                                 The Constellation program
                                                                       cancellation involved cost
 Start               Feb 2004 / O’Keefe, Griffin                       overruns and schedule delays,
                                                                       resulting in-part from: 1)
 Cancelled           Feb 2010 /Bolden                                  underfunding and altered
                                                                       phasing, 2) technical
 Sunk Cost           $9.5B + $2B cancellation                          challenges, and 3) program
                                                                       management approach and
 Phase at            Ares/Orion pre-CDR. Ares I-X flight test          constraints. The other and
 Cancellation        completed in October 2010                         overarching reason for
                                                                       cancellation was a shift in
 Prime Ctrs &        Orion / JSC, LM,                                                                  in Administration and
                                                                       Agency priorities and policy
 Contractor(s)       Ares / MSFC, ATK, Boeing                                                          the NASA leadership
                                                                       that accompanied change         team
 Acq. Vehicle        Multiple Competitive Contracts

 Official            “technical, cost, and schedule” reasons –
 Cause(s)            “lacked innovation”
 Underlying          Change in Space Policy
 Cause(s)
http://news.bbc.co.uk/2/hi/science/nature/8489097.stm
http://www.spacenews.com/civil/110614-memo-marks-end-constellation.html
http://en.wikipedia.org/wiki/Constellation_program
http://www.nasa.gov/mission_pages/constellation/news/index.html
http://www.scientificamerican.com/article.cfm?id=nasa-budget-constellation-
cancel
                                                                                                                         44
James Webb Space Telescope                                                                             Mini-Case

 James Webb Space Telescope
 Start               Planning initiated in 1996

 Cancelled                          On the Bubble
                                                                         “On the Bubble” January 2012: JWST is the subject of
 Sunk Cost           $3B + (2011)                                        heated debate within NASA House and Senate oversight
                                                                         committees threatening to consider cancellation and
 Phase at            Passed CDR – summer 2010 / 2011, the                cap future costs. The impact of JWST cost overruns on
 Cancellation        JWST program is in the final design and             other programs and projects is a major concern.
                     fabrication phase (Phase C).                        Multiple technology development issues as well as LTA
                                                                         program management have been cited as cost and
 Prime               GSFC, Northrop Grumman,                             schedule drivers.
 Contractor(s)


 Notes:              $1.6B initial/ then $5B, then $6.5B – now
                     $8.8 B (2018 launch/5 yr op)
                     Multiple re-baselines and re-planning
                     (major 2005)
 Notes:              17 international partners

http://www.jwst.nasa.gov/
http://www.pcmag.com/article/print/266679
http://www.spacepolicyonline.com/news/hall-this-is-last-opportunity-for-jwst
http://www.spacenews.com/civil/100111-congress-criticizes-spending-webb-telescope.html
http://www.wired.com/wiredscience/2011/09/astronomers-plead/
http://news.yahoo.com/scientists-fight-controversial-james-webb-space-telescope-112402158.html
                                                                                                                          45
Mini-Case
Space Station Freedom
Space Station Freedom                                          Space Station Freedom
                                                               (SSF) struggled with
Start                1984 - Early Development, 10 contracts    multiple redesigns, re-
                     signed in Sept 1988                       baselining, as well as
                                                               internal and external
Cancelled            By June 1993 after seven redesigns SSF    political conflict. SSF
                     was politically non-viable                provided a foundation for the ultimate redesign
                                                               development and implementation of the International
Sunk Cost            $11.4 B
                                                               Space Station
Phase at             Design completed – some hardware in
Cancellation         development
Prime                Boeing, PWR, McDonnell-Douglas
Contractor(s)
Cause(s)             Multiple redesigns and re-scoping
                     coupled with repeated underestimates of
                     cost coupled with uneven political
                     support

http://en.wikipedia.org/wiki/Space_Station_Freedom
http://www.russianspaceweb.com/iss_chronology.html
http://www.russianspaceweb.com/iss_us_roots.html
http://www.fas.org/spp/civil/crs/93-017.htm




                                                                                                                 46
Mini-Case
    International Space Station
                                                             The International Space Station (ISS)
  International Space Station (ISS)                          operating in orbit today survived a
                                                             serious re-evaluation in 2001.
  Start            1993 – building on the SSF                The Young report findings noted:
                   redesign efforts
                                                             “.. ISS Program’s technical             “The existing deficiencies in
                                                             achievements to date, as represented    management structure,
  “In-             IMCE (Young Report) –                     by on orbit capability, are             institutional culture, cost
  Trouble”         November 2001                             extraordinary.”                         estimating, and program control
                                                                                                     must be acknowledged and
  Notes            International Space Station (ISS)         “The existing ISS Program Plan for      corrected for the Program to move
                   Management and Cost Evaluation (IMCE)     executing the FY 02-06 budget is not    forward in a credible fashion”
                   Task Force emphasized need to develop     credible.”
                   better cost estimates and high fidelity
                   baseline for future budget planning




http://www.spacedaily.com/news/iss-01v.html                                                                     2001
http://history.nasa.gov/youngrep.pdf
http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/2002
0008687_2002004289.pdf




                                                                                                                                  47
Appendix B
DoD Mini-Case Studies



                        48
Mini-Case
 F-35 Lightening II
  F-35 Program
  Start                    November 1996

  Cancelled                Not Cancelled

  Development              Projected to be $382B for 2,443 aircraft
  Cost
  Phase at                 N/A. Currently in Development Flight
  Cancellation             Test.
                                                                                         “… In a nutshell, the F-35 program is five
  Prime                    Lockheed-Martin (airframe), Pratt-
                                                                                         to six years behind schedule. The
  Contractor(s)            Whitney (engine)
                                                                                         estimated cost to taxpayers has nearly
  Official                 Not cancelled.                                                doubled”
  Cause(s)
  Underlying               Three variants, concurrent engineering
  Cause(s)                 and test, low-TRL helmet-mounted
                           display, complex integration and life-
                           cycle support (international partners)
                           requirements, $1T Life cycle support
                           costs across 49 bases.
http://www.star-telegram.com/2011/01/29/2806149/f-35-started-with-recipe-for-
trouble.html#storylink=cpy
http://en.wikipedia.org/wiki/Lockheed_Martin_F-35_Lightning_II
http://www.defense.gov/news/newsarticle.aspx?id=62829
http://www.lse.co.uk/FinanceNews.asp?ArticleCode=eonzqi07td0tkfi&ArticleHeadline=US
_Navy_Air_Force_may_field_F35s_later_than_2016
http://www.foxnews.com/politics/2012/01/27/struggling-in-us-f-35-fighter-pushes-sales-                                                49
abroad/
Mini-Case
  A-12 Avenger II
 A-12 Program
 Start                 1984 contract award

 Cancelled             January 1991

 Sunk Cost             $5B

 Phase at              Just passed CDR in October 1990
 Cancellation                                                                  … the A-12 Avenger II - had "failed to make
                                                                               progress," were behind schedule and had
 Prime                 McDonnell-Aircraft and General                          designed a plane that was too heavy.
 Contractor(s)         Dynamics                                                Accordingly, the U.S. terminated the
 Acq. Vehicle          Firm-fixed price                                        contractors for "default,"

 Official              Breach of contract
 Cause(s)
 Underlying            Schedule delays and cost increases,
 Cause(s)              technical issues with stealth and radar
                       system, airframe overweight
http://en.wikipedia.org/wiki/McDonnell_Douglas_A-12_Avenger_II
http://news.google.com/newspapers?id=7FJSAAAAIBAJ&sjid=mzYNAAAAIBAJ&pg=3046,
1502437&dq=avenger+ii+cost&hl=en
http://www.defensenews.com/article/20110523/DEFSECT01/105230306/Supreme-
Court-Overturns-A-12-Ruling-Against-Contractors
http://www.defensenews.com/apps/pbcs.dll/article?AID=2011105300307
                                                                                                                             50
Mini-Case
    XM2001 Crusader
 Crusader Program
 Start                1QFY95

 Cancelled            2QFY02

 Sunk Cost            $2B of a planned $11B spent

 Phase at             Passed PDR in Nov 2001
 Cancellation
                                                                 “… In early May 2002, Secretary of Defense
 Prime                United Defense and General Dynamics
                                                                 Donald Rumsfeld cancelled the $11 billion USD
 Contractor(s)
                                                                 program because he considered it neither mobile
 Official             Cost increased to $25M per unit            nor precise enough…”
 Cause(s)
 Underlying           SecDef “transformational” goals.
 Cause(s)             Crusader was designed for a “different
                      strategic context”. Advanced alternative
                      systems (upgraded Paladin) were
                      available.

http://en.wikipedia.org/wiki/XM2001_Crusader




                                                                                                            51
Mini-Case
   Comanche
Comanche Program
Start              April 1991

Cancelled          February 2004
                                                                      “… the Comanche, conceived in 1983 during the Cold War,
Sunk Cost          $6.9B of a projected $39 B program                 was a victim of changing needs as the military's focus has
                                                                      shifted to the war on terrorism.”
Phase at           DOT&E
                                                                      “… keeping the Comanche "survivable" in the current
Cancellation                                                          threat environment would require design changes that
                                                                      would cost "several billion dollars" and erode the
Prime              Sikorsky, Boeing and Lockheed-Martin               chopper's stealth capability, one of the primary reasons for
Contractor(s)                                                         developing the aircraft”
Official           Schedule delays and cost overruns
Cause(s)
Underlying         Poor management, different operational
Cause(s)           environment--conceived during the cold
                   war, Iraq War lessons learned


http://articles.cnn.com/2004-02-23/us/helicopter.cancel_1_comanche-
sikorsky-aircraft-corporation-schoomaker?_s=PM:US
http://en.wikipedia.org/wiki/Boeing-Sikorsky_RAH-66_Comanche
http://www.aviationweek.com/aw/generic/story_generic.jsp?channel=
aerospacedaily&id=news/hel02244.xml
                                                                                                                            52
Mini-Case
     Future Combat Systems (FCS)
 FCS Program
 Start                 October 1999

 Cancelled             June 2009

 Sunk Cost             $18B of originally estimated $92B

 Phase at              Systems of Systems Functional Review
                                                                            "Since its inception, costs have gone up dramatically while
 Cancellation                                                               promised capability has steadily diminished”. . . . “The
                                                                            Government Accountability Office and the Congressional
 Prime                 Boeing and SAIC (lead integrators)
                                                                            Budget Office have questioned the cost and management of
 Contractor(s)                                                              Future Combat Systems”
 Official              DoD shift toward spending more on
 Cause(s)              counter-terrorism and less to prepare for
                       conventional warfare against large states
                       like China and Russia
 Underlying            Affordability. Costs have gone up
 Cause(s)              dramatically (est $233B) while promised
                       capability has steadily diminished.
http://en.wikipedia.org/wiki/Future_Combat_Systems
http://www.washingtonpost.com/wp-
dyn/content/story/2007/12/06/ST2007120602927.html
http://www.aviationweek.com/aw/generic/story.jsp?id=news/GROUND050809.xml
&headline=U.S.%20Army%20Ground%20Vehicles%20Up%20and%20Down&channel
=defense                                                                                                                           53
http://csis.org/files/media/csis/pubs/090205_fcsarmy.pdf
Mini-Case
F-22 Program
F-22 Program
Start                  April 1991 (ATF competition winner
                       announcement)
Production             July 2009
Termination
                                                                                “…The Cold War was over, it didn't make any sense to go forward
Sunk Cost              $65B for 187 aircraft                                    with the program," said Thomas Christie, a retired official who
                                                                                worked 50 years at the Pentagon. "But the Air Force built up such a
                                                                                large constituency up on the Hill that it couldn't be killed.“
Phase at               Production
Cancellation                                                                    … Two decades ago, the U.S. government planned to buy 648 of the
                                                                                fighters for $139 million apiece; the cost has almost tripled since
Prime                  Lockheed-Martin                                          then to $412 million, the Government Accountability Office said.
Contractor(s)                                                                   Recently retired Defense Secretary Robert M. Gates ended the
                                                                                purchase in 2009 at 188 planes ….
Official               5th generation fighters were required by
Cause(s)               all three services which favored the F-35
Underlying             End of the Cold War. Affordability of
Cause(s)               continuing production.

http://articles.latimes.com/2011/aug/07/business/la-fi-fighter-jets-grounded-
20110807
http://en.wikipedia.org/wiki/Lockheed_Martin_F-22_Raptor
http://www.slate.com/articles/news_and_politics/war_stories/2009/07/they_sc
rapped_the_f22.html
http://www.defensereview.com/f-22-raptor-program-cancellation-
defensereview-weighs-in/

                                                                                                                                                 54
Seawolf Program                                                                                                Mini-Case
 Seawolf Program
 Start                 August 1992 Concept Exploration and
                       Definition
 Cancelled             1995 (after U.S.S. Jimmy Carter
                       launched)
 Sunk Cost             $13B

 Phase at              Production. 3 or 29 planned submarines              …. ordered at the end of the Cold War in 1989. At one
 Cancellation          were built and deployed                             time, an intended fleet of 29 submarines was to be built
                                                                           over a ten-year period, later reduced to twelve
 Prime                 Electric Boat Division of General                   submarines. The end of the Cold War and budget
 Contractor(s)         Dynamics                                            constraints led to the cancellation in 1995 of any further
                                                                           additions to the fleet, leaving the Seawolf class limited to
 Official              Changing requirements environment and               just three boats
 Cause(s)              growing budget constraints, the superior
                       performance parameters of the Seawolf
                       class could not justify its cost
 Underlying            Faulty welding specs for HY-100 steel,
 Cause(s)              extensive rebuilding of the first hull of
                       with an estimated cost increase of
                       $68.6 million and a 1 yr schedule delay
http://www.gao.gov/assets/230/220367.pdf
http://www.globalsecurity.org/military/systems/ship/ssn-21.htm
http://www.gao.gov/archive/1995/ns95004.pdf
http://www.fas.org/programs/ssp/man/uswpns/navy/submarines/ssn21_seawolf
.html
http://www.cdi.org/issues/naval/seawolf.html
http://www.rand.org/content/dam/rand/pubs/monographs/2011/RAND_MG112                                                               55
8.2.pdf
Mini-Case
     Expeditionary Fighting Vehicle (EFV) Program
 EFV Program
 Start                 June 1996 Contract Award

 Cancelled             January 2011

 Sunk Cost             $3.3B

 Phase at              CDR in 2008, EFV was in System Design               “…the program is simply not affordable given
 Cancellation          and Development                                     likely Marine Corps procurement budgets,” said
                                                                           the general. “The procurement and
 Prime                 General Dynamics Land Systems                       operations/maintenance costs of this vehicle are
 Contractor(s)                                                             onerous” – USMC Commandant Gen J. Amos
 Type Contract         Cost Plus

 Official              Not affordable or sustainable
 Cause(s)
 Underlying            Poor management, required an
 Cause(s)              additional $12 billion to field the fleet of
                       about 550 EFVs, DoD budget cuts,
                       political pressure on DoD budget

http://www.efv.usmc.mil/
http://forums.military.com/eve/forums/a/tpc/f/8071946761/m/3520094181001
http://en.wikipedia.org/wiki/Expeditionary_Fighting_Vehicle
                                                                                                                              56

More Related Content

What's hot

Thomas.coonce
Thomas.coonceThomas.coonce
Thomas.coonceNASAPMC
 
Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1NASAPMC
 
Sean carter dan_deans
Sean carter dan_deansSean carter dan_deans
Sean carter dan_deansNASAPMC
 
Sandra smalley
Sandra smalleySandra smalley
Sandra smalleyNASAPMC
 
Backup darren elliott
Backup darren elliottBackup darren elliott
Backup darren elliottNASAPMC
 
Noneman.steven
Noneman.stevenNoneman.steven
Noneman.stevenNASAPMC
 
Kremic.tibor
Kremic.tiborKremic.tibor
Kremic.tiborNASAPMC
 
Vanessa.jeff
Vanessa.jeffVanessa.jeff
Vanessa.jeffNASAPMC
 
Cole ready aim fire impact!- status impact analysis - nasa
Cole   ready aim fire impact!- status impact analysis - nasaCole   ready aim fire impact!- status impact analysis - nasa
Cole ready aim fire impact!- status impact analysis - nasaNASAPMC
 
Hazen michael
Hazen michaelHazen michael
Hazen michaelNASAPMC
 
Lau.cheevon
Lau.cheevonLau.cheevon
Lau.cheevonNASAPMC
 
Semancik.susan
Semancik.susanSemancik.susan
Semancik.susanNASAPMC
 
Stephen.book
Stephen.bookStephen.book
Stephen.bookNASAPMC
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsenNASAPMC
 
Zimmerman barbier
Zimmerman barbierZimmerman barbier
Zimmerman barbierNASAPMC
 
Randall.taylor
Randall.taylorRandall.taylor
Randall.taylorNASAPMC
 
Smalley stigberg petze
Smalley stigberg petzeSmalley stigberg petze
Smalley stigberg petzeNASAPMC
 
Rider.stephen
Rider.stephenRider.stephen
Rider.stephenNASAPMC
 

What's hot (20)

Thomas.coonce
Thomas.coonceThomas.coonce
Thomas.coonce
 
Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1
 
Sean carter dan_deans
Sean carter dan_deansSean carter dan_deans
Sean carter dan_deans
 
Sandra smalley
Sandra smalleySandra smalley
Sandra smalley
 
Modular Construction Conference 2012
Modular Construction Conference 2012Modular Construction Conference 2012
Modular Construction Conference 2012
 
Backup darren elliott
Backup darren elliottBackup darren elliott
Backup darren elliott
 
Noneman.steven
Noneman.stevenNoneman.steven
Noneman.steven
 
Kremic.tibor
Kremic.tiborKremic.tibor
Kremic.tibor
 
Vanessa.jeff
Vanessa.jeffVanessa.jeff
Vanessa.jeff
 
Cole ready aim fire impact!- status impact analysis - nasa
Cole   ready aim fire impact!- status impact analysis - nasaCole   ready aim fire impact!- status impact analysis - nasa
Cole ready aim fire impact!- status impact analysis - nasa
 
Hazen michael
Hazen michaelHazen michael
Hazen michael
 
Lau.cheevon
Lau.cheevonLau.cheevon
Lau.cheevon
 
Modular constructionconference feb2012_a_popescu
Modular constructionconference  feb2012_a_popescuModular constructionconference  feb2012_a_popescu
Modular constructionconference feb2012_a_popescu
 
Semancik.susan
Semancik.susanSemancik.susan
Semancik.susan
 
Stephen.book
Stephen.bookStephen.book
Stephen.book
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsen
 
Zimmerman barbier
Zimmerman barbierZimmerman barbier
Zimmerman barbier
 
Randall.taylor
Randall.taylorRandall.taylor
Randall.taylor
 
Smalley stigberg petze
Smalley stigberg petzeSmalley stigberg petze
Smalley stigberg petze
 
Rider.stephen
Rider.stephenRider.stephen
Rider.stephen
 

Similar to D l engyel

Nichols.david
Nichols.davidNichols.david
Nichols.davidNASAPMC
 
Project management (A Basic Approach)
Project management (A Basic Approach)Project management (A Basic Approach)
Project management (A Basic Approach)Jed Concepcion
 
Neelam pandey project management issues
Neelam pandey   project management issuesNeelam pandey   project management issues
Neelam pandey project management issuesPMInstituteIndia
 
The Quarterbacks Dilemma
The Quarterbacks Dilemma The Quarterbacks Dilemma
The Quarterbacks Dilemma Allan Cytryn
 
Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...
Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...
Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...UNDP Climate
 
Six Sigma Green Belt Training Part 5
Six Sigma Green Belt Training Part 5Six Sigma Green Belt Training Part 5
Six Sigma Green Belt Training Part 5Skillogic Solutions
 
Managing Risk in Agile Development: It Isn’t Magic
Managing Risk in Agile Development: It Isn’t MagicManaging Risk in Agile Development: It Isn’t Magic
Managing Risk in Agile Development: It Isn’t MagicTechWell
 
Elico Solutions' Odoo ERP Project Management Implementation Approach
Elico Solutions' Odoo ERP Project Management Implementation ApproachElico Solutions' Odoo ERP Project Management Implementation Approach
Elico Solutions' Odoo ERP Project Management Implementation ApproachElico Solutions Singapore
 
Price.rick
Price.rickPrice.rick
Price.rickNASAPMC
 
Price.rick
Price.rickPrice.rick
Price.rickNASAPMC
 
Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30KristinH
 
Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30guest43777
 
Oracle eppm for asset intensive industries ppt part 1
Oracle eppm for asset intensive industries ppt part 1Oracle eppm for asset intensive industries ppt part 1
Oracle eppm for asset intensive industries ppt part 1p6academy
 
From arms race to green space
From arms race to green spaceFrom arms race to green space
From arms race to green spaceGlen Alleman
 

Similar to D l engyel (20)

lec6.ppt
lec6.pptlec6.ppt
lec6.ppt
 
Nichols.david
Nichols.davidNichols.david
Nichols.david
 
Project management (A Basic Approach)
Project management (A Basic Approach)Project management (A Basic Approach)
Project management (A Basic Approach)
 
Neelam pandey project management issues
Neelam pandey   project management issuesNeelam pandey   project management issues
Neelam pandey project management issues
 
The Quarterbacks Dilemma
The Quarterbacks Dilemma The Quarterbacks Dilemma
The Quarterbacks Dilemma
 
Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...
Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...
Developing a Project Logic/Results Framework - Session 9 Managing Project Pre...
 
Symons
SymonsSymons
Symons
 
Six Sigma Green Belt Training Part 5
Six Sigma Green Belt Training Part 5Six Sigma Green Belt Training Part 5
Six Sigma Green Belt Training Part 5
 
D07 Project Charter
D07 Project CharterD07 Project Charter
D07 Project Charter
 
Managing Risk in Agile Development: It Isn’t Magic
Managing Risk in Agile Development: It Isn’t MagicManaging Risk in Agile Development: It Isn’t Magic
Managing Risk in Agile Development: It Isn’t Magic
 
Elico Solutions' Odoo ERP Project Management Implementation Approach
Elico Solutions' Odoo ERP Project Management Implementation ApproachElico Solutions' Odoo ERP Project Management Implementation Approach
Elico Solutions' Odoo ERP Project Management Implementation Approach
 
PERIL LIBRARY
PERIL LIBRARYPERIL LIBRARY
PERIL LIBRARY
 
Price.rick
Price.rickPrice.rick
Price.rick
 
Price.rick
Price.rickPrice.rick
Price.rick
 
Risk management by YouExec
Risk management by YouExecRisk management by YouExec
Risk management by YouExec
 
Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30
 
Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30Bpm Risk Analysis 10 27 10 30
Bpm Risk Analysis 10 27 10 30
 
Risk Management
Risk ManagementRisk Management
Risk Management
 
Oracle eppm for asset intensive industries ppt part 1
Oracle eppm for asset intensive industries ppt part 1Oracle eppm for asset intensive industries ppt part 1
Oracle eppm for asset intensive industries ppt part 1
 
From arms race to green space
From arms race to green spaceFrom arms race to green space
From arms race to green space
 

More from NASAPMC

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk boNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
Yew manson
Yew mansonYew manson
Yew mansonNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)NASAPMC
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joeNASAPMC
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuartNASAPMC
 
Stock gahm
Stock gahmStock gahm
Stock gahmNASAPMC
 
Snow lee
Snow leeSnow lee
Snow leeNASAPMC
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandraNASAPMC
 
Seftas krage
Seftas krageSeftas krage
Seftas krageNASAPMC
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marcoNASAPMC
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mikeNASAPMC
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karleneNASAPMC
 
Rackley mike
Rackley mikeRackley mike
Rackley mikeNASAPMC
 
Paradis william
Paradis williamParadis william
Paradis williamNASAPMC
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeffNASAPMC
 
O'keefe william
O'keefe williamO'keefe william
O'keefe williamNASAPMC
 
Muller ralf
Muller ralfMuller ralf
Muller ralfNASAPMC
 

More from NASAPMC (20)

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk bo
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
Yew manson
Yew mansonYew manson
Yew manson
 
Wood frank
Wood frankWood frank
Wood frank
 
Wood frank
Wood frankWood frank
Wood frank
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joe
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuart
 
Stock gahm
Stock gahmStock gahm
Stock gahm
 
Snow lee
Snow leeSnow lee
Snow lee
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandra
 
Seftas krage
Seftas krageSeftas krage
Seftas krage
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marco
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mike
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karlene
 
Rackley mike
Rackley mikeRackley mike
Rackley mike
 
Paradis william
Paradis williamParadis william
Paradis william
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeff
 
O'keefe william
O'keefe williamO'keefe william
O'keefe william
 
Muller ralf
Muller ralfMuller ralf
Muller ralf
 

Recently uploaded

Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 

Recently uploaded (20)

Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 

D l engyel

  • 1. Draft - January 23 2012 Program Cancellation Failure Modes & Lessons Learned J. Steven Newman D.Sc. NASA Human Exploration & Operations Risk & Knowledge Management Support Team ARES Corporation David Lengyel Manager – NASA Human Exploration & Operations Risk & Knowledge Management PM Challenge 2012 – Orlando Fl. – February 22-23, 2012 1
  • 2. Outline • Part I: Introduction – The Need for Program Cancellation Reviews (PCRs) • Part II: Program Cancellation Overview • Part III: Program Cancellation Factors/Model • Part IV: Program Cancellation Scenarios & Examples • Part V: Proposed PCR Process 2
  • 3. Part I – Background / Call for Action • Catastrophic operational failures are documented and analyzed in-detail identifying their proximate and contributing or underlying causes • $2 Million trigger for Class A mishap formal reviews • Formal mishap review boards established in accordance with NASA Procedural Requirements document NPR 8621, Mishap and Close Call Reporting, Investigating, and Recordkeeping • No comparable process for program cancellation • Program termination announcements invariably cite cost overruns and schedule slippage as causal factors with little investigation / documentation of root causes • No formal process parallel to NPR 8621 to address program/project cancellations. Note: the term “program” is used throughout this presentation to discuss any cancelled activity – program, project, technology demonstrator • Program cancellations far exceed a mere $2 million “trigger value” by one ($20 million), two ($200 million), or even three ($2 billion) orders of magnitude • Action: Establish a formal Program Cancellation Review (PCR) process 3
  • 4. Part I - The Need & Benefits of Program Cancellation Reviews (PCRs) • Preparation for External Scrutiny / Reviews • Government Accountability Office (GAO), Office of Management & Budget (OMB), Inspector General (IG), Aerospace Safety Review Panel (ASAP) • Avoid Repeating Past Mistakes • Knowledge Capture and Transfer • Documenting and transferring Lessons Learned • Promote Increased Risk / Vulnerability Awareness within NASA PM community • Promote Greater Cultural Willingness to Look in the Mirror • Get beyond embedded NASA “Stopping Rules” • More Effective PM Training • Incorporated within APPEL curriculum • See Future Cancellations Coming In Time to Remedy • More Effective Reviews and Internal Controls • Program Management Council (PMC), Directorate Program Management Council (DPMC), Baseline Performance Review (BPR), Standing Review Board (SRB) 4
  • 5. Part I: Goals • Understand the roles of Agency leadership, Congress, and the White House in the cancellation process • Identify key factors in cancellation decisions – Analyze selected NASA and DoD Program Cancellations • Develop a model to assess cancellation vulnerability • Propose a cancellation review process 5
  • 6. Part I: Program Analysis Data-Set NASA DoD • X-33 • F-35 • X-34 • A-12 • X-37 • Crusader • X-38 Crew Rescue Vehicle • Comanche (CRV) • FCS • James Webb Space Telescope • F-22 (JWST) • Sea Wolf • Constellation (CxP) This data-set is considered sufficient to • Space Station Freedom (SSF) begin the process of identifying trends related to program cancellation. It is • International Space Station anticipated that future efforts will (ISS) expand the data set and analysis. 6
  • 8. Program Cancellation: Overview Cancellation or Not – They Decide US House and Senate White House / OSTP / Agency Leadership Program Cancellation Vulnerability Domains 8
  • 9. Congressional Leadership Considerations • Budget / budget deficit / tradeoffs • Alignment with space policy objectives • National importance or need • Affordability • Jobs in home district or home Center • Industry views 9
  • 10. Agency Leadership Considerations • Alignment with White House Space Policy Agenda • Alignment with Agency Strategic Plan • Does program addresses primary agency objectives? • Is program/project on critical path to other programs? • Does the program fulfill unique functional requirements? • Is the mission operational scenario consistent with Agency risk posture? • Are there overriding Agency-level or Program-level budget constraints? • Are there external events & constraints to consider? 10
  • 11. Program In Trouble LTA Req Management LTA Cost Estimation Cost Overrun LTA Pgm Schedule Planning LTA Funding / Cancellation Unknown Phasing Technology Challenges … or not LTA = Less Than Adequate 11
  • 12. Program in Trouble • A program “… in trouble” has started down the path toward cancellation – In Trouble usually means • Failure to deliver functional capability (and/or) • Cost Overruns – Cost Overrun is tightly coupled to Schedule Slippage – Schedule Slippage is driven by many factors, Leading drivers include: » Requirements Creep » Technology Challenges » Inadequate funding or phasing – A program “in trouble” may also have been “helped” get in trouble by external factors altering the baseline in such a way that the program may not be executable • Altered funding profile • Altered phasing of resources • Addition of requirements without allocation of funding or schedule adjustments – Mitigating factors may forestall cancellation or even prevent cancellation – One way or the other, a program in-trouble will be “in the conversation” (… on the chopping block) 12
  • 13. Program Out-of-Step New Administration Game New Changing Administrator Event(s) Shift in Senate Economic or House Imperatives 13
  • 14. Program Out-of-Step • A program “out-of-step” is no longer aligned with Agency priorities or Agency risk posture • Loss of alignment can result from: – Change in Agency leadership – Change in National Space Policy – Change in Congressional support – Public Pressure (e.g., potential adverse environmental impact) – External Factors (e.g., end of Cold War, Financial Crisis) • A private sector metaphor: The business case is no longer viable 14
  • 15. Part III: Cancellation Factors / Model 15
  • 16. Cancellation Vulnerability Scorecard (CVS) • The CVS is proposed as a notional framework to assist in the discussion and analysis of program cancellations – Derived from preliminary analysis of data set on previous slide – Mini-case studies evaluating individual programs are provide in Appendices A and B 16
  • 17. Cancellation Vulnerability Scorecard Factor High Relative Strength / Weakness Low Project Health Tech Performance G (+20) -------------------------------------(-20) R Cost/Schedule G (+20) -------------------------------------(-20) R Agency Alignment A (+15) -------------------------------------(-25) a Policy Alignment Senate S (+10) -------------------------------------(-10) s Support/Opposition House H (+10) -------------------------------------(-10) h Support/Opposition External Factor(s) G (+20) -------------------------------------(-20) R Note: The case of Agency (OSTP/White House) misalignment with Congress would be represented by High scores for 17 Agency alignment and low scores for Senate and/or House alignment – guaranteed churn and flail
  • 18. Cancellation Vulnerability Scorecard • CVS can give a general / relative sense of vulnerability • Attributes or Factors are key • Scoring values represent relative weights and are tailorable to an organization or program management environment – Technical Performance and cost given greatest weight (+/- 20) – Agency Alignment given asymmetric weighting where out of alignment can be very bad (-25) while fully aligned is +15 an expectation – Senate and House weights are bounded by +/- 10 reflecting the diffuse nature of Congressional influence – External factors (Wild Card) +/- 20 reflects potential powerful influence by an individual (e.g., support from a Congressional Committee Chairman) or an event (e.g., cancellation of a parent program) • Individual cases will play out uniquely • Cancellation outcomes also differ as discussed on next chart 18
  • 19. Cancellation “Outcomes” • A range of outcomes Options Examples from cold shutdown to X-38, X-33, delay, de-scoping, Cold storage, or transfer Done Comanche, A-12, Shutdown may allow a Crusader “cancelled” program to Future Combat re-emerge down- De-Scope System stream or continue under a different set of Delay SSF, ISS, F-35 expectations and constraints Lives Partial Buy Sea Wolf, F-22 On • Knowledge Capture / Storage X-34 (USAF/EAFB) Technology Capture is Transfer X-37 to DARPA critical – NASA and DoD both have program Re-Frame/ CxP Orion / Ares termination record Re-Name retention processes On In Trouble Bubble JWST Hill Concern 19
  • 21. Scenario Program In Trouble – Cancelled • Fails To Overcome Technical, Schedule, Budget Challenges – Cannot Deliver • Alternatives: re- base-lining, re- scoping not viable options • Weak Political Support • Not Agency Critical 21
  • 22. Scenario Program In Trouble – Lives-On • “Fixes” identified and implemented • De-scoped / Re-planned • Overcomes technical challenges • Strong political support • Agency-critical “Zombie Danger Paradigm” Mega-overrun Programs Can “Eat” other Program/ Project Budgets 22
  • 23. Scenario Program “On-the-Bubble” • Ongoing schedule / budget challenges • Discover of costly “known unknowns” (technical ) problems • One or more re-plan / re- baseline • Agency Critical • Mixed Political Support 23
  • 24. Scenario Program Out-of-Step/Cancelled • Change Driven • Policy • People • Political Environment • Economic Environment • World Environment 24
  • 25. Scenario Program Out-of-Step / Lives On Survival Scenarios • Misaligned With Agency Direction / Re- direction But Enjoying Strong Political Support • Realignment • Re-definition Of Goals & Objectives • Metamorphous • Rename • Storage • Transfer To Another Agency 25
  • 26. Examples – Cancellation Causation Summary Matrix • Mini Case Studies (see adjacent X-33 example)are presented in Appendix A and B. Summary results are provided in the matrix on the following page. 26
  • 27. C= cancelled Op=operating RB=re-baselined D= de-scoped Key Factors M=morphed T = transferred Comanche Factor Crusader Seawolf JWST A-12 X-33 X-34 X-37 X-38 F-35 F-22 EFV CxP FCS SSF ISS Tech X X X X X X X X X X X X Project Heath Performance Cost/Schedule X X X X X X X X X X X X X X Agency Alignment X X X X X X X X X X Alignment Policy Senate Support/Opp X X X X House Support/Opp X X X X Wild External Factor(s) X X X X X X X X Card C/ C/ tb C C D/ D/ Disposition C T T C d M M Op RB c C C D Op Op C 27
  • 28. Part V: Proposed Program Cancellation Review Process 28
  • 29. PCR Process Outline • Use NPR 8621.1B Mishap and Close Call Reporting as a Guide: – Selection of board members / advisors / consultants – Gather evidence and facts – Review and analysis of data – Generate recommendations – Develop report – Develop lessons learned 29
  • 30. Typical Steps in a PCR Prepare for the PCR Gather Evidence and Facts Interview Program Interview External Personnel Personnel Review and Analyze Data Draw Conclusions and Document Findings Generate Generate Lessons Recommendations Learned Develop PCR Report 30
  • 31. Mishap Report and PCR Similarities and Differences Elements Mishap Report PCR HQ and Center Contingency YES NO Plans Determine Appointing Official YES YES and Appoint Members Secure the Site & Preserve YES Preserve NASA and Contractor Evidence. Impound Data. Records and Databases Witness Statements YES YES. Inside NASA, the Contractor and Also External Organizations Contract Clause YES May Be Required Gather Physical Evidence and YES YES Facts Review and analyze Data YES YES Draw Conclusions and YES YES Recommend Findings Generate Recommendations YES YES Develop Report and Lessons YES YES Learned 31
  • 32. Review & Analysis: Expanded PCR Framework PCR- Areas of Investigation A1. Executability Tech Performance Analysis Elements A2. U.S. Navy PoPS, Probability of Program Success Cost/Schedule framework B. Better Buying Power Elements C. Alignment Agency Alignment D. Circuit Breakers Senate Support/Opposition House Support/Opposition External Factor(s) 32
  • 33. Investigation Area A-1: Executability Analysis Elements • Executability analysis – Acquisition strategy – Planning and scheduling – Requirements – Software/hardware development, integration, & test – Risk management – Metrics & performance monitoring – Integrated product team (IPT) (Bruckman 2012) effectiveness – Contractor integration & management 33
  • 34. Investigation Area A-2: U.S. Navy PoPS, Probability of Program Success framework Program Planning & Execution • Acquisition Management • Industry/Company Assessment • Total Ownership Cost Estimating • Test and Evaluation • Technology Maturity • Sustainment • Software • Contract Planning and Execution • Government Program Office Performance • Technology Protection Program Requirements Program Resources External Influencers • Parameter Status • Budget Planning • Fit in Vision • Scope Evolution • Program Advocacy. • CONOPS • Manning • Interdependencies. 34
  • 35. Investigation Area B: DoD Better Buying Power Elements Related to Requirements / Cost / Schedule Were programs wired for success of failure? Did cancelled program conduct: • Affordability Analysis • Will Cost / Should Cost Analysis • Schedule Risk Management 35
  • 36. Investigation Area C: Alignment • Lost Alignment – Weak political support / why? • Options / lessons learned / span of control / future guidance • Role of legislative affairs • Role of external affairs – Weak agency support / why? • Options / lessons learned / span of control / future guidance • Changes in agency strategic plan? • Changes in agency policies? 36
  • 37. Investigation Area D: Circuit Breakers • Role of “Circuit-breakers - Internal and External Review Processes – Internal • Program Management Council (PMC), Directorate Program Management Council (DPMC), Baseline Performance Review (BPR), Standing Review Board (SRB) – External • Government Accountability Office (GAO), Office of Management & Budget (OMB), Inspector General (IG), Aerospace Safety Review Panel (ASAP) How Effective was Review? / Were we Listening? 37
  • 38. Conclusion / Action • Implement PCR Process - It will establish and enable “the discussion” that will help avoid future cancellations – Preparation for External Scrutiny / Reviews • GAO, OMB, Inspector General – Avoid repeating past mistakes – Knowledge capture and transfer • Documenting lessons learned – Promote greater program manager (PM) self awareness • More proactive pm’s • See future cancellation coming in time to remedy – Promote greater cultural willingness to look in the mirror • Get beyond embedded NASA “stopping rules” – More effective PM training • APPEL training – More effective reviews and internal controls • DPMC, PMC, BPR, SRB 38
  • 40. Mini-Case X-33 Technology Demonstrator Test-bed The 1999 test failure X-33, SLI Program of the multi-lobed composite LH2 tank Start 1993 / Goldin doomed the program. Two options one a Cancelled March 2001 / Goldin redesign of the composite structure inconsistent with test Sunk Cost $940M introduced additional demonstration weight and created cg objectives. Phase at Manufacturing underway (Phase D) issues. The other Cancellation alternative, an Al-Li metal tank was Prime Lockheed Martin Skunk Works considered to be Contractor Acq. Vehicle Cooperative Agreement Official Cause technical Underlying Composite, complex geometry LH2 fuel Cause(s) tank problems NASA S&MA Review of X-33, March 1998 http://www.nasaspaceflight.com/2006/01/x-33venturestar-what-really-happened/ http://forum.nasaspaceflight.com/forums/thread-view.asp?tid=347&start=1 http://en.wikipedia.org/wiki/Lockheed_Martin_X-33 40
  • 41. Mini-Case X-34 Technology Demonstrator Test-bed X-34, SLI Program X-34 Cancellation: In 2001 “goal posts moved” after consecutive Start 1996 / Goldin Mars failures. Introduction of new “safety requirements” (man-in-the- Cancelled March 2001 / Goldin loop control) after the vehicle was already in operational test created Sunk Cost $378 M an untenable cost/schedule scenario. Phase at Integrated Test (Phase D), FASTRAC engine X-34 Re-emergence? Cancellation integrated , tow testing completed, captive November 2010: “The two X-34s were hauled to a carry testing underway National Test Pilot School hangar at the Mojave facility for inspections by Orbital Sciences personnel Prime Contractor Orbital Sciences to determine their viability for potential future flight programs. “ Acq. Vehicle Cooperative Agreement Official Cause “technical and schedule” reasons / (mainly involving GFE, FASTRAC engine) Underlying Change in NASA senior management risk Cause(s) posture after back-to-back Mars failures NASA Office of Safety & Mission Assurance, S&MA Review of X-33, June, 1998 http://www.nasa.gov/centers/dryden/news/FactSheets/FS-060-DFRC_prt.htm http://history.nasa.gov/monograph31.pdf http://en.wikipedia.org/wiki/Orbital_Sciences_X-34 http://forum.nasaspaceflight.com/index.php?topic=5806.30 http://www.flightglobal.com/blogs/graham-warwick/2008/02/whatever-happened-tonasas-x34-1.html http://www.wired.com/dangerroom/2010/11/grounded-nasa-space-plane-poised-for-comeback/ http://www.sciencedaily.com/releases/2000/08/000801075234.htm http://nasawatch.com/archives/2010/11/x-34-to-fly.html 41 http://www.nasa.gov/centers/dryden/status_reports/X-34s_moved_11_17_10.html
  • 42. Mini-Case X-37 Technology Demonstrator Test-bed The X-37 project progressed in a X-37 satisfactory manner, overcoming a variety of Start 1998 as Cooperative Agreement technical challenges. Change in policy / Cancelled 2003 priorities ended NASA sponsorship in 2003. The X-37B was Sunk Cost $173M cost sharing with Boeing The X-37 was launched into orbit in transferred to DARPA April 2010 and made in 2004 and continued an autonomous landing Phase at Sub-scale X-40 flights completed. NASA development. http://wl at VAFB in Dec. 2010. Cancellation transferred X-37 to DARPA in 2003. Prime Contractor Boeing Official Cause(s) Change in Agency policy direction Notes: X-37B emerging as Air Force Orbital Test Vehicle OTV (AIAA 2011) http://www.spacenews.com/launch/100423-af-launches-space-plane.html http://www.spacenews.com/commentaries/100823-civilian-role-x37b.html http://www.msfc.nasa.gov/news/x37news/index.html http://www.nasa.gov/centers/marshall/news/news/releases/2003/03-126.html http://www.msfc.nasa.gov/news/news/releases/1998/98-141.html 42
  • 43. Mini-Case X-38 Technology Demonstrator Test-bed X-38, International Space Station The X-38 project was developed to Crew Return Vehicle (CRV) serve as a potential ISS return vehicle. Start 1994 / Goldin Overcoming a variety of technical Cancelled April 2002 /OKeefe issues X-38 was on the path to succeed. Sunk Cost $1.5 B Severe cost overruns primary factor in the X-38 within the ISS cancellation. Lack of mission Phase at Three drop-test flight programs completed – program during the critical alignment was also a 2000/2001 time- factor in the decision. Cancellation par foil descent to landing on skids frame were a Prime Contractor In-House JSC / Scaled Composites (airframe) Official Cause ISS budget problems eventually led to cancellation Underlying Linkage to ISS / New Administrator / no Cause(s) policy driver for X-38 X-38/CRV All Hands, EA3/John F. Muratore, 3 Dec 2001 http://en.wikipedia.org/wiki/NASA_X-38 http://en.wikipedia.org/wiki/File:ISS_Crew_Return_Vehicle.jpg http://www.fas.org/programs/ssp/man/uswpns/air/xplanes/x38.html http://www.nasa.gov/centers/dryden/news/FactSheets/FS-038-DFRC.html 43
  • 44. Mini-Case Constellation Program Constellation Program The Constellation program cancellation involved cost Start Feb 2004 / O’Keefe, Griffin overruns and schedule delays, resulting in-part from: 1) Cancelled Feb 2010 /Bolden underfunding and altered phasing, 2) technical Sunk Cost $9.5B + $2B cancellation challenges, and 3) program management approach and Phase at Ares/Orion pre-CDR. Ares I-X flight test constraints. The other and Cancellation completed in October 2010 overarching reason for cancellation was a shift in Prime Ctrs & Orion / JSC, LM, in Administration and Agency priorities and policy Contractor(s) Ares / MSFC, ATK, Boeing the NASA leadership that accompanied change team Acq. Vehicle Multiple Competitive Contracts Official “technical, cost, and schedule” reasons – Cause(s) “lacked innovation” Underlying Change in Space Policy Cause(s) http://news.bbc.co.uk/2/hi/science/nature/8489097.stm http://www.spacenews.com/civil/110614-memo-marks-end-constellation.html http://en.wikipedia.org/wiki/Constellation_program http://www.nasa.gov/mission_pages/constellation/news/index.html http://www.scientificamerican.com/article.cfm?id=nasa-budget-constellation- cancel 44
  • 45. James Webb Space Telescope Mini-Case James Webb Space Telescope Start Planning initiated in 1996 Cancelled On the Bubble “On the Bubble” January 2012: JWST is the subject of Sunk Cost $3B + (2011) heated debate within NASA House and Senate oversight committees threatening to consider cancellation and Phase at Passed CDR – summer 2010 / 2011, the cap future costs. The impact of JWST cost overruns on Cancellation JWST program is in the final design and other programs and projects is a major concern. fabrication phase (Phase C). Multiple technology development issues as well as LTA program management have been cited as cost and Prime GSFC, Northrop Grumman, schedule drivers. Contractor(s) Notes: $1.6B initial/ then $5B, then $6.5B – now $8.8 B (2018 launch/5 yr op) Multiple re-baselines and re-planning (major 2005) Notes: 17 international partners http://www.jwst.nasa.gov/ http://www.pcmag.com/article/print/266679 http://www.spacepolicyonline.com/news/hall-this-is-last-opportunity-for-jwst http://www.spacenews.com/civil/100111-congress-criticizes-spending-webb-telescope.html http://www.wired.com/wiredscience/2011/09/astronomers-plead/ http://news.yahoo.com/scientists-fight-controversial-james-webb-space-telescope-112402158.html 45
  • 46. Mini-Case Space Station Freedom Space Station Freedom Space Station Freedom (SSF) struggled with Start 1984 - Early Development, 10 contracts multiple redesigns, re- signed in Sept 1988 baselining, as well as internal and external Cancelled By June 1993 after seven redesigns SSF political conflict. SSF was politically non-viable provided a foundation for the ultimate redesign development and implementation of the International Sunk Cost $11.4 B Space Station Phase at Design completed – some hardware in Cancellation development Prime Boeing, PWR, McDonnell-Douglas Contractor(s) Cause(s) Multiple redesigns and re-scoping coupled with repeated underestimates of cost coupled with uneven political support http://en.wikipedia.org/wiki/Space_Station_Freedom http://www.russianspaceweb.com/iss_chronology.html http://www.russianspaceweb.com/iss_us_roots.html http://www.fas.org/spp/civil/crs/93-017.htm 46
  • 47. Mini-Case International Space Station The International Space Station (ISS) International Space Station (ISS) operating in orbit today survived a serious re-evaluation in 2001. Start 1993 – building on the SSF The Young report findings noted: redesign efforts “.. ISS Program’s technical “The existing deficiencies in achievements to date, as represented management structure, “In- IMCE (Young Report) – by on orbit capability, are institutional culture, cost Trouble” November 2001 extraordinary.” estimating, and program control must be acknowledged and Notes International Space Station (ISS) “The existing ISS Program Plan for corrected for the Program to move Management and Cost Evaluation (IMCE) executing the FY 02-06 budget is not forward in a credible fashion” Task Force emphasized need to develop credible.” better cost estimates and high fidelity baseline for future budget planning http://www.spacedaily.com/news/iss-01v.html 2001 http://history.nasa.gov/youngrep.pdf http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/2002 0008687_2002004289.pdf 47
  • 49. Mini-Case F-35 Lightening II F-35 Program Start November 1996 Cancelled Not Cancelled Development Projected to be $382B for 2,443 aircraft Cost Phase at N/A. Currently in Development Flight Cancellation Test. “… In a nutshell, the F-35 program is five Prime Lockheed-Martin (airframe), Pratt- to six years behind schedule. The Contractor(s) Whitney (engine) estimated cost to taxpayers has nearly Official Not cancelled. doubled” Cause(s) Underlying Three variants, concurrent engineering Cause(s) and test, low-TRL helmet-mounted display, complex integration and life- cycle support (international partners) requirements, $1T Life cycle support costs across 49 bases. http://www.star-telegram.com/2011/01/29/2806149/f-35-started-with-recipe-for- trouble.html#storylink=cpy http://en.wikipedia.org/wiki/Lockheed_Martin_F-35_Lightning_II http://www.defense.gov/news/newsarticle.aspx?id=62829 http://www.lse.co.uk/FinanceNews.asp?ArticleCode=eonzqi07td0tkfi&ArticleHeadline=US _Navy_Air_Force_may_field_F35s_later_than_2016 http://www.foxnews.com/politics/2012/01/27/struggling-in-us-f-35-fighter-pushes-sales- 49 abroad/
  • 50. Mini-Case A-12 Avenger II A-12 Program Start 1984 contract award Cancelled January 1991 Sunk Cost $5B Phase at Just passed CDR in October 1990 Cancellation … the A-12 Avenger II - had "failed to make progress," were behind schedule and had Prime McDonnell-Aircraft and General designed a plane that was too heavy. Contractor(s) Dynamics Accordingly, the U.S. terminated the Acq. Vehicle Firm-fixed price contractors for "default," Official Breach of contract Cause(s) Underlying Schedule delays and cost increases, Cause(s) technical issues with stealth and radar system, airframe overweight http://en.wikipedia.org/wiki/McDonnell_Douglas_A-12_Avenger_II http://news.google.com/newspapers?id=7FJSAAAAIBAJ&sjid=mzYNAAAAIBAJ&pg=3046, 1502437&dq=avenger+ii+cost&hl=en http://www.defensenews.com/article/20110523/DEFSECT01/105230306/Supreme- Court-Overturns-A-12-Ruling-Against-Contractors http://www.defensenews.com/apps/pbcs.dll/article?AID=2011105300307 50
  • 51. Mini-Case XM2001 Crusader Crusader Program Start 1QFY95 Cancelled 2QFY02 Sunk Cost $2B of a planned $11B spent Phase at Passed PDR in Nov 2001 Cancellation “… In early May 2002, Secretary of Defense Prime United Defense and General Dynamics Donald Rumsfeld cancelled the $11 billion USD Contractor(s) program because he considered it neither mobile Official Cost increased to $25M per unit nor precise enough…” Cause(s) Underlying SecDef “transformational” goals. Cause(s) Crusader was designed for a “different strategic context”. Advanced alternative systems (upgraded Paladin) were available. http://en.wikipedia.org/wiki/XM2001_Crusader 51
  • 52. Mini-Case Comanche Comanche Program Start April 1991 Cancelled February 2004 “… the Comanche, conceived in 1983 during the Cold War, Sunk Cost $6.9B of a projected $39 B program was a victim of changing needs as the military's focus has shifted to the war on terrorism.” Phase at DOT&E “… keeping the Comanche "survivable" in the current Cancellation threat environment would require design changes that would cost "several billion dollars" and erode the Prime Sikorsky, Boeing and Lockheed-Martin chopper's stealth capability, one of the primary reasons for Contractor(s) developing the aircraft” Official Schedule delays and cost overruns Cause(s) Underlying Poor management, different operational Cause(s) environment--conceived during the cold war, Iraq War lessons learned http://articles.cnn.com/2004-02-23/us/helicopter.cancel_1_comanche- sikorsky-aircraft-corporation-schoomaker?_s=PM:US http://en.wikipedia.org/wiki/Boeing-Sikorsky_RAH-66_Comanche http://www.aviationweek.com/aw/generic/story_generic.jsp?channel= aerospacedaily&id=news/hel02244.xml 52
  • 53. Mini-Case Future Combat Systems (FCS) FCS Program Start October 1999 Cancelled June 2009 Sunk Cost $18B of originally estimated $92B Phase at Systems of Systems Functional Review "Since its inception, costs have gone up dramatically while Cancellation promised capability has steadily diminished”. . . . “The Government Accountability Office and the Congressional Prime Boeing and SAIC (lead integrators) Budget Office have questioned the cost and management of Contractor(s) Future Combat Systems” Official DoD shift toward spending more on Cause(s) counter-terrorism and less to prepare for conventional warfare against large states like China and Russia Underlying Affordability. Costs have gone up Cause(s) dramatically (est $233B) while promised capability has steadily diminished. http://en.wikipedia.org/wiki/Future_Combat_Systems http://www.washingtonpost.com/wp- dyn/content/story/2007/12/06/ST2007120602927.html http://www.aviationweek.com/aw/generic/story.jsp?id=news/GROUND050809.xml &headline=U.S.%20Army%20Ground%20Vehicles%20Up%20and%20Down&channel =defense 53 http://csis.org/files/media/csis/pubs/090205_fcsarmy.pdf
  • 54. Mini-Case F-22 Program F-22 Program Start April 1991 (ATF competition winner announcement) Production July 2009 Termination “…The Cold War was over, it didn't make any sense to go forward Sunk Cost $65B for 187 aircraft with the program," said Thomas Christie, a retired official who worked 50 years at the Pentagon. "But the Air Force built up such a large constituency up on the Hill that it couldn't be killed.“ Phase at Production Cancellation … Two decades ago, the U.S. government planned to buy 648 of the fighters for $139 million apiece; the cost has almost tripled since Prime Lockheed-Martin then to $412 million, the Government Accountability Office said. Contractor(s) Recently retired Defense Secretary Robert M. Gates ended the purchase in 2009 at 188 planes …. Official 5th generation fighters were required by Cause(s) all three services which favored the F-35 Underlying End of the Cold War. Affordability of Cause(s) continuing production. http://articles.latimes.com/2011/aug/07/business/la-fi-fighter-jets-grounded- 20110807 http://en.wikipedia.org/wiki/Lockheed_Martin_F-22_Raptor http://www.slate.com/articles/news_and_politics/war_stories/2009/07/they_sc rapped_the_f22.html http://www.defensereview.com/f-22-raptor-program-cancellation- defensereview-weighs-in/ 54
  • 55. Seawolf Program Mini-Case Seawolf Program Start August 1992 Concept Exploration and Definition Cancelled 1995 (after U.S.S. Jimmy Carter launched) Sunk Cost $13B Phase at Production. 3 or 29 planned submarines …. ordered at the end of the Cold War in 1989. At one Cancellation were built and deployed time, an intended fleet of 29 submarines was to be built over a ten-year period, later reduced to twelve Prime Electric Boat Division of General submarines. The end of the Cold War and budget Contractor(s) Dynamics constraints led to the cancellation in 1995 of any further additions to the fleet, leaving the Seawolf class limited to Official Changing requirements environment and just three boats Cause(s) growing budget constraints, the superior performance parameters of the Seawolf class could not justify its cost Underlying Faulty welding specs for HY-100 steel, Cause(s) extensive rebuilding of the first hull of with an estimated cost increase of $68.6 million and a 1 yr schedule delay http://www.gao.gov/assets/230/220367.pdf http://www.globalsecurity.org/military/systems/ship/ssn-21.htm http://www.gao.gov/archive/1995/ns95004.pdf http://www.fas.org/programs/ssp/man/uswpns/navy/submarines/ssn21_seawolf .html http://www.cdi.org/issues/naval/seawolf.html http://www.rand.org/content/dam/rand/pubs/monographs/2011/RAND_MG112 55 8.2.pdf
  • 56. Mini-Case Expeditionary Fighting Vehicle (EFV) Program EFV Program Start June 1996 Contract Award Cancelled January 2011 Sunk Cost $3.3B Phase at CDR in 2008, EFV was in System Design “…the program is simply not affordable given Cancellation and Development likely Marine Corps procurement budgets,” said the general. “The procurement and Prime General Dynamics Land Systems operations/maintenance costs of this vehicle are Contractor(s) onerous” – USMC Commandant Gen J. Amos Type Contract Cost Plus Official Not affordable or sustainable Cause(s) Underlying Poor management, required an Cause(s) additional $12 billion to field the fleet of about 550 EFVs, DoD budget cuts, political pressure on DoD budget http://www.efv.usmc.mil/ http://forums.military.com/eve/forums/a/tpc/f/8071946761/m/3520094181001 http://en.wikipedia.org/wiki/Expeditionary_Fighting_Vehicle 56