SlideShare une entreprise Scribd logo
1  sur  23
Télécharger pour lire hors ligne
Review Policy Evolution
Current and upcoming upgrades to independent
        review policies and processes




          Dr. James Ortiz, Director
   Independent Program Assessment Office
                  Eight Annual NASA
          Program Management Challenge 2011
                   February 9, 2011
                                               Page 1
Introduction


• The Independent Program Assessment Office (IPAO) manages
  the independent review of the Agency's Programs and
  projects at life-cycle milestones to ensure the highest
  probability of mission success.

• This presentation provides an overview of the Agency’s policy
  changes to improve the effectiveness and efficiency of the
  Agency Review Processes.

• The bulk of these changes are being considered as part of the
  upcoming revision to NPR 7120.5 (the “E” version).




                                                                  Page 2
Outline

• Agency policy context
• Change Areas
   – Standard TOR
   – Lifecycle Review Timelines (1-step; 2-step)
   – Lifecycle Products
   – Review Criteria
   – Maturity Tables
   – Team Composition and Balance
   – Convening Authorities
   – Readiness Assessment
   – Decision Memos
   – Changes to Lifecycle Reviews
• Summary
                                                   Page 3
Agency Policy Context for
  Independent Review




                            Page 4
Independent Review Policy Context


• Agency policy context
   – The “Why”
       » NPD 1000.0; NPD 1000.5
   – The “What”
       » NPR 7120.5; NPR 7123.1
   – The “How”
       » SRB Handbook
       » IPAO System Operating Procedures




                                            Page 5
Policy Flow-down


                                                                        NPD 1000.0
                                                                        NPD 1000.3
                                                                        NPD 1000.5




                              NPD 7120.4                   NPD 8700.1                     NPD 8900.5A                      Mission Support Office




                                           NPR 7120.5D                                                                                     Support Organization
         NPR 7123.1A                                                 OSMA    Directives                 OCHMO Directives
                                             and NID                                                                                             Directives


                          1        2                                     3                               4                                    5




                                                         NASA -SP 2009 -10 -015 - HQ          Center Engineering                       Mission Directorate
     NASA -SP -6105
                                                             SRB HANDBOOK                        Requirements                             Requirements
SYSTEMS ENGINEERING
      HANDBOOK
                                                                                                    7                                         6

                           IPAO 1000
                      IPAO MANAGEMENT
                              PLAN                                                        LEGEND:
                                                                                                                                                     DIRECT
                                                                                          1: ENGINEERING REQUIREMENTS
                                                                                          2: PROGRAM/PROJECT
                             IPAO 2000                                                    MANAGEMENT REQUIREMENTS
                       STANDARD OPERATING                                                                                                            INDIRECT
                                                                                          3. SAFETY AND MISSION
                            PROCEDURE                                                     ASSURANCE REQUIREMENTS
                                                                                          4. HEALTH AND MEDICAL REQTS
                                                                                          5. MSO FUNCTIONAL REQTS
                                                                                          6. MD REQUIREMENTS
                                                                                          7. CENTER REQUIREMENTS



                                                                                                                                                                Page 6
The Governance and Strategic
                 Management Handbook (NPD 1000.0A)

• NPD 1000.0A
    • sets forth NASA’s governance framework with which the Agency manages mission,
      roles, and responsibilities
• Governance principles include Checks and Balances:
    • Independent Life-cycle Review
    • Requirements tailoring
    • Dissenting Opinion process
• Benefits of successful Independent Life-cycle Reviews:
    • Agency receives independent assurance that they are on-track
    • NASA senior management receives:
        • Independent validation at key decision points of the Program/project’s readiness to proceed
          into the next phase of its life-cycle
        • Examination of externally-imposed impediments to Program/project success to be removed
    • Agency provides external stakeholders assurance we can deliver to our
      commitments
    • Preparation for a life-cycle review milestone allows for a holistic examination by the
      Program/project and the review team.
                                                                                                 Page 7
Change Areas for Independent
          Review




                               Page 8
New ToR Concept

• ToR and the SRB nominations will be combined into a single
  document
    – Intent is to streamline process for both
• ToR will no longer contain a long list of roles and
  responsibilities
    – Document will basically refer to 7120.5, 7123.1, and 1000.5 for
      guidance
• All appendices and plans for all ILCRs for a P/p will be
  completed up front
    – Schedules will be determined in terms of delta from a review rather
      than absolute dates
    – Once signed, ToR only needs to be addressed again for major
      changes to P/p or issues surrounding the review
•   A “3-step” timeline for delivery of programmatic product to
    support the reviews (refer to next page)

                                                                            Page 9
Programmatic Deliveries

•   Cost and schedule information in support of Independent Lifecycle
    Reviews (ILCRs) are delivered to the SRB in three progressive
    deliveries as outlined below
          Item                                 Content**                                    Timeline
      Data          Existing project management data including working technical      100 days prior to
      Delivery 1    baseline description; risk list/matrix; WBS, WBS dictionary;      LCR*
                    Master Equipment List; Power Equipment List; schedule;
                    planning budget by year and phase; workforce estimate; and
                    special facilities/resources required.



      Data          Preliminary delivery of data formally required for the review,    60 days prior to LCR*
      Delivery 2    including BOEs, a functional JCL model and supporting data (as
                    applicable), and available cost/schedule performance data


      Data          Final JCL results and/or budget (if no JCL) and supporting data   20 days prior to LCR*
      Delivery 3    and updated risk list matrix.


       * For two-step LCRs, the timeline is with respect to the second step of the LCR.
       ** The list of the programmatic cost and schedule data for each review is found in the Standing
       Review Board Handbook.


                                                                                                              Page 10
One Step PDR
                           Life Cycle Review Overview

                                                                                                                                  KDP-C
     KDP-B

                                                                                      (2)
                                                                      Quick Look Report


               PDR Readiness
                Assessment(2)                                   PDR-LCR
                                (30- 90 days)                                               (30 Days)

          -Required prior to LCR                    Technical Baseline with C/S/R
          -Report to DA for life cycle reviews      and Integrated Assessment of
           preceding KDP B&C and during any          Technical and Programmatic
           major replan or rebaseline (3)                      Baseline



Not To Scale

                                Programmatic Data Drops to
                                  SRB (includes JCL Model)                                   P/p         Center          MD
                                 Deliveries start at 100 days
                                      before site review
                                                                                             Brief        Brief          Brief

         (2)
CheckPoint if needed.                     Periodic SRB Involvement as Appropriate
                                                                                              FOOTNOTES:
                                                                                              1. A One Step Review may be used for any LCR
                                                                                              2. Appendix I provides information on the readiness
                                                                                              assessment, quick-look reports and checkpoints associated with
                                                                                              life cycle reviews
                                                                                              3.Version 0 - page 11 reviews report to Chief Engineer if
                                                                                                   For all other life cycle
                                                                                              significant unresolvable disagreements                 Page 11
                                                                                                                                                   Page 11
Two Step PDR
                              Life Cycle Review Overview
                                                                                                                                              KDP-
   KDP-B                                                          PDR LCR                                                                      C


                                                                                        (2)
                                                                      Quick Look Report                                       (2)
                                                                                                       Quick Look Report

               PDR Readiness
                                                                              Independent Integrated
                Assessment (2)                         PDR
                                                                                 PDR Assessment
                                 (30-90 days)                       (1-6 months)                                 (30 Days)


  -Required prior to LCR                        Technical Baseline                       Integrated
  -Report to DA for life cycle reviews              with Cost,                         Assessment of
   preceding KDP B&C and during any               Schedule, and                        Technical and
   major replan or rebaseline (3)                Risk Information                      Programmatic
                                                                                          Baseline

Not To Scale

                          Programmatic Data Drops to              Resolve Tech
                            SRB (includes JCL Model)          Issues/Risks, Update                     P/p         Center           MD
                                                             Cost/Schedule Baseline
                                                                                                       Brief        Brief           Brief

             (2)
 CheckPoint if needed.                            Periodic SRB Involvement as Appropriate
                                                                                                        FOOTNOTES:
                                                                                                        1. A Two Step Review may be used for any LCR
                                                                                                        2. Appendix I provides information on the readiness
                                                                                                        assessment, quick-look reports and checkpoints associated with
                                                                                                        life cycle reviews
                                                                                                        3. For all other life cycle reviews report to Chief Engineer if 12
                                                                                                         Version 0 - page 12                                      Page
                                                                                                        significant unresolvable disagreements                   Page 12
Lifecycle Products

Chapter 4 of the new version of 7120.5 goes to great length to
  describe the products expected during each phase of a flight
  project, the following text is pulled from the draft (August,
  2010) as an example:


4.5.1 Purpose: During Phase B, the project team completes its
   preliminary design and technology development and
   establishes the associated schedule and life cycle cost for the
   project.


4.5.2 Requirements: During Phase B, the project manager and the
   project team shall:
  -- Enumerates 32 specific items that must be addressed leading
  up to KDP-C, including detailed description of technical
  baseline and the cost and schedule estimates that match the
  design.
                                                                     Page 13
Updated LCR Criteria


• Alignment with and contribution to Agency strategic goals and
  adequacy of requirements flow down from those
   – Scope includes alignment of program/project requirements/designs with
     Agency strategic goals, program requirements, and constraints, mission
     needs and success criteria; allocation of program requirements to projects;
     and proactive management of changes in program/project scope and
     shortfalls.

• Adequacy of management approach
   – Scope includes program/project authorization, management framework and
     plans, acquisition strategies, and internal and external agreements.

• Adequacy of technical approach, as defined by NPR 7123.1
  entrance and success criteria
   – Scope includes flow down of project requirements to systems/subsystems;
     architecture and design; and operations concepts that respond to and
     satisfy imposed requirements and mission needs.


                                                                                   Page 14
Updated LCR Criteria


• Adequacy of the integrated cost and schedule estimate and
  funding strategy in accordance with NPD 1000.5
   – Scope includes cost and schedule control plans; cost and schedule
     baselines that are consistent with the program/project
     requirements, assumptions, risks, and margins; basis of estimate; JCL
     (when required); and alignment with planned budgets.

• Adequacy and availability of resources other than budget
   – Scope includes planning, availability, competency and stability of
     staffing, and infrastructure requirements.

• Adequacy of the risk management approach and risk
  identification and mitigation per NPR 8000.4
   – Scope includes risk management control plans, open and accepted
     risks, risk assessments, risk mitigation plans, and resources for
     managing/mitigating risks.



                                                                             Page 15
Maturity Tables
                                            (New Concept in 7120.5E)
                                                                                            Expected Maturity State by Review Element
             Associated                                                                                                                                                                       Overall Expected
KDP Review    Lifecycle     LCR Objectives         AgencyStrategic                                                                                                                             Maturity State
                                                                             Managemen t               Technical          Budget and        Resource Other
                                                                                                                                                    s
               Review                                 Goals &                                                                                                        Risk Management            @ Next KDP
                                                                              Approach                 Approach            Schedule          Than Budget
                                                     Outcomes
                          To evalua the
                                     te            The proposed           The Project FAD has      One or more          Credible risk-      Infrastrucureand
                                                                                                                                                      t              Driving risks an
                                                                                                                                                                                    d
                          feasibility o t
                                        f he       Projec has merit,is
                                                         t                been approved and        technicalconcepts    informed options    unique resource          mitigationoptions
                          proposed missi n o       withinthe              the management           that respond to      exist tha fit
                                                                                                                                 t          needs, such as           have been identified
                          concept and its          Agency/Program         framew is in
                                                                                  ork              missio needs are
                                                                                                          n             withindesired       special skills or rare   and are mana  geable;       Overall KDPA
                          fulfillmen of the
                                     t             scope, and initial     place;key interfaces     identifi d and
                                                                                                           e            schedule and        materials,have           the approach for         Expected Maturity:
                          Program’s needs and      objective and
                                                            s             and partnerships         appearfeasible.      available funding   been identified and      managing these risks      Projec addresses
                                                                                                                                                                                                     t
  KDP A        MCR        objectives to
                                      ;            requirements are       have been identified ;                        profile.            are likely available.    is adequate.             critical NASAneed
                          determine whether        appropriate.           and appropriate                                                                                                      and can likely be
                          the maturityof the                              plansfor Phase A are                                                                                                    achiev as
                                                                                                                                                                                                        ed
                          concept and                                     in place.                                                                                                                conceived.
                          associat d plannin
                                   e         g
                          are sufficient to
                          beginPhase A.
                          To evalua whether
                                     te            Preliminary            Preliminary Project      Functionaland        Credible            Preliminary staffing     Significantmission,
                          the functionaland        requirements           Plan is appropriately    perfo rmance         preliminary         and essential            technical,costand
                          perfo  rmance            incorporate            matur to support
                                                                                 e                 requir ements have   cost/sc hedule      infrastructure           schedule risks hav
                                                                                                                                                                                      e
                          requir  ements           program                conceptual desig n       been defined, and    estima are
                                                                                                                               tes          requirements have        been identified;
                          defined for the          requirements and       phase and                the requirements     supported by a      been identified and      viablemitigation
                          system are               constraint, and are
                                                            s             preliminary              will satisfy the     documented BOE      documented;              strategieshave been
                          responsive to the        responsive to          acquisitio strateg
                                                                                    n        y     mission.             and areconsistent   preliminary sources      defined; a
                SRR
                          Program’s                missio needs.
                                                         n                is defined.                                   w/ driving          have been                preliminaryproces s
                          requir  ements on the                                                                         assumptions,        identifi d.
                                                                                                                                                   e                 and resources exist
                          proje and
                                ct                                                                                      risks, syst m
                                                                                                                                  e                                  to effectivel
                                                                                                                                                                                 y
                                                                                                                                                                                                Overall KD BP
                          represent achievabl  e                                                                        requir ements,                               manag or mitigat
                                                                                                                                                                            e          e
                                                                                                                                                                                               Expected State:
                          capabilities.                                                                                 design options,                              them.
                                                                                                                                                                                              Proposed systems
                                                                                                                        and available
  KDP B                                                                                                                 fundingand
                                                                                                                                                                                              are feasiblewithin
                                                                                                                                                                                              available res urces
                                                                                                                                                                                                           o
                                                                                                                        schedule profile.
                                                                                                                                                                                               with acceptable
                          To evalua the
                                    te             Mission/System         Preliminary Project      Driving              Credible            Availability,            Significantmission ,            risk.
                          credibility and          requir  ements,        Plan isappropriately     requirements have    cost/sc hedule      competencyand            development, cost,
                          responsiveness of        design approaches,     matur to support
                                                                                 e                 been defined, and    estima are
                                                                                                                               tes          Stability of staffing
                                                                                                                                                                ,    schedule and safety
                          the proposed             and conceptual         preliminarydesig n       credibl system
                                                                                                          e             supported by a      essential                risks are id
                                                                                                                                                                                entified
                          mission/system           design incorporate     phase, technology        architectu and
                                                                                                            res         documented BOE      infrastructur and
                                                                                                                                                          e          and asses sed;
               MDR        architectu to the
                                     re            program                development plans        operatingconcept s   and areconsistent   additi nal
                                                                                                                                                  o                  mitigationplans hav  e
                          program                  requir  ements and     are adequate, and        respond to them.     with driving        resour are
                                                                                                                                                   ces               been defined; a
                          requirements and         constraint, and will
                                                               s          acquisitio strateg
                                                                                     n      y                           assumptions,        adequate for             processand
                          constrain including
                                    ts,            fulfill th missi n
                                                             e     o      is approved and                               risks, sysem
                                                                                                                                 t          remaining lifeycle
                                                                                                                                                           c         resour exis to
                                                                                                                                                                            ces     t
                          available resources      needs and missio  n    initiated.                                    requir ements,      phases.                  effectively manage
                          to determine.            succes criteria.
                                                           s                                                            conceptual design                            or mitigat them.
                                                                                                                                                                               e



                                                                                                                                                                                                        Page 16
SRB Composition and Balance


• SRB teams need to be balanced along three dimensions:
   – Competency
   – Currency (includes current knowledge of Agency PM policies)
   – Independence

• In addition, SRBs need to meet the independent assessment
  needs of the following:
   – Center (including TA responsibilities)
   – Mission Directorate (expected technical, cost and schedule performance)
   – Agency (overall readiness to proceed)

• The SRB team balance fits the particular situation of each
  project and thus is not prescribed.

• SRB team balance is arrived via a collaborative, iterative;
  nomination and approval process (see next chart)

                                                                               Page 17
SRB Nomination Procedures

• SRB Chair and Review Manager (RM) are nominated first
   – Program SRBs: Chair nominations initiated by the Mission Directorate
   – Project SRBs: Chair nominations initiated by the Center
   – Review Manager nominations initiated by IPAO
   – Nominations are socialized with all affected parties including the Program
     and Projects
• SRB Chair and RM nomination concurred by the Convening
  Authorities (CAs) and approved by the Decision Authority (DA)
• SRB Team
   – Chair and RM are responsible for facilitating the nomination and approval
     of SRB team members working closely with the Centers and the Mission
     Directorates
   – Nominated SRB team members have to be justified for appropriate balance
     of currency, competency and independence
   – Nominated team members have to be vetted for Organizational and
     Personal Conflict of Interests (OCI/PCI) by the Legal and Procurement
     Offices
   – SRB teams are concurred by the CAs and approved by the DA.
                                                                                  Page 18
Convening Authorities




          Table 2-5 NASA Convening Authorities for Standing Review Board




                              Decision Authority      Technical Authority*          Associate
                               NASA                                Center         Administrator,
                                AA       MDAA      NASA CE       Director***          IPCE
Establish SRB,   Programs     Approve   Approve     Approve       Approve            Approve
Approve ToR.
                 Category     Approve   Approve     Concur         Approve           Approve
Approve
                 1 Projects
Chairperson,
RM, and Other    Category               Approve                    Approve          Approve**
Board Members    2 Projects
                 Category               Approve                    Approve
                 3 Projects




           Center Directors are now a Convening Authority at the Program Level.




                                                                                                   Page 19
Readiness to Proceed Meeting


• Present NASA Policy requires the SRB Chair to meet with the
  P/p manager prior to any Site Visit to determine the projects
  readiness to proceed to the review
• This meeting should generally have the following
  characteristics:
   – It is an informal dialogue between the parties not a “review”
   – Intent is to determine if the project will be ready at the time of the review,
     not at the time of the meeting
   – Deliverables are discussed in terms of the project’s progress towards
     completion
   – Programmatic data delivery should be discussed
   – Result of meeting is an email to the Director of IPAO, who then forwards
     comments to OCE and the Convening Authorities
• It should be held 30-90 days days prior to the planned review
  date to allow for course correction as need be

                                                                                      Page 20
Decision Memos

• Associate Administrator request that Program internal and external
  commitment decisions made by GPMCs (A/DPMCs) be documented
  in real time at the governing board

• Have a record so that all involved parties have a common
  understanding of project decisions and commitments vs.
  historically having varying recall of commitments

• Used to support external reporting requirements

• Establish a process whereby management commits to the Decision
  Authority decisions from the GPMC

• Quickly document the GPMC Decision Authority decisions at the
  conclusion of GPMCs or soon thereafter

                                                                Page 21
Changes to SRB-Chaired LCRs

•   Participate-In and/or Conduct clarified One- or Two-Step Reviews
     –   Change - PLARs and CERRs are not required to be conducted by the SRB unless requested in
         the project plan

•   NEW – Perform LCR Readiness Assessment
     –   Use clarified LCR maturity state, i.e., entrance criteria

•   Use clarified LCR requirements by phase, i.e., success criteria, in the conduct
    of reviews.
•   Deliver clarified Quick-Look Reports
•   NEW – Conduct Checkpoints between ILCR and KDP as necessary
•   Assess Directorate, Program or Project developed confidence level
    estimating & budgeting (as opposed to performing “antagonistic”
    independent estimates). May use adjusted P/p risks or SRB unique risks.
     –   Phase B - range of cost & schedule estimates w/ associated confidence levels
     –   Phase C/D - 70 percent joint cost and schedule confidence level (JCL)

•   Provide specific inputs to KDP Decision Memorandum that may include
    Management Agreement and Agency Plan/Agency Baseline Commitment
•   NEW – Use “Terms of Reference” Template
                                                                                             Page 22
Summary


•   The independent life cycle review process is an integral part of the
    Agency’s check and balances built into the NASA governance
    structure and complements the programmatic and technical lines of
    command and authority.
•   The independent lifecycle review process is encoded as part of NASA
    policy direction; its requirements are stipulated in policy requirements;
    and guidance to reviews team and implementing personnel is provided
    in handbooks and operating procedures.
•   Its processes are continuously assessed for improvement by IPAO and
    its stakeholders. Changes are incorporated in policy and procedure.
•   The independent lifecycle review process helps ensure the highest
    probability of success of the Agency’s program and projects.




                                                                            Page 23

Contenu connexe

Tendances

Borchardt.poole.majerowicz
Borchardt.poole.majerowiczBorchardt.poole.majerowicz
Borchardt.poole.majerowiczNASAPMC
 
Smalley stigberg petze
Smalley stigberg petzeSmalley stigberg petze
Smalley stigberg petzeNASAPMC
 
Bauer.frank
Bauer.frankBauer.frank
Bauer.frankNASAPMC
 
Daisy.mueller
Daisy.muellerDaisy.mueller
Daisy.muellerNASAPMC
 
Bilardo.vince
Bilardo.vinceBilardo.vince
Bilardo.vinceNASAPMC
 
Corcoran webster
Corcoran websterCorcoran webster
Corcoran websterNASAPMC
 
Ray hines stegeman
Ray hines stegemanRay hines stegeman
Ray hines stegemanNASAPMC
 
Tim.honeycutt
Tim.honeycuttTim.honeycutt
Tim.honeycuttNASAPMC
 
Carmichael.kevin
Carmichael.kevinCarmichael.kevin
Carmichael.kevinNASAPMC
 
Fujieh.maura
Fujieh.mauraFujieh.maura
Fujieh.mauraNASAPMC
 
Ortiz.james
Ortiz.jamesOrtiz.james
Ortiz.jamesNASAPMC
 
Dennehy richard mrozinski
Dennehy richard mrozinskiDennehy richard mrozinski
Dennehy richard mrozinskiNASAPMC
 
William.tippin.update
William.tippin.updateWilliam.tippin.update
William.tippin.updateNASAPMC
 
Les.sorge
Les.sorgeLes.sorge
Les.sorgeNASAPMC
 
Ortiz.james
Ortiz.jamesOrtiz.james
Ortiz.jamesNASAPMC
 
Fayssal.safie
Fayssal.safieFayssal.safie
Fayssal.safieNASAPMC
 
Grammier.richard
Grammier.richardGrammier.richard
Grammier.richardNASAPMC
 
Bladwin.kristen
Bladwin.kristenBladwin.kristen
Bladwin.kristenNASAPMC
 
Backup darren elliott
Backup darren elliottBackup darren elliott
Backup darren elliottNASAPMC
 
Brantley.wayne
Brantley.wayneBrantley.wayne
Brantley.wayneNASAPMC
 

Tendances (20)

Borchardt.poole.majerowicz
Borchardt.poole.majerowiczBorchardt.poole.majerowicz
Borchardt.poole.majerowicz
 
Smalley stigberg petze
Smalley stigberg petzeSmalley stigberg petze
Smalley stigberg petze
 
Bauer.frank
Bauer.frankBauer.frank
Bauer.frank
 
Daisy.mueller
Daisy.muellerDaisy.mueller
Daisy.mueller
 
Bilardo.vince
Bilardo.vinceBilardo.vince
Bilardo.vince
 
Corcoran webster
Corcoran websterCorcoran webster
Corcoran webster
 
Ray hines stegeman
Ray hines stegemanRay hines stegeman
Ray hines stegeman
 
Tim.honeycutt
Tim.honeycuttTim.honeycutt
Tim.honeycutt
 
Carmichael.kevin
Carmichael.kevinCarmichael.kevin
Carmichael.kevin
 
Fujieh.maura
Fujieh.mauraFujieh.maura
Fujieh.maura
 
Ortiz.james
Ortiz.jamesOrtiz.james
Ortiz.james
 
Dennehy richard mrozinski
Dennehy richard mrozinskiDennehy richard mrozinski
Dennehy richard mrozinski
 
William.tippin.update
William.tippin.updateWilliam.tippin.update
William.tippin.update
 
Les.sorge
Les.sorgeLes.sorge
Les.sorge
 
Ortiz.james
Ortiz.jamesOrtiz.james
Ortiz.james
 
Fayssal.safie
Fayssal.safieFayssal.safie
Fayssal.safie
 
Grammier.richard
Grammier.richardGrammier.richard
Grammier.richard
 
Bladwin.kristen
Bladwin.kristenBladwin.kristen
Bladwin.kristen
 
Backup darren elliott
Backup darren elliottBackup darren elliott
Backup darren elliott
 
Brantley.wayne
Brantley.wayneBrantley.wayne
Brantley.wayne
 

En vedette

Dillon.r.tinsley.c.rogers.e
Dillon.r.tinsley.c.rogers.eDillon.r.tinsley.c.rogers.e
Dillon.r.tinsley.c.rogers.eNASAPMC
 
Stacy.cusack
Stacy.cusackStacy.cusack
Stacy.cusackNASAPMC
 
Debbie.dusterwald
Debbie.dusterwaldDebbie.dusterwald
Debbie.dusterwaldNASAPMC
 
Coonce.tom
Coonce.tomCoonce.tom
Coonce.tomNASAPMC
 
Grubbs teams and digital collaboration pmc2012
Grubbs teams and digital collaboration pmc2012Grubbs teams and digital collaboration pmc2012
Grubbs teams and digital collaboration pmc2012NASAPMC
 
John.emond
John.emondJohn.emond
John.emondNASAPMC
 
Serrato.be be
Serrato.be beSerrato.be be
Serrato.be beNASAPMC
 
Rippe.carpio
Rippe.carpioRippe.carpio
Rippe.carpioNASAPMC
 
Neiberding
NeiberdingNeiberding
NeiberdingNASAPMC
 
Diffoot fayful
Diffoot fayfulDiffoot fayful
Diffoot fayfulNASAPMC
 
John.olson
John.olsonJohn.olson
John.olsonNASAPMC
 
Harrison.g.poole.k
Harrison.g.poole.kHarrison.g.poole.k
Harrison.g.poole.kNASAPMC
 
Kirsch.mike
Kirsch.mikeKirsch.mike
Kirsch.mikeNASAPMC
 
Brockhurst.lane
Brockhurst.laneBrockhurst.lane
Brockhurst.laneNASAPMC
 
Backup norm.smith
Backup norm.smithBackup norm.smith
Backup norm.smithNASAPMC
 
M washington
M washingtonM washington
M washingtonNASAPMC
 
K.pagel.beene
K.pagel.beeneK.pagel.beene
K.pagel.beeneNASAPMC
 
C null 01-17-2011
C null 01-17-2011C null 01-17-2011
C null 01-17-2011NASAPMC
 
B ackroyd
B ackroydB ackroyd
B ackroydNASAPMC
 

En vedette (20)

Dillon.r.tinsley.c.rogers.e
Dillon.r.tinsley.c.rogers.eDillon.r.tinsley.c.rogers.e
Dillon.r.tinsley.c.rogers.e
 
Stacy.cusack
Stacy.cusackStacy.cusack
Stacy.cusack
 
Debbie.dusterwald
Debbie.dusterwaldDebbie.dusterwald
Debbie.dusterwald
 
Coonce.tom
Coonce.tomCoonce.tom
Coonce.tom
 
Grubbs teams and digital collaboration pmc2012
Grubbs teams and digital collaboration pmc2012Grubbs teams and digital collaboration pmc2012
Grubbs teams and digital collaboration pmc2012
 
John.emond
John.emondJohn.emond
John.emond
 
Serrato.be be
Serrato.be beSerrato.be be
Serrato.be be
 
Rippe.carpio
Rippe.carpioRippe.carpio
Rippe.carpio
 
Neiberding
NeiberdingNeiberding
Neiberding
 
Diffoot fayful
Diffoot fayfulDiffoot fayful
Diffoot fayful
 
John.olson
John.olsonJohn.olson
John.olson
 
Harrison.g.poole.k
Harrison.g.poole.kHarrison.g.poole.k
Harrison.g.poole.k
 
Kirsch.mike
Kirsch.mikeKirsch.mike
Kirsch.mike
 
Brockhurst.lane
Brockhurst.laneBrockhurst.lane
Brockhurst.lane
 
Backup norm.smith
Backup norm.smithBackup norm.smith
Backup norm.smith
 
M washington
M washingtonM washington
M washington
 
Heard
HeardHeard
Heard
 
K.pagel.beene
K.pagel.beeneK.pagel.beene
K.pagel.beene
 
C null 01-17-2011
C null 01-17-2011C null 01-17-2011
C null 01-17-2011
 
B ackroyd
B ackroydB ackroyd
B ackroyd
 

Similaire à Ortiz

Ortiz.james
Ortiz.jamesOrtiz.james
Ortiz.jamesNASAPMC
 
Dezfuli.homayoon
Dezfuli.homayoonDezfuli.homayoon
Dezfuli.homayoonNASAPMC
 
Reliability Through Correlation and Colaboration
Reliability Through Correlation and ColaborationReliability Through Correlation and Colaboration
Reliability Through Correlation and ColaborationChad Broussard
 
Healy edward
Healy edwardHealy edward
Healy edwardNASAPMC
 
HITSC 2010 06-30 slides
HITSC 2010 06-30 slidesHITSC 2010 06-30 slides
HITSC 2010 06-30 slidesBrian Ahier
 
Npr 7120.5 nasa space flight program and project management handbook
Npr 7120.5   nasa space flight program and project management handbookNpr 7120.5   nasa space flight program and project management handbook
Npr 7120.5 nasa space flight program and project management handbookssuser10cb48
 
Blythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 finalBlythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 finalNASAPMC
 
Pharmacovigilance Surge Resource Calculator
Pharmacovigilance Surge Resource CalculatorPharmacovigilance Surge Resource Calculator
Pharmacovigilance Surge Resource CalculatorTimothy Roe
 
Kelly crumbley
Kelly crumbleyKelly crumbley
Kelly crumbleyNASAPMC
 
Calloway amer v2
Calloway amer v2Calloway amer v2
Calloway amer v2NASAPMC
 
Calloway amer v2
Calloway amer v2Calloway amer v2
Calloway amer v2NASAPMC
 
Blythe.mike
Blythe.mikeBlythe.mike
Blythe.mikeNASAPMC
 
Hihn.jarius
Hihn.jariusHihn.jarius
Hihn.jariusNASAPMC
 
Keer.beth
Keer.bethKeer.beth
Keer.bethNASAPMC
 
Apcpdcl
ApcpdclApcpdcl
ApcpdclJehara
 
Sandra.smalley
Sandra.smalleySandra.smalley
Sandra.smalleyNASAPMC
 
Standards based interoperability g gorbach arc orlando 2008
Standards based interoperability g gorbach arc orlando 2008Standards based interoperability g gorbach arc orlando 2008
Standards based interoperability g gorbach arc orlando 2008ARC Advisory Group
 
Amer.tahani
Amer.tahaniAmer.tahani
Amer.tahaniNASAPMC
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsenNASAPMC
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsenNASAPMC
 

Similaire à Ortiz (20)

Ortiz.james
Ortiz.jamesOrtiz.james
Ortiz.james
 
Dezfuli.homayoon
Dezfuli.homayoonDezfuli.homayoon
Dezfuli.homayoon
 
Reliability Through Correlation and Colaboration
Reliability Through Correlation and ColaborationReliability Through Correlation and Colaboration
Reliability Through Correlation and Colaboration
 
Healy edward
Healy edwardHealy edward
Healy edward
 
HITSC 2010 06-30 slides
HITSC 2010 06-30 slidesHITSC 2010 06-30 slides
HITSC 2010 06-30 slides
 
Npr 7120.5 nasa space flight program and project management handbook
Npr 7120.5   nasa space flight program and project management handbookNpr 7120.5   nasa space flight program and project management handbook
Npr 7120.5 nasa space flight program and project management handbook
 
Blythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 finalBlythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 final
 
Pharmacovigilance Surge Resource Calculator
Pharmacovigilance Surge Resource CalculatorPharmacovigilance Surge Resource Calculator
Pharmacovigilance Surge Resource Calculator
 
Kelly crumbley
Kelly crumbleyKelly crumbley
Kelly crumbley
 
Calloway amer v2
Calloway amer v2Calloway amer v2
Calloway amer v2
 
Calloway amer v2
Calloway amer v2Calloway amer v2
Calloway amer v2
 
Blythe.mike
Blythe.mikeBlythe.mike
Blythe.mike
 
Hihn.jarius
Hihn.jariusHihn.jarius
Hihn.jarius
 
Keer.beth
Keer.bethKeer.beth
Keer.beth
 
Apcpdcl
ApcpdclApcpdcl
Apcpdcl
 
Sandra.smalley
Sandra.smalleySandra.smalley
Sandra.smalley
 
Standards based interoperability g gorbach arc orlando 2008
Standards based interoperability g gorbach arc orlando 2008Standards based interoperability g gorbach arc orlando 2008
Standards based interoperability g gorbach arc orlando 2008
 
Amer.tahani
Amer.tahaniAmer.tahani
Amer.tahani
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsen
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsen
 

Plus de NASAPMC

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk boNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
Yew manson
Yew mansonYew manson
Yew mansonNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)NASAPMC
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joeNASAPMC
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuartNASAPMC
 
Stock gahm
Stock gahmStock gahm
Stock gahmNASAPMC
 
Snow lee
Snow leeSnow lee
Snow leeNASAPMC
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandraNASAPMC
 
Seftas krage
Seftas krageSeftas krage
Seftas krageNASAPMC
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marcoNASAPMC
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mikeNASAPMC
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karleneNASAPMC
 
Rackley mike
Rackley mikeRackley mike
Rackley mikeNASAPMC
 
Paradis william
Paradis williamParadis william
Paradis williamNASAPMC
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeffNASAPMC
 
O'keefe william
O'keefe williamO'keefe william
O'keefe williamNASAPMC
 
Muller ralf
Muller ralfMuller ralf
Muller ralfNASAPMC
 

Plus de NASAPMC (20)

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk bo
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
Yew manson
Yew mansonYew manson
Yew manson
 
Wood frank
Wood frankWood frank
Wood frank
 
Wood frank
Wood frankWood frank
Wood frank
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joe
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuart
 
Stock gahm
Stock gahmStock gahm
Stock gahm
 
Snow lee
Snow leeSnow lee
Snow lee
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandra
 
Seftas krage
Seftas krageSeftas krage
Seftas krage
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marco
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mike
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karlene
 
Rackley mike
Rackley mikeRackley mike
Rackley mike
 
Paradis william
Paradis williamParadis william
Paradis william
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeff
 
O'keefe william
O'keefe williamO'keefe william
O'keefe william
 
Muller ralf
Muller ralfMuller ralf
Muller ralf
 

Dernier

Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 

Dernier (20)

Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 

Ortiz

  • 1. Review Policy Evolution Current and upcoming upgrades to independent review policies and processes Dr. James Ortiz, Director Independent Program Assessment Office Eight Annual NASA Program Management Challenge 2011 February 9, 2011 Page 1
  • 2. Introduction • The Independent Program Assessment Office (IPAO) manages the independent review of the Agency's Programs and projects at life-cycle milestones to ensure the highest probability of mission success. • This presentation provides an overview of the Agency’s policy changes to improve the effectiveness and efficiency of the Agency Review Processes. • The bulk of these changes are being considered as part of the upcoming revision to NPR 7120.5 (the “E” version). Page 2
  • 3. Outline • Agency policy context • Change Areas – Standard TOR – Lifecycle Review Timelines (1-step; 2-step) – Lifecycle Products – Review Criteria – Maturity Tables – Team Composition and Balance – Convening Authorities – Readiness Assessment – Decision Memos – Changes to Lifecycle Reviews • Summary Page 3
  • 4. Agency Policy Context for Independent Review Page 4
  • 5. Independent Review Policy Context • Agency policy context – The “Why” » NPD 1000.0; NPD 1000.5 – The “What” » NPR 7120.5; NPR 7123.1 – The “How” » SRB Handbook » IPAO System Operating Procedures Page 5
  • 6. Policy Flow-down NPD 1000.0 NPD 1000.3 NPD 1000.5 NPD 7120.4 NPD 8700.1 NPD 8900.5A Mission Support Office NPR 7120.5D Support Organization NPR 7123.1A OSMA Directives OCHMO Directives and NID Directives 1 2 3 4 5 NASA -SP 2009 -10 -015 - HQ Center Engineering Mission Directorate NASA -SP -6105 SRB HANDBOOK Requirements Requirements SYSTEMS ENGINEERING HANDBOOK 7 6 IPAO 1000 IPAO MANAGEMENT PLAN LEGEND: DIRECT 1: ENGINEERING REQUIREMENTS 2: PROGRAM/PROJECT IPAO 2000 MANAGEMENT REQUIREMENTS STANDARD OPERATING INDIRECT 3. SAFETY AND MISSION PROCEDURE ASSURANCE REQUIREMENTS 4. HEALTH AND MEDICAL REQTS 5. MSO FUNCTIONAL REQTS 6. MD REQUIREMENTS 7. CENTER REQUIREMENTS Page 6
  • 7. The Governance and Strategic Management Handbook (NPD 1000.0A) • NPD 1000.0A • sets forth NASA’s governance framework with which the Agency manages mission, roles, and responsibilities • Governance principles include Checks and Balances: • Independent Life-cycle Review • Requirements tailoring • Dissenting Opinion process • Benefits of successful Independent Life-cycle Reviews: • Agency receives independent assurance that they are on-track • NASA senior management receives: • Independent validation at key decision points of the Program/project’s readiness to proceed into the next phase of its life-cycle • Examination of externally-imposed impediments to Program/project success to be removed • Agency provides external stakeholders assurance we can deliver to our commitments • Preparation for a life-cycle review milestone allows for a holistic examination by the Program/project and the review team. Page 7
  • 8. Change Areas for Independent Review Page 8
  • 9. New ToR Concept • ToR and the SRB nominations will be combined into a single document – Intent is to streamline process for both • ToR will no longer contain a long list of roles and responsibilities – Document will basically refer to 7120.5, 7123.1, and 1000.5 for guidance • All appendices and plans for all ILCRs for a P/p will be completed up front – Schedules will be determined in terms of delta from a review rather than absolute dates – Once signed, ToR only needs to be addressed again for major changes to P/p or issues surrounding the review • A “3-step” timeline for delivery of programmatic product to support the reviews (refer to next page) Page 9
  • 10. Programmatic Deliveries • Cost and schedule information in support of Independent Lifecycle Reviews (ILCRs) are delivered to the SRB in three progressive deliveries as outlined below Item Content** Timeline Data Existing project management data including working technical 100 days prior to Delivery 1 baseline description; risk list/matrix; WBS, WBS dictionary; LCR* Master Equipment List; Power Equipment List; schedule; planning budget by year and phase; workforce estimate; and special facilities/resources required. Data Preliminary delivery of data formally required for the review, 60 days prior to LCR* Delivery 2 including BOEs, a functional JCL model and supporting data (as applicable), and available cost/schedule performance data Data Final JCL results and/or budget (if no JCL) and supporting data 20 days prior to LCR* Delivery 3 and updated risk list matrix. * For two-step LCRs, the timeline is with respect to the second step of the LCR. ** The list of the programmatic cost and schedule data for each review is found in the Standing Review Board Handbook. Page 10
  • 11. One Step PDR Life Cycle Review Overview KDP-C KDP-B (2) Quick Look Report PDR Readiness Assessment(2) PDR-LCR (30- 90 days) (30 Days) -Required prior to LCR Technical Baseline with C/S/R -Report to DA for life cycle reviews and Integrated Assessment of preceding KDP B&C and during any Technical and Programmatic major replan or rebaseline (3) Baseline Not To Scale Programmatic Data Drops to SRB (includes JCL Model) P/p Center MD Deliveries start at 100 days before site review Brief Brief Brief (2) CheckPoint if needed. Periodic SRB Involvement as Appropriate FOOTNOTES: 1. A One Step Review may be used for any LCR 2. Appendix I provides information on the readiness assessment, quick-look reports and checkpoints associated with life cycle reviews 3.Version 0 - page 11 reviews report to Chief Engineer if For all other life cycle significant unresolvable disagreements Page 11 Page 11
  • 12. Two Step PDR Life Cycle Review Overview KDP- KDP-B PDR LCR C (2) Quick Look Report (2) Quick Look Report PDR Readiness Independent Integrated Assessment (2) PDR PDR Assessment (30-90 days) (1-6 months) (30 Days) -Required prior to LCR Technical Baseline Integrated -Report to DA for life cycle reviews with Cost, Assessment of preceding KDP B&C and during any Schedule, and Technical and major replan or rebaseline (3) Risk Information Programmatic Baseline Not To Scale Programmatic Data Drops to Resolve Tech SRB (includes JCL Model) Issues/Risks, Update P/p Center MD Cost/Schedule Baseline Brief Brief Brief (2) CheckPoint if needed. Periodic SRB Involvement as Appropriate FOOTNOTES: 1. A Two Step Review may be used for any LCR 2. Appendix I provides information on the readiness assessment, quick-look reports and checkpoints associated with life cycle reviews 3. For all other life cycle reviews report to Chief Engineer if 12 Version 0 - page 12 Page significant unresolvable disagreements Page 12
  • 13. Lifecycle Products Chapter 4 of the new version of 7120.5 goes to great length to describe the products expected during each phase of a flight project, the following text is pulled from the draft (August, 2010) as an example: 4.5.1 Purpose: During Phase B, the project team completes its preliminary design and technology development and establishes the associated schedule and life cycle cost for the project. 4.5.2 Requirements: During Phase B, the project manager and the project team shall: -- Enumerates 32 specific items that must be addressed leading up to KDP-C, including detailed description of technical baseline and the cost and schedule estimates that match the design. Page 13
  • 14. Updated LCR Criteria • Alignment with and contribution to Agency strategic goals and adequacy of requirements flow down from those – Scope includes alignment of program/project requirements/designs with Agency strategic goals, program requirements, and constraints, mission needs and success criteria; allocation of program requirements to projects; and proactive management of changes in program/project scope and shortfalls. • Adequacy of management approach – Scope includes program/project authorization, management framework and plans, acquisition strategies, and internal and external agreements. • Adequacy of technical approach, as defined by NPR 7123.1 entrance and success criteria – Scope includes flow down of project requirements to systems/subsystems; architecture and design; and operations concepts that respond to and satisfy imposed requirements and mission needs. Page 14
  • 15. Updated LCR Criteria • Adequacy of the integrated cost and schedule estimate and funding strategy in accordance with NPD 1000.5 – Scope includes cost and schedule control plans; cost and schedule baselines that are consistent with the program/project requirements, assumptions, risks, and margins; basis of estimate; JCL (when required); and alignment with planned budgets. • Adequacy and availability of resources other than budget – Scope includes planning, availability, competency and stability of staffing, and infrastructure requirements. • Adequacy of the risk management approach and risk identification and mitigation per NPR 8000.4 – Scope includes risk management control plans, open and accepted risks, risk assessments, risk mitigation plans, and resources for managing/mitigating risks. Page 15
  • 16. Maturity Tables (New Concept in 7120.5E) Expected Maturity State by Review Element Associated Overall Expected KDP Review Lifecycle LCR Objectives AgencyStrategic Maturity State Managemen t Technical Budget and Resource Other s Review Goals & Risk Management @ Next KDP Approach Approach Schedule Than Budget Outcomes To evalua the te The proposed The Project FAD has One or more Credible risk- Infrastrucureand t Driving risks an d feasibility o t f he Projec has merit,is t been approved and technicalconcepts informed options unique resource mitigationoptions proposed missi n o withinthe the management that respond to exist tha fit t needs, such as have been identified concept and its Agency/Program framew is in ork missio needs are n withindesired special skills or rare and are mana geable; Overall KDPA fulfillmen of the t scope, and initial place;key interfaces identifi d and e schedule and materials,have the approach for Expected Maturity: Program’s needs and objective and s and partnerships appearfeasible. available funding been identified and managing these risks Projec addresses t KDP A MCR objectives to ; requirements are have been identified ; profile. are likely available. is adequate. critical NASAneed determine whether appropriate. and appropriate and can likely be the maturityof the plansfor Phase A are achiev as ed concept and in place. conceived. associat d plannin e g are sufficient to beginPhase A. To evalua whether te Preliminary Preliminary Project Functionaland Credible Preliminary staffing Significantmission, the functionaland requirements Plan is appropriately perfo rmance preliminary and essential technical,costand perfo rmance incorporate matur to support e requir ements have cost/sc hedule infrastructure schedule risks hav e requir ements program conceptual desig n been defined, and estima are tes requirements have been identified; defined for the requirements and phase and the requirements supported by a been identified and viablemitigation system are constraint, and are s preliminary will satisfy the documented BOE documented; strategieshave been responsive to the responsive to acquisitio strateg n y mission. and areconsistent preliminary sources defined; a SRR Program’s missio needs. n is defined. w/ driving have been preliminaryproces s requir ements on the assumptions, identifi d. e and resources exist proje and ct risks, syst m e to effectivel y Overall KD BP represent achievabl e requir ements, manag or mitigat e e Expected State: capabilities. design options, them. Proposed systems and available KDP B fundingand are feasiblewithin available res urces o schedule profile. with acceptable To evalua the te Mission/System Preliminary Project Driving Credible Availability, Significantmission , risk. credibility and requir ements, Plan isappropriately requirements have cost/sc hedule competencyand development, cost, responsiveness of design approaches, matur to support e been defined, and estima are tes Stability of staffing , schedule and safety the proposed and conceptual preliminarydesig n credibl system e supported by a essential risks are id entified mission/system design incorporate phase, technology architectu and res documented BOE infrastructur and e and asses sed; MDR architectu to the re program development plans operatingconcept s and areconsistent additi nal o mitigationplans hav e program requir ements and are adequate, and respond to them. with driving resour are ces been defined; a requirements and constraint, and will s acquisitio strateg n y assumptions, adequate for processand constrain including ts, fulfill th missi n e o is approved and risks, sysem t remaining lifeycle c resour exis to ces t available resources needs and missio n initiated. requir ements, phases. effectively manage to determine. succes criteria. s conceptual design or mitigat them. e Page 16
  • 17. SRB Composition and Balance • SRB teams need to be balanced along three dimensions: – Competency – Currency (includes current knowledge of Agency PM policies) – Independence • In addition, SRBs need to meet the independent assessment needs of the following: – Center (including TA responsibilities) – Mission Directorate (expected technical, cost and schedule performance) – Agency (overall readiness to proceed) • The SRB team balance fits the particular situation of each project and thus is not prescribed. • SRB team balance is arrived via a collaborative, iterative; nomination and approval process (see next chart) Page 17
  • 18. SRB Nomination Procedures • SRB Chair and Review Manager (RM) are nominated first – Program SRBs: Chair nominations initiated by the Mission Directorate – Project SRBs: Chair nominations initiated by the Center – Review Manager nominations initiated by IPAO – Nominations are socialized with all affected parties including the Program and Projects • SRB Chair and RM nomination concurred by the Convening Authorities (CAs) and approved by the Decision Authority (DA) • SRB Team – Chair and RM are responsible for facilitating the nomination and approval of SRB team members working closely with the Centers and the Mission Directorates – Nominated SRB team members have to be justified for appropriate balance of currency, competency and independence – Nominated team members have to be vetted for Organizational and Personal Conflict of Interests (OCI/PCI) by the Legal and Procurement Offices – SRB teams are concurred by the CAs and approved by the DA. Page 18
  • 19. Convening Authorities Table 2-5 NASA Convening Authorities for Standing Review Board Decision Authority Technical Authority* Associate NASA Center Administrator, AA MDAA NASA CE Director*** IPCE Establish SRB, Programs Approve Approve Approve Approve Approve Approve ToR. Category Approve Approve Concur Approve Approve Approve 1 Projects Chairperson, RM, and Other Category Approve Approve Approve** Board Members 2 Projects Category Approve Approve 3 Projects Center Directors are now a Convening Authority at the Program Level. Page 19
  • 20. Readiness to Proceed Meeting • Present NASA Policy requires the SRB Chair to meet with the P/p manager prior to any Site Visit to determine the projects readiness to proceed to the review • This meeting should generally have the following characteristics: – It is an informal dialogue between the parties not a “review” – Intent is to determine if the project will be ready at the time of the review, not at the time of the meeting – Deliverables are discussed in terms of the project’s progress towards completion – Programmatic data delivery should be discussed – Result of meeting is an email to the Director of IPAO, who then forwards comments to OCE and the Convening Authorities • It should be held 30-90 days days prior to the planned review date to allow for course correction as need be Page 20
  • 21. Decision Memos • Associate Administrator request that Program internal and external commitment decisions made by GPMCs (A/DPMCs) be documented in real time at the governing board • Have a record so that all involved parties have a common understanding of project decisions and commitments vs. historically having varying recall of commitments • Used to support external reporting requirements • Establish a process whereby management commits to the Decision Authority decisions from the GPMC • Quickly document the GPMC Decision Authority decisions at the conclusion of GPMCs or soon thereafter Page 21
  • 22. Changes to SRB-Chaired LCRs • Participate-In and/or Conduct clarified One- or Two-Step Reviews – Change - PLARs and CERRs are not required to be conducted by the SRB unless requested in the project plan • NEW – Perform LCR Readiness Assessment – Use clarified LCR maturity state, i.e., entrance criteria • Use clarified LCR requirements by phase, i.e., success criteria, in the conduct of reviews. • Deliver clarified Quick-Look Reports • NEW – Conduct Checkpoints between ILCR and KDP as necessary • Assess Directorate, Program or Project developed confidence level estimating & budgeting (as opposed to performing “antagonistic” independent estimates). May use adjusted P/p risks or SRB unique risks. – Phase B - range of cost & schedule estimates w/ associated confidence levels – Phase C/D - 70 percent joint cost and schedule confidence level (JCL) • Provide specific inputs to KDP Decision Memorandum that may include Management Agreement and Agency Plan/Agency Baseline Commitment • NEW – Use “Terms of Reference” Template Page 22
  • 23. Summary • The independent life cycle review process is an integral part of the Agency’s check and balances built into the NASA governance structure and complements the programmatic and technical lines of command and authority. • The independent lifecycle review process is encoded as part of NASA policy direction; its requirements are stipulated in policy requirements; and guidance to reviews team and implementing personnel is provided in handbooks and operating procedures. • Its processes are continuously assessed for improvement by IPAO and its stakeholders. Changes are incorporated in policy and procedure. • The independent lifecycle review process helps ensure the highest probability of success of the Agency’s program and projects. Page 23