Hundreds of DoD contractors and commercial organizations have adopted the Capability Maturity Model Integrated (CMMI). Their published results show improved cost and schedule performance. Despite these results, there is still community debate over the value of CMMI, and whether CMMI ratings provide sufficient guarantees of program performance. Much of the confusion results from: (1) inaccurate CMMI ratings; (2) over-estimating the benefits that CMMI provides a customer; and (3) contractors not living up to their CMMI rating. This program will explore each of these topics individually. Does CMMI Benefit the Customer? What are the true benefits of Level X – to the customer? This presentation will explore the issues, by examining the fundamental principles behind the model, from a customer perspective. Costs and benefits to the customer will be emphasized, focusing on the relationship of CMMI practices to productivity, predictability and speed, as well as the cost of implementing CMMI-compliant processes. Timelines for impacting program performance, and practical tips and techniques for realizing the benefits will be highlighted. How High Maturity Projects Fail Process maturity supposed helps project deliver predictably on-time and on-budget. Why then, have we had a string of project failures from supposedly mature organizations, even ones that have achieved high maturity (Levels 4 and 5)? The presentation will examine the reasons why projects failure, and their relationship to process maturity. Some failures are due to non-process effects -- people, products, and technology. Mature processes diminish some of these effects and amplifying others. Some of the failures are due to projects not using the assets and capabilities of a mature organization. This presentation will identify the needed assets and discuss critical project start-up activities. How to Get Contractors to Live Up to Their CMMI Ratings Like many sports stars and teams, it is possible to have tremendous capabilities, but not live up to those in practice. Similarly, project teams may fail to live up to the capabilities implied by their CMMI level. This presentation describes simple, but effective ways to ensure CMMI practices are used on a project, and provide measurable benefits to the customer. Methods presented are based on techniques used successful by Northrop Grumman Corporation to monitor and manage their key suppliers.

1. Process Improvement
in the Aerospace Industry
CMMI and Lean Six Sigma
USC CS510 – 9 Nov 2010
Rick Hefner, Ph.D.
Northrop Grumman Corporation
Rick.Hefner@ngc.com

2. Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
2

3. NDIA Top 5 Systems Engineering Issues
(2006)
• Key systems engineering practices known to be effective are not
consistently applied across all phases of the program life cycle.
• Insufficient systems engineering is applied early in the program life cycle,
compromising the foundation for initial requirements and architecture
development.
• Requirements are not always well-managed, including the effective
translation from capabilities statements into executable requirements to
achieve successful acquisition programs.
• The quantity and quality of systems engineering expertise is insufficient
to meet the demands of the government and the defense industry.
• Collaborative environments, including SE tools, are inadequate to
effectively execute SE at the joint capability, system of systems (SoS),
and system levels.
3 Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera

4. Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
– Agile
• Northrop Grumman Approach
4

5. Heritage of Standards for Systems
Engineering
2002 2002
ISO/IEC ISO/IEC
15504 19760
(FDIS) (PDTR)
1998
1994 EIA 2002
EIA / IS 632 ISO/IEC
632 15288
(Full Std)
1994
(Interim Standard) (FDIS)
1974 Mil-Std- 1998
1994
1969 Mil-Std- 499B
IEEE 1998
EIA/IS
Mil-Std- 499A (Not Released) 731
1220 IEEE
499 SE CM
(Trial Use) 1220
(Interim Standard)
(Full Std)
2002
CMMI-
Legend SE/SW/IPPD
Supersedes
Source for Standards for Systems Engineering, Jerry Lake, 2002
5

6. The Frameworks Quagmire
Sarah A. Sheard, Software Productivity Consortium
6 http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp

7. Two Complimentary Approaches
to Process Improvement
Model-Driven (e.g., CMMI) Data-Driven (e.g., Lean Six Sigma)
• Determine the industry best • Clarify what your customer
practice wants (Voice of Customer)
– Benchmarking, models – Critical to Quality (CTQs)
• Compare your current practices to • Determine what your processes
the model can do (Voice of Process)
– Appraisal, education – Statistical Process Control
• Identify and prioritize • Identify and prioritize
improvement opportunities improvement opportunities
– Implementation – Causal analysis of data
– Institutionalization
• Anticipate your customers/
• Look for ways to optimize the competitors (Voice of Business)
7 processes – Design for Six Sigma

8. Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
8

9. What is the Capability Maturity Model
Integrated?
• The CMMI is a collection of industry best-practices for
engineering, services, acquisition, project management, support, and
process management
– Developed under the sponsorship of DoD
– Consistent with DoD and commercial standards
Three Constellations sharing common components and structure
• CMMI for Development - used by engineering organizations
• CMMI for Acquisition - used by buyers (e.g., govt. agencies)
• CMMI for Services - used by service providers (e.g., help desk)
9

10. The Basic Building Blocks of CMMI –
22 Process Areas
Implemented by Implemented by
each project the organization
Project Management Engineering Support Process Management
• Project Planning • Requirements • Configuration • Organizational
Development Management Process Focus
• Project
Monitoring and • Requirements • Process and • Organizational
Control Management Product Quality Process
• Technical Assurance Definition
• Supplier
Agreement Solution • Measurement • Organizational
Management • Product and Analysis Training
Integration • Decision Analysis • Organizational
• Integrated
• Verification and Resolution Process
Project
• Causal Analysis Performance
Management) • Validation
and Resolution • Organizational
• Risk Innovation and
Management Deployment
• Quantitative
Project
Management
10

11. Expected Practices Provide Guidance
for Implementation & Institutionalization
Project Planning – Implementation Project Planning - Institutionalization
SG 1 Establish Estimates GG 2 Institutionalize a Managed Process
SP 1.1 Estimate the Scope of the GP 2.1 Establish an Organizational
Project Policy
SP 1.2 Establish Estimates of Work GP 2.2 Plan the Process
Product and Task Attributes GP 2.3 Provide Resources
SP 1.3 Define Project Life Cycle GP 2.4 Assign Responsibility
SP 1.4 Determine Estimates of Effort GP 2.5 Train People
and Cost GP 2.6 Manage Configurations
SG 2 Develop a Project Plan GP 2.7 Identify and Involve Relevant
SP 2.1 Establish the Budget and Stakeholders
Schedule GP 2.8 Monitor and Control the Process
SP 2.2 Identify Project Risks GP 2.9 Objectively Evaluate Adherence
SP 2.3 Plan for Data Management GP 2.10 Review Status with Higher
SP 2.4 Plan for Project Resources Level Management
SP 2.5 Plan for Needed Knowledge and GG 3 Institutionalize a Defined Process
Skills GP 3.1 Establish a Defined Process
SP 2.6 Plan Stakeholder Involvement GP 3.2 Collect Improvement Information
SP 2.7 Establish the Project Plan
SG 3 Obtain Commitment to the Plan
SP 3.1 Review Plans that Affect the
Project
SP 3.2 Reconcile Work and Resource
Levels
SP 3.3 Obtain Plan Commitment
11

12. Practice Ratings for the
Organization/Projects
12

13. How is the CMMI Used for Process
Improvement?
IDEAL
Model
www.sei.cmu.edu/ideal/
13

14. Typical CMMI Benefits Cited in Literature
• Reduced costs
– 33% decrease in the average
cost to fix a defect (Boeing)
– 20% reduction in unit
software costs (Lockheed • Greater Productivity
Martin) – 25-30% increase in
productivity within 3 years
• Faster Schedules (Lockheed
– 50% reduction in release Martin, Harris, Siemens)
turnaround time (Boeing)
– 60% reduction in re-work • Higher Quality
following test (Boeing) – 50% reduction of software
defects (Lockheed Martin)
• Customer Satisfaction
– 55% increase in award fees
(Lockheed Martin)
14

15. Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
15

16. What is Lean Six Sigma (LSS)?
• Lean Six Sigma is a
powerful approach to
improving the work we do
• LSS improvement projects
are performed by teams
• Teams use a set of tools
and techniques to
understand problems and
find solutions
• Lean Six Sigma integrates +
tools and techniques from
two proven process
improvement methods
16

17. Six Sigma
• A management philosophy based on meeting business
objectives by reducing variation
– A disciplined, data-driven methodology for decision making and
process improvement
• To increase process performance, you have to decrease
variation • Greater
Too early Too late Too early Too late predictability in
the process
Defects Defects • Less waste and
rework, which
Reduce
lowers costs
Delivery Time
variation • Products and
Delivery Time
services that
Spread of variation Spread of variation perform better
too wide compared to narrow compared to and last longer
specifications specifications
• Happier
17
customers

18. DMAIC Roadmap
Define Measure Analyze Improve Control
Define Identify Explore Identify Define
project needed data possible control
scope data solutions method
Establish Obtain Characterize Select Implement
formal data set process & solution
project problem
Document
Evaluate Implement
data quality Update (pilot as
improvement needed)
project scope
Summarize & scale
& baseline Evaluate
data
[Hallowell-Siviy 05]
18

19. DMAIC Toolkit
Define Measure Analyze Improve Control
Benchmark GQIM and Cause & Effect Design of Statistical
Contract/Charter Indicator Diagrams/ Matrix Experiments Controls:
Templates Failure Modes & Modeling Control
Kano Model
Data Collection Effects Analysis ANOVA Charts
Voice of the Methods
Customer Statistical Tolerancing Time Series
Measurement Inference methods
Voice of the System Robust Design
Business Reliability
Evaluation Analysis Systems
Quality Function Thinking Non-Statistical
Deployment Root Cause Controls:
Analysis, includi Decision & Risk
Analysis Procedural
ng 5 Whys adherence
Hypothesis Test PSM Perform
Analysis Model Performance
Mgmt
Preventive
measures
19

20. Design for Six Sigma (e.g., DMADV)
Define Measure Analyze Design Verify
Define Identify Explore Develop Evaluate
project customers data detailed pilot
scope design
Scale-up
Research Design Refine
Establish design
VOC solution predicted
formal
project performance
Document
Benchmark
Predict
performance Develop
pilot
Quantify
CTQs
20

21. Lean
• Series of tools and techniques refined by Toyota and called
the “Toyota Production System”
– Called “Lean” by Womack, Jones and Roos in The Machine That
Changed the World
• Focused on increasing efficiency by eliminating non-value
added process steps and wasteful practices
• Being adopted world-wide by both manufacturing and
transactional based organizations
• Utilizes tools like “Value Stream Mapping,” “Just in Time” and
“Kaizen”
LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME
21

22. Wastes in Production
CORRECTION
•
MOTION
WAITING Repair or
Rework Any wasted motion
Any non-work time to pick up parts or
waiting for tools, stack parts. Also
supplies, parts, etc.. wasted walking
Types
PROCESSING OVERPRODUCTION
of
Producing more
Doing more work than Waste than is needed
is necessary
before it is needed
INVENTORY
CONVEYANCE
Maintaining excess
inventory of raw mat’ls, Wasted effort to transport
parts in process, or materials, parts, or
finished goods. finished goods into or
out of storage, or
between
processes.
22

23. Organizational Adoption:
Roles & Responsibilities
• Champions – Facilitate the leadership,
implementation, and deployment
• Sponsors – Provide resources
• Process Owners – Responsible for
the processes being improved
• Master Black Belts – Serve as mentors for Black Belts
• Black Belts – Lead major Six Sigma projects
– Typically requires 4 weeks of training
• Green Belts – Lead minor Six Sigma teams, or serve on
improvement teams under a Black Belt
– Typically requires 2 weeks of training
23

24. A Typical Lean Six Sigma Project
in Aerospace
The organization notes that systems integration has been problematic
on past projects (budget/schedule overruns)
A Six Sigma team is formed to scope the problem, collect data from
past projects, and determine the root cause(s)
The team’s analysis of the historical data indicates that ineffective peer
reviews are leaving significant errors to be found in test
Procedures and criteria for better peer reviews are written, using best
practices from past projects
A pilot project uses the new peer review procedures and criteria, and
collects data to verify they solve the problem
The organization’s standard process and training is modified to
incorporate the procedures and criteria, to prevent similar problems on
future projects
24

25. Agenda
• Current Challenges Facing the Aerospace Industry
• Current Industry Approaches
– Capability Maturity Model Integrated
– Lean Six Sigma
• Northrop Grumman Approach
25

26. Northrop Grumman Approach:
Mission Success Requires Multiple Approaches
Risk Management Dashboards for
Enterprise-Wide
Systems Engineering Program Measurement
Independent Reviews Effectiveness
Communications &
& Cost Estimates
Best-Practice Sharing
Training, Tools, & Mission
Assurance &
Robust Governance Model
Templates
Enterprise (Policies, Processes,
Excellence Procedures)
Process Operations
Effectiveness Effectiveness
CMMI Level 5 for
Software, Systems, and
Services
ISO 9001 and AS-9100
Certification
Six Sigma
26

27. Organizational Infrastructure Required for
CMMI Level 3
Policies, Processes, Process Group Training Program Process Improvement
Templates & Tools
Measurement Repositories Best-Practice Libraries Audits & Appraisals Communications
Predictive Modeling
Defects per component
25
20
15
UCL
10
5 _
X
0
1 11 21 31 41 51 61 71
Component #
Developing and maintaining mature processes requires
significant time and investment in infrastructure
27

28. Northrop Grumman Approach:
Institutionalizing Our Improvements
We systematically analyze quality and We improve our process assets
process data and trends to determine based on internal and external best
how to improve our processes practices
Deployed
Analysis Disposition to
Information programs
ISO/AS9100
Findings Industry
• Systems/ Policy
Software
Standards Configuration
CMMI Appraisal Control Board
Engineering Internal
Findings Process Group Process
Best Practices
Customer • QMS Working Six Sigma
Comments Group Procedures
Projects
Independent • Program
External Checklists and
Audits Management
Advisory Board Best Practices Guides
Lessons Learned
& Metrics Templates and
Examples
Tools eToolkit msCAS PAL
Increasing
StartIt! My MS Portal Workbench PCDB
program
28 efficiency

29. Northrop Grumman Approach:
Lessons Learned
• Multiple improvement initiatives helps encourage a change in behavior as
opposed to “achieving a level”
– Reinforces that change (improvement) is a way of life
• Benefits results from institutionalizing local improvements across the
wider organization
– CMMI establishes the needed mechanisms
• CMMI and Lean Six Sigma compliment each other
– CMMI can yield behaviors without benefits
– Lean Six Sigma improvements based solely on data may miss innovative
improvements (assumes a local optimum)
• Training over half the staff as Lean Six Sigma Green Belts has resulted in
a change of language and culture
– Voice of Customer, data-driven decisions, causal analysis, etc.
– Better to understand/use tools in everyday work than to adopt the “religion”
29