Advanced tutorial with a focus on value and making sure everything done related to reliability provides value

1. Effective Reliability Program
Traits and Management
Fred Schenkelberg
Ops A La Carte, LLC

2. Reliability Engineering Management
Fred Schenkelberg
Senior Reliability Consultant
Ops A La Carte, LLC
(408) 710-8248
fms@opsalacarte.com
2013 RAMS – Tutorial 4A - Schenkelberg 2

3. Tutorial Objectives
 To outline the key traits for the effective
management of a reliability program.
 To make you think about how to implement
reliability engineering within an organization.
2013 RAMS – Tutorial 4A - Schenkelberg 3

4. Upstairs/Downstairs
2013 RAMS – Tutorial 4A - Schenkelberg 4

5. HP‟s Design for Reliability Story
Which activities have impact?

6. DFR Survey
SURVEY CHECKLIST
Scoring: 4 = 100%, top priority Engineering:
3 = >75, use expected Documented design cycle
2 = 25 - 75%, variable use Reliability goal budgeting
1 = <25%, occasional use Priority of reliability improvement
0 = not done or discontinued DFR training programs
Management: Preferred technology program
Goal setting for division Component qualification testing
Priority of Quality & Reliab. OEM selection & qualif. Testing
Mgmnt attention & follow up Physical failure analysis
Root cause analysis
Manufacturing: Statistical engineering experiments
Design for Manufacturability Design & stress derating rules
Priority of Q & R goals Design reviews & checking
Ownership of Q & R goals Failure rate estimation
Quality training programs Thermal design & measurements
SPC & SQC use Worst case analysis
Internal process audits Failure Modes & Effects Analysis
Supplier process audits Environmental (margin) testing
Incoming inspection Highly Accel. Stress Testing
Product burn-in Design defect tracking
Defect Tracking Lessons-learned database
Corrective action
2013 RAMS – Tutorial 4A - Schenkelberg 6

7. Results
widespread use
 environmental test manual
 product lifecycle
range of use
 module goal setting
 derating rules
limited use
 DFR training
 physics of failure analysis
2013 RAMS – Tutorial 4A - Schenkelberg
7

8. Findings
 ODM concerns
how to convey needs
and get reliable products?
 time to market priority
urgent versus important
 management structures
many ways to organize roles
 mature products & scores
when only select tools apply
2013 RAMS – Tutorial 4A - Schenkelberg
8

9. Observations
best practices worst practices
 goal setting  repair & warranty
 prediction invisible
 statistics  lessons learned capture
 golden nuggets  single owner of product
 first look process
reliability
 multiple defect tracking
systems
2013 RAMS – Tutorial 4A - Schenkelberg
9

10. QUESTIONS?
2013 RAMS – Tutorial 4A - Schenkelberg
10

11. Reliability Philosophies
Two fundamental methods to achieving high
product reliability

12. Build, Test, Fix
 In any design there are a finite number of flaws.
 If we find them, we can remove the flaw.
 Rapid prototyping
 HALT
 Large field trials or „beta‟ testing
 Reliability growth modeling
2013 RAMS – Tutorial 4A - Schenkelberg 12

13. Analytical Approach
 Develop goals
 Model expected failure mechanisms
 Conduct accelerated life tests
 Conduct reliability demonstration tests
 Routinely update system level model
 Balance of simulation/testing to increase ability of
reliability model to predict field performance.
2013 RAMS – Tutorial 4A - Schenkelberg 13

14. Issues with each approach
Build, Test, Fix Analytical
 Uncertain if design is good  Fix mostly known flaws
enough  ALT‟s take too long
 Limited prototypes means  RDT‟s take even longer
limited flaws discovered  Models have large
 Unable to plan for warranty uncertainty with new
or field service technology and
environments
2013 RAMS – Tutorial 4A - Schenkelberg 14

15. Balanced approach
Goal
Plan
FMEA Prediction
HALT RDT/ALT
Verification
Review
2013 RAMS – Tutorial 4A - Schenkelberg 15

16. Balanced approach
Goal
Plan
FMEA Prediction
HALT RDT/ALT
Verification
Review
2013 RAMS – Tutorial 4A - Schenkelberg 16

17. Balanced approach
Goal
Plan
FMEA Prediction
HALT RDT/ALT
Verification
Review
2013 RAMS – Tutorial 4A - Schenkelberg 17

18. Balanced approach
Goal
Plan
FMEA Prediction
HALT RDT/ALT
Verification
Review
2013 RAMS – Tutorial 4A - Schenkelberg 18

19. QUESTIONS?
2013 RAMS – Tutorial 4A - Schenkelberg
19

20. Reliability Goal Setting
Establish the target in an engineering
meaningful manner

21. Reliability Definition
 Reliability is often considered quality over
time
 Reliability is the probability of a product
performing its intended function over its
specified period of usage, and under
specified operating conditions, in a manner
that meets or exceeds customer
expectations.
2013 RAMS – Tutorial 4A - Schenkelberg 21

22. Intended Function
2013 RAMS – Tutorial 4A - Schenkelberg 22

23. Environment
2013 RAMS – Tutorial 4A - Schenkelberg 23

24. Duration
2013 RAMS – Tutorial 4A - Schenkelberg 24

25. Probability
2013 RAMS – Tutorial 4A - Schenkelberg 25

26. Reliability Goal-Setting
 Reliability Goals can be derived from
 Customer-specified or implied requirements
 Internally-specified or self-imposed requirements
(usually based on trying to be better than previous
products)
 Benchmarking against competition
2013 RAMS – Tutorial 4A - Schenkelberg 26

27. Example Exercise
 Elements of Product Requirements Document
 Take notes to build a reliability goal statement
2013 RAMS – Tutorial 4A - Schenkelberg 27

28. Goal Statement exercise
 In groups of two or three draft a reliability goal
 Note the missing information and draft questions
to get the missing information
 This is a brand new product with no field history
– how would you apportion the system goal to the
various subsystems?
(regulator, valve, control circuitry, and enclosure)
2013 RAMS – Tutorial 4A - Schenkelberg 28

29. Reliability Goals & Metrics Summary
 A reliability metric is often something that
organization can measure on a relatively
short, periodic basis:
 Predicted failure rate (during design phase)
 Field failure rate
 Warranty
 Actual field return rate
 Dead on Arrival rate
29
(v5) 2013 RAMS – Tutorial 4A - Schenkelberg 29

30. Fully-Stated Reliability Goals
 System goal at multiple points
 Supporting metrics during development and field
 Apportionment to appropriate level
 Provide connections to overall business
plan, contracts, customer expectations, and
include any assumptions concerning financials
 Benefit: clear target for development, vendor
and production teams. 30
2013 RAMS – Tutorial 4A - Schenkelberg 30

31. Reliability Goal
 Let‟s say we expect a
 t
few failures in one
R (t )  e 
year.
 Less than 2% ln(. 98 )   8760 / 
 Laboratory environ.
 XYZ function  XYZ function for one
year with 98% reliability
in the lab.
 Assuming constant
 (MTBF is 433,605 hrs.)
failure rate
2013 RAMS – Tutorial 4A - Schenkelberg 31

32. Other Points in Time
 Also consider other business relevant points in
time
 Infant mortality, out of box type failures
 Shipping damage
 Component defects, manufacturing defects
 Wear out related failures
 Bearings, connectors, solder joints, e-caps
2013 RAMS – Tutorial 4A - Schenkelberg 32

33. Break Down Overall Goal
 Let‟s look at example
 A computer with a one year warranty and the
business model requires less than 5% failures
within the first year.
 A desktop business computer in office environment
with 95% reliability at one year.
2013 RAMS – Tutorial 4A - Schenkelberg 33

34. Break Down the Goal, (continued)
 For simplicity consider five major elements
of the computer
 CPU/motherboard
 Hard Disk Drive
 Power Supply
 Monitor
 Bios, firmware
 For starters, let‟s give each sub-system the
same goal
2013 RAMS – Tutorial 4A - Schenkelberg 34

35. Apportionment of Goals
Computer
R = 0.95
CPU HDD P/S Monitor Bios
R = 0.99 R = 0.99 R = 0.99 R = 0.99 R = 0.99
Assuming failures within each sub-system are independent,
the simple multiplication of the reliabilities should result in
meeting the system goal
0.99 * 0.99 * 0.99 * 0.99 * 0.99 = 0.95
Given no history or vendor data – this is just a starting point.
2013 RAMS – Tutorial 4A - Schenkelberg
35

36. Estimate Reliability
 The next step is to determine the sub-system
reliability.
 Historical data from similar products
 Reliability estimates/test data by vendors
 In house reliability testing
 At first estimates are crude, refine as needed to
make good decisions.
2013 RAMS – Tutorial 4A - Schenkelberg 36

37. Apportionment of Goals
Computer
R = 0.95
Goals
CPU HDD P/S Monitor Bios
R = 0.99 R = 0.99 R = 0.99 R = 0.99 R = 0.99
Estimates
CPU HDD P/S Monitor Bios
R = 0.96 R = 0.98 R = 0.999 R = 0.99 R = 0.999
First pass estimates do not meet system goal. Now what?
2013 RAMS – Tutorial 4A - Schenkelberg 37

38. Resolving the Gap
 CPU goal 99% est. 96%  Use the simple reliability
 Largest gap, lowest model to determine if
estimate reliability improvements
 First, will the known will impact the system
issues bridge the reliability. i.e. changing
difference? the bios reliability form
99.9% to 99.99% will not
 In not enough, then use significantly alter the
FMEA and HALT to system reliability result.
populate Pareto of what
to fix  Invest in improvements
that will impact the
 Third, validate system reliability.
improvements
2013 RAMS – Tutorial 4A - Schenkelberg
38

39. Resolving the Gap, (continued)
 HDD goal 0.99 est. 0.98  When the relationship of
the failure mode and either
 Small gap, clear path to design or environmental
resolve conditions exist we do not
need FMEA or HALT –
go straight to design
 HDD reliability and improvements.
operating temperature are
related. Lowering the
internal temperature the  Use ALT to validate the
HDD experiences will model and/or design
improve performance. improvements.
2013 RAMS – Tutorial 4A - Schenkelberg 39

40. Resolving the Gap, (continued)
 P/S goal 0.99 est. 0.999  For any subsystem that
exceeds the reliability goal,
 Estimate over the goal explore potential cost
 Further improvement not savings by reducing the
cost effective given reliability performance.
minimal impact to system  This is only done when there
reliability. is accurate reliability
estimates and significant cost
 Possible to reduce savings.
reliability (select less
expensive model) and use
savings to improve
CPU/motherboard.
2013 RAMS – Tutorial 4A - Schenkelberg 40

41. Progression of Estimates
Initial Engineering Guess or Estimate
Test Data
Vendor Data
Actual Field
Data
2013 RAMS – Tutorial 4A - Schenkelberg
41

42. Reliability Goals & Metrics Summary
 A reliability goal includes each of the four
elements of the reliability definition.
 Intended function
 Environment (including use profile)
 Duration
 Probability of success
 [Customer expectations]
2013 RAMS – Tutorial 4A - Schenkelberg 42

43. Reliability Planning
Selecting the minimum set of tools to
achieve the reliability goals

44. Planning Introduction
 Mil Hdbk 785 task 1
“The purpose of this task is to develop a reliability
program which identifies, and ties together, all
program management tasks required to
accomplish program requirements.”
2013 RAMS – Tutorial 4A - Schenkelberg 44

45. Fully Stated Reliability Goals
 System goal at multiple points
 Supporting metrics during development and field
 Apportionment to appropriate level
 Provide connections to overall business plan,
contracts, customer expectations, and include any
assumptions concerning financials
 Benefit: clear target for development, vendor and
production teams.
2013 RAMS – Tutorial 4A - Schenkelberg 45

46. Medicine
"The abdomen, the chest, and the brain will be
forever shut from the intrusion of the wise and
humane surgeon"
Sir John Erichsen
leading British surgeon
1837
2013 RAMS – Tutorial 4A - Schenkelberg 46

47. Gap Analysis
 Estimate/review current reliability of system
against the next project goal
 The difference is the gap to close
 That gap is what the plan needs to bridge
2013 RAMS – Tutorial 4A - Schenkelberg 47

48. Path to close gap
 This is the „art‟ of our profession and each project
needs a unique solution.
 Just because the plan succeeded for the last
project, it may not work for the current one
 Timelines change
 Goals and risks change
 Business objectives and customer expectations change
 The organization has grown/lost capabilities
2013 RAMS – Tutorial 4A - Schenkelberg 48

49. If, (what is your situation)
When starting a project, consider the goals,
constraints, etc. and look at the entire horizontal
process.
Then,
 Let‟s find a few options to consider
2013 RAMS – Tutorial 4A - Schenkelberg 49

50. Exercise
 Identify a circumstance and an approach to
building the reliability plan.
 What will be the biggest challenges to
implementing the plan?
 Separate from the plan, what will you do as the
reliability engineer do to overcome the obstacles?
2013 RAMS – Tutorial 4A - Schenkelberg 50

51. Close on Planning Discussion
 Introduction to Planning
 Fully stated reliability goals
 Constraints
 Timeline
 Prototype samples
 Capabilities (skills and maturity)
 Current state and gap to goal
 Paths to close the gap
 Investments
 Dual paths
 Tolerance for risk
2013 RAMS – Tutorial 4A - Schenkelberg 51

52. Television
"People will soon get tired of staring at a
plywood box every night."
Darryl F. Zanuck
Twentieth Century-
Fox, 1946
2013 RAMS – Tutorial 4A - Schenkelberg 52

53. Reliability Value
How to speak in management‟s language

54. A Reliability Engineer‟s Use of
Warranty Cost Information
Fred Schenkelberg

55. Introduction
 Many (most, all?) products have a warranty
 Examples of how to use this information in your
reliability engineering work
2013 RAMS – Tutorial 4A - Schenkelberg 55

56. Electric Light
“Good enough for our transatlantic friends, but
unworthy of the attention of practical or
scientific men.”
British Parliament report on Edison’s work
1878
2013 RAMS – Tutorial 4A - Schenkelberg 56

57. Overview
 Warranty as a percentage of revenue.
 Warranty as a cost per unit.
 Who owns warranty?
 How much warranty expense is right?
 What is the right investment to reduce warranty?
2013 RAMS – Tutorial 4A - Schenkelberg 57

58. Warranty Week
www.warrantyweek.com
2013 RAMS – Tutorial 4A - Schenkelberg 58

59. Computers
“There is no reason for any individual to have a
computer in their home.”
Ken Olson
Digital Equipment Corp.
1977
2013 RAMS – Tutorial 4A - Schenkelberg 59

60. Reliability Specifications Example
 Given two fan datasheets
 Fan A has a mean time to fail of 4645 hours
 Fan B has a mean time to fail of 300 hours
 Both same price, etc.
 Choose one to maximize reliability
at 100 hours
2013 RAMS – Tutorial 4A - Schenkelberg 60

61. Reliability Specifications Example
 Consulting an internal fan expert, you are
advised to get more information
 Fan A has a Weibull time to fail shape
parameter of 0.8
 Fan B has a Weibull time to fail shape
parameter of 3.0
 1 
   1  
 
  
2013 RAMS – Tutorial 4A - Schenkelberg 61

62. Reliability Specifications Example
 Fan A has a scale parameter of 4100 hours
 Fan B has a scale parameter of 336 hours
 Use the Weibull Reliability function

 t /  
R (t )  e
 Fan A reliability at 100 hours is 0.95
 Fan B reliability at 100 hours is 0.974
2013 RAMS – Tutorial 4A - Schenkelberg 62

63. Reliability Specifications Example
 Given two fan datasheets
 Fan A has a mean time to fail of 4645 hours
 Fan B has a mean time to fail of 300 hours
 What about later, say 1000 hours?
 Fan A reliability at 1000 hours is 0.723
 Fan B reliability at 1000 hours is 3.5E-12
2013 RAMS – Tutorial 4A - Schenkelberg 63

64. The Telephone
"That's an amazing invention, but who
would ever want to use one of them?"
Rutherford Hayes
U.S. President, 1876
2013 RAMS – Tutorial 4A - Schenkelberg 64

65. The Cost Reduction Example
 Given a FET that costs 10 cents, a new
procurement engineer finds a new FET vendor
that only charges 5 cents.
 Switch?
 What else to consider?
2013 RAMS – Tutorial 4A - Schenkelberg 65

66. The Cost Reduction Example
 Given a FET that costs 10 cents, a new
procurement engineer finds a new FET vendor
that only charges 5 cents.
 $0.05 FET has MTBF of 50,000 hours
 $0.10 FET has MTBF of 75,000 hours
 1000 hours of operation
 Shipping 1000 units
 Cost to repair unit $250
2013 RAMS – Tutorial 4A - Schenkelberg 66

67. The Cost Reduction Example
 Total Cost of $0.10 FET
 1000 
 
R 0 .10 1000   e  75 , 000 
 0 . 987
 #Failed = (1-0.987) 1000 units = 13.25
 Cost of Repairs = 250*13 = $3250
 Total Cost = $3250+0.10*1000 = $3350
2013 RAMS – Tutorial 4A - Schenkelberg 67

68. The Cost Reduction Example
 Total Cost of $0.05 FET
 1000 
 
R 0 .05 1000   e  50 , 000 
 0 . 98
 #Failed = (1-0.98) 1000 units = 20
 Cost of Repairs = 250*20 = $5000
 Total Cost = $5000+0.05*1000 = $5050
2013 RAMS – Tutorial 4A - Schenkelberg 68

69. The Cost Reduction Example
 Total Cost of $0.50 FET
 1000 
 
R 0 .50 1000   e  100 , 000 
 0 . 99
 #Failed = (1-0.99) 1000 units = 10
 Cost of Repairs = 250*10 = $2500
 Total Cost = $2500+0.50*1000 = $3000
2013 RAMS – Tutorial 4A - Schenkelberg 69

70. The Cost Reduction Example
 Result?
FET Repair Total
Cost Cost Cost
$0.10 $3250 $3350
75,000 hrs
$0.05 $5000 $5050
50,000 hrs
$0.50 $2500 $3000
100,000hrs
2013 RAMS – Tutorial 4A - Schenkelberg 70

71. Aviation
"The popular mind often pictures gigantic flying
machines speeding across the Atlantic and
carrying innumerable passengers...it seems
safe to say that such ideas are wholly
visionary."
Wm. Henry Pickering
Harvard astronomer, 1908
2013 RAMS – Tutorial 4A - Schenkelberg 71

72. Component Challenges
 Cost driving manufacturing to low labor cost
areas of the world
 Pb-free causing redesign/reformulation
 Outsourced design and manufacturing facilities
gaining “commodity‟ component selection
 Other than yield - who‟s watching Quality,
Reliability and Warranty?
2013 RAMS – Tutorial 4A - Schenkelberg 72

73. Component Challenges
 P50 formula error example
 Cracked ceramic capacitors
2013 RAMS – Tutorial 4A - Schenkelberg 73

74. Component Challenges
 Trust and verify solution
 Build strong, technically verifiable, language into
purchase contracts
 Check construction and formulation on periodic
basis
2013 RAMS – Tutorial 4A - Schenkelberg 74

75. Nuclear Energy
"Nuclear powered vacuum cleaners will
probably be a reality within 10 years."
Alex Lewyt
vacuum cleaner
manufacturer,1955
2013 RAMS – Tutorial 4A - Schenkelberg 75

76. Where to Get More Information
 Newsletter and seminars
http://Warrantyweek.com
 “Warranty Cost: An Introduction”
http://quanterion.com/ReliabilityQues/V3N3.html
 “Economics of Reliability,” Chapter 4 of
Handbook of Reliability Engineering and Management, 2nd
Ed by Ireson, Coombs and Moss.
2013 RAMS – Tutorial 4A - Schenkelberg 76

77. Reliability Engineering Value
How to determine „value add‟ or ROI

78. “All metrics are wrong, some are useful.”
2013 RAMS – Tutorial 4A - Schenkelberg

79. value
2013 RAMS – Tutorial 4A - Schenkelberg

80. Terms
 Value
 An amount considered to be a suitable equivalent for
something else; a fair price or return for goods or
services
 Value Add
 The return or result of individual, team or product
investment
 Value Capture
 Value add documentation related directly to merger
 Warranty Reduction
 Lower failure rates leading to fewer claims
2013 RAMS – Tutorial 4A - Schenkelberg 80

81. How is value requested?
 Quarterly review: What have you done for me
lately?
 Checkpoint meeting: Are we on track to meet
goals?
 Budget: Which option provides best ROI?
 Annual review: What is your impact?
2013 RAMS – Tutorial 4A - Schenkelberg 81

82. current status
2013 RAMS – Tutorial 4A - Schenkelberg

83. Warranty – The Big Picture
”American manufacturers spent over $25 billion in 2004
honoring their product warranties, an increase of 4.8% from
the levels seen in 2003. However, an incredible 63% of U.S.-
based product manufacturers actually saw a decrease in their
claims rates as a percentage of sales. Only 35% saw an
increase and 2% saw no change, according to the latest
statistics compiled by Warranty Week.”
Eric Arnum, Warranty Week
www.warrantyweek.com, May 27th, 2005
2013 RAMS – Tutorial 4A - Schenkelberg 83

84. document value
2013 RAMS – Tutorial 4A - Schenkelberg

85. VALUE ADDED/ROI QUESTIONAIRE
Savings/Impact/Benefit
1. Risk / cost / warranty a. Has the work directly identified or mitigated a field related problem
reduction
b. If so estimate the probable cost of the field problem in $ (i.e. units affected
x repair cost)
c. Has the probability of field related problems been reduced?
d. If so give a guide by how much and the estimated cost of avoidance (i.e.
Estimate 1000 units per month failure at $50 each reduced by 5%)
e. Has work provided processes which will reduce the risk of field failures in
subsequent products?
2. TTM impact: a. Did work help you meet or beat your TTM goals?
b. Did work identify any problems which would have impacted your TTM?
c. Has the use of tools/techniques identified issues which would of impacted
TTM?
d. If the above are applicable please identify type of problems and estimate
TTM impact in days/weeks/months
e. What is the estimated cost of a delay in TTM?
f. What is the opportunity in $ of additional income from an early TTM?
2013 RAMS – Tutorial 4A - Schenkelberg
85

86. VALUE ADDED/ROI QUESTIONAIRE
Savings/Impact/Benefit
a. Did work help you accelerate or meet your Time to Volume
3. TT Volume impact:
goals?
b. If applicable what is the estimated $ impact of avoiding the
TTV issues that were identified
4. Material costs: a. Did we avoid or save any direct product material or test
equipment costs?
b. If so please identify type and cost
5. TCE: a. Has the work contributed to the TCE of your product?
b. If so identify how? i.e. estimated number of customer calls
avoided
c. If you have a TCE cost model what is the estimated $ impact
of the identified improvement
6.Opportunity Cost a. If engineers from the business had been used to do this work
would they have not been able do other product related work. I.e.
delivered new functions?
7. Indirect Impact: a. What advantages did internal work provide over an external
consultancy? (i.e. time, cost, contractual issues, Intellectual
Property, response time)
2013 RAMS – Tutorial 4A - Schenkelberg 86

87. “I fall back dazzled at beholding myself all rosy red,
At having, I myself, caused the sun to rise”
Edmund Rostand (1868-1918)
2013 RAMS – Tutorial 4A - Schenkelberg
87

88. VALUE ADDED/ROI QUESTIONAIRE
Savings/Impact/Benefit
8. Engineering effort a. How long would it have taken your team to undertake the
work provided. Take into account research time and whether you
saved:
had the skills available
b. If you did not have the skills available how many people
would have needed to be recruited to undertake the work?
c. How long would it take for these people to become
productive?
d. Estimate training cost associated with new personnel
9. Misc a. Please identify any other benefits or cost savings from using
our resources
2013 RAMS – Tutorial 4A - Schenkelberg
88

89. “Gross national product measures neither the health of our
children, the quality of their education, nor the joy of their play
It measures neither the beauty of our poetry, nor the strength
of our marriages.
It is indifferent to the decency of our factories and the safety
of our streets alike.
It measures neither our wisdom nor our learning, neither our
wit nor our courage, neither our compassion or our devotion
to country.
It measures everything in short, except that which makes life
worth living, and it can tell us everything about our country
except those things which make us proud to be part of it.”
Robert Kennedy
2013 RAMS – Tutorial 4A - Schenkelberg
89

90. Your „value case‟
 Problem statement
 Work done to solve problem
 Value statement(s)
2013 RAMS – Tutorial 4A - Schenkelberg 90

91. Reliability Maturity
How to understand an organization‟s
reliability culture

92. Maturity Matrix
 Handout Matrix
 Based on Quality Management Maturity Grid
from Quality is Free, c 1979 by Philip B. Crosby
2013 RAMS – Tutorial 4A - Schenkelberg 92

93. Measurement Categories
 Management Understanding and Attitude
 Business objectives and language
 Attention and investments
 Reliability Status
 Position and stature
 Location and influence
2013 RAMS – Tutorial 4A - Schenkelberg 93

94. Measurement Categories
 Problem Handling
 Proactive or Reactive
 Cost of „Un‟ Reliability
 Understanding and influence of metrics
 Local budget or total product cost
 Feedback Process
 Predictions, reliability testing
 Failure analysis, time to detection
2013 RAMS – Tutorial 4A - Schenkelberg 94

95. Measurement Categories
 DFR program status
 Exists separately or integrated
 Template or customized
 Summation of Reliability Posture
 How does the organization talk about reliability?
2013 RAMS – Tutorial 4A - Schenkelberg 95

96. Stage I Uncertainty
 Management – blame others
 Status – hidden or doesn‟t exist
 Problems – may have good fire fighting
 Cost – unknown and no influence
 Feedback – customer returns & complaints
 DFR – doesn‟t exist even with designers
 Summation – “Reliability must be ok, since
customer‟s are buying our products.”
2013 RAMS – Tutorial 4A - Schenkelberg 96

97. Stage II Awakening
 Management – important w/o resources
 Status – champion recognized
 Problems – organized fire fighting
 Cost – generally warranty only
 Feedback – disorganized, antidotal
 DFR – trying some tools
 Summation – “We really should make more
reliable products.”
2013 RAMS – Tutorial 4A - Schenkelberg 97

98. Stage III Enlightenment
 Management – Support and encouragement
 Status – Senior staff influence
 Problems – Systematic and reactive
 Cost – Starting to track cost of un-reliability
 Feedback – ALT and modeling, root cause
 DFR – program of reliability activities
 Summation – “We can see how these tools help
our product‟s field performance.”
2013 RAMS – Tutorial 4A - Schenkelberg 98

99. Stage IV Wisdom
 Management – Personally involved, leading
 Status – Senior manager, major role
 Problems – found and resolved quickly
 Cost – understanding of major drivers
 Feedback – selective testing in risk areas
 DFR – Part of products get designed
 Summation – “We avoid most field reliability
issues”
2013 RAMS – Tutorial 4A - Schenkelberg 99

100. Stage V Certainty
 Management – Considered core capability
 Status – thought leader in company
 Problems – Only a few issue, & expected
 Cost – Accurate and decreasing
 Feedback – Testing & field support models
 DFR – Normal part of company business
 Summation – “We do get surprised by the few
field failures that occur.”
2013 RAMS – Tutorial 4A - Schenkelberg 100

101. Why do we need to know Maturity?
 Recommendations need to match the
organizations capabilities
 From current state build path toward the right one
step at a time
 Value proposition for changes address
management approach to reliability
2013 RAMS – Tutorial 4A - Schenkelberg 101

102. How to determine maturity?
 Self assessment
 Small team from across organization
 Each marks blocks that describe their maturity
 Team determine Stage description by consensus
 Observation from within an organization
 As an individual trying to position changes
 Informally conduct self assessment
2013 RAMS – Tutorial 4A - Schenkelberg 102

103. How to determine maturity?
 Assessment Interviews
 Conduct interviews to understand current reliability
activities
 Review and summarize interviews
 Interpret results onto maturity matrix
2013 RAMS – Tutorial 4A - Schenkelberg 103

104.  What are your questions?
2013 RAMS – Tutorial 4A - Schenkelberg 104

105. Reliability Assessment
Using a survey to quickly understand the
organization‟s reliability program

106. survey approach
 selecting survey topics choosing interviewees
 interview format  hw r&d manager
 data collection  hw r&d engineer
 business unit summary  reliability manager
 immediate follow up  reliability engineer
 analysis  procurement
 review  manufacturing
 key stakeholder reporting
2013 RAMS – Tutorial 4A - Schenkelberg
106

107. survey form & scoring
DFR Methods Survey
Scoring: 4 = 100%, top priority, always done
3 = >75%, use normally, expected
2 = 25% - 75%, variable use
1 = <25%, only occasional use
0 = not done or discontinued
- = not visible, no comment
Management:
 Goal setting for division
 Priority of quality & reliability improvement
 Management attention & follow up (goal ownership)
Design:
 Documented hardware design cycle
 Goal setting by product or module
2013 RAMS – Tutorial 4A - Schenkelberg 107

108. design survey topics
Design:
 Documented hardware design cycle
 Goal setting by product or module
 Priority of Q&R vs. performance, cost, schedule
 Design for Reliability (DFR) training
 Preferred technology selection/standardization
 Component qualification testing
 OEM selection & testing to equal HP requirements
 Fault Tree Analysis/Rel. Block Diagrams (FTA/RBD)
 Failure/root cause analysis
 Statistically-designed engineering experiments
 Accelerated Stress/Life Testing (ALT)
 Design & derating rules
2013 RAMS – Tutorial 4A - Schenkelberg 108

109. design survey topics
 Design reviews/design rule checking
 Finite Element Analysis (FEA) or simulations
 Failure rate estimation/prediction
 Thermal design & measurements
 Design tolerance analysis
 Failure Modes & Effects Analysis (FMEA)
 Environmental (design margin) testing
 Highly accelerated life testing (HALT)
 Physics of Failure analysis
 Lessons-learned database
 Design Defect Tracking (DDT)
 Ownership of quality & reliability goals
2013 RAMS – Tutorial 4A - Schenkelberg 109

110. manufacturing survey topics
Manufacturing:
 Design for manufacturability (DFM)
 Priority of Q&R vs. schedule & cost
 Quality training programs
 Statistical Process Control (SPC/SQC)
 Total Quality Management (TQM)
 HP process audits (written reports)
 Vendor (& OEM) process audits, TQRDCE
 Incoming inspection/sampling
 Component burn-in
 Assembly-level environmental stress screening (ESS)
 Product-level environmental stress screening (ESS)
 Defect Detection & Tracking (DD&T)
 Corrective Action Reports
 Ownership of quality & reliability goals
2013 RAMS – Tutorial 4A - Schenkelberg 110

111. Aircraft Company Example
 AC, Inc. a private jet manufacturer, develops,
manufactures, sells and provides support for
aircraft, throughout the intended life cycle. The
product design process is dominated by the ability
to meet FAA certification requirements. This
product is high cost and very low volume.
 Handout, AC, Inc. Survey Summary
 Determine maturity stage and make
recommendations
2013 RAMS – Tutorial 4A - Schenkelberg 111

112. AC, Inc. key points
 MTBF metrics
 Excellent field data
 Very limited sample sizes
 Reactive mode to improvement activities
2013 RAMS – Tutorial 4A - Schenkelberg 112

113. AC, Inc. Recommendations
 Use Reliability rather than MTBF. Establish fully stated
reliability goal in terms of the probability of components
and aircraft successfully performing as expected under
stated conditions for two or more defined time periods.
Reliability is a metric that does not have a dependence on
a particular lifetime distribution and is intuitively
interpreted by engineers correctly. Using multiple time
marks, it promotes the use of lifetime distributions rather
than single parameter descriptions. Once engineers are
using lifetime distributions, calculating confidence
intervals is a natural extension.
2013 RAMS – Tutorial 4A - Schenkelberg 113

114. AC, Inc. Recommendations
 Build and support an aircraft reliability model. Use the historical
data, lifetime distributions (not MTBF), RBD (reliability block
diagramming) and simple mathematics to quickly create a basic
reliability model. An extension of the model would be to
incorporate the various environmental factors, flight profiles,
and the influence of other relevant variables on failure rates. For
example, some systems experience damaging stress during
takeoffs and landings, others only while in flight, some only
when landing in high temperature and humidity climates. Ideally
for each component the model would incorporate historical field
history along with environmental and component data. Even a
very simple model that enables the design and procurement
teams to evaluate options is well worth the effort to build and
support. Most importantly a reliability model provides feedback
very quickly to the design team during the design process.
2013 RAMS – Tutorial 4A - Schenkelberg 114

115. Additional Reading
 Practical Reliability Engineering, 4th Edition,
Patrick D. T. O‟Connor, 2002
 Improving Product Reliability: Strategies and
Implementation, Mark A. Levin and Ted T. Kalal,
2003
 Quality is Free: The Art of Making Quality
Certain, Philip B. Crosby, 1979
 Design Paradigms: Case Histories of Error and
Judgment in Engineering, Henry Petroski, 1994
2013 RAMS – Tutorial 4A - Schenkelberg 115