Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Measurement System Analysis

Measurement System Analysis
Part 3: Continuous Measurements
Week 2
Knorr-Bremse Group
About this Module
Based on this technique you can assess and judge
t t h b tt d ib d imeasurement systems much better as described in
the ISO 9000 standard.
• Part 1: Introduction of Measurement System Analysis
– Concept definition and describing the basic termsConcept definition and describing the basic terms
• Part 2: Attributive Measurements
– Kappa Analysis
• Part 3: Continuous Measurements
– The Gage R&R Study
• Some exercises
Knorr-Bremse Group 05 BB W1 Cont. Measurement 08, D. Szemkus/H. Winkler Page 2/36

The DMAIC Cycle
Control
Maintain Improvements
SPC
Define
Project charterSPC
Control Plans
Documentation
Project charter
(SMART)
Business Score Card
QFD + VOC
D QFD VOC
Strategic Goals
Project strategy
C M
Measure
Baseline AnalysisImprove
AI
Baseline Analysis
Process Map
C + E Matrix
M t
Analyze
Definition of critical
p
Adjustment to the
Optimum
FMEA
Measurement
System
Process Capability
Definition of critical
Inputs
FMEA
Statistical Tests
Statistical Tests
Simulation
Tolerancing
Process CapabilityStatistical Tests
Multi-Vari Studies
Regression
Effects of Measurement Error
Measurement System
Average
Bias -Determined through
“Calibration Study”
Accuracy
µ µ µtotal product measurement= +p
V i bilit
Measurement System
Variability - Determined
through “R&R Study”
Variability Precision
222 222
tmeasuremenproducttotal σσσ +=

Index - P/T (Precision to Tolerance)
Usually expressed as
percent100
6
/ ×
×
=TP MSσ
percent100/ ×=
Tolerance
TP
• This value addresses, what percent of the Tolerance is
taken by the measurement error (Precision).taken by the measurement error (Precision).
• It includes both repeatability and reproducibility.
• P/T < 10% Measurement system excellent
• P/T < 20% Measurement system acceptableP/T 20% Measurement system acceptable
• P/T < 30% Measurement system marginal acceptable
Note: 6 standard deviations accounts for 99,73 % of MS variation. 5.15
standard deviations accounts for 99% of MS variation (Industry standard)
standard deviations accounts for 99% of MS variation. (Industry standard).
Relation Precision to Tolerance
P/T
Product tolerance
LSL USL
V i i f h
Product tolerance
LSL USL
Variation of the
measurement
system
P/T = 20%
system
P/T = 100%
P/T = 200%

Use and Interpretation of P/T
The P/T-Ratio is the most often used evaluation for judging the
precision of a measurement system.
– This is telling us how good the the system is working
regarding the specification.
– But specifications might be to wide or to small
I h f j d i d i– In the area of judging a product to an important customer
specification the P/T-Ratio is the best evaluation.
– The measurement error σMS includes 2 components
• Repeatability – Variation caused by the measurement devicep y y
• Reproducibility – Variation caused by the operator
– Regarding process capability and process improvement the
single use of the P/T-Ratio can create a wrong feeling of
safety
safety.
Index %R&R (Precision to Total Variation)
100&% ×= MS
RR
σ Usually expressed as
percent100&% ×=
Total
RR
σ
percent
• This value addresses, what percent of the total variation is
taken by the measurement error (Precision).y ( )
• It includes both repeatability and reproducibility.
• P/T < 10% Measurement system excellent
• P/T < 20% Measurement system acceptabley p
• P/T < 30% Measurement system marginal acceptable
%R&R is the best possibility for a Black Belt, to asses the capability of
measurement systems. Based on the %R&R we can decide where to
concentrate for improvements. Process or measurement system

Repeatability & Reproducibility
%R&R
Ob d i ti
%R&R = 20%
Observed process variation
Variation of the
measurementmeasurement
system
%R&R = 75%
%R&R = 100%
Use and Interpretation of %R&R
%R&R is the best tool for estimating the capability of a measurement
system during process improvement activities.
– The %R&R – ratio estimates the capability of the
measurement system in relation to the total process variationmeasurement system in relation to the total process variation.
That means, it gives us the information how good the real
process / product variation can be identified.
– The %R&R includes 2 components
• Repeatability – Variation caused by the measurement device
R d ibili V i i d b h• Reproducibility – Variation caused by the operator
– The use of statistical tools (and their conclusion) can beThe use of statistical tools (and their conclusion) can be
influenced by high %R&R values. Therefore, the capability of
the used measurement system should be checked in
d
advance.

Comparison of P/T and %R&R
Product tolerance
LSL USL
Ob d i tiObserved process variation
%R&R = 20%P/T = 20%
%R&R = 50%P/T = 50%
%R&R = 100%P/T = 100% %R&R = 100%P/T = 100%
Measurement system variation
Product tolerance
LSL USL
Ob d i tiObserved process variation
P/T = 50% %R&R = 25%
P/T = 100% %R&R = 50%
P/T = 200% %R&R = 100%P/T = 200% %R&R = 100%

Product tolerance
LSL USL
Observed
process variationprocess variation
P/T = 10% %R&R = 20%
P/T = 20% %R&R = 40%
P/T = 50% %R&R = 100%P/T = 50% %R&R = 100%
Relation % R&R - Cp
A simplified illustration of the relation % R&R and Cp
Th l ti f h th f ll i lThe evaluation of a process shows the following values:
Mean = 204,33 with a StDev. = 2,31
The specification limits are LSL = 194 and USL = 210
This is equivalent to a process capability of 3 Sigmaq p p y g
with a cp = 1,16 and a cpk = 0.82 The evaluation of the measurement
system results in a R&R = 54,67 %
If the portion of the measurement system variation can be reduced,
the overall capability can be predicted as follows:
Gage R&R cp cpk
54,67 1,16 0,82
30 1,30 0,92
15 1,36 0,96
0 1 38 0 98
0 1,38 0,98

Still Other Statistical Indexes
The Signal-to-Noise Ratio (S/N Ratio) relates the product variation to the
measurement system variation. The S/N Ratio should be as large as
possible.
S / N Ratio =
σ P
S / N Ratio =
σ MS
The Discrimination Index provides the number of divisions that the
Measurement System can accurately measure across the part (sample)Measurement System can accurately measure across the part (sample)
variation. If this index is less than 5, then it is marginal acceptable for a
study. If the index is 2, then it is equivalent to a go/no-go gage. We would
like to see a value ≥ 5 (QS 9000 recommendation)like to see a value ≥ 5 (QS 9000 recommendation).
σ p⎛
⎜
⎞
Discrim =
σ
σ
p
ms
⎛
⎝
⎜⎜
⎞
⎠
⎟ * .141ndc =
„nnumber of ddistinct ccategories“
Number of Distinct Categories (ndc)
• Assesses the sensitivity or effective resolution of a measurement system .
• How many categories of parts can the measurement system distinguish?
ndc Process control For analysis
Appropriate only, if process
Not appropriate for the
variation is very small
compared to the
specification.
assessment of process
parameter or indices.
<‐‐‐ 1 category ‐‐‐>
Control charts can be used
assessment of process
<‐ 2‐4 categories ‐>
Control charts can be used
marginal.
parameter or indices.
Only rough estimation can be
made.catego es
Control charts can be used Recommended for production
<‐ >=5 categories ‐>
without restriction. and improvement activities.
Source: QS-9000 MSA Hand Book

Effects of P/T and S/N Ratios
• The effect P/T on Cpk
– Large P/T reduces the process Cpk from the true value tog p pk
some smaller observed value.
• The effect of P/T on misclassification probabilitye e ec o / o sc ass ca o p obab y
– Large P/T increases the probability that we will misclassify
product as defective although it’s good and vice versa.g g
• The effect of S/N ratio on control chart sensitivity
Small S/N increases the time before an out of control process– Small S/N increases the time before an out-of-control process
is detected by a control chart
• The effect of the ndc (number of distinct categories)• The effect of the ndc (number of distinct categories)
– If the ndc = 2, only attribute data are available and sample
sizes must be largersizes must be larger.
– If the ndc is 4 to 10, then discrimination is finer and sample
sizes can be smaller
sizes can be smaller.
Gage R & R Study & Sample Size
• Usually perform the study with 2 – 3 persons (inspectors, appraiser)
• At least 10 samples (parts/products) are required for a studyp (p p ) q y
• Important! Take samples out of the process which covers the normal
process variation (± 3 σ). Example: The manufacturing process producesp ( ) p g p p
a material with a thickness from 200 +/- 15 µm. Samples should be taken
over the range of 185 – 215 µm, independent if the parts are within the
specification or not!specification or not!
• Caution! If you manufacture with a certain process different parts
(different products) you have to perform a Gage R&R study for each(different products) you have to perform a Gage R&R study for each
subgroup.
Example: You manufacture three different products with a thickness inExample: You manufacture three different products with a thickness in
the range from 150 – 1000 µm, with different specification windows. All
result are checked with the same gage. You should run three studies
over the product range as described aboveover the product range as described above.
If you mix these samples the %R&R value will be artificial low, that
means artificial good!
means artificial good!

Gage R & R Study & Sample Size
• Every sample will be measured by each operator 3 times (3 trials per
operator) in a random order and “blind”.
• The total effort of the measurement system study is depending from the
complexity and the costs of the analysis.
• Number of samples
– Select enough samples so thatg p
number of samples (S) x number of operators (O) > 15
– If not practical or possible, choose number of trials so that:
• if S x O < 4, trials = 6
if S x O < 8 trials = 4• if S x O < 8, trials = 4
Procedure for Performing an R&R Study
1. Use a calibrated gage.
2. Have the first operator measure all the samples in random order once.
3. Have the second operator measure all the samples in random order
once.
4. Continue until all operators have measured the samples once
(this is trial 1).
5. Repeat steps 1-3 for the required number of trials. Make sure that the
operator will not see his own results or from the others operators.
6 U i Mi it b th G R&R ( d) ith th ANOVA M th d f th6. Use in Minitab the Gage R&R (crossed) with the ANOVA Method for the
evaluation of measurement system capability.
– Repeatability (Displays the variation of the gage (equipment variation)– Repeatability (Displays the variation of the gage (equipment variation)
– Reproducibility (Displays the variation between the operators)
Standard deviations of each of the above– Standard deviations of each of the above
– %R&R and %P/T
7 A l lt d d t i f ll ti if
7. Analyse results and determine follow-up action, if any.

Example: Thickness Measurement
Trial Operator Part Thickness Dicke µm
1 Mary 1 8 203,2
1 Mary 2 8 203,2
1 Mary 3 8 203,2
We manufacture a product with a nominal1 Mary 4 8 203,2
1 Mary 5 8,2 208,28
1 Mary 6 8 203,2
1 Mary 7 8,1 205,74
We manufacture a product with a nominal
thickness of 200 µm. The specified
tolerance is + - 15 µm.
1 Mary 8 8 203,2
1 Mary 9 8,2 208,28
1 Mary 10 8,1 205,74
2 Mary 1 8 203,2
For the measurement of the thickness we
use a micrometer. Calibration of the
i t i i d 3 th2 Mary 2 8 203,2
2 Mary 3 8,1 205,74
2 Mary 4 7,9 200,66
2 Mary 5 8,2 208,28
micrometer is required every 3 month.
In order to evaluate the measurement
t bilit 10 l h b2 Mary 6 8,1 205,74
2 Mary 7 8,2 208,28
2 Mary 8 8 203,2
2 Mary 9 8,1 205,74
system capability, 10 samples have been
measured from 3 operator 3 times.
2 Mary 10 8,1 205,74
3 Mary 1 8 203,2
3 Mary 2 8 203,2
3 Mary 3 8,1 205,74
File: Thickness Measurement mtw
The format for the evaluation in Minitab. We need columns for the
lt f th l d f th t
File: Thickness Measurement.mtw
results, for the samples and for the operators.
Gage R & R Study Evaluation
Stat
>Quality tools
>Gage Study
>Gage R&R Study crossed...
Here you can document the informationHere you can document the information
regarding study and gage.
Important if you run more studies This window is to tell Minitab the
This window is to tell Minitab the
tolerance window.

Gage R&R - ANOVA-Tables
The session window shows 2 ANOVA-tables.
Two-Way ANOVA Table With Interaction
Source DF SS MS F P
Part 9 315 412 35 0457 22 3350 0 000
If there is no significant
Part 9 315,412 35,0457 22,3350 0,000
Operator 2 10,466 5,2330 3,3350 0,059
Part * Operator 18 28,244 1,5691 1,0944 0,380
Repeatability 60 86,021 1,4337
Operator x Part
interaction, Minitab is
running a second
Total 89 440,142
Alpha to remove interaction term = 0,25
g
ANOVA without the
interaction term.
Alpha to remove interaction term 0,25
Two-Way ANOVA Table Without Interaction
This gives more power
to the operator
Source DF SS MS F P
Part 9 315,412 35,0457 23,9230 0,000
Operator 2 10,466 5,2330 3,5721 0,033
evaluation, because the
DF can be distributed
more accurate.p , , , ,
Repeatability 78 114,265 1,4649
Total 89 440,142
more accurate.
Gage R&R Evaluation
• We are interested in the % Gage R&R for process
improvement efforts
Gage R&R
improvement efforts.
• We are interested in the % P/T
for acceptance of theGage R&R
Process tolerance = 30
p
measurement instrument in
production.
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 1,26116 7,5670 54,67 25,22
Repeatability 1,21035 7,2621 52,47 24,21Repeatability 1,21035 7,2621 52,47 24,21
Reproducibility 0,35440 2,1264 15,36 7,09
Operator 0,35440 2,1264 15,36 7,09
Part-To-Part 1,93163 11,5898 83,73 38,63
T t l V i ti 2 30689 13 8413 100 00 46 14Total Variation 2,30689 13,8413 100,00 46,14
Number of Distinct Categories = 2g
• Attention – watch for the small number of the Distinct
Categories! For the use of a process improvement
thi b h t b 5 t i i !
this number has to be 5 at minimum!

Graphical Evaluation
Gage name:
Reported by :
Tolerance:
Gage R&R (ANOVA) for Thickness µm
80 % Contribution
% S d
210
Date of study : Misc:
Components of Variation Thickness µm by Part
40
0
Percent
% Study Var
% Tolerance
205
200
Part-to-PartReprodRepeatGage R&R
0
e
Fred Joe Mary
10987654321
Part
210
R Chart by Operator
Thickness µm by Operator
4
2
0
SampleRange
_
R=1,439
UCL=3,705
LCL=0
MJF d
205
200
208
an
Fred Joe Mary
MaryJoeFred
Operator
208
Fred
Operator
Xbar Chart by Operator
Operator * Part Interaction
204
200
SampleMea
__
X=204,329
UCL=205,801
LCL=202,856
10987654321
204
200
Average Fred
Joe
Mary
200 10987654321
Part
Graphical Evaluation Control Chart
Reported by :
Tolerance:
Gage R&R (ANOVA) for Thickness µm
• The Range Chart shows the effects in gradual resolution.
80 % Contribution
% S d
210
Misc:
g g
• We like to see a minimum of 3 distinctive levels within the limits
40
0
Percent
% Study Var
% Tolerance
205
200
• Based on the high samplePart-to-PartReprodRepeatGage R&R
0
e
Fred Joe MaryFred Joe Mary
10987654321
Part
210
R Chart by Operator
Based on the high sample
size the X-bar chart should
show points out of the limits.
4
2
0
SampleRange
_
R=1,439
UCL=3,705
LCL=0
205
200
• This is indicating the part to
part variation.
208
an
Fred Joe MaryFred Joe Mary
MaryJoeFred
Operator
208
Fred
Operator
p
• Only a few points out of the
limits indicates that not the
204
200
SampleMea
__
X=204,329
UCL=205,801
LCL=202,856
10987654321
204
200
Average
Fred
Joe
Mary
limits indicates that not the
whole variation is assessed.
200 10987654321
Part

Interpretation of the Average (x-bar) Chart
• If the average value for each operator is different, we may
have a problem with the reproducibility.have a problem with the reproducibility.
• We want, that more average values are outside of the
control limits but for all operators equally! This indicates
more part to part variation which we like to see in a
measurement system studymeasurement system study.
• The major portion of the dots shall be outside the controlThe major portion of the dots shall be outside the control
limits!
– If this is the case and if the R-chart is in control, it indicates a
capable measurement system. In addition we will determine
how much variation (%) of the total variation is caused by thehow much variation (%) of the total variation is caused by the
measurement system.
Interpretation of the Range Chart
• You assume insufficient resolution of the measurement
system if:system if:
– the R-chart shows less than 3 distinctive levels within
th t l li itthe control limits
– more than 4 levels for the Range available, but moreg ,
than 1/4 of the values are 0
• The repetition precision is questionable if the R chart• The repetition precision is questionable, if the R-chart
displays events outside the control limits.
• If the range of one operator is out of control, but not for the
others, than the method is questionable
• If the range for all operators out of control the system is
sensitive against the methodology of the operators
sensitive against the methodology of the operators

Interpretation of the Variation Causes
Gage name:
Reported by :
Tolerance:
Gage R&R (ANOVA) for Thickness µm• This bar chart helps to understand the variation blocks of the study.
• Because of the bad resolution it is recommended to look to the numbers
80 % Contribution
210
Gage name:
Date of study :
Tolerance:
Misc:
in the session window.
80
40
Percent
% Study Var
% Tolerance
210
205
200
Part-to-PartReprodRepeatGage R&R
0
Fred Joe Mary
10987654321
200
Part
210
R Chart by Operator
E h t i t d b4
2
0
SampleRange
_
R=1,439
UCL=3,705
LCL=0
210
205
200
• Each operator is represented by one
line.
• Crossing lines are indicating a Interpretation of the Interaction0 LCL=0
208
n
Fred Joe Mary
MaryJoeFred
200
Operator
208 Operator
• Crossing lines are indicating a
significant interaction.
• We like to see parallel lines for all
Interpretation of the Interaction
204
SampleMean
__
X=204,329
UCL=205,801
LCL=202,856
204
200
Average Fred
Joe
Mary
We like to see parallel lines for all
operators.
• Significant interactions need to be
200 10987654321
Part
g
investigated and eliminated.
The Gage Run Chart
A nice tool to understand Operator-Sample-Trial-Interaction is the
Gage Run Chart.
Stat
>Quality tools
>Gage Study>Gage Study
>Gage Run Chart...

The Gage R&R Run Chart
Gage name:
Date of study :
Reported by :
Tolerance:
Misc:
Gage Run Chart of Thickness µm by Part, Operator
1 2 3 4 5 O t
Date of study : Misc:
210
207
1 2 3 4 5 O perator
Mary
Fred
Joe
nessµm
Mean 204
201
6 7 8 9 10
Thickn
210
207
6 7 8 9 10
204
201
Mean
Operator
Panel variable: Part
This diagram shows the mean for each sample by “Trial by Operator". Watch
This diagram shows the mean for each sample by Trial by Operator . Watch
for anomalies! – we want to see flat lines at equal values for all operators!
Create a Gage R&R Study Worksheet
Stat
>Quality tools
>Gage Study
>Create Gage R&R Study Worksheet...

Gage Study: Example
After the defined actions had been completed, a new
gage study has been conducted.
File: New Thickness.mtw
Please evaluate the study individually.
- How are the results differing?
- Interpret the graphical evaluations.
Investigation Report Gage R&R Study
Every measurement system study shall be documented
together with the results The following information shalltogether with the results. The following information shall
be included:
T t– Target
– Description of the gage / the measurement system
– Process- / product description and specification
P d t f th t d– Procedure to perform the study
– Results:
• Calibration, Accuracy, Precision, P/T and % R&R
C l i– Conclusions
– Recommendations for improvements

Measurement System Analysis Questions
• Have you picked the right measurement system? Is this measurement
system associated with either critical inputs or outputs?
• What do the precision, accuracy, tolerance, P/T ratio, %R&R and trend
chart look like?
• What are the sources of variation and what is the measurement error?
• What needs to be done to improve this system?What needs to be done to improve this system?
• Have we informed the right people of our results?
• Who owns this measurement system?
• Who owns trouble shooting?g
• Does this system have a control plan in place?
• What’s the calibration frequency? Is that frequent enough?
• Do identical systems match?
Summary
• It is very important to know about the measurement system BEFORE we
start process improvement activities.
• Be careful in the study planning process and develop an adequate
sample plan.
• Analyse the measurement system regarding the effects of operators,
parts and trials.
• Take care that the measurement system has a good discrimination.
• Run always a study to document the status for variation caused byRun always a study to document the status for variation caused by
repeatability and reproducibility.
• Measuring is a process itself It covers more than just the gage Try toMeasuring is a process itself. It covers more than just the gage. Try to
understand the reasons for variation of the process and minimise it.

Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Measurement System Analysis

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Measurement System Analysis

Similaire à Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Measurement System Analysis (20)

Plus de J. García - Verdugo

Plus de J. García - Verdugo (20)

Dernier

Dernier (20)

Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Measurement System Analysis