1. UNCLASSIFIED / FOUO
UNCLASSIFIED / FOUO
National Guard
Black Belt Training
Module 23
Measurement
System Analysis (MSA)
Introduction
UNCLASSIFIED / FOUO
UNCLASSIFIED / FOUO

2. UNCLASSIFIED / FOUO
CPI Roadmap – Measure
8-STEP PROCESS
6. See
1.Validate 2. Identify 3. Set 4. Determine 5. Develop 7. Confirm 8. Standardize
Counter-
the Performance Improvement Root Counter- Results Successful
Measures
Problem Gaps Targets Cause Measures & Process Processes
Through
Define Measure Analyze Improve Control
TOOLS
•Process Mapping
ACTIVITIES
• Map Current Process / Go & See •Process Cycle Efficiency/TOC
• Identify Key Input, Process, Output Metrics •Little’s Law
• Develop Operational Definitions •Operational Definitions
• Develop Data Collection Plan •Data Collection Plan
• Validate Measurement System •Statistical Sampling
• Collect Baseline Data •Measurement System Analysis
• Identify Performance Gaps •TPM
• Estimate Financial/Operational Benefits •Generic Pull
• Determine Process Stability/Capability •Setup Reduction
• Complete Measure Tollgate •Control Charts
•Histograms
•Constraint Identification
•Process Capability
Note: Activities and tools vary by project. Lists provided here are not necessarily all-inclusive. UNCLASSIFIED / FOUO 2

3. UNCLASSIFIED / FOUO
Learning Objectives
 Understand the importance of good measurements
 Understand the language of measurement
 Understand the types of variation in measurement
systems
UNCLASSIFIED / FOUO 3

4. UNCLASSIFIED / FOUO
Exercise: The Three Rs
UNCLASSIFIED / FOUO 4

5. UNCLASSIFIED / FOUO
Examples
 The Hale Koa Hotel manager wants to reduce
customer check-in time
 The VA wants to reduce VA Home Loan Guarantee
Program processing errors
 The Army Community Service organization wants to
improve its customer service performance
 A VA Hospital is interested in finding ways to improve
in-patient and out-patient care
UNCLASSIFIED / FOUO 5

6. UNCLASSIFIED / FOUO
Why Is MSA Important?
 Our ability to assess the performance of a process we wish
to improve is only as good as our ability to measure it
 The measurement system is our “eyes” for our process
 We need to be able to see the performance of our
process clearly in order to improve it
 Sometimes, improving the ability to measure our
process results in immediate process improvements
Can you trust your measurements to tell you the truth?
Measurement System Analysis UNCLASSIFIED / FOUO 6

7. UNCLASSIFIED / FOUO
Sources Of Observed Process Variation
Observed Variation
Observed Variation
Actual Process Variation
Actual Process Variation Measurement Variation
Measurement Variation
- Long-term Process Variation
- Short-term Process Variation
Variance
Variance Variance
Variance
Due to Instrument
Due to Instrument Due to Operators
Due to Operators
- Repeatability - Reproducibility
- Calibration
- Stability
- Linearity
The variation due to the measurement system must be identified first,
then separated from actual process variation
UNCLASSIFIED / FOUO 7

8. UNCLASSIFIED / FOUO
Variation Is Additive
Measured values
Actual values
s2 Observed = s2 Measurement + s2 Part + s2 Error
s2 Measurement = s2 Observed – s2 Part – s2 Error
s2 Measurement = s2 Repeatability + s2 Reproducibility + s2 Error
UNCLASSIFIED / FOUO 8

9. UNCLASSIFIED / FOUO
Why Worry About Measurement Variation?
 Consider the reasons why we measure:
Assist in
Verify process How might measurement continuous
conformity to variation affect these decisions? improvement
specifications
activities
What if the amount of
Process
Process measurement variation
is unknown
Measurement
Measurement
?
Measurement variation can make our process capabilities appear worse than they are.
UNCLASSIFIED / FOUO 9

10. UNCLASSIFIED / FOUO
Measurement Variation
 Measurement Variation is broken down into two components: (The
two Rs of Gage R&R)
 Reproducibility (Equipment or Gage or Operator Variability)
 Different individuals get different measurements for the same thing
 Repeatability (Equipment or Gage or Operator Variability)
 A given individual gets different measurements for the same thing when
measured multiple times
 The tool we use to determine the magnitude of these two sources of
measurement system variation is called Gage R&R
UNCLASSIFIED / FOUO 10

11. UNCLASSIFIED / FOUO
Reproducibility (Operators’ Precision)
 Reproducibility is the variation in the average of the
measurements made by different operators using the
same measuring instrument when measuring the
identical characteristic on the same part
Inspector A
s  s s
2
m
2
g
2
o
Inspector B Inspector C
UNCLASSIFIED / FOUO 11

12. UNCLASSIFIED / FOUO
Repeatability (Gage Precision)
 Repeatability is the variation between successive
measurements of the same part, same characteristic,
by the same person using the same equipment
(gage). Also known as test /re-test error, used as an
estimate of short-term variation.
Ideal Process Target
s  s s
2
m
2
g
2
o
UNCLASSIFIED / FOUO 12

13. UNCLASSIFIED / FOUO
Measurement Error
Gage R & R variation is the percentage that Generally recognized criteria for
measurement variation (repeatability and gage acceptability is when
reproducibility) represents of the variation observed Gage R & R variability to process
in the process variability is :
Under 10%: Acceptable gage
10% to 30%: Might be
Observed Measurements acceptable
Over 30%: Gage is
unacceptable and should be
corrected or replaced
True Values Measurement Error
Bias Gage R&R Stability Discrimination Linearity
Repeatability Reproducibility
Operator Operator * Part
UNCLASSIFIED / FOUO 13

14. UNCLASSIFIED / FOUO
Bias (Instrument Accuracy)
 Bias is the difference between the observed average
value of measurements and the master value. The master
value is determined by precise measurement typically by
calibration tools linked to an accepted, traceable reference
standard.
Master Value (Reference Standard)
Average Value
UNCLASSIFIED / FOUO 14

15. UNCLASSIFIED / FOUO
Stability
 Stability = If measurements do not change or drift
over time, the instrument is considered to be stable
Time One
Time Two
UNCLASSIFIED / FOUO 15

16. UNCLASSIFIED / FOUO
Discrimination
 Discrimination is the capability of detecting small changes in the
characteristic being measured
 The instrument may not be appropriate to identify process variation or quantify
individual part characteristic values if the discrimination is unacceptable
 If an instrument does not allow differentiation between common variation in
the process and special cause variation, it is unsatisfactory
.28 .28 Ruler .28 .28
.279 .282 Caliper .282 .279
.2794 .2822 Micrometer .2819 .2791
UNCLASSIFIED / FOUO 16

17. UNCLASSIFIED / FOUO
Linearity
A measure of the difference in bias (or offset) over the range of
the sample characteristic the instrument is expected to see
determines linearity. If the bias is constant over the range of
measurements, then linearity is good.
 Over what range of values for a given characteristic can the
device be used?
 When the measurement equipment is used to measure a wide range of
values, linearity is a concern.
Measurement
Variation
Low High
End Measurement Scale End
UNCLASSIFIED / FOUO 17

18. UNCLASSIFIED / FOUO
Name That Problem!
Master Value
Instrument 1
Instrument 2
Average Value
Master Value
(Reference Standard)
Time One
Time Two
.28
1. Discrimination
.279 2. Bias/Accuracy
.2791
3. Repeatability
4. Reproducibility
Inspector A 5. Instrument Bias
Inspector B
Inspector C
6. Stability
UNCLASSIFIED / FOUO 18

19. UNCLASSIFIED / FOUO
Measurement Systems Analysis Template
The Measurement System used to collect data has been calibrated and is considered to have no potential for significant
errors. The data collection tool is reliable, can be counted on, has good resolution, shows no signs of bias and is stable.
Type of
Measurement Description Considerations to this Project
Error
The ability of the measurement Work hours can be measured to <.25
Discrimination
system to divide measurements into hours. Radar usage measure to +- 2
(resolution)
“data categories” minute.
The difference between an observed No bias - Work hours and radar start-
Bias average measurement result and a stop times consistent through
reference value population.
No bias of work hours and radar
Stability The change in bias over time
usage data.
Not an issue. Labor and radar usage
Repeatability The extent variability is consistent is historical and felt to be accurate
enough for insight and analysis.
- Example - Remarks in usage data deemed not
Different appraisers produce reproducible, therefore were not
Reproducibility
consistent results considered in determining which
radars were used in each op
Variation The difference between parts Required Deliverable process.
N/a to this
UNCLASSIFIED / FOUO 19

20. UNCLASSIFIED / FOUO
Reported by :
Gage name:
Tolerance:
Measurement Systems Analysis Template Date of study :
Misc:
Gage R&R (ANOVA) for Response
Gage R&R
Components of Variation Response by Part
%Contribution
Source VarComp (of VarComp) 100 % Contribution
% Study Var
10.00
Total Gage R&R 0.0015896 3.70
Percent
Repeatability 0.0005567 1.29 9.75
Reproducibility 0.0010330 2.40 50
Operator 0.0003418 0.79 9.50
Operator*Part 0.0006912 1.61
0
Part-To-Part 0.0414247 96.30 Gage R&R Repeat Reprod Part-to-Part 1 2 3 4 5 6 7 8 9 10
Total Variation 0.0430143 100.00 Part
R Chart by Operator
Study Var %Study Var Response by Operator
1 2 3
Source StdDev (SD) (6 * SD) (%SV) UCL=0.1073
0.10 10.00
Total Gage R&R 0.039870 0.23922 19.22
Sample Range
Repeatability 0.023594 0.14156 11.38
Reproducibility 0.032140 0.19284 15.50 _ 9.75
0.05
Operator 0.018488 0.11093 8.91 R=0.0417
Operator*Part 0.026290 0.15774 12.68 9.50
Part-To-Part 0.203531 1.22118 98.13 0.00 LCL=0
1 2 3
Total Variation 0.207399 1.24439 100.00
Operator
Xbar Chart by Operator
Number of Distinct Categories = 7 1 2 3 Operator * Part Interaction
10.00 10.00 Operator
Sample Mean
UCL=9.8422
_ 1
The Measurement _
Average
2
X=9.7996 9.75
9.75 LCL=9.7569
3
System is acceptable
9.50
with the Total Gage 9.50
1 2 3 4 5 6 7 8 9 10
R&R % Contribution Part
<10% - Example - Optional BB DeliverableUNCLASSIFIED / FOUO 20

21. UNCLASSIFIED / FOUO
Takeaways
 It is important to be able to rely on the accuracy and precision
of the measurement system to make good decisions
 Understand the various types of measurement system variation
 Eliminate as much of the variation in the measurement system
as possible to focus on and improve the true cause of variation
in process performance
UNCLASSIFIED / FOUO 21

22. UNCLASSIFIED / FOUO
What other comments or questions
do you have?
UNCLASSIFIED / FOUO 22