1. Training Material for M EASUREMENT S YSTEM A NALYSIS

5. Introduction: What is a measurement process General Process Measurement Process Measurement: The assignment of a numerical value to material things to represent the relations among them with respect to a particular process. Measurement Process: The process of assigning the numerical value to material things. Operation Output Input Measurement Analysis Value Decision Process to be Managed

6. Introduction: What are the variations of measurement process

7. Introduction: What are the variations of measurement process Measurement(Observed) Value = Actual Value + Variance of The Measurement System 2 σ obs = 2 σ actual + σ variance of the measurement system 2

17. For extreme cases, a minimum of two appraisers can be used, but this is strongly discouraged as a less accurate estimate of measurement variation will result. 5. Let appraiser A measure 10 parts in a random order while you record the data noting the concealed marking. Let appraisers B and C measure the same 10 parts Note: Do not allow the appraisers to witness each other performing the measurement. The reason is the same as why the unit markings are concealed, TO PREVENT BIAS. 6. Repeat the measurements for all three appraisers, but this time present the samples to each in a random order different from the original measurements. This is to again help reduce bias in the measurements. Analysis Techniques: Variable Gage Analysis …… 10 Parts 3 Appraisers 3 Trials

19. Analysis Techniques: Variable Gage Analysis The average range for each operator is then computed. The average of the measurements taken by an operator is calculated. A control chart of ranges is created. The centerline represents the average range for all operators in the study, while the upper and lower control limit constants are based on the number of times each operator measured each part (trials).

20. Analysis Techniques: Variable Gage Analysis The centerline and control limits are graphed onto a control chart and the calculated ranges are then plotted on the control chart. The range control chart is examined to determine measurement process stability. If any of the plotted ranges fall outside the control limits the measurement process is not stable , and further analysis should not take place. However, it is common to have the particular operator re-measure the particular process output again and use that data if it is in-control.

21. Analysis Techniques: Variable Gage Analysis Repeatability - Equipment Variation (E.V.) The constant d 2 * is based on the number of measurements used to compute the individual ranges(n) or trials, the number of parts in the study, and the number of different conditions under study. The constant K 1 is based on the number of times a part was repeatedly measured (trials). The equipment variation is often compared to the process output tolerance or process output variation to determine a percent equipment variation (%EV).

22. Analysis Techniques: Variable Gage Analysis Reproducibility - Appraiser Variation(A.V.) X diff is the difference between the largest average reading by an operator and the smallest average reading by an operator. The constant K 2 is based on the number of different conditions analyzed. The appraiser variation is often compared to the process output tolerance or process output variation to determine a percent appraiser variation (%AV).

23. Analysis Techniques: Variable Gage Analysis Repeatability and Reproducibility( Gage R&R) The gage error (R&R) is compared to the process output tolerance to estimate the precision to tolerance ratio (P/T ratio). This is important to determine if the measurement system can discriminate between good and bad output. The basic interest of studying the measurement process is to determine if the measurement system is capable of measuring a process output characteristic with its own unique variability. This is know as the Percent R&R (P/P ratio, %R&R), and calculated as follows:

24. Analysis Techniques: Variable Gage Analysis Process or Total Variation: If the process output variation (  m ) is not known, the total variation can be estimated using the data in the study. First the part variation is determined: Rp is the range of the part averages, while K 3 is a constant based on the number of parts in the study. The total variation (TV) is just the square root of the sum of the squares of R&R and the part variation

26. Analysis Techniques: Variable Gage Analysis Part: Operator: Interaction: Repeatability:

27. Analysis Techniques: Variable Gage Analysis Total: The gage R&R statistics are then calculated as follows: Measurement Error: Part: Operator: Interaction: Reproducibility : Repeatability : Measurement Error : Total :

32. Analysis Techniques: Variable Gage Analysis 7) Calculate the goodness of fit statistic:

33. Analysis Techniques: Variable Gage Analysis 8) Determine linearity and percent linearity: Linearity = Slope x Process variation(  m ) %Linearity = 100[linearity/Process Variation] The acceptability criteria of Bias, Linearity depend on Quality Control Plan, characteristic being measured and gage speciality, suggested criteria of ESG is as following: Under 5% - acceptable 5% to 15% - may be acceptable based upon importance of application, cost of measurement device, cost of repairs, etc., Over 15% - Considered not acceptable - every effort should be made to improve the system The stability is determined through the use of a control chart. It is important to note that, when using control charts, one must not only watch for points that fall outside of the control limits, but also care other special cause signals such as trends and centerline hugging.Guideline for the detection of such signals can be found in many publications on SPC.

35. Analysis Techniques: Attribute Gage Study Acceptability criteria: If all measurement results (four per part) agree, the gage is acceptable. If the measurement results do not agree, the gage can not be accepted, it must be improved and re-evaluated. Conclusion: Because table 1 listed measurement results are not whole agreement, at part 15# and 17#, appraiser’s decisions are not agree. so the battery length gage can not be used and must be improved and re-evaluated.

38. Table 2 Filler gage measuring result

39. Table 2 Filler gage measuring result

40. Table 2 Filler gage measuring result Analysis Techniques: Attribute Gage Study In order to determine the level of agreement among the appraisers, we applied Cohen’s Kappa which is used to assess inter-rater reliability when observing or otherwise coding qualitative/categorical variables. It can measure the agreement between the evaluations of two raters when both are rating the same object.

41. Step 1. Organize the score into a contingency table. Since the variable being rated has two categories, the contingency table will be a 2*2 table: Table 3 Analysis Techniques: Attribute Gage Study A*B Cross-Tabulation Table 3

42. Analysis Techniques: Attribute Gage Study Step 2. Compute the row totals (sum across the values on the same row) and column totals of the observed frequencies. Step 3 Compute the overall total (show in the table 3). As a computational check, be sure that the row totals and the column totals sum to the same value for the overall total, and the overall total matches the number of cases in the original data set. Step 4 Compute the total number of agreements by summing the values in the diagonal cells of the table. Σa = 53+ 89 = 142 Step 5 Compute the expected frequency for the number of agreements that would have been expected by chance for each coding category. ef = = = 21.6 Repeat the formula for other cell, we got other expected count (show in the table 3). row total * col total overall total 59 * 55 150

43. Step 6 Compute the sum of the expected frequencies of agreement by chance. Σef = 21.6+57.6 = 79.2 Step 7 Compute Kappa K = = = 0.89 Step 8 Evaluate Kappa - A general rule of thumb is that values of kappa greater than 0.75 indicate good to excellent agreement; values less than 0.4 indicate poor agreement. Repeat above step, we can got following kappa measures for the appraisers: Table 4 Analysis Techniques: Attribute Gage Study Σ a- Σef N - Σef 142 - 79.2 150 - 79.2 Table 4

44. Using the same steps to calculated the kappa measure to determine the agreement of each appraiser to the reference decision: Table 5 Total summary on Table 6: Analysis Techniques: Attribute Gage Study Table 5

45. Analysis Techniques: Attribute Gage Study

46. Analysis Techniques: Attribute Gage Study The AIAG MSA reference manual edition 3 provides acceptability criteria for each appraisers results: Definition: False Alarm – The number of times of which the operator (s) identify a good sample as a bad one. Miss – The number of times of which the operators identify a bad sample as a good one.

47. Analysis Techniques: Attribute Gage Study So summarizing all the information of the example with this table: Table 7 Number of correct decisions Total opportunities for a decision Effectiveness = Number of False Alarm Total opportunities for a decision False Alarm Rate = Number of False Alarm Total opportunities for a decision Miss Rate =

48. Analysis Techniques: Attribute Gage Study Conclusion: The measurement system was acceptable with appraiser B, marginal with appraiser A, and unacceptable for C. So we shall determine if there is a misunderstanding with appraiser C that requires further training and then need to re-do MSA. The final decision criteria should be based on the impact to the remaining process and final customer. Generally, the measurement system is acceptable if all 3 factors are acceptable or marginal. Minitab also can perform attribute gage analysis, but it didn’t declare the acceptability criteria, so it is not recognized by QS9000 standard.

50. Table 8 Signal Detection Table for Filler Gage

51. Table 8 Signal Detection Table for Filler Gage

56. Analysis Techniques: Attribute Gage Study Example: We use a filler gage to measure the fitting gap between battery and hand phone which specification is 0~0.2mm. The number of accepts for each part are: Table 10 Table 10