Measurement System Analysis: Common Mistakes and How to Avoid Them

Measurement System Analysis (MSA) is the practice of evaluating whether your measurement process is capable of reliably detecting the variation you are trying to measure. Before investing in data collection and analysis, MSA asks a fundamental question: can we trust the data? If the variation in your measurements is due to the measurement system itself rather than variation in the process you are trying to understand, all downstream analysis is compromised.

The most common MSA technique is Gauge Repeatability and Reproducibility (Gauge R&R), which evaluates two components of measurement variation: repeatability (will the same person measuring the same part multiple times get the same result?) and reproducibility (will different people measuring the same part get the same result?). Here are the common mistakes teams make with MSA.

Mistake 1: Skipping MSA Because the Measurement Process Seems Obvious

Many teams skip MSA for measurements that seem straightforward -- lengths, weights, counts, times. The assumption is that if the measurement process is well-defined, it must be reliable. This is wrong. A well-defined measurement process can still have significant repeatability or reproducibility problems. A torque specification that requires a specific gauge orientation, a timing measurement that requires a judgment about when an event starts or ends, a visual inspection that requires a trained interpretation of ambiguous evidence -- all of these can have substantial measurement system variation even when the process is clearly described. Do the MSA.

Mistake 2: Using Too Few Samples or Operators

A Gauge R&R study requires a minimum of 10 samples and 2 operators (ideally 3) to produce reliable results. Teams often reduce the sample count or use a single operator to save time. A Gauge R&R study with 5 samples and 1 operator does not provide statistically reliable estimates of repeatability or reproducibility. If the results suggest the measurement system is acceptable, the conclusion cannot be trusted. Run the study properly.

Mistake 3: Misinterpreting the Results

• The standard acceptance criteria for Gauge R&R is: below 10% is acceptable, 10-30% may be acceptable depending on context, and above 30% is unacceptable. But the denominator matters. Gauge R&R percentage is calculated relative to the total observed variation (process variation plus measurement system variation). A process with very low inherent variation will have a high Gauge R&R percentage even if the measurement system is quite precise. The relevant question is whether the measurement system can distinguish between parts that are within specification and parts that are not.

• A high Gauge R&R percentage from reproducibility (different operators getting different results) is easier to fix than a high percentage from repeatability (the same operator getting different results). Reproducibility problems often indicate training or procedure issues; repeatability problems often indicate equipment calibration or condition issues.

• Do not accept a failing Gauge R&R result as evidence that the measurement system is adequate. A common rationalisation is 'the Gauge R&R is above 30% but we have been using this measurement for years.' The Gauge R&R result is evidence about the measurement system's reliability -- the length of time the system has been in use is not.

XNM applies Six Sigma analytical methods, including Measurement System Analysis and process capability assessment, to public-sector and capital-project environments. Reach out to XNM's strategic advisory team to discuss data quality and measurement system evaluation for your organisation.