Saturday, September 27, 2014

Don't waste time on a full Gage R&R study until the data says you are ready

We support a lot of small volume manufacturing facilities (delivering low rate, high complexity products), which presents different challenges for implementing Six Sigma.

When you only produce one product per day or per week, it can be difficult to gather a good statistical sample for any analysis. In addition, even if the parts were available, the measurements can be complex with numerous data points, so the time to collect the data for each sample can take from 15 minutes, up to 8 hours or longer!

I'd like to share a best practice we've discovered with Gage Repeatability and Reproducibility (R&R) studies that can help all businesses save time and money with smaller sample sizes, not just those in the low volume production businesses.

I'll assume you have some knowledge of a Gage R&R study. If not, check out this page >>>

Let's assume you need to perform a Gage R&R on a new piece of test equipment.

We recommend a study that will require at least 30 total samples in the experiment. This allows us to gather a significant number of experimental runs to understand what is happening. You may require more, but I would suggest starting with 30, and evaluating the results before adding more runs/samples.

If we have 2 technicians running the equipment (only one piece of equipment), then typically we would take 10 parts, 2 technicians and 3 repeat measurements. That is a standard Gage R&R setup. That would be 10 x 2 x 3 = 60 total samples, which exceeds our 30 sample minimum.

What if each sample takes 2 hours to complete? Our original study will take at least 120 hours. Do you think your company would let you tie up the equipment for that long, and prevent 10 parts from being shipped? Highly doubtful in a low volume environment.

However, we should try to reduce the size of our study, and use another combination of parts, technicians and repeats to get closer to 30 samples. We could select one of the following options:

  • 5 parts x 2 technicians x 3 repeats = 30 samples
  • 8 parts x 2 technicians x 2 repeats = 32 samples
  • 6 parts x 3 technicians x 2 repeats = 30 samples
  • Or any other combination you can think of...

Which one is best? It depends on your situation. If you have a lot of uniqueness in your parts, I would select more parts for your study. If you think technicians may be driving variation, you should try to find a 3rd technician to include. If you suspect repeatability issues, then more repeat measurements may be preferred. This is where the expertise of the technicians, managers, engineers and experts can assist.

Let's select the option with 5 parts, 2 technicians and 3 repeats. Even though we have reduced the study down to 30 samples, it will still require 60 hours of testing to complete the Gage R&R, and we will be holding up 5 parts during that time. That's not what your production team will want to hear.

What can we do?

We suggest you conduct a partial Gage R&R, and evaluate those results before completing the full Gage R&R.

A partial Gage R&R would be a much smaller version of our full study. Instead of 5 parts, we should start with 3, and instead of 3 repeats, use only 2 repeats. We still want at least 2 parts, 2 repeats and 2 technicians as a minimum, so we get some estimates for repeatability and reproducibility.

This would create a Gage R&R study of 12 samples (3 parts x 2 technicians x 2 repeats). Now we have reduced the test time to 24 hours, and are only holding up 3 parts. Compared to the alternatives, that should sound pretty good to your production team.

How can we do this and still properly evaluate the equipment? 

The trick is that we might find enough variation in the partial Gage R&R results that we should stop the study, and go work the issues, before we complete the full Gage R&R. No sense wasting time gathering more data showing the same problem!

However, if the partial Gage R&R show favorable results (% of Tolerance and study variation below acceptable levels), then you will need to complete the full Gage R&R in order to ensure those results hold up with more parts, more technicians and/or more repeats.

The savings will come into effect only if there are problems with the measurement system. If there are no problems, then there will be no savings, as the full Gage R&R will still need to be completed. The nice thing is that you don't need to start over from scratch, you would just expand the partial study until it matched the setup of the full study. For our example, you would simply continue the study with the additional 2 parts, and add one additional repeat run to the study.

Let's see how this works with an example, using sample data from Minitab.

After running the full Gage R&R (30 samples), we come up with the following results (charts generated using the MSA Assistant feature in Minitab version 17):

gauge r&r

what is msa

The results show an unacceptable Gage R&R, as the % of Tolerance is above 30% (calculated at 59.7%), and the % of study variation is showing 63.1%. Most of the variation is coming from reproducibility (technician) at 56%.

The question we want to know is whether our partial Gage R&R study would have detected the same problems with the measurement system.

When we condense the data set down to 12 samples, and re-run the analysis, here are the results.




The results come out very similar!

% of Tolerance (Full) = 59.7%
% of Tolerance (Partial) = 64.5%

% of Study Variation (Full) = 63.1%
% of Study Variation (Partial) = 65.0%

% of Tolerance for Reproducibility (Full) =  56.0%
% of Tolerance for Reproducibility (Partial) =  63.5%

Since the measurement system contains reproducibility issues, we benefited by not running the full Gage R&R study. Now we need to investigate and resolve the technician issues, and conduct another Gage R&R when we feel those issues are resolved.

On a side note, do not run a Gage R&R unless you think it will be successful. If you know there are calibration problems, mismatching of equipment, outdated software installed, worn out parts, and differences in techniques used, then resolve those issues first. Otherwise, you'll have to re-run the study again later, after those improvements are made. The initial study would be a complete waste if it told you things you already suspected would be a problem. There are enough unknown variables inside a measurement system that you should deal with the obvious and known variables first.

After we make improvements to our measurement system, we would still need to run a partial Gage R&R the 2nd time, but again, do not complete the full Gage R&R until the partial Gage R&R shows results that are acceptable. Your 2nd study may find additional problems, or prove that the improvements were not effective, so we should stop and fix those right away.

Once you get acceptable results from the partial Gage R&R, only then should you continue to the full Gage R&R.

Bottom line, do not misread this and conclude that you only need to run a partial Gage R&R. A full Gage R&R is still needed to ensure all the variation in the additional parts and repeat measurements have been uncovered. But, until the measurement system issues are fully resolved, don't waste time doing a full study until the data tells you when you are ready to do so.

This concept can be applied to capability studies as well. Ideally, we would like to have 100-300 samples from a stable process produced from all of our sources of variation in the process, before we calculate an accurate number for capability indices (Cpk and Ppk). That may be easy to do in high volume production environments, but that is very difficult in low volume industries. Even getting a statistically valid sample of 30 is near impossible.

What we suggest is to gather the first 5 samples, and calculate capability. If the small samples show a problem (mean near the limits, large variation compared to limits), that might be enough information to dig into the problem. If the results are good (mean near the target, variation small compared to limits), then you will need to wait until at least 30 samples are generated before drawing any long term conclusions from the Cpk/Ppk values.

Remember, 30 samples is ideal, but 5 samples is better than only one sample, which is better than none at all!

In summary, in order to save time and money conducting a Gage R&R study, we suggest you follow these three recommendations:

1) Setup your full Gage R&R study to run only 30 samples - then decide if more are needed
2) Do not run a Gage R&R if you suspect it will not pass - address known issues first
3) Run a partial Gage R&R first, then if they are acceptable, complete the full study, otherwise go address the identified issues

If you'd like to learn more about Gage R&R studies, check out our training class >>>

Has anyone had experience trying this approach? We would love to hear from you! Add your comments below...

Saturday, September 20, 2014

What does golfing have to do with Six Sigma?

Here at BPI, we are always looking for ways to make Lean and Six Sigma concepts easier to understand for those getting started.

We will be creating a series of videos with examples that might resonate with you on basic terms and concepts. Feel free to use these videos in your training class, or send them to your attendees to help reinforce concepts you've already taught or coached.

The following video puts Six Sigma in terms of golfing, where the distance of the tee shot is being measured, compared to the limits of the golf hole.



The process of teeing off can be evaluated in Six Sigma terms based on how likely the shot will land between two sand traps.

We also explain how to connect these concepts back to the real world, such as in business with a forecasting budget process, with limits of +/- 10%.

Are there good examples you've heard, that you'd like us to capture in animation format? Let us know in the comments below...