Specification Setting: Setting Acceptance Criteria from Statistics of the Data

This article shows how Probabilistic Tolerance Intervals of the form, "We are 99% confident that 99% of the measurements will fall within the calculated tolerance limits" can be used to set acceptance limits using production data that are approximately Normally distributed. If the production measurements are concentrations of residual compounds that are present in very low concentrations, it may be appropriate to set acceptance limits by fitting a Poisson or an Exponential Distribution.

The International Conference on Harmonization (ICH) Q6 defines specifications as "a list of tests, references to analytical procedures, and appropriate acceptance criteria which are numerical limits, ranges, or other criteria for the tests described."1 These numerical limits and ranges are for engineering criteria such as dimensions or hardness, performance criteria such as the quantity of drug delivered by a device, and toxicological criteria such as the maximum permitted level of a residual chemical compound. Ideally, the acceptance criteria will be set to meet well-established requirements such as those for product performance. However, we do not always have the necessary information to do this and we are often forced to establish the likely range of acceptable values to be seen in production data.

When reviewing acceptance criteria set using preproduction data, regulators tend to favor the 3-sigma limits that are used for control charts. In practice, we have found that limits set using data from a small number of batches are almost always too tight. Thus, we want to increase the limits to 3.5 or 3.8-sigma to allow for the variability of estimating sigma from small samples. This variability was recognized many years ago, as the following quote from Deming indicates:

Shewhart, of his Statistical Method from the Viewpoint of Quality Control (The Graduate School, Department of Agriculture, Washington, 1939) demonstrates how futile it is to attempt to estimate the magnitude of sigma from samples of 4, but that samples of 100 are excellent, while samples of 1000 give practically perfect results.2

This article describes how to calculate "3-sigma" acceptance criteria for sample sizes below 200. In addition to these acceptance criteria for data that can be categorized, as "Distributed like Normal," a method is described for discrete data that can be described as "Distributed like a Poisson or Exponential."

ACCEPTANCE CRITERIA FOR MEASUREMENTS "DISTRIBUTED LIKE NORMAL"

Figure 1. Histogram of 1,3-diacetyl benzene

The measured levels of residual 1,3-diacetyl benzene in rubber seals usually have an approximately Normal distribution. The histogram of the level in 62 batches of seals is shown in Figure 1.

The 62 values appear to be sampled from a Normal distribution. If we knew that the data are Normally distributed and we knew the mean and standard deviation, we could use a tolerance interval to establish the likely range of acceptable values.

Tolerance intervals are similar to the confidence intervals on the mean that many engineers and scientists use and understand. Confidence limits indicate the range that we expect to contain the mean, whereas tolerance intervals indicate the range that we expect to contain a specified percentage of the population.

If we know the mean (μ) and standard deviation (σ) of a Normal distribution, we can calculate that 2.5% of the distribution will be above 1.96 * σ and 2.5% of the distribution will be below –1.96 * σ. Thus, the lower and upper limits for a 95% tolerance interval are μ – 1.96 * σ and μ + 1.96 * σ.

Figure 2. A 95% two-sided interval

A two-sided tolerance interval is relevant if we require acceptable values to be less than an upper limit and more than a lower limit. A one-sided tolerance interval is relevant if we require acceptable values to be less than an upper limit or more than a lower limit. A 95% two-sided interval is shown in Figure 2.

The one-sided 95% intervals are similar, but we expect 5% of the values to be above the upper limit or 5% below the lower limit. For one-sided 95% intervals, the upper 95% limit is μ + 1.64 * σ and the lower 95% limit is μ – 1.64 * σ.