False Positives in Statistical Process Control

Hello good people of the world! This post is about Statistical Process Control (SPC), and what false positives, or type I errors, mean for SPC results.

A false positive or type I error may be simply described as an error where a correlation is believed to be seen, although it does not actually exist. In statistics, this is called falsely rejecting the null hypothesis. Details can be found, for instance, on Wikipedia.

In SPC, a type I error is concluding a process is out of control when it is not.

Continued Process Validation, as defined in the FDA’s 2011 guidance on Process Validation (available for download here), includes statistical methods that may be subject to type I errors, particularly SPC control charts.

The problem is when using multiple univariate control charts, each with limit of 3x the standard deviation, the probability of getting a type I error can increase rather quickly. For example, with one control chart, the probability is 0.27%, but with 10 variables the overall probability quickly increases to 2.67%, meaning you’re going to have a type I error in one out of every 37 runs. That could mean once a month or more your Quality Unit is chasing it’s tail, trying to find the cause of the out-of-specification result, when the reality is simply type I error.

How to improve? Here are some ways:

Set an overall limit to something small (e.g. 0.27%)

Use control charts that require small size shifts for detection (e.g. CUSUM charts)

Use multivariate analysis instead of univariate – critical if variables are not independent

Do not trigger a Quality System event for every out-of-specification result – have a intermediate process in place to avoid overburdening the Quality Unit

What methods do you use for Statistical Process Control? What insight has your SPC program revealed about your process? Leave a comment below and please share this post with whomever you think would benefit.