Friday, December 27, 2013

Type I and Type II Errors: Lay Explanation

I'm reading a book titled Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming by Naomi Oreskes and Erik M. Conway and I thought their explanation of Type I and Type II errors was particularly clear. Although somewhat long, what they wrote is re-presented below (pp. 156-7):

The 95 percent confidence level is a social convention, a value judgement. And the value it reflects is one that says that the worst mistake a scientist can make is to fool herself: to think an effect is real when it is not. Statisticians call this a type 1 error. You can think of it as being gullible, naive, or having undue faith in your own ideas. To avoid it, scientists place the burden of proof on the person claiming a cause and effect. But there's another kind of error -- type 2 -- where you miss effects that are really there. You can think of that as being excessively skeptical or overly cautious. Conventional statistics is set up to be skeptical and avoid type 1 errors. The 95 percent confidence standard means that there is only 1 chance in 20 that you believe something that isn't true. That is a very high bar. It reflects a scientific worldview in which skepticism is a virtue, credulity is not. As one web site puts it, "A type 1 error is often considered to be more serious, and therefore more important to avoid, than a type 2 error." In fact, some statisticians claim that type 2 errors aren't really errors at all, just missed opportunities.

Is a type 1 error more serious that a type 2? Maybe yes, maybe no. It depends on your point of view. The fear of type 1 errors asks up to play dumb. That makes sense when we really don't know what's going on in the world -- as in the early stages of a scientific investigation. This preference also makes sense in a court of law, where we presume innocence to protect citizens from oppressive governments and overzealous prosecutors. However, when applied to evaluating environmental hazards, the fear of gullibility can make us excessively skeptical and insufficiently cautious. It places the burden of proof on the victim -- rather than, for example, the manufacturer of a harmful product -- and we may fail to protect some people who are really getting hurt.

There are many, many statistical texts that provide mathematical and symbolic definitions of Type I and Type II errors but when a lay book nicely articulates the definitions, it is worth noting (and remembering).