Hervé Abdi:

The Bonferonni and Šidák Corrections

With 3 tests, we find that the probability of not making a Type I error on all tests is:

.95 × .95 × .95 = (1 − .05)3 = (1 − α)3 .

For a family of C tests, the probability of not making a Type I error for the whole family is: (1 − α)C .

For our example, the probability of not making a Type I error on the family is

(1 − α)C = (1 − .05)10 = .599 .

Now, what we are looking for is the probability of making one or more Type I errors on the family of tests. This event is the com- plement of the event not making a Type I error on the family and therefore it is equal to

1 − (1 − α)C .

For our example, we find

1 − (1 − .05)10 = .401 .

So, with an α level of .05 for each of the 10 tests, the probability of wrongly rejecting the null hypothesis is .401.

This example makes clear the need to distinguish between two meanings of α when performing multiple tests:

•

The probability of making a Type I error when dealing only with a specific test. This probability is denoted α[PT ] (pro- nounced “alpha per test"). It is also called the testwise alpha.

•

The probability of making at least one Type I error for the whole family of tests. This probability is denoted α[PF ] (pro- nounced “alpha per family of tests”). It is also called the fam- ilywise or the experimentwise alpha.