Multiple Comparisons

David C. Howell

The material in the section on the one-way analysis of variance
generally offers good opportunities to look at multiple comparison techniques at the same
time.

I was asked a question about unequal sample sizes with multiple
comparisons, and could not help writing a new web page. Most of that
material is in the text, but it presented slightly differently here. The
link is Unequal n's and mult. comp.

A common question concerns the use of multiple comparison techniques when
the variable in question is a repeated measures variable. I have addressed
this problem in Multiple Comparison Procedures with Repeated Measures.
I argue that in most cases you do not want something like a traditional
Tukey test, although it is possible to accomplish that if it is really
necessary.

The example on the sampling distribution of F
allows you to change the means of the populations (and especially the spacing of those
means within the range from lowest to highest) and note the effects on multiple comparison
tests.

An example of multiple comparison procedures with unequal n's and unequal variances can
be found at SmokingPreg.html. It
is based on a study by Solomon, Secker-Walker, Skelly, and Flynn (1996) on reducing
smoking in pregnant women. It involves substantial hand calculation.

An interesting example of a trend analysis is based on a modification of a study by
Nolen-Hoeksema and Morrow (1991), and looks at stress as a function of time after a major earthquake.

A great example about the practical implications of power analysis can be
found at Utts-Salaries.html.
Utts presents an example of a situation where, because of low power, an
nonsignificant difference can still be important. I think that this example
has something to say about multiple comparisons, but not necessarily in the
way that we usually discuss such comparisons.