Earlier in this chapter, you saw how researchers can
do certain things in an effort to see whether a statistically significant
finding is also meaningful in a practical sense. Unfortunately, many researchers
do not rely on computed effect size indices
or power analyses to help them avoid the mistake of "Making a mountain
out of a molehill." They simply use the six-step version of hypothesis
testing and then get excited if the results are statistically significant.

Having results turn out to be statistically significant
can cause researchers to go into a trance in which they willing allow
the tail to wag the dog. This is what happened, I think, to
the researchers who conducted a study comparing the attitudes of two
groups of women. In their technical report, they first indicated that
the means turned out equal to 67.88 and 71.24 (on a scale that ranged
from 17 to 85) and then stated, "Despite the small difference in means,
there was a significant difference."

To me, the final 11words of the previous paragraph
conjure up the image of statistical procedures functioning as some kind
of magic powder that can be sprinkled on one's data and transform a molehill of
a mean difference into a mountain
that deserves others' attention. However, statistical analyses lack that
kind of magical power Had the researchers who obtained those means of
67.88 and 71.24 not been blinded by the allure of statistical significance,
they would have focused their attention on the
small difference and not the significant difference. And had they done
this, their final words might have been, "Although there was a significant
difference, the difference in means was small."