In this chapter, the focus has been on the data-based
p-level. If the p is very small, you may be tempted to think that there
is a large effect in the data or, stated differently, that Ho
is false by a mile. Similarly, you may be tempted to think that ps that
are not small turned out that way because there was a small effect in
the data or, stated differently, because the null hypothesis (if false
at all) was false by a small amount. Resist these temptations! For any
given degree of discrepancy between the sample evidence and Ho,
there is an inverse relationship between the sample size and the data-based
p-level. For example, consider a study in which a researcher is concerned
with Pearson's product-moment correlation, conducts a two-tailed test
of Ho: r = 0.00, and finds that r = .40
for the sample. Now, if the size of that sample is 10, then p is .24.
If n = 30, then p = .03. And if n = 500, then p = .00006. Now consider
the same study but this time with a very small difference between Ho
and the sample evidence. For the case where Ho:
r = 0.00 and r = .03, the p-value will turn out to be very small if the
sample size is very large. If n = 25,000, p < .00002.