Top 10 Mistakes of Hypothesis Tests

Summary:
Know your enemy! These are the most common mistakes students
make in their hypothesis tests on quizzes. Know the right things to do
instead!

10. Hypothesis missing μ or p or has the wrong one

It’s really not hard if you just think about your data.
Numeric data have means μ; binomial data have percents or
proportions p.

9. Incorrect TI-83/84 inputs

If your H0 has “25” in it, that’s what you should put
for μo. If your H1 or Ha has “≠” in it,
that’s what should go in the hypothesis on your TI screen.

And hey! write down all your inputs.

8. H1 contains = or H0 doesn’t

H0 comes first and it must contain an = sign.
(Some books use ≥, =, and ≤ in H0.)
Ha or H1 comes second and it must contain >,
≠, or <.

7. H1 has > or < instead of ≠

If the problem asks you whether something “is” a number or whether
two things are “different”, you need to test = and ≠ in your
hypotheses. Don’t make assumptions that only > or < matters.

There’s no cool memory trick, because every problem is
worded differently. Just make it a habit to read each
problem carefully and notice whether it’s asking for a two-tailed test
(≠) or a one-tailed test (> or <).

Another common problem is misreading “at least” as ≤
instead of the correct ≥, or misreading “no more than” as ≥
instead of the correct ≤. The Symbol
Sheet has some common phrases for the inequalities, but again the
best practice is just to read the problem carefully and think about
what you’re writing.

6. Sample data in hypotheses

The hypotheses must always contain the number that’s part of the
claim, never any number from the sample.

Think about it logically! You’re not testing the
sample — you know the sample. You’re testing
whether something is true about the general population that your
sample came from.

A related problem is picking <, ≠, or > for your
H1 by looking at the sample data. Again, remember that
everything about the hypotheses is based on what you want to
know, not on the data you actually find.

5. Using a z test instead of a t test

Be very sure you know the population standard
deviation σ before you use a z test. If you don’t know
the standard deviation of the population, you can’t use
z and you must use t.

Again, no magic bullet here. You need to read the problem
carefully.

4. Failing to check requirements

Our procedures need the
sampling distribution to be normal, and you’ve learned procedures or
rules of thumb to test that. If you don’t make the test, or if the
data don’t pass the test, you can’t use a z test or t
test.

3. Misreading a small p-value

When the p-value is small, your calculator may
show it in scientific notation, such as 7.7321E-5. (This is how the
calculator displays 7.7321×10-5.)
Don’t pull a boneheaded move and write
p-value = 7.7321.
Like any probability, a p-value can’t be > 1.
And if you write that, you fail to reject the H0 that you should
reject.

2. Comparing p to α wrong

I see a lot of papers with α = 0.01 and
p = 0.0275, then “p < α”. Everything
else is worthless if you get this comparison backward!

Some students write the values of p and α above the
symbols, or next to them: “p > α
(0.0257 > 0.01)”.
That’s perfectly acceptable, and it can
help you make the comparisons correctly.

And the #1 mistake of hypothesis testing ...

1. Reaching a conclusion when p > α

When p > α, you fail to reject H0
(and you don’t even mention H1). No
conclusion is possible when p > α. If
H0 was “the machine is okay” and H1 was “the
machine is broken”, your only possible conclusion is

We can’t tell, at the ____ level of significance,
whether the machine is okay or broken.

When p > α you have to write your
conclusion in neutral language, not leaning one way or the
other.

Don’t say the machine “might” be anything, or “could” be
anything. And especially don’t say “we can’t prove it’s broken” or “we
can’t prove it’s okay.” Both of those are true, but they’re only half
the truth and they lead the reader to a wrong conclusion. (The most
effective way to lie is to tell only part of the truth.)