How to Compute the Power of a Hypothesis Test

Specify the critical parameter value. The critical parameter value is an
alternative to the value specified in the null hypothesis. The difference
between the critical parameter value and the value from the null hypothesis is called the
effect size. That is, the effect size is equal to the critical
parameter value minus the value from the null hypothesis.

Compute power. Assume that the true population parameter is equal to the
critical parameter value, rather than the value specified in the null hypothesis. Based
on that assumption, compute the probability that the sample estimate of the
population parameter will fall outside the region of acceptance. That
probability is the power of the test.

The following examples illustrate how this works. The first example involves a
mean score; and the second example, a proportion.

Sample Planning Wizard

The steps required to compute the power of a hypothesis test are not trivial.
They can be time-consuming and complex. Stat Trek's
Sample Planning Wizard does this work for you - quickly, easily, and
error-free. In addition to constructing a confidence interval, the Wizard
creates a summary report that lists key findings and documents analytical
techniques. Whenever you need to construct a confidence interval, consider
using the Sample Planning Wizard. The Sample Planning Wizard is a premium tool
available only to registered users. >
Learn more

Example 1: Power of the Hypothesis Test of a Mean Score

Two inventors have developed a new, energy-efficient lawn mower engine. One
inventor says that the engine will run continuously for 5 hours (300 minutes)
on a single gallon of regular gasoline. Suppose a random sample of 50 engines
is tested. The engines run for an average of 295 minutes, with a standard
deviation of 20 minutes. The inventor tests the null hypothesis that the mean
run time is 300 minutes against the alternative hypothesis that the mean run
time is not 300 minutes, using a 0.05 level of significance.

The other inventor says that the new engine will run continuously for only 290
minutes on a gallon of gasoline. Find the power of the test to reject the null
hypothesis, if the second inventor is correct.

Solution: The steps required to compute power are presented below.

Define the region of acceptance. In a previous lesson, we
showed that the region of acceptance for this problem consists of the values
between 294.45 and 305.55 (see previous
lesson).

Specify the critical parameter value. The null hypothesis tests the
hypothesis that the run time of the engine is 300 minutes. We are interested in
determining the probability that the hypothesis test will reject the null
hypothesis, if the true run time is actually 290 minutes. Therefore, the
critical parameter value is 290. (Another way to express the critical parameter value is through
effect size. The effect size is equal to the critical parameter value minus the
hypothesized value. Thus, effect size is equal to 290 - 300 or -10.)

Compute power. The power of the test is the probability of
rejecting the null hypothesis, assuming that the true population mean is equal
to the critical parameter value. Since the region of acceptance is 294.45 to 305.55, the
null hypothesis will be rejected when the sampled run time is less than 294.45
or greater than 305.55.

Therefore, we need to compute the probability that the sampled run time will be
less than 294.45 or greater than 305.55. To do this, we make the following
assumptions:

The sampling distribution of the mean is normally distributed. (Because the
sample size is relatively large, this assumption can be justified by the
central limit theorem.)

The
standard error of the sampling distribution is 2.83. The standard
error of the sampling distribution was computed in a previous lesson (see
previous lesson).

Given these assumptions, we first assess the probability that the sample run
time will be less than 294.45. This is easy to do, using the
Normal Calculator. We enter the following values into the calculator:
value = 294.45; mean = 290; and standard deviation = 2.83. Given these inputs,
we find that the cumulative probability is 0.94207551. This means the
probability that the sample mean will be less than 294.45 is 0.942.

Next, we assess the probability that the sample mean is greater than 305.55.
Again, we use the Normal Calculator. We
enter the following values into the calculator: normal random variable = 305.55; mean = 290; and
standard deviation = 2.83. Given these inputs, we find that the probability
that the sample mean is less than 305.55 (i.e., the cumulative probability) is
0.99999998. Thus, the probability that the sample mean is greater than 305.55
is 1 - 0.99999998 or 0.00000002.

The power of the test is the sum of these probabilities: 0.94207551 +
0.00000002 = 0.94207553. This means that if the true average run time of the
new engine were 290 minutes, we would correctly reject the hypothesis that the
run time was 300 minutes 94.2 percent of the time. Hence, the probability of a
Type II error would be very small. Specifically, it would be 1 minus 0.942 or
0.058.

Example 2: Power of the Hypothesis Test of a Proportion

A major corporation offers a large bonus to all of its employees if at least 80
percent of the corporation's 1,000,000 customers are very satisfied. The
company conducts a survey of 100 randomly sampled customers to determine
whether or not to pay the bonus. The null hypothesis states that the proportion
of very satisfied customers is at least 0.80. If the null hypothesis cannot be
rejected, given a significance level of 0.05, the company pays the bonus.

Suppose the true proportion of satisfied customers is 0.75. Find the power of
the test to reject the null hypothesis.

Solution: The steps required to compute power are presented below.

Define the region of acceptance. In a previous lesson, we
showed that the region of acceptance for this problem consists of the values
between 0.734 and 1.00. (see previous
lesson).

Specify the critical parameter value. The null hypothesis tests the
hypothesis that the proportion of very satisfied customers is 0.80. We are
interested in determining the probability that the hypothesis test will reject
the null hypothesis, if the true satisfaction level is 0.75. Therefore, the
critical parameter value is 0.75. (Another way to express the critical parameter value is through
effect size. The effect size is equal to the critical parameter value minus the
hypothesized value. Thus, effect size is equal to [0.75 - 0.80] or - 0.05.)

Compute power. The power of the test is the probability of
rejecting the null hypothesis, assuming that the true population proportion is
equal to the critical parameter value. Since the region of acceptance is 0.734 to 1.00,
the null hypothesis will be rejected when the sample proportion is less than
0.734.

Therefore, we need to compute the probability that the sample proportion will
be less than 0.734. To do this, we take the following steps:

Assume that the sampling distribution of the mean is normally distributed.
(Because the sample size is relatively large, this assumption can be justified by
the central limit
theorem.)

Assume that the mean of the
sampling distribution is the critical parameter value, 0.75. (This assumption is
justified because, for the purpose of calculating power, we assume that the
true population proportion is equal to the critical parameter value. And the mean of all
possible sample proportions is equal to the population proportion. Hence, the
mean of the sampling distribution is equal to the critical parameter value.)

Compute the standard deviation of the sampling distribution.
In a
previous lesson,
we showed that the standard deviation of the sample estimate of a
proportion σP is:

σP = sqrt[ P * ( 1 - P; ) / n ]

where P is the population proportion and n is the
sample size. Therefore,

σP = sqrt[ ( 0.75 * 0.25 ) / 100 ] = 0.0433

Following these steps, we can assess the probability that the probability that
the sample proportion will be less than 0.734. This is easy to do, using the
Normal Calculator. We enter the following values into the calculator:
normal random variable = 0.734; mean = 0.75; and standard deviation = 0.0433. Given these inputs,
we find that the cumulative probability is 0.36. This means that if the true
population proportion is 0.75, then the probability that the sample proportion
will be less than 0.734 is 0.36. Thus, the power of the test is 0.36, which
means that the probability of making a Type II error is 1 - 0.36, which
equals 0.64.