Inferences about Two Means: Dependent Samples

In all study designs it is always critical to clearly identify whether samples we compare come from dependent or independent populations. There is a general formulation for the significance testing when the samples are independent. The fact that there may be uncountable many different types of dependencies prevents us from having a similar analysis protocol for all dependent sample cases. However, in one specific case - paired samples - we have the theory to generalize the significance testing analysis protocol. Two populations (or samples) are dependent because of pairing (or paired) if they are linked in some way, usually by a direct relationship. For example, measure the weight of subjects before and after a six month diet.

Paired Designs

These are the most common Paired Designs where the idea of pairing is that members of a pair are similar to each other with respect to extraneous variables.

Recall that for a random sample {} of the process, the population mean may be estimated by the sample average, .

The standard error of is given by .

Analysis Protocol for Paired Designs

To study paired data we would like to examine the differences between each pair. Suppose {} and {} represent to paired samples. The we want to study the difference sample {}. Notice the effect of the pairings of each and .

Now we can clearly see that the group effect (group differences) are directly represented in the {di} sequence. The one-sample T test is the proper strategy to analyze the difference sample {di}, if the and samples come from Normal distributions.

Because we are focusing on the differences, we can use the same reasoning as we did in the single sample case to calculate the standard error (i.e., the standard deviation of the sampling distribution of ) of .

The interval estimation of the difference of two means (or Confidence intervals) is constructed as follows. Choose a confidence level (1 − α)100%, where α is small (e.g., 0.1, 0.05, 0.025, 0.01, 0.001, etc.). Then a (1 − α)100% confidence interval for μ1 − μ2 is defined in terms of the T-distribution:

Both the confidence intervals and the hypothesis testing methods in the paired design require Normality of both samples. If these parametric assumptions are invalid we must use a not-parametric (distribution free test), even if the latter is less powerful.

Test Statistics

If the two populations that the {} and {} samples were drawn from are approximately Normal, then the Test statistics is:

.

Effects of Ignoring the Pairing

The SE estimate will be smaller for correctly paired data. If we look at the within each sample at the data we notice variation from one subject to the next. This information gets incorporated into the SE for the independent t-test via s1 and s2. The original reason we paired was to try to control for some of this inter-subject variation, which is not of interest in the paired design. Notice that the inter-subject variation has no influence on the SE for the paired test, because only the differences were used in the calculation. The price of pairing is smaller degrees of freedom of the T-test. However, this can be compensated with a smaller SE if we had paired correctly.

Pairing is used to reduce bias and increase precision in our inference. By matching/blocking we can control variation due to extraneous variables. For example, if two groups are matched on age, then a comparison between the groups is free of any bias due to a difference in age distribution. Pairing is a strategy of design, not an analysis tool. Pairing needs to be carried out before the data are observed. It is not correct to use the observations to make pairs after the data has been collected.

Example

Suppose we measure the thickness of plaque (mm) in the carotid artery of 10 randomly selected patients with mild atherosclerotic disease. Two measurements are taken, thickness before treatment with Vitamin E (baseline) and after two years of taking Vitamin E daily. Formulate testable hypothesis and make inference about the effect of the treatment at α = 0.05.

Inference

Null Hypothesis: Ho:μbefore − μafter = 0

(One-sided) alternative Research Hypotheses: H1:μbefore − μafter > 0.

Test statistics: We can use the sample summary statistics to compute the T-statistic:

.

p − value = P(T(df = 9) > To = 5.4022) = 0.000216 for this (one-sided) test. Therefore, we can reject the null hypothesis at α = 0.05! The left white area at the tails of the T(df=9) distribution depict graphically the probability of interest, which represents the strenght of the evidence (in the data) against the Null hypothesis. In this case, this area is 0.000216, which is much smaller than the initially set Type I error α = 0.05 and we reject the null hypothesis.

Conclusion

These data show that the true mean thickness of plaque after two years of treatment with Vitamin E is statistically significantly different than before the treatment (p =0.000216). In other words, vitamin E appears to be an effective in changing carotid artery plaque after treatment. The practical effect does appear to be < 60 microns; however, this may be clinically sufficient and justify patient treatment.

Paired test Validity

Both the confidence intervals and the hypothesis testing methods in the paired design require Normality of both samples. If these parametric assumptions are invalid we must use a not-parametric (distribution free test), even if the latter is less powerful.

The plots below indicate that Normal assumptions are not unreasonable for these data, and hence we may be justified in using the one-sample T-test in this case.

Paired vs. Independent Testing

Suppose we accidentally analyzed the groups independently (using the independent T-test) rather than using this paired test (this would be an incorrect way of analyzing this before-after data). How would this change our results and findings?