If P is high, should you pool?

In the nested t test example, the P value for Rats(Treatment) was 0.0239. Using the usual definition of statistical significance as P<0.05, you can reject the null hypothesis that all the rats with each treatment group have the same mean.

In the nested one-way ANOVA example, the P value for Herd(Treatment) is 0.1231. Since this is greater than the traditional threshold of 0.05, you cannot reject the null hypotheses that all the herds within each treatment group have the same mean packed cell volume.

Because that P value is "high", should you conclude that there is no difference between the herds, so pool the data and run a regular t test? Difficult question.

•The attraction of this approach is that the confidence interval for the difference between means will be narrower and the P value will be smaller.

•The problem with this approach is that a high P value does not prove the herds (for this example) have identical means, it just says you don't have strong evidence that the means are not identical.

•Some statisticians suggest never pooling, seeing pooling as essentially a trick to get a lower P value for the main comparison. Others statisticians cautiously recommend pooling, but only when the P value for the nested factor is quite high (perhaps greater than 0.25 or even greater than 0.75).