This is discussion forum for physicians, researchers, and other healthcare professionals interested in the epistemology of medical knowledge, the limitations of the evidence, how clinical trials evidence is generated, disseminated, and incorporated into clinical practice, how the evidence should optimally be incorporated into practice, and what the value of the evidence is to science, individual patients, and society.

Thursday, April 7, 2011

I awoke this morning to a press release from the AMA, and a front page NYT article declaring that, in a post-trial follow-up of the WHI study, CEE reduces breast cancer in the entire cohort of post-hysterectomy patients, and lowers CHD (coronary heart disease) risk in the youngest age stratum studied.

One need look no further than the data in figures 2 and 5 to see that it's a Type I statistical error (a signigicant result is found by chance when the null hypothesis is true and there is in reality no effect) - that's why.

For the love of Jehovah, did this really make the headlines? The P-value for the breast cancer risk is....well, they don't give a P-value, but the upper bound of the 95% CI is 0.95 so the P-value is about 0.04, BARELY significant. Seriously? This is one of FIFTEEN (15) comparisons in Table 2 alone. Corrected for multiple comparisons, this is NOT a statistically significant effect, NOT EVEN CLOSE. I think I'm having PTSD from flashbacks to the NETT trial.

And table 5? There are TEN outcomes with THREE age strata for each outcome, so, what, 30 comparisons? And look at the width of the 95% CI for the youngest age stratum in the CHD outcome - wide. So there weren't a lot of patients in that group.And nevermind the lack of an a priori hypothesis, or a legitimate reason to think some difference based on age strata might make biological sense.

Bad old habits die hard. Like a former colleague is fond of pointing out, don't assume that because an investigator does not have drug company ties that s/he is not biased. Government funding and entire careers are at stake if an idea that you've been pursuing for years yields to the truth and dies off. Gotta keep stoking the coals of incinerated ideas long enough to get tenure!

No “legitimate reason to think some difference based on age strata might make biological sense”…really?How about CEE works for primary prevention (so prevents women from getting same risk as men if you supplement estrogen at the time of menopause) but it doesn’t work for secondary prevention (women aged 60-69 who have been exposed to 10 years of menopause)

One of the biggest criticisms of WHI was that it enrolled lots of women not getting HRT - those who were 10 years post-menopausal – as opposed to perimenopausal women.

What I don’t understand is why they used age strata – rather than time since menopause as the unit of analysis.

I thought you were the one arguing that when mortality is the outcome, any improvement matters if the intervention is cheap and statistical significance be damned….There appears to be a dose-response hereDeath and CEE group – 50-59y – HR 0.73, 60-69y HR 1.04, 70-79y HR 1.12

What this says to me is that the WHI should have been better designed and not so focused on just meeting recruitment numbers. Why in the hell were they enrolling 70 year olds (22% were 70 or older!) to start with?