The HSPH researchers argue that Flegal’s study failed to account for the influence of smoking on obesity-related mortality. They suggest that smokers are on average thinner than non-smokers, and at the same time at a much higher risk of premature death. Thus, according to the argument, by including smokers in her study, Flegal doesn’t incorporate the true health advantages of being thinner. The HSPH researchers suggest a similar line or reasoning for those with chronic diseases. These people tend to be thin, but also unhealthy. In their words:

But determining the precise range of BMI associated with lowest mortality can be difficult, because the approach that researchers use to conduct their analyses can bias their findings. One such problem is a phenomenon that researchers call “reverse causation”: Low body weight often results from chronic disease, rather than being a cause of chronic disease. The weight loss may have been unintentional as a result of the underlying disease process; or the weight loss may have been intentional, because patients with serious conditions often become motivated for the first time to lose weight. Regardless, because of this phenomenon, people with a BMI below 25 are a mix of healthy individuals and those who are ill and have lost weight due to their disease. Leaner people are also more likely to smoke than their heavier counterparts. If researchers fail to account for both reverse causation and the adverse effects of smoking, they will find artificially inflated mortality rates among lean people, thus diminishing the harmful impact of overweight and obesity.

To addresse these issues, the Harvard researchers offer three solutions. First, they suggest limiting the sample to “never smokers.” Second, they want to exclude anyone with a preexisting chronic disease. And third, they advocate excluding “early deaths” to further account for the effect of chronic disease. Dr. Flegal’s final figures do not incorporate these elements. In her original JAMA paper, Flegal explains why:

To examine whether the increased relative risks at lower BMI levels might be related to possible weight loss associated with illness and increased mortality, which could also have decreased the relative risks associated with overweight and obesity, we repeated analyses excluding the first 3 or the first 5 years of deaths and found little change in the relative risk estimates (data not shown). We also repeated analyses including only individuals who never smoked and found that the elevated relative risks for the lowest BMI category persisted and that other relative risks were not systematically different.

If these biases due to illness-induced weight loss or residual confounding by smoking or prevalent illness at baseline had been operating, one would expect to see a lower relative risk for underweight (closer to 1) and a higher relative risk for overweight and for obesity after controlling for baseline health status, smoking, and early deaths. However, the relative risks did not follow this pattern. Thus, these analyses of the effects of exclusions and of stratification by health status for the combined data set did not suggest that the results for the full data set were affected in any important or systematic way by residual confounding due to smoking or to prevalent illness at baseline. These analyses did not suggest that Flegal et al (2005) had overestimated the risks associated with underweight or underestimated the risks associated with overweight or obesity.

In other words, Flegal crunched her data just as the HSPH folks would have liked, and found that it made no real difference. She still found that being overweight or slightly obese (BMI between 30 and 35, also referred to as “grade 1 obesity”) had no impact on mortality (relative risk below 1.0):

For ages under 70, the relative risks for overweight varied from 0.40 to 0.91. For never-smokers ages 25-69 years in excellent or very good health at baseline, after excluding the first 3 years of deaths, the relative risk for overweight was 0.45 (significantly below 1.0). For grade 1 obesity for younger never-smokers, regardless of baseline health status and regardless of excluding the first 3 years of deaths, the relative risks were always below 1.0.

Is Flegal’s result the product of some “flaw,” as the HSPH academics assert? Well, research from Loyola University backs Flegal up on smoking:

As for the claim that Flegal should have excluded early deaths, a frequently-cited report surveying data from 29 other studies and more than 1.9 million subjects concluded:

…either pre-existing disease does not confound the BMI-mortality association or eliminating early deaths [as Harvard suggests] is inefficient for reducing that confounding … these results suggest that [excluding early deaths] may not be advisable because, to the extent that is has any effect at all, the magnitude of this effect is minimal, of questionable clinical significance, and of ambiguous meaning.

One obvious logical problem is that, although most people die after 75 years of age, it is cumulative obesity exposure rather than weight at a specific older age that contributes the most to the higher mortality rates associated with obesity … Thus, the relative risks calculated from the oldest age groups do not reflect the true long-term impact of obesity on mortality.

[The Harvard researchers] speculate that the number of deaths attributable to obesity in the United States may be underestimated when relative risks are calculated on the basis of current body mass index (BMI). They cite no data to support their speculations, but instead invoke the notion of “reverse causality.” They hypothesize that relative risks are lowered by obese people who become ill, lose weight because of this illness to become normal weight, and die shortly thereafter of the underlying illness, surviving just long enough to be included in the study. However, this reverse-causation hypothesis is unlikely to be the correct explanation for the lower relative risks in the elderly for 2 reasons: first because data show that exclusion of preexisting illness has little effect on relative risk estimates, and second because weight loss from obesity to normal weight is relatively uncommon.

An entirely separate argument made by the HSPH researchers is that data used in the Flegal study did not provide for a long enough follow-up time on individual subjects. They insist that a shorter follow-up period doesn’t capture the full consequences of obesity. But once again Flegal anticipated and responded to this point in her original study. While suggesting that the issue requires further examination, she wrote:

To examine whether the higher relative risks in NHANES I might be due to the longer follow-up in NHANES I, we compared the relative risks from the first phase of NHANES I through the 1982-1984 follow-up with the relative risks from NHANES II and III. Thus, the follow-up period was similar for all surveys (10 years for NHANES I, 14 years for NHANES II, 9 years for NHANES III). The NHANES I relative risks over the first 10 years of follow-up were higher in almost every BMI-age subgroup than were the relative risks from the other surveys (data not shown). Thus, even after controlling for length of follow-up, NHANES I tended to have higher relative risks than the other surveys … the relative risk for total mortality in weight-stable individuals in the latter part of the NHANES I follow-up were similar to relative risks in the earlier follow-up period.

Flegal also pointed out that even in the Allison study, which predicted a much higher risk of death from obesity, “Across the 6 cohorts used by Allison et al, there was no relation between the length of follow-up in a cohort and the relative risk in the cohort.”

In sum, the HSPH critique is speculative at best, contradicted by many previous studies, and does not apply to Flegal’s data. In her JAMA study she wrote, quite appropriately:

We undertook additional analyses to examine whether our estimates of excess deaths might have been affected by factors such as length of follow-up, weight stability, weight loss caused by illness, or smoking status … Taken together, these analyses suggest that differences in length of follow-up, weight loss because of underlying illness, or counfounding by smoking status did not have a major impact on our estimates of excess deaths.