N of 1 Nutrition Part 4: The Elephant in the Room

Nutritional epidemiology has many shortcomings when it comes to acting as a basis for public health nutrition policy. But you don’t have to take Walter Willett’s word for it. Apart from the weaknesses in the methodology, there is one great big elephant in the nutrition epidemiology room that no one really wants to talk about: our current culture-wide “health prescription.”

You don’t have to care about or read about nutrition to know that “fat is bad” and “whole grains are good” [1,2]. Whether or not you follow the nutrition part of the current “health prescription” is likely to depend on a host of other factors related to general “health prescription” adherence, which in turn may have a much larger impact on your health than your actual nutritional choices. This is especially true because variation in intake and/or variation in risk related to intake are frequently quite small.

For example, in a study relating French fry consumption to type 2 diabetes, the women who ate the least amount of French fries ate 0 servings per day while the women who ate the most ate 0.14 servings per day or about 5 French fries per day (i.e. not a big difference in intake) [3]. The risk of developing type 2 diabetes among 5-fries a day piggies was observed to be .21 times greater than the risk among the no-fry zone ladies (i.e. not a big variation in risk).

Okay, everyone knows that French fries are “bad for you.” But these ladies ate them anyway. Were there other factors related to general “health prescription” adherence which may have had an impact on their risk of diabetes?

The French fry eaters also “tended to have a higher dietary glycemic load and higher intakes of red meat, refined grain, and total calories. They were more likely to smoke but were less likely to take multivitamins and postmenopausal hormone therapy.” (They also exercised less.) In other words, the French fry eaters, within a context of a known “health prescription” had chosen to ignore a number of healthy lifestyle recommendations, not just the ones related to French fries.

If you think of our current default diet recommendation as the “placebo” (although its effects may not be exactly benign), it is clear that people who fail to comply with dietary prohibitions against red meat, saturated fats, and “junk” food like French fries may also be more likely to have other poor self-care habits, like smoking and not exercising. That poor health care habits are related to poor health is of no surprise to anyone.

Statistical people

In their statistical manipulation of a dataset, nutritional epidemiologists attempt to “control” for confounding variables (confounders), such as differences in health behavior. A confounder is something that may be related to both the hypothesized cause under investigation (i.e. French fry eating) and the outcome (i.e. type 2 diabetes). As such, it muddies the water when you are trying to figure out exactly what causes what.

When statisticians “control” or “adjust” for these confounders in a data set, they essentially “pretend” (that’s the exact word my biostats professor used) that the other qualities that any given individual brings to a data set are now equalized and that the specific factor under investigation—diet—has been isolated. Well, it has and it hasn’t. The “statistical humans” created by computer programs that now have equalized risk factors are a mirage; these people do not exist. The people who contributed the data that ostensibly demonstrates that “French fries increase risk of type 2 diabetes” are the exact same people who had other behaviors that may also contribute to increased risk of diabetes. (Please note: I chose this example, rather than “red meat causes heart disease” because there are many plausible explanations for French fries causing type 2 diabetes, it is just that you aren’t going to find evidence for them using this approach.)

Most nutritional epidemiology articles contain some version the following statement in their conclusions:

“We cannot rule out the possibility of unknown or residual confounding.”

Meaning: We can not rule out the possibility that our results can be explained by factors that we failed to fully take into account. Like the elephant in the room.

That this is actually the case becomes apparent when hypotheses that seem iron-clad in observational studies are put to the test in experimental conditions.

Lack of experimental confirmation

If ever there was a field about which you could say “for every study there is an equal and opposite study,” it is nutritional epidemiology–although experimental results are generally considered “more equal” than observational data. Associations that link specific nutrients to the prevention of specific diseases can be (relatively) strong and consistent in the context of nutritional epidemiology observational data, but absent in experimental situations. Epidemiological studies suggested that beta carotene could prevent cancer; experimental evidence suggested just the opposite and in fact, smokers given beta carotene supplements had increased risk of cancer [6]. Epidemiological studies suggest that low-fat, high-carb diets are related to a healthy weight. This may be the case, but experimental evidence shows that reducing carbs and increasing fat is more effective for weight loss [7, 8]. In one study, when experiment participants added carbs back into their diet (the increase in calories from 2 months to 12 months is entirely accounted for–and then some–by carbohydrate), they regained the weight they had lost.*

Data from [7]

Kenneth Rothman, in his book Epidemiology: An Introduction, emphasizes the importance of applying Karl Popper’s philosophy of refutationism to epidemiology:

“The refutationist philosophy postulates that all scientific knowledge is tentative in that it may one day need to be refined or even discarded. Under this philosophy, what we call scientific knowledge is a body of as yet unrefuted hypotheses that appear to explain existing observations.” [9]

Rothman makes the point that there is an asymmetry when it comes to refuting hypotheses based on observations: a single contrary observation carries more weight in judging whether or not a hypothesis is false than a hundred observations that suggest that it is true.

If the current nutrition paradigm needs to be “refined or even discarded,” how will we acquire the knowledge we need to create a better system? How can we move away from “statistical people” towards a perspective that encompasses the individual variations in genetics, culture, and lifestyle that have such a tremendous impact on health?

Tune in next time for the final episode of N of 1 nutrition when I ask the all-important question: What the heck does n of 1 nutrition have to do with public health?

*This doesn’t mean that carbs are evil–some of my best friends are carbs–but that the conditions in a population that are associated with a healthy weight and the conditions in an experiment to that lead to increased weight loss are very different.

11 thoughts on “N of 1 Nutrition Part 4: The Elephant in the Room”

I love the asterisk – “the conditions in a population that are associated with a healthy weight and the conditions in an experiment to that lead to increased weight loss are very different.” I’ve always understood intuitively that just because healthy people do x thing, it doesn’t mean that doing x thing will make an unhealthy person healthy (or substitute anything else for “healthy”). I see attempts to link behaviors and outcomes so often in diet/food commercials: “women who eat Special K for breakfast have smaller waist measurements than women who don’t” or something along those lines.

There are so many aspects that don’t make sense – I’ve spent the last 10 years mimicking the supposed behavior of normal-weight people, and it hasn’t caused me to be normal weight. At the same time, when I observe my friends and acquaintances who ARE normal weight, I see that they eat junk food and fast food, rarely eat anything like vegetables or oatmeal, drink a lot of soda, smoke cigarettes for breakfast, and exercise one week per year right before they go on a beach vacation. (I’m kind of mixing up “health” and weight, but you see what I mean.). If that isn’t a case of “other factors” being involved, I don’t know what is.

If a+b=c, then c-a=b and c-b=a *
this seems to be not necessarily true when it comes to nutrition.

LOL, That’s probably as science-y as anybody needs to get! Equating a relationship with cause & effect is a such common and misleading occurrence in reporting nutrition findings–I literally wish there were a law against it (and I’m not a big fan of laws in general, plus my husband would say that you can’t legislate intelligence). Proving true cause & effect in chronic disease and obesity is so complicated & not even necessarily helpful. We’ve got to learn to do a better job of getting objective information to people so they can figure it out for themselves. Getting health claims off of boxes of cereal would be an excellent place to start.

LOL that is TOO funny! I’d been working on this post for a week–when I start writing about the flaws in nutrition epi, I get carried away–and finally managed to get out there. We must be channeling the same cosmic force: “Why look! Elephant!”

And yes, you are right about “controlling.” I kept waiting for this concept to make sense to me in class, and when my professor finally used the word “pretend,” it did.

so funny, Sunday I checked to see if you had posted something new. Oh, must be busy….. I read something else into your prof’s comment. As in, “lets pretend that we are really dealing with a fully-populated orthogonal design space here, with an actual design that will allow us to deal with each factor independent of the other.” Fat chance to get a “design” like that unless it is actually designed that way.

What I kept running across was that nutrition epidemiologists use data from studies that weren’t even designed to investigate the factor of interest they were trying to look at, much less separate that factor out from the factors with which it would be intertwined.