That Flawed Stanford Study

I tried to ignore the month-old “Stanford study.” I really did. It made so little sense that I thought it would have little impact.

That was dumb of me, and I’m sorry.

The study, which suggested — incredibly — that there is no “strong evidence that organic foods are significantly more nutritious than conventional foods,” caused as great an uproar as anything that has happened, food-wise, this year. (By comparison, the Alzheimer’s/diabetes link I wrote about last week was ignored.)

That’s because headlines (and, of course, tweets) matter. The Stanford study was not only an exercise in misdirection, it was a headline generator. By providing “useful” and “counterintuitive” information about organic food, it played right into the hands of the news hungry while conveniently obscuring important features of organic agriculture.
If I may play with metaphor for a moment, the study was like declaring guns no more dangerous than baseball bats when it comes to blunt-object head injuries. It was the equivalent of comparing milk and Elmer’s glue on the basis of whiteness. It did, in short, miss the point. Even Crystal Smith-Spangler, a Stanford co-author, perfectly captured the narrowness of the study when she said: “some believe that organic food is always healthier and more nutritious. We were a little surprised that we didn’t find that.” That’s because they didn’t look — or even worse, they ignored.

In fact, the Stanford study — actually a meta-study, an analysis of more than 200 existing studies — does say that “consumption of organic foods may reduce exposure to pesticide residues and antibiotic-resistant bacteria.”

Since that’s largely why people eat organic foods, what’s the big deal? Especially if we refer to common definitions of “nutritious” and point out that, in general, nutritious food promotes health and good condition. How can something that reduces your exposure to pesticides and antibiotic-resistant bacteria not be “more nutritious” than food that doesn’t?

Because the study narrowly defines “nutritious” as containing more vitamins. Dr. Dena Bravata, the study’s senior author, conceded that there are other reasons why people opt for organic (the aforementioned pesticides and bacteria chief among them) but said that if the decision between buying organic or conventional food were based on nutrients, “there is not robust evidence to choose one or the other.” By which standard you can claim that, based on nutrients, Frosted Flakes are a better choice than an apple.

But they’re not. And overlooking these key factors allows the authors to imply that there isn’t “robust” evidence to choose organic food over conventional. (Which for many people there is.) Under the convenient cover of helping consumers make informed choices, the study constructed a set of criteria that would easily allow them to cut “organic” down to size.

“The researchers started with a narrow set of assumptions and arrived at entirely predictable conclusions. Stanford should be ashamed of the lack of expertise about food and farming among the researchers, a low level of academic rigor in the study, its biased conclusions, and lack of transparency about the industry ties of the major researchers on the study. Normally we busy people would simply ignore another useless academic study, but this study was so aggressively spun by the PR masters that it requires a response.”

When Clark says “aggressively spun by the PR masters,” this is what she means: a Google search for “Stanford Annals of Internal Medicine” gave me these six results in the top seven:

Yet even within its narrow framework it appears the Stanford study was incorrect. Last year Kirsten Brandt, a researcher from Newcastle University, published a similar analysis of existing studies and wound up with the opposite result, concluding that organic foods are actually more nutritious. In combing through the Stanford study she’s not only noticed a critical error in properly identifying a class of nutrients, a spelling error indicative of biochemical incompetence (or at least an egregious oversight) that skewed one important result, but also that the researchers curiously excluded evaluating many nutrients that she found to be considerably higher in organic foods.

Even the Web site of Stanford’s Freeman Spogli Institute for International Studies (which supported the research) features an article right above that about the new study that says “study confirms value of organic farming” and details how conventional agriculture is much more likely to contaminate drinking water with nitrates, which “can cause serious illness in humans, particularly small children.” What’s healthy and nutritious again?

That the authors of the study chose to focus on a trivial aspect of the organic versus conventional comparison is regrettable. That they published a study that would so obviously be construed as a blanket knock against organic agriculture is willfully misleading and dangerous. That so many leading news agencies fall for this stuff is scary.

Clark is right: this junk science deserves a response. Ignoring it isn’t enough. I apologize.