Search Casual Kitchen

Readers, I've developed a cognitive rule of thumb for "scientific studies" that I now use whenever I hear or read about any study. Here it is:

Heuristic: Any study you see, ever, anywhere, that happens to reach you through the media is wrong in some fundamentally significant way. You may safely ignore it.

My first halting step toward this admittedly cynical mental rule came a long time ago: when "scientists" decided--and later undecided--that margarine was better for you than butter. I took another halting step toward this cynical rule when the "don't eat too many eggs" study came out, a study that blissfully ignored the fact that dietary cholesterol is not blood serum cholesterol.

My steps became a lot less halting thereafter, as I thought through what "scientists" used to think was true. Things like:

* "Healthy whole grains."
* An entirely upside down food pyramid.
* Recommendations for statin meds because of a "studies show" link to cardiovascular health that turned out to be wholly imaginary.
* The fact that the medical profession doesn't seem to know what normal blood pressure is.
* That toast causes cancer.

There are many, many more examples, of course, some amusing, some not funny at all, some actually life-or-death. And all wrong. And if you have even a cursory understanding of decline effects[1] and the great crisis of reproducibility,[2] you will lose any remaining faith in "studies show" science.

Still more severe examples include South Korea's misguided war on thyroid cancer[3], or the highly counterintuitive discoveries that annual mammography screenings have zero effect on life expectancy and that, shockingly, prostate cancer screenings negatively affect life quality and life expectancy.[4]

And when it seems "studies show" science couldn't be any more wronger, it gets worse: We've discovered that even some of the most important and foundational studies in twentieth century psychology cannot be reproduced.[5] We're seeing major, domain-shattering acts of sheer data fabrication--see vaccines[6] and social psychology[7] for two object examples. Even in the food industry, one of America's best known dietary scientists, Cornell University's Brian Wansink, just got caught torturing his data to find links that don't really exist.[8]

And this is to say nothing of the various types of errors that inevitably show up in the study industry, as well as the rampant errors common to media coverage trumpeting study findings. These include errors like baseline risk error (an increase in a risk that's too small to matter is still too small to matter), p-hacking (mining data for statistical anomalies first, then forming a post hoc hypothesis on something that is almost assuredly spurious), and many others, some of which we've already discussed elsewhere at Casual Kitchen. And don't forget the well-known pressure to "publish or perish" in academia, which is most likely the prime driver of many of the industry's data fabrication and data-torturing scandals.

Last but certainly not least, there's the structural fact that the media--and it does not matter which media--inevitably oversimplifies or exaggerates all study claims to the point of making them into anti-information. And I have yet to read an article in my life saying "XYZ perfectly normal everyday activity shows no link to cancer." It's always the opposite.

You can safely ignore it all.

Finally, for any readers who consider this article to be somehow anti-science, keep in mind: "studies show" science is not science. It never was.

3 comments:

Nice summary. Science is not a straight line to the truth. Implementation is subject to all of the flaws of human nature. It takes decades and an immense amount of independent research to tease out low correlation effects.

Science also suffers for two things: an overeducated public, and an undereducated public. The former has been on top of scientific studies far too early in the process. Without reproducability, the public shouldn't even hear about it. The undereducated part is that they (we?) don't know that we need to look at meta-studies and understand statistics- something most with doctorates truly don't. Correlation, causation, confounding factors, understanding the motivators for finding "something", dodging oversimplifications, and understanding the slow timeline for real knowledge are all something the general public doesn't comprehend.

I like Scott Adams statement (I donk't recall the quote): Don't trust anyone who is absolutely sure about a complex issue. I guess that means the word "All" in your title is flawed!

There was a great article a while back in the NYer as well, that discusses the failure of many scientific studies to reproduce previously held "truths" that many other theorems were based upon for years afterward. It's really something how people seem to think that science is "truth" when actually it is the process of discovery. A lot of ego is involved in this faulty construction of reality so many hold dear.

Support Me Via Amazon!

Blog Archive

About Me

TEXT TEST

Having trouble with eating healthy and carrying out a daily exercise routine? Then you need motivation and natural herbs to help you to get to the healthiness you require. 5htp controls your appetite and mood. Visit Holland and Barrett today for 5HTP.