You May Now Ignore All Scientific Studies

Readers, I've developed a cognitive rule of thumb for "scientific studies" that I now use whenever I hear or read about any study. Here it is:

Heuristic: Any study you see, ever, anywhere, that happens to reach you through the media is wrong in some fundamentally significant way. You may safely ignore it.

My first halting step toward this admittedly cynical mental rule came a long time ago: when "scientists" decided--but then later undecided--that margarine was better for you than butter. I took another halting step toward this cynical rule when the "don't eat eggs" study came out, a study that blissfully ignored the fact that dietary cholesterol is not blood serum cholesterol.

My steps became a lot less halting thereafter, as I thought through what "scientists" used to think was true. Things like:

* "Healthy whole grains."
* An entirely upside down food pyramid.
* Recommendations for statin meds because of a "studies show" link to cardiovascular health that turned out to be wholly imaginary.
* The fact that the medical profession can't seem to define normal blood pressure.
* That toast causes cancer (yes, I'm serious: Google it).

There are many, many more examples, of course, some amusing, some not funny at all, some actually life-or-death. And all wrong. And if you have even a cursory understanding of decline effects[1] and the great crisis of reproducibility,[2] you will lose any remaining faith in "studies show" science.

Still more severe examples include South Korea's misguided war on thyroid cancer[3], or the highly counterintuitive discoveries that annual mammography screenings have zero effect on life expectancy and that, shockingly, prostate cancer screenings negatively affect life quality and life expectancy.[4]

And when it seems "studies show" science couldn't be any more wronger, it gets worse: We've discovered that even some of the most important and foundational studies in twentieth century psychology cannot be reproduced.[5] We're seeing major, domain-shattering acts of sheer data fabrication--see vaccines[6] and social psychology[7] for two object examples. Even in the food industry, one of America's best known dietary scientists, Cornell University's Brian Wansink, just got caught fabricating and torturing data to find links that don't really exist.[8]

And this is to say nothing of the various types of errors that inevitably show up in the study industry, as well as the rampant errors common to media coverage trumpeting study findings. These include errors like baseline risk error (an increase in a risk that's too small to matter is still too small to matter), p-hacking (mining data for statistical anomalies first, then forming a post hoc hypothesis on something that is almost assuredly spurious), and many others, some of which we've already discussed elsewhere at Casual Kitchen. And don't forget the well-known pressure to "publish or perish" in academia, which is most likely the prime driver of many of the industry's data fabrication and data-torturing scandals.

Last but certainly not least, there's the structural fact that the media--and it does not matter which media--inevitably oversimplifies or exaggerates all study claims to the point of making them into anti-information. And I have yet to read an article in my life saying "XYZ perfectly normal everyday activity shows no link to cancer." It's always the opposite.

You can safely ignore it all.

Finally, for any readers who consider this article to be somehow anti-science, keep in mind: "studies show" science is not science. It never was.



Footnotes/Resources:
[1] A readable article on the "Decline Effect," a genre of the problem of reproducibility.

[2] More good articles on the reproducibility crisis here and here.

[3] Korea and its misguided search for thyroid cancer as a textbook example of two types of silent risks: overscreening risk and overdiagnosis risk.

[4] For excellent discussions of why prostate screenings and mammography screenings carry silent risks and do not extend life, see Gerd Giggerenzer's book Risk Savvy and Gilbert Welch's book Less Medicine, More Health.

[5] Classic and foundational studies in pyschology cannot be successfully replicated.

[6] See for example the famously fraudulent “vaccines cause autism” study. Ironically, the paper pointing out the fraudulence itself had to be later corrected because it never acknowledged financial support from MMR vaccine makers. (!)

[7] “2011: A Year of Horrors” in social psychology.

[8] Bombshell allegations of data mining, p-hacking and other types of malfeasance shatter the credibility of dietary science professor Brain Wansink's entire department at Cornell University: here, here, and most depressingly: here.