NPR logo Should You Trust That New Medical Study?

Science

Should You Trust That New Medical Study?

Consumers should be more skeptical about individual medical studies.
Alexander Raths/iStockphoto
Consumers should be more skeptical about individual medical studies.
Alexander Raths/iStockphoto

News of medical studies fill the headlines and airwaves — often in blatant contradiction. We've all seen it: One week, coffee helps cure cancer; the next, it causes it.

From a consumer's perspective, the situation can be very confusing and potentially damaging — for example, in a case where someone with a serious illness believes and follows the wrong lead.

In a recent article for Vox, journalist Julia Belluz sounds the alarm, calling attention to research arriving at a disconcerting — but perhaps not surprising — conclusion: You should not trust news about an isolated medical study promoting a new cure, or blaming this or that food group.

Instead of rolling our eyes and bad-mouthing science and scientists, there are a few lessons to be learned here, which apply not only to the medical sciences but to science in general — and to how people should interpret scientific news.

First, science is incremental. We can go back to the early 20th century and compare what was going on then with now. Clearly, the medical sciences have made vast improvements, from antibiotics and antiseptic prevention to surgical techniques and diagnostic tools.

While the majority of individual studies tend to be either premature or badly formulated, the truth — or what we can make of it — eventually comes out. It is, thus, best to be patient and not to adopt a new promised cure the minute you hear about it. There are many factors that put pressure on researchers to publish and to promote their research, and most of these factors don't help science.

Second, skepticism is the way to go. As historian of science Naomi Oreskes says in Belluz's article, "What makes it news is that it's new. ... My view would be that brand new results would be the most likely to be wrong."

Right on. We should infer the efficacy of a new drug or the benefits or harms of foods from a sample of studies, not a single new one. Of course, most people don't have the time or the inclination to go through the exercise. When it comes to health, we want to believe in a new cure, for obvious reasons. Our skepticism must be doubled precisely to prevent being misled by hope. (Although hope and a positive attitude are known contributors to healing.) The responsibility, thus, rests with scientists and the media to promote the news carefully — and with the general consumer to keep the news in perspective.

Third, if possible, look at the results statistically. So, if there are 10 studies claiming that red wine prevents cancer and two claiming it causes it, odds are definitely in favor of prevention. Ideally, we would have the perfect study, with perfectly controlled conditions and a perfect control group. Unfortunately, this is not easy with human subjects and, so, we must do the best we can with what we have. An informed decision is always better than blindly following the latest trend.

Scientists approach consensus after weighing the evidence. The initial hype tends to go away, making room for careful inference and analysis. In the end, there is an answer — and we should pursue it with diligence. Clearly, taking the long view of history, we are going the right way.


Marcelo Gleiser is a theoretical physicist and cosmologist — and professor of natural philosophy, physics and astronomy at Dartmouth College. He is the co-founder of 13.7, a prolific author of papers and essays, and active promoter of science to the general public. His latest book is The Island of Knowledge: The Limits of Science and the Search for Meaning. You can keep up with Marcelo on Facebook and Twitter: @mgleiser.