Pressure To Publish Leads To Shoddy Science And Bad Medicine : Shots - Health News U.S. taxpayers pay $30 billion a year to fund biomedical research aimed at finding better treatments. But competition for scarce funding and tenure may be prompting some scientists to cut corners.
NPR logo

How Flawed Science Is Undermining Good Medicine

  • Download
  • <iframe src="https://www.npr.org/player/embed/522262881/522826578" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
How Flawed Science Is Undermining Good Medicine

How Flawed Science Is Undermining Good Medicine

  • Download
  • <iframe src="https://www.npr.org/player/embed/522262881/522826578" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript

DAVID GREENE, HOST:

And you know if NPR had a life-or-death beat, this reporter would definitely be on it.

(SOUNDBITE OF ARCHIVED BROADCAST MONTAGE)

RICHARD HARRIS, BYLINE: Sepsis is the leading cause of death in the hospital...

Recommend low-dose CT scans to help detect lung cancer.

Researchers at Johns Hopkins are testing out new immunotherapy agents paired with traditional chemotherapy.

GREENE: That is the voice of NPR's Richard Harris. He covers biomedical research, the science that leads to cures and treatments for diseases. It's important stuff. And I had this eye-opening exchange with him in the studio the other day.

So you report on biomedical research a lot. And now you have a book about how much biomedical research is wrong. Are you going back through (laughter) some of your stories and going, uh-oh?

HARRIS: Absolutely, yeah. A lot of what we've reported over the years - and not just us, everybody - a lot of that is actually wrong. It seemed right at the time, but it has not stood up to the test of time.

GREENE: So the title of Richard Harris' new book says it all: "Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, And Wastes Billions." Richard found a stunning lack of rigor in the field he covers. He told me about one researcher at the biotech company Amgen who did something seemingly simple. He tried to replicate other people's work.

HARRIS: And he talked about how he had looked at 53 incredibly promising studies that could all lead to new drugs. And of those 53, he tried to reproduce them all. And he found that he could only reproduce six. Even...

GREENE: Six of the 53...

HARRIS: Six of 53...

GREENE: ...Could be reproduced.

HARRIS: ...Even with the help, in many cases, of the original scientists who did the studies.

GREENE: All right. There's this problem with reproducibility. Richard also found a culture in which researchers are under enormous pressure. Budget constraints mean less funding to go around. And researchers are increasingly driven to publish really quickly, even if it means getting things wrong. And researchers are also making mistakes in the lab, like with mice, that can lead to mistaken conclusions.

One of the things you talk about is - you kind of remind us that there are real people behind these stories. And there was a guy named Tom Murphy - healthy rugby player, I think. He's diagnosed with ALS, Lou Gehrig's disease. Tell me what happens - because he becomes part of an experimental drug program, right?

HARRIS: He does. Yeah. He's diagnosed, and he says - what do I do now? And he goes off to his doctor who says, well, there are a couple of experimental therapies you could try. And he signs up for one of them. It's a drug called Dex. And he's very hopeful, and it seems to be helping him. And they run these experiments with Dex. And it turns out that it doesn't work. In fact, none of the ALS drugs worked.

And there's a group in Cambridge, Mass., the ALS Therapy Development Institute, that went back and reviewed all of those studies and realized all the initial studies were wrong. They used very few mice. They weren't taking enough care to think about the genetics of the mice and a lot of other problems. This therapy institute came away with the conclusion that essentially none of them really were realistic at all.

GREENE: So this guy in his 50s has this terrible disease and ends up in a drug experiment that probably never should have happened just because scientists sort of farther back during the process were not being careful with the number of mice they were using.

HARRIS: That's right.

GREENE: I was struck because for all of the many problems that you pointed out, you know, when it comes to reproducibility, when it comes to making sure you're using, you know, the right number of mice, one of the big problems is that the delight has been taken out of biomedical research. What do you mean by that? And why should we be concerned about that?

HARRIS: Yeah. I think a lot of people go into science out of a sense of wonder. And unfortunately, what happens to all of us - this is not unique to scientists - but what happens in science in particular is people feel the career pressures. And they realize it isn't just about exploring and and having big ideas and so on it's - they have to have research that gives them progress toward their first job, toward tenure, toward the next grant and so on.

And those pressures are different from just sort of exploring and understanding fundamental biology. And so I talked to a guy named Henry Bourne at UC San Francisco who was mourning the lack of delight in biomedical research as he has witnessed it over the years.

GREENE: And why is delight a good thing to have in this type of research? Why would that make the research better and more reliable?

HARRIS: I think if you're focusing on delight, you're also focusing on the truth, ultimately, because that's what you're after. That's what everyone wants to understand in science. And the less you're focusing on delight, the less maybe you're aiming at the truth and more you are inadvertently, often, aiming at other goals - career goals, financial goals and so on - that may give you a fruitful life as an individual but may produce less of value to us as a society.

GREENE: Richard, how bad is this problem, if you could explain to our listeners? Like, if they hear, you know, tomorrow the latest new biomedical research on something, should they just not trust it? Should they question it? Like, I mean, where are we?

HARRIS: I think it is good to question it. Every time you hear something like this, just remember it's all contingent. And here is a study, and it may not stand the test of time. I think that's healthy. And I think people intuitively know that. If you say - oh, is coffee good for you or bad for you? - people will say oh, well, just wait a week, and we'll get the other answer.

(LAUGHTER)

HARRIS: So there is an element of that. And I think people understand that. And I think scientists do too. I think scientists, when they read the literature, they realize - oh, probably half of this is wrong. And it's just not knowing which half is the vexing part.

GREENE: Well, so your book is coming out when President Trump has a budget that he's proposing with significant cuts to the National Institutes of Health. If you're arguing that much of the science is bad, is cutting that budget justified in some way?

HARRIS: I suppose you could say if you could identify the wasteful projects and you could cut that, that would serve a purpose. But this is not the way it works. What's going to happen if you cut the NIH budget is you're going to shrink that already very small pool of money even smaller. And you're going to increase the competitive pressures. You're going to increase all of these perverse incentives that have put us in the position where we are to begin with. So I think that it's actually going to be devastating to biomedical research if the cuts are anything like President Trump had suggested in his initial push.

GREENE: So you're saying it could be justified if there's actually a way to figure out where the money is being wasted but that that's just not something that's very easy to do.

HARRIS: Right. You know, people don't run around with flags saying - oh, I'm the guy who's not getting it right, you know. And in fact, my book has stories of top scientists, Nobel laureates, who have made mistakes over the years. So you know, it's a very appealing idea, obviously, to say, oh, well, let's just identify the waste and root it out. But it's not the way science works.

GREENE: Pointing out so many problems and flaws in a field that you cover - does that make you any less excited to get up and cover this?

HARRIS: No, I don't. There's still a lot of great science that's going on. What I am shy of doing now is covering the individual studies that come out - that one study or another study, you know, you never really know how strong those are. But the overall trends are really interesting. And, I mean, just the culture of science is fascinating to take a look at and to watch people as they're saying - we have a problem. How are we going to fix it?

GREENE: Richard, thanks a lot.

HARRIS: My pleasure.

(ULRICH SCHNAUSS AND JONAS MUNK'S 'INTERVENTION STJERNER")

Copyright © 2017 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.