ROBERT SIEGEL, HOST:
Here's a problem for scientists trying to develop new treatments and cures for diseases - they often find it difficult if not impossible to reproduce some of the more exciting published results. Well, now researchers have tried to put a dollar value on the size of that problem, and as NPR's Richard Harris reports, that is proving to be tricky.
RICHARD HARRIS, BYLINE: Leonard Freedman was a lab scientist who decided to turn his attention to making the scientific enterprise more efficient. He started a nonprofit called the Global Biological Standards Institute, and recently, he set out with two economists to put a dollar value on the size of the reproducibility problem facing biomedicine.
LEONARD FREEDMAN: We initially were asking a very simple question, which is we simply wanted to know how much money is being spent each year on basic preclinical research that is not reproducible.
HARRIS: That turned out to be a very difficult question because there are only a few studies that address it and they aren't directly comparable. But in a paper published in the journal PLOS Biology, he and his colleagues report their best estimate.
FREEDMAN: You have a very large number - $28 billion.
HARRIS: Yes, billion with a B - every year. Taken at face value, that implies about half of all laboratory medical research in the United States cannot be reproduced in other scientists' labs. That's a contentious conclusion. There are some fields of research where reproducibility is an obvious problem. For example, many scientists who use human cells grown in the lab are actually using cells that aren't what they appear to be. A widely used cell labeled as breast cancer is actually a melanoma cell, and there are hundreds of similar examples.
FREEDMAN: The NIH currently funds about $3.7 billion annually on research using cell lines, so if a quarter of these projects apparently use misidentified or contaminated cells, that's a large number of the $3.7 billion.
HARRIS: Freedman argues that if scientists would run some simple tests on their cells, they could eliminate a billion dollars of faulty experiments each year. Other causes of trouble they identified include poorly designed experiments, poorly documented lab protocols and inappropriate data analysis. Stefano Bertuzzi, who runs the American Society for Cell Biology, acknowledges these are all real problems in his field.
STEFANO BERTUZZI: However, the problem I have with this study is that it wants to be a little bit sensational.
HARRIS: One problem is it lumps together fatal flaws - like using contaminated cells - with much less serious weaknesses. Some studies are labeled as irreproducible because they give poor descriptions of their methods, making it difficult for other scientists to repeat the experiment.
BERTUZZI: So while this is - could be a nuisance, it doesn't mean that the study is flawed or is wrong. The problem is folding all of this into a monetary value that gives the impression that this is wasted money. It's not correct.
HARRIS: For his part, Leonard Freedman agrees that the $28 billion doesn't represent wasted effort, though it may be easy to jump to that conclusion.
FREEDMAN: I hope that this work is not misconstrued.
HARRIS: Instead, he hopes that the dollar values he has calculated will help focus attention on areas where improvements will yield a big bang for the buck. And not coincidentally, his organization has launched a campaign that's trying to get scientists to make sure that they know the true identity of the cells they are studying in their labs. Richard Harris, NPR News.
NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.