AUDIE CORNISH, HOST:
When you show up at the doctor's office with a serious medical condition, chances are your doctor will decide what treatment to offer by drawing on a specific kind of scientific study, one that pulls together information from many individual studies. Now, this technique helps sort through research that's often confusing and sometimes contradictory. But for a long time it's been considered the gold standard for scientific evidence. Researchers are now raising concerns that many of these studies are tainted by commercial interests. NPR's Richard Harris explains.
RICHARD HARRIS, BYLINE: Doctors used to rely on each other's opinions to figure out what worked and didn't work in medicine. More recently they turned to evidence from controlled clinical trials to guard treatments. But these days there are so many clinical trials; medical researchers have come to rely on techniques the combine evidence from many studies to come up with a sharper answer. These are called meta-analyses and systematic analyses.
JOHN IOANNIDIS: They are extremely important. They're trying to make some sense out of very convoluted scientific and medical literature.
HARRIS: John Ioannidis at Stanford is a big fan of this type of analysis, but he says unfortunately things have gotten out of hand. The scientific literature is now overflowing with these kinds of studies. To illustrate this point, Dr. Ioannidis recently looked at meta-analyses of antidepressant drugs.
IOANNIDIS: The problem is that there's just too many meta-analyses. There's 185 of them published in the literature within seven years, which means about 25 of them published every year for the very same drugs and then the very same indication, major depression.
HARRIS: What's worse, they're increasingly being generated by scientists who have financial interests in the outcome.
IOANNIDIS: About 80 percent of them have been funded or have some other conflicts of interest with manufacturers of these drugs. If you look at what their conclusions are, those that have been authored by industry employees - practically all of them with one exception - have claimed that there's no caveats about antidepressants.
HARRIS: You have to read deep into the studies to hunt for warnings about potential suicide risks, for example. That's potentially misleading doctors who turn to this kind of analysis to get a quick take on what works and what doesn't. Ioannidis says the drug industry has started using meta-analysis for commercial purposes rather than as a disinterested look at the evidence.
IOANNIDIS: They can get the results or at least the interpretation that fits their needs. So you have the most powerful and most prestigious design in current medical evidence, and it can be easily manipulated as an advertisement, as a marketing tool.
HARRIS: And it's defeating one of the main purposes of these studies, which is to make an overflowing scientific literature more manageable. Brown University psychiatry professor Peter Kramer, famous for his book "Listening To Prozac," took a deep dive into meta-analysis when he was writing his latest book, "Ordinarily Well." He found the situation was even worse than Ioannidis suggests.
PETER KRAMER: In some ways my doubts are stronger than his.
HARRIS: Some analyses he looked at were trying to parse differences that were simply too fine - for example, comparing two very similar antidepressants. You could get any answer you want depending upon how you set up the study.
Dr. Kramer says the problems aren't simply commercial conflicts of interest. For example, he saw biases among academics who were committed to the idea that placebos are just as good as actual drugs for depression.
KRAMER: Even on that side of the equation where there's no adverse sponsorship - it's just an allegiance to an idea - I thought, you know, it wasn't always the case that everybody's hands were on the table.
HARRIS: But financial conflicts are easier to identify. Five years ago, the National Academies Institute of Medicine suggested dozens of standards that should apply to systematic reviews of the research literature. Dr. Alfred Berg from the University of Washington chaired one of the committees. Money was one big concern.
ALFRED BERG: If there's obvious sponsorship from an organization that might have a financial conflict, that's probably not a good idea.
HARRIS: Would you say that people who have financial conflicts of interest should actually just stop doing systematic reviews and meta-analysis? Is that a reasonable suggestion?
BERG: (Laughter) Of course it's a reasonable suggestion. Is it going to happen in my lifetime - probably not.
HARRIS: Fortunately scientific journals are doing a better job of making sure researchers disclose their financial interest. It's up to readers now to take heed. Richard Harris, NPR News.
NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.