MELISSA BLOCK, host:
Love lights up one area of the brain. Rejection lights up another. At least that's what it looks like in images produced by the latest brain scanners. These pictures of our brains at work appear crisp and precise.
But as NPR's Jon Hamilton reports, the truth behind them is a little fuzzier.
JON HAMILTON: You've seen the images, gray cross sections of the human head with vivid bursts of red or yellow showing where the brain seems to be responding to something. The pictures are so clear, so compelling, that even brain scientists like Chris Baker of the National Institutes of Health find them seductive.
Dr. CHRIS BAKER (National Institutes of Health): I look at these images of people's brains and I stop and I think, this is really amazing. I'm actually finding a way to look at what's going on in the brain.
HAMILTON: Then Baker starts thinking about all the ways these images can mislead a casual viewer. Most of the pretty pictures come from a technology called Functional Magnetic Resonance Imaging, or fMRI. It detects tiny changes in the amount of oxygen carried by blood to brain cells. More oxygen generally means more brain activity.
But it takes a whole lot of computer processing and human judgment to get from oxygen levels to a snapshot of love in the brain. Baker says the challenges begin the second you slide someone into a scanner and the machine starts looking for signals that suggest increased brain activity.
Dr. BAKER: The problem with function imaging is that the signals we're trying to get at are quite weak and there's a lot of noise.
HAMILTON: Noise as in irrelevant information, not actual sounds.
Dr. BAKER: So one way to think about that is like listening to a radio station which has a lot of static. So, sometimes, you know, the noise will sort of obscure being able to hear the people, or even being able to pick up the station at all.
HAMILTON: Some of the noise comes from the equipment itself. More static comes from the person in the scanner. Every heartbeat affects the flow of blood, which changes the signal. Every tiny head movement blurs the image. Then there's the noise from all the stuff the brain is doing that's not related to the experiment. And an fMRI study isn't as simple as listening to just one faint radio station from one place in the brain.
Scientists want a three-dimensional map of activity throughout the brain. So a typical scan gathers data from 100,000 different places, and it's up to the researcher to decide which spots have something to say.
Dr. BAKER: You're tuning the radio, looking across different frequencies. And for each frequency you go to, you're trying to pick up and see, was there really somebody speaking there? Did I hear a song at that particular point in time?
HAMILTON: If so, what kind of song? Was it about love or rejection or nothing at all? Baker says the signals are often so weak that researchers have to stimulate a person's brain over and over again to see any pattern. Let's say you want to know which areas of the brain respond to faces, something Baker has actually investigated.
Dr. BAKER: For the presentation of a single face, we may not be able to pick up anything at all. And typically, we're going to have to present lots and lots of faces to be able to sort of then look across all of those presentations, the faces, on average, do we see this increase in response?
HAMILTON: Then they'll have to repeat the experiment on 15 or 20 more people to see if the same area lights up in everyone or not. And face recognition is a pretty simple brain process, compared with experiencing an emotion like love or telling a lie, which is why scientists are wary of scans that claim to identify these things.
FMRI studies frequently produce billions of data points, many of which represent static, not a true signal. Baker says that with an ocean of noisy data, it's possible to find patterns produced purely by chance.
Dr. BAKER: In the worst case scenario, there are situations in which you could apparently produce a signal out of just noise.
HAMILTON: Baker doesn't think that happens very often, but he showed how it could in a recent paper published in Nature Neuroscience. That paper is one of two published this year that suggest the results of many fMRI studies aren't as robust as they appear.
Hal Pashler, a professor of psychology and cognitive science at UC San Diego, is an author of the other paper. It was published in the journal Perspectives on Psychological Science. Pashler says he and his co-authors wrote the paper because they were seeing results from fMRI studies that seemed just too good to be true.
The team began to ask why and zeroed in on a statistical practice sometimes called double dipping - it's a kind of circular reasoning. Pashler says the easiest way to understand double dipping is to imagine something that has nothing to do with brain imaging.
Professor HAL PASHLER (Psychology and Cognitive Science, University of California, San Diego): Suppose you wanted to figure out whether anyone in a town was able to predict the temperature three weeks from now. So you'd get everybody in a town, you get them all to make a prediction, then you come along three weeks later and you look at how well they did. So you might find there's 50 or 100 people who did just amazingly well. Maybe they got the temperature just spot on.
HAMILTON: You might conclude that all of these people are just brilliant at predicting the temperature. But Pashler says that's probably not the case because some of the people probably got it right purely by chance.
Prof. PASHLER: So if you test them again and have them do it again in six weeks, you're going to find that they'll do better than most people in the town. But they won't do as well as you think they did based on the data you collected when you first picked them out.
HAMILTON: Pashler and Baker say that many fMRI studies do something a lot like this when they dip into the same data twice: once to pick out which parts of the brain are responding, and again to measure how strong the response is. The scientists say that's probably made a lot of results appear stronger than they really are.
The papers criticizing fMRI studies have been the subject of furious debate in the neuroscience world. But even the strongest defenders of fMRI acknowledge there are problems and a need to exercise great caution when transforming messy data into tidy images. Pashler says it's not just the public that gets misled by overstated results.
Prof. PASHLER: When you find that something that really matters, like anxiety or depression, is linked to some very specific hotspot in the brain, that's obviously a clue that investigators and funding agencies are going to use in deciding where to put their efforts in the future.
HAMILTON: Pashler says that could mean some areas of the brain are getting attention they don't deserve, while other potentially important areas are being ignored, all because of images that aren't as clear as they seem.
Jon Hamilton, NPR News.