Want To Make Better Predictions? Researchers Explore How
STEVE INSKEEP, HOST:
Let me make a prediction. Sometime soon, you will hear someone make a prediction about the outcome of the presidential election, for example, or who's going to win the World Series. It turns out, a common way we make those predictions is ineffective. NPR's social science correspondent Shankar Vedantam has been looking into this.
SHANKAR VEDANTAM, BYLINE: Hi, Steve.
INSKEEP: So we're talking about you and me at a bar, just talking about the future. What do we do wrong?
VEDANTAM: We break up predictions into their elements. If we want to say who's going to win the Super Bowl this year, we say, let's follow the progress of each team. Let's think about how each team will do, and that'll tell us who's going to end up in the Super Bowl. Or if we want to know who's going to win the presidential election, we say, let's look at the performance of the party state by state, and that's going to tell us who's going to win nationally.
INSKEEP: Which sounds like what would make the discussion more fun. I'm going to get into the details. It's ESPN's "SportsCenter," I'm going into all these nuances...
INSKEEP: ...Of different players, or I'm looking at different electoral votes in different states.
VEDANTAM: That's right. But for most people who are using their intuitions, you know, sitting around chatting, it's potentially counterproductive. I was speaking with Theresa Kelly. She studies consumer behavior at Washington University in St. Louis, along with Joseph Simmons of the University of Pennsylvania. She conducted a series of 19 experiments. Many of them involved sports prediction.
She divided volunteers at random into groups. She gave both groups the track records of teams, and she asked one group to simply predict who won the next game. Then she asked people in the other group to think about the details of game. What's the final score? How's the pitcher going to perform? And then, who's going to win? People who focused on the details were less likely to call the correct winner than those who didn't think about all the details involved in the prediction.
INSKEEP: Wait a minute, why did we screw it up?
VEDANTAM: People are not very good at weighing different details in a prediction. Here's Kelly.
THERESA KELLY: Humans have a very, very hard time understanding correlations between different pieces of information. So I might know that the pitcher's very important and the team records are very important, but I don't know how to account for the fact that the quality of the pitcher directly contributes to the team record.
INSKEEP: So should we forget about the quality of the pitcher or the quality of a particular performance in a political debate and just go with our gut?
VEDANTAM: Well, like everything else about human behavior, Steve, there are caveats. If you pay too much attention to the drama of battleground state polls, you can undercount the overall state of the race. Kelly says that what she does in her own life, when it comes to making predictions about how she might complete an assignment, for example, is she simply focuses on past performance rather than the details.
KELLY: Rather than try to think about all the circumstances surrounding this present attempt to complete the assignment, I'll just say, well, how long has it taken me to complete similar assignments in the past? In general, that's a better way to make predictions because there's all kinds of reasons why thinking through the details of the case at hand can lead you astray.
VEDANTAM: Now, Steve, if you can actually weigh all the individual details like a statistician, you can fine tune the picture by paying attention to the details. But as they say in the commercials, this might not be something to try at home.
INSKEEP: OK, Shankar, thanks very much.
VEDANTAM: Thanks, Steve.
INSKEEP: Shankar Vedantam is host of the podcast Hidden Brain.
NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.