How To Be A 'Super Forecaster'

Experts and pundits' predictions of the future aren't particularly accurate, according to University of Pennsylvania professor Philip Tetlock. He tells host Rachel Martin that making accurate predictions might be a trainable skill.

Copyright © 2013 NPR. For personal, noncommercial use only. See Terms of Use. For other uses, prior permission required.

RACHEL MARTIN, HOST:

In the world of politics, knowing the future would be really valuable. But it turns out that experts and pundits aren't much better than readers of palms or tarot cards when it comes to predicting the future. And the more confident the expert, the less accurate their guesses. Psychologist Philip Tetlock has been studying this for decades, collecting millions of political predictions and then looking at who's right. Tetlock is a professor at the University of Pennsylvania. He joined us from Philadelphia. And I asked him why the so-called experts might be getting it wrong.

PHILIP TETLOCK: Up to a point, knowledge helps. But knowledge often doesn't help with as much as experts think it does. So, one way to measure whether experts are overconfident is to get them to express their judgments on a probability scale and get them to say, are you 90 percent confident it's going to rain today; 70 percent, 60 percent. And if you do that many times, you can build up a database and a neutral judge keeps score, which is not the way it works typically in politics. In politics, each side - the liberals and the conservatives - they have their own scoring card. Imagine a baseball game, right, where each team has its own umpires. Politics the game is somewhat like that. Each side has its own scorekeepers, and not surprisingly we often have very different conclusions about who won the game at the end.

MARTIN: What makes them so inaccurate?

TETLOCK: We had a rare occasion for assessing how inaccurate many pundits could be during the 2012 election when Nate Silver, formerly of the New York Times - I think he's with ESPN now - he conducted some very careful statistical analyses of polls and reached the conclusion that even though the polls are pretty close, Obama has a 75 or 80 percent likelihood of winning the election. And a lot of pundits jumped on him and said, oh, it couldn't possibly be the case. Many people considered that to have been a debacle for the pundits.

MARTIN: So, who's really good at doing this, at predicting the future?

TETLOCK: Well, it helps to be a bit on the modest side. There are just some things nobody can predict. Accidents happen. Things happen that are just beyond the foreseeable. But beyond that, it is possible to cultivate the skill. We can show experimentally and pretty decisively that it is possible to train people to become better users of probability scales. They become more nuanced in their use of probability scales.

MARTIN: As I understand it, Philip, you engaged in this research along with your wife; is that right?

TETLOCK: Yes, Barbara, Barbara Mellers.

MARTIN: How does this play out, I wonder - if it does at all - in your personal relationship? Can you just off the cuff say, yeah, I think it's going to rain tomorrow? The other one calls the other out on that?

TETLOCK: Barbara's a much better intuitive statistician than I am. So, I defer to her on most quantitative issues.

MARTIN: Professor Philip Tetlock. He talked to us about making predictions - how we can do it better. He joined us from Philadelphia. Thanks so much for talking with us.

TETLOCK: My pleasure.

(SOUNDBITE OF MUSIC)

MARTIN: This is NPR News.

Copyright © 2013 NPR. All rights reserved. No quotes from the materials contained herein may be used in any media without attribution to NPR. This transcript is provided for personal, noncommercial use only, pursuant to our Terms of Use. Any other use requires NPR's prior permission. Visit our permissions page for further information.

NPR transcripts are created on a rush deadline by a contractor for NPR, and accuracy and availability may vary. This text may not be in its final form and may be updated or revised in the future. Please be aware that the authoritative record of NPR's programming is the audio.

Comments

 

Please keep your community civil. All comments must follow the NPR.org Community rules and terms of use, and will be moderated prior to posting. NPR reserves the right to use the comments we receive, in whole or in part, and to use the commenter's name and location, in any medium. See also the Terms of Use, Privacy Policy and Community FAQ.

Support comes from: