Pundits Vs. Machine: Who Did Better At Predicting Campaign Controversies? : All Tech Considered We pitted two political pundits against an algorithm to compete at predicting the biggest issues to arise over a month in the presidential election. Now we find out who won.
NPR logo

Pundits Vs. Machine: Who Did Better At Predicting Campaign Controversies?

  • Download
  • <iframe src="https://www.npr.org/player/embed/498280965/498292102" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
Pundits Vs. Machine: Who Did Better At Predicting Campaign Controversies?

Pundits Vs. Machine: Who Did Better At Predicting Campaign Controversies?

  • Download
  • <iframe src="https://www.npr.org/player/embed/498280965/498292102" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript

ROBERT SIEGEL, HOST:

A month ago, we began an experiment. We pitted two political journalists against a computer. Today on All Tech Considered - the results.

(SOUNDBITE OF MUSIC)

SIEGEL: The challenge was to predict what controversies would dog each of the major party presidential candidates over the course of a month. NPR's Laura Sydell tells us now who did better, humans or machine.

LAURA SYDELL, BYLINE: The two humans and the computers each got to predict five issues per presidential candidate that would get a lot of coverage between September 12 and October 12. That means it covers only a few days of the release of the Donald Trump lewd comments about women. Given that, the top of the list wasn't dominated by that controversy.

Here's a taste of the contest - predictions from Jonah Goldberg of the National Review, Simon Maloy of Salon and the computer. First, Trump...

(SOUNDBITE OF ARCHIVED BROADCAST)

JONAH GOLDBERG: A lot of discussion about his tax return.

SIMON MALOY: I think Trump will at some point say something sexist about Hillary Clinton.

COMPUTER-GENERATED VOICE: Building the wall along the Mexican border.

SYDELL: ...And now Clinton...

(SOUNDBITE OF ARCHIVED BROADCAST)

GOLDBERG: Debate mishap - I think it would be inappropriately laughing at something she shouldn't laugh at.

MALOY: There'll be a ton of coverage the moments that Clinton makes her first public comments on her health status.

COMPUTER-GENERATED VOICE: And No. 1 for Hillary is her email scandal of course.

(SOUNDBITE OF BELL)

SYDELL: Actually the computer was wrong. By volume of coverage, the No. 1 issue that plagued Clinton was her health, giving Simon Maloy the edge.

DAN BUCZACZER: Simon was actually winning for the first five days. He's this green line here.

SYDELL: Dan Buczaczer is with Quid, the data-analytics company that created the software for this contest. He's standing in front of a chart with the winning predictions color coded. But the contest wasn't about just one prediction. Each contestant made five of them. And when that's taken into account...

(SOUNDBITE OF BELL)

BUCZACZER: ...The winner actually was Quid.

SYDELL: So the computer wins an NPR T-shirt - not sure what size. The computer won despite a whole lot of stuff about the last month that was not predictable - Trump's attacks on Miss Universe, the leaked NBC tape. Simon Maloy thought Trump would say something sexist about Hillary Clinton. But that wasn't quite right. Buczaczer says the computer showed a general trend of sexist comments, but Quid didn't think that was enough to make a detailed prediction.

BUCZACZER: We'd seen a lot of information about controversies between Trump and women, but we did not realize that there was a tape on a bus with Billy Bush that was about to be released.

SYDELL: Ultimately the contest was judged by whose predictions got the most coverage. Both humans got some things totally wrong. The computer did better. Buczaczer says the edge the computer has is that it looks back over a long period of time and sees which issues keep coming up. And that...

BUCZACZER: ...Meant those are going to continue to get coverage in the press, and they allowed us to make what I would call some safer picks because we had that background and that knowledge.

SYDELL: In fact being too swayed by what's in the media may be one of the reasons the political pundits are especially bad at predictions. Dan Gardner is co-author of "Superforecasting: The Art and Science of Prediction."

DAN GARDNER: The pundit on TV doesn't have only an interest in making an accurate forecast. That pundit has an interest in making an entertaining forecast or a forecast that will attract attention.

SYDELL: Gardner says that the best human forecasters are the ones who bring a lot of curiosity and humility to the process. Or maybe they've got the help of a computer. Quid's Dan Buczaczer says that is actually how Quid's data is meant to be used.

BUCZACZER: If Jonah or Simon want to come in, look at something through the lens of Quid and then filter it through their expertise, it becomes even that much more powerful. It's really that combination.

SYDELL: And at a time when analytics and big data are being used to predict all kinds of things, Buczaczer and others think it's important to remember computers help expand the human mind. They don't replace it. Laura Sydell, NPR News.

Copyright © 2016 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.