Polling: The Math Behind The Madness Who's up? Who's down? Who cares? Just how did this ubiquitous staple of political coverage come to be? And just what is that margin of error thing?
NPR logo

Polling: The Math Behind The Madness

  • Download
  • <iframe src="https://www.npr.org/player/embed/96432107/96432073" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
Polling: The Math Behind The Madness

Polling: The Math Behind The Madness

  • Download
  • <iframe src="https://www.npr.org/player/embed/96432107/96432073" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


This is Weekend Edition from NPR News. I'm Alison Stewart. Coming up, we buckle down with Nixon musically. But first, polls, polls, polls. Who's up? Who's down? Who cares? We wanted to find out just how this ubiquitous staple of political coverage came to be and who figured out the method behind the polling madness. And who would be the guy who would know the answer to something like that? Our math guy, Keith Devlin, of course. He joins us from Stanford University's studios in California. Hi, Keith.

Professor KEITH DEVLIN (Mathematics, Stanford University): Hi, Alison. Nice to be with you.

STEWART: Thanks for joining us. So tell me, when do the mathematics of polling first come to light?

Professor DEVLIN: You know, it's actually much more recent than people think. Right up to the middle of the 17th century, any mathematician that you approached would have said that it's impossible to predict the future. You can apply numbers and you can count things in the world we live in and in the past, but you simply can't use mathematics to predict the future. That all changed in the 17th century when Pierre de Fermat, a very famous mathematician, answered (unintelligible) a curious little problem called the problem of the unfinished game which was a problem about how you divide the pot when a gambling game has to be abandoned before it's finished.

STEWART: Oh, something important.

Professor DEVLIN: You'd have thought that that had no application whatsoever. But that actually opened a floodgate of, essentially, the modern society that we live in. Because by solving that one problem, Fermat showed that you actually can apply mathematics to predict the future. And in fact, within 25 or 30 years of that problem being solved, you have all of the trappings of modern society with sort of risk management, futures prediction, insurance, annuities. Everything we now take for granted came on the heels - literally on the heels of that one mathematical result in the 17th century.

And one of the particular results that was obtained in that rush of new mathematics was by a Swiss mathematician called Jacob Bernoulli. And he showed that if you can take a random sample from a population, and then providing there are enough people in that sample, what information you get from that sample will actually be representative of the whole. And in fact, you only need to look at maybe a thousand people, providing they're chosen randomly from the population, a thousand people can get you a pretty good prediction of what might happen in somewhere like Florida or Ohio in an election.

STEWART: But the key word you used was "random."

Professor DEVLIN: Oh, you're absolutely right. That is really key. And in fact, in 1948 when pretty well all the major polls including Gallup predicted that Thomas Dewey was going to defeat Harry S. Truman in the presidential election - in fact, they predicted it was going to be a landslide - as we all know, that didn't happen. What happened, as you correctly noted, Alison, was that they didn't pick a sufficiently random sample.

In fact, what they did in that case was use a telephone, this new device that was available to everyone - or they thought everyone - for reaching people. They phoned people and asked them who they were going to vote for. But back then only the more wealthy people owned telephones. The wealthy people were more right-wing leaning. They were favoring Dewey. And so the sample wasn't at all random. In fact, it was a sample predominantly of Jewish supporters.

STEWART: Keith Devlin, math guy, I need you to explain these three words to me, margin of error.

Professor DEVLIN: Yes, well, predicting the future - this thing that goes back to the 17th century - we don't mean we can say what will happen tomorrow. What we do is we attach numerical probabilities or likelihoods to what will happen tomorrow. And when we do that, we have to state what the margin of error is because this is probabilistic prediction. So when you carry out a poll, you have to use statistical techniques to estimate how likely you are to be wrong.

That figure of asking 1,000 people being reliable - providing you choose them correctly in a random way - that would give you typically what's known as a three percent margin of error. What that means is that if you kept on polling in that way, then 95 percent of the time the answer you get will be within three percent of the correct answer. So you've got to realize there are two percentages there. The answer you get will be within three percent of the true answer 95 percent of the time. That other five percent of the time, the answer you get will be outside that margin of error. So, even the margin of error itself is not an absolute figure.

STEWART: Why is it a thousand people you need to speak to, to be able to reach some sort of truth?

Professor DEVLIN: It turns out that if you're happy with the three percent margin of error 95 percent of the time, then a well-selected, randomly selected group of a thousand does turn out to be predictive. If you want a more accurate result, if you want a one percent margin of error, you can get it. But then you've got to interview 10,000 people. So typically these days the pollsters settle on around a thousand because experience has taught them that that gives us fairly reliable results most of the time.

STEWART: Keith Devlin, Stanford University math professor and author of the recently published "The Unfinished Game." Thanks so much, Keith.

Professor DEVLIN: My pleasure, Alison.

Copyright © 2008 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.