NPR Chats : Election 2004
The Numerology of Opinion Polls
Wed., October 20th
Numerology: The study of the occult significance of numbers -- Merriam-Webster Online Dictionary.
As the campaign season draws toward a close, we are awash in polls -- and in contradictory findings. What makes a poll "good" or "bad?" Does our current political process rely too heavily on polls? What can we foresee as we enter the "home stretch?"
Andrew Kohut is the president of the Pew Research Center, in Washington, D.C. Kohut was president of The Gallup Organization from 1979 to 1989.
He is a public opinion consultant and analyst for NPR and the NewsHour with Jim Lehrer on PBS. He has written widely about public opinion for leading newspapers and magazines, as well as for scholarly journals.
Full transcript of this chat:
Andrew Kohut: Good afternoon, happy to be with you -- any questions?
a listener: A large increase in voter registration has occurred and record turnout is expected, are any polls tracking these new voters?
Andrew Kohut: Our polls show a 15% point increase in voter registration among 18-29 year olds. We are anticipating a larger than usual turnout in part because of new registrations, and also because voters new and old see this election as a particularly important one.
pam-richland: What are the limitations of political polling in a very close race?
Andrew Kohut: The margin of sampling error is an obvious one.
Andrew Kohut: But we also have to worry about how undecided voters are likely to vote and whether we have a good reflection of the composition of the electorate, that is the turn out factor.
Andrew Kohut: And of course, last minute trends that take place after our last poll.
a listener: I am a cell phone user only and I plan to vote. I know many people in the same situation and other considering such a move. Can the polls be accurate if the pollsters are not considering our vote??
Andrew Kohut: We do not include people who only have cell phones in our sampling. This should not be a major problem in this election because so few people in this category. The largest estimate is 6% of the voting age population. We would estimate less than half of that population would vote given its age profile. Which means we are talking at most about 3% of the voting population. This 3% would have to be radically different from people with land lines for it to have a noticeable impact on our estimate of the vote.
bob-davis: I've heard that many people refuse to participate in phone polls. Does this have a large effect on the reliability of polls results?
Andrew Kohut: Refusal rates are in fact increasing.
Andrew Kohut: Our experiments and studies show that the techniques we use to cope with refusals are still working.
Andrew Kohut: Our polls continue to match known characteristics of the population from the Census Bureau and other official databases.
a listener: What, if any, research has been conducted to determine what effect releasing poll results as the election nears may have on the results of that election? We have heard much about releasing exit poll data from the East Coast before the West Coast polls close. Cokie Roberts' report on October 18th seemed to indicate that she believes that there is a connection between reported poll data and perceptions of candidate "momentum.": Do any studies support this inference?
Andrew Kohut: There is no evidence that pre-election polls influence how people are going to vote. Certainly if a race is declared over based on exit polls while voters are still going to the polls that could discourage voting. This happened to west coast voters in 1980 when President Carter acknowledged defeat while polls were still open out there. Participation fell off sharply.
casey: Do refusal rates include the "Do Not Call List?"
Andrew Kohut: Pollsters do not have to observe the "do not call list" we are not selling anything.
Andrew Kohut: Its my hope that the "Do not call" list will reduce the number of unwanted calls people get and they will be more receptive to opinion polls.
a listener: If I am registered on party line, do pollsters assume I will vote this way? I find myself looking closely at both sides when it comes to senate and house, or governor.
Andrew Kohut: No, not at all, we ask people for whom they would vote for if the election were being held today. We donít ask questions about party registration. We ask people whether they think of themselves as Republicans or Democrats, not how they are registered.
ryan-normal: how often are the polls correct in guessing the person who actually gets elected
Andrew Kohut: First of all, we don't guess.
Andrew Kohut: We take objective opinion surveys using systematic methods.
Andrew Kohut: The record of the final results of the national polls compared to election results are very good. Go to ncpp.org to see for yourself.
a listener: The media often "calls" the election based on exit polls. How accurate are exit polls? Is there any bias in the process?
Andrew Kohut: The exit polls have a pretty good record, but they are not perfect. Most media election 'calls' are based in part on exit polls but also on actual vote counts from samples of key precincts. Overall the networks have an excellent record of calling races with a few notable exceptions.
bjork1234: When polls are conducted do you believe there is a fair amount of people from all ages called or do the pollsters center only around one age group?
Andrew Kohut: The age distribution of respondents in our surveys and in other major polls reflect the age profile of the adult public according to the Census.
a listener: Why are the most widely reported polls national surveys when presidents are chosen state by state. Why isn't there more talk of who is ahead in the electoral vote count?
Andrew Kohut: Because you need a poll in each state to count electoral votes. Generally, the popular vote is a pretty good indication of who is going to win the election. We have only had a few cases in our history (including 2000) when the winner of the popular vote did not win a plurality of electoral votes.
smooty40587: if pollsters call and the person doesn't answer does the pollster call back or just skip the number?
Andrew Kohut: We make up to 8 calls to every number that we draw at random, so we make quite an effort to interview people who are not often at home.
a listener: I have seen the term likely voters often. How do you determine it if it's solely based on past voting record won't it hide passion/or strength of feeling voters feel about their candidate..? Apparently there is no easy way to measure how strongly they feel for their candidate
Andrew Kohut: We determine likely voters by asking a battery of questions that cover such things as past voting behavior, interest in politics, interest in the campaign, and intentions to vote.
a listener: According to the State Department there are an estimated 4 to 10 million Americans living abroad who are eligible to vote. They are obviously not counted in phone polls. What effect will they have on the election?
Andrew Kohut: Unfortunately, telephone surveys miss people who live outside the country but they are a very small number and would have to be radically different from the rest of the population to effect the pollís estimate of the popular vote.
stuart-cincinnati: I am trying to understand how one well-known polling organization can consistently be out of step with every other national poll? What can go wrong?
Andrew Kohut: The polls recently have been showing differing results.
Andrew Kohut: Probably because voter opinions have been unsettled, these same polls were showing comparable results earlier in the year when the public was not rethinking its point of view on the candidates.
Andrew Kohut: Sometimes polls will differ because of timing or because of major methodological differences, I think its more that small differences in methodology create large differences in answers when public opinion is unsettled.
Andrew Kohut: And it has been unsettled since the conventions.
susan -- gaithersburg md: According to the Pew website, "interviewers asked to speak to the 'youngest male 18 or older who is at home.'" If there is no eligible man at home, interviewers asked to speak with "'the oldest woman 18 or older who is at home.'" Why does the interviewer ask to speak to a man first? Hasn't it been shown that Bush is favored by men and Kerry by women? If a man answers the questions, does the interviewer then ask to speak to a woman? If not, isn't this method of asking to speak to a man first, skewing the results? Why doesn't the interviewer ask to speak to a man on the odd-numbered calls and a woman on the even-numbered calls? With the computerized system used by pollers, it seems this should be easy to do.
Andrew Kohut: We have found that the best way to get a representative sample by age and sex is to use this technique which Gallup pioneered many years ago. We sometimes do a random selection method but that has the draw back of encouraging refusals because it is time consuming and tries the patience of some potential respondents.
pc-cavecreek: Apparently only the "battleground" states will decide the outcome.: Are there polls that focus exclusively on them?
Andrew Kohut: State polling organizations and some national polling organizations are covering individual battleground states. Go to pollingreport.com or realclearpolitics.com for battleground state results.
a listener: What confidence level is typically used in polls?
edwes: "likely voters," then, does not include newly registered voters,
Andrew Kohut: We can calculate the sampling error for a survey which takes into account that the survey is a sample and not a census of the population. This is called chance error or random error. It does not take into account error as a result of bias in the sample or the measurement error associated with the questioning.
Andrew Kohut: We have a lot a questions about new voters. Since we are doing random sampling and not working from old registration lists our polls are picking up the increasing number of new voters who have registered this year.
nicki: How can we be sure that pollers are polling diverse people?
Andrew Kohut: Our large samples reflect the diversity the population and match up with Census results. We work very hard to make sure that there are no systematic biases in our sample and respondent selection methods. Most national polls look like a microcosm of our diverse national population.
bork1234: When asking what party a person is while polling them do you find that afterwards more republicans or more democrats respond to your polls?
Andrew Kohut: We generally find equal numbers of Republicans, Democrats and Independents. But these results tend to vary in response to events. When things are going well for one party we get more people saying they think of themselves as members of that party and opposition members go into denial. For example after the successful Republican Convention, most of the polls showed an uptick in the percentage of people calling themselves Republicans.
laurama: What group of voters do you believe will affect the election most this year?
Andrew Kohut: Our polls have found that female voters and white Catholics have been swinging back and forth between the candidates. Among women, for example, Kerry gets as much support as Gore did in 2000 when he is doing well, but women divide more evenly or lean to Bush at other times.
Andrew Kohut: Same with white Catholics.
finitestudenttoday: It has been unsettled but a large percentage of voters constantly vote either Republican or Democrat, isn't that enough not to be swung by a few undecided voters?
Andrew Kohut: Yes Republicans and Democrats know who they are backing this year, but we still have many Independents and swing voters. Our latest poll still finds 14% of likely voters in the persuadable category.
geometryteacher2: If you combine cell-phone only citizens, those without a phone, those abroad, and those who work odd hours and are not able to answer the phone, you wind up with more than "a very small number." How radically different would these people have to vote to skew the polls beyond a +- 5%?
Andrew Kohut: First, we contact people who work odd hours by doing some interviewing during weekdays. The cell phone onlys and the living abroads are still small enough in number so that their opinions would have to be radically different and the election otherwise very close for them to have a meaningful impact. Cell phone onlys are likely to be a growing problem for pollsters which we will have to address in future elections.
dave-tempe: Who do you find voters age 18-25 favoring?
Andrew Kohut: This age group which will be more represented in this election, has been swinging back and forth. We have seen sizable margins for Kerry and at other times this age group favors Bush.
edwes: In your experience, is 14% an unusually high number of "persuadable" voters?
Andrew Kohut: This year the swing or persuadable voters have been a smaller percentage of the electorate than usual. But still its a large enough group for one of the candidates to win a decisive victory.
marie-ohio: Do the masses really want fear factor Bush?
Andrew Kohut: Swing voters continue to favor President Bush over John Kerry for dealing with the war on terrorism. This has been so throughout the campaign --it represents one of President Bush's most important strong points in the eyes of voters. However, these same swing voters prefer Kerry on the bread and butter issues. That's why they are still swinging.
kate-claremont: Do you find that woman favor Bush more on issues relating to terror and safety as compared to men?
Andrew Kohut: No they don't favor Bush more than men, but they give higher priority to terrorism as an issue and worry more about it than men.
james-ashford: Do you subscribe to the 50% rule for incumbents, and if you do, how do you interpret its implications for the race?
Andrew Kohut: The fact that President Bush is below 50% on his approval score represents a real opportunity for Sen. Kerry. However, it is not fully realized because many swing voters still remain uncomfortable with Kerry.
james-ashford: This election, should we believe the maxim that the incumbent gets his last polling number and the challenger gets the rest?
Andrew Kohut: Undecideds often break for the challenger but there have been some elections, 1976 for example, where there have been dramatic exceptions to this rule.
papridge: How do you weigh an opinon from a 45 year old republican compared to an undeclared 18 year old?
Andrew Kohut: One person, one vote regardless race, age and/or party disposition.
lisa r -- harvard ma: Whether or not a candidate is invited to participate in a national debate is dependent on how well they are polling - candidates must be polling at least 15% to be included. Given that many polls do not include "third" party candidates - or rather, they do not include the full list of candidates - how can this method successfully determine who should be involved in a debate? It seems the media's choice of candidates becomes the default.
Andrew Kohut: Actually just about all polls have included Nader. And the Gallup poll made a special effort this year to test support levels for three or four other "third party" candidates. They just didn't get much support in a high stakes election where the public is reluctant to back a candidate with little chance of winning.
eliza-pittsburgh: Have you ever exchanged opinion polling data with other professionals across the country? Were there any issues of differences in data?
Andrew Kohut: Most pollsters archive their data at the Roper Center at the University of Conn. so we have access to other polls. I frequently mention other poll results in interviews I do on NPR because I think it's important to look at what all the polls are showing not just the Pew survey.
pam-richland: What are the methodological differences among the polls?
Andrew Kohut: Right now the major differences have to do with how to determine likely voters. But there are even differences between polls that use comparable methods to determine likely voters. This I think reflects volatility in the polls due to lack of certainty in the candidates and also participation.
npr online: Andrew is answering what will be our last question for today. Please stand by.
eliza-pittsburgh: How easy is it for you to tweak numbers to show what you want to in a poll?
Andrew Kohut: Not too difficult.: But my objective is to do a good job of reflecting public opinion not reflecting my own preferences. Also however their are plenty of other polls that would soon provide a corrective if one pollster decided to cook the data.
npr online: Thank you, Andrew, for this fascinating discussion about the impact of opinion polls on our political process generally and on the 2004 election in particular.
Andrew Kohut: You're welcome -- and thank you for the great questions today.