The Numbers

A Run at the Latest Data from ABC's Poobah of Polling, Gary Langer

Gary Langer is director of polling at ABC News, where he's covered the beat of public opinion for nearly 20 years - conducting and analyzing ABC News polls, evaluating data from other sources and setting the news division's standards for poll reporting. Langer has won two Emmy awards for ABC's reporting of public opinion polls in Iraq, and The Numbers blog was honored this year as winner of the 2008 Iowa Gallup Award for Excellent Journalism Using Polls.

ARCHIVES

November 2009
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30          

« Previous | Main | Next »

New Hampshire's Polling Fiasco

January 09, 2008 12:05 AM

There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong. We need to know why.

But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened - about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen - a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities - such as their own failings in sampling and likely voter modeling.

There have been previous races that misstated support for black candidates in biracial races. But most of those were long ago, and there have been plenty of polls in biracial races that were accurate. (For more on past problems with polls in biracial races, see this blog I wrote for Freakonomics last May.) And there was no overstatement of Obama in Iowa polls.

On the other hand, the pre-election polls in the New Hampshire Republican race were accurate. The problem was isolated to the Democratic side - where, it should be noted, we have not just one groundbreaking candidate in Barack Obama, but also another, in Hillary Clinton.

A starting point for this analysis will be to look at every significant Democratic subgroup in the New Hampshire pre-election polls, and see how those polls did in estimating the size of those groups and their vote choices. The polls' estimates of turnout overall will be relevant as well.

In the end there may be no smoking gun. Those polls may have been accurate, but done in by a superior get-out-the-vote effort, or by very late deciders whose motivations may or may not ever be known. They may have been inaccurate because of bad modeling, compromised sampling, or simply an overabundance of enthusiasm for Obama on the heels of his Iowa victory that led his would-be supporters to overstate their propensity to turn out. (A function, perhaps, of youth.)

Prof. Jon Krosnick of Stanford University has another argument: That the order of names on the New Hampshire ballot - in which, by random draw, Clinton was toward the top, Obama at the bottom - netted her about 3 percentage points more than she'd have gotten otherwise. That's not enough to explain the gap in some of the polls, which presumably randomized candidate names, but it might hold part of the answer.

The data may tell us; it may not. What's beyond question is that it is incumbent on us - and particularly on the producers of the New Hampshire pre-election polls - to look at the data, and to look closely, and to do it without prejudging.

...Wednesday afternoon p.s.: Some folks are suggesting that "late deciders" made the difference - a common explanation for poor estimates. But the exit poll doesn't support the notion. Remove voters who decided on Tuesday and the New Hampshire exit poll result is Clinton +2 exactly her actual margin. (Among those who decided "just today" it was Clinton +3.) Next theory.

January 9, 2008 in 2008 Primaries, Polling Methods | Permalink | User Comments (887)

User Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

New Hampshire is a pretty short bus ride from Mass. or NY--with the lax voting rules in NH don't you suppose the Clintonistas helped a few voters find their ways to the polls??

Posted by: Mark | Jan 9, 2008 1:07:09 AM

Polls are always off IMO! Its funny how the pundits were telling us it was all over after Iowa. This country should have some type of super tuesday every 4 years to start off the election process. Then alternate the states every 4 years. It should not be a few states tell us who is in and who is out.

Posted by: Riverrat712 | Jan 9, 2008 1:08:14 AM

I'm with Eric on this. They don't call IT the "Clinton Family" ever. Ever! Do they? They call IT the "Clinton Machine."

Highly speculative indeed but if the shoe fits...

Posted by: David | Jan 9, 2008 1:08:37 AM

polls are simply baloney.
the guys that sell them won't say
so because they make them rich.
**********************************
they are wrong every year.
they change daily.
what use are they?

Posted by: david petraglia | Jan 9, 2008 1:09:41 AM

"could it be that the Republicans WANT Hillary to get the nomination and are desperate enough to make it happen"

Huh.. take off your aluminum foil hat. As far as all the pundit's thinking is the Republicans want Obama to win.

Posted by: JC | Jan 9, 2008 1:09:59 AM

Our poll here in Iowa before the caucus ran by the Des Moines Register was dead on with the Clinton/Obama results. Why are the NH polls so off? I guess when the votes are counted behind the scenes vs. out in the open like the caucus, then results can be WAY different.

Posted by: JR from Iowa | Jan 9, 2008 1:10:49 AM

I feel so naive. I, at
34, felt hope for the first timein our elections. I saw someone who might actually be for real. I read Judiciary Watch, and am not claiming I think he's perfect, but I found hope in him. This primary was stolen. Huge lead yesterday, exit polls talking about voting for change. Could we please not let this go.

Posted by: Joe B | Jan 9, 2008 1:12:45 AM

I completely agree with the sentiments on this board. How someone comes back from an 8 point deficit and scores 4 points in the positive overnight(total of 12 points) is unheard of. The crying moment didn't change that huge of an audience.

Particularly given the fact that these poll averages nailed ALL the other candidates. I'm the furthest thing from a conspiracy theorist, but this wreaks of fraud. Even more coincidental is that Bill Clinton says it's going to be close an hour before the polls close. How would he know this if all the polls were saying otherwise?

I know no one in the media probably wants to even throw this out -- but someone's got to do some serious journalism on who has accountability for the electronic voting systems in our country.

From Sourcewatch:

Since a County Commissioner can easily have or permit physical access, and the Accuvote communicates using modem technology whose vulnerability is well-known, there is no way of guaranteeing that electoral fraud has not been enabled by these machines by any machines other than an actual hand recount of the paper ballots that these machines scan and actually count.

Posted by: JRW | Jan 9, 2008 1:14:15 AM

I LIVE IN TEXAS... TONIGHT I WENT OUTSIDE, TRIPPED ON LOG AND HURT MY KNEE.. I KNOW THOSE DAMN CLINTONS DID IT.....GET A LIFE YOU CRY BABIES...OR TELL YOUR DADDY TOM DELAY...

Posted by: MOE | Jan 9, 2008 1:15:23 AM

Seems pretty clear to me. The polls gave Obama the lead with women by 3, yet Obama lost that demographic by 13.

Posted by: Scott | Jan 9, 2008 1:16:19 AM

Hillary had her "Oprah" moment in that cafe and it turned on a lot of old ladies. Either that or it was fraud,
which is nothing new to liberals, especially, the Clintons. I think Bill got an operative in there somehow. Either way, I don't trust women that vote for a woman like Hillary on the basis of her "welling up".

Posted by: Randy | Jan 9, 2008 1:16:25 AM

i have seen inaccurate polls. but i have never seen anything like this. it is not sufficient to say that all of the independents voted for mccain in the republican primary, because their turnout was so much lower.
it cannot be a coincidence that democratic turnout was so much higher than expected and that hillary's numbers were also so much higher than expected. where did those voters come from?
this whole thing smells.

Posted by: brendan | Jan 9, 2008 1:17:03 AM

I don't think anyone should consider those in New Hampshire as "independently minded anti-establishment" voters as they clearly picked the two biggest establishment candidates they could have. If they really did "vote against the media" it's a sad day indeed; do they really feel rebellious by voting for Clinton ? A tough bunch they say; yeah, a tough bunch of automatons.

Personally I think the first poster was correct: the Diabold machines.

Posted by: Dante | Jan 9, 2008 1:19:27 AM

The people have spoken in NH, the pundits, the big media and the rigged pollsters that can easy manipulate people opinion all of then LOST! The incredible biased media in favor of OBama, the outragious biased against Clinton and the total write offf of Edwards. Their manipulative tactis lost. the people won. Now the same pundits, media should focus in the issues instead and we'll have a campaign; but they never learn the lesson.
And what of the Obama wave? That wave crested and has turned. His candiate will have more scrutiny and he shold be held to the same standards than any other politician running for the Presidency.
Good night for democracy in the US. Shame on CNN's Blitzer among others, MSNBC starting with Chris Mathews, ah...and Tapper from ABC. They should learn to be real unbiased journalist!

Posted by: Mark | Jan 9, 2008 1:19:41 AM

This is extremely disturbing, because it
may influence someone who already has been told that their candidate will loose to stay home, when their candidate
may still have a chance to win. And let's face the fact that the media has
clearly favored Obama, and they told us
he would win. So, did they lie, or intentionally mislead us to influence the final result?

Posted by: Rick | Jan 9, 2008 1:19:44 AM

The way I see it. Its a wide open race for both partys. One note: Voter turnout sets records With ballots from 12 percent of voting precincts still to be counted, about 453,000 residents had cast votes, breaking the previous primary turnout record of 396,385 ballots cast in 2000.

State officials predicted that when all ballots were counted, the total would surpass 500,000.

I not going to get into a contest of who is the best candidate but to see these numbers of voters who have come out to vote in amazing

Posted by: Riverrat712 | Jan 9, 2008 1:20:07 AM

Hillary's win was Hillary tearing up. It changed the discussion from "Obama is bringing in the youth vote for the first time" to "Hillary is a human being".

There is no tactical explanation. It is big picture/strategic.

Not knocking Hillary for her tearing. I've run 3rd party Libertarian three times and every election night when I get 1% I've had the same physiological/psychological reaction, not crying but def welling up. It is real.

Regardless, I'm a Republican and no doubt Obama is the best Dem. President McCain the Democrats aren't unhappy with; same feeling for President Obama from the Republican side.

Clinton is Kerry, a self-serving, well-marrying politican that regular people just don't like. Dems should stop giving their pres. nomination to the machine that always loses on negatives in the General. Vote for a leader. Obama is a leader.

Posted by: Scott Jeffrey | Jan 9, 2008 1:21:38 AM

two words....independent voters. With McCain making a decent showing in Iowa, he probably swung alot of independent voters in NH that would have otherwise voted Obama. I'd love to have a breakdown of the actual numbers broken down by declared party who voted for who. I'd put nothing past the Clintons.....but voter fraud in NH.....nah, not buying into it.

Posted by: ZigZagMan | Jan 9, 2008 1:21:40 AM

One thing that is getting understatedis the biracial polling and the historical errors -- polls often overstate black candidates. What's different about Iowa? It is not a secret vote, so people can't secretly "switch" in the voting booth.

Posted by: ews3 | Jan 9, 2008 1:22:12 AM

It is without question racism .... remember in Iowa -- everything is out in the open here it is behind closed doors....

Posted by: China Vacation | Jan 9, 2008 1:22:17 AM

Post a comment