« Previous | Main | Next »

A New Hampshire Post-Mortem

February 29, 2008 11:57 AM

Pollsters shed some light on their New Hampshire problem last night, with the Gallup Organization reporting that half the misstatement in its final pre-election poll was caused by its likely voter modeling. But other pollsters differed, agreeing chiefly that the causes of the meltdown remain elusive.

The discussion came at a meeting of the New York Chapter of the American Association for Public Opinion Research, whose national organization separately is looking into the failure. Each of nine polls completed the Sunday or Monday before the Jan. 8 New Hampshire primary showed Barack Obama numerically ahead, by 3 to 13 points, averaging 8. He lost by 2.

Gallup, whose final poll had Obama ahead by 13 points, had a closer 5-point Obama lead among people who described themselves as registered voters. That means its likely voter modeling, used to produce a more accurate estimate of who’ll actually vote, instead introduced error.

Gallup’s editor-in-chief, Frank Newport, said the modeling included factors such as enthusiasm and attention to the race, both of which may have increased for Obama and slacked off for Hillary Clinton after Obama’s Jan. 3 victory in Iowa. Unlikely voters – those excluded from the model – were much better for Clinton. “Obviously that was a cause for the incorrect likely voter numbers that Gallup put out,” he said.

Given Obama’s lead even among self-reported registered voters, Newport suggested two possible additional factors: late changes in voter preferences and a more effective get-out-the-vote effort by the Clinton campaign. (Presumably bias in self-reporting of registration is possible as well.)

Another polling director, Lee Miringoff of the Marist College Institute for Public Opinion, said likely voter modeling was not a factor in his poll, which showed an 8-point Obama lead. Instead, he said, “Our data suggest there was some kind of late shift to Hillary Clinton among women.”

That conclusion runs counter to data from the exit poll, which showed no large swing to Clinton among late deciders; and two final pre-election polls that if anything found an even larger Obama lead Monday night. The rest of the polls, Gallup’s and Marist’s included, were completed Sunday, a shortcoming Miringoff said he will not repeat.

A third panelist, Sarah Dutton of CBS News, whose poll had a 7-point Obama lead, similarly surmised that the New Hampshire polls “picked up a post-Iowa bounce” for Obama that didn’t carry through to Election Day. She said pollsters may have taken false comfort from the fact that all their data were similar, failing to scrutinize warning signs such as greater strength of support on Clinton’s side.

Otherwise, she said, “We might have had a little bit of an alarm bell saying, gee, we have disproportionate strength of support for these candidates.” Dutton said her analysis was limited by the sample size of the CBS poll, 323 respondents reinterviewed from a November survey.

Newport and Miringoff based their conclusions partly on post-election polls in which they called back respondents to their pre-election polls in an effort to see where those polls went wrong. Analysis of those data is not complete, though Newport said Gallup hopes to post some conclusions on its website next week.

Both said their callback polls reached about two-thirds of the original poll respondents; they hadn’t yet weighted these samples to adjust for the noncoverage, a step that could improve their analysis.

Newport said the people he recontacted divided 37-35 percent for Obama over Clinton in the pre-election poll, while reporting that they actually voted for Clinton over Obama by 39-37 percent, suggesting some changed their minds after the pre-election poll was done. At the same time, he noted the hazards of recall and self-reporting errors in post-election polls.

Marist’s callback poll found a self-reported vote of 37-31 percent in Clinton’s favor (a wider lead than Gallup’s, and than the actual result) compared with its pre-election poll (among all respondents, not just those recontacted on callback) of 36-28 percent for Obama. Twenty percent said they’d changed their minds, of whom nearly half went to Clinton vs. two in 10 for Obama, supporting the notion of a late shift in her favor, he said.

However, the New Hampshire exit poll asked people when they’d “finally decided” whom to support; 17 percent said it’d been in the last day, and they split 39-36 percent for Clinton, not nearly enough to account for the Obama overstatement in the polls. An additional 21 percent said they’d decided in the last three days, and they split 37-34 percent for Obama, again insufficient to explain the pre-election polls.

Dutton suggested that the exit poll question may have been difficult to answer accurately for people who’d switched their preference once or more.

Newport said eight in 10 voters contacted in his callback poll reported having seen the video clip in which Clinton became emotional the day before the election, and more said it made them react positively toward her than negatively. “The video may have had a positive impact,” he said. (He told a questioner that the poll didn’t ask respondents when they’d seen the video – before or after the election, when it received frequent replay.)

Miringoff, similarly, reported that eight in 10 had seen the video, and women in particular reacted to it more positively (31 percent) than negatively (11 percent).

Newport also said Clinton supporters also were more apt to say they’d been contacted by her campaign and offered assistance getting to their polling places, a possible measure of get-out-the-vote effectiveness.

In the future, Miringoff said he’d poll through the Monday before Election Day, and Newport said Gallup would re-evaluate the likely voter model it used in New Hampshire. Both also pledged further data analysis; so far, “What we have are inferential or suggestive data. None of these are huge numbers,” Newport said. “There’s no smoking gun.”

February 29, 2008 in 2008 Primaries, Polling Methods | Permalink | User Comments (29)

User Comments

Posted by: Angel | Mar 2, 2008 11:49:04 PM

Posted by: R. M. | Mar 3, 2008 12:52:43 AM

Posted by: John Nienstedt | Mar 3, 2008 12:57:35 AM

Posted by: John London | Mar 3, 2008 10:30:07 AM

Posted by: Angel | Mar 3, 2008 2:57:23 PM

Posted by: JB | Mar 4, 2008 4:40:23 AM

Posted by: Gary | Mar 7, 2008 6:15:32 AM

Posted by: Angel | Mar 7, 2008 9:10:18 AM

Posted by: Angel | Mar 7, 2008 2:28:51 PM

Post a comment

Name:

Email Address:

URL:

Remember personal info?

Comments: