Big Three Bailout: A Data Drive-Through

December 03, 2008 11:39 AM

This week’s attention on government loans for the Big Three automakers makes a drive-through of available data worthwhile. Buckle up: There are some road hazards.

The data point I like best - don’t be surprised - is from our own ABC News poll Nov. 23, in which we found 57 percent of Americans opposed to $25 billion in loans for the Big Three, just 35 percent in favor. Strong opponents outnumbered strong supporters by 2-1.

We got there with a pro/con question that summarized arguments for and against the loan. It’s an approach I favor for issues that are new, complex, carry innate positive attribute bias, or in which the arguments and/or counterarguments may not be apparent. Here’s what we asked:

ABC News, 11/23: The big three automakers in the United States have asked for a 25 billion dollar loan from the government. Some people say (it's a bailout those companies don't deserve, and that they'd be better off reorganizing under bankruptcy laws). Other people say (it's necessary to protect auto workers and save a key part of the U.S. economy). On balance, do you support or oppose this plan? Do you feel that way strongly or somewhat?

The only other independent data we've seen on this is from a Gallup poll Nov. 16, with a different, more evenly divided, result – 47 percent in favor, 49 percent opposed. This question was simpler, did not name the loan amount, used the positive word “assistance” and, to my ear, gave a reason for the loan but not a reason against. Take a look:

Gallup, 11/16: Would you favor or oppose the federal government giving major financial assistance to the big three U.S. automotive companies if they are close to going broke or declaring bankruptcy?

Lastly there was a General Motors-sponsored poll from Hart Research, Nov. 12, that found 55 percent support for the loan. If the news release didn’t say that G.M. had nothing to do with the questionnaire I’d be inclined to suggest it’s at least as good at manufacturing data as it is at manufacturing vehicles.

This effort primed respondents by asking an unbalanced question on the importance of the auto industry to the U.S. economy (three chances to call it important, two otherwise) and then asking how much harm would be done (to jobs, the economy, America's world standing and consumer choice) if the auto industry went broke, before getting to the loan question - which itself, in my reading, had a little positive attribute bias thrown in.

Hart Research for G.M., 11/12 (After questions on importance of auto industry and potential harm of its failure): Do you believe the government should or should not provide loans to America's automakers so they have the money to manufacture vehicles?

I’ve written about this kind of thing before, noting how poll results on providing government funds for failing financial firms depended quite a bit on the question wording – “bail out” or “investing,” for instance. These results aren’t contradictory; they’re simply different, and in their differences they give us a useful understanding of the language that may be used on both sides to try to steer public opinion in their favor.

Afternoon update:

CNN today released another poll result on this, very similar to our own, with Americans opposing Big Three assistance by 61-36 percent. (Ours, per above, was 57-35.)

I should note that if you put three pollsters in a room you'll get four ways to ask a question; quibbling about wording is half the job. Here's CNN's:

CNN, 12/2: The major U.S. auto companies have asked the government for a program that would provide them with several billion dollars in assistance. The auto companies say they may go into bankruptcy without that assistance. Based on what you have read or heard, do you favor or oppose this program?

My observations: On one hand, possibly imparting a negative attribute, this doesn't specify that we're talking about a loan; on the other, possibly imparting positives to the proposal, it lowballs the amount and doesn't make clear that bankruptcy can be non-fatal. (And then there's the question of both CNN and Gallup's "favor"/oppose, vs. my own strong preference, "support"/oppose - fodder for a full-day colloquium in and of itself.) These may make no difference, or may cancel each other out. The result does underscore the reassuring fact that different ways of getting at an issue, if balanced, more often than not yield similar answers.

December 3, 2008 | Permalink | User Comments (11) | TrackBack (0)

The Flip States

November 26, 2008 11:33 AM

Our assistant polling director, Peyton Craighill, has drilled into the exit poll data to examine the 2008 flip states – those that went for George W. Bush in 2004 but for Barack Obama this year. As we’ve seen nationally, age, partisanship and race were all key factors – including, notably, a sharp shift among Hispanics. His summary follows.

By Peyton Craighill

Nine states with 112 electoral votes made the difference for Barack Obama this year – the flip states John Kerry lost to George W. Bush in 2004, but Obama won. Together, Florida, Ohio, North Carolina, Virginia, Indiana, Colorado, Iowa, New Mexico and Nevada went to Obama by 52-47 percent, a turnaround from Bush’s 53-46 percent four years ago.

The question: What changed?

Exit poll results show that the swing to Obama in these states was largely a reflection of voter preferences across the country: a flailing economy, broad dissatisfaction with Bush and the war in Iraq, changes in partisan identification, energized young voters and an increased Democratic advantage among Hispanic voters.

RACE – Hispanics accounted for 9 percent of voters in these nine states compared with their 7 percent share in 2004. And they favored Obama by 26 points, 62-36 percent, as opposed to a 5-point Kerry advantage, 52-47 percent, in 2004.

This 21-point increase was the largest change in support in any group in these flip states. Hispanics in the rest of the country increased their Democratic preference a bit more modestly, from a 23-point Kerry advantage to 38 points for Obama.

In three of the nine states, Hispanics made up a significant portion of the electorate, and their change in support was crucial to Obama’s victory. In Florida, where Hispanic turnout was stable at 14 percent, these voters supported Obama by 57-42 percent. In 2004 they went to Bush by 56-44 percent.

Hispanics in New Mexico increased their turnout by 9 points to 41 percent of voters, the most of any state, and favored Obama by 69-30 percent, more than triple Kerry’s 12-point margin, 56-44 percent. And in Nevada Hispanics accounted for 15 percent of voters, up 5 points, favoring Obama by 76-22 percent vs. Kerry’s 60-39 percent in 2004.

Obama improved among blacks as well, 94-6 percent in these nine states vs. Kerry’s 86-13 percent. And while Obama lost among whites in these states, it was by a closer margin, 55-43 percent for McCain vs. Bush’s 61-39 percent. (That’s similar to the 2004 to 2008 shift among whites in the rest of the country.)

AGE – Another key to Obama’s success was the support of young voters – and in these nine states their support was especially important. As was the case nationally, voters under 30 in the flip states didn’t increase their turnout from four years ago, but did increase their support for the Democratic candidate; they split 63-36 percent, compared with 54-44 percent in 2004. Obama improved sharply among middle-aged voters too, while losing seniors by 9 points.

The net result: If no one under age 30 had turned out on Election Day, McCain would have drawn even with Obama (49-49 percent) in these nine key states. In the rest of the country, by contrast, Obama won narrowly, 51-48 percent, among all voters 30-plus. (Looking state by state, young voters appear particularly to have made the difference for Obama in two of the nine flip states, Indiana and North Carolina.)

PARTY – Changes in partisan identification were big enough to move these states from a Republican to a Democratic advantage. In 2004 the partisan turnout was 40 percent Republican, 35 percent Democratic and 25 percent independent. This year Democratic identification pulled ahead of Republican by 37 percent to 33 percent, with independents inching up to 30 percent.

Not only did Republican identification decline by 7 points, but Obama was slightly more successful winning GOP defectors – he narrowed the gap among Republican voters in these states from an 88-point deficit in 2004 to a 79-point deficit this year. In the rest of the country, by contrast, McCain had about the same advantage among Republicans that Bush did in 2004.

Ideological identification didn’t change compared with 2004, but vote choice among liberals, moderates and conservatives all trended in Obama’s direction.

BUSH/IRAQ – Bush was a major drag on McCain. In 2004 Bush had a 55 percent job approval rating in these nine states. This year that was virtually cut in half, down to 29 percent. And Obama beat McCain by 69-29 percent among Bush disapprovers.

The Iraq war, while nowhere near as important as the economy, also hurt McCain. Four years ago 55 percent in the flip states approved of the war; that was down to 41 percent this year. And the 58 percent who disapprove of the war backed Obama by 79-19 percent.

Click here for a pdf table with some of the data in this summary.

November 26, 2008 | Permalink | User Comments (4) | TrackBack (0)

Socrates in Jeopardyland

November 20, 2008 10:59 AM

Get a hard seat, a bright light, and ponder these questions:

13. Socrates, Plato, Aristotle, and Aquinas would concur that:
a. all moral and political truth is relative to one’s time and place
b. moral ideas are best explained as material accidents or byproducts of evolution
c. values originating in one’s conscience cannot be judged by others
d. Christianity is the only true religion and should rule the state
e. certain permanent moral and political truths are accessible to human reason

27. Free markets typically secure more economic prosperity than government’s centralized planning because:
a. the price system utilizes more local knowledge of means and ends
b. markets rely upon coercion, whereas government relies upon voluntary compliance with the law
c. more tax revenue can be generated from free enterprise
d. property rights and contracts are best enforced by the market system
e. government planners are too cautious in spending taxpayers’ money

29. A flood-control levee (or national defense) is considered a public good because:
a. citizens value it as much as bread and medicine
b. a resident can benefit from it without directly paying for it
c. government construction contracts increase employment
d. insurance companies cannot afford to replace all houses after a flood
e. government pays for its construction, not citizens

OK, these are from:
a. The latest SAT exam
b. Barack Obama’s vetting packet for prospective Cabinet members
c. A new poll purporting to tell us that Americans are bozos

The answer is “c,” and if your head’s spinning, join the club. 

This little beauty comes from the Intercollegiate Studies Institute, an outfit that today is releasing its annual survey on “American Civic Literacy.” Its purported result: “The majority of Americans - including elected officials - failed a test of basic knowledge about American history and economics.”

The reality is that the ISI itself has failed a test of basic knowledge about the definition and measurement of just what knowledge is. I’ve blogged about this before – see it here – but the key point is that these folks are confusing knowledge (the ability to draw on information to make considered judgments) with recall (the ability to recite disassociated facts); and then doubling down by using an inappropriate method of measurement.

The headline result in this report says "many Americans thought the phrase ‘government of the people, by the people, for the people’ came from the U.S. Constitution or Declaration of Independence and not its actual source - Lincoln's Gettysburg Address.” It does not tell us why being able to recite the precise source of this (or any) historical quote is important. The logic, it seems, is that more recall is better than less recall - the Jeopardy school of "knowledge."

Leaving aside that concept – or whether being able to spit back a comparative assessment of Socrates, Plato, Aristotle, and Aquinas constitutes “basic knowledge” – just imagine trying to answer these questions in a telephone survey. You have to comprehend the root question, then keep each response option in your head so you can compare and choose. On the phone.

Two thousand five hundred and eight Americans took this survey, answering – I am not kidding – 118 questions. What they got for their trouble was to be excoriated as dumbos. Me, I’d give ’em medals for sheer fortitude. And the booby prize to the perpetrators of this so-called research.

November 20, 2008 | Permalink | User Comments (15) | TrackBack (0)

Race Revisited

November 19, 2008 3:23 PM

Running through the exit poll data for a presentation this weekend led me to another way to approach the issue of race in the presidential election – one that, as it comes out, underscores the notion that it mattered less than you might have thought.

Few whites, 7 percent, said race was an important factor in their vote; those who said so favored John McCain by a 2-1 margin, 66-33 percent. That compares to a 58-40 percent McCain advantage among the additional 10 percent who called race a minor factor, and 53-44 percent among the 82 percent of whites who said it wasn’t a factor at all.

So who are these racially motivated white McCain supporters?

Compared with other whites (see table below), those who called race at least somewhat of a factor are slightly more apt to be rural or small town residents, a bit more apt to be men, 10 points more likely to be Southerners and evangelicals and 13 points more likely to identify themselves as conservatives.

But there’s one far bigger difference: Whites who say race was a factor in their choice, and who voted for McCain, are almost twice as likely as other whites to be Republicans – 67 percent, compared with 35 percent of others. And being a Republican, as it happens, is the single best predictor of voting for McCain – 91 percent of white Republicans did so.

               Whites, race a factor,  Other 
                 supported McCain      whites
Small town/rural       26%               21%
Men                    54                47
Southerners            40                30
Evangelicals           44                34
Conservatives          48                35
Republicans            67                35

(The differences listed above are very similar when we look at the smaller group of whites who called race not just a factor, but an important one.)

Since the standout attribute of whites who called race a factor is that they’re very disproportionately Republicans, there’d seem to be a good chance that their partisanship - rather than the race factor – accounts for the difference in their vote preference. To test it in a regression analysis all we need is the exit poll dataset, which we’re awaiting. More to come.

It’s also been noted pretty widely that the white vote was disproportionately for McCain in just one region, the South. Whites there voted 69-30 percent in McCain’s favor, vs. an even, 49-49 percent split between McCain and Barack Obama among whites in the rest of the country. But that’s not unusual: Southern whites went for George W. Bush by nearly identical margins, 70-29 percent in 2004 and 67-31 percent in 2000. Bill Clinton, himself a Southern white, did better, but still lost them, by 15 and 20 points, in his elections. And Democrats were hammered in this group in 1980-1988. That makes it hard to blame racism, and easier to conclude that a lot of Southern whites just don’t like Democrats, particularly Northern Democrats. (Indeed, 46 percent of Southern white voters this year were Republicans, as opposed to 36 percent of whites in the rest of the country.)

Note, too, that Southern whites were no more apt than whites elsewhere to call race a factor in their vote. And in fact there was a bigger pro-McCain effect among racially motivated white voters in the East and Midwest than in the South.

           Whites – McCain vote
       Race a factor   Not a factor
All         61%           53%
East        57            44
Midwest     60            49
South       71            66
West        48            49

I’d add one other observation; at a presentation I attended last week a speaker referred to race-a-factor, pro-McCain white voters as “intolerant.” I’m not at all sure I agree. There is such a thing as affinity voting; indeed blacks were twice as apt as whites to call race an important factor in their vote. Blacks can have an affinity for Obama, as can whites for McCain, without intolerance coming into it.

Ultimately, as I blogged just after the election, one of the most fascinating results is the fact that Obama could lose whites by 12 points and still be elected president by a comfortable margin. That’s a testament to the nation’s changing demographic profile, and another indication that racial intolerance, to the extent it did exist on Nov. 4, was marginalized.

November 19, 2008 | Permalink | User Comments (15) | TrackBack (0)

The Gay Marriage Vote

November 19, 2008 10:23 AM

When Californians voted by 52-48 percent for a gay marriage ban Nov. 4, it wasn’t the first time: A similar measure passed by 61-39 percent in 2000, then was rejected by the state’s courts. The new version bypassed the courts by amending the state’s Constitution.

What changed in the two votes? Support for banning gay marriage dropped among most groups, producing the narrower margin. But it increased in one, African-Americans, and that contributed greatly to the measure’s success: Non-blacks voted 50-50 on the measure this year, while blacks backed it by a 40-point margin.

In the item below, Nik Bonovich, an election polling analyst here at the ABC News Polling Unit, dissects some of the changes in California’s vote on gay marriage from 2000 to 2008.

By Nik Bonovich

Perhaps the most notable change in the gay marriage vote was among blacks: While most other groups moved away from a ban on gay marriage, African-Americans moved toward it, voting 70-30 percent in favor this year, compared with 59-41 percent eight years ago.

Strikingly, blacks broadly favored the gay marriage ban despite their almost unanimous support for Barack Obama, who’d opposed the initiative, Proposition 8. Indeed, among non-black Obama voters in California, 74 percent opposed Prop. 8. Blacks were more aligned with John McCain’s voters, who favored it overwhelmingly, 84-16 percent.

The vote among blacks is not surprising: In a national ABC/Post poll in late 2007, blacks opposed gay civil unions by 58-36 percent. (Whites were in favor, 55-41.) In earlier polling we've done specifically on gay marriage, blacks have been even more broadly opposed, 66-31 percent - very similar to their latest vote in California.

Blacks pushed the measure ahead not only by their lopsided support, but also because, with Obama at the top of the ticket, they increased their share of the electorate to 10 percent, up from the usual 6 or 7 percent in the state.

Most other shifts were against the measure. Among whites, support for banning gay marriage dropped from 60 percent in 2000 to 49 percent this year; among Hispanics, from 63 percent to 53 percent. Support for the ban dropped among men from 64 percent in 2000 to 53 percent this year; among women, from 57 percent to 52 percent.

Young adults generally are more supportive of gay rights. In the 2000 vote (held in the primaries, in which young voter turnout was low) voters under age 30 divided evenly on the issue. This year young voters shifted sharply, 61-39 percent against the measure. Seniors, on the other hand, supported the ban by precisely the same margin.

There was little or no change from 2000 among conservatives or Republicans, both of whom very broadly supported the ban, or among independents, who fairly narrowly opposed it. But opposition rose sharply among Democrats, liberals and moderates alike, up 10, 11 and 15 points, respectively.

Supporters of the ban reportedly suggested that schools would incorporate same-sex marriage into lesson plans. Whether that had an effect is unclear. But voters with children under 18 at home supported the proposition by 64-36 percent, while those without minor children opposed it, 56-44 percent. Similarly, 62 percent of unmarried voters opposed the proposition, while 60 percent of married voters supported it.

In one of the sharpest divisions, 81 percent of evangelical white Protestants in California supported the ban, while among voters with no religious affiliation, 90 percent opposed it. The two groups were about the same size, 17 and 16 percent of voters, respectively.

Voters in Florida and Arizona also approved constitutional amendments banning gay marriage. In Arizona, Proposition 102 passed by 56-44 percent; in Florida, Amendment 2 passed by 62-38 percent. African-Americans in Florida voted similarly to blacks in California, supporting the ban by 71-29 percent. (There was an insufficient sample of blacks in Arizona to estimate their vote.) But, unlike California, majorities of whites in Florida and Arizona also supported banning gay marriage, as did most Hispanics.

Click here for a table comparing the 2000 and 2008 California votes by group.

November 19, 2008 | Permalink | User Comments (25) | TrackBack (0)

Ideological Underpinnings

November 11, 2008 11:31 AM

Republicans pondering their fate and future have noted that while there was a partisan shift in voter turnout this year, there wasn’t an ideological one: Conservatives still outnumber liberals by 50 percent. It’s true – but the comfort may be, let’s say, a cool one.

Thirty-four percent of voters in the national exit poll identified themselves as conservatives and 22 percent as liberals, with the plurality, 44 percent, as moderates. That makes this more a center-right than a center-left country – a reason Democrats, even with all the advantages they can muster, don’t win presidential elections by double-digits.

But there are other results to consider. One is that liberals hit their high water mark this year in exit polls since 1976 – not significantly different from 2004’s 21 percent, but up from their lows, 17 or 18 percent, in 1980 through ’88. Conservatives peaked at 36 percent in 1984, though they’ve been lower than this year, 29 percent in 2000.

Moreover, while conservatives are a bigger group among whites – 37 percent of that population, vs. 20 percent liberals – whites, as noted in our election-night analysis, make up a declining share of the voting population. Self-described conservatives this year accounted for smaller shares of Hispanics (29 percent), blacks (20 percent) and Asian-Americans (also 20 percent) alike. Liberals in fact outnumbered conservatives among blacks and Asian-Americans, and fell just 3 points short among Hispanics.

As important as the size of these groups is their vote, and that’s where the data are most telling. Take moderates, that big group in the middle: Barack Obama won more of them than any candidate in exit polls dating back 28 years, 60 percent. He also won liberals by the widest margin on record. And he pulled in 20 percent of conservatives, up from John Kerry’s 15 percent four years ago. (Obama also won 17 percent of George W. Bush’s supporters in 2004, double John McCain’s share of Kerry voters).

Ideology, moreover, is difficult to define precisely. Conservatives once favored states’ rights, limited government and balanced budgets; Democrats, federal mandates and expensive entitlements. Those distinctions have grown fuzzier.

A vote driven by the perceived importance of future Supreme Court appointees might be seen as an ideological vote; among those who called such appointments the single most important factor in their vote, Obama won by 16 points. Views on taxes might be seen as ideological, but among voters who were persuaded that McCain would not raise their taxes, 40 percent voted for Obama anyway. Opposition to government aid for failing financial firms may be ideological; but Obama won voters who support and “somewhat” oppose the program alike. (Even among “strong” opponents, he drew 46 percent.)

One thing we do know is that a substantial part of conservative ideology correlates with religious belief: evangelical Christians accounted for 43 percent of all conservative voters. Another is the fact that whatever their self-described ideology, well under half of voters, 42 percent, called Obama “too liberal,” despite McCain’s best efforts to raise that alarm. (Seventy-four percent of conservatives called Obama too liberal, but just 36 percent of moderates agreed.)

There’s also the exit poll’s flawed question on the role of government – whether it should “do more to solve problems” or “is doing too many things better left to businesses.” The challenge in making sense of this question is that it’s dependent on what problems people are thinking about, what government they’re thinking about, and the fact that government and businesses aren’t the only players in the problem-solving game.

In any case some of the heavy hitters on the current political agenda hardly militate for government inaction; few argue for a laissez faire approach to the economic crisis (sure enough, the more finanically stressed voters are, the more likely to say government should do more), and national security is hardly a private-sector pursuit. On the other hand, government action can raise all sorts of concerns across the ideological spectrum, depending, again, on which government, and what action.

So the result? Fifty-one percent of voters this year said government should do more to solve problems, up from 41 percent when the question first was asked in the 1996 exit poll and over half, albeit barely, for the first time. Fewer, 43 percent, said the government is doing too many things better left to businesses, down 10 points from its peak in 2000.

What’s interesting is that the sense government should “do more” is up across the ideological spectrum – compared to 2000, up by 8 points among liberals, by 10 points among moderates and by 8 points among conservatives as well (albeit just to 29 percent). That suggests that what the rebuilding Republicans may want to consider isn’t just the size of ideological groups – but also their evolving sentiments about the role of government, as the problems facing that government change.

November 11, 2008 | Permalink | User Comments (8) | TrackBack (0)

The Key, or the Chimera?

November 10, 2008 10:39 AM

(See end of post for afternoon addition.)

One of the items that prompted my post on the chimera of perfect polls was a Fox News piece quoting Brian Schaffner, a political science professor at the University of Massachusetts. I found the quotes and paraphrases alarming:

“All the polls predicted Obama would win, but the degree to which they got the margin is the key to successful polling, said Brian Schaffner…

“‘A real bad prediction is if you picked the wrong guy. And then it really comes down to – for a pollster – it comes down to bragging rights. …There’s a significant difference between a 5 to 6-point win and a 10-point win.’

“‘In 2004, Gallup did the best in terms of pegging the final outcome.’

“‘Pollsters who did call cell phone-only respondents tended to overstate Obama’s support somewhat.’”

The chief point of my post was to dispute directly the notion that nailing the horse race, much less the margin, is “the key” to successful polling. While a good estimate is necessary, pinpoint precision is a fiction. The notion of “bragging rights” and “pegging the final outcome” deflects us from the real aim of serious research. (And I’d add that given the sample sizes at hand there is not a “significant difference” between, let’s say, 52-46 percent and 54-44 percent, a 6-point vs. a 10-point margin.)

Nor is it correct that polls that took the trouble to include cell-only respondents “tended to overstate” Obama. Among the final polls that included cell-phone respondents, ABC/Post had Obama at 53, Gallup 53, CBS 51, Pew 49 – none an overstatement. (I am referring to polls, not projections. Gallup projected Obama at 55, Pew at 52.)

Since in the Fox posting Schaffner sure sounded like a proponent of micrometer-level evaluations of horse race estimates to establish the “best poll,” I invited him to reply to my posting. His comment almost speaks for itself, with two closing thoughts to follow.

Schaffner:

“I think the point you make in your column is absolutely correct, and it is one that I stress whenever I teach my students about the science of polling. The true value of public opinion polls – good public opinion polls – will always be as a way to understand why, not predict how, Americans cast their votes.

“However, I do think there is value in stepping back after an election and using the actual vote outcome to help us assess what we can learn about the science of polling. Indeed, I did not plot the performances of different survey organizations at Pollster.com in order to serve as referee for some pollster bragging rights contest. Rather, I did so in an effort to better understand the challenge that the cell phone only population poses to our ability to draw valid inferences about the American public.

“Elections provide one of the few moments when we get a census of public opinion that we can use as a benchmark with which to compare our survey estimates. Thus, pollsters and political scientists should absolutely use these moments to learn how to improve the science of polling. Why did polls interviewing cell phone only respondents fare about the same as those excluding that population?  Why did every poll released in the last weekend of the campaign understate McCain’s support? These are just two issues that pollsters need to evaluate in the coming months and only by comparing the polls to the actual results are we able to identify such questions. You are correct that we shouldn’t obsess over which pollster came closest to pegging the actual result, but that doesn’t mean that we shouldn’t evaluate why some polls came closer than others.”

My final comments start with a correction. McCain’s support was not understated in all the final polls, and indeed exactly as many “understated” Obama as did McCain (playing the game in which 1 or 2 points is an understatement). That’s chiefly a function of the level of undecideds they chose to report, itself a function of polling technique (and, to my taste, a too-convenient route to fiddling with allocations).

Of greater concern is Schaffner’s suggestion that including cell-only respondents is problematic because it didn’t improve estimates. In a non-significant way this is so; without cell-onlies our final Obama-McCain estimate was 52-45 percent rather than 53-44 percent, as I reported in our final pre-election analysis. Examining why is a fine thing. But taking it to the next step – arguing that noncoverage is desirable if it “improves” the allegedly precious horse-race estimate - would be precisely the wrong direction to take.

Afternoon addition:

Prof. Schaffner asked to add the following "with a note that my original statement was written when I thought you were referring to my Pollster.com post" (linked above):

"I was called by a FoxNews.com reporter on Thursday morning after the election. The reporter asked some questions about the accuracy of the pre-election polls, which I answered to the best of my knowledge. In my discussion with the reporter, I continually emphasized that assessing the accuracy of the polls was particularly important for helping us determine whether there was a "Bradley Effect" or a "Cell Phone Only" effect in the polling. At the end of the phone call, the reporter told me that she was going to try to post the story within an hour. I was immediately concerned because this suggested to me that my comments were not going to be used to help her formulate a story on polling, but rather to fit into a story she already had in mind (if not on paper).

"During my conversation with the reporter, I did discuss how the polling industry often judges which pollsters performed best in predicting the outcome. However, I was not making any normative judgement as to whether I thought that the concept of competing for bragging rights based on predictive accuracy was desirable. As I noted in my earlier response to Gary, I generally find the question of predictive accuracy to be more novelty than anything.

"Finally, during the interview I did say that it seemed as though pollsters calling cell phone only respondents overstated Obama's support. However, I had not looked at the data closely at that point (which I admitted to her) and it is now clear to me that there was no such pattern. I should not have speculated about the pattern before I had investigated it more closely.

"Ultimately, Gary and I may have some disagreement about the value of using election results as a benchmark for evaluating survey methods. However, my guess is that if Gary had been privvy to my entire interview with the reporter rather than simply the remarks that made it into the story, he would have found far less to quibble with."

November 10, 2008 | Permalink | User Comments (3) | TrackBack (0)

Work, Widgets and Perfect Polls

November 06, 2008 2:30 PM

The lists of “best pre-election polls” and the news releases trumpeting polling perfection are starting to roll in. They’re an exercise in vacuity of the highest order. And computed foolishly, as well.

The problem is not just in the silliness of these lists, but in the damage they do to the real task before us. Elevating the inconsequential through meaningless distinctions is not just a waste of time; it breeds misconception of the purpose and value of survey research.

The lists that I’m describing evaluate (to stretch the meaning of that word) pre-election polls on the basis of which came closest to the actual vote. They live in the fictional world of pinpoint accuracy in pre-election polling. They mistake estimates for something akin to laser surgery. They encourage the horse race lottery as a pollster’s path to fame and fortune. And they ignore the apparently minor matter of substance.

Don’t get me wrong: Given the intense if misguided focus on final pre-election poll estimates, we take ours extremely seriously, and I'm pleased we did very well again this year. Our estimate of Barack Obama’s vote was 53 percent; in the latest raw vote totals that’s what he got. Our estimate for John McCain was 44 percent; he got 46.

This demonstrates what ABC News pre-election polls have been demonstrating since 1984: We can interview a couple of thousand people and reliably report within a couple of points what 130 million are going to do. That is a rewarding affirmation of the principles of inferential statistics. It’s also as good as it gets.

At ABC News we don't stake our claim on the horse race; never have, never will. Yes, a well-founded estimate must be close; if it isn’t, something’s wrong. (See New Hampshire.)  But even junk polls can have a good day, as some did this year. Three guys at the gas station can guess the horse race, too; that doesn’t lift them over the hurdle of reliability we demand. Polling is not a game of darts.

Turning the horse-race estimate into the golden ring of survey research values the result above the rigor of the methods used to reach that result. That can have the dangerous effect of encouraging even good pollsters to drop their devotion to empirical data and indulge in a last-minute round of spitball for the numbers that seem right, empiricism be damned. I saw an unseemly scramble in some data this Sunday and Monday that made me… wonder. Especially given the stability of our own daily results.

Even worse, making horse race estimates the sine qua non of pre-election polling does damage to what should be our true aim. Some on these “best poll” lists – even those that bothered with the niceties of rigorous methodology – asked essentially nothing but the horse race. This kind of commodity polling reduces the product to a numerical widget, with a shelf-life of 15 minutes and a value of 15 cents.

We poll at ABC to advance our understanding of the election and add meaningful narrative to our news coverage – not to dwell monotonously on the score of the game. We interviewed a total of 10,213 randomly selected adults in the 19 days before the election, and many thousands in the months before. As I blogged when reporting our final data early Tuesday morning, our purpose was to explore how the public came to its choices. We tried to identify the issues and candidate attributes that mattered, to measure voters’ responses to the thrust and parry of the contest, to get a look inside the campaigns’ own playbooks as they formed and made their appeals and to have in hand a reliable alternative to the spin and speculation that rush in when empirical data are absent.

We reported on the overwhelming role of the economy in this election and the way it cut through demographic groups in influencing vote preferences. We looked at the questions of Obama’s experience and race, McCain’s age and temperament, the tone of their campaigns, the long shadow of George W. Bush and the short one of Sarah Palin. We saw and reported the unusual gap in Democratic vs. Republican allegiance, the surge in early voting, the dramatic tilt among young voters and its influence on the outcome. Yes, our final estimate was substantive, too. But to mistake it for our purpose is to mistake the package for its contents.

Many of our colleagues, thankfully, undertook the same kind of work. Many producers of horse-race commodity polls did not. Nor did the online aggregators bother to distinguish between the two, or even to distinguish between what are and are not reliable data in the first place. Just when we need it least, they, like these best-poll lists, dumb us down.

The lists add insult to injury by going about their own task in a distorting way. Many of them gauge accuracy on the basis of the reported gap between the candidates. Through this absurd procedure a poll that had Obama at 8 percent and McCain at 1 percent would be knighted as dead-on, since it nailed the gap – ignoring the outstanding 91 percent of respondents. If you’re going to engage in this game, the appropriate measure is to see how close the estimate of each candidate's support came to his actual support.

Better yet, give the horse race a rest. Look for good methodology, and when you have it, look for the substance that separates real work from widgets.

November 6, 2008 | Permalink | User Comments (7) | TrackBack (0)

Is it Transformational?

November 05, 2008 4:06 PM

The key question of the 2008 presidential election is whether it represents just a change in administration – or a change in our politics. Is it a reactive election like 1992, or a transformational one like the New Deal, the Great Society and the Reagan Revolution?

Time will tell. But the possibility is there.

There are three reasons this election may represent more than simply a one-time protest against an unpopular incumbent and a poor economy. One is the youth vote; another, the possibility of partisan realignment; and the third, the role of race. Each is worth a look.

Young voters, age 18 to 29, did not turn out in disproportionate numbers; they accounted for 18 percent of voters, compared with 17 percent the past three elections. But their vote was astonishingly lopsided, 66-32 percent for Barack Obama over John McCain – a 34-point gap. The previous biggest margins among young voters were 19 points for Bill Clinton in 1996 and 19 points for Ronald Reagan in 1984. John Kerry, counting on a boost from young voters in 2004, won them by just 9 points. Al Gore essentially split them evenly with George W. Bush in 2000.

First-time voters – two-thirds of them under age 30 – voted similarly this year, 69-30 percent for Obama, again far surpassing Clinton’s 20-point margin among first-timers in 1996, Al Gore’s tepid 9 points in 2000, Kerry’s 7 points four years ago.

We know that voting is habit forming; its best predictor is having voted previously. The question here is whether the Democratic preference of young and first-time voters in 2008 carries on in their age cohort. If so it could have long-lived implications.

Next is partisanship. Reagan forged a fundamental shift in political party allegiance in this country, one that lasted a generation – until Obama upended it yesterday. As reported in our full exit poll analysis, In the 1980 election Democrats outnumbered Republicans by 15 percentage points, 45-30 percent. Reagan won his “Reagan Democrats,” and four years later they’d become Republicans; the Democrats’ advantage in 1984 contracted to a mere 2 points, 38-36 percent. And there it roughly stayed: 3 points in 1988 and 1992, 4 points in 1996 and 2000 and then pure parity in 2004, when 37 percent of voters were Democrats, 37 percent Republicans.

This year, the shift: Democrats accounted for 39 percent of voters in this election; Republicans, 32 percent – their lowest turnout in 28 years. If it’s a one-off, it means little. If it endures, like the Reagan transformation, it would mean much.

Finally there’s race. The country is changing: In 1976, 90 percent of voters were white. That has declined in every presidential election since, to the point where this year white voters slipped under a quarter of the electorate, 74 percent. That’s one reason Obama could lose whites by 12 percentage points yet still win the election.

Obama does not appear to have lost whites chiefly because of his race; after all, Kerry lost them by 17 points, Gore by 12, Mike Dukakis by 19, Walter Mondale by 29, Jimmy Carter, in 1980, by 20. But it’s true, too, that previous Democratic winners did better with whites – Carter lost them by 5 points in 1976, Clinton by 1 in his first election and by 3 in his second. Given the stiff headwinds for the Republican Party this year, it’s fair to wonder why Obama didn’t do better with whites.

Affinity voting may be part of it – the notion that some voters, perhaps less rooted in ideology, may be inclined to support the candidate who seems to have the most in common with their own experience. Greek-Americans for Dukakis. Blacks for Obama. Some whites, yes, for McCain. And certainly, yes, racism may be a part as well.

Regardless, with whites at 74 percent of voters and shrinking, purely race-based voting by whites matters less. And those whites who voted against Obama out of racial discomfort now have four years to think about it. The fascinating question – the one that will answer whether we're seeing a transformation – is what they, like the young, first-time and partisan voters of 2008, do in 2012.

November 5, 2008 in Favorite Posts | Permalink | User Comments (53) | TrackBack (0)

Final Tracking

November 04, 2008 1:28 AM

With our last night of interviewing complete we’re ending our ABC News/Washington Post tracking poll with our final horse race estimate unchanged at 53-44 percent, Obama-McCain. That’s among 2,304 likely voters interviewed Friday through Monday.

We’ve interviewed a total of 10,213 randomly selected adults in the last 19 days; on the odd chance you were among them, thank you. Our purpose hasn’t been to dwell on the horse race, but rather to further our understanding of how the public has come to its choices in the 2008 election – to identify the issues and candidate attributes that mattered, to measure voters’ responses to the thrust and parry of the race, to get a look inside the campaigns’ own playbooks as they’ve formed and made their appeals and to have an alternative to the spin and speculation that rush in when empirical data are absent.

Public attitudes never are more important than in national elections. Our aim at the ABC News Polling Unit this year, as in presidential elections since 1984, has been simple: Trying our best to understand them.

November 4, 2008 | Permalink | User Comments (43) | TrackBack (0)