« July 2007 | Main | September 2007 »

Online Ballots: Let the Clicker Beware

August 28, 2007 2:05 PM

Feathers have been flying lately about a couple of online ballots posted on ABCNews.com after the recent ABC-sponsored presidential debates in Iowa. The alleged “winners,” if you choose to buy into these compilations of clicks, were Republican Ron Paul and Democrat Dennis Kucinich. Cue howls of outrage from their respective partisans, protesting that ABC hasn’t adequately reported the so-called results.

As my 12-year-old would say, chillax.

The reality is that these things are not polls or surveys, nor does ABC News identify them as such. They are called “online ballots.” They’re posted to encourage a sense of community and participation. But while the clicks they receive are tallied, they never are percentaged. And they’re supposed to carry this disclaimer: “Not a scientific survey. For entertainment only.”

Hold that thought – “for entertainment only” – as we dig a little deeper. Because ABC policy goes further, to require that any subsequent use of these ballots, beyond simply posting them, must note their vulnerability to outside manipulation.

That’s the real issue with online click-ins. Pointing out that they’re produced by self-selection and thus not reliably representative of any broader population is true, and fatal; but it sounds like a technicality – a talking point for the Sominex-sponsored Third International Colloquium on Inferential Statistics. More fatal (so to speak), because it’s at once easier to grasp and more immediately threatening, is the fact that these things can be, and often are, intentionally manipulated by groups or individuals with an interest in the outcome.

It happens all the time. People who want to stuff the ballot box just forward around the click-in’s URL, burying it in an orchestrated cascade of votes for the favored person, position or point of view. Others go a step further, building automated voting bots that jack up the tally for the pre-selected winner. Did this or that debate "winner," say, get clicks from 15,343 people – or from one person clicking 15,343 times? It can be impossible to tell.

Often we don’t know for sure when campaigns to manipulate online ballots occur; there aren’t always smoking guns. We have, though, found a posting on a meetup.com page, urging readers to vote for Paul in the Iowa debate ballot ("Ron Paul Winning ABC Debate Poll! Vote Now!") and to distribute the link elsewhere. The call to arms: “Lets keep RON PAUL ON TOP!” (sic). And Kucinich links to the ballot from his own campaign website; the headline reads, “Kucinich's Lead Keeps Increasing - ABC Debate Poll.”

None of this is remotely new. With the help of a hyperactive online community, Alan Keyes smashed the opposition as winner of a Republican debate in New Hampshire in December 1999, with 49 percent (against five opponents) in a Fox News/Vote.com online ballot. Sadly for the clickers, Quinnipiac University conducted a real poll (that is, a representative, random-sample telephone survey) on the same debate; Keyes got 13 percent, far behind John McCain (who, as it happens, went on to win the primary).

Nearly a year later, on Oct. 4, 2000, the day of the first presidential debate in the general election, Republican National Committee Chairman Jim Nicholson sent an e-mail to his entire membership, urging the nation’s Republicans to click in on ABCNews.com and CNN.com ballots to show their support for George W. Bush. That night, 58 percent in the ABCNews.com ballot picked Bush as the debate’s winner; by contrast, in actual polls by ABC News, Gallup and NBC News, he got 39, 41 and 36 percent, respectively. (Our prohibition on percentaging click-ins came later. Its aim simply is to make them less attractive to would-be manipulators.)

Some members of the FreeRepublic.com online community even have a name for this - they call it "freeping" - and when their side is losing they go so far as to "reverse freep," boosting the other side's vote to make the result so lopsided it's unbelievable. "REVERSE FREEP...the fix is in...they are cheating so vote for Kerry...99-1 invalidates online polls since they are invalid anyway," read one posting in the heat of the 2004 presidential campaign.

The gamesmanship goes far beyond election politics. For some it's a business: then-New York Attorney General Eliot Spitzer's settlement in a payola case against Universal Music Group in 2006 disclosed that UMG had paid a "request company" to "jack TRL (MTV's "Total Request Live") for Lindsay (Lohan)." I’ve got a thick sheaf of manipulated online ballots on everything from Tom Cruise to Don Imus to Drum Corps International. (Some joker jacked the results of the lineup ballot for their 2005 Masters of the Summer Music Games in Murfreesboro, Tenn. How low can you get?)

Rich Morin, former polling director at The Washington Post, wrote a priceless description of the issue nearly a decade ago, featuring, among others, Hank the Angry, Drunken Dwarf as People magazine’s Most Beautiful Person of 1998. (The piece still lives here, in what Morin presciently called “the world’s newest and most cluttered attic.”)

Today it’s online click-ins; in an earlier time it was 1-800 call-ins. Back in 1990, for instance, USA Today invited its readers to call in and say whether they liked or disliked Donald Trump. We love the Donald, came back the headline – a smashing 81-19 percent rout in Trump’s favor. At least until a correction appeared in the paper a month later, reporting that an audit had found that 5,640 of the 7,802 pro-Trump calls had come from precisely two phone numbers at an insurance company in Cincinnati owned by an admirer of Trump’s. Thank heaven for redial.

For all its internet fizz, then, this really is just old snake oil in new bottles. Sometimes it's fun and games. Other times it rises to a more serious level – misinformation, even downright disinformation. If it's your thing, click away. Just remember: When it comes to online ballots, if there’s a buck to be made or a point to be scored, chances are very good that someone, somewhere, has a finger on the scale. It's not remotely a scientific survey. And it is, decidedly, for entertainment only.

August 28, 2007 in Favorite Posts | Permalink | User Comments (29) | TrackBack (0)

Gonzales and the Base

August 27, 2007 11:21 AM

While the controversy that today ended Alberto Gonzales’ tenure as attorney general has not been front and center of public concerns, he’s hardly been popular – perhaps most tellingly, lacking a strong partisan base that might otherwise have shored him up.

In an ABC News/Washington Post poll we conducted in June, 52 percent of Americans disapproved of the way Gonzales handled the U.S. attorney firings, while only half as many, 25 percent, approved. Barely over a third, moreover (35 percent), were prepared to say Gonzales should keep his job; 43 percent though he should lose it.

On both these questions, quite a lot of people – 23 and 22 percent, respectively – had no opinion, an indication of inattention to the issue. To some it's probably seemed like more of the usual political fare from Washington, overshadowed by more pressing issues, especially the 800-pound gorilla of current public concerns, the war in Iraq.

All the same, the lack of partisan support for Gonzales among Republicans and conservatives is telling, and perhaps sealed his fate. In our June poll just 38 percent of Republicans approved of how he handled the U.S. attorney firings; nearly as many, 32 percent, disapproved. And just under half of Republicans, 48 percent, said Gonzales should keep his job - a decidedly weak showing in the base. (Criticism of Gonzalez on both fronts soared among independents and Democrats alike.)

The numbers were similar among ideological groups; just 43 percent of conservatives were ready to say Gonzales should keep his job, while his support, naturally, slipped lower among moderates (32 percent) and liberals (27 percent) alike. Nor were George W. Bush’s expressions of support much help, given that Bush himself these days has little spare popularity to loan to his subordinates.

We don’t see any polling specifically on suggestions that Gonzales may have misled Congress – or even perjured himself – in testimony about National Security Agency surveillance programs. But the subject of truthfulness is a sensitive one for this administration; again in our June poll, 60 percent of Americans said they don’t feel they can trust the administration to honestly and accurately report intelligence about security threats facing the United States. Given that credibility gap – and whatever Bush's expressions of support for Gonzales – an attorney general accused of misleading Congress was probably the last thing this White House needed.

August 27, 2007 | Permalink | User Comments (0) | TrackBack (0)

Polling in Iowa: To List, or Not To List

August 03, 2007 12:34 PM

Our poll on the presidential race in Iowa is one of those in which as much thinking went into the sample design as the data analysis. And in the end, in climbing this particular hill, it’s fair to say we did not take the standard route.

Sampling for Iowa caucus polls tends to be based on the secretary of state’s list of all registered voters in the state. This kind of list-based sampling is popular in many quarters because it’s efficient (read: less expensive) in reaching likely voters – you don’t have to call people who aren’t registered to vote, and you don’t have to wonder if people are reporting their registration accurately. They’re either on the list, or not.

But I have problems with list-based sampling. Coverage is a real concern – and frankly an under-discussed one – in survey research. This refers to the share of the population that’s included in the sample. In the best research, everyone in the population you’re studying has a chance of inclusion. When you exclude people – noncoverage – you run the risk of introducing bias.

International polls are a good example. A lot of second-rate research in foreign countries is based on urban-only samples, because it’s cheaper and easier to produce, and it’s generally carried out by market research firms that care more about markets than about full populations. Many of these polls have vast noncoverage – everyone who lives outside a major city is systematically excluded from the sample. Imagine an urban-only poll in the United States. President Kerry would love the results.

So to Iowa: According to a colleague, J. Ann Selzer of Selzer & Co. in Des Moines, who consulted with us on our poll (and polls for the Des Moines Register), about 15 percent of the names listed on the Iowa secretary of state’s list don’t have phone numbers in the file. That’s noncoverage.

An additional seven percent are excluded from the file because they’ve been designated as “inactive” registereds. They might have moved – out of the state, or simply within it – or died, or just not have voted in a long while. They’re not covered – some appropriately, but some not so.

Then there’s the accuracy of phone numbers that are on the state’s list. The Pew Research Center used the list in a poll it conducted among Iowa Democrats in late 2003. According to sample disposition data posted on its website, it found 17 percent of the numbers to be non-working, including a few that connected to faxes or businesses. Further noncoverage.

Add it up, and it’s a lot of noncoverage – certainly enough, potentially, to affect estimates. And it’s a lot more noncoverage than you see in polls done the old-fashioned way, by randomly dialing a sample of all possible landline telephone numbers. Yes, cell phones are excluded in a landline sample, but that produces far less noncoverage than in list-based sampling. (Lists can include cell phones, which in theory is a good thing. But by law cell numbers can't be called via automated dialer - and list users don’t know what is and isn’t a cell number in the list.)

Another concern with list-based sampling is how to weight the data – to match it up with population norms. Some weight to sex and age as available on the secretary of state’s list. Others (such as Pew in 2003) don’t apply any sample-balancing weights at all. Good quality national polls, by contrast, are weighted to Census norms, customarily age, race, sex and education. Weighting to empirical population data is like truing up a wheel.

Maybe the best argument for list-based sampling is that you don’t have to waste resources calling up a whole lot of people who aren’t registered to vote. In Iowa, though, this doesn’t make much sense to me, because registration is very high – 89 percent of the voting-age population is registered, including 83 percent identified as “active.” It saves you some calls, but not all that many. (It's also been suggested that to poll accurately in Iowa you have to go beyond the registered voter list and buy a list of actual previous caucus-attenders. That's just bizarre, because it means that first-time caucus goers are entirely excluded. And in the 2004 entrance poll, 55 percent of Democratic attendees said it was their first-ever Iowa caucus.)

This brings up another issue in election polling, likely voter screening. Some polls of likely caucus-goers, or likely voters elsewhere, may include lots of people who aren’t really likely to vote at all. Drilling down, again, is more difficult and more expensive. But if you’re claiming to home in on likely voters, you want to do it seriously. Anyone producing a poll of "likely voters" should be prepared to answer this question: What share of the voting-age population do they represent?

List-based sampling, carefully done, can produce a good estimate; Selzer’s Iowa Poll had the order of finish right in the 2004 Democratic caucuses. But – you can see where this is headed – we chose to do our own Iowa survey by old-fashioned random-digit dialing. We drilled down well into the population to get a meaningful estimate of likely caucus-goers on each side. (See details in the poll reports.) And we've produced what I feel is a distinctive poll of likely caucus-goers in Iowa – not just in its analysis of the data, but in the rigor of the sample behind it.

August 3, 2007 | Permalink | User Comments (0) | TrackBack (0)