The Numbers
A Run at the Latest Data from ABC's Poobah of Polling, Gary Langer
Gary Langer is director of polling at ABC News, where he's covered the beat of public opinion for nearly 20 years - conducting and analyzing ABC News polls, evaluating data from other sources and setting the news division's standards for poll reporting. Langer is a two-time Emmy award winner, both for ABC's reporting of public opinion polls in Iraq.
FAVORITES
RECENT POSTS
MONTHLY ARCHIVES
| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| 7 | 8 | 9 | 10 | 11 | 12 | 13 |
| 14 | 15 | 16 | 17 | 18 | 19 | 20 |
| 21 | 22 | 23 | 24 | 25 | 26 | 27 |
| 28 | 29 | 30 | 31 |
« Previous | Main | Next »
Cell-Onlies: Report on a Test
September 19, 2008 10:36 AM
10/2 update: In respose to requests for details, I'm attaching a list of landline and landline-plus-cell-only comparisons here.
-
Including cell-phone only respondents in a political poll produces a negligible impact on overall results.
An ABC News/Washington Post poll done to test the approach found slight changes at most when cell-only respondents were included with a traditional land-line telephone sample. Across 100 response categories the biggest difference was 2 points, which occurred in three cases. The rounded difference was 0 in 56 of the comparisons, and 1 point in 41.
In the hothouse of an election campaign the horse race is of particular interest. Registered voters in this late August poll divided between Barack Obama and John McCain by 49-43 percent in the landline sample and 49-42 percent with cell-only respondents included. Among likely voters, it was 49-45 percent among landline respondents and 50-44 percent with cell-only respondents included.
The precise changes were 0 for Obama and -1 for McCain among registered voters, +0.7 for Obama and -0.8 for McCain among likely voters. For context, the survey’s margin of sampling error is +/- 3 percentage points for registered voters, +/-3.5 for likely voters.
Cell-only respondents accounted for an estimated 15 percent of the adult population in the most recent federal data, with their numbers growing. They can be and increasingly are being included in national polls, though other studies, similarly, have not shown a significant impact.
In this survey, George W. Bush’s approval rating was an identical 30-66 percent, approve-disapprove, with and without the cell-only sample. The poll’s “right direction-wrong track” reading was an identical 19-78 percent in both approaches. The numbers of Democrats and Republicans in each sample were identical. Identical numbers said the Iraq war was not worth fighting, and identical numbers had favorable and unfavorable impressions of McCain and Obama.
The economy was called the most important election issue by 43 percent in the landline sample, compared with 41 percent with cell-only respondents included. In another 2-point difference, the number who said Obama had spent more time attacking his opponent than addressing the issues was 29 percent in the landline sample, vs. 27 percent with cell-only respondents included.
There are two reasons for the negligible effect. One is that cell-only respondents are not different enough from the landline population – and at 15 percent, there are not enough of them – for their inclusion to make much difference in overall estimates. The other is that customary weighting to Census norms helps account for their absence.
For example, while cell-only respondents are disproportionately young (54 percent are under age 30), the views of young cell-only respondents are very similar to those of young landline respondents. And since a landline sample is corrected to the proper proportion of under-30s, it’s in effect adjusted for the absence of young cell-only users.
In this poll, for instance, among registered voters, 57 percent of cell-only respondents under age 30 supported Obama, compared with 64 percent of landline respondents under 30; the combined number is 62 percent. (The margin of sampling error is higher for this and any smaller-size subgroup.)
Another factor is that cell-only respondents are less apt to be registered or likely voters, further mitigating their impact in polls of these groups. Sixty-one percent of cell-only respondents in this poll said they were registered, compared with 78 percent of landline respondents. Sixty-seven percent of registered cell-only respondents said they were certain to vote in November, compared with 84 percent of landline respondents. And registered cell-only respondents were 14 points less likely to be following the election closely, 70 percent to 84 percent. The prevalence of young people in the cell-only population is one reason for these differences.
Inclusion of cell respondents is an early-stage practice with a variety of approaches in use. Some polls include all cell phone respondents, including those who also have a landline at home. Those polls may or may not be weighted for double-coverage of people who can be reached both ways. Others, such as this test, screen cell respondents to identify and interview only those who don’t have landlines. The number of cell- and/or cell-only respondents included may also vary; the 15 percent cell-only figure used in this survey is based on data from the in-person National Health Information Survey conducted by the Centers for Disease Control in December 2007, reported in May 2008.
There also are different approaches in combining cell and landline samples. In this survey, cell phone numbers were stratified by region, and cell-only respondents were weighted by region using NHIS estimates. Region weights were then adjusted for the landline sample to accommodate the addition of cell-only respondents. The combined landline and cell-only data were weighted to Census norms for the full population.
Different approaches to sampling and weighting can produce different results, and while these approaches are being tested it’s best to regard the inclusion of cell and/or cell-only respondents as experimental. But if the number of cell-only Americans continues to grow, the preparation will prove to have been necessary.
September 19, 2008 | Permalink | User Comments (8)
TrackBack URL for this entry:
http://www.typepad.com/t/trackback/433071/33627764
Listed below are links to weblogs that reference Cell-Onlies: Report on a Test:
Thanks - This answers a question that comes up all the time and is very informative and useful information.
Posted by: Mark David Richards | Sep 19, 2008 11:27:09 AM
Wait -- a 50% shift in the gap between the two candidates among likely voters is "negligible"?
In a race that appears extremely close, a methodology difference that opens up an extra 2 points between Obama and McCain has HUGE implications.
I'm baffled that you could write off a shift like that as "negligible."
Posted by: Aaron | Sep 19, 2008 11:42:56 AM
another thing that i will ad is that if i see a unidentified or a company name calling me on my phone that i have no ties to, i wont answer....
what is the caller id of these polling places
if it says cbs polling center or quinnipac polling center i would answer
i think other cell phone users would do similar, although i can only speak for myself
Posted by: Bhrandon | Sep 19, 2008 12:18:38 PM
"57 percent of cell-only respondents under age 30 supported Obama, compared with 64 percent of landline respondents under 30;"
Defeating the myth: Most people thought Obama' support among young voters with cell-only was higher than among young voters with landline phones.
Good job
Posted by: Angel | Sep 19, 2008 12:36:13 PM
Obama: "I'll just set this one out until I know the results. That way, I can freely criticize those who actually acted."
Posted by: dl | Sep 19, 2008 12:44:20 PM
These "neglible" differences are greater than the margins that decided the last two presidential elections. I think they're important.
Posted by: karela | Sep 19, 2008 3:40:21 PM
BING AREN'T THEY SUPPOSE CAMPAIGN IN A PLACE WHERE THEY'RE NOT IN THE LEAD, IF IT WAS INNER CITY TAMPA AND 60,000 SHOWED UP, THEN THAT WOULD BE WONDERFUL, THATS SAYING SOMETHING, BUT NOT A MILITARY RETIREMENT COMMUNITY PLEASE! OF COURSE EVERYONE WOULD SHOW UP
Posted by: SHE MI | Sep 22, 2008 1:36:31 PM
I totally agree with the comments of Aaron and the comments of Bhrandon. Yes, Mr. Langer may be a two-time Emmy Award winner but, perhaps he may be a little too old to really connect with the progressiveness of technology and the technological mindset of the cell only phone user in conducting polls.
He states he conducted a poll, "Across 100 response categories." I would think that you would have to have a larger response base to reach the conclusions express in this article. It is too quick to rely on what small data was collected.
Anyone doing sound research, on fickle cell only phone users who base the answering of the phone on what their caller id indicates, is only fooling themselves and relying on faulty data. Another mechanism has to be used in sorting through the data.
Time will tell, when the election results come in; when the nation sees a blowout and debate how off the polls really were. This is my prediction.
Posted by: Sheree | Sep 29, 2008 9:11:49 PM
Post a comment
