How to make polling data dance on command
Polling news is the perennial temptation of political journalism. It’s tough to keep up with the flood of polling data during high election season. Attention is drawn to “shock polls” with dubious methodology. Local polls in swing states can ultimately prove more significant than national numbers. Movement and momentum can be more important than today’s snapshot… but obsessive news coverage of today’s poll numbers influences momentum. You can turn to averages instead of individual polls, in search of more reliable numbers, but outlier polls distort the averages.
And there is great disagreement, among both partisan and relatively impartial observers, over what the polling numbers for a successful challenger or incumbent “should be,” weeks ahead of the election. Nationally, it’s a virtual dead heat right now, on the eve of the first presidential debate. Tuesday’s Rasmussen tracking poll calls it a one-point race, 48-47 Obama-Romney. Is that bad for Romney, given the general awfulness of Obama’s record, and all the grim foreign and domestic news currently filling the airwaves? Or is it bad for Obama, because successful incumbents are generally a bit further ahead at this point in the race, with higher approval ratings? It will all seem so very clear in retrospect.
There are always questions raised about polling methodology, but this particular election has seen a remarkable number of just plain crazy samples. Democrats are routinely oversampled at rates far beyond anything justified by either the 2008 or 2010 electoral results, producing ludicrous manufactured “edges” for Obama. This is even less believable in light of reports that Republican registration has been increasing in many states, while Democrat registration has been falling.
There is strong suspicion among conservatives that this was done to create perceived momentum in Obama’s favor, depress the Republican base during a highly polarized election, influence the shocking number of people casting early ballots, and/or terminate a growing narrative of Obama failure that was giving the Democrat base a serious case of the blues. One might even wonder if the media is trying to set the stage for post-election chaos, creating a scenario where enraged Democrats contest a Romney victory because it defies weeks of polling data favorable to Obama. Something like that happened in 2004, when Bush’s solid win over John Kerry contradicted polling news swirling across the left side of the Internet on Election Day, leading to charges of electoral theft.
Two recent, deeply flawed but widely reported polls bring everything wrong with these surveys into focus. The Washington Post actually reported as serious news – in fact, as headline news – a swing-state poll with only 169 respondents and an 8 percent margin of error. This poll was presented as evidence that while all the national polls showed a tight race, and the final demise of Obama’s “convention bounce,” Obama was supposedly building up big leads in the battleground states.
“In recent weeks, the media have reported on the ‘debate’ about skewed polling. But, there really is no debate. They are, in fact, inflating likely Democrat turnout. And when that isn’t enough to support Obama, they reach down for meaningless statistics within their poll to boost him. This is fraud,” growled Mike Flynn of Breitbart News. I found myself wondering if all 169 respondents were sitting in the same Starbuck’s coffee shop when the poll was taken. Oddly enough, the media never seems eager to create headlines with a polling sample the size of a jetliner’s passenger list that skews, even gently, towards Republicans.
The other weird poll of the day is from Qunnipiac. In defiance of other polls showing big Romney leads among independent voters, this poll found Romney only 2 points ahead with indies, and claimed to discover an 18-point lead for Obama among women. This led to an overall four-point lead for Obama.
Now, Obama may have a very significant lead among women (and Romney has one among men) but 18 points is difficult to swallow. The Pew Research Center found closer to a 12-point gap using 2011 data, and the trend was moving toward Republicans.
But then bloggers got hold of the sample data for the Quinnipiac poll, and discovered it was simply absurd: only 24 percent Republican, against 30 percent Democrat and 38 percent independent. That bears no resemblance whatsoever to the American electorate. Once again, the corrupted sample renders the poll’s conclusions meaningless.
Writing at the Washington Examiner, respected number-cruncher and self-described “recovering pollster” Michael Barone notes that in an age of extensive call screening and cell phone usage, “the Pew Research Center reports that it’s getting only 9 percent of the people it contacts to respond to its questions. That’s compared with 36 percent in 1997.” (Emphasis mine.)
Barone doubts that major media polls are deliberately slanted, but he makes an interesting point that since getting responses to telephone polls has become so difficult, fleeting surges of voter enthusiasm have more of an impact on the results. “There is evidence that since the Charlotte, N.C., convention, Democrats have become more motivated to vote and have narrowed the advantage in enthusiasm Republicans have had since 2010. In which case more Democrats may be passing through screening questions and getting polled,” he suggests.
Whatever the true nature of the polling landscape, it’s probably a mistake to assume the polls are collectively and entirely worthless, or simply cherry-pick the ones you like to reinforce your hopes about the current standing of your preferred candidate. The problem is that media reports tend to credit the polls with an authority they simply do not possess – not when they’re based on 9 percent response rates, and even the most sincere efforts to weight the results by party affiliation are as scientific as astrology. General movement over a period of days is probably a more worthy topic of observation… but that’s not going to generate any juicy headlines, or give heads floating above round tables anything exciting to talk about.
And the reporting, in turn, influences subsequent polls… especially in an environment where voter enthusiasm has such a strong effect on poll response rates. Today’s headlines are increasingly built around polls that primarily reflect the respondents’ best guess about what everyone else thinks of yesterday’s polling headlines.