Another fast-solidifying bit of conventional wisdom from the 2012 election is that “the polls were right,” and anyone who questioned them was foolish.
But that’s not true. “The polls” weren’t “right.” A lot of them accurately predicted the general outcome; very few of them got the margin of victory right. Some were clearly much more accurate than others. For example, the final pre-election CNN poll showed a tie race with a +11 Democrat sample and a 22-point lead for Mitt Romney among independents. The race wasn’t tied, Democrats didn’t have an 11-point turnout advantage, and Romney only won independents by about 5 points.
The event the polls were “right” about was essentially a binary choice: either Obama would win, or Romney would win. No matter how wrong it might have been about the specifics, every polling series had, in essence, a 50-50 chance of getting the ultimate outcome right. Actually, quite a few of them projected inscrutable razor-edge ties that meant the coin toss of the election would have landed on edge, and that is not what happened.
Elections are conducted over time. Some of the polls that were right about the ultimate outcome were wrong at various points during the campaign. And polls are quantum events – they both observe the news and influence it. In fact, polls comprise the bulk of the news during the final weeks of an election. The media talks about little else, especially when other news would damage the candidate they favor, but his polls don’t look too bad.
And everyone gets drawn into the game. Campaign professionals are understandably hungry for instant feedback on the success of their efforts. Even we outlaw right-winger blogger types respond to whatever dominates the news; we want to cover what everyone is talking about. A month out from a historic election, we want a functioning crystal ball as much as anyone else in the media. Prophecy has always been an incredibly powerful temptation.
Here’s a thought experiment: suppose Mitt Romney was actually pretty well in the lead during the weeks that many big polls were stuffing big Democrat over-samples into their models and declaring ties or small Obama leads. Suppose this had been reported universally, and all of the big media operations put Romney 5 or 6 points ahead for many days in a row. The headlines would have been all about Obama’s impending doom, and that might have prevented him from building the momentum he needed to position himself for a win. A lot more of his dejected voters might have stayed home, thinking there was no way he could win. The final electorate might not have been Democrat +6. Some serious surveys taken before the election had predicted something much closer to an even split, but as we now know, a great many Republican voters stayed home.
Neil Stevens at the Unlikely Voter blog noticed that in the course of taking a victory lap about the accuracy of his polls, Public Policy Polling director Tom Jensen allowed that his operation projected certain levels of African-American, Hispanic, and young voter turnout on what amounted to a “well-informed but still not entirely empirical hunch.”
“I don???t remember anyone willing to say PPP was actively rigging the polls to reach chosen results,” grumbles Stevens, “but there it is in black and white. Jensen decided in advance what he wanted the electorate to look like, and so tweaked the numbers until he got what he wanted.”
And in the course of doing so, created the conditions that helped the right number of voters appear at the polls on Election Day. This is not to say the whole election was a fantasia whipped up by cooked polling data. Many forces were at play; that’s how national elections work. But to offer another thought experiment, suppose the Republicans had somehow taken over all the big polling operations (where are the Koch Brothers when you really need them?) and pumped out a stream of cooked polls showing huge Romney leads. That probably would have significantly affected turnout on Election Day, wouldn’t it?
For my part, I come away from Election 2012 with roughly the same opinion of polls that I entered with: they’re not very precise at individual moments, even when they manage to be generally accurate in the broadest strokes over the long run, and in any event even the best of them are measuring unpredictable sentiments from a population that doesn’t really like talking to pollsters. The polling picture only really seems to firm up in the last few days before an election, and elections last for a few days thanks to early voting, so opinion polls seem to achieve their greatest accuracy concurrently with “the only poll that really counts.” It’s not that they’re useless before that, and I think most of the sampling errors are mistaken guesswork rather than sinister conspiracies, but collectively they strike me as educated guesses that inevitably change the variables in the equation they are trying to solve.