In the latter days of the 2014 campaign, it became a bit of intellectual comfort food for Democrats to believe that the polls were skewed toward Republicans. They mocked Republicans for unskewing polls in 2012, but then indulged in the same pretenses themselves, for the same reason: it’s the only way to keep your voters motivated when the forecasts predict tough political weather coming your way. Pundits may grumble about how poll-unskewing efforts lead to poor op-ed predictions, but from the standpoint of political leaders, there’s really no downside to telling your voters they still have a chance to prevail in a tough race. In fact, that’s thought to be the best possible mindset under all circumstances, as movement leaders fret about “complacency” causing indolent voters to sit home on Election Day, if they think their party has the election in the bag. It’s best to keep the faithful from believing that voting is either futile or unnecessary.
As it happens, the Democrats were basically right about the 2014 polls being off… but they were wrong about the direction. The polls were widely slanted against the Republicans. That’s why there were so many shocker races, in which candidates thought to be dead in the water managed to win slender victories or force recounts.
Also, a lot of the marquee Republican victories were much more solid than polls predicted. Senator Pat Roberts of Kansas, for example, was said to be a dead heat with “independent” Greg Orman, who was actually a modest favorite to win when the polls opened on Election Day. Roberts ended up winning by a very substantial 52-43 margin. In this case, it has been suggested that many Kansas voters were essentially being less than completely candid with pollsters – they were unhappy with Roberts, but when push came to shove, they voted for him anyway.
Okay, but what about the new Senate Majority Leader, Mitch McConnell, blowing out Allison Grimes in Kentucky by an eye-popping 15 points. Grimes was supposed to be a serious threat, occasionally popping out with small leads in the polls. The last round of polling before the election gave McConnell an average lead of about seven points. He more than doubled that margin.
All the pollsters thought Republican David Perdue in Georgia had a margin-of-error lead at best, with no real chance of avoiding a runoff against Democrat Michelle Nunn. He ended up crushing her by 8 points, no runoff necessary. Tom Cotton was favored to win in Arkansas, but not by an atomic 17 points.
A month ago, mainstream-media reports and left-wing blogs sizzled with rumors that Republican Mike Rounds in South Dakota was in serious trouble against independent Larry Pressler. That was based almost entirely on one outlier poll; Rounds soon retook a solid 8 to 10 point lead. He ended up winning by 21 points. Even in close races, Republicans won against the odds, as with Thom Tillis edging out Kay Hagan in North Carolina. The Beltway media is headquartered right next door to Virginia, where Democrat Mark Warner was supposed to cruise to an eight- or nine-point victory, but ended up in a thousand-vote nail-biter against Ed Gillespie.
In fact, according to polling guru Nate Silver of FiveThirtyEight, this year’s polling “overestimated the Democrats’ performance by 4 percentage points,” in both Senate and gubernatorial races. Silver makes the case that polling bias tends to run toward one party or the other – by roughly the same average margin! – but the favored party changes on a “largely unpredictable” basis – it benefited Republicans in 1998, 2006, and 2012, but ran against them in 1994, 2002, and 2014. He observes that the polling on gubernatorial races was actually much worse this time around than on the Senate races, citing the example of Republican Larry Hogan collecting a nine-point victory in Maryland despite never having led the race in a single nonpartisan poll.
Why does this happen? Silver offers a few theories, and is game enough to pin some of the blame on polling aggregators, such as himself:
It may be that pollster “herding” — the tendency of polls to mirror one another’s results rather than being independent — has become a more pronounced problem. Polling aggregators, including FiveThirtyEight, may be contributing to it. A fly-by-night pollster using a dubious methodology can look up the FiveThirtyEight or Upshot or HuffPost Pollster or Real Clear Politics polling consensus and tweak their assumptions so as to match it — but sometimes the polling consensus is wrong.
It’s equally important for polling analysts to recognize that this bias can just as easily run in either direction. It probably isn’t predictable ahead of time.
To the extent polling bias is predictable, it may call for assessing nonpolling factors — the so-called “fundamentals” — along with the polls in each race. One simple factor is the overall partisanship of a state as measured by its past voting history. In the past, Republicans have tended to outperform their polls in red states while Democrats have done so in blue states.
That’s an interesting notion I’d like to hear elaborated upon in the future: pollsters tend to under-estimate the strength of parties on favorable political terrain? Are they over-compensating for local preferences they somehow deem insincere, or is there a strong tendency toward the phenomenon that might have caused predictions to fail so spectacularly in Kansas: grumpy but essentially loyal Party voters dumping on their candidate during the race, but coming home to him in the end?
The seriously malignant effects of polling biases are the way they change party behavior, leading lost causes to absorb excessive resources while unsuspected opportunities are missed; and the way polling shapes media coverage, which in turn tends to make an impression on party strategists, and especially party donors. In this election cycle, Democrat donors were suckered into throwing a lot of money at candidates who never really had a chance, but looked viable because of that four- or five-point Democrat polling bump. Republicans missed a great opportunity in Virginia because they didn’t think Ed Gillespie’s brave campaign would come so close.
Perhaps wave elections confound pollsters because the wave is built by breaking news and mounting public perceptions; a losing party stuck in political quicksand can make it situation worse by flailing around. Nate Silver promises to make his FiveThirtyEight model better by working around the four-point bias as best he can, although he takes pains to point out that it’s difficult to tell which party the bias will point toward in advance. And how can one factor out the meta-distortion of polling averages feeding off each others’ results and assumptions? Polls have always been self-creating quantum news events – the poll becomes the story, shifting the very attitudes it was intended to measure. The media wants drama, so polls tighten as races draw near, and “registered voter” screens give way to far more accurate “likely voter” models. As with many other endeavors, the people with the best information tend to be the ones putting cold cash behind their data… but they’re also the wizards who don’t want to reveal too much about their secret divination methods to competitors.
