Discuss as:

When pollsters get it wrong

For the bustling industry comprised of political pundits, pollsters, strategists and scribes, the weeks before Election Day are a flurry of predictions about who will win the nation’s most competitive races.

When there’s plentiful public polling and stable national trends to base their assumptions upon, their prognostications are usually more or less close to the mark. But politics is unpredictable, and pollsters use different methods to determine who’s up and who’s down. Every once in a while, they get it wrong.

The famous erroneous 1948 “Dewey defeats Truman” banner headline in the first edition of the Chicago Tribune was the result of polls and conventional wisdom that turned out to be dramatically off base. After Los Angeles Mayor Tom Bradley’s unanticipated loss in the 1982 California governors’ race, many blamed racial bias undetected in public opinion polls. After Barack Obama’s dramatic win in the Iowa caucuses in 2008, the media’s overwhelming assumption that he would continue his march to victory days later in New Hampshire was due to polling. Those assumptions were shattered when Hillary Clinton wound up winning -- prompting weeks of media navel-gazing and questions of "how did we get it so wrong?"

In this midterm cycle, new poll results have hit political reporters’ e-mail inboxes at a fast and furious pace. But a quick look at the wide disparities between different polls of the same races indicates that they can’t all be right. Four recent public polls in the tight Nevada Senate race show Senate Majority leader Harry Reid everywhere from down four points to up three. Much-discussed “tightening” in the Pennsylvania Senate race two weeks ago gave way to another wide lead for Republican Pat Toomey days later.

Part of the reason for the disparity between the results, says Democratic pollster Mark Mellman, is because every polling company has its own method to determine which voters are “likely” to turn out.

“Some of the measures that people use to determine whether someone’s a likely voter sometimes have no relationship whatsoever to whether people actually do turn out or not,” Mellman said.

His example: Some pollsters use voters’ self-reported “enthusiasm” as an indicator of which voters are most likely to make it to the polls. But even though it seems like fired-up voters should be the most reliable ones, there isn’t always an exact correlation. Voters who are lukewarm at best about their candidates may still feel an obligation to cast a ballot; passionate citizens who are contacted by pollsters in mid-October may ultimately procrastinate when Election Day comes.

Another reason for variation is that some polls, especially those that use automated recordings rather than live questioners, cannot legally contact cell phone users. That means that many pollsters are limited to contacting landline users, who are likely to be older and less transient.

(The NBC News political unit, like many other major news organizations, does not use automated interactive voice response (IVR) polls in determining its race ratings. There are a plethora of such surveys, in part because IVR polls can cost as little as 10 percent of the cost of a live-interview poll.)

Additionally, survey information that is collected in the course of a single night tends to be biased towards what Mellman calls “the easy-to-reach” – voters who are more likely to be home in the evenings to answer the phone. “People that are harder to reach are much more likely to vote for Democratic candidates,” Mellman argues. “If you just look at the people you get on the first dial, you have a sample that looks much more favorable towards Republican candidates.”

While there are many different public polls of statewide gubernatorial and Senate races, there are far fewer widely-released surveys of House contests. Campaigns conduct internal polls that often offer fairly accurate snapshots of races, but they generally only release them to the media if the numbers are good for their candidate.

“You’ve got a lot of House polls that are selectively cherry-picked for the press,” says David Wasserman, who tracks House races for the Cook Political Report. That makes the states of individual races much more difficult for voters to wrap their heads around, he said.

He added that midterm elections, because they don’t have the reliably high turnout of election years with a presidential race at the top of the ticket, can be more prone to surprises.

“Smaller electorates are more difficult to predict,” said Wasserman, whose latest rankings show a whopping 100 competitive House races, including 50 tossups.

Still, although individual races may be difficult to gauge, most pundits agree that Republicans are poised to take control of the House but that the 10 seats they would need to take the Senate are likely out of reach.

And there’s one thing that’s for sure.

“Democrats will have a tough night,” said NBC’s Chuck Todd. “It's how tough that is the unknown.”