Political Polling Redeemed

by DJI

“I recognized that there is wisdom in seeing the world from a different viewpoint.” – Nate Silver

How did Nate Silver, who burnished the reputation of pollsters by using polling data, accurately predict the election outcome in all 50 states?

The Tricky Part

The whole premise of polling is that by taking a representative sample of a given population, you can draw conclusions about the attitudes and behaviors of the whole. (You can argue about how “representative” is defined, and there are different ways to set about reaching your sample, but the statistics themselves are standard.) However, polls do not tell the whole story – not by a long shot.

Nate Silver used polling data from multiple organizations as the fundamental building blocks of his successful algorithm. Aggregating polls, as the Huffington Post did, has the effect of increasing the sample sizes, and therefore the statistical accuracy of the research; it also minimizes the risk that one or two polls might be skewed. But Nate did not just aggregate all available polling data. He was a bit picky about which polls he included, and those that were, he weighted, using past performance records, fielding techniques and methodology, and how each firm defined “representative” in his evaluations.

He also layered in economic information to inform his predictions such as job growth, income levels, productivity, consumption, inflation, and projected GDP. Some might argue about whether or not this qualifies as “Big Data”. Maybe it’s somewhere in between: reportedly he ran his predictive models on “a beat-up laptop” at home, but obviously the average person can’t do this.

The Simple – But Hard – Part

However, there are things anyone can do to improve the accuracy of their predictions. In his blogs, interviews, and in his book, The Signal and the Noise, Nate emphasizes the importance of analyzing facts honestly. Instead of dismissing information that doesn’t support the narrative you would like, look for consensus in the data.

In a world where we have way too much information, it’s much easier to pick and choose the stuff you like. As researchers, we often caution our clients about making the same mistake, especially in observing focus groups. It’s very easy to focus on the respondents who more closely mirror your own point of view. Nate believes that successful marketers must consider a more diverse range of possible strategies and over time, learn how to recognize when you have enough information, and the right types of information, to warrant a strategy change.