How to Think About Polls In One Tweet
There was a ton of wisdom about how to read polling these days from leading Democratic pollster John Anzalone yesterday, in a tweet about a Monmouth survey showing Joe Biden, Bernie Sanders and Elizabeth Warren tied at 20% apiece:
Four generalizable lessons here:
Don’t over-react – Anzalone is imploring us: don’t read too much into one poll. Different pollsters use different methods, sample sizes, and weighting. That leads to different results. He’s not saying a single poll is meaningless; just that you can be a lot more confident about concluding anything when you can look at results across different polls (though even polling averages can be off).
That means you, media: That Monmouth poll got big headlines. The Morning Consult poll taken at the same time with its much bigger sample and 33-20-15 result for Biden, Sanders, and Warren respectively got barely a whisper. Well, duh. One’s a surprise that gets clicks and shares, the other’s a yawn because it matches other polls. Media is a crucial part of our democracy, but they also love “race is tightening” and “comeback” narratives (who doesn’t?) and polls that feed a buzzy storyline. So just bear that in mind when clicking.
And you too, Democrats: The reactions to the tweet above ranged from dismissive to outraged, mostly from Bernie supporters (Anzalone is Biden’s pollster…and it’s fair to consider the source, just note that he is highly respected in the field and saying something totally reasonable here). People: chill. Most of what we’re seeing in polling this early is just a mix of name recognition and lightly-formed opinions that are guaranteed to evolve over time anyway. Treat a single poll like a single baseball game at the beginning of the 162-game season.
Glance at the fine print: When you see a new poll, especially one that has a surprising result, take ten extra seconds to look at how many people were surveyed. A 400-person survey is pretty common, and gives a +/- 5% margin of error, but that’s a pretty big range: statistically, the Monmouth result is saying that the “true” answer could just as easily be 25-15-15. You get a little extra confidence from polls of 1,000 or more with a +/-3% margin, like CNN, Economist/YouGov or Fox News. You might also notice if the pollster called landline phones only (tends to pick up older people), or internet only (for which the methods have not been proven as extensively), which can affect results. That’s not to say that only results which look like the “consensus” are right. But try to look for more than one data point before concluding anything.
