February 8, 2016
Ignore the polls. This may seem like an odd request from someone who constantly writes about polls and believes almost any survey has at least some nugget of value. Not on Monday, at least on the GOP side. Feel free to look at any Democratic poll of your choosing to see if Hillary is making any progress closing the gap.
Here’s why you should discount any poll released Monday as well as any news story focusing on a poll from the GOP side:Anything that includes polling before the debate is useless
Completely pre-debate polls are fine. At least they have the benefit of being uninfluenced by the debate. You know they’re a starting point, and nothing more. Any Monday tracking poll has pre-debate data and post-debate data combined.
A non-tracking poll (one that isn’t a daily rollover, and gets taken whenever the pollster feels like it) is either going to include pre-debate data or was taken on Sunday only. That’s no good either for the below reason.
Tracking polls released on Monday are dependent on data from Super Bowl Sunday
The game didn’t start until 6:30pm in New Hampshire, and most Granite Staters are probably still mad the Patriots lost, so you can argue they weren’t really into the game. Super Bowl is still a national holiday of sorts. People start planning earlier in the day, heading over to their watch site, etc.
Not a good day to poll. It’s not quite as bad as Thanksgiving or Christmas, but not the time to have everyone’s best attention. There’s too good a chance of getting voters who are disproportionately following the election. A normal, only somewhat engaged voter is likely ignoring the phone.
When that’s your only post-debate sample, you have a problem. It’s not like the pollsters have any choice. This is a fluke of the calendar, but it fouls the data.
Outside parties leak surveys that match a helpful narrative
A John Kasich-supporting super PAC leaked the results of a snap poll they took on Sunday. It shows Donald Trump out ahead by 20 points. He’s followed by Kasich, then Jeb Bush and a sinking Marco Rubio. Ted Cruz and Chris Christie tie for fifth.
Very dangerous. The numbers completely make sense and are in line with what I would have expected after the debate (Cruz is slightly lower, but close enough). It’s confirmation bias.
The result matches my impression, so I’d want to run with it. The poll was Sunday only, they surveyed 500 people and possibly caught them early enough to avoid Super Bowl bias. Here’s the catch.
If the results weren’t to Kasich’s benefit, they wouldn’t have leaked the results. We don’t know how they determined the split between Republicans and Independents. Kasich runs far better with Independent voters. He may have more support among voters paying closer attention to the campaign, the same people who are more likely to answer a poll on Super Sunday.
Mostly, it’s just that only good news gets leaked.
It’s too easy to see what you want in the data
We all have our theories about what happened on Saturday. Whether you saw the debate or just the clips of Rubio/Christie, you’re expecting a certain outcome. Maybe it’s a collapse for Marco, falling from a decently strong second to fifth or sixth. Perhaps you think Kasich is surging. Possibly a Bush renaissance.
If a tracking poll has Rubio down a couple points, but it covers three days, you can very easily multiply by three, see he’s down 6 and reach the conclusion Tuesday will go badly for him. But it’s small sample size, and we don’t know if he will rally to make Saturday less important.
Even Tuesday morning polls will require a bunch of estimating and adjusting, but there’s nothing we can do about that. Four candidates (Rubio, Cruz, Kasich, and Bush) were bunched prior to the debate. Christie could join them based on his debate. At least by Tuesday it’s a balance of signal and noise. Monday is mostly noise.