Last month, Nate Silver of FiveThirtyEight did an analysis of polling errors in the 2020 election cycle, and I discussed it along with related data at some length. Now comes an interesting report from a group of five major Democratic pollsters that confirms much of what Nate had to say, while offering one new theory I find fascinating: that like voting by mail, poll response in 2020 may have been significantly skewed by Donald Trump’s disinformation.
To be clear, the pollsters issuing this report are campaign pollsters, not the media-sponsored outfits whose data FiveThirtyEight (and political writers) typically consumes. These are people whose findings guided, and apparently misguided, actual candidates; it says a lot for them that they are engaged in this very public self-critique, but it also shows that 2020 polling errors really were systemic, and really did (on average) show Democrats from Joe Biden on down doing better than the actual results indicated. And that was despite efforts to address similar errors in 2016, per the report:
Following thorough investigation into polling error in 2016, we all adjusted our weighting protocols to ensure we had enough white non-college voters, and polling seemed to improve in midterm and odd-year elections. But now, as we dig into 2020, we find an error that is not so easy to correct. We saw that in more Democratic states and districts, and some closely divided states like Georgia and Arizona, the data were quite good. But in more Republican areas, the data were often wrong, sometimes egregiously so.
The Democratic pollsters are cautious about reaching any conclusions, but suggest their errors were probably attributable more to sampling problems than to any late trends or turnout surprises. But in discussing the apparent relative reluctance of some Trump voters to respond to surveys, they don’t focus on the largely disproven “shy Trump voter” hypothesis (i.e., that Trump voters don’t want to admit their preference for the man to strangers on the phone) but rather on Trump’s own efforts to make pollsters (at least those who aren’t biased in his direction) disreputable.
High-quality social science surveys suggest Americans’ trust in each other has been falling for decades. As some analysts have suggested, Trump may have helped turn this into a problem for pollsters by attracting distrustful voters and making his most ardent supporters even more distrustful of other people, of the media, and perhaps even polling itself. That, in turn, could have made his supporters less likely to answer polls.
Politico quotes one pollster who put it bluntly:
“Trump went after the polls,” said [a] Democratic pollster involved in the partnership. “He was really pretty overt to those that were listening about some of his distrust of polls or media.”
The good news for pollsters is that Trump may have run his last campaign. The bad news is that even if he doesn’t run for president again, Trumpism is ascendant in the party he conquered in 2016. So the aggressive polarization characteristic of Trumpism, and with it efforts to convince voters only to trust the most partisan of institutions, could undermine participation by Republican voters in all sorts of activities that their opinion-leaders choose to demonize, whether it’s voting by mail or involvement in “fake” polls. The implications of that hypothesis, of course, go far beyond the reliability of public-opinion research: The power to manipulate distrustful voters that Trump demonstrated really does undermine democracy in fundamental ways, as the events of January 6 demonstrated.
As for the pollsters, they are admitting the status quo may not be sustainable:
How do we get people to participate in polls, if they won’t answer our phones, or respond to surveys online? We don’t have that answer yet. What we can tell you is that together, we are going to embark on a number of experiments over the course of this year. Along with progressive institutions who want to get to the bottom of this problem, we are going to put every solution, no matter how difficult, on the table.
That could include, they say, basic methodological changes like a return to door-to-door canvassing, or financial incentives for survey participation. Pollsters may privately hope if Trump goes away, things will return to normal without such drastic measures, just as Republican voters may return to past levels of participation in voting by mail. But again, the more we learn about the 45th president’s ability to weaponize distrust, the more likely it is that imitators will follow, and the harder it will become to dispassionately analyze political events, including elections.
