In 2016 and 2020, Trafalgar Group did what many more other pollsters could not: come close to accurately portraying America’s support for Donald Trump. In 2022, the company’s polls once again showed a picture of Republican strength, helping create a widespread impression that a “red wave” was about to crest. In the end, Trafalgar missed — by a lot. And several more mainstream pollsters, who had gotten things so wrong before, enjoyed a triumphantly accurate cycle.
Weeks before the midterms, I had spoken with Trafalgar’s CEO, Robert Cahaly, who predicted Republican victory, and said he wanted to be the “Elon Musk of polling.” I spoke with him again this week about what went wrong with his polling and where he goes from here.
Your polling results this year were a lot less accurate than they were in 2020. You consistently overestimated Republican support by quite a bit — an average of 7.5 percent in the Senate races. In Michigan, you had Tudor Dixon beating Gretchen Whitmer by one percent, and she ended up losing by 12. You had Washington senator Patty Murray up by one, and she ended up winning by 15. And so forth.
The thing is if you look at the last three weeks, nobody was right. We had two things happen. We had a lack of other polling this fall — I’m sure you’ve noticed that the big networks didn’t do anything in the last couple of weeks, which is surprising.
Yes, it was mostly lesser-known outfits in the last two or three weeks.
And that’s not necessarily our fault. They’re usually there, and they aren’t there. I mean, if you want to ask what I think happened — and we’re going to spend a lot of time studying this — but on first blush, Republicans have no idea how to do “get out the vote.” The Democrats are very good at it.
Our polls in Ohio weren’t very far off. North Carolina, Missouri, and even the race for governor in Nevada — those were all relatively close. And even the races we lost, we had all of those within two or three points. But the bottom line is the Democrats had better overall strategy. The Republicans just did not strategize well. It is hard to anticipate in polling one party doing that great a job at getting the vote out in targeted states and the other party doing that terrible of a job at getting the vote out in targeted states.
How does this all accord with what you’ve said about “submerged” Republicans, who don’t want to speak with pollsters and are being underestimated by polls? Do you still stand by that thesis?
Yeah, I still think those voters were there. The description I gave every media outlet is I thought that vote could be between half a point and five points. And it was just simply outdone by a great get out the vote. It is irrelevant when the other side is doing the work they do. This ought to be a lesson.
Perfect example — look at New Hampshire. You can argue that we didn’t know what we’re doing. Fine. You can argue that Insider Advantage doesn’t know what they’re doing. But can you tell me that the University of New Hampshire, with their record in New Hampshire, doesn’t know what they’re doing? You can’t. Can you tell me that Saint Anselm’s College doesn’t know what they’re doing? You can’t.
Those surveys were off in New Hampshire. But outlets like CBS News, Marist, the New York Times–Siena Poll—
No, this year, they didn’t poll in the last few weeks. This isn’t apples to apples.
Okay, but the Times’ final polls of Pennsylvania and Arizona and Georgia were almost exactly correct. You have critiqued live-caller polls that some other places do, which are very time-consuming and expensive, because you said that people who answer those polls are not representative of the average voter.
No, no, I didn’t say it exactly that way. What I said was people who answer those polls who are Gen Z and millennials aren’t representative of average voters. Those polls are more vulnerable to what’s called the social-desirability bias. And they are. These are two accepted concepts. Now, the way that we do our polls, texting back and forth — it is, if not as expensive, more expensive than even doing live calls. So it’s not a money thing.
But I do think that they’re not very representative of younger people because people just really don’t answer them. And that’s just logic. Do you know any young people who answer the phone and take 30 questions? And that’s all I said. I didn’t say they didn’t represent average voters. I said long questionnaires, however, do not represent average voters. That is what I said. And there’s a difference.
You will also notice our final generic ballot is tied for first and exactly what happened.
Yes, the generic ballot was more accurate. You mentioned Democrats’ get-out-the-vote operation. But Republicans came out too, and independents voted for Democrats. Probably narrowly, but they did.
Absolutely, they did.
And in a lot of these races where the GOP nominated fairly divisive, extreme figures, Democrats won some Republican votes.
There’s no question about that.
Your model didn’t see that coming. So how do you—
I would say it’s less our methodology and more our turnout model. And it is very clear to me that when, in the future, I see this kind of attention to get out the vote on one side, our turnout model must reflect that. Our turnout model just didn’t have it there.
And here’s what kind of bugs me: This turnout of young people from campuses didn’t happen in 2020 because they weren’t on the campuses. But this is the same thing that Democrats did in 2018, and it was very successful. And we should have just anticipated they would do it again.
But it seemed like turnout was good generally, right? This is the new reality of midterms — there’s high turnout, ever since Trump.
But even 2018 was no comparison to 2020.
Sure, but that’s a presidential election.
But 2018 will likely be no comparison to this one.
But what needs to happen is when something isn’t right, you have to figure out what part isn’t right and say how to fix it. And as we’re able to get the list of exactly who voted — those will be available in almost every state within the next few months — I plan on spending these next few months looking at exactly who turned out, seeing how far our model was off, and making adjustments and fixing it. I’m not satisfied with this.
You mentioned last time you want to be right more than anything else.
Yeah. The other day somebody said, “A lot of Republicans say that your polls helped motivate Republicans to turn out.” I was like, “I don’t care.” That’s not my goal. I don’t care whether they turn out or not. What I care about is whether I was right, and I care about how to fix it.
We also spoke last time about how you’ve gotten some flak for transparency. Will you be upfront about how you’re going to change your model based on the results of this election?
Well, again, we’re talking about two different things. There’s the methodology, which I will not change, because if that were the problem, then how would we have been right all these years?
This obviously is not an exact science, all this stuff. It seems like something where you need to keep adjusting all the time.
That’s not the weakness, though. The weakness was our turnout model. You have your methodology, to collect and process your samples, and then you have the turnout, your model of who’s going to vote. The two halves of the sandwich.
So our methodology will not change, and we’ll adjust our turnout model. Like I said, the two sides to the sandwich. You don’t throw out the top side that hasn’t really had a problem, you throw out the bottom side. Because when you look at some of the places that did have the expected turnout, they weren’t that off. When you look at places that did not have the expected turnout, they were very off. So we’ve got to adjust that.
I noticed you haven’t tweeted since Election Day, whereas you typically have a lively presence there. Are you just letting the dust settle?
Yeah, I want to wait for the last election to be settled. We’re working up a statement, what we’re going to put out. You can get really bogged down in who says what. If you’ll notice, we don’t usually even respond to what most people say. We just put out our numbers as we have them.
So you’re full speed into 2024. All this doesn’t give you too much pause going forward?
If all the other mainstream pollers could fail much worse than anything that happened to us this year, I don’t see how, when we have an average record so much better than them, we should stop because we had a bad cycle. I’m a Cowboys fan, and I don’t quit cheering for the Cowboys when they had a bad season. Believe me, they’ve had a few. If you do this year in and year out and it’s always bad? I can see thinking you’d want to do something else. We’re just not there yet. Not even close.
This interview has been edited for length and clarity.
