In 2016, the first year of its existence, the Trafalgar Group was one of the few polling outfits to predict that Donald Trump would beat Hillary Clinton. In 2020, Trafalgar incorrectly predicted another Trump win but accurately depicted the race as neck and neck in places like Wisconsin, where many pollsters again drastically underestimated Trump’s support. Although Trafalgar has missed the mark widely on its share of races over the years — usually overestimating Republican chances — it has become an object of fascination for its seeming ability to capture a segment of the population that has eluded other surveyors. But because of the company’s opaque methods and predictably GOP-leaning results, Trafalgar is also a frequent target of criticism for more mainstream pollsters. I spoke with Trafalgar founder Robert Cahaly about what makes his polls different, why he’s not more transparent about his methods, and how he thinks the midterms will shake out.
If you could name one overarching reason why you’ve been more successful in some of these races than ABC, CNN, Fox News — pick your pollster — what would it be? What are you doing right and they doing wrong, in your view?
Their system is based on a model and numbers, and mine is based on understanding people. The biggest single thing is that a 25-, 40-, 60-question poll — when you’re trying to go deep down and understand issues, that can be a very effective tool. But asking 20 or 30 questions to get a horse race right is a joke. Average people will not participate. You will oversample college-educated people. You will oversample people who have lots of free time after a nine-to-five workday and not people who have various blue-collar jobs with strange schedules.
The No. 1 question we get is, “How long is this gonna take?” And when our answer is, “Oh, listen, it’s just five or six questions — less than three minutes,” people hang on. The other thing is we give people various methods to participate. Yes, you could get a phone call. You also could get a text that says, “You have 48 hours to finish this poll.” And our texts are different from other texts. We text people each question one at a time so they don’t have to click a link to take them somewhere. We make it very easy and trustworthy, and we give them a certain amount of time. And the same thing with email — they can do it at their convenience.
There’s literally a national TV-news network who brags that all their polls are live callers only. I’m going to ask you a question, person to person, because I believe in people more than I believe in numbers: Do you know a normal member of Gen Z or a millennial who would answer a phone call from a number they don’t know and then speak to somebody to answer 30, 40 questions on a random day?
No.
No. So who the hell are they talking to? They can’t possibly make a real poll when they do that. I know how to get to those people and give them ways to participate that make it convenient. The best banks in America recognized years ago some people want to come indoors and make deposits, some want to go to an ATM, some want to do it on a phone, and some want to use an app, and they accommodate everybody and their different comfort levels with technology and trust, and that’s why those banks succeeded. And polling has to do the same thing.
As you also undoubtedly know, a lot of other pollsters have dismissed your work over the years, calling you partisan and not rigorous.
Let’s talk about Virginia for a second, where I was two-tenths of a point from perfect. And how about New Jersey, where I had the only poll in the ball game that was 1.3 points away? They can say what they want, but the numbers are the numbers.
My answer to the question about being partisan is very simple. Anybody who thinks I’d make a poll be wrong so a Republican would win — which would make me look bad for getting it wrong — doesn’t understand my competitive nature. They don’t understand me. I want to be right more than anything. I want to be the Nick Saban of polling. I want to be the Elon Musk of polling. I want to be the best guy there was. It is my goal to be the best way more than any partisan agenda, and people who know me know that.
You’re very confident about your polls, I’ve noticed. You advertise your results as “the real poll numbers.” Especially after all the failures of recent years, lots of pollsters say, “You know, here’s what we think is gonna happen, but here’s a big caveat at the top. We don’t really know.” But you lean into it.
Because I think we’ve built a better mousetrap over the years.
There’s been a lot of criticism about your methodology over the years, mostly around the fact that you don’t show your work as much as other pollsters do.
You say “show your work” — what does that mean?
Cross tabs that drill down into the details …
Oh, we do that. We do often give out the cross tabs to certain members of the media. But there are lots of people who are our paid subscribers, just like Rasmussen has paid subscribers, and what they get for being members are those details.
Why not be a little more transparent with everyone just to shut people up?
People who criticize us — if they really want to see all that, they can do it. They can certainly sign up and become a member; we’ll give them a way to do it. And if you wanted to take a poll at some point in the future and you’re a credible member of the media, I’ll send you the whole thing of cross tabs. I do that all the time with various outlets.
That’s interesting, because there’s a notion that you’re hiding something. The New York Times wrote in 2020 that pollsters don’t take you seriously because you’re so “shadowy.” That’s at odds with what you’re saying.
Well, do I give other pollsters my work? No. Let ’em guess. All they are is guys riding ponies in the Pony Express, criticizing the telegraph lines, saying, “I don’t know how that works.”
These guys can all say what they want, but nobody says that I avoid talking to the media and that I’m in a hole somewhere, you know? I’ll talk to anybody because I think we’ve got a great story to tell, and they just hate it, and that’s okay. I think it’s funny they hate it.
You appear on a lot of right-wing outlets, and you’ve worked for Republican officials in the past. You said partisanship wouldn’t affect your polls, but I’m wondering if you use it at all to get people to talk to you.
I think in 2016, nobody knew who we were. Since 2016, especially in ’20, was it beneficial that they felt like they could talk to us? Yes. I think there’s something to that. But at the same time, we didn’t run into left-wing people who were afraid to talk to us.
Where do you see the race for Congress right now? The narrative has been that Democrats saw some surprising momentum in the aftermath of the Dobbs decision but that now Republicans are more in the driver’s seat, which is what you might expect in a midterm. Do you see a red wave happening or something a bit more muddled?
I think there’s a good chance there’s a red wave. The theory I’ve been putting forth is there are a lot of Republicans who are really, really hesitant to participate in polls. And this is different than in 2016, when they were kind of shy if they were voting for Trump. And it’s different than in ’20, where you just had to work a little harder to get to them. To find the majority of Republicans, you had to just dig deeper; you had to make a lot more calls. They were unwilling to take polls, so many of them.
But now we are finding that there’s a segment of Republicans who we’re not going to be able to reach. A lot of people tell us, with what’s going on … Whether this is justified or not, I don’t know the answer, but their opinion is that — based on what they see on television, what they hear, and what they read — the FBI’s keeping track of what’s happening on social media. They hear things like the government has told the bank to keep track of gun purchases. So it’s not a jump in their mind to think that they might be keeping track of what people say in polls. And so they’re saying, “We’re not taking them.” People who would have, in years past, put a sign in the yard or put a sticker on their car or posted things on social media, they’re just submerged. They’re underwater. They’re not telling anybody anything.
This is what people are telling you? It’s not just a theory — it’s something you’ve heard out in the field?
It’s what people are saying. Here’s another interesting fact: Since Biden’s speech where he declared that he thought the MAGA Republicans were such a threat, we had six different people reach out to us with a screenshot of a poll or a recording of them talking to someone and asking, “Hey, is this you? I didn’t want to take it ’cause I wasn’t sure, and I’m very nervous.” In all the years we’ve been doing this, that’s never happened, and it happens six times since Biden’s speech? It’s completely unreal to us. Let me also clarify that, in every case, it was our poll. It was never someone pretending to be us, which they thought it was. We did look into each one of them because if people were pretending to be us, we wanted to know, but we saw no evidence of that. But I think it’s a mind-set of a lot of quote-unquote “MAGA voters” that somebody’s trying to keep track of.
So is there going to be a wave? I think yes. How big it is, I think the pollsters — including us — will have a hard time measuring it because these voters just literally aren’t reachable.
I think mainstream pollsters would certainly agree with you that it’s very difficult to measure the support of the mostly white working-class base that Republicans now rely on. But as you undoubtedly know, the whole “shy Trump voter” thing is a controversial theory. The consensus seems to be that it doesn’t exist and that people weren’t actually shy — it’s just that the white working-class population has been undersampled in polls. What do you think of that idea?
I think they’re just wrong, and I’ve always said that. Anyone who says that there weren’t shy voters apparently has never spoken to any of their friends who were for Trump and were reluctant to say so. Real people inherently know it existed because they all know people who acted that way. They all know people who were for Trump who didn’t want to put a sticker on their car. So all these guys can say it doesn’t exist, but I’m just not buying it. I’ve always disagreed with them. Social-desirability bias is a real thing. People call it the Bradley effect. You can do plenty of research on it.
But that’s another phenomenon a lot of people believe is overblown.
Well, they think that, but they’re wrong. When all the people who think it wasn’t real are the people who said Hillary was going to be president, take what they say with a grain of salt.
I do take all polling with a grain of salt these days.
Yeah. You should. You absolutely should, and this is coming from a guy who does it and had the lowest error rate of any of them consistently for the last six years.
But I just think this is different. Again, I’m giving you opinions of people that we have polled. In 2020, were there examples of pro-life activists having the FBI come to their house with guns and take people away? No. In 2020, were there examples of people who were willing to turn themselves in but had to go through an FBI-style raid on their homes? No. In 2020, did people think that the FBI was working with Facebook? No. In 2020, did the president of the United States say that MAGA supporters were a threat to America? No.
I think he was saying elected officials were the threat, not supporters.
No, I don’t think that.
That’s not what you took away from it.
It doesn’t matter what I took away from it — it’s a matter of what they took away from it. In my opinion, no polling outfit is making a list of MAGA Republicans; I do not believe that is the case, and I know it’s not the case with us because we have a very unique privacy policy, which is that we guarantee people that no one gets their information. But I do believe there are people who think that could happen.
I saw this interview you did with Split Ticket recently, where you talked about your methodology. You have a certain population of people you want to hit, and then when you hit that population, you end the poll. Is that accurate to say? That’s different from what most pollsters do.
It’s not exactly that. What I say is we try to put together a really good likely voter list. Obviously, the way we do this is very expensive, and so we try not to do a lot of unnecessary work. So especially with the texts and the emails, we say the poll ends at a certain time and date, and we have to do that with texts and emails; we can’t leave it wide open. Let’s say, after the first day, we haven’t gotten near enough responses — we’ll send a reminder. So we’re not always going to get the same number, and we don’t cut it off at a certain number. It’s that the number of reminders we send has to do with how close we are to getting what we need. If we get what we need without sending any reminders, we get it, and if we don’t, then we send reminders, but the cutoff date is the cutoff date. So our goal is to get over 1,000, but sometimes it’ll be 1,150, it’ll be 1,080, sometimes 1,022. How much we bug people to try to get them to participate has to do with how close we are to hitting our goal.
This is par for the course for Trafalgar, but you’ve published a couple of polls this cycle that are way different than anyone else’s, including one where Kathy Hochul, the incumbent governor here in New York, is only up by 2, while the FiveThirtyEight average has her up by 14. And you have one where Democratic senator Patty Murray of Washington is up by 2, while the FiveThirtyEight average is 10. How confident do you feel that these sorts of things are not outliers and that they’re an accurate depiction of those races?
What I would say first is if you remember when I first published that the Oz race was close, they all said we were crazy, and then everybody else started to realize it was close. And when we said that Masters was close, everybody said, “Yeah, that’s a double-digit race.” And now they realize it’s not. And we said that Kari Lake had a chance at winning. Everybody said that was crazy, and now the mainstream-media outlets are saying it’s a toss-up.
I’ve noticed that these mainstream-media outlets seem very hesitant to poll the New York and Washington races because, if they do, they’ll find what we’re finding, and they don’t want to publish that.
Because it goes against their agenda, in your view?
Yes, absolutely. I know there are bosses out there who would lose their mind, and we live in a world where there’s just certain things you can’t do and certain things you can’t say anymore. And it’s not that anyone tells them, “You’d better not find that”; they just fear what would happen to them if they did.
I think it might go in the other direction because the polling has been so poor in the past few years that people want to get it right this time.
But how many of them got fired? How many outlets switched pollsters?
Very few.
That’s my point. So if your job was to get it right, they’d be unemployed. If your job was to push an agenda, they’d still be there. And I hate to say it, but I think a lot of them are people pushing political agendas in the way they write stories, in the way they do everything. And let me be clear, I don’t just mean on the left. I mean on the right too. I absolutely mean on the right.
But places like the Times, who got it wrong in 2016 and 2020, they offered lots of explanations for their mistakes. I’d think they wouldn’t care about being right or wrong. I get the sense that they’re trying their best and just failing at at least one aspect of this — reaching white working-class voters — not that they’re Democratic Party operatives.
And I’m not saying that. You’ve probably never heard me criticizing the New York Times because I’m a paid subscriber, for God’s sake. Every time they call me, I call them back. That’s not a place I’m very critical of. Some of them I don’t feel as confident about. I think there are probably very good people that work everywhere, but they worry about what their career looks like if they say something so far out of a narrative. There’s a lot of people hesitant to say things and worry … I think Kanye West was just talking about all the pressure he got from Hollywood to say and think certain things. People who aren’t careful with what they say and say something stupid pay the price for it.
Well, he did tweet out antisemitic remarks, so I don’t know if that’s the person you want to use as an example.
And, again, if you say things like that, there’s gonna be a price for that, and that’s unacceptable.
But I guarantee you there are people who probably have looked at those races, seen things close to what we’ve seen, and decided, I’m not gonna post that. So my question to you is, why haven’t they? I mean, have you seen how few polls there are on those two races?
If I had to guess, I think it’s because they’re seen as blowouts, and they probably want to put their resources more into places like Georgia and Wisconsin. I’m not saying they will be blowouts, but that’s probably the thinking.
You know, I respect that because I think that way about Florida. I don’t believe there’s real races going on in Florida, so that makes sense to me.
I’m not one of those guys who totally hates the media. One thing I tell everybody that works for us is that if all you do is consume conservative media, it will affect what you do. That is literally not allowed. I tell them, “I want to see that you’re getting a diversity of opinion, whether it’s the shows you watch or the articles you read.” I think the reason we are better at this is because we understand people, and you can’t understand people if you only listen to one side. And I think that’s the No. 1 problem in the whole country: People don’t talk to each other. And it’s ridiculous because there’s so much more consensus than anybody realizes.
This interview has been edited for length and clarity.
More From This Series
- Everyone Biden Has Granted Presidential Pardons and Commutations
- Trump Is Threatening to Invade Panama, Take Back Canal
- What Happened to Texas Congresswoman Kay Granger?
