Donald Trump sent a shock wave through the polling industry — not by beating his polls (as he often has) but by suing a legendary pollster.
Trump is going after Ann Selzer, a highly respected pollster who predicted Barack Obama’s meteoric rise in the 2008 Iowa Democratic caucuses and saw Trump’s strength in 2020 when others underestimated him. Selzer whiffed in 2024 — predicting that Kamala Harris would win Iowa by three points, days before Trump won by 13 points. Trump’s suit alleges that Selzer faked the numbers to harm his campaign.
This argument is both silly and scary. From my perspective (as a pollster, not a lawyer) a flawed methodology — not criminal intent — likely led Selzer to miss Trump voters in Iowa. But the fact that Trump brought the lawsuit at all is troubling. If politicians sue pollsters for mistakes, pollsters will stop publishing surprising results and the whole political system will suffer.
If Trump really wanted to understand why Selzer’s poll missed, he could start with her post-mortem.
After the election, Selzer wrote a detailed report examining her statistical and methodological choices and testing if other decisions could have led her to a more accurate result. Her most striking finding: In her poll, self-reported Biden 2020 voters outnumbered Trump 2020 voters, 41% to 40%. In reality, Trump won Iowa by eight points that year.
When most political pollsters see data like this, we weight by party or past vote. That is, we believe that Republicans or Trump’s 2020 voters are less likely to respond to polls than Democrats, so we expect the initial data to underestimate Trump. So we give the Republicans who do respond a bit more of a voice in the poll, and the result better reflects the views of the whole electorate.
But Selzer didn’t adjust her Biden+1 sample. She has long believed that interviewing voters randomly and applying as few statistical adjustments as possible leads to the most accurate results. That strategy let her beat the competition in the past — but, this year, it misled her.
Selzer calculated that if she had weighted her poll to get a more reasonable balance of Trump’s and Biden’s 2020 supporters, Trump would have led Harris in her poll by six points — erasing more than half of her 16 point error. That’s not a perfect result. But a Trump+6 poll in a Trump+13 state would qualify as a temporary embarrassment — not a career-ending error.
This explanation — that Selzer whiffed because she made a bad statistical choice — is much more plausible than Trump’s argument that Selzer fabricated the poll to harm his campaign. But Trump isn’t just suing Selzer. He’s trying to insert himself into the polling industry’s feedback loop.
The polling industry works because we self-police. Every two years, we test which pollsters can accurately predict an election — and learn which methodologies actually reveal what the public thinks. Pollsters who perform well, such as Selzer in 2020, earn public praise (and, in some cases, new paying clients). Those that miss, such as Selzer in 2024, suffer reputational (and, if they lose clients, financial) costs.
This feedback loop isn’t perfect. But it encourages pollsters to strive for accuracy and it offers rewards to those who are willing to break from the consensus and make a risky prediction.
If politicians like Trump enter that feedback loop, they could seriously damage the industry. Some pollsters will stop publishing any surprising results out of fear of political retribution. The price of polls could go up, as some pollsters fearfully sock away money for a potential legal fight. Trump doesn’t need to sue every pollster to change the shape of the industry. If politicians can credibly threaten to drag any pollster through a damaging, costly lawsuit at any time, damage will be done.
And if pollsters become more cautious than we already are, the public will lose out. Polls can alert the media to a new, exciting candidate that’s catching fire. They can tell politicians which policies Americans actually want. They chart big social changes that, without survey data, can only be captured by “vibes.” But polls can only do these things if pollsters have the room to take risks and get it wrong sometimes — without getting sued for it.
This article was originally published on MSNBC.com
