Less than a week before Election Day, polls show Kamala Harris and Donald Trump as the frontrunners for the US presidency in intense heat—An unsatisfying result for those looking for a way to reduce the suspense.
“The stakes are very high,” says David Karpf, who studies technology and elections at George Washington University. But, he says, the polls can only tell us the same thing that they have been predicting for a year and a half: “it seems that it will be close”.
Polls are the basic elements before elections coverage and post-election analysis in the US The results of these political polls drive news cycles campaign strategyand can influence the decisions of potential donors and voters. However, they are increasingly precarious.
About supporting science journalism
If you like this article, please consider supporting our award-winning journalism subscribe. By purchasing a subscription, you’re helping to ensure a future of impactful stories about the discoveries and ideas that shape our world.
“Today, we’re using this very vulnerable technique” to make big mistakes, says Michael Bailey, a professor of American government at Georgetown University and author of the latest book. Surveys at a Crossroads: Rethinking Modern Survey Research.
Those mistakes may be familiar to those who followed the last two presidential elections, when polls underestimated Trump’s support. Pollsters hope to learn from their mistakes, but their results are still a judgment call. Here’s why.
People don’t answer polls anymore
For decades, pollsters have faced a “constant crisis” of declining response rates, Karpf says. Surveys are only as good as their samples: the wider and more representative the public that answers the survey calls, the better the data. The ubiquity of the landline in the latter half of the 20th century was a unique gift to pollsters, who could rely on response rates of about 60 percent from randomly dialed telephone numbers to hear a representative portion of the population, Bailey explained.
Technological changes today—including caller ID, the rise of texting, and the proliferation of spam—have led to very few people picking up the phone or responding to unsolicited text messages. Even those who are well respected New York Times/Siena College’s survey gets about a 1 percent response rate, Bailey noted. In many ways, survey respondents are rare, and this self-selection can significantly bias the results in unknowable but profound ways.
“The game is over. When you have a 1 percent response rate, you don’t have a random sample,” says Bailey.
To turn that limited data into useful information, surveys rely on increasingly complex models, Karpf says. These techniques “trigger” the responses of some participants to match their skewed sample with the general voting population on key variables such as age, race, gender, and political affiliation. This allows pollsters to, in theory, extrapolate information about the general electorate from a few biased responses.
In the golden age of random sampling, polls “were based on a scientific method, with a defined procedure that would produce a defined probabilistic result,” says Bailey. On the other hand, “now you just throw model decision after model decision” at the raw survey data and hope your assumptions hold true.
The assumptions in these models can easily go wrong
Pollsters are generally making defensible and good-faith decisions about how to expand and compress their data in the form of voters. But these are still guesses, and reasonable minds may differ.
“Although they are all reasonable assumptions, they are different. We don’t know which hypotheses are correct,” says Karpf.
Pollsters’ accuracy is based on how voters will actually look on November 5. This is fundamentally unknown and something pollsters have gotten wrong for the previous two elections. in 2016 88 percent of country surveys Hillary Clinton’s support overflowed. Studies found that white voters without a college education lost significant pockets of support for Trump, largely because their data was not weighted based on education.
So the 2020 polls weighted education. However, they experienced a similar problem, this time excluding the inclusion of other non-demographic factors. The polls were correct that Joe Biden would win, but 93 percent of them exceeded his lead. “It didn’t feel like such a disaster, but from an accuracy perspective … it’s chilling,” says Bailey. “You see the same problem happen again.”
The 2020 election showed that there are aspects of Trump’s support that could not be fully accounted for by the demographic variables pollsters relied on. So this year, many are using a more serious compensating technique: weighting respondents’ answers based on how recently they voted, a method called vote weighting. This brings the 2024 polls into line with the 2020 turnout, effectively inflating Trump’s support.
The polls are “leaning hard” in the weighting of the electoral votes this time, Bailey said. But this technique has some key limitations. First, it’s not clear that the voters of 2024 will look like 2020, especially given the high turnout of female voters after the Supreme Court overturned the 2022 midterms. Roe v. Wade. “For women voters in particular, it’s an existential and core values issue,” said Anna Greenberg, a pollster at the Democratic Polling House. GQR in an interview Ms. Magazine. Some pollsters are betting that this year’s election will be more like those midterms and their data is being weighted accordingly.
Voters can also change a lot in four years: voters die, many new voters turn 18 and become eligible to vote, and many move to different states. Also, people may not give reliable answers when asked who they voted for four years ago.
But Bailey is most concerned about a more fundamental problem with the technique. Pollsters need to get the correct percentage of Trump voters in their samples, they also need to get the “right” Trump voters. If their sample doesn’t include a representative sample of former Trump voters, weighting the electoral votes won’t fix that. “Imagine you voted for Donald Trump in 2020, but you’re fed up. They may not be responding to surveys now,” says Bailey. The same dynamic could be at play on the Democrats’ side as well. All of this can lead to poll results being skewed, even with voting recall weighting.
Recall vote weighting polls would be less accurate in every election since 2004, The New York Times reports. But many pollsters are choosing to use the technique now to avoid making the same mistake they did in 2016 and 2020.
“Fool me twice, I guess, don’t fool me a third time,” Bailey says.
Even election simulations won’t tell you much
If individual surveys are unreliable, what about survey aggregators? These sites aggregate results from dozens to hundreds of polls, many of which run the style of election simulation popularized by Nate Silver, founder of FiveThirtyEight (now 538).. These aggregators take polling data and run simulations of an election about 10,000 times to predict the likely outcome.
For the average person, these simulations are not very helpful. In 2016 FiveThirtyEight reported that Clinton won the race 71.4 percent of the time. What should observers do now that we live in a 28.6 percent reality where Trump won? These attempts to predict election results, rather than presenting a snapshot of candidate support at a given point in time, were heavily criticized at an American Association for Public Opinion Research meeting. the report that after the 2016 voting failures.
“There was a big presence in the media and political discourse about these poll aggregators….They were using inputs in their models with unknown errors, and they weren’t being really transparent,” said report author Kristen Olson. at the University of Nebraska-Lincoln news release. Forecasting models, the report notes, “attempt to predict a future event. As the 2016 election proved, this can be a tough exercise.”
Karpf cautions against election anxiety by refreshing 538 or other poll aggregators. “That kind of modeling exercise tells us very little,” he says.
Also, Bailey points out, they can aggregate unreliable sources. “I was showing my class today, if you look at 538 … and some of them (surveyors) start clicking, ‘Who are these guys?’ And it amazes me that in some cases they have no idea who they are,” he says. Some are biased, and others don’t have a method on their websites, which is “very far from that scientific ideal, on balance.”
The election is very close now
Another problem is that the elections are very close today. “It’s enough that if (the result) is 3 percent in Trump’s direction, it’s going to look like a blowout for Trump,” and vice versa for Harris, Karpf says. That’s statistically within the poll’s margin of error, but the public may perceive it as a big miss.
And as FiveThirtyEight notes on its prediction page, “a close race in the polls doesn’t necessarily mean the outcome will be close.” Although each candidate’s odds appear to be about the same right now, the winner could take the presidency by a significant margin in the electoral college.
“The difficulty for the public right now is that we’re looking at these polls, wondering what the future will be,” and they can’t answer that, says Karpf.
It is a high-stakes time for surveys to be at a methodological crossroads. Overhanging all these debates is the threat of political violence. If the polls underestimate Trump’s support again and he loses, even if the final results are within the margin of error, they are likely to be used to claim election fraud, Karpf says. While this problem is not for pollsters to solve, it is part of the reality they face and can decide whether or not to use recall vote weighting.
In this election, more than ever, looking at the polls has given little comfort to anyone. For the next week, Karpf recommends drinking water, sleeping well and not checking social media. What will happen on election day will not be known until it happens.
Even pollsters agree. As Greenberg advised Ms. Magazine“Try not to look at the polls… It’s been stable, more than anything. Not much has really changed since September. Everything is close.”
