In all ways Harris vs. Trump poll numbers could be wrong

Topline

Poll experts worry that polls in the final days of the election could be headed for a similar result – another possible theory why polls in 2024 could be accurate or flawed.

Important facts

Important background

In political polling, dozens of companies survey thousands of people in many different ways, from email surveys to phone calls to online surveys. Pollsters often use a combination of outreach methods. They then use various methods to make survey data more accurate, such as weighting it or interview tactics to ensure more trustworthy results.

Are the polls “herd driving” or are they consistent?

“Herding” explains how pollsters want to protect their reputations by focusing on a generally accepted election result rather than an outlier that could be terribly wrong. The effect occurs more often in the last days of the election. On October 29, Silver raised concerns about herd behavior on leads by one point. There “should be more variance,” Silver writes. In contrast, some surveys have outliers. Silver often points out that the Times/Siena College poll represents a rare exception to the “herding” practice of pollsters. His predictions aren't perfect (in 2020, President Joe Biden was predicted to win by nine points, even though he won by just 4.5 points), but outliers are published.

Are surveys timely?

Some polls are conducted in the final days of the election, but almost no polls represent Election Day. The Times/Siena poll, for example, releases its latest national poll near the end of October and is based on data dated (at the latest) October 23rd. And by the time November 5 rolls around, states have been collecting early votes for days or weeks. For example, California begins early voting 29 days before the election. This was deemed a “temporal (or time-dependent) error” by FiveThirtyEight. If something significant happens in the days before the election, the polls cannot take it into account. However, a surprise rarely affects the election in the eleventh hour.

What error rate do surveys have?

Data is not perfect. FiveThirtyEight has stated on its website that even if an “infinite” number of citizens were polled immediately on Election Day, “the average poll would still miss the final margin in the race by about 2 points.” Many polls tracked by Forbes have a margin of error between two and five points. This is particularly relevant in an undecided election like 2024, where a few percentage points could decide the election.

What kind of biases do pollsters fear may influence survey responses?

Recency bias is the tendency to believe that the person who won last will win again. Voters may predict Trump will be the next president because he won in 2016. Non-response bias takes into account certain groups that show no interest in participating in surveys. Historically, Trump supporters have tended to be less responsive to polls and poll results, suggesting that polls ultimately underestimate them. But this year, some Democrats are wondering (and hoping) polls are also undercounting their traditional key demographics, including black voters and young voters.

Pollsters “weight” their polls more heavily – and because of this, some are concerned that the polls may be biased.

Weighting highlights survey data, such as a respondent's gender or age, that pollsters believe is underrepresented. In the 2018 midterm elections, pollsters focused on education to reduce state polling inaccuracies in the 2016 presidential election poll results, which, according to the New York Times, did not accurately reach voters with less education. After 2020, pollsters increasingly began weighting by “recalled vote,” or who the respondent had previously voted for, to ensure they were capturing enough Republicans. But some experts worry that the emphasis on “recalled voting” has flaws. According to the New York Times, respondents may not accurately remember their vote and incorrectly state that they voted for the winning candidate. In some elections, the “recalled vote” could distort your predictions because it reflects opinions about a previous election. Still, two out of three polls in September used this technique, reports the New York Times.

What is the “shy voter theory”?

The “Shy Voter Theory” suggests that Trump supporters are not always honest about their voting plans, perhaps because they do not want to admit to a pollster that they support Trump. That means polls may be underestimating Trump's popularity in the 2024 election. Some say this theory explains the inaccurate poll numbers in 2016 and 2020. But it may not have any bearing on today's polls, Silver writes in the New York Times. Many Trump voters are ardent public fans and have no reservations about expressing their support.

What is the “Bradley Effect”?

The “Bradley Effect” theory would be a boon for Trump. The theory – named after Los Angeles Mayor Tom Bradley, a black Democrat who lost the 1982 California gubernatorial race to Republican Attorney General George Deukmejian – says that white voters who describe themselves as “undecided” sometimes simply don't concede want them not to vote for a black candidate. The Obama campaign spoke out against this belief and somewhat refuted it by winning the presidency by relatively large margins in 2008 and 2012. However, some believe this theory could apply to the 2024 election and Harris as the first Black female presidential candidate. A similar, gendered effect was blamed in 2016, when undecided voters ultimately tended not to vote for Clinton, Silver writes.

What is the “unified theory”?

The Unified Theory explains why presidential election polls are less accurate than midterm elections, based on Trump voters' propensity to ignore polls. According to this theory, polls in 2018 and 2022 were more accurate because the midterm elections included more engaged voters — those most likely to respond to the polls. In contrast, there is a larger pool of potentially unresponsive, less engaged voters who tend to vote only in presidential elections. That means the 2016-2020 pattern of polls undercounting Trump voters — who are typically less engaged — could continue in 2024.

What is the “Patchwork Theory”?

The “patchwork theory” suggests that the mistakes made in the 2016 and 2020 polls are completely independent of each other and have nothing to do with the midterm elections. Both elections faced a number of unique factors that caused the polls to be wrong. Some argue that polls overrepresenting college graduates and undecided voters tipped the 2016 polls in Clinton's favor. In 2020, the pandemic's far-reaching impact on people's lifestyles and choices drastically influenced polls, others point out.

Could polls be overreacting to the “Trump effect”?

This story always points out that poll inaccuracies in 2016 and 2020 are largely a concern in this election cycle. That's largely because of Trump's eager, usually less engaged voter base, which has historically been underrepresented in polls. As journalist Nate Cohn points out in his recent New York Times article, the concern among these pollsters may be that they are underestimating Harris or overestimating support for Trump. The dead heat of the election portrayed in the polls could be over-tweaked to accommodate Trump supporters of past elections.