One of the biggest open questions is why recent Democratic special election performances continue to paint a completely different electoral picture than what current polling suggests. Polls continue to show an effectively tied race between Joe Biden and Donald Trump, and Biden’s approval remains stuck in the low 40s. On the other hand, special elections don’t just show a Democratic environment — taken at face value, they would be picking up on a looming blue wave of a magnitude not seen since 2008.
The first explanation for this contrast is to attribute it to the changing coalitions in the Trump era — as the Democratic Party has swapped white working-class voters for formerly Republican college-educated suburbanites, it has exchanged one of the groups with the least reliable turnout for the most consistently-voting demographic. As a result, Democrats now arguably enjoy a clear advantage in irregular and off-cycle elections, low-turnout races where only the highest-propensity voters tend to show up. But these gains are less likely to be reflective of the actual environment, as lower-propensity Republican voters are more likely to turn out for a heavily-contested general election than they are for an obscure state legislative special election.
This is a tempting explanation, and there is likely some truth to it. It is true that this Democratic coalition is a significantly more robust one in terms of turnout, at least among whites, than any that has existed in recent memory. Many of their voters are hyper-engaged and are more likely to vote. This also helps explain why irregular voters actually voted Republican in 2020, and why Biden does better among registered voters than he does among adults in 2024 polling.
The problem with using this theory to explain everything is that the composition of the partisan coalitions of 2023 are broadly identical to the coalitions in 2021 and the first half of 2022, before the Dobbs v. Jackson ruling was released. Back then, the median special election suggested a swing to the right from 2020, and this corresponded with some extremely underwhelming Democratic performances in Virginia and New Jersey. Looking at a plot of a rolling 3-month average of special election swing versus time, weighted by votes cast provides crucial context here.
Here, you can see two clear inflection points. The first is when Biden withdrew from Afghanistan, after which Democratic fortunes in special elections began plummeting along with his approval rating. This portended a very bad November for the party in both Virginia and New Jersey. Although it’s highly unlikely that the environment itself shifted 15 points in a month, I think the correlation and the directionality of the shift itself is what’s useful — as Biden’s approvals declined towards the end of 2021, Republicans began to do significantly better in special elections, setting the stage for their overperformances in November.
Democratic special election performances were a bit less apocalyptic in 2022, but the elections still generally suggested a rightward shift from 2020 overall for the first half of the year. This brings us to the second major inflection point on the graph: the release of the Supreme Court ruling on Dobbs v. Jackson, which cleared the way for extreme abortion bans in multiple states. A discontinuity is introduced in the chart to show the obvious environmental shift that happened — immediately after this event, Democrats began gaining across the board, and the average special election in summer 2022 saw a 5 point swing to the left from 2020: a total reversal of the special election environment from 2021 into the first half of 2022.
Both the 2021 post-Afghanistan dip and the 2022 post-Dobbs spike suggest that the special elections picked up on the directional shifts in the environment. In fact, the 2022 overperformances correlated to a fairly good midterm for Democrats, considering the circumstances. They put up the best performance for the president’s party in two decades, as they won independents and saw their battleground candidates broadly outperform 2020 margins in many key races.
One counter readers may raise is that the 2022 special election swings, taken at face value, didn’t merely imply a less Republican night than 2021 — they also seemed to imply a blue wave, given that they were seeing swings significantly to the left of the margins observed in the D+4 environment of 2020. But with a national environment of R+1.6, the nation actually swung 6 points to the right on the whole. So, what accounts for the discrepancy?
One easy explanation is abysmal Democratic underperformances, localized to safe states like Florida, California, Alabama, Mississippi, and New York. This had the effect of tanking the party’s national margin. However, in battlegrounds like Michigan and Pennsylvania, Democrats actually gained on Biden’s 2020 margins. This helps explain some of the national rightward shift, but it does not explain why Democrats didn’t see the massive gains that special election margins seemingly indicated on the surface.
For this, it may be useful to note that the specials were held in summer 2022, whereas the midterms were held in November. According to FiveThirtyEight, while Democrats gained 4 points in polling between June and September 2022, they then lost 3 points of ground between September and November. In other words, as Democrats stacked up overperformances in special elections, the party simultaneously clawed back significant ground in the generic ballot. But polling suggested that a portion of those gains were ceded back to Republicans towards the end of the cycle, as undecided voters began to coalesce against the president’s party, and that came after the last special elections for the cycle were held.
Given the correlation observed between the specials and the polling during the period that we had both measures available to us, I’d be willing to bet that if we had special elections in October 2022, we would have seen the GOP regain some ground from their summer nadir. I’d also be willing to bet that if the midterms were held in September 2022, Democrats would have held the House and won the popular vote (and this is a view publicly given merit by people like Nate Cohn of the New York Times).
Special elections are a noisy dataset, and a 5 point shift in a special election clearly doesn’t correspond to a 5 point swing in the national environment. But this is not the argument being made, and focusing on this strawman risks obscuring valuable data. The question is whether special election margins have directional value still in indicating what the environment is. Here, it seems that the answer is yes. The latter half of 2021 showed a real turn against Biden and Democrats, and the declining Democratic performances in specials were followed by a Democratic collapse in both Virginia and New Jersey. Similarly, the shift after Dobbs showed Republican gains stalling and reversing, as voters began reacting to a perceived overreach from a conservative Supreme Court.
This is why it is likely safe to conclude that regardless of your views on the built-in advantage Democrats enjoy in low-turnout special elections, there is some clear value that they provide in gauging the environment and how it changes over time. It is difficult to quantify exactly what environment a D+9 special election swing correlates to, but what we can probably infer is that it is significantly more Democratic than the one that gave consistent R+3 swings a year ago.
This brings us to the open question of what the special election performances today mean. In 2023, Democrats are outperforming Biden’s 2020 margins by a whopping 8 points. These gains don’t seem to be turnout-driven, either — even in cases where Republicans are turning out, in some cases more than Democrats, the Democrats are still outperforming. However, polling paints a very different picture and suggests a close national environment.
There is no easy answer to the question of which source to trust, but I think the evidence is stronger for the specials, at least directionally. Early polling has plenty of undecideds, doesn’t have nearly as much predictive value, and tends to be extremely noisy. Polls also depend heavily on the sample they are built from, and the task of predicting a future environment clouds the certainty which hard numbers inevitably convey. Special elections, however, have baselines that we can compare to in past overperformance patterns (i.e. whether the party is overperforming by more or less in the average special than they did a year or two ago), and they do rely on real votes cast by voters.
As previously mentioned, specials can be noisy, and so it’s important to remember that their main value isn’t necessarily in the exact magnitude of overperformances. It’s in the difference between overperformances across time, which theoretically controls for coalitional effects that could give one party an advantage based on its base’s voting regularity. The true lesson from the D+8 overperformances of 2023 is not that we are suddenly in a D+12 environment (remember, the overperformances are compared to Biden’s 4-point win). Instead, it is that we are in an environment that is almost certainly more Democratic than 2022’s (at least in terms of persuasion), and quite possibly even more Democratic than 2020’s.
Whether that holds is anyone’s guess. If it does, however, it likely bodes well for Democratic chances in the 2023 elections, especially in states like Kentucky and Virginia.
Acknowledgements: The raw data for this piece was provided by Ethan Chen (@ECaliberSeven).
I’m a software engineer and a computer scientist (UC Berkeley class of 2019 BA, class of 2020 MS) who has an interest in machine learning, politics, and electoral data. I’m a partner at Split Ticket, handle our Senate races, and make many kinds of electoral models.