Were the Polls Wrong

As usual I am reading a lot about how the latest shocking election outcome shows the polls were wrong and that polling has become unreliable (no links I’ve been reading this on Twitter). This is not a new assessment of the polls. It was widely argued (again Twitter now I cite Yglesias by name) that the polls were probably wrong.

The alleged errors are exact opposites. Before the election, it was widely (to universally) asserted that polls implied an over optimistic assessment of Democrats’ chances (in particular to hold the Senate). I became very aware of this compulsively comparing different fivethirtyeight forecasts

Note here Steve Benen “In the runup to Election Day 2022, Republicans were optimistic about retaking the upper chamber, and forecasts suggested it was the most likely outcome. “

The forecasts (sic) to which he refers is the deluxe fivethirtyeight forecast which gave the Republicans a 59% chance to take the Senate. Now, first of all, an outcome with rated probability 49% should not be a shocker (and it is clear that the Senate outcome shocked a lot of people). But also the deluxe forecasts are made combining “expert” forecasts with the classic forecasts which are based on “polls, fundraising, past voting patterns, and more”. The classic forecast gave the Republicans a 51% chance of winning the majority. The polls only forecast gave them a 50% chance. The polls weren’t wrong (at least not as processed by fivethirtyeight which weights pollsters based on past performance and removes something which is a weighted average of an estimated house effect and zero.

Similarly, the polls said 2016 was too close to call (or at least fivethirtyeight gave Trump a 28% chance) and showed Brexit to be an almost exact tie as it was. In each case, the conventional wisdom was not based on polls, was held with great confidence, was wrong, and lead to little (not zero this time) self criticism.

The people dismiss polls and claim to know better before the election, turn out to know less well than the pollster, and blame the polls for their error.

I am getting tired of this.