6
votes
The polls weren't great this year and that was always a possibility
Link information
This data is scraped automatically and may be incorrect.
- Title
- The Polls Weren't Great. But That's Pretty Normal.
- Authors
- Nate Silver
- Published
- Nov 11 2020
- Word count
- 2858 words
The idea that someone will be able to forecast weeks or even months ahead of time how an election will end up when millions of people decide if they're going to vote and what they're going to vote the day before or the day of the election -- That's not a reasonable ask.
The polls are what they are. They have limitations. When people lie to themselves, they're also going lie to pollsters or researchers getting self-reported data. Generally polls don't misrepresent themselves.
Irresponsible use of polls is my gripe. We're talking media use of polls, people viewing swings in individual polls that are within margins of error as shifts in attitude.
We're in a time of change. During the last several terms, things have changed substantially in 4 years. That means polling also faces challenges. Are assumptions and corrections the same as society changes and different people are the ones who go to the polls?
Edward Bernays would beg to differ.
Could you elaborate a little more for someone who doesn't know who this is and why you bring them up in this context?
Edward bernays is the nephew(?) of freud who developed the focus group and really developed us political propaganda to the next level. There is a great bbc documentary entitled “century of the self” that is available on youtube.
The point is that it is not sonfar fetched toquery a group of people amd usefully determine from that what millions if people will decide to do in the future. He would probably say pollster have simply lost their ability to ask the right people, the right questions in the right way. They just need to adapt beyond trying to account for more fundamental shifts through mere refinements in statistical analysis.
Nate Silver wrote and pushed - daily - article about why we should trust the polls this time around. Walking that back now rubs me pretty wrong. I get that its all probabilities and the nature of this work is that all outcomes are possible, but at the same time they make big claims about what we should expect and their business model is for us to trust their forecast, so when they are wrong, turning around and saying "well see we also said that us being wrong was possible" and shifting blame to the pollsters feels like some CYA in action.
They specifically grade pollsters to make their model more accurate per their view. People use fivethirtyeight's grade for pollsters in all discourse I've seen. Why should we even trust those grades anymore.
In the end it wasn't massively off, but damn does it not instill confidence.
Where are those articles? I looked back and the articles Nate published had the exact same kind of hedging.
https://fivethirtyeight.com/features/final-2020-presidential-election-forecast/
https://fivethirtyeight.com/features/final-2020-presidential-election-forecast/
Having trouble finding the exact article. I recall it being linked in the preamble text for weeks on the election forecast page. I wonder if they changed the headline to be less strongly worded. They often linked to it saying "you can trust the polls this year".
I guess I don't see what the point is of all of this is if all he can say is "Well maybe, but also maybe not, but trust me, but also if I am wrong, it's not my fault."
I'm stuck on his pollster rating setup. He claims the authority to rate pollsters, but when several elections come out with pollsters being way off, he's just like "well y'know polls aren't always great" - but yet fivethirtyeight rates them?? Idk, I just don't have the patience for so much walking back, two major presidential elections in a row.
Biden won, which was his prediction, but still, it ruins my view of their work. The massive asterisks on all their articles makes it harder.
What do you mean? This isn't a binary "polls are 100% incorrect and have zero correlation with reality" vs "polls are 100% and completely predictive of the final result".
If you went by polling averages, you would call 48/50 of the Presidential races correctly. Only Florida and North Carolina were wrong, and only Florida was a significantly wrong swing (+3 D to +3 R).
Polls can simultaneously be useful (and they are, after all they're the only form of live update we have in this political system; neither you, nor politicians want to be running blind for 4 years before we have data) and have uncertainty.
All polls have uncertainty. The ratings come about by running the pollsters through their historical performance and grading them based on how well they correlated with reality.
They also walked back in 2012? When the national polling error was +3 D, approximately the same as the Trump polling error now, just for Obama instead? What changed now?
Polls are useful, I don't believe otherwise. I think Fivethirtyeight and Nate Silver are overblown in their perceived authority on the matter, given that they make it their business to rate and rank pollsters then continuously put out articles that boil down to "we might be right or we might be wrong in our assessment of the quality of polls".
I don't think they have zero value. I don't think polls are useless. I see no reason to lift 538 above anything else. Hell, some of the closer polls were their lower ranked ones. Especially in 2016.
I find the positioning poor. They headline their prediction model and interactive apps based on their model, then publish articles saying "but this might be totally wrong". Which like, of course it could be, but they don't drive traffic and make money by being in the business of being "maybe wrong" often. All of that can be true, but that doesn't make it better.
It just all feels gross. Sports stats is one thing, but this is politics, in which policy which impact peoples lives is at stake.
Nothing. Idk what your point is here. I wasn't following them during that election and have no opinion. This feels like a weird "gotcha" and I don't know what you're getting at. Sounds like more of the same
A broken clock is still right twice a day. That's not evidence of anything in particular. They have actual metrics that they use to evaluate effectiveness. Fact of the matter is, they still have the best polling averages, and the best polling ratings.
I don't understand what you're saying, though. Yeah, they highlight their prediction model. Prediction model. And on the page for that prediction model, they tried extremely hard to get people to understand how it works. And how it works is that probabilistic events... are probabilistic. When someone has a 90% to win, that's 90%, not 100%.
I seriously don't understand how it's gross. There is not a single page on 538 that tries to mislead anyone; their frontpage election model says, and said (if you want to backdate with internet archive). What do you want them to do?
If people choose to ignore the words they write and play numbers, that's really their problem.
And as anyone who has played XCOM can tell you, a 90% chance can often feel more like 10% when you only get one go at it, and is actually not that great of odds when the outcome going your way is critical, and failure will be catastrophic.
90% chances can feel like an absolutely sure thing to most people, but anyone who plays the odds frequently enough comes to understand pretty quickly that the 1/10 outcome is actually a lot less rare than most assume it to be. And hell, even getting that 1/10 several times in a row isn't all that rare when you play those same odds repeatedly.
"Historical standards" for reference, according to the article linked above
Presidential
State/Congressional level