alt Hacker News> I did too much bragging in the media and didn’t anticipate the extent to which public opinion toward FiveThirtyEight would shift once we became a corporate-backed incumbent rather than an eccentric upstart
Can’t speak for everyone else, but it wasn’t this for me. It was about 2016 presidential that lost me.
He tries to justify this later about how theirs was better than other outlets but I don’t care. Call it emotional, naive, unfair or whatever you want, but regardless I had zero interest in reading any of their predictions or analyses after that.
Not even mad, just that to my experience they had one job and they didn’t fulfill it at the most important time. They went from appearing insightful to just one opinion amongst so many others.
Replies
If I say you have a 50% chance to win a coin flip and you lose it, that doesn't mean I'm wrong.
A key thing though is 538 did regularly test the calibration their models: https://web.archive.org/web/20190410030104/https://fivethirt...
> "What you’ll find, though, is that our calibration has generally been very, very good. For instance, out of the 5,589 events (between sports and politics combined) that we said had a 70 chance of happening (rounded to the nearest 5 percent), they in fact occurred 71 percent of the time. Or of the 55,853 events that we said had about a 5 percent chance of occurring, they happened 4 percent of the time."
Huh. I first started listening to 538 in the run-up to the 2016 election and started really paying attention to them precisely because their 30% figure was so much higher than all the other polls. It was shocking to me then (and still is now reading your comment) that people didn’t seem to understand that 30% in the context of that particular election and that particular candidate suggested a remarkably high chance of winning, not a really low chance of winning. It’s a strange thing where people seem to think that less than 50% = not happening.
I really think a majority of NYTimes and ABCnews consumers don't know the difference between a 2/3 chance (super close) of winning and 2/3 of the vote (a landslide).
That’s like saying “there was only a 30 percent chance of rain today and it rained, so I will never look at the weather forecast again.”
Because they said trump only had a 30% chance to win?
What if they had said 49%? Would that have made their prediction worthless?
We should have a drinking game in Nate Silver thread anyone complains about 2016 prediction. Then everyone piles on to point out how probabilities work.
538 used the example of Trump having approximately the same chance of winning the 2016 presidential election as the Cavaliers had of winning the NBA championship round vs the Warriors. Both Trump and the Cavaliers won with a ~25% predicted chance.
538 made very clear with this analogy that both Trump and the Cavs were underdogs, and that both had a solid chance of winning.
A dice roll has a 16.6% chance of landing on any given side, meaning an 83% chance of not landing on that side.
If you guessed a "two", and it landed on "two, I wouldn't really be that impressed, even though there was an 83% probability going against you.
I think this gets to the core of why a lot of this election prediction stuff doesn't work. People just don't parse the numbers the way the authors intend.
FiveThirtyEight had Trump at a 30% chance of winning, and he won. The model wasn't wrong. The less likely of two outcomes occurred. Even if they'd had him at 1% they still wouldn't technically have been wrong though I think complaints might be more warranted.
If they had Trump at 49% would you have still been angry? What about at 51%? Would it have been okay then?
FiveThirtyEight gave Trump a 30% chance. Their reporting did make clear that with margin was within the range of a normal polling error. And sometimes you get more than a normal polling error.
It doesn't help that the US has a terrible election system that often leads to small margins in some states being decisive.
> Call it emotional, naive, unfair or whatever you want
Yep definitely all those.
Why is it so hard to admit 30% is not 0%?
As a younger man I would have been with the commenters mansplaining probability, but I've aged into realizing that thinking of the election like a marble pulled from an urn whose contents we have probed with polling is just as bad as thinking of it as deterministic. The reason people read fivethirtyeight, probability-savvy or not, was almost entirely to be told what was going to happen, which is sort of incompatible with feeling you can do anything about it. In that way it's probably worse than old-fashioned pundit-driven horse race coverage because it has an air of scientific authority.
> Call it emotional, naive, unfair or whatever you want
We would need a pass from the mods lol.
If you use 538 data, on average you make money betting. Its more correct than not
Nate was a huge outlier in that prediction, he gave trump a better chance than almost anyone else that I can recall, so why are you mad at him about that?
What made me mad is Nate seemed to turn into a MAGA troll himself after that election.
I had complaints about 538, especially the early days, but don't understand this critique at all. A 30% chance hitting is completely unremarkable, and it was a perfectly reasonable reading of the evidence at the time. Nate isn't wrong that conventional wisdom was way off, with even supposedly statistical models giving Hillary a 99% chance of winning.
Elections, like many things, have some inherent uncertainty. A several point polling error is normal, so a candidate who is down a couple points on election day has a decent shot of winning.