Is 538 the worst prediction site in the world?

[hoosierdaddy34]

You sure? At what point in 2016?

I remember them predicting Hilary to win


But piggybacking on the op, I was just thinking about this.

538 presents their predictions in a very neat and professional way, plus they don’t just present the data, they go to the extra step of interpreting the data and giving the reader a conclusion, they’re just always wrong lol

Nate Silver nailed 99 out of 100 elections Presidential states in 2008, and 2012. He missed like 1 senate race in that time period as well. He still had Trump at 33% in 2016. And had a very strong 2018 mid-term election.

So you clearly do not know and are just making crap up.

Fair enough, I get your point. I’m not too much into politics so don’t really pay much attention to that stuff.

Back to the sports side. I know they’re just models and not guarantees, but I can’t think of a single instance of where I saw a model/ prediction on 538 that was out of the ordinary and turned out to be true.

I feel like every time I’ve looked at there models and there was some sort of outlier (Marcus Smart has superstar potential, rockets a favorite to win the Championship), it has not turned out to be a correct prediction.


That’s all I’m saying. But no, I haven’t gone in depth to their proprietary algorithms, the point is that they’re supposed to be smarter than me and do the math for me (the stupid consumer), to show me what I’m missing.

I’ll agree their basketball models have been terrible. They need a lot of work on those.

[WestGOAT]

You sure? At what point in 2016?

I remember them predicting Hilary to win


But piggybacking on the op, I was just thinking about this.

538 presents their predictions in a very neat and professional way, plus they don’t just present the data, they go to the extra step of interpreting the data and giving the reader a conclusion, they’re just always wrong lol

Nate Silver nailed 99 out of 100 elections Presidential states in 2008, and 2012. He missed like 1 senate race in that time period as well. He still had Trump at 33% in 2016. And had a very strong 2018 mid-term election.

So you clearly do not know and are just making crap up.

Fair enough, I get your point. I’m not too much into politics so don’t really pay much attention to that stuff.

Back to the sports side. I know they’re just models and not guarantees, but I can’t think of a single instance of where I saw a model/ prediction on 538 that was out of the ordinary and turned out to be true.

I feel like every time I’ve looked at there models and there was some sort of outlier (Marcus Smart has superstar potential, rockets a favorite to win the Championship), it has not turned out to be a correct prediction.


That’s all I’m saying. But no, I haven’t gone in depth to their proprietary algorithms, the point is that they’re supposed to be smarter than me and do the math for me (the stupid consumer), to show me what I’m missing.

I think they were also one of the few sports media to predict the rise of Warriors after hiring Steve Kerr (and acquiring Iggy), even predicting that they'd be one of the favourites to win the title:
https://www.google.com/amp/s/fivethirtyeight.com/features/2014-nba-preview-the-rise-of-the-warriors/amp/

So their models are not that bad.

[TinmanZBoy]

Not true. They were actually one of the few organizations that gave Trump the best odds to beat Hilary, they basically gave him 2-1 odds (33.33% chance).

The likes of NYT, CNN etc were the ones hopelessly wrong.

You sure? At what point in 2016?

I remember them predicting Hilary to win


But piggybacking on the op, I was just thinking about this.

538 presents their predictions in a very neat and professional way, plus they don’t just present the data, they go to the extra step of interpreting the data and giving the reader a conclusion, they’re just always wrong lol

https://projects.fivethirtyeight.com/2016-election-forecast/

This was their election day forecast.
Trump to win @ 28.6% (which was similar to the odds the Trump team have themselves).

They'll probably admit they and everyone else is new to predicting sports models and it's a very new science.

maths is not science, is more about logic...the world is composed of things unexpected, in every facet...

[Johnny Bball]

Their models are especially bad preseason when they presume a guy will have the same numbers on a new team (or the year prior) without taking into account his role, changing roles, who he plays with, minutes that will be played vs were played, and who he did it for all last year and a myriad of other factors. It create a complete distortion.

[Dr Aki]

They're upfront about what their models measure and what they say.

However, if you ask anyone who's works in data modelling, they'll tell you it's almost always a case of "garbage in, garbage out".

In other words, the models ASSUMES past performance is indicative of true performance, and ergo future performance.

The models do not specifically account for:
1. Playoff basketball being a fundamentally different game to RS basketball
2. Playoff defensive and offensive scheming for specific matchups and teams
3. Players/teams performing above or below their RS baselines
4. Plain old simple variance

Once you understand this concept, their models makes perfect sense as to why they project results the way they do, and why they are fundamentally flawed.

But such is the ever evolving world of statistics big data modelling and machine learning. There will be a better model, and a better one after that, and so on until there's one that's so good, it'll be a better actual predictor than pundits and experts

[zimpy27]

Yes they seem to always be wrong

[SalmonsSuperfan]

Nate Silver had his moment in the sun during the 2012 election when he accurately predicted the winner in every state, then leveraged his nerd fame into a **** buzzfeed+stats type publication. The content has been all downhill since, I’m surprised the site didn’t just fold after “100% chance Clinton wins election.”

What a sellout. And to think I wrote him in for president in 2014. Or maybe it was lil b, I kinda forget

[Jedzz]

https://projects.fivethirtyeight.com/2020-nba-predictions/

Their predictions and forecasts make 0 sense.

They are no better than fivethirtyseven and no worse than fivethirtynine.

[G35]

This is what happens when you rely solely on numbers conjured out of nowhere...these folks make Charles Barkley look like a clairvoyant

Wait for the mass dump of excuses saying "no one of relevance solely uses numbers" or "anyone using stats without context should not be taken seriously"

538 is becoming as reliable as Rotten Tomatoes......

[G35]

They're upfront about what their models measure and what they say.

However, if you ask anyone who's works in data modelling, they'll tell you it's almost always a case of "garbage in, garbage out".

In other words, the models ASSUMES past performance is indicative of true performance, and ergo future performance.

The models do not specifically account for:
1. Playoff basketball being a fundamentally different game to RS basketball
2. Playoff defensive and offensive scheming for specific matchups and teams
3. Players/teams performing above or below their RS baselines
4. Plain old simple variance

Once you understand this concept, their models makes perfect sense as to why they project results the way they do, and why they are fundamentally flawed.

But such is the ever evolving world of statistics big data modelling and machine learning. There will be a better model, and a better one after that, and so on until there's one that's so good, it'll be a better actual predictor than pundits and experts

+1

But to be fair quite a few posters here do the same thing assuming the RS is in any similar to the PS.

What you do in the PS validates or invalidates what happens in the RS......

[picko]

They are unfairly criticised for their 2016 election forecast - mostly by people that don't understand probabilities.

These playoff forecasts are really quite bizarre though. The Lakers / Rockets forecasts are particularly hard to understand given everything we saw during the season. That suggests that there are some issues regarding calibration and model parameters.

[bwgood77]

https://projects.fivethirtyeight.com/2020-nba-predictions/

Their predictions and forecasts make 0 sense.

They had Houston as a heavy favorite to make the finals and beat Lakers.

I think pre season going in and even during much of the season they didnt have Lakers as a top 3 seed maybe even top 4. I dont know how the hell the ods are even working the way they are now.

From the Bucks/Rockets/Clips/Raps and i think Sixers even earlier and some Jazz love theyve been wrong wrong wrong

they also had Hilary Clinton at 90 something % to beat Trump.............

Which ones make zero sense, the RAPTOR one based on their player ratings, the ELO one based on play to date, or both?

[Kurtz]

These models are pretty good - they just can't account for things like injuries, matchups, player development or guys like Lebron playing in 2nd gear during the season.

[Hroz]

You sure? At what point in 2016?

I remember them predicting Hilary to win


But piggybacking on the op, I was just thinking about this.

538 presents their predictions in a very neat and professional way, plus they don’t just present the data, they go to the extra step of interpreting the data and giving the reader a conclusion, they’re just always wrong lol

https://projects.fivethirtyeight.com/2016-election-forecast/

This was their election day forecast.
Trump to win @ 28.6% (which was similar to the odds the Trump team have themselves).

So they thought Trump had the same odds of winning as Doncic has of hitting a game winning three?

Yup & Doncic has hit a lot of those.
What happened was the polls were like 44% Hillary to 40% Trump

5% 3rd party
With like 10% undecideds.

The 10% undecided looked to have gone 2-1 to Trump.

And given Trump's electoral college advantage atm.
That's how it closed out.

[Hroz]

Nate Silver had his moment in the sun during the 2012 election when he accurately predicted the winner in every state, then leveraged his nerd fame into a **** buzzfeed+stats type publication. The content has been all downhill since, I’m surprised the site didn’t just fold after “100% chance Clinton wins election.”

What a sellout. And to think I wrote him in for president in 2014. Or maybe it was lil b, I kinda forget

Huh?
100%

This is the election day forecast they gave.
https://projects.fivethirtyeight.com/2016-election-forecast/

Also president of what in 2014?
Definitely wasn't the United States.

[GSP]

https://projects.fivethirtyeight.com/2020-nba-predictions/

Heat 76% chance to beat Lakers LOL

last time to laugh at this terrible model for this season

[Jables]

Find an actual model that works better. Bringing up politics just shows how badly you understand predictive modelling, 538 gave Trump a higher chance than any other model did.