EPM updated its formula... Luka was 4th, now 1st for 2023-24

[FrodoBaggins]

Now:

Before:

A pretty dramatic change. It makes you wonder how much stock should be placed in the various all-in-ones/one-number metrics available if the so-called publicly available best-in-class is subject to changes like this.

Makes a lot of the top 100 project discussion seem a bit silly to me with how stat-based most of it is. But that's just me. The more basketball exposure and experience I gain, the less strongly I feel about data as a definitive answer.

Previous EPM Ranking:

#1 Embiid +9.2

#2 SGA +7.7

#3 Jokic +7.6

#4 Doncic +7.5

#5 Giannis +5.9

Current EPM ranking

#1 Doncic +10.0

#2 SGA +9.5

#3 Jokic +8.9

#4 Giannis +7.8

#5 Embiid +7.8

A Reddit discussion about the topic: https://reddit.com/r/nba/comments/1xtowvl/epm_widely_considered_the_most_reliable_advanced/

[Mavrelous]

Every mathematical model, especially ones that incorporate statistical regression to isolate the needed outcome from independent variables is subject to change, the fact the authors of the model adjusted their model means they are actually improving it, not that it's useless.
Most of the prodcts you use daily have the models used in their designs change, it doesn't mean the prodcuts weren't designed correctly, your 10 y/o laptop works just fine.
Luka has always been a challenge to these models because the inherent assumption is that there is a base level of competency in roster building and setup, and while most stars do have this base level, in Luka's case due to starting from an asset depleted startpoint, he had to work with not even sub optimal, but outright ineffecient lineups that distorted the on/off signal that went into these model.
In the 23/24 season, Luka had +4 on/off pre deadline and +20 on/off post deadline, it wasn't a product of him playing any different, just Kyrie and Lively came back from prolonged injuries, and 2 additions in PJ and Gafford that closed the holes on roster.
I'd argue this statistical anomaly should have been a trigger to adjust the model back then, but better late than never...
This further strengthens what many argued back then, Luka was the actual MVP that season.

[migya]

Now:

Before:

A pretty dramatic change. It makes you wonder how much stock should be placed in the various all-in-ones/one-number metrics available if the so-called publicly available best-in-class is subject to changes like this.

Makes a lot of the top 100 project discussion seem a bit silly to me with how stat-based most of it is. But that's just me. The more basketball exposure and experience I gain, the less strongly I feel about data as a definitive answer.

Previous EPM Ranking:

#1 Embiid +9.2

#2 SGA +7.7

#3 Jokic +7.6

#4 Doncic +7.5

#5 Giannis +5.9

Current EPM ranking

#1 Doncic +10.0

#2 SGA +9.5

#3 Jokic +8.9

#4 Giannis +7.8

#5 Embiid +7.8

A Reddit discussion about the topic: https://reddit.com/r/nba/comments/1xtowvl/epm_widely_considered_the_most_reliable_advanced/

For me, I wonder how all this advanced stat/metric/impact numbers are calculated. I don't follow this that much, only what I've seen on here mostly, but did see a few months ago that there is three different sites with the rapm type metric and all have different number values for the same player, so not to be taken seriously.

[TheGOATRises007]

I would disagree with the notion that Luka had the highest impact that RS.

Interesting change.

[Doctor MJ]

Wow. This is a massive change. Was there any explanation given for how EPM changed its formula? If not, put me down for saying it’s a problem.

This was the whole deal with ESPN’s RPM. Makes sense that no formula is perfect, but when the public can’t see the formula itself, we need it to be consistent for it to have any meaning.

If the same stat argues one way first, and then the other way next, and we don’t know how it works or what changed, then the degree to which it changes sets a new level of uncertainty in our use.


Sent from my iPhone using Tapatalk

[Doctor MJ]

Every mathematical model, especially ones that incorporate statistical regression to isolate the needed outcome from independent variables is subject to change, the fact the authors of the model adjusted their model means they are actually improving it, not that it's useless.
Most of the prodcts you use daily have the models used in their designs change, it doesn't mean the prodcuts weren't designed correctly, your 10 y/o laptop works just fine.
Luka has always been a challenge to these models because the inherent assumption is that there is a base level of competency in roster building and setup, and while most stars do have this base level, in Luka's case due to starting from an asset depleted startpoint, he had to work with not even sub optimal, but outright ineffecient lineups that distorted the on/off signal that went into these model.
In the 23/24 season, Luka had +4 on/off pre deadline and +20 on/off post deadline, it wasn't a product of him playing any different, just Kyrie and Lively came back from prolonged injuries, and 2 additions in PJ and Gafford that closed the holes on roster.
I'd argue this statistical anomaly should have been a trigger to adjust the model back then, but better late than never...
This further strengthens what many argued back then, Luka was the actual MVP that season.

So not to open up the old can of worms but from a pure impact standpoint point, the fact that Luka looked way better statistically later in the year as context changed doesn’t mean we should ignore what came before.

If it wasn’t about Luka personally improving, then it was about worse vs better fit around him, and both the worse and better fits/impact were real.

I’m honestly very curious how they changed the EPM formula and what would cause this. I could see a time-decay thing going on where they decided to make their formula be biased toward later events in the season. If that’s what they did it’s an understandable move, but it doesn’t make metrics who treat the entire span equally flawed. Just answering different questions.


Sent from my iPhone using Tapatalk

[Natural_Bridge]

Advanced metrics are an exercise in marketing, they have to be adjusted to match the new popular narrative lest they be left behind by the newer formula that spits out the list that everyone wants

They have never been worth anything

[Doctor MJ]

Reading up on a little, looks like the changes come from running things again and new weights leading to better prediction success given all the data they have now.

This is a reasonable gold standard to have for a stat maker generally and in particular at this time where gambling is being championed by the league.

It doesn’t explain what caused the changes though, and the reality is that they may not even know.

It does seem clear that they are using time-decay models, so the scenario I described above about weighing the latter part of the season more than the former is plausible, but even if that’s a factor, we don’t know if that’s the only thing going on.


Sent from my iPhone using Tapatalk

[eminence]

For most - just look at On and On/Off - nearly as effective and not at all difficult to understand.

Sure it won't be as predictive after 5 games or whatever, but who cares?

[Mavrelous]

e old can of worms but from a pure impact standpoint point, the fact that Luka looked way better statistically later in the year as context changed doesn’t mean we should ignore what came before.

If it wasn’t about Luka personally improving, then it was about worse vs better fit around him, and both the worse and better fits/impact were real.

I’m honestly very curious how they changed the EPM formula and what would cause this. I could see a time-decay thing going on where they decided to make their formula be biased toward later events in the season. If that’s what they did it’s an understandable move, but it doesn’t make metrics who treat the entire span equally flawed. Just answering different questions.
Sent from my iPhone using Tapatalk

He didn't look way better statistically in general, he was the same except in one specific stat, that is very context dependent, and that is being used as a major component in all catch all metrics.
The metric is supposed to measure player's impact, in a team sport, this is an impossible task to do, and how it's done is much more art than science.

[Doctor MJ]

But yeah, this is why I’m always so cautious about putting too much stock in equations we can’t see inside of, and why I’ve advocated for APM/RAPM with longer sample:

We know what goes into APM. There’s no mystery there.

RAPM adds a little mystery in which I hate, and it’s why RAPM from a new source is always suspect if it deviates from norms. But generally RAPM seems to have standardized through Englemann’s approach getting picked up by others, and so now we’re at a time when I don’t worry as much.

Anything that adds box score in makes me way more cautious. No doubt that in a single year model you gain reliability by adding the box score component, and I am interested in mid-season data so I’ve been cautiously using stats like EPM for a while now for that purpose. But I also haven’t been having to deal with major changes in a major hybrid stat like this in a good while, and so now that we are again, I expect I’ll use them less for a while.


Sent from my iPhone using Tapatalk

[Doctor MJ]

e old can of worms but from a pure impact standpoint point, the fact that Luka looked way better statistically later in the year as context changed doesn’t mean we should ignore what came before.

If it wasn’t about Luka personally improving, then it was about worse vs better fit around him, and both the worse and better fits/impact were real.

I’m honestly very curious how they changed the EPM formula and what would cause this. I could see a time-decay thing going on where they decided to make their formula be biased toward later events in the season. If that’s what they did it’s an understandable move, but it doesn’t make metrics who treat the entire span equally flawed. Just answering different questions.
Sent from my iPhone using Tapatalk

He didn't look way better statistically in general, he was the same except in one specific stat, that is very context dependent, and that is being used as a major component in all catch all metrics.
The metric is supposed to measure player's impact, in a team sport, this is an impossible task to do, and how it's done is much more art than science.

I'd say I agree with most of this but I know we disagree broadly.

From my perspective the fact that a player has different impact in two different contexts is what we should expect in all walks of life, and so there's nothing actually weird about it. In terms of what it means for achievement, I count all that happens, and while the playoffs do provide a way for a guy to really shift my ranking, on the whole there's nothing problematic about treating the entire season like it matters.

[JimmyFromNz]

Whilst I like referencing EPM. This is a prime example of the blind faith placed in these models when arguing (often on the head of a pin) the distinctions in player profiles. Are we now supposed to revise our view of Luka?

I don't agree with the earlier post dismissing the analytics, but the idea many of these formulas are first and foremost market/commercially driven (we are being sold 'advanced' formulas in the format of 'simplicity') does not receive as much attention as it should.

[Doctor MJ]

Whilst I like referencing EPM. This is a prime example of the blind faith placed in these models when arguing (often on the head of a pin) the distinctions in player profiles. Are we now supposed to revise our view of Luka?

I don't agree with the earlier post dismissing the analytics, but the idea many of these formulas are first and foremost market/commercially driven (we are being sold 'advanced' formulas in the format of 'simplicity') does not receive as much attention as it should.

So, to the question: Yeah, that's the hard truth we have to face:

If we end up relying on data to make assessments, and something problematic is revealed about the data, then we should absolutely try to re-assess. What's so hard is that the human brain doesn't keep these factoids all distinct in our brains, and so it may have influenced us in the past and yet we don't even know.

This isn't a problem that's unique to basketball analytics of course - it's the problem with our proverbial crazy uncle who seemed like a genius when we were young but then we realize is a crackpot. How do we go and weed out crazy-uncle influence on us when we're not even sure where it is?

The only way forward really is to seek to see all of our beliefs as not-for-sure, to entertain challenges to them, and to take pride in being able to uncover prior beliefs we had that turned out to be false.

[DonaldSanders]

A pretty dramatic change. It makes you wonder how much stock should be placed in the various all-in-ones/one-number metrics available if the so-called publicly available best-in-class is subject to changes like this.

Makes a lot of the top 100 project discussion seem a bit silly to me with how stat-based most of it is. But that's just me. The more basketball exposure and experience I gain, the less strongly I feel about data as a definitive answer.

As someone who likes EPM, this is a good demonstration of why I only use all-in-one metrics to show what "tier" or grouping a guy belongs in. Using them as a definitive ranking of players is absurd. I never thought Embiid was MVP because he led in EPM, but I did think it meant he had a strong MVP case.

I have to say though, this is a bigger change than I would have expected. We had what looked like a top 4 group before, now it's more like a top 6 group with the #1 guy dropping to 5th. It makes me wonder if there is some revisionist stat changes to make the model match what the creator thinks is 'correct.' I don't know if that is happening, but it makes you wonder with how much Embiid has fallen in public eye. It definitely makes me feel a bit less confident about the stat.

And the idea that play later in the regular season is worth more than earlier, I really hope that's not true. If anything it's often the opposite, especially in the tanking era -- guys end up playing teams who are trying to lose at the end of the year and put up insane stats. I'm really curious (and a bit nervous) how the formula works.

[VanWest82]

I'm surprised Snarr decided to put so much emphasis on bitching at the refs but it's his stat so he can do whatever he likes i guess. Maybe we need a substat called Snarrled.

[jalengreen]

It's impossible for me to engage with this update without some circularity. My instinct is something like "wow, that doesn't align with my eye test. I felt that 2023-24 Embiid was a tier above anyone else when he was on the floor." But I also surely knew about Embiid's EPM that season and so I cannot rule out that my perception at the time was in part shaped by what I knew of his EPM. Anybody indexing on EPM to form opinions is thus inherently prone to disagreeing with major adjustments because they formed opinions based on earlier versions of the metric!

The closer you align your opinions to EPM -> the more skepticism you will have toward changes in EPM because they go against the opinions you've aligned to the old version. That's not to say that you cannot genuinely disagree - I think I'm doing so when I say I think the first version had it right with Embiid - but you can't really know for sure that you're not biased by the outdated EPM values.

[Mavrelous]

e old can of worms but from a pure impact standpoint point, the fact that Luka looked way better statistically later in the year as context changed doesn’t mean we should ignore what came before.

If it wasn’t about Luka personally improving, then it was about worse vs better fit around him, and both the worse and better fits/impact were real.

I’m honestly very curious how they changed the EPM formula and what would cause this. I could see a time-decay thing going on where they decided to make their formula be biased toward later events in the season. If that’s what they did it’s an understandable move, but it doesn’t make metrics who treat the entire span equally flawed. Just answering different questions.
Sent from my iPhone using Tapatalk

He didn't look way better statistically in general, he was the same except in one specific stat, that is very context dependent, and that is being used as a major component in all catch all metrics.
The metric is supposed to measure player's impact, in a team sport, this is an impossible task to do, and how it's done is much more art than science.

I'd say I agree with most of this but I know we disagree broadly.

From my perspective the fact that a player has different impact in two different contexts is what we should expect in all walks of life, and so there's nothing actually weird about it. In terms of what it means for achievement, I count all that happens, and while the playoffs do provide a way for a guy to really shift my ranking, on the whole there's nothing problematic about treating the entire season like it matters.

I'll never agree with or understand this approach.
If context shouldn't be accounted for, then why use RAPM? On/off should be good enough, the only reason to use RAPM is to isolate the players contribution, and if, as you claim, you agree that lineup optimization is also a factor, then it is as important as the regression on the on/off lineups.

[jagz]

Advanced metrics are an exercise in marketing, they have to be adjusted to match the new popular narrative lest they be left behind by the newer formula that spits out the list that everyone wants

They have never been worth anything

They're not completely useless. They just were never intended as player comparators (despite the way they are used by 97% of analyitcs-dependent fans).

They were designed to help teams gain advantages in highly specific in-game situations. But, somewhere along the way, they became contorted into these all-in-one metrics that purported to establish player rankings.

[Doctor MJ]

He didn't look way better statistically in general, he was the same except in one specific stat, that is very context dependent, and that is being used as a major component in all catch all metrics.
The metric is supposed to measure player's impact, in a team sport, this is an impossible task to do, and how it's done is much more art than science.

I'd say I agree with most of this but I know we disagree broadly.

From my perspective the fact that a player has different impact in two different contexts is what we should expect in all walks of life, and so there's nothing actually weird about it. In terms of what it means for achievement, I count all that happens, and while the playoffs do provide a way for a guy to really shift my ranking, on the whole there's nothing problematic about treating the entire season like it matters.

I'll never agree with or understand this approach.
If context shouldn't be accounted for, then why use RAPM? On/off should be good enough, the only reason to use RAPM is to isolate the players contribution, and if, as you claim, you agree that lineup optimization is also a factor, then it is as important as the regression on the on/off lineups.

I’m saying value is context dependent, and since value is what we’re talking about when we talk about something like Most Valuable Player, it doesn’t make sense to talk about MVP and reject the stats that - like value - are context dependent.

Re: “lineup optimization as important as regression”, these are not two things I’d be looking to compare in this way, but certainly fit is central to value.

I’ve said since Luka got traded that the team wouldn’t be a contender until LeBron was gone. It wasn’t because LeBron was a bad player, it was because of fit concerns.

Those fit concerns would have been moot if Luka could have magically turned himself into a better fit with LeBron like KG did in Boston, and had Luka magically done so he would have been a strong MVP contender last year.

But he couldn’t, so the Lakers weren’t a serious team, and thus Luka wasn’t a serious contender for MVP as he didn’t add the value he would have needed to to be so Valuable.

Now that the Lakers made good fit moves, Luka is adding value like an MVP.

This is part of the broader phenomenon where stars form superteams together and then are mystified when the team isn’t as good as they thought it would be. The stars basically assume each of them would carry their value with them to the new context, but we’re disappointed because it doesn’t…because value is context dependent.


Sent from my iPhone using Tapatalk