Engelmann Playoff only RAPM (1997-2024)

[letskissbro]

This came out a couple days ago:
https://docs.google.com/spreadsheets/d/1KiGykvmgXmDv5ibAtobHui-DfjjRhpBueHrJMD8v3vk/htmlview

The previous 1997-2024 set Engelmann put out was RS + PS. This is just PS isolated. There is no aging curve applied this time according to Engelmann.

Top 10 overall:

Spoiler:

1. LeBron James : 7.7
2. Draymond Green : 6.3
3. Manu Ginobili : 5.9
4. Tim Duncan : 5.6
5. Kevin Durant : 5.2
6. Stephen Curry : 4.9
7. Jamal Murray : 4.5
8. Ray Allen : 4.2
9. James Harden : 4.2
10. Jayson Tatum: 4.1

Top 10 ORAPM

Spoiler:

1. LeBron James : 5.4
2. James Harden : 5.1
3. Stephen Curry : 4.5
4. Kevin Durant : 4.3
5. Ray Allen : 4.0
6. Jamal Murray : 3.8
7. Kyrie Irving : 3.8
8. Channing Frye : 3.7
9. Chauncey Billups : 3.4
10. Steve Nash : 3.3

Top 10 DRAPM:

Spoiler:

1. Ben Wallace : -3.3
2. Draymond Green: -3.2
3. Manu Ginobili : -3.2
4. David Robinson : -2.9
5. Tim Duncan : -2.7
6. P.J. Tucker : -2.7
7. Giannis Antetokounmpo : -2.5
8. Jason Collins : -2.4
9. LeBron James : -2.3
10. Kevin Garnett : -2.3

2020s Notables *1000+ MP* (ORAPM|DRAPM|Total):

Spoiler:

7. Jamal Murray : 3.8 | -0.7 | 4.5
11. Jayson Tatum : 1.9 | -2.2 | 4.1
14. Joel Embiid : 1.9 | -1.8 | 3.7
42. Giannis Antetokounmpo : -0.4 | -2.5 | 2.1
71. Jalen Brunson : 2.4 | 0.7 | 1.7
84. Devin Booker : 1.5 | 0 | 1.5
93. Nikola Jokic : 2.3 | 1 | 1.3
95. Jaylen Brown : 0.3 | -0.9 | 1.2
102. Bam Adebayo : 1 | -0.2 | 1.2
132. Donovan Mitchell : 1 | 0.2 | 0.8
190. Luka Doncic : 0.5 | 0.2 | 0.3

[EmpireFalls]

Channing Frye…

[GSP]

The gap b/w Draymond and Steph is nuts considering how much their mins overlap

But Steph didnt play in a number of playoff games tho that Warriors still dominated

Murray and Jokic too

[lessthanjake]

Given how tiny playoff off samples are, I think this is mostly extremely noisy to the point of being not very useful, except perhaps to look at the small number of people with tons of playoff games in this data set (guys like LeBron, Steph, Draymond, Duncan, and Ginobili), and even some of those wouldn’t be much less noisy than single-season RAPM (which is super noisy). And it’s actually probably even worse than just looking at the off minutes would lead you to believe, because, even for those players with lots of playoff minutes, the model will be trying to control for the presence of other players that have a very small amount of playoff data. So, basically, the adjustments the model makes compared to on-off will be mostly based on garbage.

[Doctor MJ]

The gap b/w Draymond and Steph is nuts considering how much their mins overlap

But Steph didnt play in a number of playoff games tho that Warriors still dominated

Murray and Jokic too

My analysis suggests that if you were to just chop out the first round games, the gap between Dray & Steph may basically disappear.

Jokic is interesting with this because it really doesn't look good for him...but at this point it's hard for me to actually see it as a serious concern. While it's certainly more impressive to have a big On/Off in any circumstance, most of the time guys who play huge minutes in a losing effort have bad Ons, and so as long as I'm seeing a guy do amazing things as a matter of course, and demonstrating that he can lead teams to great playoff success this way, it doesn't seem that big of a deal to me.

[Doctor MJ]

On the dark side, here are the lowest ranked guys who were named all-star and/or All-NBA in their career:

Tom Gugliotta -4.4
Anthony Mason -4.4
DeMar DeRozan -4.3
Andre Drummond -4.0
Caron Butler -4.0
Stephon Marbury -3.9
Wally Szczerbiak -3.9
Zydrunas Ilgauskas -3.8
Gordon Hayward -3.5
Clifford Robinson -3.3

[jalengreen]

On the dark side, here are the lowest ranked guys who were named all-star and/or All-NBA in their career:

Tom Gugliotta -4.4
Anthony Mason -4.4
DeMar DeRozan -4.3
Andre Drummond -4.0
Caron Butler -4.0
Stephon Marbury -3.9
Wally Szczerbiak -3.9
Zydrunas Ilgauskas -3.8
Gordon Hayward -3.5
Clifford Robinson -3.3

[OldSchoolNoBull]

On the dark side, here are the lowest ranked guys who were named all-star and/or All-NBA in their career:

Tom Gugliotta -4.4
Anthony Mason -4.4
DeMar DeRozan -4.3
Andre Drummond -4.0
Caron Butler -4.0
Stephon Marbury -3.9
Wally Szczerbiak -3.9
Zydrunas Ilgauskas -3.8
Gordon Hayward -3.5
Clifford Robinson -3.3

It looks like he just missed your cutoff, but...

Camelo Anthony: #1414, -3.1

And not that I take too much from the small sample sizes, but a couple other observations:

1. Chris Webber is just outside the Top 100, at 1.0. Meanwhile Bosh is #1302 at -2.4, and Horford and Gobert are also lower than Webber(though still in the Top 200). Just wanted to give C-Webb some love since he rarely gets any around here.

2. #71 seems low for Kobe.

3. #397 seems WAY low for Paul Pierce.

4. Tony Parker doesn't look great - #357, -0.6 - especially in comparison to his teammates Duncan, Manu, DRob, Kawhi, and even Danny Green, who are all in the Top 20 and all 3.5+.

5. Dwight Howard all the way down at #874, at -1.3.

[homecourtloss]

The gap b/w Draymond and Steph is nuts considering how much their mins overlap

But Steph didnt play in a number of playoff games tho that Warriors still dominated

Murray and Jokic too

Draymond has to be the most underrated, under-appreciated, over-maligned player by casuals (and some non-casuals for specific reasons) player in NBA history. What the Warriors did in 2015 and 2016 without curry on court with Dray including entire series should have put to rest the doubts about Draymond. Some will say “there’s a lot of minutes against lower competiton” and while that may be true, it’s difficult to dominate any playoff opposition except for a few ATGs who can. For everyone else, you’re just hoping to survive and get wins, not be +13 NRtg

2015+2016 playoffs

[iggymcfrack]

OK, based on this I’m immediately deciding that I’ve underrated Kevin Durant.

Old bottom of my top 25:
23. Wade
24. Kobe
25. KD

New bottom of my top 25:
23. KD
24. Wade
25. Kobe

Open to moving KD up further or finding an excuse to move Kobe ahead of Wade to keep him at #24 too.

Manu’s gonna have to pass AD for #29 and Harden’s gonna have to maybe pass AD too and move up to #30.

[iggymcfrack]

Really interesting to think about what the implications of this are for Draymond. I kinda had an inkling he was more crucial to the Warriors than regular season impact numbers would suggest. And now that I think about it, his defense always was a lot more valuable in the playoffs than the regular season. I think I’m going to have start thinking seriously about a top 30 all-time case for him.

[NO-KG-AI]

On the dark side, here are the lowest ranked guys who were named all-star and/or All-NBA in their career:

Tom Gugliotta -4.4
Anthony Mason -4.4
DeMar DeRozan -4.3
Andre Drummond -4.0
Caron Butler -4.0
Stephon Marbury -3.9
Wally Szczerbiak -3.9
Zydrunas Ilgauskas -3.8
Gordon Hayward -3.5
Clifford Robinson -3.3

Basically all of KG’s “star” teammates from Minnesota

[DraymondGold]

Cool stuff! I agree with the earlier statements that this is interesting, and worth doing to see how it compares to the RS-only or full-season versions... but that this is one of the less informative RAPMs made by Engelmann. No offense meant to Engelmann -- he's one of the great NBA statisticians, and I'm glad he made this RAPM -- but I want to caution over interpreting this relative to his prior RAPMs.

Limitations:
1) Full-career RAPM vs single/several-season RAPM: Doing a full-data RAPM (1997-2024) without any aging curve or anything of the sort treats these players as constant over their full career... over such a long timespan like this (over 25 seasons!) is obviously a faulty assumption. 2006 LeBron is a different player from 2016 LeBron. 2006 Kobe is a different player from 2016 Kobe. Treating the players as the same over their full career makes it far less clear what signal actually comes out at the end. Some sort of possession-weighted estimate of the players' average value?

Now, in regular season or full-season data, we have enough randomly sampled games between so many lineups and opponents that we probably get a reasonably accurate (albeit high uncertainty) version of the average value of a player. But that's far less true in playoff-only data.
2) Lack of sufficient playoff samples: A) Star players play more minutes, so the off-minutes and off-lineups are limited for them. B) Teams don't always make the playoffs, giving us a far reduced uneven sampling of their playoffs. Consider two players who are equal in value across age, and get worse as they get older. If one player gets older and their team misses the playoffs while another player gets older and their team makes the playoffs, we'd likely see the first player as rated better in this metric... when it's simply because they happened to make the playoffs, not because they're actually better. C) The lineups played and faced are not evenly or randomly sampled. Teams only face a few opponents each year at most. Worse still, there's only a single team each year that plays each other across conferences. This is far from the ideal case where each player and each lineup gets enough time against each other player and each other lineup that the we have a sufficient sample to calculate a fair RAPM. With no RS data used or RS prior, this limitation C seems particularly crippling to the accuracy of this exercise.

3) Lack of error bars given: the best full-career RAPMs give error bars, which are pretty crucial to interpreting how much better one player is from another. We don't have error bars here, unlike some other versions of full-career RAPM.

So it's an interesting exercise. It's definitely worth doing, as having a truly accurate and stable playoff RAPM would be one of the most useful stats when evaluating a player. But I fear the limitations are exacerbated by being playoff-only without any RS data whatsoever (particularly 2c), which undermines some of the usefulness.

The top tier of players are probably some of the most valuable playoff performers -- so this is nice evidence to support the playoff value of LeBron, Draymond, Manu, Duncan, Durant, and Curry. Likewise, if a player here is ranked especially different from expectations or from their RS-only full-career RAPM, that might be informative (or at a minimum, worth considering whether one of the limitations above applies to that player). But I would definitely hesitate to trust the exact ordering, even moreso without knowing the uncertainty bars.

Re: Draymond vs Curry, here's the raw plus minus data from pbpstats.com from the years they both made the playoffs:
13-25 Playoff Draymond:
On: +973 = +8.3 per 100
Off: -146 = -3.0 per 100
On-Off: +11.3 per 100

13-25 Playoff Curry:
On: +954 = +8.3 per 100
Off: -145 = -3.0 per 100
On-Off: +11.3 per 100

So if we don't filter for just the games they both played, they end up looking nearly identical in both On and On-Off. And given how close they are, it's not crazy for one of them to sneak ahead. But what if we just look at their plus minus profile in the games they both played?

13-25 Playoff Draymond (games Curry/Draymond both played):
On: +861 = +8.1 per 100
Off: -107 = -2.5 per 100
On-Off: +10.6 per 100

13-25 Playoff Curry (games Curry/Draymond both played):
On: +954 = +8.2 per 100
Off: -200 = -6.1 per 100
On-Off: +14.3 per 100

Suddenly Curry looks like the clear highest impact player for the Warriors, with a tiny advantage in On rating and a clear advantage in On-Off (which correlates slightly more strongly with RAPM than On). So as DoctorMJ said, Draymond got to strengthen his playoff data against easier first-round opponents... while Curry's missed games cause his sample to be biased low, as he only faced harder opponents.

This kind of biased sample seems like exactly the kind of thing a PS-only RAPM would struggle to capture. For first-round-exit teams, we lack any regular season data or prior, which might be used accurately measure that the first-round exit teams are worse than the late-playoffs teams. Worse yet, if these first-round exit teams are perennially first-round exit teams or occasionally missing the playoffs, then the sample size overall for them is relatively smaller, and again a playoff-only full-career RAPM doesn't gain as much information on them to see that they're easier opponents for the Draymond-only Warriors.

To be clear, I agree with others than Draymond is highly underrated by the casual fan, and one of the true high-impact playoff players of the modern era. As the second to third most valuable player on the most dominant dynasty of the era, we would expect him to be one of the most valuable players of the era.
(Note by value here, I mean value in their specific role... which some might argue was particularly suited or particularly favorable for Draymond relative to there rest of the top tier of players, or that Draymond's impact is more dependent on playing this role and less versatile relative to the rest of the top tier of players here ).
Draymond is arguably the best defender of this era (I would say he is), and was a clear positive on offense in the role he played for much of his playoff career. His chemistry with Curry and Klay was one of the key parts of the Warriors' success. It's seriously impressive that a Draymond-reliant Warriors could perform well in the first round without Curry.

But I don't think this is compelling evidence to think Draymond was more valuable to the playoff Warriors than Curry was over the course of their careers. I think the sample is just being biased here by the easier first-round opponents and the easier schedule that Draymond faced, which is exactly the kind of thing a playoff-only RAPM would miss if it ignores all regular season data.

[parsnips33]

Draymond has to be the most underrated, under-appreciated, over-maligned player by casuals (and some non-casuals for specific reasons) player in NBA history.

It's definitely not just casuals. Even some Warriors fans, who ostensibly watch the team play, don't get it.

Even if he was an angelic boy scout, it'd be hard to appreciate just how good he is. But then he makes it even harder by being a massive villain. Defies rating, the most interesting man in basketball

[homecourtloss]

Draymond has to be the most underrated, under-appreciated, over-maligned player by casuals (and some non-casuals for specific reasons) player in NBA history.

It's definitely not just casuals. Even some Warriors fans, who ostensibly watch the team play, don't get it.

Even if he was an angelic boy scout, it'd be hard to appreciate just how good he is. But then he makes it even harder by being a massive villain. Defies rating, the most interesting man in basketball

You are a Warriors fan, and you appreciate him. There are others who are Curry fans who downplay him for obvious reasons despite the overwhelming avalanche of evidence that supports his case as a top 50-60 player ever. Last season was a prime example of this –- even older, he was at worst at top 3 defender in the league. He could QB defenses until he's LeBron's age.

[falcolombardi]

Given how tiny playoff off samples are, I think this is mostly extremely noisy to the point of being not very useful, except perhaps to look at the small number of people with tons of playoff games in this data set (guys like LeBron, Steph, Draymond, Duncan, and Ginobili), and even some of those wouldn’t be much less noisy than single-season RAPM (which is super noisy). And it’s actually probably even worse than just looking at the off minutes would lead you to believe, because, even for those players with lots of playoff minutes, the model will be trying to control for the presence of other players that have a very small amount of playoff data. So, basically, the adjustments the model makes compared to on-off will be mostly based on garbage.

.

I agree completely, noisy samples like this or like the great work squared and others did with pre 2000's rapm are nice to have but extremely flawed to actually use in comparision to bigger and complete data sets of other players (or the same player*)

[homecourtloss]

it's always interesting that when there is some partial RAPM data released by squared or partial on/off data calculated by somebody or released by squared or somebody else and even if you have incomplete data or small off samples or whatever, as long as those data sets show certain players looking good there is no "cautioning," no "not very useful," "too small amount of data," for the various other caveats that you see not only in this thread, but every thread. After the usual "cautioning" and "take it with a grain of salt" and "small/tiny off samples," you see long posts talking about why the data isn't useful or not to be taken too seriously even though those very same cautioning or precautions aren't done with other data that also has small and/or incomplete sample sizes. In fact, those are the threads are responded to with "this is amazing work!" "this data really shows something," "GOAT peak GOAT signals!"

I am 100% sure that had this particular set not had LeBron far away at the top or curry below Draymond or Jokić nowhere near the top which is reflective of his playoffs on/off, etc., these very same cautioning, but of course not be given.

Consistency would be nice.

[lessthanjake]

Given how tiny playoff off samples are, I think this is mostly extremely noisy to the point of being not very useful, except perhaps to look at the small number of people with tons of playoff games in this data set (guys like LeBron, Steph, Draymond, Duncan, and Ginobili), and even some of those wouldn’t be much less noisy than single-season RAPM (which is super noisy). And it’s actually probably even worse than just looking at the off minutes would lead you to believe, because, even for those players with lots of playoff minutes, the model will be trying to control for the presence of other players that have a very small amount of playoff data. So, basically, the adjustments the model makes compared to on-off will be mostly based on garbage.

.

I agree completely, noisy samples like this or like the great work squared and others did with pre 2000's rapm are nice to have but extremely flawed to actually use in comparision to bigger and complete data sets of other players (or the same player*)

Yep—sample-size problems exist for Squared’s RAPM too. Sample-size issues are inherently a matter of degree though, and at this point with Squared’s RAPM the samples are generally bigger than these playoff samples are. Squared’s RAPM also doesn’t have the other issue I described, where “off” samples for superstars in the playoffs tend to be more consumed by garbage time. Not to mention that the playoff “off” sample will tend to be noticeably smaller for any given number of games played, because stars play more minutes per game in the playoffs. Because of these factors, the noise in a playoff sample is very likely going to be significantly higher than the noise in the same number of games in the regular season. When combined with the fact that the number of games in the overall Squared sample is at the point where it’s generally larger than these playoff samples, and I think there’s reason to think that sample-size issues are a noticeably smaller concern there. But it is certainly still a concern.

Of course, sample-size issues are not the only issue with Squared’s RAPM. Notably, because it does not include all games, Squared’s RAPM also has potential sampling error that isn’t here for most players in the data set (there is sampling error for players who played playoff games before 1997, but that’s not most players in the 1997-2024 span). So there’s good reason to take Squared’s data with a grain of salt, even if the sample-size issue isn’t as much of a problem as it is here.

[lessthanjake]

it's always interesting that when there is some partial RAPM data released by squared or partial on/off data calculated by somebody or released by squared or somebody else and even if you have incomplete data or small off samples or whatever, as long as those data sets show certain players looking good there is no "cautioning," no "not very useful," "too small amount of data," for the various other caveats that you see not only in this thread, but every thread. After the usual "cautioning" and cook take it with a grain of salt" and "small/tiny off samples," you see long posts talking about why the data isn't useful or not to be taken too seriously even though those very same cautioning or precautions aren't done with other data that also has small and/or incomplete sample sizes. In fact, those are the threats are responded to with "this is amazing work!" "this dara really shows something."

I am 100% sure that had this particular set not had LeBron far away at the top or curry below Draymond or Jokić nowhere near the top which is reflective of his playoffs on/off, etc., these very same cautioning, but of course not be given.

Consistency would be nice.

I’m sorry but this is a bit maddening. It seems fairly clear that you’re at least in part meaning to refer to me, since your post comes after a post that responded to me about this same issue (not to mention that you frequently make snide remarks directed at me, and your post’s other examples of Curry/Draymond and Jokic completely align with this referring to me). But you’ll find that I’m actually really consistent in caveating and cautioning regarding Squared’s data. I can’t even count how many times I’ve done so. Indeed, I actually kind of doubt there’s anyone on these forums who has done so more times than I have. I even just did it in response to Falco’s post, before I saw your post (ironically, I even used some of the exact same language you claim is never used regarding Squared, specifically saying it should be taken with “a grain of salt”)! The idea that “those same cautioning and precautions aren’t done with other data” is just manifestly false if you’re referring to me (which you definitely are). You’ll also find that I’m very consistent about expressing concerns regarding playoff RAPM, not just in this thread. Again, the idea that my concerns in this arena are motivated by the results of any particular measure is nonsense that is completely divorced from my actual post history. It’s just a complete straw man, and I find it maddening because it’s not the first time you’ve made posts like this. You tend to make posts where you are obviously talking about me without explicitly saying you’re talking about me, while making assertions/accusations about my post history that are just objectively untrue. I’d appreciate you not doing so. I can understand that you probably have not have read all my posts and that that could potentially be a good faith cause of your incorrect impressions, but even if that’s the case I’d say that you shouldn’t make broad assertions/accusations about my posts if you haven’t actually done your homework enough to know that what you’re saying is accurate.

[DraymondGold]

Given how tiny playoff off samples are, I think this is mostly extremely noisy to the point of being not very useful, except perhaps to look at the small number of people with tons of playoff games in this data set (guys like LeBron, Steph, Draymond, Duncan, and Ginobili), and even some of those wouldn’t be much less noisy than single-season RAPM (which is super noisy). And it’s actually probably even worse than just looking at the off minutes would lead you to believe, because, even for those players with lots of playoff minutes, the model will be trying to control for the presence of other players that have a very small amount of playoff data. So, basically, the adjustments the model makes compared to on-off will be mostly based on garbage.

.

I agree completely, noisy samples like this or like the great work squared and others did with pre 2000's rapm are nice to have but extremely flawed to actually use in comparision to bigger and complete data sets of other players (or the same player*)

Yep—sample-size problems exist for Squared’s RAPM too. Sample-size issues are inherently a matter of degree though, and at this point with Squared’s RAPM the samples are generally bigger than these playoff samples are. Squared’s RAPM also doesn’t have the other issue I described, where “off” samples for superstars in the playoffs tend to be more consumed by garbage time. Not to mention that the playoff “off” sample will tend to be noticeably smaller for any given number of games played, because stars play more minutes per game in the playoffs. Because of these factors, the noise in a playoff sample is very likely going to be significantly higher than the noise in the same number of games in the regular season. When combined with the fact that the number of games in the overall Squared sample is at the point where it’s generally larger than these playoff samples, and I think there’s reason to think that sample-size issues are a noticeably smaller concern there. But it is certainly still a concern.

Of course, sample-size issues are not the only issue with Squared’s RAPM. Notably, because it does not include all games, Squared’s RAPM also has potential sampling error that isn’t here for most players in the data set (there is sampling error for players who played playoff games before 1997, but that’s not most players in the 1997-2024 span). So there’s good reason to take Squared’s data with a grain of salt, even if the sample-size issue isn’t as much of a problem as it is here.

The other thing to emphasize with the limitations with playoff-only RAPM is just how limiting the lack of opponent diversity is.

Players face -- at maximum -- four different teams each season. That's miniscule. Even across careers, that's miniscule. Conferences face each other *once* in playoff-only single season samples, and those are only from a single team in each conference. The only plus minus signal we get to scale eastern conference vs western conference players are the finals, and the times players switch conferences. Again, utterly minuscule, and highly limiting.... and completely different from the Squared2020 data we have. These kind of limitations to the number of opposing lineups makes accurate and unbiased RAPM calculations vastly more difficult, and again it's far more pervasive to this playoff-only data than the historical data.

As an aside, it's a bummer homecourtloss would rather make passive aggressive, bad-faith complaints about you jake and presumably about me as well... without actually reading and processing the points we made in this very thread. I actually made this point above -- which distinguishes Engelmann's playoff-only RAPM from Squared2020's work -- in my previous post in this thread, which they seem to have ignored. It's not the first time they've taken to randomly insulting other people on this forum, and indeed far from the first time other discord members have taken to insulting people and accusing them of anti-LeBron bias. Just odd stuff. Oh well -- hopefully bad takes like this don't derail the rest of this thread, like they did the latest peaks project ...