RealGM

There’s been a lot of assertions made in these forums about whose impact is most underestimated or overestimated by box metrics. This is a relevant inquiry, because while box metrics correlate with impact in general, that doesn’t mean it is accurately estimating impact for a specific player. That matters when using box measures to evaluate players. And it also matters when using box-impact hybrids (i.e. all-in-ones) to evaluate players, since those basically use a box measure as a prior, and having a prior that overestimates or underestimates a player will affect the result for that player.

As far as I’m aware though, while a lot of assertions have been made about who box measures might be overestimating or underestimating, no one has really looked at the most obvious way to systematically look into this, which is the RAPM/BPM graphs that Basketball-Reference provides on its About Box Plus Minus page (https://www.basketball-reference.com/about/bpm2.html). There, Basketball-Reference has plotted five-year RAPM on the x-axis and BPM on the y-axis, using the five-year periods of 1997-2001, 2002-2006, 2007-2011, and 2012-2016.

We can look at the plot and pick out individual data points to see specific players’ RAPM and BPM values in those timeframes. This allows us to compare those values for specific players, and actually analyze whose five-year RAPM most underperforms or overperforms BPM (which is the premier box metric).

I have done this, looking at 30 prominent players. For some of these players, the data included some time periods where they were largely not actually star players, which didn’t seem informative. So the rule I used was that I looked at a time period for a player if they got MVP votes in multiple seasons in that five-year span. There were a few players I looked at that that didn’t really make sense for, since they didn’t actually get many MVP votes. So, specifically for Ray Allen, I looked at all the timeframes where he made more than one all-star game, and for Manu Ginobili I just looked at every timespan. For Draymond and Klay, I just looked at the only relevant timespan that existed for them, even though they did not get multiple years of MVP votes in that timeframe (i.e. 2012-2016).

Here’s a list of time periods for each player, organized by the percentage by which their RAPM was higher (or lower) than their BPM. So, for instance, if the player’s RAPM value in a time period was 10% higher than their BPM in that time period, then it would be +10%. If RAPM was 10% lower than BPM, it would be -10%.

Players’ Five-Year Time Periods Ranked by Percent that their RAPM is Above their BPM

1. 2012-2016 Klay Thompson: +261.50%
2. 2012-2016 Draymond Green: +124.20%
3. 2007-2011 Steve Nash: +117.50%
4. 2007-2011 Ray Allen: +103.45%
5. 2007-2011 Chris Bosh: +89.30%
6. 2007-2011 Dwight Howard: +82.50%
7. 1997-2001 Tim Duncan: +72.00%
8. 2007-2011 Dirk Nowitzki: +53.60%
9. 2012-2016 Manu Ginobili: +52.80%
10. 2007-2011 Kevin Garnett: +51.90%
11. 1997-2001 Kevin Garnett: +50.00%
12. 1997-2001 Ray Allen: +48.30%
13. 1997-2001 Reggie Miller: +46.20%
14. 1997-2001 Jason Kidd: +45.90%
15. 2012-2016 Dirk Nowitzki: +43.80%
16. 2002-2006 Steve Nash: +42.10%
17. 2002-2006 Ray Allen: +41.90%
18. 2002-2006 Dirk Nowitzki: +36.90%
19. 2002-2006 Kevin Garnett: +29.80%
20. 2012-2016 Russell Westbrook: +27.10%
21. 2012-2016 Stephen Curry: +22.50%
22. 2002-2006 Jason Kidd: +21.60%
23. 2012-2016 Kevin Love: +21.30%
24. 1997-2001 Gary Payton: +21.2%
25. 1997-2001 Michael Jordan: +20.70%
26. 2002-2006 Manu Ginobili: +20.00%
27. 2012-2016 LeBron James: +19.80%
28. 2002-2006 Tim Duncan: +18.90%
29. 2007-2011 Kobe Bryant: +17.50%
30. 2012-2016 Chris Paul: +16.70%
31. 2002-2006 Shaquille O’Neal: +13.00%
32. 1997-2001 John Stockton: +12.50%
33. 2002-2006 Dwyane Wade: +7.69%
34. 2007-2011 LeBron James: +7.69%
35. 1997-2001 Shaquille O’Neal: +2.67%
36. 2002-2006 Kobe Bryant: +1.72%
37. 2007-2011 Manu Ginobili: +0.00%
38. 2012-2016 Kevin Durant: -1.10%
39. 2007-2011 Tim Duncan: -1.80%
40. 1997-2001 David Robinson: -4.60%
41. 2012-2016 Tim Duncan: -7.50%
42. 2012-2016 James Harden: -8.10%
43. 2002-2006 LeBron James: -10.60%
44. 1997-2001 Scottie Pippen: -17.07%
45. 2007-2011 Dwyane Wade: -20.50%
45. 2007-2011 Chris Paul: -21.70%
47. 1997-2001 Karl Malone: -22.90%
48. 2012-2016 Kawhi Leonard: -23.20%
49. 1997-2001 Hakeem Olajuwon: -37.50%
50. 1997-2001 Kobe Bryant: -54.50%
51. 2007-2011 Kevin Durant: -80.00%
52. 2012-2016 Dwyane Wade: -83.80%
53. 2012-2016 Kobe Bryant: -121.74%

The effect here in these individual time periods can vary a fair bit for an individual player. So, in order to look at players overall, I ran the geometric mean of each player’s values. For this, I used their RAPM divided by their BPM. So, for instance, if someone had a RAPM of 2.0 and a BPM of 4.0, then the value for that time period for geometric-mean purposes would be 0.5.

Players Ranked by Geometric Mean of their RAPM Overperformance Compared to BPM

1. Klay Thompson: +261.50%
2. Draymond Green: +124.20%
3. Chris Bosh: +89.30%
4. Dwight Howard: +82.50%
5. Steve Nash: +75.80%
6. Ray Allen: +62.38%
7. Reggie Miller: +46.20%
8. Dirk Nowitzki: +44.61%
9. Kevin Garnett: +43.54%
10. Jason Kidd: +33.20%
11. Russell Westbrook: +27.10%
12. Stephen Curry: +22.50%
13. Manu Ginobili: +22.40%
14. Kevin Love: +21.30%
15. Gary Payton: +21.20%
16. Michael Jordan: +20.70%
17. Tim Duncan: +16.75%
18. John Stockton: +12.50%
19. Shaquille O’Neal: +7.71%
20. LeBron James: +4.87%
21. Chris Paul: -4.40%
22. David Robinson: -4.60%
23. James Harden: -8.10%
24. Scottie Pippen: -17.07%
25. Karl Malone: -22.90%
26. Kawhi Leonard: -23.20%
27. Hakeem Olajuwon: -37.50%
28. Dwyane Wade: -48.24%
29. Kevin Durant: -55.53%

I note that, for these purposes, I left Kobe off, since his 2012-2016 RAPM being negative throws off the geometric mean calculation—but I note that Kobe’s number here without that time period would be -18.38%, and overall with the 2012-2016 value, Kobe would obviously end up at the very bottom here.

Please also note that, for these numbers for each player, you can find a version using the average of the simple difference between RAPM and BPM in a subsequent post here: https://forums.realgm.com/boards/viewtopic.php?p=118891014#p118891014. That method changes the order a little at the margins, but not in any particularly significant way.

_________________

Takeaways

What are the takeaways here? Well, I’m sure there could be quite a lot since there’s a lot of information.

One obvious takeaway is that the oft-stated claim that LeBron James is particularly undersold by box metrics like BPM appears to be wrong. In fact, he comes in towards the bottom of the players listed here, when it comes to the percent by which his RAPM overperforms his BPM.

Another takeaway might be to look at how well off-ball scorers do. Specifically, with Klay Thompson, Dirk Nowitzki, Ray Allen, Reggie Miller, and Stephen Curry all coming out at the higher end of this (and triangle Jordan being notably underestimated by BPM as well—albeit not by quite as much as these other guys), it seems rather clear that the impact of off-ball scorers tend to be underestimated by box stats (or at least by BPM in particular). I’ll note that I also noticed Kyle Korver being a player on the chart whose RAPM far outdid his BPM, so that’s another data point for this concept. Of course, this all makes sense, since those players’ off-ball gravity doesn’t make it into the box score.

The takeaway regarding great big-man defenders is a bit mixed. You have guys like Draymond Green, Dwight Howard, and Kevin Garnett coming out near the top, which would be consistent with the conventional wisdom that elite big-man defenders are underestimated in box metrics. And, while I didn’t list him here, I did notice Mutombo being similar in this regard. But, at the same time, we see David Robinson and Hakeem Olajuwon coming out near the bottom. So the conclusion isn’t clear cut, though on balance most of these sorts of players do seem to be underrated by box measures. Perhaps the distinction with Robinson and Hakeem is that they were also volume scorers—which would tend to get them more box-score credit—but Dwight and Garnett scored a good bit too, so that doesn’t give us a bright-line rule that great big-man defenders that score a lot aren’t underrated by BPM.

Another observation is that Kidd, Westbrook, and Payton all come out on the high end here. Perhaps this suggests that star point guards who do not shoot very efficiently are a bit underrated by box measures (i.e. perhaps they get penalized for their relative lack of scoring efficiency more than they really should).

With Steve Nash, I’m not sure that there’s a broader conclusion to take from him being near the top. I have never quite understood why his BPM is so low (especially when plenty of pass-first PGs have very high BPMs), but if you look at his BPM it is surprisingly low, so his RAPM outpacing it by a lot is unsurprising. There’s a similar thing with Bosh, who just has a surprisingly low BPM for reasons I can’t quite pinpoint.

When it comes to the bottom of this list, it does include some players with only one time period here (Karl Malone, Kawhi, Hakeem, Harden, Pippen, and Robinson), so I suppose it’s possible this wouldn’t be the case for them with more than one time period. Meanwhile, Durant being at the bottom isn’t very surprising—I think the conventional wisdom on him is that his box data outpaces his impact. Ditto with Kobe. Wade gets pretty hurt here by a time period that includes post-prime years coming in with a big BPM-to-RAPM drop, though he’d still be around -7.5% even without that time period.

Anyways, I’m sure there’s other interesting things people may notice here.

Using percentage change doesn't really make sense here, as zero isn't the bottom - it's just league average. That's going to skew results for guys who are closer to average than the top-end elite guys as their denominators will be smaller.

As both metrics should be on the same scale (in theory) just a simple difference between them seems more reasonable.

The overall takeaways would be similar. Perhaps the big man defense issue is partly how aggressively they look to block shots (though Howard looked for blocks plenty).

giberish wrote:Using percentage change doesn't really make sense here, as zero isn't the bottom - it's just league average. That's going to skew results for guys who are closer to average than the top-end elite guys as their denominators will be smaller.

As both metrics should be on the same scale (in theory) just a simple difference between them seems more reasonable.

The overall takeaways would be similar. Perhaps the big man defense issue is partly how aggressively they look to block shots (though Howard looked for blocks plenty).

I’m not sure I agree that looking at the numerical difference instead of the percentage difference is better, since I think percentage change makes it easier to draw inferences about what is happening with specific categories of players. Notably, I also prefer to use the geometric mean, and you can’t do that if some of the numbers are negative (which they will often be if using a simple difference). One would need to just use the mean instead. All that said, it doesn’t make a huge difference here either way, since I’ve tallied up these numbers for really good players (i.e. the denominator is pretty large for everyone). The order changes a little bit on the margins, but not in a radical way. And I do see your point that there’s reason to prefer that methodology instead.

So, in the interests of presenting the data in another way that people might find helpful, here’s the average for each player based on the numerical difference between RAPM and DPM.

Average of Players’ Difference between RAPM and BPM

1. Draymond Green: +4.10
2. Klay Thompson: +3.40
3. Dwight Howard: +3.30
4. Steve Nash: +3.15
5. Kevin Garnett: +2.57
6. Chris Bosh: +2.50
7. Dirk Nowitzki: +2.27
8. Ray Allen: +2.07
9. Reggie Miller: +1.80
10. Stephen Curry: +1.80
11. Michael Jordan: +1.70
12. Russell Westbrook: +1.60
13. Jason Kidd: +1.40
14. Gary Payton: +1.10
15. Kevin Love: +1.00
16. Manu Ginobili: +1.00
17. Tim Duncan: +0.90
18. John Stockton: +0.80
19. LeBron James: +0.67
20. Shaquille O’Neal: +0.45
21. Chris Paul: -0.20
22. David Robinson: -0.30
23. James Harden: -0.50
24. Scottie Pippen: -0.70
25. Kobe Bryant: -0.88
26. Hakeem Olajuwon: -1.20
27. Kawhi Leonard: -1.30
28. Dwyane Wade: -1.43
29. Karl Malone: -1.60
30. Kevin Durant: -1.65

Because this method doesn’t really change the order in any genuinely significant way, I think the takeaways are essentially the same either way.

I should also note that this analysis is, of course, as compared to one specific form of RAPM. Granted, that RAPM was run by the Director of Basketball Analytics for the Dallas Mavericks, so it is likely done well. But there’s still methodological decisions with RAPM that can reasonably go either way, and which would result in at least somewhat different RAPM values. So while this is comparing BPM to one particular form of RAPM, the results might be a little different if comparing to a different form of RAPM.

Just to add to this point about off-ball scorers being hugely underrated by BPM, a few other notable examples:

- 2002-2006 Richard Hamilton had a RAPM of +3.1, but a BPM of only +0.5. Similarly, 2007-2011 Richard Hamilton had a RAPM of +1.6 and a BPM of only +0.6. So the geometric mean of the percent by which his RAPM outdoes his BPM is +306.61%. And the average of the simple difference between his RAPM and BPM is +1.80.

- 2012-2016 Kyle Korver had a RAPM of +7.8, but a BPM of only +1.3. This is obviously a massive difference. In 2007-2011, his RAPM was +0.7 and his BPM was +0.4. So the geometric mean here is +224.04%. And the average of the simple difference between his RAPM and BPM is +3.4.

- 2012-2016 J.J. Redick had a RAPM of +2.7, but a BPM of only +0.9. So his RAPM was 200% higher, and +1.80 higher in terms of simple difference.

lessthanjake wrote:As far as I’m aware though, while a lot of assertions have been made about who box measures might be overestimating or underestimating, no one has really looked at the most obvious way to systematically look into this, which is the RAPM/BPM graphs that Basketball-Reference

I mean...we have an even simpler way of looking at who is over/underrating players in the metrics the "assertions" are made about with filtering between pure impact components and the overall score and/or the box-only and the overall score.

RAPM most underperforms or overperforms BPM (which is the premier box metric).

By what bar is BPM the most prominent? Surely you're not going to ignore what we've already documented looking at other all-in-ones to make wierd general conclusions...

Takeaways

What are the takeaways here? Well, I’m sure there could be quite a lot since there’s a lot of information.

One obvious takeaway is that the oft-stated claim that LeBron James is particularly undersold by box metrics like BPM appears to be wrong. In fact, he comes in towards the bottom of the players listed here, when it comes to the percent by which his RAPM overperforms his BPM.

Another takeaway might be to look at how well off-ball scorers do. Specifically, with Klay Thompson, Dirk Nowitzki, Ray Allen, Reggie Miller, and Stephen Curry all coming out at the higher end of this (and triangle Jordan being notably underestimated by BPM as well—albeit not by quite as much as these other guys), it seems rather clear that the impact of off-ball scorers tend to be underestimated by box stats (or at least by BPM in particular). I’ll note that I also noticed Kyle Korver being a player on the chart whose RAPM far outdid his BPM, so that’s another data point for this concept. Of course, this all makes sense, since those players’ off-ball gravity doesn’t make it into the box score.

I guess if we ignore LEBRON, RAPTOR, and MAMBA all showing the exact opposite relationship... with Steph Curry's rapm being lower and Lebron's RAPM being higher. I don't really know why you would try to use BPM as a representation for all box-stats in the first place here...

That said, I find this particular metric underrating Steph more a bit surprising considering that in this RAPM set Lebron performs much better with the top 2 5-year grades of anyone including a significantly higher 12-16 mark. The gap in "underrated" is marginal (22 vs 19) but Lebron has a strong RAPM advantage in this set to start with. BPM is probably uniquely favorable to Lebron compared to other all-in-ones

OhayoKD wrote:

lessthanjake wrote:As far as I’m aware though, while a lot of assertions have been made about who box measures might be overestimating or underestimating, no one has really looked at the most obvious way to systematically look into this, which is the RAPM/BPM graphs that Basketball-Reference

I mean...we have an even simpler way of looking at who is over/underrating players in the metrics the "assertions" are made about with filtering between pure impact components and the overall score and/or the box-only and the overall score.

RAPM most underperforms or overperforms BPM (which is the premier box metric).

By what bar is BPM the most prominent? Surely you're not going to ignore what we've already documented looking at other all-in-ones to make wierd general conclusions...

Takeaways

What are the takeaways here? Well, I’m sure there could be quite a lot since there’s a lot of information.

One obvious takeaway is that the oft-stated claim that LeBron James is particularly undersold by box metrics like BPM appears to be wrong. In fact, he comes in towards the bottom of the players listed here, when it comes to the percent by which his RAPM overperforms his BPM.

Another takeaway might be to look at how well off-ball scorers do. Specifically, with Klay Thompson, Dirk Nowitzki, Ray Allen, Reggie Miller, and Stephen Curry all coming out at the higher end of this (and triangle Jordan being notably underestimated by BPM as well—albeit not by quite as much as these other guys), it seems rather clear that the impact of off-ball scorers tend to be underestimated by box stats (or at least by BPM in particular). I’ll note that I also noticed Kyle Korver being a player on the chart whose RAPM far outdid his BPM, so that’s another data point for this concept. Of course, this all makes sense, since those players’ off-ball gravity doesn’t make it into the box score.

I guess if we ignore LEBRON, RAPTOR, and MAMBA all showing the exact opposite relationship... with Steph Curry's rapm being lower and Lebron's RAPM being higher. I don't really know why you would try to use BPM as a representation for all box-stats in the first place here...

You’re not making a particularly valid point here, except with regards to MAMBA (which I’ll discuss further below, in what is, I think, the most important part of my response here).

As an initial matter, I’ll note that we can’t actually test your statements about LEBRON and RAPTOR, because RAPTOR doesn’t exist anymore and LEBRON doesn’t actually parse out its components. But let’s just ignore that and assume for now that you’re right that LeBron would look better in the impact component of those metrics compared to the box component. It’d still be missing the point, and is based on a misunderstanding of how these metrics work and what this thread is about. The various components of those metrics are designed to work together to be as accurate as possible. Accuracy is measured as against long-term RAPM (usually five-year RAPM). The box components of the metrics are used because they actually get the output substantially closer to long-term RAPM than you’d get if you just used short-term RAPM (or whatever the “impact component” actually is—for RAPTOR, it was not actually RAPM) without a box component. In other words, by arguing for the use of just the “impact component” of these metrics, you’re just arguing for something that is demonstrably further away in general from long-term RAPM than it would be if you added the box component. So it doesn’t really follow to suggest that we should take that “impact component” as more indicative of what long-term RAPM would say compared to the box component. The “impact component” in all-in-ones is certainly going to correlate with long-term RAPM, but the box component makes the whole thing correlate more. How could it be that short-term RAPM data could be better for someone than box numbers, while that wouldn’t necessarily be the case for long-term RAPM data? Well, short-term RAPM is noisy, and isn’t just noisy for the player at issue but for everyone that it’s adjusting for—just making the whole thing much less accurate. Basically, the “impact component” of LEBRON and RAPTOR isn’t long-term RAPM, so it doesn’t tell us how long-term RAPM compares to box components. And this thread is about how long-term RAPM compares to box components.

I’ll note that the above becomes increasingly obvious when we realize that there’s tons of long-term RAPM data that goes against the conclusion you’re trying to glean about long-term RAPM from all-in-one impact components. For instance, we have lots of five-year RAPM that has prime Steph ahead of LeBron. Of course, as I’ve mentioned, there’s different ways to do RAPM—with lots of methodological decisions that affect the output—so that’s not the case for every metric (more on that below). But the inference you’re trying to draw about long-term RAPM from year-by-year “impact components” of metrics is, by and large, not actually consistent with the sources of long-term RAPM we have.

As for MAMBA, that’s actually different and a more sensical point, since we apparently know from the creator that the box component of that metric brings down LeBron compared to that creator’s long-term RAPM more than it does for other star players, and that that creator’s long-term RAPM has LeBron at the top of the heap in the timeframes it looks at. So that actually is an example of the opposite of what we see in this thread, where we see that BPM actually underrates most stars more than LeBron, when compared to long-term RAPM developed by the Director of Basketball Analytics at the Dallas Mavericks. There’s two possible explanations for this. Either the box component the MAMBA creator used is substantially less good for LeBron than BPM, or the RAPM the MAMBA creator looked at is substantially better for LeBron than the RAPM of the Director of Basketball Analytics at the Dallas Mavericks. If it’s the former, then it means that the arguments you have made about LeBron’s impact and box data is not generalizable at all, but instead is specific to the box component of that one particular new metric. Meanwhile, if it’s the latter, then that just goes to the point that not all RAPM is the same. We know that the MAMBA creator’s RAPM was time-decay RAPM, and that that is functionally very similar to PI RAPM that uses the player’s previous RAPM values as the prior. We also know that such PI RAPM is abnormally good for LeBron (it’s obvious if you compare RAPM sets). Meanwhile, we know that the MAMBA creator’s RAPM did not adjust for the rubberband effect—which is a methodological decision that almost certainly hurts a player like Steph quite a lot, whose team was so often ahead by huge amounts. So it’s not a surprise for the MAMBA creator’s RAPM to be relatively quite favorable to LeBron.

So where does that leave us? Well, we know that, compared to the five-year RAPM of the Director of Basketball Analytics at the Dallas Mavericks, BPM underrates most stars compared to RAPM more than it underrates LeBron. That said, we also know that there are likely ways to run RAPM that are more favorable to LeBron than that. We might even be able to come up with RAPM sets that are favorable enough to LeBron that he actually looks more underrated by box measures than the vast majority of stars (it seems that the MAMBA creator did exactly that). But, at that point, the argument you’ve made about box measures underrating LeBron becomes transparently dependent on using the most convenient RAPM sets to make your point. Which makes it an argument that very clearly cannot in good faith be generalized to the kind of “box underrates LeBron’s impact” claim that you’ve tried to pass off. It may well be the case that there are some box measures that underrate LeBron—after all, there’s lots of different box measures and they come to different outputs, so there’s surely ones that underrate essentially anyone—but you’ve attempted to make a much more general argument that LeBron’s impact abnormally outpaces his box data, which is clearly not actually a viable claim and no longer should be made.

That said, I find this particular metric underrating Steph more a bit surprising considering that in this RAPM set Lebron performs much better with the top 2 5-year grades of anyone including a significantly higher 12-16 mark. The gap in "underrated" is marginal (22 vs 19) but Lebron has a strong RAPM advantage in this set to start with. BPM is probably uniquely favorable to Lebron compared to other all-in-ones

I don’t think it’s surprising at all that LeBron performs really well in absolute terms in this RAPM. For one thing, as is obvious from the analysis in this thread, he obviously performs the best in box terms in those timeframes as well. And that’s not surprising. There’s not really anyone we’d expect to do better than peak LeBron in the 1997-2001, 2002-2006, 2007-2011, and 2012-2016 timeframes. 2012-2016 is too early to fully capture Steph at his best (since it includes a couple non-prime years), and 1997-2001 only captures the very end of Jordan’s prime. The only other major greats who this RAPM actually captures in their best years are Garnett and Duncan. As with lots of other RAPM, Garnett gets close to LeBron but isn’t quite there. Overall, I don’t think the actual RAPM output here is at all surprising. And certainly, based on your professed theory that box data underrates LeBron’s impact, you’d have objectively expected LeBron to be further ahead in these timeframes than he actually is.

As for BPM being more favorable to LeBron than other box measures, it’s possible, but it’s not like he really does uniquely well in BPM. He led the league in it a bunch of times, but he’s typically been ranked about 5th in the league (and worse than that in more recent years, of course) for over a decade now. His prime BPM is lower than Jordan’s and Jokic’s. It’s a measure he looks great in, but it’s not a measure that is supportive of him being the best basketball player, so it’d be a pretty remarkable (and damning) thing for it to nevertheless be “uniquely favorable” to him.

Pssh, not even the Dallas Mavericks listen to the Dallas Mavericks Analytics Department.

I kid I kid, I'm sure James ran a fine RAPM, it certainly looked reasonable to me when Daniel was doing BPM 2.0.

lessthanjake wrote:

OhayoKD wrote:

lessthanjake wrote:As far as I’m aware though, while a lot of assertions have been made about who box measures might be overestimating or underestimating, no one has really looked at the most obvious way to systematically look into this, which is the RAPM/BPM graphs that Basketball-Reference

I mean...we have an even simpler way of looking at who is over/underrating players in the metrics the "assertions" are made about with filtering between pure impact components and the overall score and/or the box-only and the overall score.

RAPM most underperforms or overperforms BPM (which is the premier box metric).

By what bar is BPM the most prominent? Surely you're not going to ignore what we've already documented looking at other all-in-ones to make wierd general conclusions...

Takeaways

What are the takeaways here? Well, I’m sure there could be quite a lot since there’s a lot of information.

One obvious takeaway is that the oft-stated claim that LeBron James is particularly undersold by box metrics like BPM appears to be wrong. In fact, he comes in towards the bottom of the players listed here, when it comes to the percent by which his RAPM overperforms his BPM.

Another takeaway might be to look at how well off-ball scorers do. Specifically, with Klay Thompson, Dirk Nowitzki, Ray Allen, Reggie Miller, and Stephen Curry all coming out at the higher end of this (and triangle Jordan being notably underestimated by BPM as well—albeit not by quite as much as these other guys), it seems rather clear that the impact of off-ball scorers tend to be underestimated by box stats (or at least by BPM in particular). I’ll note that I also noticed Kyle Korver being a player on the chart whose RAPM far outdid his BPM, so that’s another data point for this concept. Of course, this all makes sense, since those players’ off-ball gravity doesn’t make it into the box score.

I guess if we ignore LEBRON, RAPTOR, and MAMBA all showing the exact opposite relationship... with Steph Curry's rapm being lower and Lebron's RAPM being higher. I don't really know why you would try to use BPM as a representation for all box-stats in the first place here...

You’re not making a particularly valid point here, except with regards to MAMBA (which I’ll discuss further below, in what is, I think, the most important part of my response here).

As an initial matter, I’ll note that we can’t actually test your statements about LEBRON and RAPTOR, because RAPTOR doesn’t exist anymore and LEBRON doesn’t actually parse out its components.

These statements were made 2 years ago with screenshots to go with them in the top 100 with you acknowledging those same RAPTOR on/off ratings. Are you trying to play dumb here?

Regardless before going offline 538 were nice enough to provide us the breakdowns here:
https://github.com/fivethirtyeight/data/blob/master/nba-raptor/modern_RAPTOR_by_team.csv

But let’s just ignore that and assume for now that you’re right that LeBron would look better in the impact component of those metrics compared to the box component. It’d still be missing the point, and is based on a misunderstanding of how these metrics work and what this thread is about. The various components of those metrics are designed to work together to be as accurate as possible. Accuracy is measured as against long-term RAPM (usually five-year RAPM).

Sorry are you referring to the long-term rapm which...checks notes...Lebron is almost always grading as the databall-outlier while Steph isn't even top 3?

You keep repeating this wall of text over and over again but the counter is and always will be the same. Accuracy writ large for every nba player does not make a stat unbiased in a comparison with twp. Especially when you aren't testing it vs approaches that would be biased towards the other player (in case you're wondering, that's the condition that needs to be fulfilled for those correlations to qualify the outputs as actual evidence).

If you wish to say the made-up stuff is more relevant than actual winning, you can, but what is not in doubt is it's only the made-up stuff which steph preforms better or comparably in RAPTOR or MAMBA from 15-17.

(or whatever the “impact component” actually is—for RAPTOR, it was not actually RAPM) without a box component.

Man if only they told us what the impact component is:
https://fivethirtyeight.com/features/how-our-raptor-metric-works/

They took a set of on/off inputs very similar to what is used for RAPM and regressed it to a 6-year rapm set that creating a process that is different from rapm ways which they consider insignificant.

I'm not sure how exactly you think that translates to the box-score not bolstering Steph significantly relative to his RAPM

I’ll note that the above becomes increasingly obvious when we realize that there’s tons of long-term RAPM data that goes against the conclusion you’re trying to glean about long-term RAPM from all-in-one impact components. For instance, we have lots of five-year RAPM that has prime Steph ahead of LeBron.

Which of course, does not go against anything, since the claim was not "Lebron beats Steph's during his best rapm period in every set", but that he outperforms boxscores which...every hybrid that provides both an impact and box-component seeing steph benefit relative to lebron in box-only and lebron benefit relative to steph in impact-only, supports.

This is like arguing because Lebron does way better than Steph in the RAPM BPM uses, it's inconsistent to argue Steph is underrated more than Lebron in BPM. And then you try to take things many steps further.

Which makes it an argument that very clearly cannot in good faith be generalized to the kind of “box underrates LeBron’s impact” claim that you’ve tried to pass off. It may well be the case that there are some box measures that underrate LeBron—after all, there’s lots of different box measures and they come to different outputs, so there’s surely ones that underrate essentially anyone—but you’ve attempted to make a much more general argument that LeBron’s impact abnormally outpaces his box data, which is clearly not actually a viable claim and no longer should be made.

So let's see here.

-> We have a data-set where the box-metric in question underrates Lebron's rapm
-> We have a bunch of hybrids where the box-components are far outpaced by "essentially rapm" on/off components
-> We have a proper RAPM which massively outpaces it's corresponding box-input
-> We have Lebron consistently massively outpacing that already box-outpacing rapm when we look at full-games and seasons.

And your argument here is "Steph while doing far worse in rapm was underrated a little more during the only 5-year stretch he does well in here in this specific metric while outpacing his rapm in a bunch of others and therefore Lebron's impact does not abnormally outpace his box data even though I just made a block of text trying to justify using box-scores which comically underperform Lebron's corresponding impact inputs"

Hmm, let me think about this...



Yeah, no. Your argument is bad and you should feel bad for making it.

There’s not really anyone we’d expect to do better than peak LeBron in the 1997-2001, 2002-2006, 2007-2011, and 2012-2016 timeframes. 2012-2016 is too early to fully capture Steph at his best (since it includes a couple non-prime years)

The sets which include Steph's other prime years still have him grading out below duncan and kg consistently lol. Two players Lebron has sets where he literally posts not one, not two, not three, not four, not five, not six, but 7 better 5-year stretches than.

What are you trying to accomplish here?

The only player who has ever threatened Lebron's RAPM beyond cherrypicking time-frames is Kevin Garnett who Steph never matches. Moreover that threat Garnett offers almost always dissapates if you apply any sort of volume consideration because it is largely based on low-minute stretches in Boston where he platooned with the Celtics other stars (and accordingly massively overperforms his raw signal).

You've been trying to throw partial and unevenly sampled RAPM data to frame Jordan as a challenger, but the latest Squared dump just put a Magic-sized hole in that.

And of course if we look at full-games/large chunks of seasons/full seasons instead of game snippets, Lebron's only in sight of unreplicated signals from Bill Walton and more generally Bill Russell with an in-era srs adjustment.

If you want to kick Lebron off his empirical throne, you're going to have to a whole lot better than posting an rapm set where he's still torching everyone.

His prime BPM is lower than Jordan’s and Jokic’s.

And absolutely torches both when you account for seasonal averages lol:



The full quote was "uniquely favorable relative to other all-in-ones". As your own data shows, even when he's the at the absolute apex, it still underrates him.

OhayoKD wrote:

But let’s just ignore that and assume for now that you’re right that LeBron would look better in the impact component of those metrics compared to the box component. It’d still be missing the point, and is based on a misunderstanding of how these metrics work and what this thread is about. The various components of those metrics are designed to work together to be as accurate as possible. Accuracy is measured as against long-term RAPM (usually five-year RAPM).

Sorry are you referring to the long-term rapm which...checks notes...Lebron is almost always grading as the databall-outlier while Steph isn't even top 3?

That’s just false, or at least requires you to cherry-pick which RAPM sets you decide to use. Of course, the claim I’ve long made regarding Steph isn’t even that his RAPM peaks out as high as LeBron’s does, but rather that prime Steph outpaced LeBron during that same time period. And there’s a bunch of RAPM sets that show that. I’ve posted them a bunch of times, and you just get upset and start talking about how posting fulsome data is somehow bad, and how we should only consider the specific data sources (or even just components of specific data sources) that are most favorable to your preferred conclusion. It’s a path we’ve been down so many times that there’s really no point, nor is that what this thread is about.

You keep repeating this wall of text over and over again but the counter is and always will be the same. Accuracy writ large for every nba player does not make a stat unbiased in a comparison with twp. Especially when you aren't testing it vs approaches that would be biased towards the other player (in case you're wondering, that's the condition that needs to be fulfilled for those correlations to qualify the outputs as actual evidence).

Yes, I agree that accuracy writ large does not make a stat unbiased in a comparison with two players. That’s part of the point of this thread. But you seem to fail to understand (or be unwilling to understand because it would require you to back down from a dumb argument that you feel like you need) that the single-year “impact component” of an all-in-one is not the same thing as long-term RAPM. The point of all-in-ones is to try to approximate long-term RAPM as closely as possible despite the small sample size, by combining a noisy and inaccurate impact component with a box component. That impact component is not the same as long-term RAPM, and if it actually correlated really well with it, there’d be no need for the box component. Your argument just conflates two separate things. An all-in-one’s “impact component” is not the same as long-term RAPM. Please get that through your head.

If you wish to say the made-up stuff is more relevant than actual winning, you can, but what is not in doubt is it's only the made-up stuff which steph preforms better or comparably in RAPTOR or MAMBA from 15-17.

Again, that’s just not true. In fact, since you seem to define “made-up stuff” to be any prior, it’s really the opposite, since RAPM with no prior is amongst the stuff that prime Steph does best at (and definitely outpaces LeBron in the same time periods). We’ve been over this so many times that it’s shocking you still say the same things, despite me having laid out incredibly large amounts of information on this. But maybe “made-up stuff” to you doesn’t include priors that boost LeBron relative to others—it wouldn’t be surprising at all if that were your position, since your only view on data is to twist yourself in a pretzel to justify not considering things that would lead to conclusions you don’t like, even if that requires not considering most information. Maybe now that you know it’s biased in favor of LeBron you’ll decide that BPM isn’t “made-up stuff”!

(or whatever the “impact component” actually is—for RAPTOR, it was not actually RAPM) without a box component.

Man if only they told us what the impact component is:
https://fivethirtyeight.com/features/how-our-raptor-metric-works/

Spoiler:

In comparison to “Box” RAPTOR, calculating a player’s “On-Off” RAPTOR rating is relatively simple. To calculate it, we undertook essentially the same process as for “Box” RAPTOR, regressing various offensive and defensive ratings against Davis’s six-year RAPM estimates. RAPM can be replicated quite effectively using three types of on-court ratings.20

The player’s offensive and defensive ratings — that is, how many points the team scored and allowed per 100 possessions while he was on the floor, adjusted for strength of opposition.
The player’s courtmates’ weighted average offensive and defensive ratings21 when they weren’t sharing the floor with the player. The average is weighted by the number of possessions that the courtmate shared with the player, multiplied by the number of possessions that the courtmate had without the player.22
Finally, the player’s courtmates’ other courtmates’ weighted average offensive and defensive ratings.23 Whereas a player’s raw offensive and defensive ratings (step 1) are associated with positive coefficients (It’s good if a player’s team is outscoring its opponent while he’s on the floor!) and the player’s courtmates’ ratings (step 2) are associated with negative coefficients (It’s a bad sign for a player’s if his teammates are doing well even when he’s not on the floor!), the ratings of his courtmates’ other courtmates are a positive indicator. (I know this part is a little confusing.) Essentially, high courtmates’ courtmates’ ratings mean the teammates who seemed like they were doing well without the player on the floor may only have been doing well because they were paired with other good teammates.
We find that further iterations (i.e., looking at a player’s courtmates’ courtmates’ courtmates’ ratings) don’t contribute toward predicting RAPM.

We also find that this comparatively simple way to evaluate a player’s on-court/off-court impact not only replicates RAPM extremely well in sample but also predicts out-of-sample RAPM as well or slightly better than RAPM itself, depending on the regression specification. In other words, RAPM doesn’t appear to add much value as compared with computationally simpler approaches to evaluating on-court/off-court ratings.

They took a set of on/off inputs very similar to what is used for RAPM and regressed it to a 6-year rapm set that creating a process that is different from rapm ways which they consider insignificant.

I’m not sure what you think you’re arguing about here. You’re just quoting a portion of my post in which I noted the objectively true fact that RAPTOR’s impact component was not actually RAPM. I was not making any argument about that, but simply stating an objective fact. The fact that you’re latching onto that to try to argue about it in some weird way is emblematic of the level to which you grasp at straws in these discussions.

I’ll note that the above becomes increasingly obvious when we realize that there’s tons of long-term RAPM data that goes against the conclusion you’re trying to glean about long-term RAPM from all-in-one impact components. For instance, we have lots of five-year RAPM that has prime Steph ahead of LeBron.

Which of course, does not go against anything, since the claim was not "Lebron beats Steph's during his best rapm period in every set", but that he outperforms boxscores which...every hybrid that provides both an impact and box-component seeing steph benefit relative to lebron in box-only and lebron benefit relative to steph in impact-only, supports.

This is like arguing because Lebron does way better than Steph in the RAPM BPM uses, it's inconsistent to argue Steph is underrated more than Lebron in BPM. And then you try to take things many steps further.

Again, you seem to lack an understanding of the very basic fact that the “impact component” in single-season all-in-ones is fundamentally not the same thing as long-term RAPM. Please get that through your head, and you may understand that your argument is fundamentally a non-sequitur. Of course, you basically have to make that argument because the actual long-term RAPM we have generally doesn’t support the conclusion you want to reach (as per comprehensive data I’ve previously compiled on many occasions), so you have to cling to something else.

I’ll just briefly illustrate this for you. It is very possible that, in an all-in-one that has an impact component and box component, someone could do better in the box component than the impact component and nevertheless have all of it undershoot their actual long-term RAPM. Because the “impact component” and long-term RAPM are not the same thing.

Which makes it an argument that very clearly cannot in good faith be generalized to the kind of “box underrates LeBron’s impact” claim that you’ve tried to pass off. It may well be the case that there are some box measures that underrate LeBron—after all, there’s lots of different box measures and they come to different outputs, so there’s surely ones that underrate essentially anyone—but you’ve attempted to make a much more general argument that LeBron’s impact abnormally outpaces his box data, which is clearly not actually a viable claim and no longer should be made.

So let's see here.

-> We have a data-set where the box-metric in question underrates Lebron's rapm
-> We have a bunch of hybrids where the box-components are far outpaced by "essentially rapm" on/off components
-> We have a proper RAPM which massively outpaces it's corresponding box-input
-> We have Lebron consistently massively outpacing that already box-outpacing rapm when we look at full-games and seasons.

I’ll tackle these bullet points in order:

1. I’m not sure what you’re talking about here. Are you referring to the data in this thread? The RAPM and BPM are not scaled the same. The line of best fit has a slope of 0.675534 rather than 1.0, and LeBron’s data is all above that line (i.e. his BPM is higher relative to RAPM than the average player).

2. Again, you fail to understand that the impact components of all-in-ones are not long-term RAPM. They’re extremely noisy and inaccurate one-year impact components, which correlate with long-term RAPM but are definitely not the same thing. And those components actually correlate far better with long-term RAPM if we add a box component—indeed, 538 specifically said that RAPTOR weighted the box component more highly because it actually correlated *better* with long-term RAPM. Either way, the impact components of all-in-ones are not long-term RAPM, but rather are super noisy things that no one in their right mind cares much about on their own.

3. I assume you’re referring to MAMBA. I addressed that in an earlier post. We know that the MAMBA creator logically must be looking at long-term RAPM that is particularly favorable to LeBron (and we have a good idea the methodological decisions that had that effect). Again, if your point is dependent on using the most convenient RAPM sets, then it’s not a good faith argument. “LeBron’s most favorable RAPM sets outpace his box data compared to other players” really is not the same thing as “LeBron’s impact categorically outpaces his box data compared to other players.”

4. This is just about WOWY data, which is outside the scope of this thread so isn’t worth discussing at length here, but I’ll just note that we know from DraymondGold’s work that WOWY signals are basically overcome by noise.

And your argument here is "Steph while doing far worse in rapm was underrated a little more during the only 5-year stretch he does well in here in this specific metric while outpacing his rapm in a bunch of others and therefore Lebron's impact does not abnormally outpace his box data even though I just made a block of text trying to justify using box-scores which comically underperform Lebron's corresponding impact inputs"

Hmm, let me think about this...



Yeah, no. Your argument is bad and you should feel bad for making it.

Lol, I really don’t think Steph doing a little worse than LeBron in RAPM in 2012-2016 is the data point you think it is. I would certainly hope prime LeBron would do better in a time period that includes two pre-prime years for Steph! But Steph’s impact data compared to LeBron is also not the point of this thread, so it’s not a rabbit hole I want to go down with you again—we’ve done that so many times in the past and I’m extremely comfortable with how those discussions have gone.

The fact that Steph and most other stars did better in RAPM compared to BPM than LeBron did is clear evidence of a box metric that LeBron is actually overrated in compared to other stars. Which fundamentally makes a “LeBron’s impact outpaces his box data compared to other players” argument wrong. Again, I’m sure we could come up with box components that would underrate him compared to other stars. There are enough box components out there that we should expect everyone will be overrated by some and underrated by some. That’s why it is best to look at all information at our disposal, so that those instances can cancel out as much as possible. But the idea that you’ve tried to pass off that LeBron is categorically underrated by box data compared to other stars is objectively false and not an argument you should ever make again. If you’d like to make arguments along those lines about specific metrics, that may be more tenable (though it seems your arguments on that basically just come down to conflating impact components with long-term RAPM, so they’re pretty bad arguments themselves). But the notion that LeBron’s impact categorically outpaces his box data more than other stars is untenable in light of the fact that the opposite is true for the most prominent box metric there is.

There’s not really anyone we’d expect to do better than peak LeBron in the 1997-2001, 2002-2006, 2007-2011, and 2012-2016 timeframes. 2012-2016 is too early to fully capture Steph at his best (since it includes a couple non-prime years)

The sets which include Steph's other prime years still have him grading out below duncan and kg consistently lol. Two players Lebron has sets where he literally posts not one, not two, not three, not four, not five, not six, but 7 better 5-year stretches than.

What are you trying to accomplish here?

The only player who has ever threatened Lebron's RAPM beyond cherrypicking time-frames is Kevin Garnett who Steph never matches. Moreover that threat Garnett offers almost always dissapates if you apply any sort of volume consideration because it is largely based on low-minute stretches in Boston where he platooned with the Celtics other stars (and accordingly massively overperforms his raw signal).

I’m not making an argument in this thread about who is ahead of who in RAPM (though some of my responses to you above touch on that, since you have decided to raise it—likely due to your own insecurity as to how our prior discussions on that subject have gone for you). What I’m talking about is very obviously about how players do in long-term RAPM relative to box metrics (and specifically BPM). This is something you have made categorical claims about, and those categorical claims are clearly false.

Beyond just LeBron James (this may be news to you, but there is actually more to basketball discussion than that), I also think the data in this thread can help give us information about what types of players in general are underrated by BPM. For instance, we see pretty consistently that BPM underrates off-ball scorers. This is pretty helpful to be aware of, particularly when looking at players before the play-by-play era. Knowing that BPM likely underrates certain types of players is good to know when trying to assess the impact of players prior to 1997.

You've been trying to throw partial and unevenly sampled RAPM data to frame Jordan as a challenger, but the latest Squared dump just put a Magic-sized hole in that.

Lol, not the topic of this thread, but yes, Jordan being ranked #1 in a RAPM set and being miles ahead of anyone that isn’t one of the top several players of all time really is a “hole.”

The full quote was "uniquely favorable relative to other all-in-ones". As your own data shows, even when he's the at the absolute apex, it still underrates him.

Again, the slope of the line of best fit here is not 1. LeBron’s BPM is higher relative to RAPM than it is for the average player. And, as demonstrated in this thread, it’s higher relative to RAPM than it is for most major stars too. The idea that it “still underrates him” is based on an incorrect notion that BPM and RAPM are scaled exactly the same.

lessthanjake wrote:

OhayoKD wrote:

But let’s just ignore that and assume for now that you’re right that LeBron would look better in the impact component of those metrics compared to the box component. It’d still be missing the point, and is based on a misunderstanding of how these metrics work and what this thread is about. The various components of those metrics are designed to work together to be as accurate as possible. Accuracy is measured as against long-term RAPM (usually five-year RAPM).

Sorry are you referring to the long-term rapm which...checks notes...Lebron is almost always grading as the databall-outlier while Steph isn't even top 3?

That’s just false, or at least requires you to cherry-pick which RAPM sets you decide to use.

And by "cherrypick" you mean...include seasons before 2013? Because you have yet to provide a sourced set provided where he even matches duncan or kg over similar time-frames.

Of course, the claim I’ve long made regarding Steph isn’t even that his RAPM peaks out as high as LeBron’s does

That is indeed what you've argued.

You keep repeating this wall of text over and over again but the counter is and always will be the same. Accuracy writ large for every nba player does not make a stat unbiased in a comparison with twp. Especially when you aren't testing it vs approaches that would be biased towards the other player (in case you're wondering, that's the condition that needs to be fulfilled for those correlations to qualify the outputs as actual evidence).

Yes, I agree that accuracy writ large does not make a stat unbiased in a comparison with two players. That’s part of the point of this thread. But you seem to fail to understand (or be unwilling to understand because it would require you to back down from a dumb argument that you feel like you need) that the single-year “impact component” of an all-in-one is not the same thing as long-term RAPM.

The impact component you just described as "whatever that is"? LEBRON uses RAPM. MAMBA uses RAPM. RAPTOR uses a process they describe as "basically RAPM". None of these involve box-scores. And all of them have Steph benefit from a box-prior which hurts Lebron. These are not the straws you're looking for.

You found a all-in-one where Steph is marginally hurt more while falling well behind in both the box-stat and the rapm it's based off. How in the world does that apply to all the all-in-ones you dump specifically because their outputs will outpace Steph's performance in the actual impact data they're adding bias to?

I’ll note that the above becomes increasingly obvious when we realize that there’s tons of long-term RAPM data that goes against the conclusion you’re trying to glean about long-term RAPM from all-in-one impact components. For instance, we have lots of five-year RAPM that has prime Steph ahead of LeBron.

Which of course, does not go against anything, since the claim was not "Lebron beats Steph's during his best rapm period in every set", but that he outperforms boxscores which...every hybrid that provides both an impact and box-component seeing steph benefit relative to lebron in box-only and lebron benefit relative to steph in impact-only, supports.

This is like arguing because Lebron does way better than Steph in the RAPM BPM uses, it's inconsistent to argue Steph is underrated more than Lebron in BPM. And then you try to take things many steps further.

Again, you seem to lack an understanding of the very basic fact that the “impact component” in single-season all-in-ones is fundamentally not the same thing as long-term RAPM.

Two of them literally use RAPM. The third uses "basically RAPM". Please explain which step in the third you think took Steph from being well ahead of Lebron to being well behind (as putting in the box component leads to)?

Your proof Steph is the secretly underrated one is an all-in-one which is marginally biased against Lebron where both the all-in-one and the corresponding RAPM strongly favors Lebron. if you want to argue specifically, as he was getting whacked by lebron on both fronts, he was a bit more underrated? Cool.

But those are not the time-frames your Steph-arguments are actually based on. And those are not the time-frames over which "Lebron is more underrated than Steph by the box-score" is argued for.

And then there are the larger chunks which show Lebron absolutely smoking his RAPM and box-score consistently because of context like staggering and playing a position up during 4 years of his prime.

Again, the slope of the line of best fit here is not 1. LeBron’s BPM is higher relative to RAPM than it is for the average player. And, as demonstrated in this thread, it’s higher relative to RAPM than it is for most major stars too. The idea that it “still underrates him” is based on an incorrect notion that BPM and RAPM are scaled exactly the same.

Higher relative to the average, significantly lower relative to the median as the coloring denotes. If you are using the former, then Steph's 12-16 is also overrated by the stat and they're close in overratedness for the stretch actually relevant to what you're pushing.

OhayoKD wrote:

lessthanjake wrote:

OhayoKD wrote:

Sorry are you referring to the long-term rapm which...checks notes...Lebron is almost always grading as the databall-outlier while Steph isn't even top 3?

That’s just false, or at least requires you to cherry-pick which RAPM sets you decide to use.

And by "cherrypick" you mean...include seasons before 2013? Because you have yet to provide a sourced set provided where he even matches duncan or kg over similar time-frames.

Of course, the claim I’ve long made regarding Steph isn’t even that his RAPM peaks out as high as LeBron’s does

That is indeed what you've argued.

This is false, and you know that. You can go back to our many prior discussions about this and see that it is false. The only thing that has truth to it here is that I have not “include[d] seasons before 2013,” which is a pretty unremarkable statement when my point has been that *prime Steph* outpaced LeBron during that same time period. Somehow in this response you simultaneously deny that that’s what I’ve argued while making an assertion that is true precisely because that is what I’ve argued. Totally bizarre stuff. Given how many times we’ve been over this, either you have a severe reading comprehension impairment, or you’re willfully obfuscating because you have no good argument. Either way, it is only tangentially relevant to this thread, and you nevertheless appear to insist upon bringing it up here because you feel a real need to relitigate prior discussions that have not gone well for you. I am very happy to just direct you and others to go search our post histories for the many prior discussions we’ve had on this.

You keep repeating this wall of text over and over again but the counter is and always will be the same. Accuracy writ large for every nba player does not make a stat unbiased in a comparison with twp. Especially when you aren't testing it vs approaches that would be biased towards the other player (in case you're wondering, that's the condition that needs to be fulfilled for those correlations to qualify the outputs as actual evidence).

Yes, I agree that accuracy writ large does not make a stat unbiased in a comparison with two players. That’s part of the point of this thread. But you seem to fail to understand (or be unwilling to understand because it would require you to back down from a dumb argument that you feel like you need) that the single-year “impact component” of an all-in-one is not the same thing as long-term RAPM.

The impact component you just described as "whatever that is"? LEBRON uses RAPM. MAMBA uses RAPM. RAPTOR uses a process they describe as "basically RAPM". None of these involve box-scores. And all of them have Steph benefit from a box-prior which hurts Lebron. These are not the straws you're looking for.

You found a all-in-one where Steph is marginally hurt more while falling well behind in both the box-stat and the rapm it's based off. How in the world does that apply to all the all-in-ones you dump specifically because their outputs will outpace Steph's performance in the actual impact data they're adding bias to?

Which of course, does not go against anything, since the claim was not "Lebron beats Steph's during his best rapm period in every set", but that he outperforms boxscores which...every hybrid that provides both an impact and box-component seeing steph benefit relative to lebron in box-only and lebron benefit relative to steph in impact-only, supports.

This is like arguing because Lebron does way better than Steph in the RAPM BPM uses, it's inconsistent to argue Steph is underrated more than Lebron in BPM. And then you try to take things many steps further.

Again, you seem to lack an understanding of the very basic fact that the “impact component” in single-season all-in-ones is fundamentally not the same thing as long-term RAPM.

Two of them literally use RAPM. The third uses "basically RAPM". Please explain which step in the third you think took Steph from being well ahead of Lebron to being well behind (as putting in the box component leads to)?

Your proof Steph is the secretly underrated one is an all-in-one which is marginally biased against Lebron where both the all-in-one and the corresponding RAPM strongly favors Lebron. if you want to argue specifically, as he was getting whacked by lebron on both fronts, he was a bit more underrated? Cool.

But those are not the time-frames your Steph-arguments are actually based on. And those are not the time-frames over which "Lebron is more underrated than Steph by the box-score" is argued for.

And then there are the larger chunks which show Lebron absolutely smoking his RAPM and box-score consistently because of context like staggering and playing a position up during 4 years of his prime.

You’re completely missing the point, and I’m not sure how much more clearly I can spell it out for you. The impact components of all-in-ones are not long-term RAPM. They aren’t long-term anything. They’re short-term RAPM or RAPM-adjacent measures, which are extremely noisy and inaccurate (because the data for each player, and each player that is being adjusted for for that player is in small samples). These measures will certainly correlate with long-term RAPM, because they’re using raw data that will go into long-term RAPM, but they are not the same thing at all. If they were, then there’d be no need for a box component in these measures (and it certainly wouldn’t be the case that the box component actually gets weighed more and is more accurate as to long-term RAPM than the impact component is). You’re pointing to something that is absolutely not meant to be used on its own, and acting like it is something that it clearly isn’t. I find it difficult to believe you aren’t aware of that, given the fact that I’ve already spelled it out. But I guess it’s like the Upton Sinclair quote: “It is difficult to get a man to understand something when his salary depends upon his not understanding it.”

Again, the slope of the line of best fit here is not 1. LeBron’s BPM is higher relative to RAPM than it is for the average player. And, as demonstrated in this thread, it’s higher relative to RAPM than it is for most major stars too. The idea that it “still underrates him” is based on an incorrect notion that BPM and RAPM are scaled exactly the same.

Higher relative to the average, significantly lower relative to the median as the coloring denotes. If you are using the former, then Steph's 12-16 is also overrated by the stat and they're close in overratedness for the stretch actually relevant to what you're pushing.

Yep, using the line of best fit, Steph is overrated by BPM too, as is virtually every superstar. It’s just that the vast majority of them are not as overrated by BPM as LeBron James is. Of course, since virtually every discussion here is about comparing star players, the most important question is whether someone is overrated by BPM *relative to other stars.* For LeBron, the answer overall is yes. For Steph, the answer is no.

I also note that you are misunderstanding what the coloring means. The coloring is just indicating whether the RAPM or BPM is a higher number. We know LeBron’s RAPM number is higher than his BPM in two of the three time periods. The coloring tells us nothing more than that.

Dwight was a beast in his Orlando prime despite glaring weaknesses. +7.3 RAPM (5th) in the 2007-2011 period. +3.8 ORAPM; +3.5 DRAPM. I wonder if BPM undersells his off-ball paint scoring game (rolls, cuts, lobs, putbacks, deep post seals) much like the off-ball shooters you've suggested. It makes me wonder how McHale's RAPM vs. BPM would look given his quick-hitting off-ball pivot play. Many of the best post players operate/operated like catch-and-shoot finishers except in the paint.

I believe one of the primary stated goals of BPM 2.0 was to reign in the largest outliers while keeping a good overall fit.

Using just the basic box it's almost impossible to 'properly' measure a guy like Korver and not shoot Steph through the roof, so blame Steph for off-ball shooters generally not getting their due. Guess there weren't enough off-ball shooters to throw off the overall fit.

OhayoKD wrote:

lessthanjake wrote:

OhayoKD wrote:I mean...we have an even simpler way of looking at who is over/underrating players in the metrics the "assertions" are made about with filtering between pure impact components and the overall score and/or the box-only and the overall score.


By what bar is BPM the most prominent? Surely you're not going to ignore what we've already documented looking at other all-in-ones to make wierd general conclusions...


I guess if we ignore LEBRON, RAPTOR, and MAMBA all showing the exact opposite relationship... with Steph Curry's rapm being lower and Lebron's RAPM being higher. I don't really know why you would try to use BPM as a representation for all box-stats in the first place here...

You’re not making a particularly valid point here, except with regards to MAMBA (which I’ll discuss further below, in what is, I think, the most important part of my response here).

As an initial matter, I’ll note that we can’t actually test your statements about LEBRON and RAPTOR, because RAPTOR doesn’t exist anymore and LEBRON doesn’t actually parse out its components.

These statements were made 2 years ago with screenshots to go with them in the top 100 with you acknowledging those same RAPTOR on/off ratings. Are you trying to play dumb here?

Regardless before going offline 538 were nice enough to provide us the breakdowns here:
https://github.com/fivethirtyeight/data/blob/master/nba-raptor/modern_RAPTOR_by_team.csv

You know, it’s funny that you provided that link to the RAPTOR breakdowns. It’s very helpful, and I was able to run down the RAPTOR components for LeBron and Steph for all the seasons included in that data (which is 2014-2022, with me not including 2020 for Steph because he barely played).

Specifically, I looked at the Raptor Box Total and compared to the RAPTOR On-Off Total for both players. I found that, on average, Steph’s RAPTOR On-Off Total was +1.90 higher than his RAPTOR Box Total, while LeBron’s RAPTOR On-Off Total was +1.87 higher than his RAPTOR on-off.

In other words, Steph’s RAPTOR on-off component averaged outpacing his RAPTOR box component by a little more than LeBron’s did (though the numbers are very similar). It takes a long time to compile these numbers, so I’ve not done this for other players, but this is certainly contrary to your arguments, and is consistent with the fact that Steph’s long-term RAPM outpaces his BPM by more than LeBron’s did on average.

So, even if we took seriously this obviously flawed/dumb “comparing yearly all-in-one component values to determine whether a player’s box outpaces his long-term impact” thing, it would just lead to the opposite conclusion than you’re arguing. It should be quite obvious at this point that you really just shouldn’t be making this argument that LeBron’s impact categorically outpaces his box data relative to other star players.

As a sidenote, I will also point out that, leaving aside the box component, Steph’s RAPTOR on-off averages being far higher than LeBron’s, which adds another data point in favor of prime Steph being more impactful than LeBron in the same years (albeit a data point that is merely a component of a metric, which obviously isn’t particularly useful).

lessthanjake wrote:Yep, using the line of best fit, Steph is overrated by BPM too, as is virtually every superstar. It’s just that the vast majority of them are not as overrated by BPM as LeBron James is. Of course, since virtually every discussion here is about comparing star players, the most important question is whether someone is overrated by BPM *relative to other stars.* For LeBron, the answer overall is yes. For Steph, the answer is no.

No, not really:

Using DOC's RAPM VORP vs BPM VORP with Lebron as a baseline we can see that Steph is basically in the same boat in terms of BPM bias for his career.

lessthanjake wrote:

OhayoKD wrote:

lessthanjake wrote:
You’re not making a particularly valid point here, except with regards to MAMBA (which I’ll discuss further below, in what is, I think, the most important part of my response here).

As an initial matter, I’ll note that we can’t actually test your statements about LEBRON and RAPTOR, because RAPTOR doesn’t exist anymore and LEBRON doesn’t actually parse out its components.

These statements were made 2 years ago with screenshots to go with them in the top 100 with you acknowledging those same RAPTOR on/off ratings. Are you trying to play dumb here?

Regardless before going offline 538 were nice enough to provide us the breakdowns here:
https://github.com/fivethirtyeight/data/blob/master/nba-raptor/modern_RAPTOR_by_team.csv

You know, it’s funny that you provided that link to the RAPTOR breakdowns. It’s very helpful, and I was able to run down the RAPTOR components for LeBron and Steph for all the seasons included in that data (which is 2014-2022, with me not including 2020 for Steph because he barely played).

Yeah, I'm sure it was that and not Steph posting a -4 in RAPTOR on/off along with a 10-point box underperformance.

For the same reason now Lebron's "prime" apparently extends to 2022 while we can't count anything for Steph before 2013.

What is obfuscated here is during Steph's best prime stretch, Lebron outperforms his box-component significantly more (as in literally doubling his box-component in 2 of 4 seasons) to the point his on/off is probably higher (allowing for the possibility weighing playoff possessions equally instead of double, bridges the gap)

That leaves us with

-> 2018, where Steph plays half the season while Lebron plays every single game for a team that gives up after 4 games without him in 2019 because they are outscored by 15-points a game when they try to win.
-> 2019, where the Lakers season gets completely derailed with Lebron getting injured and then he comes back on a minutes restriction as a shell of himself.
-> 2020, where Steph looks incapable of carrying anything and the Warriors accordingly punt the season while Lebron, again, doubles his box-component, has great game-level impact, and leads a good team without his best teammate.
-> 2021, where Lebron is leading the best team in the league with his Davis playing limited minutes before solomon hill ends his prime.

And given the .03 gap, I'm guessing even including all of that still wasn't enough to produce "the box actually favors Lebron" without throwing in
"prime" season in 2022.

OhayoKD wrote:

lessthanjake wrote:

OhayoKD wrote:These statements were made 2 years ago with screenshots to go with them in the top 100 with you acknowledging those same RAPTOR on/off ratings. Are you trying to play dumb here?

Regardless before going offline 538 were nice enough to provide us the breakdowns here:
https://github.com/fivethirtyeight/data/blob/master/nba-raptor/modern_RAPTOR_by_team.csv

You know, it’s funny that you provided that link to the RAPTOR breakdowns. It’s very helpful, and I was able to run down the RAPTOR components for LeBron and Steph for all the seasons included in that data (which is 2014-2022, with me not including 2020 for Steph because he barely played).

Yeah, I'm sure it was that and not Steph posting a -4 in RAPTOR on/off along with a 10-point box underperformance.

For the same reason now Lebron's "prime" apparently extends to 2022 while we can't count anything for Steph before 2013.

What is obfuscated here is during Steph's best prime stretch, Lebron outperforms his box-component significantly more (as in literally doubling his box-component in 2 of 4 seasons) to the point his on/off is probably higher (allowing for the possibility weighing playoff possessions equally instead of double, bridges the gap)

Spoiler:

Lebron
2014 RS -> Box Lebron +4.4, Impact Lebron +4.7
2014 playoffs -> Box Lebron +6.8, Impact Lebron +9.4
2015 RS -> Box Lebron +4, Impact Lebron + 8.2
2015 playoffs -> Box Lebron +4.4, Impact Lebron +4.6
2016 RS -> Box Lebron, +6, Impact Lebron +12
2016 PO -> Box Lebron +9, Impact Lebron +14
2017 RS -> Box Lebron +5, Impact Lebron +11
2017 PO -> Box Lebron +9, Impact Lebron +21

Steph
2014 RS -> Box Lebron +7.4, Impact Lebron, +8.7
2014 PO -> Box Lebron +5.6, Impact Lebron, +27.2
2015 RS -> Box Lebron +10, Impact Lebron +13
2015 PO -> Box Lebron +8.2, Impact Lebron +9
2016 RS -> Box Lebron +13.2, Impact Lebron +13.4
2016 PO -> Box Lebron, +7.32, Impact Lebron +0.9
2017 RS ->Box Lebron, +7.2, Impact Lebron +12
 2017 PO -> Box Lebron, +11.8, Impact Lebron +13

That leaves us with

-> 2018, where Steph plays half the season while Lebron plays every single game for a team that gives up after 4 games without him in 2019 because they are outscored by 15-points a game when they try to win.
-> 2019, where the Lakers season gets completely derailed with Lebron getting injured and then he comes back on a minutes restriction as a shell of himself.
-> 2020, where Steph looks incapable of carrying anything and the Warriors accordingly punt the season while Lebron, again, doubles his box-component, has great game-level impact, and leads a good team without his best teammate.
-> 2021, where Lebron is leading the best team in the league with his Davis playing limited minutes before solomon hill ends his prime.

And given the .03 gap, I'm guessing even including all of that still wasn't enough to produce "the box actually favors Lebron" without throwing in
"prime" season in 2022.

So your argument is just to take a noisy measure and transparently cherry pick years that happen to support your argument, along with ham-fisted rationalizations for ignoring everything else? Classic. This is just not a serious argument.

I think it’s worth recapping the full scope of the utter absurdity of your argument here:

As an initial matter, you are looking at a measure that has multiple inputs that work together to produce an output that is meant to be an accurate approximation of impact. You have decided that one of those inputs should actually be regarded not as an input, but rather as the thing the overall metric is trying to produce as its output. On its face, that is silly. It’s made even worse by the fact that the description of the metric specifically says that the other component is better at accurately approximating that output and therefore is weighed more in the metric. So we have an obviously stupid premise. But then it gets worse. Because, even accepting that stupid premise, the measure does not actually support your argument. So, even applying your stupid premise, you are forced to cherry pick what years to look at in order to try to get to your preferred conclusion. At this point we’re definitely in laughable territory. But, not to be content with that, to put the cherry on top you appear to be arguing that we should seriously use Steph’s RAPTOR numbers in a season in which he played in 5 games. This is one of the worst and most shameless arguments I’ve ever seen on these forums.

OhayoKD wrote:

lessthanjake wrote:Yep, using the line of best fit, Steph is overrated by BPM too, as is virtually every superstar. It’s just that the vast majority of them are not as overrated by BPM as LeBron James is. Of course, since virtually every discussion here is about comparing star players, the most important question is whether someone is overrated by BPM *relative to other stars.* For LeBron, the answer overall is yes. For Steph, the answer is no.

No, not really:

Using DOC's RAPM VORP vs BPM VORP with Lebron as a baseline we can see that Steph is basically in the same boat in terms of BPM bias for his career.

Umm, you realize that this chart is quite supportive of the point in this thread (at least with regards to LeBron—would need to dive further into it to see the extent to which it agrees or disagrees as to other players) and contrary to your argument that LeBron’s impact outpaces his box numbers, right? That uses LeBron as a baseline, so every player with a ratio above 1.0 is underestimated by BPM compared to LeBron. And, lo and behold, the vast majority of the players in that chart (including Steph) have a ratio above 1.0.

More generally, do you have a link to that data? And can you clarify for me what RAPM it is using (I’m not familiar with “DOC’s RAPM VORP”)? I am actually interested in looking at this data further, since it is aiming at measuring something quite similar to what I compiled in this thread.

There are a few major issues with comparing RAPM to BPM:

1. RAPM and BPM have different distributions. Both may nominally have a league average player at 0, and map replacement players similarly (~ -3 for RAPM, ~ -2 for BPM), but the number of players with very big seasons (say +5) is much different.
2. Because of the way BPM is designed, it's going to weight offensive possessions (i.e. possessions with offensive outcomes) disproportionately. So comparing 1:1 is problematic, because a miss because of "offensive data missing from box" vs "defensive data missing from box" looks a lot different. In RAPM, you will have players who separate themselves based on one-directional output (the O/D assignment for RAPM is actually nearly identical from last time to the total I ran it...one has +1/-1, the other is just +1 for everyone).
3. There is no such thing as multi-season BPM...it's just a weighted average of multiple years of data. RAPM is a regression with a larger dataset. So multiple years makes RAPM MORE informative, and makes BPM LESS informative.
4. BPM includes hard-coded player roles, which inform coefficients. Most flavors of RAPM do NOT include such arbitrary assignments, unless you are using a dataset as a prior.

The value of BPM on its own is up to the end-user. A better comparison would be calculating the difference between "Pure RAPM" and "Box-Score Informed RAPM" (either RPM or xRAPM -- though J.E. has blurred the lines on these).

ceiling raiser wrote:There are a few major issues with comparing RAPM to BPM:

1. RAPM and BPM have different distributions. Both may nominally have a league average player at 0, and map replacement players similarly (~ -3 for RAPM, ~ -2 for BPM), but the number of players with very big seasons (say +5) is much different.
2. Because of the way BPM is designed, it's going to weight offensive possessions (i.e. possessions with offensive outcomes) disproportionately. So comparing 1:1 is problematic, because a miss because of "offensive data missing from box" vs "defensive data missing from box" looks a lot different. In RAPM, you will have players who separate themselves based on one-directional output (the O/D assignment for RAPM is actually nearly identical from last time to the total I ran it...one has +1/-1, the other is just +1 for everyone).
3. There is no such thing as multi-season BPM...it's just a weighted average of multiple years of data. RAPM is a regression with a larger dataset. So multiple years makes RAPM MORE informative, and makes BPM LESS informative.
4. BPM includes hard-coded player roles, which inform coefficients. Most flavors of RAPM do NOT include such arbitrary assignments, unless you are using a dataset as a prior.

The value of BPM on its own is up to the end-user. A better comparison would be calculating the difference between "Pure RAPM" and "Box-Score Informed RAPM" (either RPM or xRAPM -- though J.E. has blurred the lines on these).

These are all fair points. But I don’t think any of this indicates that looking at who does better or worse than other players in BPM relative to RAPM isn’t informative, at least with regards to the specific questions of (1) whether claims people make on these forums about impact-correlated box metrics like BPM underrating certain players compared to impact are actually directionally right; and (2) whether we think certain player archetypes are underrated or overrated by BPM, such that we might look at BPM for pre-1997 players and bump up or down our mental approximation of what we think that tends to suggest about their impact. Regarding #1, this directly tests that premise, though I grant that scaling being different and BPM not being inherently multi-season makes the analysis not completely precise (certainly I agree that the specific numerical difference between a player’s RAPM and BPM isn’t actually indicating BPM is overrating or underrating impact per 100 possessions by that exact amount). I don’t see good reason why it wouldn’t be generally directionally right about what players are underrated or overrated by BPM compared to RAPM, even if we might not take the specific numerical differences in a player’s RAPM and BPM numbers as being particularly meaningful. Regarding #2, I don’t think there’s really any other way to try to determine what players might be overrated or underrated by pre-1997 BPM than by trying to look at what types of players are overrated or underrated compared to RAPM in post-1997 BPM.

As for comparing “pure RAPM” with “box-score informed RAPM,” that’s another interesting avenue to look at a similar thing. I’d imagine that if the RAPM was being informed by BPM then it’d end up to the benefit of a very similar set of players that seem to benefit from BPM in this analysis, though I take your point about weighted-averages and whatnot, so it’s not guaranteed to just correlate exactly in terms of who it helps the most/least.

One problem with comparing “pure RAPM” and “box-score informed RAPM” is that there’s a whole bunch of other relevant methodological decisions that go into RAPM. For instance, let’s say Player A does better in “pure RAPM” than “box-score informed RAPM” but the “pure RAPM” also has a rubberband adjustment and the “box-score informed RAPM” doesn’t. We wouldn’t really be able to tell which change is causing the difference. Ideally, the best way to test it would be to have “pure RAPM” and “box-score informed RAPM” where the only methodological difference is the existence of a box prior, but I don’t think we actually have any natural experiment like that. If we do, then I’d certainly be interested in seeing it, though!