OhayoKD wrote:Just want to note I appreciate your civility here. That said...Chanel Bomber wrote:OhayoKD wrote:It’s a shame if RAPTOR from the 1990s doesn’t incorporate +/- data. It’d kind of defeat the purpose.
I think RAPM’s an interesting metric which shows some relevant trends but the calculation doesn’t adjust for environment nearly enough (compared to RAPTOR for instance which does a much better job of extracting noise). I wouldn’t use RAPM at face value in this exercise is my point.
Unfortunately RAPM is a little --too-- inclusive to be as accurate writ large(needs very long stretches to stabalize). RAPTOR is also advantaged with player-tracking. However, all that said, when tested against metrics which make use of similar tech:
https://dunksandthrees.com/blog/metric-comparisonMetric accuracy was compared overall and in the context of changing rosters. EPM and RPM, which were the only metrics that used RAPM directly with a Bayesian prior, consistently performed the best among all metrics, with EPM taking the lead overall. RAPTOR was the clear third-place metric with the revamped BPM putting the pressure on in fourth place. New player metrics using the latest methodologies and data are better built for today’s game.
We find that RAPTOR is both less accurate and [/b]less stable[/b] than data which more directly inputs RAPM. We also find box-aggregates like PER still lose out to RAPM on both fronts(BPM 2.0 matches) despite their theoretical advantages being "stability" and "accuracy" over smaller time-frames. Even though RAPM is far more inclusive(which one hand means it is noisy, but on the otherhand means it avoids most systematic biases), it still mantains an advantage in terms of both accuracy and stability.
This does not include PIPM which is currently owned by the Wizards and to my knowledge, also outperforms RAPTOR with it's defensive scores being more closely tied to DRAPM(something something linear vs branching). Maybe that's why, but IIRC Ben says it's better at capturing defensive value even without on/off data.
All of this is to say, I do not know RAPTOR is the most "promising" of approaches. FWIW, I do not think it's well respected in analytical circles and I've always had some suspicion towards it because in the-lead-up to it being revealed, it's creator was saying that the "competition" so to speak did not respect two-way wings enough and that kawhi leonard was the best defender in the league(as of 2019).
Moving on...You mention cherry-picked stats but I don’t think there is yet an all-encompassing metric that incorporates finely-adjusted data that we can use for cross-generation comparisons. Although I need to learn more about WOWYR. But as a general point, we also need to think critically about the accuracy of those metrics by the way they are put together.
There is not. I linked my own appraisal of WOWYR in the post you replied to, but even if you like the adjustments, it's still working off tiny samples(for bill russell for example, all those adjustments are being applied to 2.2 games a season of off).
And this gets us to the uncomfortable reality of historic player assessment. Uncertainty is inevitable. That would still be true if we had RAPM or RAPM derivatives(more on that later), but if you want to compare the "value" of players historically, you just need to get comfortable with that.
We can mitigate this issue somewhat by
-> looking for replication(bonus if it's across contexts)
-> tracking what skills generally produce value(can look at data-ball as well) and then map that onto established skill-sets
-> using "production" to internally scale players and teammates
-> using "production" to map to similar players we have more data for(hakeem -> duncan)
-> applying knowledge and context
As an example of point 2, we can look at the greatest defenses in history, the defensive kings of data-ball, real-world signals(Marc Gasol joins raptors -> atg d, Gasol leaves Raptors -> average d), and see how things unfold when teammates join or leave(Kawhi Leave, Raptors stay an atg d) and reasonably conclude that paint-protection>>>>>perimiter man d(can also look to what happened to the clippers defense without strong rim-protection and two elite "two-way wings" the next few postseasons).
Similarly we can extrapolate that guards do not move-the-needle like wings do and wings do not move the needle like bigs do(giannis coasts bucks defense collapses to average, giannis goes all out, bucks return to atg in the playoffs). The notable exception here is probably CP3, someone who by many accounts, serves as a on-court coach who directs his teammates where to go. This is a pretty rare trait among players, also shared by Draymond Green. Another guy who is relatively undersized relative to the results(greatest defender of his generation?)
(PS: you get one guess which non-big is an outlier regarding those two traits and has a consistently sees their defenses fall off without them)
The point is we are not bound to just looking at what happens to a team when x player leaves. That said, I think it's probably good to start there, rather than to just make assumptions regarding the values of different contributions(which is what you implicitly do when you use PER as a heat-check). Reality is noisy, but it is not biased, and should not just be disregarded when inconvenient for our priors. And to be clear, this does not just apply to box-stuff, which gets us to...An over-reliance on impact metrics then also opens the door for questions like is Jokic or Curry the GOAT (over James and Jordan)? If you’re willing to have these discussions, then fair enough.
...Okay I have seen this alot and(feel free to check me if it does not apply to you), but I believe for Steph this claim is largely based on
-> Comparing the highest single-year scores on apm or apm derivatives(04 kg single highest score on JE'S RAPM SET! Best RS ever!)
-> Raw plus-minus and on/off?
I think people need to remember that RAPM(and it's derivatives) are artificial. They are also(like on/off) prone to issues with colinearity. While they make adjustments they are still susceptible to wonky rotations. Beyond that, they also curve down outliers. What you should be looking for in something like LEBRON, EPM or RAPM(ideally over extended samples) is how often a player hits at or near the top historically. It is there to establish a baseline. Not to establish how the best year of Draymond compares to the best year of Steph or CP3.
Per LEBRON, a "state of the art" apm derivative, here are the most valuable seasons(overall) since 2010:
Here are the most valuable seasons(per-possession) since 2010:
notice how best single-scoring years look around the same? That is not real. RAPM is not a substitute for real-world signals. The main benefit of something like WOWY(especially if you are looking at seasons without a player), is that you can truly see what happens to a team when a player leaves.
Adjustments and all, the premise of APM is ultimately still that “winning on the court is good, as is seeing your team become worse without you on the court”. RAPM can approximate that in most cases(given enough time), and it is advantaged in terms of stability(and probably even accuracy looking at hundreds of players), but it is not a replacement for the real thing.
RAPM is also a rate stat. KG may have scored the highest in 2008, but he averaged substantially less minutes than his peers(. Something to keep in mind for those who think Kobe was undeserving of his MVP(he was not).
All that considered, here's an example of a 1-year "impact" comparison I did 2 days ago:Spoiler:
Feel free to do your own adjustments and come to a different conclusion, but good empirical comps do not start and end with RAPM.
I would also note that while Jokic's 1-year stuff looks awesome(2023) it does not look that good when we extend the sample(shotchart 3-year rapm and cheema career rapm favors embid oddly enough, raw-stuff still looks great(lineup-ratings, extended wowy, lineups and wowy without Murray, ect.).
I think from a 1-year "impact only" perspective(only looking at the rs), the potential standouts are Kareem(72, 74, 77), Lebron(9/10), Walton(77), Wilt(67, era-tresholds come into play here), and Russell(pick a year honestly). Steph's 2016(and 2015-2017) looks awesome, but not like the "greatest in nba history" outlier people potray it as. Obviously the pre-data ball stuff is super-murky, but I see a strong possiblity there(can elaborate for those who are curious). Accounting for the postseason makes things more interesting(A bunch of Wilt years, For Kareem, 71 and 72 potentially jumps, 12 and 15-17 Lebron, ect.)
Thanks for the time and effort you put into this. I honestly won’t have time to immerse myself this week, but this is really interesting and I hope to find some time next weekend to absorb, reflect, and engage. Because this will take at least a couple hours to advance the discussion in a meaningful way. Cheers.
PS1: To be clear, I personally don’t use PER.
PS2: I agree that RAPTOR is flawed and always assumed that it was intended for the general public moreso than analytical circles. I just have limited access to these resources (different metrics) and have found RAPTOR to be quite remarkable if interpreted through the lens of players’ roles and not just at face value. It does a decent job, especially at challenging preconceptions based on narratives rather than data. But I’d love to extend my scope, of course.
