Nitro1118 wrote:Doctor MJ, if you would find time to answer a couple of questions about RAPM, it would be greatly appreciated...
1) You seem to be a huge believer in weighing RAPM data when it comes to player analysis. However, since it essentially gauges a player's impact to your average team, isn't it very problematic that it only strips data from playing on a 15 man roster for any given season, when there are thousands of different player combinations in the entire league? Sure, it can make predictions, but how reliable can such a prediction be when it uses probably less than 1% of data from the entire theoretical pie? And that is not accounting coaching variability, and even stupid things from how a player reacts to a certain city, how good the training staff is, etc...
2) To what degree of importance do you place impact stats? For instance, look at a player like 2013-current LeBron. In those seasons, he's had a tendency of coasting on both sides of the ball until either his team falls into a hole, or coasting an entire game if his entire team is involved in a blowout situation. So, this can shave a few points off his impact stats compared to other stars. So, let's say he and Chris Paul are a +5 for the season, but Paul plays at consistent high effort while LeBron alternates between coasting and high...just looking at the RAPM, they are equals. However, LeBron's peak impact is higher and he has a tendency to "flip that switch" when needed, so do you factor in that contrxt when reviewing these stats. In that example, LeBron should clearly be the victor, since his peak impact is higher.
On a broader note, the point differential thing is why I feel those Miami teams get underrated in all-time arguments. Their coasting mentality often killed their point differential, but when they needed to flip the switch, they played much better than their+/- differential indicates.
3) At what +/- differential between 2 players do you look at as a big enough gap does your opinion sway in comparing Player A to Player B? For instance, let's say Player A appears to clearly have a lead in major box score stats and a good chunk of advanced stats, and the eye test also tells you he is likely the superior player...but his RAPM data is -2.5 worse than Player B. Is that enough of a gap to sway your opinion. Keep in mind, a middle of the road playoff team like Toronto has a +4 point differential, so 2-3pts doesn't necessarily dictate a W or L.
Basically, I want to be more of a believer in +/- stats, but I just have huge issues in stats that try to account for variables that are simply impossible to do so with the concrete data we currently have. I look at it as just another tool in the toolbox, but right now don't even really value it any more than box score stats, PER, etc... Because at least with those stats they are not predictive in nature, and I can simply use the eye test/situational stats if possible to place those stats within context. I have much greater issue in doing so with RAPM. However, with a guy as smart as you putting so much faith in RAPM, I really want to "get it."
Thanks in advance!
Hi Nitro, always happy to help:
1) Sounds like what you're bothered by is what some of us call the goodness problem, or the distinction between value & goodness. The reality is that +/- stats only ever tell us about what a guy can do in the context he's in, and that's far from the only context in the basketball world. They thus shed much more light on value rather than goodness.
What I tend to point out to people is that the same is true for all stats. A player's ability to score in one context will be different from another context, and hence the question of how truly good a player is in scoring is a deeper problem than that. This isn't to say I don't use box score stats of course, but rather that when you really think about it, +/- doesn't have new problems that box score stats don't have. It has basically the same issues, but has some worse than the box score and some better. In the end then, it just makes the most sense to use everything you've got to help you understand what's going on.
2) How important are impact stats to me? It's a great question, but not one that can be answered as well as I'd like. People tend to see me as someone who advocates that impact stats are the holy grail, and then call me a hypocrite when I respond to criticisms of an impact stat by talking about context beyond the stat, when the reality is that all that stuff is going on in my head whenever I'm doing my actual analysis - and that's my priority: RealGM and other places are fun, motivational, and educational, but it all means very little if things don't make sense to me when I'm looking at what's going on by myself. I don't know if that makes sense, but my personality is that when I say things don't "feel right", I don't just mean it as a synonym for not understanding something, I'm saying they actually make me uncomfortable until I can make sense of them. Perhaps everyone feels like that, but I tend to keep looking at a thing until I no longer feel intellectual vertigo as the sight.
Okay so that probably sounds pretentious as hell. Let me try again: I made this spreadsheet, because being able to see all this data to me tells me more about the value a guy actually contributed over time than any other spreadsheet I could think to make. In this sense, impact stats are the paramount stat in my arsenal. They remain however a first-pass thing. If someone asks me what I think about who had a better career between two players of the era, I'll absolutely look at my spreadsheet to compare. But I also already know who those players are as players if they spent any significant amount of time in the league so I'm reviewing year by year their history as I'm reviewing year by year the +/- data. To the extent I cannot really remember their history, then I'll pop over to b-r and refresh my memory, but that typically only happens when we really get into the nitty gritty.
Last note on this: If it's not clear, part of what I'm talking bout when talking about a player's history, is knowing what they were doing on the court, how they did it, how often they did it, and who they did it with. Nick Collison putting up some huge RAPM numbers absolutely made me think highly of him, and made me believe he was underrated by most. It also made me wonder if maybe he should have had a bigger role on the Thunder. But it never made me think "Collison is the true star of the team." A player like that can have the advantage of being played only when he supplies a lineup advantage, and when he's really working to that effect, he's often something of a stealth impactor succeeding in part because the opposing team doesn't realize how much he's burning them. This is a farcry from being able to provide big impact for your team for as many minutes as possible despite the opponent doing all they can to stop it.
And so in this sense, the box score trumps the +/-. While it's quite possible for a Collison to have more impact than some in a starring role who aren't really working cohesively with their team, he's not seriously in the same ballpark as players with heavy primacy putting +/- numbers that appear to be in the same ballpark as his.
Re: flipping the switch. That's a concern definitely, but I think you planted the seeds to the solution: You have to have a sense already for when a player might be in something less than a killer mode in order to account for it, but that's not so hard. Not saying I catch it every time, just saying, if we're seriously having a discussion about a guy's +/- history, we can figure such things out typically.
With that said, as with all other applications of "flipping the switch", there's a serious danger of giving too much power to the idea...and I really think I actually did that when analyzing LeBron's Heat. When they were on their huge winning streak in '12-13, I started having discussion with people about whether this was truly an all-time great team despite the fact the stats in general didn't say so. In the end as sports fans, what's tough is that we'll never truly know. I can point to the fact that those Heatles really never looked ultra-dominant in any playoffs to explain why in retrospect I don't really think the team was vastly superior to it's stats, and why I now see them as not really any kind of GOAT candidate, but then others can talk about injuries etc too.
To end with this, I'd just point out that I assumed I'd see a pretty major "flipping the switch" issue with Shaq during his lazy years, because to me he is THE flip-switch guy in my basketball watching life. Turns out, not so much. His piss poor attitude undoubtedly had a major effect on morale, but on the court, the dude had major impact basically always, and the same frankly has been true of LeBron.
3) How big of a +/- gap is big enough to sway the comparison? Again a great question, but I'd be lying if I had a specific number in my head. It's just too fluid for that. Let me go through Real Plus Minus from last year though just to shed some light. Of course I was paying attention all throughout the season, so it's not like I just looked at these numbers at the end and went from there - my qualitative sense for these things comes first.
So, the RPM leaderboard last year as sorted by the "WINS" column that factors in how much guys actually played:
1. Harden
2. Curry
3. LeBron
4. Paul
5. Davis
6. Green
7. Westbrook
8. Kawhi
9. Middleton
10. DeAndre
My top 5 and 5 honorable mentions in alpha order determined after the Finals:
1. Curry
2. LeBron
3. Harden
4. Paul
5. Davis
HM: Marc Gasol
HM: Green
HM: Kawhi
HM: Wall
HM: Westbrook
So you can see a big similarity indicating the importance of the stat. Know that "+/- stats", and "Real Plus Minus" definitely weren't synonyms there. I looked at other +/- stats too.
Focusing on differences:
1. Curry beat Harden pretty easily in my book. Partly it was because he played less than Harden not because he couldn't play more, but because his team didn't need him too. The much bigger deal though to me was that Curry was helping a much better team, and I felt they style he was playing scaled far better with a great team than Harden's style did. I'm actually a big Harden fan, and I think he could re-pattern his game to be better suited at leading a contender, but that change is far from trivial, and even with its successful transition, I don't think it'd work as well as Curry's set of strengths.
2. LeBron beat Harden - not easily, but also not with any kind of great soul searching. With a guy like LeBron who has already proven he can have insane impact, the question is less about his specific numbers, and more about him giving his team what was needed. Taken over the entire year, Harden had more impact in the sense that he "lifted" his team more, but LeBron did what he needed to, and when he needed to do more than Harden, I felt like he did it.
3. Incidentally, Harden vs Paul was a very tough one for me and one I still wonder if I was too hard on Paul about. I don't like admitting that a little bit of luck may have swayed me, but I think I'd have had Paul at #3 had the Clippers won the Rocket series. I really have more faith in Paul playing his style, than Harden playing the style he's played in Houston, but when Harden contributes more lift through the year, and his team wins out in the playoffs, to me that's when I need to start saying "Yeah, most times, Paul's team goes farther...but they failed to do so this time, and there are consequences for this."
4. As mentioned, I prefer not to rank 6-10, so while Westbrook looks like he dropped based on where I put him, he didn't necessarily...I was however very critical of him last year. To me he played a style that really doesn't lead to a good basketball team, and if you look at RAPM that's not shaped by box score like RPM does, I think you see that. (Apologies, I don't remember what the link was for '14-15 RAPM data, sufficed to say, it was much more critical of him in general and defense specifically.)
With that said, being able to lead a bad team to be decent is still a major accomplishment by most standards, and I didn't feel like their were 10 guys more impressive than Westbrook last year.
5. Middleton missed my top 10, but I have a ton of respect for him. He's of this new breed with Kawhi and Green that in the past would have been mistaken for being far more limited than they actually are. But still, playing that role while helping lead elite teams is considerably more impressive than doing it on a team that isn't really accomplishing that much. Yes the Bucks had a great defense last year, and Middleton was a big part of that, but some of that was stealth success. Teams weren't going into Milwaukee desperately trying to figure out how they could win. There were many ways to skin that particular cat, and hence while Middleton was impressive, he wasn't quite as impressive as the +/- stats would indicate.
6. DeAndre even more so than Westbrook is someone I was skeptical of for box score issue reasons. Defensive rebounds & blocks gained without smart play can actually be a really bad sign. Real Plus Minus factors in those stats directly, and thus DeAndre looks pretty solid on defense, but when you actually look at regression data specifically on the Four Factors of defense, you see it's a problem.
7. As for Gasol & Wall, I tend to find it pretty hard to single guys out toward the bottom of the Top 10, so I could have gone with other guys here, but I think those guys are quite good.
