Narf wrote:Doctor MJ wrote:Narf wrote:DRAPM is one of the worst stats I've seen.
Sorry to be inflammatory, but you should know whenever I read "X is one of the worst stats", that to me makes me prone to interpret all that follows as "I don't understand how to use stats but I'm really sure there's something very wrong with this".
And when I see that, I think you're arrogant, self righteous, and trying to hide the fact that you can't make a statistically relevant argument to back up anything you say. That's fallacy of course, but I don't feel like arguing stats with someone who disregards anything about me without a civil discussion or anything resembling a benefit of the doubt to my intelligence. So I have no interest in responding after this.
Yeah I should have tried to find a better way to say it. I'm sorry.
The point is really just that it doesn't make sense to talk about ranking how "good" various stats are when they don't tell us the same story as each other. If you're inclined to throw out +/- stats because you see some data in there you think is unreasonable then you need to change your perspective on how to incorporate various stats into your analyses. Something can only be justified as useless to you if you've can a superior version of the same type of tool handy, which in the case of RAPM you basically can't. You can be cautious about using it, you can criticize others for putting too much faith in it, but if you don't use it then you're ignoring a set of data that there's no rational basis for ignoring.
Narf wrote:And if you didn't catch it, I clearly said the 538 analysis was "crap". I just disagree with the statements people made about steals not having much value in determining a players defense. I think "gambling for steals" is thrown out there way too much, and the correlation between steals and good defense is stronger than people think.
Anecdotally, I can show a group of players that are greatly mischaracterized by DRAPM by how they play with 4 other players. When a far lesser DRAPM player is replaced with them the defense gets better, meaning the only thing that's changed is a "better" defender replaced a "worse" defender and DRAPM had it backwards. You could do the same for DVORP, but I personally have found DVORP to have a stronger correlation between ACTUAL defense and it's rating. It's not about how the number is created, it's about whether or not that number does what it's supposed to do.
There's also very large fluctuations between players leaving one team and going to another with new players around them. Kevin Garnett was a better defender in Minnesota, but DRAPM says he's been a better defender since then. This is someone I watched his whole career, I don't need to prove that statistically to you. Not that KG wasn't a great defender in Boston, just that he was even better in Minnesota. But his teammates weren't. So because of his teammates, his DRAPM went up.
Well fundamentally with RAPM we're still just talking about a variation on player "lift" (meaning how is the player in question lifting his team compared to what they'd do without them), and most typically the potential bias here runs in the other direction. If you're on a team with bad players, you can make a night & day difference but joining players already capable of functioning well without you means that there's going to be redundancy and diminishing returns.
What the data tells us about Garnett's defense in the two cities is that Garnett actually had more lift when surrounded by better talent, and that's so rare as to be extremely noteworthy. If you think you see this a lot in RAPM, you're confused.
Speaking more on what happened with Garnett. To the extent you're saying he wasn't actually a more capable defensive player in Boston than he was in Minny, I quite agree. What we're talking about is him having greater defensive efficacy in Boston than in Minny. And that was based on him being allowed to focus more on defense in Boston than he did as the two-way star of Minnesota, as well as getting to play for Thibodeau who is probably the best defensive mind of the basketball world. It's good to note that context, but it's not a flaw in RAPM that it tells us "Wow, Garnett's really having huge impact on defense in Boston!".
Narf wrote:In essense, this is no different than +/-. But we don't pretend that +/- is without context. You know you have to look at who is replacing them to get a feel for what that defensive +/- is. Actual defense is measured by how many points the team gives up with that player on the floor. Individual contributions are a little harder to verify. But if the defense is terrible with a player on, and average with his replacement on, he's a bad defender any way you cut it. He's also a bad defender if his replacement is a bad defender and they have the same defensive points per possession allowed.
And clearly +/- is not accurate unless put into context.
Just jumping in here because it begs me to say how much of my ability to use +/- in all forms (raw, on/off, APM, RAPM, xRAPM, etc) is based on my ability to read the context. You're not wrong at all on this, it's just you went down a wrong turn when you chose to look at a particular stat in terms of "good" & "bad" instead of just using the noise to develop your intuition for the context further.
And I'm sorry if I continue to sound condescending, it's just that I've got quite a lot of experience in this and I've seen so many people talk like you're currently doing, and sometimes I don't give enough cushion in my words to keep them from feeling their pride being violently attacked. There's no way around it though, I think I do know better than you on this, and since you didn't come in here making your own content but rather criticizing something I'm using, I get impatient sometimes.
Narf wrote:You have to have some understanding of the overall defense, some understanding of the value of his replacement, and some understanding of the players they play most of their minutes with (for instance, Brewer plays most of his minutes with Kevin Martin, which significantly lowers his +/- and DRAPM
Jumping in again just to say: There's absolutely no reason to think that being on the floor with a bad defender makes your RAPM more likely to be worse. The entire reason why we have "adjusted" +/- is combat that issue.
Narf wrote:....making both of them fairly inaccurate). But that's easier to look up with +/-, with DRAPM people just assume it's right. Defensive matrixes don't just have outliers. They are terrible, and simply get it wrong far too often to be used as if it is an authoritative number. It's like counting someone's championships to claim they were a great player. As if Karl Malone wasn't greater then most of the PFs that received rings. Or if KG had spent his entire career in Minnesota and never won one.
I agree with you. I figured this is the source of your frustration, but I have to step in when you go from there and literally call a stat bad because others use it improperly, especially when you say that in response to a conversation started by me and I didn't use it like that. When you reach the point that you assume someone is using a stat poorly without any evidence they are, you've got an issue. When you don't simply assume but respond by starting with "RAPM is one of the worst stats I've seen." you're broadcasting to the world your issue.
And so when I respond to you by letting you know how that looks it's tough love man.
Narf wrote:Great defensive teams make great DRAPM, whether you're a great individual defender or not. The individual player is not greater on that team then he was on the last one, he just had a better team that year.
82games.com keeps track of 5 man units and actual defensive points per 100 possessions while those units are in. Knock yourself out comparing drapm with players and their replacements in 5 man rotations if you'd like. Then (gasp) look at those same numbers the last year and the year before. They won't correlate well.
And no, I didn't get my opinion from some article you found. I had it long before I read anyone else who agreed with it. It wasn't hard to look at DRAPM's ratings and see clear as day they were terrible based on the players I already knew. I think both DRAPM and DVORP are so flawed they are nearly worthless. But of the 2, DRAPM is more flawed. Feel free to disagree.
I would say the fundamental thing you need to consider here is that even though we talk about pluses and minuses, your basic mathematical terms, we know that all of this is much more complicated than that. It's still over simplistic, but start thinking about the synergy & redundancy of sets of players in terms of multipication, division, exponents, etc.
When Garnett went to Boston, you didn't simply add a defender of X1 quality to a bunch of guys of X2, X3, X4, etc quality, and the clearest sign of that is that no one expected the team to dominate the league with their defense. If you had told people that Garnett would get the box score stats he did in Boston in the summer of 2007, no one would have been impressed, and no one would have talked about the tremendous effect that would mean he was having on the Boston defense.
Hence, it was an absolute shock to the basketball world, and a clear sign that we need stats that help us embrace such nonlinear relationships in the sport. The +/- family of stats helps us do that, and the proof of that is in the fact that it is in those stats that you see some manner of individual explanation for the shocking emergent property that was Boston's defensive dominance.
Now, clearly you'd see one rebuttal to this being that +/- is simply echoing the team success. Okay, and what's the problem there? When you talk about year to year variance in +/- data that goes along with year to year variance in overall team defensive efficacy, obviously something important is changing, and if you've been around basketball enough you know that something may have nothing to do with an actual roster change. That +/- stats can actually see something and say something when problems happen simply makes it useful.
Another rebuttal: Well if it's just an effect of team success, then it doesn't say anything about actual causality. Of course it doesn't, and that's why you need context. +/- data is either a way to confirm hunches or a way to tell you need to look closer at something. It is not the entire analysis. I agree with you that sometimes people use it as such and that's a problem.
But what you also need to understand, particularly in a context like this, is that people have been at this quite a while, and when that's the case they use shorthand. A player's RAPM isn't ever alone enough for me to say something big about him, but 'round these parts analysis of, say, Garnett's defensive strengths and role as a defender in Minny & Boston has been talked about so much that it hardly seems necessarily to re-has all that every time I mention a stat of his where he looks good.
Also ftr, I was asking what stat you were using when you refer to DVORP. VORP is a baseball term that's been claimed by multiple basketball statisticians and none of them have really gotten that much traction because people aren't really sold that they do what they say (calculate "value over replacement player").
