Talk Me Down: How is Deng not MVP of the Bulls?

[alucryts]

I am not going to put too much more time into this beyond the video analysis in the coming weeks, but the problem I have with people using +/- and saying find me a number that proves it wrong is that no such numerical evidence exists besides plus minus that does a good job of separating out individual players defense. Out of all the stats out there, +/- is the best as nothing even comes close. That's the problem inherent in almost any argument revolving the stat; there is no arguing with the number with other numbers! No number can describe individual defenders as well as it can. When it is wrong, there is nothing numerically to prove it wrong. When it is right, there is nothing numerically to prove it right. Plus minus is the end all be all of numerical individual defense. I do not feel that it does a good job at all in describing this either. It's sort of a deal where I feel it is the best of the worst.

I feel like a lot of posters take it as the gospel and that it cannot be wrong because it is "results" and ignore the fact that it can be wrong. We can always go on to any advanced stat and pick out players who just are not ranked correctly at all. This shows (obviously) that the stat has some error as they all do, and that error can vary from very great to very little going from player to player. How do we know the error associated with any player? How do we know if Deng is +4.0 +/- 100 or +4.0 +/- 0.1?

When a stat that has no opposition or additional numerical context is used to say something that a great number of neutral observers disagree with, why is the burden on me to prove a stat wrong that is unasailable when game tape and defensive analysis say that the conclusion is ludicrous?

A side question I have is, how does any +/- stat accomodate for teammates or opponents mathematically speaking? Is a pseudo SRS type of analysis used?

[alucryts]

Mysticbb's contextual/analytic contributions outside of +/- have had to do with a combination of Deng's ability to play good iso D, his ability to funnel/steer his opponent towards the help better than usual, and his ability to make good defensive rotations with very few mistakes. Alucryts, you may not agree with mysticbb's points, but they HAVE been stated several times in this thread. It hasn't all been +/-.

Out of that list, his iso D is not helping his case as the Bulls never truly play 1 on 1 defense, his ability to steer and funnel as well as few mistakes is seen in both Brewer and Rose in a very comparable fashion. None of this tells me Deng is the DPOY defensive MVP; It says he is a great perimeter defender.

My interpretation is that Mystic, on the other hand, would argue that though Noah's ROLE is very important to the defensive set, that how well Noah FULFILLS that role is in question. Noah's role likely is more important than Deng's role, but if Deng does his job on defense enough BETTER than Noah does his job, Deng could still be the more impactful defender.

My problem is that if Noah reduces his ability by 10%, it trickles down to a greater reduction in the perimeter players ability. It's kind of like first option on offense vs third option; using raw stats to compare them doesn't exactly work. If Noah or any big doesn't do their job, Deng or any perimeter defender can't do their job. This relationship does not hold up in reverse.

As an experimental scientist by profession I would strenuously disagree with that assertion. Making conclusions based upon experimental observation is at the heart of the entire scientific method, and I'd dare say that arguing against that method of analysis being useful would by far be the more "Russell's teacup" stance to take in this debate.

As an engineer going into research, my life revolves around quantifying and using error much like I assume yours does. When I see numbers as they are presented as absolute values, without errors or standard deviations attached, the number means nothing. I spent a great deal of time developing an optical measurement system that could measure something at around 150 microns to within 45 microns of standard deviation……if I took 10 measurements, I got a decent average. That sucked. I spent a lot of time changing that 45 all the way down to 5 microns. The system still spits out the same average, but it is much more robust. Based on a paper mystic provided me, the error involved in predictions with this stat is enormous. Because of this, why does the average mean anything at all? We should be examining the range of numbers that involve the first standard deviation, and try to put context to find out where in that range the player falls. That imo would be the best approach to plus minus. If the argument is then that you can’t attach an error or standard deviation equivalent to it, then I really can’t take that stat very seriously at all. It’s why I almost never refer to PER or things like that in any serious comparisons. It is a fun stat to watch, but I don’t take it’s comparison abilities seriously.

I think that +/- is useful, but I do not think it is robust enough to say something like Deng is the DPOY. That is a wild statement backed by numbers that are very questionable.

[Doctor MJ]

It is very easy to remove Deng from the defense on offense. If he was truly the cause of the great defense, offensive teams would remove him from the play and go to work on our defense. You can easily remove one perimeter player's impact from a defense. Teams don't remove perimeter players from defense because they are not the cause of great defense; they only help it.

I would urge caution on that kind of statement, and here's why:

The impact of all players is really quite tiny compared to what we naturally think. Just a few points per game. So while an offense can drastically reduce the direct impact of really any defender (even the bigs), at what cost to the optimal running of their offense? As others mentioned, if the defense put Deng on LeBron, then Deng is going to be involved as long as LeBron is involved, and LeBron's offense is obviously too valuable to give up.

[Doctor MJ]

A side question I have is, how does any +/- stat accomodate for teammates or opponents mathematically speaking? Is a pseudo SRS type of analysis used?

It's possible I'm misunderstanding the question, but to what I think you're asking:

The idea behind adjusted +/- is to essentially turn correlation with the scoreboard into a massive, massive algebra equation: Many unknowns, many (more) equations. So every player is assumed to be a variable with a certain value, and when you add the players on the court together, the goal is to get as close to what actually happened to the scoreboard as possible.

So then, the source of confusion many have is that "adjusted +/-" doesn't actually take +/- and adjust it, and in that sense it's actually a pretty bad name. Rather, it just implicitly factors all teammates and opponents in all over the league, all at once.

EDIT: So, not sure how that makes you feel about the stat, but to me this is all VERY good things. One severe danger with any "advanced" stat is that it becomes chalked full of assumptions that then cloud analysis even if the assumptions were reasonable ones. APM is actually an incredibly streamlined process that doesn't really leave much room for "doing it wrong". It's not perfect, but it's biggest flaws are really right out in the open.

[Doctor MJ]

The biggest problem I have with plus minus arguments is that they are all versions of Russell's teapot.

Russell's teapot, sometimes called the celestial teapot or cosmic teapot, is an analogy first coined by the philosopher Bertrand Russell (1872–1970) to illustrate the idea that the philosophic burden of proof lies upon a person making scientifically unfalsifiable claims rather than shifting the burden of proof to others, specifically in the case of religion. Russell wrote that if he claimed that a teapot were orbiting the Sun somewhere in space between the Earth and Mars, it would be nonsensical for him to expect others not to doubt him on the grounds that they could not prove him wrong. Russell's teapot is still referred to in discussions concerning the existence of God and has drawn some criticism for comparing the unfalsifiability of a teapot to God.

You say that Deng has, for a random made up example, a +4.0 impact on defense ranking him like a DPOY. My argument is that this +4.0 is not only riddled with variance, but it is also in error with correlation and causation. Let's for a second assume 100% accuracy of +4.0 to completely remove variance from the equation. My argument is that +4.0 is not caused by Deng himself entirely. My argument is that a big enough portion of that +4.0 is caused by his team mates and his system to remove him not from being a good defender, but from being the singular MVP of the Bulls (or DPOY). There is no way to prove you wrong under your logic with my opinion because of the plus minus bunker you have put up. Since you are the one here making an extreme claim (Deng is the MVP of the Bulls and DPOY being akin to the teapot in space), the burden of proof is on you. You are not right because no one can provide proof greater than plus minus in your eyes. It is up to you to provide proof beyond plus minus to satisfy your claim. This is the same as saying the person claiming there is a teapot in space is right because no one can prove him wrong. You have insufficient proof.

I also have the burden of proof of showing that Noah is the Bulls best defender. I will attempt to do this with video over spring break for me.

Well, I think the thing is that sports analysis is chalked full of "religious" thinking along these lines. The people quoting +/- are at times too dogmatic, and too optimistic, but they've at least got something objective to point to. As such I really don't think it makes sense to single them out with this issue - however, what you speak is indeed an issue with the +/- along with pretty much every other group.

[alucryts]

It is very easy to remove Deng from the defense on offense. If he was truly the cause of the great defense, offensive teams would remove him from the play and go to work on our defense. You can easily remove one perimeter player's impact from a defense. Teams don't remove perimeter players from defense because they are not the cause of great defense; they only help it.

I would urge caution on that kind of statement, and here's why:

The impact of all players is really quite tiny compared to what we naturally think. Just a few points per game. So while an offense can drastically reduce the direct impact of really any defender (even the bigs), at what cost to the optimal running of their offense? As others mentioned, if the defense put Deng on LeBron, then Deng is going to be involved as long as LeBron is involved, and LeBron's offense is obviously too valuable to give up.

Yes, I amended this statement to say it is easy to remove his help defense and focus on the rest of the defense. Removing him entirely is quite impossible without large detriment. This argument was to say that because it is his help defense that is being pointed to as why the Bulls defense is so good, why don't offenses remove him and attack a much softer defense?

EDIT: So, not sure how that makes you feel about the stat, but to me this is all VERY good things. One severe danger with any "advanced" stat is that it becomes chalked full of assumptions that then cloud analysis even if the assumptions were reasonable ones. APM is actually an incredibly streamlined process that doesn't really leave much room for "doing it wrong". It's not perfect, but it's biggest flaws are really right out in the open.

So it is kind of an SRS-ish process. I see what it is a bit better with that then. I will always come back to the fact on this that no matter what you do to a number, its error sticks with it throughout. If the data being plugged in is in error, then the data coming out is in error. I would be interested to see what the difference in error is between before and after if such a thing is even possible to put into a number at all. It's the idea that a lot of these advanced stats can't produce any type of error associated with the number that comes out the back end without a ridiculous amount of effort that turns me off to a lot of advanced stats like +/-, PER, eff, etc.

Well, I think the thing is that sports analysis is chalked full of "religious" thinking along these lines. The people quoting +/- are at times too dogmatic, and too optimistic, but they've at least got something objective to point to. As such I really don't think it makes sense to single them out with this issue - however, what you speak is indeed an issue with the +/- along with pretty much every other group.

I agree. Like I said in a previous post, because it is the only numerical value to describe this, essentially any reasonable conclusion based in the numbers is unassailable by the numbers. This is where I brought up the Teapot analogy; just because the number is unassailable doesn't mean it is correct.

At this point I think that all I can do on this is agree to disagree with those on the advanced stats side of the argument. I feel the stats are flawed and riddled with error, and that my watching and understanding of defense paints the Deng is a DPOY candidate argument ludicrous. On the other hand, the advanced stat crowd feels that without another source of truly objective data, you cannot argue the results really.

[Doctor MJ]

If the data being plugged in is in error, then the data coming out is in error. I would be interested to see what the difference in error is between before and after if such a thing is even possible to put into a number at all. It's the idea that a lot of these advanced stats can't produce any type of error associated with the number that comes out the back end without a ridiculous amount of effort that turns me off to a lot of advanced stats like +/-, PER, eff, etc..

I'm not sure I'm really understanding here. Two things come to mind that I'll say:

-There's no doubt that reliability is an issue with +/- stats, and even though longer studies help this a good deal, I've never felt comfortable treating them as gospel. What they do provide though is something that's actually valid which is huge. It's funny when I see stat people who are really fixated on the box score stats to me because in the end, we can never really know what value went into to each act that resulted in the scorekeepers mark. Focusing simply on the scoreboard itself truly means that if we had enough data, we'd have something real.

This might seem weak given that we don't have perfect data, but I'm coming from a perspective where I just realized that the box score stats where rather ridiculously biased for and against certain types of players, and +/- offered something that truly has no such bias.

-When I realized how they got the "error" for +/- I was really disappointed, and then annoyed with myself for thinking there was more to it than that. Truly all it gives us is how imperfect we know the data has to be at a minimum, and there's really no reason to think that the actual error is right around that minimum threshold. At the most fundamental level, standard APM has to assume that a player is playing just as effectively all the time, which is obviously pretty far from the case. In that sense, your approach of not putting enough stock in APM for it to be your default position is clearly warranted.

And truthfully, I don't know if my default position is different for the short term. Long term I clearly put more stock in it. I'll also say this: The top tier superstar have so reliably become APM superstars that I don't feel comfortable anointing a new superstar until I see him prove it in the APM. This is not the same as assuming someone is NOT a true superstar if we don't see that data when he emerges, it's just a kind of purgatory. And when I then see a player truly be unable to ever have that happen for years (a la Melo & Amare), to me that becomes a big deal.

At this point I think that all I can do on this is agree to disagree with those on the advanced stats side of the argument. I feel the stats are flawed and riddled with error, and that my watching and understanding of defense paints the Deng is a DPOY candidate argument ludicrous. On the other hand, the advanced stat crowd feels that without another source of truly objective data, you cannot argue the results really.

Just keep in mind that we've got a spectrum of people here. mystic's data crunching abilities make me green with envy, and I respect him a ton on that, but tech ability aside, we clearly have different philosophies about how where to default our opinions.

I think you've made great arguments in this thread that I'm going to keep rattling around in my head, and I remain reluctant to anoint Deng as a true DPOY guy even as he is on my shortlist right now. I'll also say that while I may well end up ranking Deng ahead of Rose on an MVP list this year, were I drafting GM, I still would not seriously consider starting a franchise around Deng over Rose.

[Rerisen]

The thing is, it didn't happen in those 10 games without Rose. What do you expect? That the Bulls will just play against better than average teams all of the sudden without Rose? It makes no sense. The Bulls will play the majority of their games against worse than average teams, the same like they are doing with Rose. The Bulls will still play the great teams as often and will very likely lose the most of those games without Rose. That's what the results are telling us, that is what RAPM is telling us and that is what your personal belief is telling you too. So, where exactly is the problem? Do you expect the Bulls to play a much stronger schedule just because Rose isn't there?

Over a full season yes, the Bulls would play a tougher schedule than in those 10 games, and would be in many more close games than they were in that period.

10 games is not a reliable sample - even if it happens to more or less agree with the past season. How many other off base results for other players probably exist in those 10 games? But we are to accept the Rose and Deng values just because they produced similar results? Heck, even Deng's 1000 or so minutes without Rose for all of last year is not a great sample size, considering a full year for a player is upwards of 3000 minutes and +/- rarely want to rely on even a full year of minutes to make quality predictions.

I think if we looked at say how the Heat fared without Wade this year, it would also suggest rather strange things about the value of Wade, probably that the Heat were better without him, just eye-balling their 8-1 record, including wins against much better teams than the Bulls beat without Rose.

I appreciate that you went on to explain your actual rating system of players would involve a combination of box score metrics and on/off results. But it seems you didn't do this in your initial prediction in this thread, but rather just spit at me what the RAPM differences were between Rose/Deng as if that alone was setting the outcome you believe would happen without either, in a permanent fashion.

Just try to understand that not everyone agrees that the best answers for this new scenario lie in limited past numbers we already have, that were built in vastly different circumstances.

As I told you before, I do not believe absolute player value exists regardless of circumstance.

And that once situations are changing profoundly, player value also changes, and thus specific number predictions based on those (themselves supposed) previous values become suspect.

I hoped you would understand my position via the LeBron example, that shows how differently a player’s value can be perceived via a stat like RAPM though drastic circumstance changes, even though the likelihood is that the player didn’t change that much at all.

I believe our Rose-less and Deng-less Bulls represent a similar style upheaval that requires a little less rote thinking and methods of prediction.

[alucryts]

A good post with good points; mystic and I just aren't quite cut from the same cloth. What I would love to see, is for someone to make a super +/- much like Synergy has done with basic stats. What would happen is someone who understands the impact of all 5 players on the court on both ends would watch every play and split up the +2 among all of the players on the floor. So for a pick and roll dunk by Boozer for example:

Rose and his defender get +0.75/-0.75
Rip and his defender get +0.15/-0.15
Deng and his defender get +0.15/-0.15
Boozer and his defender get +0.75/-0.75
Noah and his defender get +0.2/-0.2

or some combination of constant credit that makes sense based on the play type. I think that this would make the plus minus stat a lot more robust.

Regardless, once I do the defensive videos, I'll post a link to the thread I make in here.

[Doctor MJ]

A good post with good points; mystic and I just aren't quite cut from the same cloth. What I would love to see, is for someone to make a super +/- much like Synergy has done with basic stats. What would happen is someone who understands the impact of all 5 players on the court on both ends would watch every play and split up the +2 among all of the players on the floor. So for a pick and roll dunk by Boozer for example:

Rose and his defender get +0.75/-0.75
Rip and his defender get +0.15/-0.15
Deng and his defender get +0.15/-0.15
Boozer and his defender get +0.75/-0.75
Noah and his defender get +0.2/-0.2

or some combination of constant credit that makes sense based on the play type. I think that this would make the plus minus stat a lot more robust.

Regardless, once I do the defensive videos, I'll post a link to the thread I make in here.

Well, I think what you're talking about isn't an improved +/-, it's just a form of game tracking. That's not meant as an insult, I just think these are two parallel methods that must both be used.

[Rerisen]

Surely you know the answer to this question. Omer is one of the best defensive centers in the NBA - not exactly a fair baseline to establish value over replacement.

And yes, Omer has an argument for being the better defensive player. But, he is such a liablity in other aspects, that he can't stay on the floor for extended minutes.

Yes. Omer Asik is an interesting player. And just as your said, extremely valuable in a limited context.

If Tom Thibodeau went purely by +/- unit strength, his starting lineup should be:

Rose/Brewer/Deng/Gibson/Asik

Because that was the Bulls best lineup with over 100 minutes last year, and also his best lineup in the playoffs last year over 20 minutes. The problem is Thibs used this lineup mostly when the Bulls had leads, to choke out games as they went down to the wire. It worked well because at the end of the game like this, it was fine to turn Rose loose offensively for short periods.

Rose did all the scoring, everyone else defended. But this was a situational lineup, that could not sustain over longer periods, because the scoring burden on Rose would be just so immense. He would tire out, and the pressure on him and that lineup when the team was actually losing or in competitive games, would turn against the Bulls. Tom Thibodeau must also believe this, or similarly, because the Bulls certainly have the same lineup information.

This is something that you can only truly understand by watching how the Bulls function as a team. Something that has been rather lacking at times in this thread. If one wants to call that 'personal belief' in what would happen if Thibs trotted out this lineup 30 minutes a game, be my quest, but is pretty darn far from cheerleading a player, and is based on knowledge and understanding of the team.

[mysticbb]

That's the problem inherent in almost any argument revolving the stat; there is no arguing with the number with other numbers!

The issue is that individual boxscore numbers can only explain 50% of the variance of the defense. That's it! And from those numbers, defensive rebounding contributes the most, the 2nd is own turnovers, after that steals, blocked shots and scoring efficiency. So, how much do you want to put into boxscore numbers here?

Anyway, it is the only real test for metrics we have. We can "train" our model with a dataset and then procede to predict out of sample data. Well, my boxscore metric is better than RAPM in terms of predicting the difference, but it is worse at predicting the lineup offense or defense performance. As expected, the seperation between offense and defensive impact is not good via boxscore metrics. Using defensive RAPM values improves that by a lot. So, yes, defensive RAPM is better than boxscore based numbers and can explain about 90% of the variance.

A side question I have is, how does any +/- stat accomodate for teammates or opponents mathematically speaking? Is a pseudo SRS type of analysis used?

I think the issue is that you don't trust those numbers, because you don't understand how it works and can't really make sense out of the errors you found. First of all, the RMSE was for the whole unit against a whole other unit, essentially that is the error for 10 players combined. Calculating the error via bootstrap we are getting an error of 0.5 to 1.5 per player for offense and defense combined. But those are not symmetrical values like you can assume a standard error is. It also is not unbiased, so, we really can't interpret that in a meaningful way. But overall we can say that the error is around the same as it would be for a multi-year APM study.
2nd of all:

Margin = HCA + a1P1 + a2P2 + a3P3 + a4P4 + a5P5 - a6P6 - a7P7 - a8P8 - a9P9 - a10P10

We have 10 different independent variables + the HCA as intercept to determine the Margin, which is the dependent variable in the regression. And we have around 18000 game snippets over a full season to determine the around 450 coefficients "aX". So, we have an overdetermined system.
In a standard regression that would cause multiple problems with overfitting and multicollinearities. The problems can be resolved via Ridge Regression (a baysian distrubtion is assumed and the values are just kind pulled to zero or some sort of other prior) and within half a season we can get more reliable numbers than we can get for a 2yr APM study. It is really that much better than normal APM.

And it should not be a surprise that the error for the individual players adds up to about 12 overall for the whole unit vs. another whole unit. The error is NOT 12 for each player, like it seems you assume here.

That is not a "pseudo SRS method".


And Deng's iso defense helps, because that is also determined by the offense and not just by the defense. Additional to that, if Rose and Brewer would be at the same level, it HAS to show up in the results at some point.
You are also putting a lot of faith into your own ability to judge a players defensive impact by just watching them and trying to understand defensive schemes. I hardly doubt that you are capable of that, especially when you come up with the idea that it is really Noah. Seriously, what Noah provides on defense can be provided by multiple bigs in the NBA, much more than you realize. Deng's abilities on defense are not that easily replacable. If you would be better at evaluating this, you would be able to make better prediction. But so far you can't explain why your argumentation is not seen in the results. So, why is that not seen? You want to talk sample size here, but somehow RAPM does not have a big issue with that, in fact we can see that for multiple instances when a player misses about 10 games that the real results (talking about scoring margin adjusted for strength of schedule) are rather close to the predictions. And so far, when the standard deviation for the a multi game sample reached the level of the standard deviation for a whole season, the sample size seemed to be sufficient in that way that it was close between prediction and real results. It works much better than you might imagine.

My problem is that if Noah reduces his ability by 10%, it trickles down to a greater reduction in the perimeter players ability. It's kind of like first option on offense vs third option; using raw stats to compare them doesn't exactly work. If Noah or any big doesn't do their job, Deng or any perimeter defender can't do their job. This relationship does not hold up in reverse.

But the minutes with Brewer and Korver are showing that Noah's value is much more effected by changing teammates than Deng. Maybe the small difference you see is just a much bigger issue for the overall team defense than you can imagine. That is like people trying to argue that scoring at 53 TS% is basically the same as scoring at 57 TS%. For a player with around 9 shot attempts per game that makes already the difference of 2 wins a season. And do you want to tell me that you can see that kind of difference in scoring efficiency just by watching a player? Heck, Rerisen was so sure that Watson just went hot as the starter while the reality shows that this was not the case. People are overrating their own abilities to "see things", especially when it comes to a level of small difference like we see that for basketball players. There is a typical bias that we overvalue situations which we think are more important, while in the reality the situations are most times equally important. We are just seeing a whole bunch of different cognitive biases here.

Based on a paper mystic provided me, the error involved in predictions with this stat is enormous. Because of this, why does the average mean anything at all?

Because that average determines win or loss at a very high level, we are talking about R^2 of 0.9+. What we have here is your inability to really judge the meaning of that error. You are confident, because you can calculate a standard error, but you don't show any ability beyond that in terms of statistical analysis. That you have to even asked how a regression works pretty much confirms that.
The rest of your idea becomes basically useless, because of your lack of knowlegde. But that isn't the worst thing here, because with ridge regression we are approaching a level which is not even understood by a guy like Dave Berri, who is supposed to be an economic professor who is working with statistical tools. So, maybe I'm expecting way too much here.

I think that +/- is useful, but I do not think it is robust enough to say something like Deng is the DPOY. That is a wild statement backed by numbers that are very questionable.

Again, the results are not as questionable as you think, especially when we are talking about the real results really played out in reallife.

[mysticbb]

Over a full season yes, the Bulls would play a tougher schedule than in those 10 games, and would be in many more close games than they were in that period.

Not quite sure how you came up with the idea, but I meant a 82 games schedule with Rose would contain the same opponents as a 82 games schedule without Rose. The Bulls are not playing different opponents and more often against the very best teams only because Rose wouldn't be there.

10 games is not a reliable sample

Do you have any evidence to confirm that in terms of overall team performance? Or is that just your assumption here, because a lot of people seem to say that?

Heck, even Deng's 1000 or so minutes without Rose for all of last year is not a great sample size, considering a full year for a player is upwards of 3000 minutes and +/- rarely want to rely on even a full year of minutes to make quality predictions.

RAPM does not need a full year in order to be as reliable as a 2yr APM study. And with prior informed RAPM the predictive power goes up rather quickly. You can basically make reliable predictions based on prior informed RAPM after 20 something games at a similar level as after 60 games.

I appreciate that you went on to explain your actual rating system of players would involve a combination of box score metrics and on/off results. But it seems you didn't do this in your initial prediction in this thread, but rather just spit at me what the RAPM differences were between Rose/Deng as if that alone was setting the outcome you believe would happen without either, in a permanent fashion.

Because RAPM is working better for Deng than any boxscore based metric. We know why that is, because the strength of Deng is not captured well by the boxscore. For other players it is the other way around, but for players with their biggest strength on the defensive end, RAPM gives better predictions. That is/was seen for a whole lot of players, like Jason Collins, Shane Battier, Bruce Bowen or Luol Deng. When a player is not putting much numbers into the boxscore while still providing great defense, the boxscore fails to recognize that. In other instances the boxscore is closer, also because it can seperate the stuff on offense better. I can explain 99% of the variance on offense with individual boxscore numbers, but only 50% of the defense. RAPM is basically at 90% for both.

Just try to understand that not everyone agrees that the best answers for this new scenario lie in limited past numbers we already have, that were built in vastly different circumstances.

As I told you before, I do not believe absolute player value exists regardless of circumstance.

And that once situations are changing profoundly, player value also changes, and thus specific number predictions based on those (themselves supposed) previous values become suspect.

But that is true for ALL players. What you are saying is that we can't seperate between the players anyway, thus we should not try it at all. That makes your own "eye test" as much useless as any statistical analysis. The only thing I say is that statistical analysis beats the "eye test" in terms of predictive power.

I believe our Rose-less and Deng-less Bulls represent a similar style upheaval that requires a little less rote thinking and methods of prediction.

Yes, you believe that, but for James it can be shown that his productivity increases without Wade on the court. We see in the data that James has difficulties getting the touches. We don't see such things for Rose and Deng. Sorry, but your example does not work here. We have a healthy Deng providing a similar impact back in 2007 without Rose. That is not about Rose so much as you think. What I think is that you try to rationalize your personal belief here, assuming that Deng is just effected by Rose while Rose is uneffected by Deng. You are applying your own standard differently to different players, just because you "feel" it is the right thing to do. Instead of understanding the results, your approach is just dismissing those results with questionable arguments while not being able to really back those arguments up.

The James-Wade stuff is interesting and something A LOT of people predicted. The overlapping in the skillset and the fact that both are better with the ball than without would effect their overall impact level. The funny thing is that the previous impact level is again seen, when one of the players is not available, with the distinct results that James without Wade is more sucessful than Wade without James. Something we really expected based purely on numbers. The predictions for Bosh are pretty much are equal with the seen results on the Heat.
We have a case of dimishing returns for overlapping skillsets. That this is less of an issue on the Bulls with Deng and Rose should be clear. So, your example of Wade/James does not apply here.

If one wants to call that 'personal belief' in what would happen if Thibs trotted out this lineup 30 minutes a game, be my quest, but is pretty darn far from cheerleading a player, and is based on knowledge and understanding of the team.

You are talking a strawman here. That is the problem. For sure, you can argue against your own strawman, but that will not tell you much.

Do you even have understand the point about Bonner I made? Expecting the same results with increased minutes is stupid, thus your whole point is either not understanding an argument here or just trolling. Honestly, I get tired of that stuff. Deng does not play only 20 minutes or so, his role does not change much without Rose on the court, that we should expect a lower impact based on that is somehow weird. Why should we expect a lower impact by Deng without Rose? And don't think that EVERY player in this league is at his best when he can play a role suited for the player's strength? Do you think Rose on a team in which he is not the primary ball handler, where he is just put into a role of a primary defender would have the same impact? Because you must believe that this is the case, otherwise you would make the same argument for Rose; that Rose is effected by his teammates and thus we can't get much out of the informations we have anyway. But it seems that your "standard" is only applied when you see fit. That is obviously completely biased, but it doesn't seem to bother you at all.

[alucryts]

I know exactly how regressions work as I have had to derive many of them by hand as an engineer over the years, and I work with them and image processing a lot. I am going to bow out of this conversation with you because it has gone from an in depth discussion of stats vs non-stats to something that is beginning to become personal.

[Rerisen]

But that is true for ALL players. hat you are saying is that we can't seperate between the players anyway, thus we should not try it at all. That makes your own "eye test" as much useless as any statistical analysis. The only thing I say is that statistical analysis beats the "eye test" in terms of predictive power.
...
We have a case of dimishing returns for overlapping skillsets. That this is less of an issue on the Bulls with Deng and Rose should be clear. So, your example of Wade/James does not apply here.

I already told you that I believe (minus Rose for the Bulls) would be an example of the opposite effect, not the same effect. James and Wade had overlapping skills, yes exactly, many *people* predicted it, I predicted it as well, but RAPM does not understand it. It needs people to make those corrections. At which point they become very arbitrary.

Just as it would need people to understand the Bulls would have all kinds of trouble scoring without a guy that contributed more to the Bulls offense last year (via passing and scoring) than any other player to their team in the league. Of which the loss could create a mini domino effect impacting players established values with him there.

This isn't just switching Mike James for the year with John Lucas, then I would totally be on board with these methods. You are telling me we already know it would work just peachy because the Bulls do fine 5 minutes a game when Deng is out there without Rose, it's not the same thing at all.

You are still starting with a methodology based on numbers I do not believe would translate so exactly, and then working backwards off them. Factoring Rose and Deng’s value based on the last 150 or so games in which the majority of their value by these numbers were built together. Then trying to adjust off of that, based on info we don’t have, or that I took issue with, like how many more games the Bulls might lose in the clutch (you said 4 games for instance) or extra negative effect of Bogans. But I don’t believe Deng and Rose’s values would rate out the same to begin with in a full season situation without each other. So the starting point is flawed to me even before these concessions.

Now if we assume a new Bulls team without Rose has a roughly similar player, like Tony Parker say, then I could buy it a lot more. But of course would not be the case if we just are removing one say to injury.

Deng's last year pre-Rose at high value was 07, just happens to be the last year before Rose where the team had a couple of players, Ben Gordon and Kirk Hinrich, having their two career years offensively, allowing Deng to do his thing as a role player, just as he does now, but could not if he was being asked to be the 1st option on a team (or virtually tied with Boozer). How the heck is Deng going to impact defensively as much with that kind of new scoring burden? Even guys like Kobe can't. It's tone deaf to not see these things.

Next up, Derrick Rose is already a better player than last year, he’s a better passer and has a better understanding of the current team’s personnel. He’s scoring more efficiently. We can see his PER is higher despite shooting 2 less times a game, that’s not easy to do. So we are measuring moving targets here, which can’t be forecasted forward only on who these guys were a year ago. Rose is still an improving player, that has gotten better every single year. Projecting him forward at a static level would be rather silly in light of that and his talent level. And why it’s not crazy to think his impact will be more relative to Deng over the next 82 games, vs the last 150. Especially if even in your own valuation of the two of them there is only something like a .15 difference between their values per game, a mere fraction of a point.

(Sidenote: Is it me or is RAPM not already over compressing a lot of these guys? The numbers are so much smaller than APM and Raw +/-. Reducing drastic errors perhaps, but over and under selling more. Chris Paul, the league leader this year is worth under 3.5 pts to his team per game!?!?)

I mentioned this on the stats forum, but I think the basketball equivalent of the baseball stats revolution is yet a few years off. Baseball, beyond being a simpler game, with far less interactions, did not have great divide among it’s sabermetricians when it came to accept the new era of metrics, whether the value of OPS, VORP, WHIP, or whatever even newer metrics have come along since I stopped following the sport too closely.

By comparison basketball has more of a quest for the holy grail approach, with people setting off in different directions for the promised land. We might have one system that you have used, and one that John Hollinger has used and both be able to beat Vegas (also not gospel) on occasion, yet John Hollinger also thinks Luol Deng is not even a worthy All-Star and you have him as a possible league MVP candidate (short of missed games) with Wayne Winston. Wayne also the guy that wouldn’t take Kevin Durant ‘for free’ a few years ago, presumably because his +/- were garbage his first two years. But any common sense talent evaluation knew it was only a matter of time before this guy was impacting wins in a big way and that it would show. Me? I would have taken Kevin Durant.

So while different systems can all do well against the whole field via different methods, and while wildly disagreeing on certain players within, there is plenty of room for specific error within that range, that might be trumped by yes, understanding of circumstance and contexts. Which your position doesn't seem to care much for.

[mysticbb]

I already told you that I believe (minus Rose for the Bulls) would be an example of the opposite effect, not the same effect. James and Wade had overlapping skills, yes exactly, many *people* predicted it, I predicted it as well, but RAPM does not understand it. It needs people to make those corrections. At which point they become very arbitrary.

The thing is that your prediction for Bulls minus Rose are even seen in the results. The Bulls without Rose are worse offensively. As you expected it and as the numbers expected it. The numbers aren't telling you something about the skillset, we have to adjust for that. That is meant with putting "numbers into context". What you are doing here is comparing your context with "numbers without looking into context" and then procede with concluding: numbers without put into context aren't good as a tool. Yeah, NOBODY is disagreeing with that at all. Really, you are talking a strawman here, nothing more.

Just as it would need people to understand the Bulls would have all kinds of trouble scoring without a guy that contributed more to the Bulls offense last year (via passing and scoring) than any other player to their team in the league. This isn't just switching Mike James for the year with John Lucas, then I would totally be on board with these methods. You are telling me we already know it would work just peachy because the Bulls do fine 5 minutes a game when Deng is out there without Rose, it's not the same thing at all.

The results are showing that the Bulls have more trouble scoring without Rose, but their defense is basically uneffected. And when a team is 6 points better than average on defense while only being 5 points better on offense, we very well end up with a team better than average overall even without Rose. And that is seen in the results without Rose too. The interesting thing is that this is basically seen to the same degree in complete games without Rose as in those 13 minutes without Rose in the game.
And I have no idea what you mean with "You are telling me we already know it would work". What does "work" mean here? That the Bulls are clearly worse without Rose? I ask you again: How far are we apart here in term of performance level drop? We both expect the Bulls to do worse, we both expect them to do worse on offense. Those expectations are backed up by the results in games without Rose. Now, we can procede to go over to the effect by Deng. I say that without Deng the Bulls are getting worse defensively. That's what happened. What did you expect?

You are still starting with a methodology based on numbers I do not believe would translate so exactly

What does "exactly" mean in that context, because I have a hard time seeing your point here being a contradication to the points I made. But you are still coming up with words like "works" or "exactly" without attaching any deeper definition to that.
I already said that this method is not "exact" in the sense that it can completely accurate predict the outcome. It has an error and thus is is natural to be not exact. But the predictions based on that are close to the results and so far much closer than anything I read about "Noah is the most important player" and "Deng does not have a big impact".

Factoring Rose and Deng’s value based on the last 150 or so games in which the majority of their value by these numbers were built together. Then trying to adjust off of that, based on info we don’t have, or that I took issue with, like how many more games the Bulls might lose in the clutch (you said 4 games for instance) or extra negative effect of Bogans. But I don’t believe Deng and Rose’s values would rate out the same to begin with in a full season situation without each other. So the starting point is flawed to me even before these concessions.

The thing is that those adjustments had minor consequences. And yet, you are not satisfied with the results. You haven't provided any further proof for the idea that Deng is more effected by Rose than vice versa and thus Deng's RAPM value is inherently flawed. The thing is that Deng had a similar value in his healthy season 2007. The thing is that Deng is not a much different player in terms of production and efficiency with Rose than without Rose. We have 1060 minutes in which Deng played without Rose over the last 1.5 years, but that is not a big enough sample? How many minutes would be appropiate? After how many minutes do you take a player's boxscore stats serious in terms of predicting future stats?
You keep implying that Deng's role would drastically change, but that isn't really true. Deng, in the 7 games without Rose, took basically the same amount of shots per 36 minutes, as in games with Rose. The role would not change dramatically, the PG of the Bulls would still be the main ballhandler and Deng would play still off the ball much. He had a couple of touches more, but he also had basically the same assists to turnover ratio.

What I really like to know is what you are arguing against? Because I get the impression that you are trying to argue against something I haven't said. I don't say at all that the Bulls without Rose would be fine, in fact I'm saying they are losing a lot and are no longer a contender.

Next up, Derrick Rose is already a better player than last year, he’s a better passer and has a better understanding of the current team’s personnel. He’s scoring more efficiently. We can see his PER is higher despite shooting 2 less times a game, that’s not easy to do. So we are measuring moving targets here, which can’t be forecasted forward only on who these guys were a year ago. Rose is still an improving player, that has gotten better every single year. Projecting him forward at a static level would be rather silly in light of that and his talent level. And why it’s not crazy to think his impact will be more relative to Deng over the next 82 games, vs the last 150. Especially if even in your own valuation of the two of them there is only something like a .15 difference between their values per game, a mere fraction of a point.

First of all, Rose PER is also higher, because the league average unadjusted PER is lower. ;) A player can have the exact same efficiency in this season as he had last season and would post a higher PER. PER is adjusted to the league average, that is one of the reasons you can't assume that PER would represent the player development completely.
And it would be silly to assume a player has a static value, but I NEVER said something different. As I mentioned it a couple of times now, you are arguing a strawman, nothing else. And it does not improve your argumentation, if you are just continue to do that.

Anyway, player's have aging curves, that is well known and a lot of people are using that for predictions (Hollinger, btw, is one of those). I haven't find the time to do that with a big enough sample, but it should be clear (and it was proven already) that applying aging curves is improving the predictive power.

(Sidenote: Is it me or is RAPM not already over compressing a lot of these guys? The numbers are so much smaller than APM. Reducing drastic errors perhaps, but over and under selling more. Chris Paul, the league leader this year is worth under 3.5 pts to his team per game!?!?)

Yes, and I explained it before why that is the case. Honestly, read my posts carefully and you would not be surprised by the results.

I mentioned this on the stats forum, but I think the basketball equivalent of the baseball stats revolution is yet a few years off. Baseball, beyond being a simpler game, with far less interactions, did not have great divide among it’s sabermetricians when it came to accept the new era of metrics, whether the value of OPS, VORP, WHIP, or whatever even newer metrics have come along since I stopped following the sport too closely.

Yes, and the issue seems to be that more sophisticated methods are not much accepted among fans, because they are not able to understand them. The degree we are approaching here is much higher than an average education provides. I make the mistake to assume that everybody can understand the maths behind that to the same degree as I can, but the truth is, that is not the case. We see alucryts making bold statements in terms of reliability of RAPM, while not having understood how that works overall. The same is seen in your posts. Despite the fact that you haven't figured out how it works, you think you can make bold statements that it must be wrong. But the statements are based on your personal belief to be able to judge that better without providing any evidence (hard facts) to support your claims. And it seems to not bothering you that you haven't understand the underlying methods and don't have anything to disprove that. That's like arguing against theoretical mechanics while not being able to handle matrices well and aren't understand what Hamiltonian or Lagranian even mean. I doubt that you would try to lecture a theoretical physicist on that subject, but you feel you have to lecture people with more statistical knowledge about the flaws of those stats.

By comparison basketball has more of a quest for the holy grail approach, with people setting off in different directions for the promised land. We might have one system that you have used, and one that John Hollinger has used and both be able to beat Vegas (also not gospel) on occasion, yet John Hollinger also thinks Luol Deng is not even a worthy All-Star

Hollinger relies on boxscore metrics, a metric which showed to be worse in any retrodiction test. PER is worse at explaining what happen and it is worse at predicting. Hollinger can beat Vegas by using scoring margin and make adjustments based on player development curves. That is NOT the same method he uses to say that Deng is not an All-Star. Well, boxscore based stats will tell you pretty much the same.

and you have him as a possible league MVP candidate (short of missed games) with Wayne Winston.

Wayne Winston had im as MVP candidate last season, he may have him again in this season. But I don't said anything about MVP at all. Deng is not in the Top5 so far. He is clearly below James, Howard, Durant and Paul in this season so far. So, I have no idea how you come up with that statement again.

Wayne also the guy that wouldn’t take Kevin Durant ‘for free’ a few years ago, presumably because his +/- were garbage his first two years.

He said that during a specific season without saying anything about Durant's development. And I agree, I would not have taken Durant for free in order to win as many games as possible in any of his first two years. There were a lot of better players. That Durant will learn and will make less mistakes was something to expected.

But any common sense talent evaluation knew it was only a matter of time before this guy was impacting wins in a big way and that it would show. Me? I would have taken Kevin Durant. :)

But that is a different scenario. Winston wasn't asked about Durant's future potential, he was asked about the recent performance by Durant. You would have taken Durant in his first two seasons instead of say Paul Pierce in order to win the most games possible? I highly doubt that.

So while different systems can all do well against the whole field via different methods, and while wildly disagreeing on certain players within, there is plenty of room for specific error within that range, that might be trumped by yes, understanding of circumstance and contexts.

Well, and who is arguing against that?

Which your position doesn't seem to care much for.

Sorry, Rerisen, but would you please stop with such nonsense. SHOW ME ONE EXAMPLE IN WHICH I SAID THAT WE CAN IGNORE CONTEXT! Please, do me that favor before you continue to lie about that.

[Rerisen]

This is becoming fruitless repetition at this point. I’ll let your post be the last one for this part of the discussion. Because there is nothing fundamentally new in your responses that changes our differences on the topic.

Maybe I can make things clearer for you by saying I think the Bulls overperformed in the 10 games without Rose this year.

More so margin wise, than win wise though. At one point, they set a record for most road blowout margins in a row. That’s just who they are, a team that does not let up in the regular season. It’s a shame Rose didn’t get to join in on the stompings, or his ratings for this season would probably be higher.

Deng alone had +86 just vs dreadful PHX, CLE and CHA. These margins are well beyond what even a healthy Bulls team would be expected to win by.

It might be considered that numbers generated once games get past a certain ‘laugher’ margin, aren’t that valuable.

Combined (current) record of the 10 teams played without Rose is 113-195 (.367 win %, repeat opponents counted twice). What a great sample. Not only tiny for projecting a year on, but horribly unbalanced.

[mysticbb]

Maybe I can make things clearer for you by saying I think the Bulls overperformed in the 10 games without Rose this year.

Ok. By how much? And how do you explain this?

It might be considered that numbers generated once games get past a certain ‘laugher’ margin, aren’t that valuable.

Actually, that was tested and installing a cap at 12 as the max "valuable" margin did decrease the predictive power. It is better to include all games with the original scoring margin to have the best prediction.

[lorak]

great job mysticbb. yours posts are very educational, I learned a lot.

[drza]

I am not going to put too much more time into this beyond the video analysis in the coming weeks, but the problem I have with people using +/- and saying find me a number that proves it wrong is that no such numerical evidence exists besides plus minus that does a good job of separating out individual players defense. Out of all the stats out there, +/- is the best as nothing even comes close. That's the problem inherent in almost any argument revolving the stat; there is no arguing with the number with other numbers! No number can describe individual defenders as well as it can. When it is wrong, there is nothing numerically to prove it wrong. When it is right, there is nothing numerically to prove it right. Plus minus is the end all be all of numerical individual defense. I do not feel that it does a good job at all in describing this either. It's sort of a deal where I feel it is the best of the worst.

I feel like a lot of posters take it as the gospel and that it cannot be wrong because it is "results" and ignore the fact that it can be wrong. We can always go on to any advanced stat and pick out players who just are not ranked correctly at all. This shows (obviously) that the stat has some error as they all do, and that error can vary from very great to very little going from player to player. How do we know the error associated with any player? How do we know if Deng is +4.0 +/- 100 or +4.0 +/- 0.1?

When a stat that has no opposition or additional numerical context is used to say something that a great number of neutral observers disagree with, why is the burden on me to prove a stat wrong that is unasailable when game tape and defensive analysis say that the conclusion is ludicrous?

A side question I have is, how does any +/- stat accomodate for teammates or opponents mathematically speaking? Is a pseudo SRS type of analysis used?

OK, we're getting somewhere here because now the discussion has advanced. We're no longer at the point where using +/- data is analogous to something ridiculous like a teacup orbiting the sun, but we're at the point where you admit that the +/- approach is useful, the best that is currently out there "and nothing comes close" (even if it is the "best of the worst"), but you see no way to quantitatively argue it since "it" IS currently the best. This is good, because it lets me point out that you CAN quantitatively argue +/- results...USING variations of +/- results. I'll explain.

One of the big hurdles to wider-spread use of the +/- approach on these boards is the tendency for some to treat "+/-" as if it's a black box, where data comes into "the +/-" and an answer is output like magic. But it's really not like that. The "team results, +/-" approach is analogous to the "box score" approach or the "scouting" approach. Just like there are many, many ways to look at the box scores for the sake of analysis, THE SAME IS TRUE of +/- data. There IS no "the APM" that is the complete be-all for the +/- approach, any more than you would look ONLY at PER or ONLY at Win Shares and use them in the absence of any other box score data or context. In fact, let's run with that.

PER is a composite box score stat that attempts to sum up all of the box score contributions into one figure. We know that it's not a perfect measure, and that it overemphasizes volume scoring a the cost of efficiency. How do we know this? Because we can look into how PER is calculated, and identify specific components of PER where errors can be found. It has been shown that PER can be increased with increases in scoring volume as long as the person has a shooting efficiency on the order of like 33% or something. While this is obviously something that intuition would argue to be wrong, separate regression studies have shown quantitatively that shooting efficiencies that low are not conducive to team success.

So in summary, PER (a composite box score stat) has a demonstrated weakness that is shown quantitatively by looking at other box score measures that are otherwise incorporated into PER.

This is also how you would attack APM results!

The different +/- approaches also have weaknesses that are known and have been shown. This is also true for the adjusted +/- approaches. A standard one year APM calculation (the giant algebra problem that I and DocMJ have referred to in this thread) has (essentially) more unknowns than equations, which prevents us from being able to solve it without making estimates that create the error. And that error can be quantified in a standard error like what is shown on the 1-year and 2-year APM studies at basketballvalue. Like you I'm also a research engineer so I, too, appreciate the individual standard errors provided there. And for a 1, or even 2-year APM study of this type, the standard errors can be both large and quantifiable. Also, there can be a co-linearity issue if players play the majority of their minutes together so it is hard to separate one's value from another. To address some of these issues, one approach is to use multiple years of data...Ilardi's 6-year '04 - '09 study shows much, much smaller standard errors and also allows many more combinations of players ( https://docs.google.com/a/backyardbrain ... l=en#gid=0 ). But a multiple-year study is not instantaneous, and comes with the assumption that all players are essentially the same over the entire time...which of course isn't true. So, another approach is the RAPM approach of using different math techniques like ridge regression to improve the predictability of 1-year studies. Because the math tricks are estimates, by definition they aren't completely accurate and come with inherent error, BUT the way the math is performed it makes the standard errors (as reported for the standard APM calculations) not tenable.

Now, my point in saying all of that? There are different approaches even for just the composite APM stats...analogous to the difference between PER, wins shared, wins produced, etc. among the composite box score stats. There are strengths and weaknesses to each approach, but they are also DIFFERENT approaches. And then, the components that make up APM studies are ALSO different from the APM results themselves.

So, bringing it back to the specifics of Deng for this thread, it is NOT TRUE that you can't support your arguments against Deng quantitatively just because Deng's defensive APM is good. That would only be true if you were standing by your original stance that the +/- approach itself is so ridiculous as to fit the Russell's teacup narrative, which based on your response above seems to no longer be the case. No, now, if it's Deng's defensive RAPM for the last 1.3 years that you disagree with, you have lots of tools to make your case.

*You can look at multi-year APM studies instead of RAPM, to see if Deng's impact is a new thing. The main ones that I know of are Ilardi's '04-09, Englemann's 06-10, and Englemann's '08 - '11.

*You can look at Deng's history, using RAPM from other years. One of the better exchanges in this thread was you pointing out that Deng struggled a bit in '09 and '08 (but no numbers shown), then Mystic responding that this was due to injury and using 2007 results to support his case. There's meat there. Really go through Deng's earlier years, and see if you can find some examples to make your case that his impact isn't really that big without Noah around.

*You can look at specific areas of weakness of APM. For example, you say that Noah is the real defensive key, that Omer is good but couldn't replicate Noah in bigger minutes, that Deng playing with the bench are why he looks so good, that Boozer sucks and pulls Noah down, that Korver is so bad on D he skews results, etc. You've made statements like this in the thread, but you haven't supported them. So, do so. Ditch APM, and actually go into the 5-man line-ups. Show with team defensive rating that among the starting line-up groups, swapping Noah for replacements really hurts the team (Noah missed like 33 games last year, that's significant enough to show up). Show that swapping Deng out for someone else in the starting line-ups doesn't have much effect because Noah covers, but that Deng makes a huge difference to the second unit. Or show that Deng's 5-man units with the bench mob are so much better than the starting case that it skews his defensive APM results.

The point is, there are MANY statements that you make (and have made) that you could actually test very reasonably with an aspect of the +/- approach that is NOT the same as APM. You can look how Deng's measured impact has changed over time under different circumstances, using much more contextually specific measures than a composite APM score. I mean, Deng has been in the NBA fo 8 years under 5 coaches on very different teams, and we now have various +/- or 5-man units or play-by-play results for ALL of those years. If Deng really is just a guy who's been in the right place at the right time for the past 1.3 years, it really shouldn't be that difficult to quantitatively support your case with the available data. And if you go through all of the reams of data available and STILL can't find any quantitative support for your stance...while that might not conclusively prove anything, I would strongly suggest that maybe you at least consider that in this instance you might be incorrect in your intuitions.