RealGM

penbeast's sig has a quote that I absolutely love:

He uses statistics as a drunken man uses lamp-posts…
For support rather than illumination

Andrew Lang (1844-1912)

I always try to keep it in mind when I'm using stat. Recently, my usage of adjusted +/- to evaluate players has caused some people to think I'm crazy, or a hater, or a both, because of how bad Kobe Bryant looks in the stat.

So I've written an article about being in this situation as an analyst who uses statistics, as well as ironing out specifics about how I used adjusted +/- to come to my conclusions about Kobe in the MVP race.

Two key snippets:

The quote above makes a powerful point that I always try to keep in mind as I use stats. The reality is that all serious basketball watchers come to their use of a particular tool set of stats at least partially based on their own understanding and intuition of the game. Thus on a broad strokes level, we are all arguably guilty of being Lang’s drunken man. This behavior is not an inherent problem as long as it does not extend to fine scale analysis, however it’s quite easy to make that leap because doing so is very tempting and often times quite subtle. Were I to ignore Kobe’s APM, then I truly would be the drunk Lang seeks to condemn for I’d then clearly only be using the stat when it fit with my pre-conceived notions.

The players ranked ahead of Ginobili then are considered to only be on teams with inferior supporting casts compared to what Ginobili has. The salient question about Kobe’s candidacy becomes: How can one move Kobe Bryant ahead of a star on a team with a superior record, when his supporting cast is so strong that they actually give him a terrible APM – something I don’t believe we’ve never seen in the half dozen years we’ve had this type of advanced stat?

And my current answer is: I can’t. This isn’t a reflection Kobe’s capabilities, but simply based on what we’ve seen this season, Kobe’s got a supporting cast as solid as anyone else, and yet there are teams with better records. That puts a ceiling on how much credit I can give him in the MVP race.

http://asubstituteforwar.com/2011/03/19 ... lluminate/

I think that adjusted +/- in theory can be the most valuable stat. In practice though, I am not
convinced that they can do the math accurately enough to trust the numbers. Try to explain the
current Laker numbers to me. I don't think the math makes any sense.

Gasol, Odom and Bynum are all very positive and every other Laker is negative. But how is it
possible for every Laker PG/SG/SF to hurt the team? This doesn't even make sense. The real
explanation is that the other Laker bigs who hardly play are truly horrific, which completely
distorts the Laker numbers. So Bynum/Gasol/Odom get a huge bump for having terrible backups. Great. But Kobe also has a relatively terrible backup. Shannon Brown is at -5.5 for the year, while Kobe is at 4.9. But not only is Kobe not rewarded for this, somehow he is heavily punished. Also, the Kobe/Fisher gap seems completely impossible if you know anything about the Lakers. Kobe is 11 points worse than his net number, while Fisher is only 7.5 points worse. Fisher plays almost exclusively with the starters though. Kobe plays much more with Blake and Brown, which means his number should be adjusted up compared to Fisher. Instead we have the exact opposite result.


-----------------------minutes---------adj +/--------on--------off-----net
Gasol, Pau--------- 2,652.67--------11.93-------- 8.88----- 0.09----- 8.79
Bryant, Kobe------ 2,406.47---------(-5.83------- 8.35----- 3.43----- 4.92
Odom, Lamar----- 2,308.45-------- 7.64--------- 7.95----- 4.73----- 3.22
Artest, Ron------- 2,058.22--------(-2.54--------- 8.84----- 3.94----- 4.91
Fisher, Derek----- 1,959.62--------(-0.55--------- 9.90----- 2.82----- 7.08
Blake, Steve------ 1,412.33--------(-5.60--------- 3.74----- 9.05----- (-5.31
Brown, Shannon---1,373.67--------(-7.08-------- 3.50----- 9.15----- (-5.65
Bynum, Andrew----1,190.45-------11.95--------- 8.80------5.90----- 2.89
Barnes, Matt------- 897.38-------- (-0.68--------- 3.85----- 7.97-----( -4.12

Welcome laika, you have understandable questions.

I think the first thing to realize is that what makes +/- particularly useful in the NBA is that the lineups aren't rigid. Kobe isn't being replaced with Shannon Brown whenever he goes out. If he were, then at most all we could do would be to compare the two of them. In reality teams play all sorts of lineups, and in general the smaller they go the better the offense, the bigger they go the better the defense.

You're looking at part of the equation when you're thinking about adjusting numbers based on who you get to play with. However the other side of that coin is who else those guys get to play with besides you and how they do when they do this, which in turn changes how the algorithms rates the players you get to play with.

With Kobe this year, the big thing is just that the team does unexpectedly well without him, and that's the most damning thing that can happen according to this stat.

Now again, I don't use APM as gospel. Literally interpreting the numbers to say Kobe hurts the team doesn't make sense to me because of the reasons I've outlined before. However, when your teammates do really dang well without you, I think using that along with other facts to say you've got one hell of a supporting cast is not even bold, it's just reasonable.

MJ, where can I find the lowdown on how APM is calculated? I typically stick to raw +-, despite its limitations, because I have not found the APMs I've seen in the past to pass the sniff test, and I can't find how they are calculated.

For instance, in one of your articles, you mention that APM solves the problem of one player always playing with a star. But how does it solve the problem of which is the star?

On basketballvalue, I've noticed that their adjusted numbers simply don't include low minute players. I've suspected before that whatever regressions APM is using don't work well with low minute players, so perhaps that is why that site excludes them.

floppymoose wrote:MJ, where can I find the lowdown on how APM is calculated? I typically stick to raw +-, despite its limitations, because I have not found the APMs I've seen in the past to pass the sniff test, and I can't find how they are calculated.

For instance, in one of your articles, you mention that APM solves the problem of one player always playing with a star. But how does it solve the problem of which is the star?

On basketballvalue, I've noticed that their adjusted numbers simply don't include low minute players. I've suspected before that whatever regressions APM is using don't work well with low minute players, so perhaps that is why that site excludes them.

Best discussion I'm aware of on the subject:

http://sonicscentral.com/apbrmetrics/vi ... e800c13a5d

If you're looking to calculate it yourself, this page has good instructions although I had trouble with it at a certain point, and used Matlab instead of R.

http://www.countthebasket.com/blog/2008 ... lus-minus/

Re: Which one is the star? The idea is that you assign weights to each player in the league with the theory that they add up to the +/- results we actually get, and the regression corrects this until reaching an optimal point. So it automatically determines some players to be superior to others and uses this against the players they play with.

Re: Excluding the low minute players. Well it doesn't work well on them, but that's not why I agree with excluding them. For me it's the matter that I don't really care about the low minute players and I'm willing to believe that the ability of the lot of them tends to be pretty similar. We can get a more precise result for the players I do care about by smushing the scrubs altogether assuming those assumptions hold because it effectively increases sample size, so why not do it?

Doctor MJ wrote:Re: Which one is the star? The idea is that you assign weights to each player in the league with the theory that they add up to the +/- results we actually get, and the regression corrects this until reaching an optimal point. So it automatically determines some players to be superior to others and uses this against the players they play with.

First, thanks for taking the time to try to explain this. Having said that, I don't understand a word of the quoted paragraph. Assign weights how? Let's take a contrived example: two players always play together, and always sit together. Their +- is always the same because of that. There is no way to separate them in the +- data. How do you decide which is the star? What is this "assigned weight" you mention?

floppymoose wrote:

Doctor MJ wrote:Re: Which one is the star? The idea is that you assign weights to each player in the league with the theory that they add up to the +/- results we actually get, and the regression corrects this until reaching an optimal point. So it automatically determines some players to be superior to others and uses this against the players they play with.

First, thanks for taking the time to try to explain this. Having said that, I don't understand a word of the quoted paragraph. Assign weights how? Let's take a contrived example: two players always play together, and always sit together. Their +- is always the same because of that. There is no way to separate them in the +- data. How do you decide which is the star? What is this "assigned weight" you mention?

That's the BIG problem with APM. And it happens.

The other collinearity problem is when a player is ONLY substituted for another player. Then that pair of players can vary inversely from the rest of the team with no restraint. This happened with D-Howard and Gortat for several years at Orlando--their relative ratings to one another stayed about the same, but the varied inversely from the rest of their team.

Adding Ridge Regression can cut out some of the tremendous collinearity, but at the expense of losing some of the validity of the stat.

DSMok1 wrote:

floppymoose wrote:

Doctor MJ wrote:Re: Which one is the star? The idea is that you assign weights to each player in the league with the theory that they add up to the +/- results we actually get, and the regression corrects this until reaching an optimal point. So it automatically determines some players to be superior to others and uses this against the players they play with.

First, thanks for taking the time to try to explain this. Having said that, I don't understand a word of the quoted paragraph. Assign weights how? Let's take a contrived example: two players always play together, and always sit together. Their +- is always the same because of that. There is no way to separate them in the +- data. How do you decide which is the star? What is this "assigned weight" you mention?

That's the BIG problem with APM. And it happens.

The other collinearity problem is when a player is ONLY substituted for another player. Then that pair of players can vary inversely from the rest of the team with no restraint. This happened with D-Howard and Gortat for several years at Orlando--their relative ratings to one another stayed about the same, but the varied inversely from the rest of their team.

Adding Ridge Regression can cut out some of the tremendous collinearity, but at the expense of losing some of the validity of the stat.

Right and to be clear, "collinearity" is the exact name for the problem that is just one of the classic issues in statistics.

fm, understand though that the reason why this is type of stat is useful in basketball is that the collinearity is far lower than in some other sports. We get to see about 4000 minutes of each team per season, typically about 1/3rd of that is without out a star in there, which means we get to see a team more *without* a star, and the amount you get to see Peyton Manning play *total* in a season. Add in that the lineups are not tied strictly to having 1 of each of the 5 player positions in the game at all times like they would be for some other sports, and that's really quite a lot. It's not perfect, but when we look at the results over significant time spans, they look pretty darn common sensical.

I mean consider

floppymoose wrote:First, thanks for taking the time to try to explain this. Having said that, I don't understand a word of the quoted paragraph.

floppy, I appreciate the kind words, even if they're said in exasperation. I want you to know that I'm always happy to explain further. These are hard concepts to understand, and hard concepts to explain, and frankly, I could use the practice explaining to make sure I understand them.

Well the good news is that I'm a former math geek. I'm out of practice which means the vocabulary is gone and things have to be explained in close to layman's terms. But I'm pretty good at grasping the concepts. (It's why WP immediately jumped out at me as jury-rigged).

floppymoose wrote:Well the good news is that I'm a former math geek. I'm out of practice which means the vocabulary is gone and things have to be explained in close to layman's terms. But I'm pretty good at grasping the concepts. (It's why WP immediately jumped out at me as jury-rigged).

Here's a mathematical relation: FM48 > WP.

Doctor MJ wrote:
With Kobe this year, the big thing is just that the team does unexpectedly well without him, and that's the most damning thing that can happen according to this stat.

This is false.
The Lakers beat teams by 8.9 points when he plays, 2.7 points when he is out. Kobe's net +/- is one of the better ones in the league. It's funny that Kobe's net +/- is over 5 times as good as Rose's, yet Rose is the runaway MVP leader and we are talking here about whether Kobe is one of the worst players in the league.

I think it is fairly obvious that Bryant is an extreme outlier this year, and that adjusted +/- has his number completely wrong.

Artest and Fisher play the huge majority of their minutes with Kobe. This of course would create obvious collinearity problems. Both players do far worse in the small amount of time they play without Kobe.

Kobe and Fisher(plays with Kobe 97%)- 10.2 ppg in 1955 minutes
Fisher without Kobe - 5.4 ppg in 62 minutes
Kobe and Artest(plays with Kobe 89%)- 10.4 ppg in 1898 minutes
Artest without Kobe - negative 7.4 ppg in 225 minutes

Kobe's net +/- rating is over twice as high as Odom and Bynum's, yet they have vastly higher adjusted +/- ratings.

Kobe has the tenth highest ON rating in the league among starters(8th if you don't count the 2 players he carries). Kobe has the 22nd highest net +/- rating in the league. It is only when you get to the adjustment that Kobe doesn't look like an all-star. His downward adjustment is the largest of any of the top 50 players. Only a few others even come close.

Over the last 3 years Kobe has had by far the highest adjusted +/- rating on the Lakers. So you would have to believe that despite Kobe getting similar stats and the Lakers winning a similar amount of games with almost exactly the same team that Bryant mysteriously had a historically unprecedented drop in playing quality in one year. I shouldn't even have to point out how absurd this is.

I think that not only is Kobe's adj +/- number incorrect this year, but massively incorrect. By all other statistical measures besides adj +/- Kobe is still one of the top players in the league.

laika wrote:

Doctor MJ wrote:
With Kobe this year, the big thing is just that the team does unexpectedly well without him, and that's the most damning thing that can happen according to this stat.

This is false.
The Lakers beat teams by 8.9 points when he plays, 2.7 points when he is out. Kobe's net +/- is one of the better ones in the league. It's funny that Kobe's net +/- is over 5 times as good as Rose's, yet Rose is the runaway MVP leader and we are talking here about whether Kobe is one of the worst players in the league.

I don't know what to tell you man. If the team did worse without Kobe, he'd do better by this stat. It's pretty simple. You don't have to value the stat or agree with conclusions people reach from the stat, but it's no great mystery what the cause is.

floppymoose wrote:Well the good news is that I'm a former math geek. I'm out of practice which means the vocabulary is gone and things have to be explained in close to layman's terms. But I'm pretty good at grasping the concepts. (It's why WP immediately jumped out at me as jury-rigged).

You & I are in a similar boat. I'm just now really trying to do this stuff on my own.

ElGee wrote:

floppymoose wrote:Well the good news is that I'm a former math geek. I'm out of practice which means the vocabulary is gone and things have to be explained in close to layman's terms. But I'm pretty good at grasping the concepts. (It's why WP immediately jumped out at me as jury-rigged).

Here's a mathematical relation: FM48 > WP.

I want this meme to spread across the 7 seas.

Doctor MJ wrote:
I don't know what to tell you man. If the team did worse without Kobe, he'd do better by this stat. It's pretty simple. You don't have to value the stat or agree with conclusions people reach from the stat, but it's no great mystery what the cause is.

You can agree that his rating is extremely irregular and most likely wrong. He would necessarily do better by this stat only if adjusted +/- was 100% reliable and valid- which it clearly is not. I don't know why you keep talking about the team doing worse without Kobe- as I've shown in multiple ways the Lakers play significantly worse without him.

Short of a mathematical proof showing that adjusted +/- produces invalid results due to statistical errors I don't know what else I could show you. What we know-

-Adjusted +/- is known to have problems when dealing with collinearity and substitution effects. The Lakers have a large collinearity problem since 2 starters spend almost all their time playing with Bryant.
-Adj +/- can be very unreliable, with big swings for no apparent reason.
-Bryant historically has had a very good adj +/-.
-There has been remarkable continuity on the Lakers from last year to this year. The players are the same, the team performance is the same, and Kobe's stats are very similar.
-Every other statistical indicator shows Kobe is having a good year. His net +/- is good, his Per is good, his net production is good and virtually every Laker lineup plays significantly worse when Kobe is off the court. From what I can see there is nothing else that supports his adj +/- number.
-Kobe is the largest outlier among all of the top players.

Here's a hypothetical, just for fun- If adj +/- were wrong, how would you know? Unless you were a true statistical expert, then you would not. As far as I know no one has come close to proving that adj +/- is valid and reliable in every situation. If anything, people have shown situations where it would give flawed results. Of course common sense would dictate that you discard the adj +/- number when it disagrees with every other piece of evidence, as it does here.

I think it's time to let go of the lamppost in this particular case and admit that Kobe's adj +/- number is really flawed this year.

laika wrote:

Doctor MJ wrote:
I don't know what to tell you man. If the team did worse without Kobe, he'd do better by this stat. It's pretty simple. You don't have to value the stat or agree with conclusions people reach from the stat, but it's no great mystery what the cause is.

You can agree that his rating is extremely irregular and most likely wrong. He would necessarily do better by this stat only if adjusted +/- was 100% reliable and valid- which it clearly is not. I don't know why you keep talking about the team doing worse without Kobe- as I've shown in multiple ways the Lakers play significantly worse without him.

Short of a mathematical proof showing that adjusted +/- produces invalid results due to statistical errors I don't know what else I could show you. What we know-

-Adjusted +/- is known to have problems when dealing with collinearity and substitution effects. The Lakers have a large collinearity problem since 2 starters spend almost all their time playing with Bryant.
-Adj +/- can be very unreliable, with big swings for no apparent reason.
-Bryant historically has had a very good adj +/-.
-There has been remarkable continuity on the Lakers from last year to this year. The players are the same, the team performance is the same, and Kobe's stats are very similar.
-Every other statistical indicator shows Kobe is having a good year. His net +/- is good, his Per is good, his net production is good and virtually every Laker lineup plays significantly worse when Kobe is off the court. From what I can see there is nothing else that supports his adj +/- number.
-Kobe is the largest outlier among all of the top players.

Man, I keep getting hung up in conversations to night where we talk about abstract intellectual stuff, and people are getting confused with basic language. I don't mean that as an insult to you in any way, such things can happen to anyone, but it's frustrating.

When I say "Kobe's APM would be better if his teammates did worse without him", I mean "Kobe's APM would be better if his teammates did EVEN worse THAN THEY ACTUALLY DO without him".

Is it damning for the stat's math that he rates poorly given that the team does better with Kobe on the floor than when he's off the floor? Not at all. Kobe's hardly alone that this happens to. The answer is always the same: Given the players Kobe gets to play with disproportionately relative to the combinations of players who play for the Lakers with Kobe not on the floor, the gap in performance is less than expected.

It's up to each person to decide what that means to them, and clearly I don't do anything like a straight mapping of it, but the description of what's happened is clear.

The issues of reliability you speak of are real, and if you consider them so severe that the stat should be completely ignored at all times, that's up to you. I don't throw stats out like that though. The lack of reliability just means I view the numbers with a great deal of fuzziness, and draw conclusions with great caution. Kobe's stats are WAY out there to the point they are impossible for me to ignore...and the result is that I have him a few slots lower in my MVP rankings than other people. This is really not that bold folks.

Re: "Kobe's been good by this stat before, and he and the team are so similar to previous years."

Kobe's playing 5 minutes per game less this year than last year and the team's SRS is much better than last year's team. If you are not seeing a shift in the play of his supporting cast, you aren't paying close enough attention.

laika wrote:Here's a hypothetical, just for fun- If adj +/- were wrong, how would you know? Unless you were a true statistical expert, then you would not. As far as I know no one has come close to proving that adj +/- is valid and reliable in every situation. If anything, people have shown situations where it would give flawed results. Of course common sense would dictate that you discard the adj +/- number when it disagrees with every other piece of evidence, as it does here.

I think it's time to let go of the lamppost in this particular case and admit that Kobe's adj +/- number is really flawed this year.

If I were a true statistical expert? Whoa. I'd say you're thinking about this all wrong. This isn't something where if we only had a good enough statistician in the room, he could tell us which is right, and which is wrong. There is uncertainty here, which is the point where the consulting statistician comes back to the client and together they use his intuition about stats and their intuition about the context of the data to come to the most reasonable conclusions.

Am I a good enough statistician to play those two roles? Well you're free to decide whatever you want. I've got a sufficiently large ego than I'm not terribly concerned. Should you believe me? I don't know what to tell you. You want a resume? Honestly for me what this all comes down to is that I'm having fun - I'm concerned about being wrong primarily as a means for self-improvement. I don't want others to believe something simply because I believe, and you'd better believe that there is no authority that I'll defer to without cause.

Re: flawed this year. You know I don't know if I'd disagree with that statement. If the stat was perfect, I could use it as a direct proxy for how good of a year Kobe has had. I cannot, so in some sense it is flawed. Flaws don't mean you discard though, they mean you evaluate how much stock you can give the thing in question. As I've said before, out of the 450 players in the league, I have Kobe about 3 or 4 slots lower than than some other people. This is really not that big of a thing.

What is the source for the Kobe adj +-?

Looking at the raw numbers, he is third on the team:
http://basketballvalue.com/teamplayers. ... C&team=LAL

It seems hard to believe that if he is third best in raw, that he is leaching his way to a good +- via his teammates. Intuitively, I think the only way that makes sense is if Pau Gasol is the greatest basketball who ever lived, and the rest of the Lakers are all riding his coattails.

These kinds of results (and I've seen others that didn't make sense to me as well) are why I am both interested in how adj +- is calculated (so that I can figure out what the problem is) and also why I use raw +- instead (because until adj can be fixed, I think raw is more reliable^h^h^h^h^h^h^h^hvalid).

Doctor MJ wrote:

The issues of reliability you speak of are real, and if you consider them so severe that the stat should be completely ignored at all times, that's up to you. I don't throw stats out like that though. The lack of reliability just means I view the numbers with a great deal of fuzziness, and draw conclusions with great caution. Kobe's stats are WAY out there to the point they are impossible for me to ignore...and the result is that I have him a few slots lower in my MVP rankings than other people. This is really not that bold folks.

As I've said before, out of the 450 players in the league, I have Kobe about 3 or 4 slots lower than than some other people. This is really not that big of a thing.

I think we pretty much agree that in this case adj +/- has the value wrong, but are approaching it from different angles.

I don't think that adj +/- should be completely ignored. In fact, I think it theoretically can be the best stat since it most directly measures winning. However, it sometimes gives bad results. As in this case where all the available evidence disagrees then you have to discount adj +/-.

When I get time I'll try to list some Laker lineups to see where Bryant might be getting rated down. At first glance though, I'm not seeing anything. Pretty much all of the top Laker lineups include Kobe.

floppymoose wrote:What is the source for the Kobe adj +-?

Looking at the raw numbers, he is third on the team:
http://basketballvalue.com/teamplayers. ... C&team=LAL

It seems hard to believe that if he is third best in raw, that he is leaching his way to a good +- via his teammates. Intuitively, I think the only way that makes sense is if Pau Gasol is the greatest basketball who ever lived, and the rest of the Lakers are all riding his coattails.

These kinds of results (and I've seen others that didn't make sense to me as well) are why I am both interested in how adj +- is calculated (so that I can figure out what the problem is) and also why I use raw +- instead (because until adj can be fixed, I think raw is more reliable^h^h^h^h^h^h^h^hvalid).

The APM I'm using comes from basketballvalue as well.

I'm running out of things to say right now, so feel free to ask other questions - and again I'll admit, I'm not the world's expert on the thing.

As I'm guessing is becoming clear, when we talk about whether the stat is "wrong", I prefer to just go into the data rather than talk about conclusions. People can do what they want with their conclusions, but the idea that something is wrong the math troubles me.

Consider that according to nba.com, Kobe's played 2159 of his 2476 with Pau Gasol. So the vast majority of is played with Gasol. Any time you get a situation along those lines, that's going to have the potential to result is a very large difference between the two players in the APM. And all year the team's done quite a bit better with Gasol & no Kobe than with Kobe & no Gasol.

The phenomenon I'm describing is multicollinearity, and certainly not considered a good thing. You want your data to have very little multicollinearity - so for this data to have high multicollinearity diminishes how much confidence we should have in a regression-based metric like APM.

For me, that diminished confidence is still not zero'ed out, but on a scale of 0 to 1, it's certainly closer to 0.

With all the claims here I make of making "not bold" conclusions, you may wonder what the point is of my discussion, but I think people need to realize there's a larger point. As I mentioned before, Kobe's MPG are down by 5 MPG this season, and his team's SRS is significantly better. Add in that Kobe's per minute performance is worse than it was during his peak ('05-06 to '08-09). People are starting to give Kobe some serious MVP buzz right now, simply based on the fact that he's Kobe, he's still scoring, and the Lakers are the best team in the league, so there is ample reason to try to dispel the notion that this is the same Kobe performance we've always seen.