Talk Me Down: How is Deng not MVP of the Bulls?

[mysticbb]

OK, next question, how did you adjust for the strength of the opponent?

And that's where ridge regression comes in handy.

;)

[Doctor MJ]

mystic, I'd like to hear you answer his question as I can only guess what you mean. I'll give my guess, please correct as much as you need, or just start from scratch if I'm too far off the mark.

So my understanding is that normally RAPM uses an identity matrix (or at least the most simplistic form of it would). While classic APM can be said to have adjusted for opponent strength, usage of the unadjusted identity would re-introduce bias. So, it makes sense to construct a different matrix which is based on the opponent strength, which I'd imagine would be based on each opponent's RAPM, how often they played against each other, and then iterate etc.

Is that right? Is there more?

It did also seem to me that coldfish would find it important to factor in changes over the course of a season. My guess is that you'd agree that there is a clear ceiling about what RAPM can do in terms of precision on this front, but that you clearly think don't think that this is nearly as big of a deal as he does given that you chalk up perceived trends to normal quirks of a basketball season.

[mopper8]

Forgive me in advance if this seems rambling or incoherent...

RAPM still has limits in terms of its inability to evaluate skill set and team fit. Simply adjusting for the players on the court doesn't adjust for roles filled within the team context.

So, what I mean is, for example, OKC is a below-average team on the defensive glass (23rd in the league). Now, as it turns out, Collison is nothing special on the defensive glass. However, for the sake of argument, let's pretend that he is not only good on the glass, he's great there, and by far the best on the team.

I've seen studies that suggest defensive rebounding has a high correlation with winning. So ultimately, no matter who Collison (in this imaginary scenario) is paired with, his impact will always be high, because he's carrying the rebounding load, and that's a much-needed thing on a team bereft of strong rebounders.

However, put him on Orlando, and the skill he brings to the table that is so valuable to OKC is all of a sudden very redundant next to Dwight Howard, and his impact is minimized because the marginal improvement he gives you is small, because the one area where he's truly great is already covered.

Now, obviously, whatever Collison is bringing to the Thunder, it's not defensive rebounding. And as far as I know, what it is that Collison brings to the table has not been adequately described and broken down into its component parts. However, that doesn't mean we wouldn't see similar effects as imagined here if he transferred to another team. Maybe in a different defensive scheme his ability to draw charges isn't as important, or whatever.

One area where I think advanced statisticians lag is in terms of creating a typology of NBA skills, and attempting to measure all of them. Granted some things can't be quantified, or at least, would be very difficult to quantify, for example, who is a better screener, Udonis Haslem or Joakim Noah? How do you even measure that? How do you disentangle ability to screen from ability to finish on the roll or the pop? Do we even want to? It might matter if the ball-handler uses the screens mainly to find his own shot, and as such all we care about is the ability of the screener to create separation between the defender and the ball handler. But even then, the ball handler's timing is just as important as the screener's. And even if we could quantify it, how do we put a value on it? And yet, these things matter when you're trying to construct a team

In that sense, RAPM is very limited, in that even if we accept its findings, it merely tells you to what extent a guy is effective, but not why he is that effective. And when it comes to building a team, the "why" is just as important as the "how" IMO. And I think in general the complaints in this thread from Bulls fans about the conclusions being drawn is that they ignore the "why," when understanding it would (in theory) shed light on why you might prefer the lower RAPM player even if we accept the idea that in actuality Deng's presence on the court has meant more for improving the Bulls' performance than Rose's has.

[coldfish]

OK, next question, how did you adjust for the strength of the opponent?

And that's where ridge regression comes in handy.

At first I wasn't going to respond since you are just being a smartass because you know that isn't an answer. But I'll follow up, what did you run ridge regression ON?

[mysticbb]

So my understanding is that normally RAPM uses an identity matrix (or at least the most simplistic form of it would). While classic APM can be said to have adjusted for opponent strength, usage of the unadjusted identity would re-introduce bias.

The identity matrix means nothing else than just adding lambda whenever the player was on the court. The formula becomes:

Margin = HCA + a_1(lambda)P_1 + a_2(lambda)P_2 + ... - a_6(lambda)P_6 - a_10(lambda)P_10

For a OLS (ordinary least square) lambda is just zero. In both cases it boils down to matrix algebra, once with lambda=0 and once with lambda != 0. Thus, the adjustment for the strength of the opponents is intrinsic. And yes, lambda is introducing a bias here. The bias is:

bias(a_x) = lambda(M^(T)M+lambda E)^(-1)a_x

with M being the matrix, E being the identity matrix and a the coefficients.

We have to understand that there is always a lambda for which the RMSE(ridge)<RMSE(OLS). That is the very basis. Now, when we have that lambda (usually found via cross validation), we just need to run the ridge regression with the said lambda once, basically doing:

a_x = (M^(T)M+lambda E)^(-1)M^(T)R

with R being the response vector (the margin).

So, it makes sense to construct a different matrix which is based on the opponent strength, which I'd imagine would be based on each opponent's RAPM, how often they played against each other, and then iterate etc.

Unless you change the lambda, that will not change the coefficients. What you are proposing is a prior, a vector including some sort of an expected result already. If the prior are the coefficients from the ridge regression of the same dataset it will give out the same coefficients (give or take due to rounding errors of the computer we can't control). The prior = resulting coefficients.

It did also seem to me that coldfish would find it important to factor in changes over the course of a season. My guess is that you'd agree that there is a clear ceiling about what RAPM can do in terms of precision on this front, but that you clearly think don't think that this is nearly as big of a deal as he does given that you chalk up perceived trends to normal quirks of a basketball season.

Yes, no player is static and will hover around a certain value. Obviously, we can't adjust for the mood of that player from game-to-game, we don't have the informations. But for the given exercise the sample size should take care of that.


mopper, you are right, the regression can't tell you why. But that should be clear. It just gives a quantification for the "why". The numbers are worthless without interpretation, that is basically what you are saying. Those results are not intrinsic values of certain players, but the "impact" they had on the result. And somewhat likely they will have that value for a typical future timespan (typical just means according to an aging curve). Why can I say that? Well, I personally believe that coaches aren't complete idiots and would use players completely not to their strength.
But overall regarding your point: You shouldn't expect from a dishwasher that it will drive your kids to school. ;)

Assigning values to the skills of the players might be difficult, but not impossible. But we also have to keep in mind that one aspect is most times different for each player, consistency. The difference between starters and bench players is in many cases just consistency.

The effect you want to describe with your defensive rebounding example is called dimishing returns. The same effect we saw with James and Wade for example. We have to adjust for that, and I agree, that is not an easy task, but again, I would not call that impossible.


Coldfish, the regression is run on the matchup data.

Btw, I'm trying to incoperate a couple of adjustments which should increase the value of "crunch time" play. With that included I get Rose with +2.4 and Deng with +2.1, non-prior informed. The issue is the RMSE became larger (not by much) and the adjustment is arbitrary. Well, it does not look like that crunch time play is something a team should rely on, based on the data. The Dallas Mavericks from last season are obviously speaking against that.

[mopper8]

Yeah, I'm familiar with the concept of diminishing returns. I also appreciate that coaches generally aren't idiots, but I'd be curious, for example, about how JJ Barea's offensive RAPM compares this season to last. Last year he was basically the only Mav who could run a good PnR, but he did it at an elite level. This season, it seems like there are a few guys who can run the PnR for Minny, and what's more, I don't think its as important in Adelman's offense.

edit: I appreciate the thoughtful response

[mysticbb]

but I'd be curious, for example, about how JJ Barea's offensive RAPM compares this season to last. Last year he was basically the only Mav who could run a good PnR, but he did it at an elite level. This season, it seems like there are a few guys who can run the PnR for Minny, and what's more, I don't think its as important in Adelman's offense.

We should keep in mind that Barea is a bench player for the very reason I named: consistency. He is very inconsistent AND also only useful in certain matchups in order to not get completely crushed on the defensive end. One of the reasons the Mavericks in this season have a better defense than last is that Barea's minutes are now taken by a better defender (Carter, Beaubois, West).
I disagree that Barea ran the P&R at an elite level. He done that very well whenever Nowitzki was on the court, he struggled badly without him. In Adelman's offense he is basically just a scorer who is supposed to attack the rim, but he is actually not that great. His offensive value dimished, but that is something we can expect from a roleplayer who is put into a different role. But well, I would suspect Barea would generate the same offense like last season, if he would play the same role. Right now his offensive RAPM dropped from -0.8 to -1.4. Interesting is that he increased his defensive RAPM, and I'm not sure I have any sort of explanation for this. But also see him having not played that many minutes, but maybe he improved his defense? He always tried, but he is really small and made constantly mistakes. His defensive numbers on Synergy are all better than last season. Maybe Adelman is better at picking the right matchups or has the luxury to pick them more than Carlisle?

[mopper8]

I recall seeing on Synergy's free trials during the playoffs last spring that Barea was top-20 in points/possession finishing the PnR as a ball handler last year...though if you have access to Synergy data you might be able to double check that for me. That's why I said he did so at an elite level.

I've never seen any data comparing Barea/Dirk v Barea/Marion or Barea/Chandler or Barea/Haywood PnRs, but the former being by far the most effective would surprise nobody.

However, the spacing effects you get from Dirk in theory ought to be replicated by Love...but, as you pointed out, he's being used differently now, which is hurting his value.

His offensive value dimished, but that is something we can expect from a roleplayer who is put into a different role.

Ultimately, this is what I'm driving at. RAPM to me ultimately is a retrospective statistic, that tells us who did well, but it doesn't necessarily tell us what to expect going forward when guys change teams/roles/etc. Your point about this not necessarily applying to starters or stars is well-taken, though I wouldn't say I agree 100%

[Doctor MJ]

I thought it appropriate to bring up 82games' Opponents PER stat. Here are the leaders for the past 2 years. I'll comment in a separate post:

'11-12: http://www.82games.com/1112/ROLRTG3.HTM
1. Iguodala 7.3
2. Battier 7.4
3. James 9.0
(tie) Wade 9.0
(tie) Brewer 9.0

Bulls other than Brewer:
Deng 9.3
Rose 11.8
Korver 12.0
Gibson 12.0
Noah 17.4
Boozer 18.1
Asik 18.4

'10-11: http://www.82games.com/1011/ROLRTG3.HTM
1. Iguodala 9.8
2. Howard 9.9
3. Deng 10.5
4. James 10.9
5. Kirilenko 11.1
(tie) Brewer 11.1

Bulls other than Deng & Brewer:
Asik 12.3
Korver 14.1
Rose 14.2
Bogans 14.7
Gibson 14.8
Boozer 16.7
Noah 18.6

[Doctor MJ]

So this stat is imperfect for a variety of reasons, two of them:

1) It "blames" a player for what his nominal counterpart is doing, even if that player's role isn't actually to stop that counterpart.

2) PER is biased relative to certain types of players. This affects the stat normally, and affects opponent PER in reverse.

All that amounts to we really need a "degree of difficulty" to go along with this stuff at the very least.

Of course, the fact that Iguodala is the 76er who always takes the hardest job, consistently ranks near the top of the charts, and now has ridiculously good numbers while his team has the #1 defense in the league has to be considered impressive no matter how you slice it. All things considered, he's the guy I'd for DPOY right now.

Getting back to the Bulls, what we see is that the Bulls' perimeter guys known for their defense (Deng & Brewer) look excellent. Oddly though, the Bulls' big do NOT look good at all. This goes far beyond what you'd expect based on degree of difficulty. Playing opposing bigs doesn't stop guys like Howard and Garnett from looking solid on this front.

So the question: If the Bulls really are stellar on defense because of their depth of superb big man defenders, and guys like Deng & Brewer are the less important defenders, what is causing these stats to look so wacky?

[mopper8]

In theory, Bulls great D could be a combo of stellar perimeter defenders + mobile bigs who give good help, are strong on the PnR, and control the defensive glass, while being susceptible to higher scoring bigs.

[coldfish]

Coldfish, the regression is run on the matchup data.

What match up data? In order for you to run any statistical analysis, you have to boil down the match up down to numbers. What numbers do you use?

.....

@DoctorMJ:

Here is the Bulls defense in a nutshell:
- Perimeter defender on ball shields and forces his man baseline and tries to prevent the guy from crossing from one side to the other.
- Big men help early when perimeter defenders go baseline to form a double team outside the lane near the baseline.
- Perimeter defenders on the weak side of the court cheat into the lane to prevent passes to the uncovered offensive players
- Recover as quickly as possible
- If a big man gets the ball in the post, DO NOT HELP. Let the guy go one on one.

Basically, the big men are extremely important, but their impact is best seen on what they do to opposing perimeter players. The Bulls intentionally and repeatedly hang their big men out to dry.

[Doctor MJ]

Excellent stuff. Thank you coldfish (and mop!).

So would you say that the Bulls' defense is designed with the idea that since the rest of the league now rarely uses big men as volume scorers, it makes sense to systematically cheat to help contain perimeter players?

[RichardsRival3]

I hope you are watching Deng running the offense.

long-range J after long-range J

MVP my butt.

[Rerisen]

I hope you are watching Deng running the offense.

long-range J after long-range J

MVP my butt.

Against a good defense, exactly to expectation. It's a one man show out there for the most part. Deng gets some open threes, but all because of Derrick (same as the previous game in San Antonio). Without him there, neither are those quality shots.

Bulls need to get Rip in there and see how he mixes things up late.

Wayne Winston was mentioned earlier in the thread his site frequently lists the 'Impact' statistic, putting higher appreciation on crunch time. Rose was big in it last year, I think the leader in the East at the AS Break. Haven't seen it for this year. But Rose's importance in that time of the game is the same, even though they are winning a little bigger in general.

[lorak]

1) It "blames" a player for what his nominal counterpart is doing, even if that player's role isn't actually to stop that counterpart.

And even if he isn't guarding that counterpart. Because as far as I know 82games data is based on play by ply so there's no way they know who exactly guarded who on every possession.

[mysticbb]

I thought it appropriate to bring up 82games' Opponents PER stat. Here are the leaders for the past 2 years. I'll comment in a separate post:

Two issues: 1st, we don't know whether player X really guarded "his" counterpart on the given possessions. 2nd, I would really like to see the average production of the counterpart against an average defender in comparison.

The first issue can be solved by using Synergy Stats. For example: Deng 0.7 ppp (ranked 26th overall), Iguodala 0.73 ppp (ranked 45th).

What match up data? In order for you to run any statistical analysis, you have to boil down the match up down to numbers. What numbers do you use?

I described it now multiple times in this thread, no idea why you have to ask again. But anyway ...
The lineup matchups from each game. Everytime a new player comes in, a new line in the matchup file is created. We have the data of 5 home players vs. 5 away players. The basic formula is:

Margin = HCA+a_1P_1+a_2P_2+a_3P_3+a_4P_4+a_5P_5-a_6P_6-a_7P_7-a_8P_8-a_9P_9-a_10P_10

where Margin is the scoring margin adjusted to 100 possessions, HCA is the intercept, a_1 to a_5 the coefficients for the home players, a_6 to a_10 the coefficients for the away players. Obviously, the index runs from 1 to ca. 450 (one index for each player). The regression finds the best fitting coefficients in order to approximate Margin. That boils down to matrix algebra. For the ridge regression a lambda is added and the coefficients are depending on the lambda now. For each lambda you get a different set of coefficients. It was shown that there is always a lambda for which RMSE (ridge) < RMSE (OLS). The best fitting lambda can be found via crossvalidation.

Rerisen, the "impact" by Wayne Winston says a similar thing about Rose as my test with an increased weight for minutes with low margin and less minutes to play. The issue is: Winston is using OLS, which means he is running into some big problems with overfitting while also reducing the sample size in order to solve multicollinearity issues. And I have not found that the model becomes better at predicting the outcome of games, if the crunch times is weighted more heavily. I have to admit, that I don't know the exact mathematical theory behind Winston's "impact" while I choose a arbitrary baysian formula to increase importance of a game snippet with less minutes left in the game and with a closer result. As I pointed out for this season Rose gets a higher value in that model than Deng. So, there is proof for our idea that Rose is more valuable in crunch time.

[mysticbb]

Ultimately, this is what I'm driving at. RAPM to me ultimately is a retrospective statistic, that tells us who did well, but it doesn't necessarily tell us what to expect going forward when guys change teams/roles/etc. Your point about this not necessarily applying to starters or stars is well-taken, though I wouldn't say I agree 100%

Every stats is retrospective, that is not an issue alone for RAPM. But it can be shown that RAPM is better as a predictor than basically all other stats. And yes, the issue about the role becomes less important, if the players are staying within that role. If a player plays similar minutes in a similar role, without running into those dimishing returns, the value should stay rather constant.

Regarding Barea: The data clearly shows that with Nowitzki on the court Barea became a better player (in terms of production and efficiency). For Love that effect is hardly seen, Barea is a bit more efficient, but his production is less with Love on the court. Maybe the effects will equal out over the course of the season. We have to keep in mind that Barea had to deal with injuries, and it was previously shown in this thread that injuries have an effect on the performance level of a player.

[drza]

What kind of data is not available? I have a file with all different lineup matchups for the whole season. Why should I not have the ability to use that in order to adjust for opponent's strength?

Slightly OT, but a couple of questions:

1) Where did you get that file? Is it publicly available? If not, would you be willing to share?

2) How many seasons do you have? If multiple, would you be willing to share those?

I ask because I've been flirting for awhile now with doing some more in-depth calculations of my own, but the thought of trying to manually put together such a file myself from ESPN over multiple years would be time/labor intensive enough to make it impossible to do in the short term. So I would really appreciate any help you'd be willing to give on the subject.

(I wish I could have PMed you this).

[mysticbb]

1) Where did you get that file? Is it publicly available?

http://basketballvalue.com/downloads.php

From 2006 to now (excluding 2006 playoffs)


http://stats-for-the-nba.appspot.com/PBP/all.rar

2003 to 2005 + 2002 and 2006 playoffs + a couple of games in 2002

b-r.com has 2001 and 2002 completely, but is not willing to share for free. Wayne Winston has 1999 and 2000, but also not willing to share.

Btw, I have a Python based parser for ESPN and NBA pbp data. Not that difficult to write.