Flaws of Berri's Wins Produced Metric

[Witzig-Okashi]

Before stumbling to RealGM's basketball forum at a random Google Search, I use to follow the Wages of Wins crew and Berri's Wins Produced Metric.

http://wagesofwins.com/how-to-calculate-wins-produced/

I don't follow it anymore, but it isn't because I lost faith in WP per se, my interest in the site just waned. I've heard and read criticisms of the formula before, but I wasn't very open-minded to it (I think the last one I read was about Jose Calderon, and that was almost a year ago).

Does it value rebounding too much (players like Rodman and Wallace were rated highly on some of their lists, and that was some of the criticism some received)?

Is there any other flaws in it that I'm unaware of? Perhaps it's comparison to WS, PER, or some other stat of that ilk? I'm curious....

[blabla]

It's got a wide variety of flaws, ranging from
- assuming that values of certain team statistics are the same on the player level
- failing to realize that defenders that make offensive players miss shots should get some of the credit (and not just the guy who got the defensive rebound). They made some changes to this but probably not enough
- failing to understand that your metric should be able to predict well, and not just correlate with data from the past

That being said, I don't think PER is that much better as a metric, as it has it's own flaws (arbitrary weights, for one). WS also uses arbitrary weights, but at least these are closer to what the 'good' BoxScore metrics (i.e. ASPM) have found they should be

[Witzig-Okashi]

It's got a wide variety of flaws, ranging from
- assuming that values of certain team statistics are the same on the player level
- failing to realize that defenders that make offensive players miss shots should get some of the credit (and not just the guy who got the defensive rebound). They made some changes to this but probably not enough
- failing to understand that your metric should be able to predict well, and not just correlate with data from the past

These were probably one of the notable criticisms that I remember seeing now. I think one of the complaints was related to having guys like Wallace and Rodman rated over Jordan. The only guard that appeared multiple times like the bigs was 6-9 Magic Johnson, based on one of their WP-related metrics. http://wagesofwins.com/2011/08/13/defending-rodman-featuring-the-100-most-productive-seasons-since-1978/

That being said, I don't think PER is that much better as a metric, as it has it's own flaws (arbitrary weights, for one). WS also uses arbitrary weights, but at least these are closer to what the 'good' BoxScore metrics (i.e. ASPM) have found they should be

WP supporters usually took shots at Hollinger and PER when I would visit WoW and the like; I don't generally like to mention PER myself (and also use of blocks and steals/too offense-reliant), and it's probably why I became a fan of WP to begin with. Probably a knee-jerk reaction...

[Dr Positivity]

WP is based on the belief players production depends on them and not because of teammates. Essentially if basketball was as an individually driven as baseball, WP would be on the money for evaluating players. But since in the NBA player production is dependent on teammates there is holes in WP. For example boxscoregeeks.com posted an article about LMA's offense being overrated, as a high volume weak efficiency player.. While on paper league high efficiency at a low volume is more valuable than below average efficiency at a high volume, the reality is there is teammate effect changing player value. LMA draws double teams and floor stretches, which gets his teammates more open and makes them more efficient. The opposite of him is a player like Deandre Jordan who's low volume, elite efficiency WP typically loves (he overall rates as one of the best players in the league by WP). But unlike LMA he does not make his teammates more open with doubles or floor spacing, more like his teammates get him open. So understanding why LMA is more valuable than Deandre offensively is something WP has a hard time calculating, because it doesn't happen statistically. PER tries to account for this by simply giving players extra value for high volume scoring and obviously there are +/- enthusiasts, but each have their own set of concerns.

I think WP ultimately gets a little too much hate, it is not a perfect stat but it's a start and step in the right direction - and its notoriety has ultimately played its part in people adopting efficiency over volume scoring. I feel like the right approach for them was to say "WP is an output, but the input is dependent on players" - Therefore in the case of Deandre they could've been saying "On paper Deandre's stats are incredibly valuable to the Clippers, which is in part a credit to the offensive presence Chris Paul and Blake Griffin provide" etc. etc., but they are fairly committed to the input=output concept. I do feel like Arturo and Alvarez are two of the better twitter follows and they typically have a charismatic presentation on their site. I also feel like separating from the Berri Wages of Wins site may do them good since Berri always struck me as the one with the arrogance problem and a guy who got acted with rage when people questioned his precious. Also finally for people who are interested in the analytics approach to the NBA Draft, Arturo's yearly model is about as good as it gets in that department. Since drafting analytics sites leaning on indicators like steals, blocks, rebounds, etc. and adjusting for age/competition, is right in line with the regression style analysis WP is based on

[Witzig-Okashi]

WP is based on the belief players production depends on them and not because of teammates. Essentially if basketball was as an individually driven as baseball, WP would be on the money for evaluating players. But since in the NBA player production is dependent on teammates there is holes in WP. For example boxscoregeeks.com posted an article about LMA's offense being overrated, as a high volume weak efficiency player.. While on paper league high efficiency at a low volume is more valuable than below average efficiency at a high volume, the reality is there is teammate effect changing player value. LMA draws double teams and floor stretches, which gets his teammates more open and makes them more efficient. The opposite of him is a player like Deandre Jordan who's low volume, elite efficiency WP typically loves (he overall rates as one of the best players in the league by WP). But unlike LMA he does not make his teammates more open with doubles or floor spacing, more like his teammates get him open. So understanding why LMA is more valuable than Deandre offensively is something WP has a hard time calculating, because it doesn't happen statistically. PER tries to account for this by simply giving players extra value for high volume scoring and obviously there are +/- enthusiasts, but each have their own set of concerns.

I think WP ultimately gets a little too much hate, it is not a perfect stat but it's a start and step in the right direction - and its notoriety has ultimately played its part in people adopting efficiency over volume scoring. I feel like the right approach for them was to say "WP is an output, but the input is dependent on players" - Therefore in the case of Deandre they could've been saying "On paper Deandre's stats are incredibly valuable to the Clippers, which is in part a credit to the offensive presence Chris Paul and Blake Griffin provide" etc. etc., but they are fairly committed to the input=output concept. I do feel like Arturo and Alvarez are two of the better twitter follows and they typically have a charismatic presentation on their site. I also feel like separating from the Berri Wages of Wins site may do them good since Berri always struck me as the one with the arrogance problem and a guy who got acted with rage when people questioned his precious. Also finally for people who are interested in the analytics approach to the NBA Draft, Arturo's yearly model is about as good as it gets in that department. Since drafting analytics sites leaning on indicators like steals, blocks, rebounds, etc. and adjusting for age/competition, is right in line with the regression style analysis WP is based on

I've come to a bit of a head-scratching moment when evaluating both LMA and DeAndre. LMA isn't shooting the basketball like a Dirk Nowitzki, but the way he creates opportunities and spreads the floor puts you in the mindset of Dirk. In spite of DeAndre Jordan's improvements, I still not particularly sold on him as a DPOY caliber player, but that could be me still lingering on to old bad habits of his.

I guess since DeAndre is leading the league in rebounding, it would play a role in him being favored in WP. I think I've seen somebody make a case for Andre Drummond being the ROY based on WP on a site before (I need to find that link), but it wasn't on the WoW site. It seems that WP takes into account efficiency to the point that it doesn't account for how the rated players are acquiring those points. But at the same time you don't want to overrate players who are able to create their own offense. Perhaps that's what WP is trying to eliminate, but you assessment is more detailed than mine.

I do like Arturo's approach to the NBA draft, too. I'm not as analytically inclined as others, though, despite liking statistics.....I'm sure I'll check it out if time permits during July.....

[Doctor MJ]

Before stumbling to RealGM's basketball forum at a random Google Search, I use to follow the Wages of Wins crew and Berri's Wins Produced Metric.

http://wagesofwins.com/how-to-calculate-wins-produced/

I don't follow it anymore, but it isn't because I lost faith in WP per se, my interest in the site just waned. I've heard and read criticisms of the formula before, but I wasn't very open-minded to it (I think the last one I read was about Jose Calderon, and that was almost a year ago).

Does it value rebounding too much (players like Rodman and Wallace were rated highly on some of their lists, and that was some of the criticism some received)?

Is there any other flaws in it that I'm unaware of? Perhaps it's comparison to WS, PER, or some other stat of that ilk? I'm curious....

First and foremost the issue with the stat is Berry and how he promotes it. He trumpets it as a holy grail bringing on a revolution far beyond what anyone else has thought of, when the reality is it's just another stat. It's not a useless stat, but there are other stats you can use for the same purpose, and there are legit flaws in the calculation process that have been pointed out and which I've never seen him address - and to be clear he's made a real show of avoiding answering the questions.

Specific flaws I see:

1. It's predominantly a box score based stat. Those are useful, but only as useful as the box score is complete. Not Berry's fault that the box score isn't complete, but irresponsible to be too cocky given those limitations.

2. Berri's correction for this is to make an adjustment based on the team's winning. This isn't inherently a bad thing to do, but it's a broad strokes method. You're not going to get amazing precision with it, and you shouldn't pretend you are.

3. Berri's argument for his stats' superiority is based on correlation with winning...but he literally uses that correlation as part of the stat. A poster here (mysticbb), demonstrated you could make a stat with great correlation using basic stuff like minutes played. The specific issue here has a technical term and it's called overfitting. It's a very well known term in machine learning circles. Whenever I've seen Berri asked about overfitting, he never responds. It eventually got to the point that I concluded he might not know what it is, which still blows my mind, but of course he doesn't come from machine learning, he comes from economics, so maybe that's the issue.

On that note, as someone with a machine learning background, seeing him play the ivory tower card with his economics background always seems weird to me. It ain't like the people in the computer science department are going around asking economists for help on this stuff, it's always been the other way around.

Further: Him playing the ivory tower card while being a prof at CSU Bakersfield in a debate with people with degrees from Caltech and the like has always blown my mind. You can do great things anywhere - he doesn't lose the debate because he worked in Bakersfield - but him implying that he's more established than these other people simply because he's paid by a school which gives tenure based largely on teaching is bizarre.

4. His criticisms of any use of +/- stats are weird also. It's fine to point out the noise in +/- stats, but the more established guys in the NBA consulting area used them in comparing box score methods as a barometer, which very clearly dominates over Berri's choice of barometer (raw team wins and losses) in terms of noise.

Okay, now what does that actually come down to in the results? What do I "auto-correct" when I see his data?

The offensive rebounding is the big thing. He gives huge weight to it, and it very much differs from other approaches people have used to use regression analysis to find it's weight, to say nothing of common sense, or the fact that team's have largely decided offensive rebounding is LESS valuable than they'd previously thought it was given it's negative effect on transition defense.

[Witzig-Okashi]

I used to be under the impression that Berri's detractors were just offended by of his prickish behavior that you mentioned in your post. Especially those defending Hollinger and PER.

First and foremost the issue with the stat is Berry and how he promotes it. He trumpets it as a holy grail bringing on a revolution far beyond what anyone else has thought of, when the reality is it's just another stat. It's not a useless stat, but there are other stats you can use for the same purpose, and there are legit flaws in the calculation process that have been pointed out and which I've never seen him address - and to be clear he's made a real show of avoiding answering the questions.

The offensive rebounding is the big thing. He gives huge weight to it, and it very much differs from other approaches people have used to use regression analysis to find it's weight, to say nothing of common sense, or the fact that team's have largely decided offensive rebounding is LESS valuable than they'd previously thought it was given it's negative effect on transition defense.

That makes sense then. Moses Malone was rated rather high on some of the lists involving WP, too (If I can find those links)....
Though I'm confused about his distaste for +/- in terms of impact. I certainly has its flaws, but I do think it can be easier to minimize if you analyze surrounding teammates with the same metric, take account of the style/pace of the team(s), and so forth. It seems to be easier to mend errors with that rather than OP.

One example of the WP starting to become puzzling for me was it reasoning for Andre Drummond as RPOY (I seem to have contradicted myself regarding this article, thought I saw it on another site)
http://wagesofwins.com/2013/04/18/no-damian-lillard-isnt-the-rookie-of-the-year/. I didn't think much of it at the time, but I went right along with the "scoring is overrated" aspect of award winning.

Of course, Davis would have won ROY if he was healthy (I certainly thought so), but it compared Drummond's impact to James and Paul based on WP, but it just seemed missing at the time. Me, Drummond has his moments, but he's still very raw, and his blocks and steals can be misleading for his lack of consistent defensive awareness. I don't think rebounding tells the entire story, but I digress...

[Doctor MJ]

That makes sense then. Moses Malone was rated rather high on some of the lists involving WP, too (If I can find those links)....
Though I'm confused about his distaste for +/- in terms of impact. I certainly has its flaws, but I do think it can be easier to minimize if you analyze surrounding teammates with the same metric, take account of the style/pace of the team(s), and so forth. It seems to be easier to mend errors with that rather than OP.

One example of the WP starting to become puzzling for me was it reasoning for Andre Drummond as RPOY (I seem to have contradicted myself regarding this article, thought I saw it on another site)
http://wagesofwins.com/2013/04/18/no-damian-lillard-isnt-the-rookie-of-the-year/. I didn't think much of it at the time, but I went right along with the "scoring is overrated" aspect of award winning.

Of course, Davis would have won ROY if he was healthy (I certainly thought so), but it compared Drummond's impact to James and Paul based on WP, but it just seemed missing at the time. Me, Drummond has his moments, but he's still very raw, and his blocks and steals can be misleading for his lack of consistent defensive awareness. I don't think rebounding tells the entire story, but I digress...

Berri isn't alone in his dismissal of +/-. What I found when I first went over to the APBRmetrics board - god, about 9 years ago - is that a lot of the folks there really thought they could boil the game down to the box score. The combination of that and the inconsistent appearances of the guys who clearly knew more than me (Oliver, Rosenbaum, Ilardi, etc) made it a place I've never been able to completely tolerate as a regular even though it is clearly the #1 stats board on the 'net well above RealGM's stat board. Of course, Berri's a much bigger name than those other fundamentalists.

What's also vexing about Berri's stance though is how much faith he has in his winning-based correlation. If we look at winning-based analysis as a series of steps from broad to precise, basically nobody serious sticks at the "Level 1" level of siding with whoever won. "Level 2" in this type of "impact on team" analysis is raw +/- data, where a player doesn't get penalized or rewarded for what happens when he's off the court. "Level 2" is not nearly good enough of course, but it's quite clearly better than insisting on using "Level 1" in any way, shape, or form...and yet Berri insists on using "Level 1" as his barometer. This doesn't make him the most foolish guy out there, given that some guys are at the "Level 0" where they rely solely on box score stats, but it's really weird to see anyone reach "Level 1" and absolutely loyal to it when they have academic training. If we stick with this level-speak, we're at the very least to "Level 4" winning-based analysis now, and the flaws in it remain obvious to anyone who understands it.

I didn't know about the Drummond pick, but that's classic Berri for ya. Clearly I don't need to convince you that at the very least there's more to it than can be concluded by WP, but for those following let me point out:

1) Drummond played largely against backups his rookie year. it's hard for me to fathom even considering giving such a player the nod over a guy playing against starter. One can argue that he maybe WOULD have been the ROY if he'd be given the right opportunity, but he wasn't so he isn't.

2) It's noteworthy with Drummond how much of his early success is about doing basically just one thing well: He can get offensive boards, and put them in the hoop. That's not nothing, but it's also not a complete player. A guy who can't be relied upon to make the right decisions on defense, and is crippled on offense if he gets the ball for more than a brief moment, is basically not someone a contender could afford to use in any kind of serious situation. And if your all-in-one stat says he looks amazing because of how good he is at that one basic action, well that only tells you how much the all-in-one stat can't tell you.

[Doctor MJ]

ftr, I should note: I consider ROY an incredibly complex award that most people think is really simple.

While the criteria for MVP is nominally left loose, there's still the matter that the award specifies something clear: Player of Most Value. People who choose to ignore that are truly either rebels or fools.

There is no Most Valuable Rookie award. One can argue how prescient they were in their name, but if we presume that they did it with intention, I think it was very smart. I think that the reality is that in many years the Most Valuable Rookie is a role player who happened to end up in the right place, and that's not something anyone cares enough about to have an award for.

(Here's where it should be noted that many fool themselves into thinking that the Most Valuable Rookie is the guy they vote for for ROY. What we see though with regression analysis is that star rookies typically don't transform their teams. Even when it appears so based on a team tanking the prior year, the detailed analysis shows the team's improvement over the prior year remains with or without their new star.)

We have a ROY - and we don't have "2nd Years of the Year" or "8th Years of the Year" - because people want to know who the next star is, and I think they do a pretty good job of getting that right. Typically the rookie with the gaudiest stats IS the guy who has demonstrated the most star-like impression, so the ROY goes to that future star.

There is of course the fact that sometimes the voters use this to pick a guy with lower ceiling because he seems more ready right now. To some degree we can account for this in terms of who has "proven" the most star-like, as opposed to a voting that simply guesses who will end up the best. Hence a guy like Giannis this year isn't likely to win ROY even if many GMs would now pick him as the player most likely to be an NBA star. Seems reasonable.

But there are times where the fact that people don't realize the complexities before them leads them into incoherencies, and you can see this sometimes when people consider injuries.

There were people last year who chose Lillard over Davis because of the injuries, and this is absurd. It would make sense if this were a Most Valuable Rookie debate, but it really isn't, and those 18 games Davis missed most certainly weren't enough that people should have felt like they didn't have a grasp on how Davis would handle the NBA. Davis had the superior PER - a stat biased against him, had the superior WS - despite missed time, and everyone agreed his ceiling was higher, so what exactly were people anointing Lillard for?

(Granted some people would rank Lillard higher because they overrate offense, obviously that's a real component of this, and an entirely separate issue.)

Of course as I say this, and essentially roast those picking Lillard for ROY over Davis, a couple items are worth bringing up:

1) If I hold it against Drummond that he didn't play more, why is Davis immune? Forgetting games played, the MPG gap between Davis & Drummond is smaller than Lillard & Davis.

Answer: It's not just about the MPG numbers, but there is a tipping point where it becomes undeniable that the player's team doesn't think the player's skills are ready for prime time. When a guy plays 28 MPG like Davis in a year where he has to recover from injury, it doesn't necessarily say anything about his perceived limitations. The same cannot be said for someone like Drummond.

2) I'll put this in question form: "Doc, do you truly believe that rookie value isn't a separate factor at all? If one earns some MVP consideration, wouldn't that clinch ROY even if teams consider another rookie a much better prospect going forward?"

Me: "You're right".

To go further: This is part of the complexity of the situation. I boil it down as best I can, but because of the confusion of the voters, it's a bit like trying to tuck in that last corner of the bed.

All I can say is that when we see this behavior it is irrational. As noted, "Rookie of the Year" is essentially an award that gets it's importance based on how bad the eligible players are for the time being. There's little point to interpreting that as "The Most Valuable Player among the players who don't yet know how to contribute much value".

[mysticbb]

WP suffers from two flawed assumptions. Berri never showed in any of his papers/books that both of those assumptions are true. First one: that his formula for PE is actually able to count individual possessions used. Second one: That the individual boxscore entries have a marginal value which can be derived via linear regression.

The first one is easy to show that it is not true. We just take the example of a player getting an offensive rebound and making a field goal off a putback. In that case the formula for offensive efficiency (hereafter Off eff) by Dave Berri becomes:

PE = FGA+0.45*FTA+TO-ORB

Off eff = PTS/PE

Off eff = 2/1+0.45*0+0-1
Off eff = 2/0

That is not defined. We just created an example in which Berri's formula is not defined for individual players. Therefore the formula is invalid.

That is an important issue here, because the value of players is supposed to be determined by that, while it is essentially possible to score points without using a possession according to Berri's formula. This is also the main reason Drummond got such high value despite being not that efficient. Going by Berri's formula Drummond had a 158 ORtg last season, and 182 ORtg this season. That is obviously absurd. In comparison, the more efficient scorer and less turnover-prone Nowitzki comes up with a 114 ORtg, a way worse offensive player according to this. Using Oliver's formulas for individual possession gives us 119 ORtg for Drummond and 120 ORtg for Nowitzki. Makes much more sense, especially when we consider that in Oliver's approach the fact that Nowitzki has a higher usage is a positive for him, while in Berri's approach usage is obsolete. The reason: one offensive rebound decreased the amount of used possession by one. That would only be a useful approach, if an offensive rebound can be looked at as an isolated event, meaning the offensive rebounds just falls from the sky without any teammate having to do anything with it. That is obviously not true, because for an offensive rebound to be possible, someone has to take and miss a shot. In Berri's formula taking a shot is as bad as turning the ball over, which is again bs, because the chance of scoring in a possession after a turnover is 0, while in average a team still scores about 0.4 points per miss. Avoiding a turnover as an additional result of a FGA doesn't exist in Berri's approach, even though it can be shown that higher usage players are exactly doing that: decreasing the turnover rate for the overall team. Well, Berri also dismissed the usage vs. efficiency analysis by Eli Witus (which the statement is essentially based on), the current vice president of basketball operations and former head of stats for the Houston Rockets. (And that is not the only guy currently working for an NBA team Berri just dismissed; we can essentially count Barzilai, Nichols, Hollinger, Rosenbaum and a couple of more, where Witus, Barzilai, Rosenbaum and Nichols are "+/- guys").

The second one is actually disproven by Berri himself, he just doesn't realize it. Using the marginal values he calculated via regression alone gives much more value to bigs than to smalls. Which either means: those big players are per se much more valuable than smalls, or the values can't be derived via regression. Berri now says that a team needs to have those small players, which would mean in turn the only scientific reasonable conclusion: the values are bogus and linear regression can't be used to derive the marginal value of those boxscore entries. Well, Berri doesn't care about scientific methods at all, and just decided to make a positional adjustment, in that way Berri actually believes that scoring points has a different value for a team, depending on the position of the player. Forget that it is not that easy to define the position for each player, given the fact that there is overlapping and players may play different position on offense and defense, but you need to understand that according to WP making a 3 pointer is actually more valuable from a SF than from a PF, while basically all impact studies show that a stretch 4 has more impact by drawing out the interior defense than a SF, who is supposed to be on the perimeter anyway.

Overall it is also funny to note that Berri's approach leads to the fact that scoring has no value in average for a NBA player.

That Berri's justification for his method is insufficient was shown here on RealGM before. I simply used a model based on scoring per 100 possession and a defensive adjustment and showed that it has a higher correlation to wins than WP and is more stable from season to season. The underlying theory of scoring plus defense = wins is sound as well. So, it completely fulfills Berri's supposed needed requirement for a "scientific metric". It is obviously bogus to reduced the player's individual contribution to scoring alone while establishing a good correlation to winning via a defensive adjustment based on the team defensive result, but that is as much valid as eliminating scoring as value while doing a defensive adjustment to get a high correlation to winning.

I also showed that WP is less able to predict lineup performances than Win Shares or my own boxscore metric, and is on par with PER in that regard. Other tests showed that WP can't serve as a better predictor for out of sample data than PER, and is worse than Win Shares or ASPM by dsmok.

[Witzig-Okashi]

WP suffers from two flawed assumptions. Berri never showed in any of his papers/books that both of those assumptions are true. First one: that his formula for PE is actually able to count individual possessions used. Second one: That the individual boxscore entries have a marginal value which can be derived via linear regression.

The first one is easy to show that it is not true. We just take the example of a player getting an offensive rebound and making a field goal off a putback. In that case the formula for offensive efficiency (hereafter Off eff) by Dave Berri becomes:

PE = FGA+0.45*FTA+TO-ORB

Off eff = PTS/PE

Off eff = 2/1+0.45*0+0-1
Off eff = 2/0

That is not defined. We just created an example in which Berri's formula is not defined for individual players. Therefore the formula is invalid.

That is an important issue here, because the value of players is supposed to be determined by that, while it is essentially possible to score points without using a possession according to Berri's formula. This is also the main reason Drummond got such high value despite being not that efficient. Going by Berri's formula Drummond had a 158 ORtg last season, and 182 ORtg this season. That is obviously absurd. In comparison, the more efficient scorer and less turnover-prone Nowitzki comes up with a 114 ORtg, a way worse offensive player according to this. Using Oliver's formulas for individual possession gives us 119 ORtg for Drummond and 120 ORtg for Nowitzki. Makes much more sense, especially when we consider that in Oliver's approach the fact that Nowitzki has a higher usage is a positive for him, while in Berri's approach usage is obsolete. The reason: one offensive rebound decreased the amount of used possession by one. That would only be a useful approach, if an offensive rebound can be looked at as an isolated event, meaning the offensive rebounds just falls from the sky without any teammate having to do anything with it. That is obviously not true, because for an offensive rebound to be possible, someone has to take and miss a shot. In Berri's formula taking a shot is as bad as turning the ball over, which is again bs, because the chance of scoring in a possession after a turnover is 0, while in average a team still scores about 0.4 points per miss. Avoiding a turnover as an additional result of a FGA doesn't exist in Berri's approach, even though it can be shown that higher usage players are exactly doing that: decreasing the turnover rate for the overall team. Well, Berri also dismissed the usage vs. efficiency analysis by Eli Witus (which the statement is essentially based on), the current vice president of basketball operations and former head of stats for the Houston Rockets. (And that is not the only guy currently working for an NBA team Berri just dismissed; we can essentially count Barzilai, Nichols, Hollinger, Rosenbaum and a couple of more, where Witus, Barzilai, Rosenbaum and Nichols are "+/- guys").

Yes. That was also a potentially big problem I had with WP. Although I think it's admirable that there is a metric that tries to attempt to measure impact of players that do work for rebounds, and necessarily need not to be 'on-the-ball players', it's flaw is that it rules out players who aren't playing the 4 and the 5 (they could get their points off of offensive rebounds by capitalizing off of their teammates misses).

The second one is actually disproven by Berri himself, he just doesn't realize it. Using the marginal values he calculated via regression alone gives much more value to bigs than to smalls. Which either means: those big players are per se much more valuable than smalls, or the values can't be derived via regression. Berri now says that a team needs to have those small players, which would mean in turn the only scientific reasonable conclusion: the values are bogus and linear regression can't be used to derive the marginal value of those boxscore entries. Well, Berri doesn't care about scientific methods at all, and just decided to make a positional adjustment, in that way Berri actually believes that scoring points has a different value for a team, depending on the position of the player. Forget that it is not that easy to define the position for each player, given the fact that there is overlapping and players may play different position on offense and defense, but you need to understand that according to WP making a 3 pointer is actually more valuable from a SF than from a PF, while basically all impact studies show that a stretch 4 has more impact by drawing out the interior defense than a SF, who is supposed to be on the perimeter anyway.

Overall it is also funny to note that Berri's approach leads to the fact that scoring has no value in average for a NBA player.

That Berri's justification for his method is insufficient was shown here on RealGM before. I simply used a model based on scoring per 100 possession and a defensive adjustment and showed that it has a higher correlation to wins than WP and is more stable from season to season. The underlying theory of scoring plus defense = wins is sound as well. So, it completely fulfills Berri's supposed needed requirement for a "scientific metric". It is obviously bogus to reduced the player's individual contribution to scoring alone while establishing a good correlation to winning via a defensive adjustment based on the team defensive result, but that is as much valid as eliminating scoring as value while doing a defensive adjustment to get a high correlation to winning.

I also showed that WP is less able to predict lineup performances than Win Shares or my own boxscore metric, and is on par with PER in that regard. Other tests showed that WP can't serve as a better predictor for out of sample data than PER, and is worse than Win Shares or ASPM by dsmok.

So, in layman's terms, Berri's metric is flawed in being half-baked. It tries to attempt to use a metric that takes into account a perceived flaw in analysis of an indiviual players' impact on a victory by broadening the horizons, yet it falls victim to what it attempts to avoid by only taking into account the box-score stats. Am I correct?

[mysticbb]

So, in layman's terms, Berri's metric is flawed in being half-baked.

Indeed. If he would use the individual possession counting by Oliver instead of his PE, he would arrive at a much better result. Or he could simply start with the premise that all positions are equal and arrive at marginal values which are appropriate for all boxscore entries without having to apply a positional adjustment. Either way is much more consistent than what Berri did.

It tries to attempt to use a metric that takes into account a perceived flaw in analysis of an indiviual players' impact on a victory by broadening the horizons, yet it falls victim to what it attempts to avoid by only taking into account the box-score stats. Am I correct?

Not really. You can arrive at a much better solution while only using the boxscore stats. That is not the issue here. It is more inline with a flawed understanding of science on his part as well as the overfitting issue which DocMJ mentioned. By actually trying to replicate the results from the past season as good as possible, he uses data not specifically meant to describe individual players (the defensive adjustment) as well as applying the positional adjustment to cover up the flawed basic assumption that PE would describe the possessions used for individual players in a useful way. The thing is: Berri's approach is inconsistent.

And then we have Berri's personality ...

[Witzig-Okashi]

ftr, I should note: I consider ROY an incredibly complex award that most people think is really simple.

While the criteria for MVP is nominally left loose, there's still the matter that the award specifies something clear: Player of Most Value. People who choose to ignore that are truly either rebels or fools.

There is no Most Valuable Rookie award. One can argue how prescient they were in their name, but if we presume that they did it with intention, I think it was very smart. I think that the reality is that in many years the Most Valuable Rookie is a role player who happened to end up in the right place, and that's not something anyone cares enough about to have an award for.

(Here's where it should be noted that many fool themselves into thinking that the Most Valuable Rookie is the guy they vote for for ROY. What we see though with regression analysis is that star rookies typically don't transform their teams. Even when it appears so based on a team tanking the prior year, the detailed analysis shows the team's improvement over the prior year remains with or without their new star.)

We have a ROY - and we don't have "2nd Years of the Year" or "8th Years of the Year" - because people want to know who the next star is, and I think they do a pretty good job of getting that right. Typically the rookie with the gaudiest stats IS the guy who has demonstrated the most star-like impression, so the ROY goes to that future star.

There is of course the fact that sometimes the voters use this to pick a guy with lower ceiling because he seems more ready right now. To some degree we can account for this in terms of who has "proven" the most star-like, as opposed to a voting that simply guesses who will end up the best. Hence a guy like Giannis this year isn't likely to win ROY even if many GMs would now pick him as the player most likely to be an NBA star. Seems reasonable.

But there are times where the fact that people don't realize the complexities before them leads them into incoherencies, and you can see this sometimes when people consider injuries.

There were people last year who chose Lillard over Davis because of the injuries, and this is absurd. It would make sense if this were a Most Valuable Rookie debate, but it really isn't, and those 18 games Davis missed most certainly weren't enough that people should have felt like they didn't have a grasp on how Davis would handle the NBA. Davis had the superior PER - a stat biased against him, had the superior WS - despite missed time, and everyone agreed his ceiling was higher, so what exactly were people anointing Lillard for?

(Granted some people would rank Lillard higher because they overrate offense, obviously that's a real component of this, and an entirely separate issue.)

Of course as I say this, and essentially roast those picking Lillard for ROY over Davis, a couple items are worth bringing up:

1) If I hold it against Drummond that he didn't play more, why is Davis immune? Forgetting games played, the MPG gap between Davis & Drummond is smaller than Lillard & Davis.

Answer: It's not just about the MPG numbers, but there is a tipping point where it becomes undeniable that the player's team doesn't think the player's skills are ready for prime time. When a guy plays 28 MPG like Davis in a year where he has to recover from injury, it doesn't necessarily say anything about his perceived limitations. The same cannot be said for someone like Drummond.

2) I'll put this in question form: "Doc, do you truly believe that rookie value isn't a separate factor at all? If one earns some MVP consideration, wouldn't that clinch ROY even if teams consider another rookie a much better prospect going forward?"

Me: "You're right".

To go further: This is part of the complexity of the situation. I boil it down as best I can, but because of the confusion of the voters, it's a bit like trying to tuck in that last corner of the bed.

All I can say is that when we see this behavior it is irrational. As noted, "Rookie of the Year" is essentially an award that gets it's importance based on how bad the eligible players are for the time being. There's little point to interpreting that as "The Most Valuable Player among the players who don't yet know how to contribute much value".

My assumption the WP impact is so huge, Drummond is a no-brainer pick. It's utterly ridiculous to think that he's even a tier or two below having the same impact as James and Paul did in 2013, especially when taking outside issues (why Drummond wasn't getting more minutes as you mentioned) into context.

After reading this, I think I need to familiarize myself guess the regression analysis on a more frequent basis when it comes to evaluating player's impact on a team. I still have a tendency to overestimate a player's during particular seasons, such as Jason Kidd on the New Jersey Nets. The effects are still lingering, but it isn't nearly as bad as it was some odd 8 or 10 years ago as an ignorant adolescent...

Thinking about the process of the ROY and my opinions on them, I probably need to go back to analyze my thoughts on a few of them. I was the devil's advocate for Melo winning the ROY over LeBron because his team went to the playoffs and their winning record was almost as bad as Cleveland's back in '03 (not to mention that LBJ was about to make the playoffs with a record 12 games below .500). I don't necessarily feel this way now (LBJ was likely the right winner), but I haven't went back to see the actual impact of Melo's games for that 04 Nuggets team compared to that of LBJ's. And I didn't take into account the season before Melo was drafted in Denver (Camby missing games, no J. Howard and Posey, acquisition of A. Miller and more playing time for Nene, etc.) The bolded is essentially what I felt about it, you just happened to articulate it better.

I also felt and still do feel that Ben Gordon should have won two awards in '05, but perhaps it was still thought that Okafor would pan out to be a top center in the NBA (I thought that one of the reasons he wasn't selected by Orlando was his back issues anyway...)

It's been awhile since I've looked back on those seasons involving those four guys, but I need to make time to reassess. I'm pretty sure I'm overlooking some things....

[Witzig-Okashi]

So, in layman's terms, Berri's metric is flawed in being half-baked.

Indeed. If he would use the individual possession counting by Oliver instead of his PE, he would arrive at a much better result. Or he could simply start with the premise that all positions are equal and arrive at marginal values which are appropriate for all boxscore entries without having to apply a positional adjustment. Either way is much more consistent than what Berri did.

It tries to attempt to use a metric that takes into account a perceived flaw in analysis of an indiviual players' impact on a victory by broadening the horizons, yet it falls victim to what it attempts to avoid by only taking into account the box-score stats. Am I correct?

Not really. You can arrive at a much better solution while only using the boxscore stats. That is not the issue here. It is more inline with a flawed understanding of science on his part as well as the overfitting issue which DocMJ mentioned. By actually trying to replicate the results from the past season as good as possible, he uses data not specifically meant to describe individual players (the defensive adjustment) as well as applying the positional adjustment to cover up the flawed basic assumption that PE would describe the possessions used for individual players in a useful way. The thing is: Berri's approach is inconsistent.

And then we have Berri's personality ...

Okay, I understand what you mean now. Must have over-read that....

Maybe Berri likes to unwittingly present himself as a genius who tends to be an a--hole, which is kinda troubling when given some more thought...

[mysticbb]

After reading this, I think I need to familiarize myself guess the regression analysis on a more frequent basis when it comes to evaluating player's impact on a team. I still have a tendency to overestimate a player's during particular seasons, such as Jason Kidd on the New Jersey Nets.

That is actually a very interesting topic. Jason Kidd and the rebounds on the Nets, where a guy named Jason Collins had a bigger rebounding impact (positive that is) than Jason Kidd. Once you understand that, you will realise that the boxscore has its limits beyond just not accounting for several stuff at all, but also by giving the right credit to the players. Similar to hockey assists, we would actually need something like "hockey rebounds" where the guy boxing out is getting credit as well. But yeah, regression analysis on rebounding as well as watching the video tapes gives you a lot of insight on that topic, especially the Nets with Kidd and Collins.

[mysticbb]

Maybe Berri likes to unwittingly present himself as a genius who tends to be an a--hole, which is kinda troubling when given some more thought...

It helps to sell the books, I guess ... And he has surrounded himself with guys not really questioning the metric, which can give everyone the wrong perspective about the work itself. It is also a pretty young field, where the expertise is actually less in the science world than rather in such stats communities and the teams. Therefore the peer-review process lacks severe quality. But well, overall I think Berri was always pissed that the other guys got the jobs in the NBA, while he pretends that he isn't interested. That might be part of his behavior ... And it is particular funny to me that with John Hollinger and Eli Witus two guys Berri claimed to have no clue about basketball, stats and math are now working as Vice Presidents of Basketball Operations for the Grizzlies and Rockets respectively. They are now decision makers in the NBA, while Berri still writes angry posts about the supposed stupidity of coaches and GM on his blog ... some sort of poetic justice :)

[Witzig-Okashi]

After reading this, I think I need to familiarize myself guess the regression analysis on a more frequent basis when it comes to evaluating player's impact on a team. I still have a tendency to overestimate a player's during particular seasons, such as Jason Kidd on the New Jersey Nets.

That is actually a very interesting topic. Jason Kidd and the rebounds on the Nets, where a guy named Jason Collins had a bigger rebounding impact (positive that is) than Jason Kidd. Once you understand that, you will realise that the boxscore has its limits beyond just not accounting for several stuff at all, but also by giving the right credit to the players. Similar to hockey assists, we would actually need something like "hockey rebounds" where the guy boxing out is getting credit as well. But yeah, regression analysis on rebounding as well as watching the video tapes gives you a lot of insight on that topic, especially the Nets with Kidd and Collins.

That reminds of a post that was on the GB of why Roy Hibbert was averaging so low on rebounds last month. I don't think that the OP of that board had taken into account that the Pacers were leading the league it total rebounds, and Hibbert does an excellent job of boxing out to let other his teammates get rebounds, too. Not to mention Hibbs' size. It's one reason why Stephenson can nab such a high number of rebounds on a frequent basis for a 2-guard.

[Doctor MJ]

After reading this, I think I need to familiarize myself guess the regression analysis on a more frequent basis when it comes to evaluating player's impact on a team. I still have a tendency to overestimate a player's during particular seasons, such as Jason Kidd on the New Jersey Nets. The effects are still lingering, but it isn't nearly as bad as it was some odd 8 or 10 years ago as an ignorant adolescent...

Thinking about the process of the ROY and my opinions on them, I probably need to go back to analyze my thoughts on a few of them. I was the devil's advocate for Melo winning the ROY over LeBron because his team went to the playoffs and their winning record was almost as bad as Cleveland's back in '03 (not to mention that LBJ was about to make the playoffs with a record 12 games below .500). I don't necessarily feel this way now (LBJ was likely the right winner), but I haven't went back to see the actual impact of Melo's games for that 04 Nuggets team compared to that of LBJ's. And I didn't take into account the season before Melo was drafted in Denver (Camby missing games, no J. Howard and Posey, acquisition of A. Miller and more playing time for Nene, etc.) The bolded is essentially what I felt about it, you just happened to articulate it better.

I also felt and still do feel that Ben Gordon should have won two awards in '05, but perhaps it was still thought that Okafor would pan out to be a top center in the NBA (I thought that one of the reasons he wasn't selected by Orlando was his back issues anyway...)

It's been awhile since I've looked back on those seasons involving those four guys, but I need to make time to reassess. I'm pretty sure I'm overlooking some things....

I find it always good to go back and re-evaluate. If you do that whenever you learn something new, and you're active about learning new things, over time you'll really get somewhere.

Re: LeBron vs Melo ROY impact. Some numbers for you to chew on, an RAPM study from that year:

LeBron +0.32
Wade +0.21
Melo -1.17

More simplistic on/off analysis will tell you something similar, which is: Meh. I wouldn't use this data to argue against Melo getting the ROY, it's just that none of these guys were showing drastic signs that they lifted their team. And this is typically the story now that we have access to regression data: Rookie stars don't seem to impact the game like their box score stats suggest.

So my vote would go to LeBron, based on him showing us the most that year, even if there were some real issues to what he was literally doing out there.

In the following year, since you mentioned it:

Gordon +0.67
Okafor -0.98
Howard -0.51

Here we've got something interesting, and something that probably makes you feel all the more confident about your opinion: We do seem to have evidence that Gordon was pretty sizably ahead of those other guys in terms of contributing to a team, which makes sense given his important role to a real team.

Is that enough for me to side with Gordon? I'd have to think about it. As I said before, what I don't want to do is give a guy an award simply because he's in a place to be put in a useful niche. The jobs of these rookies were very different, and given that, it's tough to compare them. I don't have a problem with your take on it though.

For more perspective, the data I'm using normalizes RAPM for variance, and I've found that on average, if you're someone playing starters minutes, having a rating of +1 or better, makes you around a Top 40 guy. We haven't seen any ROY contender reach that level since we started getting this data 10+ years ago.

[Witzig-Okashi]

I find it always good to go back and re-evaluate. If you do that whenever you learn something new, and you're active about learning new things, over time you'll really get somewhere.

Re: LeBron vs Melo ROY impact. Some numbers for you to chew on, an RAPM study from that year:

LeBron +0.32
Wade +0.21
Melo -1.17

More simplistic on/off analysis will tell you something similar, which is: Meh. I wouldn't use this data to argue against Melo getting the ROY, it's just that none of these guys were showing drastic signs that they lifted their team. And this is typically the story now that we have access to regression data: Rookie stars don't seem to impact the game like their box score stats suggest.

So my vote would go to LeBron, based on him showing us the most that year, even if there were some real issues to what he was literally doing out there.

In the following year, since you mentioned it:

Gordon +0.67
Okafor -0.98
Howard -0.51

Here we've got something interesting, and something that probably makes you feel all the more confident about your opinion: We do seem to have evidence that Gordon was pretty sizably ahead of those other guys in terms of contributing to a team, which makes sense given his important role to a real team.

Is that enough for me to side with Gordon? I'd have to think about it. As I said before, what I don't want to do is give a guy an award simply because he's in a place to be put in a useful niche. The jobs of these rookies were very different, and given that, it's tough to compare them. I don't have a problem with your take on it though.

For more perspective, the data I'm using normalizes RAPM for variance, and I've found that on average, if you're someone playing starters minutes, having a rating of +1 or better, makes you around a Top 40 guy. We haven't seen any ROY contender reach that level since we started getting this data 10+ years ago.

I had forgot all about Wade during that season, excluding his moments in the playoffs that year. I didn't consider him much because I thought he was surrounded with the best amount of talent (and he probably was.) I'm certainly surprised in the gap b/t LBJ and Melo, though...I haven't had a problem with LeBron winning it for several years, but it's interesting to see some form of advance metrics favoring him...

As for '05, I guess the voters factored in that it was the first year of the franchises' existence, that Okafor had averaged a double-double along with showing promise as a top tier center (the best I can think of)....I'd like to think there's very good arguments for him that year, just didn't run across them (but I'll admit not being a fan of Okafor after the 04 title game vs GA Tech grudgingly playing a factor at the time...I'm not a zealous stan/'hater' of players anymore, and that certainly doesn't play a factor anymore). I see your point in the subjectivity in the award, though I still think it has its limits in certain cases...

Where exactly are the best sites to access regression analysis? I need to bookmark them and make use of them....

[Doctor MJ]

I had forgot all about Wade during that season, excluding his moments in the playoffs that year. I didn't consider him much because I thought he was surrounded with the best amount of talent (and he probably was.) I'm certainly surprised in the gap b/t LBJ and Melo, though...I haven't had a problem with LeBron winning it for several years, but it's interesting to see some form of advance metrics favoring him...

As for '05, I guess the voters factored in that it was the first year of the franchises' existence, that Okafor had averaged a double-double along with showing promise as a top tier center (the best I can think of)....I'd like to think there's very good arguments for him that year, just didn't run across them (but I'll admit not being a fan of Okafor after the 04 title game vs GA Tech grudgingly playing a factor at the time...I'm not a zealous stan/'hater' of players anymore, and that certainly doesn't play a factor anymore). I see your point in the subjectivity in the award, though I still think it has its limits in certain cases...

Where exactly are the best sites to access regression analysis? I need to bookmark them and make use of them....

Re: Okafor vs Gordon. Well if you go by box score-based metrics, Okafor had better PER and better WS. He did this while taking on a role more similar to an actual starring role, and he was clearly seen as the more likely star by most going into the season. Pretty easy to me to see why people would side with him. The case for Gordon is clear to me to, so I feel ya, but at this point I don't look at that year and see a clear problem with the voting.

Re: best sites for regression data. That's tough.

So first off, you should check out the APBRmetrics boards. That's where the closest thing to a braintrust hangs out.

I think visiting this old blog is good, because it tries to explain how to actually make the stats yourself:
http://www.countthebasket.com/blog/

You'll note that the blog is not current, and this is common issue with such sites: The guys get hired by NBA teams, and they stop updating their site.

Plus, even the sites that have the stats themselves posted typically aren't likely to explain them to you. So here are a couple sites worth checking out, but you might find them too opaque:

http://talkingpracticeblog.com/2012/10/24/hello-world/
http://ascreamingcomesacrossthecourt.blogspot.com/

Also here's a Google doc with some old data:
https://docs.google.com/spreadsheet/ccc ... iTEE#gid=3

And here's mysticbb's site, although I don't think it's really up-to-date right now. mystic though is about as sharp as they come in this:
http://bbmetrics.wordpress.com/