Why I'm not a WP fan

[mysticbb]

DSMok1, that is basically what I did with the correlation analysis of the player's values from 2010 and 2011. If we add those up, we get a measure for the teams. We have to add rookies and players who didn't participate in Year Y, but that isn't such a huge problem. In fact Rosenbaum&Lewin 2007 done such a study.

But the bigger question is: What are we testing here? Basically around 80% of the players from year Y are playing again major minutes in year Y+1. If we have a somewhat stable model, we should get around 80% of the variance explained by the year Y measures. We add the rookies with the current values and all the other players, and we will likely end up with 80+%. It will depend on the year, but the test with the FM48 showed 0.85 correlation coefficient. The same test with my rating PRA shows a similar correlation coefficient. And I'm pretty sure we see something in the 0.8 to 0.85 range for Win Shares, PER and Wins Produced too.

The test doesn't show us a bit more about the model's ability to evaluate individual players or helps us to find a metric which can be used to determine optimal lineups. But well, besides Berri I never read that someone said the coaches are dumb and have to distribute the minutes different. A metric like PER should just show casual fans who are not watching every game how the players from his favorite team are stacking up against the rest of the league. I doubt that Hollinger thinks his metric should be used to suggest lineup changes or a different minutes distribution.

As I pointed out in one of my comments here before a model isn't good at evaluating individual players just because of a high correlation to winning or a good consistency from year-to-year. Those things are not specifically controlled by the most models, but rather by the circumstances. All models will do worse in terms of year-to-year consistency, if there is a huge jump in minutes distribution and a lot of role changes for players. Metrics like Win Shares, Wins Produced or my rating will have trouble when a certain team wins a lot of close games, thus the scoring margin indicates a lower amount of wins. All three metrics are trying to approximate scoring margin in the end, but they are doing it in different ways. A test like yours will show not really a big difference for each of those models, except for cases in which we have heavy role changes and different minutes. In that case the model which can evaluate the players better will have the better result. And in case of the Warriors the current minutes distribution has 50 wins for them by WP48, 38 by WS/48 and 37 by my rating at the end of the season. Right now they are on their way to 37 wins. Not quite sure, but I don't see them winning 27 of their last 29 games. The expectation is that WP48 will perform slightly worse than the rest, like Rosenbaum&Lewin 2007 showed it.

[DSMok1]

I had Golden State at 41 with my ASPM predictions. http://sonicscentral.com/apbrmetrics/vi ... 2407#32407

I agree with what you said above.

How would you test validity of a given model?

[ElGee]

I can no longer respond in the discussion because I've been given the honorary dissenting blacklist. But the last comment is directed at WP critics and I think it misses some really fundamental stuff.

Been thinking on a few things and it seems to me that the criticism of WP48 goes like this:

- It undervalues scoring because scoring is hard, and WP48 assumes that someone else would score if inefficient player X didn’t take so many shots.
- It over values rebounding because someone would get the rebounds anyway.

This seems like a contradiction. Either all stats have to be achieved (scoring AND rebounding) or none – the argument that some do and some don’t seems hard to justify.

And the argument for inefficient scoring is an odd one because both Carmelo and Iverson ware inneficient not becaue fo talent but shot selection.

Carmelo has taken 1313 three pointers in his career and hit 409 at .312 and Iverson took 3383 and hit 1059 at .313.

IMHO poor shot selection that leads to inneficient scoring is the biggest issue in mis-evaluating players. That is because people bias for the ability to generate good scoring options, and forget all the contested 19 footers and 3 pointers a player takes and shouldn’t. That is one’s eyes lying to you – plain and simple.

The answers seem obvious to me. Apparently they haven't been spelled out well.

It's not that scoring is "hard" and rebounding "easy," or vice versa. It's that the WP marginal values emphasize rebounding over scoring: WP suggests rebounding is MORE IMPORTANT than scoring.

In order for that to be true, the difference between scorers throughout the league would have to be extremely small (almost as if scoring were a constant, regardless of team) and teams then separated themselves by rebounding misses more effectively and used the possessions to generate more scoring to create a margin of victory.

Only that's not the case. Not empirically, intuitively, or statistically (although rebounding is certainly important -- that's where WP's numbers come from.). I'm sure a database master like mystic can run a quick analysis on points and offensive performance across players, but I hope it's not necessary. Winning the rebounding battle helps win the game, winning the scoring battle ALWAYS wins the game. (Those are the rules )

Strip the leading scorers off of any team and don't replace them and see how well the team performs. They'll still score -- Brooke Lopez might even average 20 points a game -- but they won't score well. And the difference these days between 1.0 pt/possession on offense and 1.10 pts/possession on offense is only about 27 wins in a season.

I've used examples before, and I don't know why WP proponents can't understand or respond to them. Look at at how it handles scoring (FTs, 2s, 3s and misses)

Danny Granger has a marginal value of 0.07 WP48 from his scoring.
Joel Anthony has a marginal value of 0.01 WP48 from his scoring.

They both are shooting .554 TS%, only Granger averages 21 pts/36 and Anthony 3.3.

In other words, volume doesn't really matter to WP. LeBron averaged 27.4 pts/36 on .579 TS last year. LeBron got about 0.18 WP48 from scoring values and ~0.30 from rebounding values. Even the website's own example, Bob Lanier in 1978, only earns 0.13 per48 from his scoring marginal values and 0.49 per48 from his rebounding marginal values. It's no wonder that top rebounders at the positions do really well and volume scorers who are lacking in the other box score metrics look terrible.

I don't know why this doesn't bother WP proponents. But Berri's defense seems to be "well, the regression showed that is what matters. What you think matters is wrong." It's at that point that I have to wonder if he understands what regression is. There are two slightly important factors here:

(1) It's correlative
(2) It's limited by the box score.

Mystic just created FM48 which can claim the exact same thing. Whippidy doo! Correlation doesn't equal causation. And in this case, everyone (except Berri?) seems to agree, for obvious reasons, that the box score doesn't cover all pertinent elements of basketball.

So what he's left with is trying to assign credit for point differentials based on these limitations. He's done that with a model, and obviously the model "fits" fairly well, but it has serious limitations because of the aforementioned factors.

It's how you get results like Michael Smith (Wizards) being the best SF in basketball in 2001, and then never making another NBA team again at 28. Of course, that's not a problem for Berri because every GM and Coach in the league is confused because they are using "prior knowledge" and not WP.

[Idunkon1stdates]

the purpose of this approach (an approach used by others) is to eliminate “trolls” (i.e. people who wish to leave the same or similar comments over and over and over and over again).

I don't think that's what a troll is, dave. By that definition, every WoW blog post is a troll... actually, that would make a lot of sense. Notice how berri and his acolytes repeat themselves ad nauseum, misrepresenting the arguments against them. It really is a cult-like atmosphere.

But yeah, the comments in that GSW post were pretty insightful and far from repetitive. There was a civil discussion going on.

[azuresou1]

I hope D.Berri gets mauled by a bear, he'd deserve it.

[droponov]

the purpose of this approach (an approach used by others) is to eliminate “trolls” (i.e. people who wish to leave the same or similar comments over and over and over and over again).

I don't think that's what a troll is, dave. By that definition, every WoW blog post is a troll... actually, that would make a lot of sense. Notice how berri and his acolytes repeat themselves ad nauseum, misrepresenting the arguments against them.It really is a cult-like atmosphere.

But yeah, the comments in that GSW post were pretty insightful and far from repetitive. There was a civil discussion going on.

It's been like that from the beginning. I was banned from the blog a week after the book came out (or a week after I read it, which wasn't much longer).

Berri is well aware of the deficiencies of his metric and the dubious nature of some of his claims, I think. But it's that assertiveness, the lack of any kind of epistemological modesty, the oversimplification, the attribution of quasi-magical properties to what he does (and the combination of the simplicity of the certainty with the esoteric and almost mysterious nature of the weights), that attracts a legion of devoted followers that actually buy his books. How to build a cult 101.

David Berri is the LaRouche of basketball stats. I wouldn't pay much attention to him. He created an interesting but deeply flawed linear weighted boxscore metric. I'd simply stop talking about it.

[mysticbb]

How would you test validity of a given model?

With statistical methods? A really good question. We would need a lineup check like I did it. How does the results of specific lineups correlate with the average (or sum) of the individual player metrics. Wins Produced was way worse than Win Shares and my rating. I actually added the Top 50 lineups in minutes played for the 2007/08, 2008/09, 2009/10 season and the result is rather similar. Wins Produced ended up slightly better than PER, but worse than Win Shares and my own rating. Obviously the sample size will be an argument against that method, but to really test a metric, we have to do something similar.

Btw, your prediction for the Warriors was before the season started. How many wins does your metric predict with the current minute distribution on the Warriors?

Only that's not the case. Not empirically, intuitively, or statistically (although rebounding is certainly important -- that's where WP's numbers come from.). I'm sure a database master like mystic can run a quick analysis on points and offensive performance across players, but I hope it's not necessary.

I have to admit that I don't quite understand your idea here. If it is not too complicated, I can probably run such an analysis rather quickly. And I would gladly do so.

It's how you get results like Michael Smith (Wizards) being the best SF in basketball in 2001, and then never making another NBA team again at 28. Of course, that's not a problem for Berri because every GM and Coach in the league is confused because they are using "prior knowledge" and not WP. :cry:

Holy crap. I didn't notice that before, but that is right away funny as hell. Seriously, Michael Smith with 0.416 WP48, that is incredible. Not only was he the best SF, but the very best player in 2001 in terms of WP48. And a quick glance at his numbers in the season before, he actually declined. Unbelievable, and such a guy ends up playing in Italy. :lol:

[DSMok1]

Btw, your prediction for the Warriors was before the season started. How many wins does your metric predict with the current minute distribution on the Warriors?

Still the same, oddly enough (when using my preseason rates for the players). Most of the difference is that David Lee hasn't been himself when actually playing. In fact, that's nearly 2 pts/100 Poss difference from my preseason projections right there.

[mysticbb]

That means someone else got more minutes who had actually rather similar rates as Lee according to your metric?

[Idunkon1stdates]

the purpose of this approach (an approach used by others) is to eliminate “trolls” (i.e. people who wish to leave the same or similar comments over and over and over and over again).

I don't think that's what a troll is, dave. By that definition, every WoW blog post is a troll... actually, that would make a lot of sense. Notice how berri and his acolytes repeat themselves ad nauseum, misrepresenting the arguments against them.It really is a cult-like atmosphere.

But yeah, the comments in that GSW post were pretty insightful and far from repetitive. There was a civil discussion going on.

It's been like that from the beginning. I was banned from the blog a week after the book came out (or a week after I read it, which wasn't much longer).

Berri is well aware of the deficiencies of his metric and the dubious nature of some of his claims, I think. But it's that assertiveness, the lack of any kind of epistemological modesty, the oversimplification, the attribution of quasi-magical properties to what he does (and the combination of the simplicity of the certainty with the esoteric and almost mysterious nature of the weights), that attracts a legion of devoted followers that actually buy his books. How to build a cult 101.

David Berri is the LaRouche of basketball stats. I wouldn't pay much attention to him. He created an interesting but deeply flawed linear weighted boxscore metric. I'd simply stop talking about it.

I will admit that I fell for it for awhile. I was a casual basketball fan when his book came out. WoW opened me to a world of statistical analysis I did not know existed, and since my knowledge of the game was rudimentary, I was not able to critically analyse his work. His approach was relatively intuitive, came from a position of authority, and seemed to make sense. Above all, it was easy. He had done the "hard work." Every answer to any question came from that little formula. Well, for a brief period of time, anyway.

After the honeymoon phase (a month or two), a few things started to bother me. I noticed Berri's arrogant attitude towards outsiders, both NBA decision makers, other sports economists, and people with dissenting views posting at his blog. He was passive-aggressive, petty, and used straw man after straw man to refute their arguments. This is not how a professional should behave, much less a professional whose work relied on the scientific method. I began discussing the NBA with Berri's work in mind, which brought me into conflict with a lot of well-established beliefs, many of which I could not counter. So, I posted these arguments on his blog, thinking that if I played the devil's advocate, Berri could refute these arguments and restore my faith in the model. Not only did he fail to refute these arguments, he and his followers were impolite and mis-represented the arguments in their responses.

His treatment of my softball criticisms and the criticisms of others made me even more suspicious. So I began re-reading posts on his blog, reviews of his book, and other forums that criticized Berri's methodology. It became clear that Berri's work was widely condemned, and not just by simpletons -- by respected members of the sports economist community, many of which were professors themselves or employed by basketball teams.

Further, as my knowledge of the game grew, it was becoming harder and harder to resolve glaring problems with the metric and Berri and what I knew objectively were true. Berri says that as a player increases FGA, efficiency declines, but only modestly so. Thus, a high efficiency, low usage scorer like a Biedrins can increase his usage with little decline in efficiency. Biedrins has no jumpshot, can't hit free throws or get to the line, and has a limited number of post moves. He objectively cannot take 25 - 30 FGA and only have a limited decline in efficiency. Berri says he can. I do not disagree that players who take more shots only experience a modest decline in deficiency -- but perhaps factors such as coaching and skillset come in to play. A player who is incapable of creating those shots or scoring efficiently at that rate will not be allowed to do so. This sort of critical thinking is not appreciated by Berri. There are many other examples, little thought experiments that demonstrate the model's problems, which I naturally came upon while thinking about basketball.

At this point, maybe several months after reading his book, I still considered Wins Produced a solid basis for what wins games in the NBA. Actually, to this day, I prefer it to other box score metrics (except FM48). But then I decided to look at the inner workings of the model. Berri hides most of this information (it is scattered between his website, his published work, and his book; one must actively seek it). By now, I was familiar with Berri's less-than-honest approach, and this just further solidified my opinion of him personally. And after looking at the model in detail and then reading what others had written about it... it was clearly disingenuous.

Since I concluded that WoW was a load of hooey 4- 5 years ago, not much has changed. Berri makes the same arguments, uses the same formula, and has most of the same acolytes as he did 5 years ago. I occasionally check his blog for a laugh, and I post once or twice a year to see if the responses will ever be anything different. They aren't, of course. It's always the same: basketball is about more than scoring points, Wins Produced explains 90% of wins, NBA decision makers would make better decisions if they just read this book, etc. It does not matter what you argue. They repeat the same thing time and again.

I don't like being conned. I think a response like, "I hope D.Berri gets mauled by a bear, he'd deserve it," while inappropriate, demonstrates how people feel after they have been misled or treated like idiots. Berri likes to play the victim, but he is the one who, I think, is deliberately misleading and victimizing people by abusing his academic credentials. Or maybe he really does believe his own bull? Who knows?

That's my own experience with WoW. I am embarassed to admit that I fell for it completely for at least a month. It would take several months before I got it completely out of my system. Even now I feel icky.

[floppymoose]

Actually, to this day, I prefer it to other box score metrics (except FM48).

Nice post, but I'm hoping that part was a joke. FM48 was my idea of a metric that might equal or best WP48 in all the pillars usually used to support it: win% correlation, stability, and explainability. All while being transparently flawed to the most casual fan of basketball. (And then mystic actually proved all that. w00t!) It's an intentional straw man.

So... you're kidding. Right?

[Idunkon1stdates]

Actually, to this day, I prefer it to other box score metrics (except FM48).

Nice post, but I'm hoping that part was a joke. FM48 was my idea of a metric that might equal or best WP48 in all the pillars usually used to support it: win% correlation, stability, and explainability. All while being transparently flawed to the most casual fan of basketball. (And then mystic actually proved all that. w00t!) It's an intentional straw man.

So... you're kidding. Right?

Yes, I am.

But if you write a book about it and start a blog, I may fall victim to it.

And wow, that was a long, self-absorbed post. Hopefully, a few other people can relate to it and don't find it too boring.

[Jimmy76]

I enjoyed your post it's always interesting to hear the entire thought process behind a particular point of view.

I think of myself as someone willing to take on unconventional beliefs even if most people won't take the idea seriously or even if it flies in the face of prima facie truth. I'm not the stat wiz some are but I feel like I usually do a pretty good job of understanding what a particular stat means. In the case of wins produced it was interesting for a short time before common sense reared it's practical head and I realized that it wasn't even the sniff test wp was failing to pass, it was failing to pass the 2+2=4 test. Even before wp I was exposed to advanced stats and possession based evaluation in forms that were friendlier the to mind.

I like wins produced in terms of it's experimentation and the forward movement it represents but Berri stopping there and claiming it as dogma is like ending the progression of western thought at the medevil christian apologetics.

Can't match your length but I can match the self-absorbiness.

[Doctor MJ]

David Berri is the LaRouche of basketball stats. I wouldn't pay much attention to him.

Wow. That's pretty dead on actually.

[Twinkie defense]

The Wages of Wins folks predicted that USA would go undefeated at the World Championships. That was pretty impressive given how the US usually gets beat up in these international competitions - their previous gold medal there was in 1994 with Don Nelson. And people were laughing at the US squad and calling them the "B Team."

[azuresou1]

The Wages of Wins folks predicted that USA would go undefeated at the World Championships. That was pretty impressive given how the US usually gets beat up in these international competitions - their previous gold medal there was in 1994 with Don Nelson. And people were laughing at the US squad and calling them the "B Team."

Pretty much everyone could tell you that the US team was going to win, so no, not impressive at all.

[Doctor MJ]

The Wages of Wins folks predicted that USA would go undefeated at the World Championships. That was pretty impressive given how the US usually gets beat up in these international competitions - their previous gold medal there was in 1994 with Don Nelson. And people were laughing at the US squad and calling them the "B Team."

Hmm. No one was claiming that any other team had anywhere near the amount of talent that team USA did. The concern was fit, which had everything to do with the lack of big men - and Wins Produced LOVES rebounding big men. Yes, the team had Kevin Love, but they hardly played the dude.

I'd be interested to see the prediction you speak of.

[Twinkie defense]

I referenced the article in the Warriors board thread on the World Championships, leading up the event. I would have to track that link down. But c'mon, nobody was predicting that the team would go undefeated; they weren't even favored to win gold.

And then of course there is the famous example where, against a tide of common perception, WoW correctly predicted the (positive) impact of Philly trading AI.

It is easy to point out seemingly funky outliers, and good to question models. If someone really wants to tear down the methodology though they should be able to present alternatives that are better, and not just for a cherry-picked example but for ALL instances - their own funky outliers included. For instance, regarding Floppy's stated preference for adjusted plus/minus (which I think there are even bigger problems with), there are no doubt some players commonly perceived as being very good - or very bad - where plus/minus tells the opposite story.

[mysticbb]

I referenced the article in the Warriors board thread on the World Championships, leading up the event. I would have to track that link down. But c'mon, nobody was predicting that the team would go undefeated; they weren't even favored to win gold.

:lol:

Where did you get that from? Team U.S. was heavily favored to win it all, also because many international stars didn't play.

And then of course there is the famous example where, against a tide of common perception, WoW correctly predicted the (positive) impact of Philly trading AI.

So, Berri predicted that a team which played 10 of the 15 games on the road will improve when they have an easier schedule? No kidding, the 76ers were expected to be better after that even without trading Iverson. Their scoring margin increased a little bit (from -3.6 to -2.7) which can be nearly fully explained by the schedule.
Berri is giving you a misleading view here, because he claims he predicted them to win 35.4 after that trade, but his model is based on scoring margin not on record. Thus you have to look at the scoring margin. Berri overrated the improvment, because at the end the 76ers' scoring margin didn't improve as much as Berri predicted.

It is easy to point out seemingly funky outliers, and good to question models. If someone really wants to tear down the methodology though they should be able to present alternatives that are better, and not just for a cherry-picked example but for ALL instances - their own funky outliers included. For instance, regarding Floppy's stated preference for adjusted plus/minus (which I think there are even bigger problems with), there are no doubt some players commonly perceived as being very good - or very bad - where plus/minus tells the opposite story.

You don't quite understand science at all. It is not necessary to provide a better model (which indeed can be found in this very thread here!), but to disprove a model. You can find examples that a model is working most times, but that doesn't make them good or even right. You are falling for a trap here, a trap Berri is using, because he is intellectual dishonest about it. Nearly everything he is writing on his blog is worthless, that's the reason he is writing it on the blog and not in a scientific paper. I'm pretty sure he knows that and his only motivation is attention and book sales.

Anyway, I disproved his model already. He is using team formulas in his regression which are not useful for players. He never showed that the formulas he uses are meant for players, but he is using the results as if he can compare players with that. But if you believe that it is reasonable a player can score points without having possession of the ball, you might very well believe that Wins Produced is something useful. :)

[lorak]

Kris Humphries likes WP

When he gets double double Nets are 15-7.