Why I'm not a WP fan

[UGA Hayes]

I don't have any statistical background and don't have a dog in the fight but I think one reason Berri has followers is that his system seems to do really well with experimental models and by that I mean trades. Compared to Hollinger (WP vs PER) he seems to predict the outcomes of trades far better. I thought the OKC trade or Perkins was a pretty good Litmus test. His assertion that OKC would be a lot better with the Loss of Green and Krstic and replacement by Ibaka and PErkins would be a big advantage. He also predicted a simialr adavantage for Denver trading away Carmelo. Both seem to be good example of trading away guys whose best attribute appears to be usage and the ability to create shots even if eficiently. I also have found that it seems to predict reprecussions for injuries well too.

I think for the average fan the disconnect with alot of the other methods even if Berri method is complete BS is that they doesn't seem to correlate with wins. I think a side issue for me the uninformed is is it such a bad thing to have a modifying factor for team wins which I think is a big complaint about Berri system. Someone brought up Monta Ellis before. I think a lot of hard core fans would tend to agree with WP rating of Ellis as only average in helping produce wins as to opposed to Hollingers top 40 player.

[floppymoose]

This whole thread started because WP completely failed to predict the impact of the David Lee trade.

[EvanZ]

Berri was certainly not alone in predicting that less Green and more Ibaka would have a positive effect. Although you wouldn't know it by reading his blog.

[floppymoose]

+- was pointing out that Green wasn't really cutting it at PF. WP didn't have any monopoly on that insight.

[Idunkon1stdates]

This whole thread started because WP completely failed to predict the impact of the David Lee trade.

In WP's defense, David Lee's played like ass this year. If Lee put up his usual numbers AND the Warriors sucked ass, it would truly be a stunning indictment of the model... that said, they were predicting 50 - 55 wins for that team. Even a hampered Lee should've taken the Warriors to 45+ wins. But he didn't.

Anyway: http://dberri.wordpress.com/2011/04/22/ ... /#comments

Mike,
That is just an awful study by Birnbaum. He then cites another awful study by Eli Witus (which I already refuted in the comments on a post on Aaron Brooks — I think). You have to do more than just look at one season and look at a correlation. My study — which looked at thirty years of data and controlled for a variety of other factors — doesn’t find that large of an effect (although there is a diminishing returns effect).

Here is the problem. So many on-line studies are done by people like Birnbaum and Witus, who simply do awful work. And then people like you go around citing these studies as if they said something. Unfortunately, I can’t teach Birnbuam, Witus, and you econometrics in an blog post. So I guess we just have to live with nonsense.

I quite enjoy it when berri seethes with rage. I've seen him post nasty comments only to remove them a couple minutes later. Dude has issues.

[EvanZ]

I love how Berri equates econometrics with statistics. No other field of study or inquiry involves statistics, right?

[mysticbb]

I love how Berri equates econometrics with statistics. No other field of study or inquiry involves statistics, right? :lol:

And that statement is even funnier, if you include how Berri ignores the financial aspect completely. Like in his last piece about the Hornets: http://dberri.wordpress.com/2011/04/25/ ... ar-season/

All of this suggests that the decision-makers in New Orleans are just guessing.

No Dave, all of this suggest that the decision-makers in New Orleans cutted the costs while still being able to battle for a playoff spot. The trades had more to do with financial relief and flexibility. But how should an economy professor know that? :D

[Nivek]

Do individual basketball players even produce wins?

I'm not playing semantics here. Teams win and lose games, but I'm not convinced that wins is a suitable unit of production for individual players. I sorta think that players produce points, rebounds, assists, etc., which combined at the team level and mixed with defense and coaching produce wins. I haven't thought all this through completely, and I'm posting with very little time, so forgive me if it comes as gibberish.

Also, don't forget how much work is being done by the position adjustment. Unadjusted for position, a WP ranking would be a list of centers, followed by PFs, followed by wings, followed by PGs. The position adjustment assumes that each position contributes equally to "producing wins" when that may not be the case.

[UGA Hayes]

On Lee, is that a fair argument though? The prediction is predicated on Lee putting up the same numbers as he did in NY which he didn't. A couple of things to consider. Before his elbow infection Lee and the Warriors were both playing very well. Additionally this is the worst year Lee has had shooting wise his whole career by a good margin which is likely a fluke. His rebounding totals is also probably a little lower than normal. There doesn't strike me as any tangible reason those numbers were down other than he had an off year considering he is a fairly consistent player to this point. He missed 8 games of which his team only won once, with the onther games being blowout losses.I don't think its that outrageous to think had Lee played in those games and had his normal numbers that the team would have been closer to 50 wins. As it is WP says Lee only had a little better than average year.

Even if the methodology is total luck it does feel to me like its capturing how a player is playing in a particular year relative to his position. I think the criticism of the system based on how it predicts isn't a good way to go because you seem to be implying that the player isn't allowed to be a worse player than he was the year before.

Another person brought up Kris Humphries. Kris Humphries can't be that good right. Well he almost certainly had a fluke year but his actual numbers were awesome this year as far as shotting and rebounding, blowing away his career norms. Why should his reputation be held against his production. I wouldn't trade for him since his season was a fluke, but I'm not sure why there is a criticism that he would have produced more wins.

Now if the argument is that its easier to put up stats on bad teams or that a good rebounder needs to ecvaluated in the context of how good his teammates are than thats something different but a whole different burden of truth needs to be proven. I happen to think those things are true and take it into account when looking at the WP numbers.

[floppymoose]

On Lee, is that a fair argument though? The prediction is predicated on Lee putting up the same numbers as he did in NY which he didn't.

Short answer: yes, it's fair. Drops in WP production when a player switches teams is one of the ways WP could be wrong. While it might be a more satisfying case to have his WP numbers stay the same and still see the Warriors only win 36 games, saying that the drop in Lee's WP excuses WP's predictions is not a valid argument. Lee's WP could have gone down on his new team precisely because he wasn't as valuable as his prior WP suggested, and thus his new team used him differently.

[mysticbb]

Do individual basketball players even produce wins?

I'm not playing semantics here. Teams win and lose games, but I'm not convinced that wins is a suitable unit of production for individual players. I sorta think that players produce points, rebounds, assists, etc., which combined at the team level and mixed with defense and coaching produce wins. I haven't thought all this through completely, and I'm posting with very little time, so forgive me if it comes as gibberish. :)

No gibberish, that is in fact all true. Now some of the stats geeks (like EvanZ here or DSMok1 or myself) are trying to attribute individual players with those wins, but we are doing that in a certain way. We are calculating wins above replacement (the replacement level was part of the discussion and can differ). That means we get a value for the team with a certain player and a value with a replacement player. The difference is just the wins above (or below) replacement. No player is producing that alone, he needs 4 teammates to do that. But we want to evaluate the differences between those players by using various statistical methods.

Also, don't forget how much work is being done by the position adjustment. Unadjusted for position, a WP ranking would be a list of centers, followed by PFs, followed by wings, followed by PGs. The position adjustment assumes that each position contributes equally to "producing wins" when that may not be the case.

I completely agree. Additional to that the current NBA isn't really using the old positions anymore. Some players are playing SG on defense and SF on offense or PF on defense and SF on offense or ... Getting the positions right, especially for bigger forwards is pretty hard. And it makes a difference of an average player in WP48 whether someone is listed as a SF or PF.

On Lee, is that a fair argument though? The prediction is predicated on Lee putting up the same numbers as he did in NY which he didn't.

That is true for every other metric we can use for the prediction and no other metric will predict 48+ wins for the Warriors with the current minutes distribution. In fact my rating predicted them to have 38 wins, they ended up with 35 pyth wins.
And that the difference doesn't even show up in a correlation analysis, because WP48 has 0.77 and my rating has 0.79 as correlation between last seasons and this seasons values for the Warriors players. But overall WP48 is around double the amount off per player than my rating from year to year.
And that is something Berri also fails to understand that the correlation coefficient isn't something we can actually really use as an evaluation tool for a metric's ability to evaluate players.

A couple of things to consider. Before his elbow infection Lee and the Warriors were both playing very well. Additionally this is the worst year Lee has had shooting wise his whole career by a good margin which is likely a fluke. His rebounding totals is also probably a little lower than normal. There doesn't strike me as any tangible reason those numbers were down other than he had an off year considering he is a fairly consistent player to this point. He missed 8 games of which his team only won once, with the onther games being blowout losses.I don't think its that outrageous to think had Lee played in those games and had his normal numbers that the team would have been closer to 50 wins. As it is WP says Lee only had a little better than average year.

Blabla blub ... Minutes distribution and the values from last season predicted 48 wins for the Warriors. They are not even close. WS48 is closer, my rating is closer, heck, even PER is closer.

Even if the methodology is total luck it does feel to me like its capturing how a player is playing in a particular year relative to his position. I think the criticism of the system based on how it predicts isn't a good way to go because you seem to be implying that the player isn't allowed to be a worse player than he was the year before.

Actually that is Berri's argument why WP48 is good. If you take that argument out, Berri becomes useless.

And that you feel that way is weird, because Berri is using formulas which are working for teams, but not for players. Or do you honestly think that Humphries used 225 possession less to score his points than he in reality did? Understand the formulas Berri is using and you will see that WP48 is bogus.

Another person brought up Kris Humphries. Kris Humphries can't be that good right. Well he almost certainly had a fluke year but his actual numbers were awesome this year as far as shotting and rebounding, blowing away his career norms. Why should his reputation be held against his production. I wouldn't trade for him since his season was a fluke, but I'm not sure why there is a criticism that he would have produced more wins.

Humphries is supposed to be a better player than Dirk Nowitzki in this season. And that by a HUGE margin. According to WP48 Humphries would have given the Mavericks 7.8 wins MORE than Nowitzki in this season, if he had played his minutes. Do you actually believe this?
How does someone come up with this? Well, Humphries grabbed 225 offensive rebounds and each offensive rebounds means the player used one possession less for his "points created". Looking at the values Humphries "produced" around 850 points via scoring and assists in this season. Using the possession formula for teams (which Berri is using) he only used 551 possessions for this. That makes an offensive rating of 154.3! And that for a player who had a 55.5 TS% and a 12.1 To-R. The league average is 54.1 TS% and 11.2 To-R. So, basically Humphries is as efficent on offense like an average player, but accoring to Berri is one of the best offensive players EVER!
Unfortunately for Nowitzki: He doesn't grab that many offensive rebounds, but rather is going back on defense, which helps the Mavericks defense a lot more than the Nets get a boost by Humphries offensive rebounding. That means, even though Nowitzki has with 61.2 TS% and a 8.1 To-R a clearly better offensive efficiency than Humphries, he ends up as the worse offensive player according to Berri.

That's what is wrong with that metric. If you use a team formula and think you can evaluate players with that, you are either a moron or a liar. I guess Berri is something between ...

[UGA Hayes]

So is it implied that the WP for Humphries is from his statistical measures. For the sake of argument if Humphries was three times the defensive player that Dirk was could Berri point to that as being captured by WP and thats its not all the offensive rebounds, or does his formula actually explicitly lay it out as you just did?

Also I am still confused about the team result factor. Say Hollinger added that factor to PER would we expect it to have the same "see it correlates to winning look"

Also I don't understand how you can dismiss what I said about Lee. Why can't he have a significantly worse year, which statistically he did, and that not affect anything. That one got a closer prediction despite such obvious differences in his statistical output and games missed is not a convincing argument that your predictive model did well. In fact that is why I'm not a fan of predictive models b/c the reality is that a team does not have health or consistent variables for a whole season.

I'm just playing devil's advocate here. While I think its clear that Berris system is pretty fIawed if not an outright sham, I don't know that you guys are giving the most convincing argument.

[floppymoose]

WP doesn't really measure defense. That's why Berri has to use the "team adjustment" - to capture the collective defense of the team. And since that adjustment is applied ot everyone on the team equally, players who put up big box score stats but are poor defenders (Murphy, Lee, Love, Fortson) end up boosting their own WP while taking down the WP of their teammates.

[DSMok1]

I will note that I greatly dislike using "wins" as a unit of measurement for players as well, Nivek. I use points/100 possessions above replacement to measure player value. That is more directly applicable to players, and the conversion from points to wins isn't completely trivial.

[Doctor MJ]

Do individual basketball players even produce wins?

I'm not playing semantics here. Teams win and lose games, but I'm not convinced that wins is a suitable unit of production for individual players. I sorta think that players produce points, rebounds, assists, etc., which combined at the team level and mixed with defense and coaching produce wins. I haven't thought all this through completely, and I'm posting with very little time, so forgive me if it comes as gibberish.

Great observation. The use of "Win" is a great trick. Manages to make your stat seem more real when in fact it's just adding in a level of abstraction.

[qianlong]

First of all i love this thread, well argued positions in a sophisticated and polite way with high density of good posts and posters. So much that i wanted to participate.

Also I am still confused about the team result factor. Say Hollinger added that factor to PER would we expect it to have the same "see it correlates to winning look"

I am not sure i understood what you mean, but if i understood correctly this argument was clearly solved with the terrible and awesome FM statistic in page 5. With the defense factor used to obtain a statistic that correlated well. That part deserves a paper.

Also I don't understand how you can dismiss what I said about Lee. Why can't he have a significantly worse year, which statistically he did, and that not affect anything. That one got a closer prediction despite such obvious differences in his statistical output and games missed is not a convincing argument that your predictive model did well. In fact that is why I'm not a fan of predictive models b/c the reality is that a team does not have health or consistent variables for a whole season.

Because when you try to make predictions you don't use spikes but you use the mean. You are a bit conservative for improving players, but most of the time you are right. I hope i made my self clear.

[Nivek]

Just finished reading through the most recent posts, and I saw someone ask about the team level adjustment. The short answer is that someone could use nearly any metric and make it correlate strongly with wins by using a team level defensive adjustment.

I did this shortly after Berri published "Wins Produced." I calculated several different metrics that have been used to evaluate players, including Win Score (Berri's "simple" version of Wins Produced), PER, my own metric, points per game, and a few others. I didn't use Berri's defensive adjustment method (because he hadn't published it at that point), but created one of my own. Once applied, ALL the metrics correlated with wins at .9 or better. A few metrics got to ~.95, including PER and Win Score.

With a defensive adjustment, most of the box score metrics will correlate strongly with team wins. At the team level, of course. That doesn't mean they're doing a good job of evaluating INDIVIDUAL contributions, however.

[UGA Hayes]

Just finished reading through the most recent posts, and I saw someone ask about the team level adjustment. The short answer is that someone could use nearly any metric and make it correlate strongly with wins by using a team level defensive adjustment.

I did this shortly after Berri published "Wins Produced." I calculated several different metrics that have been used to evaluate players, including Win Score (Berri's "simple" version of Wins Produced), PER, my own metric, points per game, and a few others. I didn't use Berri's defensive adjustment method (because he hadn't published it at that point), but created one of my own. Once applied, ALL the metrics correlated with wins at .9 or better. A few metrics got to ~.95, including PER and Win Score.

With a defensive adjustment, most of the box score metrics will correlate strongly with team wins. At the team level, of course. That doesn't mean they're doing a good job of evaluating INDIVIDUAL contributions, however.

Thats interesting. One of the things I was thinking about is that if you hypothetically had 50 different statisticians and each was convinced a few statistical indicators such as rebounding with Berri was key to winning and then you adjusted for a team factor amost by accident someone would distribute credit among individual players appropriately. Like I said earlier I don't follow Berri's rankings b/c I think he uses sound formulas, but rather b/c when I have compared his evaluation of player personnel moves based on his probably faulty stats he has had a good track record especially in comparison to the biggest name out there Hollinger. Hollinger has gone on record with some pretty strong statments such as Iverson being traded to Denver made them instant contenders and has a lot of retrospectively dubious assertions like that, largely I believe b/c his system makes no effort to put an individual production in a team context. And if your system can't do that and arguably does it wrongly then what is the point?

I feel like we are talking past each other on the Lee thing and making predictions in general. You really haven't responded to my point that a prediction whether right or wrong a lot of times is meaningless b/c the prediction is based on players maintaning their production from previous seasons and health of the entire team. I'd rather see something, whether Berri does it or not, that accurately portrays who is helping a team the most during a current season.

[Nivek]

We're not talking past each other about Lee, because WE'RE not talking at all. I haven't said a thing about Lee -- you're having that conversation with someone else.

[floppymoose]

You really haven't responded to my point that a prediction whether right or wrong a lot of times is meaningless b/c the prediction is based on players maintaning their production from previous seasons and health of the entire team. I'd rather see something, whether Berri does it or not, that accurately portrays who is helping a team the most during a current season.

That was probably to me.

Predictions are the best tool we have for testing player ranking metrics. Due to the particulars of how WP works, one of the ways to test it is when an apparent WP "outlier" gets traded. We don't get that happening a lot, so there is not a lot of data to choose from there. That makes it easy for WP defenders to say that the results didn't match predictions because of other factors, rather than a problem with WP.

Another good way to test WP is to use the WP48 values and see how well they correlate to actual performance of different team lineups. If that correlation is poor, that's a red flag. I believe mystic did tat earlier in this tread.