Can someone explain Berri's Wins Produced formula?

Moderator: Doctor MJ

Chronz
Starter
Posts: 2,199
And1: 467
Joined: Jul 30, 2008

Can someone explain Berri's Wins Produced formula? 

Post#1 » by Chronz » Thu Jul 19, 2012 12:12 am

Apparently his set of linear weights are the most concrete, atleast according to what Ive read from him defending them, no other statistician is so vigorous in defending his pet stat. How can something that seems so logically sound come to such horrible conclusions (ie Rodman > MJ).

I get that he over values rebounding but he hides behind the fact that the numbers all add up in the aggregate when correlating to actual team success. How does he accomplish this? I MUST KNOW....
giberish
RealGM
Posts: 15,824
And1: 5,809
Joined: Mar 30, 2006
Location: Whereever you go - there you are

Re: Can someone explain Berri's Wins Produced formula? 

Post#2 » by giberish » Thu Jul 19, 2012 12:22 am

While I won't go into the exact details - these are the basics.

1) He finds the correlation between team stats (team-wide assists, rebounds, blocks, shooting%,) and team success.

2) He assumes that individual box-score stats all correlate linearly with team-wide stats. This is a major flaw, and the reason that most people discount his results.

3) He then scales the team-wide scores with a team's overall points for/against. This actually helps a bit , as players get a little credit for team-wide play that doesn't show up in box-score stats (though it's blurred over the whole team so it's not exact).
User avatar
rrravenred
Retired Mod
Retired Mod
Posts: 6,065
And1: 546
Joined: Feb 24, 2006
Location: Pulling at the loose threads of arguments since 2006

Re: Can someone explain Berri's Wins Produced formula? 

Post#3 » by rrravenred » Thu Jul 19, 2012 7:31 am

Not even Berri explains Wins Produced. ;)

Like all box-score based metrics, it misses a hell of a lot of qualifiers for impact and Berri famously doesn't answer criticism with anything approaching humility.
ElGee wrote:You, my friend, have shoved those words into my mouth, which is OK because I'm hungry.


Got fallacy?
CBB_Fan
Senior
Posts: 591
And1: 138
Joined: Jul 15, 2012

Re: Can someone explain Berri's Wins Produced formula? 

Post#4 » by CBB_Fan » Thu Jul 19, 2012 5:22 pm

I did something similar to his method to evaluate college basketball, but I didn't try to use it to predict player wins produced (mostly was doing a NCAA tournament evaluation). College is completely different from the NBA in that players aren't as mobile and certain stats have a much stronger correlation with wins than in the NBA (biggest example: blocked shots).

I liked doing it for college because of the much larger sample size. With 345 teams and over 10,000 games, I could get a much more comprehensive idea of what a team should do to win a basketball game than the 30 teams and a little over 2400 games in a full NBA season.

In order to correctly apply it to player you would have to count efficiency into the equation, something I don't think Berri does (or if he does, not correctly). For instance, a player that shoots to much hurts a team's chances of winning, and a player could get a ton of defensive rebounds without actually increasing his team's total # of rebounds.
azuresou1
Head Coach
Posts: 7,416
And1: 1,072
Joined: Jun 15, 2009
   

Re: Can someone explain Berri's Wins Produced formula? 

Post#5 » by azuresou1 » Thu Jul 19, 2012 7:39 pm

Berri also assumes that efficiency is all that matters when it comes to scoring, because obviously the points would just materialize from somewhere.

Literally had Landry Fields as the 8th most valuable player in 2011, simply because he grabbed a fair amount of rebounds and had a high TS%.
Chronz
Starter
Posts: 2,199
And1: 467
Joined: Jul 30, 2008

Re: Can someone explain Berri's Wins Produced formula? 

Post#6 » by Chronz » Fri Jul 20, 2012 6:56 am

So the consensus(amongst the APBR community at least) is that the very reason he defends it (that the individual markers correlate almost perfectly at the team level) is not only irrelevant but proof of its shortcomings?

Yet Berri is praised by the vast majority of people in HIS field for lending his work to be analyzed openly, which to my knowledge Hollinger/Rosenbaum do not do. Any particular reason why you value APBR community above his fellow economist?
mysticbb
Banned User
Posts: 8,205
And1: 713
Joined: May 28, 2007
Contact:
   

Re: Can someone explain Berri's Wins Produced formula? 

Post#7 » by mysticbb » Fri Jul 20, 2012 11:40 am

Chronz wrote:So the consensus(amongst the APBR community at least) is that the very reason he defends it (that the individual markers correlate almost perfectly at the team level) is not only irrelevant but proof of its shortcomings?


What? No, that is not at all the critism of WP48. The method is inconsistent, that is one issue, the other issue is that things on a team level do not scale down to the individual level. Berri assumes two specific things he NEVER showed are true. First, that each boxscore metric has an intrinsic value which can be determined by regression, and second that each player can be looked at as if he would be a "team". Neither things were shown by Berri nor are those two things true.

Berri's way of "proving" his metric is completely inconsistent and does not show anything. You can basically take the uniform numbers, adjust them by "position" and then make a team adjustment and the result will be a pretty much "perfect" correlation to the overall team wins. The high correlation is controlled by the adjustments not by the initial unadjusted metric. I showed that in another thread by just taking a scoring rate, adjust that with a team defensive adjustment and I get an even better correlation than Berri. I can make the same steps to "prove" that this metric has some value and get the very same result as Berri. Well, Berri is ignoring two very important biases: selection bias and hindsight bias.
The players are pre-selected by scouts and coaches and will in almost all cases be used in a similar way. No coach will try to make his "enforcer" to be his main "scorer" for example. The players will have similar roles and minutes in consecutive seasons. That just a simple fact. And using hindsight data to predict something which already happened is not something which is really impressive. In fact, we have a better method to predict team wins, pythagorean expectation, than the linear relationship Berri is using.

Berri's method is inconsistent, because at the beginning he is proposing that boxscore entries would have intrinsic value which can be determined by regression. Well, Berri is doing one regression and one regression only, he uses linear regression to determine the win% by using offense and defensive efficiency as independent variables. After that Berri is using league average values to determine the marginal values of each boxscore entry. So far, so good. The issue is that now we get a pretty good differentiation between players, bigs are incredible valuable while smalls are not. At that point Berri is doing something which is completely unscientific, he is introducing an adjustment for somewhat arbritrary positions. Well, if Berri would have stayed with his original hypothesis, he would have just taken the results and would have concluded that bigs are in fact more valuable than smalls. By changing that make the assumption that each "position" would be equally valuable, he also changes the marginal values for each different position. In that way 2pt scoresd by a center are less valuable than 2pt scored by a point guard. Makes no sense to everyone except to Berri and his followers.
That step also is very important in regard to the consistency. Because adjusting for position just means that boxscore entries do not have an intrinsic value, thus we can't use regression in order to determine such value. That makes the whole thing complete bogus.

Well, and at the end Berri does also assume that every player on a specific team has the same impact on the overall team defense. He is basically saying that each player is the same in terms of help or weakside defense. That is obviously complete nonsense, but Berri believes that this is true. That adjustment is rather important, because it controls the correlation between WP48 and overall team wins.

Can we show that all what I wrote is actually true? Yes, we can. Two things in which WP48 would need to perform worse are correlations to lineup performance level and in out of sample tests. WP48 can't say anything about the lineup performance, we can use a coin and will have the same ability to predict lineup performances as WP48. Other boxscore metrics such as Win Shares or my SPM are MUCH better regarding this. The other question is the out of sample test. Even when we take the real values in the respective season for rookies into account WP48 is worse than basically everything else except of PER. WP48 as a tool is worthless in terms of predicting outcome of games in the future.


Chronz wrote:Yet Berri is praised by the vast majority of people in HIS field for lending his work to be analyzed openly, which to my knowledge Hollinger/Rosenbaum do not do. Any particular reason why you value APBR community above his fellow economist?


Which fellow economists? Seriously, Berri is publishing his stuff in low-impact journals, sometimes Berri himself is in the editorial board or was. Berri is basically publishing his stuff without any kind of serious peer-review. He is not cited by any meaningful people and the only citations you can find are those from people in his staff or those who are critizing the method.
We also talk about a group of people which have not a strong mathematical background. They can use SPSS in order to make a regression, that's basically it. The APBR comunity consists of people with a PhD in math, physics, etc., a much stronger mathematical background than that of those economy people. As a geophysicist I had more math in my first 2 semesters than Berri in his whole life, just to make clear what kind of differences we are talking about.

Hollinger does not claim that his method is scientific, but his methods are all open. No idea how you can say otherwise. Rosenbaum has also described his method in great detail and the work by others is influenced by that. The difference between those +/- based people and Berri is: A lot of them are actually really working for teams and thus do not publish their results openly. But with enough math knowledge we can easily reproduce such things. Well, Aaron Barzilai is even kind enough to provide the results for all teams but the Grizzlies, because he works for the Grizzlies.
azuresou1
Head Coach
Posts: 7,416
And1: 1,072
Joined: Jun 15, 2009
   

Re: Can someone explain Berri's Wins Produced formula? 

Post#8 » by azuresou1 » Fri Jul 20, 2012 6:21 pm

Well stated mystic.

In many ways stats such as PER which are entirely first-level boxscore metrics are superior to the garbage that Berri produces, because PER doesn't claim to be the end-all of basketball statistics, and also doesn't have half a dozen terribad assumptions baked in.

That clown Wayne Winston is also bad.
giberish
RealGM
Posts: 15,824
And1: 5,809
Joined: Mar 30, 2006
Location: Whereever you go - there you are

Re: Can someone explain Berri's Wins Produced formula? 

Post#9 » by giberish » Fri Jul 20, 2012 7:07 pm

One simple test is that anyone claiming to have one number that describes exactly how good every player in the league is, without any thought necessary, is total BS.
Doctor MJ
Senior Mod
Senior Mod
Posts: 50,604
And1: 19,355
Joined: Mar 10, 2005
Location: Cali
     

Re: Can someone explain Berri's Wins Produced formula? 

Post#10 » by Doctor MJ » Fri Jul 20, 2012 7:58 pm

Chronz wrote:Apparently his set of linear weights are the most concrete, atleast according to what Ive read from him defending them, no other statistician is so vigorous in defending his pet stat. How can something that seems so logically sound come to such horrible conclusions (ie Rodman > MJ).

I get that he over values rebounding but he hides behind the fact that the numbers all add up in the aggregate when correlating to actual team success. How does he accomplish this? I MUST KNOW....


Glad to see mystic & co already get into the details on this. I have to tee off on what you already mention: His communication as a scientist is completely unacceptable.

This is a guy who has cultivated a huge internet presence, but who refuses to have any actual debate online (he's banned many of the people on this board from his forum) claiming that that needs to be done in journals. This is a guy who hid behind the shield of academic superiority, while working at CSU Bakersfield and putting down scientists who work at more prestigious universities simply because they aren't publishing their findings in his preferred journals (and meanwhile those guys actually tend to have some athletic background and work with people in the NBA).

It's bad science, either by incompetence or by malevolence (probably a bit from both) even before you get to the actual criticisms.
Getting ready for the RealGM 100 on the PC Board

Come join the WNBA Board if you're a fan!
Doctor MJ
Senior Mod
Senior Mod
Posts: 50,604
And1: 19,355
Joined: Mar 10, 2005
Location: Cali
     

Re: Can someone explain Berri's Wins Produced formula? 

Post#11 » by Doctor MJ » Fri Jul 20, 2012 8:02 pm

azuresou1 wrote:Well stated mystic.

In many ways stats such as PER which are entirely first-level boxscore metrics are superior to the garbage that Berri produces, because PER doesn't claim to be the end-all of basketball statistics, and also doesn't have half a dozen terribad assumptions baked in.

That clown Wayne Winston is also bad.


In terms of simple-mindedly buying into one's own stat, Winston is the other known guy who is just as bad as Berri. Unlike Berry though, Winston doesn't act like a fascist dictator, has worked with the NBA, and his ideas actually have contributed significantly to basketball statistics (his WINVAL system was adjusted +/- before adjusted +/- existed, and it was articles about WINVAL that started people on the internet getting to work).

Criticism of Winston is warranted, but the ninth circle of statistical hell is reserved for folks like Berri.

Return to Statistical Analysis