Are statmodel more Irrational than Irrational decision maker

Moderator: Doctor MJ

sp6r=underrated
RealGM
Posts: 17,212
And1: 8,544
Joined: Jan 20, 2007
 

Are statmodel more Irrational than Irrational decision maker 

Post#1 » by sp6r=underrated » Tue Feb 1, 2011 3:20 am

This is an old paper that I've never read before. I'm sure some of you have read it. I stumbled on this article at http://sabermetricresearch.blogspot.com ... 0of%20Wins.

It is very interesting

Recent research suggests that “statistics and analysis” typically lead to better decisions than “intuition and human intellect” in diverse areas such as choosing which students to admit to college and assessing mortality risks among cancer patients. Sports economics, with its rich data and abundant decisions to analyze has provided a fertile laboratory for studies of the efficiency of decision-making. In fact, researchers of the National Basketball Association (NBA) have used statistical models of player productivity to make strong claims about the “rationality” of NBA decision-makers. Yet these statistical models have rarely been subjected to any rigorous examination of their ability to forecast the future. We examine how well several player productivity metrics, including (a) John Hollinger’s Player Efficiency Rating, (b) Wages of Wins Wins Produced, and (c) the NBA Efficiency metric, do in predicting future team wins and future player productivity (the latter as measured by plus/minus statistics). In addition to a comprehensive examination of the player productivity metrics used by NBA statistical analysts, this paper is the first academic presentation of plus/minus statistics. Our findings provide a counterweight to much of prevailing literature and suggest that models that assume simplistic NBA decision-making often outperform more sophisticated statistical models.


http://www.stumblingonwins.com/LewinRosenbaum2007.pdf
Jimmy76
RealGM
Posts: 14,548
And1: 8
Joined: May 01, 2009

Re: Are statmodel more Irrational than Irrational decision maker 

Post#2 » by Jimmy76 » Tue Feb 1, 2011 4:15 am

A lot of that went over my head. It sounds like it confirms some commonsense statistical truths eg wins produced is not a good measuring tool and one year adj+/- is too noisy. PER and long term adj+/- run a pretty good correlation.

Essentially the writer is trying to evaluate statistical measurements by predictive ability (which to me is clearly the way to go) is how I understood it.
Doctor MJ
Senior Mod
Senior Mod
Posts: 50,782
And1: 19,479
Joined: Mar 10, 2005
Location: Cali
     

Re: Are statmodel more Irrational than Irrational decision maker 

Post#3 » by Doctor MJ » Tue Feb 1, 2011 6:43 am

Interesting. I'd read the "fights" between Berri and Rosenbaum (with pretty much every APBRmetrics person on Rosenbaum's side), but never seen the article. To see the article, on Berri's site, when it really appears to crucify Berri's work is unexpected to say the least.

The fact that WP does worse than raw MPG using the +/- barometer seems pretty damning to me. Berri's a guy who is constantly saying with great condescension that coaches don't assign minutes properly, and his metric can't even out predict performance from MPG which is essentially the wisdom of coach minus the realities of human teams?

Re: long term adjust +/- as predictor. Looks to me like no matter what you do with +/-, it's just not as useful for prediction as the other stats. This is an important practical fact, but I'd be careful with conclusions from it. Issues with a valid model with low reliability are distinct from those a less valid model with higher reliability but by the predictability metric, it all gets confused together.

I find it very interesting that PER outperforms everything else given that it's positively archaic relative to the newer stuff.

I'd be very interested to see how statistical +/- would do in this metric.
Getting ready for the RealGM 100 on the PC Board

Come join the WNBA Board if you're a fan!

Return to Statistical Analysis