dice wrote:why not just look at the unadulterated year-by-year career data in NPI form and form one's own conclusions rather than trust the validity of blended one size fits all data?
"Unadulterated" regression analysis of the year in question would be the stat APM. It is unregularized, and therefore injects no bias but is noisier. Put another way: APM is a pinnacle in valid +/- statistics, but it's not very reliable. The latter issue is big enough that at this point it's fallen out of favor and RAPM, and variations there of, are considered the state of the art on this front. Frankly I wish there was more pure APM studies available, but I don't disagree that RAPM is typically going to give you more useful information.
Once we are using RAPM, there is nothing without accuracy problems. All of them can be seen as starting a player from some place that's not based on his performance from that year, and hence result in a result somewhere between that starting place and the truth. The nice thing about the prior-informed version is that it's at least based on something we know about that player in the past whereas the non-prior informed truly is a bit of a "one size fits all" metric.
dice wrote:The prior-informed version let's the consistency between the years smooth out those bad assumptions.
does it treat all seasons as equivalent regardless of games played? that would be a major flaw
That question could mean a lot of different things, so I'm not sure what exactly you want to know.
If what you're asking is whether each prior for each player is equally valid in its assessment of the player, the answer is a clear no. It's an imperfect metric, and understanding the flaws is crucial if you're going to use it. I don't find it difficult to navigate through those issues with my analysis. It's not like you don't know if a player had something weird happen the previous year, so being aware of such issues you keep that in mind when you see what the stat says.
dice wrote:This gets concrete in a hurry when you compare Duncan to Garnett. There's a recurring theme where Duncan wins by NPI while Garnett wins by PI.
how is that possible? obviously one guy could win the occasional season NPI with the other guy winning PI every year due to smoothing, but i don't understand how a player could be consistently winning one and losing the other
All of this is a bit tough because we're using words here - and frankly not only am I not one of the top mathematical experts on the thing, I haven't thought about it in a while so I'm a bit rusty with it. I'll try again though:
The issue with RAPM in general is that it's only an improvement on APM most of the time - not all of the time. If there were no luck involved in the game, and no specific matchup-based distortions, then APM would be the perfect stat. Those things do exist though, so the mathematical technique called regularization is used to smooth things out. Outlier data in general in statistics is unlikely to continue as was done in the sample, and thus this technique is a way to diminish the impact of this data that's presumed to be noise.
If we look at Kevin Garnett in his peak Minnesota years using one year samples, the effect of this is to take him from being better than Tim Duncan by a pretty wide margin, to being behind Duncan. Or to paraphrase: "Yeah, that's a fluke, if we adjust based on what we typically see in the league, Duncan's the more proven of the two players".
Now, even before we get to using priors, there's a weird thing going on here. Seeing that happen simply once makes the skepticism absolutely warranted, but if it's happening year after year, then that's your first hint that perhaps this outlier data isn't actually noise.
So then we get into the use of priors. In general what priors do is help ensure that if a player is roughly the same player from year to year, his results will be more consistent from year to year. More signal, less noise. So, if you've got a player who has something effectively being classified as noise, but it proves to be a repeating trend from year to year, it's going to stop being dismissed as noise. And if that assumed noise is stuff that makes the player in question look better once you factor that in, well the PI metric is going to rate the player higher each year than the NPI does.
So to summarize: We've got APM, NPI RAPM, and PI RAPM. Each metric amends the previous to make the data more reliable, but there exists a particular course of events that leads to the first & third methods to agree with each other relative to the middle metric, and in such a case that indicates the middle metric is making things worse for that particular player.
Is that what happened with Garnett? Well, let's look at colts18's alternative in the next post.
