Normalized & Scaled RAPM Chronology Spreadsheet

Moderator: Doctor MJ

Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#21 » by Doctor MJ » Mon Sep 22, 2014 4:35 am

dice wrote:why not just look at the unadulterated year-by-year career data in NPI form and form one's own conclusions rather than trust the validity of blended one size fits all data?


"Unadulterated" regression analysis of the year in question would be the stat APM. It is unregularized, and therefore injects no bias but is noisier. Put another way: APM is a pinnacle in valid +/- statistics, but it's not very reliable. The latter issue is big enough that at this point it's fallen out of favor and RAPM, and variations there of, are considered the state of the art on this front. Frankly I wish there was more pure APM studies available, but I don't disagree that RAPM is typically going to give you more useful information.

Once we are using RAPM, there is nothing without accuracy problems. All of them can be seen as starting a player from some place that's not based on his performance from that year, and hence result in a result somewhere between that starting place and the truth. The nice thing about the prior-informed version is that it's at least based on something we know about that player in the past whereas the non-prior informed truly is a bit of a "one size fits all" metric.

dice wrote:
The prior-informed version let's the consistency between the years smooth out those bad assumptions.

does it treat all seasons as equivalent regardless of games played? that would be a major flaw


That question could mean a lot of different things, so I'm not sure what exactly you want to know.

If what you're asking is whether each prior for each player is equally valid in its assessment of the player, the answer is a clear no. It's an imperfect metric, and understanding the flaws is crucial if you're going to use it. I don't find it difficult to navigate through those issues with my analysis. It's not like you don't know if a player had something weird happen the previous year, so being aware of such issues you keep that in mind when you see what the stat says.

dice wrote:
This gets concrete in a hurry when you compare Duncan to Garnett. There's a recurring theme where Duncan wins by NPI while Garnett wins by PI.

how is that possible? obviously one guy could win the occasional season NPI with the other guy winning PI every year due to smoothing, but i don't understand how a player could be consistently winning one and losing the other


All of this is a bit tough because we're using words here - and frankly not only am I not one of the top mathematical experts on the thing, I haven't thought about it in a while so I'm a bit rusty with it. I'll try again though:

The issue with RAPM in general is that it's only an improvement on APM most of the time - not all of the time. If there were no luck involved in the game, and no specific matchup-based distortions, then APM would be the perfect stat. Those things do exist though, so the mathematical technique called regularization is used to smooth things out. Outlier data in general in statistics is unlikely to continue as was done in the sample, and thus this technique is a way to diminish the impact of this data that's presumed to be noise.

If we look at Kevin Garnett in his peak Minnesota years using one year samples, the effect of this is to take him from being better than Tim Duncan by a pretty wide margin, to being behind Duncan. Or to paraphrase: "Yeah, that's a fluke, if we adjust based on what we typically see in the league, Duncan's the more proven of the two players".

Now, even before we get to using priors, there's a weird thing going on here. Seeing that happen simply once makes the skepticism absolutely warranted, but if it's happening year after year, then that's your first hint that perhaps this outlier data isn't actually noise.

So then we get into the use of priors. In general what priors do is help ensure that if a player is roughly the same player from year to year, his results will be more consistent from year to year. More signal, less noise. So, if you've got a player who has something effectively being classified as noise, but it proves to be a repeating trend from year to year, it's going to stop being dismissed as noise. And if that assumed noise is stuff that makes the player in question look better once you factor that in, well the PI metric is going to rate the player higher each year than the NPI does.

So to summarize: We've got APM, NPI RAPM, and PI RAPM. Each metric amends the previous to make the data more reliable, but there exists a particular course of events that leads to the first & third methods to agree with each other relative to the middle metric, and in such a case that indicates the middle metric is making things worse for that particular player.

Is that what happened with Garnett? Well, let's look at colts18's alternative in the next post.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#22 » by Doctor MJ » Mon Sep 22, 2014 5:07 am

colts18 wrote:
dice wrote:how is that possible? obviously one guy could win the occasional season NPI with the other guy winning PI every year due to smoothing, but i don't understand how a player could be consistently winning one and losing the other


It's due to a flaw in the PI RAPM stats the Dr. MJ uses. The PI RAPM he uses had KG ahead of Duncan because a few of those years were missing like 30% of the data. In the 100% complete NPI RAPM data, Duncan finishes ahead of KG so he would have likely finished ahead of him in the PI RAPM too.


So, what colts argues is that the PI data is incomplete but the NPI data is complete, and so a truly complete PI result would agree with the NPI data.

So, if this is the case, this is possible of course.

First thing I"ll say: I really don't remember what all has been said about what specific things are incomplete. I would appreciate if colts could go into specific on that. If it seems weird that the NPI is always complete when the PI isn't, yeah, I think so too.

More importantly though is the sheer scale of the disagreement between the two stats.

Going through the years, here's who wins between Duncan & Garnett by each metric up through 2012:

Year NPI PI
1998 Duncan Garnett
1999 Duncan Garnett
2000 Duncan Garnett
2001 no PI data
2002 Duncan Duncan
2003 Duncan Garnett
2004 Duncan Garnett
2005 Duncan Duncan
2006 Duncan Duncan
2007 Duncan Duncan
2008 Garnett Garnett
2009 Garnett Garnett
2010 Duncan Garnett
2011 Garnett Garnett
2012 Garnett Garnett

If you go by NPI, Duncan wins 10-3.
If you go by PI, Garnett wins 9-4.

It's pretty insane really. When we first started seeing these trends I basically punted on what it meant because I doubted it would hold up year after year. Now that we see the two metrics disagreeing, and disagreeing in the exact same way enough times to shift the edge from a blow out one way, to a blow out in the other direction, you really can't punt any more and still insist on an opinion.

So colts is in one direction, I'm in another.

Here's the thing though: colts entire explanation is based on the assumption that there's missing data in many different years that's letting Garnett look like he has the edge when he really doesn't. I mean, we aren't even talking about the same statisticians here. colts is referencing what J.E. did in the early '00s here, whereas the '90s data is from across the court using the data the NBA later released.

You have to ask yourself: What are the odds that there's missing data in all of these years, and for some reason, the data that's lost is always the data that would make Garnett look way worse than the rest of the data indicates?

imho, the odds of this happening are very slim, and even if you personally conclude the odds are more plausible than that, it's still an explanation made entirely based on the reasoning that we just got unlucky with the incomplete sets of data. There's literally nothing in the entire line of thinking that has anything to do with the stat itself and how it's made. One could make this argument while understanding nothing about the difference between the two at all.

By contrast, as you've seen, whether or not you believe my argument, it's entirely based on how the stats in question actually work. Quite honestly, perhaps someone could come in and argue against my statement of how the stats work...but no one has. Not colts, not anyone else.

So short of that happening, it's a question of what you find more credible: That this is a natural phenomenon of the differences between the two stats, or That this is due to bad luck happening again and again.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#23 » by Doctor MJ » Mon Sep 22, 2014 5:17 am

Ah, one other thing I don't think I made clear:

Do you see how Garnett finally starts winning the NPI comparison when he goes to Boston?

The specific issue causing him to be underrated by NPI is the fact that his teammates performed so, so bad without him in Minnesota. If you go and look at 82games early in an NBA season, you'll see that a lot of players have astronomical "off" numbers. That's because a star players way more minutes than he sits, and so the sample size in the "off" data is lower, and more prone to noise creating absurd data. Even over the course of an entire season, this doesn't completely disappear.

So if we've got a player whose on/off data gives him a very high rating because it says his teammates are terrible, it makes sense to hold on to your wallet. APM does stuff to smooth this out, but it's not as good as RAPM, and in fact it turns out that with someone like Garnett it was "too good" because those teammates looked like that year after year.

Once Garnett goes to Boston and has good teammates, he stops having that factor work against him. He's getting big +/- numbers because of his own raw on-court data, like stars on good teams always have, and this isn't something that RAPM tends to dismiss as noise.

Last thing I'll say: What about 2010? Well, a one year shift in NPI, if it's backed up by something else we know to be true, is exactly the reason why for some in any given year the NPI data is going to be more meaningful than the PI data. Garnett just didn't play as well in 2010 as he did the prior year, and so using the PI data there overrates him.

So in that instance, it is indeed luck that's making that year go along with the rest of that disagreement trend. The other 5 years though? I think by far the best explanation comes from delving into how the 3 different stats I've talked about actually work.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
dice
RealGM
Posts: 38,765
And1: 10,801
Joined: Jun 30, 2003
Location: chicago

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#24 » by dice » Mon Sep 22, 2014 2:42 pm

appreciate the explanations

xRAPM: bastardized stat due to addition of boxscore data?
PI: how far back is it 'informed'? 1 year? 5 years? entire career?
"'tupelo honey' has always existed. van was the vessel and the earthly vehicle for it" - bob dylan
colts18
Head Coach
Posts: 7,226
And1: 2,997
Joined: Jun 29, 2009

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#25 » by colts18 » Mon Sep 22, 2014 3:18 pm

dice wrote:appreciate the explanations

xRAPM: bastardized stat due to addition of boxscore data?
PI: how far back is it 'informed'? 1 year? 5 years? entire career?

1. It's not a bastardized stat. It uses boxscore data to stabilize. It's more reliable than regular RAPM and more predictive of future outcomes. The reason why some think its bastardized is because some people like the fact that RAPM has no box score influence at all (its the only stat like that out there). They would rather weigh the box score stuff with RAPM theit own way

2. It depends really. You would have to ask acrossthecourt how he did it. I believe its informed as far back as the RAPM creator has data from. So for the 02-12 RAPM that J.E. created, I think its informed all the way to 2002. For the 98-00 RAPM, its informed back to 1997.
dice
RealGM
Posts: 38,765
And1: 10,801
Joined: Jun 30, 2003
Location: chicago

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#26 » by dice » Mon Sep 22, 2014 3:27 pm

colts18 wrote:
dice wrote:appreciate the explanations

xRAPM: bastardized stat due to addition of boxscore data?
PI: how far back is it 'informed'? 1 year? 5 years? entire career?

1. It's not a bastardized stat. It uses boxscore data to stabilize. It's more reliable than regular RAPM and more predictive of future outcomes.

more predictive of what? the next year's xRAPM? that doesn't help me if i don't like xRAPM

The reason why some think its bastardized is because some people like the fact that RAPM has no box score influence at all (its the only stat like that out there). They would rather weigh the box score stuff with RAPM theit own way

exactly. and by this thought process it IS bastardized. i don't want someone making their determination of how various boxscore stats should be weighted and have that mixed in with RAPM, which is supposed to be completely unbiased

2. It depends really. You would have to ask acrossthecourt how he did it. I believe its informed as far back as the RAPM creator has data from. So for the 02-12 RAPM that J.E. created, I think its informed all the way to 2002. For the 98-00 RAPM, its informed back to 1997.

this is why i don't trust the individual year numbers for PI. a guy could have a legitimately bad or "off" year and have it "smoothed" into a good year. of course, the only alternative seems to be the very noisy and unreliable NPI
"'tupelo honey' has always existed. van was the vessel and the earthly vehicle for it" - bob dylan
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#27 » by Doctor MJ » Tue Sep 23, 2014 1:21 am

dice wrote:appreciate the explanations

xRAPM: bastardized stat due to addition of boxscore data?
PI: how far back is it 'informed'? 1 year? 5 years? entire career?


xRAPM is "bastardized" RAPM, but you could also say RAPM is bastardized APM. In both cases you're sacrificing validity for reliability, and on average it's going to give you better results if you look at that stat alone.

What is so problematic to me about xRAPM as an analyst, is that I'd never only use 1 metric to do my analysis, and by adding box score data and other stuff into the mix he essentially makes me double count things in ways I can't hope to discern. Given that it came about with JE taking actual RAPM off his site, it's even worse. It's somewhat like b-r taking out their standard box score and only showing us PER. On its own it's a useful number, but it's far less useful than all those numbers separated.

None of this means I'll never end up using something called xRAPM, but it's problematic, and JE"s response to the complaints left me simply not trusting his future work. Put least damningly: Guys like us aren't his target audience. He wasn't really looking to provide data for use by internet experts, he was flexing his math muscles as advertisement for someone to pay him. I don't begrudge him that, it's just that he could achieved this without confusing a lot of people.

Re: How far informed. My understanding is that typically they use the non-prior informed data from the previous year. In general you want to be careful letting more distant stuff influence the ratings because the further away it is in time, the more you're literally letting a players of different capability influence their later selves.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#28 » by Doctor MJ » Tue Sep 23, 2014 1:37 am

dice wrote:more predictive of what? the next year's xRAPM? that doesn't help me if i don't like xRAPM


More predictive of wins the next year. As in, can you use the stat for all the players on all the teams, and accurately predict how they'll do the next year? It's a method for evaluate statistical methods that is commonly used, and it's definitely a meaningful thing.

The thing is though: That's an all or nothing method. It's basically saying, if I use this stat, and only this stat, which gets me closest to the truth. But the right thing to do isn't to use only any 1 stat, it's to use many and figure things out from there.

More concretely: Many of the players where this alloyed xRAPM stat gains a serious advantage over RAPM, are players that it's obvious to anyone who knows what they are doing that you should be careful using regression data on - meaning you shouldn't be putting much stock in RAPM, xRAPM, or any other competitor.

To me as an analyst, it's a little like an advertising campaign along the lines of "Buy Brand X Human Scales rather than Brand Y, because Brand X won't be quite as inaccurate when you use it to weigh a hamster."

It's not worth nothing, but it's somewhat like training wheels. It helps keep the novice from falling flat over, but you're not going to be using them on the mountains of France.

dice wrote:
2. It depends really. You would have to ask acrossthecourt how he did it. I believe its informed as far back as the RAPM creator has data from. So for the 02-12 RAPM that J.E. created, I think its informed all the way to 2002. For the 98-00 RAPM, its informed back to 1997.

this is why i don't trust the individual year numbers for PI. a guy could have a legitimately bad or "off" year and have it "smoothed" into a good year. of course, the only alternative seems to be the very noisy and unreliable NPI


If you're really looking to analyze a particular player in a particular year, I'd certainly advise you to look at both PI & NPI along with a whole range of other non-regression stats.

For a spreadsheet like I've made though, if the goal is to get the best understanding of how a player's impact was through his prime (assuming a typical career), the boost of reliability PI gives you is valuable and the concerns of years bleeding into each other to me seem largely moot.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
dice
RealGM
Posts: 38,765
And1: 10,801
Joined: Jun 30, 2003
Location: chicago

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#29 » by dice » Tue Sep 23, 2014 2:27 am

Doctor MJ wrote:
dice wrote:more predictive of what? the next year's xRAPM? that doesn't help me if i don't like xRAPM


More predictive of wins the next year. As in, can you use the stat for all the players on all the teams, and accurately predict how they'll do the next year? It's a method for evaluate statistical methods that is commonly used, and it's definitely a meaningful thing.

meaning that you could take a random sampling of 12-15players on various teams and reasonably predict how well they'd do if they were on a team together?

The thing is though: That's an all or nothing method. It's basically saying, if I use this stat, and only this stat, which gets me closest to the truth. But the right thing to do isn't to use only any 1 stat, it's to use many and figure things out from there.

this guy took it a step further:

http://sportskeptic.wordpress.com/2012/01/25/nba-retrodiction-contest-part-3-the-perfect-blend/
"'tupelo honey' has always existed. van was the vessel and the earthly vehicle for it" - bob dylan
dice
RealGM
Posts: 38,765
And1: 10,801
Joined: Jun 30, 2003
Location: chicago

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#30 » by dice » Tue Sep 23, 2014 2:31 am

Doctor MJ wrote:Re: How far informed. My understanding is that typically they use the non-prior informed data from the previous year.

i doubt that's true simply because NPI is wildly variable for a single season. to use that one year for PI would still result in very funky results. but what i've seen of PI is relatively stable from year to year
"'tupelo honey' has always existed. van was the vessel and the earthly vehicle for it" - bob dylan
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#31 » by Doctor MJ » Tue Sep 23, 2014 2:39 am

dice wrote:
Doctor MJ wrote:
dice wrote:more predictive of what? the next year's xRAPM? that doesn't help me if i don't like xRAPM


More predictive of wins the next year. As in, can you use the stat for all the players on all the teams, and accurately predict how they'll do the next year? It's a method for evaluate statistical methods that is commonly used, and it's definitely a meaningful thing.


meaning that you could take a random sampling of 12-15players on various teams and reasonably predict how well they'd do if they were on a team together?


More like Method X would do better than Method Y, regardless of how reasonable it all is.

Most definitely: The more you randomize the players, the worse all methods would do in a sport like this, but even predicting this stuff well when players stay on the same team is tough, and very valuable if you can do it well.

dice wrote:
The thing is though: That's an all or nothing method. It's basically saying, if I use this stat, and only this stat, which gets me closest to the truth. But the right thing to do isn't to use only any 1 stat, it's to use many and figure things out from there.

this guy took it a step further:

http://sportskeptic.wordpress.com/2012/01/25/nba-retrodiction-contest-part-3-the-perfect-blend/


Yup, I've seen his work. Like I say, cool stuff, but I think you're better off using a variety of methods plus your own head if you know your stuff.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#32 » by Doctor MJ » Tue Sep 23, 2014 2:44 am

dice wrote:
Doctor MJ wrote:Re: How far informed. My understanding is that typically they use the non-prior informed data from the previous year.

i doubt that's true simply because NPI is wildly variable for a single season. to use that one year for PI would still result in very funky results. but what i've seen of PI is relatively stable from year to year


Put it like this:

The earliest NPI data we have is from '97.
The earliest PI data we have is from '98.

That '98 PI is absolutely using the NPI data from '97 as it's prior, and therefore if the subsequent PI's use the same method, they'll be using NPI also.

Not saying that makes it a given, but that's the starting point.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#33 » by Doctor MJ » Sun Oct 5, 2014 8:09 pm

Head's up: lorak was trying to work with this data himself and couldn't duplicate my 1998 results. Comparing our data we realized that he had way more players from 1998 than me. This was probably caused by me somehow deleting them, but regardless:

This would lower the data's standard deviation I recorded for that year, and thus cause individual player's results in that year to have inflated values compared to other years.

So you know how we seemed to see some big values for 1998 and we wondered about that? This might be the reason.

I've checked all my other years from 1999 to 2012, none of them have the same issue. So just be cautious with the 1998 numbers for now.

Thank you lorak!
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#34 » by Doctor MJ » Sun Oct 5, 2014 8:24 pm

I don't have time to do all the fixes today, but here's what I'll say:

The correct values for 1998 I believe will be about 77% of what you currently see.

Some players we've discussed:

Overall Scaled RAPM:
Shaq's numbers previously 12.8, now it's 9.87, which makes it only his 2nd best number behind his 10.7 rating in 2000.
Zo's number drops from 11.9 to 9.16, which makes his 199 number of 9.95 his best performance.
Stockton's number drops from 9.03 to 6.95.
Mutombo drops from 6.89 to 5.31.
Payton drops from 7.54 to 5.81
Mookie drops from 11.0 to 8.48
Reggie from 8.32 to 6.41

Additionally on defense this drops Mutombo's outlier good 9.74 to 7.5, which is still phenomenonal but not as amazing.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
User avatar
Jaivl
Lead Assistant
Posts: 5,405
And1: 4,695
Joined: Jan 28, 2014
Location: Lugo, Spain
Contact:
   

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#35 » by Jaivl » Sun Oct 5, 2014 10:02 pm

Doctor MJ wrote:Overall Scaled RAPM:
Shaq's numbers previously 12.8, now it's 9.87, which makes it only his 2nd best number behind his 10.7 rating in 2000.
Zo's number drops from 11.9 to 9.16, which makes his 199 number of 9.95 his best performance.
Stockton's number drops from 9.03 to 6.95.
Mutombo drops from 6.89 to 5.31.
Payton drops from 7.54 to 5.81
Mookie drops from 11.0 to 8.48
Reggie from 8.32 to 6.41

Welp, it's quite a big deal for me. So maybe Deke's defensive edge is not thaaaat big. Sure it'll come in handy in the next threads of the all-time list. Thanks to both Lorak and you.
Maf wrote:I'd undestand if anyone had KG outside top ten PF's. Having him top five all-time? Often I jokingly rank Kyle Korver as the GOAT but I never try to fake serious discussion about it.

ShawnKemp96 wrote:Infact he made a lot more steals than the statisticians think.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#36 » by Doctor MJ » Sun Oct 5, 2014 10:06 pm

Jaivl wrote:
Doctor MJ wrote:Overall Scaled RAPM:
Shaq's numbers previously 12.8, now it's 9.87, which makes it only his 2nd best number behind his 10.7 rating in 2000.
Zo's number drops from 11.9 to 9.16, which makes his 199 number of 9.95 his best performance.
Stockton's number drops from 9.03 to 6.95.
Mutombo drops from 6.89 to 5.31.
Payton drops from 7.54 to 5.81
Mookie drops from 11.0 to 8.48
Reggie from 8.32 to 6.41

Welp, it's quite a big deal for me. So maybe Deke's defensive edge is not thaaaat big. Sure it'll come in handy in the next threads of the all-time list. Thanks to both Lorak and you.


Yeah, I'm kicking myself and a bit embarassed. It is a big deal, because people were using these numbers. I was using them too, but I was also trying to urge caution - not because I thought I made a mistake so much as that with anything new at the very edge, if it differs from everything else, I'd like to see it seconded.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
colts18
Head Coach
Posts: 7,226
And1: 2,997
Joined: Jun 29, 2009

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#37 » by colts18 » Tue Oct 7, 2014 5:08 pm

I noticed that 2 weeks ago. I was looking for the ratings of some Bulls players but I couldn't find a few of them. Turned out you were missing a lot of players.


Also another issue with your spreadsheet is that it lacks consistency. You will have players like Scottie Pippen or David Robinson listed 2 or 3 times. They should only be listed once.
Doctor MJ
Senior Mod
Senior Mod
Posts: 45,346
And1: 14,305
Joined: Mar 10, 2005
Location: Cali
     

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#38 » by Doctor MJ » Tue Oct 7, 2014 10:52 pm

colts18 wrote:I noticed that 2 weeks ago. I was looking for the ratings of some Bulls players but I couldn't find a few of them. Turned out you were missing a lot of players.


Also another issue with your spreadsheet is that it lacks consistency. You will have players like Scottie Pippen or David Robinson listed 2 or 3 times. They should only be listed once.


To be clear, this is 2 separate issues, one big, one small.

1998 is missing a ton of players. It's a big deal, it resulted in the data shown being considerably inflated.

Beyond that, the issue is that I'm taking data from different sources, and if the formatting was different I have to fix it in order to be perfect. I did a good amount of this, but no, I didn't fix it all.
Hey: With what's going on in the world, my fuse is shorter than it used to be, and it's leading my lose my cool and then go on self-imposed breaks from things (such as RealGM). Please try to keep it civil, and I'll be looking to do the same.
drza
Analyst
Posts: 3,506
And1: 1,819
Joined: May 22, 2001

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#39 » by drza » Wed Oct 8, 2014 3:42 am

That IS a huge catch, and I appreciate it. I've been mentally taking the 1998 data with a bit of a grain of salt, but lacking a reason for the inflated values I continued to use those numbers. It's absolutely wonderful to see a quantitative reason, and thus a correction made. The Shaq 1998 vs 2000 was bothering me, for example.

Doc, you haven't updated the spreadsheet yet, have you? Do you plan to?
Creator of the Hoops Lab: tinyurl.com/mpo2brj
Contributor to NylonCalculusDOTcom
Contributor to TYTSports: https://www.youtube.com/playlist?list=PLTbFEVCpx9shKEsZl7FcRHzpGO1dPoimk
Follow on Twitter: @ProfessorDrz
lorak
Head Coach
Posts: 6,317
And1: 2,226
Joined: Nov 23, 2009

Re: Normalized & Scaled RAPM Chronology Spreadsheet 

Post#40 » by lorak » Wed Oct 8, 2014 6:48 am

^
Before any updates we should discuss methodology Doc used - so for example if using "0" instead of real sample mean is right thing to do? If so what about also adjusting "true standard deviation"? I also would like to see how "scaled RAPM" is calculated?

There are also some issues with source data Doc used - obviously 1998 sample wasn't complete, but also basically every year done by Engelmann has some small noise (some players listed twice, some didn't play in a given year). I actually cleaned all that data, but before I make "normalized" update I would like to know answers to questions listed above.

Return to Statistical Analysis