2015 APM/RAPM/etc.

[Quotatious]

If there's no consensus about a player's peak (like LeBron's, for example), I think we should create a separate thread with a poll, opened for 24 hours, and then agree to use the year that wins the poll as his peak (or maybe not a poll, but just a separate thread where the eligible voters would provide reasoning for their choice). Anyway, I see LeBron's 2009, 2012 and 2013 seasons as roughly equal (I like '09 the most, but I can see a good argument for the other two, as well), so it shouldn't really matter which year I would select.

[Dr Spaceman]

If there's no consensus about a player's peak (like LeBron's, for example), I think we should create a separate thread with a poll, opened for 24 hours, and then agree to use the year that wins the poll as his peak (or maybe not a poll, but just a separate thread where the eligible voters would provide reasoning for their choice). Anyway, I see LeBron's 2009, 2012 and 2013 seasons as roughly equal (I like '09 the most, but I can see a good argument for the other two, as well), so it shouldn't really matter which year I wouls select.

This is a really good idea. We do this pre-project and then we have it nailed down before we go in at all, and this way everyone can be clear on where they stand. Definitely the best way to avoid vote splitting.

[Doctor MJ]

Do we have instances in previous years where multiple NPI samples are available? I'm almost certain this exact scenario played out in 2012 as well, we had NPI from like April and then got an update around the Finals which had similarly wild swings, and the conclusion at that time was the same - that NPI was just too noisy within single seasons.

My gut tells me these discussions were either in the Peaks project (it's been 3 years, we should consider this for the summer) or the POY threads for that year. I'll start skimming through them to see if I can find anything.

colts has the older 2012 NPI set archived: Link.

There was an update in I want to say June and that's the one I'm looking for.

I clearly remember Elgee complaining about the wild swing in NPI RAPM during the 2012 season. It was probably the Peaks project because we ran that in 2012. Look through ElGee's posts and you should find it.

Yeah so like you said, this is something of a known issue, which is why I wouldn't advise people to simply look at just this stat, or really just any stats.

Here's something else to ponder: APM and all other stats in the family are essentially saying "This is the simplest explanation for what we've seen in this sample.", as opposed to box score-based stats that simply accumulate over time. As a result you can visualize the results of the regression that drives these stats being something like a Chladni plate:

https://www.youtube.com/watch?v=YedgubRZva8

(Watch for at least 40 seconds, maybe with the sound down.)

Thing stay the same, stay the same, stay the same...but then every once in a while something happens that changes the balance can affect literally every thing in the entire study. You can visualize the "R" in RAPM as essentially being something that weights things in a particular direction that limits the likelihood of something like this happening by making some assumptions about where the numbers ought to be. And that's an improvement so long as those assumptions are right - the good news being that we can largely tell when those assumption should be right and when they should not. The more a player is known to have changed in ability/role/context/whatever, the more you simply have to rely on pure APM, and the less overall confidence you can have about his impact.

If me comparing these numbers to a Chladni Plate makes you feel like you want to ignore them altogether, I understand, but at least until player tracking data analytics gets really damn good, it means that you're choosing to ignore something that can be incredibly informative.

Also, if your hang up is that you want to use this stuff simply for saying "Who is actually better?", keep in mind that nothing does that perfectly, and that the real answer to how to approach that answer is to gain an understanding and an intuition about a wide variety of metrics and use them to build off of what you already know.

[Doctor MJ]

Reinforces that Harden should be the MVP

Not necessarily.

The data reinforces the idea that Harden is providing the most raw lift to his team, and if that's how you define MVP so be it.

I would recommend thought that people remember that the goal of a player isn't simply to help teams go from bad to good, but to help them become truly great. While I personally love Harden - dude's from my neck of the woods, is smart, and he's a bearded - and while I think he can be the best player on a championship team, his impact currently is based on a style of play that taken on its own seems quite unlikely to scale to team greatness.

Put another way: If I'm building a team to win a championship and I don't think the ankle issues are enough to sway my choice, I'd pick Curry over Harden simply because in addition to the active threat of him with the ball, his mere presence out there has a gravity unlike anyone else in the league that makes it I think quite likely that the players around him will continue to look fantastic, and I also think this is something that you can't really get much better at defending in the playoffs.

Put into math: Generally speaking, I'm more impressed with a guy doing this on better teams than on weaker teams.

[Doctor MJ]

I am just surprised that Harden has a massive lead in RAPM by about 20%, that's noise to me already. If Harden was the more efficient player or his team had a better SRS, or his on-off numbers were better or his WS48/BPM/RPM were better, then it would make a little sense.

Everything here has a lot of sense here, we just have to look at full context. Harden has +10.1 RAPM, Curry +8.4, but James plays on +3.5 SRS team, while Steph +10.6, so what Curry has done is more impressive. It's like - for example - improving 50 W% team by 10 wins vs 65 W% by 8 wins. "8" is lower number here, but in fact means impact is greater, because it's more difficult to improve so much better team.

If that's the case that should be factored too. SRS and team record are not hard to look up and get weighted into the model.

Actually I disagree with this completely, but the point is one that people who are in this community don't necessarily agree with me on.

My perspective: What I want is a stat that means something that can be expressed with relative simplicity. Because I start already knowing that no stat will be good enough on its own that I can forego other kinds of analysis, and the more convoluted they make the stat, the more difficult it is to combine these things in my head into something meaningful.

So APM in essence can be described as "lift": How much better is this guy making his team based on our best estimates.

RAPM and especially xRAPM/RPM add wrinkles to this that I'm quite wary about because they violate this principle of conceptual simplicity. And this isn't simply opinion, it relates to why Engelmann created xRAPM in the first place: He wants to do what you want, create the single best possible all-in-one stat. What I've tried to explain to him and to others is that his stat is only useful to the point that analysts can use it, and if you make the stat too complicated, you make that task tougher not easier. Most statisticians undoubtedly dislike this, because there's an implicit statement that that means that they themselves are not the best analysts. But generally speaking, they aren't. You need more than just algorthmic know-how to do the job of the analyst, and while the Dean Olivers of the world have the whole toolbar, most the APBRmetric guys don't.

More specifically: People saying that it's typically more impressive to do this on a better team is not the same thing as is definitively being so. As such, having a metric that looked to weight this in would mean something adding something convoluted on top of the simple & pure stat which isn't simply a problem for those who philosophically disagree with doing so, it's also a problem for someone like me because I have no reason to trust that the statistician can do a better job of it by trapping it in a math equation than I can do holistically in my head. And anything I can do in my head, I don't need a stat for.

Incidentally talk like this has left some APBRmetrics people in the past essentially saying I favor the silly narratives that dominate drunken sports conversations over the hard math, but sufficed to say my math abilities are very strong and I rely on them a lot. It's just that I also have experience in recognizing that simply setting your holistic methods into mathematical concrete doesn't mean you have something better. You have something you can share with other people, but in cases like this if you're aware and honest with yourself, you know that the process in your mind is something that more closely resembles a program rather than a formula, and it's very difficult to iron that entire process out. My hat's off to those who do it successfully, but for the most part statisticians don't actually even attempt this level of nuance.

Also GSW is 59-13, if Curry is only +8.4, they are a 51 win team without him? something doesn't add up.

As others mentioned I think, an SRS improvement of 1 does not mean 1 more win in 82 games. Just two entirely different scales. There are way to try to map the two together if you're really interested, but this doesn't really have much bearing on analysis of player performance in my experience.

[Doctor MJ]

If there's no consensus about a player's peak (like LeBron's, for example), I think we should create a separate thread with a poll, opened for 24 hours, and then agree to use the year that wins the poll as his peak (or maybe not a poll, but just a separate thread where the eligible voters would provide reasoning for their choice). Anyway, I see LeBron's 2009, 2012 and 2013 seasons as roughly equal (I like '09 the most, but I can see a good argument for the other two, as well), so it shouldn't really matter which year I wouls select.

This is a really good idea. We do this pre-project and then we have it nailed down before we go in at all, and this way everyone can be clear on where they stand. Definitely the best way to avoid vote splitting.

So here's what I'll say as the guy who ran this project and watched it die:

The real reason it died is not because it made people choose which year was the best, it was because when people are honest, they find rating a player's peak a lot harder than rating his career. This is very interesting by the way, because in an ideal analysis of the career, you'd start by accurately rating each year in a player's career, and as such career measurement SHOULD be harder than peak measurement, but in actuality that's not how we evaluate careers. In short, we cheat, and work from the top down. Only allowing peak superiority to get involved when that factor is salient enough that it dominates the comparison in question.

Okay now practically speaking I'll say a few things:

1) If you guys want to re-run this project, go for it. No one will stop you.

2) It will be much more likely to succeed though if you have a mod actively involved, and that won't be me unfortunately. Feel free to talk to other mods though.

3) In my experience as a project runner on this board for many years, the really big projects take up "energy". And so after one big one goes, there needs to be time to re-charge. As such for example, the next project I'd like to do is an analog of the RPOY but with OPOY and DPOY, but with a Top 100 project still running, and using the last fumes of energy the core group of the board has, I don't intend to do it this summer - maybe next summer instead, maybe a bit earlier in the spring.

As such you can try to do it right now, but I wouldn't be surprised if you get a lackluster response at this moment.

Nevertheless, I like your enthusiasm, and I'm really happy to have both of you on this board really getting into it and up'ing the brain power of the community.

[PaulieWal]

3) In my experience as a project runner on this board for many years, the really big projects take up "energy". And so after one big one goes, there needs to be time to re-charge. As such for example, the next project I'd like to do is an analog of the RPOY but with OPOY and DPOY, but with a Top 100 project still running, and using the last fumes of energy the core group of the board has, I don't intend to do it this summer - maybe next summer instead, maybe a bit earlier in the spring.

Doc, just to be sure we are still going to be doing POY discussion and voting still, right? I am actually really looking forward to that given there is no clear-cut best player in the league this year and it will make the discussion 10x more fun/intriguing.

[MisterHibachi]

Peaks project (it's been 3 years, we should consider this for the summer)

Absolutely we should. I wasn't around the last time they did it, and that project and the RPOY project are things I'm very sorry I missed out on. RPOY obviously can't really be redone due to logistics and attrition, but the peaks project is absolutely something that could be updated.

If I'm not mistaken, that project died last time around #30, correct? We'd have to figure out a way to prevent that, and whether we can do both that and Hibachi's project at the same time (is that still going forward?).

That was the plan. But not sure now because of Doc's comment about energy level on the board a couple posts up, but maybe the Jordan project will aim at a different membership as I'm sure most of the guys voting in the top 100 already have good knowledge of Jordan's game. I'll gauge interest and start a thread with the preliminary discussions in about a month, after my exams are over.

[Doctor MJ]

3) In my experience as a project runner on this board for many years, the really big projects take up "energy". And so after one big one goes, there needs to be time to re-charge. As such for example, the next project I'd like to do is an analog of the RPOY but with OPOY and DPOY, but with a Top 100 project still running, and using the last fumes of energy the core group of the board has, I don't intend to do it this summer - maybe next summer instead, maybe a bit earlier in the spring.

Doc, just to be sure we are still going to be doing POY discussion and voting still, right? I am actually really looking forward to that given there is no clear-cut best player in the league this year and it will make the discussion 10x more fun/intriguing.

Definitely. Actually, let me respond on the RPOY thread for that.

[SideshowBob]

I tweeted JE/Gotbuckets/TalkingPractice on whether there's an updated PI set ("Pure" RAPM, year n informed by n-1) available for 2015, but no response yet.

Kinda wish mystic was still around, he pretty frequently gave updates on his own RAPM and blended metrics, and of the guys who frequently run these numbers had the best basketball grasp IMO.

[SideshowBob]

Just a note, the data in the first post that was initially requested by fpliii from JE is multi-year RAPM, NOT PI RAPM:

So I've come to the understanding that I was mistaken earlier, the set that JE posted in this thread was 2015 multi-year RAPM, not "pure" (15 informed by 14) RAPM.

Is there a prior-informed 2015 set out there? GotBuckets.com had been posting talkingpractice's PI sets from 08-14, but both sites have yet to update for 2015 (we haven't seen IPV for this year either).

I really doubt that there'll be any significant differences between
a) 2015 RAPM informed by 2014 (which, in turn, is informed by 2013 etc)
b) Multiyear RAPM that uses data from 2015, 2014, etc and gives less weight to older seasons

If anything, b) makes more sense - at least in my mind

[acrossthecourt]

Single year NPI RAPM is really wonky and shouldn't be taken too seriously. It's useful to study when you're looking for hidden gems or weird trends, but it has a lot of issues.

I like mutli-year RAPM a lot more because you get a lot more useful information. It's not just having more years. It's about seeing more players on different teams. Guys don't switch teams as often as the do in the off-season, and lineups are mixed more year to year.

Not that NPI RAPM is completely useless. It's funny how close its prediction power is to something like PER, which people still inexplicably love.

Of course, adding box score information (and other stats) is much better.

Anyone taken a look at Got Buckets' FourFactor APM? Its multiyear, they have data for the 2015 season (that is currently using data from Feb 2013-Feb 2015), but only in offensive/defensive splits. It'd be good to compare to previous seasons (they go back to 2005).

Offense

Defense

Here is how anyone that got an MVP top 10 vote last year is doing according to this stat and I listed it from largest to smallest

Chris Paul = 9.07/3.07 = 12.14
LeBron James = 9.07/2.14 = 11.21
Stephen Curry = 12.93/-2.88 = 10.05
James Harden = 11.64/-2.29 = 9.35
Kevin Durant = 5.80/0.56 = 6.36
Lamarcus Aldridge = 2.64/1.95 = 4.59
Joakim Noah = -4.48/5.68 = 1.20
Blake Griffin = 0.10/-0.64 = -0.54
Kevin Love = -0.45/-0.78 = -1.23
Al Jefferson = -4.25/-1.61 = -5.86

Also is their a bias when it comes to big men on offense because both gasols, demarcus cousins are all in the negative, blake is barely above zero and LaMarcus is the first true big man ( PF, PF/C or C to be listed and he's only at 2.64, then Davis who is a great offensive player on a good offensive team is only at 2.08??? )

It's pretty common looking at RAPM for the past 17 or so years that big men are rarely elite on offense. Outside of a jump-shooting big guys like Dirk and specialists like Ryan Anderson, it's a pretty short list.

There's nothing "wrong" with the numbers in that respect. It's a common trend.

Morrow over Davis. I am not sure if this is noise or nonsense. lol

You have to remember that RAPM has to be used in conjunction with usage/role. Morrow is just a role player, his usage is not nearly as high as AD's, and opposing teams don't gameplan against him like they do against Davis.

RAPM results should only be compared among players of similar caliber/role/usage. Otherwise you'll often get nonsense like Nick Collison ahead of Dirk etc.

that's part of the things that advance stat should do already. Refine the model to eliminate more noise, giving better weights on different playing style, etc. Harden and Curry play the game very differently, high efficiency vs high usage, how does RAPM measure one against the other?

That's called RPM. People have been doing tweaks like this for a while.

[ceiling raiser]

Thanks to SSB, J.E. noted that he'll update his set (multi-season) after the playoffs.

I really doubt that there'll be any significant differences between
a) 2015 RAPM informed by 2014 (which, in turn, is informed by 2013 etc)
b) Multiyear RAPM that uses data from 2015, 2014, etc and gives less weight to older seasons

If anything, b) makes more sense - at least in my mind

Appreciate the response

Has that set been updated? I believe it was using data up to mid-January.

Haven't updated since, but will definitely update after the playoffs end

In the meantime, EvanZ has supplied his NPI set:

https://docs.google.com/spreadsheets/d/10sUwyeaLx3VP664y9bBu5vAHGY5Z17LEA4jkxvAnmsY/edit#gid=454809408

[colts18]

Rank Player (O+D)RAPM ORAPM DRAPM
1 James Harden 10.06 6.30 -3.75
2 Draymond Green 8.38 3.36 -5.02
3 Stephen Curry 8.35 4.97 -3.39
4 Manu Ginobili 8.22 3.06 -5.16
5 Anthony Morrow 7.97 5.64 -2.33
6 Kyle Korver 7.04 5.67 -1.36
7 Anthony Davis 6.81 2.95 -3.86
8 Kawhi Leonard 6.73 2.40 -4.33
9 Kyle Lowry 6.66 3.28 -3.39
10 Damian Lillard 6.42 3.17 -3.26
11 Zach Randolph 6.39 3.61 -2.78
12 Chris Paul 6.36 4.36 -2.00
13 LeBron James 5.91 5.12 -0.79
14 Markieff Morris 5.75 1.15 -4.60
15 Khris Middleton 5.52 1.66 -3.86
16 C.J. Miles 5.48 1.40 -4.08
17 Chandler Parsons 5.34 4.15 -1.20
18 Luol Deng 5.31 4.00 -1.31
19 Carmelo Anthony 5.27 5.55 0.28
20 Ben McLemore 5.25 1.96 -3.29

[bondom34]

Yeah, the more of the NPI I'm seeing, the more I'm starting to not believe in its real usefulness. It passes some of the general "sniff test" (Harden/Curry), then I'm seeing Z Bo with very good defense, same for Lillard (who I don't think is as bad as his rep, but not that good), Beal rating much more highly than I'd put him, etc. I'm becoming less a fan of this as I go on, as strange as it may seem. I get there's a lot of noise, but it seems the noise is starting to outweigh the usefulness to my mind.

[lorak]

So this time it looks more stable than previous version despite the fact that in both cases difference between updates was about the same (around two weeks).

[Jaivl]

Yeah, I'll wait for JE's PI set, this is full of noise. I just don't believe Harden, Ginóbili, Curry, McLemore or Lillard are close to -3 on defense.

[Keller61]

Yeah, I'll wait for JE's PI set, this is full of noise. I just don't believe Harden, Ginóbili, Curry, McLemore or Lillard are close to -3 on defense.

But isn't NPI RAPM what people really want out of +/- stats: an unbiased look at what is happening on the court? I don't see how PI RAPM actually reduces noise; it just gives biased results.

[Jaivl]

But isn't NPI RAPM what people really want out of +/- stats: an unbiased look at what is happening on the court? I don't see how PI RAPM actually reduces noise; it just gives biased results.

It would be perfect if seasons were 820 games long... with 82 games there is still a looooot of error due to sample size issues (usually bigger than the data itself), too much overlap between players, problems with separating far from the mean, etc. Hence the wonky results.

[Moonbeam]

Yeah, I'll wait for JE's PI set, this is full of noise. I just don't believe Harden, Ginóbili, Curry, McLemore or Lillard are close to -3 on defense.

But isn't NPI RAPM what people really want out of +/- stats: an unbiased look at what is happening on the court? I don't see how PI RAPM actually reduces noise; it just gives biased results.

RAPM is always biased. Ridge regression or any other regularization method introduces bias (toward 0) in order to reduce variance.