ESPN getting on the RAPM bandwagon "Real Plus Minus"

[JLei]

http://espn.go.com/nba/story/_/id/10740 ... plus-minus

http://espn.go.com/nba/statistics/rpm/_/sort/RPM

Seems they are using xRAPM for their Real Plus Minus statistic and site J.E.'s website/ techniques in the article. Naturally the next step in their more stats based coverage of the NBA now that PER is a bit more ubiquitous on their website.

Also included is Kevin Pelton's WAR metric.

[sp6r=underrated]

http://espn.go.com/nba/story/_/id/10740818/introducing-real-plus-minus

http://espn.go.com/nba/statistics/rpm/_/sort/RPM

Seems they are using xRAPM for their Real Plus Minus statistic and site J.E.'s website/ techniques in the article. Naturally the next step in their more stats based coverage of the NBA now that PER is a bit more ubiquitous on their website.

Also included is Kevin Pelton's WAR metric.

It is nice that they finally put the WARP numbers on the site. It was getting annoying reading articles in which he referenced WARP but no clear list.

[JLei]

http://espn.go.com/nba/story/_/id/10740818/introducing-real-plus-minus

http://espn.go.com/nba/statistics/rpm/_/sort/RPM

Seems they are using xRAPM for their Real Plus Minus statistic and site J.E.'s website/ techniques in the article. Naturally the next step in their more stats based coverage of the NBA now that PER is a bit more ubiquitous on their website.

Also included is Kevin Pelton's WAR metric.

It is nice that they finally put the WARP numbers on the site. It was getting annoying reading articles in which he referenced WARP but no clear list.

Pelton probably wanted to be paid for it. Since he's their new Hollinger makes sense.

[Winsome Gerbil]

And then in the very first example they try to convince you that Jamal Crawford is a net negative player in the league and has 15,000pts in this league because of a long succession of GMs and coaches who just don't know their stat formulas.

[lorak]

And then in the very first example they try to convince you that Jamal Crawford is a net negative player in the league and has 15,000pts in this league because of a long succession of GMs and coaches who just don't know their stat formulas.

RPM says Crawford is positive on offense, so 15k pts argument doesn't make sense here, because no one is criticizing his offensive impact. The problem with Crawford is on the other end of the court - and it's not surprise he is more bad on defense than good on offense.

[Winsome Gerbil]

And then in the very first example they try to convince you that Jamal Crawford is a net negative player in the league and has 15,000pts in this league because of a long succession of GMs and coaches who just don't know their stat formulas.

RPM says Crawford is positive on offense, so 15k pts argument doesn't make sense here, because no one is criticizing his offensive impact. The problem with Crawford is on the other end of the court - and it's not surprise he is more bad on defense than good on offense.

And that is no surprise...except of course that at that point you should be able to replace him with just any average NBA guard and be better off. You believe that?

here's another one: Anthony Davis's defense, oft lauded around here as potential GOATish etc., this year is sandwiched right in between Rudy Gay and Al Jefferson in effectiveness. He's less than half as effective as noted stud defender Greg Monroe.

Fun stat. Its like discovering a whole new sport with nothing whatsoever to do with tall men bouncing that boring old orange ball.

[ceiling raiser]

And then in the very first example they try to convince you that Jamal Crawford is a net negative player in the league and has 15,000pts in this league because of a long succession of GMs and coaches who just don't know their stat formulas.

RPM says Crawford is positive on offense, so 15k pts argument doesn't make sense here, because no one is criticizing his offensive impact. The problem with Crawford is on the other end of the court - and it's not surprise he is more bad on defense than good on offense.

And that is no surprise...except of course that at that point you should be able to replace him with just any average NBA guard and be better off. You believe that?

here's another one: Anthony Davis's defense, oft lauded around here as potential GOATish etc., this year is sandwiched right in between Rudy Gay and Al Jefferson in effectiveness. He's less than half as effective as noted stud defender Greg Monroe.

Fun stat. Its like discovering a whole new sport with nothing whatsoever to do with tall men bouncing that boring old orange ball.

I can't comment on this extensively since I don't know all the mechanics behind calculating xRAPM (of which RPM is a derivative), but here are some things to note based on my understanding of RAPM:

1) A slightly negative rating doesn't mean that if you replace him with any NBA guard you're better off. A rating of 0.0 is possession-weighted I believe, meaning that severe outliers (Durant, LeBron, CP3, etc.) playing a ton of possessions bring up the average. It's a mean rather than a median.

2) A lot of the hype around AD's defense is in his potential. He has incredible tools, and looks to develop into an all-time great defender (based on the RAPM data we have). Improving his discipline and adding mass to his frame (as well as improved teammates/coaching) will better his defense.

3) It's the same sport, different data. It's easy to find something that looks "wrong" with a massive data set, but in general, if something doesn't pass the smell test in your opinion, rather than worrying about the results, maybe it's time to question said smell test.

[MisterHibachi]

It has in fact been noted before on this forum that AD's defense is worse than his stats suggest. His command of the floor on defense is not equal to his stats, but that's expected for such a young guy. The hype about him is more so his potential and where he already is in his development this early in his career.

[Wizenheimer]

It has in fact been noted before on this forum that AD's defense is worse than his stats suggest. His command of the floor on defense is not equal to his stats, but that's expected for such a young guy. The hype about him is more so his potential and where he already is in his development this early in his career.

last season, IIRC, Davis was ranked somewhere around 400th in the league on defense according to Synergy. I saw him play half a dozen times last season and while his offense was surprisingly developed, his defense sucked

So, I'm not real surprised that advanced stats might show him as weak on defense. Two things about that though are that defensive stats are all a little suspect, especially if team context is removed. The other is that it's not realistic to expect a 20 year old big man to be a great defender. That will take some time. Blocked shots aren't always a sign of a great defender

[Dr Positivity]

Is this xRAPM? It seems like WARP is already an answer for +/- with other boxscore stuff

[Doctor MJ]

Glad to see ESPN get with the times. It's frankly weird to me it took them so long.

I remain frustrated though at the direction Engelmann has taken things, and now frustrated that ESPN has co-signed with him. You can't use a stat that uses this:

RPM reflects enhancements to RAPM by Engelmann, among them the use of Bayesian priors, aging curves, score of the game and extensive out-of-sample testing to improve RPM's predictive accuracy.

To say this:

LeBron has been accused of coasting at times this season, and RPM reveals where the charge may have merit.

While LeBron was aptly named to the NBA All-Defensive Team last season, his defensive impact this season has been surprisingly mediocre, at least as measured by RPM (-0.21 DRPM). His rating is consistent with the claim that James has not consistently given his best effort this season on the defensive end.

Does the same hold true on offense? No. Offensively LeBron has been as dominant as ever, and his +8.32 ORPM leads the league by a comfortable margin. This means that despite his less-than-stellar defensive contribution, LeBron's total RPM (+8.11) still rates as the NBA's highest this season.

No gahdammit. You can't use a stat that factors in previous years of work to assert that allegation of current coasting are simply wrong. Let alone freaking aging curves!

This was what I didn't like about how I knew people would use Engelmann's stats when he went in this direction, and now it's mainstream and the usage is not only happening but is being championed...

by Ilardi, who was known for making the pure APM stuff.

[mysticbb]

Let alone freaking aging curves!

What do you think is wrong with adjusting the prior up or down based on the expectated development of a player? A rookie is expected to get better while a 30+ player is expected to get worse. Adjusting that seems not just a gimmick to me, but adapting a method to something which is real.

This was what I didn't like about how I knew people would use Engelmann's stats when he went in this direction, and now it's mainstream and the usage is not only happening but is being championed...

I know you don't like it, but how about the fact that the result gives a better prediction for upcoming games in this season? Should we use the +4 James (that's what he comes out in NPI RAPM for me) instead of the +8 James (well, I have him at +7, but still the best in the league ahead of Durant and I only use current season information). I know that it would be important to only judge the current season with information from this season, but the outliers here aren't that many anyway. For me, the bigger issue is the choosen prior based on boxscore stats, which isn't mentioned on ESPN and has some really weird results. Also, using a boxscore based prior for defense is an issue, because the boxscore is really not good at determining defensive impact. It gives more stability to the results, but someone like Boozer isn't becoming a "good" defender just based on getting those defensive rebounds.
Nonetheless, I think that is a step in the right direction overall.

[lorak]

Let alone freaking aging curves!

What do you think is wrong with adjusting the prior up or down based on the expectated development of a player? A rookie is expected to get better while a 30+ player is expected to get worse. Adjusting that seems not just a gimmick to me, but adapting a method to something which is real.

But how it works exactly? For example if rookie didn't improve enough then his value is lowered?

This was what I didn't like about how I knew people would use Engelmann's stats when he went in this direction, and now it's mainstream and the usage is not only happening but is being championed...

I know you don't like it, but how about the fact that the result gives a better prediction for upcoming games in this season?

Does it also give better explanation of previous games played this season?

And what about offense/defense splits? I mean does it explain (and predict) offensive/defensive performance better than RAPM without box score?

[E-Balla]

Let alone freaking aging curves!

What do you think is wrong with adjusting the prior up or down based on the expectated development of a player? A rookie is expected to get better while a 30+ player is expected to get worse. Adjusting that seems not just a gimmick to me, but adapting a method to something which is real.

That's already accounted for by the players either playing worse or playing better.

This was what I didn't like about how I knew people would use Engelmann's stats when he went in this direction, and now it's mainstream and the usage is not only happening but is being championed...

I know you don't like it, but how about the fact that the result gives a better prediction for upcoming games in this season? Should we use the +4 James (that's what he comes out in NPI RAPM for me) instead of the +8 James (well, I have him at +7, but still the best in the league ahead of Durant and I only use current season information). I know that it would be important to only judge the current season with information from this season, but the outliers here aren't that many anyway. For me, the bigger issue is the choosen prior based on boxscore stats, which isn't mentioned on ESPN and has some really weird results. Also, using a boxscore based prior for defense is an issue, because the boxscore is really not good at determining defensive impact. It gives more stability to the results, but someone like Boozer isn't becoming a "good" defender just based on getting those defensive rebounds.
Nonetheless, I think that is a step in the right direction overall.

Yes use the +4 James because that's how he's playing right now. That +8 James is non existent right now and that's important when listing numbers for this season.

[mysticbb]

But how it works exactly? For example if rookie didn't improve enough then his value is lowered?

When you have a prior from last season and you apply the aging curve, you basically add or subtract a value based on age from that. The value depends on the respective age of the players. So, an 35 yr old +5 player may be lowered to a +4 player instead, while +2 22yr old, might get a raise to +3 (well, those numbers aren't correct, just an illustration to get the drift here). After that the prior is adjusted to the mean anyway and also the boxscore stats is applied to that before running the regression on the current season data.

Does it also give better explanation of previous games played this season?

No, it can't. There is a bias introduced based on the choosen lambda.

The bias for the coefficients would be:

Code: Select all

bias(β) = -λUβ

where U = (X^(T)X + λI)^(-1)

and β = (X^(T)X + λI)^(-1)X^(T)y


β is the coefficent vector (the result)
X is the design matrix
X^T is the transpose design matrix
y is the response vector

And what about offense/defense splits? I mean does it explain (and predict) offensive/defensive performance better than RAPM without box score?

Explain? No, but predict. Even though I'm not convinced that the defensive part is actually really showing a better prediction, at least in my test it suggest that it is not better in comparison to pure RAPM informed RAPM values. I personally wouldn't use the boxscore data, if I want to make a split between offense/defense. I tried it just based on the RAPM data on my merged ratings, but the results weren't as good as I hoped and I dropped that matter. From my perspective the overall number is the real meaningful number in order to make a prediction about the scoring margin (or get quantified information about the player's impact). In out-of-sample test my SPM (pure statistical +/- based on boxscore data) actually predicted the results better than RAPM informed RAPM values, while it couldn't match the predictive power in terms of offense and defense (both were off). Engelmann did the test with his best RAPM version at that time too and my older SPM version and came to the same conclusion. So, no idea why he went with the boxscore prior for the defensive part, but I guess it showed improved stability from yr-to-yr, which is something we can expect. And given that the yr-to-yr consistency of playing time for the majority of players is pretty high (coaches like using players in a similar fashion), the defensive value is just getting shifted from a better defender without the boxscore numbers to that boxscore number player (like Boozer getting a boost here and Noah/Deng went down). The negative effect by players changing teams is likely smaller than the positive effect coming from the increased stability.
So, from my experience the RPM value is more reliable than the splits, but those splits are nonetheless better than any other attempt available in public.

[mysticbb]

That's already accounted for by the players either playing worse or playing better.

It is not. Players getting better or worse over the summer is not included!

Yes use the +4 James because that's how he's playing right now. That +8 James is non existent right now and that's important when listing numbers for this season.

Sorry, but that is simply not true. James current performance level and the expected change in terms of scoring margin when he plays is closer to +8 than to +4. Even though in average for the season the regression can't pick that up, because it simply adds value to his teammates for the remainder of the season while they are not providing it. The simple fact is: The predictive power improves when using a prior. That's the reason for that. DocMJ is right that those values should likely not be used to determine an MVP award or something like that, but given his boxscore values, James should finish in a regression just based on a boxscore prior also better than his NPI RAPM would suggest.
Or in another way: Would you say that production and efficiency are also elements in a player evaluation?

[HeatRing2012]

is the box score prior minute adjusted (e.g. per36) ?
or is it basically benefiting players who accumulate the most stats by playing lots of minutes?

[mysticbb]

is the box score prior minute adjusted (e.g. per36) ?

Per 100 poss; like the regression is based on and the results are interpreted as (RPM = change of the scoring margin per 100 poss).

or is it basically benefiting players who accumulate the most stats by playing lots of minutes?

Well, the funny thing is that playing more minutes than average actually does have an effect on the boxscore prior, positive on defense while negative on offense. That is based on a regression of the boxscore entries in order to get the best predictive value. Overall playing more minutes has a slight positive effect. The variable here is pace-adjusted minutes played.

[GSP]

mystic what level do u have other players this season in the top 10?

[E-Balla]

It is not. Players getting better or worse over the summer is not included!

Correct me if I'm wrong but let's say I was a +3 player last season and this season I played like a +1 player due to my old age making me less effective as a player. Without the age adjustment wouldn't my RPM still be well under the +3 I was past year effectively lowering my RPM because I aged (which made me play worse). I kinda understand that it helps the numbers become more accurate for predictions but I think it screws up older guys because players like Duncan, Dirk, Pierce, and Carter haven't really seen a decline in years.

Sorry, but that is simply not true. James current performance level and the expected change in terms of scoring margin when he plays is closer to +8 than to +4. Even though in average for the season the regression can't pick that up, because it simply adds value to his teammates for the remainder of the season while they are not providing it. The simple fact is: The predictive power improves when using a prior. That's the reason for that. DocMJ is right that those values should likely not be used to determine an MVP award or something like that, but given his boxscore values, James should finish in a regression just based on a boxscore prior also better than his NPI RAPM would suggest.
Or in another way: Would you say that production and efficiency are also elements in a player evaluation?

I honestly don't understand what he was trying to accomplish by creating xRAPM. Correct me if I'm wrong but it seems like RAPM (the pure version) takes a lot less of an influence from prior than xRAPM does.

Also I don't think ESPN will be using these numbers for prediction rather than analysis. Sure using the prior raises future accuracy but when trying to gauge how someone played already (like saying Lebron has been having the best season based on his RPM) I think this leaves a bit to be desired.