SPM, APM, ASPM, RAPM, xRAPM, and RPM Question

[lverson]

Yeah, I was wondering if someone could give describe the inadequacies in relation to each other, of these six derivations of the plus-minus. Concise or long, doesn't matter, whatever's more convenient. Here's what I do know, ASPM, SPM, xRAPM and RPM all incorporate box-scores to aid in the projection process and reduce noise, with RPM apparently including a bevy of other data. I have a basic, and I do stress basic, understanding of APM and RAPM and their calculation, but I'm more confused about those others.

I guess my primary question is: ASPM improves on SPM, RAPM is an improvement on APM, with xRAPM as an alternative, so why aren't the more raw versions rendered obsolete? Is there something you lose by ridge-regressing APM or using priors for RAPM or using advanced box-score stats in ASPM or apparently accounting for even height in RPM, that invariably makes them not the obvious best solution?

Maybe I'm looking at this all too naively, but I really just want to pick one and figure out it's process. It's just that it seems like to take the best pictures, you just can't use only the newest cameras.

[bondom34]

Just me, and I'm not the utmost expert here, but my thought is that in the end, its up to the user. RAPM has the prior added, but at the same time this can be a disadvantage depending on what you want from the stat. It has better predictive power, but won't tell you who played better in a single season per se. It would likely see that season as an outlier if a slightly better player had a slightly worse season (why Lebron was still rated highly on defense last season on RAPM but not APM). I've seen sources which indicate xRAPM may be the best predictor as well, but I generally like RAPM or APM, then you can use an actual box score as a separate piece of data. xRAPM integrates the 2 together, but both can be used separately like that. RPM is another version that I don't know the full deal w/ as its ESPN's special little thing now, its useful but I wish I knew more on it.

[blabla]

Ok here goes

APM uses lineup data only (no BoxScore) and does standard linear regression with the data. Due to the data being far from perfect (many players get only subbed in/out together, or only for each other, creating a problem called 'multicollinearity') standard linear regression is awfully suited for this problem and can actually lead to worse results than just assuming all players are equal

Thus, RAPM was born, which uses a statistical trick (Regularization) to better cope with multicollinearity. It makes APM obsolete. Regularization essentially means we're assuming, from the start, for everyone to be a 0, and we need lots of data to move players away from that 0. This can potentially, especially if you have less than 3 years of data, lead to the problem that star players get a rating that is too low (i.e. we originally assumed LeBron's a 0, thus we need looots of data before we call him a +10. One year of data won't be enough)

SPM and ASPM are essentially the same, some people just like to call their SPM 'advanced' (hence the A in ASPM), but there's actually no objective way of telling whether some ASPM is better than some SPM. All SPM's use BoxScore information to rate players. Most SPM's are done using long-term RAPM and then finding weights for BoxScore stats so that the final SPM ranking is close to the RAPM ranking. Since SPM's don't use lineup info, players that look really good on (BoxScore) paper because they have similar BoxScore production to good players can get overrated in SPM's. The poster child for this problem is Javale McGee. In 12-13 he had great BoxScore stats (BLK, FG%, REB, TOV) but he also did lots of dumb stuff on the court not captured by the BoxScore, and had a negative +/-

Remember that RAPM, at first, assumes everyone's a 0 and needs convincing to move ratings away from 0. xRAPM (=RPM) uses players ratings generated from SPM as a starting point, essentially using data from both the BoxScore and the +/- world

Some people like to have RAPM just so they can look at a BoxScore-free rating system, and maybe later combine it themselves, or whatever. I'd generally go with xRAPM/RPM as it's really the best of both worlds. Mistakes by SPM get partially corrected by the lineup part of xRAPM and vice versa

Let me know if you have further questions

[lverson]

Thanks for the responses guys. There are a lot of answers to be found on the internet but they usually come with the expectation you have some knowledge of the metrics. I think I have a decent understanding of them now.

This can potentially, especially if you have less than 3 years of data, lead to the problem that star players get a rating that is too low (i.e. we originally assumed LeBron's a 0, thus we need looots of data before we call him a +10. One year of data won't be enough)

RAPM has the prior added, but at the same time this can be a disadvantage depending on what you want from the stat. It has better predictive power, but won't tell you who played better in a single season per se.

So if I'm following correctly the issue is that using priors is better for projections but it isn't always the best indicator for individual seasons, because the whole point of regularizing APM is to get rid of outliers.

Now, I've read a lot of criticism when it comes to combining box-score statistics with +/-. I'm not sure why...isn't more data always going to be useful when it comes to determining true value? Assuming the weighting for the statistics isn't completely arbitrary, like PER.

[bondom34]

Thanks for the responses guys. There are a lot of answers to be found on the internet but they usually come with the expectation you have some knowledge of the metrics. I think I have a decent understanding of them now.

This can potentially, especially if you have less than 3 years of data, lead to the problem that star players get a rating that is too low (i.e. we originally assumed LeBron's a 0, thus we need looots of data before we call him a +10. One year of data won't be enough)

RAPM has the prior added, but at the same time this can be a disadvantage depending on what you want from the stat. It has better predictive power, but won't tell you who played better in a single season per se.

So if I'm following correctly the issue is that using priors is better for projections but it isn't always the best indicator for individual seasons, because the whole point of regularizing APM is to get rid of outliers.

Now, I've read a lot of criticism when it comes to combining box-score statistics with +/-. I'm not sure why...isn't more data always going to be useful when it comes to determining true value? Assuming the weighting for the statistics isn't completely arbitrary, like PER.

In the end to me, the bolded is what rings true. Use all the data at hand and see where it leads you. For an example, here are 2 players stats for 2014 (one of whom I wasn't ever a fan of, but have come around on due to plus/minus numbers showing serious impact and realizing some strengths in his game that don't come through in box scores, the other a somewhat similar player who many feel is superior):

Player A:
RAPM: 3.10 (APM 6.18)
DRAPM: 1.57 (DAPM 2.69)
ORAPM: 1.53 (OAPM 3.48)
PPG: 9.5
FGA/G: 8.2
APG: 8.6
RPG: 4.2
TS: .491
WS/48: .107

Player B:
RAPM: 1.36 (APM 6.35)
DRAPM: 0.03 (DAPM 3.28)
ORAPM: 1.34 (OAPM 3.67)
PPG: 17.7
FGA/G: 15.7
APG: 6.1
RPG: 4.2
TS: .499
WS/48: .093

So, you can see the prior's change. I've found many people like B better than A, though they are the same age and play the same position, and would be willing to pay him more. I lean A and would pay similarly for both.

Spoiler:

Player A - Ricky Rubio

Player B - Kemba Walker

[blabla]

So if I'm following correctly the issue is that using priors is better for projections but it isn't always the best indicator for individual seasons, because the whole point of regularizing APM is to get rid of outliers.

There are two types of priors: The 'everyone's a 0'-prior which RAPM uses, and the SPM prior that xRAPM/RPM use. Using priors is indeed better for projections (with the SPM prior obviously being better than just assuming everyone's a 0), simply because a) you're not as heavily influenced by what's probably noise and b) you're not completely ignoring data (BoxScore) that has some value to it. (Single season) SPM is actually better for judging individual seasons because single year RAPM with no priors can be very wonky. RAPM is not good for judging single seasons, I'd say ever

Now, I've read a lot of criticism when it comes to combining box-score statistics with +/-. I'm not sure why...isn't more data always going to be useful when it comes to determining true value? Assuming the weighting for the statistics isn't completely arbitrary, like PER.

I guess people just grew fond of a BoxScore-free metric; and they probably wanted to combine RAPM with another BoxScore-based metric by themselves. If you're out for the best single number player metric there's definitely no reason to ignore the additional data, and as such, you should go with RPM/xRAPM

[Doctor MJ]

Feel like most things are basically covered. Couple things I'll say, because I haven't seen them answered:

Not all RAPM uses 0 as the prior. We have non-prior informed (which is the 0 one), and we have prior-informed, of which xRAPM is a variant of. As long as you're simply using RAPM data in the prior, I've never seen anyone insist on not calling it RAPM. The xRAPM label came in when a statistician start using non-RAPM data in the prior and gave it the 'x' prefix.

As far as the distinction between SPM and ASPM, I may think of it wrong, but I tend to see the former as a PER-style stat with weights determined by regression, while the latter is a true hybrid stat that actually factors in a player's +/- directly in some form along with the SPM component.

In terms of strengths and weaknesses, from my perspective:

If I could access only one of these things along with normal box score data to look at a player's career, it would be prior-informed RAPM. To me that's the most reliable non-box score metric we have generally speaking.

If I'm evaluating just the current year, but I have access to all or most of the season, I would prefer the non-prior informed RAPM. Typically in such cases, I'm looking to evaluate the most noteworthy players of the year, and with such players I've found that a prior can really distort things for a player who takes it to the next level that year.

I have a soft spot in my heart for the pure APM, and I would always love to have access to APM, and the two RAPMs (prior or no) for a given year, but I've gotten used to RAPM enough that I feel I can handle it's weaknesses in validity pretty well, and the reliability edge is strong.

As far as using SPM, I already use PER and WS as part of my box score analysis, so I'd gladly use SPM on that front if it were widely available. I don't see it as a competitor to the other stats here at all.

The hybrid stats - ASPM and xRAPM - are stats that would become the best stats around if I didn't have access to box score data...but I do have access to that data, so I've never really had a use for them. I understand to the statistician building them they seem like the next logical step forward, but as an analyst they basically just feel like me outsourcing my understanding of the game to the statmaker, which will never make sense to anyone serious.

To be clear: Me using these other stats isn't a similar outsourcing. From raw +/- all the way to prior-informed RAPM, these are just statistical techniques being applied to basketball. Once you try to combine the box score in there though, you're getting someone making decisions about how to actually way various basketball factors in, and that task is precisely what I'm trying to do myself.

One might argue that the same is true for PER & WS, but those processes are far simpler. Far less opaque from the start, and since it's never actually been a choice to use PER or raw data because I can use both, it's always been something more in the background for me...though I will say in the early years, Hollinger was annoying as hell with PER (and Berri with WP will never cease to be anything but an annoyance).

[dice]

how far back does "prior informed" data go? is there a weighting system to prior seasons that phases out older data? if so, isn't that introducing a human element as far as how to weight it? why not just use non-prior informed data for every individual season and recognize for yourself what appear to be outliers and trends?

also, given that there are more offensive box score stats and that defensive box score stats can be more deceptive, aren't these advanced metrics far more useful for defensive analysis? for example, give me points/TS%/A/TO/ORB and i have an excellent idea about offensive contributions. not nearly as much the case on the defensive end w/ blocks/steals/DRB

[jinxed]

how far back does "prior informed" data go? is there a weighting system to prior seasons that phases out older data? if so, isn't that introducing a human element as far as how to weight it? why not just use non-prior informed data for every individual season and recognize for yourself what appear to be outliers and trends?

also, given that there are more offensive box score stats and that defensive box score stats can be more deceptive, aren't these advanced metrics far more useful for defensive analysis? for example, give me points/TS%/A/TO/ORB and i have an excellent idea about offensive contributions. not nearly as much the case w/ blocks/steals/DRB

There are different types of prior-informed data. In Multi-year RAPM then the priors are previous seasons, and we have seen many versions of this 6 yrs, 3ys, 2 yrs, for example.

If we are talking about prior-informed for RPM, then it is just that single season, with the prior being that seasons SPM. Previous seasons are not taken into account.

I'm not sure what has been shown to be better in predictions..multiyear RAPM or RPM, but I'd assume the latter.

[dice]

first of all, thanks for your response

how far back does "prior informed" data go? is there a weighting system to prior seasons that phases out older data? if so, isn't that introducing a human element as far as how to weight it? why not just use non-prior informed data for every individual season and recognize for yourself what appear to be outliers and trends?

also, given that there are more offensive box score stats and that defensive box score stats can be more deceptive, aren't these advanced metrics far more useful for defensive analysis? for example, give me points/TS%/A/TO/ORB and i have an excellent idea about offensive contributions. not nearly as much the case w/ blocks/steals/DRB

There are different types of prior-informed data. In Multi-year RAPM then the priors are previous seasons, and we have seen many versions of this 6 yrs, 3ys, 2 yrs, for example.

ok, so I assume there's gotta be a human judgement of how to weight each prior season, which makes multi-year RAPM impossible for an outsider to analyze

If we are talking about prior-informed for RPM, then it is just that single season, with the prior being that seasons SPM. Previous seasons are not taken into account.

I'm not sure what has been shown to be better in predictions..multiyear RAPM or RPM, but I'd assume the latter.

isn't the whole idea of multiyear RAPM to give a smoothed out, better idea of present value?

[jinxed]

first of all, thanks for your response

how far back does "prior informed" data go? is there a weighting system to prior seasons that phases out older data? if so, isn't that introducing a human element as far as how to weight it? why not just use non-prior informed data for every individual season and recognize for yourself what appear to be outliers and trends?

also, given that there are more offensive box score stats and that defensive box score stats can be more deceptive, aren't these advanced metrics far more useful for defensive analysis? for example, give me points/TS%/A/TO/ORB and i have an excellent idea about offensive contributions. not nearly as much the case w/ blocks/steals/DRB

There are different types of prior-informed data. In Multi-year RAPM then the priors are previous seasons, and we have seen many versions of this 6 yrs, 3ys, 2 yrs, for example.

ok, so I assume there's gotta be a human judgement of how to weight each prior season, which makes multi-year RAPM impossible for an outsider to analyze

If we are talking about prior-informed for RPM, then it is just that single season, with the prior being that seasons SPM. Previous seasons are not taken into account.

I'm not sure what has been shown to be better in predictions..multiyear RAPM or RPM, but I'd assume the latter.

isn't the whole idea of multiyear RAPM to give a smoothed out, better idea of present value?

1. Usually the person who released the multi-year rapm will tell you the number of prior years and weights. I haven't seen a version for 2017 yet if anyone out there has please pass it along!

2. Multi year RAPM is definitely better than single year RAPM, I just don't know if it better than RPM

[dice]

first of all, thanks for your response

There are different types of prior-informed data. In Multi-year RAPM then the priors are previous seasons, and we have seen many versions of this 6 yrs, 3ys, 2 yrs, for example.

ok, so I assume there's gotta be a human judgement of how to weight each prior season, which makes multi-year RAPM impossible for an outsider to analyze

If we are talking about prior-informed for RPM, then it is just that single season, with the prior being that seasons SPM. Previous seasons are not taken into account.

I'm not sure what has been shown to be better in predictions..multiyear RAPM or RPM, but I'd assume the latter.

isn't the whole idea of multiyear RAPM to give a smoothed out, better idea of present value?

1. Usually the person who released the multi-year rapm will tell you the number of prior years and weights. I haven't seen a version for 2017 yet if anyone out there has please pass it along!

2. Multi year RAPM is definitely better than single year RAPM, I just don't know if it better than RPM

Engelmann released a multi-year rapm recently in a tweet. I could find it quickly up until recent days just by googling the words 'engelmann', 'multi', and 'rapm' but for some reason I can't find it anymore. I'm sure it's out there

[KqWIN]

Adding onto what others have said...

The first thing I would point out is that all of these things were some version of another.

SPM->ASPM->BPM
APM->RAPM->xRAPM=RPM

The only ones that I'd pay attention to are BPM, RAPM, and RPM. (There is a lot of great stuff out there, but these are the bedrocks).

BPM - Multivariate Regression using Advanced Box Score measures
RAPM - Ridge Regression using play by play data only
RPM - RAPM with a box score bayesian prior

(I can explain the ideology/math behind these if anyone is interested)

One thing I don't think has been mentioned yet is the design/functionality of +/- statistics. These numbers are essentially trying to predict a single players contribution to the team's +/- over 100 possessions. So let's say you have a team with players that have estimated +/- of 1, 2, 3, 4, and 5.

The estimated lineup would be 1+2+3+4+5= +15 per 100 possessions. Those are the numbers that these models are trying to estimate. They just do it in different ways.

[apapanicolaou]

Hey KqWIN, I'm quite interested in the mathematical underpinnings and find it's often not elaborated on. If you don't mind, could you dive into some of the details of how the different metrics are constructed?

-Alex