RealGM

I wanted to build upon a thought experiment raised by Ceiling Raiser in viewtopic.php?f=64&t=2311899

What is the maximum offensive and defensive impact a player could have, or could have had, measured in points per 100 possessions vs. league average? How has this changed through the history of professional basketball, and what were the rule changes that has modified this?

To put this in context, I have access to some 4-year-average, prior-informed RAPM that puts the maximum impact as +8 for offense (peak LeBron and peak Curry) and around +6.5 for defense (Bogut, Mutumbo, Green). Now peaks are likely higher than that, as that is 4 year average including regular season and playoffs. In addition, the highest rated players are still likely over-regressed towards the prior; they're pushing the limit of the statistical shrinkage. So actual maximum would definitely larger than what I reported.

I feel that with the current rules, logically the maximum offensive impact is definitely larger than the maximum defensive impact. A dominant offensive player can be involved on every possession, whether directly or indirectly, whereas an offense can "avoid" involving a dominant defensive player to some extent, and also choose an attack that targets their weakest point. This customized attack will particularly become evident in the playoffs, where weakest defensive points are typically exploited through specific gameplanning.

So--to answer my own question in part, I think that the current NBA maximum offensive impact is probably in the range of +10, and the maximum defensive impact is probably in the range of +7 or so, or perhaps even less under the microscope of the playoffs.

What does everyone think? What about the past? Changes that would impact this include illegal defense, 3 point line, and goaltending.

DSMok1 wrote:I wanted to build upon a thought experiment raised by Ceiling Raiser in viewtopic.php?f=64&t=2311899

What is the maximum offensive and defensive impact a player could have, or could have had, measured in points per 100 possessions vs. league average? How has this changed through the history of professional basketball, and what were the rule changes that has modified this?

To put this in context, I have access to some 4-year-average, prior-informed RAPM that puts the maximum impact as +8 for offense (peak LeBron and peak Curry) and around +6.5 for defense (Bogut, Mutumbo, Green). Now peaks are likely higher than that, as that is 4 year average including regular season and playoffs. In addition, the highest rated players are still likely over-regressed towards the prior; they're pushing the limit of the statistical shrinkage. So actual maximum would definitely larger than what I reported.

I feel that with the current rules, logically the maximum offensive impact is definitely larger than the maximum defensive impact. A dominant offensive player can be involved on every possession, whether directly or indirectly, whereas an offense can "avoid" involving a dominant defensive player to some extent, and also choose an attack that targets their weakest point. This customized attack will particularly become evident in the playoffs, where weakest defensive points are typically exploited through specific gameplanning.

So--to answer my own question in part, I think that the current NBA maximum offensive impact is probably in the range of +10, and the maximum defensive impact is probably in the range of +7 or so, or perhaps even less under the microscope of the playoffs.

What does everyone think? What about the past? Changes that would impact this include illegal defense, 3 point line, and goaltending.

RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

Actual real-world defensive splits can break the scale even today. The question is how "real" those are and how replicable

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

Actual real-world defensive splits can break the scale even today. The question is how "real" those are and how replicable

This is a highly-informed version of RAPM that is similar to EPM or RPM, but your point does stand. I list RAPM as a lower bound for what a maximum impact would actually be.

Note that I do not wish to include small samples/luck in this assessment. Obviously if a player goes 15/23 from 3 point range over 4 games, they will have a very high impact, beyond what would be sustainable with neutral luck.

DSMok1 wrote:

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

Actual real-world defensive splits can break the scale even today. The question is how "real" those are and how replicable

This is a highly-informed version of RAPM that is similar to EPM or RPM, but your point does stand. I list RAPM as a lower bound for what a maximum impact would actually be.

Note that I do not wish to include small samples/luck in this assessment. Obviously if a player goes 15/23 from 3 point range over 4 games, they will have a very high impact, beyond what would be sustainable with neutral luck.

there are big samples where the scale breaks too. it being prior informed or not does not help in the slightest

OhayoKD wrote:

DSMok1 wrote:

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

Actual real-world defensive splits can break the scale even today. The question is how "real" those are and how replicable

This is a highly-informed version of RAPM that is similar to EPM or RPM, but your point does stand. I list RAPM as a lower bound for what a maximum impact would actually be.

Note that I do not wish to include small samples/luck in this assessment. Obviously if a player goes 15/23 from 3 point range over 4 games, they will have a very high impact, beyond what would be sustainable with neutral luck.

there are big samples where the scale breaks too. it being prior informed or not does not help in the slightest

Prior informed definitely does help here. One of standard RAPM’s biggest flaws is assuming that every player has mean of 0. This helps a bunch for controlling the variance of the final estimates but would regularize star players much more than a prior informed RAPM.

MrVorp wrote:

OhayoKD wrote:

DSMok1 wrote:
This is a highly-informed version of RAPM that is similar to EPM or RPM, but your point does stand. I list RAPM as a lower bound for what a maximum impact would actually be.

Note that I do not wish to include small samples/luck in this assessment. Obviously if a player goes 15/23 from 3 point range over 4 games, they will have a very high impact, beyond what would be sustainable with neutral luck.

there are big samples where the scale breaks too. it being prior informed or not does not help in the slightest

Prior informed definitely does help here. One of standard RAPM’s biggest flaws is assuming that every player has mean of 0. This helps a bunch for controlling the variance of the final estimates but would regularize star players much more than a prior informed RAPM.

Controling variance is not the point here lol

DSMok1 wrote:I wanted to build upon a thought experiment raised by Ceiling Raiser in viewtopic.php?f=64&t=2311899

What is the maximum offensive and defensive impact a player could have, or could have had, measured in points per 100 possessions vs. league average? How has this changed through the history of professional basketball, and what were the rule changes that has modified this?

To put this in context, I have access to some 4-year-average, prior-informed RAPM that puts the maximum impact as +8 for offense (peak LeBron and peak Curry) and around +6.5 for defense (Bogut, Mutumbo, Green). Now peaks are likely higher than that, as that is 4 year average including regular season and playoffs. In addition, the highest rated players are still likely over-regressed towards the prior; they're pushing the limit of the statistical shrinkage. So actual maximum would definitely larger than what I reported.

I feel that with the current rules, logically the maximum offensive impact is definitely larger than the maximum defensive impact. A dominant offensive player can be involved on every possession, whether directly or indirectly, whereas an offense can "avoid" involving a dominant defensive player to some extent, and also choose an attack that targets their weakest point. This customized attack will particularly become evident in the playoffs, where weakest defensive points are typically exploited through specific gameplanning.

So--to answer my own question in part, I think that the current NBA maximum offensive impact is probably in the range of +10, and the maximum defensive impact is probably in the range of +7 or so, or perhaps even less under the microscope of the playoffs.

What does everyone think? What about the past? Changes that would impact this include illegal defense, 3 point line, and goaltending.

I concur with your process and rough conclusions for the modern game. There's obviously no true max the way the speed of light represents a speed limit in the universe, but the data I've seen looks consistent enough that we can do heuristics.

I will say that it might be helpful to think about this in terms of variance, because that's something you can calculate and get a clear sense of offense's variance compared to defense.

In terms of the evolution of all of this in practice, various thoughts:

1. When we go into deep basketball history, especially before the NBA, we see drastically different scoring scales. When we're considering games that, say, have less than half the scoring of today's game, 2 points is effectively considerably more valuable win it comes to helping your side win, and that makes us ask whether we would be better of thinking of "times divide" rather than plus minus - that is ratio-based impact.

2. Considerations of goaltending make clear the limits of real data. In theory this could have been used to dominate the game from the very first Naismith Jump, in practice it doesn't seem to have really become a thing until the '40s.

Regardless, I think it's crystal clear that in any form of basketball that allows goaltending, and has excellent goaltenders, that defense would trump offense.

3. Similarly, while in theory the 3-point line instantly should have made offense trump defense, it really took generations of coaches retiring before that got embraced.

I think it's an excellent question when offense actually usurped defense. My gut says it was the '80s, but that might be more a function of when Bird & Magic happened to be born than anything else.

4. Illegal Defenses' effects are more subtle. I think it's pretty clear at this point that pace & space would have been the optimal approach in any post-goaltending basketball world, but in practice offensive and defensive effectiveness might have swung back and forth.

I think it's more clear that Illegal Defense affected tactics used, which thus helped some players and hurt other players achieve their prominence.

5. I think it's worthwhile considering that at lower levels of basketball the dynamics shift. In shorter league, it's pretty plausible that the best defenders are those who are thieves rather than shotblockers. In fact, by general consensus the WNBA defensive GOAT is Tamika Catchings who was a thief rather than a big.

MrVorp wrote:

OhayoKD wrote:

DSMok1 wrote:
This is a highly-informed version of RAPM that is similar to EPM or RPM, but your point does stand. I list RAPM as a lower bound for what a maximum impact would actually be.

Note that I do not wish to include small samples/luck in this assessment. Obviously if a player goes 15/23 from 3 point range over 4 games, they will have a very high impact, beyond what would be sustainable with neutral luck.

there are big samples where the scale breaks too. it being prior informed or not does not help in the slightest

Prior informed definitely does help here. One of standard RAPM’s biggest flaws is assuming that every player has mean of 0. This helps a bunch for controlling the variance of the final estimates but would regularize star players much more than a prior informed RAPM.

I hav no idea how I would implement it mathematically, or whether you would get improvements, but I've been wondering whether you could get more accurate results by assuming each player has a mean of their team's full season margin of victory.

To give slightly more detail (warning: tiny bit of math) the difference between Adjusted Plus Minus and RAPM is the addition of a parameter lambda. When lambda = 0, we have APM = RAPM. But there's a problem with raw (unregressed) APM, which is that APM is super noisy and variable, which can reduce the accuracy of our measurement. As you increase lambda to infinity, all values of RAPM = 0. Somewhere in between, you get less noise and variability (so more accurate measurements!).... but then the question is what's the ideal value of lambda. You can test for the approximate best value of lambda based on your sample size/data/etc., but there's still some uncertainty here.
Source: https://squared2020.com/2017/09/18/deep-dive-on-regularized-adjusted-plus-minus-i-introductory-example/. Feel free to correct me if I said anything wrong!

So to bring this back to my idea, is there a way to create a statistic where, when lambda -> infinity, every value of RAPM = team MoV, rather than 0? This might produce less shrinkage and (maybe?) more accurate values of RAPM for whatever the ideal value of lambda is. Again, no clue whatsoever how to implement this mathematically, I'm' just spitballing here.

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

I wouldn't call RAPM useless for the task, but would agree that APM is the better tool for the job.

Over the course of a full season, maybe close to this? ?s=46&t=SSGuQgt0b5os6LeB9cXuIQ

Although that is measuring true talent so maybe if a player is a lucky he push ~+9.5 on offense and +6 defense.

Doctor MJ wrote:

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

I wouldn't call RAPM useless for the task, but would agree that APM is the better tool for the job.

I still prefer to use RAPM that is tuned appropriately...although when we're talking 4 years of data, prior-informed RAPM and APM start to converge.

From deep in the archives, here are the stabilized APM estimates Steve Illardi calculated for the 2009 season:


Here's what he said about them:

I've now had a chance to generate 2008-2009 APM ratings, using a full six-year (03-09) lineup dataset to greatly reduce estimation error.

Specifically, I've given each prior season a fractional weighting of the form:

weight = 1/(2^(YearsAgo +1)

This generates the following season-by-season weighting scheme:

2008-2009 = 1
2007-2008 = 1/4
2006-2007 = 1/8
2005-2006 = 1/16
2004-2005 = 1/32
2003-2004 = 1/64

Note that the resulting model still accords nearly 70% of the overall weight to the 2008-2009 season, with much of the rest of the weight coming from the preceding season (and all weightings tapering off exponentially as a function of time).

Obviously, the ideal is an APM model based 100% on the target 08-09 season, but (as we've seen in the past - and as posted at bv.com), such a model yields estimation errors so high as to render the estimates of only limited value. By including prior seasons' data in the model (at reduced weight), we're able to dramatically reduce estimation error, and still allow the results to primarily reflect the target season.

If interested, you can find the latest APM estimates posted at: http://spreadsheets.google.com/ccc?key=0AnGzTFTtSPx_dFVrZXdHNlNZQUJadllKUm1Ld294WkE&hl=en

- Steve

Given that several of these players were near their peak, perhaps this is close to the maximum we can see. Steve Nash over +11 on offense; KG almost +9 on defense.

MrVorp wrote:Over the course of a full season, maybe close to this?

Read on Twitter

?s=46&t=SSGuQgt0b5os6LeB9cXuIQ

Although that is measuring true talent so maybe if a player is a lucky he push ~+9.5 on offense and +6 defense.

Darko is an especially terrible way to do this....

Doctor MJ wrote:

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

I wouldn't call RAPM useless for the task, but would agree that APM is the better tool for the job.

If you are looking for a theoretical upper-bound, i'm not sure either approach works. The tools where you would get the upper-limit would be wowy/like stuff but we open a whole esptemological can of worms there.

I think the best you can do is look at general distributions and make --general-- assumptions about the distributions over longer-time periods. And there I think apm or rapm are fine(i would avoid any box-priors though)

But a conclusion like "a player cannnot be more worth more than x" is just not feasible.

DSMok1 wrote:

Doctor MJ wrote:

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

I wouldn't call RAPM useless for the task, but would agree that APM is the better tool for the job.

I still prefer to use RAPM that is tuned appropriately...although when we're talking 4 years of data, prior-informed RAPM and APM start to converge.

Re: prefer RAPM tuned appropriately. Well, I'd be curious about what all "tuned appropriately" means to you as I know you've worked with this considerably more than I have.

I'm not looking to argue that APM will on average give you better percentile results for a player, but if we're talking about which technique is more closely tied 1 algorithm point to 1 scoreboard point, the regularization of RAPM literally gets in the way of that, and so short of a tuning technique that literally undoes the regularization, I'd expect there's always some haziness there. Am i wrong?

Re: 4-year study sees convergence between RAPM & APM. Oh definitely.

OhayoKD wrote:

Doctor MJ wrote:

OhayoKD wrote:RAPM is pretty useless here. You are not going to get maximums with something that curves down outliers

I wouldn't call RAPM useless for the task, but would agree that APM is the better tool for the job.

If you are looking for a theoretical upper-bound, i'm not sure either approach works. The tools where you would get the upper-limit would be wowy/like stuff but we open a whole esptemological can of worms there.

I think the best you can do is look at general distributions and make --general-- assumptions about the distributions over longer-time periods. And there I think apm or rapm are fine(i would avoid any box-priors though)

But a conclusion like "a player cannnot be more worth more than x" is just not feasible.

Good points in general and I don't think I disagree with you.

I'd be more likely to call it a pragmatic upper-bound than a theoretical one.

OhayoKD wrote:

MrVorp wrote:Over the course of a full season, maybe close to this?

Read on Twitter

?s=46&t=SSGuQgt0b5os6LeB9cXuIQ

Although that is measuring true talent so maybe if a player is a lucky he push ~+9.5 on offense and +6 defense.

Darko is an especially terrible way to do this....

Using arguably the best public model at correctly crediting impact, taking the highest values ever recorded for it, and adding a point or so to them is a horrible way to do it? I’m not saying it’s perfect but I disagree.

MrVorp wrote:

OhayoKD wrote:

MrVorp wrote:Over the course of a full season, maybe close to this?

Read on Twitter

?s=46&t=SSGuQgt0b5os6LeB9cXuIQ

Although that is measuring true talent so maybe if a player is a lucky he push ~+9.5 on offense and +6 defense.

Darko is an especially terrible way to do this....

Using arguably the best public model at correctly crediting impact, taking the highest values ever recorded for it, and adding a point or so to them is a horrible way to do it? I’m not saying it’s perfect but I disagree.

It is the most predictive model because it maps to trajectories and curves data up and down accordingly. When you are looking for an upper-bound, yeah, it's not useful

I'd estimate the reverse, that more raw APM variants tend to (relatively slightly) rate outliers higher than WOWY type numbers (see Ilardi above, 3.5 high minute guys above +12.71 in a single season - KG for the 0.5). Approximately +10 on the team Net level (players don't play every minute). The number of WOWY swings we've seen in that range is pretty darn low (obviously it's significantly lower by definition, great players who switched teams is a subset of great players). Some number of them in situations where the team without the superstar was actively tanking (eg '97 Spurs/'03 Cavs). It does seem slightly more likely when a great player leaves a team than when a great player arrives to a team (eg LeBron leaving the Cavs both times hits that bar, but his only arrival that meets it is when he arrived as a rookie).

I don't have a particularly good explanation for why that might be the case (something around lower end players being able to scale up slightly in worse team situations), but it does seem so to me.

The toll is higher. So that would explain the long-term difference here, but I’ve noticed defense has scaled much higher in the NBA playoffs. The mental and physical stamina it takes just makes it really hard to sustain.


Just in the 10’s, Joel Embiid in the 19 ECSF and LeBron James in the 16 Finals, are the most extreme examples.

Embiid slammed Toronto’s offense to the floor for over 30 minutes a game, Kawhi was averaging over 50 points per 36 on ridiculous efficiency without Embiid in the game, and less than his RS averages on both categories with him in. Meaning, in Kawhi’s amazing ECSF, he only really dominated a quarter of the minutes due to Embiid.

Embiid tilted that series in Philly’s favor due to his defense. He was the gap closer. And has the craziest on/off results on record in the history of the sport, all while being abysmal offensively.

I think there’s something to be said for the fact that an NBA team has never won a title with a bad, or even average defense. While there’s plenty of average or bad offenses that have won titles.

Defense is a supreme gap closer in basketball and scales up seemingly infinitely. LeBron and Embiid are not top defenders of all-time, even though they are athletic freaks, but I’m sure there are plenty of similar ridiculous defensive feats throughout NBA history, even ignoring the obvious Bill Russell.