True Shooting Percentage =/= Offensive Efficiency

[bert stein]

individual ortg is very much confounded, in the sense that often captures teammate's contributions to offense that are captured by individual box stats. team ortg, however, internalizes these confound; so that fact that team ortg and inidividual ortg are based on the same principles says nothing about the problems with individual ortg.

I could not figure out what you're attempting to argue in support of the assertion that individual ortg is "very much confounded." I'd be interested to see thoughtful analysis of the problems with individual ortg, if you have thoughtful analysis to offer.

basically, IORtg suffers from the same issues as all other box-score-based measures of performance. I don't think what i say here is novel to anyone who does quant analysis in sports.

IORtg doesn't take into account events that aren't part of the box score. Since such events (e.g. making the correct first pass, setting the screen) are correlated with box stats of other players, and since players differ in how they contribute to team scoring (specifically, in the extent to which their contributions are captured by box stats), this results in two related issues.

First, non-box-score actions act as latent variables in any such analysis (eg in calculating PER). Second, player contributions via non-box-score actions aren't taken into account with IORtg. Consequently IORtg is a biased estimator of individual "offensive contribution".

[Dr Positivity]

It's scoring efficiency

I still think it's flawed when the player is a ball stopper. ie. Corey Maggette, Adrian Dantley. I would argue being a ball stopper has a huge negative effect on your offense and scoring at a high efficiency creates a neutral effect at best

[Nivek]

individual ortg is very much confounded, in the sense that often captures teammate's contributions to offense that are captured by individual box stats. team ortg, however, internalizes these confound; so that fact that team ortg and inidividual ortg are based on the same principles says nothing about the problems with individual ortg.

I could not figure out what you're attempting to argue in support of the assertion that individual ortg is "very much confounded." I'd be interested to see thoughtful analysis of the problems with individual ortg, if you have thoughtful analysis to offer.

basically, IORtg suffers from the same issues as all other box-score-based measures of performance. I don't think what i say here is novel to anyone who does quant analysis in sports.

IORtg doesn't take into account events that aren't part of the box score. Since such events (e.g. making the correct first pass, setting the screen) are correlated with box stats of other players, and since players differ in how they contribute to team scoring (specifically, in the extent to which their contributions are captured by box stats), this results in two related issues.

First, non-box-score actions act as latent variables in any such analysis (eg in calculating PER). Second, player contributions via non-box-score actions aren't taken into account with IORtg. Consequently IORtg is a biased estimator of individual "offensive contribution".

This reminds me of what Michael Jordan said when the Knicks complained he was getting favoritism from the refs, "So?"

1) NOTHING captures what you're talking about, except (kinda-sorta) +/- data, and that's the very definition of a stat that captures a teammate's contributions.

2) While setting screens, making the right pass, etc. are important activities, and a guy taking the shot surely benefits (or doesn't depending on the quality of the screen), the point of IORTG is to measure a player's SCORING efficiency. It measures how many points a guy produces compared to how many possessions he uses.

Your "complaints" about IORTG are true about every single basketball stat (box score or otherwise) with the possible exception of free throw percentage. So, I guess your point really is that we should just not bother with stats at all, right?

[bert stein]

basically, IORtg suffers from the same issues as all other box-score-based measures of performance. I don't think what i say here is novel to anyone who does quant analysis in sports.

IORtg doesn't take into account events that aren't part of the box score. Since such events (e.g. making the correct first pass, setting the screen) are correlated with box stats of other players, and since players differ in how they contribute to team scoring (specifically, in the extent to which their contributions are captured by box stats), this results in two related issues.

First, non-box-score actions act as latent variables in any such analysis (eg in calculating PER). Second, player contributions via non-box-score actions aren't taken into account with IORtg. Consequently IORtg is a biased estimator of individual "offensive contribution".

This reminds me of what Michael Jordan said when the Knicks complained he was getting favoritism from the refs, "So?"

1) NOTHING captures what you're talking about, except (kinda-sorta) +/- data, and that's the very definition of a stat that captures a teammate's contributions.

2) While setting screens, making the right pass, etc. are important activities, and a guy taking the shot surely benefits (or doesn't depending on the quality of the screen), the point of IORTG is to measure a player's SCORING efficiency. It measures how many points a guy produces compared to how many possessions he uses.

Your "complaints" about IORTG are true about every single basketball stat (box score or otherwise) with the possible exception of free throw percentage. So, I guess your point really is that we should just not bother with stats at all, right?

At what point did I claim to be proposing that we abolish the use of IORtg? All I pointed out was that the statement by you and another poster, that IORtg was not confounded, was incorrect. There's a big difference between you not understanding what confounded means and confounded variables being substantively useless.

As for your point 2 - IORtg counts the contribution of the player when making the final pass (in the case it counts as an assist), but not the contribution of the player when making the next-to-final pass, or the person when setting the screen. In what sense, other than (possibly) in degree, does the first contribution count as a measure of scoring efficiency, but not the last two? (In other words, how does the fact that IORtg tries to capture scoring efficiency excuse the omission of the last two types of contributions, when the first type of contribution is admitted?) Why does the assist count as usage but not the pass leading to the assist?

[Nivek]

Umm, I understand what confounded means. I double-checked it in the dictionary and e'erything. Ortg is not confounded by the stuff you're mentioning anymore than every other statistical measure we use -- again, with the exception of free throw percentage. It's a team game, and teammates help and hurt each other.

Your main point seems to be that the box score is imperfect. I think we can all agree on that. Despite its imperfections, there's a ton that can be learned from it, and DeanO's book (and work with NBA teams) is evidence of that.

Man, in re-reading my 2nd point, I agree -- it doesn't make much sense. I got interrupted while typing the post and I didn't express myself clearly. Sorry about that. Obviously, ortg is more than a "scoring efficiency" measure -- it's a measure of overall offensive contributions. That's a strength, and something that sets it apart from other measures. It is constrained by the limits of the box score. I think it's as good a use of the box score as we're going to find.

As for why assists count as usage and not the pass leading to the assist, it's probably because nobody counts that "hockey assist." If it was counted, I suspect Dean would have incorporated it into the stat. (Of course, then someone would probably complain that it wasn't counting the pass that led to the pass...) Same thing for screen setting. Obviously those are important activities on offense. Unfortunately, there's no data available on who does those activities with what skill and/or frequency. When I tracked defense regularly, it was easy to see that there's lots of important stuff that goes on that doesn't get counted. The choice there is to collect more data or figure out how to make use of the data available.

I guess my question back is this: If IORTG is confoundable, what stat isn't? And, if they're all "confoundable," what then?

I guess those are questions back.

[bert stein]

I agree that all box score based stats have the same problem - which is why I explicitly stated so in my previous post. But the point is to make clear the drawbacks of box score stats compared to +/- numbers or to subjective analysis. Making claims that any box score stat (that purports to measure efficiency in any useful sense) is not confoundable or not biased is just wrong, and moves us towards overreliance on such measures. In particular, IORtg will generate systematic biases for or against individuals; there will be players systematically underrated/overrated by such a measure, not just because of usage, but also because of his playing style.

Note that adjusted +/- (and variants thereof) is a less biased (albeit noiser) measure of performance than stuff like IORtg. Although it does not take into account complementarities between players, and team strategies, it gets rid most of the biases that box score stats are afflicted with. Specifically, with a sufficiently long series of data, adjusted +/- will be a (loosely speaking) better measure of performance than any possible box score stat.

In fact, the ideal box score stat to measure individual performance would be one constructed by running a regression with adjusted +/- as the independent variable and box stats as dependent variables (over a large sample of players), then using the resulting coefficients to weigh each box stat. (To obtain a measure of offensive performance, we'd also have to adjust for the additional possessions generated by offensive rebounding, either separately or by using an adjusted measure of offensive possessions in weighting the +/- numbers.) It will still be confounded, but at least it gets rid of all sorts of other problems. It should also be a less noisy measure of individual performance than adjusted +/-. Note, by the way, that this is distinct from the PER approach of correlating box stats with _team_ performance, which is badly flawed.

[mattg]

TS% is a good stat, however it is obnoxiously overrated/used here on realgm. The problem i have with TS% is that without looking at every single player across the league's TS%, the number itself tells you nothing. The only reason people know that 60%TS is good is because that's what *insert elite offensive player here* has for their TS%. Without context and comparison among other players, TS% is just a number.


I feel like eFG%% is really underused when describing scoring efficiency. Helps factor in shot selection and allows you to look at efficiency across positions easily.

[AussieBuck]

TS% is a good stat, however it is obnoxiously overrated/used here on realgm. The problem i have with TS% is that without looking at every single player across the league's TS%, the number itself tells you nothing. The only reason people know that 60%TS is good is because that's what *insert elite offensive player here* has for their TS%. Without context and comparison among other players, TS% is just a number.

Just go here: http://hoopdata.com/advancedstats.aspx and you can see the league average or find the average for different positions played or by the team or by big/small minute players. Everything you need to put TS% in context.

[Andrewchos]

eFG% > TS%

[AussieBuck]

eFG% > TS%

eFG%, where Hedo is almost as efficient as Manu.

[tclg]

its telling you different things. efg% is all about shots taken and their conversion rate. It doesnt tell you how many and ones or free throw attempts you took.

[Nivek]

bert stein: I don't share your faith in APM. It's interesting information, but it's also the very definition of a confoundable stat. I'm skeptical of claims about it not being less biased than other measures since it's a black blox stat. I'm open to seeing more evidence -- I'm just nowhere close to being convinced yet.

[mattg]

TS% is a good stat, however it is obnoxiously overrated/used here on realgm. The problem i have with TS% is that without looking at every single player across the league's TS%, the number itself tells you nothing. The only reason people know that 60%TS is good is because that's what *insert elite offensive player here* has for their TS%. Without context and comparison among other players, TS% is just a number.

Just go here: http://hoopdata.com/advancedstats.aspx and you can see the league average or find the average for different positions played or by the team or by big/small minute players. Everything you need to put TS% in context.

you missed my point. obviously i know where to find everyone's TS%. that doesn't change that you need context for that number to not be meaningless. PER is the exact same way. try and describe to someone why 60TS% is good without comparing to another player. All you're gonna say is "uhhhhhhhhhh...'.

[bert stein]

bert stein: I don't share your faith in APM. It's interesting information, but it's also the very definition of a confoundable stat. I'm skeptical of claims about it not being less biased than other measures since it's a black blox stat. I'm open to seeing more evidence -- I'm just nowhere close to being convinced yet.

Here is one way to think about it. We may decompose potential flaws for a given individual metric into one of the following types:
First, noise, whereby the metric is unreliable when sample size is small (but noise disappears for large sample size). Empirically, +/- has a disadvantage here, as I acknowledge.
Second, bias, whereby the large sample mean of the metric does not properly capture whatever we are attempting to capture (player performance). Roughly speaking, we can distinguish between three types of biases:
- biases that result from nonstationary environments. For example, a player that a comparative advantage when facing elite defense (because of his 1v1 scoring ability) but whose abilities do not shine through against typical defences may be underweighted, relative to his potential contribution to a title team. Such biases afflict both types of metrics. One might argue that adj. +/- (as opposed to raw +/-) is more susceptible to this bias that other metrics, because of the strong regularity assumptions imposed in calculating it - but I don't think this is significant compared to the fact that we can't even state the regularity assumptions implicit in box stats.
- biases that arise due to team composition / team style. Both +/- type stats and box stats suffer from this form of bias. For example, when a team is constructed around a particular player's strengths, and there are no good substitutes for that player on the bench, he will have inflated +/- and box stats relative to "true" ability. However, there are biases of this sort that are specific to box stats. For example, a playing style that relies heavily on actions that do not show up on the box score (such as zone d) will deflate box stats relative to true contribution of players that are good at such actions, but adjusted +/- will remain unbiased.
- Individual-specific biases that are independent of team composition / style. Box stats, but not adjusted +/-, are susceptible to such biases. Players who are good at actions that do not show up in box scores will be consistently underappreciated by box stats; but these actions will be accounted for in +/- type statistics. In addition, if a particular box stat overweights /underweights a particular statistic (which is virtually inevitable), it will obviously over-underappreciate each player's contribution.
In these senses, +/- is less confounded than box stats.

[Patterns]

eFG% > TS%

[mysticbb]

In these senses, +/- is less confounded than box stats.

I agree with you, but as you pointed out the error can be huge. You can easily get an error which is greater than the result. For APM you need a large sample size and even with a two year value, you can not really say that a +9 player is worse than a +16 player. With a SE of about +/- 5 that kind of difference is within the error range.

[bert stein]

I agree with you, but as you pointed out the error can be huge. You can easily get an error which is greater than the result. For APM you need a large sample size and even with a two year value, you can not really say that a +9 player is worse than a +16 player. With a SE of about +/- 5 that kind of difference is within the error range.

Obviously, I agree. Like I said, the best we can do to improve the current state of affairs is to to expand the set of variables captured in the box score to include other factors (both objective, like distance of shot from basket, and subjective, like quality of pass received) and correlate them (with appropriate interactions added) with +/- type metrics over a large sample of players, then use the resultant coefficients to generate box stat metrics. This should remove some but not all of the bias while also alleviating the noise associated with +/-.

[ElGee]

I'm glad there are folks who don't let facts get in the way of firm assertions.

Individual ortg is a terrific measure of overall offensive EFFICIENCY. In fact, it's the best overall measure of a player's efficiency on the offensive end. It does not say who the best scorer is. It doesn't say who the "best" offensive player is. It tells you how often a guy produces points for his team compared to how many possessions that guy "uses."

It's not noisy. It's not confoundable. It's not imprecise. It's the opposite of those things.

It does not explain everything. Context still matters. A great offensive player surrounded by a bunch of stumblebums will probably have a lower ortg because defensive attention will logically focus on that great player. Way it goes. Lower usage players often have higher ortg because they're being highly selective -- attempting to score only when they have a high likelihood of succeeding. A player's ortg will typically decrease as his usg% increases. That kind of information needs to considered when looking at ortg. Ortg by itself tells us is how efficient the guy is overall. When looked at in combination with usg%, we get a good sense of a guy's overall offensive ability and role with the team.

If you want to talk about a "problem" area of DeanO's work, it's the other end of the floor -- the individual drtg. Those ratings, which are based on a series of estimates from the box score, are imprecise, confoundable and noisy. Which Dean readily acknowledges. Drtg is probably the best defensive information we're going to be able to derive from the box score, and Dean deserves tons of credit for giving it a try.

Where Dean's approach on the defensive end gets interesting is with hand tracking (which I've been doing). Drtg is robust when combined with hand tracking of the forced misses and forced turnovers and free throws related to fouls a player commits. Combine those with defensive rebounds and steals, and you get excellent information into a player's defensive abilities. Still imperfect because there are aspects of defense beyond forcing misses and turnovers -- stuff like shot prevention and ball denial. But it's a HUGE improvement on what we have available now.

Fantastic post. The box captures about 10 or 20% of defensive effort IMO. I have half a dozen stats I think should be added, before we talk about hand-tracking and stuff synergy does (which is a huge component too), but oh well.

[penbeast0]

Oddly enough, I haven't noticed anecdotally (ie. without looking analytically at it) is any indication that efficiency really does drop as usage increases; instead I've looked for that and not seen it. Maybe it's because when you get a big increase in usage (as opposed to small, incremental ones) it's a result of players taking on a different role . . . maybe getting more isolations or plays run for them instead of taking the leftover shots where you don't get the ball until it's shoot or shot clock time?

Has anyone looked at this? Are there patterns that have developed and what kind of hypotheses are supported by them?

[Jimmy76]

Oddly enough, I haven't noticed anecdotally (ie. without looking analytically at it) is any indication that efficiency really does drop as usage increases; instead I've looked for that and not seen it. Maybe it's because when you get a big increase in usage (as opposed to small, incremental ones) it's a result of players taking on a different role . . . maybe getting more isolations or plays run for them instead of taking the leftover shots where you don't get the ball until it's shoot or shot clock time?

Has anyone looked at this? Are there patterns that have developed and what kind of hypotheses are supported by them?

I think skill curves might be what you're looking for

They measure efficiency increase and decrease against usage. Haven't actually read basketball on paper but I think someone like Iverson would have a linear or flat curve since his efficiency doesn't change with usage that much as opposed to someone like Korver who would drop exponentially in efficiency if you fed him 30 shots a game.