What's the best defensive metric

[EvanZ]

Have you looked at the correlation between PDSS and defensive RAPM?

Yes. But do I have to consider defensive RAPM the gold standard and then align everything else to it?

Well, in a word, yes. It *is* the gold standard. It's currently the best predictor among all metrics I have seen. You said yourself PDSS isn't a predictor. So, the question is whether there is *any* correlation whatsoever between PDSS and a rating system that is widely considered the gold standard among predictors.

BTW, I'm sure you are aware that there are statistical +/- metrics which use RAPM as their basis. Those SPM metrics are the "silver standard" - not quite as good as RAPM, itself, but not much worse. And they are not quite as much a black box.

My thought would be that PDSS could be used to create a version of advanced statistical +/-. You just need to determine the weights for the various PDSS stats, and that can be done using RAPM as a basis.

[EvanZ]

And really - we're supposed to ignore what can be observed and counted to use the evidence of a system that tells us that last year Jose Calderon was a better defender than Andrei Kirilenko? Please.

Calderon's multi-year defensive RAPM is -1.8. Kirilenko's is +1.2. That's the RAPM "gold standard".

[BorisDK1]

Well, in a word, yes. It *is* the gold standard. It's currently the best predictor among all metrics I have seen. You said yourself PDSS isn't a predictor. So, the question is whether there is *any* correlation whatsoever between PDSS and a rating system that is widely considered the gold standard among predictors.

It may be perceived to be the "gold standard", but only due to the lack of other information. Boxscore data for defense is almost non-existent; so, people have designed schema to try to measure defensive performance the only way they know how - indirectly. But that's only the case because there are no alternatives widely available - even PDSS data only exists for one team, for the past two years, and that's because I did it. So RAPM is what's used by default, not by preference. Conceptually, I think we can all admit that a noisy series of formulas measuring simply by indirect data is not superior to direct individual data.

My thought would be that PDSS could be used to create a version of advanced statistical +/-. You just need to determine the weights for the various PDSS stats, and that can be done using RAPM as a basis.

By why does an indirect system designed by engineers without any direct data taken from the basketball floor get to determine what every other system shows? That makes zero sense. RAPM does not get to determine what every other metric shows or does not show.

PDSS data tends to provide extremely "rich" information. For example, we see that DeMar DeRozan does a so-so job on the possessions he faces (he had a stop% of .525 last year), but faced few possessions (DPs% of only .145) and thus didn't produce a lot of stops. So is he a decent defender (because he doesn't allow much) or a poor one (because he produces so few stops)? You could go either way, and to boil that data down to a one-number solution probably robs us of analytical opportunities.

[BorisDK1]

Calderon's multi-year defensive RAPM is -1.8. Kirilenko's is +1.2. That's the RAPM "gold standard".

And that's even so close as to be ridiculous.

[EvanZ]

Calderon's multi-year defensive RAPM is -1.8. Kirilenko's is +1.2. That's the RAPM "gold standard".

And that's even so close as to be ridiculous.

That's not really that close. A difference of 3 points is worth 7.5 wins over the course of a season. And it's a silly comparison to begin with.

[EvanZ]

PDSS data tends to provide extremely "rich" information. For example, we see that DeMar DeRozan does a so-so job on the possessions he faces (he had a stop% of .525 last year), but faced few possessions (DPs% of only .145) and thus didn't produce a lot of stops. So is he a decent defender (because he doesn't allow much) or a poor one (because he produces so few stops)? You could go either way, and to boil that data down to a one-number solution probably robs us of analytical opportunities.

My answer to this, honestly, is so? How am I supposed to interpret this unless you place it in the context of other players at his position?

Also, whether this matters is only relevant to its effects on winning...so, what do these stats tell us in terms of wins?

Without the context of winning, it's interesting, but hard to know what to do with it. RAPM, while it's a black box and doesn't tell you why one player is better than another, at least, tells you how one player impacts the bottom line (winning) compared to other players. It's about winning in the end, isn't it?

[BorisDK1]

My answer to this, honestly, is so? How am I supposed to interpret this unless you place it in the context of other players at his position?

Maybe you also need to interpret it in the context of his team?

Honestly, I can't say much when your only response is "so"? That's not a particularly insightful response.

Also, whether this matters is only relevant to its effects on winning...so, what do these stats tell us in terms of wins?

That's the thing: if he's on a team with one or more perimeter players who bear a lot of possessions and are efficient with them, then he's probably producing wins. On a team without anybody on the perimeter who can stop anybody - like Toronto - then it's scary when he's your best perimeter defender, right? If there were one or more other good defenders on the perimeter, suddenly the fact that he's not a stop-producer becomes far less important.

Without the context of winning, it's interesting, but hard to know what to do with it. RAPM, while it's a black box and doesn't tell you why one player is better than another, at least, tells you how one player impacts the bottom line (winning) compared to other players. It's about winning in the end, isn't it?

It is about winning, that's the important thing. But at some level, winning does happen within a team context and I don't trust any single one metric to correlate individual performance to winning. When that tries to get done from an indirect standpoint, I'm even less confident in allowing it to tell the whole story (which it seems you're trying to do - to the point of insisting that what we can count ought to be set aside based on RAPM's output). I am not likely to follow that yellow brick road, personally. I'm certainly interested in looking at it when I don't have direct data - and maybe even when I do. But I certainly won't base my thinking entirely upon it.

You do raise an excellent point about the "black box". Oliver's metrics were designed, ultimately, to be tools to be used for coaches (he himself was an assistant coach in the NCAA); that's important to me, as a coach who's trying to understand and interpret and utilize metrics. PDSS data is immediately applicable and transparent for how I'm trying to understand my own teams and personnel, and deploy them accordingly.

[EvanZ]

That's the thing: if he's on a team with one or more perimeter players who bear a lot of possessions and are efficient with them, then he's probably producing wins. On a team without anybody on the perimeter who can stop anybody - like Toronto - then it's scary when he's your best perimeter defender, right? If there were one or more other good defenders on the perimeter, suddenly the fact that he's not a stop-producer becomes far less important.

How am I supposed to know that it's scary? What is the reference value for a player at his position? How does he compare to the league average? I would have to know those things before coming to any conclusion.

His defensive RAPM is -1.9. That tells me immediately he's a very bad defender.

[BorisDK1]

How am I supposed to know that it's scary? What is the reference value for a player at his position? How does he compare to the league average? I would have to know those things before coming to any conclusion.

League average by definition for 2009-10 would have been about .505 @ DPs% - by definition. League Average Stop% = 1 - League Average Floor %, which in 2009-10 would have been .495. League Average DPs% = .200, by definition. So, having one guy only 20 points above league average at less than 3/4 of expected defensive share is fairly scary.

His defensive RAPM is -1.9. That tells me immediately he's a very bad defender.

And maybe you might want to look at data other than RAPM for a richer, more functional view. Defensive RAPM also tells you that he's worse than Jose Calderon, which does fail to pass the laugh test.

Remember: RAPM has come into being due to lack of defensive data. PDSS provides that richer data, and to ignore it is probably unwise.

[EvanZ]

Remember: RAPM has come into being due to lack of defensive data. PDSS provides that richer data, and to ignore it is probably unwise.

Nice strawman. Where did I say "ignore it"?

As far as comparing Calderon and DeRozan, it doesn't make sense, because they don't play the same position. But according to RAPM they are both very bad defenders for their respective positions. A difference of 0.1 is not significant.

As far as looking at other data, I'm all for it. Never said otherwise. I routinely look at Synergy. I just don't know how to turn Synergy data into a predictive metric like RAPM.

[floppymoose]

I'm sorry, PDSS does show that. Players who help their teammates get stops get credit for those stops, and player who don't, don't.

I'm not at all convinced that is true, based on what I know about PDSS.

[BorisDK1]

I'm not at all convinced that is true, based on what I know about PDSS.

Well, I think your understanding is a little lacking.

Player A1 drives past player B1, who recovers enough to keep the drive out of a straight line. Player A5 comes over to help contest the floater A1 takes, which he misses. Result = .5 forced miss A1, .5 forced miss A5.

That's how PDSS is scored.

[BorisDK1]

Nice strawman. Where did I say "ignore it"?

When you want to rewrite the results in the light of RAPM, or jettison them entirely when they disagree with RAPM, you basically kinda are.

As far as comparing Calderon and DeRozan, it doesn't make sense, because they don't play the same position. But according to RAPM they are both very bad defenders for their respective positions. A difference of 0.1 is not significant.

The difference between their actual performance is not being captured by RAPM, and that is a problem. If you're going to give such huge explanatory power to a metric like RAPM as you seem wont to do, it has to reflect what actually happens on the basketball floor. There is a huge gap in defensive performance between Calderon and DeRozan - and their relative positions don't matter. If RAPM isn't capturing that, then there exists a huge, screaming need for better defensive data. Using RAPM as the exclusive determiner of defensive performance seems unworkable in light of this.

As far as looking at other data, I'm all for it. Never said otherwise. I routinely look at Synergy. I just don't know how to turn Synergy data into a predictive metric like RAPM.

Okay, you say that now - but just spent the better part of two pages insisting that all PDSS results either have to explain themselves in light of RAPM or simply be jettisoned in favour of it. I'm not swimming in that pool, sorry.

[EvanZ]

This is OT, but I looked at your blog, and started reading some of your duck recipes. Great stuff. I don't eat red meat, so duck is about as close as I get to it (I had ostrich once, which was even more meat-like). Anyway, cool stuff. You combine BBIQ and BBQ in one blog like no other.

[BorisDK1]

This is OT, but I looked at your blog, and started reading some of your duck recipes. Great stuff. I don't eat red meat, so duck is about as close as I get to it (I had ostrich once, which was even more meat-like). Anyway, cool stuff. You combine BBIQ and BBQ in one blog like no other.

Thanks, man! If you're local to the general Toronto area, let me know and we'll organize a dinner or something. For somebody trying to lose weight like myself, muscovy duck is really not a bad way to get a nicer piece of meat than chicken, if you need a change. For a duck, it's surprisingly light in fat and especially when done barbecue-style it's not that much business at all.

Sorry about the non-updates from the blog as of late. I'm busy getting ready for school to start and I've had some vacation lately that's just disrupted everything.

[floppymoose]

I'm not at all convinced that is true, based on what I know about PDSS.

Well, I think your understanding is a little lacking.

Player A1 drives past player B1, who recovers enough to keep the drive out of a straight line. Player A5 comes over to help contest the floater A1 takes, which he misses. Result = .5 forced miss A1, .5 forced miss A5.

That's how PDSS is scored.

That is how I imagined it worked, and it supports my prior thinking. All the time Bargs was on the court, other players were failing to pick up .5's they would have got with a better defender beside them. The result: Bargs poor play gets distributed across his teammates as well.

[BorisDK1]

That is how I imagined it worked, and it supports my prior thinking. All the time Bargs was on the court, other players were failing to pick up .5's they would have got with a better defender beside them. The result: Bargs poor play gets distributed across his teammates as well.

Bargnani's stop% of just .451 on a DPs% of only .195 with his DRat of 114.3 shows that at every level as well. Those are shameful numbers, especially for a post player.

As for the Raptors basically being victims of Bargnani exclusively, please keep in mind that the Raptors' problems are not just Bargnani. Bargnani is devastating, there's no doubt - PDSS shows that quite clearly - but he's got nothing on Calderon. The "helpable" plays that Bargnani could deal with but prefers to stand and watch unfold without challenge are really not that frequent in the course of a ball game - maybe 3 or 4, with 2 or 3 of them ending up in scores. That's not enough of a difference to explain how the Raptors give up 113 points / 100 possessions, though. Would the Raptors be better off if they were to wrap Bargnani in heavy chains and drop him to the bottom of Lake Ontario? Sure they would. But until you have a bunch of other issues dealt with as well - say, weak ball screen coverages, terrible transition coverage (Jose Calderon's MO is actually to run away from the ball in transition and make his bigs guard the ball in the middle of the floor in transition), the fact that nobody in the backcourt can keep people in front of them - it's only a piece of the puzzle, and not even close to being the majority of the puzzle.

We should look critically at Bargnani and conclude that the team is better off with him gone, absolutely. But the problems are far deeper than just Bargnani. RAPM, sadly, doesn't really catch that. I think if we were to attempt to fix this team just by ditching Bargnani, we'd be pretty stunned at how much foul trouble our bigs end up on a consistent basis and how little overall impact is made to the team. Nevertheless, he should go.

I think we need to be careful to make analysis of what actually transpires on the basketball floor our rule, and not let theories suggested by various metrics become "laws" without a little challenge.

[EvanZ]

As much as you don't like Calderon, it's not clear to me that his defense relative to other point guards is as bad as Bargnani's relative to other centers. Defense at the point guard position these days is a luxury, not an expectation. The opposite is true at center.

I'd lay about 5% of the Rap's problems on Calderon's defense, and maybe 30-40% on Bargnani's.

[floppymoose]

In support of the idea that poor d from Bargs might be more damaging than poor d from Calderon:

http://www.backpicks.com/2011/01/22/defensive-usage/

[BorisDK1]

As much as you don't like Calderon, it's not clear to me that his defense relative to other point guards is as bad as Bargnani's relative to other centers. Defense at the point guard position these days is a luxury, not an expectation. The opposite is true at center.

I'd lay about 5% of the Rap's problems on Calderon's defense, and maybe 30-40% on Bargnani's.

On what basis do you make that assertion? It seems you've come up with an ad hoc assignment of blame based on gross generalization ("defense at the point guard position is a luxury"). I would invite you again to look up at just how much Calderon gives up: it's not like he's facing a possession here or there and doing just kinda subpar with them. He's getting picked on - a lot - and basically doing so poorly that the team is only slightly better off with him out there than playing 5-on-4 on that end of the floor.

Bargnani's crushing, we all agree. All numbers align to show that. But if RAPM is colouring your vision to the point where you don't see what Calderon is doing, then you probably need to find another interpretive lens.