What's the best defensive metric

[Fenris-77]

The point is nobody's actually compiling PDSS stats right now (maybe Oliver did when he worked for the Nuggets, but it wasn't made public obviously). The DRTG that basketball reference gives is a modified form of DO's DRTG, and just uses box score stats.

Not true. BorisDK1 from the Raps board compiles PDSS stats for every Raps game, and has for a couple of seasons now. They're available for free download if you want to take a look. It's true that you can't just go look this shizz up, which makes it a less than useful vote for 'best' on my part, but when it's available I'll take it over the next best defensive metric every time (even though it doesn't track quite everything I'd like)

It's not that complicated to double check for accuracy if you're rocking a PVR either.

I'm also not really sure how PDSS ends up being a 'stupid' acronym for Project Defensive Score Sheet. It just kinda ... makes sense.

[Wannabe MEP]

I'm also not really sure how PDSS ends up being a 'stupid' acronym for Project Defensive Score Sheet. It just kinda ... makes sense.

I freely admit that I was wrong about that. I thought the name was an arbitrary nickname that grew out of the context of a discussion on your board, but not when I realized PDSS actually came from Oliver.

It's true that you can't just go look this shizz up, which makes it a less than useful vote for 'best' on my part, but when it's available I'll take it over the next best defensive metric every time (even though it doesn't track quite everything I'd like)

I think it would be an incredibly valuable tool to supplement APM/RAPM, but multi-year APM/RAPM kills it as a single metric. It's just not possible to assign credit to an individual on almost any given play.

For example: Dirk and Chandler set a staggered ball screen for Terry. With Terry and Chandler potentially diving into the paint for an easy bucket, Kidd's man sags to help. Terry kicks to Kidd, who knocks down the three. That's a very simple play with only one pass, yet four defenders were DIRECTLY involved in trying to stop it. Unless Kidd's man was completely wrong for sagging to help, which is extremely unlikely, it's basically impossible to assign blame to any one defender.

That's one example of about a million: basketball is played within these gray areas. Every screen involves at least two defenders, but almost always more. A trap isn't just about the guys who trap; it's also the guys denying would-be passes. When poor defenders like Bargs or Stat are on the court, the whole team knows they have to help, which creates openings elsewhere. A player that helps well and then quickly recovers to his man just prevented an easy bucket, but he still might have no direct role in what happens during the final moments of the shot clock, but he played a major role in reducing the other team's efficiency. This stuff can't be measured directly.

Imperfect as it is, APM/RAPM wins.

[tclg]

I try to use my eyes then check def win share and dtrg Its hard though because it is a team thing

[floppymoose]

That's one example of about a million: basketball is played within these gray areas.

Which is exactly why I don't trust synergy.

[BorisDK1]

The point is nobody's actually compiling PDSS stats right now (maybe Oliver did when he worked for the Nuggets, but it wasn't made public obviously). The DRTG that basketball reference gives is a modified form of DO's DRTG, and just uses box score stats.

I, in fact, have compiled PDSS data for the Toronto Raptors for the past two seasons. The results are downloadable here.

[BorisDK1]

I think it would be an incredibly valuable tool to supplement APM/RAPM, but multi-year APM/RAPM kills it as a single metric. It's just not possible to assign credit to an individual on almost any given play.

For example: Dirk and Chandler set a staggered ball screen for Terry. With Terry and Chandler potentially diving into the paint for an easy bucket, Kidd's man sags to help. Terry kicks to Kidd, who knocks down the three. That's a very simple play with only one pass, yet four defenders were DIRECTLY involved in trying to stop it. Unless Kidd's man was completely wrong for sagging to help, which is extremely unlikely, it's basically impossible to assign blame to any one defender.

That's very easy to assign: 1 3FGA allowed, xKidd. That's right out of Basketball on Paper. It's a simple metric: let's not get too complicated, here. In the totality of not only a ball game, but a season, the data you get from this paints you a pretty clear picture of what's going on.

That's one example of about a million: basketball is played within these gray areas. Every screen involves at least two defenders, but almost always more. A trap isn't just about the guys who trap; it's also the guys denying would-be passes. When poor defenders like Bargs or Stat are on the court, the whole team knows they have to help, which creates openings elsewhere. A player that helps well and then quickly recovers to his man just prevented an easy bucket, but he still might have no direct role in what happens during the final moments of the shot clock, but he played a major role in reducing the other team's efficiency. This stuff can't be measured directly.

Imperfect as it is, APM/RAPM wins.

In tracking PDSS, it's very easy to assign credit/blame in multiple-player scenarios. Really, it's not that hard. Sometimes it's split between two players (both positively and negatively) but this idea that the game is far too complex to be meaningfully tracked by a PDSS method doesn't ring true. And I will take it over a series of mathematical formulas to adjust indirect data.

[EvanZ]

The point is nobody's actually compiling PDSS stats right now (maybe Oliver did when he worked for the Nuggets, but it wasn't made public obviously). The DRTG that basketball reference gives is a modified form of DO's DRTG, and just uses box score stats.

I, in fact, have compiled PDSS data for the Toronto Raptors for the past two seasons. The results are downloadable here.

Thanks, it was mentioned previously (just a few posts above yours). Of course, while it is interesting to look at one team, it's difficult to put those stats into context if it's not done for the other 29 teams.

[BorisDK1]

Thanks, it was mentioned previously (just a few posts above yours). Of course, while it is interesting to look at one team, it's difficult to put those stats into context if it's not done for the other 29 teams.

The lamentable lack of data for the other teams doesn't really nullify its usefulness, though. The solution is to get more people off their butts and tracking this stuff until we do have all 1230 games tracked.

[Wannabe MEP]

BorisDK1, let me repeat: I wholeheartedly affirm the compilation of PDSS; I don't think you were wasting your time by keeping these stats, and I think it would be valuable to have this league-wide. But as a single measure of the defensive impact of a player, there's absolutely no contest. Multi-year RAPM/APM is much, much, much, much, much, much better than PDSS DRat.

It's a simple metric: let's not get too complicated, here.

The metric may be simple, but BASKETBALL is complicated. That's the whole point. This is a massive, crucial philosophical divide. I think that it is extremely unlikely that either of us can convince the other or that we can find a comfortable middle road. I fundamentally disagree with you about our ability to accurately measure all the subtleties of defense directly. I'm not sure, however, what concerns you have with RAPM/APM.

But the divide in the numbers is enormous. We only have the Raptors to look at, but take a look at the difference among bigs.

2009-2010 PDSS DRat:
Amir 109.5
Bargs 110.0
Bosh 110.0

Multi-year RAPM:
Amir +3.3
Bargs -4.2
Bosh +2.3

Your stats have these guys as essentially equivalent, while RAPM says that Amir and Bosh are both very good defenders and that Bargs is the worst defender in the league. The one-year RAPM for 2009-2010 also has Bargs as the worst defender in the league. These are incredibly consistent with the fact that Bargs' increase in minutes has correlated with the plummeting defensive rating of the Raptors overall to the worst defensive team in the league two years running, as well as the fact that in simple net rating the team has been 7.41, 9.10, and 6.45 points better defensively per 100 possessions when he is on the bench for the past three seasons. He has also been among the three worst players league-wide in overall defensive rating (basketballvalue.com) for the past two years (115.87 and 115.17), topping the list in 2009-2010. RAPM gets more accurate with a greater accumulation of data, and every single year Bargs' defensive value has fallen further as the numbers have become clearer.

Bargs certainly doesn't get all the credit for the Raptors' horrendous defense, but essentially saying that he's the same caliber defender as Amir and Bosh is completely ludicrous. He is a terrible defender.

[floppymoose]

I agree with Los Soles. On the occasions where I've heard announcers or other forum members break down defensive credit/blame for a particular play, I find that I disagree with them a very large percentage of the time. That makes stuff like Synergy and PDSS completely useless to me. I can't trust it.

[Nivek]

I used DeanO's Project Defensive Score Sheet as the basis for tracking the Wizards defense over the course of about 3 seasons. I collected additional information as well, then later worked with Dean on something that basically put PDSS on steroids. I understand floppymoose's concern, but the information was robust. I actually worked with a few teams providing them info from this kind of tracking. Front offices usually liked the data, and some coaches REALLY liked it. A few coaches didn't like it and said they wouldn't use it. Worth mentioning that comments like that invariably came from coaches who disagreed with the numbers about a particular player or two.

Anyway, I never got traction beyond 5-10 game stints because teams traditionally don't pay very much for statistical analysis. So many people want to "in" that they're willing to virtually give away their stuff. Teams weren't willing to pay me what I would need to be willing to do the tracking over a long period of time. (And it wasn't like I was asking for a max contract either. )

The real value of PDSS is NOT being able to generate a defensive rating. The real value came when we put thing on 'roids and were tracking by offensive play types, help defense, ball movement, and an array of other stuff. Being able to quantify who was good at what was great information to provide for coaches/front office types. Another value from systematic tracking: it measures effectiveness. Sometimes a player may look like he's doing a good job on film, when he isn't. That showed up in PDSS. Other times, there are guys who don't "look good," but are actually quite effective when it comes to the goal of defense: preventing the other team from scoring. That also showed up.

In many ways, I understand the coaches who said they wouldn't use it. They're typically inundated with advice. Everyone KNOWS what the coach should be doing. But a lot of the advice they get is bad. So ultimately, the coach has to trust himself and his staff.

I like Synergy and find their info useful. It's good enough (and cheap enough) that it pushes aside PDSS -- at least at the team level. Front office types think they have basically the same info they'd get from PDSS from Synergy. I don't think Synergy is as good, but I could be biased. I wish Synergy would adjust use terminology consistent with APBRmetric lingo. What they call a "possession" is what APBRmetrics would call a "play." As possession in APBRmetric parlance lasts until the other team gets the ball back. In Synergy, an offensive rebound creates a new possession. In APBRmetrics, an offensive rebound is a play within a possession -- it extends the possession, it doesn't start a new one.

[BorisDK1]

The metric may be simple, but BASKETBALL is complicated. That's the whole point. This is a massive, crucial philosophical divide. I think that it is extremely unlikely that either of us can convince the other or that we can find a comfortable middle road. I fundamentally disagree with you about our ability to accurately measure all the subtleties of defense directly. I'm not sure, however, what concerns you have with RAPM/APM.

My concerns with RAPM/APM is that it's an engineer's attempt to understand basketball. There's a lot of "noise" in however you do the adjustments, no matter how many years background there is to it. You're ultimately taking an indirect metric and subjecting it to a series of formulas, and there's only so much you can get from that.

Direct data is to be preferred, I think. And it's not that difficult to scorekeep a game with PDSS metrics. I've coached and officiated the game for years after I stopped playing, it's not that tricky. And, frankly, when you have PDSS data you have what actually happened. The game was watched, the results were tabulated - and it's reliable. My eyes probably weren't tricked, and God knows I spent way too much time to be deceitful or be deceived (thank God for PVRs).

But the divide in the numbers is enormous. We only have the Raptors to look at, but take a look at the difference among bigs.

2009-2010 PDSS DRat:
Amir 109.5
Bargs 110.0
Bosh 110.0

Multi-year RAPM:
Amir +3.3
Bargs -4.2
Bosh +2.3

Your stats have these guys as essentially equivalent, while RAPM says that Amir and Bosh are both very good defenders and that Bargs is the worst defender in the league. The one-year RAPM for 2009-2010 also has Bargs as the worst defender in the league. These are incredibly consistent with the fact that Bargs' increase in minutes has correlated with the plummeting defensive rating of the Raptors overall to the worst defensive team in the league two years running, as well as the fact that in simple net rating the team has been 7.41, 9.10, and 6.45 points better defensively per 100 possessions when he is on the bench for the past three seasons. He has also been among the three worst players league-wide in overall defensive rating (basketballvalue.com) for the past two years (115.87 and 115.17), topping the list in 2009-2010. RAPM gets more accurate with a greater accumulation of data, and every single year Bargs' defensive value has fallen further as the numbers have become clearer.

It was what it was. That one year, Bargnani actually played as hard as he ever has on the defensive end of the floor. He had to handle switches on every ball screen when Calderon was on the floor, he did a good job contesting on the perimeter, he did a decent job around the basket while too many other players did...nothing. Calderon was particularly horrendous; I'm sure you noticed his stop% was around the same level that uncontested action (i.e. "thin air") was. You'll also notice a distinct improvement in the team's performance in those games in which he didn't play. I believe the improvement was somewhere around 5 points / 100 possessions in those games.

Now, if you look at the 2010/11 data, you see Bargnani performing at a level which we more or less expect from him. Why? He didn't give a damn last year after about the first 10 games. I'm not saying Bargnani has ever been a good defender (he really hasn't, and the PDSS data showed that in a "good" year for him, he was still at a DRat of 110), but there are levels to his performance.

So maybe PDSS data has something to communicate. Maybe APM can only give you so much. I think in issues of doubt, direct data is to be preferred over indirect data, no? And just because APM indicates one thing, you can't go and ignore all data to the contrary. It might be helpful to question APMs results, too.

Bargs certainly doesn't get all the credit for the Raptors' horrendous defense, but essentially saying that he's the same caliber defender as Amir and Bosh is completely ludicrous. He is a terrible defender.

Was the same thing indicated in 2010-11?

Perhaps more careful analysis than "well, I don't like the results, so I'll toss them away" is needed.

[BorisDK1]

The real value of PDSS is NOT being able to generate a defensive rating. The real value came when we put thing on 'roids and were tracking by offensive play types, help defense, ball movement, and an array of other stuff. Being able to quantify who was good at what was great information to provide for coaches/front office types. Another value from systematic tracking: it measures effectiveness. Sometimes a player may look like he's doing a good job on film, when he isn't. That showed up in PDSS. Other times, there are guys who don't "look good," but are actually quite effective when it comes to the goal of defense: preventing the other team from scoring. That also showed up.

Nivek:

I'd imagine the exact tracking you were doing is fairly proprietary, but would you mind going into some detail? I'd love to do an enhanced version of PDSS. It is, after all, a metric designed for coaches, and if coaches are getting more rich information from your version, that's a good thing.

If you can't talk about it in public, can you email me any details at va3svd a rac . ca ?

I like Synergy and find their info useful. It's good enough (and cheap enough) that it pushes aside PDSS -- at least at the team level. Front office types think they have basically the same info they'd get from PDSS from Synergy. I don't think Synergy is as good, but I could be biased. I wish Synergy would adjust use terminology consistent with APBRmetric lingo. What they call a "possession" is what APBRmetrics would call a "play." As possession in APBRmetric parlance lasts until the other team gets the ball back. In Synergy, an offensive rebound creates a new possession. In APBRmetrics, an offensive rebound is a play within a possession -- it extends the possession, it doesn't start a new one.

Agreed entirely.

[EvanZ]

The metric may be simple, but BASKETBALL is complicated. That's the whole point. This is a massive, crucial philosophical divide. I think that it is extremely unlikely that either of us can convince the other or that we can find a comfortable middle road. I fundamentally disagree with you about our ability to accurately measure all the subtleties of defense directly. I'm not sure, however, what concerns you have with RAPM/APM.

My concerns with RAPM/APM is that it's an engineer's attempt to understand basketball. There's a lot of "noise" in however you do the adjustments, no matter how many years background there is to it. You're ultimately taking an indirect metric and subjecting it to a series of formulas, and there's only so much you can get from that.

Direct data is to be preferred, I think.

Do you have any data about the predictive power of PDSS? The main strength of RAPM - perhaps, it's only real strength - is it's predictive power. It's the best of all the metrics I've heard of so far.

[BorisDK1]

Do you have any data about the predictive power of PDSS? The main strength of RAPM - perhaps, it's only real strength - is it's predictive power. It's the best of all the metrics I've heard of so far.

It's not meant to be a predictive statistic, in the same way that ORat or DRat are not predictive statistics. It gives you an idea of how a player performed and from that you can take what you may.

[EvanZ]

Do you have any data about the predictive power of PDSS? The main strength of RAPM - perhaps, it's only real strength - is it's predictive power. It's the best of all the metrics I've heard of so far.

It's not meant to be a predictive statistic, in the same way that ORat or DRat are not predictive statistics. It gives you an idea of how a player performed and from that you can take what you may.

Have you looked at the correlation between PDSS and defensive RAPM?

[BorisDK1]

Have you looked at the correlation between PDSS and defensive RAPM?

Yes. But do I have to consider defensive RAPM the gold standard and then align everything else to it? "Oops, it's not possible that Amir Johnson allowed that score. I should have attributed that to Andrea Bargnani because despite what looked like that guy scoring over Amir, Andrea somehow mysteriously caused it to happen in some way I can't see, because defensive RAPM tells me so." Seriously - that's what's being asked to be believed in this thread.

And, having said that, I have no problem with the idea that Andrea Bargnani is a lousy defender, nor that Amir Johnson (who happens to be among my favourite players) is a good one. But I do not have confidence in defensive RAPM when I have direct data to tell me specifically what happened.

[floppymoose]

"Oops, it's not possible that Amir Johnson allowed that score. I should have attributed that to Andrea Bargnani because despite what looked like that guy scoring over Amir, Andrea somehow mysteriously caused it to happen in some way I can't see, because defensive RAPM tells me so." Seriously - that's what's being asked to be believed in this thread.

So the answer is, yes, exactly.

Seriously. Sometimes a player like Bargs causes a basket not because he did anything wrong, but because he didn't do something spectacular and unexpected.

The good defenders in this league will surprise you a couple of times a game, and erase a basket that would have happened and been blamed on someone else. If you are missing that type of player out there on the court, PDSS will never show it (properly). Someone watching the Bargs for what he does right or wrong at the micro level will miss the macro level issue: that he isn't saving you those extra couple of baskets.

[floppymoose]

I wish Synergy would adjust use terminology consistent with APBRmetric lingo. What they call a "possession" is what APBRmetrics would call a "play." As possession in APBRmetric parlance lasts until the other team gets the ball back. In Synergy, an offensive rebound creates a new possession. In APBRmetrics, an offensive rebound is a play within a possession -- it extends the possession, it doesn't start a new one.

Lol, I was just wishing more data out there was doing possessions the way Synergy apparently does.

viewtopic.php?p=28054311#p28054311

[BorisDK1]

So the answer is, yes, exactly.

Seriously. Sometimes a player like Bargs causes a basket not because he did anything wrong, but because he didn't do something spectacular and unexpected.

The good defenders in this league will surprise you a couple of times a game, and erase a basket that would have happened and been blamed on someone else. If you are missing that type of player out there on the court, PDSS will never show it (properly). Someone watching the Bargs for what he does right or wrong at the micro level will miss the macro level issue: that he isn't saving you those extra couple of baskets.

I'm sorry, PDSS does show that. Players who help their teammates get stops get credit for those stops, and player who don't, don't. Plus, players who are bad defenders don't get good DRats in a PDSS system. Look at Bargnani: in a great individual year for him (by his standards), he had a DRat of 110.0. That's horrendous. I really don't see the objections posed in this thread; PDSS tells us that obviously he isn't making his team good defensively.

And I'm sorry, any objection that starts with "forget what you saw (and documented!), look at this engineer's metric!" isn't going to carry much weight. The whole point of the metrics is to take us to the floor to ask questions of it. I find the indirect metrics get misused so much that it's not even funny.

And really - we're supposed to ignore what can be observed and counted to use the evidence of a system that tells us that last year Jose Calderon was a better defender than Andrei Kirilenko? Please.