Is PDSS a good way to analyze the Raptors defense?

[Ripp]

You're such arse it's unbelievable. Ripp has been dipping deep into condescension like, "You don't understand statistics"... blah, blah.

No, in that specific circumstance, he demonstrated that he doesn't understand that particular aspect of statistics...i.e., that more samples combat noise. That is the heart of statistics...with enough samples, your noisy observations becomes almost noiseless (or rather, your ability to estimate whatever it is you are trying to estimate becomes better and better.)

[BorisDK1]

Do you believe the on/off DRTG captures those weaknesses better than PDSS? If you believe PDSS captures that aspect better than DRTG then is it safe to assume PDSS doesn't put a heavy weight on those aspects based on his overall rating?

I think PDSS gives you direct individual data of defensive performance and brings in team performance nicely. I think when you look at the ultimate product as well as the constituent individual parts (stop%, DPoss%) you really get a taste of what certain players are and are not doing. When you have a post player bearing less than average amount of possessions while they're on the floor, like both Bosh and Bargnani did this year, you know you're having a problem with not having game-impacting help defense. When you bring the team performance into the mix, as the ultimate DRat from the PDSS data does, I think you can see pretty quickly guys who simply are not rising above the challenge of perhaps guarding their own immediate assignment particularly well.

I have no problem looking at on/off court data. I think it's nice to have that information, it dovetails with PDSS data nicely.

[BorisDK1]

No, in that specific circumstance, he demonstrated that he doesn't understand that particular aspect of statistics...i.e., that more samples combat noise. That is the heart of statistics...with enough samples, your noisy observations becomes almost noiseless (or rather, your ability to estimate whatever it is you are trying to estimate becomes better and better.)

You can have as large a sample size as you want: using data that carries zero information about individual performance as, in fact, evaluating a particular player's performance is still a faulty use of the data.

In other words, the data isn't flawed: the use of it by the interpreter is.

[Ripp]

...Zero information about individual performance?

I mean, consider the following lineups:

Ripp-Kobe-Artest-Pau-Bynum
DWill-Kobe-Artest-Pau-Bynum

Observing 1000 minutes of play from the above lineups will tell us absolutely nothing about the defensive capabilities of DWill versus Ripp? Obviously there might be certain biases and noise in those 1000 minutes, but to say that there is zero information is a pretty bold hypothesis.

[Ripp]

Sorry, I didn't even need to use that lineup example to make my point. Clearly the defense that a team plays is a function of all of their individual defensive contributions...so the information is there, though it might be hidden by other variables, noise, etc.

[BorisDK1]

Sorry, I didn't even need to use that lineup example to make my point. Clearly the defense that a team plays is a function of all of their individual defensive contributions...so the information is there, though it might be hidden by other variables, noise, etc.

There you go. At best it provides indirect data about individual defensive contribution, which you can't really determine reliably. PDSS provides that.

[Ripp]

Lots of indirect information adds up. In the coinflipping example I gave, each coin flip doesn't tell me much about the probability of the coin coming up heads. But if I get to see 1000 or 1 billion coin flips, I can determine that probability to extremely high accuracy. Lots of things work like that more generally in statistics and statistical modeling.

[BorisDK1]

Lots of indirect information adds up. In the coinflipping example I gave, each coin flip doesn't tell me much about the probability of the coin coming up heads. But if I get to see 1000 or 1 billion coin flips, I can determine that probability to extremely high accuracy. Lots of things work like that more generally in statistics and statistical modeling.

And this is where you're falling down. Given a choice between your preferred statistical method which provides only indirect evidence, and observation of what actually happened on the basketball floor, you choose your method.

I have no problem looking at on/off court data or lineup data, but using it to interpret individual performance is fraught with difficulty. Choosing that method over a method that actually tracks individual performance isn't something I'd be caught dead doing, if I want to actually speak to how an individual defends.

[Ripp]

Alright, let's agree to disagree and simply drop the matter.

[Ripp]

Err, but as an aside for supersub15...do you know if Synergy published end-of-season box scores for their defensive stats for the entire league? Or if this can somehow be extracted from their stuff?

I've not played around with the site before, and am curious.

[disoblige]

http://www.mysynergysports.com/playoffd ... /index.htm

It has regular season defensive stats of players. There is no published info of all the team for the whole season or they would be facing lawsuit from synergy.

Too bad Raptors didnt make the playoffs.

[Ripp]

^-- Cool, thanks for the heads up.

[evenflow]

..i.e., that more samples combat noise. That is the heart of statistics...with enough samples, your noisy observations becomes almost noiseless (or rather, your ability to estimate whatever it is you are trying to estimate becomes better and better.)

True, typically, all else being equal, a larger sample size will yield more accurate results. Is that the case in the 9000+ min. Andrea has played? Are all those minutes more or less equal in value?

Now you may be absolutely correct in that, with the right algorithms, even if the data is not equal in value one can still yield accurate results but than the question that I would want answered is: are the 9000 min enough of a sample size to eliminate all the variables? Just because 9000 looks like a big number doesn't necessarily mean it is a large enough sample size to eliminate all the variables or 'noise'.

[Ripp]

Well, that is a great point....9000+ minutes might not be enough. Or maybe 9000+ minutes IS enough for a great algorithm, but you are only using a good one...so you need more data to compensate for your algorithm's lack of power.

So what you do is build your model on those 9000 minutes, then test its performance on some more data that you get later. Like, you might build a model based on 2006-2010 data, and then see how well it predicts wins, team and lineup Ortg/Drtg for the 2010-2011 season. If it does a good job on these things, then you'll have a sense that your model is pretty good. If it does poorly, then either you don't have enough data, or your algorithm is bad.

[ranger001]

Lots of indirect information adds up. In the coinflipping example I gave, each coin flip doesn't tell me much about the probability of the coin coming up heads. But if I get to see 1000 or 1 billion coin flips, I can determine that probability to extremely high accuracy. Lots of things work like that more generally in statistics and statistical modeling.

And this is where you're falling down. Given a choice between your preferred statistical method which provides only indirect evidence, and observation of what actually happened on the basketball floor, you choose your method.

I have no problem looking at on/off court data or lineup data, but using it to interpret individual performance is fraught with difficulty. Choosing that method over a method that actually tracks individual performance isn't something I'd be caught dead doing, if I want to actually speak to how an individual defends.

The difficulty I see with tracking individual performance is that sometimes its difficult to measure it in a team game like basketball, sometimes a certain person works well with his teammates and just makes his team better even though its difficult to measure, e.g. setting a good screen to set another guy free, reading an offensive play and telling a teammate where to go, etc. How do you suggest we overcome this except by measuring on/off court data?

[cdel00]

whoa some strange logic in this thread

1) The result of the next event in a series of random events is still random and NOT influenced by the past (Ripp) you can flip the coin a google times and the next time you flip it there is still a 50/50 chance of heads.

2) Increasing sample size does not reduce noise it actually increases noise. Each instance of an event has noise so when you sample the event you also sample the noise in that event as you sample many events you sample many occurances of noise. What you are hoping for in a large sample size is that thenoise is evenly balanced and cancels itself out but that is not fact it is hope. The only way to remove noise is to account for it in each sampling and remove it at source.

I have more to say but i must go now hope to post more later

[Reignman]

whoa some strange logic in this thread

1) The result of the next event in a series of random events is still random and NOT influenced by the past (Ripp) you can flip the coin a google times and the next time you flip it there is still a 50/50 chance of heads.

2) Increasing sample size does not reduce noise it actually increases noise. Each instance of an event has noise so when you sample the event you also sample the noise in that event as you sample many events you sample many occurances of noise. What you are hoping for in a large sample size is that thenoise is evenly balanced and cancels itself out but that is not fact it is hope. The only way to remove noise is to account for it in each sampling and remove it at source.

I have more to say but i must go now hope to post more later

Keep in mind though that the noise in this instance applies to each of the 5 players in any unit, so the fact that Andrea stands out in a negative manner with on/off DRTG consistently over a sample of 9000+ mins suggests something different than noise right?

If noise was the determining factor for Andrea to stand out like that you'd figure it would happen to the others as well. However, using on / off it seems that it only affects those that empirically seem like weak defenders. Mainly Jose and Bargnani.

Basically, as the sample gets larger the noise that affects each individual players gets smoother and smoother till it equalizes. The question is how much of a sample do you need for that to happen? I think 9000 mins is more than enough considering it spans over 4 NBA seasons; however, others might not feel the same way.

[Ripp]

whoa some strange logic in this thread

1) The result of the next event in a series of random events is still random and NOT influenced by the past (Ripp) you can flip the coin a google times and the next time you flip it there is still a 50/50 chance of heads.

2) Increasing sample size does not reduce noise it actually increases noise. Each instance of an event has noise so when you sample the event you also sample the noise in that event as you sample many events you sample many occurances of noise. What you are hoping for in a large sample size is that thenoise is evenly balanced and cancels itself out but that is not fact it is hope. The only way to remove noise is to account for it in each sampling and remove it at source.

I have more to say but i must go now hope to post more later

1. I didn't say the coin had a 50% chance of coming up heads. It has probability P of coming up heads, and it is your task to estimate that probability to high accuracy. This is something that can be easily done.
2. But again, if you are trying to compute some underlying functional of the underlying distribution (in this case, the chance of coming up heads), then more noisy samples help.

The key that you seem to be missing is that I'm not trying to predict what exactly the sequence of coin flips will be, but what the AVERAGE behavior will be.


To make the analogy with basketball, I'm not interested in determining exactly whether a player was good/bad in this specific play, but what their AVERAGE performance is. Of course, if in 90% of plays, they are good defensively, it is very suggestive..

[BorisDK1]

Keep in mind though that the noise in this instance applies to each of the 5 players in any unit, so the fact that Andrea stands out in a negative manner with on/off DRTG consistently over a sample of 9000+ mins suggests something different than noise right?

If noise was the determining factor for Andrea to stand out like that you'd figure it would happen to the others as well. However, using on / off it seems that it only affects those that empirically seem like weak defenders. Mainly Jose and Bargnani.

Basically, as the sample gets larger the noise that affects each individual players gets smoother and smoother till it equalizes. The question is how much of a sample do you need for that to happen? I think 9000 mins is more than enough considering it spans over 4 NBA seasons; however, others might not feel the same way.

Here's the problem: on/off court and lineup analysis doesn't tell you about the quality of anybody's defense as individuals. It does tell you there was a problem / success with those lineups, but not 1) against whom that took place, 2) who contributed to that performance-wise and who did not. On/off court analysis simply does not provide that information.

To make the leap from analyzing groups to an individual from that data is not supportable. It simply doesn't measure individual production and doesn't pretend to. Some individuals want to use the data that way, but they cannot make any conclusive and concrete arguments from it. PDSS does communicate that data, however.

[Ripp]

To make the leap from analyzing groups to an individual from that data is not supportable. It simply doesn't measure individual production and doesn't pretend to. Some individuals want to use the data that way, but they cannot make any conclusive and concrete arguments from it. PDSS does communicate that data, however.

Well, the nice thing about a test-driven methodology is that you can make up seemingly crazy hypotheses that don't seem supportable, and then just go ahead and test them and see if your intuition was right or wrong.

For example, suppose we work under the hypothesis that "a team's defense is the linear sum of the defensive capabilities of each of the individual player." Seems like a pretty simple and crappy model, no? Just assuming that things just add up? However, this hypothesis leads to a linear regression model, and a pretty nifty way to rate players called adjusted +/- (APM).

How well does this seemingly mediocre model do? Well, take a look at the top lineups in minutes played for the 2009-2010 season:

http://basketballvalue.com/topunits.php?&year=2009-2010

Notice how closely the column called "1 year adjusted +/-" matches the column called "Overall Rtg" for most of the lineups listed there. So this hypothesis doesn't seem to be very far from the truth.

And APM isn't even the best model in this family of algorithms...people have come up with even better techniques...there are lots of ways to improve it substantially.