Extensive Clutch Stats

Moderators: Clyde Frazier, Doctor MJ, trex_8063, penbeast0, PaulieWal

Gideon
Pro Prospect
Posts: 830
And1: 178
Joined: Feb 29, 2012

Re: Extensive Clutch Stats 

Post#41 » by Gideon » Fri May 3, 2013 7:30 am

ElGee wrote:
Gideon wrote:
ElGee wrote:
They have the two best career on/offs in the regular season. According to BBR, Kevin Garnett is the best in the playoffs for any player career 2001-2012.


I focused on regular season on/off, because I think sample size is a huge factor with on/off... for that reason I would put much more stock in regular season on/off (which in the case of KG incorporates almost exactly 10 times the number of games as the playoffs... I assume this is similar for most top players). I've seen bizarre on/offs with even 2 or 3 full seasons of sample size, but once you're looking at multiple seasons up to a career, it starts to seem like a pretty good indicator.

Obviously, KG is a great player (I think he might be 3rd in RS on/off, anyway after Dirk and LeBron), so I'm not saying it's an anomaly he's the leader in on/off for the playoffs... but for a stat that is SO dependent on sample size, I would be much more inclined to focus more on wherever there's a significant sample size (which is clearly RS in this case). Garnett has played 114 playoff games since 2001... that's barely more than a single season, which just is not a reliable sample size for on/off.

*Actually, thinking about this more... the very best solution seems like it would be to just combine numbers from RS and PS and get the largest sample size possible for on/off. That seems like it would be the best indicator. I don't have time to do that for LeBron, Dirk, and KG at this moment, but judging by a quick glance at the numbers, I would expect those to be the top 3 guys using that metric, with LeBron first and Dirk and KG very close for 2nd.


Gideon I don't actually see on/off that way. I don't think you are increasing accuracy by increasing sample at all. I haven't tested variation after 50 games say, versus 80, but I don't see that as the issue. I think the stat just isn't particularly accurate for measuring goodness. It is what it is, which is first and foremost an indicator of what's happening with a guy on the court versus off it. That's interesting. It's valuable. It's also a conditional measure, and for the purpose of measuring player goodness, is subject to some major confounding variables like teammates and opponents (this is why people try and adjust it).

I don't see increasing the sample as changing these issues at all. And similarly, I don't see the samples for multi-year PS, which I can post tomorrow, as being subject to a small sample. For his career, KG has 4800 minutes on and 1200 minutes off -- what sample-size argument can you come up with to object to that?


A bigger sample size increases accuracy with every other stat... why wouldn't it do so with on/off?

A player who has a "true" scoring avg of 20 ppg could easily get hot and average 35 ppg over a 5-game sample; over a season, that player will almost never be as far from his "true" average as he was during that 5-game sample, but the season can certainly still be an outlier (such as Michael Adams's 26.5 ppg season); over a 10-year-career, that player's ppg is going to be much closer to a "true" average (obv variables such as era/pace, system, etc. should be considered, too, but you get the idea).

On/off isn't any different. The bigger the sample size, the more likely it is to be accurate. Flip a coin 10 times, and 80% heads could easily happen... flip the coin 1000 times, and heads is always going to be very close to 50%. With on/off it's a little more complex than the coin-flip situation, since there are variables that affect on/off and should be evaluated to put it into context (just like with the ppg example)... but that's true of every other stat in basketball, as well.

The principle of bigger sample size being more accurate for on/off can also be demonstrated directly with on/off itself by observing that many players have a single season on/off that is higher than the highest career on-off since 2001. I'm not referring to a small number of elite seasons that "deserve" to be higher than LeBron or Dirk or KG's career on/off... it's a phenomenon that clearly has to do with sample size. For example, Baron Davis's on/off for the 06/07 season is higher than Dirk and Lebron's league-best career RS on/offs. There are plenty of similar examples of single-season outliers like this. However, there are zero career outliers with a normal-length career to anywhere near this extent that I've come across... I guess it's possible I'm missing one or two, but that would still be vastly less than with single seasons.

Here's an on/off leaderboard from 07-08 (http://www.82games.com/ONSORT6.HTM). #2 and #3 on that list are Jamison and Carter... and there are plenty of other surprises/outliers on it. The career RS on/off leaderboard has Dirk, LeBron, and KG as the top three, and many other stars right behind them. There are just way fewer outliers on the career list than for any single season... I think sample size has to be the reason for this.

When we look at the PS -- (most stars have the equivalent of between about half a season to a little over a season worth of PS games to evaluate) -- there are also many outliers. For example, Chris Paul's PS career on/off (over 39 games) is -2.0. This is obviously not representative of Paul's true impact. However, Paul's RS career on/off (over a 555-game sample size) is +8.2, which is star-level and exactly the same as Tim Duncan. In this case, the bigger sample size has produced a much more accurate result, and I could show example after example like this.

I'm not saying on/off is a magic end-all, be-all stat (although I do like it more than most seem to on this forum). I've found to it be pretty valuable over a large sample size, and not very valuable over a small sample size. This is just speculation on my part, but I think the complexities involved with on/off that you touched upon (rotations, system, opponents, pace, etc.) are complex enough that a larger sample size is needed to naturally adjust for them than with many other stats that are less variable-dependent.
Keller61
RealGM
Posts: 10,128
And1: 5,041
Joined: Feb 12, 2013

Re: Extensive Clutch Stats 

Post#42 » by Keller61 » Fri May 3, 2013 9:57 am

I'm surprised by how bad Rose's '12 numbers are - I thought he was really good in the clutch last year. I guess injury issues might have hurt those numbers a bit. IIRC he had a few games where he was just horrible coming off a groin injury, but earlier in the season he was making a bunch of clutch baskets.

I'm actually impressed by Kobe's stats, specifically how efficient he is, given the tough shots he always seems to take in crunch time. One thing you notice is that with all these star players the TS% is much higher than efg%, so they are all good at getting to the line in crunch time.

Wade's '09 stats are insane, especially the blocks. He was a beast on both ends.

I wouldn't put much stock in the +/- numbers, as that depends a lot on the quality of the team, as evidenced by the numbers for Nash.

I would like to see the stats for Shaq and Dwight Howard if you have them. Thanks!
HeatRing2012
Bench Warmer
Posts: 1,326
And1: 293
Joined: Feb 27, 2011
 

Re: Extensive Clutch Stats 

Post#43 » by HeatRing2012 » Fri May 3, 2013 4:01 pm

ElGee wrote:
Gideon wrote:
HeatRing2012 wrote:good work

according to +/-, Lebron and Dirk are the greatest clutch performers in the NBA for their career. do I read that right?


LeBron and Dirk also have the two best career ON/OFFs since that stat has been tracked by basketball-reference. I think there's a strong case to be made that they are the two most impactful stars of their generation (with LeBron first, and Dirk second). It's hard to put Dirk about Duncan, and I still have Duncan ranked higher on my list at the moment, but some of these stats that are emerging are at least making me consider that question.


They have the two best career on/offs in the regular season. According to BBR, Kevin Garnett is the best in the playoffs for any player career 2001-2012.

can you provice me with a link?

btw. I like Gideons idea to mix RS and PS +/-.
however I'd propose to weight the post season more than the regular season.
a naive approach would be a comparision between post season and regular season pace. e.g. lets say league average pace in regular season shall be 95 and 85 for the post season, thus the decreased available possessions (influencing the on/off court score changing) in the post season should be weighted by 11% for the +/- impact.
the same could be done by including SRS changes from regular to post seaosn (accounting for the tougher competition). given the SRS spread of the usual regular season, this would mean that the post season could also be weighted by additional 15-20%.
which leaves us with a ~30% overall increase of the post season +/- weight compared to the regular season
(all numbers are guesses)
colts18
Head Coach
Posts: 7,434
And1: 3,255
Joined: Jun 29, 2009

Re: Extensive Clutch Stats 

Post#44 » by colts18 » Fri May 3, 2013 4:25 pm

ElGee wrote:I don't see increasing the sample as changing these issues at all. And similarly, I don't see the samples for multi-year PS, which I can post tomorrow, as being subject to a small sample. For his career, KG has 4800 minutes on and 1200 minutes off -- what sample-size argument can you come up with to object to that?

1200 minutes off is the small sample, not the 4800 minutes on. For a 40 MPG player, that 1200 minutes equals about 30 games, a good portion of it with bench players or garbage time since KG played like 95% of the important minutes for Minnesota.

The sample is small because if you include 97-2000 where KG was negative in the playoffs, his career postseason +/-suffers some. All it takes is like 100 minutes of really good or bad play to change that sample significantly.


btw, james Harden +/- in the postseason is higher than KG's, though his sample is only 400 minutes less of off time. I think if you include 97-2000, Nick Collison's +/- will rank up there, maybe as the best.
The Infamous1
Lead Assistant
Posts: 5,733
And1: 1,025
Joined: Mar 14, 2012
   

Re: Extensive Clutch Stats 

Post#45 » by The Infamous1 » Fri May 3, 2013 5:01 pm

aal04 wrote:Clutch to me is last 24 seconds of the clock.


Clutch to me is playoff performance
We can get paper longer than Pippens arms
ElGee
Assistant Coach
Posts: 4,041
And1: 1,208
Joined: Mar 08, 2010
Contact:

Re: Extensive Clutch Stats 

Post#46 » by ElGee » Fri May 3, 2013 5:42 pm

Gideon wrote:
ElGee wrote:
Gideon wrote:
I focused on regular season on/off, because I think sample size is a huge factor with on/off... for that reason I would put much more stock in regular season on/off (which in the case of KG incorporates almost exactly 10 times the number of games as the playoffs... I assume this is similar for most top players). I've seen bizarre on/offs with even 2 or 3 full seasons of sample size, but once you're looking at multiple seasons up to a career, it starts to seem like a pretty good indicator.

Obviously, KG is a great player (I think he might be 3rd in RS on/off, anyway after Dirk and LeBron), so I'm not saying it's an anomaly he's the leader in on/off for the playoffs... but for a stat that is SO dependent on sample size, I would be much more inclined to focus more on wherever there's a significant sample size (which is clearly RS in this case). Garnett has played 114 playoff games since 2001... that's barely more than a single season, which just is not a reliable sample size for on/off.

*Actually, thinking about this more... the very best solution seems like it would be to just combine numbers from RS and PS and get the largest sample size possible for on/off. That seems like it would be the best indicator. I don't have time to do that for LeBron, Dirk, and KG at this moment, but judging by a quick glance at the numbers, I would expect those to be the top 3 guys using that metric, with LeBron first and Dirk and KG very close for 2nd.


Gideon I don't actually see on/off that way. I don't think you are increasing accuracy by increasing sample at all. I haven't tested variation after 50 games say, versus 80, but I don't see that as the issue. I think the stat just isn't particularly accurate for measuring goodness. It is what it is, which is first and foremost an indicator of what's happening with a guy on the court versus off it. That's interesting. It's valuable. It's also a conditional measure, and for the purpose of measuring player goodness, is subject to some major confounding variables like teammates and opponents (this is why people try and adjust it).

I don't see increasing the sample as changing these issues at all. And similarly, I don't see the samples for multi-year PS, which I can post tomorrow, as being subject to a small sample. For his career, KG has 4800 minutes on and 1200 minutes off -- what sample-size argument can you come up with to object to that?


A bigger sample size increases accuracy with every other stat... why wouldn't it do so with on/off?

A player who has a "true" scoring avg of 20 ppg could easily get hot and average 35 ppg over a 5-game sample; over a season, that player will almost never be as far from his "true" average as he was during that 5-game sample, but the season can certainly still be an outlier (such as Michael Adams's 26.5 ppg season); over a 10-year-career, that player's ppg is going to be much closer to a "true" average (obv variables such as era/pace, system, etc. should be considered, too, but you get the idea).

On/off isn't any different. The bigger the sample size, the more likely it is to be accurate. Flip a coin 10 times, and 80% heads could easily happen... flip the coin 1000 times, and heads is always going to be very close to 50%. With on/off it's a little more complex than the coin-flip situation, since there are variables that affect on/off and should be evaluated to put it into context (just like with the ppg example)... but that's true of every other stat in basketball, as well.

The principle of bigger sample size being more accurate for on/off can also be demonstrated directly with on/off itself by observing that many players have a single season on/off that is higher than the highest career on-off since 2001. I'm not referring to a small number of elite seasons that "deserve" to be higher than LeBron or Dirk or KG's career on/off... it's a phenomenon that clearly has to do with sample size. For example, Baron Davis's on/off for the 06/07 season is higher than Dirk and Lebron's league-best career RS on/offs. There are plenty of similar examples of single-season outliers like this. However, there are zero career outliers with a normal-length career to anywhere near this extent that I've come across... I guess it's possible I'm missing one or two, but that would still be vastly less than with single seasons.

Here's an on/off leaderboard from 07-08 (http://www.82games.com/ONSORT6.HTM). #2 and #3 on that list are Jamison and Carter... and there are plenty of other surprises/outliers on it. The career RS on/off leaderboard has Dirk, LeBron, and KG as the top three, and many other stars right behind them. There are just way fewer outliers on the career list than for any single season... I think sample size has to be the reason for this.

When we look at the PS -- (most stars have the equivalent of between about half a season to a little over a season worth of PS games to evaluate) -- there are also many outliers. For example, Chris Paul's PS career on/off (over 39 games) is -2.0. This is obviously not representative of Paul's true impact. However, Paul's RS career on/off (over a 555-game sample size) is +8.2, which is star-level and exactly the same as Tim Duncan. In this case, the bigger sample size has produced a much more accurate result, and I could show example after example like this.

I'm not saying on/off is a magic end-all, be-all stat (although I do like it more than most seem to on this forum). I've found to it be pretty valuable over a large sample size, and not very valuable over a small sample size. This is just speculation on my part, but I think the complexities involved with on/off that you touched upon (rotations, system, opponents, pace, etc.) are complex enough that a larger sample size is needed to naturally adjust for them than with many other stats that are less variable-dependent.


You are making some assumptions that aren't quite right -- I've bolded them. In short, increasing sample size does not always make a stat more accurate -- you will not get closer to "true impact" by increasing sample here. For one, you are introducing other variables, the most basic of which is time (ie players change, and so do situations). But there's more, which has to do with validity of measurement...

The reason why sample size is important is because of variance. If something had no variance, we wouldn't need much of a sample size. Weight is a good example of this -- my weight fluctuates constantly but only within a band of a few pounds (water). I don't need to weigh myself over and over and over and over throughout the day (or week) to eliminate variance from my weight measurement -- it's very consistent of my "true" weight (aka "average water weight") to within about 2-3% just from a single measurement, every time!

Once you have a sufficient sample size you don't have to worry about variance, but increasing the sample does not increase the validity (accuracy) of the stat. You still have the same confounds to worry about at 2000 MP as you do at 10,000 MP. Once you have a sufficient shooting sample of FT's, you can have a really accurate idea of a players FT%, and ONLY his FT%. Increasing the sample won't tell you about his quality as a shooter (which is a related but separate issue). Similarly, once you have a sufficient sample of on/off minutes you can determine a players net on/off, and ONLY his on/off. Increasing the sample won't tell you more about his goodness as a player, which is also a related but separate issue. This is the difference between reliability and validity (we're talking about the far-left bullseye below).

Image

With the +/- family, we obviously need a good sample size to overcome variance. Without presenting any formal statistical tests on a sample, just look at the change from 50 games to the end of the season to see how much the increase sample is stabilizing the stat:

I used BBR to quickly scoop 52 players with at least 800 off minutes in the first 50 games.

18 (35%) had a change of less than 1 pt/48
30 (58%) had a change of less than 2 pts/48
39 (75%) had a change of less than 3 pts/48
49 (94%) had a change of less than 4 pts/48

The mean change from game 50 to game 82 for these players (at least 1000 min on, 800 off in first 50g) was 1.9 pts/48. The three outliers were from Miami and Denver -- only two of seven teams that improved by more than a point/48 as a team from the 50 game mark to the 82 game mark.


This is nearly identical behavior to what we see with all MOV-related variance. Which is to say, for teams without major lineup or injury changes (or some other noticeable change), at about 20 games (960 minutes) we start to see strong stabilization. The variance in MOV from 50 games into a season to 82 games -- WITHOUT accounting for injury/lineup changes -- is nearly identical to the variance we seen in individual on/off changes from 50 games to 82 games.

Variance will be an issue with an off sample of 500 MP, but as the sample increases off will be less of an issue. When you get to 800, 1000, or 1200 off minutes, (a) sample size is a small concern and (b) the on number is quite stable and useful. But once variance isn't an issue, increasing that number more won't increase accuracy as a measure of goodness...it will actually just introduce further problems like aging and different team environments.
Check out and discuss my book, now on Kindle! http://www.backpicks.com/thinking-basketball/
ElGee
Assistant Coach
Posts: 4,041
And1: 1,208
Joined: Mar 08, 2010
Contact:

Re: Extensive Clutch Stats 

Post#47 » by ElGee » Fri May 3, 2013 5:55 pm

colts18 wrote:
ElGee wrote:I don't see increasing the sample as changing these issues at all. And similarly, I don't see the samples for multi-year PS, which I can post tomorrow, as being subject to a small sample. For his career, KG has 4800 minutes on and 1200 minutes off -- what sample-size argument can you come up with to object to that?

1200 minutes off is the small sample, not the 4800 minutes on. For a 40 MPG player, that 1200 minutes equals about 30 games, a good portion of it with bench players or garbage time since KG played like 95% of the important minutes for Minnesota.

The sample is small because if you include 97-2000 where KG was negative in the playoffs, his career postseason +/-suffers some. All it takes is like 100 minutes of really good or bad play to change that sample significantly.


btw, james Harden +/- in the postseason is higher than KG's, though his sample is only 400 minutes less of off time. I think if you include 97-2000, Nick Collison's +/- will rank up there, maybe as the best.


You are reinforcing the point --

(1) 1200 MP is 25 FULL games. What kind of accuracy do you think a 25-game sample holds for team performance? How much will another game or two change it?

(2) KG's career PS on/off is 12.6 according to NBA.com. If you exclude 1997-2000 (nearly 700 on, 100 off), it's 14.4. We must have different semantic concepts if you think that including a sample from when a player was ostensibly not as valuable by every measurement leads to a small overall reduction in value is "significantly" changing a sample.

When you have a sufficient sample, outlying data WON'T significantly change the sample... which it doesn't. (His 1997-2000 on/off is -0.1.)
Check out and discuss my book, now on Kindle! http://www.backpicks.com/thinking-basketball/
Gideon
Pro Prospect
Posts: 830
And1: 178
Joined: Feb 29, 2012

Re: Extensive Clutch Stats 

Post#48 » by Gideon » Fri May 3, 2013 8:02 pm

If there is a sufficient sample size cut off for on/off (and there are definitely things where there really isn't, or at least where we will never approach that number in real life), that cut off seems to me to be way higher than one season. I think what you (El Gee) wrote about how sample size and statistics work is interesting and I'm genuinely glad I read it, but it didn't address at all the evidence I posted that smaller samples sizes are very unreliable for on/off and bigger ones are actually pretty reliable... and that was only a smidgen of what could be posted to back that up. Examples like the ones I gave are abundant for on/off, while counter-examples are flimsy and very tough to find.

There are obviously instances where a larger sample size keeps improving accuracy. I gave the coin flip example, where increasing sample size increases accuracy pretty much ad infinitum (not literally always the case, since the coin could go 2-2, making accuracy exact, but then go 5001-4999 over 10K flips; however, on average the more flips, the more accurate the outcome, which is the best we can do... also, if there is some "accuracy threshold" in this example, it's obviously a truly massive number of flips). You gave the example of weight being an instance where the larger sample size doesn't matter. I only agree with this if you're weighing a statue (in which case, once is clearly enough); people change all the time, but still have a "true" average... most people gain and lose weight at different times in their life, and even over a few months or a year. If you only test them a few times, you might find them when they're especially skinny (maybe they've been doing lots of cardio and dieting) or especially heavy (maybe a sedentary, food-binging period). Testing them a bunch of different times will be much more likely to get you close to their actual average weight.

I'm also not on board with your FT% example... there are absolutely outlier seasons (and even multi-season stretches) for FT%. Do you think Jose Calderon was ever a "true" 98% FT shooter? Of course not; he was a very good FT shooter on a massive hot streak over a small sample size. (In essence, that was his "skinny jeans" season.) The next season, Calderon figuratively let himself go, (getting the equivalent of uncharacteristically heavy for a short time in your weight example) by shooting only 79.8% from the line. Well, Calderon's "true" FT% seems to be somewhere in the high 80s. We have a pretty good overall sample size for him in his entire career, and, thus, more-or-less know this. However, if we only had one or two seasons of his to look at, there's a good chance we would misjudge his free-throw-shooting ability by a significant margin. A larger sample size seems to be much better in this case... and, I still contend, in the vast majority of cases with stats.

One thing I'm not clear about is if you disagree with my assertion that career on/off is much more accurate than single-season (or even 2 or 3 season) on/off, or if you agree career on/off IS more accurate -- (I would think it would be hard not to with all the evidence showing this) -- but feel the reason for this increased accuracy is not connected to sample size? If so, what else would be the reason? You bolded my statement about negative 2.0 not being CP3's true impact, indicating you felt that was an incorrect assumption. You actually think CP3 has hurt his team(s) in the playoffs so far in his career? That's hard for me to believe. It seems to me like there's just all sorts of evidence out there that a substantial sample size (i.e. multiple seasons) is vastly more accurate when looking at on/off.
User avatar
E-Balla
RealGM
Posts: 35,828
And1: 25,127
Joined: Dec 19, 2012
Location: The Poster Formerly Known As The Gotham City Pantalones
   

Re: Extensive Clutch Stats 

Post#49 » by E-Balla » Fri May 3, 2013 8:25 pm

Can you put up MJ's numbers from only 97 and 98 and Shaq from 99-02?
colts18
Head Coach
Posts: 7,434
And1: 3,255
Joined: Jun 29, 2009

Re: Extensive Clutch Stats 

Post#50 » by colts18 » Fri May 3, 2013 8:37 pm

GC Pantalones wrote:Can you put up MJ's numbers from only 97 and 98 and Shaq from 99-02?

He put up MJ's numbers in the OP.

Here is Shaq's per 48 numbers for the 2000-2002:

27.7 pts, 54 FG%, .560 TS%, 14.4 reb, 3.7 AST, 6.6 blk, +15.4 +/-
User avatar
E-Balla
RealGM
Posts: 35,828
And1: 25,127
Joined: Dec 19, 2012
Location: The Poster Formerly Known As The Gotham City Pantalones
   

Re: Extensive Clutch Stats 

Post#51 » by E-Balla » Fri May 3, 2013 8:44 pm

colts18 wrote:
GC Pantalones wrote:Can you put up MJ's numbers from only 97 and 98 and Shaq from 99-02?

He put up MJ's numbers in the OP.

Here is Shaq's per 48 numbers for the 2000-2002:

27.7 pts, 54 FG%, .560 TS%, 14.4 reb, 3.7 AST, 6.6 blk, +15.4 +/-

Thanks. I missed MJ somehow. That scoring :o. He was old too...
User avatar
acrossthecourt
Pro Prospect
Posts: 984
And1: 729
Joined: Feb 05, 2012
Contact:

Re: Extensive Clutch Stats 

Post#52 » by acrossthecourt » Sat May 4, 2013 12:03 am

The Infamous1 wrote:
aal04 wrote:Clutch to me is last 24 seconds of the clock.


Clutch to me is playoff performance

Clutch to me is the indefinable "will of the warrior," determined by the good ol' eye test, which remains concrete despite a bevy of so called "facts" and statistics.
Twitter: AcrossTheCourt
Website; advanced stats based with a few studies:
http://ascreamingcomesacrossthecourt.blogspot.com
Not Bias
Banned User
Posts: 2,305
And1: 63
Joined: May 05, 2010
Location: Inside of Javale Mcgee's head

Re: Extensive Clutch Stats 

Post#53 » by Not Bias » Sat May 4, 2013 12:41 am

acrossthecourt wrote:Clutch to me is the indefinable "will of the warrior," determined by the good ol' eye test, which remains concrete despite a bevy of so called "facts" and statistics.

Hmm...can't tell if your taking a shot at Kobe or not?... :dontknow:
ElGee
Assistant Coach
Posts: 4,041
And1: 1,208
Joined: Mar 08, 2010
Contact:

Re: Extensive Clutch Stats 

Post#54 » by ElGee » Sat May 4, 2013 5:25 pm

Gideon wrote:If there is a sufficient sample size cut off for on/off (and there are definitely things where there really isn't, or at least where we will never approach that number in real life), that cut off seems to me to be way higher than one season. I think what you (El Gee) wrote about how sample size and statistics work is interesting and I'm genuinely glad I read it, but it didn't address at all the evidence I posted that smaller samples sizes are very unreliable for on/off and bigger ones are actually pretty reliable... and that was only a smidgen of what could be posted to back that up. Examples like the ones I gave are abundant for on/off, while counter-examples are flimsy and very tough to find.

There are obviously instances where a larger sample size keeps improving accuracy. I gave the coin flip example, where increasing sample size increases accuracy pretty much ad infinitum (not literally always the case, since the coin could go 2-2, making accuracy exact, but then go 5001-4999 over 10K flips; however, on average the more flips, the more accurate the outcome, which is the best we can do... also, if there is some "accuracy threshold" in this example, it's obviously a truly massive number of flips). You gave the example of weight being an instance where the larger sample size doesn't matter. I only agree with this if you're weighing a statue (in which case, once is clearly enough); people change all the time, but still have a "true" average... most people gain and lose weight at different times in their life, and even over a few months or a year. If you only test them a few times, you might find them when they're especially skinny (maybe they've been doing lots of cardio and dieting) or especially heavy (maybe a sedentary, food-binging period). Testing them a bunch of different times will be much more likely to get you close to their actual average weight.

I'm also not on board with your FT% example... there are absolutely outlier seasons (and even multi-season stretches) for FT%. Do you think Jose Calderon was ever a "true" 98% FT shooter? Of course not; he was a very good FT shooter on a massive hot streak over a small sample size. (In essence, that was his "skinny jeans" season.) The next season, Calderon figuratively let himself go, (getting the equivalent of uncharacteristically heavy for a short time in your weight example) by shooting only 79.8% from the line. Well, Calderon's "true" FT% seems to be somewhere in the high 80s. We have a pretty good overall sample size for him in his entire career, and, thus, more-or-less know this. However, if we only had one or two seasons of his to look at, there's a good chance we would misjudge his free-throw-shooting ability by a significant margin. A larger sample size seems to be much better in this case... and, I still contend, in the vast majority of cases with stats.

One thing I'm not clear about is if you disagree with my assertion that career on/off is much more accurate than single-season (or even 2 or 3 season) on/off, or if you agree career on/off IS more accurate -- (I would think it would be hard not to with all the evidence showing this) -- but feel the reason for this increased accuracy is not connected to sample size? If so, what else would be the reason? You bolded my statement about negative 2.0 not being CP3's true impact, indicating you felt that was an incorrect assumption. You actually think CP3 has hurt his team(s) in the playoffs so far in his career? That's hard for me to believe. It seems to me like there's just all sorts of evidence out there that a substantial sample size (i.e. multiple seasons) is vastly more accurate when looking at on/off.


Everything should be addressed in the last post although I should have clearer. Re-read it after reading this and that will (hopefully!) be clear.

-In general, bigger sample is better. But you aren't taking the same sample (!) with on/off because circumstances change. Thus there are serious diminish returns on this idea of multi-year samples. Once we reach a certain threshold (800-1000 MP) I will no longer be too concerned with variance, but that does not mean I can increase sample and make on/off a "better" stat than what it is.

I personally like connected, multi-year situations where the circumstances change very little for on/off, but it's a starting stat for me -- I'm never going to put too much stock without digging into the lineups. My prioritization is typically "what's the on? who was on?" then "what's the off? who was off?"

-Weight of a calorically-maintaining individual can never really be precise because of water fluctuation. I chose this is an example deliberately because it's not important to find the "true" weight (however you want to define that). What's important is having a really accurate approximation of the true weight. Similarly, finding a "true" on/off isn't important because the circumstances are always in flux -- we don't have balanced lineups on and off where we are ramping up sample to reduce variance -- but having a really accurate approximation is.

eg True On/Off on 2012 Team: +12.1745. The range we observe with a sufficient sample: +11.6 to +12.7. This is sufficient since on/off is not a player "goodness" measurement.

-Calderon took 150 FT's. That's not a "sufficient" number of FT's, no, regardless of whether it was "1 year" of FT's for him.

-Career v Single-Season: We need to be careful with the word "accurate." On/Off is not an incredibly accurate indicator of "impact."
Check out and discuss my book, now on Kindle! http://www.backpicks.com/thinking-basketball/
ElGee
Assistant Coach
Posts: 4,041
And1: 1,208
Joined: Mar 08, 2010
Contact:

Re: Extensive Clutch Stats 

Post#55 » by ElGee » Sat May 4, 2013 5:28 pm

Full circle: Your original assessment was LBJ and Dirk are the two most impactful players of this generation because of on/off. I don't think that's accurate, and I was simply noting that you are leaving out a part of the sample for your conclusion, and it happens to be the part that most people obsess over (the playoffs).

Yes, the PS is small-sampled, but when you introduce the same stat you wanted to use -- career on/off -- that's much less of an issue. Hope that makes sense...
Check out and discuss my book, now on Kindle! http://www.backpicks.com/thinking-basketball/
Gideon
Pro Prospect
Posts: 830
And1: 178
Joined: Feb 29, 2012

Re: Extensive Clutch Stats 

Post#56 » by Gideon » Sun May 5, 2013 9:19 am

ElGee wrote:Full circle: Your original assessment was LBJ and Dirk are the two most impactful players of this generation because of on/off. I don't think that's accurate, and I was simply noting that you are leaving out a part of the sample for your conclusion, and it happens to be the part that most people obsess over (the playoffs).

Yes, the PS is small-sampled, but when you introduce the same stat you wanted to use -- career on/off -- that's much less of an issue. Hope that makes sense...


I get your point(s)... I don't agree with everything you posted about this, as I do think we generally tend to get closer to a good approximation of a true avg (I forgot exactly how you put it) with a bigger sample of on/off, even with changing circumstances. I posted examples that I felt supported this, but I guess they didn't do much for you.

As far as my original assessment, it was not that LBJ and Dirk were the two most impactful players of their generation because of on/off. It was that on/off was evidence pointing that direction, which viewed in concert with the other evidence we have was leading me to think more about that possibility (even though I stated I still ranked Duncan ahead of Dirk). There's a very big difference between that and just looking at on/off and thinking it gives me an exact ranking (which I wouldn't do).
aal04
Sixth Man
Posts: 1,543
And1: 559
Joined: Mar 27, 2012

Re: Extensive Clutch Stats 

Post#57 » by aal04 » Mon May 6, 2013 12:09 am

The Infamous1 wrote:
aal04 wrote:Clutch to me is last 24 seconds of the clock.


Clutch to me is playoff performance



You could have 5 LBJs in your team and win by 100pts each game, and one of them is scoring 30pts a game. Is that clutch?

Return to Player Comparisons