wigglestrue wrote:The answer to the question "Ever Accounted For?" is clearly no...so I edited the thread title, also to include the other free throw changes that have gone unexamined. I think it's kind of embarrassing for basketball stat geekdom to be inventing elaborate metrics without first addressing this most basic kind of counting problem. For all the obsessing over efficiency and possessions, no one knows (or has even seriously estimated) the statistical significance of either the 3-to-make-2/2-to-make-1 rule or the pre-penalty non-shooting-foul single free throw. Geeks, you're better than this.
I think you clearly mean your tone to be tongue-in-cheek, but it still bothers me.
First and foremost it bothers me because you're right. This is clearly something basic that really should be accounted for, and it's risky for us to be doing too much historical analysis without making such adjustments. So it bothers me because I may have come to atraditional conclusions that will actually end up being beaten by the traditional ones. This is embarrassing.
Second, though: How is it a sign of geek incompetence that the NBA has been prone to rule changes they don't document well, while recording basic data that they can't even get that right? What exactly are the geeks supposed to do have done differently? (Granting that Oliver did something about this, but I think you're right that not everyone doing estimates did.) "Get all your ducks in a row before you give conclusions" sounds like nice advice, but when you don't really know where to look to be sure you've factored everything in, at a certain point you've simply got to make due with what you have.
Third, and maybe most importantly: My concern when I read something like this thread is that people will have a reaction along the lines of "Oh, well I guess we shouldn't listen to the geeks after all", which to my mind should not be the takeaway. To my mind this is like a data analyst being hired as a consultant and then getting reprimanded by the client because the client forgot to tell the analyst something important.
Still, what it does mean is that caution is a virtue. If you see dramatically unexpected results without any causal basis given, you want to be careful running too fast with the results.
Now last, let me spend a little time analyzing the guy who clearly would seem to be the center of this: Wilt. Exaggerating the free throws he takes could inflate pace, which would then decrease estimates of actual offensive effectiveness. If the effect is large enough, then what if the entire trend of shockingly disappointing Wilt offenses is just a quirk of bad data?
Okay, well, the theory here would presumably have to entail that the exaggeration would be larger as Wilt's free throws increased, since that would increase the free throws of the team, which would put his team into the bonus which causes the 3-to-2 change.
Alright, by current estimates, Wilt's team had its huge breakthrough in '67 when he stopped volume scoring. Perhaps much of what happened is simply that the change caused less free throws which then caused less pace inflation. So if we see that the 76ers in '66 had way more free throws than in '67, this becomes a very strong possibility. To the data!
Free throw attempts:
'66 76ers: 3141
'67 76ers: 3411
So...no, the obvious test actually gives us results in the opposite direction: It would seem entirely possible then that we have UNDERestimated how big the improvement to team offense was when Wilt stopped shooting so much.
For the heck of a lit, let's go to Wilt's volume scoring peak in '62, where he shot 500 more free throws than he did in '67 while clearly playing at a much faster pace. Surely at that point the team free throws would dwarf the '67 76ers free throws:
'62 Warriors: 3207
Nope, still no trends along these lines.
Alright so where does that leave us? Same rule: Be cautious. Be cautious of relying too much on data, but also be cautious of ignoring data based on presumption that using incomplete data is worse than using none. No matter the choice, there is risk. The only way to go, is to understand everything and make your expert judgment from there.