azuresou1 wrote:Defensively, you should re-read how I'm calculating DVORP. I'm not making any assumptions about opponent data; everything is based solely on a player's stats, his team overall rating, and the league average. I already acknowledged earlier that we should expect that this would correlate extremely highly with Team DRTG. If I sound defensive, it's because I don't enjoy the implication that I'm being "suspicious" or "cheating" in my analysis.
I understand how you're calculating DVORP. The big component of DVORP is when you compare player drtg to team drtg and team drtg to lg drtg. You can just cancel these two terms out and call it player drtg compared to lg drtg. Drtg in and of itself, when you minutes weight it by player for a given team, will reconcile to team drtg. We're just stating the obvious, which is that you are largely anchoring player DVORP to team drtg, so it should reconcile fairly well. That reconciliation is only good to the extent that individual drtg is a good metric.
azuresou1 wrote:Offensively, I'm not sure how you're finding that calculated ORTG falls 3-5 pts away from actual 20% of the time. I have 2 teams that fall outside a 3 pt boundary - BOS and NYK - and one that sits right at 3 (MIL). While not ideal, that's half your stated rate.
My apologies. I was working from memory. I should have stated 2-5 pts from average. I don't have your full data set for low min players, nor did I allocate players who were traded to teams, but based upon what I do have (which still accounts for the overwhelming majority of team minutes), this is what I saw.
BOS 4.5 pts off
NYK 3.4
MIL 3.2
GSW 2.5
ATL 2.4
UTA 2.0
TOR 2.0
WAS 2.0
PHO 2.0
So 30% of the league is off by 2 pts (rounded to nearest tenth) or more. That's not good. Keep in mind some of these may not be off by as much when you include traded players or those falling below the minute threshold. Others I haven't listed may also appear too, however. This is based upon 4.11 OVORP = lg avg, which may be slightly different when including lower min players.
azuresou1 wrote:Anyways... as a validation I looked at year-over-year change at a player level, the idea being that if VORP works well, we should see either minimal unexplainable changes year-over-year.
http://www.buildingbetterball.com/2013/ ... ar-change/
Summary is that we do indeed see some significant swings in DVORP, but primarily among backup big men who get new (usually bigger) roles. However, among big minute players, the variance is significantly less. I also believe the OVORP variances observed are perfectly indicative of changes in player performance.
This type of test doesn't necessarily validate your study. It only tells you that similar players producing similar numbers get similar results year over year. The results may very well still be wrong. If you want to validate the study, I would suggest doing an out of sample test at the team level. Calculate OVORP and DVORP for 2-3 prior years for each player. From this calculate team Ortg and Drtg. Then look at the error terms between your implied team o and d rtgs and actual team o and d rtgs. Compare these error terms to other studies. Example: dsmok1 has an Off +/- adjustment he has included in his spreadsheet. A 0.4 adjustment = a team error of 2 pts (04. x 5 players on the court). Also, why do you believe that, "OVORP variances observed are perfectly indicative of changes in player performance"? Is it because you want to believe? Is it because you've tested the variances in OVORP compared to ORAPM and note similar consistency? Some other reason?
Finally, IMO, you still haven't addressed the issues that someone looking at this critically will bring up:
1) How did you select the weights in your analysis? Is there anything to it other than these are the weights you came up with on your own?
2) How can you explain that a replacement level player plays better D than an average NBA player? Did you look at the commentary I directed you to from Dsmok1? Did you consider that first or second year 10-day players aren't true replacements?
3)Have you considered at least making a forced adjustment for each players OVORP and DVORP such that the sum of player values for a team will reconcile to actual team OVORP and DVORP?
4) How did you empirically derive your position adjustment?
5) Have you attempted to validate your data vs. SPM or other means?