A Crisis of Criteria leading up to the 2023 Top 100
Posted: Sat Feb 18, 2023 7:14 pm
Hey y'all,
So I wanted to share with people where my head is at pertaining to Top 100 criteria, and hopefully get into fruitful conversation on the topic.
I'll first note that Tim made a thread in this vein already that was great, but this is something I'm just saying is a crisis for me, so I wanted to just start fresh speaking for myself.
Here's the essence of the "crisis" as I see it:
1. After decades and decades of relatively stable strategic equilibrium in the NBA, now we're in the midst of a massive shift which can be summed up fairly well by the change in ORtg.
When we look at bk-ref's year's ORtg's, we see a league that hovered within a 6 point range (102.5 to 108.5) from '78-79 to '15-16, and in the past few years we've now climbed to the point where we're approaching another 6 points beyond that.
2. If this were something caused by a 2016 rule change that simply made scoring easier, it would straight forward to at least diagnose the dilemma in an era-separated way, but the reality is that the critical rule change happened in 1979, and it just took decades for teams to embrace an adaptation which allowed that change to transform the league.
3. Adding to the analytic challenge here is that the fact that the new paradigm shift doesn't effect all players in the same way. Shouldn't be unexpected that this is so - different players have different talents and skills - but it leads to this fundamental issue:
Player A had a better career than Player B based on standards for achievement, but Player B likely would have had the superior career if coaches at the time had properly internalized the paradigm shifts we now see as clear cut, even without any further rule changes.
4. When we go to do all-time player comparisons, we can acknowledge various criteria that will lead to different ranked lists...but when doing something like the Top 100, we are choosing to put forward one particular criteria above all others, and this raises the question of which criteria we should use.
5. The most straight forward approach, I'd argue, is one that looks to completely ignore how "good" any given league is, and focus only on dominance within one's own era. But while a particular user is free to use this approach, that's not what the norm has ever been since I first participated in 2006.
6. Rather, there's always some form of era-adjustment going on which leads to - among other things - the ability for the group to leave George Mikan out of the top tier of candidates.
7. While this has always seemed pretty reasonable to me, the question of how individual posters are doing these sort of collaborations hasn't been front and center. Sometimes we describe the process as simply looking to somehow estimate how good the competition of any season's league was, and apply that as a degree of difficulty bonus/penalty relative to a guy in another era.
8. What I've always tried to do though is to try to do an analysis that actually looks at the basketball strengths and weaknesses of each player relative to each other, and imagine what they'd do in the other player's era. There are a lot of little issues to think through when you do this, and you're never going to do it perfectly, but at the very least, you're thinking about basketball details when you're making you choice between basketball players.
9. But when we experience a strategic paradigm shift that makes the player who was worse into the player who would have been better, now we end up with a problematic loop:
Historical A had a better career than Historical B
Modern C is better than Historical A at the modern game.
Historical B is better than Modern C at the modern game.
B > C > A > B
How do you decide who should be ranked first?
10. I think most have concluded already that there's no way to do it, and so you have to focus on a broader degree-of-difficulty mixed with in-era dominance, and I'm thinking I'm going to have to go in this direction myself despite my misgivings, and if I do, it will have some pretty profound changes on my voting compared to what I've done in the past.
11. Case in point, I've had Garnett ahead of Duncan on my Top 100 list since the 2014 edition, but I think it likely I'll go back to having Duncan ahead of Garnett when we do this in the 2023 edition.
Would I draft Duncan over Garnett? No.
Am I more impressed with Duncan than Garnett? No.
Do I think Duncan was better at basketball than Garnett? No.
But...
Does Duncan rank higher than Garnett based on how I evaluate season by season achievement accumulated over the course of a career? Yes.
And to put this in something a bit more concrete, I'm now going to share some historical spreadsheets I've made.
Before I do, note that I'd love to talk more about the details in these spreadsheets, but I'm not interested in being on trial for "bias". I do the best I can as I go through these analyses, but make no claim about being definitively right. Further, while it would be great if I either a) had a perfect memory or b) took really detail notes, but I don't. So I can try to explain why I put things as I did, but I won't necessarily have a ready and detailed answer for any particular spreadsheet field.
First spreadsheet: A POY-shares style document based on my own opinions for POY, OPOY, DPOY & COY going all the way back to '43-44. (ftr, I've grayed out '43-44 & '44-45 because in retrospect I've concluded that '45-46 - the first season after World War II - was really the better line of demarcation.)
Doc's Ballon d'Orange
Second spreadsheet: With the idea of Player of the Decade, a common issue is that people insist on focusing on stuff with the same second-to-last digit ('80s, '90s, etc), when I think the more meaningful thing is to do a running POD analysis. Anyone who stands out as the best over any 10-year run is a POD in my book.
So in this spreadsheet you'll see PODs (as well as OPODs, DPODs & CODs) that go from '45-46 to '54-55 all the way to '12-13 to '21-22 for men's basketball, as well as sheets for women's basketball going back to the founding of the WNBA.
I know not everyone cares about women's basketball, but from my perspective it represents a nice parallel analysis for comparison.
Doc's Ballon d'Orange PODs
Along those same lines, the last spreadsheet I've done something similar for tennis. Note that my tennis analysis here is going solely by performance in major tournaments, and is intended to be entirely separate from my own personal opinions about specific tennis players. While I actually have more competitive background in tennis than basketball, there are reasons why historical analysis in tennis is more problematic than it is for basketball.
I'll note that I do separate analysis out here by playing surface.
I should also note that in the pre-Open era, I ignore the Grand Slams in men's tennis and focus on the pro majors, because that's where the best players were. By contrast in the women's game, the top players typically stayed as amateurs until the Open era.
Tennis GS Pods
Alright, I'll leave it at that and hope this leads to good discussion.
So I wanted to share with people where my head is at pertaining to Top 100 criteria, and hopefully get into fruitful conversation on the topic.
I'll first note that Tim made a thread in this vein already that was great, but this is something I'm just saying is a crisis for me, so I wanted to just start fresh speaking for myself.
Here's the essence of the "crisis" as I see it:
1. After decades and decades of relatively stable strategic equilibrium in the NBA, now we're in the midst of a massive shift which can be summed up fairly well by the change in ORtg.
When we look at bk-ref's year's ORtg's, we see a league that hovered within a 6 point range (102.5 to 108.5) from '78-79 to '15-16, and in the past few years we've now climbed to the point where we're approaching another 6 points beyond that.
2. If this were something caused by a 2016 rule change that simply made scoring easier, it would straight forward to at least diagnose the dilemma in an era-separated way, but the reality is that the critical rule change happened in 1979, and it just took decades for teams to embrace an adaptation which allowed that change to transform the league.
3. Adding to the analytic challenge here is that the fact that the new paradigm shift doesn't effect all players in the same way. Shouldn't be unexpected that this is so - different players have different talents and skills - but it leads to this fundamental issue:
Player A had a better career than Player B based on standards for achievement, but Player B likely would have had the superior career if coaches at the time had properly internalized the paradigm shifts we now see as clear cut, even without any further rule changes.
4. When we go to do all-time player comparisons, we can acknowledge various criteria that will lead to different ranked lists...but when doing something like the Top 100, we are choosing to put forward one particular criteria above all others, and this raises the question of which criteria we should use.
5. The most straight forward approach, I'd argue, is one that looks to completely ignore how "good" any given league is, and focus only on dominance within one's own era. But while a particular user is free to use this approach, that's not what the norm has ever been since I first participated in 2006.
6. Rather, there's always some form of era-adjustment going on which leads to - among other things - the ability for the group to leave George Mikan out of the top tier of candidates.
7. While this has always seemed pretty reasonable to me, the question of how individual posters are doing these sort of collaborations hasn't been front and center. Sometimes we describe the process as simply looking to somehow estimate how good the competition of any season's league was, and apply that as a degree of difficulty bonus/penalty relative to a guy in another era.
8. What I've always tried to do though is to try to do an analysis that actually looks at the basketball strengths and weaknesses of each player relative to each other, and imagine what they'd do in the other player's era. There are a lot of little issues to think through when you do this, and you're never going to do it perfectly, but at the very least, you're thinking about basketball details when you're making you choice between basketball players.
9. But when we experience a strategic paradigm shift that makes the player who was worse into the player who would have been better, now we end up with a problematic loop:
Historical A had a better career than Historical B
Modern C is better than Historical A at the modern game.
Historical B is better than Modern C at the modern game.
B > C > A > B
How do you decide who should be ranked first?
10. I think most have concluded already that there's no way to do it, and so you have to focus on a broader degree-of-difficulty mixed with in-era dominance, and I'm thinking I'm going to have to go in this direction myself despite my misgivings, and if I do, it will have some pretty profound changes on my voting compared to what I've done in the past.
11. Case in point, I've had Garnett ahead of Duncan on my Top 100 list since the 2014 edition, but I think it likely I'll go back to having Duncan ahead of Garnett when we do this in the 2023 edition.
Would I draft Duncan over Garnett? No.
Am I more impressed with Duncan than Garnett? No.
Do I think Duncan was better at basketball than Garnett? No.
But...
Does Duncan rank higher than Garnett based on how I evaluate season by season achievement accumulated over the course of a career? Yes.
And to put this in something a bit more concrete, I'm now going to share some historical spreadsheets I've made.
Before I do, note that I'd love to talk more about the details in these spreadsheets, but I'm not interested in being on trial for "bias". I do the best I can as I go through these analyses, but make no claim about being definitively right. Further, while it would be great if I either a) had a perfect memory or b) took really detail notes, but I don't. So I can try to explain why I put things as I did, but I won't necessarily have a ready and detailed answer for any particular spreadsheet field.
First spreadsheet: A POY-shares style document based on my own opinions for POY, OPOY, DPOY & COY going all the way back to '43-44. (ftr, I've grayed out '43-44 & '44-45 because in retrospect I've concluded that '45-46 - the first season after World War II - was really the better line of demarcation.)
Doc's Ballon d'Orange
Second spreadsheet: With the idea of Player of the Decade, a common issue is that people insist on focusing on stuff with the same second-to-last digit ('80s, '90s, etc), when I think the more meaningful thing is to do a running POD analysis. Anyone who stands out as the best over any 10-year run is a POD in my book.
So in this spreadsheet you'll see PODs (as well as OPODs, DPODs & CODs) that go from '45-46 to '54-55 all the way to '12-13 to '21-22 for men's basketball, as well as sheets for women's basketball going back to the founding of the WNBA.
I know not everyone cares about women's basketball, but from my perspective it represents a nice parallel analysis for comparison.
Doc's Ballon d'Orange PODs
Along those same lines, the last spreadsheet I've done something similar for tennis. Note that my tennis analysis here is going solely by performance in major tournaments, and is intended to be entirely separate from my own personal opinions about specific tennis players. While I actually have more competitive background in tennis than basketball, there are reasons why historical analysis in tennis is more problematic than it is for basketball.
I'll note that I do separate analysis out here by playing surface.
I should also note that in the pre-Open era, I ignore the Grand Slams in men's tennis and focus on the pro majors, because that's where the best players were. By contrast in the women's game, the top players typically stayed as amateurs until the Open era.
Tennis GS Pods
Alright, I'll leave it at that and hope this leads to good discussion.