Evidence For Portability

Moderators: penbeast0, PaulieWal, Clyde Frazier, Doctor MJ, trex_8063

DraymondGold
Senior
Posts: 590
And1: 763
Joined: May 19, 2022

Evidence For Portability 

Post#1 » by DraymondGold » Tue Sep 3, 2024 2:15 am

Introduction
What is portability? Portability (sometimes called scalability) can have a variety of meanings. In the traditional meaning, popularized by Thinking Basketball, portability is how well “players carry more value on better and better teams.” (source: https://thinkingbasketball.net/2018/02/12/backpicks-goat-philosophy-of-player-ranking/ ).

A player’s value pretty irrefutably faces diminishing returns as their teammates get better and better. Different skills face different amounts of diminishing returns. Some skillsets are better at “floor raising,” taking a bad team to an average team. Other skillsets are better at “ceiling raising,” taking an average team to a championship-level team. Good defense is usually maximally portable (“most clubs need defenders”, per the article above), while offense can have a greater range of portability (“there’s only one ball”, as the adage goes). Maintaining more value on better teams is important, as generally only the most dominant teams in a season are capable of winning championships. Thus, if you’re interested in how much a player improves their teams’ championship odds, incorporating portability as one facet of your evaluation can produce more accurate results.

Of course, as proponents of portability readily say, it is far from the only factor. The overall value of a player is likely more important than that player’s portability. And the importance of a player’s portability might change depending on how good they are. In the traditional portability model, the importance of portability increases as the player gets better. This is backed up by team results throughout history: a team’s chances of winning a championship increases more as you go from 55 to 65 wins vs as you go from 45 to 55 wins, so facing fewer diminishing returns on those dominant dynasties can really improve your championship odds.

Still, there are opponents of portability who argue against certain facets of the model. Some refute how portability changes with player value, saying portability is more important for role players than star players. Others argue it’s not a significant variable (e.g. your evaluation would be no worse if you rated everyone as having +0 portability), and a minority of people even argue negative portability skills are better.

In general, I think there’s room for more detailed study of portability, the evidence for it, and how important it is in for good players and good team results. Here, I’ll provide evidence for the existence of portability and note a few interesting trends across several studies: A) trends in overall team results, B) trends in offensive team results, C) trends in player impact data, D) trends in superstar on-court offensive team results, E) film evidence, and F) expert opinions.

A. Trends in Overall team results
Good portability is supposed to be a measure of strong ceiling raising. Is this true? Let’s look at the Top 100 teams according to two overall team performance stats: ELO (which measures team record, adjusted for opponent, in the regular season + playoffs) and Sansterre’s Overall SRS (which measures team margin of victory, adjusted for opponent, in the regular season + playoffs). Then we can compare these teams’ impact metrics with the offensive portability of their best player to see if there’s a trend.

Why start with overall team performance and not offense-only results? We’ll do both. But among the top offensive players and top overall players, it’s not uncommon to have an offense-focused roster around a low-portability player or a defense-focused roster around a high-portability player. This could bias the signal, so we’ll look at overall team results first.

To be clear, I’m using a very crude model: it doesn’t include any other information besides the best player’s offensive portability score. I expected the trend to be insignificant. It turns out, there is a signal that portability improves ceiling raising.

1. ELO
-Data: Top 100 Teams according to ELO (1955–2023)
-Method: Linear Fit of portability score of top player vs Composite ELO score. Note: for this study I’m using Thinking Basketball’s older portability scores from a year or two ago.
-Slope: +6.82 [ELO / port].
The mean difference in ELO between neighboring ranks is 1.56, while the median is 1, so an improvement of +1 portability in your best player is estimated to improve a Top 100 team’s ranking by 4–7 slots.
Replacing a -2 portability with a +2 portability best player is estimated to improve a Top 100 team’s ranking by 17–28 slots.
-p-value: 0.02 (good).
This is statistically significant! This is a real feature in the team results.
-R^2: 0.05 (bad).
This trend is not sufficient to explain all the variation in the team results. Which was self-evident, even before calculating. It seems the portability of a team’s best player is a statistically significant factor, but it is obviously far from the only factor.
-Change in slope across the data: +7.16 (top 50 teams), +0.09 (bottom 50 teams).
So the portability of your best player becomes more important on better teams. This is exactly what Thinking Basketball predicted: the most dominant teams are subject to the most diminishing returns, and so gain the most benefits if their best player (who has the most value on the team) has better portability (and so faces less diminishing returns).

2. Sansterre’s Overall SRS
-Data: Top 100 Teams according to OSRS (1955–2023)
-Method: Linear Fit of portability score of top player vs the OSRS ranking value (a combination of OSRS, OSRS standard deviation, and playoff series won, which Sansterre used to rank teams in his list). Note: for this study I’m using Thinking Basketball’s older portability scores from a year or two ago.
-Slope: +0.75 [OSRS ranking score / port].
The mean difference between neighboring ranks is 0.22, while the median is 0.09, so an improvement of +1 portability in your best player is estimated to improve a Top 100 team’s ranking by 3–8 slots.
Replacing a -2 portability with a +2 portability best player is estimated to improve a Top 100 team’s ranking by 14–33 slots.
-p-value: 0.04 (good).
This is statistically significant! This is a real feature in the team results.
-R^2: 0.04 (bad).
This trend is not sufficient to explain all the variation in the team results. Which was self-evident, even before calculating. It seems the portability of a team’s best player is a statistically significant factor, but it is obviously far from the only factor.
-Change in slope across the data: +1.02 (top 50 teams), +0.18 (bottom 50 teams).
So the portability of your best player becomes more important on better teams. This is exactly what Thinking Basketball predicted: the most dominant teams are subject to the most diminishing returns, and so gain the most benefits if their best player (who has the most value on the team) has better portability (and so faces less diminishing returns).

So in both cases, teams with a more portable best player were likely to produce more dominant results. An improvement of +1 in your best player’s portability score correlates with an improvement of +3–8 ranks in the Top 100 lists, and the importance of portability seems to increase as you get to better teams. The trend is statistically significant: better portability does produce better ceiling raising.

B. Trends in Offensive team results
In Part A, we looked at trends in the best overall players’ offensive portability scores compared to the overall team results, among the best overall teams. Here, we look at trends in the best offensive players’ offensive portability scores compared to the offensive team results, among the best offensive teams. 

Teams: As before, we need a large enough sample that we’re not dominated by small-sample noise, so let’s look at the Top 50 offensive teams ever. We’ll use relative offensive rating, as the best teams ever by raw offensive rating will be primarily modern teams, which would make samples across the variety of top teams/styles/players smaller and thus noisier. As an aside, the recent boost in raw offensive ratings have been improved first and foremost by the improvement in three point shooting, which happens to be one of the most portable skills.

Players: We’ll use Thinking Basketball’s portability scores. To find each teams’ best offensive player, we’ll use Thinking Basketball’s offensive evaluations. In most cases, the most valuable offensive player on each team is clear. Generally, quibbling over if Player X is actually better offensively than Player Y in the small sample of teams where it’s ambiguous won’t change the overall results.

Prediction: The average portability score among players is 0, and we might expect it to be roughly normally distributed (so most players have +0 port, fewer players have +/- 1 port, fewer still have +/- 2 port). If portability exists, we would expect the top offensive teams to skew slightly towards positive portability when looking at the top offensive teams ever.

Among the Top 50 Teams according to RS rORTG:
-Number of best offensive players with +2 Port: 7
-Number of best offensive players with +1 Port: 8
-Number of best offensive players with +0 Port: 21
-Number of best offensive players with -1 Port: 10
-Number of best offensive players with -2 Port: 0
-(Number of best offensive players without Thinking Basketball Port scores yet: 4)
-Average port score: +0.3.
-Positive vs negative scores: 13 have positive port (7 have +2), 10 have negative port (0 have -2).

We can do a statistical test called a (one-sample) t-test to determine if the mean portability in this sample is significantly different (in the statistical sense) from the expected mean of 0. If the (one-tail) p-value from this t-test is less than 0.05, we can conclude there is a real shift towards positive portability, and that this is not just noise.
-p-value: 0.038 (statistically significant!)

Among the Top 50 Teams according to PS rORTG (minimum 10 games played):
-Number of best offensive players with +2 Port: 7
-Number of best offensive players with +1 Port: 12
-Number of best offensive players with +0 Port: 14
-Number of best offensive players with -1 Port: 10
-Number of best offensive players with -2 Port: 0
-(Number of best offensive players without Thinking Basketball Port scores yet: 7)
-Average port score: +0.4.
-Positive vs negative scores: 19 have positive port (7 have +2), 10 have negative port (0 have -2).
-p-value: 0.011 (statistically significant!)

So among the top offensive team results ever, as predicted, we find the most common value is neutral +0 portability, with the distribution skewing more towards positive portability than negative portability. In both the RS and the PS, the average portability is slightly positive, and there are more positive portability best offensive players than negative ones. Better portability in your best player seems to produce better team offenses. This trend is statistically significant. This supports the idea of portability.

The general qualitative takeaways don’t change if you change the sample (e.g. shrink the sample to the Top 25, or change the playoff minimum games played filter), and there are similar trends if you do a similar study across team playstyle (e.g. movement-based offenses vs two-man offenses vs heliocentric offenses, etc.), although the exact numbers would obviously change.

C. Trends in Player Impact Metrics
Is there a signal in the impact data of individual stars? If so, including portability in an evaluation of a player should produce a better fit to available impact metrics than evaluations that ignore portability.

Since we’re using Thinking Basketball’s concept of portability, let’s use their evaluations to filter players. Let’s take every season that’s worth 4+ in Thinking Basketball’s evaluation since 1997. There are 28 players and 164 seasons total.

To test if portability is a better model, let’s compare Thinking Basketball’s valuation and CORP without including portability vs their CORP including portability. The best model should produce a better fit with impact metrics, such as plus minus, on-off, and Augmented Plus Minus. As a reminder, lower p-value is better, and higher R^2 is better.

Fitting with Plus Minus (on):
-Raw valuation (no portability): 0.0004 p-value, 0.07 R^2
-CORP (no portability): 0.0006 p-value, 0.07 R^2
-CORP (with portability): 0.0003 p-value, 0.08 R^2

Fitting with On-off
-Raw valuation (no portability): 0.0006 p-value, 0.07 R^2
-CORP (no portability): 0.0006 p-value, 0.07 R^2
-CORP (with portability): 9e-5 p-value, 0.09 R^2

Fitting with Augmented Plus Minus (AuPM)
-Raw valuation (no portability): 6e-13 p-value, 0.28 R^2
-CORP (ignoring portability): 5e-13 p-value, 0.28 R^2
-CORP (with portability): 3e-15 p-value, 0.32 R^2

So in all three cases, including portability produced a clear improvement in how well the player evaluation fits the impact data. There’s more rigorous ways to compare models if people would like. And there’s other possible explanations (maybe higher portability players do better than lower ones for non-portability reasons). But as a first-pass, it seems if two players are otherwise equal in value, the one with higher portability tends to have better impact metrics than the one with lower portability.

D. Survey of Offensive team results when the Superstar is on-court

Overall team offensive data can be biased by bench offenses, so it’s worth looking at the offensive team data just when star players are on the court in the playoffs. It would be best to look at rORTG; but since it’s quite painstaking to calculate the proper rORTG when a star’s ON in the playoffs with differing defenses (and could be biased by e.g. starter vs bench defenses), let’s use a first-pass proxy: raw ORTG when the star’s ON. It’s imperfect, but if you remember the uncertainty bars, it’s still informative.

Since era differences will make a major difference in raw offensive rating, let’s look just at the top 8 offensive players of the pace and space era: Curry, Jokic, LeBron, Durant, Chris Paul, Harden, Luka, Kawhi. (Purely to satisfy my own curiosity, I’ll include Nash, but feel free to ignore him if you’d prefer). For context, here are the top players’ portability scores: Curry +2, Jokic +2, Durant +1, Kawhi (+1 Spurs, +0 post-Spurs), LeBron (-1 pre-2017, +0 post-2017), Harden -1, Chris Paul -1, Luka -1, Nash -1.

-Best 3-year PS ORTG when ON: Durant +120.4, Curry +120.3 > Jokic +118.5 > LeBron +118, Kawhi +117.8 > Luka +117.1 > Harden +116.1, Nash +116.1 > Chris Paul +111.6
-Best-fit linear Slope: 1.54 (better portability produces better results). p-value: 0.047 (it’s statistically significant). R^2 value: 0.51 (portability does a good job explaining the variance in the data)
(note: these fits are done removing Nash to provide a more fair same-era comparison.)

-Best 5-year PS ORTG when ON: Curry +119.4, Durant +119.3 > LeBron +117.7 > Jokic +117.3, Kawhi +117.3 > Nash +116.3 > Luka +115.8 (4-year avg) > Harden +114.7 > Chris Paul +111.4
-Best-fit linear Slope: 1.50 (better portability produces better results). p-value: 0.038 (it’s statistically significant). R^2 value: 0.54 (portability does a good job explaining the variance in the data).
(note: these fits are done removing Nash to provide a more fair same-era comparison.)

This peak team rating almost exactly follows the order predicted by portability, which itself is just assigned based on playstyle. The positive portability players are very clearly on top, the neutral portability players fall in the middle, and the negative portability players clearly lag behind.

There are similar trends if you look at other peak time ranges, like 2 or 4 years. Adding regular season data does not significantly help the negative-port players. Again, this is a crude metric. A larger sample and more context (e.g. checking relative on-court ratings) would be necessary for this to be more conclusive. But in a survey of the best recent offensive stars, the more portable stars seemed to have better on-court offensive team results. This is supportive of the existence of portability, especially when taken in conjunction with the rest of the evidence in this post.

E. Film Evidence (and various other studies)
The primary meat of this post was intended to be the new statistical analysis. However, it's worth noting that there’s also film evidence for portability. Skills like shooting (with its efficient finishing and spacing benefits), off-ball action (e.g. movement, screening, rebounding), passing, and efficient finishing tend to have high offensive portability, while traits like isolation scoring and ball-dominance tend to have low portability. Unfortunately there’s no one spot where Thinking Basketball goes through a comprehensive study of the portability of all different skillsets. However, there are a variety of places where he or others go in-depth on facets of portability. Here's a few of them.

An analysis of the value of off-ball motion:

A case study on how a variety of portable skills (motion, playmaking bigs, handoffs with screening, shooting, and pace) can produce one of the best raw Offensive Ratings ever:

The value of three point shooting should be pretty obvious to everyone on this board. It’s more efficient than long twos and so makes finishing with the help of a playmaker more valuable. The spacing makes it easier for players to drive or attack the rim, harder for the defenses to help certain actions, and more punishing when they do help with wider passing lanes and a longer distance to recover. Still, for completeness, it’s worth including some content on the three point revolution.
Spoiler:
Discussion of the Three Point Revolution:

The history of spacing:
https://thinkingbasketball.net/2017/11/02/the-history-of-nba-spacing/

A review of how recent offenses use motion (off-ball cutting) to strain defenses


F. Reported Evidence
There are numerous quotes from players, coaches, and analysts in support of portability or related ideas. Quotes are far from conclusive. Indeed, just because a player is good on-court does not mean their on-court skills translate to being good off-court analysts. Even players with good on-court basketball IQ can be faulty off-court analysts, or be biased in their evaluations by personal feelings, relations, or loyalties. Nonetheless, since it’s fun, I’ll include a survey of quotes in support of portability and related ideas.

Wilt on Bill Russell’s fit:
“I picked him [Bill Russell] as the number one center of all time, because he was a complete basketball player…. I would pick him over me, because he also helped his team to win maybe a lot more than I could help my team to win. Sometimes the mere power of you makes you more individualistic. I have said this before: Wilt Chamberlain on the Boston Celtics might not be the same. Because I would take away from Bob Cousy, from Tom Heinsohn, because I was a scorer too. And then all of a sudden they would have to pass the ball to me, and that would take the ball away from them. So sometimes less is better…. The man could score. He averaged 17 ppg, which is only about 7 points less than Kareem Abdul-Jabbar. He was the supreme rebounder. He got the ball for his team, which made it happen. He’s high in assists, and we all know he played defense. You can’t ask for any more from his position.”
So the ability to fit often requires not needing the ball in your hands so you don’t take the ball away from your teammates, being versatile with good passing and rebounding, and also defense. Defense faces minimum diminishing returns, and so always receives a maximum portability rating from Thinking Basketball, but too much individualism on offense can lead to diminishing returns and limit how much you win.

Moses on rebounding:
“You keep moving, and it’s going to pay off. He’s going to be so tired fighting you, he won’t have anything left. Things change in that money (fourth) quarter. A man figures he has boxed you out for three quarters, he’s confident. He’s dead-tired, so he lets down a little. Now he kinda forgets about me, but I’m still coming. The only time I have trouble getting a rebound is if they put two or three guys on me. If it’s only one guy, I can get the rebounds.”
So being more active off-ball on the board can tire opponents out.

Cooper on Bird:
“People always ask me who’s the hardest player I ever had to guard…. I always say Lard Bird…. With Larry, when he passed the basketball is when he became more dangerous. He was either setting a pick, coming off a pick, catching the ball, passing the ball, so he was the one you always had to stay attentive to the whole 24 seconds of that offensive play.”
So off-ball action requires greater endurance, defensive attentiveness, defensive IQ, and gives an offensive greater versatility, which can be harder to guard.

Phil Jackson on Triangle and Jordan:
“Basically I was planning to ask Michael, who had won his third scoring title in a row the previous season, to reduce the number of shots he took so that other members of the team could get more involved in the offense…. I told him that I was planning to implement the triangle... ‘You’ve got to share the spotlight with your teammates,’ I said, ‘because if you don’t, they won’t grow.’…. Sometimes I would tell him that he needed to be aggressive and set the tone for the team. Other times I’d say, 'Why don’t you try to get Scottie going so that the defenders will go after him and then you can attack?’ “
So sharing the ball, reducing too much volume scoring, and running off-ball actions helps teammates grow and stresses the defense at multiple spots on the floor, ultimately leading to winning basketball.

LeBron and Redick on 2011 Miami:
LeBron: “Obviously my first year there, played great basketball, got all the way to the finals, lose in the finals. I play like ****…. When [Spo] came back to us, he knew that in order for us to reach our potential, one I had to be 10x better than I was in that previous June finals, but [two] Chris Bosh had to go to the five…. And we had to spread. He had to start working on his corner 3 faithfully….”
“The Bosh spacing, what did that sort of unlock?” -Redick
“The cutting. Slot cuts…. It unlocked exactly what myself and D Wade thrive on….
It changed the whole team. Yeah we added Ray. Added Shane. Added Mike Miller. We added the Spacing….” -LeBron
“…. Basketball is a very organic thing. And the players and their skills have to complement each other. And Chris Bosh is a great example of that. The sacrifice to figure out, how can my skills (and maybe I have to develop some of those; you mentioned the three point shooting), how can I figure out how to complement. It’s going to make me better, it’s going to make LeBron better, it’s going to make D Wade better, and it’s going to make our team better. And that’s basketball.” - Redick
“And that’s basketball. But that also comes from (to go back to episode one) basketball IQ. Him having the basketball IQ and the knowledge of saying ‘yeah, I could still be in Toronto averaging 25 and 12, but I didn’t come here for that ****. I came here to win championships. And we **** lost in year 1. What can I do to complement my teammates? What can I do to broaden my game out to where we don’t lose in year two?’ “ - LeBron
So 3 point shooting, having floor spacers, and broadening your game to focus on skills that complement with your teammates, rather than focusing on putting up big counting stats, is essential to winning championships.

Greg Popovich on motion offenses:
“We’re always trying to move the ball from good to great (shots). Penetrate for a teammate, not necessarily for yourself.”
So ball movement and unselfish player movement can improve shot quality.

Alex Caruso on Curry:
“That dude’s a menace. He’s just, he’s impossible. It’s especially the way they run their offense and the way that him and Draymond have this telepathy. Like the guys that are back this year… they’re actually understanding how to play with him now, how he operates, how he moves, when he gives up the ball and continues to run in circles until you fall asleep and even then shoots it from 30 feet. It’s borderline— at times it’s unguardable.”
So when a star has good off-ball motion, good shooting, and good chemistry with smart teammates, all in a motion system, it becomes borderline unguardable.

Curry on Jokic:
“He puts pressure on you all game and you got to respect, not only his ability to post up and put pressure on the paint. He knocked down 3 threes tonight. He is obviously an amazing playmaker so when they’re doing all that motion around and him, everybody has to be on alert to take away passing angles and lanes because he sees the court so well. And obviously even just offensive rebounding. There’s tip ins… you play good defense and he still finds his way right to the basket…. So he’s got kind of the full package, and guys are hitting shots around him and it makes it super tough to figure out how you want to defend him.”
So when a player has portable skills like 3 point shooting, playmaking, offensive rebounding, combined with motion and shooting around them, it makes it harder to defend.

Conclusion
There are definitely limitations to a study like this. Portability is qualitative concept. There’s general principles and common ideas, but there isn’t a single universally agreed definition. Reasonable people can disagree on the exact weights for how portable different skills are (see e.g. Thinking Basketball vs CraftedNBA’s portability scores). Even if you agree on the weightings, you could still disagree in your evaluation of a specific player’s portability in a specific season (e.g. Thinking Basketball has even updated their portability ratings for certain players).

Still, a survey of data across overall team results, individual impact metrics, offensive team results, and on-court team performance finds that portability (such as the model proposed by Thinking Basketball) exists and is statistically significant. And the portability of a team’s best player seems to become more important as the overall team quality. We can back this finding up, provide deeper explanations, and explore the necessary context using additional film analysis and player discussions.

There’s also room to disagree on exactly how important portability is (although the statistical studies above give us an approximate sense for a good team’s best player). Most everyone would agree it’s far from the only factor, or the most important factor. It’s far from deterministic: portability measures skills that tend to fit better with better teammates, but fit is complex and there can definitely still be ways to maximize fit on great teams with neutral or negative portability best players. If your best player has neutral or negative offensive portability, you might still produce dominant team results and minimize diminishing returns if you e.g. (1) focus on adding defensive costars, (2) be especially cognizant of offensive fit when upgrading your offensive talent, or (3) de-synchronize your stars’ minutes so both can play more of a floor-raising role.

Nevertheless, it seems portability is a real, statistically significant feature in team results, impact metrics, and film, and dominant ceiling-raising team performances benefit from having a best player with better portability.
DraymondGold
Senior
Posts: 590
And1: 763
Joined: May 19, 2022

Re: Evidence For Portability 

Post#2 » by DraymondGold » Tue Sep 3, 2024 2:21 am

Djoker wrote:.
You asked for me to post the portability evidence I had, so here it all is! :D

As an aside for others, I know portability is a bit of a controversial topic to certain people, and debates in the past have gotten pretty heated. If you disagree with people in this thread, please be mindful and try to maintain a minimum level of maturity and civility when discussing. The thread's much more likely to be productive that way!
penbeast0
Senior Mod - NBA Player Comparisons
Senior Mod - NBA Player Comparisons
Posts: 30,163
And1: 9,777
Joined: Aug 14, 2004
Location: South Florida
 

Re: Evidence For Portability 

Post#3 » by penbeast0 » Tue Sep 3, 2024 2:47 am

A beautiful read; though I haven't read the Thinking Basketball portability studies so I'm sure I'm missing some things. Still, very clear simple writeup, thank you.
“Most people use statistics like a drunk man uses a lamppost; more for support than illumination,” Andrew Lang.
Djoker
Starter
Posts: 2,113
And1: 1,817
Joined: Sep 12, 2015
 

Re: Evidence For Portability 

Post#4 » by Djoker » Tue Sep 3, 2024 2:58 am

Epic post.

I especially like the player impact metrics and how the correlation (R squared value) with AuPM improves when portability is included. Correlation is still a bit low though which might be because AuPM isn't a perfect metric. That's kind of the issue we have in evaluating this sport too. The metrics are imperfect but every one of them gives us a signal, a data point.
JimmyFromNz
Rookie
Posts: 1,075
And1: 1,227
Joined: Jul 11, 2006
 

Re: Evidence For Portability 

Post#5 » by JimmyFromNz » Tue Sep 3, 2024 3:25 am

Impressive effort put into your background research.

It's a frustratingly impossible concept to quantify into a plausible metric - and I think we need to be comfortable with that fact. Particularly in an era where sports analytics is heavily conditioned by the drive for 'a single source of truth' that can overlook the necessary (and often only) discussion of what insight can an isolated, incomplete 'figure' provide us as part of the overall picture.

Why mention this? I liked that you referenced a multitude of measures, drew some correlations but matched/tested this with a qualitative approach for a highly qualitative part of basketball discussion. I agree with your general conclusions.

That's a mix of analytical curiosity and honesty that is becoming rarer in discussions. Well done.
User avatar
Bad Gatorade
Senior
Posts: 715
And1: 1,870
Joined: Aug 23, 2016
Location: Australia
   

Re: Evidence For Portability 

Post#6 » by Bad Gatorade » Tue Sep 3, 2024 3:22 pm

Ok, this is a really good effort, and I applaud the time that went into this, but I'm going to have to push back a bit.

I'll stick with Ben's rough definition of portability, which is "degree to which a player can maintain that impact with their playstyle on a good team." I don't fully agree with this definition to begin with, but either way.

A few notes on some of the R^2 results - some of the R^2 results are honestly tiny and it feels like they're an insufficient proof for "portability." I'll give an example:

In section 3,

Fitting with Plus Minus (on):
-Raw valuation (no portability): 0.0004 p-value, 0.07 R^2
-CORP (no portability): 0.0006 p-value, 0.07 R^2
-CORP (with portability): 0.0003 p-value, 0.08 R^2


So in all three cases, including portability produced a clear improvement in how well the player evaluation fits the impact data.


Going from an R^2 of 0.07 to 0.08 on a variable that, in itself, is already somewhat arbitrarily defined, is hardly a boon, or even noteworthy. Something like that may simply be a result of error, or bias, or whatever reason, and it could be enough to sway things in the opposite direction if evaluated separately. Rating early career Durant as +2, and Dirk as +0 for his entire career is an example, because there's little evidence that early Durant plugs into a team more neatly than Dirk. He wasn't a great passer, wasn't a better shooter/scorer than Dirk back then (even if his PPG was higher, Dirk was just as good, if not better as a scorer IMO) and he hadn't yet had team success as a lead guy the way Dirk had. However, no, 2009-2012 Durant was +2 and Dirk was +0.

There are other examples - Jerry West, in a ball dominant season where he led in assists, would get +2 portability, which is treatment that the LeBrons, CP3s, Hardens etc don't seem to get. Stockton was a +1 by his portability metric, Nash was -2, but Stockton and Nash have frequently drawn parallels due to the similarity of their scoring + assist volumes and both of them being excellent shooters. They are literally almost polar opposites here, with the exception being that Stockton played more off ball, but Nash, who was already leading the best offences ever, literally has very, very clear proof of his impact being retained on good offensive teams. It... it's a bit inconsistent, that's all, and not something I'd place too much stock in.

I don't know if these are the most recent valuations, but this is what I found when searching.

Either way, even if his portability allotments are correct, a statistically significant sample also differs from a statistically meaningful sample. A dataset can have a p-value of < 0.05, and the difference can be real, but a p-value says nothing of the magnitude of the difference. Now, you have attempted to quantify this, so do I credit you there, although I still question the magnitude of some of these metrics.

Among the Top 50 Teams according to RS rORTG:
-Number of best offensive players with +2 Port: 7
-Number of best offensive players with +1 Port: 8
-Number of best offensive players with +0 Port: 21
-Number of best offensive players with -1 Port: 10
-Number of best offensive players with -2 Port: 0
-(Number of best offensive players without Thinking Basketball Port scores yet: 4)
-Average port score: +0.3.
-Positive vs negative scores: 13 have positive port (7 have +2), 10 have negative port (0 have -2).

We can do a statistical test called a (one-sample) t-test to determine if the mean portability in this sample is significantly different (in the statistical sense) from the expected mean of 0. If the (one-tail) p-value from this t-test is less than 0.05, we can conclude there is a real shift towards positive portability, and that this is not just noise.


Just as a note - the expected mean of 0 may not reflect the actual superstar mean. It's entirely possible that the sample of 'potential #1 players' may also have very few -2 valuations depending on what valuation scale Ben has used. I'd state that a one-sample t-test is not appropriate here, and neither is assuming that the mean is 0.

Some of the on-court rating data needs a bit of scrutiny too - A dataset of 8 players (Curry, Durant, LeBron, Jokic, Kawhi, Luka, Harden, Paul) is fairly small in itself, but it's worth mentioning that Curry/Durant played on the same team in what was one of the most controversial free agent signings ever and this was their most successful impact period. However, I'd like to go a step further -

If we look at relative team ORTG, for example, Curry's best 3 year stretch was from 2017-2019, in which the team rORTG was +10.3 (weighing it on a game by game basis). Paul, who was by far the lowest out of everybody there, had his 3 year peak from 2013-2015, where the team relative ORTG was... +10.3. Suddenly, the +8.7 point difference dissipates. LeBron's team from 2015-2017 had +11.2. Jokic's highest rORTG ever was in 2023, at... +7.0. And just to top it off, since we have Nash listed there, even though he wasn't included in the regression, we've got a 3 year sample of -2 portability drain Steve Nash at... +12.0, top of the sample.

I don't disagree with the value of "portability" underneath a different definition/usage, but I strongly disagree with how it's applied. Let's have two hypothetical players -

Player A is +7 on ball and +7 off ball
Player B is +5 on ball and +7 off ball

(the number values themselves can be scaled up/down; that's not really salient to the discussion)

The way Ben considers portability, he would consider players A and B equals, because their "optimal value" would be playing off ball at +7. However, that completely ignores that playing on ball at +7 is also incredibly useful, and there have been numerous teams that have won titles (Duncan's Spurs, Hakeem's Rockets etc) simply because their offence was good enough within their team structure, even if they weren't offensively at the level of LeBron/Magic/MJ etc. However, that sort of thing is dismissed completely, and I feel like that's wrong.

A better measure of actual portability would be to consider how the player performs in different , yet reasonable circumstances. Something such as a logit model (which would give credit to Player A above, as you have the choice to use him on/off ball at equal efficacy) based on different usages/team structures would actually be a measure of portability.
I use a lot of parentheses when I post (it's a bad habit)
User avatar
eminence
RealGM
Posts: 16,848
And1: 11,685
Joined: Mar 07, 2015

Re: Evidence For Portability 

Post#7 » by eminence » Tue Sep 3, 2024 3:39 pm

I think the evidence here supports a takeaway that an eye test (or at least a half decent one like Bens) can add value to most metrics. It's essentially a little categorical variable added to adjust individual metrics when one (Ben in this case) doesn't feel the metrics quite represent their true opinion.

I found a lot of value in this approach when doing team level predictions.

I find Ben's descriptions of his eye-test adjustment (which he calls a portability grade) strange at times and a bit inconsistent, but I imagine most would find my own the same, it's hard to put words to some patterns we see (and of course we're all just wrong sometimes as well).
I bought a boat.
OhayoKD
Head Coach
Posts: 6,025
And1: 3,914
Joined: Jun 22, 2022
 

Re: Evidence For Portability 

Post#8 » by OhayoKD » Wed Sep 4, 2024 5:15 am

eminence wrote:I think the evidence here supports a takeaway that an eye test (or at least a half decent one like Bens) can add value to most metrics. It's essentially a little categorical variable added to adjust individual metrics when one (Ben in this case) doesn't feel the metrics quite represent their true opinion.

It can if you provide a basis for the adjustments your eyetest is contributing...otherwise it just distorts it.

Ben has never to my knowledge actually justified what skills he rates as the most portable empirically so he probably shouldn't be letting port play any role in his metrics. Especially when he's only looking at offense for it.
its my last message in this thread, but I just admit, that all the people, casual and analytical minds, more or less have consencus who has the weight of a rubberized duck. And its not JaivLLLL
OhayoKD
Head Coach
Posts: 6,025
And1: 3,914
Joined: Jun 22, 2022
 

Re: Evidence For Portability 

Post#9 » by OhayoKD » Wed Sep 4, 2024 5:22 am

Bad Gatorade wrote:Ok, this is a really good effort, and I applaud the time that went into this, but I'm going to have to push back a bit.

I'll stick with Ben's rough definition of portability, which is "degree to which a player can maintain that impact with their playstyle on a good team." I don't fully agree with this definition to begin with, but either way.

The correct term for that definition is "scalability". There is no reason for Ben to use "portability" there when it is commonly understood as something different.
its my last message in this thread, but I just admit, that all the people, casual and analytical minds, more or less have consencus who has the weight of a rubberized duck. And its not JaivLLLL
User avatar
eminence
RealGM
Posts: 16,848
And1: 11,685
Joined: Mar 07, 2015

Re: Evidence For Portability 

Post#10 » by eminence » Wed Sep 4, 2024 11:42 am

OhayoKD wrote:
eminence wrote:I think the evidence here supports a takeaway that an eye test (or at least a half decent one like Bens) can add value to most metrics. It's essentially a little categorical variable added to adjust individual metrics when one (Ben in this case) doesn't feel the metrics quite represent their true opinion.

It can if you provide a basis for the adjustments your eyetest is contributing...otherwise it just distorts it.

Ben has never to my knowledge actually justified what skills he rates as the most portable empirically so he probably shouldn't be letting port play any role in his metrics. Especially when he's only looking at offense for it.


Depends on the purpose for your metric. If it's for distribution like Bens are then I agree. If it's in house for increasing your own descriptive/predictive power then it works pretty well for that purpose without needing a strong justification past 'it works'.
I bought a boat.
penbeast0
Senior Mod - NBA Player Comparisons
Senior Mod - NBA Player Comparisons
Posts: 30,163
And1: 9,777
Joined: Aug 14, 2004
Location: South Florida
 

Re: Evidence For Portability 

Post#11 » by penbeast0 » Wed Sep 4, 2024 11:53 am

Bad Gatorade wrote:..


Thank you. Your posts are always well reasoned and worth reading.
“Most people use statistics like a drunk man uses a lamppost; more for support than illumination,” Andrew Lang.
OhayoKD
Head Coach
Posts: 6,025
And1: 3,914
Joined: Jun 22, 2022
 

Re: Evidence For Portability 

Post#12 » by OhayoKD » Wed Sep 4, 2024 1:12 pm

eminence wrote:
OhayoKD wrote:
eminence wrote:I think the evidence here supports a takeaway that an eye test (or at least a half decent one like Bens) can add value to most metrics. It's essentially a little categorical variable added to adjust individual metrics when one (Ben in this case) doesn't feel the metrics quite represent their true opinion.

It can if you provide a basis for the adjustments your eyetest is contributing...otherwise it just distorts it.

Ben has never to my knowledge actually justified what skills he rates as the most portable empirically so he probably shouldn't be letting port play any role in his metrics. Especially when he's only looking at offense for it.


Depends on the purpose for your metric. If it's for distribution like Bens are then I agree. If it's in house for increasing your own descriptive/predictive power then it works pretty well for that purpose without needing a strong justification past 'it works'.

In-house is fine, you will reap what you sow.

But the public distribution of what are basically the same set of inputs with minor tweaks, justified by "it's correlative" reinforces unwarranted confidence in an approach that has not actually been asked to compete with distinct approaches.

I do wish we had alternative box-scores(counting things other than who was last or second to last the touch the ball before x happened) and from there alternative inputs into alternative all-in-ones as a control group/test of "can you get to correlativeness without making the same assumptions as convention tells you to make", but no one with the clout to meaningfully push for that is.
its my last message in this thread, but I just admit, that all the people, casual and analytical minds, more or less have consencus who has the weight of a rubberized duck. And its not JaivLLLL
Bidofo
Pro Prospect
Posts: 776
And1: 975
Joined: Sep 20, 2014
     

Re: Evidence For Portability 

Post#13 » by Bidofo » Wed Sep 4, 2024 1:40 pm

Bad Gatorade wrote:Some of the on-court rating data needs a bit of scrutiny too - A dataset of 8 players (Curry, Durant, LeBron, Jokic, Kawhi, Luka, Harden, Paul) is fairly small in itself, but it's worth mentioning that Curry/Durant played on the same team in what was one of the most controversial free agent signings ever and this was their most successful impact period. However, I'd like to go a step further -

If we look at relative team ORTG, for example, Curry's best 3 year stretch was from 2017-2019, in which the team rORTG was +10.3 (weighing it on a game by game basis). Paul, who was by far the lowest out of everybody there, had his 3 year peak from 2013-2015, where the team relative ORTG was... +10.3. Suddenly, the +8.7 point difference dissipates. LeBron's team from 2015-2017 had +11.2. Jokic's highest rORTG ever was in 2023, at... +7.0. And just to top it off, since we have Nash listed there, even though he wasn't included in the regression, we've got a 3 year sample of -2 portability drain Steve Nash at... +12.0, top of the sample.

The OP was a good and interesting read and this is a great response, but yeah by far the most objectionable thing about it was the lumping of 'pace and space' players as if the league hasn't drastically changed even in the last couple of years. Honestly it gets overlooked pretty often.

2023 LA ORTG (presumably part of the sample size for Luka, Jokic): 114.8
2017 LA ORTG (sample size for Curry, Durant, LeBron, prob more): 108.8
2015 LA ORTG (Paul, maybe Harden?): 105.6
2005 LA ORTG (Nash): 106.1

It's kind of jarring imo when you look at it. Personally 2017 is the start of 'pace and space' for me, there was an 11% increase in 3PAr which might be the largest in history and the start of teams really trying to maximize the power of the 3. But the league has improved by 6 pp100 (6.5 if you include the 2024 season) since then, more than double what the league had improved by in 2017 from 2005! The most recent year that 2017 ORTG is at least 6 points better than is 1999 (102.2) and the most recent non-lockout year is...1978 (100.9)! 2015 had the second lowest ORTG (2012) since the '04 rule changes! It's no wonder that the raw ORTG rankings turned out the way that they did. Now, perhaps these year-to-year comparisons should instead be done by % change. I think the results are much the same however.

I know there were disclaimers in this part of the OP about the data being incomplete and with flaws...but to me it is SO flawed that there's no point in using it at all, especially when it seems to paint the exact opposite picture of who's outperforming LA ORTG the most.
DraymondGold
Senior
Posts: 590
And1: 763
Joined: May 19, 2022

Re: Evidence For Portability 

Post#14 » by DraymondGold » Wed Sep 4, 2024 2:41 pm

Definitions:
Bad Gatorade wrote:Ok, this is a really good effort, and I applaud the time that went into this, but I'm going to have to push back a bit.

I'll stick with Ben's rough definition of portability, which is "degree to which a player can maintain that impact with their playstyle on a good team." I don't fully agree with this definition to begin with, but either way.
Appreciate the thoughtful reply!

Yeah there's a variety of concepts adjacent to portability/scalability, and some prefer to weigh others more heavily. For example:
-'Buildability' -- how easy is it to 'build' around a superstar? Does getting the most of their value require a more specific scheme / costars / supporting cast or can you get the most of their value with a variety of contexts? From a team building perspective, how probable is it that you could get the right team context (and that the team management would be smart enough to correctly identify the right team context)?

-'Portability' (from a team-switching / 'versatility' perspective) -- how easy is it for a superstar to switch teams or switch teammates/contexts and maintain most of their value? This is sometimes what people think of when they hear the term, which is why I prefer to use the term scalability when referring to Thinking Basketball's version. People will cite either the impact metrics or team results for superstars who switch teams, which gives an appealing possibility to start quantifying this concept. The limitations are there's tons of noise with these kinds of measurements, and just because one player happened to change teams doesn't mean they adjusted better to the new context than a player who happened to stay on one team. Of course, the other concepts here are also hard to measure!

-'Portability' (from the ceiling-rising / 'scalability' perspective) -- Ben's definition, defined above. I think he makes compelling arguments personally for why ceiling raising is important for wining championships, given the history of NBA champions and how good those teams tended to be in their league. Maybe by focusing on ceiling-raising/scalability, he misses something that's only captured in buildability / portability (the team-switching/versatility version) / etc. But given how similar and fuzzy the concepts are, I'd have trouble putting a finger on what trait/skill specifically he'd miss. Open to idea here!

Lower R^2:
Bad Gatorade wrote:A few notes on some of the R^2 results - some of the R^2 results are honestly tiny and it feels like they're an insufficient proof for "portability." I'll give an example:

In section 3,

Fitting with Plus Minus (on):
-Raw valuation (no portability): 0.0004 p-value, 0.07 R^2
-CORP (no portability): 0.0006 p-value, 0.07 R^2
-CORP (with portability): 0.0003 p-value, 0.08 R^2


So in all three cases, including portability produced a clear improvement in how well the player evaluation fits the impact data.


Going from an R^2 of 0.07 to 0.08 on a variable that, in itself, is already somewhat arbitrarily defined, is hardly a boon, or even noteworthy.
Agreed! The R^2 is definitely super low. But consider the stat we're using here: raw plus minus and on-off are super noisy. Ben's evaluations are intended to be as stable as the player's actual goodness, which we generally think doesn't change nearly as massively between seasons.

Consider (picking a random star here...) Harden:
2017: Healthy Corp: 12.2%. Plus minus: +7.0.
2018: Healthy Corp: 14.5%. Plus minus: +10.5.
2019: Healthy Corp: 14.5%. Plus minus: +6.6.
2020: Healthy Corp: 13.9%. Plus minus: +5.7.
2021: Healthy Corp: 13.9%. Plus minus: +3.1.

Harden's portability doesn't change here. Thinking Basketball's evaluation varies from +4.5 to +5.0 (11% change). His CORP changes by 19% (it’s exponential to weigh peak seasons more heavily). But Harden’s raw plus minus varies from +3.1 to +10.5, a whopping 239%!

R^2 measures how much variance in one variable (the plus minus) that is explained by the other (the player evaluation, including or not including portability). The variance is very clearly not explained -- but that's because of the noise of plus minus data! I'm sure no one really thinks healthy Harden's true value changed by 239% in those years. Something like an 11–19% change seems much more appropriate to how people usually consider his 'goodness' changed in those seasons.
In short: we should expect the R^2 of any ‘goodness’ fit to raw Plus Minus to be very low! Raw plus minus is noisy!

Same thing applies to on-off, and AuPM to a lesser extent (although that metric is more stable given the box inputs). And indeed you can see the R^2 improves pretty dramatically as you get to more stable metrics. I could do this with something maximally stable like a box metric, or with a hybrid metric like DARKO/EPM, but chose to keep things more basic and impact-only for the time being.

Some Case Studies and Disagreements on Specific Player Evals:
Bad Gatorade wrote:Something like that may simply be a result of error, or bias, or whatever reason, and it could be enough to sway things in the opposite direction if evaluated separately. Rating early career Durant as +2, and Dirk as +0 for his entire career is an example, because there's little evidence that early Durant plugs into a team more neatly than Dirk. He wasn't a great passer, wasn't a better shooter/scorer than Dirk back then (even if his PPG was higher, Dirk was just as good, if not better as a scorer IMO) and he hadn't yet had team success as a lead guy the way Dirk had. However, no, 2009-2012 Durant was +2 and Dirk was +0.

There are other examples - Jerry West, in a ball dominant season where he led in assists, would get +2 portability, which is treatment that the LeBrons, CP3s, Hardens etc don't seem to get. Stockton was a +1 by his portability metric, Nash was -2, but Stockton and Nash have frequently drawn parallels due to the similarity of their scoring + assist volumes and both of them being excellent shooters. They are literally almost polar opposites here, with the exception being that Stockton played more off ball, but Nash, who was already leading the best offences ever, literally has very, very clear proof of his impact being retained on good offensive teams. It... it's a bit inconsistent, that's all, and not something I'd place too much stock in.

I don't know if these are the most recent valuations, but this is what I found when searching.
Definitely not the most recent values!

As a reminder, portability is not the raw offensive goodness, it's how well does this player scale onto better teams (taken relative to how good they are). So saying something like 'Dirk's a better shooter/scorer' or 'he hadn't yet had team success as a lead guy in the way Dirk had' doesn't necessarily invalidate portability. It might just say that you think Dirk's starting spot is higher. E.g. you might think Dirk's +6 to Durant's +5 on an average team, while Dirk's +5.5 to Durant's +4.75 on a good team. In both cases, Dirk would be the better player, but Durant faces less diminishing returns (relative to his starting spot) on a good team. In this example, Durant would have the higher portability, but Dirk would be the better overall player.

To be clear, I'm not against raising Dirk's portability. I thought it was a bit low when I saw the original score. And apparently, so does current Ben!

Updated values:
Dirk +1
Durant +1
So they're given the same portability in the updated values I used here.

As for Jerry West, I would bet Ben's argument for his higher portability would be
a) would be that he was a more active off-ball player relative to era than those other players you mentioned based on his film study, and similarly b) he's less ball-dominant.
As an example for a, he played much more of a hybrid (on and off-ball) role throughout much of his prime. The change in playstyle occurred as his team changed when he got older, so that doesn't necessarily suggest he lost the ability to be off-ball. He's also clearly the better era-relative shooter than LeBron/CP3, which adds a lot of value.
As an example for b, West's load in that season you mention was 46%, while Chris Paul's was 53% in 09, LeBron's was 56% in 09, Harden's was 66% in 2019.

That's not to say you have to agree with this assessment, but I just want to be clear that it's not an inconsistency in Ben's analysis -- it might just be a difference in which traits he values compared to you. That said, his latest assessment has gone less extreme for West's portability, perhaps for some of the reasons you mention.

Updated value:
Jerry West +1

There's similar points to raise for the Stockton/Nash point. Portability isn't the same as raw ability, and Thinking Basketball assigns Nash a much higher mean +/- than Stockton. Indeed, he considers Nash a Tier 1 all-time offensive player, within uncertainty range of being the GOAT offensive player, while he doesn't have Stockton anywhere closer to even a Top 15 offensive peak.

In terms of portability, I think the argument might be something like Stockton showed greater chemistry / fewer diminishing returns (relative to his starting value, which like you say is lower!) with Malone than Nash did with Dirk, Shaq, or Kobe/Dwight. Obviously Nash still led great offensive results with Dirk and even more so with the SSOL Suns (boosted in no small part by Nash’s high starting value), Shaq was clearly post-prime, as was Nash/Kobe when they paired up. But there does seem to be a dip in Nash’s value when Nash is paired with players who demand the ball more (the kind of thing portability cares about), and the times when Nash had his best impact required very careful offensive construction being surrounded specifically by lots of play finishers (exactly like I mentioned in my conclusions!).

Of course, we don’t get that same kind of multi-team perspective for prime Stockton, so there’s some film analysis and projection required. This is all a very high-level overview of the kinds of arguments one might make. It’s possible Thinking Basketball might give a more nuanced version of this argument in either of his Top 40 Project discussions of the players. But this is the kind of argument you might use to suggest Nash is the overall better offensive player, but with worse portability, and I’m not sure your point disproves it just yet.

Now I actually agree with you that Nash shouldn’t be a -2 port. He’s an all time shooter, which has to retain some value if he’s forced to play more of a finishing role with more of an on-ball costar. Likewise, he absolutely can develop incredible chemistry with strong finishers — he’s a GOAT level passer. So a portability score has to weigh those positive abilities with the ball dominance, and the great SSOL team offenses with the declining impact when with other more ball dominant players.

Perhaps for some of those reasons, TB updated Nash’s portability score:
Nash -1

All that to say, you certainly don't have to agree with any specific portability assessment from Thinking Basketball. But he's actually closer to your perspective, since the latest portability scores are different from the portability scores you post here, and I think some of the arguments against his scores here miss what his reasoning is behind the portability score.
A disagreement on values, on how much to weigh different skills when assessing a mean portability score, isn’t necessarily internally inconsistent.

Player Impact:
Bad Gatorade wrote:Either way, even if his portability allotments are correct, a statistically significant sample also differs from a statistically meaningful sample. A dataset can have a p-value of < 0.05, and the difference can be real, but a p-value says nothing of the magnitude of the difference. Now, you have attempted to quantify this, so do I credit you there, although I still question the magnitude of some of these metrics.
Yep! It seems like the biggest magnitude is at the overall team level (e.g. overall SRS or ELO). Which brings up an interesting question of… why?

a) Is this a case where the higher-portability stars do have better ceiling-raising chemistry but get surrounded by more defensive oriented supporting casts, while a few lower-portability stars get surrounded by more offensive oriented supporting casts? There’s certainly some examples I could point out (e.g. Bird/Curry vs Nash).

b) Another possibility: is it the teammates who benefit from portable stars, and not the superstars? Let’s say for a sec that you buy the surprisingly high p-value in comparing the portability vs the overall team results, as well as the t-tests comparing portability vs offensive team results (more on that later). That would imply those teams with more portable stars tend to (though certainly not always!) have more value. 

Where does this value come from? Does the good chemistry benefit the impact of the superstar? Or does being portable mean just not getting in the way of the costars and the supporting cast (a la Wilt’s quote in OP), allowing them to have less diminishing returns? Either/both, depending on the situation? Would impact metrics successfully assign value correctly here, or could they mis-attribute value to the other person?

This ties in with the comment from
:
Djoker wrote:.


Maybe one way forward on this would be to look at the impact metrics of co stars (say the 2nd best player) compared to the portability of the superstar, in this sample of either great overall teams / great offensive teams / great overall superstars.

What we’re trying to do here is get a model for how chemistry and fit tends to work on good teams, which is obviously very complex. I’m not surprised at all that the fit between a general model and very noisy impact data based on a very real (I would hope we all agree fit/chemistry are real!) but very complex process would produce low R^2 values. To me, the fact that portability carries stronger statistical significance (by multiple orders of magnitude(!) ) when fit to better metrics than Plus Minus (like on-off and AuPM), and that it produces statistically significant trends with overall team results is pretty compelling.

Overall Team Offense:
Bad Gatorade wrote:
Among the Top 50 Teams according to RS rORTG:
-Number of best offensive players with +2 Port: 7
-Number of best offensive players with +1 Port: 8
-Number of best offensive players with +0 Port: 21
-Number of best offensive players with -1 Port: 10
-Number of best offensive players with -2 Port: 0
-(Number of best offensive players without Thinking Basketball Port scores yet: 4)
-Average port score: +0.3.
-Positive vs negative scores: 13 have positive port (7 have +2), 10 have negative port (0 have -2).

We can do a statistical test called a (one-sample) t-test to determine if the mean portability in this sample is significantly different (in the statistical sense) from the expected mean of 0. If the (one-tail) p-value from this t-test is less than 0.05, we can conclude there is a real shift towards positive portability, and that this is not just noise.


Just as a note - the expected mean of 0 may not reflect the actual superstar mean. It's entirely possible that the sample of 'potential #1 players' may also have very few -2 valuations depending on what valuation scale Ben has used.


Indeed! You’re quite right here. While the full mean should be 0, with 0 being league average, positive being above average, negative being below average, we don’t know if there’s a bias in the evaluations or a shift in the sample we’re looking at.

But it turns out it’s basically 0. It depends what you set the threshold for ‘superstar’ to of course, but here are the means of the sampled seasons in the database:
3.25+ players (Top ~10ish players in a season): +0.026 mean port
4.0+ players (maybe top 7ish players in a season): -0.009 mean port
4.25+ players (Top 5ish players in a season): 0.004 mean port
5.0+ players: -0.056 mean port

These are all absolutely just 0 +/- uncertainty, so I think we’re good here.

Bad Gatorade wrote:I'd state that a one-sample t-test is not appropriate here, and neither is assuming that the mean is 0.
Hmm, what’s your issue with a t-test here? Do you have an alternative test to suggest?

I’m no expert on t-tests, but (https://www.scribbr.com/statistics/t-test/) for example says one-sample t-tests are to “investigate whether there’s a difference between group and a standard value or whether a subgroup belongs to a population”, which seems like exactly what we’re looking for.

Wikipedia says it tests whether the mean of a [sub]-population has a value specified in a null-hypothesis (which would be that the full-population of stars/top players have a mean of 0). Again, exactly what we're looking for.

What’s your concern here, and do you have an alternative test to suggest?

Bad Gatorade's Criticisms of Portability + an Alternate model:
Bad Gatorade wrote:
Some of the on-court rating data needs a bit of scrutiny too - A dataset of 8 players (Curry, Durant, LeBron, Jokic, Kawhi, Luka, Harden, Paul) is fairly small in itself, but it's worth mentioning that Curry/Durant played on the same team in what was one of the most controversial free agent signings ever and this was their most successful impact period. However, I'd like to go a step further -

If we look at relative team ORTG, for example, Curry's best 3 year stretch was from 2017-2019, in which the team rORTG was +10.3 (weighing it on a game by game basis). Paul, who was by far the lowest out of everybody there, had his 3 year peak from 2013-2015, where the team relative ORTG was... +10.3. Suddenly, the +8.7 point difference dissipates. LeBron's team from 2015-2017 had +11.2. Jokic's highest rORTG ever was in 2023, at... +7.0. And just to top it off, since we have Nash listed there, even though he wasn't included in the regression, we've got a 3 year sample of -2 portability drain Steve Nash at... +12.0, top of the sample.

I don't disagree with the value of "portability" underneath a different definition/usage, but I strongly disagree with how it's applied. Let's have two hypothetical players -

Player A is +7 on ball and +7 off ball
Player B is +5 on ball and +7 off ball

(the number values themselves can be scaled up/down; that's not really salient to the discussion)

The way Ben considers portability, he would consider players A and B equals, because their "optimal value" would be playing off ball at +7. However, that completely ignores that playing on ball at +7 is also incredibly useful,
A quick clarification: The last part isn’t quite right for what Thinking Basketball’s doing.

Let’s assume TB agrees on those assessments (Player A = +7 on-ball and +7 off; Player B = +5 on ball and +7 off). For ease of math, assuming they both spend 50% of the time on and off ball, the Player A’s mean value is +7 and Player B’s mean value is +6.

TB would assign Player A a mean +/- valuation of +7 (i.e. a higher starting point), and Player B +6.
TB would assign Player B a marginally higher portability since off-ball action is a greater percentage of their value relative to their starting point (it’s 58% of Player B’s value to Player A’s 50%).
(Edit: not actually sure here on second thought — they could also be assigned equal portability given the off-ball value is equal, or at least the difference in their portability might not be worth changing a full number since there’s only 5 portability grades...)
But regardless, there’s a limited weighting to portability. Portability can’t boost players arbitrarily high. There’s no team in Ben’s CORP calculator where Stockton’s higher portability means he actually adds more value than Nash — the difference in the mean +/- valuation is just too high.
(Edit: perhaps a better example here is Curry vs Miller. Both are great off ball. Curry’s great on ball. Both have the same portability grade, but Curry’s starting value (the mean +/- valuation) is appropriately much higher, and Curry would rightfully add more value to any standard team in the corp calculator than Miller. )
Likewise here, there’s no team where Player B would actually be preferred. It’s just the difference between them might reduce if the portability grade was marginally higher.

Where it would get more interesting is when Player A is actually worse off-ball (e.g. +5 off ball), or if Player A is just as good off-ball as Player B per possession, but on a smaller volume where you might have concerns how well their volume would scale up. In situations like these, you might actually end up preferring Player A or Player B depending on the exact numbers and team.

and there have been numerous teams that have won titles (Duncan's Spurs, Hakeem's Rockets etc) simply because their offence was good enough within their team structure, even if they weren't offensively at the level of LeBron/Magic/MJ etc. However, that sort of thing is dismissed completely, and I feel like that's wrong.
Great points! And I would absolutely agree that that’s wrong. But it’s not what Ben’s doing.

Like I said in my post and like Thinking Basketball said in the links included in my post / elsewhere, he includes defense in his evaluation, and he considers good defense maximally portable.

Strong defense gets minimum diminishing returns on good teams, while offense has a wider range of diminishing returns. There’s data to back this up somewhere in old TB posts, and of course it intuitively makes sense and it’s clear in the team-level championship performance like you point out.

So he needs to estimate how good a player is on offense and defense, then assess how much a player’s offense would face diminishing returns on different team levels, in order to assess their chances of winning a championship on a random team.

Bad Gatorade wrote: A better measure of actual portability would be to consider how the player performs in different , yet reasonable circumstances. Something such as a logit model (which would give credit to Player A above, as you have the choice to use him on/off ball at equal efficacy) based on different usages/team structures would actually be a measure of portability.
Interesting idea! It seems like this is prioritizing portability ~= versatility. Is that a fair summary?

Given some of my clarifications/comments, how would this method differ on the portability of someone like Jerry West? Let’s put aside the actual player evaluations for a sec and assume we agree with Ben’s assessment of each player’s skill (since we’re talking about general methodology/philosophy here). Let’s assume Ben’s right that West does have better off-ball offense relative to era than someone like Chris Paul (or at least that a greater percentage of West’s value comes from off-ball offense if you’re lower on West’s starting point). This might be from a difference in off-ball volume (and concerns with how well CP3 could scale up his off-ball volume in a different role), rating West’s finishing ability as higher (e.g. his relative efficiency is definitely better), rating his shooting as better (e.g. his era-relative shooting is definitely better), liking his chemistry with Elgin Baylor or Wilt more, being less ball-dominant (e.g. smaller load), etc.

So when West shifted to being more of an on-ball playmaker and still maintained value, wouldn’t that be a case where a player was e.g. +7 on-ball and +7 off (to use the numbers you gave, but like you say the value is arbitrary here), and thus should get positive portability? I suppose the issue could have been more with e.g. Chris Paul’s grade, but there’s at least reasons given above for why Ben might have been lower on Paul’s scalability, even if they’re not conclusive and people can still disagree. Just trying to get a sense of how your proposed model differs here.

Anyways, thanks for the thoughtful commentary! :)
User avatar
Bad Gatorade
Senior
Posts: 715
And1: 1,870
Joined: Aug 23, 2016
Location: Australia
   

Re: Evidence For Portability 

Post#15 » by Bad Gatorade » Wed Sep 4, 2024 4:42 pm

DraymondGold wrote:...


Thanks for the reply - I'll try and break it down (there's a lot to unpack here).

Regarding the low R^2

Plus minus is inherently noisy, and single season plus minus is going to be even more noisy. Ben absolutely tries to "smooth" his valuations out, so no disagreements there. I guess my greater issue is with the conclusion that going from, say, 0.07 R^2 to 0.08 R^2 by adding portability, due to the inherent noisiness, isn't really a boon for portability. Adding value to a noisy stat doesn't mean that portability is "defying the odds" or anything like that - it more so means that low R^2 values generally indicate that there's either something with the methodology that's not quite right, or the data trend isn't really there.

Regarding the specific player evaluations

Thanks for letting me know that he has updated his valuations (especially re: Dirk and Durant, because Dirk and Durant having the same value makes a lot more sense than what he had before).

I am aware that his portability is more so "how well does this player scale onto better teams", but I still find quite a few inconsistencies, even with the updated numbers. To use Nash as an example again, even though Nash's portability seems to have been updated to -1, as opposed to -2, I do disagree that portability is truly an accurate representation of Nash's portability. What Ben says is that if Nash was alongside better players, then he'd lose some of his value. The problem is that Nash is already producing top tier value on his team. What the negative portability tells me is that Nash loses some value alongside ball handlers, but that's not really a Steve Nash issue as much as it's a team building consideration. Dray is a +2 portability guy, but he would obviously lose offensive value on a team with other ball handlers.

In other words, Nash gets punished in terms of portability because there's a chance he loses value next to a high volume playmaker, but would Dray also not lose value in this circumstance?

Another example - Chris Paul had negative portability in 2015, even though he played alongside another high volume player in Blake Griffin (who was producing at a > 20 PPG and > 5 APG rate) on the best offence in the league. He had neutral portability in 2020, on a team with fairly egalitarian usage (4 guys between 17.6 and 19.0 PPG). Bear in mind that Schroder is a point guard, Shai is a combo guard (and often lead distributor) and whilst Gallo isn't a guard, he is a perimeter orientated player. That is hardly a scenario in which his impact is maximised. He gets punished when playing as the primary PG option (even though there's another high volume scorer + ball handler/passer in the starting lineup) but he doesn't get credited for his play when he's playing alongside a litany of PGs - he just didn't get "punished."

Anyway, this is somewhat a pointless exercise IMO, because I'm not going to agree with all of the valuations and we'd be here all day, so it's not really a "productive" use of this discussion time.

Regarding +/- and t-testing

So, there's no reason to believe the mean is meant to be 0, because portability itself is... subjective, and so there's no guarantee that Ben allots the portability values in a normally distributed way.

Looking at all 1,339 seasons available, we find only 13 seasons that are -2 portability (so, out of a sample of 46, we're expected to have only 0.447 results - in other words, the sample giving 0 seasons is what we'd expect). The mean is 0.14, which is not too dissimilar from the 0.26 you found for the top end data.

Now, running multiple tests (including a t-test) actually gives me a p-value > 0.05, meaning we'd retain the notion that the "top 50 (really 46)" teams aren't doing anything statistically significant enough to parse them from the pack. Of course, there is also error in the sense that the number of player seasons may also be mixed, and it's not just the "best player" but I digress.

Using the adjusted population mean, the p-values are actually quite far from 0.05 (I'm talking +0.2 one tailed, +0.4 two tailed type stuff).

So, my concerns with a t-test here are -

a) running a few normality tests (eg Shapiro-Wilk), we can see that the distribution is not "normally" distributed and so I'd automatically be hesitant running the test. Usually, you can still get away with a t-test at this sample applying the Central Limit Theorem, but I'd look elsewhere first given the distribution of the data.

b) the null hypothesis you ran actually asks if the portability of the top 46 teams stars is > 0, whereas the conclusion you're reaching is that the portability of the top 46 teams is a real phenomenon, because it's different from 0. If the average portability of top 46 stars was 0.26, but the overall portability of the average player was 0.5, or something, then your testing method would be outright telling us the wrong thing (i.e. that the top 46 team stars are more portable). It's important to cater the test to what the actual aim is.

I'd probably prefer something like a Chi-square goodness of fit test, since there are clear pots of data (-2, -1, 0, 1, 2) that you're aiming to match. Even then, it's iffy since the sample size feels subjectively small (and you'd have to pool the pots together to an extent IMO) but running something like this still results in a p-value > 0.05.

What Thinking Basketball is doing when they run portability and impact numbers

In theory, he often does this, and there are definitely players that get high portability scores for being able to play the game in multiple ways and retain impact. I don't necessarily oppose Jerry West being treated as "portable." However... he also gives these maximum scores to players such as Klay Thompson (whose on ball and off ball game has a clear discrepancy) and Draymond (ditto).

Ben obviously considers defence in his evaluation - he's clearly one of the best basketball minds out there and he has pushed forward the way people view basketball in the mainstream. I more so disagree with the way he approaches portability, and how it is attached to his player valuations. To me, it feels like more of a preferential tweak, rather than something remotely scientific.

As I've said before, I'm all for considering impact in different scenarios, because a holistic view of basketball is going to assist player evaluation, but I'm not at all convinced all of the valuations provided are considering all of the different scenarios - they sometimes feel more like a test of "how well would this player fit if you plugged them onto the Golden State Warriors" to me.
I use a lot of parentheses when I post (it's a bad habit)
OhayoKD
Head Coach
Posts: 6,025
And1: 3,914
Joined: Jun 22, 2022
 

Re: Evidence For Portability 

Post#16 » by OhayoKD » Wed Sep 4, 2024 10:33 pm

Bad Gatorade wrote:
DraymondGold wrote:What Thinking Basketball is doing when they run portability and impact numbers

In theory, he often does this, and there are definitely players that get high portability scores for being able to play the game in multiple ways and retain impact. I don't necessarily oppose Jerry West being treated as "portable." However... he also gives these maximum scores to players such as Klay Thompson (whose on ball and off ball game has a clear discrepancy) and Draymond (ditto).


Draymond makes alot of sense as a scalable (there is no real function to making portablity a synonym besides confusing people) piece...if ben was looking at any of the things he offers there (defense - simply not a factor), floor-general (also simply not factored in).

Scalability really seems to "how would players do in the warriors system" and/or "how would they do if they couldnt' handle the ball because it doesn't do anything but put you in a position to assist/score"
its my last message in this thread, but I just admit, that all the people, casual and analytical minds, more or less have consencus who has the weight of a rubberized duck. And its not JaivLLLL
SNPA
General Manager
Posts: 8,975
And1: 8,344
Joined: Apr 15, 2020

Re: Evidence For Portability 

Post#17 » by SNPA » Thu Sep 5, 2024 5:11 am

Ben’s definition it a tad odd. I only saw a clipped version, the full sentence/paragraph being quoted would be good (maybe I missed it?).

Anyways, the best interpretation I can put on it is portability means; a guy can still get his, while playing well with other good players who are getting theirs.

Taking that fundamental understanding it’s not hard to discern who fits. The better the defense, check. The better the rebounder, check. The better the passer, check. The better the shooter, check. All around game, and a winning mindset (flexible in style and approach). That’s what portability means, according to that definition IMO.

It’s a water theory. Portable players will find level because they have enough skills to fit in somehow and still be effective.
penbeast0
Senior Mod - NBA Player Comparisons
Senior Mod - NBA Player Comparisons
Posts: 30,163
And1: 9,777
Joined: Aug 14, 2004
Location: South Florida
 

Re: Evidence For Portability 

Post#18 » by penbeast0 » Thu Sep 5, 2024 11:48 am

I think of scalability as different from portability. Being scalable, to me, means you have the capacity to up your usage strongly in a featured role or to still be valuable in a non-featured role with off ball scoring, defense, etc. Chris Bosh would be a good example.

Portability, to me, means that you can be effective in different offensive and (to a lesser degree) defensive systems; less focus on defense because I don't like to penalize great rim protectors and great point of attack defenders though I probably should for consistency since two of them side by side means the one not in that primary role isn't making the same degree of impact. A great efficient scorer that is effective on and off ball, but wouldn't necessarily be as valuable in a low usage role like a Steph Curry would be my poster boy for portability without scalability.
“Most people use statistics like a drunk man uses a lamppost; more for support than illumination,” Andrew Lang.
User avatar
Heej
General Manager
Posts: 8,469
And1: 9,170
Joined: Jan 14, 2011

Re: Evidence For Portability 

Post#19 » by Heej » Thu Sep 5, 2024 2:29 pm

The problem with how people see portability now is that it's wielded as a cudgel to bludgeon players whose playstyles you don't like. And the criteria for what makes some players more portable than others just seems like Ben worked backwards from a foregone conclusion. It's easy to say that off-ball scoring or shooting is the most portable skill because it looks good on paper, especially in the regular season.

In the playoffs vs better coaching staffs and players there are certain skills (the off-ball shooting championed by Ben is an example) which are far easier to take away, which in turn makes me think they're actually less portable by definition.

The current concept of what's portable and what isn't doesn't make sense to me when you devalue contender level lead playmaking. A good playmaker can scale with any team because there might be perhaps 20 guys in the league you can trust to handle a large primary playmaking load, while nearly 90% of the league is a record scratch giving them the ball to attack a closeout in the finals :lol:

Also, defense is certainly scalable but some types are easily more scalable than others. POA defenders with good screen navigation skills in PnR? Top tier portability. POA defenders that can't actually get thru screens and can only defend isos? Trash portability and at risk of being played off the floor if they're not offensively gifted and get constantly screened out of the play (My God, that's Jarred Vanderbilt's music!)

But stuff like that doesn't really get looked at in-depth. It's all just word salad to hype up the guys you like vs the guys you don't. Also I agree with Ohayo that scalability makes way more sense as a term anyway when you consider what a leap it is going from regular season gameplans/competition to playoff level.
LeBron's NBA Cup MVP is more valuable than either of KD's Finals MVPs. This is the word of the Lord
User avatar
homecourtloss
RealGM
Posts: 11,317
And1: 18,724
Joined: Dec 29, 2012

Re: Evidence For Portability 

Post#20 » by homecourtloss » Fri Sep 6, 2024 2:28 am

Bad Gatorade wrote:
DraymondGold wrote:...


Thanks for the reply - I'll try and break it down (there's a lot to unpack here).

Regarding the low R^2

Plus minus is inherently noisy, and single season plus minus is going to be even more noisy. Ben absolutely tries to "smooth" his valuations out, so no disagreements there. I guess my greater issue is with the conclusion that going from, say, 0.07 R^2 to 0.08 R^2 by adding portability, due to the inherent noisiness, isn't really a boon for portability. Adding value to a noisy stat doesn't mean that portability is "defying the odds" or anything like that - it more so means that low R^2 values generally indicate that there's either something with the methodology that's not quite right, or the data trend isn't really there.

Regarding the specific player evaluations

Thanks for letting me know that he has updated his valuations (especially re: Dirk and Durant, because Dirk and Durant having the same value makes a lot more sense than what he had before).

I am aware that his portability is more so "how well does this player scale onto better teams", but I still find quite a few inconsistencies, even with the updated numbers. To use Nash as an example again, even though Nash's portability seems to have been updated to -1, as opposed to -2, I do disagree that portability is truly an accurate representation of Nash's portability. What Ben says is that if Nash was alongside better players, then he'd lose some of his value. The problem is that Nash is already producing top tier value on his team. What the negative portability tells me is that Nash loses some value alongside ball handlers, but that's not really a Steve Nash issue as much as it's a team building consideration. Dray is a +2 portability guy, but he would obviously lose offensive value on a team with other ball handlers.

In other words, Nash gets punished in terms of portability because there's a chance he loses value next to a high volume playmaker, but would Dray also not lose value in this circumstance?

Another example - Chris Paul had negative portability in 2015, even though he played alongside another high volume player in Blake Griffin (who was producing at a > 20 PPG and > 5 APG rate) on the best offence in the league. He had neutral portability in 2020, on a team with fairly egalitarian usage (4 guys between 17.6 and 19.0 PPG). Bear in mind that Schroder is a point guard, Shai is a combo guard (and often lead distributor) and whilst Gallo isn't a guard, he is a perimeter orientated player. That is hardly a scenario in which his impact is maximised. He gets punished when playing as the primary PG option (even though there's another high volume scorer + ball handler/passer in the starting lineup) but he doesn't get credited for his play when he's playing alongside a litany of PGs - he just didn't get "punished."

Anyway, this is somewhat a pointless exercise IMO, because I'm not going to agree with all of the valuations and we'd be here all day, so it's not really a "productive" use of this discussion time.

Regarding +/- and t-testing

So, there's no reason to believe the mean is meant to be 0, because portability itself is... subjective, and so there's no guarantee that Ben allots the portability values in a normally distributed way.

Looking at all 1,339 seasons available, we find only 13 seasons that are -2 portability (so, out of a sample of 46, we're expected to have only 0.447 results - in other words, the sample giving 0 seasons is what we'd expect). The mean is 0.14, which is not too dissimilar from the 0.26 you found for the top end data.

Now, running multiple tests (including a t-test) actually gives me a p-value > 0.05, meaning we'd retain the notion that the "top 50 (really 46)" teams aren't doing anything statistically significant enough to parse them from the pack. Of course, there is also error in the sense that the number of player seasons may also be mixed, and it's not just the "best player" but I digress.

Using the adjusted population mean, the p-values are actually quite far from 0.05 (I'm talking +0.2 one tailed, +0.4 two tailed type stuff).

So, my concerns with a t-test here are -

a) running a few normality tests (eg Shapiro-Wilk), we can see that the distribution is not "normally" distributed and so I'd automatically be hesitant running the test. Usually, you can still get away with a t-test at this sample applying the Central Limit Theorem, but I'd look elsewhere first given the distribution of the data.

b) the null hypothesis you ran actually asks if the portability of the top 46 teams stars is > 0, whereas the conclusion you're reaching is that the portability of the top 46 teams is a real phenomenon, because it's different from 0. If the average portability of top 46 stars was 0.26, but the overall portability of the average player was 0.5, or something, then your testing method would be outright telling us the wrong thing (i.e. that the top 46 team stars are more portable). It's important to cater the test to what the actual aim is.

I'd probably prefer something like a Chi-square goodness of fit test, since there are clear pots of data (-2, -1, 0, 1, 2) that you're aiming to match. Even then, it's iffy since the sample size feels subjectively small (and you'd have to pool the pots together to an extent IMO) but running something like this still results in a p-value > 0.05.

What Thinking Basketball is doing when they run portability and impact numbers

In theory, he often does this, and there are definitely players that get high portability scores for being able to play the game in multiple ways and retain impact. I don't necessarily oppose Jerry West being treated as "portable." However... he also gives these maximum scores to players such as Klay Thompson (whose on ball and off ball game has a clear discrepancy) and Draymond (ditto).

Ben obviously considers defence in his evaluation - he's clearly one of the best basketball minds out there and he has pushed forward the way people view basketball in the mainstream. I more so disagree with the way he approaches portability, and how it is attached to his player valuations. To me, it feels like more of a preferential tweak, rather than something remotely scientific.

As I've said before, I'm all for considering impact in different scenarios, because a holistic view of basketball is going to assist player evaluation, but I'm not at all convinced all of the valuations provided are considering all of the different scenarios - they sometimes feel more like a test of "how well would this player fit if you plugged them onto the Golden State Warriors" to me.


Back to back great posts in this thread, BG. A few other things I wanted to add to the comments that you have already made.

1. Low R^2 values
As Draymond on mentioned, R^2 value shows the proportion of variance in the dependent variable that is predictable from the independent variable and some cases variables . Whether or not value is .07 which means 7% of the variance is explained by the model or the value is .08 meaning 8% doesn’t really matter When we are looking at the noisiness of plus-minus statistics, especially over a single season, small fluctuations in R^2 are likely because of random variance rather than some true effect of portability. Also, as an aside, I find it amusing that in other threads, even when the R^2 value was not been used to make an argument there were comments about how the low R^2 values meant nothing was found but there’s a different reaction in this thread.

2. Subjectivity in Assigning Portability Scores
As has been mentioned so many times before, the assignment of portability scores without any empirical foundation and based on subjective judgments leaves everything to be desired. What are we actually looking at when we’re looking at subjectively assigned scores? if there’s no standardization and no objectivity, then of course the data is going to be biased and inconsistent. You and others have brought up the example of Chris Paul who is one of the great point guard defenders and defensive quarterbacks of the data ball and is also a good spot up shooter who somehow has a negative portability score even though he has proven valuable and impactful in every single context he’s ever been in. That brings up another point about defense and whether defense is going to be disregarded at all together when talking about “portability.” I also like the term scalability better and if you are going to use the term portability, then at least should say offensive portability, even though I disagree with that as well. The subjectivity basically shows personal bias and as Heej mentions, Ben seems to be working backwards from a conclusion he already has in his mind. The bottom line is that when you have these portability scores being subjective, I can’t really be used or reliable in any type of statistical models.

3. A Chi-Squared Test Is More Appropriate Than a T-Test
You can’t use a t-test if the overall distribution of the portability scores is not normally distributed; if you’re looking at these probability scores of -2, negative one, plus one, +2, etc., which are categorical, it’s highly unlikely that the data is normally distributed, especially when you have such few players at -2. A T-test assumes normality so in this case is not valid. This means a chi-squared goodness-of-fit test would be more valid than a t-test. The chi-squared test doesn't assume a normal distribution and is intentionally used to compare the frequency of things that we actually observe with the expected frequencies against whatever categories we want. If we had continuous normally distributed data, we could do that, but we don’t have it with these portability scores.

So basically, even if we do have a stat test that shows some significance, i.e., below value, the effect size might be too small anyway to have any practical relevance so even if we did have a tiny increase in R^2, it really wouldn’t mean anything. It will be A lot of work, but I hope Ben actually puts in place and objective statistical system that determines “portability.”
lessthanjake wrote:Kyrie was extremely impactful without LeBron, and basically had zero impact whatsoever if LeBron was on the court.

lessthanjake wrote: By playing in a way that prevents Kyrie from getting much impact, LeBron ensures that controlling for Kyrie has limited effect…

Return to Player Comparisons