New Impact Metric: MAMBA

[lessthanjake]

LeBron lakers tenure has largely been a disappointment. Not sure how that is even arguable

But you are the same poster who says harden was not a good a defender in 2018 and 2020, so I already know you are a narrative follower like the rest of the nba fanbase

To be fair, I dunno that I’d call anyone’s tenure on a team a disappointment if they won a title there. I think it’s probably fair to call most other individual seasons he’s had there a disappointment (probably with the exception of 2023, where the RS was fairly disappointing but they made the WCF, so it was a pretty decent year), but the 2020 title changes the conclusion about the Lakers tenure as a whole IMO.

Relatedly, I think if you count bringing in Anthony Davis as LeBron’s doing, then that probably more than counteracts the other bad decisions on its own, since AD was a very crucial part of the team winning a title.

And what are the bad decisions our being counteracted here? Coach-changes led to the teams improving

Lebron borderline tampering to get AD is well documented and initially Boston was the team people expected to get him. Lebron forcing the FO to trade for Westbrook seems little more than people assuming Lebron controls what the FO does, blatantly ignoring that Lebron was unable to make the FO pay Ty Lue enough. Just like he was unable to make the cavs trade hickson for amare stoudimire potentially gaining them an extra championship.

Outside of bizarrely deciding change is inherently a negative decision or exclusively attributing decisions deemed as negative to Lebron, there's not much of an argument as LeGM as a net negative.

I think it is highly reductive to the point of uselessness to suggest that a coaching change must’ve been good if the team’s record improved their first year. There’s so many other factors that go into that—the coach isn’t even the biggest one. And I know Lakers fans are adamant that Darvin Ham was notably bad.

In any event, the rest of what you say is basically just speculation about what LeBron did or didn’t push for. Ultimately, we can’t really know the details of any of that—we don’t have full information, and what we do have are primarily stories planted with reporters by one person or another (and likely for self-serving reasons). Which I think does make the criticisms of LeBron inherently speculative and based on some assumptions. Any such assumptions may be right and they may be wrong. I think it is true that the Lakers have made some bad decisions, and the extent to which that reflects on LeBron’s off-court decision-making is not entirely clear because we do not *really* know his culpability in those decisions.

[AEnigma]

So I messaged the creator after the typically unproductive General Board back-and-forth. This was his initial response regarding the claim that “old MAMBA was replaced with a worse version partially to boost Lebron”:

MAMBA was never less accurate. It had less real world testing. It’s actually destroying LEBRON and EPM projections I’m just too lazy to post it. The rigging it for lebron was more of a joke; I don’t even remember saying I rigged it for lebron in any way, unless it’s making the top end results make sense, but all the impact metrics have done that.

At that point I sent the actual comment in question:

The creator of MAMBA specifically said he thought the original version underrated LeBron’s post-Miami defense, and then created a revised version that improved LeBron’s standing, while noting that changes were made because LeBron “w[as] underrated in the Prior” (albeit that particular comment came specifically in a bullet point about offense, but it certainly further indicates the sentiment about how the prior was revised). Improving LeBron’s standing of course wasn’t the only reason changes were made, but I don’t think anyone could fairly read through both lengthy write ups the creator made and fail to come to the conclusion that improving LeBron’s standing was a major purpose of the revision. There’s no particular reason to prefer new MAMBA for these purposes compared to old MAMBA, except that the creator happened to like the output more (which, again, was obviously in part a result of it placing LeBron higher).

To which he responded:

Strong misread.

1. The part on improving the original version because it underrated Lebrons defense was not what I said, it was in reference to another all in one metric which clearly did so when you cross reference it to other ones and itself. All in one metrics inherently will underrate players who have more non box score impact than their box scores suggest, Lebron was one of multiple blatant examples of this where their results outpace their priors, and that is still the case. He’s also hardly the one who improved the most, and you have a consistent effect where reducing the weight of the prior improves his standing more than anyone else, which I didn’t do because it would hurt the rest of the model. Lebron is underrated in every version here pretty clearly, if I were to create a model for him I’d simply weight on court impact more.

2. The main change I made was to not only consider the players ability to be more efficient than their shot difficulty but also consider the ability to get good shots and how often they could do so.

3. The old Mamba tested worse overall and would have tested much worse in real world testing, the current one is in first place in APBR testing far above more complex predictions using other all in one metrics.

I hope but will not hold my breath that if someone claims to be interested in a metric, future questions can be asked using the contact resources available rather than baselessly inventing positions never expressed.

[lessthanjake]

So I messaged the creator after the typically unproductive General Board back-and-forth. This was his initial response regarding the claim that “old MAMBA was replaced with a worse version partially to boost Lebron”:

MAMBA was never less accurate. It had less real world testing. It’s actually destroying LEBRON and EPM projections I’m just too lazy to post it. The rigging it for lebron was more of a joke; I don’t even remember saying I rigged it for lebron in any way, unless it’s making the top end results make sense, but all the impact metrics have done that.

At that point I sent the actual comment in question:

The creator of MAMBA specifically said he thought the original version underrated LeBron’s post-Miami defense, and then created a revised version that improved LeBron’s standing, while noting that changes were made because LeBron “w[as] underrated in the Prior” (albeit that particular comment came specifically in a bullet point about offense, but it certainly further indicates the sentiment about how the prior was revised). Improving LeBron’s standing of course wasn’t the only reason changes were made, but I don’t think anyone could fairly read through both lengthy write ups the creator made and fail to come to the conclusion that improving LeBron’s standing was a major purpose of the revision. There’s no particular reason to prefer new MAMBA for these purposes compared to old MAMBA, except that the creator happened to like the output more (which, again, was obviously in part a result of it placing LeBron higher).

To which he responded:

Strong misread.

1. The part on improving the original version because it underrated Lebrons defense was not what I said, it was in reference to another all in one metric which clearly did so when you cross reference it to other ones and itself. All in one metrics inherently will underrate players who have more non box score impact than their box scores suggest, Lebron was one of multiple blatant examples of this where their results outpace their priors, and that is still the case. He’s also hardly the one who improved the most, and you have a consistent effect where reducing the weight of the prior improves his standing more than anyone else, which I didn’t do because it would hurt the rest of the model. Lebron is underrated in every version here pretty clearly, if I were to create a model for him I’d simply weight on court impact more.

2. The main change I made was to not only consider the players ability to be more efficient than their shot difficulty but also consider the ability to get good shots and how often they could do so.

3. The old Mamba tested worse overall and would have tested much worse in real world testing, the current one is in first place in APBR testing far above more complex predictions using other all in one metrics.

I hope but will not hold my breath that if someone claims to be interested in a metric, future questions can be asked using the contact resources available rather than baselessly inventing positions never expressed.

I think the most important thing here is the discussion about priors underrating LeBron. And, to echo a point I’ve already made before, I don’t think that that helps you in any discussion we’ve had about this. That’s because if the prior underrates LeBron then it must underrate the people we have compared him to even more, because in any discussion I’ve had on this metric (both as to LeBron’s standing in the league in terms of defense and a comparison with Steph), I’ve pointed out that multi-year RAPM—which is what these all-in-one metrics are tested against (usually 5-year RAPM in particular)—doesn’t make LeBron look any better for purposes of those discussions. In fact, it typically makes him look worse than new MAMBA. So the idea that LeBron is underrated by the prior doesn’t check out, or at least isn’t right for the purposes of any of the actual discussions we’ve had about this.

The statement that LeBron improved “more than anyone else” the less the prior was weighted would seem to go against that notion, which makes me wonder what multi-year RAPM this is being tested against, since that would pretty clearly not be the case if using most of the five-year RAPM we have. As I’ve alluded to, it may well be that it’s being tested against RAPM that has certain of its own basic priors or parameters that are particularly favorable to LeBron compared to other RAPM measures. In which case, the idea that priors hurt LeBron more than others would really just be a function of what RAPM set we are assuming represents the underlying truth, and that’s not a question we really know the answer to. The blog post itself suggests it’s testing against time-decayed RAPM, and mentions that it is very similar in practice to PI RAPM. And PI RAPM does tend to be higher on LeBron than other forms of RAPM, so this checks out. It’s definitely not clear this is the best form of RAPM.

As for the rest of it, I find it a bit confusing. For instance, the statement that what I’d referred to was actually “in reference to another all in one metric which clearly did so when you cross reference it to other ones and itself” is confusing. I think it’s valid to say it wasn’t *exactly* right for me to say he said post-Miami LeBron’s defense was underrated specifically in MAMBA since the statement was more general than that, but he said it was underrated “by all in ones” (not just any specific one) and MAMBA is an all-in-one, so it seemed pretty reasonable to say the statement applied to MAMBA too. And, of course, this response essentially confirms that the author thinks that that statement is true of MAMBA. So I’m not sure how I actually materially misread anything here.

I do think it’s good to have additional information that apparently old MAMBA tested worse than new MAMBA overall. So that does give reason to prefer new MAMBA, though my view on these metrics is that, while we might have a mild preference for those that test better, we should look at as many of them as we can, because testing better overall doesn’t mean it will always be most accurate on any specific question. It’s more like a slightly higher probabilistic chance that it’ll be more accurate. So I think old MAMBA is useful anyways, just like I think the multiple even less well “performing” all-in-one metrics than old MAMBA should be considered. I’d also note that new MAMBA is still supportive of arguments I’ve had relating to it (see, for instance, discussions about Steph vs. LeBron in this very thread), even if it’s a bit higher on LeBron than some metrics.

[AEnigma]

Because there is no singular version of RAPM. None of these metrics are just pulling from github or “therapmdatabase”. When he says his box inputs are what drag Lebron down, it is an internal reference based on the direct impact portion making up MAMBA, which itself needed to be coded a certain way.

[homecourtloss]

Because there is no singular version of RAPM. None of these metrics are just pulling from github or “therapmdatabase”. When he says his box inputs are what drag Lebron down, it is an internal reference based on the direct impact portion making up MAMBA, which itself needed to be coded a certain way.

Some of the wildest things that I have seen are people who did not even know what RAPM was a year or two ago now speak authoritatively about it, even though they don’t have the slightest idea how it’s calculated. also, you will notice that for whatever metric database that if a certain player is at the top of it, then there is absolutely no discussion or further inquiry about the metric itself, but effusive praise. But if a certain person is not at the top, well, then…

[lessthanjake]

Because there is no singular version of RAPM.

Yes, that’s exactly my point. When these metrics are “tested” they are tested against some particular form of RAPM. Similarly, when someone makes a statement that the prior pulls someone down, that is as compared to some particular form of RAPM. The statements effectively assume that that version of RAPM is the objective truth. But there’s no singular form of RAPM, and different RAPM sets use different methodologies and come to different results. And we don’t know which version is best or more accurate. Which means that any statement (or testing) that assumes that one particular form of RAPM is the objective truth isn’t *necessarily* right. And that’s especially true where the statement seems in logical contradiction to several other versions of RAPM. In the case of the discussions we’ve had in which MAMBA has ever come up, I’ve always pointed to many versions of RAPM that support my position (while you have not).

[lessthanjake]

Because there is no singular version of RAPM. None of these metrics are just pulling from github or “therapmdatabase”. When he says his box inputs are what drag Lebron down, it is an internal reference based on the direct impact portion making up MAMBA, which itself needed to be coded a certain way.

Some of the wildest things that I have seen are people who did not even know what RAPM was a year or two ago now speak authoritatively about it, even though they don’t have the slightest idea how it’s calculated. also, you will notice that for whatever metric database that if a certain player is at the top of it, then there is absolutely no discussion or further inquiry about the metric itself, but effusive praise. But if a certain person is not at the top, well, then…

Actually, I think you’ll find that I very consistently say that we should take all available metrics into account, and that it is people who argue for the views you agree with that explicitly push back on that and try to artificially limit consideration to only certain metrics with output that they like. I guess perhaps you’re criticizing those you substantively agree with here, but if you’re referring to me, then you’re way out of line.

[AEnigma]

Because there is no singular version of RAPM.

Yes, that’s exactly my point. When these metrics are “tested” they are tested against some particular form of RAPM. Similarly, when someone makes a statement that the prior pulls someone down, that is as compared to some particular form of RAPM. The statements effectively assume that that version of RAPM is the objective truth. But there’s no singular form of RAPM, and different RAPM sets use different methodologies and come to different results. And we don’t know which version is best or more accurate. Which means that any statement (or testing) that assumes that one particular form of RAPM is the objective truth isn’t *necessarily* right. And that’s especially true where the statement seems in logical contradiction to several other versions of RAPM. In the case of the discussions we’ve had in which MAMBA has ever come up, I’ve always pointed to many versions of RAPM that support my position (while you have not).

Because I am not in the business of compiling lists of outputs for the purpose of advancing a chosen fan narrative. You see a metric’s creator say, “My metric rates this player worse when I add box components,” and rather than engage with that concept, your reaction is to dump a compilation of irrelevant other metrics in a transparent slander attempt. That is pure ideology, but characteristically, you prefer to pretend it is actually something everyone else does.

[lessthanjake]

Because there is no singular version of RAPM.

Yes, that’s exactly my point. When these metrics are “tested” they are tested against some particular form of RAPM. Similarly, when someone makes a statement that the prior pulls someone down, that is as compared to some particular form of RAPM. The statements effectively assume that that version of RAPM is the objective truth. But there’s no singular form of RAPM, and different RAPM sets use different methodologies and come to different results. And we don’t know which version is best or more accurate. Which means that any statement (or testing) that assumes that one particular form of RAPM is the objective truth isn’t *necessarily* right. And that’s especially true where the statement seems in logical contradiction to several other versions of RAPM. In the case of the discussions we’ve had in which MAMBA has ever come up, I’ve always pointed to many versions of RAPM that support my position (while you have not).

Because I am not in the business of compiling lists of outputs for the purpose of advancing a chosen fan narrative. You see a metric’s creator say, “My metric rates this player worse when I add box components,” and rather than engage with that concept, your reaction is to dump a compilation of irrelevant other metrics in a transparent slander attempt. That is pure ideology, but characteristically, you prefer to pretend it is actually something everyone else does.

I hope you realize that compiling all available data is virtually definitionally the opposite of “advancing a chosen fan narrative.” Maybe if a “compilation” of data suggests that you’re wrong about a given viewpoint on a player, you should consider whether you may actually just be wrong, rather than objecting to the presentation of fulsome information and/or retreating to the use of only a particular metric or two that gives you output you like.

As for the fact that the MAMBA creator says that LeBron gets worse in his metric the more you weight the prior, I have no doubt that that’s true, but I find it extremely odd that you suggest I did not “engage with that concept.” I engaged with it by pointing out that, given that we have lots of RAPM output that is not consistent with that (at least as it relates to the specific topics you and I have discussed), that seems to just be a relic of the specific type of RAPM that the creator is using. This seems obviously true. If we have a bunch of multi-year RAPM data sets and MAMBA tends to be a little bit more favorable to LeBron than that available RAPM data, then it is pretty obviously the case that the only way the MAMBA prior pulls LeBron down from RAPM more than anyone else is if the RAPM that MAMBA uses is much more favorable to LeBron than a bunch of other RAPM data we have. So the MAMBA creator’s statement basically just means that the RAPM methodology he uses is very high on LeBron compared to other RAPM. And that’s a good fact for LeBron (though we don’t know how good, since I don’t think we actually have just the RAPM, and he doesn’t look the best once we add the prior). We don’t actually know what of the many versions of RAPM is the best, though.

[AEnigma]

Yes, that’s exactly my point. When these metrics are “tested” they are tested against some particular form of RAPM. Similarly, when someone makes a statement that the prior pulls someone down, that is as compared to some particular form of RAPM. The statements effectively assume that that version of RAPM is the objective truth. But there’s no singular form of RAPM, and different RAPM sets use different methodologies and come to different results. And we don’t know which version is best or more accurate. Which means that any statement (or testing) that assumes that one particular form of RAPM is the objective truth isn’t *necessarily* right. And that’s especially true where the statement seems in logical contradiction to several other versions of RAPM. In the case of the discussions we’ve had in which MAMBA has ever come up, I’ve always pointed to many versions of RAPM that support my position (while you have not).

Because I am not in the business of compiling lists of outputs for the purpose of advancing a chosen fan narrative. You see a metric’s creator say, “My metric rates this player worse when I add box components,” and rather than engage with that concept, your reaction is to dump a compilation of irrelevant other metrics in a transparent slander attempt. That is pure ideology, but characteristically, you prefer to pretend it is actually something everyone else does.

I hope you realize that compiling all available data is virtually definitionally the opposite of “advancing a chosen fan narrative.”

If you did it comprehensively and neutrally, sure. But time and time again, you act selectively and emphasise what is most favourable to the narrative you want to push.

Maybe if a “compilation” of data suggests that you’re wrong about a given viewpoint on a player, you should consider whether you may actually just be wrong,

Or maybe you should stop conflating outputs as something they are not.

rather than objecting to the presentation of fulsome information and/or retreating to the use of a particular metric or two that gives you output you like.

I thought you said you were being comprehensive?

I have repeatedly said I do not care much about all-in-ones, although not in the sense that I see them all equally. My issue is on the emphasis and on the conflation of them with some idea of objective player quality when they are all still highly dependent on circumstance (as I have repeatedly gone over).

As for the fact that the MAMBA creator says that LeBron gets worse in his metric the more you weight the prior, I have no doubt that that’s true, but I find it extremely odd that you suggest I did not “engage with that concept.” I engaged with it by pointing out that, given that we have lots of RAPM output that is not consistent with that (at least as it relates to the specific topics you and I have discussed), that seems to just be a relic of the specific type of RAPM that the creator is using. This seems obviously true. If we have a bunch of multi-year RAPM data sets and MAMBA tends to be a little bit more favorable to LeBron than that available RAPM data, then it is pretty obviously the case that the only way the MAMBA prior pulls LeBron down from RAPM more than anyone else is if the RAPM that MAMBA uses is much more favorable to LeBron than a bunch of other RAPM data we have. So the MAMBA creator’s statement basically just means that the RAPM methodology he uses is very high on LeBron compared to other RAPM. And that’s a good fact for LeBron (though we don’t know how good, since I don’t think we actually have just the RAPM, and he doesn’t look the best once we add the prior). We don’t actually know what of the many versions of RAPM is the best, though.

And why do you think he — and indeed all these metrics, regardless of their actual “preference” — would use a different form of RAPM from what you cite from github or thebasketballdatabase?

[Sofia]

Next rating metric will be named WEMBY

I’m hanging out for the ANTETOKOUNMPO rating myself, which combines effort estimates on court with production.

Analytical Net True Efficient Tracking Of Kinetic Outputs, & Unweighted Net Maximum Player Outputs

[lessthanjake]

Because I am not in the business of compiling lists of outputs for the purpose of advancing a chosen fan narrative. You see a metric’s creator say, “My metric rates this player worse when I add box components,” and rather than engage with that concept, your reaction is to dump a compilation of irrelevant other metrics in a transparent slander attempt. That is pure ideology, but characteristically, you prefer to pretend it is actually something everyone else does.

I hope you realize that compiling all available data is virtually definitionally the opposite of “advancing a chosen fan narrative.”

If you did it comprehensively and neutrally, sure. But time and time again, you act selectively and emphasise what is most favourable to the narrative you want to push.

Lol, okay. This is just a really odd response to me, given that I’m more insistent than probably anyone on these forums that we should look at *all* available data. The “compilations” of data I’ve provided have often included individual metrics that, taken by themselves, would *not* support the position I’m arguing. But the data picture as a whole does, and that is what I argue is most important, especially given that individual metrics will have various forms of error and flaws. If the standard is that I must be aware of and immediately recall every single data point in existence before compiling data in a post, then I’m sure I’ve not always met that (nor would that be reasonable to expect). And you’re more than free to point out data points I’ve missed, anytime I’ve compiled data. But the idea that I’ve systematically excluded data points that go against my views is actually demonstrably untrue. And I think you probably know that.

Meanwhile, it is people you typically agree with (and posts that you And-1) that explicitly argue that I’m wrong to compile data and to look at that entire picture. They instead argue that we should look at only information that conveniently has results they like the most—accompanied by some ham-fisted argument about why that piece of information is the only reliable data. It is very clearly not me that “selectively . . . emphazise[s] what is most favorable to the narrative” being pushed. People you agree with very explicitly do that, and I very explicitly push back on that approach, with a lot of ink having been spilt on many threads on these forums about exactly this disagreement in terms of approach. You truly are barking up the wrong tree.

rather than objecting to the presentation of fulsome information and/or retreating to the use of a particular metric or two that gives you output you like.

I thought you said you were being comprehensive?

What is your point? Me presenting fulsome information that, taken as a whole, leads to a particular conclusion is really not mutually exclusive with the existence of individual data points that are less favorable to that conclusion. That’s the whole point of looking at a compilation of data rather than just one or two individual data points! If every data point always said the same thing, then it wouldn’t be necessary!

I have repeatedly said I do not care much about all-in-ones, although not in the sense that I see them all equally. My issue is on the emphasis and on the conflation of them with some idea of objective player quality when they are all still highly dependent on circumstance (as I have repeatedly gone over).

Agreed that these metrics are dependent on circumstances. This is something I’m more than happy to talk about in any individual instance. And, indeed, I’ve talked about it many times before. For instance, I’ve talked about how, in terms of impact data, it is more favorable for a star player if the team’s offensive system and roster is specifically built to maximize him, rather than it being built to maximize the rest of the roster while assuming that the star player will still eat regardless. I think we should all probably be able to agree with that general statement, even if we might disagree on exactly how that concept applies to specific circumstances. I’ve also talked about how I think that, in a vacuum, being on a worse team may be better for purposes of impact data, because even if you control for how good a player’s teammates are, a player will actually have more effect on a bad team than a good team (because they’re filling in more gaping holes), and impact data is essentially trying to isolate out the impact of that individual player. That’s a conclusion that I’m less certain about but it makes intuitive sense to me. I’ve also talked a lot about the mental effect of teams not really trying to contend, as well as the effect on impact of things like the rubber-band effect. I could go on. I’ve talked very extensively on these forums about circumstances affecting the output of impact data.

These are all interesting topics, and there’s surely more contextual issues than those. You’ll find that I’ve been very consistent in saying that impact data is all flawed and prone to error (something that people you consistently agree with have somehow taken issue with). The fact that I’ve compiled large amounts of data reflects the fact that I think a compilation of data gives a data picture that minimizes flaws and error in the data as much as possible. But I’ve definitely never said data is perfect or all we should look at. Very much to the contrary! There may be contextual factors that affect *all* impact data, which is something that looking at all data would not account for.

To tie things back a bit, if you want to respond to a compilation of data I provide by arguing that that data goes a certain direction due to contextual factors that essentially bias all the data, then that would probably be a good discussion! Unfortunately, that’s very rarely how you or others react to compilations of data. The most common reaction is instead to say that we should only cherry-pick out certain specific pieces of data instead of looking at the whole picture, or to quibble with the compilation in some trivial way.

As for the fact that the MAMBA creator says that LeBron gets worse in his metric the more you weight the prior, I have no doubt that that’s true, but I find it extremely odd that you suggest I did not “engage with that concept.” I engaged with it by pointing out that, given that we have lots of RAPM output that is not consistent with that (at least as it relates to the specific topics you and I have discussed), that seems to just be a relic of the specific type of RAPM that the creator is using. This seems obviously true. If we have a bunch of multi-year RAPM data sets and MAMBA tends to be a little bit more favorable to LeBron than that available RAPM data, then it is pretty obviously the case that the only way the MAMBA prior pulls LeBron down from RAPM more than anyone else is if the RAPM that MAMBA uses is much more favorable to LeBron than a bunch of other RAPM data we have. So the MAMBA creator’s statement basically just means that the RAPM methodology he uses is very high on LeBron compared to other RAPM. And that’s a good fact for LeBron (though we don’t know how good, since I don’t think we actually have just the RAPM, and he doesn’t look the best once we add the prior). We don’t actually know what of the many versions of RAPM is the best, though.

And why do you think he — and indeed all these metrics, regardless of their actual “preference” — would use a different form of RAPM from what you cite from github or thebasketballdatabase?

There are plenty of logical reasons to prefer one type of RAPM over another, because there are a lot of methodological decisions involved. For instance, do we apply a minutes cutoff for players who didn’t play much (and if so, how much of one)? Do we apply a luck adjustment, and if so, what are we assuming is simply the product of luck? Do we apply an adjustment for the rubber band effect? Do we include some kind of basic prior? And, if we do include a basic prior, what is it based on? Is it based on minutes played? Is the prior based on the prior years’ RAPM? If it’s based on the prior years’ RAPM, how are you weighing each year? Are years that are further back given less weight, and what are those weights and how far are you going back? There’s of course a lot of these decisions. It is in no way clear what is the “right” answer on these methodological questions. And that’s the point! Assuming that one particular form of RAPM is the objective truth is definitely pretty tenuous, especially as it relates to a conclusion that is inconsistent with a lot of other forms of RAPM.

[AEnigma]

I hope you realize that compiling all available data is virtually definitionally the opposite of “advancing a chosen fan narrative.”

If you did it comprehensively and neutrally, sure. But time and time again, you act selectively and emphasise what is most favourable to the narrative you want to push.

Lol, okay. This is just a really odd response to me, given that I’m more insistent than probably anyone on these forums that we should look at *all* available data. The “compilations” of data I’ve provided have often included individual metrics that, taken by themselves, would *not* support the position I’m arguing. But the data picture as a whole does, and that is what I argue is most important, especially given that individual metrics will have various forms of error and flaws. If the standard is that I must be aware of and immediately recall every single data point in existence before compiling data in a post, then I’m sure I’ve not always met that (nor would that be reasonable to expect). And you’re more than free to point out data points I’ve missed, anytime I’ve compiled data.

And I regularly do.

But the idea that I’ve systematically excluded data points that go against my views is actually demonstrably untrue. And I think you probably know that.

Systematically exclude, no. Systematically talk down or ignore, yes.

Meanwhile, it is people you typically agree with (and posts that you And-1) that explicitly argue that I’m wrong to compile data and to look at that entire picture.

Compiling composites is not compiling new data, it is compiling new formulas that interpret the data.

They instead argue that we should look at only information that conveniently has results they like the most—accompanied by some ham-fisted argument about why that piece of information is the only reliable data.

Not really, no, but that is of course how you like to portray it.

It is very clearly not me that “selectively . . . emphazise[s] what is most favorable to the narrative” being pushed. People you agree with very explicitly do that, and I very explicitly push back on that approach, with a lot of ink having been spilt on many threads on these forums about exactly this disagreement in terms of approach. You truly are barking up the wrong tree.

No, I generally find them consistent regardless of whether I agree with their approach in every instant. It is why I have stopped habitually harping on those who seem to start and end their analysis with some composite of choice: regardless of whether I agree, they are not trying to manipulate anything.

rather than objecting to the presentation of fulsome information and/or retreating to the use of a particular metric or two that gives you output you like.

I thought you said you were being comprehensive?

What is your point? Me presenting fulsome information that, taken as a whole, leads to a particular conclusion is really not mutually exclusive with the existence of individual data points that are less favorable to that conclusion. That’s the whole point of looking at a compilation of data rather than just one or two individual data points! If every data point always said the same thing, then it wouldn’t be necessary!

My point is you are not comprehensive. You share what you like in selective ranges and try to paint it as more comprehensive than it is, then when excluded information is highlighted, you argue against it. And you do this because your interest is in how you can advocate for or against specific players or choice, not out of anything remotely principled.

I have repeatedly said I do not care much about all-in-ones, although not in the sense that I see them all equally. My issue is on the emphasis and on the conflation of them with some idea of objective player quality when they are all still highly dependent on circumstance (as I have repeatedly gone over).

Agreed that these metrics are dependent on circumstances. This is something I’m more than happy to talk about in any individual instance. And, indeed, I’ve talked about it many times before. For instance, I’ve talked about how, in terms of impact data, it is more favorable for a star player if the team’s offensive system and roster is specifically built to maximize him, rather than it being built to maximize the rest of the roster while assuming that the star player will still eat regardless. I think we should all probably be able to agree with that general statement, even if we might disagree on exactly how that concept applies to specific circumstances. I’ve also talked about how I think that, in a vacuum, being on a worse team may be better for purposes of impact data, because even if you control for how good a player’s teammates are, a player will actually have more effect on a bad team than a good team (because they’re filling in more gaping holes), and impact data is essentially trying to isolate out the impact of that individual player. That’s a conclusion that I’m less certain about but it makes intuitive sense to me. I’ve also talked a lot about the mental effect of teams not really trying to contend, as well as the effect on impact of things like the rubber-band effect. I could go on. I’ve talked very extensively on these forums about circumstances affecting the output of impact data.

These are all interesting topics, and there’s surely more contextual issues than those. You’ll find that I’ve been very consistent in saying that impact data is all flawed and prone to error (something that people you consistently agree with have somehow taken issue with).

No, again, I find that you are very consistent in emphasising what supports your narrative in the moment and working to dismiss or ignore that which does not for arbitrary and poorly substantiated reasons which certainly do not reflect someone sincerely interested in holistic assessment.

The fact that I’ve compiled large amounts of data reflects the fact that I think a compilation of data gives a data picture that minimizes flaws and error in the data as much as possible. But I’ve definitely never said data is perfect or all we should look at. Very much to the contrary! There may be contextual factors that affect *all* impact data, which is something that looking at all data would not account for.

To tie things back a bit, if you want to respond to a compilation of data I provide by arguing that that data goes a certain direction due to contextual factors that essentially bias all the data, then that would probably be a good discussion!

The discussion has been had and was ignored.

Unfortunately, that’s very rarely how you or others react to compilations of data. The most common reaction is instead to say that we should only cherry-pick out certain specific pieces of data instead of looking at the whole picture, or to quibble with the compilation in some trivial way.

Trivial here meaning pointing out how you highlighted everything that supported what you want “the data” to say and minimised everything that did not.

As for the fact that the MAMBA creator says that LeBron gets worse in his metric the more you weight the prior, I have no doubt that that’s true, but I find it extremely odd that you suggest I did not “engage with that concept.” I engaged with it by pointing out that, given that we have lots of RAPM output that is not consistent with that (at least as it relates to the specific topics you and I have discussed), that seems to just be a relic of the specific type of RAPM that the creator is using. This seems obviously true. If we have a bunch of multi-year RAPM data sets and MAMBA tends to be a little bit more favorable to LeBron than that available RAPM data, then it is pretty obviously the case that the only way the MAMBA prior pulls LeBron down from RAPM more than anyone else is if the RAPM that MAMBA uses is much more favorable to LeBron than a bunch of other RAPM data we have. So the MAMBA creator’s statement basically just means that the RAPM methodology he uses is very high on LeBron compared to other RAPM. And that’s a good fact for LeBron (though we don’t know how good, since I don’t think we actually have just the RAPM, and he doesn’t look the best once we add the prior). We don’t actually know what of the many versions of RAPM is the best, though.

And why do you think he — and indeed all these metrics, regardless of their actual “preference” — would use a different form of RAPM from what you cite from github or thebasketballdatabase?

There are plenty of logical reasons to prefer one type of RAPM over another, because there are a lot of methodological decisions involved. For instance, do we apply a minutes cutoff for players who didn’t play much (and if so, how much of one)? Do we apply a luck adjustment, and if so, what are we assuming is simply the product of luck? Do we apply an adjustment for the rubber band effect? Do we include some kind of basic prior? And, if we do include a basic prior, what is it based on? Is it based on minutes played? Is the prior based on the prior years’ RAPM? If it’s based on the prior years’ RAPM, how are you weighing each year? Are years that are further back given less weight, and what are those weights and how far are you going back? There’s of course a lot of these decisions. It is in no way clear what is the “right” answer on these methodological questions. And that’s the point! Assuming that one particular form of RAPM is the objective truth is definitely pretty tenuous, especially as it relates to a conclusion that is inconsistent with a lot of other forms of RAPM.

Dodge the question better.

[lessthanjake]

.

This entire post is just filled with vague accusations that are not substantiated and are effectively impossible to respond to. Your characterizations are incorrect, but this relates to countless prior discussions and I’d just say that others can read through my post history and the posts responding to me and see what they think in any individual instance.

The only thing that seems even possible to meaningfully respond to is the last part where you suggest I’ve “[d]odged the question” about RAPM sources. You seem to be implying that there’s something wrong with certain sources of RAPM that I’ve referred to. The forms of RAPM you identify definitely do not include every form of RAPM I’ve pointed to, but let’s take one example you specifically mentioned. TheBasketballDatabase RAPM specifically says the following: “The values presented on this site are ‘vanilla’ RAPM - the result of a pure ridge regression without any priors to tell the regression what the results ‘should’ look like.” So it presents RAPM with no priors. Are you suggesting that that’s not a good approach? I think there’s arguments that “vanilla” RAPM is not the best way to go. But there’s definitely arguments in favor of it too! And that’s especially true the more years we’re looking at (since priors are mostly useful to help the data perform better in smaller samples). Of course, even with “vanilla” RAPM, there’s other methodological decisions that can affect the results—including things I mentioned earlier, like minutes thresholds, luck adjustments, etc. So TheBasketballDatabase RAPM will be at least a little different from other “vanilla” RAPM. But I’ve also cited other multi-year “vanilla” RAPM measures such as the NBAshotcharts one. Overall, we actually don’t really know whether “vanilla” RAPM is better or if it’s better to have a prior (and, if so, what prior is best), nor do we know what is better in regards to other methodological choices.

Also, to take a step back, practically speaking, the reason that various all-in-ones don’t specifically use this RAPM data is that they typically run *their own* RAPM rather than taking RAPM from any other particular source. And this may perhaps be what you’re getting at. I don’t think something like TheBasketballDatabase has published its raw data (though I *believe* NBAshotcharts has, actually), so you can’t really use it to build another metric. Relatedly, while it includes a description of what it is, it does not explain every little methodological decision in a way that makes it clear *exactly* what it did. Does that mean we shouldn’t care about it or should assume it is untrustworthy? I guess you’re free to think so, but I think that’s overly cynical (not to mention that it would exclude essentially all data we have). It also definitely wouldn’t be in keeping with you and others you typically agree with sometimes citing things like Engelmann’s RAPM (which is definitely a black box that could never be used by an all-in-one).

[OhayoKD]

Mamba's creator made another post which includes a response to some of the discussion here:
https://www.teemohoop.com/mamba/mamba-2024-25
(under "Also a Strange Comment someone told me about")


Seems there's some stuff to clear-up

I’ve pointed out that multi-year RAPM—which is what these all-in-one metrics are tested against (usually 5-year RAPM in particular)—doesn’t make LeBron look any better for purposes of those discussions.

That doesn't seem to be the case for the RAPM MAMBA's using:

My reasoning for creating an all in one was I thought my approach could help somewhat offset the bias you get from priors in all in one metrics. I think it did so a little bit but certainly not completely and I think in hindsight a multilayered approach might make more sense than this, but it definitely proved to me that the bias there exists. It was somewhat shocking to see how much player’s would move the less I weighted the box score (and thus weighted box score impact more). You genuinely could get to points where Lebron was just first place all the way through the entire warriors cavs era aside from 2018 instead of Steph being the highest one in general, without even including playoffs with a still reasonable weight on the box score but just relatively more weight than normal on on court impact for an all in one metric.

...

I recall I was curious to see how “playoff mode” bron looked in this type of data from 2015-2017, so I tried compiling that data and it truly was completely ludicrous, I don’t remember. It was fairly noisy, but I remember the constant was Lebron was something insane like a +14 or +15 in a data set where no one generally exceeded +12, so that was crazy, obviously playoff data is too noisy to post an all in one playoff metric or something like that, but him being that far ahead of everyone in the dataset by a country mile was pretty funny, most people’s results had been someone muted compared to their regular season results.

...

Based upon it being such a consistent trend and the sheer magnitude of it, he’s very clearly someone underrated by this kind of thing. With how much of Curry’s impact comes from outside of the box score, I expected him to be somewhat underrated by the box score priors relative to his impact, but his priors were consistently fantastic with the ones I made even in years his raw RAPM impact were not as strong, so that was very interesting, to see he didn’t have that issue of the model potentially underrated him the same way.

Also will highlight the provided "why" for the model being adjusted:

This is just very peculiar, but I guess I’ll try to respond to it. The only real change I made on offense that impacted Lebron was an adjustment of how I used synergy data and adding in transition points over expectation, which ended up causing a huge boost in terms of accuracy. Outside of that, I mainly fixed some minor mistakes I had in the prior when compiling the box score data, and then did what EPM did in terms of setting certain limits to some data that could be extremely high or low based upon noise.

I mean, I think the name Lebron when referring the the player came up once outside of when I read out the results where I would be like: Yeah well now the guy that everyone knew was the best player in the world is now higher than George Hill so, i mean, its just a bit of a weird accusation

So to make something clear, I adjusted how I tested the metrics in terms of how I dealt with replacement players, so the testing metrics are different between posts (as you can see with EPM for example when you look at it). I had the Old MAMBA data when I was testing the projections, and the improvement between the old MAMBA data and the new MAMBA data was roughly the same as how much better old MAMBA tested compared to EPM when I did the projections the old way.

Make of that what you will.

[lessthanjake]

Mamba's creator made another post which includes a response to some of the discussion here:
https://www.teemohoop.com/mamba/mamba-2024-25
(under "Also a Strange Comment someone told me about")


Seems there's some stuff to clear-up

I’ve pointed out that multi-year RAPM—which is what these all-in-one metrics are tested against (usually 5-year RAPM in particular)—doesn’t make LeBron look any better for purposes of those discussions.

That doesn't seem to be the case for the RAPM MAMBA's using:

Yeah I think if you actually read the discussion we’ve been having, you’d find that we were all in agreement about that already since there was no other way to logically square what the MAMBA creator had said. The issue is just that this strongly suggests the RAPM that MAMBA uses is just quite a lot higher on LeBron than many other RAPM sources we have, and we have no idea what the best way to do RAPM is. FWIW, in various posts, the MAMBA creator has said that his version of RAPM is very similar to PI RAPM and also doesn’t include things like rubber band adjustments. From what I’ve seen, PI RAPM tends to be better for LeBron—especially for the early parts of this time period. And that makes sense, since that sort of RAPM will tend have a lagged effect from LeBron’s very best years, while doing the opposite for someone like Steph. But maybe PI RAPM (or this time decay methodology that is similar) is better? Who knows! Perhaps even more importantly, not adjusting for the rubber band effect is quite likely a decision that’s materially bad for prime Steph, since his teams were so good in games he played. So, given what we know, I think it’s not surprising that the MAMBA creator’s form of RAPM would be materially better for LeBron compared to Steph Curry than other forms—there’s methodological decisions that we’d expect to be to LeBron’s relative benefit, and it’s obviously the case that it is indeed better for LeBron than a lot of other forms of RAPM we have that used different methodology. We ultimately don’t know what is right, since we can’t really be sure what methodological decisions are correct. I do tend to think that adjusting for the rubber-band effect is almost certainly a good thing, though, but I guess even that is theoretically debatable since it’s not entirely clear *how* to adjust for it.

Anyways, of course this all goes to my frequent statement that we should look at all data. We don’t know what methodology is best, so the best way to look at the data picture is to try to zero out the effect of methodological differences as much as possible by looking at everything we have. Obviously there’s some ways to do RAPM that are favorable to LeBron over Steph. That’s not true of a lot of other forms of RAPM, though—with many being really favorable to Steph in this comparison. Meanwhile, with all-in-ones, box priors are added to RAPM to increase accuracy (including in MAMBA itself!), so they may well be better! And those also tend to be favorable to Steph over LeBron—including MAMBA itself! Your retort to this is often to try to point out instances where the non-box portion of these all-in-one metrics are favorable to LeBron, ignoring the fact that the metric has a box portion in order to improve it! So you’re just pointing to LeBron doing better in data that is understood to be worse, while Steph looks better in the better data. Obviously not very convincing! And if we engaged with the point further than that, we’d realize that all your point means is basically that there’s a form of single-season RAPM that LeBron looks better in, while we know that there are multiple versions of multi-year RAPM that say otherwise anyways. The overall impact data picture is squarely in Steph’s favor during his prime years. The fact that the MAMBA creator’s version of RAPM is apparently favorable to LeBron, while his actual metric that is built to be better than that is favorable to Steph is not actually a good fact for you, but it’s just one of the many data points we have.