Real Plus Minus & Other Bogus Stats

[payitforward]

I've forgotten where we were discussing RPM. &...I've forgotten who was pushing it as an outstanding player metric.

Wherever it was, I do recall promising to read & comment on the following article:

https://cornerthreehoops.wordpress.com/2014/04/17/explaining-espns-real-plus-minus/

I've now had time to do so, & I am happy to confirm my first impressions from a previous skim: RPM is completely bogus. It uses a lot of sophisticated statistical techniques to massage data in a way that provides no useful information whatever. Thus, read the following:

(A team's) player RPMs over the current season... will NOT sum to the team’s actual performance.

Is it clear to you what that means? Here's an easy example: suppose I have a starter at, say, SF who has a lousy RPM, & I go out looking for for a different SF, a guy who has an outstanding RPM -- someone whom RPM says is a much better player.

Lucky me, I find him. Better yet, I am able to trade my low-RPM starter straight up for this guy with a high RPM. Great!

Should I expect that I've improved my team? Should I expect to win more games now that I've got this guy who is so much better than my previous guy? In other words, if a guy with a high RPM plays, say, 30 minutes a game instead my old starter, who had a low RPM, can I expect my team to improve?

RPM's answer is "no." Higher RPM does not correlate with a higher number of wins.

Great, huh?

I also promised I'd read this article: https://fansided.com/2019/01/08/nylon-calculus-best-advanced-stat/

I did. It's no more helpful in giving anyone any reason to pay any attention whatever to ESPN's product Real Plus Minus.

Now, I've got an assignment for you: if you want to understand why RPM is, essentially, meaningless, read this:

https://www.clevescene.com/scene-and-heard/archives/2016/05/04/real-plus-minus-the-bogus-new-stat-everyones-using

It's not a great essay, but it does most of what's needed to cure a person of the RPM addiction.

[doclinkin]

Whereas you only need a 2 word phrase to cure someone of the "WIN score" boondoggle:

Joey Dorsey.

As in:

Memphis’s Joey Dorsey just plain puts up amazing Win Scores. Looking at his history, you’ll understand why I sighed a big breath of relief as Dorsey fouled out of the National Title Game. Dorsey, not Derrick Rose, was the engine that drove the Memphis Tigers through the regular season, and both players stepped up for a fantastic tournament run. Rose may go 30 picks earlier, but Dorsey can well prove to be the better value given the cheap projected price of a 2nd rounder.

Followed by:

Win Scores may be optimistic on Beasley and Love, but how do they measure up historically? To answer this, I dug up last year’s numbers and ran a quick comparison between Win Scores favorite big men, 2007 versus 2008. ...

It appears Beasley and Love grade out solidly higher than Oden & Durant. In general, the class of 2008 generally can claim higher Win Scores than the class of 2007.

[NatP4]

This is not worthy of an entire thread. Don’t like where this is heading in regards to the board as a whole.

[nate33]

This is not worthy of an entire thread. Don’t like where this is heading in regards to the board as a whole.

???

If you don't like the subject, don't click on the thread.

[Dat2U]

I feel like Payit must be in a good mood because he's looking to start a fight lol. No stat is perfect and over reliance on one particular stat or simple productivity measurements is flawed and will lead to wonky analysis that likely won't age well. This goes for any statistical measurement. PER, PIPM, BPM, RPM, WS40, WP40, etc, etc, etc. Position matters, role matters, eye test matters (especially defensively. Basically its a very small piece in a big picture.

[payitforward]

Whereas you only need a 2 word phrase to cure someone of the "WIN score" boondoggle:
Joey Dorsey.

As in:

Memphis’s Joey Dorsey just plain puts up amazing Win Scores. Looking at his history, you’ll understand why I sighed a big breath of relief as Dorsey fouled out of the National Title Game. Dorsey, not Derrick Rose, was the engine that drove the Memphis Tigers through the regular season, and both players stepped up for a fantastic tournament run. Rose may go 30 picks earlier, but Dorsey can well prove to be the better value given the cheap projected price of a 2nd rounder.

Followed by:

Win Scores may be optimistic on Beasley and Love, but how do they measure up historically? To answer this, I dug up last year’s numbers and ran a quick comparison between Win Scores favorite big men, 2007 versus 2008. ...

It appears Beasley and Love grade out solidly higher than Oden & Durant. In general, the class of 2008 generally can claim higher Win Scores than the class of 2007.

doc raises a good point -- tho he raises it indirectly & dismissively, which may lead to a non-rational tone of discussion. Let me see if I can re-state his point in what (to me at least) seems a more productive way.

"Win Score" made Joey Dorsey look like a good player in college, but it turned out he was not a good player in the NBA. This is absolutely correct on both counts.

As to doc's other example: Beasley & Love did look like better players than Oden & Durant! Beasley's problems had nothing to do with his basketball abilities, which were absolutely amazing. They had to do with his character & behavior.

Beasley came out after 1 year of college. The only player I've ever seen with as good Freshman numbers as Michael Beasley is Zion Williamson.

Durant also came out after 1 year & he also posted terrific numbers as a Freshman -- but, not at the level of Zion or Beasley.

[payitforward]

So... why did Win Score bomb so badly on Joey Dorsey?

Because he played in a weird way that fooled it: Dorsey only took 7 shots per 40 minutes, most of them putbacks off of the 6 offensive rebounds he got in the same 40 minutes. As a result, he shot 65% on FGs. Add the 9 defensive boards he also got, & his numbers start to look really good! &, of course, he didn't commit many turnovers either -- since he basically didn't touch the ball on offense except when he got an offensive board!

Dorsey was drafted #33 in 2008. Guys in the league were bigger than him & faster than him, & the rules were different too. He couldn't do the things he had done so well in college. Duh.

OTOH, a bigger, faster, stronger version of him was picked 2 picks later -- DeAndre Jordan -- & he was an enormous success. There were 12 Centers taken before Jordan that year. I doubt most people here could name 5 without looking back at that draft. IOW, Win Score was right about Jordan.

[payitforward]

Here is the key overall point:

Statistical measures are not substitutes for, or perfect versions of, actual reality. They are tools, that is all. You can figure out whether one of them is better overall or worse overall than some other one of them. That's all. No such tool gives you "the truth" -- especially not "the truth every time."

How can you tell whether one of these tools is better than another overall? It should be obvious....

[montestewart]

I feel like Payit must be in a good mood because he's looking to start a fight lol. No stat is perfect and over reliance on one particular stat or simple productivity measurements is flawed and will lead to wonky analysis that likely won't age well. This goes for any statistical measurement. PER, PIPM, BPM, RPM, WS40, WP40, etc, etc, etc. Position matters, role matters, eye test matters (especially defensively. Basically its a very small piece in a big picture.

Not sure how you would create such a metric, but I can see the worth in a system whose individual measure totals don't necessarily approximate team success precisely because it attempts to isolate individual abilities to determine whether a player's skills (and metric measures) are strengthened or weakened by current team, position, role, etc.

[pcbothwel]

Here is the key overall point:

Statistical measures are not substitutes for, or perfect versions of, actual reality. They are tools, that is all. You can figure out whether one of them is better overall or worse overall than some other one of them. That's all. No such tool gives you "the truth" -- especially not "the truth every time."

How can you tell whether one of these tools is better than another overall? It should be obvious....

To the Dorsey point (And the same point I had about Otto Porter)... RPM, and others, doesnt tell you what a player can potentially do... It tells you good the player at what they are ASKED to do.

In the A-Z toolbox a player may only have A-C and may be missing the other 23 tools, but if they are only asked to perform A-C and do so well, these "catch all" stats show that player to be prolific.

Otto was a great example of that. He was asked to hit open shots, rebound, play elite team defense, and good man defense. And he proved to be GREAT at all those things, which is why RPM rated him as a top 3-5 Small Forward.

He wasn't asked to create, Lock down the best perimeter defender, or drive to the basket.

[doclinkin]

So... why did Win Score bomb so badly on Joey Dorsey?

Because he played in a weird way that fooled it: Dorsey only took 7 shots per 40 minutes, most of them putbacks off of the 6 offensive rebounds he got in the same 40 minutes. As a result, he shot 65% on FGs. Add the 9 defensive boards he also got, & his numbers start to look really good! &, of course, he didn't commit many turnovers either -- since he basically didn't touch the ball on offense except when he got an offensive board!

System inflated his numbers, and in my opinion WIN score overrates offensive boards. The dribble drive motion offense that Calipari runs is predicated on 4 ball handlers on the perimeter who attack any mismatch, while a single big posts on the weakside baseline and crashes the glass on every drive. It makes giants out of the Dorsey and DeMarcus players who are isolated down low, since their only involvement in the game is to bang against a single big on a putback attempt, or receive a pass if the attack is stopped and the ballhandler elects not to pass out, or bring it outside themselves and reset the offense. The other defensive 'big' is chasing around the perimeter, and the Big opposing Dorsey or DeMarcus or whomever is usually on the strong side trying to deter that penetration. Bigs in this system are set up to succeed in one particular stat. Well, rebounding at both ends since the opponents often elect to put a perimeter defender out there. So the Big has only one Big to beat at either end. If you are a significantly good one on one rebounder, you appear as a world class dominant one in this system.

To the other point. Offensive board focus gives up transition defense. It gives up defense in general since the boardsman is trailing behind the action instead of establishing early position at the other end. I could be wrong but understand WIN score rates O-boards higher than defense, where it is my impression that players who have a high defensive board effect (either in team rebounding % numbers or on their own) prove to be better NBA players than bouncy long athletes who hustle for offensive boards.

[payitforward]

I feel like Payit must be in a good mood because he's looking to start a fight lol. No stat is perfect and over reliance on one particular stat or simple productivity measurements is flawed and will lead to wonky analysis that likely won't age well. This goes for any statistical measurement. PER, PIPM, BPM, RPM, WS40, WP40, etc, etc, etc. Position matters, role matters, eye test matters (especially defensively. Basically its a very small piece in a big picture.

I am in a good mood -- but, I'm not looking to start a fight. Thus...

Everything you write above is totally correct. Plus, situation also matters.

Not only is no stat perfect, but the concept of a stat being perfect is flawed. The only question is between one stat (or roll up of stats) & another: which one does a better job of accounting for team results?

But even if there was some fabulous new statistical roll-up that correlated 100% with a team's record, that wouldn't enable us to say that this methodology was 100% perfect. Why? Because any of these tools -- whether it's PER or Win Shares or WP48 -- is assessing players based on the numbers they put up individually. But some of what makes a team successful comes from interactions which are virtually impossible to quantify numerically.

That's true above all with respect to defense, because (as one guy writes) "...even with all the stuff we have (i.e. statistical data) we have, it’s still very hard to ultimately assign credit for these things. Because players are switching all of the time. Teams play defense as a unit.”

But, it's not just defense -- it's also the somewhat loosely-defined set of things that we describe under the term "fit." E.g. spacing -- & more. Hence, not even the best of these tools tells the whole story.

But, that doesn't make them useless -- far from it!

Except for one of them, that is....

[payitforward]

Except for one of them, that is....

There is one problem with what you write, dat, & that is your inclusion of RPM in the list of these tools.

In fact, RPM is not useful at all in evaluating player performance.

Let me repeat for clarity: RPM tells you nothing -- zip. nada. -- about how good NBA players are. You think that can't be? Read this:

Spoiler:

First, let's get obvious stuff out of the way: does RPM get the very best guys more or less right? Does it rank them up top? Why, yes, it does.

Then again... how hard is that really? do I really need much help from math & software to know that Giannis is the best player in the league? Plus, did you notice that I used the phrase "more or less" just above?

Tell me, do you think that in 2019-20, Kyle Lowry was a better player than Kawhi Leonard?

Do you think Brook Lopez was better than Bam Adebayo? How about Russell Westbrook over Jimmy Butler?

You agree? Well then, perhaps you also agree that...

Daniel Theis was better than Anthony Davis?

Oh, & if you don't think Theis was better than Davis, maybe you'd still agree with RPM that Shaquille Harrison was better than Anthony Davis? Joe Ingles? D.J. Augustin?

All those guys were better than Anthony Davis? All of them better than Bam too? All better than Devin Booker?

Do you agree?

Great! Then try this one on for size:

We know that Chris Chiozza didn't play nearly as many minutes as Brad Beal, obviously, but in the minutes he did play, do you think he was better than Beal? RPM does. In fact, it thinks he was about 6.5 times as effective as Brad.

Not just Chris -- although don't get me wrong, RPM has Chris in the top 10% of players in the league -- but also Alec Caruso, Reggie Jackson, Kris Dunn, J.J. Barea, Norman Powell, Devonte Graham, Chris Clemons, Jaylen nowell, Terence Davis, John Konchar, Damion lee, Evan Fournier... all better than Bradley Beal. Better than Jamal Murray too.

How about Dillon Brooks, Ryan Broekhoff, David Nwaba... better than Beal?

Finally, tell me this, do you think Caleb Martin, an undrafted rookie who struggled to get any time on the floor for Charlotte, was a more effective NBA player than Bradley Beal in 2019-20?