RAPTOR adjusts data to fit perceived consensus

[freethedevil]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:

"it would be nice to have a metric that looks at how teams value players."

Then he says...

"more often than not the consensus is right"

Does this make sense? Should all in one metrics adjust data to fit a general consensus?

[GeorgeMarcus]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:

"it would be nice to have a metric that looks at how teams value players."

Then he says...

"more often than not the consensus is right"

Does this make sense? Should all in one metrics adjust data to fit a general consensus?

I vote no, but the allure of an all-in-one metric is kind of a pipe dream to begin with IMO. What we really need is comprehensive RAPM data.

[Pennebaker]

That's not science.

[dhsilv2]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:

"it would be nice to have a metric that looks at how teams value players."

Then he says...

"more often than not the consensus is right"

Does this make sense? Should all in one metrics adjust data to fit a general consensus?

I vote no, but the allure of an all-in-one metric is kind of a pipe dream to begin with IMO. What we really need is comprehensive RAPM data.

More versions and consistent version of RAPM are absolutely what we need. I don't understand why Nate's not gone that direction. These all in one metrics are nice, they have some value, but they're not moving the needle. They should take the additional data from nba.com and create a more comprehensive box score metric and then show that with the fuller RAPM data, and then you can do your own analysis from there. Instead....they want to put it all in one.

[zimpy27]

Yeah the constant focus on a all-in-one metric is pushing the money towards stats guys who could do a much better job by isolating important aspects of the game.

I'd love if it you could have ranks for individual game aspects and then work on relationship-based algorithms to determine what players fit well together.

[freethedevil]

Yeah the constant focus on a all-in-one metric is pushing the money towards stats guys who could do a much better job by isolating important aspects of the game.

This is silly. We've seen plaenty of statistical advancement on "singular aspects of the game." There's no reason to chooose either/or

[Bum Adebayo]

Consensus approach is blasphemy.

[MarcusBrody]

The exciting part about the early application of advanced statistics was that they revealed some players/skills that were undervalued by consensus at the time. I understand wanting to use it to minimize variability, but it seems like it would just make the metrics much more conservative by nature and so limit their independent utility.

[ProcessDoctor]

Their model was dead on last year (TOR-GSW Finals). I don't know why they felt the need to change it, especially if they included a non-objective variable.

[Lalouie]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:

"it would be nice to have a metric that looks at how teams value players."

Then he says...

"more often than not the consensus is right"

Does this make sense? Should all in one metrics adjust data to fit a general consensus?

it makes absolute sense.

"general consensus" is probably more commonly known as the eyeball test and the eyeball test works. analytical metrics is just trying to codify what the eyeball already knows. while i got bored and stopped listening after,,,,,,2minutes, if he is tweaking to the eyeball test is it because the the metrics are missing something and the eyeball says "wait a minute. this is F'ING WRONG".

since the numbers can only give you what you put in, at it's core it is just a glorified eyeball test.

[KqWIN]

Two schools of thought at play here:

1. Use what we think is good to predict RAPM.

2. Doesn't matter what we think, use what predicts RAPM the best.

There's inherently a lot of overlap between the two. Nate is right, generally the consensus is right. RAPTOR uses the first approach, RPM uses the second. The differences are not material enough to say one is better than the other. Just something to keep in the back of your head.

[freethedevil]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:

"it would be nice to have a metric that looks at how teams value players."

Then he says...

"more often than not the consensus is right"

Does this make sense? Should all in one metrics adjust data to fit a general consensus?

it makes absolute sense.

"general consensus" is probably more commonly known as the eyeball test and the eyeball test works.

No, the eyeball test is watching the game. And the quality of the eyetest is based on how throughly you watch it, and knowing what you're watching. Incidentally, analytics guys also happen to watch the game the most extensively and know what to look for.

As a result, people who don't have the knowledge or aren't willing to do the research required for higher level discussion, default to consensus and "eyetest", because it allows them an escape from the scrutiny more detailed arguments are subject to.

Do any of the eyestest guys here actually watch and breakdown hundreds of game for hundreds of players? No. So then why do people say "the eyetest disagrees"? Because when you say, "my eyetest disagrees", you can't be refuted.

And if you say use "consensus", you can cover up that you aren't as knowledgable on the topic by saying many people think what you do.

[freethedevil]

Two schools of thought at play here:

1. Use what we think is good to predict RAPM.

2. Doesn't matter what we think, use what predicts RAPM the best.

There's inherently a lot of overlap between the two. Nate is right, generally the consensus is right. RAPTOR uses the first approach, RPM uses the second. The differences are not material enough to say one is better than the other. Just something to keep in the back of your head.

Well the better stat will be the one that predicts winning better, but we'll need time to see.

[KqWIN]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:



Then he says...



Does this make sense? Should all in one metrics adjust data to fit a general consensus?

I vote no, but the allure of an all-in-one metric is kind of a pipe dream to begin with IMO. What we really need is comprehensive RAPM data.

More versions and consistent version of RAPM are absolutely what we need. I don't understand why Nate's not gone that direction. These all in one metrics are nice, they have some value, but they're not moving the needle. They should take the additional data from nba.com and create a more comprehensive box score metric and then show that with the fuller RAPM data, and then you can do your own analysis from there. Instead....they want to put it all in one.

I don't understand this argument. RAPM is too noisy, so we use methods to predict RAPM that are more stable. That's what RPM, RAPTOR, PIPM, BPM ect all are.

[freethedevil]

Their model was dead on last year (TOR-GSW Finals). I don't know why they felt the need to change it, especially if they included a non-objective variable.

Their model barely outperformed vegas overall(which is based on what the general public will bet on) and was significantly outperformed by goldstein's pipm based model. You can't just cherrypick two teams to say a model was "spot on"

[freethedevil]

I vote no, but the allure of an all-in-one metric is kind of a pipe dream to begin with IMO. What we really need is comprehensive RAPM data.

More versions and consistent version of RAPM are absolutely what we need. I don't understand why Nate's not gone that direction. These all in one metrics are nice, they have some value, but they're not moving the needle. They should take the additional data from nba.com and create a more comprehensive box score metric and then show that with the fuller RAPM data, and then you can do your own analysis from there. Instead....they want to put it all in one.

I don't understand this argument. RAPM is too noisy, so we use methods to predict RAPM that are more stable. That's what RPM, RAPTOR, PIPM, BPM ect all are.

People don't understand what these "all in one" stats are, and they don't like when said stats disagree with them, so they do whatever they can to try and discredit them.

[Lalouie]

https://soundcloud.com/thinkingbasketball/29-nate-silver-on-raptor


He starts talking at 4:58 about this:



Then he says...



Does this make sense? Should all in one metrics adjust data to fit a general consensus?

it makes absolute sense.

"general consensus" is probably more commonly known as the eyeball test and the eyeball test works.

No, the eyeball test is watching the game. And the quality of the eyetest is based on how throughly you watch it, and knowing what you're watching. Incidentally, analytics guys also happen to watch the game the most extensively and know what to look for.

As a result, people who don't have the knowledge or aren't willing to do the research required for higher level discussion, default to consensus and "eyetest", because it allows them an escape from the scrutiny more detailed arguments are subject to.

Do any of the eyestest guys here actually watch and breakdown hundreds of game for hundreds of players? No. So then why do people say "the eyetest disagrees"? Because when you say, "my eyetest disagrees", you can't be refuted.

And if you say use "consensus", you can cover up that you aren't as knowledgable on the topic by saying many people think what you do.

the eyeball test is a gut feeling

[freethedevil]

it makes absolute sense.

"general consensus" is probably more commonly known as the eyeball test and the eyeball test works.

No, the eyeball test is watching the game. And the quality of the eyetest is based on how throughly you watch it, and knowing what you're watching. Incidentally, analytics guys also happen to watch the game the most extensively and know what to look for.

As a result, people who don't have the knowledge or aren't willing to do the research required for higher level discussion, default to consensus and "eyetest", because it allows them an escape from the scrutiny more detailed arguments are subject to.

Do any of the eyestest guys here actually watch and breakdown hundreds of game for hundreds of players? No. So then why do people say "the eyetest disagrees"? Because when you say, "my eyetest disagrees", you can't be refuted.

And if you say use "consensus", you can cover up that you aren't as knowledgable on the topic by saying many people think what you do.

the eyeball test is a gut feeling

The gut feeling that makes you feel better about players you like and feel worse about players you don't?

No they eye test is based on your eyes.

[Sixerscan]

538's basketball analytics always come off more like some fun side project as opposed to the actual analysis they do with their election coverage. (Specifically talking about their projection models, Chris Herring is a very good NBA writer) I don't think they really take it all that seriously so we shouldn't either.

[KqWIN]

More versions and consistent version of RAPM are absolutely what we need. I don't understand why Nate's not gone that direction. These all in one metrics are nice, they have some value, but they're not moving the needle. They should take the additional data from nba.com and create a more comprehensive box score metric and then show that with the fuller RAPM data, and then you can do your own analysis from there. Instead....they want to put it all in one.

I don't understand this argument. RAPM is too noisy, so we use methods to predict RAPM that are more stable. That's what RPM, RAPTOR, PIPM, BPM ect all are.

People don't understand what these "all in one" stats are, and they don't like when said stats disagree with them, so they do whatever they can to try and discredit them.

It just doesn't make sense to love RAPM, but toss aside the attempts to predict RAPM in a more stable way. These other +/- models are essentially more consistent versions of RAPM. That's actually a good way to describe them...The reason entire reason why we have them is because RAPM is noisy and not consistent. It's not like these are random numbers that give arbitrary weights.