How Unreliable is the 'Eye-Test' for NBA analysis?

[dhsilv2]

Stats can be manipulated easily.
Eye test is harder to manipulate.
Looking at both should lead you to a better conclusion than relying on one though.

IN what world are stats more easy to manipulate than eyes? Our brains literally try and adjust things to tell us what we expect to see.

[LessEyeTest]

Extremely. The number of people that can put aside their bias and preconceived notions, and the amount of people that actually are open to being wrong or correcting an opinion they held is miniscule.

This is the correct answer. Anyone who chooses to argue using "eye test" over analytics shows themselves to not be impartial and has an agenda. It's how we have people arguing Kobe > LeBron and Embiid > Jokic and they are incapable of shifting their opinions because after all, their eyes > evidence.

[dhsilv2]

Eye Test is critical and the most important part of evaluation. You have to see footwork, defense, explosion, first step etc.

You can't measure athleticism in the box scores. The box scores and analytics helps you understand what a person can be and how best to use that player.

But, as a man who pours over that data when I can, the eye test remains the most important evaluation tool. What is that player doing off ball? How about hockey assists. What kind of gravity does he generate.

Etc.

Gravity and Hockey assists are straight up stats...how does that support your eye test? Not sure we have a public gravity stat though...wish that was out there. But it's calculated and quantified and I'm sure every NBA team has it by player by year. They likely even have it by play type. And hockey assists are just straight up on NBA.com, they even have points generated by your assists and your hockey assists.

[LessEyeTest]

Stats can be manipulated easily.
Eye test is harder to manipulate.
Looking at both should lead you to a better conclusion than relying on one though.

IN what world are stats more easy to manipulate than eyes? Our brains literally try and adjust things to tell us what we expect to see.

Ignore him - that's the agenda I'm speaking about.

"Eye test is harder to manipulate" - okay, cool. In my opinion, Paul George is better than peak LeBron James. Now, do we just argue in circles about whose opinion is better and more valuable?

[TheNG]

Stats can be manipulated easily.
Eye test is harder to manipulate.
Looking at both should lead you to a better conclusion than relying on one though.

IN what world are stats more easy to manipulate than eyes? Our brains literally try and adjust things to tell us what we expect to see.

Stats capture "some" of the data, while your eyes see everything. It's their inherit nature - trying to summarize a lot of data to a final number.
Not saying there aren't cognitive biases, I agree with that, there are many known fallacies.
But no stat will tell me how a player gave up on his team with bad body language in the most crucial moment.
But again, as I said, it is better to have both. This way you can work with the difference between what they tell and gain new insights.

[reddyplayerone]

Stats can be manipulated easily.
Eye test is harder to manipulate.
Looking at both should lead you to a better conclusion than relying on one though.

IN what world are stats more easy to manipulate than eyes? Our brains literally try and adjust things to tell us what we expect to see.

Ignore him - that's the agenda I'm speaking about.

"Eye test is harder to manipulate" - okay, cool. In my opinion, Paul George is better than peak LeBron James. Now, do we just argue in circles about whose opinion is better and more valuable?

Yes?

Like I think it's really important to recognize that whichever side of the Eyes Vs. Stats argument you're on, all anyone here is doing is arguing about their opinions in circles.

[dhsilv2]

Stats can be manipulated easily.
Eye test is harder to manipulate.
Looking at both should lead you to a better conclusion than relying on one though.

IN what world are stats more easy to manipulate than eyes? Our brains literally try and adjust things to tell us what we expect to see.

Stats capture "some" of the data, while your eyes see everything. It's their inherit nature - trying to summarize a lot of data to a final number.
Not saying there aren't cognitive biases, I agree with that, there are many known fallacies.
But no stat will tell me how a player gave up on his team with bad body language on the most crucial moment.
But again, as I said, it is better to have both. This way you can work with the difference between what they tell and gain new insights.

Most people if we had you watch a game without announcers or a score board and ended the video with 5 minutes left in the 4th couldn't even tell me who's winning or losing the game. I'd actually even go further and say that the announcers actually influence the average fan's analysis more than their own eyes do watching the game itself.

Fans ball watch. Fans often have a bias for positive outcomes. Fans have massive winning bias. Fans have excitement bias. Fans have play type bias. We can go on and on. We're REALLY REALLY REALLY bad at watching games and building an analysis. That's why they have the announcers in the first place. They help guide fans eyes.

Even the statement "Most crucial moments in the game" is a type of bias. The last minute of a game only matters if you didn't take care of business the last 47 minutes. But fans have a bias for the end of the game and arbitrary additional weight to it.

We can even take this a step further. I recall Parker talking about how Tim Duncan would have these "quiet" games, and then he'd look at the box score 30-20-6 or something absurd. So here, a guy actually playing in the game, sitting court side watching when not playing...and he can't even keep up with his own teammate scoring 30 and grabbing 20 boards. I've heard guys talk about how they didn't realize how tall another player was until they were on that player's team...but 2 or 3 years earlier they'd been GUARDING that guy in the playoffs!

Now to counter that, I've seen guys like Lebron breakdown every single play of a game after it was played, so there are levels and exceptions to this stuff. But none the less the amount of bias that influences our eye tests is staggering. And not many of us have spent 20,000 hours breaking down game film like Lebron. And that includes MOST of his peers, who are often like Shaq and Chuck...oblivious half the time.

[reddyplayerone]

I'll also say that it is incredibly jarring to read fans talking about being a Fan like it's inherently a bad thing or means a person's opinions are somehow less valid or credible.

[BobbyPortisEyes]

Stats capture "some" of the data, while your eyes see everything. It's their inherit nature - trying to summarize a lot of data to a final number.

It's almost the exact opposite for both of these sentences. Your eyes will focus on a very limited set of things, mostly focusing on where the ball is and where the most movement occurs. And your brain will then extrapolate and distort what you saw as it tries to give meaning. It's definitely not the inherent nature of eyes to summarize large amounts of data, we wouldn't even be here today as a species if our eyes were for deep analysis instead of quick judgment for survival's sake.

[Capn'O]

Screw you all. Big Dog was awesome. He got a raw deal.

[GoBobs]

There are three kinds of lies

1. Lies

2. Damn Lies

3. Statistics

[azcatz11]

Eye Test >>>>> Stats

People constantly quote random advanced stats on here that no one understands or how they're calculated.

"Player X has a higher RAPTOR, LEBRON, etc. than Player Y"

I can watch the games and form a better opinion than any stats would present.

[xdrta+]

I'm perfectly fine with using the eye test as long as its my eyes that are doing the testing. They're the only ones that get it right.

[KembaWalker]

The one number “advanced impact metric” process is more or less:

Nerds fiddling with spreadsheet formulas until it spits out a list that wouldn’t get them laughed at by people who watch a lot of basketball. It hangs around for a few years because those players it measured as good stay good for a few years. Longer it goes, it starts to decay and spit out wacky results because it’s not handcrafted to get a certain list like it was when it was created. It gets retired or “updated” return to step 1

[Capn'O]

But seriously, ideally by now anyone on here would be at least integrating some sort of analytical understanding with their viewing as well, no? Like, when I'm looking at players to add to bench roles in my 30-team dynasty league I have a statistical methodology that I run before I watch tape. I can't possibly scout everyone heavily so a culling method is essential.

Also, to me recordings don't really capture a players abilities and tendencies as much as watching them live and I can only go to like one game a year and a quarter of that time I'm chatting with my buddy. So it's not even the right eye test most of the time.

[dhsilv2]

The one number “advanced impact metric” process is more or less:

Nerds fiddling with spreadsheet formulas until it spits out a list that wouldn’t get them laughed at by people who watch a lot of basketball. It hangs around for a few years because those players it measured as good stay good for a few years. Longer it goes, it starts to decay and spit out wacky results because it’s not handcrafted to get a certain list like it was when it was created. It gets retired or “updated” return to step 1

BPM/VORP is literally the only metric like that. And it was designed to mimic another metrics that's better, but requires play by play data...

Even PER was used by Hollinger to predict results and he even used it to pretty good success. The ONLY good measure of any metric is predicting future results. It's been the standard since day one and still is.

[Cavsfansince84]

The problem is when people want to say 'eye test is all I need' or 'numbers are all that matter' without any in between. I guarantee that even people who think eye test is better still look at box score stats. It's ok to watch games and use advanced stats. Both methods will have blind spots. If you can't admit that then its your own blind spot.

[ForeverTFC]

So flawed a premise.

Data. You use it as though only numbers are data. That is wrong. You also use it as though this type of data has no “bias.” Also flat wrong.

Everything not quantitive you lump into one category and call it the eye test. So everything that impacts a basketball game that can’t be put on a spreadsheet is eye test.

Everything can be expressed by numbers. Whether we are expressing it all or whether we are expressing it correctly is definitely a valid debate and rebuttal. But all data can be numbers. And numbers should play a disproportionate role; math and numbers are the fundamental tool in human development. There is no reason why basketball is any different.

[dhsilv2]

I'll also say that it is incredibly jarring to read fans talking about being a Fan like it's inherently a bad thing or means a person's opinions are somehow less valid or credible.

It's obviously a bad thing if you want to be objective. Being a fan by default makes one irrational and biased. Which is completely OK, basketball should be fun to watch. I don't want a pencil and paper in my hand all game while a track plays and then type that into excel, then move into a database to run complex math on it. I want to watch and enjoy the game. Then later when I'm wondering about something more in depth...I'll go check the stats and see what's going on.

[dhsilv2]

Eye Test >>>>> Stats

People constantly quote random advanced stats on here that no one understands or how they're calculated.

"Player X has a higher RAPTOR, LEBRON, etc. than Player Y"

I can watch the games and form a better opinion than any stats would present.

OK super simple ask then. Give me the top 200 players defensively ranked by their role (not position). Given your eyes are so good and fast, I'd assume you can do this pretty quickly. Then we can compare some results and see how it matches with the stats. Now I couldn't do this or even come close, but my eye test isn't 5 greaters stats. I'd also like to know how you're doing with vegas given those stats when they were first coming out and not public were consistently beating Vegas odds.