How Unreliable is the 'Eye-Test' for NBA analysis?

[Johnny Bball]

Depends on who's eye.

And how biased that person is. Most people here see what they want to see.

[ballzboyee]

Eye test is fine in certain scenarios. To give an example, Kenyon Martin in an interview talked about how after he retired teams would contract him to work out players. Even if he shut a guy down, he could tell if they were going to be good enough to play in the league. Nothing trumps experience, which is what the eye test really is. A scout or casual fan watching Martin shut down a prospect might think, "Oh, man, this guy sucks." In reality, Martin is thinking the opposite because he can feel the strength of the prospect's game and where it is likely heading in the future. During the interview he gave a specific example of players like this, but I can't remember now who.

Now, if we are talking about specific matchup and lineup analysis, then teams definitely need more deep dive numbers and analytics to see what is happening because there are just way more moving parts when looking at 5v5, 4v4, etc. Much harder if not impossible to analyze just by using an eye test.

[Special_Puppy]

Useful eye test=Carefully done film study done by an expert
Useless eye test=Casually watching the game live by a non-expert

[Special_Puppy]

Useful eye test=Carefully done film study done by an expert (Think Ben Taylor)
Useless eye test=Casually watching the game live by a non-expert

[EmpireFalls]

It is the test, in my opinion.

The only problem is people are biased and stupid and their eye test is filtered through that lens, and the other issue is people don’t watch anywhere near as much as they claim to. But eye test matters.

[MiltownMadness]

Its everything. If youve been watching for a long time your intuition often picks up on details before even looking at the stats. I often notice things that a casual fan wouldn't ever think of because Ive watched thousands of hours of NBA basketball. That goes for many on here.

[xinxin]

Its everything. If youve been watching for a long time your intuition often picks up on details before even looking at the stats. I often notice things that a casual fan wouldn't ever think of because Ive watched thousands of hours of NBA basketball. That goes for many on here.

Agreed. That’s why it cannot be dismissed.

To the trained “eye”, the eye test might be all you need.


Sent from my iPhone using Tapatalk

[hardenASG13]

I used to be an OKC fan, and remember arguing for years with basically the whole OKC board, who knocked my "eye test" saying playing Andre Roberson 30 minutes a game, when he couldn't shoot or dribble a basketball, was crazy. Especially next to KD and Westbrook. I was mocked for saying things like it was killing the spacing, and that shooting role players are what that and every team needed. They gave a bunch of lineup stats, said all I cared about was "pointzz" and didn't know what I was seeing. Said guys like Jerami Grant were scrubs because of percentages. Now they don't let me post there and aren't willing to discuss it, although it certainly seems like the complete direction the league has gone. But the stats didn't agree. I wonder why more contending teams aren't playing bums like Roberson 30mpg anymore in the corner next to their superstar creators? Don't they see the lineup numbers and how important individual perimeter defense is, in a league that is forced to switch almost every perimeter screen haha.

Those that just post stats and spend hours compiling them are often just trying to look smart, with no ability to understand what they are watching. Many here also put way too much into regular season stats, when it's quite clear that the playoffs are a whole different beast. You need an eye test to see what players and concepts will work in the playoffs, when the defense knows exactly what you are running and how to take it away.

[hauntedcomputer]

Numbers can misrepresent, facts can lie, and we are so loaded up with biases that it is impossible to overcome them even if you are aware of them.

In short, chaos reigns and all beliefs are delusion.

[dhsilv2]

Depends on how many times you use it. If you are only using one game, obviously it’s not enough evidence. The eye test to me is to see how someone reacts to a certain situation on a consistent basis.

If a QB can’t make certain throws against the blitz on a consistent basis, then defense can take advantage of that. Eye test would conclude, “He can’t do well under pressure.” or “He can’t go through reads.”

You'd be better off with just having his stats against the blitz and with a defender within x feet or something. That would avoid you over focusing on a few games or a big playoff moment that sticks vs looking at the data in totality. This is an example where data would be overwhelmingly and virtually indispensable while an eye test would hold little to no value.

The eye test might help with for example, the QB tends to look left to right if his first option is broken up. Or perhaps you pickup on a QB's height seems off of his labeled height and he's impacting his ability to make reads over the middle. This is where the eye might pick something up we'd have a VERY hard time quantifying.

[dhsilv2]

I think you have to use the eye test to have any kind of opinion.

For one, defense is not well quantified by statistics. You can see blocks, you can see steals, and so on, but any of the more advanced metrics usually involve the entire team to one degree or another. So you really aren't going to see half the game of basketball from looking at a box score, at least not for an individuals defense. Even blocks and steals can be very misleading when talking about how good a defensive player is.

We have break downs of every time a player is in any type of defensive set or against against any type of offensive play type. We have what that player shot against you. What they normally shoot in those scenarios. And we can gather to some degree what impact you have. We can see how many shots you challenge a game which gives us an idea of how much/good you are helping. Certainly defense isn't perfect and augmenting it with an eye test is extremely helpful, but you're massively underselling what kind of data we have.

For example I know Luka defends in isolation 1.5 times per game with the man he's defending scoring 0.84 points per possession and they score about 36% of the time doing this.

Now I'm sure there's more data out there, but not public to drill into who he's guarding. But my eyes know Luka's got poor foot speed but great strength which he uses in isolation. If I had data on foot speed like the mavs likely do, they could drill into how that breaks out between high and low foot speed guys. From there coaches can work to optimize their defense using that data. I as a fan can point out that as long as Kidd finds ways to keep Luka in optimal defensive sets, his weaknesses can be minimized.

Now maybe I see the slow feet, but it doesn't show up in the data? Now I'd have something to look further into and the data could help my eye test be more accurate.

Similarly, I know Jokic this year defends post ups 0.5 times per game. On those guys are scoring 0.81 points per possession, they score about 40.6% of the time, but he's shockingly not given up a single and one all year from a guy posting him up and only seeing a 3.1% frequency for a shooting foul while guys turn the ball over 15.6% of the time when they post him up.

We have a hell of a lot of good data on defense these days. We don't have a magic equation that says "you're a good defender" I suppose...

[Godymas]

the eye test is always a thing and you can so easily tell a quality player it isn't even funny.

just watch a game of college basketball to get an idea of what you're looking at. Like if you watched any Duke game at all this year it's so clear how far ahead of the competition Kyle Filipkowski is that it's no surprise he's projecting to be a lottery pick in the draft.

[dhsilv2]

the eye test is always a thing and you can so easily tell a quality player it isn't even funny.

just watch a game of college basketball to get an idea of what you're looking at. Like if you watched any Duke game at all this year it's so clear how far ahead of the competition Kyle Filipkowski is that it's no surprise he's projecting to be a lottery pick in the draft.

I mean sure, you don't need stats to know the difference in an elite prospect and a guy who's going to graduate with a 4 year degree and go on to be an accountant or sales guy. But likely could use stats OR the eye test and figure that out in seconds. That's not really a good example of either working well.

[slick_watts]

most people who talk about 'eye test' treat it as something mutually exclusive from quantitative analysis. most people digging deep into nba stats are also watching a lot of basketball, probably more than the people who talk about eye test all the time.

there's always value in watching basketball but i don't think it's possible for most people to just watch a game and see everything that is going on to the extent that they are as informed as someone who watches a game and also has access to / considers all the data sources we have available to us these days.

and of course there's no way anyone is watching every game.

[dhsilv2]

I tend to look at this way.

Basketball is played on the court, not in python calculations. You need to grasp how players interact, roles they have, and what they are physically. Once you grasp that, the data is going to just tell you more than your eyes can take in and process over the course of a season.

Stats don't explain why Embiid and Harden struggle in the playoffs. My eyes tell me that Harden's asthma is an issue for him and Embiid's conditioning is. Stats however tell me that I'd actually rather have Harden take those ABSURD looking step back 3's than even some conventional one's....he's just a freak with that shot. And I don't think my brain could ever in a million years process how it's even possible he could shoot that better. But once I saw consistently he was shooting a few percentage points better on those vs catch and shoot....I just had to accept it.

So yeah, looking at data is likely going to tell you more accurate and actionable information. But you need to understand the team and players before you jump into the data. Case and point...I'm not sure I could look at the Piston's data and tell you a lot. I haven't watched them this year and frankly...not sure I need to. But I'm also not going to bread down my feelings on most of their player's outside of maybe Wiseman who I watched a lot of with the Warriors. I just don't have a feel for how that team plays or what each player's roles are. Meanwhile, I find it hard to really get a feel even watching the wolves at how good some guys are. So I turn to stats to augment and assist me and trying to figure it out. A wolves fan who watches more of their games might need way less augmentation because....they watch them all the time.

And when the eye test and data conflict, I'd expect a good eye test to be supported with data. Similarly, anyone pushing data better be able to explain the data with eye test details. And from there the best argument using both will win the day.

[Masigond]

I remember a game when the Celtics were playing the Jazz in TD Banknorth Garden when I first truly realized that my eye-test is not reliable enough. I was noticing very much of Delonte West's performance. He was scoring 15 points by shooting 5/8 from the field and 5/6 FTs to go with 7 rebounds, 5 assists, 3 steals and 2 blocks. He was so very noticeable on the field in that game. But impact? The Celtics were +/- 0 with him on the field. In the same game I almost completely missed Ricky Davis having 20 points by shooting 6/12 FGs and 8/10 FTs with the Celtics outscoring the Jazz by 21 when he was on the field.

So I'm too prone to fall for spectacular plays and missing rather smooth looking performances. Definitively too prone to paying too much attention on the player on the ball, too. To really notice somewhat unspectacular players I really have to concentrate on them. It's too easy to miss great movement without the ball and all those little things that make the game easier for teammates. Boxing out, setting screens, gravity and so on. And it's very easy to excuse a player for not hitting a contested difficult shot, especially when you (like him obviously) didn't notice his teammates being in a much better position for an easy bucket,with the pass never coming for that better opportunity.
Thus overrating and underrating a player's impact is very common for many viewers as we are too prone to fall for the spectacle. I guess there are way too few to really read a game properly, especially when watching live without the control of watching a certain play on replays and in slow-motion,

[HotelVitale]

Depends on how many times you use it. If you are only using one game, obviously it’s not enough evidence. The eye test to me is to see how someone reacts to a certain situation on a consistent basis.

If a QB can’t make certain throws against the blitz on a consistent basis, then defense can take advantage of that. Eye test would conclude, “He can’t do well under pressure.” or “He can’t go through reads.”

Your example might not work for you. The best scouts in the world are wrong about that 'eye test' all the time for QBs, and everyone also uses various analytics to help with that too. Things like completion % in the flats on broken plays, or accuracy of deep post throws beyond 25 yards. Every category possible to help you suss out exactly how strong or weak a player's skillset is based on all available game data. (Simpler example: if you saw some dropped catches in some big moments, the eye test might make you think 'man this QB's receivers seem to flat out drop a ton of great throws' but basic analytics would help tell you if that was actually a meaningful or outlying amount of drops compared to other QB prospects, or just a few that stuck in your mind).

For scouting in general, I've found the average sports fan massively over-estimates the 'gut' of certain scouts (or worse GMs) when it comes to drafting. Scouts and FOs are mostly quite smart and sophisticated and aren't leaving anything on the table in their analyses, but they know what they don't know and they know that most draft picks come down to those unknowns. People always want to credit a GM for a good pick and bash them for a bad one, but most of those picks were well-informed good gambles and the player either could or couldn't perform their strengths in real time in real games. Usually that simple. (Think of why CJ Stroud was so good this year vs other recent top QB picks).

[cupcakesnake]

You need to watch basketball to know why the numbers say what they're saying.
You need numbers to check the bias of your eye test.

There aren't good basketball analysts who don't use a ton of both. Most people who tell you "I go off the eye test" aren't even being honest. They don't know the numbers so they're taking some macho, anti-nerd posture, whether they watch a lot of ball or not (often they don't watch much). I'm not sure I encounter people who say they only look at numbers, but if I'm reading articles or listening to a podcast, and the person is just listing stats without explaining what's happening on the court, I get bored.

[zero rings]

The eye test is useful for providing context to the data, but people who use it as a substitute are usually pushing some kind of agenda.

[KnicksGadfly]

My issue is that many times, when posters say they rely on the eye test, what they really mean is “I watched the film” and I really don’t want to hear anything else, not the data or any other opposing eye tests. It’s a way to sound very authoritative when it’s just reliant on one’s sole expertise.

We all know that eye tests themselves can be very diverse. Posters on this forum can look at the same player on one play and come away with completely opposing conclusions; hell, the best example of bias I can think of is this one video of Randle’s laziness at an end of game play, and 99% of posters could see where he failed and one poster kept absolving him of blame and blasting the other 4 players. I think Morey also said the Rockets got in trouble once cause one scout called Marc Gasol “manboobs” which led the Rockets not to draft him. I think teams mitigate this by having multiple people conducting eye tests - to me, if multiple people who I respect have similar eye-tests, that’s a very strong data point (and starts veering into analytics, weirdly enough). But at the same time, that’s very rare and then a decision has to be made. Still, you go into it with eyes wide open, knowing that you’ve gotten both sides of the argument.

But one sole guy doing it - no matter how much he says “eye test,” I can’t take it that seriously.