A Breakdown of the Caliper Profile

[XxIronChainzxX]

In another Bargnani thread, someone brought up the old Caliper Profile article, where basically they slobbered all over Bargnani's knob as being the most incredible, elite, fearless demon-monster this side of Wilt Chamberlain. Here's the relevant section:

"The test showed that Bargnani is virtually oblivious to what others think of him. And his tremendous ability to block out such potentially negative pressures enables him to focus completely on the task at hand. So the expectations and anxieties that come with being the No.1 pick, or the only Italian-born player in the league, or even taking a game-winning shot, don't even register with him.

Then, there's this from Dr Greenberg, the founder of the Caliper Test:

That's the kind of thing we're very, very proud of. We did some … I wouldn't say “arm twisting,” because they liked him, but they knew they'd get a lot of criticism for drafting him. I can tell you right now: He will be one of the top eight or ten centers in NBA history.”

The link to a pdf. of the profile itself is here: http://www.caliperonline.com/brochures/ ... 0Guide.pdf.

The majority of my degree work is in neuroscience, but I have a pretty solid basis now in psychology and especially psychometrics because of my general area of interest, so I took a look at what the Caliper Profile actually measures, and whether or not this is pseudoscientific BS.

At any rate, here are the profile aspects: Abstract Reasoning, Accommodation, Aggressiveness, Assertiveness, Cautiousness, Ego-Drive, Ego-Strength Resilience, Empathy, External Structure, Flexibility, Gregariousness, Idea Orientation, Risk Taking, Self-Structure/Self-Discipline, Scepticism, Sociability, Thoroughness and Urgency.

Here's the basketball relevant portion:

Based on what the article says, Bargnani's rated as being oblivious to what others think of him. This seems like a mix of their measure on Ego-Strength Resilience and Accommodation. Both sound like BS models that have no legitimate scientific basis used by the profile. Basically, Bargnani could well have tested very well on this measure. But as a scientist, I would say you should trust this as an accurate description of who he is basically as much as you should trust the opinion of your local witch doctor.

They don't say much else about his test result, but at least in that portion, Bargnani performed well on the equivalent of a dude asking you questions about how much you care what others think about you and then translating this into "he will the the most cold blooded assassin in professional sports."

[Wiz Khalifa]

im too lazy to read all of this
someone summarize this huge post

[XxIronChainzxX]

The short version is that the Caliper test doesn't sound like great science, so the fact that Bargnani comes out as a barely human demi-god on it might not mean, really, anything. I thought it'd be interested to talk about it since BC hyped this up a fair bit early on as why Bargs would rock.

[mercy]

I believe in Andrea !!

[Boogie!]

you do realize that the one paragraph doesn't sum up his entire caliper test results right? as a journalist, he pointed out the most obvious and most relevant-to-the-article aspect of the test. when i read that paragraph i don't assume bargnani is gonna be a good player because he doesn't care what people think of him. if it were that simple, i doubt the caliper test would be as popular as it is. but i realize this is ONE of the factors that will help bargnani be successful in the league.

[HighOctane]

1. Would there have been any language barriers in his test?
2. Do blank stares count for extra points?
3. Is there the possibility of him just absent mindedly not answering the questions properly?

[darth_federer]

This was what the owner said.

“Teammate Anthony Parker cuts through all the psycho-babble. He says, "He's absolutely fearless. Why? I think he was just born with it." Bargnani himself brushes off talk of his extraordinary self-confidence. "I'm just a normal person," he says. But the trait is real, measurable, in fact.

"For years, NBA teams have used the Caliper Profile to evaluate potential draft picks. The Caliper is a personality profile used by numerous corporations and organizations to measure one's capacity to excel in specific situations. Over the past 24 years, Caliper has assessed more than 20,000 athletes, including NBA players from Detroit, San Antonio, Denver and Phoenix. Colangelo has long been sold on the system. When he heard how Bargnani measured up, he nearly dropped the phone.

"They said his upside and potential were off the charts," Colangelo says from the tunnel of the Air Canada Centre as Bargnani drains a three against the Cavaliers. "They said, 'Out of all the athletes we've profiled, we've never seen anything like this.'

"The test showed that Bargnani is virtually oblivious to what others think of him. And his tremendous ability to block out such potentially negative pressures enables him to focus completely on the task at hand. So the expectations and anxieties that come with being the No.1 pick, or the only Italian-born player in the league, or even taking a game-winning shot, don't even register with him.

Then, there's this from Dr Greenberg, the founder of the Caliper Test:

That's the kind of thing we're very, very proud of. We did some … I wouldn't say “arm twisting,” because they liked him, but they knew they'd get a lot of criticism for drafting him. I can tell you right now: He will be one of the top eight or ten centers in NBA history.”

http://sports.espn.go.com/nba/news/stor ... g/bargnani

[Joga_Bonito]

The test just shows what type of personality Bargnani has but just because we know what kind of guy he is does not mean it will translate to what kind of player he is or will be. There are so many other variables than the athlete's personality.

[ponder276]

Bargnani does indeed seem virtually oblivious, especially on defense. He is able to block out distractions such as players driving to the rack, and maintain his focus on guarding empty space.

[ImissJordan]

Yeah, oblivious is a good word to describe Andrea Bargnani.

And LMFAO at Bargnani being one of the top 8-10 centres in the history of the NBA; he isn't even one of the top 8-10 centres in the game right now.

[34_fifty]

You know, as a transformed fan that is looking at the future with a positive outlook, I must say that this is encouraging.

[Ripp]

How the caliper test has it been validated? Is there a correlation between high performance on it and NBA greatness?

For example, you can correlate IQ scores with SAT scores or College Gpa...if you have a high IQ, you'll likely have a high SAT score, or great grades in college.

If the Caliper test has been used on 20k athletes, is their any correlation between the results and actually performance? Sounds like a bunch of voodoo to me, unless this sort of validation has been done. Not only voodoo, but a trap for foolish GMs (e.g., Bryan Colangelo.)

[freamon]

Bargnani does indeed seem virtually oblivious, especially on defense. He is able to block out distractions such as players driving to the rack, and maintain his focus on guarding empty space.

Read my mind.

"Andrea! Rotate!"
"I am oblivious to your expectations of me."

[freakoutguy]

if this test was a multiple choice one, could Andrea have fluked out by randomly selecting the answers

[chuckerz]

Andrea on court demeanor kind of reminds me of Tim Duncan in his young days. Where they both kind of seem disinterested with the stoic look. Bargnani with his mouth open all the time while Tim Duncan has his blank stare. The only difference is that Duncan is mentally tough and can handle criticism from a coach like Popovich while Bargnani couldn't handle it with the Smitch and he started to excel only when triano came in as interim head coach. So the Caliper test fails.

[ruckus]

The thing is, from what tidbits we were given on Bargs' caliper test, they have proven to be fairly accurate. He doesn't seem to care what other people say about him. He keeps chugging along progressing at his own pace and not the pace that others are expecting him to grow at.

Although, the part about his confidence doesn't seem to hold. We all saw how Bargs' was shaken when Smitch was messing with his minutes.

[StopitLeo]

I need to see a graph of a player's Caliper score vs. various stats like PER, ORTG, DRTG. My feeling is that will be a very weak correlation.

[lobosloboslobos]

It's easy to dis these kind of personality profile tests as empty psycho-babble. I certainly used to do it myself. Until I had a fascinating experience with one about 10 years ago that really made me change my opinion. I won't go into details here, but it seriously transformed a group of 10-15 people who took it by revealing previously unknown tensions and alignments in values and personalities. I was shocked at how accurate and useful it was.

[darth_federer]

It's easy to dis these kind of personality profile tests as empty psycho-babble. I certainly used to do it myself. Until I had a fascinating experience with one about 10 years ago that really made me change my opinion. I won't go into details here, but it seriously transformed a group of 10-15 people who took it by revealing previously unknown tensions and alignments in values and personalities. I was shocked at how accurate and useful it was.

http://www.psychologies.co.uk/articles/ ... awakening/

[XxIronChainzxX]

The test just shows what type of personality Bargnani has but just because we know what kind of guy he is does not mean it will translate to what kind of player he is or will be. There are so many other variables than the athlete's personality.

I agree, but the thing from that quote is that the creator of the Caliper Profile said Bargnani would be a top 8-10 C in the NBA, and worked to sell Bargnani to Colangelo based on that. We can't know what went on, but if someone is saying stuff that strong, we ought to ask ourselves, does the test sound like a good one?

And I think the problem is that it doesn't, not when it doesn't look at justified scientific measures. I did a lot of digging around last night and I have a good idea why they measure what they do, and the problem is that they use a lot of outdated or otherwise discredited constructs, like learned helplessness.

How the caliper test has it been validated? Is there a correlation between high performance on it and NBA greatness?

That shouldn't be enough, though.

For example, you can correlate IQ scores with SAT scores or College Gpa...if you have a high IQ, you'll likely have a high SAT score, or great grades in college.

Right, but the thing is, IQ is a general ability. It's like ... correlating athletic ability with NBA success. You'll find the more athletic ability you have, the more successful you'll likely be, ceteris parabilis.

The Caliper is a personality profile, and their correlation with success is really, really low. As far as I know, even things like IQ and conscientiousness, which are huge predictors of success, only correlate @ 0.3 with success, which basically means that they can explain about 10% of why you were as successful as you were.

We don't know what the predictive strength of this test is, but I'm betting it's not very high. The thing is, an interview is useless as a method of evaluation, so it makes sense for a Corporation to use these profiles, because they're more likely to get good employees that way. It's likely stupid for the NBA to do it, because they have way better methods of evaluation that are more reliable, like scouting (that you could do for years), workouts, etc.