Book Me For A Speech

My Writing and Ranting

Press Room

Good Books

« d.school Alum Laura Jones Selected by BusinessWeek as one of "21 People Who Will Change Business" | Main | Flawed Self-Evaluations: David Dunning's Facinating Work »

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b75569e20120a6180b20970b

Listed below are links to weblogs that reference Selecting Talent: The Upshot from 85 Years of Research:

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

watch a video

I found your blog through a comment on 37Signals regarding interview questions involving puzzles. How I wish some of these companies saw this. I am pretty sure that the folks inside Google do almost similar tasks as I do. While I would like to work there, I don't think I'd be able to crack the puzzle they ask.

Avidblogreader

I can't help it, I think it's important to make the following comment:

You have a blog and write serious commentary about serious subjects. Don't insult your readers by posting something that you didn't seriously proofread.

Skeptic

As I recall from my statistics class, 0.5 or less was considered a weak correlation rapidly diminishing in significance. So even the top seven or eight tests listed here are marginally useful, the rest junk. Not very encouraging to say the least.

Ellie

Rookie question:

Can anyone tell me what what a structured interview consists off compared to an unstructured one?

Brian

Hi Bob,

I feel this post certainly passes the litmus test as not snake-oil. So many "consultants" out there focus on those less predictive measures because it would be very hard to charge a company to look for smart people that can demonstrate that they can perform their work. Perhaps hiring is not a black art after all.

Arun

Look up page 227 of Handbook of Industrial, Work and Organizational Psychology: Personnel psychology
By Neil Anderson on books.google.com

You will see Schmidt and Hunter's table reproduced.

Table 11.2 Meta-analytic predictive validity of g and a second predictor for job performance.

The column headings are

Predictor
Validity (r)
Multiple R
R - r (superscript l, subscript g)

I produce the first 3 columns of the first few rows

g, .51,
Work sample test, .54, .63
Integrity tests, .41, .65
Conscientiousness tests, .31, .60
Structured interviews, .51, .63

By itself, Work sample test seems to have more validity (0.54) than g (0.51)

It seems to me that this blog post may have high g, but it has low integrity.

Arun

This is the article abstract:

"This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed."

http://aprtestingservice.com/ tes...hmidtHunter1998

Doesn't seem to be saying what the blog article says.

juegos de estrategia

nice article of sutton

greetings

Rob G

As you point out, this research was conducted in 1998, so myself and other I/O Psychologists have known about this for a while. To me one of the most interesting points on this is Years of Experience. In the rank order it is pretty low, and when reading the article the effect size is low as well. Yet go to your favorite career website (Monster, Careerbuilder, The Ladders, etc.) or go to your own internal job post site. Years of experience will almost always be listed first in terms of requirements, and utilized first in resume screening by recruiters and hiring managers. A sub-par predictor becomes the first and usually most important screening criteria. Talk about not utilizing evidence-based management!

Jan

As I understand it, you are very likely to hire someone who resembles your own personality. You want someone who looks upon the world the way you do, because then you will understand them, and they will understand you.

Management teams are known for something called homosocial regeneration, which basically is that everyone behaves, thinks and dresses alike, because it gives comfort and a false sense of security (not the same thing as group think).

Maybe we should look more at how creative a person is (if it's a creative job), and less on value-systems and personality. I think this is an awfully difficult subject.

sk

Very interesting. The fact that GMA tests are most predictive shows why filtering based on university is commonly used. A school is an imperfect proxy for SAT, GRE, or GMAT tests, which in turn are imperfect proxies for the controversial (at least to this layman) IQ.

Although interviews may not be the most successful predictor in determining the ability of a person to perform a job, it seems they would be the most effective way to insure employees fit the culture. In particular, a way to avoid (or attract) assholes as your culture sees fit.

So, I wonder how much of a role company cultural fit or social interactions played in the various studies used to perform the meta analysis.

It is also interesting that these results show age discrimination (a growing problem) to be the least effective way to create an able workforce.

That's a nice, thought-provoking blog entry for a Friday.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Scaling up Excellence

Good Boss Bad Boss

No Asshole Rule

Hard Facts

Weird Ideas

Knowing -Doing Gap

http://bobsutton.typepad.com/.a/6a00d83451b75569e20168e878040d970c-pi

The No Asshole Rule:Articles and Stories