Why interviews are a lousy way to judge candidates

We've had many discussions here about the hiring process, with great contributions from those who do the interviewing/hiring and those who have been through the process. Here's an interesting article that suggests that most organizations are doing it wrong.

... managers are consistently overconfident in their ability to identify the best candidates using a job interview. We cling to the fanciful notion that we can perfectly predict future job performance, despite overwhelming evidence against it. We all want to believe that we are good judges of character, yet we do not bother to collect the evidence we would need to test that belief. Rather, we rely on gut intuitions about whom to hire.

Click to expand...

I must say though that while the writer has pinpointed a problem, I'm not big on his solution of trying to make the process more measurable.

My company have utilized structured interviews for years. It forces the candidates to draw on past experience from their professional and/or personal life rather than answer hypotheticals based on what they think the interviewer wants to hear. It's harder for people (IMO) to make-up a lie on the spot if the question requires them to tell you a specific example from their past experience. It's easier to catch them in a lie too.

We don't do IQ tests. For some industries, it might make sense to do one. We require a writing sample since providing written narratives is a big part of the job. It's astonishing to me how poorly some people spell or the amount of grammatical errors that I read from a simple writing sample.

Cynical me sees this article as the writer finding an excuse to promote his research. Cynical me also says that MBA students interviewing each other to predict who will do well on the midterm is not even close to being a reliable replication of the conditions/expectations of a job interview. Ironic when the writer is complaining about the lack of reliability of hiring tests in the real world.

IMHO the real world problem is not the lack of reliability of hiring measures, but the politicking and the "like hires like" mentality in hiring that leads people to overlook the useful information that *does* come out of hiring tests and candidate assessments.

I must say though that while the writer has pinpointed a problem, I'm not big on his solution of trying to make the process more measurable.

Thoughts?

Click to expand...

I actually like his solution. I've always thought interviews were biased towards extroverted, sociable types. As an introvert, I've always felt at a distinct disadvantage in the interview process. I think making the process more measurable will help a lot in negating that bias.

Cynical me sees this article as the writer finding an excuse to promote his research. Cynical me also says that MBA students interviewing each other to predict who will do well on the midterm is not even close to being a reliable replication of the conditions/expectations of a job interview. Ironic when the writer is complaining about the lack of reliability of hiring tests in the real world.

Click to expand...

Agree - I think he's right about the problem, but he started to lose me with his solution. As a researcher, of course he's going to advocate a process that has few variables and is measurable.

What he doesn't really address, other than saying that intelligence tests are applicable to most jobs (and even that I question), is that different jobs don't just require different skill sets (which he does acknowledge), but different interview styles (which he does not).

Many jobs do require a lot of interaction, whether with coworkers, suppliers, customers or others, and therefore, how a person presents themselves in an interview can be hugely important. Maybe it doesn't equate to how they will perform long term (his hypothesis, which I agree with to an extent), but it does tell you a lot about how they'll do on a sales call, dealing with a new customer, or working with a diverse range of colleagues.

My company have utilized structured interviews for years. It forces the candidates to draw on past experience from their professional and/or personal life rather than answer hypotheticals based on what they think the interviewer wants to hear. It's harder for people (IMO) to make-up a lie on the spot if the question requires them to tell you a specific example from their past experience. It's easier to catch them in a lie too.

Click to expand...

We do this - we call it STAR. Situation, Task, Action, Result. We present the situation and ask for an example related to that situation that presents the task performed, the action taken and the result.

The one major work around that candidates use is to give examples performed by their co-workers instead of themselves. We have used STAR for a long time, so people know the drill - so it is possible to lie when you know how the interview will be structured and are a little creative with it.

This is both on and off topic - but one thing I have consistently seen in interviews / hiring over the years is that people want to hire people like themselves. I understand this is not done consciously but it is pervasive IME.

To me, that is the advantage of a STAR like system. It won't elminate the internal bias but I think it helps somewhat reduce it. Of course, I still have seen people override STAR for the candidate they like - or weigh certain questions more heavily that favor their "like me" candidate.

My brother's company does personality testing too - he says it works freakishly good to identify good fits. For example, if you are applying for an analyst job, you have to have the analyst personality traits as defined by this test. He has overridden it twice - both times he regretted it.

We don't do IQ tests. For some industries, it might make sense to do one.

Click to expand...

Can I skip the IQ test if I give them my Mensa card?

I *really* don't think an IQ test is necessarily a good measure of who you should hire. And where are you going to draw the IQ-line, and for which jobs - and what backup do you have re: drawing those lines? What if specific groups do more poorly on IQ tests than others do - and not for reasons of intelligence? Total landmine, this one.

My last company did that too. Now I used whenever I go on an interview whether the interviewer asks the question that way or not. So if they say "what would you if you were in X situation", I reply with "when I worked at Y, we had a similar situation and this is what I did and how it turned out and what I would the same or differently if presented with that situation again."

I think answering this way makes you seem like a more solid candidate because it reminds them you have experience doing what they want you to do.

I don't think IQ tests tell anything about what a person is like to work with or if they have the skills to do the job and I don't think I've ever taken any test as part of the interview process that I thought was worth a damn.

I also have found that if I ignore my gut feelings about a candidate and hire them anyway, I'm always sorry because whatever I was worried about turns out to be 10x the problem I thought it would be.

I don't think IQ tests tell anything about what a person is like to work with or if they have the skills to do the job and I don't think I've ever taken any test as part of the interview process that I thought was worth a damn.

Click to expand...

Me either, and I'd frankly find it a bit invasive.

I've worked with some people who are clearly very intelligent who weren't good performers, and I've worked with people who couldn't find my city on a map or give me the square root of 9, but were fabulous workers - at all levels.

Another one that isn't necessarily a good measure is education. Some people simply can't afford to go to the best schools, and shouldn't be judged as lesser because they went to a lesser school, or for that matter didn't go to college at all. For several years I worked with a brilliant CEO of a multi-billion dollar company who taught me a ton about business and management. We needed a bio for something, and I found out that he had just a high school diploma. He had literally worked his way from the bottom to the top of this company.

I once had a Supervisor ask a candidate, "If you were a Disney character, which one would you be and why?" I don't think there was any value to it, but the answers were always entertaining.

Click to expand...

I think that sort of question is to see how the interviewee can think quickly on his feet.
My answer for animal was a Swan. They paddle like crazy underneath the surface to get where they need to go, but all anyone can see is them gliding smoothly on top.

This is both on and off topic - but one thing I have consistently seen in interviews / hiring over the years is that people want to hire people like themselves.

Click to expand...

I would tweak that a bit to say people want to hire those they feel comfortable being around and who complement the workers/working style already there. People spend a lot of time at work, and they want those hours to be relatively enjoyable. If you're working with someone who is productive, but unfriendly, that doesn't necessarily make for an enjoyable work environment for others.

Each place of work has its unique culture, so it's not just a matter of being able to do task A, B, and C--one also has to be a good fit for the overall work environment. And that's where things like personality and gut instinct come in.

I would tweak that a bit to say people want to hire those they like and feel comfortable being around. People spend a lot of time at work, and they want those hours to be relatively enjoyable. If you're working with someone who is productive, but unfriendly, that doesn't necessarily make for an enjoyable work environment for others.

Each place of work has its unique culture, so it's not just a matter of being able to do task A, B, and C--one also has to be a good fit for the overall work environment. And that's where things like personality and gut instinct come in.

Click to expand...

This is right on the money. My youngest daughter has a small corporate job. Her company has taken her for the last 3 years back to the university she was graduated from to do informational sessions. This year, they took her to those and then again a few months later to do actual interviews, she did twelve during one day. She said that it was usually the director or vice presidential level worker who had done these interviews. But she thought it was a good idea for her to do them because she could tell over the length of each individual session which candidates, most of whom were usually of the approximately same level of intelligence and background, would fit right in with the culture of her company, and who would not. I am sure she wasn't the only interview these kids had, but I can see where her opinion would be of value. They hire a group of about 25 kids each year, and they go through a training program for two years but even stay friends and connect beyond that initial period. In fact, she received her first promotion beyond the training program when being recommended for an interview by someone in the "class" above her. She would not have heard about the job otherwise, and she beat out all the guys who applied.

I would tweak that a bit to say people want to hire those they feel comfortable being around and who complement the workers/working style already there. People spend a lot of time at work, and they want those hours to be relatively enjoyable. If you're working with someone who is productive, but unfriendly, that doesn't necessarily make for an enjoyable work environment for others.

Each place of work has its unique culture, so it's not just a matter of being able to do task A, B, and C--one also has to be a good fit for the overall work environment. And that's where things like personality and gut instinct come in.

Click to expand...

It's even more critical in today's dot-com era. My mom's old boss now works at Facebook and she had lunch with her there recently. Over there, nobody has any privacy. There are no walls, no dividers, obviously no cubicles, but that's the point. They want everyone to work together. If you want to talk to your coworker, he's only a foot away from you. Heck, you want to talk to Mr. Zuckerburg, he's on the floor along with the rest of the programmers.

My mom mentioned it would drive her crazy, but she's the kind of person who only needs to be given a job and goes off in a quiet corner to do it. Since that style of working is obviously not for everybody, an interview for fit absolutely makes sense.

My bf works for a very small startup tech company which involves everyone in interviews, and it's absolutely for fit. They have turned down brilliant programmers before because the fit wasn't right.

Hiring people is such a tough thing because if you get it wrong it can really cause massive problems in the workplace.

We had a couple of people in the last couple of years in our department who on first impressions presented extremely well. But as time went on we discovered one was a selfish and nasacistic brat who we ended up nicknaming Veruca Salt because her catchphrase was "it's not fair". And the other would be a nice as pie to your face and say horrible things about you behind you back. Both only last around 12 months with us. Needless to say it was "all our fault" that they left. Unfortunately I don't think an interview would pick up those kinds of things.

The latest couple of people we have are great - very professional, no bullshit - they just do their jobs. And they have no personality disorders.

I would tweak that a bit to say people want to hire those they feel comfortable being around and who complement the workers/working style already there.

Click to expand...

Absolutely, but this can be problematic when "complement the style already there" is operationalized as "gets along with me, and too bad if they're an *ss to everyone else". Or when it's operationalized as "lives in same neighbourhood as me", "goes to same church as me", "belongs to same club as me", "likes the same sports teams I do", etc.etc.. And when things like these are used to eliminate candidates who are perfectly likeable and have appropriate qualifications but don't meet these criteria for "fit".

I once had a Supervisor ask a candidate, "If you were a Disney character, which one would you be and why?" I don't think there was any value to it, but the answers were always entertaining.

Click to expand...

My sister is a headhunter of sorts and she asks questions like that when she gets bored with canned responses. She said it often cuts the tension and the candidates relax a little and show more of their true personalities.

I tend to avoid hiring people like myself. I find that their flaws jump out so much more obviously. I'd rather hire someone who can enthusiastically approach all of the things I'd rather not be doing.

As someone who tests 100% introvert but can pass as a mild extrovert, I try to give people a balance of exercises:
1) Phone interview
2) Writing sample
3) In-person interview
4) Timed analysis/written exercise
5) Homework assignment due a week later

By the time I get to #5, I'm usually confident I'm going to hire someone. In the cases where I have declined someone after #5, I've had serious doubts about whether the writing sample submitted was actually done by the same person. And/or that the person really wanted the job.

I once had a Supervisor ask a candidate, "If you were a Disney character, which one would you be and why?" I don't think there was any value to it, but the answers were always entertaining.

Click to expand...

I would seriously question whether I wanted to work for a company that asked such ridiculous questions. Interviews are a two-way street (at least they are with good economies and people who are employed).

I'd be curious to see how different cultures/groups react to such questions, too. I suspect these questions run a high risk of bias and/or cultural relativism.

We had a couple of people in the last couple of years in our department who on first impressions presented extremely well. But as time went on we discovered one was a selfish and nasacistic brat who we ended up nicknaming Veruca Salt because her catchphrase was "it's not fair". And the other would be a nice as pie to your face and say horrible things about you behind you back. Both only last around 12 months with us. Needless to say it was "all our fault" that they left. Unfortunately I don't think an interview would pick up those kinds of things.

Click to expand...

Next time try asking if they need shuttles to get to work. Maybe that way you can weed out some whiny, negative candidates with attitude problems.

Absolutely, but this can be problematic when "complement the style already there" is operationalized as "gets along with me, and too bad if they're an *ss to everyone else". Or when it's operationalized as "lives in same neighbourhood as me", "goes to same church as me", "belongs to same club as me", "likes the same sports teams I do", etc.etc.. And when things like these are used to eliminate candidates who are perfectly likeable and have appropriate qualifications but don't meet these criteria for "fit".

Click to expand...

At my last place of full time employment, the key factors were 1) Are you a graduate of this high school? (male or female) and 2) Are you willing to flirt with, giggle at, toss your hair, and seductively pat the principal on the arm while speaking to him? (female).

One of those things would get you hired. Two of them would make you a department head and director of curriculum as a second year teacher new to the faculty.