Avoiding Cognitive Biases in Hiring

by Lowell Vaughn

The human brain is often compared to a computer. There’s one problem with this analogy: if we treat it like a real computer, the human brain is remarkably buggy. It has all kinds of defective logic and can be thrown off in a myriad of ways. If it were a silicon based computer, the QE team would never let it ship. There are so many bugs in our brains that people have created lists of the common ones – they call them cognitive biases. In all areas of human decision making, from managing baseball teams to managing hedge funds, there has been great interest in recent years in reducing the impact of these biases.

Blind Code Reviews

Of course when people hear “biases”, they often think of biases based on gender or ethnicity. Of course, we want to avoid those biases in hiring (after all, it’s not just the right thing to do, it’s the law). Going back to the 1950’s and the Boston Symphony, many times people have found that people who were hiring were in fact biased, even when they believed themselves not to be.

It’s probably impossible to completely avoid any biases that people bring to the table when hiring. (After all, you do actually need to meet the person and talk with them at some point in the process.) However, we have made a change that eliminates it in the early stages.

All of our software developer candidates for R&D positions at WP Engine are given a short “take-home test” problem to solve. It is then reviewed by two of our engineers to decide if the candidate should come in for an in-person interview. In the fall of 2016, we started redacting all personally identifiable information from the submissions these engineers review. They just know that there’s a submission, and they vote to either bring them in or not. In the case of a tie, we bring in a third reviewer.

Avoid turning it into a highly stressful affair

Another bug in the human brain is that it gets dumber under stress. One manifestation of this is during on-site interviews, where it’s common for tech companies to place interviewees under more stress than they would be on the job and have them code in a manner that they never code on the job. This leads to a hiring process that is less predictive of an engineer’s real-world performance.

We try to minimize this in a few ways. One way is reducing an emphasis on whiteboard coding. It turns out that because “whiteboard compiler technology” is lagging, we never actually write our code on a whiteboard while working. How someone does with a whiteboard isn’t necessarily indicative of how they’ll do in real life. Many people also find this much more stressful than normal coding. What we do is we have the candidate bring in a laptop (or provide them with one if needed) and have them work on their coding example in front of us. This actually shows more of their skill than a whiteboard and is less stressful.

“Throwing numbers”

A cluster of biases we like to avoid are Authority Bias, Anchoring, and the Bandwagon Effect. Authority Bias comes about when people sway their own opinion based on the opinion of someone in authority (be that either because the person is a manager or because the person is more senior in the organization). The bandwagon effect and anchoring are related biases where the first opinion given will have a disproportionate impact and the remaining opinions will cluster around that one and each other. For example, the first person who gives their opinion of a candidate will have a disproportionate impact on the rest of the discussion (and people will alter their opinion based on it). Humans are evolved to go along with the group. The downside to this is that valuable, dissenting opinions can get downplayed.

To avoid this, the interviewers do not talk amongst themselves* after an interview. We have a round-table meeting where all of the interviewers gather. At the start of the meeting, the recruiter counts to three and all the interviewers hold up between 1 and 4 fingers based on their opinion of the candidate:

1. Strong no. Based on the interview, the interviewer can’t be convinced that we should ever hire this candidate for this position.

2. Weak no. Feels like we probably shouldn’t hire this person, but could be talked into it.

3. Weak yes. We should hire this person, but the interviewer has some reservations about it.

4. Strong yes. Is this person still in the building? If so, we should tackle them in the parking garage and make them an offer before they go.

One thing to note here is that there are no half numbers. Especially, there is no 2.5. Everyone has to say “yes” or “no” before knowing what everyone else has to say, thus making sure we see everyone’s opinion before it’s influenced by either a more senior person or the group. We then go around and give our reasoning for our number. Sometimes this will move other people’s opinions one way or another over the 2-3 barrier (e.g., He said what? Yeah, we shouldn’t hire him).

There’s another bias we try and avoid here, which is that after you’re exposed to something or someone, you tend to view them too favorably. To counteract this, we try to err the side of throwing a 2. 1’s are not technically a veto, but someone getting hired when a 1 is thrown is rare, indeed. Unless you feel that strongly, leave the option open. Conversely, hiring the wrong person is bad for the company and ultimately bad for them (who wants to explain why they got fired after three months on the job?). So, if there’s doubt, don’t hire them. The ones you should have hired will find jobs anyway (the privilege of being an engineer is that finding work isn’t hard).

What does it all mean?

The human brain is a remarkable thing. It has the power to send us to the moon and create the Internet, but it also has the ability to do really silly things. When we’re hiring, we’re trying to get optimal decisions using this horrible hardware. The good news is that the brain is smart enough to see its own weaknesses and design systems around them. At WP Engine, we’ve worked hard to build those systems to try and hire the best people possible. If you’d like to see the process first hand, we’re hiring!

* Unless it is to ask a later interviewer to cover something they feel they didn’t get time to cover.