Tuesday, February 21, 2006

Early Amazon: Interviews

Fast growth means lots of hiring. And there was a lot of hiring at Amazon. At many times, I was doing 1-3 interviews every day.

Different people had different interview techniques. I spent a lot of time refining mine. I did much reading on different strategies, tried borrowing ideas from Microsoft and elsewhere, and pulled other Amazonians into long discussions and debates about hiring.

In the end, I settled on looking for three things: enthusiasm, creativity, competence.

On enthusiasm, I wanted to see that the candidate had done at least minimal research into Amazon, the business, and the website. I was shocked by how many people came in to interview at Amazon who had never used the website. Here you are, considering spending the next few years at Amazon, and you didn't do some basic investigation beforehand? C'mon, that's just lame.

On creativity, I usually looked for ideas on how to improve the site. It didn't matter if we had already thought of the ideas. I realize how hard it is to come up with cool ideas from the outside. But I wanted to see some exploration of how things could be done better.

On competence, I merely attempted to verify what they said on their resume. If they claimed to be an expert in C++, could they answer a couple introductory level questions about C++? If they claim to know Python, can they write a trivial 5-10 line program in Python? If they have a degree in computer science, can they talk about algorithms and data structures? On some project they said they did in the past, can they talk about it in depth and with enthusiasm for all the little details?

By the way, exploring someone's knowledge doesn't necessarily require knowledge of it yourself. You can just keep asking questions, diving deeper and deeper. If they really understand the problem, they should be able to explain it to others, to teach people about the problem.

Eventually, you should get to a point where they say "I don't know" to a question. That's a great sign. Knowing what you know isn't as important as knowing what you don't know. It is a sign of real understanding when someone can openly discuss where their knowledge ends.

Reading over these questions now, it may sound like an easy filter. Minimal enthusiasm for and research of Amazon.com, a couple creative ideas on improving Amazon, and don't lie or exaggerate on your resume. Wouldn't everyone pass that?

As it turns out, no. Offhand, I'd guess that I rejected about 90% of candidates during phone screens and another 90% during face-to-face interviews. A filter for enthusiastic, creative, competent people seemed to be a surprisingly harsh one.

Of these three things, I think the single biggest predictor of success at Amazon.com was enthusiasm.

Amazon was a chaotic environment. Getting things done required initiative. Amazon needed people who would grab problems by the throat and never let go. Part of my work every day was to find them.

I hope your interview style was not of the kind shown in "Pirates of the Silicon Valley" (http://www.imdb.com/title/tt0168122/ where Steve Jobs intereviewd "an IBM typ" guy. A movie every nerd must see!

I'm loving this series about Amazon back in the day. I liked this one in particular because I too have found that those three things together make a great filter for interviews! I've gone through hundreds of interviews only to find a small handful who have done any form or research on the company. Some, like you said, haven't even seen the meat of our website or really know what we're about... And when you add on that with creativity and really knowing the stuff you said you know... you're set on finding a good guy!!

"By the way, exploring someone's knowledge doesn't necessarily require knowledge of it yourself. You can just keep asking questions, diving deeper and deeper."

I call complete bull on that. This happened to me on my interview. The person asking me the questions obviously had no clue but was following the same line of thought presented in this blog's posting.

Since they had no knowledge of the material, they weren't able to comprehend any of the answers. I DID however try to shift strategies and explain the answers without jargon and tried explain behaviors rather than syntax or implementation... and regarding implementation, it's pretty sad when the interviewer doesn't know the API he's trying to ask me about himself nor doesn't know what an event handler is -- if you call interviewees lame for not researching Amazon prior to the interview, what does that say about an interviewer that doesn't do his homework neither?

The interviewer was bent on diving deeper and deeper despite not possessing anything other than a 30,000 foot view of the material he was trying to grill me about. Eventually I just had to give up and say I don't know after about 30 minutes of unproductive back and forth on what was essentially the same thing. Ultimately, that was nothing more than 30 minutes of wasted time. He learned nothing about me -- whether or not I’m smart and can I get things done? Playing "games" in interviews is so 1990s. What do you think my impression of that little section of Amazon is now? There are much better ways of determining enthusiasm than hazing your interviewee. If you want grownups and professionals working for you, conduct your interview in a commensurate manner.

When I was at Amazon a decade ago, I would spend at least an hour preparing for each interview, which I figured was the least I could do given the effort the interviewee has to do to come to an interview. I would skim publications the person had done, look at anything that was publicly visible on the features they said they worked on at other companies, and look at any open source code they had available.

Putting in that level of preparation as an interviewer, sadly, is not always done. I've been in interviews from the other side, as the interviewee, where some interviewers don't know who I am and obviously haven't even looked at my resume, not to mention all the other writing I have done, before sitting down with me. Most recent example of that was at Google. When this happens, it still shocks me, even having seen it a few times.

Interviews go both ways. I have rejected working with teams and companies based on the interviews I have had. Life is too short not to work with good people.