Thursday, March 22, 2007

Howto Pass a Silicon Valley Software Engineering Interview

I do a fair bit of interviewing. This probably averages about 2 to 3 interviews per week - mostly for Java developers. I'm also giving a birds-of-a-feather at the Software Developer's conference tonite in the Santa Clara Hyatt bar on just this topic (7:30pm if you're interested).

Now mind you, being an experienced interviewER does not necessarily give you relevant information on being an experienced interviewEE. However, the latter is hard to gain a lot of experience at. Because of course, if you're good at it, you tend to not do it for very long (i.e., you get hired).

Before Google, I interviewed at several startups in the valley. And combining that experience plus what I know about how I interview at Google and how I interviewed at Preemptive I do have a few guidelines.

1) Don't interview at your dream job first.

If you haven't interviewed in awhile, your first interview is likely not going to be great. It's not because you're not a crackshot developer or a math whiz. It's just because you aren't familiar with the whole process. From getting used to jumping from topic to topic all the way to saying why you want the job. Its always a good idea to interview at your 3rd or 4th choice first.

2) Be positive - no swearing.

You will get asked about your last job. Saying your manager sucked and the dev team was a mess wins you nothing. I've seen candidates attempt to put down some technical faction or previous employers seeking my solidarity with them. Nuh uh. Doesn't happen.

Why are you leaving your current job? Simple - because this new job is a better opportunity. Your last job was a fine career builder but this one's business model or development principles or philosophy or job description or reputation suits you way better. Not to mention your skillset can bring significantly more value in this new position.

Don't tell your new employee what's wrong with their products - mention you hope you can (non-specifically) improve such products. Even if asked what you'd improve, phrase it such that it is indeed an improvement and not a fix for something you think is terrible.

Also - forget technical religion. If you love Agile say so - but don't pretend its the only solution. Millions of people get work done on windows, linux, agile, waterfall, C++, java, .NET everyday. All are solutions - sure, some are better than others and defensible positions are great - but unfounded zealousness is not.

And, I am amazed I need to write this - don't swear. Its a respect issue. You don't know your interviewer. Some people don't mind swearing - some do. There's really no need.

3) Check your attitude at the door.

As I've said in previous articles, if you are the smartest person at where you work - QUIT. Similarly, its a silly idea to join a company where you will be the smartest person the day you start. Therefore, if you ARE smart, you will be looking to join a company where you AREN'T the smartest person. Therefore - you should leave your arrogance about how great you are outside.

I ask most every candidate to rate their Java and C++ skills on a scale of 1-10. Then I write that down for the next interviewer. At Google, you never know who your interviewers are going to be. If you say you're a Java or C++ expert that rates a 10 - you darn well better be - because you never know - your next interviewer could be Josh Bloch, Matt Austern, Guido van Rossum,or Ken Thompson. Or worse, someone else you've never heard of that's a super crackshot - and there are plenty of those.

Again, I'm amazed at people that give their interviewer attitude. It's such an obviously stupid act that I have to question the person's intelligence in addition to being annoyed by their arrogance.

4) Be passionate about development.

I have a dirty secret - if Google stopped paying me tomorrow, I'd still come to work (unless they like took my badge too and got Hector the security guard to watch out for me - that dude could smoke me). God forbid that while sleeping I have a dream that solves a coding problem I've had the day before. When this has happened in the past, I found myself sitting awake in bed with my mind racing. I was too excited to sleep and figured I might as well drive into work and start implementing the solution (despite the fact that it happened to be 4am).

The number one thing I look at on resumes (and I don't look at resumes all that much) is extra-cirricular coding activities. I want to hire engineers that I want work with. And those engineers are passionate about cool algorithms, slick code, and new ideas. They do that stuff in their spare time - its not just a job, its what they do because they love it.

5) APIs really don't impress.

People seem so proud they know a lot of APIs. As far as I know, APIs were designed to be easy to learn. I'd really rather hire a smart person that I know can learn most any API than one that brags about the few they already know.

I'm not saying knowing APIs is bad - it's not. It's just not the most interesting thing on your resume.

6) Know algorithms and data structures.

One theme in Silicon Valley is massive amounts of data. And its not always of the classic relational database type of data. Its massively huge datasets that require plenty of processing (imagine the graphs made for page rank).

One of my favorite/fun interview questions (actually, probably one of my ex-favorites given that posting it here means its too well-known) is simply how to sort some objects. The absolute beauty of this question is that very many software engineer interviewees have given me a suboptimal answer for this - whereas my mom (who, despite making crazy good pirogis, has zero computer training) got it right.

Here it is, as I ask it of engineers, and as I asked it of my mom:

Engineer's version: Say you had a million objects in memory (assume we have no memory constraints) all of type UniversityStudent. These objects have two fields:

String name;int numberOfYearsOld;

What is the fastest way to sort these objects by "numberOfYearsOld"?

... So.. whats the answer? quicksort? mergesort? whats the running time?The most common answer I get is something like quicksort with an average running time of O(nlogn). For a million objects, that's something like 20million operations (comparisons) to do the sort.

Mom's version: Say you came into a very large room with a million papers in a stack. On each paper is written the name and number-of-years-old of a given student. Whats the fastest way to sort these papers by hand by number-of-years-old?

Mom made stacks. A stack of 18yr olds. A stack of 19yr olds, etc. She needed a possible of about 100 stacks maybe (ages 10-110). How many times do you need to look at each paper? Once. Right. That's 1 million operations or O(n). Go mom.

Of course, Mom got it right because she had no preconceived notions about the problem or sorting in general. A candidate that memorized sorting algorithms before coming to the interview probably robotically responded with O(nlogn) without really thinking about the problem.

If you've written plenty of code, you should be familiar with when to use what data structures and to know their runtime characteristics. You should know that a hashtable's worst case search time is linear - and you should have an idea how to avoid it. And why you might use a binary tree instead of a hashtable even though it's an O(logn) lookup. And that O(1) is effectively the same as O(100). Surely the subtleties are situation dependent - but that's why you understand it - to apply it in the right situation.

This is all datastruct and algorithms 101. I perpetually hear developers tell me that they learned that stuff in school but now forgot it. Personally, I wonder what the hell they have been coding? If you've just been gluing APIs together then thats nice, but its not very interesting. Even if you don't interact with them directly, knowing data structures and algorithms is key to understanding performance. This is not premature optimization - this is choosing the right tool for the job. And that choice is often wonderfully subtle.

And if you decide to do 20million operations when you could very easily instead do 1million, eventually we're going to have some problems.

7) Be an engineer that your interviewer would want to work with.

I don't know exactly what that means because it will vary with every interview and interviewer. Obviously be genuine but be passionate.

8) Know the language you say you do.

I try to phrase all language questions as things that I think any developer that has worked in a language for a year couldn't possibly not know. Tell me about wait, notify, and notifyAll (3 methods every Java class ever created has).

I'm not worried if you don't know Java generics, but if you say you do, I'll ask for some code.

Good interviewing coding questions in my opinion should be meaty, do something, and take no more than a whiteboard.

-------------

There really is a spectrum of good to poor engineers. And the one theme that runs through it all is passion. Not for a given language or system - but for problem solving. And building things. Certainly a good degree doesn't hurt but I promise you its not the whole story. I have flunked MIT Ph.Ds. and recommended-for-hire people with very modest formal educations.

As I've said before - the interview is very very honest. Its about you, the whiteboard, and what you can do.

Bottom line is I want smart, passionate, crackshot developers. They're out there and I want them here - partially because they'll help to make my company better. But also because they're very likely going to be smarter than me - and working with them is going to make me better.

These are all great recommendations. I really like #3 and #4. Those hit the nail on the head. I look for those two almost immediately. At my previous gig I was doing a lot of interviewing and I'm starting to do more at thew new job and you'd be shocked at how little passion most developers show.

The programming quiz was also a good one. I use one where I essentially have them implement a weighted round robin load balancing algorithm and you'd be surprised at the responses I get. Most try to over complicate the problem to no end but some just refuse to do it because they don't see the "practical application" -- I pretty much flunk those right away. Attitude problem..they don't know the answer so they think their response is somehow better? Buh bye. :)

I don't really understand how being passionate and genuine fits together with not being negative? To me being passionate always entails criticism, and even if constructive, it's always negative. I won't improve anything, including myself, if I don't see anything bad!

I'll give you an example. Let's assume that I'm the smartest guy at where I work, and let's say that I have no passionate co-workers or managers around me and I'm sick and tired of being THE ONLY ONE WHO CARES. Would you want me to explain this to you as I feel it, at the risk of being negative, or would you want me to concoct a lame fable about how I'm looking for new challenges even though my current job is a "fine career builder"? Would you want me to show you how passionate I really am, or would you want to see how clever a bullshitter I am? How genuine would that be? You're not looking for honest and passionate people, you're looking for form fitting bullshitters! ;)

Anyway, I guess there are as many ways of being a successful interviewEE as there are interviewERs. You never know what you'll be judged on. But if you have nothing negative to say about you're current job, I don't understand why you'd want to leave it?! And I wouldn't buy just any tame tale you tell me about it.

Thanks for these insights. I've been thinking about applying to Google, and your post makes it sound less imposing than it might be otherwise.

The first question my company asks interviewees (who are usually in school or have recently graduated) goes through the performance differences between LinkedList and ArrayList, how to write an Iterator, and how concurrent access should be handled. We can usually tell within five minutes whether someone will rock, bomb, or struggle through the interview admirably. Good engineers answer easy questions quickly. Bad engineers would be lost on a hard question anyway, so starting easy doesn't make them feel so bad and doesn't involve as much socratic frustration by the interviewers.

This advice applies to many, many different kinds of jobs and to so many other things you do. Okay, minus the specifics. But here we have all sorts of good advice, from practical positive thinking to practical Applied Zen. Write the whole darned thing on a piece of paper and read it all once a day for a week. Repeat a month later.

Paul, this is great. The Tin Man already had a heart when he went looking for one. Anybody who thinks he needs or wants Passion already has passion. What is passion? It's the alternative to television.

I'd love to work with you. Not because I'm an engineer—I'm not—but because life is worth living when you're with people who think life is worth living.

Thanks for this very insightful article. I am actually using it in order to prepare myself for some interviews.

I must say however that I don't quite agree with some of the points.

I believe that it is very easy to get sucked into positions for a long time where you do glue APIs together. If for instance you work as a J2EE developer you will gradualy be removed from lower level programming concpets such as bit manipulation ,memory managment or any algorithm development. I am sure there a a few occasions where that is not the case , but the majority of these positions will not make that a neccessity. In fact the whole intent of the J2EE was to make the developer concentrate on the business logic without worryint to much about security or memory management. Most programming jobs these days involve devleoping some sort of an n-tier app where you read some user input , perform some business logic and the update the database. I do admit however that game programming, low level embedded software or firmware development still require the programmer to be on top of his game in those areas, but to believe that someone is not a good programmer because they did not get the chance to be exposed to those areas is in my opinion not correct.

My quesion to you is, do you believe that someone applying for google should be at a level where they are extremelty proficient in those areas as you proposed or is rather that you expect them to study those concepts to get to that level for the interview?

Also you mentioned that at the interview it's all about the white board and what you can do. Although I do believe that someone who does perform well at the white board is probably a good candiate I do not believe that everyone who doesn't is not. I myself can solve a lot of problems if I am given some space to do it. when people watch me however I tend to choke due to peformance anxiety. Now I don't believe that this has anything to do with my intellectual ability and I cannot think of any situation at work where you have to solve a problem while people are watching you. So don't you think that this exercise might cause lot of potetnialy great employees to be overlooked?

For the sample with Students/Ages I'd never use anything from API (or forgotten algos...). I don't think in Array.sort() terms or any List etc. `!`The correct answer depends on papers, rooms, and baskets. Or may be on physical data structure? Wouldn't we get Mom's solution by indexing it? What do we call 'index' indeed... and why do we need to sort arrays... I've never ever sorted arrays!

About scale 0 to 10: answer is simple, there is no right answer. Some interviewers like determinant & confident people scaling themselves 10; some people like 'thinking' fuzzy roving 'may be... between 7 and 8...'; something goes wrong if after answer '10!' I'll get interviewed by Josh. It is not the only functional dependency.

I had a very talented student who preferred to earn money being a SysAdmin in another university... We had a competition, so-called 'Olimpiada' (in Russian), and the task was:Given a number of rectangles with coordinates, calculate the space of union (sorry for my English, smth. measured in square feets for instance).

He created the algo easily! He just printed all squares on a screen, white shapes on a black screen, then he counted total number of white dots:)))Of course this algo has limitations... size of screen... but it works for different shapes.

Ok, here some thoughts... I was asked questions which definitely do not have correct answer, such as a question about sentence retrieval without language parameter (in some languages some single characters have equivalent character sequences)... and of course I am not ready to accept any kind of possible offer, it's very complicated, Google is still startup speculating on reselling Ads.

About famous authors willing to work for Google: I have a lot of books, but I've never read any of them from-cover-to-cover. Even "Thinking in Java". But books help sometimes and I have a lot of them, including some new Flex staff, and even very old book "Inside JVM".

After 3rd interview I can say: you still do not have skilled interviewers. "Good employee" is not the same term as "expert", and "good employee" has very specific meaning. About "thinking process" - nice idea, many people just learned some staff without any ability to write simple algo from scratch and to optimize it.

Anyway...

I believe companies like Goggle can develop future employees starting at Universities.

Thanks for a great post. I lost in the final round of interviews at Google - mainly I think it went too plain. Lot of practical advice in your post...I am sure it will help me in my second attempt at Google (if they call again).

By the way the link to the "Notes from my BOF can be found HERE" is broken. If it was some interview preparation material, kindly put it back.

Are you saying the recommended interview behaviors you mention, correlate to Google success? (both for Google goals, and employee goals)

I'm trying to read between the lines.

Or are you saying it's all the same for software engineering everywhere.

I guess I'm wondering about this "google interview" stuff. If there's something unique about Google's needs, then I would think the things you think are important should reflect that uniqueness? or no?

I see a lot of attitude in the post. What have they been doing since college if they don't remember data structures and algorithms? They have been coding solving all other problems that doesn't necessarily involve algorithm analysis. Programmers of that kind are also on high demand. I have worked for a big company and has great technology and in the past decade a need for doing algorithm analysis hardly ever came up. I am not a junkie in that company, have a great reputation, I am seen as a problem solver, highly productive and overall a valuable person in the teams I've worked with.

I have been coding for over 20 years in many different languages. My code works and is delivered on time. The efficient compact and elegant code frequently contains bugs and comes in late. My sloppy redundant code works on time. Do I need to change? As far as Sorting. I would implement Comparable and use Collections.sort not really caring what log n would be.

"I perpetually hear developers tell me that they learned that stuff in school but now forgot it. Personally, I wonder what the hell they have been coding?"

I didn't know whether to laugh or cry when i read this. Do you honestly think every developer is writing data structures? Most solutions which require data structures can be easily solved by existing open-source or publicly available solutions or even wrapped in the framework you're working with. Not a heck of a lot of companies are providing gigantic search engines serving hundreds of millions of users per day, and the need for a specifically-designed data structure is very low. Why reinvent the code when it's readily available and tested, especially when you're on a tight deadline?

I do realize that this an important skill to have and it is a tremendous benefit at Google. It most definitely had to be emphasized in your article. However, you didn't have to sound like a God-like Google genius and degrade everyone who is beneath you by saying that whatever you do equates to nothing if you don't write data structures. That's like saying "If you don't code in NotePad you're really not a programmer."

I agree with your sentiment that my wording was a bit snooty. But I still agree with the underlying principle.

That is, programming is data structures. I'm by no means thinking you are or even should be writing your own hashtable - but even a few levels above that you are (or at least I am) thinking about how data is internally stored and how it interacts.

Immediately post-google I was CTO at a mortgage start-up. I'm hard pressed to think of something farther from writing hashtables. And it was indeed often a "glue API #1 to API #2" type environment.

But even then, there were day to day instances of considering the performance and size constraints of dealing with the data. (And surprisingly, I did have to develop one highly combinatorial algorithm to do "reverse mortgage qualification" - that is find all permutations of financial changes a user could do to qualify for a given rate).

And, I will point out that the article was purposely titled to be for Silicon Valley.

I am trying to post second time, I believe it is important...I recently found excellent post http://blog.notdot.net/2007/4/Damn-Cool-Algorithms-Part-1-BK-Trees(another Googler); I was inspired, thinking all night about this simplest thing, next day I contributed important code to Apache Lucene (full-text indexing service). It is BK-Tree structure, it allows "quick-find" similarity using distance (spellchecker, etc.)Even at Google, it took 34 years to find it (probably Google employees spend some time in university libraries?)

I think smart (really smart!) guys could do it... just reinventing the wheel without asking for applauds and without trying to Google it, and without trying to patent it.

Of course such companies as Google need developers who can write custom rt.jar! (such as Joshua Bloch?)

For instance, in "crawler-like" applications we can't use java.net.URL class because it is synchronized.

About job interviews... Good luck all! I am not looking for permanent job.Most developers are forced to focus on implementing "business use cases" without thinking of String class bottlenecks for specific "robot use case". Others know nothing but something specific, like ja.io internals...

I really like your article, Paul, 3 years later it still resonates. I've decided, working in Silicon Valley for 15+ years, that there are two kinds of interviewers: they want "rock stars" or they want "employees". The rock stars are what you call "crackshot" and can invent a new data structure. Employees- more practical, a bit more pragmatic, as the earlier commenter said, they may turn in long verbose code but it works and has less bugs.

I also have to admit I really dislike academic questions. This isn't a quiz in college, this is real life, where sadly there are very few isolated instances of "a million students with grades". I can say the moments I've run into those situations, I'm delighted and eager to write an algorithm. As one successful hiring manager said- "I want someone who gets stuff done." Vs. the pretty code.

I don't give a shit how smart you are or clever you are. What I care about when I'm hiring developers is can you deliver the features in my product road map or not? And by when? That's the only thing that matters.

Your criteria for evaluating reeks of intellectual snobbery and is completely oblivious to what matters to the business. The ability to get shit done when it needs to be done, giving the company tools it needs to make moree money, these are the things that matter.

Making people solve bullshit CS problems does not address the problems of the business. It just makes you feel smarter than the other person when they can't solve it, or gives you some circle-jerk satisfaction when they can. It's a waste of time for most business software developer positions and rarely provide little insight to a person's real-world problem solving ability.

And who gives a shit if the developer is passionate? Passion is highly overrated. Give me someone productive over someone who's passionate any day of the week. If they are one in the same, great, but passion is bullshit.

Knowing APIs is important. They help you get things done faster. The faster you get shit done, the better off the business is going to be. The more APIs you know, the more things you can get done faster. Is knowing APIs impressive from a nerd point of view? Maybe not. But to a business, that knowledge is valuable.

My point is this. You completely ignore what the business actually needs to be successful in your criteria, but rather focus on hiring people you want to work with. That's a huge mistake and irresponsible on your part.

If I interviewed you and you came in talking some bullshit about how you once sorted a massive collection of objects in X ms, I'd tell you to get the fuck out of my office unless you can tell me how you made your last company more money by doing it.

This kind of interview suits for start ups and product company but not service based company in INDIA.recently i worked with start up company where work environment is quiet different compare to service base company which i never expected. I have done reasonably good in the interview process but coming to real work i failed so lost my job with in a month.Shall i blame myself for not doing well or my boss who took interview? because his inability to evaluate me in the interview process....

martti vh, maybe you need to understand the difference between being critical versus being negative? The distinction is clear and comes through in an interview.

That said, in my interviews I ask people to give me examples where they had a negative experience and how they handled it. If they tell me they never had one, that raises a red flag. The successful interviewee will be critical (not negative) in their answer - not only with respect to the situation, but in how they may have handled it better.

Now I know what to answer politely to such an interviewer: "I have never had any negative experience" LOL ;-) Most important: say it slowly... with serious face... and with LOL at the end. Remember, only 7-12% of communication is passed by words; the rest is voice tone, body language.For sure, if they like you (and they want you) they will ask more detailed question. Don't waste time!

I think it's sad how every interview article describes how its up to the interviewEE to convince the interviewER that he/she is great.

Its sad because that's the general attitude of HR people and why I think all HR people should be fired. HR people are like union workers. They don't care about finding a good person. They just want to be entertained and paid for it too.

In reality its a two way street, but you talk about the arrogance of interviewEEs and say nothing of the arrogance of interviewERs.

Hi Paul,Usually I do not read article on blogs, but I would like to say that this write-up very pressured me to check out and do it! Your writing taste has been surprised me. Thank you, quite nice article.

You are awesome and that is a cool post. I love the language expression and to be honest, I was in those shoes except that i went for my dream Job first (An Interview at thoughtworks) and blew it bad. Thanx for the heads up