One of the hardest things to do when hiring a software developer is to establish his or her level of expertise. It's pretty easy to learn what their attitude is like after a bit of time in an interview. But actual programming experience is tough. Some companies rely on various tests, but in my experience, those tests often evaluate rote memorization that the modern development environment does not require (e.g., between the IDE's autocomplete, F1, and the Internet, library knowledge is not as important as it used to be). Here are some questions that are good to ask a developer, and why they are good questions. If you don't feel comfortable judging the answers on some of the more technical questions, have one of your senior developers sit in as well. These questions are designed to be generic and allow you to "pre-screen" a candidate before you get to questions specific to your projects.

CS 101 questions

I had assumed for a long time that everyone billing themselves as a "programmer" had a certain baseline level of knowledge. I discovered that this was not the case at all! While not every developer has a degree in computer science, it should be expected that every developer understands certain fundamental ideas. While the ideas may not always come up in day-to-day programming, they are important for developers to understand. Some of these questions I like to ask are:

Explain the difference between "equality" and "equivalence" (credits to TechRepublic member Tony Patton for this question).

What is the difference between "pass by value" and "pass by reference"? How are these ideas different in object-oriented systems and procedural systems?

Describe "polymorphism."

Compare and contrast "pessimistic locking" and "optimistic locking."

Any candidate who cannot successfully answer the first two is "entry level" at best. The second two should be answerable for any "intermediate" developer.

Thought process

Much has been made of some companies like Microsoft and Google and the intense tests they use to evaluate how candidates approach problem solving. While those tests are well and good, the fact is that few managers and companies have the time to spend a full day interviewing job candidates and having them play puzzle games. All the same, it's important to get an idea of how well someone can think about a problem and communicate the solution. In my experience, the best example of this kind of question is "The Chicken Question." I first encountered it on a job interview about three years ago, and I thought it was an excellent question. You ask the candidate, "if you had your way, how would you design a chicken, and why?" There really are no "right" or "wrong" answers here, but a candidate who knows how to approach a problem well will be able to give a detailed answer. Some candidates will like to play "philosopher" (or worse, "heckler") and ask questions like, "Are we sure that we need to design a chicken in the first place?" These kinds of answers can give important insight into their work attitude as well. This question is a good one for candidates looking at senior developer or software architect positions. Of course, it doesn't have to be a chicken, either. It could be any kind of animal, a piece of machinery, etc.

Whiteboard programming

Something that has become less common in interviews (as far as I can tell) is "whiteboard programming," where the candidate is given a problem to solve and asked to use pseudo-code on a whiteboard to write an application to solve the problem, explaining along the way why they are doing what they are doing. In some regards, this is a more hands-on version of "the chicken question" and shows what the person would be like as a developer. You can use a variety of different scenarios for this, maybe something applicable to your environment or perhaps something more generic. The keys to a successful question here is to make sure that it can easily be expressed in a whiteboard's worth of pseudo-code and does not involve too much library knowledge. After all, you're looking for someone to demonstrate how he thinks about programming, not how well he functions without any tools! Some examples that might be appropriate whiteboard problems:

Yes, these are fairly basic questions, but you would be shocked at how few developers can give a decent, short piece of pseudo-code that handles them! Feel free to pull an example from the realm that your specific position hires from, of course.

One interview I went to had a unique twist on whiteboard programming: Instead of explaining the problem to me and having me write the pseudo-code, they wrote pseudo-code and asked me to explain what it actually accomplished. This is an excellent tool and makes a good test for someone's ability to delve into code and start troubleshooting or making changes.

Code reviews

Code reviews are a common part of the development process. Something that I've seen in interviews that can be effective is to show the candidate a piece of code and have her review it. Of course, the code should not be perfect, but at the same time, the flaws should not be the kind of things that a compiler will pick up like a missing period. Instead, you should be looking for a candidate who can make optimizations, find "fence posting errors" (and other common mistakes), and so on. This will show that that person understands the difference between good code and poor code and know how to write good code.

"The usual suspects"

In most interviews, there are a few standard questions that get asked. Things like, "What is the hardest challenge you have faced in your current position?" and "What do you consider your biggest weakness?" These questions are fairly worn out, but they do serve a purpose all the same. It's helpful if you focus the question on development. For example, instead of asking, "What's your biggest weakness?" try asking, "What aspect of Web development do you feel weakest in?" (or whatever kind of development your project is in). If your candidate lacks the exact set of skills that you're looking for, don't ask them, "Do you think you could come up to speed quickly in these tools?" Because, of course, she will say she can. Instead, ask for examples of how she has been able to perform jobs despite lacking the skill set at the beginning.

Closely examine the experience

One of the biggest problems when evaluating resumes is that candidates sometimes turn even the slightest association with a project into a central role. For example, the person who acted as a scribe in one brainstorming session might list "extensive specification development experience" on his resume. You really need to ask specifics about the past experience, particularly anything that interested you in the resume. Drill down into the answers. The following dialogue between Susan (the interviewer) and Kevin (the candidate) should give you an idea of how this goes:

Susan: On your resume, it says that you were key in developing a new CRM system. Tell me a bit about that.
Kevin: It was a Web application written in ASP.NET with C# and Microsoft SQL Server.
Susan: What was your role in that?
Kevin: I focused on the data access layers and business logic and worked with a designer to tie it to the UI.
Susan: What technologies did you use to communicate with the database, and where was the business logic stored?
Kevin: Our DBA set up the initial table structure, and I used NHibernate to work with the database. From there, I created custom data access objects that exposed only the appropriate methods and functions, and I performed some additional data validation and transformations that were not appropriate to do in NHibernate or in the database itself.

As you can see, Susan was able to get to the heart of the matter very quickly. If Kevin had not been able to clearly define his role in the project, the technologies used, their limitations, and so on, it would have been a sign that Kevin's resume overstated his expertise.

Conclusion

Hiring a developer is not easy. And when you hire the wrong person, it can take a while to find out that that person has been making a mess of things, and sometimes even longer to reverse the damage. In an ideal world, we could use a lot more time than the typical interview to get to know the candidate, but we rarely have that luxury. These tips should get you on the right track to separating the poor developers from the good ones.