I admit, that this is traditionally a security blog, however I am going to digress for a post. I truly hate the concept of a coding test during an interview. I hate them as both an interviewer and an interviewee. Joel Spolsky & Jeff Atwood would probably disagree with me, however I truly believe programing tests during an interview are pointless. I have come to this conclusion after 8 years in the software industry both, taking tests as well as giving them. The reasons that I believe these tests are pointless are: The tests don’t really prove anything; coding tests don’t simulate a real work environment; they’re a waste of time.

1. Prove nothing

I’ve often heard the arguments that you wouldn’t hire a contractor, any other contractor without previously seeing their work.Or you need to see and evaluate one’s style to see if the candidate is a good fit, you want to ensure that the candidate can actually code.

The test during an actual interview does nothing to really prove or establish any of these three points. Often times during the interview the candidate gets X amount of time to accomplish a specific task. Software today has become so diverse with so many different industries writing many types of software across many different technologies that the candidate may not be versed in all the specific technologies require to accomplish their task in the given time frame.

Does that mean that they’re a bad programmer? Of course not. Does it mean they’re incapable of working for your company? of course not.

When you truly consider the role of a developer or engineer within your organization, writing the code is actually the easy part and the end solution to the problem. This raises the question are you hiring an engineer to solve problems or are you hiring an engineer to crank out code. The obvious answer should be the former. Therefore the way an engineer actually writes code is less relevant then how they solve the problem especially in today’s industry where every company has their own standards, practices and format.

If the candidate has been employed for longer then an year, at a previous software/technology company and provided that they’re not a fresh graduate it stands to reason that they can actually write code. So too isn’t this why you check their references and follow up with their past.

2. Doesn’t model your environment

In what organization does a member of your development team actually only have X amount of time to solve a problem and write some actual code. So why are you asking a candidate to perform under pressure, already being stressed out from the interview, to write code and be evaluated in a short amount of time? Do you really think that you’re getting the best possible result from that candidate, now if one doesn’t handle interview situations well, you might pass them over because they didn’t perform on their test well enough yet, you might hire a weaker engineer, because they are capable of performing in that 1 situation better.

In the environment the engineer has a chance to think through the problem clearly design an excellent solution and lastly write the code which will illustrate the situation as needed. So why not design a test and circumstances that mirror an actual environment and allow the candidate to actual produce in the best situation.

3. Waste of Time

Often the candidate is going to write bad code. Often time the developer shows up isn’t mentally prepared to write the code. Now you as an interviewing team have to evaluate bad code that may or may not actually solve the problem that your test was designed to evaluate.

Often time these tests are designed to be so short, such that the candidate can complete it within the given time frame. The test is pointless, without meaning, often times it doesn’t expose the candidate to real problems & technologies that your organization actually works with. Nor does it actually allow the organization a chance to evaluate a candidate’s architectural abilities.

A Solution

Rather then springing a coding test on the candidate during the interview, why not E-mail the candidate the test a few days before as part of the screening process or the night before. This approach will give the candidate the time required to think about the problem plan out a solution, and actually implement the feature. They may or may not actually finish it early and therefore have time to refine and refactor it into an even better solution on your behalf.

Once the candidate has sent the test back the interviewing team can perform a code review analyze the test and prepare a standard set of questions to ask the candidate for the in person interview. Or the team can perform the code review during the interview itself. It will quickly become obvious through a discussion whether the candidate implemented the test themselves or if they hired someone else to implement the test for them. This approach would actually allow the time for a more detailed test which would test architecture abilities as well as, coding abilities, giving a much more complete and reliable solution.

Given the onset of DevExpress for visual studio, or other free IDE’s these days there’s really no excuse why a potential candidate could not grab an IDE, if they don’t already have one installed, which is telling in and of itself. There’s really no reason why a candidate cannot pick up an IDE and actually write some code at home in the evening before the test.

What am I advocating for? Scrapping the test during the interview and allowing for a more detailed thorough take home test that models your work environment and exposes the candidate to the technologies that your organization actually uses.

Share

About the Author

I am a Sr Engineer for a major security firm; I have been developing software professionally for 8 years now; I've worked for start ups, small companies, large companies, myself, education. Currently the company I work for has 7,000+ employees worldwide. I am responsible for our platform security, I write code, implement features, educate other engineers about security, I perform security reviews, threat modeling, continue to educate myself on the latest software. By night, I actively work to educate other developers about security and security issues. I also founded a local chapter of OWASP which I organize and run.

I cut my teeth developing in C++ and it's still where my heart is with development, lately I've been writing a lot of C# code & some java, but I do have a project or two coming out in C++ /DiectX 11 whenever I get the time.

When I am not developing code I am spending my time with my wife and daughter or I am lost deep in the woods some where on a camping trip with friends. If you can't find me with a GPS and a SPOT device then chances are I am on the Rugby pitch playing Rugby and having a great time doing so.

Comments and Discussions

I whole-heartedly agree that the usual code-based interview is a crock of .... (well, you know what I was going to say. ).
Anyone having any influence on the hiring process at their company should read your article!

One issue, though, with giving out a test before-hand is that this gives the candidate time to "cheat", so to speak. There should be a process applied on the candidate's resulting code or algorithm that weeds out the ones that don't actually "know" anything, but are gurus of the GoogleTAO(tm). Perhaps make them explain exactly how their code solves the given problem, or discuss their thought process and any issues they encountered while designing the code. This should make the "cheaters" quite uncomfortable, while the good ones will be happy to talk about it.

Google is a great tool for any developer (it's saved my bacon a couple of times!), but it shouldn't be the basis for every single solution a developer creates, and any pre-interview home-based testing would need to take that into account.

That's certainly something to think about, I think you could come up with cheating cases, the first being extensive use of Google, the second, paying someone to take the test for you as in through Odesk etc.

There's a couple of ways to address this, firstly the coding test should not be the entire process and interview if it is you're failing to conduct a proper evaluation. You've got references, experiences etc still to check. In Canada it's actually against the law to give a negative reference, however that's hardly ever enforced and even more difficult to prove. One of my favourite questions to ask, is "would you hire this person again?" I've had people tell me no, which I would not evaluate as positive or negative, and I've recommended hiring people after their previous employer said they wouldn't hire them again. If the response is no, then I leave it at that if the response is yes, then I follow up with why?

More on the technical side of things, and getting back to the test. I think 1 can gather from some well placed questions. Another favourite question is asking someone to justify their implementation, if they don't have a clue because they cheated maybe they can BS their way through this question. Also asking them how they might change it? That's a harder question to BS your way through, or given more time how would you improve it? I also like to make them think and take their code, and tell them I want to extend it to add this functionality. I don't expect them to code it up on the spot I expect them to have some knowledge on how they'd do something.

For example. How would you make this class, or these classes so that they could be inherited ed, in .NET they should reply with "sealed" of course, the next obvious question is well what if someone on your team needs to extend your functionality and it's wrapped up in a dll, well then they should use extension methods, so on and so forth.

I really believe this is a far better way of conducting an interview during a coding examination

"In Canada it's actually against the law to give a negative reference, " ...

That seems like an odd thing. I mean if the negative reference was not the [whole] truth, it would be unethical and be a basis for the law. However, in the scenario that you owned a company and wanted to hire someone, and lets say this applicant was always late, stole company property, and repetitively sexually harassed co-workers, all leading to their former job termination (last week).. Wouldn't it be unethical for the government to deny you that relevant information before you hire them (or for that mater, allow them near your work center unattended)? Let's also assume legal prosecution never occurred (not worth the previous employer's time/money), so there would be no criminal record (assuming that is allowed to be asked about either). It kind of reminds me of another law I heard about a long time ago (I think it was Germany), where it was/(is?) illegal for a company advertising their product to "compare" it to another brand's product.. What? Why??

Anyway.. on to my real post..

What about the option of taking actual cases of past [non-trivial] "problems" in the company (that were already solved), and present that to the interviewee (along with information about the resources that were available at the time [but not the actual resources]), and give them a fixed amount of time to come up with methods of "attacking the problem" and/or solutions. They would not be expected to have a solution, but their answers could give some insight of how they might perform (since in this "test", the suggested steps responded can be directly compared to the original problem and the way it was solved). While not a sole picture of their potential value, an indicator if they "would have" been better, the same, or worse than the other employees. Of course this technique would be of little value for cases were analysis isn't an important part of the job.

Instead of sending a coding test by email, for screening purposes it would be best to use an online programming testing service. These services are not about identifying who you should hire but rather about cost-effectively identifying who you should not hire before you spend any time of your own interviewing or trialling this person.

One of the first customers of TestDome.com stated that he now talks to 80% less people.