The Online Code Test

Lately I’ve been reading a lotabout how hiring in tech sucks. More specifically, the interview process for developers is known to be pretty inadequate towards finding great developers who ship great code and contribute to great workplace environments.

I’ve gone through the interview process at several companies since graduating from code school in November, and overall have had a mixed experience. I had some good experiences, but also some duds. What really sticks out to me is being asked on two occasions to submit a timed, online coding exercise. Both times it was a terrible process that I really don’t feel gave even a halfway decent gauge of my knowledge and skills.

The first time was through a recruiting firm. This test was, I suppose, their way of vetting their candidates before submitting them for positions. It was an all multiple choice, timed test based on Python. I believe I had two hours to complete 50 multiple choice questions. I was confused by this test because I wasn’t actually writing code? Most of the questions were something along the lines of “Why won’t this code compile?” or “What kind of output would you expect from this code?” or “What code would do X thing?”. While this works in theory, in practice I just used Google and the Python REPL a lot. I actually scored pretty well on this test and was put into their pool of potential developers as a result.

My problem with this test is that while I did well, it didn’t show how I would approach problems or that I could actually find working solutions to given problems. It really only tested that I could read basic Python and use the REPL, and I guess that does say something, but still doesn’t give a good idea of my development skills.

The second time was a far more unpleasant experience. For this test, I used Codility and was given a little under two hours to write code to solve three different problems. Once again, in theory this sounds like a decent way to give code tests. The applicants are actually writing code! It’s graded by Codility, so in theory is unbiased. While I DO think that this model has much better potential I also had several problems with it.

For reference, the job I was applying to was in C#, they knew and were cool with the fact that I knew exactly zero C#.

Problem 1) I only had a few language options with which to perform the code sample, any C language (C, C++, C#), Java, Javascript, or Ruby. At the time I was interviewing, I had never used any of these except a tiny amount of Ruby, so I reluctantly chose Ruby for the sample.

Problem 2) With only 110 minutes from the time I saw the problems to the time they were due, I had to figure out how to use an unfamiliar language to solve decently complex problems. I knew the basic syntax of Ruby, but definitely was much slower writing Ruby and was pretty unfamiliar with the standard library. While I was able to actually find working solutions to all three problems (with about 45 seconds to spare), I definitely don’t think it was comprehensive, beautiful, or anywhere near my best work.

I think the choice of forcing language options with a time constraint was kind of crap. To be clear, I would have been happy to do the code sample in any language. If they had asked for a code sample in C# (which would have made sense if the job was in C#), I feel confident that not given a strict time limit, I could have submitted a pretty decent sample.

My irritation with this code sample is also with the company who I interviewed with. If you know I don’t know ANY C# and you’re still very willing to interview me, why force me to write a timed sample with it? Had I not known the bare amount of Ruby I knew, I would have gone into the sample effectively blind and would have probably done much worse than I did.

Once again, while I can see what they were trying to achieve with this test, I still don’t think it was an accurate portrayal of my skills as a developer. It was an accurate portrayal of how well I can write code to solve admittedly asinine problems with a tight time limit in an unfamiliar language. If that at all relates to my day to day work with the company in question, then I don’t think I want that job anyway.

I have been asked to do code exercises on two other occasions that were actually pretty good representations. Once for my current job, and at another company I interviewed with. On both occasions I wasn’t given a strict time limit, and the samples were to be written in the languages that the companies used on a day to day basis. As a bonus, the exercises were also pretty relevant to the kind of code I’d be writing at both companies. These weren’t asinine problems at all! In my current role, I’ve actually written production code that’s pretty similar to the code I wrote in my exercise.

While I understand the extra time involved in giving candidates good coding exercises, isn’t it worth it to get an actual good representation of the people you’re trying to hire?