What kind of skills determine a person that is capable of debugging code with ease? Some time ago my friend carried out an interview with a relatively good programmer. The programmer got hired. He could write good code, understood the frameworks and design patterns. The thing he was missing was - debugging skills. He could not debug at all and finding problems with his or someone else's code was huge pain for him.

Since then we are thinking about how we can assess or estimate a person's debugging skills.

So the first question is: what skills determine if a person can effectively debug software?

I was actually given code to debug, on a computer, at one interview. They had modified the source code to tar or gzip or something, and wanted to see how I would debug it.
–
birryreeMar 1 '12 at 13:35

4

The only way to be sure is to let him or her do it "live". Let him or her know in advance what your development environment is and that there will be a simple coding and debug test.
–
user1249Mar 1 '12 at 15:03

16 Answers
16

If the first thing the person wants to do is look at the code and step through it with a debugger that person is not a great troubleshooter.

If you don't already have a plan of action and you dive into the debugger blind you are basically Easter Egging. This is true for ANY kind of troubleshooting.

In an interview situation a person that asks how the system operates and asks about history of the system would be somebody that might be a good troubleshooter. A person that thinks system first and mechanics second could be a good troubleshooter.

+1 for a good perspective on this. I agree but when they DO think mechanics second, they better now how to be able to use the tools. Just like in cars, an engineer who doesn't understand or can't use a mechanics tools is not a qualified engineer at all.
–
maple_shaft♦Mar 1 '12 at 14:45

13

This answer doesn't leave any room for instinctual debugging. Someone who's worked with enough systems, types of code, or languages, will often be able to "smell" their way towards whatever is funky going on. Sometimes you don't need to know the ins and outs of the system to be able to find its flaws.
–
JordanMar 1 '12 at 18:28

7

I have to disagree 100% with your black and white if the first thing they do comment. I would say that if a person thinks that firing up the debugger is not a good first option in some cases then that person is also not a good troubleshooter. If the problem is that communications stopped, the first thing I'm going to do is fire up the debugger and figure out which processes/threads/tasks are blocked or stopped working. Another reason for firing up the debugger is to try and see if the problem is repeatable. Once you know how to break the system finding the solution becomes much easier.
–
DunkMar 1 '12 at 20:37

3

@ElGringoGrande he was suggesting the opposite of that, from what I am reading. The point is that people become naturally better at debugging as they become more experienced. The tools or methodology isn't as important as how effective they are. That's why your answer is incomplete. There are many valid ways to debug, including pulling up a chair and running through the code, asking questions, et al. I've effectively debugged large PHP programs with print. I don't like doing it, but it really isn't as much about the tool as it is knowledge of how systems generally work.
–
JordanMar 1 '12 at 22:29

I would argue that the best measure of a good software developer in a particular language or framework is the ability to critically analyze complex problems and to have good debugging skills in the language or framework. They must be able to demonstrate low level debugging as well as proficiency in high level debugging with common debugging tools.

This means creating a scenario for them that demonstrates high aptitude of debugging tools in their chosen IDE. You should look for things like:

Running sandboxed application or server in debug mode or building application with symbols for debugging

Making available and demonstrating remote debugging ports or debugging of non-sandboxed application that was built with symbols (if applicable to language)

Use of expressions or variable watches for monitoring variable values or references

Use of ad-hoc variable value or reference or pointer manipulation in real time

Demonstrate ability to step into, over and out of application flow

Critical evaluation of call stack

Debugging multi-threaded applications and comprehending this.

Other debugging strategies without tools should be demonstrated as well, such as analyzing logs and source code, as well as ability to perform some low level debugging without the use of an IDE as well.

I'm of the opinion that debugging multi-threaded applications is an entirely different real than debugging single-threaded apps. Different, and much, much harder.
–
Bruce EdigerMar 1 '12 at 14:45

20

@JarrodRoberson Brian Kernighan and Rob Pike wrote in The Practice of Programming that they still prefer print statements to debuggers because it's less transitory. Many people prefer a good logging system that gives them a detailed view of the code path without halting the program mid-run. It's also easier to read a log file then debug a core dump. So your litmus test might reject some good programmers because not everybody agrees debuggers are as useful as alternative debugging approaches (logs, unit tests, assertions)
–
MetricSystemMar 1 '12 at 15:33

12

Debug print statements are fine and can be preferable to a debugger especially where concurrency is involved. Your problem with them might really be with a slow compiler.
–
Ricky ClarksonMar 1 '12 at 15:40

If you want to see if a programmer can debug, give them code to fix. It's the same approach if you want to see if they can write code. Give them a problem and have them write code.

Now, I'm confused about this programmer who has no problems writing code but fails misserably when asked to debug. Does this person regurgitate code examples or just sticks to areas he has experience like reading and writing to a database? Unless they get the code right the first time, they can't fix it?

Maybe the person just doesn't like debugging and makes no effort? I'm no good at this so stop asking me to do it - learned helplessness.

Working on an existing code base requires looking through code, documention and possibly making some notes and desgins of your own.

I know we think of debugging as fixing production code that has failed, but I need to debug code while I'm writing it. Either this person isn't a very good programmer or they just prefer to write new code. Don't we all.

We all debug our programs. In the beginning it's easy because the program is small and it's easy to have it all in your head. But as it grows and gets more complex debugging becomes harder. And now - some people can still handle that and find a bug in reasonable amount of time and some people are just lost. They do not know where to focus and how to narrow down to find it...
–
Michal B.Mar 1 '12 at 15:01

1

@MichalB.: We all debug our programs, but some persons will exhibit a principled approach while other just randomly tweak things and see what happen.
–
Matthieu M.Mar 1 '12 at 17:39

I've often given candidates hypothetical situations... for example, a production system has stopped responding. What do you do? They might answer "check the logs" and I say "the logs show nothing abnormal, except that there is nothing written in them since the problem started happening". And so it continues, until I'm satisfied that I've assessed the candidates ability to problem solve.

i look at puzzle and i am like ewwwwwwww. you really dont have any thing better than "gotcha" stuff ? and how does "seniority" come into the picture ? are older ppl supposed to solve more difficult puzzles? are all good programmers (or debuggers) a fan of sudoku ? you might end up annoying some really good programmers and bad mouthing you all over town. gotcha questions are an insult <period> please come up with something better men.
–
WildlingMar 1 '12 at 18:23

I will share a experience together with a recruits perspective about test of a candidate skills in debugging. I onced got on a interview that had three stages. The second stage was a "practical case". I didn't know more at that moment. While there I was informed there are a system that stopped working and they don't know. Some bugs is lying behind.

It was arranged as a remote desktop to a old testing environment. Probably to a unplugged or isolated environment. The project was a few webforms with some ASP.NET controls and related Code-file code. The codefile referred to a kind of business layer for which i just have a dll, no source code and method descriptions. The Webforms did the CRUD functions you can expect. Also a small search function. The business layer, in turn, talked to Views and SP in a sql server.

They broked some parts at different levels. I was given a paper with symptoms. "Can't search" "The 'region' field disappeared after last update" and such. Such as you can receive from your users.

I don't remember all details but at least a table field got renamed, which lead to a broken SP, which was used by the search function. That means no error in VS and no BL source code to trace field names. A SELECT parameter against Sqlcommand was misspelled and caused a webform to malfunction. Also a field was omitted which was the missing field in GridView (Autogeneratecolumns). A ASP.NET Button was referred to something who must meant to be a duplicated, enhanced, method and "forgot" to point button to new method.

Also such minor thing using title in a html tag that don't allow it. Also opposite ALT tag was omitted in a control that required it. There was also some errors with uncorrect closed html tags but which did not malfunction. Not sure if all those was a pure playhouse-project-error or perhaps same project for different recruitments. I never asked. The level of difficulty should of course match the need of the recruit.

Such test should probably be screened (not followed) to see, after interview, how debugging was done. For myself at that stage, I found the test a little ridiculous, but that's would also be the big point. If it was or wasn't, should worth a lot have the candidate in right place.

*I think that test was proved the candidates/my skills to *
* Analyze a foreign system
* Use a minimal of information to find errors and bug
* Under time stress and without someone help you, code assumed corrections
* Different levels of knowledge;
** sql db and stored procedures,
** use of dll in project,
** asp.net technique,
** layered architecture
** problem-oriented aspect

But also the more obvious things like handle the developer environment, find and understand Db Server Management tool. Surely there are candidates that looks really nice on paper but, in practice, could stuck on such tasks.

I pick an actual problem I encountered that's relevant to the position and I present it to the candidate much as it was presented to me. Of course I offer them some general background, and a small amount of relevant documentation like a code snippet or a schematic diagram.

I tell them their job is to solve the problem and I offer to answer any technical questions they have and tell them the outcome of any experiments they want to perform. If they say, "I'd put a scope probe here," then I will sketch them a trace of what they might find. If they want to insert a printf in a loop I will tell them that it never comes out (!) or first prints "7" and then repeatedly "5". If they get so far off in the weeds that I can't give meaningful answers I will admit that we're on the wrong track and back up to somewhere else. If they become stuck I'll ask leading questions or give hints until we can move on.

What I want to see is orderly thought processes, determination to get to the solution, well considered questions and experiments, and ideally a successful identification of the problem. Sometimes I choose problems that are too complex for someone to fully debug in a one hour interview and at the end I give them the real answer. At that point I am looking for a reaction that shows that they were engaged with the problem and experience that "aha" moment and gratification at getting to the cause. The best candidates will spontaneously ask followup questions at that point, trying to link their mental map of the problem with what was really going on.

All this tells you is that the person can analyze code and binaries to find a potential null pointer reference. It doesn't actually demonstrate skilled use of common debugging tools.
–
maple_shaft♦Mar 1 '12 at 13:54

If you have your candidates do a preliminary code tests, ask them to modify the code during the interview to either solve a bug or add a new feature or better yet both. If you make the code test specifications fairly vague, that would make it easier to create test cases with "bugs" in then.

Finding "the bug" in a little code snippet is a very artificial situation. I suppose it might be helpful in the same way that puzzles and brain teasers are.

A more comprehensive approach would ask behavioral questions about how the candidate has performed debugging in the past citing specific incidents and then following-up with questions.

Someone who is good at trouble shooting will be able to talk about more than just the debug facilities in the IDE. What about... the bug-reporting tools, end-user interaction, reproducing the bug, logfile analysis, verification?

There is MUCH MORE to debugging than tracing through a block of code and any evaluation of someone's skill in debugging should refect that.

I tend to ask people to describe to me the most difficult bug they ever had to track down and fix and what they did to find it and fix it. I also know that if the most difficult bug was something that you would expect only a beginner to find difficult, then likely they are not good troubleshooters (unless this is an interview for entry level). If it is something genuinely difficult and they describe their thought process as they tried to track it down, then I can get a feel for what their skill level is. What has alaways amazed me is the sheer number of people who get a "deer in the headlights" look and can't think of a single example of something they did that was complicated. Well sorry someone who leaves the hard problems for someone else to fix is not someone I'm interested in for anything except a straight-out-of-school, very junior level job.