Writing Code On Whiteboards Is Hard

Work has been crazy lately and I haven't had much time to work on SimpleScript. It's a lot of work starting from scratch! Right now I'm writing a templatized hash table for the binders and other lookup tables. I haven't written templatized C++ for a loooong time and its slow going. I hope to have the named item logic done over the weekend, so that I can explain how the module system works next week.

Until then, here's an article I wrote a while back on interviewing that I never got around to putting up on the blog before now.

Writing Code On Whiteboards Is Hard

As I've mentioned before, I occasionally interview candidates for development positions on my team and occasionally other Visual Studio teams. Plenty of people have written plenty of web pages on interviewing at Microsoft, so I won't rehash the whole story here. What I wanted to mention today was some words of advice for candidates for development positions. This is by no means complete -- I want to concentrate on one important aspect of the interview.

Dev candidates: if you've done any reading at all, you know that most of your interviews will involve writing some code on a whiteboard. A word of advice: writing code on whiteboards is HARD. Practice!

It's hard in part because you don't have intellisense or syntax colouring or any of the other tools at your disposal that you normally do when writing code. I know that, and I'll take that into account. I don't care if you write memset(&b, cb, 0x00) when you mean memset(&b, 0x00, cb) -- I don't remember that stuff either. I don't care if you forget semis or make other syntactical gaffes. I'm not looking for people who can spit out syntactically perfect code at the drop of a hat; that's not the point of the coding exercise at all. I'm trying to find out how you solve problems. Do you:

rush right in and start coding without thinking the problem through first?

find areas where the problem is ambiguous and clarify them, or do you make a bunch of assumptions?

break the problem down into pieces?

do the easy pieces, paint yourself into a corner, and attempt to handwave your way out of the situation, or work on the hard stuff first?

have any confidence that your solution is correct?

do anything to prove to yourself and me that it is correct?

It would be wise to make it easy for me to see that you're a well-organized person with good problem solving skills. Leave lots of space -- you might need to stick in some extra lines. Write slowly and clearly. Explain what you're doing as you do it. If the code is clear, well-organized, legible, and at least vaguely syntactically correct, it will be a lot easier on both of us to tell whether the code is well designed and bug free. Walk through some simple test cases -- don't just dash down some code and say “yep, that's correct!“

The vast majority of the coding problems people pose do not require any "aha!" insights or rocket-science algorithms. No one is going to ask you to implement a 4-5 Runge-Kutta differential equation solver. We're going to ask you to implement a hash table or replace every instance of "foo" in a string with "bar". Eric Carter has a copy of Schaum's Programming With C++in his office because that thing is a gold mine for interview questions. Got a technical interview coming up? Pick up a copy of any introductory programming text, pick a few problems at random, and solve them on a whiteboard. Heck, I'll flip through it at random right now. Here are some sample problems taken from various places in the book:

implement a method which determines how many characters the string pointer must be incremented to point to the trailing null.

implement function that takes an array of integers and returns the maximum, minimum and average.

implement a template for generating stack classes

implement a program that shuffles an array into a random order

Any of those would be a highly typical coding question. And before you say "that's easy!" about any of them, think about what's missing from those problem statements before you write the code.

That string pointer: What encoding is it pointing to? 7-bit ASCII chars, UTF-8, UTF-16, some ANSI encoding? Are you going to ask the interviewer, or are you going to assume that you're writing C code for some operating system that existed in the Before Time when there were no strings that contained Chinese characters? Hint: Microsoft writes very little C code targetting PDP-11 machines. Ditto for programs that never have to deal with non-Roman character sets.

That less-than operator: Do you have to handle nonreduced fractions like 2/4 ? Illegal fractions like 2/0? Can you assume anything about the size of the integers?

That array you're shuffling: do we care if a hacker can predict the sequence? Do we need crypto-strength randomness, or reproducible pseudo-randomness? Is this for an online poker game for real money or a test case for a sort algorithm?

And so on. None of these problems is well-defined enough that I'd feel comfortable writing production quality code without at least some clarification.

Candidates often make the mistake of concentrating on the code, like the code exists in a vacuum for its own sake. That's a very unrealistic assumption! Code not only has to be well engineered and correct, it has to be maintainable, testable, and solve a real customer problem. In your interview, when you're done writing the code, think about:

how would a tester attack it? Is the design even testable?

does it handle stuff that hostile/buggy callers are going to throw at it? null pointers, large denominators, huge arrays?

does it work well with other technologies? that is, does it use the conventions of COM or ATL, or does it work against them?

how would you know whether this was the code the customer wanted or not?

Now, this is not to say that someone who automatically thinks char * when they hear “string“ is an automatic no-hire. But I am a lot more inclined to believe that “experienced in COM programming“ on your resume if you acknowledge the existence of BSTRs! Also, I recognize that fresh-out-of-school candidates often have very little experience with these sorts of “real world“ considerations; I cut them some slack in those areas and concentrate on their raw intellectual horsepower, coding talent and long-term potential.

Finally, let me reiterate that technical interviews are hard, and even bright people screw up on them sometimes. Two teams no-hired me when I interviewed for full-time positions here! But that's another story.

I got a no-hire on shell...but I got an offer from the client group and I really wanted to work on the window manager bits anyway. I didn't practice the whiteboarding at all(my whiteboard here is usually just doodles or diagrams), and I had a rough time at it..Chris Anderson was really forgiving of the fact that I started on the right side and then continued the same function on the left side of the board. Writing code on a white-board is definately something worth practice.

I don't work for MS nor have I interviewed there, but I have interviewed and been interviewed at other places. Here are my 2 cents:

1. Make sure you understand exactly what you are asked to do - ask questions if you need to.
2. Don't be a showoff, do what you are asked to do
3. Ask if you can choose the language. Ask if you can use pseudo-code.
4. Write clearly (don't write cursive)
5. It's OK to stop and think

An aside: any particular reason you can't use hash_map for your templatized hash table?

I've been going back and forth between using hash_map and rolling my own. The former is easier, the latter is perhaps more interesting from a pedagogic point of view.

In looking at the original script engine code, there are lots of places where we needed something goofy -- for instance, JScript name tables are basically hash tables but they use a move-to-front-on-access algorithm to improve performance.

Given that I'm going for clarity and not perf, I'm starting to lean back towards the STL implementation...

I remember an interview at ObjectSpace (now Recursion Software) years ago and being asked to write code on a whiteboard. It's the one and only time in over twenty years that I've ever been asked to "code-on-demand" and I point-blank refused. It just isn't a realistic test of people's abilities, IMO. I was quite happy to whiteboard a UML model (of the game of Monopoly, as I recall) but coding isn't a "spectator sport" in my book.

The same interview ended with them sitting me down in a room on my own with a spec and a Windows PC and Visual C++ and being told to "code what the spec says". That was just about the final straw for me. I wrestled with VC++ for five minutes (I had never used it and rarely used Windows at all - and that was not the environment I would have been using in the job they wanted me for anyway as it wasn't a programming role) and decided it was time to end the interview. I walked out and caught my plane home (to England).

They'd flown me out for the interview and after I "stormed out in disgust" they called me up and apologized and asked if they could persuade me to take the job anyway(!). I'm very glad that no company since has repeated that particular interview process...

Dan Shappir

15 Apr 2004 2:53 PM

Don't knock STL performance. With a modern optimizing compiler like VC++ you’d be hard pressed to beat it. You may be able to hand-craft a container that is particularly suited for the task at hand, but even then the difference would probably be marginal.

Given your JScript example, today I'd probably use hash_map (or the less efficient but more portable map) and also maintain a direct pointer to the last item accessed. I've actually used such a structure in a recent project:

We have an application that for historical reasons stores its data in a text file using the ini format. The ini file was getting to be very big and the load time was becoming excessive. I rewrote the loading + parsing code (reads the data into a hash of hashes, and can also write the data back) using STL. The load and parse time for a 12MB ini file (more than 10000 sections) was 1.2 second (2.4GHz Pentium 4 with 500MB RAM using Windows XP).

Dan Shappir

15 Apr 2004 2:59 PM

> coding isn't a "spectator sport" in my book

Actually I agree. I've given people tests where they have sat alone in a room and wrote some code on paper. I've never had an interviewee code on a whiteboard in front of me. I have had them use a whiteboard to explain the structure of systems they have designed in the past.

Re: STL maps -- the other remaining problem that I'm going to have to look at is the threading model. The script engines use a pretty goofy threading model, but I think for most cases we can probably ensure a single-writer-multiple-reader model.

Also, you might be surprised at how overstated some people's resumes are. I once had a candidate who described themselves as "nine out of ten" C++ programmer who could not write the line of code that determined the absolute difference between two floats. I had a candidate with a solid seven years of programming experience who had only a vague idea of what "recursion" was.

Freaky -- Chris Sells says virtually the exact same thing about whiteboard coding in his Channel 9 tour of MSDN: "The board is a terrible place to write code ... there's no syntax completion, there's no coloring, there's no IntelliSense ..."

It's probably not necessary to roll-yer-own map classes for the example engine here. Code for structure walking and garbage collection is much simpler when you don't have to worry about maintaining complex invariants in the data structures. But it would be nice to have some discussion on the data structure design tradeoffs for prototype/slot happy languages like JScript. A lot of the in-depth looks at language implementation hit static scoped, static typed, or compiled languages which is apples and oranges to a project like this.

Dan, not to knock STL here, but there's engine code that gets called millions of times per second and it's important to understand what the compiler is spitting out. I get tired head staring at a pile of maps and iters and templates and trying to figure out if it's going to spill my registers. I guess I just don't eat C++ for breakfast.

I've been contract programming for many years now and have never been asked to code on a whiteboard during an interview. I agree with the "spectator sport" comment. If asked, I suspect I'd do like Sean and walk out. With a few well chosen questions, you can usually determine a programmers aptitude without resorting to a whiteboard code review.

> With a few well chosen questions, you can usually determine a programmers aptitude without resorting to a whiteboard code review.

**************

You're hiring a juggler for your kid's birthday party. I show up for an interview.

Me: "Hi, I'm a juggler."

You: "Can you juggle?"

Me: "Yep."

You: "Three objects? Four?"

Me: "Yep."

You: "Five?"

Me: "I can do a five beanbag multiplexing trick, but not five ball cascade."

You: "Clubs? Torches?"

Me: "Yep."

You: "What is 'Mills Mess'?"

Me: "A three ball pattern in which one ball does the forward cascade pattern, one ball does reverse cascade, and one ball does underarm chops, while at the same time crossing and uncrossing the arms. The arms begin crossed. The corresponding pattern without the arm crossings is relatively easy."

You: "Who is widely considered to be the greatest juggler in history?"

Me: "Most people would say Enrico Rastelli, 1896-1931 -- who could do eight plates or ten balls or fourteen rings -- but I'm personally more fond of WC Fields. Less technical brilliance, more comedic brilliance."

You: "Super, sounds good. One more thing -- here are three rubber balls. Could you do a three ball full shower pattern for me?"

Me: "In all my years I've never been asked to actually juggle at an interview! And such a simple trick! How insulting! I'm out of here!"

***********

OK, juggling IS a spectator sport, but I hope my point is taken. I don't know any "well chosen questions" that tell me whether someone can juggle or not.

Similarly, I don't know what these magical "well chosen questions" are that tell me whether a person can actually write code; I'd love to hear them.

It's interesting how my perspective on whiteboard coding, and programming tests changed when I switched from interviewee to interviewer. I'm sure that the programming test we currently use eliminates some people who are great programmers, but don't test well. On the other hand, it eliminates far more people who talk a good game, but can't code their way out of a paper bag, and we're willing to give up the rare occurance of the former to save a lot of wasted time dealing with the latter.

On the other hand, Eric, given the state of IDEs these days, do you really care if someone can write code, or are you more interested in their design and problem solving skills? I don't think anyone would particularly object to designing something on a whiteboard, or to explaining a previous design they worked on, and the rationales behind it (as far as their NDAs would allow, of course).

Yes, I do care, very much whether they can write code, but I am also very interested in problem solving skills.

When I interview experienced "industry" candidates I usually ask a very design-heavy question. The question tests ability to deal with ambiguous situations, make tradeoffs, understand customer needs, deal with existing badly designed infrastructure by finding bugs and working around design flaws. Often we don't write a single line of code, sometimes we do, but it is pretty high-level.

I give fresh-out-of-college "campus" candidates a simple mathematical problem that can be solved in two or three lines of C. The question tests attention to detail, ability to reason about the purpose of code rather than just what it does, ability to come up with test cases, etc.

However, many of my colleagues do not break it down like that, and ask everyone to do a coding problem.