Programming Practices: Part 1 – Watching from a distance

I figured I’d start the series with a more abstract post about what watching me code looks like. It was interesting to observe myself doing it because I didn’t really realize the degree to which I do some things. If I were to summarize my overall approach to coding with a simple description, I’d call it “Annealing” – from the Encarta dictionary: “metallurgy craft transitive and intransitive verb to subject an alloy, metal, or glass to a process of heating and slow cooling to make it tougher and less brittle.” Or in more of a software sense, “Simulated Annealing” – from Wikipedia: “a technique for searching for a solution in a space otherwise too large for “ordinary” search methods to yield results”.

What does that mean?

Let me try talking about it as a set of principles and techniques:

1) Working code every few hours – I find I can’t stand to have code that is in pieces all over the floor. I don’t ever like to be far from code that works. My coding is defined by a sequence of “stable” points that are no more than a few hours apart. A “stable point” is a state where all of the code compiles, it runs and it does something useful. It may not be anywhere near doing what the ultimate application is intended to do but it does some fraction of it. A quantum of work transitions from one stable point to the next.

I find that as I approach a problem, I think about how I am going to build the code in bite sized chunks. It’s kind of funny to watch but if I get a few hours into something and I’m not very close to a “stable point”, I actually get anxious. I start getting nervous and looking for ways to stub out or cauterize parts of the application so I can get back to something I can build and hit F5 on. I think part of this is related to why I got into programming in the first place – I’m a huge fan of the “instant gratification” that you get programming. I can think of no other profession where you can start with nothing and in a few short hours have built something really useful. But more practically, it’s an important way to make sure you don’t get lost in the weeds of a solution.

2) Prove the concept – When I’m getting ready to leave a stable point, I find the first thing I want to do is map my path to the next stable point. I choose some specific, containable goal – add a dialog, a menu item, move an operation to a background thread, add a control to a form, something. Before I enter the chaos between stable points I want to prove to myself that I know how to get to the next one. I often start, particularly if I’m working with API surface I’m unfamiliar with, by doing a rapid prototype of the core concepts that I need. I don’t care what the code looks like. I don’t modularize anything. I don’t worry about comments, variable names, code organization, error handling, performance, anything. I just want to touch each of the core APIs that I need to use and prove that I can get access to the information that I need and understand roughly what I need to do with it.

I usually get it all the way to running code and step through it in the debugger so that I can see all of the information flow and ensure that I understand all of the side effects, etc. To a first approximation, I then throw it all away. Sometimes, I’ll just comment out the routine(s) that I wrote and keep them around a while for reference and a source of code snippets. I step back and conceptualize an organization of the code that feels clean and I then start writing the code “for real”.

At some level this might feel like a waste of time but I believe it is a huge time saver for me. It usually doesn’t take me more than 15-20 minutes or so to explore the key concepts of the few hour leap from one stable point to the next and I often learn a ton in the process and make significant shifts in my approach as a result. It saves me from getting an hour or two into something and realizing that some assumption I made at the beginning was invalid and I need to redo everything I just did. I think it also helps ensure that I end up with better organized code when I’m done.

3) Refactor, Refactor, Refactor – I never really realized how much I just naturally do this. I’ll bet the net effect is that I write any given stable point to stable point quantum 2 or 3 times in the few hours I work on it. I start with a conceptual picture of the algorithm but I don’t try to plan out everything. I start writing the code organized in a way that feels the “cleanest” but as soon as I realize something’s not right, I refactor. I’m constantly splitting methods, reordering code, reorganizing data structures, building abstractions, etc. This is the main reason I started by talking about annealing – because that’s what the process feels like to me. I keep jiggling the code, gradually reducing the temperature until all of the crystals line up just right and I have a very clean, maintainable, reusable, fast, robust, … implementation.

I now realize that I used to do this even before “refactoring tools” were invented but I am finding some of the refactoring tools to be very useful in saving me some typing – though they have a long way to go to make me really happy. The VS ones, at least, don’t automatically do everything I want and don’t always have the flexibility for me to control it. Examples – Extract method often leaves me with a lot of clean up to do.

It doesn’t give me the ability to control parameter ordering (and as you’ll see from this series, I’m anal about everything). I am very particular about the ordering of parameters.

It doesn’t allow me to control the types of parameters to the method. Every method should have an abstraction or contract of what it operates on and what it does. I might want to type a parameter as a base class of the passed object. I might want to pass a one or more members of an object that is used rather than the whole object (because the type of the object is not part of the method’s contract).

It doesn’t allow me to extract to a different class.

It rarely puts the method where I want it in the class (more on this in future posts).

Lastly, I find that before I extract the method, I actually want to type the call to the extracted function. It’s part of making sure I understand the factoring of the code and what the contract is. I wish I could type the call site, then select the code I want to extract and say – extract it such that the call matches what I just typed. But instead, of course, it extracts it, adds another call site, which I immediately delete and then go fix up the parameter types and order, etc.

As I write the code, I comment lightly. Because I refactor so much, I don’t want to waste time writing and rewriting comments, method contract descriptions, etc. But once I’ve gotten the code just right, I polish it. I go back over all of it making sure I like the abstractions and the flow, I comment everything and I move all of the methods that I’ve written to their permanent resting place in the class – I’m very particular about the order of methods in a class (did I mention I’m anal about everything? :)).

4) Step through everything in the debugger – Before I start polishing a quantum of work, I walk through everything in the debugger. I conceive and write or manually execute test cases for every interesting scenario and I step through all of them in the debugger. It’s not sufficient that the code works. I’m a firm believer that you don’t actually understand your code if you don’t understand exactly how and why it works and the only way to do that is to step through it in the debugger and see what it is actually doing.

You’d be stunned to realize how often the results of code match what you expect but the execution path is nothing like what you expect. This is one of my greatest secrets to getting high quality, high performance code that I never have to come back and revisit again. When it’s done, it’s done done 🙂

5) Paper and pencil – The last thing that I’ll mention that’s perhaps a bit odd and I think you’d notice if you sat and watched me code is that I always have a pencil and paper in front of me. As I’m coding, I’m constantly drawing pictures of data structures, writing out sample data, hand walking through the algorithm on paper to see how things work. I suspect it’s just a quirk of mine but there’s no better way for me to think through the algorithms as I go. When I’m done with a quantum of work, I always have a sheet of paper full of scribbles. I generally extract some of the value and then just throw the paper away. Some of the scribbles become part of the documentation – pictures, algorithm descriptions, etc. Some become the source of test cases that I keep to serve as regression tests in the event that I ever need to come back to the code.

Summary

At a very high level, that’s what watching me code looks like. Some of what I’ve written here, I think of as quirks of my personality – heavy use of pencil and paper, for example. But most of it I consider important practices to produce high quality code efficiently. I can imagine a lot of people looking at this and saying, “wow, that sounds expensive” and if you are banging out a one off project with a short lifetime, it’s really not worth it. But if you are writing code that is part of a significant application, is going to live for years and potentially have other people needing to understand it, I believe these practices are invaluable.

I’m not sure I’ve captured everything. Perhaps in the discussions that follow and subsequent posts I’ll think of a few more “high level” things. We’ll see.

Tags

Join the conversation

Sounds very close to what I am doing right now! I am sitting at my desk getting ready to code with a note book open and a pencil ready.

Are you using TDD or some other form of test first development? Stepping from Quantum to Quantum is a helpful aspect of TDD as the unit tests naturally guide you into the next steps. Refactoring is also supposed to be a core part of the workflow and with tests in place drastic refactorings are much safer. The debugging aspect does not fit into TDD quite as well but I often find myself in the same situation, stepping through code to make sure I know exactly what path is being evaluated.

Thank you for your post.

8 years ago

Ron Harding

this sounds a lot like me as well. As electronic engineer, i cross trained in 2004 to MS .NET. at that time, i was primarily familiar with coding microcontrollers, no OOP, linear coding, much of it in assembly. Also, much of this coding i did was interrupt driven. It took me time to learn the whole OOP, but i eventually got it down by writing programs. My first job, I designed and built a semi-automatic test system from the ground up. this system was intended to test an ethernet to two way radio remote control/portal(linux box web server).

this application was targeted towards two way radio service technicians, to assist in troubleshooting units by having semi-automatic test of the system. so a c# control gui, interface hardware to UUT and some 3rd party s/w that my app. called made up the system.

approach:

1. since the system was intended to test a two way radio controller, i had to build a test fixture that ‘looks like’ a connected two way radio. I used Kenwood’s proprietary packet passing protocol using byte command sequences to key the radio, listen for responses and other functionality.

2. i used the on-board diagnostics to perform some of the testing. I used a virtual oscilloscope and signal generator to output a waveform from the connected virtual two way radio and then play it back. The system stored digital audio so I needed a quick way to verify that the audio was performing properly. There are specific performance criteria used to evaluate audio performance; in this case record audio, play it back and look at Signal to noise ratio and Total Harmonic Distortion.

3. as i mentioned i designed the code to be ‘context sensitive’ to an audience of two way radio service technicians. Also, this system was state machine, as many test systems lend themselves to that kind of construct.

so, in all, i have provided contract services of this type to 3 clients so far. it has been successful. However, i was laid off back last july 09.

i have been getting calls the past couple of weeks, and have done some job pre-screening coding.

i am prepariing demo for job interview. i am designing remote control for my Agilent arbitrary waveform generator. This equipment allows me to send out any type of waveform in any condition; in this case a signal with specific noise on top. the user will be a developer that inputs specific math equations that represent the signal. this signal gets converted to individual samples and sent to the generator to be output. you would use such a setup where you need to perform standards testing by injecting noise into the signal stream to exercise communication protocols.

so that is my bag right now.

have a good day.

8 years ago

Carl D

Very similar to the way I code too, except instead of pencil and paper, I usually end up with a ton of Post-It notes. I usually develop with at least one instance of Word, Excel and Visio open in addition to VSm, so most of my notes end up in Word documents that serve as raw materials for the documentation. I generally keep all those word, excel and visio documents in source control right alongside the code.

8 years ago

Idji

yes, I identify with all of this as well – Constantly Refactoring and simplifying and splitting methods.

Obviously this blog has attracted like-minded people.

I do step lots through the debugger just to be sure it is doing what i think, but what is new is stepping deliberately through every use-case through the debugger.

However order of methods and parameter order are not so important to me. I use "go to reference" or the drop-down lists to find the method again.

8 years ago

Craig Lebowitz

My practices (including the beginning anxiety) are very similar. I never thought about the instant gratification part, but I definitely get the ‘rush’ from coding that many programmers feel. Building things up from the simplest possible working example is a great motivator and helps you recognize and mull over the abstractions that are going to be used later. Also, it always helps to be able to have something to show the approximates the desired result (even if done ‘wrong’) to show a collaborator. Nice post!

8 years ago

Craig Lebowitz

My practices (including the beginning anxiety) are very similar. I never thought about the instant gratification part, but I definitely get the ‘rush’ from coding that many programmers feel. Building things up from the simplest possible working example is a great motivator and helps you recognize and mull over the abstractions that are going to be used later. Also, it always helps to be able to have something to show the approximates the desired result (even if done ‘wrong’) to show a collaborator. Nice post!

Wow, I guess I could have written this post, it describes almost step by step the approach I have. 🙂

For the last one I always push myself first to do it mentally and if I fail then I take the paper and pencil.

As I refactor a lot, the comments come at the end, when I feel the code stable.

Refactoring is triggered when you feel/realize the code you just wrote doesn’t fullfill the criterias anymore (or when you realize they changed!:)).

As for the F5 all the time thing, I guess 5 years working without a debugger in protected mode assembler made me love having a great debugger on my side! And the static representation of your code rarely matches the flow you can observe while debugging.

I would say now I’m glad there’s the Layer Diagram and Arhictecture validation in VS10, it saves me a big deal of work (especially when working in a team) to keep things clean at a high level.

As for the position of methods/properties in the class I love resharper and its File Structure view that let me reorganize everything smoothly and put regions everywhere the way I want!

8 years ago

Kevin

Bah. My strategy is to quickly peruse the API and start coding the grand architecture, hoping to get it right the first time through.

I just started experimenting with it, but so far the "generate from usage" feature seems more promising to me than "extract method" – even as a refactoring tool.

I have code I want to move out, so I write the new line to call out to the (not-yet-existing) method, click Refactor->Generate…, then move the code into the stubbed out function. Seems to require less cleanup (though that assumes variable names/types are mostly staying the same).

8 years ago

jalf

Paper and pencil are probably among the tools I’d feel most crippled without. Partly for just sketching out data structures, class diagrams or even class names, partly for writing down notes and keywords to remember and revisit later, and partly just because fiddling around with the pencil is a nice way to keep my hands occupied while I’m thinking. Don’t underestimate the last one! 🙂

8 years ago

Walter Sassano

Like most of the previous posts, this feels like I could have written it. It describes my process almost exactly. Great article and nice to know I have company.

8 years ago

whatknott

You mention you’re very particular about the order of your methods. Could you elaborate on how you order them?

As smaclell mentioned, TDD (or at least unit testing) could be beneficial. I used to step through code in the debugger to manually test it. The problem was that it took a lot of time, and I had to redo it if I changed the code. This made me resistant to change. Unit tests solved this problem for me.

I often (but not always) do that too. I find diffing the changes before I check in to not only be a good way to tell if I left any rubbish but it’s also a good opportunity to give it one last high level overview to see if I forgot to do anything.