How Studying SICP Made Me a Better Programmer

While working through the problems of SICP section 2-2 this week, I had a flash of insight. Things that had frustrated and stymied me for years were suddenly becoming trivial. One data structures problem that had particularly bugged me had become pretty easy, too– though I had to apply everything I’d learned in SICP about recursion, first class functions, and the closure property to do it. Here’s a quick tour of my proof of concept from the REPL prompt:

We’re starting out by defining a hash-table to store all of our work in. I guess I could have stored everything in a global variable so that I wouldn’t have to keep referencing the store all of the time, but I figured it was worth it not to have all of my functions hard coded to a particular variable. With the above code we assigned division-a and division-b to a company we like to call Mega-Corp. We created some factories, distribution centers, and a main office while we were at it.

Here we take a look at what we’re actually storing in our hash table. Each item in the hash is a list in the format of (WidgetName Parent-Widget Children-Widgets-List Properties-Hash-Table). You can see above that Mega-Corp has children, but no parent. Division-a has both a parent and children. And Factory-1 has a parent but no children.

Here we extend our data structure even further by adding some office staff groups. We can extend the structure as much we want– and in principle, there’s nothing preventing us from writing procedures to copy and combine such data structures however we choose. That wide-open combinatorial possibility is what the closure property is all about.

Here we’re using a recursive procedure to apply the plus function to all of the property values from a certain point in the hierarchy down. The zero in the s-expressions there is an identity value– just like the ones we coded in SICP section 1-3. I had to implement that just in case there were all NIL’s in my hash table for a certain property. With a careful use of property names, you can get some benefits of stratified design (covered in SICP section 2-2) in this kind of data structure. We can code in all kinds of information about the separate divisions at one level… and way down the hierarchy we can have all kinds of other information stored about the employees. While working at either end, we can effectively ignore all of the other levels. At the same time we can “roll up” related values however we like at any level we care to focus on.

(There’s probably a few other things I could have done to make this better, faster, or slicker. If you’re working through SICP, too, please check out this complete code file and let me know if you have any tips or suggestions. I used Common Lisp instead of Scheme, though. I really needed to stop a while and apply what I was learning in SICP to problems that I’d developed a personal stake in. Most people tend to abandon the book by section 2-2, so I better get back to work if I want to finish it!)

Back in the day, I would have avoided a recursive data structure like this if I was working on a similar task. I would have (horrors!) hard-coded the hierarchy at maybe 3 or 4 levels deep, coded all my tables as separate entities, and more than likely built in some work-arounds for the places where such choices were inadequate. (Thinking that OOP was the “right way,” I would have coded a class file for companies, divisions, offices, and employees– and then been stuck storing and retrieving data in as many separate RDBMS tables and figuring out how to map them back and forth somehow. Writing nested hash tables to a file, on the other hand, is trivial in Common Lisp.) The end result would have been a maintenance nightmare…. It’s a good thing I was too lazy to finish any of those pet projects back then! Funny, I could see clearly the problems of coding 50 slightly different forms when one generic form could be rigged up to do the same thing, but I couldn’t break through my mental limitations to think three-dimensionally about data structures. Thanks to many painful hours spent working through SICP, I can now synthesize recursion, first class functions, and the closure property to make more elegant solutions to things that used to be next-to-impossible for me.

8 Responses to “How Studying SICP Made Me a Better Programmer”

FYI, such data structures are the bread-and-butter of Perl, that even has special “notation” for naming them. For example: HoHoA is Hash of Hash of Array, or a Hash table where each value is a Hash table, in which each value is an array.

It doesn’t seem to me that the structure you describe here is itself problematic with OOP. It could be handled adequately with a something like the XML DOM. The difficulty arises when you want to have different semantics for different entities in the hierarchy. What can Lisp do for us here? You hint at “wide-open combinatorial possibility” but it would be nice to see some examples.

I can see that generic functions might create some possibilities – especially multiple dispatch when it comes to parent-child relationships; but I’m not sure that takes us much beyond what a dynamically-typed OO language provides (I only have a superficial familiarity with Lisp).

How does Lisp support the application of operations to this data structure. Are you planning a follow-up post?

You’re probably right about OOP not being at fault. Not being fluent in recursion would of course limit my ability to do things like this in any language. On the other hand, the desire to make a fancy object model using lots of inheritance works against the kind of generic approach that seems “obvious” after a couple of chapters of SICP. Nevertheless, that doesn’t prevent me from taking what I learn in Lisp and using it become a more savvy OOP guy.

There’s several other things I’d like to do with this structure beyond just storing property values. As I learn more from SICP, I will be sure to try to apply it and detail my progress in follow up posts on this theme.

[…] is considered one of the great computer science books. Some people claim it will make you a better programmer. It was the entry-level computer science subject at MIT and it’s still used in universities […]