Software Without Magic, Zittertagen & Marginalia

Archive for April, 2015

For the next couple of months this blog is going to take a short detour into CS education. I started my career as a history teacher, and James Patton asked me to use those skills to help him review CS. I’ve agreed to help him out. Each Monday I will be posting a new lesson. Follow along over on his blog as he does the work, and on Fridays I will post my solution. The Friday post will also include my design notes, and any extra content left out of Monday’s Lesson.
My basic approach is to make use of things I learned in both education and development. CI provides some interesting ideas for automated grading. Sal Khan has already shown the benefit of computer generated practice problems. And Agile has suggested fantastic ways to organize this effort.

My starting point for unit/module creation this is the observation. “In both BBD(Backwards By Design) and TDD(Test Driven Development) we start with a Red Test.” I have copied that sentence from one working notebook to another at least a dozen times. It highlights something I all to often forget, One cannot achieve a goal one has not defined. The Red Test defines what evidence will show that we have achieved a goal, and allows us to begin.

Once we have a Red Test we develop tools to help our student achieve that goal. When our student is software we just write code that matches up with the goal. When our student is a human this phase is much more complex. We write and give lectures. Create worksheets, assign readings, lead activities to give the students a chance to practice.

Once we finish all this, we measure our progress. That is run our tests again. If we have done our job then the Red Test is now a Green Test, and we can work on a new goal. If we haven’t, then we know that we need to refine and perfect our techniques. In either case we have a good idea of what our student knows and is capable of doing.

Defining the goal is the hardest part of the cycle. What prior knowledge can we assume? What larger goal lead the student to us, and what short run goal will bring them closer?

I know that on Friday I promised you all that I would be sharing my own software short stack. But as I was trying to build and document my stack I found myself fighting against it. My initial efforts to define a short stack borrowed from suckless & considered harmful.

Both of these lists advocate small composable tools. Something I have been well convinced is a virtue in a toolkit. Most of them are written in C or similar low-level language, and are explicitly trying to be both simple and fast. When I tried these tools however I found that they got in my way.

The suckless tools I examined were close to the metal, written in C, and blazing fast. Both ST and DWM brag about how few lines of code they have. As a direct result of this both of them demand that you already know C to do any customization at all. ST lead me on a wild-goose chase tracking down behavior of my backspace key. DWM demanded I interact with the keyboard at a low level if I wanted to customize it.

The ‘considered harmful’ suggestions were often more confusing. Quite often, they seem to be pushing Plan 9’s way of doing things or proposing we adopt new ways of doing things even though a well supported de-facto standard is widely used. One particularly egregious example suggested that we replace `head` with `sed 11q`. That does work, it’s clever and demonstrates that the inventor understands an existing tool well. It also demands that a new user learn a bit of sed or accept a magic incantation, to something quite basic like peek at the top of a file.

Both lists inspired my explorations of Short Stacks, and in the past few weeks I’ve read through both sites quite often. They also both demand that users be quite advanced to be very productive. After a few weeks trying to adapt to them I realized that I was wasting time. Instead of picking a set of tools to use and solving problems, I was shaving a yak. To make things worse, this yak did not need shaving!

I already have a mostly functional toolkit that does all of those things. My terminal already gives me the *nix, vim, git, python, short-stack I cited Friday. I could have very likely already solved a couple of interesting side-problems in Python with the time I’ve spent yak-shaving and picking tools.

It is easy to get lost finding just trying to use your tools, or how to do something you know is simple. Both of these delays are a kind of mental stack overflow. Instead of solving the problem you are recursing to solve the blocking meta-problem.

I had reached for these tools after getting fed up wit .net and it’s associated tools at work. Visual studio is a great IDE, but it is anything but a short stack. Particularly after you install the all but essential extra tools: Resharper, Web Essentials, Fiddler, Postman, Powershell, Powergui, and so on. Even on my beefy workstation I spend a lot of time waiting for things to load, compile, or update.

In trying to explore alternatives to the high-level but slow tools I use at work, I ran into a conflict. There are at least two ways to measure the depth of a stack. The first is ‘distance to metal’. How close is your work to the actual platform on which your solution will run? The second is ‘conceptual RAM’. How many things do you have to keep track of to complete your goal?

These ideas of stack depth have different implications when building your personal workspace. Noticing these and working out these implications is what led me delay defining my own base stack. Whatever I stack I define and put the effort into automating needs to meet several needs. It must be fast. Even on my wimpy Chromebook 2. It cannot steal mental ram from the task at hand.Nor may it lead me on a quest for the finest Yak’s Wool.

For the moment, my stack will remain manual and tied to the web. I will keep using my crouton as a *nix shell, using my adapted dotfiles, on my Chromebook to build my side project. Do my writing in Google Docs, and publish on WordPress. I’ll also be trying to solve my idea of stack ranking and ELO to help estimate task & issue priorites in Python over the next week.

I will also continue to work on defining a “basic short-stack” that it is worth automating by next Monday. Until then what tools have you used that demand massive amounts of mental ram, or despite being nice are just too slow to use day to day?

About 5 years ago I first wrote some software with the idea that this might be a career for me. I had been a Linux user for years when I found myself writing software to support myself. For the first time, I considered development as a career. And I started to drown in the sea of known unknowns. Whatever I needed to do feel enormous. It felt as though things that are basic, like serving an HTML page were an elephant I had 30 min to eat. Heaven aid me if I needed to use c, or other hard(read: compiled) languages. I had only the vaguest idea of how to use a compiler and as far as I knew `make` was a part of `dpkg’. An acquaintance in the local open source community saved me. Clint Savage showed me tools and technology that were easy to understand. He was also patient, explaining to me when I had found something I should ignore for now. Without his mentoring I never would have learned to manage the complexity in modern development.

A few years later I found Levinux. This tiny Linux image is Mike Levin’s software education project. He coined the term Short Stack to describe the environment Levinux provides. A kind of bare bones environment that I imagine the elder neckbeards toiling in when they invented the internet.

In Mike’s explanation, of the Short Stack, its defining advantage is how easy it is to learn. When you are working on a well-defined stack(Unix, vim, git & Python in levinix) there is less to learn. Less to learn means you can learn these tools deeper. Even better, if you pick your tools with care you can count on having access to them on almost any hardware. Similarly choosing battle-hardened tools can help you be sure they will stick around. These factors combine to make these skills more transferable than the framework of the week. Add c/c++ to this stack and you have some of the most valuable skills in software & IT.

These two experiences stuck with me. They taught me that my sense of mastery depends on my ability to understand the context I am working in. This idea seems obvious, almost tautological. “Developing software in a context you understand is easier than doing the same in a context you do not understand.” Seeing this written out helps us see what it implies, and how we can use these ideas to help us.

The big idea here, and one that I haven’t seen explicitly made yet is, “The easiest way to make your development environment understandable is to make it small.” A small environment is easier to learn, easier to maintain, easier to deploy, and more secure. This seems to be the underlying drive behind containers. It motivates the popularity of micro-frameworks and new Linux distributions. It has even driven attempts to re-write portions of the stack to try and make them more understandable.

This all leads us to the question, how do you get such a small environment and what is worthy of adding to your short-stack? On Monday, I will share my software short stack, and the criteria I use to decide if something is worth adding. In the meantime what are your thoughts on short-stack development, and what tools do you use that absolutely must have to work productively