There’s a long tradition (since 1968, according to Wikipedia) of looking at software development as an engineering discipline. But calling software development engineering is something like adhering to the letter of the law, rather than to the spirit

Software development is about acquiring knowledge

If you think of software development as strictly an engineering discipline, you might be inclined to believe that the job of a software engineer is to apply his or her previous knowledge and training toward solving a particular problem. And you’d be right, to a point, because knowledge and training are important aspects of software development.

But you’d also be missing the key factor, that software development is mostly about acquiring new knowledge. Software developers are always engaged in the following pursuits:

Discovering what to build

Discovering how to build it

This goes a long way toward explaining why so much software is custom software. If software development were entirely an engineering discipline, it is conceivable that we would be able to construct just about any software application by plugging together the proper parts, chosen from a catalog of well-known, existing components. The fact that we aren’t even close to being able to do that is an indication that we still have a lot of discovery ahead.

Discovering what to build

How do we discover what to build? By working closely (and continually, if possible) with the customer. By getting feedback as often as possible from real users, and by constantly applying that feedback to the product under development.

Discovering how to build it

From the developer’s perspective, this is a career-long pursuit, as new techniques for building software are always appearing. As software developers, we can provide the greatest benefit to our clients by being aware of new developments in the field, but tempering that with the client’s needs to meet specific schedules and cost targets.

For any particular product, we can practice Test Driven Development (TDD) and encourage emergent design, so that we avoid imposing our own pre-conceived (engineering) notions on the product, and instead develop just what the client needs (and no more).

In software development there are many ways to transfer the knowledge about how to build a product to the people who do the actual building. Production can be severely hampered, however, if that knowledge is being produced more rapidly than it can be consumed. This is the knowledge transfer bottleneck.

I recently hosted a workshop that let participants experience three different ways of transferring knowlege in a production environment. The product, in this case, was a paper airplane of unusual design. The idea was to try different ways of transferring the knowledge about how to build the airplane from the “chief designer” (me) to the production workers, and to compare the relative productivity of the different methods, which were:

Documentation – The workers were given written instructions (22 steps worth) for building the airplane.

Reverse Engineering – The workers were given a completed airplane which they could study in order to reproduce the steps required to build it.

Mentoring – The “chief designer” built an airplane step by step and the workers replicated each step as it was performed.

The experiment was conducted in two phases. In the first phase, all 8 participants used the Documentation method. In the second phase, one team of 4 tried Reverse Engineering, while the other team of 4 tried Mentoring.

The results were interesting. Using the Documentation method, only one person out of a total of 8 came close to being able to build the airplane at all in the 5-minute period allotted.

Using the Reverse Engineering method, 1 person out of a total of 4 produced a completed airplane in 5 minutes.

Using the Mentoring method, each of 4 team members produced a completed airplane, and in less than the 5 minutes available.

The knowledge transfer bottleneck in software

In a software development effort, knowledge transfer takes place all the time, and it’s easy to imagine a software developer in the “chief designer” role described in the exercise above.

Let’s say I’m a developer who has discovered, and written the code to implement, a technique for binding some data to the controls in a user interface, and that this technique forms a pattern that my fellow developers want to know about. If you were one of my fellow developers, would you rather I (a) gave you a document I had written about the technique, (b) told you where the code was and suggested you figure it out for yourself, or (c) paired with you to implement the pattern for a new set of data?

Now, certainly, pairing with you takes more of my time, and might seem less efficient from my viewpoint. After all, I could be off designing the next pattern, and the one after that. But the productivity of the team as a whole, rather than my personal productivity, is what’s important. And mentoring helps increase the team’s productivity by avoiding knowledge transfer bottlenecks.