Thursday, 18 March 2010

A quote and a rethink

“Rule of Modularity: The only way to write complex software that won’t fall on its face is to build it out of simple modules connected by well-defined interfaces, so that most problems are local and you can have some hope of fixing or optimizing a part without breaking the whole." -The Art of Unix Programming

Now, looking at what's been going on with data-oriented development I see that there are some words that though at the time pertinent, actually cause an inflexibility of interpretation. An inflexibility that will allow many to point and laugh at the data-oriented crowd. The problem is with the natural interpretation of modules and interfaces.

Modules and interfaces were just a way of saying break down the complex stuff into smaller easier to manage stuff, but the meaning has been lost as we have solidified what a module and an interface means. Let's go back and think about this again. Breaking a problem of many entities down was solved a long time ago by database engineers, they invented many tools, and even created a generic but powerful interface to manipulate their data. This could be called modularising the idea of persistence. SQL was the interface.

We can do the same, ignore the words "modules and interfaces" instead concentrate on the idea, separated and distinct techniques for processing objects.If we allow ourselves to define objects as the streams or arrays of data, then all we need to do is write a lot of processes that operate on them. Each process does something, usually something somewhere between simple and complex as we don't want to waste data throughput on tediously simple problems (which is what OO advocates normally), and we don't want too much data per row (as that will cause cache issues). Which is why the data-oriented approach works really well with the old Unix programming quote, but only if you try to distil the essence of the quote, not just use the words blindly.

So, a refactored quote.

"Rule of Modularity: The only way to write complex software that won’t fall on its face is to build it out of simple data definitions connected by well-defined transforms, so that most problems are locally defined and you can have some hope of fixing or optimizing a single link without breaking the whole chain." -The Art of Unix Programming (revisited for modern hardware architecture - me)