I get very annoyed about politicians being held to account for admitting they were wrong, rather than forcefully challenged when they were wrong in the first place. Unless they lied, if someone was wrong and admits it, they should be congratulated. They have grown as a human being.

I am about to do something very similar. I’m going to start confessing some wrong things I used to think, that the world has come to agree with me about. I feel I should congratulate you all.

You can’t design a Database without knowing how it will be used

I was taught at university that you could create a single abstract data model of an organisation’s data. “The word database has no plural”, I was told. I tried to create a model of all street furniture (signs and lighting) in Staffordshire, in my second job. I couldn’t do it. I concluded that it was impossible to know what was entities and what was attributes. I now know this is because models are always created for a purpose. If you aren’t yet aware of that purpose, you can’t design for it. My suspicion was confirmed in a talk at Wolverhampton University by Michael ‘JSD’ Jackson. The revelation seemed a big shock to the large team from the Inland Revenue. I guess they had made unconscious assumptions about likely processes.

Relations don’t understand time

(They would probably say the same about me.) A transaction acting across multiple tables is assumed to be instantaneous. This worried me. A complex calculation requiring reads could not be guaranteed to be consistent unless all accessed tables are locked against writes, throughout the transaction. Jackson also confirmed that the Relational Model has no concept of time. A dirty fix is data warehousing which achieves consistency without locking by the trade-off of guaranteeing the data is old.

The Object Model doesn’t generalise

I’d stopped developing software by the time I heard about the Object Oriented Programming paradigm. I could see a lot of sense in OOP for simulating real-world objects. Software could be designed to be more modular when the data structures representing the state of a real-world object and the code which handled state-change were kept in a black box with a sign on that said “Beware of the leopard”. I couldn’t grasp how people filled the space between the objects with imaginary software objects that followed the same restrictions, or why they needed to.

A new wave of Functional Programming has introduced immutable data structures. I have recently learned through Clojure author Rich Hickey’s videos that reflecting state-change by mutating the value of variables is now a sin punishable by a career in Java programming. Functional Programmers have apparently always agreed with me that not all data structures belong in an object

There are others I’m still waiting for everyone to catch up on:

The Writable Web is a bad idea

The Web wasn’t designed for this isn’t very good at it. Throwing complexity bombs at an over-simplified model rarely helps.

I learned to ‘programme computers’ long ago, almost before there was no “me” in ‘program’ and certainly before I knew how to ‘team’. I had a very brief and unsuccessful exposure to functional programming in LISP (not Lisp) then stopped. I did other ‘Data Processing’ things.

In recent years I’ve been working as an analyst, alongside people who write code according to the object model. I think I have a feel for objects but never having written code in an object-oriented language, I can’t be sure. I decided to try, in the Python language, then got distracted by the shiny Clojure language which is functional. I feel that right now I’m approximately equally confused by objects and functions, so I thought I’d write this quickly before I know what I’m talking about. I can come back later to laugh at my naivety, along with the rest of you.

Like the person who wrote this https://news.ycombinator.com/item?id=4246018 ,
“I’ve been watching some talks online recently by Rich Hickey of Clojure fame”
The post asks “So if I follow Hickey’s advice, how am I supposed to represent a book? As a vector of vectors of vectors of vectors of strings? If so, then how do I prevent a change in the representation of the Book from breaking client code?”

I found the question very interesting because representing ‘books’ in a functional language is exactly what I want to do. I think differently to the author because I’m not yet trapped inside the object paradigm. I can see that ‘book’ is a real-world class of objects, a very specific and limited implementation of the representation of a small subset of all the information in the world. That’s what my ‘book’ was going to be about and why I’m now playing with functions instead of writing it.

Objects are good at simulation of real life systems. They encapsulate small sub-systems of a process and it’s local data into an object. What I always struggled to understand was what you did with the data that didn’t want to be enclosed – “information wants to be free”. People seem to cope by inventing objects that don’t really exist: to be data shepherds.

Functions are good at abstraction. A book is a single output format from something much richer. That’s what I want to write. Data and processes are complex. Objects and functions are simplifying models; there may be others.

I graduated just after the Structured Programming War was won. I was probably the first generation to be taught to program by someone who actually knew how; to be warned of the real and present danger of the GOTO statement and to be exposed first to a language that didn’t need it. I didn’t need to fall back to assembler when the going got tough or to be able to read hex dumps or deal with physical memory constraints. I entered the computing profession just as people were starting to re-brand their programmers as ‘software engineers’ and academics were talking of ‘formal methods’ then ‘iterative development’ and ‘prototyping’ as we lost faith and retreated, as the techniques borrowed from other engineering disciplines continued to disappoint when applied to software.

After some years away from software development, I returned to find ‘Agile’, ‘Lean’ and ‘Software Craftsmanship’. We’d surrendered to the chaos, accepted that we weren’t designers of great engineering works but software whittlers. I was pleased that we’d dropped the pretence that we knew what we were doing but disappointed that we’d been reduced to hand-weaving our systems like hipsters.

There had been another good change too: The Object Model. The thrust of software engineering had often been decomposition but our model had been the parts breakdown structure, the skeletal parts of a dead system. Objects allowed us to model running systems, the process network at the heart of computation. I’ve recently seen a claim that the Unix command line interface with its pipes and redirection was the first object system. Unix begat GNU and Free software and Linux and close to zero costs for the ‘means of production’ of software. I don’t believe that Agile or Lean start-ups could have happened in a world without objects, the Internet or Free software. Do you know how much work it takes to order software on a tape from the US by post? I do.

So here we are, in our loft rooms, on a hand crafted loom made of driftwood and sweat, iterating towards a vague idea emerging out of someone’s misty musings and watching our salary eroded towards the cost of production. Is this why I studied Computer Science for 3 years? Who turned my profession into a hobby activity?