From: Simon Wistow
Date: 10:59 on 28 Aug 2003
Subject: software with surprising depths of superficiality
I'm actually talking about a particular bit of code that I've inherited
but I've seen this phenonomem elsewhere so I'll paraphrase.
I'm not sure if there's a neat buzzword for this particular species -
neither lasagne nor spaghetti code seems to work. Perhaps 'spirelli'
code.
Anyway, lurching back to the point. I've inherited this code. The task
seems simple enough - indeed I have a suspicion that if I was to
reimplement it from scratch it would take a week or two. Instead, on and
off, I've been poking at it for nigh on six months now.
Now to be honest this is a week here and a week there but still. Six
months. You could rewrite Emacs in Perl in that space of time. Or
Mailman :)
Why is this code so bad? Well, its lineage for a start - I can
practically see its evolutionary tree.
Currently it is a rag tag collection of shell and Perl scripts. Some of
the Perl scripts obviously used to be shell scripts to the extent that
they actually SHELL OUT TO OTHER PERL SCRIPTS. Not for forking reasons,
oh no. Just because, well, the original author had partaken of the
crack pipe and not drunk deeply from the cup of refactoring. I have to
concede that they did move some of their 'require'-d code into modules
but the design of those, plus the fact that they lurk in various lower
case namespaces or, and I'm not sure if this is better or worse, in the
Test:: namespace, means that this is not enough to let them off the
hook.
Even as I write this I'm struggling to explain why this has caused me so
many problems. On paper, and I know this because I had to write it down
yesterday, it all seems perfectly easy. But, obviously there is,
otherwise I wouldn't be writing this.
My two, no wait, three major issues with this stuff are these :
1. The code is *incredibly* brittle
This to the point that if I leave it for a week and then run it again I
can practically guarantee that it will not work, often failing silently,
and I will spend a full day poking around trying to diagnose the
problem. Often there will seem to be no reason for this.
2. It's amazingly hard to follow.
There are several reasons for this - my particular favourite being the
lack of indenting, the terse variable names, the overuse of hash
references (which are normally fine but become incredibly difficult to
follow when nested 8 deep and passed between required files to functions
with 10 arguments with no consitent naming scheme or ordering), the fact
that there are many "actions at a distance" not least the use of
Environment Variables as some sort of perverse global variable system,
like the author went "well, I'm told globals are bad so if I put then in
ENV then nobody will know". Truly, the list goes on and on. I haven't
even mentioned the lack of comments and the long, undocumented regexes
with unprintable characters in. On and the fact that various libraries
are in various people's home directories.
Testing is also a problem because a trial run takes 4 hours and requires
a full days worth of data meaning that a fix checked in on Monday won't
be checkable until after lunch Wednesday.
3. I'm not allowed to refactor.
Apparently the main reason I'm not allowed to is that then it won't be
'trust worthy'. I've mentioned Unit tests but these have been roundly
ignored. Also, the code has been internationalised from the original
which is also being developed (don't ask why there are mutliple
branches). So I can't refactor unless I want my life to be a manual
patch applying hell. Hell, I can't even use Perl Tidy (I tried -
waaaaaay too many spurious deltas)
Actually, there's a fourth.
4. It makes me feel shit
I know I'm not a bad programmer[0] (stop sniggering Richard). I've
written far more compilcated programs than this. Yet it has single
handedly been the most disasterous project I've ever worked on. I seem
to find it impossible to give, let alone keep, a deadline. It saps my
confidence and I have huge problems forcing myself to work on it which
in turn makes me feel worse. Et cetera, ad nauseaum, ad infinitum.
Right, that's enough moping.

From: =?ISO-8859-1?Q?Ask_Bj=F8rn_Hansen?=
Date: 11:05 on 28 Aug 2003
Subject: Re: software with surprising depths of superficiality
On Thursday, Aug 28, 2003, at 02:59 America/Los_Angeles, Simon Wistow
wrote:
> Right, that's enough moping.
And now to look for New Place of Work!
Sounds like you need it very badly. :-) At least get an offer or two,
then you can tell your local PHB that you either get support and time
to refactor The Monster - or you are out.
- ask

From: Piers Cawley
Date: 13:29 on 28 Aug 2003
Subject: Re: software with surprising depths of superficiality
Ask Bj=F8rn Hansen <ask@xxxxxxxxxx.xxx> writes:
> On Thursday, Aug 28, 2003, at 02:59 America/Los_Angeles, Simon Wistow> wrote:
>
>> Right, that's enough moping.
>
> And now to look for New Place of Work!
>
> Sounds like you need it very badly. :-) At least get an offer or two,> then you can tell your local PHB that you either get support and time> to refactor The Monster - or you are out.
Ah, the old "If you can't change your organization, change your
organization." technique.

From: Ann Barcomb
Date: 11:34 on 28 Aug 2003
Subject: Re: software with surprising depths of superficiality
We've probably all had to maintain horrible software, so even if I
haven't met your particular burden, you have my sympathy.
I think I'll avoid listing which jobs and places of employment were
responsible for these disasters, but I have at one time or another,
had to maintain all of the following:
* poor directory structure (modules, logs and cgi in the same directory)
* variables shared between scripts and modules without ever being explicitly
passed
* the same sort of data (such as translations) stored in multiple
locations--sometimes in a file, sometimes in a database
* credit card numbers stored unencrypted in databases
* code littered with exceptions for projects or features which have since
expired or faded away
* unused code which hasn't been deleted
* unused database fields
* modules which export all 200 functions by default
* hardcoded deployment information scattered across dozens of files
* no revision control in sight
* no tests
* no documentation
* multiple versions of the same application which differ just enough to
make maintaining each a seperate nightmare
* no development environment; everything is done on the live server
* multiple HTML pages embeded in a single script
* thousands of lines of code without a subroutine
* subroutines which neither accept arguments nor return values, but set
global variables
* dry run code which shares nothing whatsoever with the real run code
* code which avoided nested data structures by having hash values which
were in fact the names of the other hashes
(NOT: $hash{foo} = \%bar, BUT: $hash{foo} = 'bar')
* subroutines which can take dozens of optional arguments, with each specifying
a slightly different result in the routine (without using named
arguments)
* OO modules that did nothing which required OO, but instead exported a
bunch of 'methods'
* passwords in plain text in cookies, databases and URLs
* really odd home-grown templating/cgi syntax
* error messages with creative values like 'ERROR 2'
* single letter variable names (when variables > alphabet, use 2-character
names)
I think I have to stop. I'm getting depressed.

From: Piers Cawley
Date: 13:32 on 28 Aug 2003
Subject: Re: software with surprising depths of superficiality
Simon Wistow <simon@xxxxxxxxxx.xxx> writes:
> 3. I'm not allowed to refactor. > Apparently the main reason I'm not allowed to is that then it won't be> 'trust worthy'. I've mentioned Unit tests but these have been roundly> ignored. Also, the code has been internationalised from the original> which is also being developed (don't ask why there are mutliple> branches). So I can't refactor unless I want my life to be a manual> patch applying hell. Hell, I can't even use Perl Tidy (I tried -> waaaaaay too many spurious deltas)
Have you been in touch with the other team? Directly, with the
developers. Maybe they feel just as bad about it as you do, but
daren't touch anything because there are other developers like
you. Maybe they're not, but you can't know unless you communicate.
At best you'll all be able to present a united front to management
when you say it needs refactoring. At worst, you'll know they're a
bunch of hacks and you can start pushing (hard) to be reassigned or
ship out.

From: David Cantrell
Date: 21:52 on 28 Aug 2003
Subject: Re: software with surprising depths of superficiality
Simon Wistow wrote:
> Currently it is a rag tag collection of shell and Perl scripts. Some of > the Perl scripts obviously used to be shell scripts to the extent that > they actually SHELL OUT TO OTHER PERL SCRIPTS.
Gosh, that sounds familiar. [looks around, hopes $colleague isn't on
this list] One of the many things on my to-do list at work is replacing
such an abortion. Well, actually it's to re-jig it so that it does a
whole bunch more, but I'm damned if I'm hacking on that mess, so a
re-write it is. And the re-write will not involve any of that XML crap.
But in the mean time, I have far more interesting things to do, thank
$deity.
> 2. It's amazingly hard to follow. > There are several reasons for this?- my particular favourite being the> lack of indenting, the terse variable names, the overuse of hash> references (which are normally fine but become incredibly difficult to> follow when nested 8 deep
Are deeply nested data structures that naughty? With sensible keys I
find them easy to use, even if you do things like ...
$foop->{boing}->{$foop->{wibbles}->{$wibble}}
Provided the keys are sensible - which does not preclude being terse - I
see little to be gained from creating temporary variables to hold the
individual bits of that just to make that one little bit of data access
shorter. It certainly won't make the code easier to read IMO.