So your client asks you to write some code, so you do. He then changes the specs on you, as expected, and you diligently implement his new features like a good little lad. Except... the new features kind of conflict with the old features, so now your code is a mess. You really want to go back and fix it, but he keeps requesting new things and every time you finish cleaning something, it winds up a mess again.

What do you do? Stop being an OCD maniac and just accept that your code is going to wind up a mess no matter what you do, and just keep tacking on features to this monstrosity? Save the cleaning for version 2?

I think this is a good place to apply the 80/20 rule.
–
Mark CNov 26 '10 at 0:30

7

@Roopesh: It isn't ugly in the first place, it's when you keep tacking things on that it becomes ugly. With the added features, you realize that the core structure could have been designed differently to better support those features. And at that point you can either go back and re-write large chunks of foundation, or just add the feature. Usually there isn't enough time to back and re-write half your program.
–
MarkNov 27 '10 at 7:52

17 Answers
17

But in all seriousness: estimation padding is your friend. I generally do a decent realistic estimate, then double it. This might sound excessive, and sometimes it is, but it's better to overestimate a bit and even look a tad slow sometimes - than to leave bad impressions by turning out buggy code and always blowing your estimates. And of course, you incur technical debt by letting the codebase get hacky.

Another (related) tip: Always estimate seemingly tiny, no brainer tasks to a decent sized block. Say, for example - an item which you are almost sure will just be a simple one line 30 second change - give it 1 hour (or maybe whatever the lowest time block is on your timesheet or CR system, eg 15 mins/0.25 hour). And give half-day or 1 day blocks for the slightly bigger but still relatively trivial items.

The reason for this is mainly psychological: I find that if you get in the habit of quickly hacking up small changes, the work feels rushed, and you never end up sitting back, taking stock, and refactoring stuff that needs to be refactored. Also, on a practical level: small-but-non-trivial changes sometimes blow out, and you don't want to be constantly feeling like you're behind schedule and putting out bug fires. It's part and parcel of why code bases get hacky over time.

Finally, always remember that people don't have to know that you are padding your estimates somewhat. As long as you're a competent developer and you're cranking out work at a decent pace, this padding won't be noticeable. ie Don't tell the PHB "My initial estimate is that it'll take two hours, but give me half a day". Just tell him "I think it'll take about half a day." and leave it there.

@muntoo: Took me a second to realize what that did... didn't see the for. Cute ;)
–
MarkNov 26 '10 at 3:16

3

I'm sure this depends on the manager, but you don't necessarily need to lie. The CTO and I have an understanding; he knows that I can give a reasonable estimate but only with about 50% confidence; if I put in a fudge factor then I can give the same estimate with a 90% confidence level. And it turns out that, over a long period of time, most people prefer reliable estimates to naïvely optimistic ones, even if they don't admit or realize it, so he gives the pessimistic estimate to his boss unless it's an emergency.
–
AaronaughtNov 26 '10 at 3:52

2

The point that absolutely nothing takes less than about half an hour is very well made. Even if a single change to the code takes 5 minutes there's a raft of overhead that goes with.
–
MurphNov 26 '10 at 9:08

2

@Murph - spot on. I refuse any commercial estimate of less than half a day. By the time the developer has grabbed the right code, made the change, run the unit tests, passed the build to test and test have sanity checked it, NOTHING takes 5 minutes.
–
Jon HopkinsNov 26 '10 at 10:30

I think that's a reasonable way of handling the problem too. The pain they subject to the developers needs to be passed back as additional cost. Management and the sales force have to buy into that philosophy too, otherwise the developers will get the shaft and be subjected to ever worsening code-bases.
–
the Tin ManNov 26 '10 at 1:24

Is this estimation padding? It looks like time to implement a new feature while keeping code quality to me.
–
David ThornleyNov 28 '10 at 3:21

2

I think this is mostly the right approach, but I would characterize it differently. They are hiring you to develop professional quality code. That means you need to build into your estimates time for "doing it right". Don't make estimates based on the time it would take you if you stayed up all night hacking and declared "complete" as soon as it ran correctly the first time. This may mean in a competitive situation you will be underbid sometimes. That's ok. You will develop a reputation for delivering quality and consistency and will ultimately win. Play the long game.
–
Brandon DuRetteMar 3 '11 at 14:16

Try to do proper redesign while integrating new features. There is no later. Without the redesign you are constantly adding more and more friction for further changes and new features.

At some point you'll come to a grinding near halt where everything seems to take ages. Most companies probably go for the big rewrite at this point, version 2. It has pretty poor economics and is a good moment for your customer to try a different development party if they feel inclined.

With all the comments about over-estimating I think that there's a modest amount of point (well opportunity) being missed.

Its not about estimating the time taken make the change (just) and then adding some, its about estimating the time required to modify the code (refactor!) to bring it to a point where the change may be safely made and to then make the change (probably somewhat munged together). Ok, this amounts to the same thing... but there's no question of fudging or stretching or over-estimating, its simply a matter of saying that in order to do this I first need to do that and this is how long it will take in total. Key here is that you work on those parts of the system that the change is dependent on and no more - if there's horrid code elsewhere... tough, catch it when you're there.

To come back to the original question a bit - after a lot of years its come down to this for me, when you implement something unless you know (not believe, not expect (suspect?), not think but know) that additional stuff is also required then you should do what you need to implement that requirement and no more in as tidy and elegant a fashion as you can.

When you come to implement the next thing - some while later - you take the steps required to bring the codebase (and the database and whatever) to the state needed to implement that functionality as in as tidy and elegant a fashion as you can. This refactoring is where you deal with the mess that arises naturally as a project evolves - and hopefully avoid creating more mess (or at least keep the level consistent).

One of the discussion areas here is "Technical Debt" - its like an overdraft, you have to pay it back and the longer you leave it the more interest (in this case time required to correct) you will accrue - which gives you a good argument for spending some of your time minimising the technical debt.

This is also where unit testing and other automated testing starts to come in (if I could do as well as I say I'm fairly certain I would be a happier person!) combined with a proper build server (that can run at least some of your tests). Combined with those - but of value in an of themselves - are patterns like dependency injection and inversion of control (never quite sure how much the "same" those two are) because they make it easier to change the plumbing and hence deal with changes in isolation.

Lastly - remember, if it ain't broke don't fix it. Tidying your code purely for the sake of tidying it may be satisfying but its also an opportunity to introduce errors so whilst it can be painful if you don't need to change it and are not building upon it it may be better to leave some lumps alone - the opportunity to fix or replace will roll around eventually!

If the customer changes the specification that's fine, that's his right, however it is a change and it needs to be charged for (or costed in whatever manner is appropriate to the project structure / relationship).

The estimate for that change should include the cost of the necessary refactoring. The client may well baulk at what seems to be a high cost but at that point you need to explain to him that because the code is already half written there are elements that need to be rewritten to ensure it's robust and supportable in the future and that if it's not done then he's likely to have problems down the line with future support or changes becoming even more expensive.

2) Refactoring Should Be Done Such That Is Provides Genuine Long Term Benefit to the Client

When considering refactoring you always need to consider what's actually needed and what's important and ensure that the refactoring work provides genuine long term value for money.

After all, we should do these things so the code remains extendable and supportable in the medium / long term to ensure that the client's investment remains valid rather than out of any drive for theoretical perfection. Refactoring work (and the corresponding estimates) should be done with this as the scope, and not just because you now think there might be a slightly better way to do it.

Charge by the hour and if he wants changes say that is fine but incorporate the time required to write good code into the equation. Also remember writing neater code pays off long term when you have to maintain it. Saving time now could cost you later.

Some programmers suggest that a way of controlling that problem with clients is to have client sign and authorize the initial specification. THEN, when they request a requirement change which is not in the initial spec, you tell them that you need to go through the contract and project timetable in order to calculate additional costs and time delays, then make an annex to the contract. Apparently it does wonders in preventing clients in insisting on new (unpredicted) features.

+1; It can work, however it also runs the danger of alienating your client by being too inflexible. To some degree whether you can do this or not depends on the type (size) of the project and the client's expectations.
–
Ken HendersonNov 26 '10 at 14:01

I have the following comment in a code base that I'm currently working on:

/*
* Every time I see this function, I want to take a shower.
*/

I know, very well the situation you are describing. What I do is try (my best) to wait until things settle down and any kind of 'creep' has 'crept' all that it's going to do. By that time, you probably have something usable released, and you can take some time to clean things up and implement stuff a bit differently.

You can't run around cleaning up lots of little messes repeatedly. That just triples your work, and frustration. Wait for it to become one bigger, but hardly moving mess and then you can do something about it.

It all depends on HOW you read the spec. Its easy to think of them as stone tablets, but in reality most specs change. When you design your code, take a view on how likely that each part of the spec is going to change. Over time, you will get quite good at predicting this.

Having got into the mess, experience and judgement is very important. Are you writing new bugs because of this spaghetti code? is it taking longer to implement? these would point to doing a tactical refactor.

for the future, it sounds like you need to work in partnership with your customer. Saying to them, "look this product is expanding significantly beyond the original spec. Whilst the original design was good for that level, expanding it in direction X and directions Y need some restructuring in the design" handled well, and you will even get your customer to pay for it.

Initial proper design cannot help avoiding the problem. And it is almost impossible (or very-very hard) to consider all future "maybe" requirements. So after some time the Big Re-factoring will arrive. And the best solution is to re-write everything.

In a few words: instead of fitting a turret onto the red Ferrari, re-consider the requirements and build a tank.

I think writing software needs to go hand in hand with business needs. If it is a throwaway project (like a prototype that needs to be built in a week, with new inputs coming in every day), then there is no need to worry about code maintainability and other stuff - time is crucial and you just need to push your code out of the door as fast as you can.

But if you are writing a long term app, then it makes sense to consider all this, because there is a sizable impact on how long it takes to build new features, to fix existing bugs, to integrate into other applications and other stuff - and this translates into business impact (because of more time required later and more cost).

So it is better to sensitize the decision maker to the actual costs of not refactoring code whenever necessary - in my experience, if the costs and time impact of both the options are explained in measurable terms to the decision owner, then the decision can be a no-brainer. Don't expect people to tell you 'yeah go ahead write beautiful code, even though it takes twice as much time and gives me no extra benefit'. It just doesn't work that way.

Make it part of your process, I call it "extreme refactoring" and it's going to be big! ;) Just do stuff quickly and when enough new features has been added that there's scar tissue, refactor it. Continually ask yourself "Now if I had started from scratch, how would I have done it"

People who think they can design and think about everything up front is mostly fooling themselves, you (and your client) always learn things as you go along. Use those lessons.

As you you're a good programmer you'll be able to refactor pretty quickly and as you do it continually the code will start take it's "proper form", meaning it will become more flexible with less dependencies.

Clients might be miffed if they knew you were "wasting time" reworking stuff so it helps not asking/telling and being really quick about it.

Code developed this way will save you loads of time in the end and will make it increasingly easier to add new features.

I'd also say that one of the biggest reasons for bad code is the fear some programmers have on doing larger structural refactoring, and the longer you wait the worse it gets.

Aka refactor as soon as possible: for example when ugly code comes from rushing for a deadline, I'd refactor after the deadline because you can't (or shouldn't at least) add more features until the existing code is maintainable, otherwise it's going to make it that much harder to debug future codes.

I don't mean praying. I mean, make sure there is a business guy (ie project manager or equivalent) that you can place as padding between you and the client. If the client is demanding too much let the business guy put the foot down and be ready to wield the "it's do-able but I'm not sure whether that fits within the scope of the specification, see [business guy]".

In a normal project flow, the general specification should be frozen before serious development takes place.

Many clients will continue to drive for changes/improvements/enhancements as long as you let them. Many will abuse that ability to the max because it makes them feel like they're getting the most for their money (even if it sabotages your project).

Have a person dedicated to perfecting and freezing the specification early on and enforcing it later on.

There's nothing wrong with doing a little extra for a little good karma with the client but be ready to defer to a higher power when they get out of hand. If the specification requires a ridiculous amount of changes, maybe it's time to go back through the business loop and re-assess the contract and/or add additions to the contract (with fair monetary compensation).

The fact that you're having this issue has little to do with how you code. It's a sign that your project manager is underutilized on the project (whether it's your fault, his fault, or both).

Like others have said in many answers, adding a time buffer for contingencies is also necessary on any project but determining that should be decided behind closed doors before the specification is frozen and delivered to the client by the PM.

Learn from the code with your team, see what can be refactored and reused next time, then go have a beer.

From a in-development perspective:

Patiently explain why development has stopped, and explain why it cannot continue until all specs are on the table and understood. Then, go have a beer.

From a planning perspective:

Request all specs up front and work with everyone to have clear understanding of development path. Involve client / stakeholders as closely as possible to make sure everyones on the same page. Later that night, get everyone beers. Tomorrow, start the project.