The Art of Maintenance Programming

Introduction

I have seen and read numerous books and articles that cover new development, including design and architecture, coding, and project planning and management. Not so for humble and overlooked maintenance programming. Maintenance is often mentioned in studies on software life cycle, but actual design and coding techniques are usually taught with active development in mind, rather than maintenance.

Here, I am not going even to try to define maintenance programming, or to quote someone else’s definition. Instead, I’ll just list some characteristics of a maintained software as opposed to an actively developed one:

It has been deployed and used.

The original developers may have left the organization or switched to other tasks. There are less people involved in maintaining the product than developing it.

Expectations from maintained software are generally lower. Maintenance usually includes fixing bugs as reported by users, and adding some features, but it is not the “flagship”.

Maintained code is often developed with legacy or even unsupported technologies. The quality of code has usually deteriorated due to last-minute bug fixes and other hacks.

Maintenance Programming Good Practices

The tips I offer here are the result of my own experience. You may find they don’t apply well in your situation, or even more likely, that some of them apply and some of them don’t. If you have any additional advices, please post them at the article’s forum.

1. Get to Like It

How can anyone like maintenance programming? Developers dream of being chief architects in their teams, and when they end up maintaining existing software, it feels almost like punishment. It doesn’t have to be: maintenance programming has its own set of challenges, and requires a lot of creativity, adaptability, patience, discipline, and good communication. It can be good for your career as well: having bombastic entries like “Architected n-tier 24/7 enterprise application” in your resume looks nice, but employers actually value people who solve problems; and maintenance programming can be a good chance to demonstrate your problem-solving skills.

2. Get Familiar with the Functionality of the Software You Maintain

It is crucial to know the functionality of the software you maintain. Read the existing documentation, and make sure to try it for yourself. If you have a QA department, try to dig out the test plan, and go through it. Find out how the software is being used in practice and make sure you know the most common scenarios of use. There will be cases when users come to you requesting features that already exist, they just don’t know about them. In other cases, there will be handy workarounds. If you are a power user of the software you maintain, chances are you will spend less time in the debugger.

3. Get Familiar with the Architecture of the Software You Maintain

You would expect to inherit detailed and up-to date design documentation, including requirements, functional specifications, high-level and low-level design, as well as well-commented code, right? In reality, chances of that happening are close to zero. It is much more likely you get a little, if any, documentation, and even that will probably be outdated. Code, or some parts of it, may be commented, but you can’t be sure even the comments are up to date; sometimes they can actually be deceptive if the code was changed in the meantime.

Look at the code; try to understand the roles of classes, modules and components. Use the debugger: step through different scenarios of use, and see what happens when different parts of code are executed. Be sure to write down your findings and organize them in some form; it can be some formal method like UML diagrams, or something informal but easy to understand. Think of getting familiar with the architecture as of an ongoing process, rather than one time thing. You will have the opportunity to know the system even better when you start to fix bugs and add features; be sure to document your findings in this stage as well.

4. Communicate with the Users

I can’t stress enough the importance of this. Many software developers are introvert persons who prefer to deal with technology rather than with people. However, the software is written for people to use it, and in the case of maintained software, we already have users. It is crucial to find out how they use the software, why they use it, what problems are they trying to solve, what functionality they need. To find that out, don’t wait for them to come to you; be proactive and go to them. Try to establish simple and effective procedures for submitting bug reports and enhancement requests. When they report a problem, get back to them immediately, even if you don’t have a solution at that moment – just let them know you are looking into it and not ignoring them. Keep them updated about the status of their problem. Finally, be honest; if you can’t meet their request for any reason, tell them.

5. If Possible, Communicate with Original Developers

If you are lucky, some of the original developers are still with the company. Sure, they have moved forward and want to forget about this old crappy software you are maintaining, but they can be a tremendous help and save a lot of your time. Don’t expect them to do the actual coding for you, but they can throw you an idea how to accomplish something, explain you why something works the way it works, and provide you with lots of helpful hints. Again, dealing with them requires “people skills” that many developers lack, but it is not something we can’t learn.

6. Keep the History of Changes

There are different and not mutually exclusive ways to keep the history of changes. The most obvious thing to do is to comment each and every check-in to your source control system (I am not even going to consider the possibility you don’t use a source control system). Some people prefer to put the list of changes at the top of each file. It may be convenient to have a centralized list of changes in a spreadsheet as well. Whatever approach you take, be serious about it. An accurate history of changes is invaluable for successful maintenance, even if nobody else before you took care of it, and you get only a partial list of changes.

7. Be Conservative

Many times during the maintenance work, you are going to get frustrated with the existing code in some module, and just want to toss it away and rewrite it from scratch. Before you actually do that, ask yourself some questions:

How would it benefit your organization and its customers? The fact that some code is ugly, or looks ugly to you is not per se a reason to rewrite it. If a customer wants a feature that can’t be added to existing code, that could be a reason to consider rewriting it.

Are you sure you understand what this code does and how it works?

Are you sure you understand how it interacts with other parts of the system, and which modules and functionality depend on the code you want to rewrite?

Are you sure you can make it better? Maybe you run into the same problems the original developers did, and make it same or even worse.

The fact is that whenever we make a change to existing code, no matter how trivial the change is, there is a risk of introducing bugs. Therefore, we want to be as conservative as possible given the circumstances. When a new feature is requested, we would want to implement it in a separate module if possible and keep the old code unchanged. Don’t fix it if it isn’t broken. If it is broken, fix it rather than rewrite it from scratch. Keep the changes as local as possible – you don’t want to find out that you improved one part of the system and broke another.

Refrain from refactoring. While it can be a useful technique during active development, in maintenance phase, it will probably bring you only troubles. Even if you don’t break anything through refactoring, you will complicate your history of changes and screw up the differences between versions in your source control system (you do have a source control system, right?).

Also, when you do change the code or add new features, try to do it the right way immediately. Too many times I’ve seen people say “let’s just make a quick hack now to meet the deadline, and we can rewrite it properly later”. Well, “later” usually never comes, for various reasons. If you can, do it properly; if not, be honest with yourself and don’t mention “later” as an excuse.

8. Test after Every Change You Make

Probably the first coding task you need to do after taking some software to maintain is to write a comprehensive regression-test suite for it; that is unless you inherited one already, but in general, you are not going to be that lucky. It is true that automated tests cannot guarantee the code is bug-free, but it is a poor excuse to avoid them; they will not uncover all bugs, but they will help you find many and bring you confidence that the core functionality of the software you maintain is not broken after you do a change. A good practice is that for each defect report you get, write a test-case to reproduce it. That way, you will know if you reintroduce a bug when fixing something else.

Sometimes, depending on the maintained software, it may take too long to run the whole regression-test suite after every little change you want to check-in to your source control system (which you do have, right?). In that case, it may be useful to take a subset of regression tests and make a “smoke-test” that would cover only a subset of test cases from the regression-test suite. You would run this smoke-test after every change you make, and the regression-test overnight or during weekends.

9. Adopt Existing Code Conventions Even If You Don’t Like Them

Code conventions are something people get religious about, and in general, it is hard to make someone change his mind about which coding conventions to use. Even with languages that come with suggested coding conventions from the authors (like Java, Visual Basic and C#), developers still argue about them. With languages where authors leave the conventions to developers (C, C++, Perl), the matter of choosing a convention quickly becomes a war.

I know I am not going to convince anybody, but try to stick to whatever conventions exist in the code you maintain. Even if there is no uniform convention all over the system, chances are that the conventions are uniform at least on module level, since in most cases, one person develops one module. In the worst case scenario, someone other than the original developer already altered the code and used other conventions, so you have a horrible mix of conventions to deal with. Even in that case, adopt one of the existing conventions, and don’t introduce yet another one.

Conclusion

Maintenance programming has its own set of challenges, and these challenges need to be addressed in a disciplined manner, just as with active development. In this article, I present some practices that I found useful in my work; this is by no means an attempt to write the definite guide to the art of maintenance programming.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

Comments and Discussions

Great article. I love maintenance programming. This is because that is how I learned about all the different ways code could be written when I was a pup programmer. It's especially useful to determine what NOT to do as much or more than what you SHOULD be doing. Great article.

The good thing is that I have many ideas The bad thing is that after evaluating them thoroughly, most of them turn out to be inappropriate, impractical, inelegant, overcomplicated or even silly! Normally less than 5% of the ideas, which I am researching and evaluating in my limited spare time, are good enough to justify the publication of a CodeProject article. Currently, I am still researching and rejecting ideas...

Early in my career, I tested nuclear reactor control software under some extremely strict guidelines (think CMM 5+). Once I left that position, I was aghast at how many "gurus" had no idea how to track down a bug or to prevent them in their own code. I once had an argument with a Ph.D. Computer Scientist that insisted that it's impossible to write bug-free code. Self-fulfilling prophecy perhaps?

It's always been my belief that anyone serious about software development should spend at least a year in the "humiliating" position of maintaining and testing a software base. This humility goes a long way in that person's future, and may prevent them from growing that stupid ponytail :p

I used to resent working on other people's code, until I swallowed my pride and realized they might have something to teach. I've learned some fascinating things about architecture and abstraction since adopting this approach.

I no longer take over someone else's work assuming I can do it better. But instead anticipate learning something.

To state the obvious: a software solution provides a solution to business problems. Business requirements change, and the solution needs to change in lockstep with it.

As such I would suggest a very robust way of managing requirements on an ongoing basis, and having complete traceabiliy through requirements all to way to implemented code. To put it another way, rather than thinking of you application as a finished house getting the odd lick of paint, think of it as a living organism, conforming to requirements that are thought through regularly. Fearlessly ensure that the design of the application is the best for these vigorously monitored requirements - almost like how the human body replaces all of it's cells every two years. I find it almost needless to say that test suites on all levels (unit, interaction, user acceptance etc.) form inherent part of the solution, and are kept current as well (read mor on test-driven development).

If not, you are working the solution into the ground, and will need to find another 'maintenance' position within a few years.

If done in the above way, there is very little difference between 'maintenance' and new development.

If you do find yourself in the kind of project for which these tips make sense (I did make some basic assumptions about it!), my 'tip' would be to urgently look for better work. Apart from raising an important issue, these ideas, in software engineering terms, seem very outdated (I'm talking about a decade!)

(The idea about creating a regression suite is a good one, although I have yet to come across a project where this is not in place, where it would be less work to create the test suite than rewrite. Also, if it is not in place, the culture of the company for which it was created is normally to blame, and their expectations would be such that I doubt that you would be able to motivate the cost involved.)

While I am not going to disagree with you that in ideal world things should be handled that way, I have never even heard of an organization with such a high level of planning, let alone worked for one. In real world, people do get old applications to maintain and they need to cope with problems specific to maintenance programming.

I have made some assumptions, one being that the article is aimed at 'beginner' architects rather than 'beginner' programmers, due to the fact that it is part of the 'design' section of the site. Another has been that we are probably not talking about real 'legacy' apps written in Cobol and the like as this is a website devoted to more modern technologies.If you're an architect, the responsibility to make these changes happen lies with you. If you're a programmer, my suggestion was to try and find an architect to work under that knows to make these sort of changes.I'll concede one single point, and that is that in most organisations I know of maintenance is seen as a little 'second rate'. As such they are not looking for architects telling them that change is needed. What I tried to point out though is that exactly this represents a good opportunity to use a maintenance position to move on in your career, as in reality, there is not that much difference between maintenance and green fields. By changing your viewpoint you could actually use new development guidelines on a legacy project, as from an architectural point of view there are more similarities between maintenance and green fields than differences.Architects on the maintenance app when it was originally written followed the guidelines available at that time. These have moved on, hence new apps are developed differently. But these same guidelines can be used to architect maintenance apps, because apps are more like living organisms and less like buildings, and as such can 'change their cells'. Does this make any sense?It is a little like the story of Theseus' ship, where the enitre ship has been rebuilt over the years by replacing all the wood with other pieces. On an architectural level there's no need for the architecture to end up looking the same as it has, and if you have a process that always keeps the code in synch with the design, this will end up looking different as well.From experience however I know that doing the above is not easy.

thinkpod wrote:I have made some assumptions, one being that the article is aimed at 'beginner' architects rather than 'beginner' programmers, due to the fact that it is part of the 'design' section of the site.

This article is aimed at software engineers who find themselves in a position to maintain existing applications. They are typically responsible for both design and codingthinkpod wrote:as in reality, there is not that much difference between maintenance and green fields.I beg to disagree. I have worked on both, and they are very different. With maintenance you are typically constrained not only by existing design, but also with tight budget and lack of time to do anything "big".

thinkpod wrote:It is a little like the story of Theseus' ship, where the enitre ship has been rebuilt over the years by replacing all the wood with other pieces.I don't know about you, but my employer is not willing to pay me for rebuilding any existing application piece by piece. They expect me to introduce new features and fix bugs as requested by customers. Investing money into rewriting an application from scratch was pretty common in marry 90's, but not today. Companies don't see a business value in rewriting applications that already work; "architecture" means nothing to them.

thinkpod wrote:As such they are not looking for architects telling them that change is needed. What I tried to point out though is that exactly this represents a good opportunity to use a maintenance position to move on in your career

I was mostly "architecting" in the beginning of my career. These days I worry less about architecture and more about solving real-life problems, and that attitude turned out to be very good for my career.

Good article, glad t see tht someone else out there has some common sence when it comes to maintenance.

But I think I have to disagree with some of your comments on code layout. I think consistent code layout only really exists on newer code. I deal with a lot of code that is 15 to 20 years old, and the only sensable forst step when trying work out what the code actually does is to spend time re-layout the module, (even found a few bugs this way!).

When doing this you should keep the overall structure the same, no new structures, functions, procedures etc.

I've gotten to the point where I almost feel like I don't care where the curly-braces are. If it's seemingly randomly formatted, it is tougher, but usually it's semi-consistant, and the key is to use the editor's brace-matching function to verify what I think I'm seeing.

In general I found this article to be pragmatic and reasonable. For long-term software maintenance, there are probably some additional things - for example, keep track of the number of defects fixed in each source module, and consider a rewrite when some threshold is reached.

Also, when you discover some fact or detail about the way the software works, include it in the source as a comment (so you don't forget six months from now!). This is especially true of any prerequisites, or special settings.

Finally, adopt some systematic approach to logging within the app, that you can selectively enable if you need more information. Since many apps today are client-server, the usual "breakpoint" debugging techniques may not be the most efficient at finding or identifying a problem, especially if it only shows up in production.

Hans Dietrich wrote:In general I found this article to be pragmatic and reasonable.

Thanks for your kind words, Hans.

Hans Dietrich wrote:keep track of the number of defects fixed in each source module, and consider a rewrite when some threshold is reached.

That's too rigid, IMHO. I don't think there is a strong correlation between a number of fixed bugs and the quality of code afterwards.

Hans Dietrich wrote:Also, when you discover some fact or detail about the way the software works, include it in the source as a comment (so you don't forget six months from now!).

Absolutely. I mentioned this in my article, although I didn't specified that it must be documented through a comment.

Hans Dietrich wrote:Finally, adopt some systematic approach to logging within the app, that you can selectively enable if you need more information.

That's a good one, although ideally logging should be included during active development, and used during maintenance. In a project I worked on, we logged all exceptions and that proved to be very helpful.

On the topic of logging exceptions, it can be extremely valuable to be able to analyze crashes in depth after the event. If you've never done this before then you might not appreciate the importance of it, but when software has been deployed, the users aren't necessarily interested in letting you poke around on their machine when it goes wrong, and if they aren't programmers then they probably don't have a decent debugger installed anyway.

Distribute the PDB along with the EXE, throw in a copy of dbghelp.dll, and write out a callstack and minidump for every crash (except stack overflows). Including the PDB and dbghelp.dll from whatever compiler version you're using helps enormously in generating good callstacks in optimized builds. I don't think it's required for generating minidumps, but even so, you'll want access to the right PDB when you do come to use the minidump later on, and if it's bundled in the same install program then it's that much easier.

Callstacks and minidumps are non-intrusive to add to existing programs, so it's quite practical to do in a maintenance environment, and it can be hugely valuable. More intrusively, but very valuable if you can fit it in: Arrange some automatic upload system for the logfiles. Minidumps are more problematic as they can be pretty large, but if you leave a callstack at the end of your logfile then you have a cheap indication of roughly what went wrong, and you can arrange to get a copy of the minidump later on.

Of course, the less the users know about this stuff the better. Don't ask them to try to send only the interesting bits of the logfile, or to copy the callstack out of a dialog or something like that, because they'll get scared and (worse) get it wrong! And don't tell them you're filling their hard disks with minidumps, you know where that would lead...

Nemanja Trifunovic wrote:Hans Dietrich wrote:keep track of the number of defects fixed in each source module, and consider a rewrite when some threshold is reached.

That's too rigid, IMHO. I don't think there is a strong correlation between a number of fixed bugs and the quality of code afterwards.

Actually there is a thing called Error Prone Modules--modules or chunks of code that never really worked right and are constantly being fixed. The fixes and kludges pile up until the module is incomprehensible, which leads to more bugs, etc. These really are good candidates for a rewrite. Just as the 80/20 rule applies to performance (80% of the work is done in 20% of the code) the same tends to apply to bugs.

But, as you said, it is important to resist the Not-Written-By-Me urge to rewrite code...

Test BEFORE you make a change. You wouldn't want to discover a problem that previously existed while testing your change. If the problem existed before and it wasn't caused by your change, you can't prove that the problem wasn't introduced by your change.

Brad Bruce wrote:Test BEFORE you make a change.In that case you would have duplicate tests against the same code, right? If you test after a change, and before the next one, the only thing that can change in between is the tests suite.

Brad Bruce wrote:If the problem existed before and it wasn't caused by your change, you can't prove that the problem wasn't introduced by your change

To be more specific, when you are about to correct a problem, write a quick unit-test that reproduces the problem you are supposed to fix. Run it _BEFORE_ you make changes to be sure that it does FAIL. Then make your changes and run the SAME test as before to insure it PASSES.

Now, and this is the part I REALLY want to stress. Never EVER remove any of these unit-tests. Always run them all before starting a change. That way, you know absolutely that your new changes didn't break anything.

1. Write a test case to reproduce it.2. Fix the bug.3. Include the test case in your regression-test suite.3. Run the whole regression suite and make sure all the tests pass.4. Change the bug status to "fixed"

Nemanja Trifunovic wrote:1. Write a test case to reproduce it.2. Fix the bug.3. Include the test case in your regression-test suite.3. Run the whole regression suite and make sure all the tests pass.4. Change the bug status to "fixed"

This is something I hope to accomplish in an company Standard Operating Procedure list. I got a lot of negative feedback from other developers over requesting this kind of activity. My preference in creating the test to display the bug is to show you actually, really, truly understand the bug. Sometimes the "bug" is a symptom of another issue, just writing the regression test sometimes creates a lightbulb that says, "wait, I think the real cause is a step back."

I do both new development and maintenance. I normally reverse the development concept:

new development:write it (code development)prove it (regression test)

maintenance:prove it (regression test)fix it (code development)

_________________________Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)