StatCounter

Watch Your Amount of Technical Debt

With the ongoing catastrophes in the financial and credit markets of late, this is a perhaps a good time to review the concept/metaphor of technical debt; particularly when it appears increasingly likely that IT budgets going into at least next year will hold steady or, in other situations, get cut back. First, some history on how this concept came about.

"...We believe this process leads to the most appropriate product in the
shortest possible time. There are pitfalls, however. Not everyone is
comfortable with sudden changes in direction, especially programmers.
Here the best antidote is more complete familiarity with the product and its
implementation. Changes will be accepted when they are recognized as
do-able.

Another, more serious pitfall is the failure to consolidate. Although
immature code may work fine and be completely acceptable to the
customer, excess quantities will make a program unmasterable, leading to
extreme specialization of programmers and finally an inflexible product.
Shipping first time code is like going into debt. A little debt speeds
development so long as it is paid back promptly with a rewrite. Objects
make the cost of this transaction tolerable. The danger occurs when the
debt is not repaid. Every minute spent on not-quite-right code counts as
interest on that debt. Entire engineering organizations can be brought to a
stand-still under the debt load of an unconsolidated implementation, object-
oriented or otherwise."

"...You have a piece of functionality that you need to add to your
system. You see two ways to do it, one is quick to do but is messy -
you are sure that it will make further changes harder in the future.
The other results in a cleaner design, but will take longer to put in
place.

Technical Debt is a wonderful metaphor developed by Ward
Cunningham to help us think about this problem. In this metaphor,
doing things the quick and dirty way sets us up with a technical debt,
which is similar to a financial debt. Like a financial debt, the
technical debt incurs interest payments, which come in the form of the
extra effort that we have to do in future development because of the
quick and dirty design choice. We can choose to continue paying the
interest, or we can pay down the principal by refactoring the quick
and dirty design into the better design. Although it costs to pay down
the principal, we gain by reduced interest payments in the future.

The metaphor also explains why it may be sensible to do the quick
and dirty approach. Just as a business incurs some debt to take
advantage of a market opportunity developers may incur technical debt
to hit an important deadline. The all too common problem is that
development organizations let their debt get out of control and spend
most of their future development effort paying crippling interest
payments."

Given the current economy and its impact - now and on-going, on IT initiatives and projects, it is vital that we keep this concept in mind as we plan, design, and execute them. Martin mentions the 'extra effort' required to refactor because of quick-and-dirty choices made initially. What that also means in some cases is increased costs and time-to-market, and neither of these are very palatable to IT execs and project sponsors who are under increasingly shrill budget and time pressures.

It is also interesting to note that this metaphor applies no matter what software development methodology is employed. If, for example, an agile project requires a number of iterations to refactor codebases without adding additional business functionality, the technical debt incurred in the previous iterations is being addressed and 'paid down' - usually with additional budget and time, but nonetheless paid off.

Is technical debt bad and to be avoided? The answer I usually give is no, as long as its: a) managed properly; and b) generally 'paid down' in a time/cost sensitive manner as projects progress to completion. All projects incur some form of technical debt as they proceed. The real problem with technical debt is when it, like subprime mortgage loans and collateralized debt obligations, becomes toxic - toxic to the business, the development teams, the IT organization, etc. Here are some forms of technical debt that can get progressively toxic over time:

Code complexity increases to the point that modifications cannot be made without introducing significant defects, and the time/cost to refactor means that the effort is essentially a complete redesign/re-write.

The code does the 80% of functionality that was 'nice to have,' but non-critical, fairly well and the 20% of critical functionality poorly.

Architecture and design activities up-front are given short-shrift, and development teams 'go their own way' in developing systems - the amount of technical debt incurred can get particularly gruesome in situations where multiple development teams are engaged to build large systems without enough architectural and design coordination between the teams.

Situations where business analysts, product owners, sponsors, or major stakeholders will not (or cannot) decide on major functionality issues, leaving the development teams to guess and proceed without clear motives or direction. This happens more often than we care to admit.

There are lots of valid reasons for incurring technical debt, and as long as the debts do not become toxic, and are 'paid down' properly, the projects incurring them should proceed well. However, given the current situation with the economy, and the reaction to it in most of our workplaces, it behooves us as IT and project management professionals to not only keep this concept front-and-center during our projects, but to deal appropriately with it so it doesn't get out of hand and make our organizations look like the AIG or Lehman Brothers of IT.

TrackBack

Comments

Thank you for this post. Current trends in technical debt do mirror those in financial debt. So many executives seem so willing to risk so much on complexities they don't care to fully understand. Thanks again, this is a very important concept to bring to light.

I like the parallel you described between technical and financial debt. I could not have understood it without your clear explanation. They are alike in ways I never thought of like having to finish off a project using technical shortcuts or something to take advantage of some market opportunity the same way one would go and borrow funds to cover some transaction that is beneficial only at that moment.