Got Technical Debt?

It’s an old concept that I remember talking about, maybe circa 2000… That sort of insidious growth of “software barnacles” that slow down development, or worse, bankrupt your company.

Am I Infected With Technical Debt?

If you think you have technical debt, but aren’t sure… Then follow this lengthy process to determine the prognosis with certainty.

Do you suspect you have Technical Debt?

If YES, then you have Technical Debt

If NO, then you likely still have Technical Debt

Funny, aren’t I? This stems from a time when I worked with a chef, and asked “Tommy, do you think this smells bad?” “Throw it out,” he said, without giving it a sniff or a second thought. In other words, the risk of being wrong wasn’t worth the cost of the “waste.” If you think you have technical debt, then simply assume you do have some technical debt, and deal with it – maybe not as brusquely say “throw it out.”

Good Technical Debt

This is best described by a true story from when I was at TogetherSoft.

It’s been a while, and my memory might be a bit fuzzy, and the actual dialog is just a guess, but this actually happened. Sales meeting at a Large Financial Firm in San Francisco, ca., 1999. Me and the sales rep.

Client says::

“If we can get Feature X, then we will buy $1,000,000 worth of your Together ControlCenter [UML Modeling / IDE]”

Jon goes back to the dev shop in Russia:

“Team, here’s a new feature we need, and I have hacked together an approach.”

“Ewww, Jon, you know there is a better way. We must change our architecture to support that.”

“Yes, but that will delay our getting this $1MM deal, and my hack will work.”

“Oh! Then let’s do the hack. Just promise me that we’ll do it right ASAP!”

“Indeed, I have already submitted an issue for the next release to immediately rip out this hack and do it the right way. In the mean time, we’ll get One Million Dollars, and we’ll get feedback on the feature.”

Now that is good technical debt! Done for a very positive reason, and not left in the code base to get worse over time. Just like some debt in business is good, so too in software. But if you leave the bad stuff in place for too long, it will turn into BAD debt!

Bad Technical Debt

This is the “classic” problem easily identified by any or all of the following symptoms:

Just look at this code, it sucks

Change in one area causes breakage in (seemingly) unrelated area

We have a lot of bugs, especially in that area of code over there

Developers know to avoid changes in certain parts of the code base

It costs a fortune to add new features

Etc.

Often, the code base starts off good enough, but somehow erodes over time. Maybe the new feature requires a bit of architectural changes, maybe a new table or two. But you find a way to “jam” it in that will “work” for now. Or maybe you don’t even realize there is a better alternative because you are either not skilled enough, or are not wise enough (grin).

In either case, this sort of “one-off” side-stepping of the proper solution, while seemingly innocent enough, can have additional, unforeseen consequences. Even if you catalogued this debt, can you really understand how it might compound in the future? Maybe the next developer comes along and builds off of your one-off, tacking on yet more debt. And then before both of those “stains” are cleaned up, two more small tweaks are built on top of the existing not-so-good code.

And so it grows, in a typically non-linear fashion. A few years later, a new member joins, points out the cruft, and your team wonders how it got so out of hand… One stain at a time, that’s how.

Don’t be that guy. Stop it before it happens.

Measuring Technical Debt

Manually, you can track technical debt as you run across it. That is, submit a “ticket” to address the technical debt, describing things like:

where it is found

what negative impacts does it have now

what negative impacts will it have in the future

cost to fix

etc.

Code Metrics

While applying metrics tools is a good start (I championed adding dozens of audits and metrics to our Together ControlCenter tool), it is lacking in one major fundamental piece: tying the metrics to the real world.

That is, just because I can tell you the horribly complex bits of your code (my favorite metric is cyclomatic complexity), that doesn’t directly translate to your technical debt. Now, if that complexity lies in a heavily-used area or a frequently changing area of your code, now you have a potential handle on “costs” associated with that complexity.

“Hey, boss, if we clean up this core module, we can get features out the door much faster. It really slows us down every time we have to wade through this muck.”

Michael Feathers has a well-liked book out there on tackling Legacy Code.

SonarQube has built an algorithm to express the technical debt lurking inside code using their plug-in.

Macro Metrics

You can start to track high-level metrics on your projects:

Feature “throughput” (of course, size matters, in terms of comparisons) to see if you are slowing down or compare against other projects.

Bug count

Consecutive days running without a major event that everyone hates

Etc.

Naturally, some of these could be normalized using number of devs, or usage statistics, or whatever makes sense for your situation and metrics. If you have lots of projects going on, try out a few different metrics to see if any trends emerge.

Of course, if only we had a “tape measure” for technical debt, life would be easy. Alas, at most, we’re just guessing at the current costs, and especially guessing at the future compounding costs.

Another idea is to measure how good or bad the code base is by providing a survey to the development team. Track this subjective measure over time.

Magical Metrics

You can point Sonar at your source. And then have it find all of your technical debt so you can start fixing it. Just kidding! Though sonar will help you tailor the metrics it gathers, and help you manage “technical debt,” it still requires the intervention of a sentient being to make the right calls.

Planning A Way Out

Much easier said than done — especially if you have spent years digging yourself into this mess.

Catalog your debt. Make it just as visible as your features and bug lists.

Determine the root causes for the different types of debt, and see if you can detect a pattern and fix it!

Hopefully, the business is helping drive the choices that are made regarding what the development team works on next. And hopefully, the development team keeps any critical-to-fix technical debt at the forefront of the conversations regarding iteration/release planning.

Of course, hope is never a good strategy, Best to come to some agreement about allotting time to tackle the priority-ranked list of technical debt items.

Prevention is the Best Remedy

Though you can catalog, prioritize, and start to eliminate your existing Technical Debt, that is all for naught unless you get a handle on preventing it in the first place!

It would be demoralizing for your team to be diligently cleaning up the Technical Debt muck, only to find new projects adding it faster than it is being removed.

Eliminate the causes of the Technical Debt. Eliminate the rewards of allowing Technical Debt. If a manager gets rewarded for getting a project done ahead of schedule regardless of the quality, guess what you just encouraged? The same thing could happen if you give $1000 for every bug fixed — you’ll get more bugs produced.

Avoiding the technical debt requires a holistic set of practices that conspire to help maintain a lean code base:

Get better at developing software in general. I mean, adopt a truly agile process that involves the business so that trade-offs between quality, schedule, and incurring technical debt can be more easily made.

Keep the design simple

Follow the domain

Use BDD to ensure you have your code protected by acceptance-level tests

Culture is Key

If your organization rewards folks for developing software that strikes the proper balance between quality and cost and schedule, then it should be easy to not let Technical Debt build up. However, if your company thrives on jumping to the next “brush fire” to excitedly stomp out; or never has the guts to tackle the technical debt identified, then either

change your own little piece of the world and do the right thing

change the organization

buy more rolls of duct tape to keep your head from exploding

or find a better job where software is treated more like a valuable business asset

What, No Silver Bullets?

At the risk of upsetting those of you who came looking here for a silver bullet to magically detect and destroy Technical Debt, you’re going to be disappointed.

It took work, inattention to detail, possibly poor craftsmanship, to build up the bad debt. It will take work, paying attention, and having professional craftsmen to pay off the debt.

I was going to ask you if posts like http://xprogramming.com/blog/2009/02/19/technical-debt-2/ by Ron Jeffries (which exemplifies the view that technical debt is better dealt with by refactoring daily rather than by asking the customer for permission to do a ‘big refactoring’story) would change the content of your ‘Planning A Way Out’ section, but then I noticed you left a comment on that post, so I guess it has not influenced your opinion much.

Hi Philip,
Ward’s and Ron’s “discussion” seemed to be about avoiding the sort of “insidious” technical debt that creeps in over time during development. I maintain that many people are happily coding away, not recognizing that they are leaving technical debt in the code/design.

Regarding the other article, I can certainly agree that you shouldn’t merely go around and clean up technical debt in the code in an ad hoc fashion — unless you truly have nothing better to do. If you are working in that code area, you should consider cleaning up some or all of the debt — depending on the severity.

My rule of thumb: if it is hard to add a feature (especially one that seems straightforward), then maybe you need to “fix” the design/code to make it easier to add the new feature. So do the clean-up/refactoring, and then add you new feature.

Great message. Admitting it is the first step. I think that identifying it could be a good second. The analysis we provide could be of great help. Have you had any experience with source code analysis and reducing technical debt? We have, though I think it is strange that so much visual/manual inspection is going on. Why do you think this is?

My name is Bhavin, and I work for Packt Publishing. We have recently published a new book titled “Instant Cucumber BDD How-to” and was wondering if you would be interested in featuring the book on your blog/website in the form of a book review.

I came across your profile through a Google search and see that you have published useful articles on the topic. Considering your expertise in Cucumber BDD, I feel you’d be one of the best persons who could provide an honest, unbiased review of the book. I’m confident that your thoughts about this book will prove to be an interesting read.

In case this subject interests you and you’d be interested in writing a review on your blog or Amazon (if a review on your blog is not possible), do let me know. I have with me limited e-copies to give and I’d be delighted to send you a complimentary e-copy of the book on your confirmation.

Please feel free to ask if you have any doubts about reviewing, timelines or otherwise, I’ll be glad to help.