Retiring technical debt can be a super wicked problem

Last updated on December 11th, 2018 at 07:11 am

In my last post I provided a list of attributes of wicked problems [Rittel 1973], and listed the reasons why I feel that technical debt retirement projects can qualify as wicked problems. As a quick review, here are the attributes of a wicked problem as Rittel and Webber see them, rephrased for brevity:

There is no clear problem statement

There’s no way to tell when you’ve “solved” it

Solutions aren’t right/wrong, but good/bad

There’s no ultimate test of a solution

You can’t learn by trial-and-error

There’s no way to describe the set of possible solutions

Every problem is unique

Every problem can be seen as a symptom of another problem

How you explain the problem determines what solutions you investigate

The planner (or designer) is accountable for the consequences of trying a solution

A subset of wicked problems can be viewed as super wicked [Levin 2012]. Levin, et al. list the following four properties of super wicked problems. With each one, I’ve added my thoughts about how retiring technical debt can qualify as a super wicked problem.

Time is running out

Two pilots line up their F/A-22 Raptor aircraft behind a KC-10 Extender to refuel while en route to Hill Air Force Base, Utah. When Hurricane Michael made landfall just after noon local time on October 10, 2018, it swept almost directly over Tyndall Air Force Base. Tyndall is the base for the U.S. Air Force 325th Fighter Wing, which has primary responsibility for air dominance training for pilots of F-22s, and maintenance personnel and air battle managers. 55 F-22s are based at Tyndall. When Hurricane Michael was approaching, 33 of the F-22s were repositioned to Wright-Patterson Air Force Base near Dayton, Ohio [Phillips 2018a]. According to U.S. Senator Bill Nelson (D-Florida), the aircraft that were left behind were undergoing maintenance and were not in condition to be flown [Gabriel 2018]. Some of these aircraft are known to have been damaged in the storm. At this writing, the extent of any damage hasn’t been disclosed, but it’s believed that the aircraft are repairable.Given the reality of anthropogenic climate change [Cook 2016], further increases in storm intensity and frequency are likely, and air bases located in vulnerable coastal regions are at risk [Phillips 2018b]. Because these bases were constructed during the period before the general recognition of anthropogenic climate change, they now constitute a technical debt. Relocating them would undoubtedly be a wicked problem, and possibly a super wicked problem. However, if federal policies continue to fail to account for anthropogenic climate change, those policies could prevent relocating these bases. If that happens, the policies themselves represent a technical debt. Changing federal policy to account for anthropogenic climate change is certainly a super wicked problem. U.S. Air Force photo by TSgt Ben Bloker, courtesy Wikipedia

The problem presents difficulty that has an inherent timescale. For example, many believe that climate change will become irreversible if it isn’t dealt with successfully within 30 years.

Technical debt retirement can have an inherent time scale, though it varies with the debt in question. For example, Microsoft ended mainstream support for Windows 7 in January 2015. At that point, by any definition, every computer running Windows 7 incurred a technical debt. Yet by September 2018, almost four years later, 46.7% of all computers running Windows were running Windows 7, and that number represented an increase from the previous month by 0.6% [Keizer 2018]. The safe-operability window for Windows 7 is closing fast. Well, to be honest, no faster than the calendar. But the end of standard support is January 2020, which represents the inherent timescale for this kind of technical debt. The consequences are much more serious than just converting to Windows 10. All applications running on those machines are also potentially reaching end of life, unless they operate correctly on Windows 10. And all users of converted machines will need to learn how to use the new Windows 10 OS and any applications that must be updated or replaced. So there’s also a training issue, a learning curve, and period of elevated user error rates to deal with, too.

Other technical debts might also have explicitly inherent timescales, of course. But for enterprises whose securities are publicly traded, all technical debt has a shared inherent timescale associated with the cost of retiring it. For any technical debt that must eventually be retired, if the cost of retirement is high enough to be noticed as a decline in net income in enterprise financial reports, the impact of debt retirement can be severe and negative. Letting debt remain in place, unaddressed and growing, can be a financially dangerous strategy. A safer strategy is to spread the cost of retirement across as many fiscal quarters as possible. For Windows 10 conversions, there are only about five fiscal quarters remaining for that purpose. For other technical debts, the number of fiscal quarters available for diluting the costs of retirement might be more—or less.

Those who cause the problem also seek to provide a solution

The phrase “seek to provide a solution” might be somewhat tactful. I expect that some super wicked problems have the property that those who cause the problem exert some degree of control over what kinds of solutions are acceptable, or even discussible. In many cases, this represents a conflict of interest that can prevent the organization from deploying the more effective options.

That conflict of interest is certainly present in the context of technical debt retirement projects. Technical debt formation and persistence are due, in part, to a failure to commit resources to retiring it, or, at least, to inhibiting its formation. That failure is the responsibility of those in leadership roles in the enterprise. Typically, these are the same people who must decide to commit resources to retire technical debt in the future.

The central authority needed to address it is weak or non-existent

Again, I find this description unnecessarily limiting. I would prefer a phrasing such as, “The central authority, for whatever reason, chooses to exert, or is unable to exert, its authority in furtherance of solution, or even investigation.” In other words, the central authority need not be weak for it to be a source of difficulty in addressing the super wicked problem. It need only choose not to act. This can happen when those who cause the problem are the people who constitute the central authority, or they capture the central authority, or they capture the function to which the central authority has delegated responsibility for solution.

With respect to technical debt retirement, consider this scenario. At AMUFC, A Made-Up Fictitious Corporation, the sales and marketing functions have repeatedly struggled with the engineering function for shares of budget resources. Engineering has argued repeatedly, and unsuccessfully, that it needs additional resources to address the technical debt that has accumulated in several products. But the CEO is a former VP Sales, and a close friend of the CFO. Together, they have always decided to defer technical debt retirement in favor of new products and enhancements favored by the VP Marketing, by customers, and by investors.

Scenarios like this are common. Enterprise leadership is strong, but not inclined to address the technical debt retirement issue.

Partly as a result, policy responses discount the future irrationally

Irrational discounting of future costs and benefits occurs when policies are deployed that give too much emphasis to producing short-term benefits and/or to avoiding short-term costs or inconveniences. Benefits are pulled in from the future towards the present; costs and inconveniences are pushed out toward the future and deferred. One form of this discounting scheme—one of many—is hyperbolic discounting.

This tendency is one way of distracting attention from the actual problem. It is the principal tactic that enables the persistence of technical debt, and the means by which enterprises repeatedly defer attention to the problem of retiring technical debt.

Both the problem of managing technical debt and the problem of designing technical debt retirement projects, exhibit all of these properties to some degree. It’s likely, in my view, that these problems are super wicked problems.

Intervention strategies for super wicked problems

Levin, et al., recommend four distinct strategies for resolving super wicked problems. They are all approaches to devising policies that are difficult to alter, thus committing the organization to a particular path forward.

Exploit lock-in

Lock-in is usually regarded as dysfunctional adherence to a strategy or course of action despite the existence of superior alternatives [Brenner 2011]. It occurs when a policy confers some kind of immediate benefit on a subset of the population. If that benefit is significant, if the population subset would be harmed by alterations of the policy that remove the benefit, and if the subset has enough political power to defend the benefit, the policy will be “locked in” and thus difficult to change. Levin, et al., suggest that this phenomenon can serve a beneficial purpose by protecting a constructive policy, thus preventing its abandonment.

Most technical debt retirement efforts are explicitly designed to focus solely on retiring the debt, with all (or most) of the benefit appearing in the form of increased engineering productivity, decreased sources of frustration for engineers, or increased engineering agility. Benefits for non-engineering stakeholders tend to be indirect. To establish policies that exploit lock-in we must craft them so that they provide ongoing, direct benefit to the most politically powerful stakeholders. For example, addressing first the forms of technical debt that are most likely to lead to product innovations that non-engineering stakeholders would value highly could cause those stakeholders to favor further technical debt retirement efforts.

Positive feedback

This strategy makes policies durable when people or organizations already supporting the policy derive some kind of increased benefit, leading to others not yet supporting or covered by the policy to decide to support it. This mechanism is sometimes known as a “network effect.” When network effects are present, the value of a product or service increases as the size of the population using it increases [Shapiro 1998].

To exploit network effects when devising technical debt retirement efforts, focus on retiring the kinds of technical debts that confer benefits on stakeholders of platform assets. A platform asset is an asset that supports multiple other assets. Examples: an application development tool suite, a product line architecture, or an enterprise data network. Platform assets that support collaboration communities are more likely to generate network effects.

Increasing Returns

Policies and interventions that enable increasing returns to the population are more likely to be durable than those that offer steady returns. Because people adapt to steady levels of stimuli, policies that produce a change in the context only during the period immediately following initial adoption of those policies are less likely to maintain popular support than are policies that continue to provide increasing returns as long as they’re in place.

But among policies that provide increasing returns, Levin, et al., identify two types. Type I policies, which are less durable, confer their benefits on an existing population of supporters. Type I policies don’t cause others to become supporters. Type II policies also confer benefits on their supporters, but they do cause others to become supporters. Type II policies are thus far more durable than are Type I, because they foster growth in the supporting population.

Framing technical debt retirement projects as individual projects with the objective of retiring a specified kind of technical debt is likely to lead to the enterprise population viewing the effort as a Type I policy at best. But framing each project as a phase of a longer-term effort could position the larger effort as a Type II policy, if the larger effort is designed to affect increasing portions of the enterprise population.

Self-reinforcing

A self-reinforcing policy is a policy that creates a dynamic that causes that policy to be maintained. Reinforcement can come about either as a result of increases in the benefits the policy generates, or by causing increases in the cost of rescinding the policy, or a combination of both effects. As with the strategy of Increasing Returns, there are two types of self-reinforcing policies. Type I self-reinforcing policies focus on maintaining support for the policy within the subset of the population consisting of its original supporters. In analogy with Type II Increasing Returns policies, Type II Self-reinforcing policies affect both the original supporting population and portions of the population not yet affected directly by the policy.

To exploit self-reinforcement, technical debt retirement programs must emphasize retiring debts that have curtailed organizational agility in recognizable ways, or which have prevented introduction of capabilities that the population values. Communicating these objectives is an important part of the program, because self-reinforcing popular support is possible only if the population understands the strategy and how it benefits the enterprise.

Because of the essential uniqueness of any wicked problem (Proposition 7 of Rittel and Webber), it is futile to attempt to apply as a template any retirement program that worked for some other organization, or for some other portion of a given organization at an earlier time with a different form of technical debt. But these four strategies, implemented carefully and communicated widely and effectively within the organization, can build organizational commitment to a long-term technical debt retirement program, even though retiring technical debt may be a super wicked problem.

Contact me on social media:

Reference posts

A collection of definitions of terms as we use them in this blog, with links to longer discussions of each term. Along with each definition is a link to a post that discusses that term in more detail. Read more…

Welcome to Technical Debt for Policymakers. What you’ll find here are resources, insights, and conversations of interest to policymakers who are concerned with managing technical debt within their organizations. Read more…

When retiring one kind of technical debt from an asset, auxiliary technical debt is any other kind of technical debt in the asset. It can be tempting to try to retire auxiliary technical debt too. Sometimes that’s wise, but it can lead to scope creep. Rules of engagement can control this temptation. [More]

To retire technical debt, we need to know where it is. And if service disruptions are necessary, we need to know who will be affected. Here’s a survey of some of the issues, and suggestions for resolving them. [More]

For some assets, we can’t allow debt to persist, and we can’t afford replacements. We must retire the debt. This post begins exploring what it takes to design projects to retire technical debt in irreplaceable assets. [More]

By carefully observing what happens when we actually try to retire some kinds of technical debt, we can better understand the degree of the degree of wickedness of the effort. That understanding helps manage risk in technical debt retirement projects, reducing costs and speeding execution. [More]