Michael Portwood

Understanding Technical Debt

"Technical Debt" is a term coined by Ward Cunningham to describe the obligations incurred by a software organization when it chooses an expedient design or construction approach that increases complexity and is more costly in the long term. The term is useful for communicating with technical teams and provides a powerful metaphor for describing this concept to non-technical stakeholders.

What is Technical Debt?

While the initial definition helps, it does not describe the sources of technical debt. There are two sources:

• Unintentional (Type 1) This debt is the result of doing a poor job. For example, a design approach turns out to be error-prone or a developer produces poor code. In most cases, this debt is incurred unknowingly. For example, your company acquires a business that has accumulated significant technical debt not identified until after the acquisition. Sometimes, ironically, this debt can be created when a team stumbles in its efforts to rewrite a debt-laden system and inadvertently creates more debt.
• Intentional (Type 2) An organization makes a conscious decision to optimize for the present rather than for the future. "If we don’t get this release done on time, there won’t be a next release" is a common refrain – and often a compelling one. Example decisions would be:
  
  "We don’t have time to reconcile these two databases, so we’ll write some glue code that keeps them synchronized for now and reconcile them after we ship."
  
  "We have some code written by a contractor that doesn’t follow our coding standards. We’ll clean that up later."
  
  "We didn’t have time to write all the unit tests for the code we wrote the last 2 months of the project. We’ll write those tests after the release."

It is important to recognize that all systems carry technical debt. Our goal should be to minimize Unintentional (Type 1) technical debt by putting processes in place to identify and avoid this debt source. The impact of unintentional technical debt is similar to being shocked at the end of the month when a bill arrives with unexpectedly high charges and interest rate.

Short-Term Versus Long-Term Debt

Companies maintain both short-term and long-term debt. In financial terms, short-term debt covers situations like gaps between receivables (payments from customers) and expenses (payroll). This debt is taken on when you are owed money and just don’t have it now. It is expected to be paid off quickly. The technical equivalent seems straightforward. Short-term technical debt is the debt taken on tactically and reactively, usually as a late-stage measure to get a specific release shipped. (We’ll call this Type 2.A.)

Companies also take on debt strategically and proactively–investing in new capital equipment, like a new factory or office building. Again, the technical equivalent seems straightforward: "We don’t think we’re going to need to support a second platform for at least five years, so this release can be built assuming we’re supporting only one platform." (We’ll call this Type 2.B.)

The implication is that short-term debt should be paid off quickly, perhaps as the first part of the next release cycle, whereas long-term debt can be carried for years.

Incurring Strategic Technical Debt

When technical debt is strategically incurred, the fundamental reason is always that the development cost today is viewed as more expensive than the cost will be in the future. This can be true for any of several reasons:

• Time to Market- When time to market is critical, incurring an extra $1 in development cost might equate to preventing a $10 revenue loss. Even if the development cost for the same work rises to $5 later, incurring the $1 debt now is a sound business decision.
• Capital Preservation- When working with limited or fixed investments, every dollar counts. Deferring an expense until later can preserve capital for delivering critical features to customers. This source of technical debt can be paid off at a future date once the business conditions improve. Using this rationale is particularly important for startups or established businesses that expect or are experiencing contraction.
• Delaying Development Expense- Unlike financial debt, when a system is retired, all of its technical debt is retired with it. Once a system has been taken out of production, there is no difference between a "clean and correct" and "quick and dirty" solution. Consequently, near the end of a system’s service life, it becomes increasingly difficult to cost-justify investing in anything other than what’s most expedient.

Be Sure You Are Incurring The Right Kind of Short-Term Technical Debt

Some technical debt is taken on in large chunks: "We don’t have time to implement this the right way; just hack it in and we’ll fix it after we ship." This is a large debt that can be tracked and managed. (We’ll call this Type 2.A.1.)

Other technical debt accumulates from taking hundreds or thousands of small shortcuts– generic variable names, sparse comments, poor object decomposition, ignoring coding conventions, and so on. This is similar to credit card debt. It’s easy to incur, it adds up faster than you think, and it’s harder to track and manage after it has been incurred. (We’ll call this Type 2.A.2.)

Both of these kinds of short-term technical debt are commonly incurred in response to the directive, "Get it out the door as quickly as possible," "Get ‘er done," or "Do the needful." Short-term technical debt from small shortcuts (2.A.2) doesn’t pay off and should be avoided.

Debt Service

One of the important implications of technical debt is that it must be serviced. That is, once you incur a debt there will be interest charges in the form of rework.

If the technical debt grows large enough, eventually the company will spend more on debt servicing than it invests in increasing the value of its other assets. A common example is a legacy code base in which so much work goes into keeping a production system running (e.g. "servicing the debt") that there is little time left over to add new system capabilities. With financial debt, analysts talk about the "debt ratio," which is equal to total debt divided by total assets. Higher debt ratios are seen as more risky, which seems true for technical debt, too.

Attitudes Toward Technical Debt

Like financial debt, different companies have different philosophies about the usefulness of technical debt. Some companies avoid taking on any debt at all; others see debt as a useful tool and want to use it wisely.

Commercial teams generally have a higher tolerance for technical debt than do their technical teams. Executives want to understand the tradeoffs involved, whereas some technical teams believe that the only correct amount of technical debt is zero. This avoidance by technical teams results from failed communications with the commercial teams regarding the existence and implications of technical debt.

Most agree that it is reasonable to incur debt late in a release cycle, but commercial teams sometimes resists accounting for the time needed to pay it off on the next release cycle. The main issue seems to be that, unlike financial debt, technical debt is less visible, so people have an easier time ignoring it.

How do You Make an Organization’s Debt Load More Visible?

One way is to maintain an initiative (or project) technical debt list. Each time a technical debt is incurred, the tasks needed to pay off that debt are entered into the tracking system along with an estimated effort and schedule. The debt backlog is then tracked and any unresolved debt more than 90 days old is treated as critically past due.

This approach can be used to increase visibility into the debt load and debt service work that needs to occur within or across release cycles. It also provides a useful safeguard against accumulating the "credit card debt" of a mountain of tiny shortcuts mentioned earlier. You can simply tell the team, "If the shortcut you are considering is too minor to add to the debt-service list/product backlog, then it’s too minor to make a difference; don’t take that shortcut. We only want to take shortcuts that we can track and repair later."

Technological Relevance is not Technical Debt

Some organizations inappropriately lump costs associated with technological relevance as a kind of technical debt. The goal of budgeting effort for technological relevance is to avoid product obsolescence. These expenses generally focus on work required to assure systems work properly with future versions of operating systems, third-party libraries, programming language improvements, etc. Most product lifecycle plans cannot account for or predict third party release cycles, or appropriately account for the impacts of these externally driven changes. Consider accounting for technological relevance separately from technological debt.

Ability to Take on Debt Safely Varies

Different teams will have different technical debt credit ratings. The credit rating reflects a team’s ability to pay off technical debt after it has been incurred.

A key factor in ability to pay off technical debt is the debt level a team takes on unintentionally. That is, how much Type 1 debt. The less technical debt a team creates for itself through unintentional low-quality work, the more debt a team can safely absorb for intentional reasons.

One innovative approach is to track technical debt versus team velocity. Once a team’s velocity begins to drop as a result of servicing its technical debt, the team focuses on reducing its debt until its velocity recovers. Another approach is to track rework, and use that as a measure of how much debt a team is accumulating. This remaining debt should be managed as part of the initiative backlog with a high priority.

Retiring Debt

"Working off debt" can be motivational and good for team morale because the results are measurable and tangible. A good approach when short-term debt has been incurred is to take the next development iteration and pay it off.

The ability to pay off debt depends at least in part on the kind of software the team is developing. If a team incurs short-term debt on a web application, a new release can easily be rolled out as the team completes its debt-reduction work. However, if a team incurs short-term debt in avionics firmware— the pay off of which requires replacing a box on an airplane – that team should have a higher bar for taking on any short-term debt. This is like a minimum payment– if your minimum payment is 3% of your balance, that’s no problem. If the minimum payment were $1000 regardless of your balance, you would think hard about taking on any debt at all.

Technical Debt Taxonomy

The classification presented above is complicated and can be confusing. Here’s a summary of the kinds of technical debt. Note that the red cells should be avoided and the yellow cells should be minimized and managed.

Communicating about Technical Debt

The technical debt vocabulary provides a way to communicate with non-technical teams in an area that has traditionally suffered from opacity. Shifting the dialog from a technical to a financial vocabulary provides a clear, understandable framework for these discussions. Although the technical debt terminology is not currently in widespread use, it resonates with executives and non-technical stakeholders. It also makes sense to technical teams that are often all-too-aware of the debt load their organization is carrying.

Here are some suggestions for communicating about technical debt to stakeholders:

• Use an organization’s maintenance budget as a rough proxy for its technical debt service load.
• Discuss debt in terms of money rather than features. For example, "40% of our current development budget is going into supporting previous releases," or "We’re currently spending $2.3 million per year servicing our technical debt."
• Be sure you’re taking on the right kind of debt. Not all technical debts are equal. Some debts result from good business decisions; others are the result of sloppy technical practices or bad communication about what debt the business intends to take on. The only kinds of healthy debt are Types 2.A.1 and 2.B.
• Treat the discussion about debt as an ongoing dialog rather than a single discussion. You might need several discussions before the nuances of the metaphor fully sink in.

Find more interesting articles By Michael Portwood at www.MichaelPortwood.com