Monday, January 27, 2014

The business case for technical debt reduction

When I teach solution architecture classes, technical debt is always a very popular topic among practicing architects. Technical debt is a metaphor that transposes the concepts of loan and interest to IT based solutions. It respresents work that should be done in order to deliver a consistent, maintainable solution. As long as the work has not been done, the solution is in debt, which means that some stakeholders pay interest in the form of e.g. extra effort needed for simple changes, or higher support fees. Repaying the loan’s principal means doing the work needed to remove the debt: this could mean e.g refactoring software or upgrading hardware. As soon as that work has been done, the stakeholders stop paying the interest, just like when a loan has been repayed.

Architectural debt
The technical debt metaphor has been very popular in the software development world, where it refers mainly to low code quality or unnecesary complexity. Tools like SONAR now have functionality that analyses source code to measure such “implementation debt”. For architects, however, other types of technical debt may be more interesting. Aside from implementation debt, there is “architectural debt”: this is typically structural in nature, or represents a technology gap.

An example of structural architectural debt is when an architectural principle like “all applications should use the Enterprise Service Bus (ESB) to exchange data” is temporarily violated. An architect could decide to allow direct access from application A to application B’s database if A needs data of B that have not yet been exposed through the ESB, and doing it properly would mean missing an important deadline. The interest in this case is caused by reduction of control of the information flow through the application landscape, and potential errors being introduced by teams that are not aware of the shortcut. The principal is the refactoring that needs to be done later on: changing applications A and B to route the data through the ESB, and configuring the ESB. This type of debt cannot be measured in the software code of either application: it is structural in nature. The same is true of technology debt,  when a solution uses obsolete hard- or software products that cause potential failures and risks (interest) and needs to be upgraded (principal).

Business case
One of the problems many architects face is convincing their stakeholders to reduce technical debt, mainly because the debt is invisible to the end-user (see Philippe Kruchten’s categorization). Making the case for technical debt reduction in technical terms will usually not convince the business stakeholders. That requires translation into economic terms – in other words, a business case.

If one has unlimited resources (time, budget, staff), the business case for repaying technical debt is quite simple: the longer you wait, the more interest you pay, so the economic optimum is immediate repayment. The only exception is when the solution is planned to be decommissioned, and the total interest to be paid over the remaining lifespan is lower than the cost of repayment. Usually, however, resources are limited and the business case for technical debt reduction needs to compete with other solution improvements, such as new features.

In all cases, proper representation of the interest is crucial to making a compelling case. In case of structural or technology debt, it is often hard to quantify the extra costs caused by the debt. The difficulty is compounded by the inherent uncertainty: things might go wrong, but they might also go smoothly, even with the technical debt present. One often hears arguments like “we’ve run this application on this platform for 15 years, and it has never caused us any problems – so why invest in an upgrade?”.

The risk factor
The key to making the business case for technical debt reduction is to account for the risk caused by the debt. The proper way to calculate the total expected cost of uncertain failure is the well-known risk exposure formula: E(S) = p(S) x C(S), where p(S) is the probability of failure scenario S occurring, and C(S) is the cost incurred when S occurs. By summing up the risk exposure E over all possible failure scenarios S caused by the technical debt, you come as close as statistically possible to an accurate prediction of the expected cost of failure.

I recently encountered a situation where a large transportation company was running some of their core business systems on ancient mini-computers. Spare parts were very hard to get by, and the manufacturer had put severe limitations on their maintenance contract. The organization in question had a hard time making the business case for migrating the system to a modern, virtualized, blade-based solution: the cost of the old platforms was so low that the ROI for the migration looked negative. The risk of failure, however, was substantial: a single missing spare part could potentially break the company by disabling their core system for a few days. Including that risk exposure in the technical debt interest leads to a completely different business case.

Forgetting the risk factor in the business case for technical debt reduction is a common mistake, which can lead to very wrong decisions. The organization in the example above instinctively knew they had to make the upgrade anyway, but the omission of the risk in the business case did lead to unnecesary delays in the decision making process.

In short: architects arguing for technical debt reduction should make sure that they articulate the risk component in the interest. This will help them convince stakeholders to give proper priority to things they might otherwise find less interesting for being invisible to end-users.

3 comments:

  1. Good story and probably the only viable way of addressing this.

    However, it still won't be a clear cut discussion. Quantified risks often still loses from the potential revenue of a prospect willing to sign on condition that a particular feature will be added. And the problem is, the longer you postpone addressing the technical debt, the more difficult the discussion becomes.

    It really requires a good mix of opportunism and long term vision, which is not available by default.

    ReplyDelete
  2. Nice point. But a good architect should always take risk into account. Maybe its even one of the most important drivers for software architecture in general. With the implication that the introduction of technical debt can be a conscious choice upfront also. But somehow it does not feel natural for architects to feel happy with a sloppy design :-)

    ReplyDelete
  3. Technical debt from an IT operations perspective includes many consequences that business should care about: 1) unnecessary server and network capacity expenditures, 2) service degradation that users tolerate, and 3) low signal-to-noise ratio that can obscure anomalous activity (with either security or performance implications). This blog post gives examples of technical debt as expressed through DNS, LDAP, web, database, and network waste and inefficiency: http://www.extrahop.com/post/blog/paying-down-technical-debt-in-your-it-infrastructure/

    ReplyDelete