Technical debt is similar to new features

If we accept the premise that technical debt is the difference between what we want the code to be and what the code actually is, we can treat technical debt like new requirements.

Most development shops have a process for new requirements. That process usually involves describing the new feature, estimating the benefits (additional revenue, reduced expenses), estimating the effort, scheduling the work, testing, and deploying the new requirements in a new version.

An organization can use a similar process for technical debt. Technical debt must be described ("module X fails to meet code style rules 12, 14, and 17" or "library Y is obsolete"), estimated for benefits, estimated for effort, scheduled, tested, and deployed.

Technical debt does vary from new requirements in two ways. First, new features are often about revenue or cost reduction, and technical debt is about the reduction of risk. This difference often makes new requirements more desirable; revenue is often better than less risk, especially for a working system.

The second difference is the sponsors for changes. New features often have organizational sponsors. They may have titles such as "product owner", "VP of marketing", or "user representative". These sponsors make the case for the new feature, presenting the benefits to the managers who control the project budget.

In contrast, the sponsors for technical debt are (almost always) developers. It is the developers who see the problems in the code, the developers who want to improve things.

But "improving the code" is not a goal that makes sense for the business. Managers are concerned with four general topics: money, time, risk, and politics. "Improving the code" is not one of them. To get managers to "buy in" to the effort to reduce technical debt, developers must make the case in management terms. They have to convert "improvement to the code" into one of money, time, risk, and politics.

A change to eliminate technical debt may, on occasion, offer a reduction in cost, but usually the benefit is a reduction in risk. To get time and resources to reduce technical debt, developers must present a clear case about the risk caused by the current code, and possible costs (money, time, delayed releases) in the future.

If developers can make their case, then the process for technical debt is quite similar to the process for a new feature. Changes for technical debt can be identified, described, tracked, prioritized, estimated, scheduled, implemented, and deployed.

Technical debt is not a mysterious aspect of software development. It can be described and measured, and the effort to reduce it can be estimated, prioritized, scheduled, and implemented -- just like the efforts for  new features. The big difference between technical debt and new features is that technical debt is about risk, and new features are about money. Any organization must balance the needs of money, time, risk, and politics -- that's management.

Technical debt is what we want it to be

We like to think that technical debt is an attribute of software.

Wikipedia provides a definition: A concept in software development that reflects the implied cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer.

I believe that technical debt is not derived from software, but is instead derived from our expectation of the software.

Or more specifically, technical debt is code that violates a coding style or architectural guideline.

If we accept this definition of technical debt, something interesting happens.

Technical debt becomes not an aspect of software, but an aspect of our desire for the software.

Moreover, it can change over time.

Consider a hypothetical project with code written in Python. There is a substantial code base, and there are no rules for code style. The development team is unaware of the generally accepted rules for Python code, nor do they use pycodestyle.

The code performs as expected with no defects. (I did say that this was a hypothetical project.)

Yet the code itself is a jumbled mess of classes and functions, with poor names for variables and lines that exceed Python's recommend 80 characters. The development team is okay with the state of the code. (After all the code does run and it does produce the correct results.)

The code in the project has no technical debt. There is nothing in the code that developers or managers want to change.

Now we change the scene: The team becomes aware of code style rules and the pycodestyle package. They discuss the idea of code standards and agree to use the pycodestyle package to review their code.

Now the code has technical debt.

We did not change the code. The code is exactly the same as before the introduction of code style rules.

Yet now it has technical debt.

Not because the code changed. Because the expectations of the developers changed. Simply changing the rules made technical debt appear.

Technical debt does not follow the rules of accounting. One does not use double-entry bookkeeping to track it. There is no "law of conservation of technical debt".

Technical debt is what we say it is. Or more precisely, technical debt is what we want the code to be that it is not. The key phrase is "what we want".

Technical debt is not an absolute attribute of the code. It depends on the expectation of the developers.

Tech debt considered possibly not harmful

Current wisdom holds that tech debt (poorly-implemented programs or sections of programs) is bad, and should be avoided. Much as been said about "clean code" and keeping the code in a good state of repair at all times. The Agile Development methodology recognizes the need for refactoring, to reduce tech debt.

Everyone agrees that good code is good (for the project and for the company) and bad code is bad (also, for the project and the company).

Except possibly me.

Which is to say, I am not convinced that every project should take steps to avoid or reduce tech debt. I am of the opinion that some projects should avoid or reduce tech debt, and other projects should not.

Which projects should avoid tech debt -- and which projects should allow it -- is an interesting question, and not always easy to answer. But the answers lie within another question: why is tech debt bad?

Tech debt is bad, we all agree (including myself), in that it increases the development cost. The forms that tech debt takes -- poorly written programs, older programming languages or programs that depend on old versions of interpreters or compilers -- slows the development of new features and fixes to existing features. Tech debt also makes for a brittle code base, such that a small change in one section can have large effects throughout the entire system. Thus, even the smallest of changes must be carefully analyzed, carefully designed, carefully implemented, carefully tested, carefully reviewed, and carefully tested again. Each of those "carefully" operations requires time and effort.

But preventing or fixing tech debt also has a cost. It diverts development resources from adding new features into fixing old code. That diversion can delay the implementation of new features (if you keep the size of the development team constant) or increase the cost of the development team (if you add members).

The decision to reduce tech debt depends on one thing, and one thing only: the value that the organization places on the software. And the value of software, while it can be calculated with the rules of accounting for capital expenditures and depreciation, is really dependent on how you use the software.

Any code base can have value from the following uses:

• Using the application (that is, running it) for company business
• Taking pieces of the code for use in other applications
• Copying design of the code for use in other applications (in a different language)
• Selling the code to another organization

If you are actively using the software (and most likely maintaining it with small fixes and possibly large enhancements), then the effects of tech debt will drive up the development costs, and tech debt should be evaluated and, when reasonable, reduced.

If you are not using the software, but intend to use pieces of the software in other systems, then the cost of tech debt must be discounted. Only the pieces that will be transferred should be considered. The remaining pieces, which will be discarded, have no intrinsic value and you should not fix their tech debt.

A different calculation applies for the transfer of not code but design. Transferring a poor design from one system to another is maintaining a poor design. But it may be more effective to fix the tech debt on the receiving end, rather than in the source system.

If you are selling the code, and this is a one-time event, then I see little incentive to improve the code. Odds are that the purchaser will not evaluate the quality of the code, or provide a higher purchase price for the improved code.

If you sell code often -- and perhaps are in the business of selling code -- then your code is your product, and you should look to remove tech debt from your code. Your code is your offering to your customers, and your reputation is built on it.

The one scenario that I did not list is the decommissioning of software. If your software has a short life (and you must define "short" as it varies from organization to organization) with no use after that life, then any investment will have a limited time for a return. We don't fix cars that are about the be hauled off to the junkyard, and we shouldn't fix software that we are about to discard.

The decision to avoid or reduce tech debt depends on the future use of the software. For some systems, this is easy: long-lived code such as the Linux kernel, or Microsoft Word, or an accounting system, all benefit from reduced tech debt. Other, short-lived code (such as a short script that is discarded at the end of the day) gain little from refactoring and improvements.

The difficult part in this is determining the future of the software. But once you know that, you know how much effort you should put into the removal of tech debt.

Technical debt is a bet

A long time ago, I worked in the IT department of a small, regional bank with 14 or so branch offices.

The IT team was proud of their mainframe-based online teller network. All teller transactions were cleared through the system and it prevented the fraud of someone making withdrawals at different branches, withdrawals that would exceed the account balance. (We might think little of such a system today, but in that time it was an impressive system.)

But the system had technical debt. It was written in IBM's assembler language, and it was extremely difficult to change. At the core of the system was the branch table. Not "branch" as in "jump instruction", but "branch" as in "branch office". The table allowed for 20 branch offices, and no more.

Lots of code was built around the branch table, and that code had built-in dependencies on the size of the table. In other words, the entire system "knew" that the size of the branch table was 20.

Things were okay as long as the bank had 20 (or fewer) branches. Which they did.

Until the president retired, and a new president took the helm. The new president wanted to expand the bank, and he did, acquiring a few branch offices from other banks.

The IT team started working on the expansion of the branch table. It wasn't an immediate problem, but they knew that the limit would be exceeded. They had to expand the table.

After months of analysis, coding, and tests, the IT team came to a difficult realization: they were unable to expand the branch table. The director of data processing had to inform the president. (I imaging the meeting was not pleasant.)

Technical debt exists in your systems, but it is a bet against you competitors.

It doesn't matter if the debt is your reliance on an out-of-date compiler, an old version of an operating system, or lot of messy source code.

Each of these is a form of technical debt, and each of these is a drag on agility. It slows your ability to respond to changes in the market, changes in technology, and competition. Yet in the end, it is only the competition that matters.

Does the technical debt of your existing system -- the hard-to-read code, the magic build machine, the inconsistent database schema -- slow you in responding to the competition?

It doesn't have to be a new product from the competition. It could be something that affects the entire market, such as legislation, to which you and your competition must respond. Your technical debt may delay that response. Does your competition have similar technical debt, such that their response will also be delayed? Are you sure?

That's the risk of technical debt.

Measuring technical debt is not enough

I've been working on the issue of technical debt. Technical debt is a big problem, one that affects many projects. (In short, technical debt is the result of short-cuts, quick fixes, and poor design. It leads to code that is complicated, difficult to understand, and risky to change.) Technical debt can exist within source code, a database schema, an interface specification, or any other aspect of a technical product. It can be large or small; it tends to start small and grow over time.

Technical debt is usually easy to identify. Typical indicators are:

• Poorly named variables, functions or classes
• Poorly formatted code is another
• Duplicate code
• An excessive use of 'true' and 'false' constants in code
• Functions with multiple 'return' statements
• Functions with any number of 'continue' statements
• A flat, wide class hierarchy
• A tall, narrow class hierarchy
• Cyclic dependencies among classes

Most of these indicators can be measured -- with automatic means. (The meaningfulness of names cannot.) Measuring these indicators gives you an estimate of your technical debt. Measuring them over time gives you an estimate of the trajectory of technical debt -- how fast it is growing.

But measurements are not enough. What it comes down to is this: I can quantify the technical debt, but not the benefits of fixing it. And that's a problem.

It's a problem because very few people find such information appealing. I'm describing (and quantifying) a problem, and not quantifying (or even describing) the benefits of the solution. Who would buy anything with that kind of a sales pitch?

What we need is a measure of the cost of technical debt. We need to measure the effect of technical debt on our projects. Does it increase the time needed to add new features? Does it increase the risk of defects? Does it drive developers to other projects (thus increasing staff turnover)?

Intuitively, I know (or at least I believe) that we want to reduce technical debt. We want our code to be "clean".

But measuring the effect of technical debt is hard.