Instead of simplicity, measure complexity

The IEEE Computer Society devoted their November magazine issue to "Simplicity in IT". Simplicity is a desirable trait, but I have found that one cannot measure it. Instead, one must measure its opposite: complexity.

Some qualities cannot be measured. I learned this lesson as a sysadmin, managing disk space for multiple users and groups. We had large but finite disk resources (resources are always finite), shared by different teams. Despite the large disk resources, the combined usage of the teams exceeded our resources -- in other words, we "ran out of free space". My job was to figure out "where the space had gone".

I quickly learned that the goal of "where the space had gone" was the wrong one. It is impossible to measure, because space doesn't "go" anywhere. I substituted new metrics: who is using space, and how much, and how does that compare to their usage last week? These were possible to measure, and more useful. A developer who uses more than four times the next developer, and more than ten times the average developer is (probably) working inefficiently.

The metric "disk space used by developer" is measurable. The metric "change in usage from last week" is also measurable. In contrast, the metric "where did the unallocated space go" is not.

The measure of simplicity is similar. Instead of measuring simplicity, measure the opposite: complexity. Instead of asking "why is our application (or code, or UI, or database schema) not simple?", ask instead "where is the complexity?"

Complexity in source code can be easily measured. There are a number of commercial tools, a number of open source tools, and I have written a few tools for my own use. Anyone who wants to measure the complexity of their system has tools available to them.

Measuring the change in complexity (such as the change from one week to the next) involves taking measurements at one time and storing them, then taking measurements at a later time and comparing them against the earlier measurements. That is a little more complex that merely taking measurements, but not much more complicated.

Identifying the complex areas of your system give you an indicator. It shows you the sections of your system that you must change to achieve simplicity. That work may be easy, or may be difficult; a measure of complexity merely points to the problem areas.

* * * *

When I measure code, I measure the following:

• Lines of code
• Source lines of code (non-comments)
• Cyclomatic complexity
• Boolean constants
• Number of directly referenced classes
• Number of indirectly referenced classes
• Number of directly dependent classes
• Number of indirectly dependent classes
• Class interface complexity (a count of member variables and public functions)

I find that these metrics let me quickly identify the "problem classes" -- the classes that cause the most defects. I can work on those classes and simplify the system.

Measuring technical debt is not enough

I've been working on the issue of technical debt. Technical debt is a big problem, one that affects many projects. (In short, technical debt is the result of short-cuts, quick fixes, and poor design. It leads to code that is complicated, difficult to understand, and risky to change.) Technical debt can exist within source code, a database schema, an interface specification, or any other aspect of a technical product. It can be large or small; it tends to start small and grow over time.

Technical debt is usually easy to identify. Typical indicators are:

• Poorly named variables, functions or classes
• Poorly formatted code is another
• Duplicate code
• An excessive use of 'true' and 'false' constants in code
• Functions with multiple 'return' statements
• Functions with any number of 'continue' statements
• A flat, wide class hierarchy
• A tall, narrow class hierarchy
• Cyclic dependencies among classes

Most of these indicators can be measured -- with automatic means. (The meaningfulness of names cannot.) Measuring these indicators gives you an estimate of your technical debt. Measuring them over time gives you an estimate of the trajectory of technical debt -- how fast it is growing.

But measurements are not enough. What it comes down to is this: I can quantify the technical debt, but not the benefits of fixing it. And that's a problem.

It's a problem because very few people find such information appealing. I'm describing (and quantifying) a problem, and not quantifying (or even describing) the benefits of the solution. Who would buy anything with that kind of a sales pitch?

What we need is a measure of the cost of technical debt. We need to measure the effect of technical debt on our projects. Does it increase the time needed to add new features? Does it increase the risk of defects? Does it drive developers to other projects (thus increasing staff turnover)?

Intuitively, I know (or at least I believe) that we want to reduce technical debt. We want our code to be "clean".

But measuring the effect of technical debt is hard.

Measure code complexity

We measure many things on development projects, from the cost to the time to user satisfaction. Yet we do not measure the complexity of our code.

One might find this surprising. After all, complexity of code is closely tied to quality (or so I like to believe) and also an indication of future effort (simple code is easier to change than complicated code).

The problem is not in the measurement of complexity. We have numerous techniques and tools, spanning the range from "lines of code" to function points. There are commercial tools and open source tools that measure complexity.

No, the problem is not in techniques or tools.

It is a matter of will. We don't measure complexity because, in short, we don't want to.

I can think of a few reasons that discourage the measurement of source code complexity.

- The measurement of complexity is a negative one. That is, more complexity is worse. A result of 170 is better than a result of 270, and this inverted scale is awkward. We are trained to like positive measurements, like baseball scores. (Perhaps the golf enthusiasts would see more interest if they changed their scoring system.)

- There is no direct way to connect complexity to cost. While we understand that a complicated code base is harder to maintain that a simple one, we have no way of converting that extra complexity into dollars. If we reduce our complexity from 270 to 170 (or 37 percent), do we reduce the cost of development by the same percentage? Why or why not? (I suspect that there is a lot to be learned in this area. Perhaps several Masters theses can be derived from it.)

- Not knowing the complexity shifts risk from managers to developers. In organizations with antagonistic relations between managers and developers, a willful ignorance of code complexity pushes risk onto developers. Estimates, if made by managers, will ignore complexity. Estimates made by developers may be optimistic (or pessimistic) but may be adjusted by managers. In either case, schedule delays will be the fault of the developer, not the manager.

- Developers (in shops with poor management relations) may avoid the use of any metrics, fearing that they will be used for performance evaluations.

Looking forward, I can see a time when we do measure code complexity.

- A company considering the acquisition of software (including the source code), may want an unbiased opinion of the code. They may not completely trust the seller (who is biased towards the sale) and they may not trust their own people (who may be biased against 'outside' software).

- A project team may want to identify complex areas of their code, to identify high-risk areas.

- A development team may wish to estimate the effort for maintaining code, and may include the complexity as a factor in that effort.

The tools are available.

I believe that we will, eventually, consider complexity analysis a regular part of software development. Perhaps it will start small, like the adoption of version control and automated testing. Both of those techniques were at one time considered new and unproven. Today, they are considered 'best practices'.

Airships and software

Airships (that is, dirigibles, blimps, and balloons) and software have more in common than one might think. Yet we think of them as two very different things, and we even think about thinking about them in different ways.

Both airships and software must be engineered, and the designs must account for various trade-offs. For airships, one must consider the weight of the materials, the shape, and the size. A larger airship weighs more, yet has more buoyancy and can carry more cargo. Yet a larger airship is affected more by wind and is less maneuverable. Lighter materials tend to be less durable than heavy ones; the trade-off is long-term cost against short-term performance.

The design of software has trade-offs: some designs are cheaper to construct in the short term yet more expensive to maintain in the long term. Some programming languages allow for the better construction of a system -- and others even require it. Comparing C++ to Java, one can see that Java encourages better designs up front, while C++ merely allows for them.

I have observed a number of shops and a number of projects. Most (if not all) have given little thought to the programming language. The typical project picks a programming language based on the current knowledge of the people present on the team -- or possibly the latest "fad" language.

Selecting the programming language is important. More important than current knowledge or the current fad. I admit that learning a new language has a cost. Yet picking a language based on nothing more than team's current knowledge seems a poor way to run a project.

The effect does not stop at languages. We (as an industry) tend to use what we know for many aspects of projects: programming languages, database, front-end design, and even project management. If we (as an organization) have been using the waterfall process, we tend to keep using it. If we (as an organization) have been using an SQL database, we tend to keep using it.

Using "what we know" makes some sense. It is a reasonable course of action -- in some situations. But there comes a time when "the way we have always done it" does not work. There comes a time when new technologies are more cost-effective. Yet sticking with "what we know" means we have no experience with "the new stuff".

If we have no experience with the new technologies, how do we know when they are cost-effective?

We have (as an industry) been using relative measures of technologies. We don't know the absolute cost of the technologies for software. (We do know the absolute cost of technologies for airships -- an airship needs so many yards of material and weighs so much. It carries so much. It has so much resistance and so much wind shear.)

Actually, the problem is worse than relative measures. We have no measures at all (for many projects) and we rely on our gut feelings. A project manager picks the language and database and project process based on his feelings. There are no metrics!

I'm not sure why we treat software projects so differently from other engineering projects. I strongly believe that it cannot continue. The cost of picking the wrong language, the wrong database, the wrong management process will increase over time.

We had better start measuring things. The sooner we do, we can learn the right things to measure.