Estimates of woe

Software development is in crisis. Our projects overrun estimates on a consistent and too-frequent basis. The typical project overruns estimates by as much as forty-six percent or fifty-six percent, depending on your source studies. (Possibly more, if you find the right studies.)

The schedule overrun crisis, and its companion the budget overrun crisis, has been with us for decades.

But first, a question: If a project runs long, that is it takes longer than the initial estimate, which is incorrect? Was the project run incorrectly? Or was the estimate incorrect? (There is nothing magical about estimates. I can come up with incorrect estimates, such as driving from New York to Chicago in three hours.)

Most solutions I have seen have tacitly assumed that estimates were correct and the project management needs improvement. Managers, with their one hammer of project management, could see the problem as a project-management nail. I hear no programmers or practitioners asking for better project management. Of course, with their software development hammer, they may see the problem as a software development nail.

I see no demand for better estimating skills. I see no critical analysis of estimates and the means used to create them.

Here is a simple (and probably wrong) reason for the lack of desire to improve estimating skills: No one wants them. Programmers don't want them, because estimating is not fun (estimates are not programming, and only programming amounts to fun). Managers don't want better estimating techniques, because they don't want to lose the ability to fudge estimates to meet business goals. Let's look at estimates before continuing that thought.

Estimates for repetitive activities are easy. Housing construction, automobile assembly, and light manufacturing all have repetitive processes. (One car of a certain make and model is identical to another of the same make and model, one house in a development tract is almost identical to the house next door.) We've been building houses, cars, and consumer goods for decades, and we know how much time and resources went into previous models.

Estimates for non-repeating activities are harder. The time and skill for creating original art is hard to predict. So is the time for building a bridge or tunnel in a difficult location. (Bridges and tunnels are hard to begin with, but a difficult location adds uncertainties.)

But all estimates must have a bottom, some grounding in resources and elementary activities. For houses, you need to know about concrete, lumber, plumbing, and wiring. For cars, you need to know about sheet metal, windshields, wiring, suspension systems, ... the list goes on, but you can get to the bottom.

Software development is different. The estimates for development (at least the estimates that I have seen) have had no grounding. The project manager asks the team leads for estimates, the team leads ask developers, and developers, with no one left below them, have to provide some numbers. But they have nothing on which to base them. So they make up numbers, which are provided to the team leads. The team leads review them and make suggest changes (possibly based on their development experience and possibly based on their knowledge of a business-imposed schedule. Team leads forward their numbers to the project manager, who also reviews them based on development experience (if any) and knowledge of the business desires, adjusting them to meet the business needs.

Sometimes managers will keep asking development teams for "better" estimates, waiting until the team "gets it right". Some managers will dictate the solution to the team; others will specify a solution and then ask for an estimate. (I suspect the last is to allow the development team to "buy in" to the estimate, and to allow the manager a scapegoat should the project run over.)

So here is my theory: managers claim they want accurate estimates, but really don't. Managers want estimates that they can adjust. Accurate, grounded-in-reality estimates would mean that some projects could not be completed in time for business objectives. Estimates from the current "make the numbers fit" system allow managers to adjust estimates and promise to deliver systems on time. Delivering on time is good, but making the promise is better. Telling the business that they cannot have something is a bad role. Project managers who tell their business that the request cannot be completed on time find that they are given different assignments -- sometimes at different employers. Project managers who make the promise, and then show that they made every effort to complete the project, get to keep their jobs (usually). Many projects run over schedule and over budget, so businesses have little choice but to accept it as the norm.

As long as we have that set of incentives, we will have adjusted estimates. And as long as we have adjusted estimates, we will have overruns.