Just how smart is your process?

Does your organization have a process? Specifically, does it have a process for the development of software?

The American mindset is one of process over skill. Define a good process and you don't need talented (that is, expensive) people. Instead of creative people, you can staff your teams with non-creative (that is, low wage) employees, and still get the same results. Or so the thinking goes.

The trend goes back to the scientific management movement of the early twentieth century.

For some tasks, the de-skilling of the workforce may make sense. Jobs that consist of repeated, well-defined steps, jobs with no unexpected factors, jobs that require no thought or creativity, can be handled by a process.

The creation of software is generally unrepeatable, has poorly-defined steps, has unexpected factors and events, and requires a great deal of thought. Yet many organizations (especially large organizations) attempt to define processes to make software development repeatable and predictable.

These organizations confuse software development with the project of software development. While the act of software development is unpredictable, a project for software development can be fit into a process. The project management tasks (status reports, personnel assignment, skills assessment, cost calculations, etc.) can be made routine. You most likely want them routine and standardized, to allow for meaningful comparison of one project to another.

Yet the core aspect of software development remains creative, and you cannot create a process for creative acts. (Well, you can create a process, and inflict it upon your people, but you will have little success with it.) Programming is an art more than science, and by definition an art is something outside of the realm of repeated processes.

Some organizations define a process that uses very specific requirements or design documents, removing all ambiguity and leaving the programming to low-skilled individuals. While this method appears to solve the "programming is an art" problem, it merely shifts the creative aspect to another group of individuals. This group (usually the "architects", "chief engineers", "tech leads", or "analysts") are doing the actual programming. (Perhaps not programming in FORTRAN or C#, but programming in English.) Shifting the creative work away from the coders introduces several problems, including the risk of poor run-time performance and the risk of specifying impossible solutions. Coders, the folks who wrangle the compiler, have the advantage of knowing that their solutions will either work or not work -- the computer tells them so. Architects and analysts who "program in English" have no such accurate and absolute feedback.

Successful management of software development consists not of reducing every task to a well-defined, repeatable set of steps, but of dividing tasks into the "repeatable" and "creative" groups, and managing both groups. For the repeatable tasks, use tools and techniques to automate the tasks and make them as friction-free as possible. For the creative tasks, provide well-defined goals and allow your teams to work on imaginative solutions.