AI and programmer productivity

In the sixty years of IT, we have seen a number of productivity tools and techniques. Now we're looking at "artificial intelligence" as a way to improve the productivity of programmers. Google, Microsoft, and others are pushing their AI tools upon developers.

Will AI really work? Will it improve programmer productivity? It's not their first attempt to improve the productivity of programmers. Let's look at the history of programming and some of the ways we have improved (or attempted to improve) productivity.

Start with hardware: The first electronic computers were "programmed" by wiring. That is, the hardware was built to perform specific calculations. When you wanted a different calculation, you had to rewire the computer. This was the first form of programming. It's not really an improvement, but we have to start somewhere. Why not at the beginning?

Plug boards: The first productivity improvement was the "plug board". It was a physical board that was plugged into the computer, and it held the wiring for the specific problem. To change a computer's program, one could easily remove one plug board and install a different one. The computer became a general calculation device and the plug boards held the specific calculations - one could call then "programs".

Programs in memory: The next advance was changing the program from wiring (or plug boards) into values stored in the computer. A program consisting of numeric codes could be loaded into memory and then executed. No more plug boards! Programs could be loaded via switches on the computer's front panel, or from prepared paper tapes or punch cards.

Assemblers: But creating the long lists of numbers was tedious. Each operation required its own numeric code, such as 1 to add a number and 5 to store a number to memory. Programmers had to first decide on the sequence of operations and then convert those operations to numeric values. To help programmers (to improve performance) we invented the assembler. The assembler was a program that converted text op-codes into the numeric values that are executed by the computer. (The assembler was also a program that created another program!) Each computer model had its own set of numeric codes and its own assembler.

The first assemblers converted text operation codes to the proper numeric values. But programs are more than just operation codes. Many operations need additional information, such as a numeric constant (for the operation of "add 1 to the accumulator") or a memory address (for the operation "store the value in the accumulator to memory location 1008). It made sense to use names instead of the raw values, so we could write "store accumulator into location named 'total' " as STA TOTAL instead of STA 1008,

Symbols provided to benefits. First, referring to a memory location as "TOTAL" instead of its numeric address made the programs more readable. Second, as the program was revised, the location for TOTAL changed (it was 1008 in the first version, then 1010 in the second version because we needed some memory for other values, and 1015 in a third version). As the real address of TOTAL moved, the symbolic assembler kept up with the changes and the programmer didn't have to worry about them.

Each of those techniques improved productivity and eased the jobs of programmers. But we didn't stop there.

Programming languages: After assemblers, we invented the notion of programming "languages". There were many languages in those early days; Fortran and Cobol are two that we still use today, albeit with enhancements and changes to syntax. We called these "high level" languages to distinguish them from the "low level" assemblers.

We created compilers to convert programs written in high level languages into either machine code or into assembly code (which could then be converted to machine code by the assemblers). We still use high level languages today. We have Rust and Go and C++, which all follow the same process as the early compilers.

But after the invention of programming languages, things changed. The ideas and techniques for improving productivity focussed on the programmers and how they used the compilers, how they stored "source code" (the text programs), and how they interacted with the computer.

Structured programming: Structured programming was not a new language, or a new compiler or assembler, or even a program. It was a set of techniques to write programs that could be better understood by the original author and others. We had decided that programs were hard to read because the sequence of execution was hard to follow, and it was hard to follow because of the GOTO operation, which changed the sequence of control to another part of the program. Avoiding the GOTO became a goal of "good programming". GOTO statements were replaced with IF/THEN/ELSE, WHILE, and SWITCH/CASE statements. These rules were enforced in Pascal (which had a constrained GOTO) and implemented in PL/I, C, and C++ (but they still allowed unconstrained GOTO).

The IDE (integrated development environment:  Prior to the IDE, work on programs was divided between a text editor and the compiler. You would run the editor, make changes, save the file, and exit the text editor. Then your would run the compiler, get a list of errors, and note them. Then run the editor again and fix the errors, then run the compiler again. The development process consisted of alternately running the editor and the compiler. The IDE combined the editor and compiler into a single program, so you could edit and compile quickly. Popularized by Turbo Pascal in the 1980s, they had existed prior in the UCSD p-System. One could even say that BASIC was the first IDE, as it let one edit a program, run the program, and diagnose errors without leaving the BASIC program.

Fourth generation languages: Higher, more abstract than the third generation languages (Cobol, Fortran). SQL is a fourth-generation language, probably the only one we use today. The others were discarded due to poor performance and inability to handle low-level 

Program generators: If compilers take source code and convert it to assembly language (and some of them did), then we can apply the same trick and create a configuration file and feed it into a program generator which generates a program in a third level language (which can then be compiled as usual). Program generators failed bot because they couldn't do the job, but because the very high level languages were very limited in their capabilities. As one colleague said, "program generators do what they do very well, but nothing beyond that".

Source control systems: From the first source control system (sccs) to today's modern tool (git) source control kept previous versions of code and allowed programmers to compare the current version to earlier versions of the code. It allowed programmers to commit changes into a central repository and easily share their updates with other members of the team.

UML (Universal Modelling Language): Similar to program generators, UML was a notation for specifying computation. (The 'U' for 'universal' was the result of combining multiple competing modelling notations.) UML wasn't a programming language -- it wasn't fed into a generator which created the program; instead, it was used by human programmers to create the programs in traditional programming languages. UML was more generic than the configuration files for program generators. But it was not adopted by the industry, for reasons of money, time, and politics.

Object-oriented programming: A way to organize source code for large systems. One might say that it was "structured programming but bigger". Object-oriented programming is one the biggest successes in programming.

Function points: A way to measure the effort to develop programs from the requirements. Function points were a tool not for programmers but for project managers. They calculated estimates for effort, based on easily identified aspects such as inputs, processing steps, and outputs. This was advertised as an improvement over the previous method of intuition or just plain guessing. Function points were unbiased and un-overly-optimistic, and the approach should have been welcomed by managers. Yet managers eschewed function points. There were challenges (tools for all languages were not available, or were expensive) but I believe that the real reason was that the analyses provided by the tools were often higher than managers wanted. Managers did not like the estimates from the function point reports, and reverted back to the older technique of guessing (which could give a number that managers did like).

Looking back, we can see that we have tried various ideas for improving productivity. Many succeeded, some did not.

But from what I've seen, AI seems to be closest to the fourth generation languages and program generators of the past. It creates programs from a specification (an input prompt). Compared to the program generators of the 1970s and 1980s, today's AI tools are much more sophisticated and can generate many more types of programs. Yet they are still limited to the input data used to train them, and AI can go only so far in creating programs. I expect that we will quickly find those limits and become disappointed with AI.

I suspect that AI has a place in programming, probably with junior developers, as aides to develop simple programs. I have yet to be convinced that AI will handle the creation of large-scale, complex systems -- at least not today's version of AI. Future versions of AI may be able to generate large, complex applications; I will wait and see.

A lesson from the 1960s

The recent push for AI (artificial intelligence) is forcing us to learn a lesson -- or rather, re-learn a lesson that we learned back in the early days of computing.

In the 1960s, when computers were the shiny new thing, we adored computers. They were superior at computing (compared to us humans) and could calculate much faster and more accurately than us. Computers became the subject of books, movies, magazine articles, and even television programs. They were depicted as large, efficient, and always correct (or so we thought).

We trusted computers. That was the first phase of our relationship.

Yet after some time, we learned that computers were not infallible. They could "make mistakes". Many problems within organizations were blamed on "computer error". It was a convenient excuse, and one that was not easily challenged. They became scapegoats. That was the second phase of our relationship.

Given more time, we realized that computers were tools, and like any tools, they could be used or misused, that they were good at some tasks and not others. We also learned that the quality of a computer's output depended on two things: the quality of the program and the quality of the data. Both had to be correct for the results to be correct. Relatively few people worked on the programs; more people worked on the data being fed into computers. 

This was the third phase of our relationship with computers. We recognized that their output was based on the input. We began to check our input data. We began to select sources for our data based on the quality of the data. We even invented a saying: "Garbage in yields garbage out".

That was the 1960s.

Fast-forward to the 2020s. Look carefully at our relationship with AI and see how it matches that first phase of the 1960s relationship with computers. AI is the shiny new thing. We adore it. We trust it.

We don't recognize that it is a tool, and like any tool it is good at some things and not others. We don't recognize that the quality of its output depends on the quality of its input.

We build large language models and train them on any data that we can find. We don't curate the data. We don't ensure that it is correct.

The rule from the 1960s still holds. Garbage in yields garbage out. We have to re-learn that rule.