The end of the C++ party

In the future, historians of programming languages will draw a line and say: "this is the point that C++ began it's decline". And that point will be prior to today. The party is over for C++, although many of the partygoers are still drinking punch and throwing streamers in the air.

Peter Seibel's blog excerpts comments from the just-released Coders and Work. He lists multiple comments about the C++ language, all of them from detractors.

C++ has had a history of negative comments. It's early history, as a quiet project and before the internet and related twitterness, saw comments about C++ through e-mails and usenet. As people became interested in C++, there were more comments (some positive and some negative) but there was the feeling that C++ was the future and it was the place to go. Negative comments, when made, were either directed to the difficultly of learning a new paradigm (object-oriented programming), the implementation (the compiler and libraries), or the support tools (the IDE and debugger). C++ was the shiny new thing.

The arrival of IBM OS/2 and Microsoft Windows also made C++ attractive. OS/2 and Windows use an event-driven model, and object-oriented programs fare better than procedural programs. Microsoft's support for C++ (among other languages) also made it a "safe" choice.

The novelty of a new programming language is a powerful drug, and C++ was a new language. Managers may have been reluctant to move to it (the risks of unknown territory and longer ramp-up for developers) and some programmers too (charges of larger executables and "inefficient generated code") but eventually we (as an industry) adopted it. The euphoria of the new was replaced with the optimism of the next release: "Yes," we told ourselves, "we're having difficulties, but the problem is in our compiler, or our own expertise. Next year will be better!"

And for a while, the next year was better. And the year after that one was better too, because we were becoming better object-oriented programmers and the compilers were getting better.

But there were those who complained. And those who doubted. And there were those who took action.

Sun introduced Java, another object-oriented programming language. For a while, it held the allure of "the new thing". It had its rough spots (performance, IDE) but we overcame them and newer versions were better. And C++ was no longer the one and only choice for object-oriented programming. (I'm ignoring the earlier languages such as LISP and Scheme. They never entered the mainstream.)

Once we had Java, we could look at C++ in a different light. C++ was not the shining superhero that we desired. He was just another shlub that happened to do some things well. C++ was demoted from "all-wonderful" to "just another tool", much to the delight of the early complainers.

Other languages emerged. Python. Ruby. Objective-C. Haskell. All were object-oriented, but none powerful enough to dislodge C++. The killer (for C++) was Microsoft's C# language. The introduction of C# (and .NET) struck two blows against C++.

First, C# was viewed as a Java clone. Microsoft failed at embracing and extending Java, so they created a direct competitor. By doing so, they gave Java (and its JVM) the stamp of legitimacy.

Second, Microsoft made C# their premier language, demoting C++ below Visual Basic. (Count the number of sample code fragments on the Microsoft web site.) Now Microsoft was saying that C++ wasn't the shiny new thing.

We (in the programming industry) examined our problems with C++, discussed them, debated, them, and arrived at a conclusion: problems have been solved, but the one problem remaining is that C++ is a difficult language. The next version of the compiler will not fix that problem. Nor will more design patterns. Nor will user groups.

The C++ party is over. People are leaving. Not just the folks in Coders at Work, but regular programmers. Companies are finding it hard to hire C++ programmers. Recruiters tell me that C++ programmers want to move on to other things. We as a profession have decided to if not abandon C++, at least give it a smaller role.

Which presents a problem for the owners of C++ systems.

The decision to leave C++ has been made at the programmer level. Programmers want out. Very few college graduates learn C++ (or want to learn it).

But the owners of systems (businessmen and managers) have not made the decision to leave C++. For the most part, they want to keep their (now legacy) applications running. They see nothing wrong with C++, just as they saw nothing wrong with C and FORTRAN and COBOL and dBase V. C++ works for them.

In a bizarre, almost Marxist twist, the workers are leaving owners with the means of production (the compilers and IDEs of C++) and moving on to other tools.

C++ has been elevated to the rank of "elder language", joining COBOL and possibly FORTRAN. From this point on, I expect that the majority of comments on C++ will be negative. We have decided to put it out to pasture, to retire it.

There is too much code written in C++ to simply abandon it. Businesses have to maintain their code. Some open source projects will continue to use it. But it will be used grudgingly, as a concession to practicalities. Linux won't be converted to a new language... but the successor to Linux will use something other than C++.

Glass houses

I just went through the experience of renewing my IEEE (and IEEE Computer Society) membership with the IEEE web pages. The transaction was, in a word, embarrassing.

Here is my experience:

- After I logged in, the web site complained that I was attempting to start a second session and left me with an empty window. I had to re-load the renewal page to continue. (Not simple press the "reload" button, but re-select the IEEE URL again.)

- The few pages to process the renewal were straightforward, until I reached the "checkout" page. This page had a collection of errors.

- After entering my credit card number, the site informed me that I had too many characters in the number. I had entered the number with spaces, just as it appears on my credit card and my statements. The site also erased my entry, forcing me to re-enter the entire number.

- I used the "auto-fill" button to retrieve the stored address. The auto-fill did not enter a value for the country, however, and nor could I, as the field was disabled. Only after adjusting the street address could I select a country.

- After clicking the "process" button, the web site informed me that I had an invalid value in the "state/province" field. I dutifully reviewed the value supplied by the auto-fill routine, changed it from "MD" to "MD".

- That action fixed the problem with the state/province field, but the web site then erased my credit card number. After entering the credit card number again (the third time), I was able to renew my membership.

If the IEEE (and by association the IEEE Computer Society) cannot create and maintain a check-out web site, a function that has been with us for the past ten years and is considered elementary, then they have little credibility for advice on software design and construction. More than that, if the IEEE cannot get "the basics" right, how can anyone trust them for the advanced concepts?

A cell phone is not a land-line phone

When you call a land line, you call a place. When you call a cell phone, you call a person.

I heard this idea at a recent O'Reilly conference. (It was either E-Tech or OSCON, but I don't remember. Nor do I remember the speaker.)

In the good ole days, calling a place was the same as calling a person. Mostly. A typical (working-class) person could have two locations: home and office. To discuss business, you called them at their office. To discuss other matters, you called them at their home.

A funny thing happened on the way to the Twenty-first Century: people became mobile, and technology became mobile too.

Mobility is not a new idea. Indeed, one can look at the technological and social changes of the Twentieth Century to see the trend of increasing mobility. Trains, airplanes, hotels, reservation systems... the arrow points from "stay in one place" to "move among locations". Modern-day cell phones and portable internet tablets are logical steps in a long chain.

People have become mobile and businesses will become mobile too.

Yet many people (and many organizations) cling to the old notion of "a person has a place and only one place". Even stronger is the idea "a business has a place and only one place (except for branch offices and subsidiaries)". Our state and federal governments have coded these notions into laws, with concepts of "state of residence" and "permanent address". Many businesses tie their customers to locations, and then build an internal organization based on that assumption (regional sales reps, for example). For customers that have large physical assets such as factories and warehouses, this makes some sense. But for the lightweight customer, one without the anchoring assets, it does not. (Yet businesses -- and governments -- will insist on a declared permanent address because their systems need it.)

Newer businesses are not encumbered with this idea. Twitter and LiveJournal, for example, don't care about your location. They don't have to assess your property, send tax bills, or deliver physical goods. Facebook does allow you to specify a location, but as a convenience for finding other people in your social network. (Facebook does limit you to one physical location, though, so I cannot add my summer home.)

Some businesses go so far as to tie an account to a physical location. Land line phones for one, from the old billing practices of charging based on distance called. At least one large shipping company uses the "you are always in this place" concept, since it also uses "charge based on distance" model.

For moving physical boxes in the real world, this may make some sense, but telephone service has all but completely moved to the "pure minutes" model, with no notion of distance. (Calling across country borders is more expensive, but this is a function of politics and rate tariffs and not technology.)

We have separated a person from a single location. Soon we will detach businesses from single locations.

Limits to Growth

Did you know that Isaac Newton, esteemed scientist and notable man of the Church, once estimated the theoretical maximum height for trees? I didn't, until I read a recent magazine article. It claimed that he calculated the maximum height as 300 feet, using strength and weight formulas.

I have found no other reference to confirm this event, but perhaps the truth of the event is less important than the idea that one can calculate a theoretical maximum.

For trees, the calculation is straightforward. Weight is a function of volume, and can be charted as a line graph. Strength is a function of the cross-section of the tree, and can also be charted as a line graph. The two lines are not parallel, however, and the point at which they cross is the theoretical maximum. (There are a few other factors, such as the density of the wood, and they can be included in the calculation.) The intersection point is the limit, beyond which no tree can grow.

Let's move from trees to software. Are there limits to software? Can we calculate the maximum size of a program or system? Here the computations are more complex. I'm not referring to arbitrary limits such as the maximum modules a compiler can handle (although those limits seem to be relegated to our past) but to the size of a program, or of a system of programs.

It's hard to say that there are limits to the size of programs. Our industry, over the past sixty years, has seen programs and systems grow in size and complexity. In the early days, a program of a few hundred lines of code was considered large. Today we have systems with hundreds of millions of lines of code. There seems to be no upper limit.

If we cannot identify absolute limits for programs or systems, can we identify limits to programming teams? It's quite easy to see that a programming team of one person would be limited to the one of a single individual. That individual might be extremely talented and extremely hard-working, or might be an average performer. A team of programmers, in theory, can perform more work than a single programmer. Using simple logic, we could simply add programmers until we achieve the needed size.

Readers of The Mythical Man-Month by Fred Brooks will recognize the fallacy of that logic. Adding programmers to a team increases capacity, but also increases the communication load. More programmers need more coordination. Their contributions increase linearly, but coordination effort increases faster than linearly. (Metcalfe's law which indicates that communication channels increase as the square of the participants, works against you here.) You have a graph with two lines, and at some point they cross. Beyond that point, your project spends more time communication than coding, and each additional programmer costs more than they produce.

I don't have numbers. Brooks indicated that a good team size was about seven people. That's probably a shock to the managers of large, multi-million LOC projects and their teams of dozens (hundreds?) of programmers. Perhaps Brooks is wrong, and the number is higher.

The important thing is to monitor the complexity. Knowing the trend helps one plan for resources and measure efficiency. Here's my list of important factors. These are the things I would measure:

- The complexity of the data
- The complexity of the operations on the data
- The power of the programming language
- The power of the development tools (debuggers, automated tests)
- The talent of people on the team (programmers, testers, and managers)
- The communication mechanisms used by the team (e-mail, phone, video conference)
- The coordination mechanisms used by the team (meetings, code reviews, documents)
- The rate at which changes are made to the code
- The quality of the code
- The rate at which code is refactored

The last two factors are often overlooked. Changes made to the code can be of high or low quality. High-quality changes are elegant and easy to maintain. Low-quality changes get the work done, but leave the code difficult to maintain. Refactoring improves the code quality while keeping the feature set constant. Hastily-made changes often leave you in a technical hole. These two factors measure the rate at which you are climbing out of the hole. If you aren't measuring these two factors, then your team is probably digging the hole deeper.

So, as a manager, are you measuring these factors?

Or are you digging the hole deeper?

Why IT has difficulty with estimates

Estimating has always been a difficult task in IT. Especially for development efforts. How long will it take to write the program? How much will it cost? How many people do we need? For decades, we have struggled with estimates and project plans. Development projects run over allotted time (and over allotted budget). Why?

I observe that the problem with estimates is on the development side of IT. The other major parts of IT, support and operations, have loads that can be reliably estimated. For support, we have experience with the number of customers who call and the complexity of their issues. For operations, we have the experience of nightly jobs and the time it takes to run them. It's only on the development side, where we gather requirements, prepare designs, and do the programming that we have the problem with estimates. (I'm including testing as part of the development effort.)

The process of estimation works for repeated tasks. That is, you can form a reasonable estimate for a task that you have performed before. The more often you have performed the task, the better your estimate.

For example, most people have very good estimates for the amount of time they need for their morning commute. We know when to leave to arrive at the office on time. Every once in a while our estimate is incorrect, due to an unforeseen event such as traffic delays or water main breaks, but on average we do a pretty good job.

We're not perfect at estimates. We cannot make them out of nothing. We need some initial values, some basis for the estimate. When we are just hired and are making our first trips to the new office, we allow extra time. We leave early and probably arrive early -- or perhaps we leave at what we think is a good time and arrive late. We try different departure times and eventually find one that works for us. Once we have a repeating process, we can estimate the duration.

Hold that thought while I shift to a different topic. I'll come back to estimates, I promise.

The fundamental job of IT is to automate tasks. The task could be anything, from updating patient records to processing the day's sales transactions. It could be monitoring a set of servers and restarting jobs when necessary. It could be serving custom web pages. It is not a specific kind of task that we automate, it is the repetition of *any* task.

Once we identify a repeating task, we automate it. That's what we do. We develop programs, scripts, and sometimes even new hardware to automate well-defined, repeating tasks.

Once a task has been automated, it becomes part of the operation. As an operation task, it is run on a regular schedule with an expected duration. We can plan for the CPU load, network load, and other resources. And it is no longer part of the development task set.

The repeating tasks, the well-defined tasks, the tasks that can be predicted, move to operations. The tasks remaining for development -- the ones that need estimates -- are the ones that have are not repeating. They are new. They are not well-defined. They cover unexplored territory.

And here's where estimates come back into the discussion. Since we are constantly identifying processes that can be automated, automating them, and moving them from development to operations, the well-defined, repeatable tasks fall out of the development basket, leaving the ill-defined and non-repeating tasks. These are the tasks that cannot be estimated, since they are not well-defined and repeating.

Well, you *can* write down some numbers and call them estimates. But without experience to validate your numbers, I'm not sure how you can call them anything but guesses.

Your data are not nails

Data comes in different shapes, sizes, and with different levels of structure. The containers we select for data should respect those shapes and sizes, not force the data into a different form. But all too often, we pick one form for data and force-fit all types of data into that form. The result is data that is hard to understand, because the natural form has been replaced with the imposed form.

This post, for example, is small and has little structure (beyond paragraphs, sentences, and words). The "natural" form is the one your reading now. Forcing the text into another form, such as XML, would reduce our comprehension of the data. (Unless we converted the text back into "plain" format.)

One poor choice that I saw (and later changed) was the selection of XML for build scripts. It was a system that I inherited, one that was used by a development team to perform the compile and packaging steps for a large C++/MFC application. 

The thinking behind the choice or XML was twofold: XML allowed for some structure (it was thought there would be some) and XML was the shiny new thing. (There were some other shiny new things in the system, including Java, a web server, RMI, EJB, and reflection. It turns out that I got rid of all of the shiny things and the build system still worked.)

I can't blame the designers for succumbing to XML. Even Microsoft has gone a bit XML-happy with their configuration files for projects in Visual Studio.

It's easy to pick a single form and force all data into that form. It's also comfortable. You know that a single tool (or application) will serve your needs. But anyone who has used word processors and spreadsheets knows that the form of data lets us understand it.

Some data is structured, some is free-flowing. Some data is large, some is small. Some data consists of repeated structures, other data has multiple items with structure but each item has its own structure.

For build scripts, we found that text files were the most understandable, most flexible, and most useful form. Scripts are (typically) of moderate size. Converting the XML scripts to text saw the size of scripts shrink, from 20,000 lines to about 2200 lines. The smaller scripts were much easier to maintain, and the time for simple changes dropped from weeks to hours. (Mostly for testing. The time for script changes dropped to minutes.)

Small data sets with no to light structure fit well in text files. Possibly INI files, which have a little more structure to them.

Small to medium data sets with heavy structure fit into XML files.

Large data sets with homogeneous items fit well in relational databases.

Large data sets with heterogeneous items fit better into network databases or graph databases. (The "No SQL" movement can give you information about these databases.)

Don't think of all data as a set of nails, with your One True Format as the hammer. Use forms that make your team effective. Respect the data, and it will respect you.

My brain is now small

My brain is now smaller than the programming languages I use.

I started programming in 1976, on a PDP-8/e computer with timeshare BASIC. (Real BASIC, not this Visual Basic thing for wimps. BASIC has line numbers and variable names limited to a letter and an optional digit.)

Since then I have used various languages, operating systems, and environments. I've used HDOS, CP/M, and UCSD p-System on microcomputers (with BASIC, 8080 assembly language, and C), DECsystem-10s (with TOPS-10 and FORTRAN and Pascal), MS-DOS (with C and C++), and Windows (with C++, Java, Perl, and a few other languages).

In each case, I have struggled with the language (and run-time, and environment). The implementation has been the limiting factor. My efforts have been (mostly) beating the language/environment/implementation into submission to get the job done.

That has now changed.

I've been using the Ruby language for a few small projects. Not Ruby on Rails, which is the web framework that uses Ruby as the underlying language, but the Ruby language itself. It is a simple scripting language, like Perl or Python. It has a clean syntax and object-oriented concepts are baked in, not stuck on like in Perl. But that's not the most important point.

Ruby lets me get work done.

The language and run-time library are simple, elegant, and most importantly, capable. It lets me do work and stays out of my way. This is a pleasant change.

And somewhat frightening.

With Ruby, the limiting factor is not the language. The limiting factor is my programming skills. And while my programming skills are good, they are the result of working with stunted languages for the past thirty-odd years. The really neat stuff, the higher-order programming concepts, I have not learned, because the languages that I used could not support them.

Now I can.

I'm frightened, and excited.