Fitzpatrick's Fabulous Future

Best if viewed in...

2012-01-22T18:25:00.000-08:00

Some web sites display the phrase "best if viewed in Internet Explorer 6 or higher".

In the past, this phrase would anger me. I could assume that the web site work work only with Internet Explorer, and my browser of choice (either Opera or Firefox) would fail in some way.

Now the phrase amuses me.

I ask myself, in today's world, why would someone put "works best with IE6" on their web site? What are they hoping to accomplish? What message are they sending?

Internet Explorer version 6 is old. Even Microsoft recommends a later version of Internet Explorer.

Today's browser "market" consists of IE, Firefox, Chrome, and Safari at a bare minimum, and Opera and Lynx for the truly browser-aware. Building a web site for only one browser is unthinkable.

So the message is from an earlier age.

I also suspect that the message was put on sites that performed transactions of some sort, sites that were more than brochure-ware. These sites have users who are attempting to perform some task, either shopping or submitting time cards or recording information. My guess is that the message was a disclaimer, designed to first reduce the number of users with other browsers, and second to provide an easy "out" for the help desk of said web site when people using the "wrong". (Anyone complaining would be pointed to the notice and told to use IE6. Support call closed, user issue, no problem!)

Today, the notion of turning away customers because they have a different browser is ... unusual. It is a rare company that can decline paying customers.

I read the "best if viewed in" phrase now as an indicator, a measure of a web site's age and maintenance. Only web sites designed and built in the late 1990s (and possibly early 2000s) would have this message. Therefore, a site that still bears this message was built in that era and has had no (major) maintenance. Any maintenance that has been performed (if any) has been specific and limited to the task at hand.

In other words, the site is not living on "internet time". It's owner is not updating the site, modifying it to meet new business conditions or leverage new technologies. The web site is... old.

I use the phrase as an indication a company, of how "with it" they are.

Premature optimization?

2012-01-21T05:01:00.000-08:00

It has been said that premature optimization is the root of all evil. (I read this as a loose translation of "optimizing too early can cause you grief", not as "it makes you a bad person".)

We often optimize our programs and systems. We admire system designers who can build systems that work smoothly and with minimal resources -- that is, systems that are optimized.

But what are optimizations? Most of the time, they are not pure optimizations (which use the smallest amount of system resources) but trade-offs (which use one resource in lieu of another). A simple trade-off optimization is caching: using memory (a cheap resource) to avoid database lookups (an expensive operation).

Optimization, or the selection of a specific set of trade-offs, is a good thing as long as the underlying assumptions hold.

Let us consider a long-standing tool in the developer world: version control systems (VCSs). We have used these tools for forty years, starting with SCCS and moving through various generations (RCS, CVS, PVCS, SourceSafe, Subversion, to name a few).

Many version control systems store revisions to files not as whole files but as 'deltas', the changes from one version to another. This decision is a trade-off: using computations (generating the change list) and reducing disk usage. (The list of differences is often smaller than the revised file.)

This trade-off relied on several assumptions:

The files stored in the VCS would be text files
The changes from one version to another would be a small fraction of the file
Disk space was expensive (compared to the user's time)

It turns out that, some forty years later, these assumptions do not always hold. We are using version control systems for more than source code, and some files that are not text files. (Non-text files are handled poorly by the 'delta' calculation logic, and most VCSs simply give up and store the entire file.) User time is expensive (and getting more so) and disk space is cheap (and also getting more so).

The trade-offs made by version control systems are now working against us. We grumble while our systems generate deltas. We care little that the Microsoft Word document files are stored in their entirety.

The latest version control systems ('git' is an example) do away with the notion of deltas. They store the entire file, with various techniques to compress the file and to de-duplicate data. (We still care about disk usage.)

The notion of storing revisions as deltas was an optimization. It is a notion that we are now moving away from. Was it a premature optimization? Was it a trade-off that we made in error? Is it an example of "the root of all evil"?

I think that the answer is no. At the time, with the technology that we had, using deltas was a good trade-off. It reduced our use of resources, and one can justify the claim of optimization. And most importantly, it worked for a long period of time.

An optimization becomes "bad" when the the underlying assumptions fail. At that point, the system is "upside down", or de-optimized. (Some might say "pessimized".) When that happens, we want to re-design the system to use a better technique (and thus reduce our use of resources). The cost of that change is part of the equation, and must be tallied. A long-running optimization with a low cost of change is good; a short-lived optimization (especially one with a high 'fix' cost at the end) is bad.

Optimizations are like leased cars. You can get by for a period of time with lower payments, but in the end you must turn in the car (or purchase it). Knowing the length of the lease and the tail-end costs is important in your decision. Optimizing without knowing the costs, in my mind, is the root of all evil.

Microsoft discovers the need for an ecosystem?

2012-01-08T19:30:00.000-08:00

I believe that we have seen a significant change in Microsoft's philosophy in 2011, one that will lead to a number of changes in the software market.

Allow me to point to two events in the past year:

The decision to support HTML5 and JavaScript as first-level development tools. This is a big change from the C#-centric world of .NET.
The (somewhat quieter) decision to bundle InstallShield LE with Visual Studio, and drop the Microsoft-built install packager. This is also a big change in the "we supply everything" strategy.

These two changes mark a significant shift in Microsoft's strategy. For the past decades (since the introduction of Windows), Microsoft has had a strategy of being the sole source for all Windows-based software. Microsoft was determined to be the dominant supplier in every market, from operating system to office applications, from development applications to business applications.

This "provide everything" strategy has been successful. If you look at a random PC running Windows today, I expect that you will find that it runs Microsoft applications, with possibly a few home-grown applications and possible some applications from Adobe.

Here's a history of the big products in the Windows world:

Microsoft made Word the best -- or at the least most popular -- word processor, beating the competition from WordStar, WordPerfect, and every other competitor.
Microsoft made Excel the best -- or the most popular -- spreadsheet, winning the battle with Lotus 1-2-3 and defeating smaller competitors like Borland's Sprint.
Microsoft's PowerPoint is the standard presentation software for Windows. There are no competitors to speak of.
Microsoft's Access (and later, SQL Server) pushed aside all other database engines: dBase, Paradox, Reflex, R:base, dbVista, and others. A few small competitors (like Faircom) exist in niche markets. IBM's DB2 and Oracle's products still exist, but are less "Windows products" and more "Windows implementations" of multi-platform products.
Microsoft made Visual Studio the IDE of choice, driving Borland out of the market.
Microsoft purchased the SourceSafe product, integrated it with Visual Studio, and made it the dominate version control system for Windows. (Microsoft recently replaced the old SourceSafe product with Team Foundation Server.) Other players exist, but only as a small portion of the market.
Microsoft overpowered Netscape, which re-incarnated itself as the open-source Mozilla project. Browsers are conspicuous in their plurality in Windows; no other major application has this status.
Microsoft absorbed network functions into Windows. (Remember Novell Netware?)
Microsoft absorbed virus-checking into Windows.
Microsoft built media capabilities into Windows, eliminating third-party music players and video players.
Microsoft developed SilverLight to compete with Adobe Flash, and has rolled out several successful, capable versions. I suspect that they would soon displace Adobe, had it not been for HTML5 trumping both Flash and SilverLight.

I'm charging of abuse of monopoly power, nor of using internal technical information to develop superior products. (Others have raised such issues.) I am looking at the results of Microsoft's actions, regardless of their methods. And the results are these: the Microsoft ecosystem is dying.

Compared to the ecosystem for the Apple iPhone/iPad market, the ecosystem for Microsoft is small. Lots of people, from individual developers to large companies, are developing apps for IOS. In contrast, few folks are building applications for Windows.

Comparing the Windows ecosystem of today against the Windows ecosystem of two decades ago shows the same pattern: fewer developers today.

This is no surprise. The only winning strategy in the commercial market is to become big. One cannot succeed by staying a small company. (Yes, I recognize that there are lots of small companies writing software for Windows. But are they successful? I humbly submit that they have plans to become larger, and are simply waiting for the "right market" or the "right opportunity".)

The problem with the Windows ecosystem is that Microsoft kills any company that becomes too large. (How large is "too large" is defined by Microsoft.) How can one even consider Windows as a long-term environment for a product? You either stay small or become large and get crushed by the Microsoft empire. And there are opportunities in the IOS and Android markets. Developers have noticed the possibilities.

And I think Microsoft has realized this.

The past few years Microsoft has been claiming that they have a large ecosystem; the claim is to impress (or soothe) corporate buyers. But looking at the products on the market and the attendance at Microsoft conferences and fairs (and looking closely at the demographics and not just the numbers) one can see that Microsoft is not winning the hearts of new developers.

I think that the ecosystem has been changing quietly over the past decade (since the introduction of Linux and the original iMac computers). Developers have been moving to the non-Microsoft platforms in response to the expense, the "buy one Microsoft thing and you need another Microsoft thing" dependencies of products, and the threat of competition from Microsoft.

After a decade of quiet changes, the difference is significant. Microsoft recognizes it, and realizes that they need to change.

I expect that Microsoft will change from a "we supply everything" shop and focus on items of strategic importance. Those items will be money-makers and important system components. Here's a plan for Microsoft:

Keep Windows, but make significant revisions. Windows is necessary for the Microsoft world. The brand is valuable. Lots of customers are committed to it and are not willing to move to other platforms. But large customers want better security and easier administration, and small customers want lower expenses and easier administration.
Keep ActiveDirectory. It competes with LDAP and is easier to administrate.
Keep Exchange for large customers. Microsoft must offer a cloud-based e-mail/calendar solution for smaller customers.
Keep parts of Microsoft Office. Word and Excel can compete with the Open Office and Libre Office products. Drop PowerPoint. Keep Visio.
Keep Visual Studio and C#. Drop Visual Basic. Increase support for F#.
Drop Internet Explorer. (It offers no strategic value, and other browsers do not harm Microsoft's web offerings.)
Drop IIS. (It offers no strategic value.)
Enhance SQL Server and Access (the front-end GUI) to support data in the cloud as well as local data. Provide data in the cloud so that customers need only Access on their hardware.
Develop the Microsoft App Store (or whatever they call it) and allow others to sell applications.
Develop an update system that updates all apps, not just the Microsoft-supplied programs.

The decision to drop products like Visual Basic, IE, and IIS may strike some as foolish. It may be my personal bias that dictates the decision for VB, but IE and IIS offer no revenue to Microsoft. The need for IE is long gone; the winning strategy is to have superior web applications, not control of the browser. Microsoft would be better focussing their efforts on superior web applications.

One risk of dropping these products is that once customers convert to other products they may see value in non-Microsoft products. By pushing customers to non-Microsoft products, Microsoft legitimizes non-Microsoft solutions. That is a risk that Microsoft must counter by providing superior value in the products it retains in its offerings.

Despite the risks, I think that this is a good direction for Microsoft. By letting non-Microsoft products thrive, Microsoft can restore developers' faith in the Microsoft ecosystem and encourage them to consider it for new projects. I see it as the best way for Microsoft to succeed.

Predictions for 2012

2012-01-07T10:01:00.000-08:00

Happy new year!

The turning of the year provides a time to pause, look back, and look ahead. Looking ahead can be fun, since we can make predictions.

Here are my predictions for computing in the coming year:

With the rise of mobile apps, we will see changes in project requirements and in the desires of candidates.

The best talent will work on mobile apps. The best talent will -- as always -- work on the "cool new stuff". The "cool new stuff" will be mobile apps. The C#/.NET and Java applications will be considered "that old stuff". Look for the bright, creative programmers and designers to flock to companies building mobile apps. Companies maintaining legacy applications will have to hire the less enthusiastic workers.

Less funding for desktop applications. Desktop applications will be demoted to "legacy" status. Expect a reduced emphasis on their staffing. These projects will be viewed as less important to the organization, and will see less training, less tolerance for "Fast Company"-style project teams, and lower compensation. Desktop projects will be the standard, routine, bureaucratic (and boring) projects of classic legacy shops. The C# programmers will be sitting next to, eating lunch with, and reminiscing with, the COBOL programmers.

More interest in system architects. Mobile applications are a combination of front end apps (the iPhone and iPad apps) and back-end systems that store and supply data. Applications like Facebook and Twitter work only because the front end app can call upon the back end systems to obtain data (updates submitted by other users). Successful applications will need people who can visualize, describe, and lead the team in building mobile applications.

More interest in generalists. Companies will look to bring on people skilled in multiple areas (coding, testing, and user interfaces). They will be less interested in specialists who know a single area -- with a few exceptions of the "hot new technologies".

Continued fracturing of the tech world. Amazon.com, Apple, and Google will continue to build their walled gardens of devices, apps, and media. Music and books available from Amazon.com will not be usable in the Apple world (although available on the iPod and iPad in the Amazon.com Kindle app). Music and books from Apple will not be available on Amazon.com Kindles and Google devices. Consumers will continue to accept this model. (Although like 33 RPM LPs and 45 PRM singles, consumers will eventually want a music and books on multiple devices. But that is a year or two away.)

Cloud computing will be big, popular, and confused. Different cloud suppliers offer different types of cloud services. Amazon.com's EC2 offering is a set of virtual machines that allow one to "build up" from there, installing operating systems and applications. Microsoft's Azure is a set of virtual machines with Windows/.NET and one may build applications starting at a higher level that Amazon's offering. Salesforce.com offers their cloud platform that lets one build applications at an even higher level. Lots of folks will want cloud computing, and vendors will supply it -- in the form that the vendor offers. When people from different "clouds" meet, they will be confused that the "other guy's cloud" is different from theirs.

Virtualization will fade into the background. It will be useful in large shops, and it will not disappear. It is necessary for cloud computing. But it will not be the big star. Instead, it will be a quiet, necessary technology, joining the ranks of power management, DASD management, telecommunications, and network administration. Companies will need smart, capable people to make it work, but they will be reluctant to pay for them.

Telework will exist, quietly. I expect that the phrase "telework" will be reserved for traditional "everyone works in the office" companies that allow some employees to work in remote locations. For them, the default will be "work in the office" and the exception will be "telework". In contrast, small companies (especially start-ups) will leverage faster networks, chat and videoconferencing, mobile devices, and social networks. Their standard mode of operation will be "work from wherever" but they won't think of themselves as offering "telework". From their point of view, it will simply be "how we do business", and they won't need a word to distinguish it. (They may, however, create a word to describe folks who insist on working in company-supplied space every day. Look for new companies to call these people "in-house employees" or "residents".)

Understand the sea change of the iPad. The single-app interface works for people consuming information. The old-fashioned multi-windowed desktop interface works for people composing and creating information. This change leads to a very different approach to the design of applications. This year people will understand the value of the "swipe" interface and the strengths of the "keyboard" interface.

Voice recognition will be the hot new tech. With the success of "Siri" (and Android's voice recognizer "Majel"), expect interest in voice recognition technology and apps designed for voice.

Content delivery becomes important. Content distributors (Amazon.com, Google, and Apple) become more important, as they provide exclusive content within their walled gardens. The old model of a large market in which anyone can write and sell software will change to a market controlled by the delivery channels. The model becomes one similar to the movie industry (a few studios producing and releasing almost all movies) and the record industry (a few record labels producing and releasing almost all music) and the book industry (a few publishing houses... you get the idea).

Content creation becomes more professional. With content delivery controlled by the few major players, the business model becomes less "anyone can put on a show" and more of "who do you know". Consumers and companies will have higher expectations of content and the abilities of those who prepare it.

Amateur producers will still exist, but with less perceived value. Content that is deemed "professional" (that is, for sale on the market) will be developed by professional teams. Other content (such as the day-to-day internal memos and letters) will be composed by amateur content creators: the typical office worker equipped with a word processor, a spreadsheet, and e-mail will be viewed as less important, since they provide no revenue.

Microsoft must provide content provider and enable professional creators. Microsoft's business has been to supply tools to amateur content creators. Their roots of BASIC, DOS, Windows, Office, and Visual Basic let anyone (with or without specific degrees or certifications) create content for the market. With the rise of the "professional content creator", expect Microsoft to supply tools labeled (and priced) for professionals.

Interest in new programming languages. Expect a transition from the object-oriented languages (C++, Java, C#) to a new breed of languages that introduce ideas from functional programming. Languages such as Scala, Lua, Python, and Ruby will gain in popularity. C# will have a long life -- but not the C# we know today. Microsoft has added functional programming capabilities to the .NET platform, and modified the C# language to use them. C# will continue to change as Microsoft adapts to the market.

The new year brings lots of changes and a bit of uncertainty, and that's how it should be.

Vox populi

2012-01-04T19:28:00.000-08:00

Apple's "Siri" has shown that voice recognition is not only possible but also practical (and fun). Voice-driven apps are naturals for the smartphone and tablet world, where keyboards and mice are foreigners. A voice-driven app solves the problem of composition on a tablet app: most smartphone and tablet apps are better for the consumption of data, with desktop apps better for composing data.

The new world of voice-driven apps will bring a number of changes.

Voice-driven apps require a new design. One cannot simple replace a keyboard and mouse with voice-driven commands; the user experience is too different.

The (relatively) commonplace task of designing a GUI for a program becomes problematic with voice-directed apps. Do we use commands such as "place a button below the list box"? This may lead to a renaissance of SHRDLU and the old "Adventure" and "Zelda" programs, with their commands of "go north" and "take everything but the snake".

The testing of apps will change, and require new tools and techniques. How does one test a voice-driven app? Do you have people speak the commands, or do you have pre-recorded commands and play them back to the app? How can you build a suite of automated tests for voice-driven apps?

Our notions of the workplace will change. For the past four decades, we have built workplaces from cubicles. Cubicles are sufficient for people to type on keyboards, but are poor environments for speaking to computers. With voice-driven apps, and everyone speaking at their computer, the noise in the typical office increases. Voice-recognition software may be able to filter out background noise and voices; humans may have a harder time of it. Do we change our workplaces to individual offices?

What about the folks working at home, or working in co-working locations? We'll need quiet places to perform our work. At home, that may mean a separate room with a door. Co-working sites may change to suites of hard-walled offices.

The introvert/extrovert gap may be significant with voice-driven apps. Introverts emphasize written text, and they spend a lot of time composing their words. Extroverts speak readily, with less weight on their ideas. The extroverts share their ideas early and look to the group to help shape the final concepts; introverts think up front and have already decided by the time they hand you the document. I expect that extroverts will be more comfortable (or perhaps less uncomfortable) with voice-driven apps.

Do we combine voice-driven with gesture-driven? Microsoft's Kinect has shown itself to be capable and reliable. Perhaps we will have a combination of voice and gesture to control computers and to create content. I can easily see the layout of a GUI being developed by a programmer with voice-driven and gesture-driven commands. "Place a new button at the bottom of the screen", says the developer, and the IDE will create a new button. "Change the text to 'Cancel'" says the developer, "Now move the button to the right" and the programmer gestures with his hands, pushing an imaginary button to the right until it is in the desired position.

Voice recognition is here. I see lots of changes, for developers, testers, and users.

The wonder of Git

2011-12-30T06:11:00.000-08:00

I say "git" in the title of this post, but this is really about distributed version control systems (DVCS).

Git is easy to install and set up. It's easy to learn, and easy to use. (One can make the same claim of other programs, such as Mercurial.)

It's not the simply installation or operation that I find interesting about git. What I find interesting is the organization of the repositories.

Git (and possibly Mercurial and other DVCS packages) allows for a hierarchical collection of repositories. With a hierarchical arrangement, a project starts with a single repository, and then as people join the project they clone the original repository to form their own. They are the committers for their repositories, and the project owner remains the committer for the top-most repository. (This description is a gross over-simplification; there can be multiple committers and more interactions between project members. But bear with me.)

The traditional, "heavyweight" version control systems (PVCS, Visual SourceSafe, TFS) use a single repository. Projects that use these products tend to allow everyone on the project to check in changes -- there are no committers, no one specifically assigned to review changes and approve them. One can set policies to limit check-in privileges, although the mechanisms are clunky. One can set a policy to manually review all code changes, but the VCS provides no support for this policy -- it is enforced from the outside.

The hierarchical arrangement of multiple repositories aligns "commit" privileges with position in the organization. If you own a repository, you are responsible for changes; you are the committer. (Again, this is a simplification.)

Once you approve your changes, you can "send them up" to the next higher level of the repository hierarchy. Git supports this operation, bundling your changes and sending them automatically.

Git supports the synchronization of your repository with the rest of the organization, so you get changes made by others. You may have to resolve conflicts, but they would exist only in areas of the code in which you work.

The capabilities of distributed version control systems supports your organization. They align responsibility with position, requiring more responsibility with authority. (If you want to manage a large part of the code, you must be prepared to review changes for that code.) In contrast, the older version control systems provide nothing in the way of support, and sometimes require effort to manage the project as you would like.

This is a subtle difference, one that is not discussed. I suspect that there will be a quiet revolution, as projects move from the old tools to the new.

The character of programming languages

2011-12-17T14:40:00.000-08:00

Many languages use C-style blocks denoted with braces (the characters '{' and '}').

The BCPL programming language was the first language to use braces as part of its syntax. Earlier languages (notably COBOL, FORTRAN, algol, and LISP) did not use the brace characters.

Earlier languages did not use brace characters because the characters did not exist, at least not as defined characters. There was little in the way of standards for character sets, with each vendor (and sometimes each system) using its own character set. For a language to run on multiple computers, one had to limit the characters used in the language to those available on all planned platforms. Thus, FORTRAN uses uppercase letters and parentheses but not square brackets.

With the introduction of the ASCII and EBCDIC character sets, things changed. A standard character set (well, two standards) let one assume the existence of all of the defined characters.

First published in 1963, the character sets predate the effort to build BCPL in 1966. Thus, when BCPL was designed, the brace characters were present and ready to be used. They also have the virtue of not being used for anything before.

Our character sets define, to some extent, the syntax of our languages.

In open source, can there be more than one?

2011-12-12T18:12:00.001-08:00

In the commercial market, multiple products is considered a healthy sign. The prevailing logic states that competition is a good thing, giving us the best possible product.

Vendors must position their products with a balance of different features and compatible operations. A word processor must provide some unique set of functions, but must provide some core set of functions to be considered a word processor. It must provide some compatibility to be considered useful to new customers (perhaps to convert their existing files). A bold, new approach to letter-writing, an approach that varies from the conventions of current products, will have a difficult time gaining acceptance. A word processor that performs the basic tasks of existing word processors, that is ten percent better at most things, and that offers a few new ideas, has a better chance of success. The commercial market allows for different, similar products.

The commercial market also has the risk of failure. Building a system on a product (say, a compiler or a version control system) builds in the risk of that product. Companies fail, and products are discontinued (even when the vendor succeeds). The user must choose carefully from the available products.

In the open source ecosystem, the dynamics are different. Multiple products (or projects) are not viewed as a necessity. Consider the popular open source solutions for different tasks: Linux, LibreOffice, GIMP, gcc, SendMail, and NFS. There are competing offerings for these functions, but the "market" has settled on these projects. The chances of a project replacing the Linux kernel, or the GIMP package, are low. (Although not zero, as LibreOffice recently replaced OpenOffice.)

Open source is not monolithic, nor is it limited to single solutions. There are competing ideas for scripting languages (Perl, Python, Ruby) and editors (vi and Emacs). There are competing ideas for databases (MySQL and PostGres, not to mention CouchDB).

I think that it is harder for an open source project to remain independent from the lead project than it is for a commercial product to remain independent from the market leader.

In open source, your ideas (and source code) are available. A small project that is mostly compatible with a large project can be absorbed into the large project. To remain independent, a project must remain different in some core aspect. The languages Perl, Python, and Ruby are all different. The editors vi and Emacs are different. Because of their differences, they can continue to exist as independent projects.

For most software functions, I believe that there is a "Highlander effect": there can be only one. There will be one wildly popular kernel, one wildly popular office suite, one wildly popular C++ compiler.

When there are "competing" open source projects, they will either eventually merge or eventually distance themselves (as with the case of vi and Emacs).

A popular open source project can "absorb" other, similar open source projects.

This effect will give a degree of stability to the ecosystem. One can build systems on top of the popular solutions. A system built with Linux, GNU utilities, gcc, and Python will endure for many years.

Tradeoffs

2011-12-11T13:52:00.001-08:00

It used to be that we had to write small, fast programs. Processors were slow, storage media (punch cards, tape drives, disc drives) were even slower, and memory was limited. In such a world, programmers were rewarded for tight code, and DP managers were rewarded for maintaining systems at utilization rates of ninety to ninety-five percent of machine capacity. The reason was that a higher rate meant that you needed more equipment, and a lower rate meant that you had purchased (or more likely, leased) too much equipment.

In that world, programmers had to make tradeoffs when creating systems. Readable code might not be fast, and fast code might not be readable (and often the two were true). Fast code won out over readable (slower) code. Small code that squeezed the most out of the hardware won out over readable (less efficient) code. The tradeoffs were reasonable.

The world has changed. Computers have become more powerful. Networks are faster and more reliable. Databases are faster, and we have multiple choices of database designs -- not everything is a flat file or a set of related tables. Equipment is cheap, almost commodities.

This change means that the focus of costs now shifts. Equipment is not the big cost item. CPU time is not the big cost item. Telecommunications is not the big cost item.

The big problem of application development, the big expense that concerns managers, the thing that will get attention, will be maintenance: the time and cost to modify or enhance an existing system.

The biggest factor in maintenance costs, in my mind, is the readability of the code. Readable code is easy to change (possibly). Opaque code is impossible to change (certainly).

Some folks look to documentation, such as design or architecture documents. I put little value in documentation; I have always found the code to be the final and most accurate description of the system. Documents suffer from aging: they were correct some but the system has been modified. Documents suffer from imprecision: they specify some but not all of the details. Documents suffer from inaccuracy: they specify what the author thought the system was doing, not what the system actually does.

Sometimes documentation can be useful. The business requirements of a system can be useful. But I find "System architecture" and "Design overview" documents useless.

If the code is to be the documentation for itself, then it must be readable.

Readability is a slippery concept. Different programmers have different ideas about "readability". What is readable to me may not be readable to you. Over my career, my ideas of readability have changed, as I learned new programming techniques (structured programming, object-oriented programming, functional programming), and even as I learned more about a language (my current ideas of "readable" C++ code are very different from my early ideas of "readable" C++ code).

I won't define readability. I will let each project decide on a meaningful definition of readability. I will list a few ideas that will let teams improve the readability of their code (however they define it).

Version control for source code A shop that is not using version control is not serious about software development. There are several reliable, well-documented and well supported, popular systems for version control. Version control lets multiple team members work together and coordinate their changes.

Automated builds An automated build lets you build the system reliably, consistently, and at low effort. You want the product for the customer to be built with a reliable and consistent method.

Any developer can build the system Developers need to build the system to run their tests. They need a reliable, consistent, low-effort, method to do that. And it has to work with their development environment, allowing them to change code and debug the system.

Automated testing Like version control, automated testing is necessary for a modern shop. You want to test the product before you send it to your customers, and you want the testing to be consistent and reliable. (You also want it easy to run.)

Any developer can test the system Developers need to know that their changes affect only the behaviors that they intend, and no other parts of the system. They need to use the tests to ensure that their changes have no unintended side-effects. Low-effort automated tests let them run the tests often.

Acceptance of refactoring To improve code, complicated classes and modules must be changed into sets of smaller, simpler classes and modules. Refactoring changes the code without changing the external behavior of the code. If I start with a system that passes its tests (automated tests, right?) and I refactor it, it should pass the same tests. When I can rearrange code, without changing the behavior, I can make the code more readable.

Incentives for developers to use all of the above Any project that discourages developers from using automated builds or automated tests, either explicitly or implicitly, will see little or no improvements in readability.

But the biggest technique for readable code is that the organization -- its developers and managers -- must want readable code. If the organization is more concerned with "delivering a quality product" or "meeting the quarterly numbers", then they will trade off readability for those goals.

Is "cheap and easy" a good thing?

2011-11-30T18:13:00.001-08:00

In the IT industry, we are all about developing (and adopting) new techniques. The techniques often start as manual processes, often slow, expensive, and unreliable. We automate these processes, and eventually, the processes become cheap and easy. One would think that this path is a good thing.

But there is a dark spot.

Consider two aspects of software development: backups and version control.

More often than I like, I encounter projects that do not use a version control system. And many times, I encounter shops that have no process for creating backup copies of data.

In the early days of PCs, backups were expensive and consumed time and resources. The history of version control systems is similar. The earliest (primitive) systems were followed by (expensive) commercial solutions (that also consumed time and resources).

But the early objections to backups and version control no longer hold. There are solutions that are freely available, easy to use, easy to administer, and mostly automatic. Disk space and network connections are plentiful.

These solutions do require some effort and some administration. Nothing is completely free, or completely automatic. But the costs are significantly less than they were.

The resistance to version control is, then, only in the mindset of the project manager (or chief programmer, or architect, or whoever is running the show). If a project is not using version control, its because the project manager thinks that not using version control will be faster (or cheaper, or better) than using version control. If a shop is not making backup copies of important data, its because the manager thinks that not making backups is cheaper than making backups.

It is not enough for a solution to be cheap and easy. A solution has to be recognized as cheap and easy, and recognized as the right thing to do. The problem facing "infrastructure" items like backups and version control is that as they become cheap and easy, they also fade into the background. Solutions that "run themselves" require little in the way of attention from managers, who rightfully focus their efforts on running the business.

When solutions become cheap and easy (and reliable), they fall off of managers' radar. I suspect that few magazine articles talk about backup systems. (The ones that do probably discuss compliance with regulations for specific environments.) Today's articles on version control talk about the benefits of the new technologies (distributed version control systems), not the necessity of version control.

So here is the fading pain effect: We start with a need. We develop solutions, and make those tasks easier and more reliable, and we reduce the pain. As the pain is reduced, the visibility of the tasks drops. As the visibility drops, the importance assigned by managers drops. As the importance drops, fewer resources are assigned to the task. Resources are allocated to other, bigger pains. ("The squeaky wheel gets the grease.")

Beyond that, there seems to be a "window of awareness" for technical infrastructure solutions. When we invent techniques (version control, for example), there is a certain level of discussion and awareness of the techniques. As we improve the tools, the discussions become fewer, and at some point they live only in obscure corners of web forums. Shops that have adopted the techniques continue to use them, but shops that did not adopt the techniques have little impetus to adopt them, since they (the solutions) are no longer discussed.

So if you're a shop and you're "muddling through" with a manual solution (or no solution), you eventually stop getting the message that there are automated solutions. At this point, it is likely that you will never adopt the technology.

And this is why I think that "cheap and easy" may not always be a good thing.

Programming and the fullness of time

2011-11-19T13:43:00.001-08:00

Sometimes when writing code, the right thing to do is to wait for the technology to improve.

On a previous project, we had the challenge of (after the system had been constructed) making the programs run faster. Once the user saw the performance of the system, they wanted a faster version. It was a reasonable request, although the performance was sluggish, not horrible. The system was usable, it just wasn't "snappy".

So we set about devising a solution. We knew the code, and we knew that making the system faster would not be easy. Improved performance would require changing much of the code. Complicating the issue was the tool that we had used: a code-generation package that created a lot of the code for us. Once we started modifying the generated code, we could no longer use the generator. Or we could track all changes and apply them to later generated versions of the system. Neither path was appealing.

We debated various approaches, and the project management bureaucracy was such that we *could* debate various approaches without showing progress in code changes. That is, we could stall or "run the clock out".

It turned out that doing nothing was the right thing to do. By making no changes to the code, but simply waiting for PCs to become faster, the problem was solved.

So now we come to Google's Chromebook, the portable PC with only a browser.

One of the criticisms against the Chromebook is the lack of off-line capabilities for Google Docs. This is a fair criticism; the Chromebook is useful only when connected to the internet, and internet connections are not available everywhere.

Yet an off-line mode for Google Docs may be the wrong solution to the problem. The cost of developing such a solution is not trivial. Google might invest several months (with multiple people) developing and testing the off-line mode.

But what if networking becomes ubiquitous? Or at least more available? If that were to happen, then the need for off-line processing is reduced (if not eliminated). The solution to "how do I process documents when I am not connected" is solved not by creating a new solution, but by waiting for the surrounding technology to improve.

Google has an interesting decision ahead of them. They can build the off-line capabilities into their Docs applications. (I suspect it would require a fair amount of Javascript and hacks for storing large sets of data.) Or they can do nothing and hope that network coverage improves. (By "do nothing", I mean work on other projects.)

These decisions are easy to review in hindsight, they are cloudy on the front end. If I were them, I would be looking at the effort for off-line processing, the possible side benefits from that solution, and the rate of network coverage. Right now, I see no clear "winning" choice; no obvious solution that is significantly better than others. Which doesn't mean that Google should simply wait for network coverage to get better -- but it also means that Google shouldn't count that idea out.

Programming languages exist to make programming easy

2011-11-13T08:40:00.001-08:00

We create programming languages to make programming easy.

After the invention of the electronic computer, we invented FORTRAN and COBOL. Both languages made the act of programming easy. (Easier than assembly language, the only other game in town.) FORTRAN made it easy to perform numeric computations, and despite the horror of its input/output methods, it also made it easier to read and write numerical values. COBOL also made it easy to perform computations and input/output operations; it was slanted towards structured data (records containing fields) and readability (longer variable names, and verbose language keywords).

After the invention of time-sharing (and a shortage of skilled programmers), we invented BASIC, a teaching language that linguistically sat between FORTRAN and COBOL.

After the invention of minicomputers (and the ability for schools and research groups to purchase them), we invented the C language, which combined structured programming concepts from Algol and Pascal with the low-level access of assembly language. The combination allowed researchers to connect computers to laboratory equipment and write efficient programs for processing data.

After the invention of graphics terminals and PCs, we invented Microsoft Windows and the Visual Basic language to program applications in Windows. The earlier languages of C and C++ made programming in Windows possible, but Visual Basic was the language that made it easy.

After PCs became powerful enough, we invented Java, which leverage the power to run interpreted byte-code programs, but also (and more significantly) handle threaded applications. Support for threading was built into the Java language.

With the invention of networking, we created HTML and web browsers and Javascript.

I have left out equipment (microcomputers with CP/M, the Xerox Alto, the Apple Macintosh) and languages (LISP, RPG, C#, and others). I'm looking at the large trend using a few data points. If your favorite computer or language is missing, please forgive my arbitrary selections.

We create languages to make tasks easier for ourselves. As we develop new hardware, larger data sets, and new ways of connecting data and devices, we need new languages to handle the capabilities of these new inventions.

Looking forward, what can we see? What new hardware will stimulate the creation of new languages?

Cloud computing is big, and will lead to creative solutions. We're already seeing new languages that have increased rigor in the form of functional programming. We moved from non-structured programming to structured programming to object-oriented programming; in the future I expect us to move to functional programming. Functional programming is a good fit for cloud computing, with its immutable objects and no-side-effect functions.

Mobile programming is popular, but I don't expect a language for mobile apps. Instead, I expect new languages for mobile devices. The Java, C#, and Objective-C languages (from Google, Microsoft, and Apple, respectively) will mutate into languages better suited to small, mobile devices that must run applications in a secure manner. I expect that security, not performance, will be the driver for change.

Big data is on the rise. We'll see new languages to handle the collection, synchronization, querying, and analysis of large data sets. The language 'Processing' is a start in that direction, letting us render data in a visual form. The invention of NoSQL databases is also a start; look for a 'NoSQL standard' language (or possibly several).

The new languages will allow us to handle new challenges. But that doesn't mean that the old languages will go away. Those languages were designed to handle specific challenges, and they handle them well. So well that new languages have not displaced them. (Billing systems are still in COBOL, scientists still use Fortran, and lots of Microsoft Windows applications are still running in Visual Basic.) New languages are optimized for different criteria and cannot always handle the older tasks; I would not want to write a billing system in C, for example.

As the 'space' of our challenges expands, we invent languages to fill that space. Let's invent some languages and meet some new challenges!

The insignificance of significant figures in programming languages

2011-11-10T18:56:00.001-08:00

If a city with a population figure of 500,000 gets three more residents, the population figure is... 500,000, not 500,003. The reasoning is this: the original figure was accurate only to the first digit (the hundred-thousands digit). It has a finite precision, and adding a number that is smaller than the precision has no affect on the original number.

Significant figures is not the same as "number of decimal places", although many people do confuse the two.

Significant figures are needed for calculations with measured quantities. Measurements will have some degree of imprecision, and the rigor of significant figures keeps our calculations honest. The rules for significant figures are more complex (and subtle) than a simple "use 3 decimal places". The number of decimal places will vary, and some calculations may affect positions to the left of the decimal point. (As in our "city with 500,000 residents" example.)

For a better description of significant figures, see the wikipedia page.

Applications such as Microsoft Excel (or LibreOffice Calc) have no built-in support for significant figures. Nor, to my knowledge, are there plug-ins or extensions to support calculations with significant figures.

Perhaps the lack of support for significant figures is caused by a lack of demand. Most spreadsheets are built to handle money, which is counted (not measured) and therefore does not fall under the domain of significant figures. (Monetary figures are considered to be exact, in most applications.)

Perhaps the lack of support is driven by the confusion between "decimal places" and "significant figures".

But perhaps the biggest reason for a lack of support for significant figures in applications is this: There is no support for significant figures in popular languages.

A quick search for C++, Java, Python, and Ruby yield no such corresponding packages. Interestingly, the only language that had a package for significant figures was Perl: CPAN has the Math::SigFigs package.

So the better question is: Why do programming languages have no support for significant figures?

Advertisements in the brave new world of Swipeville

2011-11-08T19:36:00.000-08:00

We've seen the different types of ads in Webland: banner ads, side-bar ads, in-text ads, pop-up ads, pop-under ads... all of the types.

My favorite are the "your page is loading" ads which block the (completely loaded, who do you think you are kidding) content page. I like them because, with multi-tab browsers, I can view a different page while the ad-covered page is "loading" while the advertisement times out. With multiple tabs, I can avoid the delay and essentially skip the advertisement.

This all changes in the world of phones and tablets. (I call this new world "Swipeville".) Classic desktop browsers gave us tabbed windows; the new platforms do not. The phone and tablet browsers have one window and one "tab", much like the early desktop browsers.

In this world, we cannot escape advertisements by using multiple tabs. Nor can we look at another window (such as another application) while our page "loads". Since apps own the entire screen, they are either running or not -- switching to another app means that the browser stops, and switching back re-runs the page load/draw operation.

Which means that advertisements will be less avoidable, and therefore (possibly) more effective.

Or they may be less effective; the psychology of cell phone ads is, I think, poorly understood. Regardless of effectiveness, we will be seeing more of them.

Picking a programming language

2011-11-06T12:40:00.000-08:00

In the great, ongoing debate of language superiority, many factors are considered ... and brandished. The discussions of languages are sometimes heated. My purpose here is to provide some musings in a cool light.

The popular languages of the day are (in order provided by Tiobe Software): Java, C, C++, PHP, C#, Objective C, Visual Basic, Python, Perl, and Javascript.

But instead of arguing about the sequence of these languages (or even other candidates for inclusion), let's look at the attributes that make languages popular. Here's a list of some considerations:

Platform: which platforms (Windows, OSX, iOS, Android, Linux) support the language
Performance: how well the programs perform at run-time (whether compiled or interpreted)
Readability: how well programs written by programmers can be read by other programmers
Reliability: how consistently the written programs perform
Cost: here I mean direct costs: the cost of the compiler and tools (and ongoing costs for support and licenses)
Market support: how much support is available from vendors, groups, and developers

How well do languages match these criteria? Let's try some free association.

For performance, the language of choice is C++. Some might argue that Objective-C provides better performance, but I think the argument would come only from developers in the OSX and iOS platforms.

Readability is a difficult notion, and subject to a lot of, well, subjectivity. My impression is that most programmers will claim that their favorite language is eminently readable, if only one takes the time to learn it. To get around this bias, I propose that people will pick as second-best in readability the language Python, and I choose that as the most readable language.

I submit that reliability among languages is a neutral item. Compilers and interpreters for all of these languages are quite good, and programs perform -- for the most part -- consistently.

For cost, all of these languages are available in no-cost options. There are commercial versions for C# (Microsoft's Visual Studio) and Objective-C (Apple's developer kit), and one would think that such costs would give boosts to the other languages. And it does, but cost alone is not enough to "make" or "break" a language. Which brings us to market support.

The support of Microsoft and Apple for C# and Objective-C make those languages appealing. The Microsoft tools have a lot of followers: companies that specify them as standards and developers who know and keep active in the C# language.

Peering into the future, what can we see?

I think that the Perl/Python tussle will end up going to Python. Right now, Perl has better market support: the CPAN libraries and lots of developers. These factors can change, and are changing. O'Reilly has been printing (and selling) lots of books on Python. People have been starting projects in Python. In contrast, Perl loses on readability, something that is hard to change.

The Java/C# tussle is more about market support and less about readability and performance. These languages are about the same in readability, performance, and reliability. Microsoft has made C# the prince of languages for development in Windows; we need to see what Oracle will do with Java.

Apple had designated Objective-C, C, and C++ as the only languages suitable for iOS, but is relaxing their rules. I expect some change in the popularity of iOS programming languages.

But what about those other popular languages, the ones I have not mentioned? What about C, Visual Basic, PHP, and Javascript? Each have their fanbase (companies and developers) and each have a fair rating in performance, reliability, and market support.

I expect that Javascript will become more popular, continuing the current trend. The others I think will fade gradually. Expect to see less market support (fewer books, fewer updates to tools) and more conversion projects (from Visual Basic to C#, for example). But also expect a long life from these languages. The old languages of Fortran and COBOL are still with us.

Which language you pick for your project is a choice that you should make consciously. You must weigh many factors -- more than are listed here -- and live with the consequences of that decision. I encourage you to think of these factors, think of other factors, and discuss them with your colleagues.

Mobile first, desktop second (maybe)

2011-11-01T20:02:00.000-07:00

Mobile computing has arrived, and is no longer a second-class citizen. In fact, it is the desktop that may be the second-class application.

A long time ago, desktop applications were the only game in town. Then mobile arrived, and it was granted a small presence: usually m.whatever.com, with some custom scripts to generate a limited set of web pages.

Now, the mobile app is the leader. If you are starting a project, start with mobile, and if you have time, build in the "plain" version later. Focus on your customers; for new apps, customers are mobile devices: iPhones, iPads, Android phones, and tablets. You can add the desktop browser version later, after you get the core running.

Bring your own device

2011-10-31T19:11:00.000-07:00

The typical policy for corporate networks is simple: corporation-supplied equipment is allowed, and everything else is forbidden. Do not attach your own computers or cell phones, do not connect your own tablet computers, do not plug in your own thumb drives. Only corporate-approved (and corporate-supplied) equipment is allowed, because that enables security.

The typical policy for corporate networks is changing.

This change has been brought about by reality. Corporations cannot keep up with the plethora of devices available (iPods, iPads, Android phones, tablets, ... what have you...) but must improve efficiency of their employees. New devices improve that efficiency.

In the struggle between security and efficiency... the winner is efficiency.

IBM is allowing employees to attach their own equipment to the corporate network. This makes sense for IBM, since they advise other companies in the effective use of resources. IBM *has* to make this work, in order for them to retain credibility. After all, if IBM cannot make this work, they cannot counsel other companies and advise that those companies open their networks to employee-owned equipment.

Non-consulting corporations (that is, most corporations) don't have the pressure to make this change. They can choose to keep their networks "pure" and free from non-approved equipment.

For a while.

Instead of marketing pressure, companies will face pressure from within. It will come from new hires, who expect to use their smartphones and tablets. It will come from "average" employees, who want to use readily-available equipment to get the job done.

More and more, people within the company will question the rules put in place by the IT group, rules that limit their choices of hardware.

And once "alien" hardware is approved, software will follow. At first, the software will be the operating systems and closely-bound utilities (Mac OSX and iTunes, for example). Eventually, the demand for other utilities (Google Docs, Google App Engine, Python) will overwhelm the IT forces holding back the tide.

IT can approach this change with grace, or with resistance. But face it they will, and adjust to it they must.

Small is the new big thing

2011-10-26T12:37:00.000-07:00

Applications are big, out of necessity. Apps are small, and should be.

Applications are programs that do everything you need. Microsoft Word and Microsoft Excel are applications: They let you compose documents (or spreadsheets), manipulate them, and store them. Visual Studio is an application: It lets you compose programs, compile them, and test them. Everything you need is baked into the application, except for the low-level functionality provided by the operating system.

Apps, in contrast, contain just enough logic to get the desired data and present it to the user.

A smartphone app is not a complete application; except for the most trivial of programs, it is the user interface to an application.

The Facebook app is a small program that talks to Facebook servers and presents data. Twitter apps talk to the Twitter servers. The New York Times talks to their servers. Simple apps such as a calculator app or rudimentary games can run without back-ends, but I suspect that popular games like "Angry Birds" store data on servers.

Applications contained everything: core logic, user interface, and data storage. Apps are components in a larger system.

We've seen distributed systems before: client-server systems and web applications divide data storage and core logic from user interface and validation logic. These application designs allowed for a single front-end; current system design allows for multiple user interfaces: iPhone, iPad, Android, and web. Multiple front ends are necessary; there is no clear leader, no "IBM PC" standard.

To omit a popular platform is to walk away from business.

Small front ends are better than large front ends. A small, simple front end can be ported quickly to new platforms. It can be updated more rapidly, to stay competitive. Large, complex apps can be ported to new platforms, but as with everything else, a large program requires more effort to port.

Small apps allow a company to move quickly to new platforms.

With a dynamic market of user interface devices, an effective company must adopt new platforms or face reduced revenue. Small user interfaces (apps) allow a company to quickly adopt new platforms.

If you want to succeed, think small.

Steve Jobs, Dennis Ritchie, John McCarthy, and Daniel McCracken

2011-10-24T18:54:00.000-07:00

We lost four significant people from the computing world this year.

Steve Jobs needed no introduction. Everyone new him as that slightly crazy guy from Apple, the one who would show off new products while always wearing a black mock-turtleneck shirt.

Dennis Ritchie was well-known by the geeks. Articles comparing him to Steve Jobs were wrong: Ritchie co-created Unix and C somewhat before Steve Jobs founded Apple. Many languages (C++, Java, C#) are descendants of C. Linux, Android, Apple iOS, and Apple OSX are descendants of Unix.

John McCarthy was know by the true geeks. He built a lot of AI, and created a language called LISP. Modern languages (Python, Ruby, Scala, and even C# and C++) are beginning to incorporate ideas from the LISP language.

Daniel McCracken is the unsung hero of the group. He is unknown even among true geeks. His work predates the others (except McCarthy), and had a greater influence on the industry than possibly all of them. McCracken wrote books on FORTRAN and COBOL, books that were understandable and comprehensive. He made it possible for the very early programmers to learn their craft -- not just the syntax but the craft of programming.

The next time you write a "for" loop with the control variable named "i", or see a "for" loop with the control variable named "i", you can thank Daniel McCracken. It was his work that set that convention and taught the first set of programmers.

Functional programming pays off (part 2)

2011-10-23T17:55:00.000-07:00

We continue to gain from our use of functional programming techniques.

Using just the "immutable object" technique, we've improved our code and made our programming lives easier. Immutable objects have given us two benefits this week.

The first benefit: less code. We revised our test framework to use immutable objects. Rather than instantiating a test object (which exercises the true object under test) and asking it to run the tests, we now instantiate the test object and it runs the tests immediately. We then simply ask it for the results. Our new code is simpler than before, and contains fewer lines of code.

The second benefit: we can extract classes from one program and add them to another -- and do it easily. This is a big win. Often (too often), extracting a class from one program is difficult, because of dependencies and side effects. The one class requires other classes, not just direct dependencies but classes "to the side" and "above" in order to function. In the end, one must import most of the original system!

With immutable objects, we have eliminated side effects. Our code has no "side" or "above" dependencies, and has fewer direct dependencies. Thus, it is much easier for us to move a class from one program into another.

We took advantage of both of these effects this week, re-organizing our code. We were productive because our code used immutable objects.

Engineering vs. craft

2011-10-19T19:49:00.000-07:00

Some folks consider the development of software to be a craft; others claim that it is engineering.

As much as I would like for software development to be engineering, I consider it a craft.

Engineering is a craft that must work within measurable constraints, and must optimize some measurable attributes. For example, bridges must support a specific, measurable load, and minimize the materials used in construction (again, measurable quantities).

We do not do this for software.

We manage not software but software development. That is, we measure the cost and time of the development effort, but we do not measure the software itself. (The one exception is measuring the quality of the software, but that is a difficult measurement and we usually measure the number and severity of defects, which is a negative measure.)

If we are to engineer software, then we must measure the software. (We can -- and should -- measure the development effort. Those are necessary measurements. But they are not, by themselves, sufficient for engineering.)

What can we measure in software? Here are some suggestions:

- Lines of code
- Number of classes
- Number of methods
- Average size of classes
- Complexity (cyclomatic, McCabe, or whatever metric you like)
- "Boolean complexity" (the number of boolean constants used within code that are not part of initialization)
- The fraction of classes that are immutable

Some might find the notion of measuring lines of code abhorrent. I will argue that it is not the metric that is evil, it is the use of it to rank and rate programmers. The misuse of metrics is all too easy and can lead to poor code. (You get what you measure and reward.)

Why do we not measure these things? (Or any other aspect of code?)

Probably because there is no way to connect these metrics to project cost. In the end, project cost is what matters. Without a translation from lines of code (or any other metric) to cost, the metrics are meaningless. The code may be one class of ten thousand lines, or one hundred classes of one hundred lines each; without a conversion factor, the cost of each design is the same. (And the cost of each design is effectively zero, since we cannot convert design decisions into costs.)

Our current capabilities do not allow us to assign cost to design, or code size, or code complexity. The only costs we can measure are the development costs: number of programmers, time for testing, and number of defects.

One day in the future we will be able to convert complexity to cost. When we do, we will move from craft to engineering.

SOA is not DOA

2011-10-11T18:45:00.000-07:00

SOA (service oriented architecture) is not dead. It is alive and well.

Mobile apps use it. iPhone apps that get data from a server (e-mail or Twitter, for example) use web services -- a service oriented architecture.

SOA was the big thing back in 2006. So why do we not hear about it today?

I suspect it had nothing to do with SOA's marketability.

I suspect that no one talks about SOA because no one makes money from it.

Object oriented programming was an opportunity to make money. Programmers had to learn new techniques and new languages; tool vendors had to provide new compilers, debuggers, and IDEs.

Java was a new programming language. Programmers had to learn it. Vendors provided new compilers and IDEs.

UML was big, for a while. Vendors provided tools; architects, programmers, and analysts learned it.

The "retraining load" for SOA is smaller, limited mostly to the architects of systems. (And there are far fewer architects than programmers or analysts.) SOA has no direct affect on programmers.

With no large-scale training programs for SOA (and no large-scale training budgets for SOA), vendors had no incentive to advertise it. They were better off hawking new versions of compilers.

Thus, SOA quietly faded into the background.

But it's not dead.

Mobile apps use SOA to get work done. iPhones and Android phones talk to servers, using web services. This design is SOA. We may not call it that, but that's what it is.

When the hype of SOA vanished, lots of companies dropped interest in SOA. Now, to move their applications to the mobile world, they will have to learn SOA.

So don't count SOA among the dead.

On the other hand, don't count on it for your profits. You need it, but it is infrastructure, like electricity and running water. I know of few companies that count on those utilities as a competitive advantage.

Talent is not a commodity

2011-10-10T18:10:00.000-07:00

Some companies treat their staff as a commodity. You know the symptoms: rigid job titles, detail (although often inaccurate) job descriptions, and a bureaucracy for hiring people. The underlying idea is that people, like any other commodity, can be hired for specific tasks at specific times. The management-speak for this idea is "just in time provisioning of staff".

Unfortunately for the managers, talented individuals are not stocked on shelves. They must be found and recruited. While companies (hiring companies and staffing companies) have built an infrastructure of resumes and keyword searches, the selection of candidates is lengthy and unpredictable. Hiring a good programmer is different from ordering a box of paper.

The "talent is a commodity" mindset leads to the "exact match" mindset. The "exact match" mindset leads hiring managers (and Human Resource managers) to the conclusion that the only person for the job is the "right fit" with the "complete set of skills". It is an approach that avoids mistakes, turning away candidates for the smallest of reasons. ("We listed eight skills for this position, and you have only seven. Sorry, you're not the person for us!")

Biasing your hiring decisions against mistakes means that you lose out on opportunities. It also means that you delay bringing a person on board. You can wait until you find a person with the exact right skills. Depending on the package (and it's popularity), it may take some time before you find the person.

It might take you six months -- or longer -- to find an exact match. And you may never find an exact match. Instead, with a deadline looming, you compromise on a candidate that has skills that are "close enough".

I once had a recruiter from half-way across the country call me, because my resume listed the package GraphViz. GraphViz generates and manipulates network graphs, and while used by lots of people, it is rarely listed on resumes. Therefore, recruiters cannot find people with the exact match to the desired skills -- the keyword match fails.

Of course, when you bring this person on board, you are under a tight schedule. You need the person to perform immediately. They do their best, but even that may be insufficient to learn the technologies and your current system. (Not to mention the corporate culture.) The approach has a high risk of mistakes (low quality deliverable), slow performance (again, a low quality deliverable), cost overruns from overtime (high expenses), and possibly a late delivery.

Let's consider an alternative sequence.

Instead of looking for an exact match, you find a bright programmer who has the ability to learn the specialized skill. Pay that person for a week (or three) to learn the package. Then have them start on integrating the package into your system.

You should be able to find someone in a few weeks, much less than the six months or more for the exact match. (If you cannot find a bright programmer in a week or two, you have other problems.)

Compromising on specific skills (while keeping excellence in general skills) provides some advantages.

You start earlier, which means you can identify problems earlier.

Your costs may be slightly higher, since you're paying for more time. On the other hand, you may be able to find a person at a lower rate. And even at the higher rate, a few months over a long term of employment is not that significant.

You invest in the person (by paying him to learn something new), and the person will recognize that. (You're hiring a clever person, remember?)

You can consider talent as an "off-the-shelf" commodity, something that can be hired on demand. For commonly used skills, this is a workable model. But for obscure skills, or a long list of skills, the model works poorly. Good managers know how and when to compromise on small objectives to meet the larger goals.

What Microsoft's past can tell us about Windows 8

2011-10-08T15:10:00.000-07:00

Microsoft Windows 8 changes a lot of assumptions about Windows. It especially affects developers. The familiar Windows API has been deprecated, and Microsoft now offers WinRT (the "Windows Runtime").

What will it be like? What will it offer?

I have a guess.

This is a guess. As such, I could be right or wrong. I have seen none of Microsoft's announcements or documentation for Windows 8, so I might be wrong at this very moment.

Microsoft is good at building better versions of competitors' products.

Let's look at Microsoft products and see how they compare to the competition.

MS-DOS was a bigger, better CP/M.

Windows was a better (although perhaps not bigger) version of IBM's OS/2 Presentation Manager.

Windows 3.1 included a better version of Novell's Netware.

Word was a bigger version of Wordstar and Wordperfect.

Excel was a bigger, better version of Lotus 1-2-3.

Visual Studio was a bigger, better version of Borland's TurboPascal IDE.

C# was a better version of Java.

Microsoft is not so much an innovator as it is an "improver", one who refines an idea.

It might just be that Windows 8 will be not an Innovative New Thing, but instead a Bigger Better Version of Some Existing Thing -- and not a bigger, better version of Windows 7, but a bigger, better version of someone else's operating system.

That operating system may just be Unix, or Linux, or NetBSD.

Microsoft can't simply take the code to Linux and "improve" it into WinRT; doing so would violate the Linux license.

But Microsoft has an agreement with Novell (yes, the same Novell that saw it's Netware product killed by Windows 3.1), and Novell has the copyright to Unix. That may give Microsoft a way to use Unix code.

It just may be that Microsoft's WinRT will be very Unix-like, with a kernel and a separate graphics layer, modules and drivers, and an efficient set of system calls. WinRT may be nothing more than a bigger, better version of Unix.

And that may be a good thing.

What have you done for you lately?

2011-10-04T19:28:00.000-07:00

The cynical question that one asks of another is "What have you done for me lately?".

A better question to ask of oneself is: "What have I done for me lately?".

We should each be learning new things: new technologies, new languages, new business skills... something.

Companies provide employees with performance reviews (or assessments, or evaluations, or some such thing). One item (often given a low weighting factor) is "training". (Personally, I think it should be considered "education"... but that is another issue.)

I like to give myself an assessment each year, and look at education. I expect to learn something new each year.

I start each year with a list of cool stuff that sounds interesting. The items could be new programming languages, or a different technologies, or interpersonal skills. I refer to that list during the year; sometimes I add or change things. (I don't hold myself to the original list -- technology changes to quickly.)

Employers and companies all too often take little action to help their employees improve. That doesn't mean that you get a free pass -- it means that you must be proactive. Don't wait for someone to tell you to learn a new skill; by then it will be too late. Look around, pick some skills, and start learning.

What are you doing for you?

The end of the PC age?

2011-10-02T10:09:00.000-07:00

Are we approaching the end of the PC age? It seems odd to see the end of the age, as I was there at the beginning. The idea that a technology should have a shorter lifespan than a human leads one to various contemplations.

But perhaps the idea is not so strange. Other technologies have come and gone: videotape recorders, hand-held calculators, Firewire, and the space shuttle come to mind. (And by "gone", I mean "used in limited quantities, if at all". The space shuttles are gone; VCRs and calculators are still in use but considered curiosities.

Personal computers are still around, of course. People use them in the office and at home. They are entrenched in the office, and I think that they will remain present for at least a decade. Home use, in contrast, will decline quickly, with personal computers replaced by game consoles, cell phones, and tablets. Computing will remain in the office and in the home.

But here's the thing: People do not think of cell phones and tablets as personal computers.

Cell phones and tablets are cool computing devices, but they are not "personal computers". Even Macbooks and iMac computers are not "personal computers". The term "PC" was strongly associated with IBM (with "clone" for other brands) and Microsoft DOS (and later, Windows).

People have come to associate the term "personal computer" with a desktop or laptop computer of a certain size and weight, of any brand, running Microsoft Windows. Computing devices in other forms, or running other operating systems, are not "personal computers": they are something else: a Macbook, a cell phone, an iPad... something. But not a PC.

Microsoft's Windows 8 offers a very different experience from the "classic Windows". I believe that this difference is enough to break the idea of a "personal computer". That is, a tablet running Windows 8 will be considered a "tablet" and not a "PC". New desktop computers with touchscreens will be considered computers, but probably not "PCs". Only the older computers with keyboards and mice (and no touchscreen) will be considered "personal computers".

Microsoft has the opportunity to brand these new touchscreen computers. I suggest that they take advantage of this opportunity. I recognize that their track record with product names has been poor ("Zune", "Kin", and the ever-awful "Bob") but they must do something.

The term "personal computer" is becoming a reference to a legacy device, to our father's computing equipment. Personal computers were once the Cool New Thing, but no more.

Functional programming pays off

2011-09-30T19:03:00.000-07:00

I've been using the "immutable objects" technique from functional programming. This technique starts with object-oriented programming and constrains objects to immutables, objects that cannot change once constructed. This is quite different from the traditional object-oriented programming approach, in which objects can change their state.

With the immutable object style, objects can be constructed but not modified. This is constraining, yet it is also freeing. Once we have an object, I know that I can re-arrange code and not affect the object -- it is immutable, after all. Re-arranging code lets us simplify the higher-level functions and make the code more readable.

The new technique was not "natural" -- no change in programming techniques ever is -- and it took some effort to change. I started at the bottom of the object hierarchy, which let me modify objects with no dependencies. This approach was important. I could change the bottom-most objects and not affect the other (high-level) objects. It let me introduce the concept gradually, and with minimal ripples.

Over the past few weeks, I have extended the immutable style upwards, and now most classes are immutable. This change has already yielded results; we can debug problems faster and change the system design quickly, and each time we know that we are introducing no new defects. (A comprehensive set of tests helps, too.)

We now have code that can be read by our (non-programming) subject matter experts, code that works, and code that can be easily changed. This is a win for all of us.

I expect immutable object programming to become popular, and soon.

The end of the word processor?

2011-09-26T19:34:00.000-07:00

Do word processors have a future?

Desktop PCs (and laptop PCs) were built for composers, people who assembled information. The earliest applications for PCs were word processors (Wordstar, Wordperfect, and others), spreadsheets (Visicalc, Supercalc, Lotus 1-2-3), databases (dBase II), and development tools (BASIC, assembly language, Pascal, C, and others).

Cell phones and tablets are built for consumers, people who view information but do not create information. The early apps were games, instant messaging, music players, cameras, Twitter, and perhaps the New York Times.

One does not compose documents on a cell phone. Nor does one compose spreadsheets, or presentations. It is a platform for consuming content, not creating it.

Which leads to the question: in the new world of phones and tablets, how do we create content?

I think that the answer is: we create it on a different platform.

Perhaps we create it on servers. Or perhaps we create it on the few desktops that still exist. (I suspect that desktop PCs will not go away, but like mainframes will exist in small numbers.)

But not everyone will be composing. I expect that only a small percentage of computer users will be creating content. The typical home user will not need Microsoft Word. One does not need a word processor to run a household today -- and I suspect one never really needed one.

This shrinks the market for word processors and spreadsheets. More specifically, this shrinks the market for Microsoft Office. And I think Microsoft knows this. It knows that the home market for Microsoft Office is evaporating. (The business side is a different case. More on that in another post.)

Word processors replaced typewriters. They were a convenient way of collecting text for printing on paper. With the advent of the internet and e-mail, we eliminated the step of printing text on paper. That leaves us with the idea of collecting text and sharing it with people -- and one does not need a word processor for that.

Does Windows 8 make Windows our new legacy?

2011-09-25T08:07:00.000-07:00

Windows 8 is a big change from all previous versions of Windows. Not only is the user interface significantly different (tiled Windows are a thing of the past), the underlying APIs are different. The Win32 and Win APIs that we have used for years (decades?) have been replaced by WinRT.

Windows 8 does not completely drop support for WinAPI and tiled windows. They are supported in the Desktop app, an app available under Metro and WinAPI.

But the position of WinAPI has moved from premiere to second. It is now "the old thing". And by extension, applications that use WinAPI are now "the old thing". In other words, legacy applications.

If the platform on which we build our application becomes obsolete, so do our applications. It matters not what we think of our applications. We may love them or despise them, respect them for their profitability or fear them for the maintenance headaches. But the ability to support them rests on the platform; without it, our applications vanish.

Microsoft has signalled that they are removing the WinAPI platform. To live, our applications must move to the new platform.

Microsoft closes the door to Windows

2011-09-19T16:23:00.000-07:00

Microsoft's primer for Metro yields lots of interesting information. What I find most interesting is that Microsoft is "closing the door" for Windows development. And by that, I mean that Microsoft is exerting more control over the market. I see two limitations:

Microsoft becomes a gatekeeper, evaluating and approving those apps that become available in the Windows App Store
Microsoft limits the languages for development

Gating apps for distribution is possibly a mistake. The great strength of Windows was its open market. Anyone could develop and sell (or give away) a Windows application. The entry price was not free (one needed the tools and the knowledge) but the market was open to all comers. Microsoft made some attempt to ensure quality, with such things as "Ready for Win95" requirements, but always let anyone develop and distribute software.

The approval process is limited to the Metro side of Windows 8; I believe that classic Windows apps which run under the Desktop app will follow the old model. Yet I believe that the Desktop app is a transition device, akin to the "DOS box" that was in Windows 95 and later versions. The Desktop becomes the place for "legacy" apps, apps with a limited life span. Metro is the future, and Microsoft will build tools and support for the brave new Metro world.

Enterprises and developers will be able to create and install their own apps without going through the Windows App Store. I'm guessing that Microsoft will limit this capability, requiring developers to sign their apps and allowing them to install only their own apps -- they won't be able to install an app from a different developer. (I'm guessing the same will hold for enterprises, too.)

So as a Windows developer, I can build and test my app on my hardware -- but I cannot give it (or sell it) to you, unless I go through the Windows App Store.

And I'm guessing that the ability to install self-signed apps will come for a price: a developer license, or maybe a signing license. (Probably multiple levels of license, from developer to enterprise, with different price tags.)

Microsoft also limits the languages for Metro apps to a few: C++, C#, Visual Basic, and Javascript. This is an interesting development, given the genesis of .NET. The announcement of .NET emphasized the plurality of languages (some from Microsoft and some from third parties) which contrasted .NET against Java's "one language for everyone" design.

With these two changes, it becomes clear that Linux is the only open platform, allowing choice for development languages and an open market. Microsoft joins Apple and Google/Android in the closed and controlled market for apps.

If we define the beginning of the PC revolution as the IMSAI 8800 (from 1977) with its totally open architecture, we can mark today as another step towards the end of that revolution.

Business must be simple enough for a cell phone

2011-09-15T20:15:00.000-07:00

Is your business simple enough to serve customers on a cell phone? If not, you may want to think about changing your business.

In the "good old days", companies had large mainframe computers and those computers allowed for complex businesses. Computers allowed banks to pay interest based on the day of deposit and the day of withdrawal. they allowed telephone companies to set rates beyond the simple commercial and residential categories. The best example is airlines, with their complex systems for reservations and fares.

Personal computers, despite their name, were never simple. The applications ranged from simple to complex, and even the set-up of the equipment required some expertise. The typical PC application is a complex beast. Even the simple Windows applications of Notepad and Calculator have nooks and crannies, little features that are unobvious.

The World Wide Web changed the idea of complexity. Web pages can be simple or complicated, but a business on web pages can afford only so much complexity. When customers use "self-service" web sites, complexity is your enemy. A customer will accept only so much complexity; after that they call your help desk.

Cell phones and tablet computers have set a new bar for simplicity in applications. An app on a cell phone must be simple; the size of the display constrains the complexity. Tablet computers are following the cell phone model, not the desktop PC model.

I''m convinced that Microsoft recognizes this; their new Windows 8 and its "Metro" interface is geared for simpler applications.

Consumers have come to expect a simple experience. Most Microsoft applications are complicated, from their file formats to the installation routines to their "average" use. This complexity is a failure of the operating system that was going to make computers "easy to use" and "intuitive". If you are following the Microsoft model of applications (multiple windows, lots of dialogs), you will have a difficult time living in the brave new world of cell phone and tablet apps.

But it's not just your software. If your software is complicated, it probably means that your business is complicated. (For example, airline reservations.) Complex businesses require complex software, and complex software does not fit in the cell phone interface.

Smart phones have been out for several years. If you cannot offer your business to customers on a cell phone (because it's too complicated), you may want to think about your business. The cell phone and the tablet are the new location for business. If you cannot fit there, you will be unable to survive.

Microsoft Metro is not their PS/2 -- its worse

2011-09-13T19:17:00.000-07:00

Microsoft is releasing more information about Windows 8 and its "Metro" interface. Metro is very different from the "traditional" Windows interface. It is more like a cell phone or tablet interface, displaying one application at a time (no tiled windows) and built to handle touch screen gestures.

IBM introduced the PS/2 (and OS/2) in 1987, six years after the introduction of the original IBM PC. The PS/2 was a step up from the PC/XT/AT product line, addressing multiple problems with that line. The PS/2 had a different floppy drive, a smarter buss, better video, a smaller keyboard connector, and a built-in mouse port, to name the big improvements.

The problem was, no one followed IBM. This was in part due to IBM's licensing arrangements. The original PC was "open", at least to hardware. It took a while for Compaq and other manufacturers to develop a compatible BIOS which allowed them to build computers with sufficient compatible behavior to run popular software.

Instead of following IBM, people followed Compaq, which introduced their "Deskpro" line. The Deskpros were fast PCs that used the traditional connectors and busses, with faster processors and more memory. Compaq also beat IBM with the first 80386-based PC, the "Deskpro 386".

IBM found that it was not the market leader. (You're not a leader when you say, "Let's go this way" and no one follows.)

Now Microsoft is introducing Windows 8 and Metro. Are they pulling a "PS/2"? The answer seems to be "no".

Metro is a big change. Metro is certainly different from the Windows interface introduced in Windows 1.0 and enhanced in Windows 95. The shift from tiled windows to single-app visibility is a large one. This change is as big as the PC-to-PS/2 change.

Microsoft is committing to Metro. Windows 8 and Metro include a "Legacy Windows" app, in which one can run old-style Windows applications. The Microsoft propaganda says that legacy apps will be supported in Windows 8, and I am sure that they will. But let's not fool ourselves: The new stuff will be in Metro.

Metro is not a bold new paradigm. Microsoft is shifting to the new paradigm introduced by iOS and Android. In this way, Microsoft is not leading the market, but following it.

The Microsoft App Store is not a bold new paradigm. An app store is, again, following the market. Yet it also alienates the software distribution channel. When software is sold on-line and not in boxes, the retailers (Best Buy, etc.) have nothing to sell.

The market will not revert to older designs. The market rejected IBM's PS/2 and selected Compaq (with its old-style designs) as their new leader. With Windows 8 and Metro, Microsoft is validating the existing markets for iOS and Android apps. The only old-style provider would be Linux, which offers multi-tiled desktop applications, and I see few people (and even fewer companies) abandoning Windows for Linux.

The PC did, eventually, mutate to something quite similar to the PS/2 design. The new keyboard and mouse connectors were adopted. The PS/2 Micro-channel bus was not adopted, but the PCI bus was. The VGA standard was adopted and quickly surpassed with Super VGA, WVGA, XVGA, and UVGA and a host of others. Everyone used the new 1.44MB 3.5" floppy disk standard. The only thing lacking from the PS/2 was the orange switch for power.

Will something similar happen with operating systems and software? I think that the answer is "yes". Will Microsoft be the company to lead us? I'm not so sure. Apple and Android have a commanding presence.

So I don't know that "Metro" is Microsoft's "PS/2 event". But I do think that it is a significant change.

Ugly business makes for ugly code

2011-09-12T05:28:00.000-07:00

A number of associates bemoan the state of their code. More specifically, they complain of the complexity of the code, and of the time and effort required to implement changes and new features.

The complexity of code is the sum of two elements: the complexity of the business and the complexity of the source code. Complexity of the business is decided by the management of the business; complexity of the code is a bit more subtle.

A program can be written in multiple ways; some simple and some complex. In my experience, non-trivial programs can be complicated or simple (some might use the word "elegant"), but the time for solutions is different. Complicated solutions can be written quickly; elegant solutions require more time. A "quick and dirty" solution is just what is sounds like: hastily assembled code that gets the job done but is a jumble of logic.

Simple code, code that is easy to maintain, is harder to write than jumbled code. Simple code is the result of jumbled code that is refined and improved through multiple iterations of design. I know of no programmers that write simple, elegant code from the get-go. They all start with messy code, confirm that they have the correct behavior, and then refactor the code. Those refinements take time and skill.

The one truth about simple code is this:

If you want simple code, you have to pay for it.

But there is a limit to the simplicity of your code. That limit is defined by your business. Your business has operational requirements; your programs must meet those requirements.

The one truth about complexity is this:

No amount of programming will simplify complex business rules. If you have a complex business, you will have complex programs.

How you choose to run your business is, well, your business. But consider this: complex programs cost more to maintain than simple programs. The business benefit of complex rules must yield enough additional revenue to cover the cost of the additional maintenance; if not, you are spending money for a net negative return on investment.

Simple is the new black

2011-09-05T18:13:00.000-07:00

If you haven't noticed, we've had a paradigm shift.

We've changed our expectations of computer programs, from comprehensive and complex to simple and east-to-use.

In the old model, we had large, complicated programs that were accompanied by manuals (installation manuals, reference manuals, operations manuals) and how-to books ("Learn Microsoft Word in 21 Days!"). Recent versions of software had no printed manual but large help files and comprehensive web pages.

The value of the software was based on the weight of the box and accompanying materials. Small, light packages were valued less than heavy packages. An application with lots of documentation was worth more than an application with little documentation. Enterprise applications were worth even more, since they required not only manuals but dedicated administrators and specialists to teach the regular users.

Smartphones and tablets changed that model. They defined and validated a different method to evaluate software.

In the new model, we use software without referring to manuals. In the new model, we expect software to work for us. In the new model, we value results.

This bodes ill for programs such as Microsoft Word and Microsoft Excel. These programs are complex. They offer many features, and lots of control over the document (or spreadsheet), but they require a "ramp-up" time. We're no longer willing to pay that time. Software has been swallowed into the age of "immediate self-gratification".

It also bodes ill for enterprise software. Well, enterprise software that provides no value to the enterprise. Large corporations may put up with complicated software, but it must prove itself. We no longer send people to week-long classes for the use of software. The software has to work immediately, and people have to be productive immediately.

Complex and comprehensive is not sufficient. Software has to offer value, and it must be immediate and recognized. It must provide value to users (and the enterprise). And it must do it from "minute one", from when we start using the software. Forcing people to learn the habits and quirks of the software is "out"; making people effective immediately is "in".

Microsoft's big opportunity

2011-09-01T18:13:00.000-07:00

Apple has had lots of success with iPhones, iPods, and iPads. They have redefined the computer for the consumer market. Microsoft has not kept up with Apple, and one can say that Apple has surpassed Microsoft with its products.

They key word in that paragraph is "consumer". Apple has excellent products for consumers, people who use computers and digital goods. But Apple is not so good at infrastructure (consider the XServer) and composition tools. Apple has had to go to outside companies for tools to create digital media: Adobe for PDF and Photoshop, Microsoft for its Office suite.

Microsoft has a long history of products that let people create stuff. The earliest was their BASIC interpreter, followed quickly by compilers for COBOL, FORTRAN, and C. Some were developed in-house, others were acquired, but Microsoft was there and ready to supply the builders. Microsoft also has Visual Studio, one of the best environments for developing programs. Beyond programming tools, Microsoft offers its Office suite that allows people to create documents, spreadsheets, presentations, databases, and project plans.

Apple's products are clearly for consumers. The iPad is a wonderful device, but it is not meant for the creator of goods. It's on-screen keyboard is insufficient for true development work. (Yes, you can get a bluetooth keyboard. But at that point, why not a laptop?)

Apple has shown that it is not interested in the market for builders (at least not beyond OSX and iOS platforms). This is Microsoft's opportunity: They can be the supplier of premier development tools for Windows and OSX.

They need not stop at Apple platforms. The Linux tools for development are good, but they are not as good as Microsoft's tools. Here too, Microsoft has the opportunity to become the leader in composition/creation tools. (I'm including compilers in this list, not simply documents and spreadsheets.)

To win these markets, Microsoft must move away from the "Windows and only Windows" mindset. It is an attitude that forces them to build everything: the operating system, the composition tools, and the consumer products. And they haven't done such a great job at all of that.

There is more to it that simply building tools for other platforms. Microsoft has alienated the users of those other platforms, and must reconcile those bad feelings. Microsoft also has licensing issues to work out -- the Linux crowd expects software for free -- but I think their recent "Express" products may lead the way. The solution is within Microsoft's reach.

Getting old

2011-08-29T16:19:00.001-07:00

Predicting the future of technology is difficult. Some technologies endure, others disappear quickly. Linux is twenty years old. Windows XP is ten, although bits of the Windows code base go back to Windows NT (and possibly Windows 3.1 or even MS-DOS). Yet the "CueCat", Microsoft "Bob", and IBM "TopView" all vanished in short order.

One aspect of technology is easy to predict: our systems will be a mix of old and new tech.

Technology has always been a mix of new and old. In the Eldar Days of PCs (the era of the Apple II, CP/M, and companies named Cromemco and Northstar), computer systems were blends of technology. Computers were powered by microprocessors that were low-end even in the day, to reduce the cost. We stored data on floppy disks (technology from the minicomputer age), on cassette tapes (a new twist on old, cheap hardware), and some folks used paper tape (tech from the 1960s).

Terminals were scrounged keyboards and displays, often the display was a television with 40 characters of uppercase characters per line. The better-off could afford systems with built-in equipment like the Radio Shack TRS-80 and the Commodore PET.

The software was a mix. Most systems had some form of Microsoft BASIC built into ROM; advanced systems allowed for the operating system to be loaded from disk. CP/M was popular and new to the microcomputer era, but it borrowed from DEC's operating systems. Apple had their DOS, Heathkit had their own HDOS but also supported CP/M and the UCSD p-System.

We all used BASIC. We knew it was from the timesharing era, despite Microsoft's extensions. When not using BASIC, we used assembly language and we knew it was ancient. A few brave souls ventured into Digital Research's CBASIC, FORTH, or a version of Tiny C.

The original IBM PC was a mix of off-the-shelf and new equipment. The keyboard came from the earlier IBM System/23, albeit with different labels on the keys. The motherboard was new. The operating system (MS-DOS) was new, but a clone of CP/M.

Our current modern equipment uses mixed-age technologies. Modern PCs have just now lost the old PS/2 keyboard and mouse ports (dating back to the 1987 IBM PS/2) and the serial and parallel ports (dating back to the original 1981 IBM PC with the same connectors and earlier with larger, more rugged connectors).

Apple has done a good job at moving technology forward. The iPhone and iPad devices have little in the way of legacy hardware and software. Not bad for a company whose first serious product had a built-in keyboard but needed a television to display 24 rows of 40 uppercase characters.

A successful project requires diverse skills

2011-08-28T13:31:00.000-07:00

My introduction to the Pentaho suite (and specifically the "Spoon" and "Kettle" tools) gave me some insight into necessary skills for a successful project.

For a successful development project, you need several, diverse skills. All are important. Without any of them, the project will suffer and possibly fail.

Assuming that your project will use some tools (compilers, web servers, whatever), you need the skills necessary to use the tools. How the they work. How they open files and read data. Which widgets are included and what they do. (Microsoft .NET has a problem with its widgets: there are too many of them. When building a solution, you must either use pre-built widgets or make your own. The .NET class collection is larger than a single person can comprehend, so you always fear that you are missing out on a simpler solution from a more powerful widget.)

Second is knowledge of the business resources. What data is available? How is the data named? Where is it stored? Most importantly, what does it mean? As with widgets, some data domains have "too much" data, such that a single person can never be sure that they have the right data. (Large organizations tend to have analysts dedicated to the storage and retrieval of data.) Beyond data, you need awareness of servers and network resources.

Third is knowledge of the business environment. What external requirements (regulations) affect your solution? What are the self-imposed requirements (policies)? What about authentication and data retention?

Fourth is the ability to interact with business folks and understand the requirements for the specific task or tasks. (What, exactly, should be on the report? What is its purpose? Who will be using it? What decisions do they make?)

Finally, we have programming skills. The notions of iteration, selection, and computation are needed to build a workable solution. You can lump in the skills of iterative development (extreme programming, agile development, or any set you like). Once you understand the tool and the data available, you must compose a solution. It's one thing to know the tools and the problem domain, it's another to assemble a solution. It is these skills that make one a programmer.

Some organizations break the development of a system into multiple tasks, assigning different pieces to different individuals. This division of labor allows for specialization and (so managers hope) efficiency. Yet it also opens the organization to other problems: you need an architect overseeing the entire system to ensure that the pieces fit, and it is easy for an organization to fall into political bickering over the different subtasks.

Regardless of your approach (one person or a team of people), you need all of these skills.

Farewell Steve Jobs

2011-08-25T18:17:00.000-07:00

Jobs was the last of the original titans of microcomputers. There were many folks in the early days, but only a few known by name. Steve Wozniak, Gary Kildall, and Bill Gates were the others.

Those titans (the known-by-name and the unsung) made the microcomputer revolution possible. Companies like Apple, Microsoft, Digital Research, Radio Shack, Commodore, Heathkit, and even TI and Sinclair all made personal computing possible in the late 1970s and early 1980s.

There are few titans today. Yes, we have Steve Ballmer at Microsoft and Larry Ellison at Oracle, but they are business folks, not technologists. The open source community has its set (Linus Torvalds, Eric S Raymond, and others) but the world is different. The later titans are smaller, building on the shoulders of their earlier kin.

Steve Jobs and Apple taught us some valuable lessons:

Design counts: The design of a product is important. People will pay for well-designed products (and avoid other products).

Quality counts: People will pay for quality. Apple products have been priced higher than corresponding PC products, and people buy them.

Try things and learn from mistakes: Apple tried many things. There were several incarnations of the iPod before it become popular.

One can enter an established market: Apple entered the market with its iPod well after MP3 players were established and "the norm". It also entered the market with its iPhone.

One can create new markets: The iPad was a new thing, something previously unseen. Apple made the market for it.

Drop technology when it doesn't help: Apple products have mutated over the years, losing features that most folks would say are required for backwards compatibility. AppleTalk, the PS/2-style keyboard and mouse ports, RS-232 serial ports, Centronics parallel printer ports, even Firewire have all been eliminated from the Apple line.

Use marketing to your advantage: Apple uses marketing strategically, coordinating it with products. It also uses it as a weapon, raising Apple above the level of the average technology companies.

Replace your own products: Apple constantly introduces new products to replace existing Apple products. They don't wait for someone else to challenge them; they constantly raise the bar.

Focus on the customer: Apple has focussed on the customer and their experience with the product. Their customer experience beats any product, commercial or open source.

Apple must now live without Steve Jobs. And not only Apple, but all of us. Steve Jobs' influence was not merely within Apple but extended to the entire computing world.

Immutable Object Programming

2011-08-22T17:43:00.000-07:00

I've been working with "Immutable Object Programming" and becoming more impressed with it.

Immutable Object Programming is object-oriented programming with objects that, once created, do not change. It is a technique used in functional programming, and I borrowed it as a transition from traditional object-oriented programming to functional programming.

Immutable Object Programming (IOP) enforces a discipline on the programmer, much like structured programming enforced a discipline on programmers. With IOP, one must assemble all components of an object prior to its creation. The approach of traditional object-oriented programming allows for objects to change state, and this is not possible with IOP. With IOP, you do not want an object to change state. Instead, you want a new object, often an object of a different type. Thus, when you have new information, you construct a new object from the old, adding the information and creating a new object of a similar but different type. (For example, a Sale object and a payment are used to construct a CompletedSale object.)

IOP yields programs that have lots of classes and the logic is mostly linear. The majority of statements are assignment statements -- often creating an object, and the logic for iteration and decisions are contained within the constructor code.

As a programmer, I have a good feeling about the programs I write using IOP techniques. It is a feeling of certainty, a feeling that the code is correct. It is a good feeling.

I experienced this feeling once before, when I learned structured programming techniques. At the time, my programs were muddled and difficult to follow. With structured programming techniques, my programs became understandable.

I have not had that feeling since. I did not experience it with object-oriented programming; OOP was difficult to learn and not clarifying.

You can use immutable object programming immediately; it requires no new compiler or language. It requires a certain level of discipline, and a willingness to change. I use it with the C# language; it works with any modern language. (For this conversation, C++ is omitted from the set of modern languages.) I started with the bottom layer of our objects, the ones that are self-contained. Once the "elementary" objects were made immutable, I moved up a layer to the next set of objects. Within a few weeks I was at the highest level of objects in our code.

Iterating over a set is better than looping

2011-08-15T18:18:00.000-07:00

When coding, I find it better to use the "foreach" iterator that the "for" loop.

The two are similar but not identical. The "for" operation is a loop for a fixed number of times; the "foreach" operation is applied to a set and repeats the contained code once for each member of the set. A "for" loop will often be used to achieve the same goal, but there is no guarantee that the number of iterations will match the size of the set. A "foreach" iteration is guaranteed to match the set.

For example, I was reviewing code with a colleague today. The code was:

for (int i = 0; i < max_size; i++)

{

for (int j = 0; j < struct_size; j++, i++)

{

item[i] = // some value

}

This is an unusual construct. It differs from the normal nested loop:

The inner loop increments both index values (i and j)
The inner loop contains assignments based on index i, but not j

What's happening here is that the j loop is used as a counter, and the index i is used as an index into the entire structure.

This is a fragile construct; the value max_size must contain the size of the entire structure "item". Normally the max_size would contain the number of larger elements, each element containing a struct_size number of items. Changing the size of item requires understanding (and remembering) this bit of code, since it must change (or at least the initialization of max_size must change).

Changing this code to "foreach" iterators would make it more robust. It also requires us to think about the structures involved. In the previous code, all we know is that we have "max_size" number of items. If the set is truly a linear set, then a single "foreach" is enough to initialize them. (So is a single "for" loop.) If the set actually consists of a set of items (a set within a set), then we have code that looks like:

foreach (Item_set i in larger_set)

{

foreach (Item j in i)

{

j = // some value

}

Of course, once you make this transformation, you often want to change the variable names. The names "i" and "j" are useful for indices, but with iterators we can use names that represent the actual structures:

foreach (Item_set item_set in larger_set)

{

foreach (Item item in item_set)

{

item = // some value

}

Changing from "for" to "foreach" forces us to think about the true structure of our data and align our code with that structure. It encourages us to pick meaningful names for our iteration operations. Finally, it gives us code that is more robust and resilient to change.

I think that this is a win all the way around.

The web and recruiting

2011-08-14T16:51:00.000-07:00

When times are hard, and lots of people are seeking employment, companies have an easy time of hiring (for those companies that are even hiring). When times are good, more people are employed and fewer people seek employment. Companies wishing to hire folks have a more difficult time, since there are fewer people "in the market". The "market" of available people swings from a "buyer's market" to a "seller's market" as people are more and less available.

The traditional method of finding people is through staffing agencies and recruiting firms. I suspect that the internet and the web will change this. Companies and candidates can use new means of advertising and broadcasting to make information available, either about open positions or available talent. From Facebook and LinkedIn to contributions to open source projects, candidates make a wealth of information available.

As I see it, companies will fall into two general categories. I call them "group A" and "group B".

Group A companies will use the web to identify candidates and reach out to them. They will use the web as a means of finding the right people. Group A companies use the web-enabled information.

Group B companies will use the web in a passive role. They will post their open positions and wait for candidates to apply. (And they will probably demand that the candidate submit their resume in Word format only. They won't accept a PDF or ODT file.) They may use web sites to check on candidates, probably looking for pictures of the person dressed as a pirate. I expect that they will do little to review contributions to open source projects.

When it comes to finding talent, the group A companies have the advantage. They can evaluate candidates early on and make offers to the people who have the skills that they seek. The group B companies have a harder time, as they have to filter through the applications and review candidates.

I suspect that, between the two strategies, the group A companies will be more effective and have the smaller effort. It's more work for each candidate, but less work overall. The group B companies will spend less time on each applicant, but more time overall. (And by spending less time on each candidate, they make less effective decisions.)

I also suspect that both groups will think that their strategy is the lesser effort and more effective.

So which company do you want to be? And which company would you rather work with?

A measure of quality

2011-08-10T20:56:00.000-07:00

I propose that quality of code (source code) is inversely correlated to duplications within the code. That is, the more duplications, the worse the code. Good code will have few or no duplications.

The traditional argument against duplicate code is the increased risk of defects. When I copy and paste code, I copy not only the code but also all defects within the copied code. (Or if requirements later change and we must modify the copied code, we may miss one of the duplicate locations and make an incomplete set of changes.)

Modern languages allow for the consolidation of duplicate code. Subroutines and functions, parent classes, and code blocks as first-class entities allow the development team to eliminate the duplication of code.

So let us assume that duplicate code is bad. Is it possible to measure (or even detect) code duplications? The answer is yes. I have done it.

Is it easy to detect duplicate code? Again, the answer is yes. Most developers, after some experience with the code base, will know if there are duplicate sections of code. But is there an automated way to detect duplicate code?

And what about measuring duplicate code? Is it easy (or even possible) to create a metric of duplicate code?

Let's handle these separately.

Identifying duplicate blocks of code within a system can be viewed as a scaled-up version of the same problem between two files. Given two separate source files, how can one find the duplicate blocks of code? The method I used was to run a custom program on the two files, a program that identified common blocks of code. The program operated like 'diff', but in reverse: instead of finding differences, it found common blocks. (And in fact that is how we wrote our program. We wrote 'diff', and then changed it to output the common blocks and not the different blocks.)

Writing our 'anti-diff' utility (we called it 'common') was hard enough. Writing it in such a way that it was fast was another challenge. (You can learn about some of the techniques by looking for 'how is grep fast' articles on the web.)

Once the problem has been solved for two files, you can scale it up to all of the files in your project. But be careful! After a moment's thought, you realize that to find all of the common blocks of code, you must compare every file against every other file, and this algorithm scales O(n-squared). This is a bad factor for scaling, and we solved it by throwing hardware at the problem. (Fortunately, the algorithm is parallizable.)

After more thought, you realize that there may be common blocks within a single file, and that you need a special case (and a special utility) to detect them. You are relieved that this special case scales at O(n).

Eventually, you have a process that identifies the duplicate blocks of code within your source code.

The task of identifying duplications may be hard, but assigning a metric is open to debate. Should a block of 10 lines duplicated twice (for a total of three occurrences) count the same as a block of 15 lines duplicated once? Is the longer duplication worse? Or is the more frequent duplication the more severe?

We picked a set of "badness factors" and used them to generate reports. We didn't care too much about the specific factors, or the "quantity vs. length" problem. For us, it was more important to use a consistent set of factors, get a consistent set of metrics, and observe the overall trend. (Which went up for a while, and then levelled off and later decreased as we requested a reduction in duplicate code. Having the reports of the most serious problems was helpful in convincing the development team to address the problem.)

In the end, one must review the costs and the benefits. Was this effort of identifying duplicate code worth the cost? We like to think that it is, for four reasons:

We reduced our code base: we identified and eliminated duplicate code.

We corrected defects: We identified near-identical code and found that the near-duplicates were true duplicates, some with fixes and some without. We combined the code and ensured that all code paths had the right fixes.

We demonstrated an interest in the quality of the code: Rather than focus on only the behavior of the code, we took an active interest in the quality of our source code.

We obtained a leading indicator of quality: Regression tests are lagging indicators of quality, observable only after the coding is complete. We can measure duplicate code from the source code, and from the first day of the project, getting measurements immediately.

We believe that we get the behavior that we reward. By imposing soft penalties for duplicate code, measuring the code, and distributing that information, we changed the behavior of the development team and improved the quality of our code. We made it easy to eliminate the duplicate code, by providing lists of the duplicate code and the locations within the code base.

My next PC won't be real

2011-08-07T18:14:00.000-07:00

After working a bit with Eclipse and the Android SDK, I have come to the realization that my PC is a bit ... lame ... and needs to be replaced. The PC is an old Dell Optiplex that I found in the "giving place" in my apartment building. Someone else was throwing it away, and I adopted it. I replaced the disc and pushed the memory to the limit. It served me well for the past few years, but it cannot handle the development tools effectively.

The question is: what new PC do I select? Should it be a PC? Or a Mac? Or a tablet? I'm thinking that the new PC will be none of these.

Perhaps my new PC will be a virtual PC in a cloud. Instead of buying a physical PC and installing it at home, I can rent a virtual PC on a cloud service (for example, amazon.com's EC2, or VMware's cloud). I don't need the PC -- all I need is the processing power to drive Eclipse and the Android SDK.

Think about it... I don't need the PC box sitting in my room. I don't need the cables and wires. All I need is processing power... and I don't really care about the location of the processor. With the internet, I can run my applications anywhere.

I *do* need a way to talk to the virtual PC. I need a mechanism to control Eclipse and see my files. (Really, I want a tablet app that lets me talk to Eclipse in the cloud.)

In the eldar days (before the first IBM PC), the only folks using microcomputers were determined hobbyists and a few really determined business users. The latter thought that they wanted computers, but all they really wanted was the business applications.

Today, people use computers but only from inertia, and Apple has shown another path. People want processing power, not computers. They don't care about CPUs or memory... all they want is for their apps to work.

Current cloud apps are offering a glimpse of the future of apps. Phones and tablets have made the purchase and installation of apps easy and inexpensive. GMail and Google Docs have made apps available wherever there is an internet connection.

Why would anyone go through the trouble of buying and installing a physical PC?

The current cloud offerings are built around servers. Amazon.com EC2 is designed for servers (build your own, but talk to it like a server). Google App Engine is built to run web apps.

The next wave will be personal computers in the cloud. Computers that can run plain apps and talk to you in a VNC client, or maybe even a browser.

The following wave will be personal apps in the cloud... and apps will mutate to live in the cloud. The app will have two parts: a small client on your phone or tablet and a back end on the server in the cloud.

Personal or impersonal?

2011-08-02T20:58:00.000-07:00

There are two ways to design software: impersonal and personal. Interestingly, the same two methods apply to businesses.

Consider airline travel and the instructions given to passengers. Anyone who has travelled on a commercial flight knows these instructions (how to fasten and wear a seatbelt, floatation devices, oxygen masks, and no smoking in lavatories). Airlines are mandated to provide these instructions to passengers.

Many airlines view this task as a cost, and do everything that they can to minimize that cost. The provide the instructions on recorded video, which reduces the cost and ensures a consistent delivery of the message. In doing so, they make the procedure impersonal. Some might argue that it is inhuman.

Southwest Airlines takes a different approach. On all of their flights, the flight attendants provide the message. There is no recording, no impersonal message. People deliver the message. Some crews take the opportunity to customize the message, adding humorous comments to the instructions. The experience on Southwest is more enjoyable.

The same concept applies to software. Now, I don't expect you to visit each of your customers and provide humorous instructions for the use of your software. I will ask you to think about the experience that you provide to your customers. Is it consistent and impersonal?

The change from desktop PC to laptop saw no real change in the user experience. The change from laptop PC to smart phone (or tablet) is larger; customers have a more intimate relationship with these devices. Impersonal applications will find little traction in the smartphone and tablet market.

You can make the customer experience what you want it to be. The question is, what do you want it to be?

Virtual companies

2011-07-28T22:24:00.000-07:00

Some companies consider themselves "virtual". That is, they have no central office, no single place of performing work. They have no permanent office. Employees perform work in their own homes, or in any convenient location, from the local coffee shop to a co-working site.

Not every company can be a virtual company. The assembly of physical goods (like automobiles or lamps) requires that people work in a factory. Taxi drivers must be in taxis. Plumbers must visit the site of the plumbing.

Virtual companies are not so much virtual as they are distributed. The employees (and work performed) is distributed in various locations. This leads to some interesting changes in the management of companies.

The companies that can be distributed are the ones that deal with information, with bits that can be easily transported to different locations. Software development is an obvious candidate; others include accounting, insurance claim processing, banking, X-ray analysis, product support, and economic analysis.

First is the obvious technology changes. Employees must have the equipment and skills for communication and coordination of effort. Telephones, e-mail, instant messaging, video conferencing are necessary communication tools. Collaboration tools such as Google Docs are needed, too.

There are changes in the hiring employees. In the old (central location) style of a company, candidates would be drawn from a pool of local talent. Some companies may choose to re-locate people to their area, once an offer was made and accepted. Other companies won't. In contrast, the new (distributed) style allows for people to work from any location.

Payroll is affected: If I am in one state and hire a person living in another state, what taxes are paid? More specifically, where is the work performed?

More than these, there is the question of performance review. Many shops have some form of performance review, and larger shops have elaborate and bureaucratic processes. Despite the forms and meetings, many evaluations by managers are reduced to the managers opinion of the employee, and that opinion is often guided by the employee's arrival and departure times.

With a distributed workforce, a manager cannot simply walk past cubes to see who is working late.

Some managers may resort to tricks, such as sending requests at 4:55 (or five minutes prior to the employee's shift end, in whatever time zone) to see if the employee responds. Some companies may set up arrangements to "clock in" and "clock out". I suspect that these tricks will annoy everyone involved without contributing to productivity.

In a distributed company, managers will have to measure employees based on their contributions -- which is probably what they should have been doing all along. This may be difficult for some managers, as they will have to learn about their employees' tasks and deliverables. It may be difficult for employees, since they will have to explain their work to their supervisors (and also not boost their score by simply working late two nights a week).

Distributed work is here. Some companies are using it, to their advantage. They have a larger pool of talent from which to draw. Limiting your talent to "the locals" is convenient (and follows the tradition) but may not be good enough to compete in the future.

You can choose to use distributed work, or you can choose to ignore it. But you must choose.

Open source drives innovation

2011-07-28T07:22:00.001-07:00

A long time ago, in a galaxy far, far away... there was a great battle for software. The empires of closed-source software were challenged by a loosely-connected alliance of open source rebels. Mighty and brave were the deeds of many in the alliance.

Open source has succeeded. It is no longer the rebel alliance. It won, in the sense that it is established and legitimate in the eyes of individuals, businesses, and governments. (It did not achieve world domination; if that was your goal then open source still has work to do. But that is another topic.)

If open source is not the rebel alliance, if it is not the subversive movement pushing for change, then what is it?

I believe it to be the research arm of the software industry. It is the laboratory in which new technologies are developed.

Consider these developments, all from open source and not from the "industry" as we know it:

Agile development: The polar opposite of the "big design up front" techniques used by traditional development shops. It was necessary for open source projects, since they lack the command-and-control structure necessary to implement BDUF.

NoSQL databases: Data stores for non-structured data. Big organizations have spent oodles of time trying to structure their data, just as they try to structure everything else. Open source has built memcached, CouchDB, and other approaches to data stores.

Scripting languages: Starting with the Unix shell scripts, through Perl, Python, and Ruby, open source has given us these languages. Closed source has given us Java, C#, and a new version of C++.

Distributed version control: Closed source was content with centralized version control. Only open source projects gave us Git and the idea of a distributed version control system.

Distributed projects: Open source runs on volunteers (usually) and it makes little sense to force contributors into a single location. Open source projects have contributors from around the world, virtual organizations that use the best available people.

Open source is still pushing change. Not merely pushing for change, but driving change. It is driving change at the technology level, which is why I consider it the research arm of the software industry.

The incredible shrinking program

2011-07-25T20:53:00.001-07:00

Computer programs have been shrinking. They have been doing so since the beginning of the computer age. You may think this claim strange, given that computers are much bigger than earlier eras and programs certainly look bigger, with their millions of lines of source code. And you are right -- computer programs have gotten bigger. Yet they have also gotten smaller.

In absolute terms, computer programs are larger. They have more lines of source code, larger memory footprints, and greater complexity.

Relative to the size of the computer, however, computer programs are smaller.

The earliest computers were programmed with plugboards, so the "software" was hardware. For these computers, there was only a thin boundary between the machine and the program, so in a sense the program was the machine.

Computers in the 1940s and 1950s were programmable in the sense we have today, with the hardware and the software being two different kinds of things. The program was loaded into memory and executed, often by an operator. While running, the program was the only thing in memory -- there was no operating system or monitor. The program had to perform all tasks, from input to processing to output.

With the 1960s we saw the advent of operating systems. The operating system contained common functions and the program called the operating system to perform actions. The model of hardware, operating system, and application program was a solid one, and continues to this day.

But notice that the program is now smaller than the machine. The application program is constrained by the operating system. Combined, the program and operating system fill the machine. (For time-sharing systems, the combination is the operating system and all running programs.) Thus, the program has become smaller, giving some functions over to the operating system.

Microcomputers followed this path. The earliest microcomputers (the Altair, the IMSAI, and others of the late 1970s) were processors, memory, and front panels that allowed a person to load a program and run it. This was the same model as the 1940s electronic computers. Hobbyists quickly developed "disk operating systems" although one can argue that the first true operating systems were IBM's OS/2, Microsoft's Windows NT, and variants of Unix, all of which needed the Intel 80386 processor to be effective. As with the mainframe path, the application shrunk and gave up processing to the operating system.

The iOS and Android operating systems extend this trend. Programs are even smaller under these operating systems, yielding more control and code to the operating system.

In the earlier model, the operating system launches a program and the program runs until completion (by signalling the operating system that it is finished). There is one entry point for the program; there can be many exit points.

The model used by iOS and Android is different. Instead of starting the program and letting it run to completion, the operating system issues multiple calls to a program. Instead of a single entry point, a program has multiple (well-defined) entry points. Anyone familiar with event-driven programming will recognize the similarity: Windows sends programs messages about various events (to a single entry point) and the program must fan them out to separate routines. iOS and Android had removed the top layer of that program and fan out the messages themselves.

Thus, programs have once again yielded code to the operating system, and have shrunk in size.

I think that this is a good change. It removes "boilerplate" code from applications and puts the multiple copies in a single place. Application code can focus on the business problem and ignore an infrastructure issue. Programs gain a measure of uniformity.

The model of the operating system lasted for fifty years (twenty in the PC world) and served us well. I want to think of the new model as something different: an operating system with pluggable tasks. I think it will serve us for a long time.

The tablet revolution

2011-07-24T21:06:00.000-07:00

When Apple introduced the iPad, I was not impressed.

I was comparing the iPad to e-readers, specifically the Kindle and the Nook. And for reading, I think that the Kindle and other e-readers are superior to the iPad.

But as a general computation device, the iPad (or tablets in general) are superior not only to e-readers but also to traditional desktop PCs and laptop PCs. Significantly superior. Superior enough to cause a change in the ideas about computing hardware.

The iPad and tablets (and iPods and cell phones) use touchscreens, something that traditional computers have avoided. The touchscreen experience is very different from the screen-and-mouse experience. The touchscreen experience is more intimate and more immediate; the mouse experience is clunky. Why should I have to drag the mouse pointer all the way over to a scroll bar when I can simply reach out the drag the screen?

Apple has, once again, defined the modern computing interface. Apple defined the "mouse" UI with the Lisa and then the Macintosh computers, stealing a bunch of ideas from Xerox (who in turn had lifted them from Doug Englebart).

The mouse interface was introduced in 1983, and it took a while to become popular. Once it became the standard, it was *the* standard way to interact with computers. The introduction of the iPhone/iPod/iPad interface set a new standard, one that is being adopted quickly. Apple is moving the interface to the newest version of OSX (the "Lion" release) and Microsoft is doing the same thing with its "Metro" interface for Windows 8.

The new interface expects a touchscreen. While some folks may try to fudge the interface with a plain display and a mouse, I believe that we will see a fairly rapid conversion to touchscreens.

Converting the hardware is the smaller of the two problems. The bigger problem is the software. Our current software is designed for the mouse interface, not the touch interface. Apple and Microsoft may craft solutions that let older (soon to be derided as legacy) apps to run in the new environment, but there will be some apps that fail to make the transition.

The conversion of software will also give new players an opportunity to take market share. We may see Microsoft lose its dominant position with Office.

I expect two, parallel tracks of acceptance: the home user and the business user. Home users will adopt the new UI quickly -- they have already done so on their cell phones, so changing the operating system to look like a cell phone is probably viewed as an improvement.

Business users, on the other hand, will face a number of challenges. First will be the cost of upgrading equipment with touchscreens. Related to that will be the training issues (probably minimal) and the politics associated with the distribution of the new equipment. If a company must roll out new equipment in phases (perhaps over several years) there will be squabblings over the selection of employees to get the new hardware.

Businesses also have to integrate the new hardware and software into their organization. New hardware can be adopted quickly; new software takes longer. The support teams must learn the software and the methods for resolving problems. The new software must be configured to confirm to existing standards for security, disaster recovery, and data retention. New versions of apps must be acquired and rolled out -- but only to folks with the new equipment.

The fate of developers is hard to predict. The new user interfaces have proven themselves in the consumer space. I suspect that they can work in the business space. I'm unsure of their soundness for developers. Our programming environments, tools, and even languages are designed for keyboards, not swipes on a screen. What kinds of computers will programmers use?

One possibility is that developers will use tradition-style PCs, with tradition keyboards and traditional operating systems. But this will put PCs into a niche market (developers only) and drive the prices up. Developers have for a long time been riding on the wave of popular (low-priced commodity) equipment; I'm not sure how they will adapt to a premium market.

Another possibility, albeit small, is that developers will develop new languages that fit into the new user experience. This is not precedented: BASIC was designed to fit into timesharing systems and Visual Basic was designed for programming in Windows.

Either way, it will be interesting.

The collision of cloud and cryptography

2011-07-20T18:55:00.000-07:00

We have an impending collision: cryptography and cloud computing.

The big problem with most cryptographic systems is the "key problem". Most systems use a "symmetric key", in which the same key is used to encode and decode the message. This presents a problem: for me to send you an encoded message, I must first send you the key. But anyone who has the key can decode the message, so how to send you -- confidentially -- the key? I cannot send it as a message, because either it is encrypted or it is not. If I send the key encrypted, then you cannot decrypt it, because you need the key! If I send it unencrypted, then anyone who sees the message (including intermediate e-mail servers) have access to the key and can decrypt any successive, encrypted message.

This problem was solved (or so we thought) by asymmetric keys. With asymmetric keys, two keys are used: one to encrypt and a different one to decrypt. Technically, one key is the "private key" and the second is the "public key". The two work together, and while either can be used to encrypt (with the other used to decrypt), the two are a matched set. Only the other key in the pair can decrypt a message encrypted with the other. The private key is kept private and the public key is published to the world.

The asymmetric key approach allows for flexibility. If we both have a pair of keys, I can use your public key (which is readily available, since it is public) and only you can decrypt the message. Thus, I know only you can read the message. You can send me a confidential message by using my public key; only I can decrypt the message.

Tangentially, I can encrypt the message with my private key, and when you decrypt it with my public key, you know that the message is from me. No one else can encrypt a message that is decryptable with my public key, because they do not have the private key.

The nature of the private key is the problem with cloud computing. The cryptography system relies on a private key, a number that you keep, well, private. It remains in your domain and you must not reveal it to anyone. But cloud computing pulls computing away from a person and into "the cloud", into a system of provisioned servers that are outside of my domain.

Let's look at a simpler case: e-mail. The same concepts apply to cloud systems.

If I want to encrypt my e-mail messages, the process is fairly easy. I can program my private key into my e-mail client, and the e-mail client will use that key to encrypt all outbound messages. (Sophisticated e-mail clients allow me to enable and disable encryption on each message I send.) This all works, as long as the e-mail client is on my equipment -- remember, one cannot give the private key to anyone else.

To use encryption with a web-based e-mail system, I would have to give the e-mail system my private key. (I have to use the private key for encryption, since the public key -- the other key -- must be used for decryption. My recipients cannot have my private key, since it must remain private.) But if I give my private key to the e-mail system (whether it be GMail, Hotmail, or Yahoo! mail) then someone other than me has my private key! This is unacceptable to the encryption strategy.

For web-based e-mail, the best I can do is 1) compose the e-mail, 2) copy it to my local system, 3) encrypt the message on my local system, and 4) paste the encrypted message into the e-mail client. This allows my to keep my private key in my local domain and send encrypted messages.

You can see that this solution is a kludge, and while it works for e-mail it fails miserably for cloud applications. (Cloud apps do not always allow for the cutting and pasting of data at the appropriate points in processing.)

With the current technology of cryptography, there is no way around this dilemma. I must keep my private keys private, and use them on my equipment. (That is, equipment that is completely under my control.) To use encryption with cloud applications, I need a a local processor, a mechanism to ship data to my local processor, the ability to encrypt the data, and a means to send the encrypted data back to the cloud application.

I don't see anyone working on any of these mechanisms. Sadly, the cloud was designed with no thought for personal encryption. (Cloud apps can encrypt data between themselves, but that the is application encrypting the data with whatever means -- if any -- and only good for other cloud applications. It does not handle encryption at the user level.)

Encryption (or more specifically, the "public key infrastructure" or PKI) is useful for sending confidential messages (messages that only the recipient can read) and signed messages (messages that anyone can read but are guaranteed to be from me). We need these abilities, especially for commerce. Yet cloud computing works against these goals. Cloud computing, with its "move everything to someone else's servers" is at odds with the "private key on my processor" requirement of PKI.

And that is the conflict between cloud computing and encryption.

The Nixon of programming languages

2011-07-09T13:57:00.000-07:00

Do we need a language to kick around?

It seems that we do. From the earliest days of computing, people have been critical of specific programming languages.

Those who had learned machine language and assembly code were skeptics of FORTRAN and horrified at the output of the COBOL compiler.

When I joined the field, BASIC, Pascal, and C were in the ascendant, yet each had their fans. In the microcomputer arena, BASIC was dominant and thus admired by many and despised by many (with some folks living in both camps). Pascal and C had their followers, and there were other languages for the explorers (CBASIC and Forth). The clear winner in the "most despised" race was BASIC.

In the golden age of Microsoft Windows, the dominant languages were Pascal (briefly), followed by C, and then a tussle between Visual Basic and C++. Both Visual Basic and C++ were liked and disliked, with strong loyalties.

Sun and Microsoft introduced Java and C#, which pulled people away from the Visual Basic and C++ arena and into a new, complex dispute. The argument of language superiority was clouded by the assets of the run-time system and the backing vendor. To this day, people have strong preferences for one over the other.

Today we see discussions, with new languages Scala, Clojure, and Lua compared to Python, Ruby, and Java. But these discussions are less heated and more educational. They are civilized discourse.

My theory is that we use languages as a proxy for independence, and our arguments are not about language or compiler but about our ability to survive. Using FORTRAN or COBOL meant committing to IBM (despite the portability of the languages), and people feared IBM.

In the microcomputer age, programming in BASIC meant committing to Microsoft, but the relationship was complex. Microsoft owned the language, but Digital Research owned CP/M (the de facto standard operating system), so we had two brutes to fear.

Now that Oracle has purchased Sun and acquired Java, I expect the Java/C# disputes to increase. Sun was the rebel alliance against the imperial Microsoft, but Oracle is a second empire. Both can threaten the independence of smaller organizations.

I also expect that more people will be kicking Java. Those who want independence will look at newer languages; those who want security will look to Java or C#. It may be a imagined security, since the vendor can pull the syntactical rug out from under your project at any time; consider changes in Visual Basic over time.

The new languages have no large empire behind them. (Yes, Scala and Clojure live in the Java run-time, but they are not viewed as tools of the empire.) With no bogeyman behind them, there is little reason to castigate them. They have little power over us.

It is the power of large vendors that we fear. So yes, as long as we have large vendors backing languages, there will be languages to kick around.

The increasing cost of C++ and other old tech

2011-07-06T19:58:00.000-07:00

If you are running a project that uses C++, you may want to think about its costs. As I see it, the cost of C++ is increasing.

The cost of compilers and libraries is remaining constant. (Whether you are using Microsoft's Visual Studio, the open source gcc, or a different toolset.)

The increasing cost is not in the tools, but in the people.

First and most obvious: the time to build applications in C++ is longer than the time to build applications in modern languages. (Notice how I have not-so-subtlely omitted C++ from the set of "modern" languages. But modern or not, programming in C++ takes more time.) This drives up your costs.

Recent college graduates don't learn C++; they learn Java or Python. You can hire recent graduates and ask them to learn C++, but I suspect few will want to work on C++ to any large degree. The programmers who know C++ are the more senior folks, programmers with more experience and higher salary expectations. This drives up your costs.

Not all senior folks admit to knowing C++. Some of my colleagues have removed C++ from their resume, because they want to work on projects with different languages. This removes them from the pool of talent, reducing the number of available C++ programmers. Finding programmers is harder and takes longer. This drives up your cost.

This affect is not limited to C++. Other "old tech" suffers the same fate. Think about COBOL, Visual Basic, Windows API programming, the Sybase database, Powerbuilder, and any of a number of older technologies. Each was popular in their heyday; each is still around but with a much-diminished pool of talent.

When technologies become "old", they become expensive. Eventually, they become so expensive that the product must either change to a different technology set or be discontinued.

As a product manager, how do you project your development and maintenance costs? Do you assume a flat cost model (perhaps with modest increases to match inflation)? Or do you project increasing costs as labor becomes scarce? Do you assume that a single technology will last the life of the product, or do you plan for a migration to a new technology?

Technologies become less popular over time. The assumption that a set of unchanging technology will carry a product over its entire life is naive -- unless your product life is shorter than the technology cycle. Effective project managers will plan for change.

Looking out of the Windows

2011-07-01T19:08:00.000-07:00

According to this article in Computerworld, Microsoft is claiming success with Internet Explorer 9, in the form "the most popular modern browser on Windows 7", a phrase that excludes IE6 and IE7.

This is an awkward phrase to use to describe success. It's akin to the phrase "we're the best widget producer except for those other guys who produce widgets offshore".

I can think of two reasons for Microsoft to use such a phrase:

1) They want a phrase that shows that they are winning. Their phrasing, with all of the implied conditions, does indeed show that Microsoft is winning. But it is less than the simple "we're number one!".

2) Microsoft views success through Windows 7, or perhaps through currently marketed products. In this view, older products (Windows XP, IE6) don't count. And there may be some sense in that view, since Microsoft is in the business of selling software, and previously sold packages are nice but not part of this month's "P&L" statement.

The former, despite the dripping residue of marketing, is I think the healthier attitude. It is a focussed and specific measurement, sounds good, and pushes us to the line of deception without crossing it.

The latter is the more harmful. It is a self-deception, a measurement of what is selling today with no regard for the damage (or good) done yesterday, a willful ignorance of the effects of the company on the ecosystem.

The bottom line is that Internet Explorer is losing market share. This may not be the doom that it once was, with apps for iPhone and Android phones increasing in popularity. Indeed, the true danger may be in Microsoft's inability to build a market for Windows Phone 7 and their unwillingness to build apps for iOS and Android devices.

Run your business like an open source project

2011-06-27T15:59:00.000-07:00

Businesses must make decisions about the different activities that they perform. They often classify activities as "drivers" and "supporting tasks" (or as "profit centers" and "cost centers"). Profit centers get funding; they are essential to the business and usually have a measurable return on investment (ROI). Cost centers get funding, grudgingly. They are considered a necessary evil and by definition they cannot provide ROI.

If you want to improve your ability to classify drivers and supporting utilities, look at open source projects. Not the software, but the projects that build the software. These projects often run with little or no funding, yet they succeed in delivering usable, quality software. They must be doing something right.

Any project that delivers usable, quality software is doing several things right. From setting objectives to managing resources, from marketing to coordinating activities, successful projects are... well, successful.

Open source projects are very good at identifying the drivers in their project, and focussing on them. They are also good at separating the supporting tasks, and either outsourcing them or automating them. Through outsourcing and automation, the project can drive the cost of supporting tasks to zero. Here, "outsourcing" does not mean "hire someone else" but "use external resources" which could be open web sites like Google Docs.

Open source projects use metrics to help them make decisions. They automate testing, which allows them to run tests often (and inexpensively). The results of the tests inform them of the quality of their product.

Open source projects don't think that they need special versions of everything. if a plain version control system is usable and lets them achieve their goals, they use it. If a plain compiler gets the job done, they use it. If allowing people to work around the world on schedules of their own choosing gets the job done, the project allows contributors to work around the world on their own schedule.

Very few open source projects -- successful open source projects -- can afford egos and custom stationery.

So are they all that different from a business?

The other disappearing papers

2011-06-24T08:02:00.000-07:00

The internet age has brought about the demise of several newspapers, and threatens the entire industry. Many folks have blogged about this and even newspaper reporters have written stories about it.

But there is another set of papers that is threatened by the internet, a set of papers about which no one (to my knowledge) has worried, blogged, or reported.

That set of papers is the rental apartment advertising books.

You know these books. They live in flimsy (but apparently indestructably flimsy) boxes on streetcorners in downtown neighborhoods. The list page after page of beautiful apartment buildings, each nicer than the last.

At some point, the internet is going to eat these advertising books. As with newspapers, they are expensive to produce and distribute. Their demise is inevitable; it will be a simple business decision to ax them when they produce less revenue than cost.

Their disappearance will not cause hue and cry. There will be no period of mourning, no ceremony for their passing. They are, unlike newspapers, artifacts of pure advertising. They have no "content", in the internet/web sense.

These little booklets of advertisements are enablers of commerce, nothing more.

And when they are gone, it means that we have built other mechanisms to enable commerce, and people have accepted those other mechanisms. Those mechanisms might be web pages, or e-mail lists, or spiffy iPhone apps, or something else. But they will exist, for commerce will exist.

The demise of the paper advertisement will signal the acceptance of the on-line advertisement.

The parts are worth more than the whole

2011-06-20T21:32:00.000-07:00

Some number of years ago, I would attend computer fairs. There was a particularly nice one near Trenton, NJ. The fair had a large flea market with people selling old, used hardware (and some software). Just about anything would be available, from PC parts to old eight-inch floppy disk drives to telephone equipment to minicomputer line printers... lots of things.

One of the most frustrating things was looking at old hardware that had been built for a specific system configuration. The device (whatever it was) would have various connectors; some for data and some for power. These connectors were not standard but custom, used only by that device (and the thing to which it attached).

Such custom connectors may have made sense -- perhaps there was no standard, or the standard did not meet the needs of the system. Or perhaps the manufacturer used a non-standard connector to lock customers into their hardware. Whatever the reason, the non-standard connectors effected the components when the system was decommissioned: they were unusable.

In contrast, the IBM PC used standard parts (the IBM PC set a new standard, but it was adopted by the industry) and parts, when a system was decommissioned, were still usable. When one upgraded from an IBM PC to an IBM PC-AT, one could swap in the old video and accessory cards. One could keep the display monitor, and the printer. The parts had value after the system lost its value.

Software systems have similar behavior. Software systems are built with components and layers (or so the pretty architecture diagrams lead us to believe) but these "components" often have the equivalent of the custom connectors of the old one-system-only hardware. Such custom components can be used within the system, but when the system is decommissioned, the components cannot be used elsewhere. Thus, when the system is decommissioned, the value of the components drops to zero. A component that can be used by one and only one system has no value outside of that system.

A good system architect will see not only the system but the components, and ensure that the components (as well as the system) have value. That means that the components must use standard APIs and be able to stand alone, so other systems can use them. A re-usable component must accept data in a reasonably neutral format, not one that is specific to a system or one that uses obscure object types. A re-usable component must run with a minimal number of dependencies, not the entire system. A re-usable component must be tested independently of the one system, to ensure that it can be run independently of the one system.

Building re-usable components helps retain value of software, and also retains value of people. Re-usable components can be used in other systems. (By definition.) Using components in other systems leverages the knowledge of the development team across multiple systems. In the long term, this can reduce your staffing costs.

From the bottom up

2011-06-19T04:25:00.000-07:00

Fixing software is hard. Experienced managers of development projects have memories (often painful memories) of projects to redesign a system. Redesign efforts are hard to justify, difficult to plan, and risk breaking code that works. All too often, a redesign project ends late, over budget, and with code that is just as hard to maintain. The result is the trifecta of project failures.

A big part of the failure is the approach to redesign. Projects to fix code often use one of the following approaches:

- Re-write the entire program (or system)
- Improve code along functional components (the "room by room" approach)
- Improve code as you modify it ("fix it as we make other changes")

Re-writing the system is expensive. Often too expensive to justify. ("You're going to spend how many dollars and how many months? And the result will be the same as what we have now?" ask the senior managers. And there is no good answer.) Plus, you have the risk of missing important functionality.

Changing the system in parts, akin to replacing tires on a car, is appealing but never seems to work. Components have too many connections to other components, and to truly fix a component one must fix the other components... and you're back to re-writing the entire system. (Except now you don't have the money or the budget for it.)

Fixing code as you make other changes (as you are implementing new functionality) doesn't work. It also has the "dependency effect" of the component approach, where you cannot fix code in one module without fixing code in a bunch of other modules.

Managers are poor at planning redesign projects, because they manage software like they manage the organization: from the top down. While top-down works for people, it does not work for code.

People organizations are, compared to software, fairly simple. Even a large company such as Wal-mart with its one million employees is small, compared to software projects of many million lines of code. Lines of code are not the same as people, but the measures are reasonable for comparison. Any line of code may "talk" with any other line of code; that is not true for most organizations.

People organizations are, compared to software, fairly smart. People are sentient beings and can act on their own. A corporate manager can issue directives to his reports and expect them to carry out orders. Software, on the other hand, cannot re-organize itself. Except for a few specialized AI systems, software must be guided.

I have worked on a number of projects. In each case, the only method that worked -- really worked -- was the "bottom up" approach.

In the bottom up approach, you start with the smallest components (functions, classes, whatever makes up your system), redesign them to make them "clean", and then move upwards. And to do this, you must know about your code.

You start with a survey the entire code. You need this survey to build a tree of the dependencies between functions and classes. Use the tree to identify the code at the bottom of the tree -- code that stands alone and does not depend on other code (except for standard libraries). That is the code to redesign first. Once you fix the bottom code, move up to the next layer of code -- code that depends only on the newly-cleaned code. Repeat this process until you reach the top.

Why the "bottom up" method for software?

Software is all about dependencies. Fixing software is more about changing dependencies than it is fixing code. Redesigning software changes the dependencies; improving code in the middle of a system will affect code above and below. Changes can affect more areas of the code, like ripples spreading across the surface of a lake. Starting at the bottom lets us change the code and worry about the ripples in only one direction.

Why not start at the top (with ripples in only one direction)? Changes at the top run the risk of leading us in the wrong direction. We think that we know how the software should be organized. But this impression is wrong. If we have learned anything from the past sixty years of software development, it is that we do not know, up front, how to organize software. If we start at the top, we will most likely pick the wrong design.

On the other hand, starting at the bottom lets us redesign the smallest components of the system, and leverage our knowledge of their current uses within the system. That extra knowledge lets us build better components. Once the smallest components are "clean", we can move to the next level of components and redesign them using knowledge of their use and clean subcomponents.

If anything, starting from the bottom reduces risk. By working on small components, a team can improve the code and make definite gains on the redesign of the system. The smaller objectives of bottom-up redesign are easier to accomplish and easier to measure. Once made, the changes are a permanent part of the system, easing future development efforts.

Software is not people; it does not act like people and it cannot be managed like people. Software is its own thing, and must be managed properly. Redesign projects from the top down are large, expensive, and risky. Projects from the bottom up have a better chance of success.

Modern times for websites

2011-06-15T18:44:00.000-07:00

I visited a website recently; it recommended using Internet Explorer version 6.0 or higher. I found the idea nostolgic and amusing.

The website was clearly designed several years ago, when there were good reasons to require specific browsers.

But today, such a statement makes one look foolish. Firefox, Safari, Chrome, and Opera all perform well. Internet Explorer's share of the market has dropped, and one must design a website for the bunch, not the one.

There is no good reason to design a website for a single browser, nor demand IE6 as a minimum. Even Microsoft is requesting that people move on from IE6!

Websites are dynamic, and even websites with static content must change. They must grow with the standards or die.

Better programming with immutable objects

2011-06-11T14:11:00.000-07:00

Recent buzz in the development community has discussed functional programming languages. These languages have a different approach to the organization of code. They are not object-oriented languages, nor are they procedural. Functional programming languages include Haskell, Erlang, and F#.

Working with old-fashioned object-oriented languages (C++ and C#), I suffer from language envy. I can see the shiny new languages but use the old, dented languages for my daily work. (Of course, I recognize that at one time C# and even C++ were the new, shiny languages. I also recognize that at some point in the future, Haskell and F# will be considered the old, dented languages. But for now, the grass is greener over the fence and in the yard of the new languages.)

I may be suffering from envy, but I don't sit and curse the darkness. Instead, I try to use the techniques of the new languages in my current toolset. I did this when programming in C and gazing longingly at C++; I wrote code in a style that I called "thing-oriented" programming. My thing-oriented programming let me identify key objects in the code (data structures, not objects in the OO sense) and collect and organize functions by the objects they manipulated. It was a step towards object-oriented programming. (Eventually I did learn C++ and work on C++ projects.)

With the introduction of C# and .NET, I used the same idea. For this transition, I wrote class libraries in C++ that mimicked the class libraries in .NET. (Not the entire .NET collection, just the basic classes to get the work done.) The work was a bit difficult, as we were not allowed to use STL. But like "thing-oriented programming", it was a step towards .NET and made the transition easier.

So here I am again, and this time the target is functional programming languages. I want to transition to functional programming from object-oriented programming, but don't have the tools. (The day job uses "classic" OO languages, and I struggle to find time outside of the office.)

My transition technique is to use "immutable object programming".

This has some interesting effects:

1) To construct an object, all data must be present. I cannot instantiate an object and later give it more information. Once an object is instantiated, it is complete and final. (I can construct other objects from built objects, so I can construct a "sales price" object and later use it to construct a "discounted sales price" object. But I cannot wedge in a discount to the "sales price".

2) I find myself thinking about the sequence of operations and the dependencies between objects.

3) My programs are designed in three phases: collect data (and construct objects), construct other objects from the collected data, and produce output. This is a pattern that goes back to the beginnings of the programming age. It may be a natural fit for the "immutable object programming" style, or it may be driven by the calculations of the systems.

4) My thinking is more rigorous. Not necessarily up front, but the end result (the final code) is more disciplined. I write some code, and sometimes take non-immutable shortcuts (or even OO shortcuts). These shortcuts are obviously wrong, but they yield the correct output. Once I have the correct output (and tests to check that output), I revise the code and remove the shortcuts.

5) My code is easy to debug. When the output is incorrect, I know that an object has incorrect data. (Or possibly the output routine is incorrect.) If the object has incorrect data, I know that it was constructed improperly. (There is no other method that modifies the object.) Finding the problem is a simple matter of identifying the incorrect object and then identifying the construction of that object.

6) My code is easy to change. I have made a number of changes to the code, and all have been simple and obvious. (And therefore, they worked.) These were not trivial changes, either; some were large and affected multiple modules.

I will admit that I do not understand functional programming languages. I have created some small programs in Haskell, but not enough that I "grok" the concept. My "immutable object programming" trick may be a step towards functional programming or it may be a step away from functional programming. It feels right, and I like the results.

The tweet is more powerful than the sword

2011-06-08T19:52:00.000-07:00

People are surprised that companies respond quickly to complaints and questions posted to them on Twitter.

I find the rapid response predictable, once you think about it.

E-mails are, for the most part, one-to-one messages. A complaint e-mail especially so. I'm not going to CC: friends on a complaint e-mail to my bank, or my insurance company, because that simply loads their inboxes with clutter. An e-mail is a private conversation between you and the company. The company can delay response or ignore it completely at practically no cost.

Tweets, on the other hand, are a one-to-many message. They are not a letter to a friend (or business) but are published to many people. They are viewed by one's followers, and some people have hundreds of thousands of followers.

If Twitter were that simple, then companies could evaluate the importance of the Tweeter and answer only those tweets from users with large followings.

But it's more complex. All tweets are part of the TwitterStream, and anyone can look at any message (just about). I recently tweeted about my bank and used a hashtag to identify them. The bank was not following me (and is still not following me, from what I can tell) but they found the message anyway, and responded. Tweets are public postings, messages published to potentially the entire world. Tweets with hashtags are advertisements, not approved by the marketing department.

With messages going out to the globe, a company has no choice but to respond, and respond quickly. If they don't, the next tweet might just be "hey, #companyX ignored my tweet. is anyone home?", which translates to more negative advertising.

I think every company, organization, and government office should be watching the TwitterStream for messages about themselves. You want to know what people are saying about you.

Windows 8 shows Microsoft's big problem

2011-06-02T18:57:00.001-07:00

We have seen the future, and for Windows, the future looks a lot like Windows Phone 7. Microsoft has let folks see glimpses of the not-yet-released version of Windows. Some have raved and others have yawned.

A long time ago, Windows was created with three goals in mind: provide a graphical interface for PCs, counter the threat from from the new Macintosh computer, and hide the command line interface from users.

Windows 8 continues those objectives. It provides a nice-looking interface, it counters the recent advances made by Apple, and once again it hides complexity from the user.

But Windows 8 is not Microsoft using the same strategy as Apple. For the iPods and iPhones, Apple created a new operating system, a new user interface, and a new way of computing. It made a clean break from the Mac OSX line. Microsoft, in contrast, is betting on backwards-compatibility, and Windows 8, with its shiny interface, can run older Windows programs.

Backwards compatibility is a blessing and a curse for Microsoft. It allows their customers to move gradually to the new operating system, keeping their existing applications. Yet it means that Microsoft must support a lot of complicated (and sometimes just plain wrong) APIs. (For example, the Windows Registry will be present in Windows 8, despite almost universal hatred of the thing. But too many programs depend on it, and Microsoft is stuck with it.)

Microsoft has trained customers to expect backwards compatibility.

Apple has trained its customers to expect new platforms. The history of Apple products in littered with abandoned platforms: Apple II DOS, the original Mac OS, the later Mac OS version 6 through 9, the Motorola processors, and the Power PC chips. When Apple introduced a new iOS operating system for the iPod and iPhone, no one blinked.

Microsoft could never get away with anything like that. The release of Windows Vista with its limited support for devices brought howls.

Microsoft's fate, like it or not, is tied to Windows. Which means that Microsoft must drag Windows to any market that it chooses to enter. The XBOX runs Windows, the Zune runs (OK, ran) Windows, the Media Center runs Windows... everything Microsoft does runs Windows.

Or to put it another way, Microsoft does everything with Windows. And the consequences of that are... Windows must be extended, stretched, folded, spindled, and mutilated into any new product.

Microsoft's growth is limited by its ability to expand Windows. And that is Microsoft's big problem.

Fast food software

2011-05-31T19:12:00.000-07:00

Our industry of software development has used the "large software" model for development. By "large software", I mean software that costs a lot. Not just in purchase cost (or support licenses) but in care and feeding.

I recently attended a science fiction convention. I noticed that the con used little in the way of tech. The folks running the con used computers, but few attendees used them. The con made no effort to interact with people via tech. The con had a static web site and a PDF with a list of events.

In contrast, the O'Reilly conferences use tech to involve people. They have a web site. You can create a profile. The web site has a schedule of sessions, and you can create your personal schedule, picking those sessions that interest you.

The difference between the two organizations is technical firepower. O'Reilly has the wherewithal to hire folks and make the interaction happen. Science fiction conventions are often volunteer-run and have little technical expertise on staff.

I suspect that a lot of small organizations, companies, and government agencies are in similar situations. They probably use PCs with Windows, MS-Office, and Internet Explorer... because that's the software that is on the PC when they take it out of the box. Small organizations don't have IT support teams and cannot adopt today's high-tech solutions (the effort outstrips the one guy who sets up the PCs).

Small shops are too often run on a bunch of PCs, a router, and shared spreadsheets. (Or worse, passed-around spreadsheets, and we're not really sure who has the latest changes.) They don't have the expertise to install and support custom-made systems. (I suspect that many small shops don't have the experience to install MS-Office and track licenses and activation codes.)

Smartphones and tablets (and the cloud) are a possible solution here. The model provided by smartphones and tablets is a good fit for these small organizations. Installation of software is handled by a single click. (No license, no activation code, no special instructions.) Smartphone and tablet software is also priced within the budget of a small organization. The local Mom-and-Pop store will never, ever, purchase and install an ERP system (nor should they) but they can record expenses on their phones.

The business model for small software is different from the business model for large software. Large software is like a dinner at a high-end restaurant: You have personal service and you meal is prepared according to your specifications. Small software is fast food: You pick a set of items from a limited menu and your meal is handed to you from a stack of prepared items, with little or no personalization. It is not elegant... but it is fast, predictable, and cheap.

With the right combination of front-end software (on tablets and phones) and back-end software (in the cloud), small organizations will be able to use tech and involve their customers. (And involve their customers in a way that is easy for customers to use.)

There is a market here. It's different from the current market, in terms of tech and business. It is a high-volume, low-margin market. And for those who exploit it capably, very profitable.

Tablets and cloud, perfect together

2011-05-30T18:45:00.000-07:00

If you want to see the future of computing, look at tablet computers and cloud computing. These two technologies complement each other and provide exciting opportunities for new applications.

Tablet computers (riding on wi-fi and cell networks) provide computation power to people when they want, where they want. Cloud computing can provide the support for tablet applications, either for music or video or plain ole' apps. Alone, either platform is nice but not so hot. Combined, the two will do well.

The two technologies will mean excitement for developers, too. Cloud apps are different from web apps and PC apps, and I believe a new language will be selected for the standard for cloud apps. (Just which language is not clear at the moment, but I suspect a functional programming language.)

I don't see PC apps or web apps disappearing, but i do see less emphasis on them. I suspect that we have seen all of the major PC apps, and that only custom, highly specific applications will be developed for PCs. There will be no new "killer app" for PCs, no new "Lotus 1-2-3" or "Word", no new PC app that everyone must have. We may see new apps for the (non-tablet) web, but every new web app will have a tablet/phone counterpart.

The tablets (and smartphones) rule! We will see new tablet/phone apps backed by the cloud. Some will have web counterparts, but many will not.

The tablet/cloud future awaits!

Build on clouds

2011-05-26T19:50:00.000-07:00

Folks have put effort (some folks lots of effort) into making the cloud look and behave like plain old PC applications. Not just the look and feel of apps, but the techniques to build those apps. Microsoft has designed their tools to make the development of cloud apps a lot like the development of PC applications. (Their tools also make the development of web apps similar to the development of PC apps.)

I think that this is the wrong approach.

I recognize the benefits of similar environments and tools. A single method is easier to learn and support. One can move from development on one project to development on another (different platform) project.

But there is nothing in the PC app development model that makes it the right model for all platforms.

The PC app toolset has a long history. We started with the tools and techniques of the mainframe and minicomputer platforms, but didn't keep them. We adopted the notions of compilers, interactive command lines, and version control, and then built our own tools that leveraged PC resources: the IDE, interactive debuggers, and GUI designers. We abandoned the processes of desk-checking software and symbol cross-references; we built modern processes, from code reviews to automated testing.

With cloud computing, expect new methods and tools that leverage the resources of the cloud. You won't use the new techniques if you stay in the PC mindset.

If you must have a single development platform, make it the cloud. That is, make the process and tools for the development of PC apps look and feel like the process for the development of cloud apps. Instead of adapting the cloud to the PC app process, create a process for cloud apps and then adapt it to the PC app process.

Files are out, web services are in

2011-05-25T16:15:00.000-07:00

The trusty file has been with us (that is, us PC users) since before the dawn of time. (If we consider the "dawn of time" to be the introduction of the IBM PC.) It has been the workhorse of data containers, going back to the elder days of CP/M and DEC PDP-11 computers.

While files have served us well, they are poor citizens of cloud computing. Files insist on being, on existing, at some location (usually on a disk). Because they have a known physical location, they are problematic for the cloud. A file is a singular thing, existing on a singular host. Even files on a device such as a SAN are on a singular host -- the SAN, arguably, is a host. A single point of access is a bottleneck and a threat to performance.

But it turns out that we don't need files. Files are nothing more that a collection of bytes. Some consider them a stream. Others will think of them as a collection of not bytes but characters.

We need collections (or streams) of bytes (or characters). But we don't have to house those collections in files.

In the cloud, we can use web services. Web services can provide collections of bytes (or characters) just like files. Web services can be distributed across multiple hosts, eliminating the bottleneck of files. Unlike files, web services are identified by URIs, and this gives us flexibility. A "file" provided by a URI can come from one server or from one of many (although you must ensure that all servers agree upon the contents of the file).

The design of a classic PC application involved the specification of input and output files. The design of a cloud application must involve the specification of web services.

Discoveries about discovery

2011-05-22T04:48:00.000-07:00

Recent developments in tech have created an automated process to handle "discovery", the process of reviewing materials for a legal case.

One might think that law firms will adopt the technology, as a way to reduce costs. Or one might think that law firms will *not* adopt the tech, believing that they are traditional and unwilling to change. Or perhaps one might think that law firms are risk-averse, and do not want to try new tech that could miss something and cause the loss of a case.

I have a different outlook. I think law firms will avoid the tech of automated discovery, for economic reasons.

Law firms use the time-and-materials business model. They bill by the hour: the more hours, the higher the bill. Law firms have constructed their hourly rate to cover their costs (including labor) and to provide profit. Thus, each billable hour generates profit (not just revenue) for the firm. A reduction in labor (billable hours) equates to a reduction in profit.

Industries that have adopted technology (specifically to reduce labor hours) are industries that sell products with prices fixed by the market. Automobiles, hair dryers, books, computers... these are all sold at the market price. Profit is revenue less the cost of goods and manufacturing. The costs of labor and goods does not drive the price, and a manufacturer must manage costs to live within the market price. A reduction in labor hours equates to an increase in profit.

Thus, we can expect that businesses selling goods (or services) at prices dictated by the market will adopt cost-reducing techniques and technologies. They will computerize accounting systems, automate assembly lines with robots, and outsource software development.

We can also expect that businesses selling services on a time-and-materials basis will *not* adopt technologies that reduce labor hours. (They may adopt techniques that reduce labor costs, replacing highly paid workers with lower-wage workers. But a reduction in billable hours is unlikely.)

What of this can we apply to software development?

The software development industry is a complex one. Some software is created and sold on a time-and-materials basis. Some is sold in the market. Some is given away. Some software that is sold is not sold in a free market; Microsoft enjoys a monopoly position with Windows and Office. (A monopoly that is perhaps less strong now than ten years ago, yet still part of the complexities of the market.)

Companies that build software with the time-and-materials business model have little incentive to reduce the workload in their projects. These projects benefit from high labor efforts, and therefore we can expect them to use little in the way of effort-reducing techniques. Companies that build software for sale on the open market have strong incentives to minimize their costs. Cost reduction methods include:

outsourcing
agile development (pair programming, automated tests)
modern languages (Python, Ruby, Scala, Lua, Haskell)

Companies in the time-and-materials business model do not need such cost-reducing measures. If you're not working with the above techniques, then you're probably on a time-and-materials project. Or you're in a company that will soon be out of business.

Cannonballs and fence posts

2011-05-14T05:35:00.000-07:00

Imagine a world that has unstoppable cannon balls. Once launched, the cannonball is not stopped, or even slowed, by anything it encounters. It continues on, even impervious to friction. Also imagine that this world has immovable fence posts. Once placed, and immovable fence post cannot be moved by any means. What happens when an unstoppable cannonball collides with an immovable fence post?

The riddle is not unrelated to the announcement of Google's Chromebook and its automatic updates.

Unlike Windows and Mac OSX, the Chromebook applies updates without asking for permission -- it finds them and loads them as a matter of its general operation. Windows and OSX (and even Linux) follow a different model, and inform the user of an update but allow the user to decline the update (or at least defer it).

System administrators for large corporations (and even medium-size ones) want the latter model. They want to ensure that their operations continue, and they want control over updates. A good systems administrator will test updates on a few systems before releasing it to the entire corporation.

But individuals (and possibly small companies) want automatic updates. For them, the workload of monitoring, testing, and releasing updates is a burden. They choose to trust the supplier, and gain the time and effort that would go into verifying updates.

So here we have the two opposing forces: automatic updates (the unstoppable cannonball) and controlled updates (the immovable fencepost).

The answer to the riddle is a bit of a let-down: It is not that one overpowers the other, but that the question is not valid. If such a thing as an unstoppable cannonball exists, then by definition there can be no such thing as an immovable fence post. (And vice-versa.)

The answer to the current debate about Chromebook and automatic updates is less clear. I expect that individuals will look favorably on automatic updates and large enterprises will continue to use the "test and then apply" method. I think the two can both exist in our world.

Look for consumer devices to adopt automatic updates. Look for commercial software to stay with manual updates. Software that bridges the two worlds (Linux) will allow users to set the update method.

Are you an A-list programmer? Do you care?

2011-05-11T20:21:00.000-07:00

Programmers are a diverse lot. Not merely in gender, ethnic background, or age, but in programming talent. Tests of programming skills show a large difference between the best and worst programmers: some tests report a factor of twenty-five.

Such a large difference may offend the sensibilities of some managers. (Especially managers in organizations that consider programming staff to be interchangeable cogs.) Yet I think that we can agree that some programmers are better than others. I classify programmers into three tiers.

The A-list programmers are those few who are superb at the craft. Not only productive, they are knowledgeable and creative. These are the folks who create Twitter and FaceBook and Amazon.com. They do not fit in to large, stodgy bureaucracies; the only large companies they stay with are the ones that they found. These folks work with the cutting edge languages. Today, that list is: Haskell, Scala, Lua, and perhaps Python. These programmers are mobile. If the A-list programmer does not like his current work, he moves on to something more appealing.

The B-list programmers are hard-working and passionate, but they work at a less entrepreneurial level than the A-list folks. They don't found companies, but they do have lots of ideas and creativity. They don't use the cutting-edge languages, but do use modern languages such as Ruby, Python, Objective-C, or Perl. They are not as mobile as the A-list programmers, but they do move from opportunity to opportunity.

The C-list folks are the determined programmers. They are not entrepreneurs, and they don't have ideas that are burning to get out. They are quietly sensible, and know that the paycheck is there to pay the rent. These folks avoid the (chaotic) start-ups and are comfortable with the (predictable) large bureaucracy. They use the languages that the corporation deems "acceptable": Java, C#, or maybe C++ or Visual Basic. (The last two are from legacy systems which must be maintained but not converted to new languages.) These folks move rarely, being risk-averse and preferring the known devil to the unknown demon.

The difference in performance matches the risk tolerance of the company. Small start-ups require talented performers, and cannot survive mediocrity. Large corporations find it difficult to tolerate creative folks who break the rules, and prefer conformance over performance.

I am sure that there are exceptions to these broad categories. The correlation of performance and language selection is yet to be shown, and some projects will require the use of an older language. But I think this alignment is pretty accurate. The top performers are where they are because they pick the best tools -- and languages -- for the job. The worst performers know only one language, try to solve every problem with it, and have picked a language that is "safe" (one that matches lots of job openings). They are not on the cutting edge of programming.

Where you fit into this scheme is, for the most part, up to you. If you are willing to take risks and are creative, you can be an A-list programmer. (You need to be a good programmer, too.) If you prefer a safer solution, a large company with older technology is a better fit for you. (Although I don't know that larger companies are safer, in these days.)

Constants aren't, variables won't

2011-05-10T19:40:00.001-07:00

We like to think that standards are fixed and unchanging. Yet they shift and change over time, like sands on the shore.

Hardware has certainly changed. In fact, it has changed so much that today's PCs share nothing with the original IBM PC. Take any part of the today's PC and you find that it will not connect or attach to the original IBM PC. The keyboard is different (USB was not available on the IBM PC), the display is different (the IBM PC had monochrome or CGA, not VGA and certainly not DVI), the internal cards use a different buss... the list goes on.

The software platform has changed, too. From DOS, to Windows 3.1, to Windows 95, to Windows NT, and on to Windows XP and now Windows 7. While some early PC programs still run under Windows 7, no one uses them for serious work.

Development tools have changed. Original PC programs were written in assembly language. The early versions of Windows used Pascal. Later versions used C and then C++. Now The dominant language is C#.

Our standards change. Our tools change. With this environment, we must adopt development practices to accommodate change. Yet too many shops treat systems as fixed solutions: define the requirements, build the system test it, and release it. They allow for defect corrections and enhancements (often called "maintenance") but they do not plan for changes to platforms or tools. Instead, they wait for new platforms and tools to become dominant, and then they rush to bring their system to the new environment. People rushing and working under pressure to mutate a system into a new form; the results are often dismal.

Forward-looking shops will prepare for change. They will encourage people to learn new technologies. They will strive for clean designs. They will monitor changes in technology and tools. They will look outward and observe others in their industry, in other industries, and in academia.

One does not have to predict the future precisely. But one must be prepared for the future.

It's a poor atom blaster that doesn't scale

2011-05-03T19:51:00.000-07:00

In Isaac Asimov's "Foundation" novel, the character Salvor Hardin utters the phrase "It's a poor atom blaster that doesn't point both ways.". The same can be said of "scaling" the process of adjusting an application or system to handle a different workload.

Most people think of scaling in the upward direction. Having built a prototype or proof-of-concept system, they ask themselves "will this design 'scale'?", meaning "will this design hold up under a large workload?". Frequently the system under discussion is a database, but it can be a social network or just about anything.

Yet scaling works in two directions. One can create a system that works well for a single user but fails with many users, and one can create a system that works for many users but fails for a small number. And the system does not have to be a database or even a system at all.

One example is Microsoft's products for version control. Let's consider the Visual SourceSafe and Team Foundation Server products. The former is (was, as it is no longer sold) usable by small teams, perhaps up to twenty people. Beyond that, it is hard to manage user rights and the underlying database. The latter (Team Foundation Server) on the other hand is built for large teams -- but is unsuitable for a small team of less than five. TFS has a fairly heavy administration cost, and it sucks up time and energy. A large team can afford to dedicate a person or two to the administration; a small team cannot.

Languages are also affected by scaling. Microsoft's Visual Basic languages (prior to VB.NET) were very good for small Windows applications. Simple applications could be made quite easily, but complex applications were harder. Visual Basic had some object-oriented constructs, but it was mostly procedural and one needed a lot of discipline to create a middling-complex application. Also, Visual Basic had some behaviors that limited the complexity of applications, such as its inability to display modal dialogs within modal dialogs.

Microsoft's Visual C++, in contrast to Visual Basic, was better suited for complex applications. It had full object-oriented constructs. It allowed any number of dialogs (modal or non-modal). But it had a high start-up cost. A single person could use it, but it was better for teams.

Scaling down is important for languages. Developers need tools to perform all sorts of tasks, and some applications are sensible for single users. Microsoft's tools are designed for large problems with large solutions implemented by large teams. Java is in a better situation, with language hacks like Scala and Groovy.

The current crop of scripting languages (Perl, Python, and Ruby) scale well, allowing single developers, small teams, and large teams. (Perl perhaps not quite so easily with large teams.)

With the rise of smartphones, apps have to move to smaller screens and virtual keyboards. (And virtual keyboards make a difference -- one does not type a term paper on a cell phone.) The assumptions of large screen and full keyboards do not apply in the cell phone arena.

Scaling down is just as important as scaling up. Scaling applies to users, data, and developers. Not every application, system, and development effort must scale from smartphone to cloud, but the system designers should look both ways before choosing their strategies.

The peak of PC apps

2011-05-01T16:42:00.000-07:00

There has been talk about "peak oil", the notion that oil production peaks at a specific point and then declines as supplies are depleted. I think that there is a similar notion of "peak software" for a platform. When a platform is introduced, there is some initial interest and some software. As interest in the platform grows, the number of applications grows.

PC applications fell into five categories: office applications (word processing, spreadsheets, presentations), development (compilers, interpreters, debuggers, and IDEs), games, and specialty applications (CAD/CAM, Fedex and UPS shipping, mortgage document preparation, etc.), and web browsers.

The first (office applications) has generic software, something that everyone can use. The second (development) is software used by the tool-makers, a small but diverse (and enthusiastic) crowd. Games, like development tools, appealed to a small but diverse crowd. Interestingly, web browsers are used by many folks; as a tool to get to a web page or web app they are not the end but a means.

The rise of PC applications started prior to the PC, with the microcomputers of the late 1970s: the Apple II, the TRS-80, the Commodore PET, and others. Dominant programs at the time were compilers and interpreters (BASIC, mostly), word processors (Wordstar), spreadsheets (Visicalc), games, and some business software (accounting packages).

PC applications increased with the introduction of the IBM PC (Lotus 1-2-3, Wordperfect, and more business applications).

But something happened in the late 1990s: the number of applications stopped increasing. Part of this was due to Microsoft's dominance in office applications and development tools (who wants to compete with Microsoft?). Another part of this change was due to the appearance of the world wide web and web applications. I like to think that the "best and brightest" developers moved from PC development (a routine and dull area) to the web (a bright, shiny future).

Fast-forward to 2011, and the development of PC applications has all but stopped. Aside from updates to existing applications, I see nothing in the way of development. And even updates are few: Microsoft releases new versions of Windows and Office, but what other general software is updated?

I expect that the home PC market is about to collapse, with most folks moving to either game consoles or smartphones -- or maybe tablets. The few remaining members of the home PC community will be the folks who started the PC craze: hobbyists and hard-core enthusiasts.

Businesses will continue to use PCs and PC applications. They have large investments in such applications and cannot easily transition to smartphones and tablets. Their processes are set up for PC development; their standards define PCs (and mainframes) but not tablets. And tablet apps are very different from PC apps -- as PC apps are different from mainframe apps -- so the transition will require a change not of hardware but of mindset.

It's clear that Microsoft is prepared for this transition. They have geared their offerings for businesses, home game users, and home web users. They are prepared for the disappearance of the home PC.

Apple, too, is prepared for this change, with it's line of iPhones, iPods, and iPads.

The real question is: what happens to Linux? Linux has a "business model" that is built on the use of PCs, usually old PCs that are too small for the latest version of Windows. The hobbyists and enthusiasts may want to use Linux, but how will they get hardware? And without hardware, how will they run Linux?

Speaking in tongues

2011-04-30T11:51:00.000-07:00

In the 1960s, Gerald Weinberg wrote "The Psychology of Computer Programming", in which he observed that programs are not only written but read (by the author and other programmers). He also observed that programming languages are written but not spoken. No one ever "says something in Fortran".

It is an observation that was true at the time, yet I think we now have programming languages that can -- almost -- be considered a spoken language.

The early languages of FORTRAN and COBOL were meant to be written and read (silently). FORTRAN had the terse syntax that persists in most languages today; COBOL had a wordy syntax with the explicit purpose of readability. But no one intended for people to read COBOL out loud. It's wordiness was for the benefit of programming managers and interchangeable programmers.

FORTRAN offered a basic loop with the code:

DO I = 1, 10

BASIC improved upon that syntax with:

FOR I = 1 TO 10

Notice that in BASIC, the label denoting the end of the iterating block has been removed. (BASIC uses a separate statement, NEXT, to mark the end of the block.)

C and C++ use the "for" loop:

for (int i = 0; i < 10; i++)

But all of these are in the realm of iterating a known number of times. BASIC, C, and C++ also have "while" loops, to iterate while a condition is true. But "for" loops and "while" loops still require a fair amount of thinking about the looping. A different notion of iterating is "for each of these things", and multiple languages have such constructs.

C++/STL introduced the concept of iterators to the language, and allowed for more concise (and consistent) notation for iterating over a set of objects. The STL provided more than a set of convenient classes -- it provided a set of common idioms for frequently-used constructs. These common idioms allow a person to translate the code

for (Object::const_iter iter(objects.begin()); iter != objects.end(); iter++)

{

iter->do_something()

}

into "for each member of the set of objects, do something". Yet this approach of common idioms still requires familiarity with the language and the work of translation from the C++ idiom to the English.

Microsoft has not ignored this trend. Visual Basic had (and C# has) the "foreach" keyword for iteration, and a recent update to the .NET Framework introduced the Enumerable.Range(start, count) iterator. The latter is a bit clumsy, but serviceable.

The Ruby language uses the method "each()" to iterate over a collection. This is eminently more readable than the C++/STL version: it requires no translation of idiom.

These constructs allow us to read the program -- out loud -- more easily and with less translation.

Speaking a program -- perhaps the proper verb is "orating" -- is important. Our brain's speech centers process information differently that the visual centers. When we orate, we think differently than when we read to ourselves. This different thought process helps us identify errors in our logic. (I suspect that this effect is at work when we explain a failing program to another programmer and during the explanation we understand the failure. I think it also is at work with the pair-programming concept of Extreme Programming and Agile Development, helping programmers identify errors early.)

I don't expect programming languages to be used for grand poetry (or even bad poetry) any time soon. But the concept of orating a program may be within reach.

Drucker's Moron

2011-04-27T19:35:00.000-07:00

In the 1960s, Peter Drucker wrote a series of essays on management and technology. In one essay, he made an observation that holds today:

We are beginning to realize that the computer makes no decisions; it only carries out orders. It's a total moron, and therein lies its strength. It forces us to think, to set the criteria. The stupider the tool, the brighter the master has to be... (the emphasis is mine)

The problems we have with computers, even to this day, have this concept at their core.

Our programming languages have improved in their abilities to structure operations, our data representations have become self-describing, and our operating systems have gained fancy GUI interfaces. Yet the fundamental problem is this: the computer is only as good as its instructions, and the quality of those instructions depends on the people writing the programs, their understanding of the technology, and their understanding of the business needs. People who don't understand the technology will produce programs that operate inconsistently and poorly. People who don't understand the business need (whether it be a general ledger, a database, a word processor, or a web search engine) will produce a program that at best solves the wrong need.

The quality of our people define the quality of the solution. Good programmers (and GUI designers, and database architects) will be able to create good solutions. A good solution is not guaranteed, only allowed. On the other hand, poor programmers (or GUI designers, or database architects) will yield a poor program -- there is no possibility of a good result.

Single platform is so 90s

2011-04-24T07:44:00.001-07:00

Ah, for those days of yesteryear when things were simple. The software market was easy: build your application to run under Windows and ignore everyone else.

Life is complicated these days. There are three major PC platforms: Windows, OSX, and Linux. Targeting an application to a single platform locks you out of the others. And while Windows enjoys a large share of desktops, one should not ignore a platform simply because its market share is small. All it takes to ruin your single-platform day is for a large potential customer to ask "Does it run on X?" where X is not your platform. Telling a customer "no" is not winning strategy.

But life is even more complicated than three desktop platforms. Apple's success with iPhones and iPads, RIM's success with BlackBerry devices, and the plethora of Android phones (and soon to be tablets) has created more platforms for individuals. These new devices have not worked their way into the office (except for the BlackBerry) but they will soon be there. Androids and iPhones will be there shortly after a CEO wants to read e-mail, and declines the IT-proffered BlackBerry.

The single-platform strategy is no longer viable. To be viewed as a serious contender, providers will have to support multiple platforms. They will have to support the desktop (Windows, OSX, and Linux), the web (Internet Explorer, FireFox, Safari, and Chrome), smartphones (iPhone, Android, and BlackBerry), and tablets (iPad and Android). Their services will have to make sense on these different platforms; stuffing your app into a browser window and claiming that it works on all platforms won't do it. Your customers won't buy it, and your competitors that do support the platforms will win the business.

Web presentations for everyone

2011-04-20T08:29:00.000-07:00

And now for a little bit of news about nifty new software.

The software is "Big Blue Button", a web conferencing package.

What is the Big Blue Button?

Big Blue Button (http://code.google.com/p/bigbluebutton/) is a collection of software that lets one host presentations on the web, much like WebEx or GoToMeeting. The presenter can provide audio and video, and even share his desktop. More than that, attendees can ask questions (either in text message or audio) and can chat through IM to the presenter or each other.

Neat feature: real-time translation of instant messages. Big Blue Button uses the Google Translation API and lets people chat across language boundaries. If I set my language to "English" and you set yours to "Spanish", I can type a message in English and you see the translated version. (You can also see my original English version.)

The one feature that is needed: recording. Other products in this arena let one record the meeting for playback later. The folks at Linux-ETC recognize the need and are working on it.

Is Big Blue Button ready for everyone?

The presentation I attended had a few problems, one significant enough to crash the software on the presenters PC. The software is perhaps not quite ready for prime time. Large, respectable, and stodgy corporations will probably choose the safer option of WebEx. But smaller teams (and start-ups) may want to look at Big Blue Button.

How will Big Blue Button change things?

Big Blue Button reduces the cost of on-line presentations, and provides another method for coordinating remote teams. It makes it easier to share knowledge, and the startup investment is small. Therefore, Big Blue Button will make things easier for companies to out-source projects. If you are running your off-shore projects with e-mail, voice-mail, and audio conference calls, you may want to look at Big Blue Button's capabilities.

Developers working at home may want to neaten their office-in-the-home. A disorganized office sounds just as good as a well-organized office, but video changes the game. People get professional head shots for LinkedIn and Facebook to maintain their brand. Two-way web audio/visual connections will create the need for professional-level studios (or something that looks like a professional studio). One will need a proper background, a good microphone and webcam, lighting that is flattering, and a way to block external noise like street traffic. I expect that we will see a small industry of "video consultants" to set up office studios and home studios.

The 3% solution

2011-04-17T15:48:00.000-07:00

Banks, at least in the old days of the 1980s, did not want all of their customers to pay their credit card bills. They wanted a default rate of something greater than zero. Banks wanted about three percent of their credit card customers to default -- to make no payments.

Their logic was as follows: The bank could enact strict credit checks and lend money to only those customers who would pay them back. But doing so would reduce the number of customers with credit cards, and thereby reduce the profits from credit cards. If banks loosened the rules and allowed more customers to get credit cards, some would default -- but only a small number of customers. Many of the "new" customers would pay, even though their credit rating was not so great. More customers would repay loans than would default, and the bank would earn more profits overall. Allowing for some failure increased the total profits.

In software development, we strive to reduce the number of defects in software. In the management of software development projects, we strive to control the cost, the delivery schedule, the features delivered, and the quality. These are all good objectives, but sometimes we can focus too much on the process and lose sight of the end goal.

Organizations enact policies to control projects. Some organizations have informal policies; others have stringent policies. Some projects (such as control systems for atomic energy plants) need very specific policies. But not every project requires such minute specification and control.

Yet some organizations push for (and carry out) very specific procedures, in an attempt to eliminate all variations from the project plan. This is equivalent to a bank limiting credit cards and loans to only those customers with the best credit scores. The process can work, but it reduces your effectiveness. For the bank, it shuts out a large number of customers (some of whom will default). For the software development effort, it limits the project to those features that are guaranteed to succeed. In both arenas, the organization achieves less than it could.

As I see it, the best strategy is to allow for failure -- a small number of failures -- with a process to recover and correct after a failure. Attempting to implement twenty features and succeeding with sixteen (failing with four) is better than attempting (and succeeding with) a more conservative feature set of ten.

If your goal is to avoid failure, then the best strategy is to change nothing. If your goal is to advance, then you must accept some risk of failure. The important thing is not to avoid failure but to recover from failure.

Web 3.0 will make celebrities of us all

2011-04-16T16:11:00.001-07:00

The wonderful thing about the web (and the terrible thing about the web) is that it changes. The original web was simple, with static documents that linked to each other.

The next version of the web gave us transactions. It was a big step up: We could buy stuff! Books, CDs, and eventually batteries, household items, groceries, and even cars. But this version of the web did not let us share with others, except for e-mail.

With social media came "Web 2.0" and the ability to share information with others. From reviews of books and music to on-line journals and blogs, from friend sites (Friendster, MySpace, and Facebook) to micro-blogging sites (Twitter and Identi.ca), we built circles of friends and shared our experiences and opinions. But the sharing of information was not very granular. Reviews were available for anyone to see. Journals and blogs were visible to the world, unless we locked things to friends and family.

Twitter changed the game and let people sign up (and unsubscribe) without our say-so, but stayed within the Web 2.0 realm. We were still sharing data to everyone.

I think that "Web 3.0" will be different. It won't happen in the browser, but instead will happen on cell phones. The next generation of apps to connect us will be cell phone apps, not web browser apps. (Although there may be a web version of the cell phone app.)

And Web 3.0 will be defined not only by the primary platform (a cell phone and not the web browser) but also by the degree of control we have on sharing information. With Web 3.0, we will have a big say in who gets to see information, and there will be different levels of openness. For example, there is already a grocery list app that spouses can share. The list is shared between spouses (or significant others) and either person can add or remove items from the list -- but the list is visible to only the parties involved. This makes sense -- who really needs to see my grocery list? Yet it is useful to share it with my spouse.

Web 3.0 will have multiple sets of "friends", from spouses to family to close friends to distant friends and then to business associates. We will be aware of our different "circles of friends" and apps will be aware of them too. When we make information available, we will select the desired level of sharing.

With multiple circles of family and friends will come various politics. Will people be offended when they are placed in an outer circle and not in an inner circle? Will people lobby for more intimacy? Will they cry out when they are "demoted" to an outer circle? We will soon learn the answers to these questions.

The magazine "People" needs the movie industry to create material to fill pages. The movie industry needs the magazine "People" to have a venue for the news, pseudo-news, gossip about actors in the industry. It's a tidy little symbiosis. With Web 3.0, we will each have our own outlet for news. Perhaps not "People" magazine (or a magazine of any name) but web pages, public blogs, and friend-locked blogs. We will each be a celebrity, to some degree, with our own publicity and innermost group of friends. And also the drama that comes with celebrityhood.

If a C++ standard falls in a forest

2011-04-10T13:52:00.000-07:00

The next C++ standard (C++0X) is out, or just about out. Should we rejoice? Should we wait in anticipation for the release of new compilers?

The new standard has a lot of good things in it. Things that I want... or would want, if I were developing in C++. (And to some extent, I am still developing in C++, depending on the client.)

But I'm not sure that the future is so bright for C++ developers. The standard has been all but settled and published. The next steps would be for compiler implementers to release new versions of their products and then for developers to incorporate the new features into their code. That's what happened with the last release of the C++ standard.

The C++ environment has changed. After the previous standards update, the major C++ compiler vendors were Microsoft, Borland, IBM, and Intel. Microsoft and Borland were competing fiercely for the Windows developer.

Today, in the Windows market, the only compiler vendor is Microsoft. Borland is gone, IBM has moved to Java, and Intel was never a big player. I see no one that can provide competitive pressure to Microsoft.

And I doubt that Microsoft has much to gain from a new version of a C++ compiler. They are pushing their customers to C#; releasing a C++ compiler would send a mixed message to developers. I think Microsoft may leave the new C++ standard unimplemented.

The other big player in the C++ compiler arena is GNU, with the 'gcc' collection of compilers. But I don't see GNU as a major competitor to Microsoft; certainly not big enough to push Microsoft into implementing the new standard.

There is still a market for C++ -- just smaller than it used to be. Two decades ago, every serious piece of software for Windows was developed in C++. That market has been fragmented into separate camps for C#, Java, Objective-C, Python, and PHP. C++ is useful, but not universal.

We still need C++. While business apps and smartphone apps can be written in the new languages, the new languages themselves are, I suspect, written in C++. The only language compiler that may be written in something other than C++ might be Microsoft's C# compiler (and of course the Python interpreter for the PyPy project).

But C++ may become more of a specialized language, a tool used only be a small subset of the development community. As such, the per-unit cost for C++ compilers may increase. further encouraging C++ users to move to other languages.

Faster than a hand-held calculator?

2011-04-06T20:39:00.000-07:00

A fried of mine recently gave me an old HP 35 calculator. For a geek like me, a true HP calculator is a kingly gift. I cleaned off some grime and replaced the internal batteries (a bit tricky but possible) and now have a working calculator.

While using it, I was impressed with the speed of its response. Even for the sophisticated operations, the answers are displayed immediately. (As in "before my finger is off the button".)

Thinking about the experience, I was impressed with this speed because I have been using various computers, and all have been sluggish. The computers are a varied lot and include a collection of operating systems. The hardware ranges from ten years old to just purchased, and the operating systems include Windows, OSX, and Linux.

All of the PCs, despite their youth or modern operating systems, are slow to respond. Launching a program on any of them (even Windows Explorer) sees a delay. (The Apple MacBook is particularly gratuitous with special effects of windows swooping up and down, but Windows 7 is not without its swoopiness.)

Yet the calculator, designed and built in the early 1970s, is faster than the computers of today. The HP 35 has a custom Mostek chip running at no better than 1MHz and probably slower than that, with perhaps 32 bytes of read/write memory and less than 1K of ROM.

Yet the calculator is much faster than the PC.

Comparing a hand-held calculator against a personal computer is awkward at best. The calculator is a specific-use device; the personal computer is a general-use device, capable of much more than arithmetic calculations. And if I run the calculator program on the PC, its response is just as fast as the calculator. (It's the launching of the program that takes time.)

I recognize than comparing the launching of programs against the operation of a calculator is unfair or even meaningless, much like comparing the taste of orange juice against the handling of a mini-van.

But there is a psychological effect here. I was impressed with the HP 35. I don't remember being impressed by a PC program in a long time. (The last PC software that impressed me was the setup program for the Apple Airport, and that was at least four years ago.)

Is it possible to create impressive software? Such software would have to be easy to use, have an elegant interface, and fast. (And correct.)

When building software, we tend to focus on the "correct" part of the requirements, and leave the "wow" factors out. The approach is utilitarian, and possibly efficient, but little more than that.

Y2K is not dead

2011-04-03T19:20:00.001-07:00

The problem with Y2K was not that programs were going to calculate dates incorrectly. The miscalculation of a date is a program defect, a "bug", and all sorts of programs have all sorts of defects.

The problem with Y2K was that a large number of programs would have a defect (the same defect, or similar defects, as it happened) at the same time. It was the large number of programs affected at the same time that made Y2K the crisis what it was. (That, and our willingness to ignore the problems -- we knew about them since the 1960s -- until we had to address them.)

There are other problems of a similar nature to Y2K.

The year 3000 Most obvious is the "Y3K" problem. We solved the year 2000 problem with a series of patches and temporary fixes. A few programs were fixed with a long-term solution. After January 1, 2000, we all looked back, breathed a sigh of relief, and told ourselves that we would not let that problem happen again. Yet as early as March we dropped the "20" from our years and started using two-digit years in our documents and data. These documents, programs, and datasets will cause problems in the year 3000.

For those of you who are not worried about "Y3K" problems because the date is so far ahead, let me remind you that that thinking is exactly what got us into the Y2K mess.

The year 10000 A more distant and more subtle problem. The programs that were fixed "properly" for Y2K will work in 3000, and int 4000, but may fail in 10000.

The astute reader has realized that the "Y10K" problem is the same as Y2K, but at a larger time scale. The very astute reader has surmised there is no permanent solution for storage of dates, at least not with our current calendar and notations.

The Microsoft COM problem Not related to a specific date (at least not yet), this problem is similar to Y2K in that it will affect a large number of systems at the same time and it is a known problem.

At some point, Microsoft will stop supporting COM. I don't know when that time is, and I don't know that Microsoft has made such an announcement, but I am fairly certain that they will stop supporting COM -- if for no other reason than to get rid of the Windows registry, which COM uses.

COM was a solution to the problem of multiple DLLs, and it worked (clumsily) for a long time. It still works, but the design of .NET makes COM unnecessary. A "modern" Windows application will use .NET, not COM. Well, it might use COM and Microsoft's "Interop" techniques to talk to COM components, but only because those components have not been ported to .NET.

Yet a lot of systems use COM. Adobe products use COM to connect to objects in their scripting language. An unknown number of in-house systems use COM to connect components into systems. Even Microsoft uses COM for some of its products. COM is not dead -- although one can make the argument that it should be.

I'm sure that Microsoft is working to migrate all of its products and applications from COM into .NET. Microsoft has a large product line and a large code base, and such a migration may take years and possibly decades.

But know this: at some point, COM will stop. (It will cease to be. It will depart. It will meet its maker. It will be a dead parrot.) When COM stops, all of the systems that rely on COM will stop, too. Microsoft will have converted their applications to use .NET. Will your systems be ready?

My purpose is not to panic people. I believe that COM will be supported for a long time. You have time to identify your systems, plan for their conversion, and implement the changes long before Microsoft drops support for COM. My purpose is to alert you to this change and get you to make plans.

Eventually, you will have to change your applications. You can change them on your schedule, or on Microsoft's schedule. Which do you prefer?

The Opposite of Batch

2011-04-02T08:57:00.001-07:00

We of the PC revolution like to think that the opposite of big, hulking mainframes are the nimble and friendly personal computers of our trade. Mainframes are expensive, hard to program, hard to use, and come with bureaucracies and rules. Personal computers, in contrast, are affordable, easy to program, easy to use, and one is free to do what one will with the PC -- the only rules are the ones we impose upon ourselves.

Yet mainframes and personal computers have one element in common: Both have fixed resources. The processor, the memory, the disk for storage... all are of fixed capacity. (Changeable only by taking the entire system off-line and adding or removing components.)

Mainframes are used by many people, and the resources must be allocated to the different users. The batch systems used by large mainframes are a means of allocating resources efficiently (at least from the perspective of the hardware). Users must wait their turn, submit their request, and wait for the results. The notion of batch is necessary because there are more requests than computing resources.

Personal computers provide interactive programs by providing more computing resources than a single person requires. The user can start jobs (programs) whenever he wants, because there is always spare capacity. The processor is fast enough, the memory is large enough, and the disk is also large enough.

But personal computers still offer fixed capacity. We rarely notice it, since we rarely bump up against the limits. (Although when we do, we often become irritated. Also, our personal systems perform poorly when they require more resources than available. Try to boot a Windows PC with a completely full hard disk.)

The true opposite of batch is not interactive, but flexible resources -- resources that can change as we need them. Such a design is provided by cloud computing. With cloud computing, we can increase or decrease the number of processors, the number of web server instances, the memory, our data store -- all of our resources -- without taking the system off-line. Our computing platform becomes elastic, expanding or contracting to meet our needs, rather than our needs adjusting to fit the fixed-size platform. Perhaps a better name for cloud computing would have been "balloon computing", since our resources can grow or shrink like a balloon.

This inversion of shape -- the system conforms to our needs, not our needs to the system -- is the revolutionary change offered by cloud computing. It will allow us to think of computing in different ways, to design new types of systems. We will have less thought of hardware constraints and more thought for problem design and business constraints. Cloud computing will free us from the drudgery of system design for hardware -- and let us pick up the drudgery of system design for business logic.

With cloud computing, IT becomes a better partner for the business. IT can enable faster business processes, more efficient supply chains, and better market predictions.

The rise and fall of page numbers

2011-03-29T19:26:00.000-07:00

How important is the page number? It is a small thing, and we often read often overlook the page numbers of our texts. Yet page numbers are a relatively new invention.

A long time ago (a very long time ago), knowledge was passed on through the oral tradition, with scholars memorizing and speaking. There was no writing (and therefore no pages or page numbers).

A long time ago (not quite as long as the age of memorization) we humans started writing. We quickly developed parchment and the dominant form was the scroll -- a long piece of parchment that was rolled.

In the first century B.C.E. the codex gained popularity. Instead of a single scroll of parchment, the codex consisted of a set of flat pages. Often made of papyrus, it had the virtue of being two-sided. One could write on both sides of the papyrus. (One could write on both sides of a scroll, but it was not practical.)

Page numbers were introduced in the sixteenth century B.C.E., to aid in cross-referencing documents.

That's a long time for a new concept to arise, especially to our twenty-first century treadmill of new products.

But back to page numbers. E-readers -- a twenty-first century doodad -- make life difficult for page numbers.

It is not pages that lead to the invention of page numbers, but pages with consistent text that make page numbers useful. Every time you refer to the page, the text must be the same as the time before. Without this consistency, page numbers are useless.

E-readers break that consistency. While different editions of a book might use different layouts (and therefore different page numbering), once a printed book was printed, the page numbers stayed put. But e-readers play by a different set of rules. They allow for different sizes of text, so the "pages" they present on their displays change. A document of ten pages in small typeface may be twenty pages in a larger typeface.

Page assignments in printed books are static; page assignments on e-readers are dynamic. Or in the current programmer lingo, printed books use early binding for page numbers and e-readers use late binding. ("Binding" in the "assignment" sense and not the "leaves and signatures into a folio" sense.)

Interesting, this problem did not arise with word processors. I suspect that is because word processors are used more for composition and less for reading, while e-readers are used primarily (if not solely) for reading. Wordstar, Wordperfect, and Microsoft Word are all quite good at assigning page numbers, mostly as an afterthought. For the author, page numbers are nothing more than a bit of stuff that is tacked on at the end.

I suspect that we will see the demise of page numbers. Their original purpose -- cross referencing -- is really a matter of marking destinations within the text, and we can do that with hypertext links and other techniques. Page numbers were a compromise, the best that was possible with the technology of the day, a hack that happened to work.

We still need the ability to refer to specific locations within texts. With other techniques, we can drop page numbers. Oh, I expect that they will stick around for a while. As long as we use printed documents for reference we will need page numbers. I expect a new class of completely on-line information to use pageless (or page-number-less) formats.

While page numbers will disappear, the two structures that currently depend on them (tables of contents and indices) will remain. These are useful, and they will change into structures that can get us quickly to the desired place of the document. They will do it without page numbers, though.

Structure thy data

2011-03-27T15:56:00.000-07:00

In the 1980s movie "Labyrinth", there is one scene that shows an image of David Bowie carved in rock. The camera shifts, and you see that the image was actually an illusion. It is not a single rock with the image carved, but three different rocks each with a piece of the image. When viewed from the exact right angle -- when the three rocks line up, and you see the image. When viewed from a different angle, the image disappears.

The movie "The Incredibles" has a scene that does the same thing, but in reverse. Mr. Incredible, in a cave, is looking at a set of rocks. As he shifts his position, writing carved into the rocks forms the word "Kronos". The word is visible from only one position in the cave.

In both movies, data is visible from a specific position, but not from others. This affect can happen to real-life data too.

For example, document files can contain headings, paragraphs, and tables. The data contained within the document often looks good to a human viewing the document, but is difficult (or perhaps impossible) to read by another computer system.

I'm not talking about file formats, and the difficulties in parsing .DOC and .PDF formats. Difficulties in parsing certain file types do exist, and have been reduced by the use of XML-based formats. But beyond the reading of formats, there is the challenge of extracting the structure of the data.

Documents contain headers, paragraphs, and tables. In order to properly interpret and process a document, you must identify these different parts of the document. To identify the parts of the documents, the information in the file must be present. And with our current tools and techniques, there is in way to ensure your document has this structure or verify that another author's document contains this information.

The "proper" way to create a document with this information (in, say, Microsoft Word) is to use styles to mark different parts of the text. Marking the document heading as "Heading" and section and sub-section headings as "Heading 1" and "Heading 2" places this meta-information in the document.

The "improper" way to create a document is to ignore styles and use low-level font commands to change the appearance of text. One can change the typeface, the weight, and the size of text. For each section heading, one can set specific options for typeface attributes. Its less efficient than using styles, but perhaps easier to understand.

I put the words "proper" and "improper" in quotes because there is no standard for using Microsoft Word (or other word processors). One is free to use styles or the low-level commands. And this is part of the problem. But I will ignore the need for best practices and focus on the aspect of data alignment.

Documents using the "proper" method (styles), contain meta information that can be used by other applications to interpret the documents. Documents using the "improper" method have no certain way of interpreting data. (One can make assumptions based on certain patterns, but it relies on consistency in the application of low-level formatting commands.) The data is unstructured.

Unstructured data is like the rocks in "Labyrinth" and words on rocks in "The Incredibles". From the exact right angle, the image is visible. But from any other angle, the image is not. The unstructured data in a document is visible and sensible to a human looking at the text rendered by the word processor on a screen (or on a printed page), but is not sensible to a computer program.

We have achieved much in the development of computer programs. Not just word processors, but spreadsheets, databases, accounting systems, instant messaging, image manipulation, and many more applications. We are at a point (and possible past it) that we should tolerate unstructured data. We should use structured data and encourage it when structured options are available.

Data endures, often long beyond the life of the applications that created the data. To rely on the original programs to read the data is foolish. To assume that only humans will read the data is arrogant.

Getting social media right

2011-03-24T19:30:00.000-07:00

A lot of companies have jumped on the social media bandwagon, from Facebook to Twitter to what-have-you. And a lot of companies have gotten it wrong.

They mistake social media as another path for them to send information to customers and potential customers. And one certainly can do that. (With the benefit of knowing that the people following you are actually interested in your products and services.) But that's where a lot of companies stop.

It's a rather arrogant approach, thinking that everyone will be listening to you, with nothing to say of their own.

The companies that really "get" social media will be doing two things: First, they will be talking to people (and not at them). Second, they will be listening.

Social media lets everyone talk. If you really want to leverage social media, recognize its potential for communications from you to your customers, from your customers to you, and from customers to other customers.

Listening to your customers. Now, there's a novel idea!

Microsoft Office becomes hip because Facebook did

2011-03-22T18:18:00.000-07:00

The new Microsoft Office 2010 edition has a number of improvements, ranging from animations on the opening splash screen to collaborative capabilities. Microsoft Outlook has a special feature, one that I have not seen in the other products of the suite.

Microsoft Outlook has a social networking feel to it, with incoming e-mails showing avatars of the sender and their immediate network. That's a neat trick, since few people in our office have the latest version of MS Office, and therefore have had no opportunity to set up a social network. (And to be honest, the avatars are all generic "shadow" avatars.)

But this is a change to look like a social network without actually being one.

It turns out that you as the user don't define your social network in MS Outlook -- the software uses the organization's network defined in the Active Directory server. People are members of departments (or groups, or branches, or whatever) and MS Outlook relies on this structure. Which means that the social network image of MS Outlook is just that: an image. It's not my social network (where I can friend and unfriend people), it is the company's definition of a social network: the people in my assigned organizational unit.

It looks cool, until you realize what's happening. Is this an attempt at making a staid (dare one say "quaint") interface more hip? is it a way to make MS Outlook more acceptable to the generation that was raised on Fascebook? Perhaps.

Hipness aside, this change demonstrates that Microsoft felt compelled to change Outlook. Perhaps the change was driven by the collaboration capabilities, but I feel that Microsoft is attempting to compete with Facebook. Which means that Facebook is driving the design of software, not Microsoft.

Improve quality of code by changing staff

2011-03-20T16:00:00.000-07:00

In the not-so-distant past, I worked on a small project and reviewed staffing with the project manager. The project was a C++ application that was a component in other systems within the organization.

The code and the procedures were complicated. A new member of the team required at least six months before he or she could be productive, and often the "ramp up" time was closer to a year.

I discussed many issues with the Project Leader. At one point, we discussed the expected tenure of a team member. We agreed that it should be about five years.

Simple math shows that for an average tenure of five years, a team of ten people should "rotate out" two people every year. This project manager did not understand that calculation. More specifically, the Project Leader was unwilling to "swap out" two people of the team that year. This makes some sense, since the two replacement folks would require the better part of the year before becoming productive.

And swapping out another two people in the next year would mean that another two newcomers would need the better part of a year to come up to speed. With my "average tenure of five years" plan, every year would see two newcomers and those newcomers would require time to become familiar with the procedures and the technologies of the project.

So while this Project Leader agreed with the idea of a five-year tenure for team members, she wanted to keep people on the project as long as possible -- and more than five years. Extending the tenure of a person on the project reduced the cost of training a "newbie".

Yet a set of newcomers may be a good thing for a development project. (And perhaps any type of project.)

With a steady influx of new members (and a steady departure of experienced folks), a project must make accommodations for the newcomers. When a project is so complicated, and so difficult to learn, that it takes the better part of a year before a professional skilled in the craft can be productive then adding team members is expensive. The desire to hold on to experienced staff is large.

A project that has a lower "ramp up" cost is better prepared to accept newcomers.

So is the "ramp up" cost high because project difficult to learn. or is the project difficult to learn because there are few newcomers (and therefore little incentive to design the project to be friendly to newcomers)?

The implicit assumption is that the complexity of a project is an innate quality of the project, a result of the requirements and technology. I don't think that is quite true.

Let's turn the idea on its head. Let's work with the premise that the complexity of a project (its processes and its code) is an attribute of the project and one that can be managed, much as the total cost, delivery schedule, quality, and total effort. Therefore, the complexity of a project can be controlled by the project managers. As an attribute, the complexity can be measured and reviewed, and managers can direct changes to ensure that the complexity stays within expected values.

If managers can control the complexity of a project, then they can control the "ramp up" cost for new team members.

Of course, managing the complexity of a project means adding it to the other attributes of a project. There is no free lunch, and devoting effort to one attribute means less effort to other attributes. Reducing complexity may mean less effort for quality assurance, or less effort for support. It may even mean a longer development cycle.

I'm not saying that complexity can be managed for free. I'm asserting that complexity is an attribute of a project can can be managed. How one trades off effort of one attribute against another attribute is what project management is really about.

The end of phone calls?

2011-03-19T10:58:00.000-07:00

Is it possible that we are seeing the end of phone calls? The notion is hard to accept. Many companies, government organizations, and individuals run on phone calls.

Here's what I see:

Voice calls are a generational thing The "younger generation" (anyone under 30) uses cell phones, and uses cell phones a lot. But most of the use is text messages. Followed by taking (or sharing) pictures. Followed by surfing the web. Voice calls are way down on the list of uses. Folks over 30 make voice calls; folks under 30 send text messages.

Organizations use voice calls within the organization They use e-mail and some voice for conversations outside of the organization. Organizations use e-mail for communicating with people in other time zones.

Voice calling is cheap The typical phone plan offers some number of free minutes (300, 550, 700... the plans vary) but most folks use only a fraction of their free minutes. In the early days of cell phones, users and vendors worried about minutes of usage (it was all voice calls back then). Today we have enough minutes in the plan. We don't worry about minutes. Voice calls, essentially, are "free", which means that there is little demand for them.

Vendors are focussing on data, not voice The big debates are now about data caps and throttling. Voice minutes and coverage are not part of the debate.

Voicemail makes voice calls asynchronous A traditional voice call is synchronous: both parties must be present and participating at the same time. With voicemail, one party can leave a message and the second party can respond at a later (more convenient) time. Voicemail converts phone calls to the time context of e-mail.

Voicemail as a tactic Companies have been using voicemail for decades, as have individuals. Individuals are now using voicemail tactically, allowing calls to "roll over" to voicemail even though they could answer the phone. I myself let my home (wireline) phone go to voicemail all of the time; family and friends know to call me on my cell phone, so a call to the home phone means someone other than family and friends.

But not everyone is abandoning phone calls. Here are the people and organizations who are still using voice calls:

Political campaigns Every few years, the polit-callers make their cases.

Advertisers Despite being on the "do not call" list, some companies call -- and some leave messages.

Recruiters Some send e-mails, and some call. Even after I gently nudge them towards e-mail, the ones who call continue to use voice calls. This may be a extrovert behavior, where folks prefer talking over written communication.

I generally find that I do not want to answer the phone. Odds are that the call will be an annoyance.

I also find that I have little reason to call a person. I will call a company for a specific reason (to activate a credit card, perhaps) but I make few phone calls to friends. Calls to family are often to the older folks who are reluctant to use text messages.

I think that the lesson here is: all good things come to an end, and when a technology becomes free, it is close to its demise.

Microsoft should recognize "good enough"

2011-03-15T18:33:00.000-07:00

The good folks in the tech support group came by today and upgraded my Microsoft Office from version 2007 to version 2010. The experience left me wondering why Microsoft bothered to introduce a new version of MS Office.

I understand the reasons for version 2007. It was a big change. It introduced the "ribbon", a new way of presenting the GUI menus to users. It also introduced the MS OOXML file formats, which made reading (and writing) files for MS Office much easier. Customers, third parties, and I suspect Microsoft, all benefitted.

With version 2010, I don't see the big changes. There *are* changes. Small changes. The non-intuitive "Office button" has been changed to a tab for the ribbon, which makes it closer to the old "File" menu that was the standard for GUI applications. The "skin" has changed its color scheme from silvery-blue to silvery-gray. There is a new menu item called "Team", which apparently lets one use Microsoft's Team Foundation Server to store and share documents. (But no connection to Microsoft SharePoint or Microsoft Live -- at least none that I can see.)

So I have to wonder: why release a new version? What's in it for Microsoft? More importantly, how do their customers benefit?

And I am not sure that we even need a new version of MS Office.

I suspect that we (in the app industry) have gotten pretty good at word processing, spreadsheets, and note-taking. Improvements beyond fancy skins will be hard to come by. We can do only so much with fonts, justification, and formulas. (In some ways, Intuit has the same problem with Quicken and Quickbooks. Each year they introduce new versions that are advertised as easier to use. Yet accounting has been around for centuries and we don't really need new, imaginative approaches.)

Looking at the situation from another angle: Word processing and spreadsheets, for the most part, are "gen 2" computer applications, and we are moving to "gen 3" apps.

"Gen 1" applications were the centralized, business oriented accounting and inventory apps, where users submitted well-defined transactions to well-defined central databases and received well-defined reports. "Gen 2" applications are the early PC applications with users controlling their own unstructured data and using it for their own (possibly unstructured) analysis. ("Islands of information", in the 1990s term.) Word processing and spreadsheets fit into this category.

The "gen 3" applications of shared data ("web 2.0", social media) have quite different motivators and usages. LiveJournal, Facebook, and Twitter fall into this group. The earlier applications will not fit into this group, no matter how hard you (or even Google) push.

We're done with serious development of the "gen 2" applications. We've made them "good enough". It's time to move on to new ideas.

Microsoft should recognize that MS Office is good enough. Their history has been one of shipping software when it was deemed "good enough". Yet now that it is a large income stream, they seem determined to maintain it. They need to move on.

Because the rest of us have.

The rate of change

2011-03-13T12:05:00.000-07:00

In the good old days, technology changed. (Yeah, it still changes.) But it changed at a slow rate. When computers were large, hulking beasts that filled entire rooms, changes occurred over years and were small. You might get a new FORTRAN or COBOL compiler -- but not a new language -- every half-decade or so. (FORTRAN-66 was followed by FORTRAN-77, for example.)

Today, computers get faster and more powerful every six months. (Consider the Apple iPad, with the second version released prior to a year after the first.) Microsoft has released compilers for C# in 2001, 2003, 2005, 2008, and 2010 -- about one every two years. And not only do we get compilers, but we also get new languages. In 1995, Java was the new thing. In 2001, it was C#. Now we have Ruby, Python, Erlang, Haskell, Scala, Lua, and a bunch more. Not all of these will be the "next big thing", but one (or possibly more) will be.

Organizations can absorb change at a certain rate, and no faster. Each organization has its own rate. Some companies are faster than others. Larger companies take more time, since they have more people involved in decisions and more legacy applications. Small companies with fewer people and "software assets" can adopt new technologies quicker. Start-ups with a small number of employees and a few lines of code can move the quickest.

We're in a situation where technology changes faster than most companies can absorb the change. In most (big-ish) companies, managers don't really work with technology but make decisions about it. They make decisions based on their experience, which they got when they were managers-to-be and still working with technology. So managers in today's organizations think that tech works like a C++ compiler (or maybe a Java JVM), and senior managers think that tech works like a COBOL compiler and IMS database. (I imagine that MBA graduates who have no direct experience with tech believe that tech works as a series of commoditized black boxes that are replaceable at the proper cost.)

This is a big deal. If managers cannot value technology and make good judgements, then the decision-making process within companies becomes political, with different groups pushing for their positions and advocating certain directions. Solutions are selected for perceived benefits and the results can be vastly different from the desired outcome. Mistakes can be very expensive for a company, and possibly fatal to the project or the company.

So what is a company to do? One could hire managers who have deep technical knowledge and keep abreast of changes, but such managers are hard to find and hard to keep. One could create a separate team of technologists to set a technical direction for the company, but this can devolve into a special interest group within the company and create additional politics.

I think the best thing a company can do is set a general direction to keep the company technically capable, to set an expectation of all employees to be technically aware, and to reward those teams that demonstrate the ability to manage technology changes. Instead of dictating a specific solution, look for and encourage specific behaviors.

Stupid wizard tricks

2011-03-12T14:48:00.000-08:00

Sometimes well-intentioned designs are less than effective.

For example, the Lenovo "select a PC" wizard on their web site. I am in the market for a laptop PC, and visited the Lenovo web site to view their wares. (I have had good experiences with IBM Thinkpads, so Lenovo has an advantage in the selection process.)

After viewing several web pages and being overwhelmed by the choices of PCs, I chose to use the Lenovo web "wizard" (my term, not theirs) to select a laptop. I had been looking at a netbook PC, and I knew that I did *not* want a netbook PC. I want a full-blown laptop PC. But their product line is larger than I can hold in my head at one time, so I ran their web wizard to ask me questions and pick "the right PC for me".

There web wizard is an awesome construct. It asks lots of questions, and then asks a second round of "balance factor X against factor Y" questions where X and Y are attributes like screen size and battery life.

Finally you get to the recommendation. This is, by scientific analysis of your answers to the multitude of questions, the best PC for you. And for me, the web wizard selected... the exact netbook that I had been looking at before, the one that I knew I did not want!

So here I am with the exact wrong answer. The web wizard, with its tiny brain, has decided that I need a netbook. Yet my gut tells me that I do not want it. What to do?

There is no "make adjustments" option. My only option is to start the entire web wizard (with its multitude of questions) from the beginning. I choose not to go through that again -- now it is an ordeal, not an assistant.

The end result: I did *not* (and to this date have not) purchased a laptop (or any other computer) from Lenovo. This is a :FAIL all around.

The moral for software developers: Use wizards -- that is, one-way selection processes of guided questions -- with care, and when you do, keep them short. The larger picture is to allow the user a degree of control, and the ability to make adjustments. Had the wizard displayed possible answers along the way, and let me narrow the set as I select attributes, I probably would be the owner (a happy owner) of a Lenovo laptop PC today.

Did amazon.com invent cloud computing?

2011-03-10T17:55:00.000-08:00

I was the recipient of an interesting idea at the local CloudCamp un-conference.

Amazon.com offers their EC2 virtual servers. At first, it was offered with no guarantees: yeah you can run things on EC2 servers, but they could crash at any time. It was cheap, but not reliable. What was a developer to do?

The price for amazon.com's EC2 service was too low to ignore, so developers did what they always do: they built software around the problems. For cheap servers that could crash at any time, that meant building software that was tolerant of servers "disappearing" at any moment.

That's a big part of cloud computing. With "the cloud", a server can drop off-line at any time. Your application must continue. The initial level of reliability of EC2 was low, and forced developers to think about portable processes, processes that could hop from server to server and continue the work. This idea (along with others) made cloud computing possible.

Prior to the idea of portable processes (and crashing servers), applications were built on the model of "the server is reliable". After EC2, software was built with the idea of "the server is not guaranteed". It's quite a change.

So we may have amazon.com to thank for the cloud.

Hybrid for functional languages

2011-03-09T20:03:00.000-08:00

Are there hybrid functional languages? Languages that correspond to the hybrid object-oriented programming languages of C++, Object Pascal, and Visual Basic? A quick survey of the web (if there is such a thing as a quick survey) yields a page (from stackoverflow.com) that lists the following languages: Scala, Clojure, F#, Ruby, OCaml, Common LISP, Nemerle, Smalltalk, and O'Haskell. A few other web pages list these languages as hybrids.

So yes, there are hybrid functional languages. And this is important.

The jump from the "classic" languages of C++, C#, and Java to pure functional languages (Haskell, Erlang) is a large one. We were able to move from procedural languages (C, Pascal, and BASIC) to object-oriented languages (Java, C#) by using hybrid languages as an intermediate step. (A rather expensive intermediate step, as we look back at the mountains of hard-to-maintain C++ code we have created, but it was a step.)

We humans, for the most part and in most situations, tend to do better with small steps. Small changes in our programming languages are more acceptable to us that large changes. (Hence the migration from FORTRAN II to FORTRAN IV to FORTRAN 66 and then to Fortran 77, and not a migration from FORTRAN II to Pascal.)

The hybrid object-oriented languages became popular because they were useful.

I expect that hybrid functional languages will become popular for the same reason. We will want to move to the functional languages, to reduce programming time and improve reliability. The jump to a pure functional language is large, often requiring not only a complete re-write of the application but a re-thinking of our thinking for the application. Hybrid functional languages will allow a more gentle transition to functional programming.

Exactly *which* hybrid functional languages become popular is quite hard to predict. Microsoft will push F#, or possibly extend C# with functional qualities. The Java crowd will like Scala. The Ruby enthusiasts will like... Ruby. I'm not sure what the Apple camp will adopt.

Apple, with its devotion to C, C++, and Objective-C is in an interesting position. There is no clear path to functional programming on the Apple platform. This may present a problem for Apple. (Or maybe not, as they may choose to support a functional or hybrid functional language in the future.)

One level down

2011-03-06T14:08:00.000-08:00

A while back, I was build-master for a large project. The project consisted of twenty or so Visual C++ projects ("solutions", in Microsoft's terms) and five C#/.NET projects.

As build master, I had to maintain the build scripts and the system that ran them. The build system itself was a complicated application: A Java program with dozens of classes, XML files for the scripts, and an interface that ran on a web page. Maintaining the build system was expensive, and we chose to re-write the system. The resulting system was a simpler collection of batch files. The batch files looked something like this:


CD project-directory-1
MSDEV /build /solution project-1.sln /project Release
CD ..
CD project-directory-2
MSDEV /build /solution project-2.sln /project Release
CD ..
... repeat for all twenty-five projects

The one feature we needed in the system was for it to stop on an error. That is, if a Visual C++ solution failed to compile, we wanted the build system to stop and report the failure, not continue on and attempt to build the rest of the projects.

Batch files in Windows are not good at stopping. In fact, they are very good at continuing on. You can force a batch file to stop. Here's our first attempt:


CD project-directory-1
MSDEV /build /solution project-1.sln /project Release
IF %ERRORLEVEL% NEQ 0 EXIT /B 1
CD ..
CD project-directory-2
MSDEV /build /solution project-2.sln /project Release
IF %ERRORLEVEL% NEQ 0 EXIT /B 1
CD ..
... repeat for all twenty-five projects

This solution is pretty ugly, since it mixes in the control of the execution with the tasks of the execution. (Not to mention the repetitiveness of the 'IF/EXIT' command.) The problem was pervasive: we wanted our scripts to stop after a failure in any part of the process, not just compiling projects. Thus we needed 'IF/EXIT' lines sprinkled in the early phases of the job when we were getting files from version control and in the later part of the job when we were bundling files into an install package.

After a bit of thought and several discussions, we implemented a different solution. We wrote our own command processor, one that would feed commands to CMD.EXE one at a time, and check the results of each command. When a command failed, our command processor would stop and report the error.

The result was a much simpler script. We took out the 'IF/EXIT' lines, and the script once again focussed on the task of building our projects.

With our new command processor in place, we added logic to capture the output of the called programs. We captured the output of the compilers, the source control utilities, and the install packager. This allowed for an even simpler and more focussed script, since we removed the '>log.txt' and '2>errlog.txt' clauses on each line.

Looking back, I realize that our solution was to move the logic for error detection down one level. It took the problem out of the script space and into the space of the command processing.

Sometimes, pushing a problem to a different level is the right thing to do.



Governing the speed of business
2011-03-05T17:31:00.000-08:00
Governance of IT projects ensures that they bring value to the organization, by aligning priorities and coordinating resources for the best value to the organization. They do this by enforcing a set of standard processes for each project. The standards often specify the technology to be used, the project evaluation (including cost/benefit analysis), and the reporting of project progress.

A side effect of IT governance is the slow pace at which projects can move. The governance process includes meetings and conference calls between various groups, and buy-in from the involved teams. All stakeholders are included, from users to testers and support teams. Meetings between all of these groups requires time, and frequently the governance organization schedules meetings on a regular basis (say, once a month) to review project progress and discuss new projects. Projects must live on "governance time", and fit within the schedule of the governance organization. But the benefit is that the project meets the needs of the organization.

So the IT governance process slows projects to ensure quality and coordination.

Interestingly, early steam engines (well, some steam engines) had governors. They were used to limit the speed of the engine. They slowed the engine to ensure that the engine provided the proper power. (Or, one could say "to meet the needs of the organization".)

The problem with (traditional) IT governance is that it slows the project. Any project. Every project.

Traditional IT governance is a bureaucratic process. It obtains quality, coordination, and consistency at the cost of time.

It also constrains solutions to projects of a certain size, or of a size within a certain range of sizes.

IT projects come in all sizes, from the two-person, four-hour "let's try this" to the hundred-person, multi-year global development and implementation projects for thousands of users. But IT governance often uses a standard project template for projects, specifying the general steps for a project, the reporting requirements, and the technologies involved. Any project must go through the governance process, and like Play-Doh being pushed through a mold, the project is shaped into a form selected by the governance group. Small projects are increased to include users and deployment teams. Large projects are divided into smaller ones, to allow for deliverables within the standard timeframe.

In other words, the governance process "normalizes" all projects. Small projects are made standard-size, and large projects are made standard-size too.

This normalization makes sense from the view of the governance committee (who wants to see a consistent set of projects and reports) but not from the view of the business. Let's look at a hypothetical example:

Senior Manager (to project leader): "Customers keep calling and asking for an iPhone app. Can we get something built before the end of the quarter?"

Project Leader: "Possibly. We have a few folks who have been experimenting with iPhone apps. Let me look at schedules and see what I can arrange."

:: one week later ::

PL (to SM): "I can move people from the "Huron" and "Erie" projects, and they could build the iPhone app and deliver it in two weeks. But that means delaying "Huron" and "Erie", and the PMO refused to adjust the schedule for those projects. And even if they did, the standards committee needs two months to review the new tools for iPhone apps before allowing us to use them. And then the security group rejected our proposal, since they have not tested the iPhone in our environment. They want six months (and funding) to build a lab and hire four analysts and a manager for iPhone apps. Oh, and the PMO called and wants to discuss the iPhone app at their monthly project review meetings, and they can fit us in to the schedule in three months."

An idea brought "under control" by the governance process. Standardized and approved.

But perhaps not what the organization needs.


The disappearing notion of storage
2011-03-02T18:11:00.000-08:00
In the beginning was the processor. And it was good. But the processor needed data to act upon, and lo! there was memory (in the form of mercury delay lines). But the processor and the memory worked only while powered, and we needed a way to store the data in the long term (that is, when the machine was not powered) and so there came storage devices.

Storage devices evolved along a tortuous path. From punch cards to magnetic tape, from paper tape to DECtapes (which were not quite the same as plain magtapes), and then to magnetic platters that were eventually called "discs".

The path for microcomputers was slightly different. Microcomputers started with nothing, had a short stint with paper tape and cassette tape, took off with floppy disks, then hard disks, and finally (eons later) flash thumb drives and solid-state disks (SSDs).

A computer needs storage because everything that we want to store is bigger than memory. The company's accounts would not fit in the 4K of memory (especially with the general ledger program and tape libraries sitting in memory too) so the data had to live somewhere. Removable media (paper tape, magtape, disks) made for an auxiliary memory of virtually infinite capacity.

But what happens when a computer's main memory becomes large enough to hold everything?

I've made the argument (in another forum) that virtual memory is sensible only when the CPU's addressable memory space is larger than physical memory. If the processor can address 16MB of memory (including all virtual pages) and the computer contains 4MB of memory, then virtual memory works. You can allocate memory up to the 4MB limit, and then swap out memory and effectively use 16MB of memory. But give that processor 16MB of memory, and there is no need for virtual memory -- in fact it is not possible to use virtual memory, since you can never have a page fault. (I'm ignoring the CPU designs that reserve address bits for virtual memory. Assume that every address bit can be used to address real memory.)

Computer memory has been limited in size due to a number of factors, one being manufacturing capabilities and costs. It wasn't possible (read 'feasible with the budget') to obtain more than a paltry few kilobytes of memory. External storage was slow but cheap.

The cost factor has just about disappeared. With solid-state disks, we now have lots and lots if bits. They happen to be organized into an entity that pretends to be a storage device, but let's think about this. Why pretend to be a storage device when the CPU can address memory directly?

Here's what I see happening: CPUs will change over time, and will be equipped with larger and larger addressing capabilities. This will require a change to physical architecture and instruction sets, so I expect the change will occur over decades. But in the end, I expect a computer to consist of a CPU, memory, video, and a network port. No USB ports. No memory sticks. No serial port. No keyboard port. And no hard disk drive. You will load applications and data onto your computer. They will be stored in memory. No muss, no fuss!

Such changes will mean changes for programs. We won't need the elaborate caching schemes. We won't need "save file" dialogs. We won't need disk de-blocking routines or special drivers for spinning metal platters.

Not everything changes. We will still need backup copies of data. We will still need transactions, to ensure that a complete set of changes is applied. We will still need names for data sets (we call them 'file names' today) and we will still need to grant permission to use selected parts of our data store.

So I wouldn't toss the concept of file systems just yet. But be prepared for changes in hardware to rock the boat.


Mine is bigger than yours
2011-02-26T18:54:00.000-08:00
People have noticed that apps for the iPhone (and Android phones) are easier to use than the standard PC applications. They are easier to install, easier to run, and they tend to run without crashing.

Why are the two so different? I think that the answer started in marketing.

PCs and cell phones are advertised differently. Personal computers are sold on power; cell phones are sold on convenience.

In 1981, the IBM PC entered a market that already had other computers, and the vast majority of users were men. The IBM PC competed on power: a faster and more capable CPU, more memory, more pixels on the screen, more sales offices, ... you get the idea.

Software for the IBM PC competed on the same basis. Word processors advertised the number of fonts, spreadsheet advertised the number of cells, compilers advertised the number of features. Advertising in the PC age (for hardware and software) was a game of "mine is bigger than yours".

The Apple iPhone, on the other hand, entered a market devoid of direct competition. Yes, there were other cell phones, but the iPhone set a new standard for cell phones. Whereas the IBM PC was perhaps thirty percent better than the Apple II and TRS-80 computers, the iPhone was more than one hundred percent better. It was a new creature.

I'm sure that Apple knew that the market for iPhones would include women as well as men. That may have pushed them to market convenience. Or not; I don't know their motivation. it is a difference that we can analyze for a long time.

The bottom line is that the iPhone was sold on the basis of "easy to use" and "coolness". It was not sold on "bigger and therefore better".

It's relatively easy to deliver "bigger and better". (Not completely easy, as IBM and later Microsoft learned from the need for backwards-compatibility, but possible.) Delivering "coolness" is hard. "Coolness" has a psychological component that is absent from the pure "big hardware" solution.

The coolness aspect is what Microsoft products need. Microsoft has improved Windows, designed game systems, and built cell phones and music players, but all within the mindset of "bigger than the other guy". Even their Visual Studio IDE is bigger and more capable than Eclipse and the other IDEs, but not cooler. It's just bigger.

If Microsoft wants to compete in the new age, they must learn coolness. They must forget "mine is bigger than yours" and make products truly desirable.

Come to think of it, if *you* want to compete, you must learn coolness.


CPU time rides again!
2011-02-23T19:51:00.000-08:00
A long time ago, when computers were large, hulking beasts (and I mean truly large, hulking beasts, the types that filled rooms), there was the notion of "CPU time". Not only was there "CPU time", but there was a cost associated with CPU usage. In dollars.

CPU time was expensive and computations were precious. So expensive and so precious, in fact, that early IBM programmers were taught that when performing a "multiply" operation, one should load registers with the larger number in one particular register and the smaller number in a different register. While the operations "3 times 5" and "5 times 3" yield the same results, the early processors did not consider them identical. The multiplication operation was a series of add operations, and "3 times 5" was performed as five "add" operations, while "5 times 3" was performed as three "add" operations. The difference was two "add" operations. Not much, but the difference was larger for larger numbers. Repeated through the program, the total difference was significant. (That is, measurable in dollars.)

Advances in technology and the PC changed that mindset. Personal computers didn't have the notion of "CPU time". In part because the hardware didn't support the capture of CPU time, but also because the user didn't care. People cared about getting the job done, not about minimizing CPU time and maximizing the number of jobs run. There was only one job the user (who was also the system administrator) cared about -- the program that they were running.

For the past thirty years, people have not known or cared about CPU usage and program efficiency. I should rephrase that to "people in the PC/DOS/Windows world". Folks in the web world have cared about performance and still care about performance. But let's focus on the PC folks.

The PC folks have had a free ride for the past three decades, not worrying about performance. Oh, a few folks have worried: developers from the "old world" who learned frugality and programmers with really large data processing needs. But the vast majority of PC users have gotten by with the attitude of "if the program is slow, buy a faster PC".

This attitude is in for a change. The cause of the change? Virtualization.

With virtualization, PCs cease to be stand-alone machines. They become an "image" running under a virtualization engine. (That engine could be Virtual PC, VMWare, Virtualbox, Xen, or a few others. The engine doesn't matter; this issue applies to all of them.)

By shifting from a stand-alone machine to a job in a virtualization host, the PC becomes a job in a datacenter. It also becomes someone else's headache. The PC user is no longer the administrator. (Actually, the role of administrator in corporations shifted long ago, with Windows NT, domain controllers, centralized authentication, and group policies. Virtualization shifts the burden of CPU management to the central support team.)

The system administrators for virtualized PCs are true administrators, not PC owners who have the role thrust upon them. Real sysadmins pay attention to lots of performance indicators, including CPU usage, disk activity, and network activity. They pay attention because the operations cost money.

With virtual PCs, the processing occurs in the datacenter, and sysadmins will quickly spot the inefficient applications. The programs that consume lots of CPU and I/O will make themselves known, by standing out from the others.

Here's what I see happening:

- The shift to virtual PCs will continue, with today's PC users migrating to low-cost PCs and using Remote Desktop Connection (for windows) and Virtual Network Computing (for Linux) to connect to virtualized hosts. Users will keep their current applications.

- Some applications will exhibit poor response through RDP and VNC. These will be the applications with poorly written GUI routines, programs that require the virtualization software to perform extra work to make them work.

- Users will complain to the system administrators, who will tweak settings but in general be unable to fix the problem.

- Some applications will consume lots of CPU or I/O operations. System administrators will identify them and ask users to fix their applications. Users (for the most part) will have no clue about performance of their applications, either because they were written by someone else or because the user has no experience with performance programming.

- At this point, most folks (users and sysadmins) are frustrated with the changes enforced by management and the lack of fixes for performance issues. But folks will carry on.

- System administrators will provide reports on resource usage. Reports will be broken down by subunits within the organization, and show the cost of resources consumed by each subgroup.

- Some shops will introduce charge-back systems, to allocate usage charges to organization groups. The charged groups may ignore the charges at first, or consider them an uncontrollable cost of business. I expect pressure to reduce expenses will get managers looking at costs.

- Eventually, someone will observe that application Y performs well under virtualization (that is, more cheaply) while application X does not. Applications X and Y provide the same functions (say, word processing) and are mostly equivalent.

- Once the system administrators learn about the performance difference, they will push for the more efficient application. Armed with statistics and cost figures, they will be in a good position to advocate the adoption of application Y as an organization standard.

- User teams and managers will be willing to adopt the proposed application, to reduce their monthly charges.

And over time, the market will reward those applications that perform well under virtualization. Notice that this change occurs without marketing. It also forces the trade-off of features against performance, something that has been absent from the PC world.

Your job, if you are building applications, is to build the 'Y' version. You want an application that wins on performance. You do not want the 'X' version.

You have to measure your application and learn how to write programs that are efficient. You need the tools to measure your application's performance, environments in which to test, and the desire to run these tests and improve your application. You will have a new set of requirements for your application: performance requirements. All while meeting the same (unreduced) set of functional requirements.

Remember, "3 times 5" is not the same as "5 times 3".


Whining for SQL
2011-02-20T16:09:00.000-08:00
Several folks have been whining (and I do mean whining) about the lack of SQL in cloud computing. The common arguments are that SQL is the standard for database access, that it is familiar (meaning that a lot of people have learned it), that it is needed to build applications effectively.

To these arguments, I say "bunk".

SQL has a limited history in the computing age. It become popular in the 1980s. prior to then, we got along without SQL quite well. SQL is not needed to access data or build applications effectively.

I will pause here to disclose my bias: I dislike SQL. I think that the language is ugly, and I don't like ugly languages.

As I see it, SQL was the result of two forces. One was the popularity of relational databases, which in turn were driven by a desire to reduce redundant data. The second force was the desire to divide the work of application development cleanly between database design and application design. I'm not sure that either of these forces applies in today's world of technology.

Cloud applications may not need SQL at all. We may be able to create new methods of accessing data for cloud applications. (And we seem to be well on our way doing so.) Insisting that cloud apps use SQL is a misguided attempt at keeping an old (and ugly ... did I mention ugly?) data access mechanism. Similar thinking was common in the early days of microcomputers (the pre-IBM PC days) when people strove to implement FORTRAN and COBOL compilers on microcomputers and build systems for general ledger and inventory.

Google has no incentive to bring SQL to the cloud. Nor do Amazon.com and Salesforce.com. The players who do have incentive for SQL in the cloud are the vendors selling SQL databases: Microsoft and Oracle. I expect that they will find a way -- some way, any way -- to use SQL in cloud apps. And I expect those ways to be expensive.

But despite the efforts of Microsoft and Oracle, I expect cloud apps to thrive without SQL.


Should managers read code?
2011-02-16T20:04:00.000-08:00
A goal of the COBOL programming language was reducing the effort of coding to the point that managers could write their own programs. Perhaps this was a disguised form of a different goal -- the elimination of programmers entirely. Whatever the motivation, COBOL failed to deliver the result of "managers write code". All subsequent efforts (report writers, fourth-generation languages, natural language query systems, etc.) have also failed to make programming easy enough for managers.

After failing to make managers write code (or code writable by managers), the industry moved to a new model, one that separated managers and programmers into distinct manager and worker roles. With this separation, managers no longer wrote code or even looked at code, but instead performed project management tasks. This arrangement puts managers in the position of hiring, managing, and assessing programmers without direct contact of their work.

Perhaps the complete aversion to code is inappropriate. Perhaps managers needn't write code but can still review code, if the code is readable.

I will admit that most code is unreadable -- by programmers as well as managers. And I will admit that object-oriented code, if well written, is easier to understand than procedural code. (For any but the most trivial of tasks.) What would it take for managers to read and understand code?

I believe that the code must be in a language with minimal administrative code and readable syntax. The code must be tied to business concepts, not programming syntax.

In other words, something other than C++.

The code must be well-written object-oriented code, in either Java or C# (or possibly Python or Ruby). The code must be written to be read, thus have meaningful names for variables and classes and methods. The code must be organized into comprehensible chunks -- no thousand-line methods or hundred-method classes.

I see benefits to manager-readable code. First, managers will be able to comprehend the code and verify that the code is performing the work as expected. Second, programmers will have an easier time of fixing defects, since the code will match the real world data entities. (This was a highly touted attribute of object-oriented code.) A possible third benefit is for managers to understand the difficulties of fixing some defects, as they can see how the code does not "bend" in the desired direction. But this perhaps depends more on a manager willing to accept that the effort is hard, and not so much the ability to understand the problem.

Beyond those advantages, an organization can benefit in other ways. The organization can open the code to other teams -- other development teams, support teams, and testing teams. These teams can learn from the code and better understand failures. They can also comment on the code and provide additional insights and improvements. In a limited way, the code becomes "open source" -- but only within the organization.

If those advantages aren't enough, the other advantage I see is for newcomers to the team. They can learn the business by reading the code. A new member of the programming team has to learn many things beyond the location of the rest rooms and the cafeteria, and time spent learning business rules can be reduced by making the business rules visible in the code.

So I think that managers shouldn't be forced to write code, but we should strive to write code that can be read by managers. Readable code is understandable code.


Back around again
2011-02-08T19:07:00.001-08:00
A long time ago, when laptop computers first came on to the scene, the technology for LCD screens was different from today. The early screens worked (at a lower resolution that today's screens, of course) but could be viewed only from straight-on. Side viewers would not see the display.
With the (much ballyhooed) introduction of thin-film-transistors (TFTs), LCD displays were viewable at a wider angle. The development of TFTs was not easy, and it took a lot of work to get us there.

Now all LCD screens are viewable from a wide angle.

And we apparently don't want that.

I just saw an advertisement for an LCD screen cover that blocks the view from the side. It prevents another person from viewing the contents of your screen. The market is for executives travelling on airplanes, where a random person may sit next to them.

If we had just done nothing, we wouldn't have this problem and need this solution. Sometimes doing nothing is the right thing to do.

(Of course this ignores all of the people who *do* want to share their screen and let others view their information. but I think the point is worth considering. Sometimes, standing in one place is the best course of action.)




Excuse me, but your batch slip is showing
2011-02-02T17:56:00.001-08:00
The larger, older companies have a problem. They computerized their systems a while ago, and now they have slow, customer-unfriendly systems. This is caused by technology: the early automated systems were (are) batch systems, and they exhibit certain patterns. Delays in output is one such pattern.

Here are some examples.

I submitted a stock trade order through the Chase web site on Sunday evening. I understood that the trade would occur during business hours in New York, so the earliest possible time of the trade would be 9:30 on Monday morning.

The confirmation e-mail of my trade arrived at 2:25 on Tuesday morning. I'm pretty sure that the trade was not completed in the wee hours of the morning. I'm guessing that the trade occurred shortly after the market opened on Monday, and probably no later that 10:00. Yet Chase needed more than 16 hours to send the confirmation e-mail.

I'm guessing that the person executing the trade had confirmation quite a bit sooner than 16 hours. Chase apparently thinks that customers can wait.

In another case, Chase sent me a "year end summary available" notice... on February 2. (At 2:28 in the morning. Apparently Chase sends all of its e-mails starting at 2:00 in the morning.) I can understand that Chase would want to wait a few days, to let merchants send transactions and get a complete picture of the year. But waiting a whole month seems a bit extreme. (I'm guessing that Chase waits until the end of the billing cycle in January, then waits a bit more, and then sends the notification.)

The culprit here is the batch processing model. With batch processing, the workload is divided. Front-end systems collect data and back-end systems run jobs at specific intervals. These back-end systems are responsible for updating amounts and sending e-mails. Certain operations occur "on-line", that is, immediately. These operations are considered expensive and are therefore limited to a selected set of data and performed for a selected set of users -- generally not customers. The result is that Chase defines customers as second-class citizens.

I am picking on Chase here, but other large companies that uses batch processing and have the same "second-class citizen" attitude towards their customers.

Large companies have relied on batch processing for decades. And for decades, batch processing has been good enough. But no more. A new kid in town has raised the bar, and customers are not going to be satisfied with being treated as second class.

That new kid is Facebook.

Facebook (and other social networking sites) have built systems that provide immediate feedback for results. When I post my status on Facebook, I receive responses from friends within minutes. (Not days, and not hours. Certainly not months!)

Banks and other companies do not have the excuse of size or transaction volume to use as excuses. Facebook has over 500 million users; I know of no banks with similar customer bases. Facebook processes more messages than all of the transactions processed by banks -- combined!

The new generation of customers has been raised on Facebook and its immediate response model. They will be unhappy with "business the way we've always done it" and "you're a customer and you will wait". The bank that provides immediate feedback will win lots of customers.

This is not a technology problem. This is a management problem. The management at Chase is satisfied with the performance of their systems. (That is, they are satisfied that their systems are offending customers, since a small enough number complain.)

At some point the customers will revolt, but it will happen unevenly: mostly the younger customers will take their business elsewhere. Since younger customer tend to have smaller account balances, the averages will show an increase in assets per customer. This will be perceived by Chase management as a good thing. ("Average assets per customer is up, boss!")

But customer demographics is a funny thing. Companies with few young customers tend to have short lives. Their older customers remain (until they pass into the next dimension), and without a new crop of young customers to replace them, the customer base eventually shrinks. In the end, the company collapses into a fit of bankruptcy and acquisition.

The strategy for survival is to improve customer communication to a level close to Facebook. When the stock trade is complete, let me know, by e-mail, text message, Twitter, Facebook, or other method of my choice. Send me a year-end statement on January 4. Let me configure my account and notifications they way I want them, not the way your archaic batch system allows.

Or not.