Our technology shapes our systems

In the old days, computer programs were fairly linear things. They processed data in a linear fashion, and the source code often appeared in a linear fashion.

Early business applications of computers were for accounting applications: general ledger, accounts payable, payroll, inventory... etc. These systems were often designed with a master file and one or more transaction files. The master file held information about customers, accounts, and inventory, and the transaction files held information about, well, transactions, discrete events that changed something in the master file. (For example, a bank's checking accounts would have balances in the master file, and records in the transaction file would adjust those balances. Deposits would increase a balance, checks or fees would decrease a balance.)

The files were not stored on modern devices such as USB drives or even floppy disks. In the early days, the master file was on magnetic tape, and the transactions were on punch cards.

The thing about magnetic tape is that you must run through it from beginning to end. (Much like a tour through an Ikea store.) You cannot jump around from one position to another; you must start with the first record, then process the second record, and in sequence process every record until the last.

The same holds for punch cards. Paper punch cards were placed in a hopper and read and processed one at a time.

You might wonder how you can handle processing of accounts with such restrictions in place. One pass through the master file? And only one pass through the transactions? How can we match transactions to master records if we cannot move to the proper record?

The trick was to align the input files, keeping the master file sorted and sorting the transactions before starting the update process. With a bit of thought, you can imagine a system that reads a master record and a transaction record, compares the account numbers on each (both records need a key for matching) and if they match then updates the master record and moves on to the next transaction. If they don't match then the system stores the master record (on another tape, the output tape) and runs the comparison again. The algorithm does work (although I have simplified it somewhat) and this was a common model for program design.

The rise of direct-access storage devices and complex data structures has changed programming. As processors became less expensive and more powerful, as programming languages became more expressive and allowed complex data structures such as lists and trees, as memory became available to hold complex data structures in their entirety, our model for programming became more complex. No longer were programs limited to the simple cycle of "read-master, read-transaction, compare, update, write-master, repeat".

Programming in that model (perhaps we could call it the "Transaction Pattern") was easy and low-risk because clever people figured out the algorithm and other people could copy it.

This notion of a common system model is not unique to 1960s-style programming. Microsoft Windows programs at the API level follow a specific pattern of messages sent by the Windows core "message pump". Android programs use a similar technique.

Tablet/cloud systems will probably develop one (or perhaps a handful) of common patterns, repeated (perhaps with some variations) for the majority of applications. The trick will be to identify the patterns that let us leverage the platform with minimal thought and risk. Keep your eyes open for common templates for systems. When you find one that works, when you find one that lets lots of people leverage the cleverness of a few individuals, stick with it.

I'm guessing that the system model will not be a purely linear one, as we had in the 1960s. But it may have linear aspects, with message queues serializing transactions and updates.