Microsoft discovers the need for an ecosystem?

I believe that we have seen a significant change in Microsoft's philosophy in 2011, one that will lead to a number of changes in the software market.

Allow me to point to two events in the past year:

• The decision to support HTML5 and JavaScript as first-level development tools. This is a big change from the C#-centric world of .NET.
• The (somewhat quieter) decision to bundle InstallShield LE with Visual Studio, and drop the Microsoft-built install packager. This is also a big change in the "we supply everything" strategy.

These two changes mark a significant shift in Microsoft's strategy. For the past decades (since the introduction of Windows), Microsoft has had a strategy of being the sole source for all Windows-based software. Microsoft was determined to be the dominant supplier in every market, from operating system to office applications, from development applications to business applications.

This "provide everything" strategy has been successful. If you look at a random PC running Windows today, I expect that you will find that it runs Microsoft applications, with possibly a few home-grown applications and possible some applications from Adobe.

Here's a history of the big products in the Windows world:

• Microsoft made Word the best -- or at the least most popular -- word processor, beating the competition from WordStar, WordPerfect, and every other competitor.
• Microsoft made Excel the best -- or the most popular -- spreadsheet, winning the battle with Lotus 1-2-3 and defeating smaller competitors like Borland's Sprint.
• Microsoft's PowerPoint is the standard presentation software for Windows. There are no competitors to speak of.
• Microsoft's Access (and later, SQL Server) pushed aside all other database engines: dBase, Paradox, Reflex, R:base, dbVista, and others. A few small competitors (like Faircom) exist in niche markets. IBM's DB2 and Oracle's products still exist, but are less "Windows products" and more "Windows implementations" of multi-platform products.
• Microsoft made Visual Studio the IDE of choice, driving Borland out of the market.
• Microsoft purchased the SourceSafe product, integrated it with Visual Studio, and made it the dominate version control system for Windows. (Microsoft recently replaced the old SourceSafe product with Team Foundation Server.) Other players exist, but only as a small portion of the market.
• Microsoft overpowered Netscape, which re-incarnated itself as the open-source Mozilla project. Browsers are conspicuous in their plurality in Windows; no other major application has this status.
• Microsoft absorbed network functions into Windows. (Remember Novell Netware?)
• Microsoft absorbed virus-checking into Windows.
• Microsoft built media capabilities into Windows, eliminating third-party music players and video players.
• Microsoft developed SilverLight to compete with Adobe Flash, and has rolled out several successful, capable versions. I suspect that they would soon displace Adobe, had it not been for HTML5 trumping both Flash and SilverLight.

I'm charging of abuse of monopoly power, nor of using internal technical information to develop superior products. (Others have raised such issues.) I am looking at the results of Microsoft's actions, regardless of their methods. And the results are these: the Microsoft ecosystem is dying.

Compared to the ecosystem for the Apple iPhone/iPad market, the ecosystem for Microsoft is small. Lots of people, from individual developers to large companies, are developing apps for IOS. In contrast, few folks are building applications for Windows.

Comparing the Windows ecosystem of today against the Windows ecosystem of two decades ago shows the same pattern: fewer developers today.


This is no surprise. The only winning strategy in the commercial market is to become big. One cannot succeed by staying a small company. (Yes, I recognize that there are lots of small companies writing software for Windows. But are they successful? I humbly submit that they have plans to become larger, and are simply waiting for the "right market" or the "right opportunity".)


The problem with the Windows ecosystem is that Microsoft kills any company that becomes too large. (How large is "too large" is defined by Microsoft.) How can one even consider Windows as a long-term environment for a product? You either stay small or become large and get crushed by the Microsoft empire. And there are opportunities in the IOS and Android markets. Developers have noticed the possibilities.

And I think Microsoft has realized this.

The past few years Microsoft has been claiming that they have a large ecosystem; the claim is to impress (or soothe) corporate buyers. But looking at the products on the market and the attendance at Microsoft conferences and fairs (and looking closely at the demographics and not just the numbers) one can see that Microsoft is not winning the hearts of new developers.

I think that the ecosystem has been changing quietly over the past decade (since the introduction of Linux and the original iMac computers). Developers have been moving to the non-Microsoft platforms in response to the expense, the "buy one Microsoft thing and you need another Microsoft thing" dependencies of products, and the threat of competition from Microsoft.

After a decade of quiet changes, the difference is significant. Microsoft recognizes it, and realizes that they need to change.

I expect that Microsoft will change from a "we supply everything" shop and focus on items of strategic importance. Those items will be money-makers and important system components. Here's a plan for Microsoft:

• Keep Windows, but make significant revisions. Windows is necessary for the Microsoft world. The brand is valuable. Lots of customers are committed to it and are not willing to move to other platforms. But large customers want better security and easier administration, and small customers want lower expenses and easier administration.
• Keep ActiveDirectory. It competes with LDAP and is easier to administrate.
• Keep Exchange for large customers. Microsoft must offer a cloud-based e-mail/calendar solution for smaller customers.
• Keep parts of Microsoft Office. Word and Excel can compete with the Open Office and Libre Office products. Drop PowerPoint. Keep Visio.
• Keep Visual Studio and C#. Drop Visual Basic. Increase support for F#.
• Drop Internet Explorer. (It offers no strategic value, and other browsers do not harm Microsoft's web offerings.)
• Drop IIS. (It offers no strategic value.)
• Enhance SQL Server and Access (the front-end GUI) to support data in the cloud as well as local data. Provide data in the cloud so that customers need only Access on their hardware.
• Develop the Microsoft App Store (or whatever they call it) and allow others to sell applications.
• Develop an update system that updates all apps, not just the Microsoft-supplied programs.

The decision to drop products like Visual Basic, IE, and IIS may strike some as foolish. It may be my personal bias that dictates the decision for VB, but IE and IIS offer no revenue to Microsoft. The need for IE is long gone; the winning strategy is to have superior web applications, not control of the browser. Microsoft would be better focussing their efforts on superior web applications.

One risk of dropping these products is that once customers convert to other products they may see value in non-Microsoft products. By pushing customers to non-Microsoft products, Microsoft legitimizes non-Microsoft solutions. That is a risk that Microsoft must counter by providing superior value in the products it retains in its offerings.

Despite the risks, I think that this is a good direction for Microsoft. By letting non-Microsoft products thrive, Microsoft can restore developers' faith in the Microsoft ecosystem and encourage them to consider it for new projects. I see it as the best way for Microsoft to succeed.

Predictions for 2012

Happy new year!

The turning of the year provides a time to pause, look back, and look ahead. Looking ahead can be fun, since we can make predictions.

Here are my predictions for computing in the coming year:

With the rise of mobile apps, we will see changes in project requirements and in the desires of candidates.

The best talent will work on mobile apps. The best talent will -- as always -- work on the "cool new stuff". The "cool new stuff" will be mobile apps. The C#/.NET and Java applications will be considered "that old stuff". Look for the bright, creative programmers and designers to flock to companies building mobile apps. Companies maintaining legacy applications will have to hire the less enthusiastic workers.

Less funding for desktop applications. Desktop applications will be demoted to "legacy" status. Expect a reduced emphasis on their staffing. These projects will be viewed as less important to the organization, and will see less training, less tolerance for "Fast Company"-style project teams, and lower compensation. Desktop projects will be the standard, routine, bureaucratic (and boring) projects of classic legacy shops. The C# programmers will be sitting next to, eating lunch with, and reminiscing with, the COBOL programmers.

More interest in system architects. Mobile applications are a combination of front end apps (the iPhone and iPad apps) and back-end systems that store and supply data. Applications like Facebook and Twitter work only because the front end app can call upon the back end systems to obtain data (updates submitted by other users). Successful applications will need people who can visualize, describe, and lead the team in building mobile applications.

More interest in generalists. Companies will look to bring on people skilled in multiple areas (coding, testing, and user interfaces). They will be less interested in specialists who know a single area -- with a few exceptions of the "hot new technologies".

Continued fracturing of the tech world. Amazon.com, Apple, and Google will continue to build their walled gardens of devices, apps, and media. Music and books available from Amazon.com will not be usable in the Apple world (although available on the iPod and iPad in the Amazon.com Kindle app). Music and books from Apple will not be available on Amazon.com Kindles and Google devices. Consumers will continue to accept this model. (Although like 33 RPM LPs and 45 PRM singles, consumers will eventually want a music and books on multiple devices. But that is a year or two away.)

Cloud computing will be big, popular, and confused. Different cloud suppliers offer different types of cloud services. Amazon.com's EC2 offering is a set of virtual machines that allow one to "build up" from there, installing operating systems and applications. Microsoft's Azure is a set of virtual machines with Windows/.NET and one may build applications starting at a higher level that Amazon's offering. Salesforce.com offers their cloud platform that lets one build applications at an even higher level. Lots of folks will want cloud computing, and vendors will supply it -- in the form that the vendor offers. When people from different "clouds" meet, they will be confused that the "other guy's cloud" is different from theirs.

Virtualization will fade into the background. It will be useful in large shops, and it will not disappear. It is necessary for cloud computing. But it will not be the big star. Instead, it will be a quiet, necessary technology, joining the ranks of power management, DASD management, telecommunications, and network administration. Companies will need smart, capable people to make it work, but they will be reluctant to pay for them.

Telework will exist, quietly. I expect that the phrase "telework" will be reserved for traditional "everyone works in the office" companies that allow some employees to work in remote locations. For them, the default will be "work in the office" and the exception will be "telework". In contrast, small companies (especially start-ups) will leverage faster networks, chat and videoconferencing, mobile devices, and social networks. Their standard mode of operation will be "work from wherever" but they won't think of themselves as offering "telework". From their point of view, it will simply be "how we do business", and they won't need a word to distinguish it. (They may, however, create a word to describe folks who insist on working in company-supplied space every day. Look for new companies to call these people "in-house employees" or "residents".)

Understand the sea change of the iPad. The single-app interface works for people consuming information. The old-fashioned multi-windowed desktop interface works for people composing and creating information. This change leads to a very different approach to the design of applications. This year people will understand the value of the "swipe" interface and the strengths of the "keyboard" interface.

Voice recognition will be the hot new tech. With the success of "Siri" (and Android's voice recognizer "Majel"), expect interest in voice recognition technology and apps designed for voice.

Content delivery becomes important. Content distributors (Amazon.com, Google, and Apple) become more important, as they provide exclusive content within their walled gardens. The old model of a large market in which anyone can write and sell software will change to a market controlled by the delivery channels. The model becomes one similar to the movie industry (a few studios producing and releasing almost all movies) and the record industry (a few record labels producing and releasing almost all music) and the book industry (a few publishing houses... you get the idea).

Content creation becomes more professional. With content delivery controlled by the few major players, the business model becomes less "anyone can put on a show" and more of "who do you know". Consumers and companies will have higher expectations of content and the abilities of those who prepare it.

Amateur producers will still exist, but with less perceived value. Content that is deemed "professional" (that is, for sale on the market) will be developed by professional teams. Other content (such as the day-to-day internal memos and letters) will be composed by amateur content creators: the typical office worker equipped with a word processor, a spreadsheet, and e-mail will be viewed as less important, since they provide no revenue.

Microsoft must provide content provider and enable professional creators. Microsoft's business has been to supply tools to amateur content creators. Their roots of BASIC, DOS, Windows, Office, and Visual Basic let anyone (with or without specific degrees or certifications) create content for the market. With the rise of the "professional content creator", expect Microsoft to supply tools labeled (and priced) for professionals.

Interest in new programming languages. Expect a transition from the object-oriented languages (C++, Java, C#) to a new breed of languages that introduce ideas from functional programming. Languages such as Scala, Lua, Python, and Ruby will gain in popularity. C# will have a long life -- but not the C# we know today. Microsoft has added functional programming capabilities to the .NET platform, and modified the C# language to use them. C# will continue to change as Microsoft adapts to the market.

The new year brings lots of changes and a bit of uncertainty, and that's how it should be.

Vox populi

Apple's "Siri" has shown that voice recognition is not only possible but also practical (and fun). Voice-driven apps are naturals for the smartphone and tablet world, where keyboards and mice are foreigners. A voice-driven app solves the problem of composition on a tablet app: most smartphone and tablet apps are better for the consumption of data, with desktop apps better for composing data.

The new world of voice-driven apps will bring a number of changes.

Voice-driven apps require a new design. One cannot simple replace a keyboard and mouse with voice-driven commands; the user experience is too different.

The (relatively) commonplace task of designing a GUI for a program becomes problematic with voice-directed apps. Do we use commands such as "place a button below the list box"? This may lead to a renaissance of SHRDLU and the old "Adventure" and "Zelda" programs, with their commands of "go north" and "take everything but the snake".

The testing of apps will change, and require new tools and techniques. How does one test a voice-driven app? Do you have people speak the commands, or do you have pre-recorded commands and play them back to the app? How can you build a suite of automated tests for voice-driven apps?

Our notions of the workplace will change. For the past four decades, we have built workplaces from cubicles. Cubicles are sufficient for people to type on keyboards, but are poor environments for speaking to computers. With voice-driven apps, and everyone speaking at their computer, the noise in the typical office increases. Voice-recognition software may be able to filter out background noise and voices; humans may have a harder time of it. Do we change our workplaces to individual offices?

What about the folks working at home, or working in co-working locations? We'll need quiet places to perform our work. At home, that may mean a separate room with a door. Co-working sites may change to suites of hard-walled offices.

The introvert/extrovert gap may be significant with voice-driven apps. Introverts emphasize written text, and they spend a lot of time composing their words. Extroverts speak readily, with less weight on their ideas. The extroverts share their ideas early and look to the group to help shape the final concepts; introverts think up front and have already decided by the time they hand you the document. I expect that extroverts will be more comfortable (or perhaps less uncomfortable) with voice-driven apps.

Do we combine voice-driven with gesture-driven? Microsoft's Kinect has shown itself to be capable and reliable. Perhaps we will have a combination of voice and gesture to control computers and to create content. I can easily see the layout of a GUI being developed by a programmer with voice-driven and gesture-driven commands. "Place a new button at the bottom of the screen", says the developer, and the IDE will create a new button. "Change the text to 'Cancel'" says the developer, "Now move the button to the right" and the programmer gestures with his hands, pushing an imaginary button to the right until it is in the desired position.

Voice recognition is here. I see lots of changes, for developers, testers, and users.

The wonder of Git

I say "git" in the title of this post, but this is really about distributed version control systems (DVCS).

Git is easy to install and set up. It's easy to learn, and easy to use. (One can make the same claim of other programs, such as Mercurial.)

It's not the simply installation or operation that I find interesting about git. What I find interesting is the organization of the repositories.

Git (and possibly Mercurial and other DVCS packages) allows for a hierarchical collection of repositories. With a hierarchical arrangement, a project starts with a single repository, and then as people join the project they clone the original repository to form their own. They are the committers for their repositories, and the project owner remains the committer for the top-most repository. (This description is a gross over-simplification; there can be multiple committers and more interactions between project members. But bear with me.)

The traditional, "heavyweight" version control systems (PVCS, Visual SourceSafe, TFS) use a single repository. Projects that use these products tend to allow everyone on the project to check in changes -- there are no committers, no one specifically assigned to review changes and approve them. One can set policies to limit check-in privileges, although the mechanisms are clunky. One can set a policy to manually review all code changes, but the VCS provides no support for this policy -- it is enforced from the outside.

The hierarchical arrangement of multiple repositories aligns "commit" privileges with position in the organization. If you own a repository, you are responsible for changes; you are the committer. (Again, this is a simplification.)

Once you approve your changes, you can "send them up" to the next higher level of the repository hierarchy. Git supports this operation, bundling your changes and sending them automatically.

Git supports the synchronization of your repository with the rest of the organization, so you get changes made by others. You may have to resolve conflicts, but they would exist only in areas of the code in which you work.

The capabilities of distributed version control systems supports your organization. They align responsibility with position, requiring more responsibility with authority. (If you want to manage a large part of the code, you must be prepared to review changes for that code.) In contrast, the older version control systems provide nothing in the way of support, and sometimes require effort to manage the project as you would like.

This is a subtle difference, one that is not discussed. I suspect that there will be a quiet revolution, as projects move from the old tools to the new.

The character of programming languages

Many languages use C-style blocks denoted with braces (the characters '{' and '}').

The BCPL programming language was the first language to use braces as part of its syntax. Earlier languages (notably COBOL, FORTRAN, algol, and LISP) did not use the brace characters.

Earlier languages did not use brace characters because the characters did not exist, at least not as defined characters. There was little in the way of standards for character sets, with each vendor (and sometimes each system) using its own character set. For a language to run on multiple computers, one had to limit the characters used in the language to those available on all planned platforms. Thus, FORTRAN uses uppercase letters and parentheses but not square brackets.

With the introduction of the ASCII and EBCDIC character sets, things changed. A standard character set (well, two standards) let one assume the existence of all of the defined characters.

First published in 1963, the character sets predate the effort to build BCPL in 1966. Thus, when BCPL was designed, the brace characters were present and ready to be used. They also have the virtue of not being used for anything before.

Our character sets define, to some extent, the syntax of our languages.

In open source, can there be more than one?

In the commercial market, multiple products is considered a healthy sign. The prevailing logic states that competition is a good thing, giving us the best possible product.

Vendors must position their products with a balance of different features and compatible operations. A word processor must provide some unique set of functions, but must provide some core set of functions to be considered a word processor. It must provide some compatibility to be considered useful to new customers (perhaps to convert their existing files). A bold, new approach to letter-writing, an approach that varies from the conventions of current products, will have a difficult time gaining acceptance. A word processor that performs the basic tasks of existing word processors, that is ten percent better at most things, and that offers a few new ideas, has a better chance of success. The commercial market allows for different, similar products.

The commercial market also has the risk of failure. Building a system on a product (say, a compiler or a  version control system) builds in the risk of that product. Companies fail, and products are discontinued (even when the vendor succeeds). The user must choose carefully from the available products.

In the open source ecosystem, the dynamics are different. Multiple products (or projects) are not viewed as a necessity. Consider the popular open source solutions for different tasks: Linux, LibreOffice, GIMP, gcc, SendMail, and NFS. There are competing offerings for these functions, but the "market" has settled on these projects. The chances of a project replacing the Linux kernel, or the GIMP package, are low. (Although not zero, as LibreOffice recently replaced OpenOffice.)

Open source is not monolithic, nor is it limited to single solutions. There are competing ideas for scripting languages (Perl, Python, Ruby) and editors (vi and Emacs). There are competing ideas for databases (MySQL and PostGres, not to mention CouchDB).

I think that it is harder for an open source project to remain independent from the lead project than it is for a commercial product to remain independent from the market leader.

In open source, your ideas (and source code) are available. A small project that is mostly compatible with a large project can be absorbed into the large project. To remain independent, a project must remain different in some core aspect. The languages Perl, Python, and Ruby are all different. The editors vi and Emacs are different. Because of their differences, they can continue to exist as independent projects.

For most software functions, I believe that there is a "Highlander effect": there can be only one. There will be one wildly popular kernel, one wildly popular office suite, one wildly popular C++ compiler.

When there are "competing" open source projects, they will either eventually merge or eventually distance themselves (as with the case of vi and Emacs).

A popular open source project can "absorb" other, similar open source projects.

This effect will give a degree of stability to the ecosystem. One can build systems on top of the popular solutions. A system built with Linux, GNU utilities, gcc, and Python will endure for many years.

Tradeoffs

It used to be that we had to write small, fast programs. Processors were slow, storage media (punch cards, tape drives, disc drives) were even slower, and memory was limited. In such a world, programmers were rewarded for tight code, and DP managers were rewarded for maintaining systems at utilization rates of ninety to ninety-five percent of machine capacity. The reason was that a higher rate meant that you needed more equipment, and a lower rate meant that you had purchased (or more likely, leased) too much equipment.

In that world, programmers had to make tradeoffs when creating systems. Readable code might not be fast, and fast code might not be readable (and often the two were true). Fast code won out over readable (slower) code. Small code that squeezed the most out of the hardware won out over readable (less efficient) code. The tradeoffs were reasonable.

The world has changed. Computers have become more powerful. Networks are faster and more reliable. Databases are faster, and we have multiple choices of database designs -- not everything is a flat file or a set of related tables. Equipment is cheap, almost commodities.

This change means that the focus of costs now shifts. Equipment is not the big cost item. CPU time is not the big cost item. Telecommunications is not the big cost item.

The big problem of application development, the big expense that concerns managers, the thing that will get attention, will be maintenance: the time and cost to modify or enhance an existing system.

The biggest factor in maintenance costs, in my mind, is the readability of the code. Readable code is easy to change (possibly). Opaque code is impossible to change (certainly).

Some folks look to documentation, such as design or architecture documents. I put little value in documentation; I have always found the code to be the final and most accurate description of the system. Documents suffer from aging: they were correct some but the system has been modified. Documents suffer from imprecision: they specify some but not all of the details. Documents suffer from inaccuracy: they specify what the author thought the system was doing, not what the system actually does.

Sometimes documentation can be useful. The business requirements of a system can be useful. But I find "System architecture" and "Design overview" documents useless.

If the code is to be the documentation for itself, then it must be readable.

Readability is a slippery concept. Different programmers have different ideas about "readability". What is readable to me may not be readable to you. Over my career, my ideas of readability have changed, as I learned new programming techniques (structured programming, object-oriented programming, functional programming), and even as I learned more about a language (my current ideas of "readable" C++ code are very different from my early ideas of "readable" C++ code).

I won't define readability. I will let each project decide on a meaningful definition of readability. I will list a few ideas that will let teams improve the readability of their code (however they define it).

Version control for source code A shop that is not using version control is not serious about software development. There are several reliable, well-documented and well supported, popular systems for version control. Version control lets multiple team members work together and coordinate their changes.

Automated builds An automated build lets you build the system reliably, consistently, and at low effort. You want the product for the customer to be built with a reliable and consistent method.

Any developer can build the system Developers need to build the system to run their tests. They need a reliable, consistent, low-effort, method to do that. And it has to work with their development environment, allowing them to change code and debug the system.

Automated testing Like version control, automated testing is necessary for a modern shop. You want to test the product before you send it to your customers, and you want the testing to be consistent and reliable. (You also want it easy to run.)

Any developer can test the system Developers need to know that their changes affect only the behaviors that they intend, and no other parts of the system. They need to use the tests to ensure that their changes have no unintended side-effects. Low-effort automated tests let them run the tests often.

Acceptance of refactoring To improve code, complicated classes and modules must be changed into sets of smaller, simpler classes and modules. Refactoring changes the code without changing the external behavior of the code. If I start with a system that passes its tests (automated tests, right?) and I refactor it, it should pass the same tests. When I can rearrange code, without changing the behavior, I can make the code more readable.

Incentives for developers to use all of the above Any project that discourages developers from using automated builds or automated tests, either explicitly or implicitly, will see little or no improvements in readability.

But the biggest technique for readable code is that the organization -- its developers and managers -- must want readable code. If the organization is more concerned with "delivering a quality product" or "meeting the quarterly numbers", then they will trade off readability for those goals.