Sunday, September 23, 2012

Project management is more than Microsoft Project

Microsoft Project is a nice tool, but I find that it is insufficient for managing projects. That is, it has incomplete information about the project and everything that I must manage.

When I manage projects, I think about the tasks for the project, delivery dates, and total effort. (All these items are in Microsoft Project.) I also worry about the skills of people on the team, including their technical skills, their familiarity with the code, and their people skills. (These items are not in Microsoft Project.)

Microsoft Project has elaborate information on the computed aspects of the project (total hours, etc.) but it does not know about the people. It assumes that people are interchangeable units.

Yet people are not interchangeable cogs. Individuals have specific knowledge about technology; some programmers know C++ and others PHP (and a few know both). Team members know the requirements for parts of the system but it is rare that a person knows the requirements for all areas. Some people get along with others, some are better at negotiating than others, and some have excellent skills at finding defects. I have to balance these skills and ensure that the different subteams have the right mix for their objectives (and I suspect that you have to balance those skills and needs too).

Yet Microsoft Project (and other project management software) provides no support for those kinds of analysis.

I find that simple techniques can support my decisions. For example, I often create a grid (by hand, on a blank piece of paper, with nothing more than a pen) with system subcomponents along one axis and people on the other. I then place an 'X' in each box for the match of a person with a subcomponent. When a person knows the subcomponent, I add an 'X'; when the person does not I leave the cell blank. (One can perform this exercise in a spreadsheet, if one desires a high-tech solution.)

This simple tool provides a summary of coverage for the system. I can quickly see which subcomponents have knowledge across the team and which can be supported by only a few (or perhaps even only one) team members. I use that knowledge when re-assigning or transferring team members.

Microsoft Project is clearly "necessary but not sufficient" for effective project management, yet I meet many managers who use Microsoft Project and only Microsoft Project. They perform no additional analyses. Consequently, they treat team members as interchangeable units and they act on that belief. And they often have difficulties meeting deadlines.

I sometimes think that Microsoft Project (and other project management software) and its blindness to individual skills and talents causes us managers to think of people as interchangeable units. If we have no way to note the differences between team members and our software cannot distinguish them, then we will think along those lines. Our tools shape our thoughts.

I also think, from time to time, that many managers want individuals to be interchangeable. Such a condition is easier to manage, requiring less thought. Changing project plans is easier, as there are fewer constraints.

But just because the management of a project is easier does not mean that we are more effective. Ignoring the tech and people skills of our team members does not mean that those skills go away or become meaningless. They may be ignored by the tool, but they still exist.

Wednesday, September 19, 2012

How to improve spreadsheets

Spreadsheets are the sharks of the IT world. They evolved before the IBM PC (the first spreadsheet ran on an Apple II) and have, for the most part, remain unchanged. They have migrated from the 6502 processor to the 8080, the Z-80, the 8086, and today's set. They have added graphs and fonts and extravagant file formats. They have grown from 256 rows to a rather large number. But they remain engines of data and formulas, with the data in well-defined grids and formulas with well-defined characteristics. They are sharks, evolved to a level of efficiency that has yet to be surpassed.

Spreadsheets get a number of things right:

- The syntax is easy to learn and consistent
- The data types are limited: numeric values, string values, and formulas
- Feedback for changes is immediate (no compiling or test scripts)
- Cells can only read values from other cells; they cannot assign a value to another cell

Immediate feedback is important. Teams using dynamic languages are at risk of a slew of problems; they use automated tests to identify errors. Agile processes emphasize the frequent use of tests. Spreadsheets provide this feedback without the need for tests.

The last item is most important. Cells can assign values to themselves, but they cannot assign values to other cells. This means that spreadsheets have no shared mutable state, no update collisions, no race conditions. That gives them stability.

Yet spreadsheets get a number of things wrong:

- All data is global
- Organization of data is dependent on the composer's skills and discipline

Our current spreadsheets are like the C language: fast, powerful, and dangerous. In C, one can do just about anything with the underlying machine. The C language lets you convert data from one form to another, and point to just about anything. It is quite powerful, but you have to know what you are doing.

Spreadsheets are not that dangerous. They don't have pointers, and the only things you can reference are (type safe because they are all of one type) cells within the spreadsheet.

But spreadsheets have the element of "you have to know what you are doing". The global nature of the data allows for formulas to refer to any cell (initialized or not) with little or no warning about nonsensical operations. In this sense, spreadsheet programming (in formulas) is much like C.

At first, I thought that the concepts of structured programming would improve spreadsheets. This is a false lead. Structured programming organizes code into sequences, iterations, and alternate paths. It clarifies code, but the formulas in spreadsheets are not a Turing-complete programming language. Structured programming can offer little to spreadsheets.

Instead, I think the concept of data encapsulation (from object-oriented programming) may help us advance the spreadsheet.

Spreadsheet authors tend to organize data into ranges. They may provide names for these ranges or leave them unnamed, but they will cluster their data into areas. Subsections of the grid will be used for specific types of data (for example, contact names in one range, regional sales in another).

For small spreadsheets, the "everything on one grid" concept works. Larger spreadsheets can see data split across pages (or tabs, depending on your spreadsheet manufacturer).

The problem with the spreadsheet grid is that it is, up to a point, infinite. We can add data to it without concern for the organization or structure. This becomes a problem over time; after a number of updates and revisions the effort to keep data organized becomes large.

An advanced spreadsheet would recognize that data is not stored in grids but in ranges, and would provide ranges as the key building block. Current spreadsheets let you define ranges, but the ability to operate on ranges is limited. In my new species of spreadsheet, the range would be the organizational unit, and ranges would not be infinite, empty grids. (They could expand, but only as a result of conscious action.)

Ranges are closer to tables in a database. Just as one can define a table and provide data for that table, one could define a range and provide data for that range. Unlike databases, ranges can be easily extended horizontally (more columns), re-sequenced, formatted, and edited. Unlike grids, ranges can be separated or re-combined to build new applications. Ranges must provide for local addresses (within the range) and external addresses (data within other ranges). Formulas must be able to read values from the current range and also from other grids.

If we do it right, ranges will be able to live in the cloud, being called in when needed and stored when not. Ranges will also be members of one application (or multiple applications), serving data to whatever application needs it.

Any improved spreadsheet will have to retain the advantages of the current tools. The immediacy of spreadsheets is a big advantage, allowing users of all skill levels to become proficient in a short time. Changing from grid-based spreadsheets to range-based spreadsheets must allow for this immediacy. This is a function of the UI, something that must be designed carefully.

I think that this new form of spreadsheet it possible, and offers some advantages. Now all I need is some time to implement it.

Sunday, September 16, 2012

The Agile and Waterfall Impedance Mismatch

During a lunchtime conversation with a colleague, we chatted about project management and gained some insights into the Waterfall and Agile project management techniques.

Our first insight was that an organization cannot split its project management methods. It cannot run some projects with a Waterfall process and other projects with Agile processes. (At least not if the projects must coordinate deliverables.) A company that uses Waterfall processes may be tempted to run a pilot project with Agile processes -- but if that pilot project "plugs in" to other projects, the result will be failure.

The problem is that Waterfall and Agile make two different promises. Waterfall promises a specific set of functionality, with a specific level of quality, delivered on a specific date. Agile makes no such promise;  it promises to add functionality and have a product that is always ready to ship (that is a high level of quality), albeit with an unknown set of functionality. The Agile process adds small bits of functionality and waits to get them correct before adding others -- thus ensuring that everything that has been added is working as expected, but not promising to delivery everything desired by a specific date. (I am simplifying things here. Agile enthusiasts will point out that there is quite a bit more to Agile processes.)

Waterfall processes make promises that are very specific in terms of feature set, quality, and delivery time -- but are not that good at keeping them. Hence, we have a large number of projects that are late or have low quality. Agile makes promises that are specific in terms of quality, and are good at keeping them. But the promises of the Agile processes are limited to quality; they do not propose the specifics that are promised by Waterfall.

With two different promises, it is no surprise that waterfall and agile processes don't co-exist. (There are other reasons that the two methods fail to cooperate, including the "design up front" of Waterfall and the "design as you go" of Agile.)

Our second insight was that transitioning an IT shop from Waterfall to Agile methods should not be accomplished by pilot projects.

Pilot projects are suitable to introduce people to the methods -- but those pilot projects must exist in isolation from the "regular" projects. Such projects were easy to establish in the earlier age of separate systems -- "islands of automation" -- that gave each system a measure of independence. Today, it is rare to see an IT system that exists in isolation.

Rather that use pilot projects, we like the idea of introducing ideas from Agile into the standard process used within the company. Our first choice is automated testing, the unit-level automated testing that can be performed by developers. Giving each developer the ability to run automated tests introduces them to a core practice of Agile, without creating an impedance mismatch.

After automated testing, we like the idea of allowing refactoring. Often omitted from plans for Waterfall projects, refactoring is another key practice of Agile development. Once unit tests are in place, developers can refactor with confidence.

Our third insight relates to project methods and project size. We think (and this is speculation) that Agile is better suited to small projects (and small systems) and Waterfall may be better suited to large systems. Thus, if you have large, complex systems, you may want to stay with Waterfall; if you have small systems (or even small applications) then you may want to use Agile.

We think that this relationship is correlation, not causal. That is, you can pick one side of the equation and drive the other. If you have large systems, you will end up with Waterfall. But the equation works in both directions, and you don't have to start with the size of your systems. If you use Agile methods, you will end up with smaller, collaborating systems.

Now, we realize that a lot of companies have large systems. We don't believe that by switching overnight to Agile methods will lead to smaller systems overnight. A base of large systems contains a certain amount of inertia, and requires a certain effect to redesign into smaller systems.

What we do believe is that you have a choice, that you can choose to use Agile (or Waterfall) methods (carefully, avoiding impedance mismatches), and that you can change the size of your systems. You can manage the size and complexity of your systems by selecting your processes.

These are choices that managers should accept and make carefully. They should not assume that "the system is the system" and "things are big and complicated and will always be so".

Thursday, September 13, 2012

The ultimate desktop OS

The phrase "ultimate desktop OS" is inspiring and attention-getting. While we might think that the "ultimate" desktop operating system is an unreachable dream, it is possible that it can exist, that it does exist, and that we have seen it.

That ultimate desktop operating system may be, in fact, Windows 7.

It is quite possible that Windows 7 is the peak of desktop operating systems. Its successor, Windows 8, is geared for tablets, not desktops. (And now you see why I have been carefully using the phrase "desktop operating system".)

Some might argue that it is not Windows 7 that is the "bestest" operating system for desktops, that the award for "best desktop operating system" should go to Windows XP, or perhaps Ubuntu Linux 10.04. These are worthy contenders for the title.

I won't quibble about the selection.

Instead, I will observe that desktop PCs have peaked, that they have reached their potential, and the future belongs to another device. (In my mind, that device is the tablet.)

Should you dispute this idea, let me ask you this: If you were to build a new app, something from scratch (not a re-hash of e-mail or word processing), would you build it for the desktop or for the tablet? I would build it for the tablet, and I think a majority of developers would agree.

And that is why I say that desktop operating systems have peaked. The future belongs to the tablet. (And the cloud, for back-end processing.)

If tablets are the future -- and I believe that they are -- then it really doesn't matter that Microsoft releases a new version of Windows for desktops. (Who gets excited when IBM releases a new version of MVS?) Yes, some folks will welcome the new version of Windows, but they will be a minority.

Instead of new versions of Windows, we will be looking for new versions of iOS and Android.

Tuesday, September 11, 2012

Apple got patch management right, at a cost

Looking at Apple's iOS and Microsoft's update services for Windows, one can see that Apple developed the more effective answer.

Apple's solution for iOS devices (iPads, iPhones, and iPods) covers all apps. With iTunes as the gateway for all apps, all apps can be updated through iTunes. One solution covers all apps.

In contrast, Microsoft's Windows has a messier solution. Windows update services cover Microsoft products, but other (non-Microsoft) products must provide their own solutions. The result is a hodge-podge of update methods, and while most use InstallShield they each have their eccentricities. Small-scale shops can handle the variance, but large shops have large headaches with patch management.

Patch management (or "update management", or "version management") are the headache that was solved with iOS and iTunes. It is also solved with Microsoft's new WinRT app manager (the Microsoft App Store) and is mostly solved with the major Linux distributions. (I say "mostly solved for the Linux distributions" since one can install software from outside the known repositories.)

The grand patch management solutions of iTunes and App Store provide a central location, a single method, for updating apps. The traditional Windows environment allowed for multiple applications from multiple vendors, and there was no one update manager that handled all of them.

But the neat solutions offered by Apple's iTunes and Microsoft's App Store come at a price. That price is part variety and part control. The centralized iTunes and App Store methods limit apps to those approved by the service administrators. Apple can reject any app for any reason (and has been accused of rejecting apps for trivial reasons). I'm certain that Microsoft's App Store will have similar challenges. Apple and Microsoft have "gatekeeper" control over their new distribution mechanisms: apps they like are permitted, apps they dislike are omitted.

Linux, with its strong emphasis on system administrators, has taken a different route. The Linux distros have central repositories for installation and update of packages, but the software managers use a "repository" model which sees software packages (which must conform to certain technical specifications) stored in repositories, and the set of repositories is open. That is, one can select the repositories for your software and receive updates from multiple sources. (The construction and maintenance of a repository takes some work, but the rules for building it are available and open.)

Patch management is important. Software is vulnerable to exploits, and patches can minimize one's exposure. It is something that Windows has worried and frustrated Windows administrators since the beginning of Windows.

In the future, when we look back, I think that the big contribution of iOS and Windows 8 will not be the new UI with its swipes and taps, nor the small form factor, but the patch management. A unified system for updates will be considered necessary for any "modern" system. Apple's iOS delivered that.


Friday, September 7, 2012

Windows 8 is version 1.0

Lots of people have critiqued Windows 8, and they seem to have forgotten the "rule of three" for Microsoft products. (The "rule of three" states that the early versions of new products are lame and that the third version of the product is the potent one. This rule also applies to the movie "The Maltese Falcon".)

Windows 8 is a step in a new direction for Microsoft. It defines a new UI and a new method for distributing software. It makes older versions of Windows obsolete.

There are a lot of changes, and I don't expect Microsoft to get it right on the first release. (They didn't with Windows, or with Visual Basic, or with .NET, either.)

What I *do* expect is that Microsoft will learn from their experience and release new versions. I expect a follow-on to Windows 8. Perhaps it will be called Windows 9. Perhaps it will be called Windows 8.5. Or Windows 8.1. Or maybe "Windows Tablet" or "Windows Mobile". Or maybe something prosaic like "Windows Touch". I don't know the exact name.

What I do know is that Microsoft releases new versions of its software. I expect a new version of its mobile device software, suitable for phones and tablets. I expect that Microsoft will fix the problems of the first release (what we call "Windows 8").

So I am not judging Microsoft solely on its Windows 8 product. I believe that they will release another version, and that later version will fix problems -- and change things.

Laud Microsoft or laugh at them. But don't expect them to stand still.

Wednesday, September 5, 2012

Microsoft's new revenue model

Windows 8 brings a number of changes to software. A new UI, a new Windows API, and a spiffy new logo. But there is one other change that has gotten little discussion.

Windows 8 brings a new revenue model for Microsoft.

Microsoft's former revenue model was to charge end users, server operators, and developers. That is, Microsoft charged fees for software that they provided (think Office), for programs running on servers (think SQL Server and Exchange), and for development tools (think Visual Studio). It is a system that has worked since the beginning of the PC era -- but it has a hole.

The hole in this revenue model has been third-party software. Were I to build software and sell it, I would pay Microsoft for the development tools (third-party development tools have long been pushed aside) but after that I was free to sell my software to anyone, with no royalties to Microsoft. The revenue collection point (the "tollbooth") was on the development tools.

Windows 8 moves the tollbooth for third-party software. Microsoft has moved the tollbooth from the receiving dock on third-party manufacturing facilities to the front door of the retail center (the Microsoft App Store, conveniently operated only by Microsoft).

This is the model used by Apple. Apple charges nominal fees for development tools and mandates that all apps are distributed through iTunes, where it can collect a part of the action.

One can assume that Microsoft will reduce the cost for development tools, perhaps even open-sourcing parts of their development tools. Doing so is in their best interest. The more people they have developing software for Windows 8 and selling (through the Microsoft App Store), the better for Microsoft. It is better for Microsoft to collect 30% on the sale of $5 retail units (sold in millions) than 100% on the sale of $500 development kits (sold in thousands).

Likewise, look to see Microsoft reduce the complexity of development for Windows 8. Instead of a large API with intricate calls, look to see a simplified API with easy-to-understand calls. A complicated API is good when you are selling the development tools; a simple API is better when you are selling (or collecting on) the retail units. You want to encourage the development of new apps.

I expect that the Windows 8 API will undergo a number of changes in the first few versions. Unlike the classic Windows API, the changes will simplify the API, not make it more complex. Microsoft has strong incentives to make it easy for the development of Windows 8 apps. (Besides the revenue model, Microsoft must compete with Apple and Android.)

The new model holds for the Windows 8 retail apps, the ones sold through the Microsoft App Store. For applications that run on servers, the old revenue model applies. Expect a bifurcation of Microsoft products and pricing, with cheap and easy-to-use tools for the development of retail apps and expensive and hard-to-use tools for server-based applications. (Until Microsoft releases their Microsoft Server App Store, which lets it collect revenue on the sale and use of server-based apps.)