Agile, Waterfall, and Risk

For some years (decades, really), software development has used an agile approach to project management. The Agile method sees short iterations that each focus on a single feature, with the entire team reviewing progress and selecting the feature for the next iteration. Over time, a complete system evolves. The advantage is that the entire team (programmers, managers, salespersons, etc.) learn about the business problem, the functions of the system, and the capabilities of the team. The team can change course (hence the name "agile") as they develop each feature.

Prior to Agile, for some years (decades, really), software development used the "waterfall" approach to project management. The Waterfall method starts with a set of requirements and a schedule, and moves through different phases for analysis, design, coding, testing, and deployment. The important aspect is the schedule. The Waterfall method promises to deliver a complete system on the specified date.

This last aspect of Waterfall is quite different from Agile. The Agile method makes no promise to deliver a completed system on a specific date. It does promise that each iteration ends with a working system that implements the features selected by the team. Thus, a system developed with Agile is always working -- although incomplete -- whereas a system developed with Waterfall is not guaranteed to work until the delivery date.

(It has been observed that while the Waterfall method promises a complete, working system on the specified delivery date, it is quite poor at keeping that promise. Many projects overrun both schedule and budget.)

Here is where risk comes into play.

With Agile, the risk is shared by the entire team, key among these are developers and managers. An agile project has no specified delivery date, but more often than not senior managers (those above the agile-involved managers) have a date in mind. (And probably a budget, too.) Agile projects can easily overrun these unstated expectations. When they do, the agile-involved managers are part of the group held responsible for the failure. Managers have some risk.

But look at the Waterfall project. When a waterfall project fails (that is, runs over schedule or budget) the managers have way to distance themselves from the failure. They can say (honestly) that they provided the developers with a list of requirements and a schedule (and a budget) and that the developers failed to meet meet the "contract" of the waterfall project. Managers can deflect the risk to the development team.

(For some reason, we rarely question the feasibility of the schedule, or the consistency and completeness of the requirements, or the budget assigned to the project. These are considered "good", and any delay or shortcoming is therefore the fault of the developers.)

Managers want to avoid risk -- or at least transfer it to another group. Therefore, I predict that in the commercial space, projects will slowly revert from Agile methods to Waterfall methods.

WFH and the real estate crisis

Over the past decades, companies (that is, employers) have shifted responsibilities (and risk) to their employees.

Employer companies replaced company-funded (and company-managed) pension plans with employee-funded (and employee-managed) 401-k retirement plans.

Employer companies have shifted the cost of medical insurance to employees. The company-run (and company-funded) benefit plan is a thing of the past. Today, the hiring process includes a form for the employee to select insurance options and authorize payment via payroll deduction.

Some companies have shifted other forms of risk to employees. Restaurants and fast-food companies, subject to large swings in demand during the day, have shifted staffing risk to employees. Instead of a steady forty-hour workweek with eight-hour shifts, employers now schedule employees with "just in time" methods, informing employees of their schedule one day in advance. Employees cannot plan for many activities, as they may (or may not) be scheduled to work in any future day.

In all of these changes, the employer shifted the risk to the employees.

Now we come to another form of risk: real estate. It may seem strange that real estate could be a risk. And it isn't; the risk is the loans companies have to buy real estate.

Many companies cannot afford the loans for their buildings. Here's why: A sizable number of companies have allowed employees to work from home (or locations of their own choosing), and away from the office. As a result, those companies need less office space than they needed in the past. So they rent less space.

It's not the tenant companies that have the risk of real estate loans -- it's the landlord companies. They made the loans and purchased the buildings.

But risk is risk, and it won't take long for landlord companies to shift this risk away from themselves. But this shift won't be easy, and it won't be like the previous shifts.

A building has two (or perhaps more) companies. One that owns the building (the landlord company), and a second (the tenant company) that leases space within. (The owner could be a group of companies in a joint effort. And a large building could have more than one tenant.)

But notice that this risk has two levels of corporations: the landlord company and the tenant company. The landlord company has employees, but they are not the employees who work in the building. Shifting the risk to them makes no sense.

The employees who work in the building are employees of the tenant company, and they have no direct connection to the landlord company. The landlord company cannot shift the risk to them, either.

Thus, the shift of risk (if a shift does occur) must move between the two companies. For the risk of real estate, the landlord company must shift the risk to its tenant companies.

That shift is difficult. It occurs not between an employer and employee, but between two companies. Shifting risk from employer to employee is relatively easy, due to the imbalance of power between the two. Shifting risk between companies is difficult: the tenant company can hire lawyers and fight the change.

If the owning company is successful, and does manage to shift the risk to its tenant company, then one might assume that the tenant company would want to shift the risk to its employees. That shift is also difficult, because there is little to change in the employment arrangement. Medical benefits and pension plans were easy to change, because employees were receiving something. With the risk of building ownership (or more specifically the risk of a lower value for the building) the employee is currently receiving... nothing. The have no share in the building, no part of the revenue, no interest in the transaction.

Savvy readers will have already thought of other ways of hedging the risk of real estate loans (or the risk of reduced demand for real estate). There are other ways; most involve some form of insurance. With them, the landlord company purchases a policy or some other instrument. The risk is shifted to a third company (the insurer) with payments.

I expect that the insurance option will be the one adopted by most companies. It works, it follows existing patterns of business, and it offers predictable payments to mitigate risk.

Sometimes you can shift risk to employees. Sometimes you can't.

Locked out of the walled gardens

A dark side of the walled gardens of technology is becoming visible.

The old paradigm was one of home ownership. You purchased your equipment (a PC) and then you installed whatever software you wanted. You decided. There were constraints, of course. PCs typically ran Windows (or before that, DOS) and the software had to run under the operating system. PCs had various configurations and the software had to fit (a floppy-only PC could not run large software that required a hard drive, for example).

Once installed, you had to see to the care and feeding of the software. You had to ensure that updates were applied. You had to ensure that data you exchanged with other users was compatible with their systems.

The benefit of this paradigm is the open market and the freedoms that come with it. Vendors are free to enter the market and offer their wares. You are free to choose among those products. You could pick from a number of word processors, spreadsheets, databases, compilers and IDEs, project managers, and other product categories.

The walled gardens of iOS and Android (and soon Windows and MacOS X) provide a different paradigm. If the old paradigm was one of home ownership, the new paradigm is one of renting an apartment. You still have a place for your stuff, yet a lot of the tedious chores of ownership have been removed.

With Apple's walled garden of iOS (and the gatekeeper iTunes), updates are automatic, and software is guaranteed to be compatible. The same holds for Google's Android garden and its gatekeeper 'Play Store'. They guard against 'unfriendly' software.

But the price of living inside the walled garden is that one loses the open market. Only selected vendors may enter the market, and those that do may offer a limited selection of products. Apple and Google enforce requirements for products in their walled gardens, through their registration and gatekeepers. Apple forbids a number of products in iOS and limits others. Web browsers, for example, must use Apple's WebKit engine and not install their own; Apple also forbids programming languages or scripting languages.

We're now seeing the Flash technology being pushed out of the walled gardens. Apple has prohibited it from the beginning. Google has deprecated it on YouTube. How long before Microsoft kicks it out of its garden?

The expulsion of Flash may foreshadow other exclusions of technology. Apple could, at any time, remove Microsoft's apps from iTunes. Google could remove Adobe apps from the Play Store. Microsoft could kick out Oracle apps from the (soon to be revived, I think) Microsoft App Store.

The ability to remove apps from the garden is something that the enterprise folks will want to think about. Building a business on a walled garden has the risk of existing at the whim of the gardener.