Agile, Waterfall, and Risk

For some years (decades, really), software development has used an agile approach to project management. The Agile method sees short iterations that each focus on a single feature, with the entire team reviewing progress and selecting the feature for the next iteration. Over time, a complete system evolves. The advantage is that the entire team (programmers, managers, salespersons, etc.) learn about the business problem, the functions of the system, and the capabilities of the team. The team can change course (hence the name "agile") as they develop each feature.

Prior to Agile, for some years (decades, really), software development used the "waterfall" approach to project management. The Waterfall method starts with a set of requirements and a schedule, and moves through different phases for analysis, design, coding, testing, and deployment. The important aspect is the schedule. The Waterfall method promises to deliver a complete system on the specified date.

This last aspect of Waterfall is quite different from Agile. The Agile method makes no promise to deliver a completed system on a specific date. It does promise that each iteration ends with a working system that implements the features selected by the team. Thus, a system developed with Agile is always working -- although incomplete -- whereas a system developed with Waterfall is not guaranteed to work until the delivery date.

(It has been observed that while the Waterfall method promises a complete, working system on the specified delivery date, it is quite poor at keeping that promise. Many projects overrun both schedule and budget.)

Here is where risk comes into play.

With Agile, the risk is shared by the entire team, key among these are developers and managers. An agile project has no specified delivery date, but more often than not senior managers (those above the agile-involved managers) have a date in mind. (And probably a budget, too.) Agile projects can easily overrun these unstated expectations. When they do, the agile-involved managers are part of the group held responsible for the failure. Managers have some risk.

But look at the Waterfall project. When a waterfall project fails (that is, runs over schedule or budget) the managers have way to distance themselves from the failure. They can say (honestly) that they provided the developers with a list of requirements and a schedule (and a budget) and that the developers failed to meet meet the "contract" of the waterfall project. Managers can deflect the risk to the development team.

(For some reason, we rarely question the feasibility of the schedule, or the consistency and completeness of the requirements, or the budget assigned to the project. These are considered "good", and any delay or shortcoming is therefore the fault of the developers.)

Managers want to avoid risk -- or at least transfer it to another group. Therefore, I predict that in the commercial space, projects will slowly revert from Agile methods to Waterfall methods.

The fall and possible rise of UML

Lots of folks are discussing UML, and specifically the death of UML. What killed UML? Lots of people have different ideas. I have some ideas too. Rather than pin the failure on one reason, I have a bunch.

First, our methods changed, and UML was not a good fit with newer methods. UML was created in the world of large-scale waterfall projects. It works well with those projects, with design up front (disparagingly called "Big Design Up Front") as a precursor to coding. UML does not work well with Agile methods, where design and coding occur in parallel. UML assigns value to code; the idea of up-front design is to build the right code form the start and not revise it. In the UML world, changes to code are expensive and to be avoided. UML also attaches value to the designs, with identical attachments to designs and the desire to avoid changes to designs. (Although changes to designs are preferred over changes to code.)

UML works well with object-oriented programming, but not with cloud computing (small scripts instead of big code).

Second, UML entailed costs. UML notation was difficult to learn. Or at least required some time to learn. The tools took time to learn, and they also cost significant sums. The mindset was "invest now (by learning UML and buying the tools) to prevent more costly mistakes later". At the time, there were charts showing the cost of a mistake, and comparing the cost of detecting the mistake at different points in the project. A mistake detected early (say in requirements or design) was less expensive than a mistake detected later (say in coding or testing). Mistakes detected after deployment were the most expensive. This effect justified the expense of UML tools.

But UML tools were expensive, and not everyone on the team got UML tools. The tools were reserved for the designers; coders were limited to printed copies of UML diagrams. This lead to the notion that designers were special and worth more than programmers. (The elite received UML tools; the plebes did not.) This in turn lead to resentment by programmers.

A third (and often overlooked) reason was the expense for designers. When programmers performed both design and programming, their salaries covered both activities. UML formalized the design process and required a subteam of designers, and each of those designers required a salary. (And they often wanted salaries higher than those of programmers.)

A fourth (and also often overlooked) reason was the added delay to the development process.

UML created an additional step in the waterfall process. Theoretically, it did not, because UML was simply formalized design documents. But in practice, UML did create an additional step.

Before UML, a project would have the formal steps of requirements, design, coding, testing, and deployment. That's what managers thought they had. In reality, the steps were slightly different than those formal steps. The actual steps were requirements, design and coding, testing, and deployment.

Before UML: requirements -> design and code -> test -> deploy

Notice that the steps of design and code are one step, not two. It was an activity performed by the programming team. As it was a single team, people could move from designing to coding and back again, revising the design as they developed the code.

UML and a formal design deliverable changed the process to the five steps the managers thought they had:

With UML: requirements -> design -> code -> rest -> deploy

UML forced the separation of design from coding, and in doing so, changed the (informal) four-step process to a five-step process.

Programmers were used to designing as well as programming. With UML, programmers could not unilaterally change the design; they had to push back against the design. This set up conflicts between designers and programmers. Sometimes the designers "gave in" and allowed a change; other times they "held fast" and programmers had to build something they considered wrong. In either case, such differences introduced delays and political struggles when there were none before.

Those are my observations for UML, and why it failed: new methods not suitable for UML, direct expense of tools and training, direct expense of designers, and a slower development process.

* * * * *

In a way, I am sorry for the loss of UML. I think it can be a helpful tool. But not in the way it was implemented.

UML was added to a project as a design aid, and one that occurred prior to coding. Perhaps it is better to have UML as a diagnostic instead of an aspiration. That is, instead of creating UML and then generating code from UML, create code and then generate UML from the code.

In this way, UML could be a kind of "super lint" that reports on the design of the system.

There was "round-tripping" which allowed for UML to be converted to code, and then that code converted back to UML. That is not the same; it leaves UML as the center for design. And round-tripping never really worked the way we needed. A one-way code-to-UML diagnostic puts code at the center and UML as a tool to assist the programmers. (That's my bias as a programmer showing.)

A code-to-UML diagnostic could be helpful to Agile projects, just as 'lint' and other style checkers are. The tools may be less expensive (we've gotten better at tools, and a diagnostic tool is easier to build than a UML editor). We would not have a separate design team, avoiding that expense (and the associated politics). And a diagnostic tool would not slow the development process -- or at least not so much.

Maybe we will see such a tool. If we do, it will have to be developed by the open-source community. (That is, an individual who wants to scratch an itch, much like Perl, or Python, or Linux.) I don't see a large corporation building one; I don't see a business model for it.

Anyone want to scratch an itch?

What Agile and Waterfall have in common

Agile and Waterfall are often described in contrasts: Waterfall is large and bureaucratic, Agile is light and nimble. Waterfall is old, Agile is new. Waterfall is... you get the idea.

But Waterfall and Agile have things in common.

First and most obvious, Waterfall and Agile are both used to manage development projects. They both deliver software.

I'm sure that they are both used to deliver things other than software. They are tools for managing projects, not limited to software projects.

But those are the obvious common elements.

An overlooked commonality is the task of defining small project steps. For Waterfall, this is the design phase, in which requirements are translated into system design. The complete set of requirements can paint a broad picture of the system, providing a general shape and contours. (Individual requirements can be quite specific, with details on input data, calculations, and output data.)

Breaking down the large idea of the system into smaller, code-able pieces is a talent required for Waterfall. It is how you move from requirements to coding.

Agile also needs that talent. In contrast to Waterfall, Agile does not envision the completed system and does not break that large picture into smaller segments. Instead, Agile asks the team to start with a small piece of the system (most often a core function) and build that single piece.

This focus on a single task is, essentially, the same as the design phase in Waterfall. It converts a requirement (or user story, or use case, or whatever small unit is convenient) into design for code.

The difference between Waterfall and Agile is obvious: Waterfall converts all requirements in one large batch before any coding is started, and Agile performs the conversions serially, seeing one requirement all the way to coding and testing (or more properly, testing and then coding!) before starting the next.

So whether you use Waterfall or Agile, you need the ability to "zoom in" from requirements to design, and then on to tests and code. Waterfall and Agile are different, but the differences are more in the sequence of performing tasks and not the tasks themselves.

Agile can be cheap; Waterfall is expensive

Agile can be cheap, but Waterfall will always be expensive.

Here's why:

Waterfall starts its process with an estimate. The Waterfall method uses a set of phases (analysis, design, coding, testing, and deployment) which are executed according to a fixed schedule. Many Waterfall projects assign specific times to each phase. Waterfall needs this planning because it makes a promise: deliver a set of features on a specific date.

But notice that Waterfall begins with an estimate: the features that can be implemented in a specific time frame. That estimate is crucial to the success of the project. What is necessary to obtain that estimate?

Only people with knowledge and experience can provide a meaningful estimate. (One could, foolishly, ask an inexperienced person for the estimate, but that estimate has no value.)

What knowledge does that experienced person need? Here are some ideas:

- The existing code
- The programming language and tools used
- The different teams involved in development and testing
- The procedures and techniques used to coordinate efforts
- The terms and concepts used by the business

With knowledge of these, a person can provide a reasonable estimate for the effort.

These areas of knowledge do not come easily. They can be learned only by working on the project and in different capacities.

In other words, the estimate must be provided by a senior member of the team.

In other words, the team must have at least one senior member.

Waterfall relies on team members having knowledge about the business, the code, and the development processes.

Agile, in contrast, does not rely on that experience. Agile is designed to allow inexperienced people work on the project.

Thus, Agile projects can get by without senior, experienced team members, but Waterfall projects must have at least one (and probably more) senior team members. Since senior personnel are more expensive than junior, and Waterfall requires senior personnel, we can see that Waterfall projects will, on average, cost more than Agile projects. (At least in terms of per-person costs.)

Do not take this to mean that you should run all projects with Agile methods. Waterfall may be more expensive, but it provides different value. It promises a specific set of functionality on a specific date, a promise that Agile does not make. If you need the promises of Waterfall, it may be worth the extra cost (higher wages). This is a business decision, similar to using proprietary tools over open-source tools, or leasing premium office space in the suburbs over discount office space in a not-so-nice part of town.

Which method you choose is up to you. But be aware that they are not the same, not only in terms of deliverables but in staffing requirements. Keep those differences in mind when you make your decision.

Agile is for startups; waterfall for established projects

The Agile method was conceived as a revolt against the Waterfall method. Agile was going to be everything that Waterfall was not: lightweight, simple, free of bureaucracy, and successful. In retrospect, Agile was successful, but that doesn't mean that Waterfall is obsolete.

It turns out that Agile is good for some situations, and Waterfall is good for others.

Agile is effective when the functionality of the completed system is not known in advance. The Agile method moves forward in small steps which explore functionality, with reviews after each step. The Agile method also allows for changes in direction after each review. Agile provides flexibility.

Waterfall is effective when the functionality and the delivery date is known in advance. The Waterfall method starts with a set of requirements and executes a plan for analysis, design, coding, testing, and deployment. It commits to delivering that functionality on the delivery date, and does not allow (in its pure form) for changes in direction. Waterfall provides predictability.

Established companies, which have an ongoing business with procedures and policies, often want the predictability that Waterfall offers. They have business partners and customers and people to whom they make commitments (such as "a new feature will be ready for the new season"). They know in detail the change they want and they know precisely when they want the change to be effective. For them, the Waterfall method is appropriate.

Start-ups, on the other hand, are building their business model and are not always certain of how it will work. Many start-ups adjust their initial vision to obtain profitability. Start-ups don't have customers and business partners, or at least not with long-standing relationships and specific expectations. Start-ups expect to make changes to their plan. While they want to start offering services and products quickly (to generate income) they don't have a specific date in mind (other than the "end of runway" date when they run out of cash). Their world is very different from the world of the established company. They need the flexibility of Agile.

The rise of Agile did not mean the death of Waterfall -- despite some intentions of the instigators. Both have strengths, and each can be used effectively. (Of course, each can be mis-used, too. It is quite easy to manage a project "into the ground" with the wrong method, or a poorly applied method.)

The moral is: know what is best for your project and use the right methods.

Agile is all about code quality

Agile promises clean code. That's the purpose of the 'refactor' phase. After creating a test and modifying the code, the developer refactors the code to eliminate compromises made during the changes.

But how much refactoring is enough? One might flippantly say "as much as it takes" but that's not an answer.

For many shops, the answer seems to be "as much as the developer thinks is needed". Other shops allow refactoring until the end of the development cycle. The first is subjective and opens the development team to the risk of spending too much time on refactoring and not enough on adding features. The second is arbitrary and risks short-changing the refactoring phase and allowing messy code to remain in the system.

Agile removes risk by creating automated tests, creating them before modifying the code, and having developers run those automated tests after all changes. Developers must ensure that all tests pass; they cannot move on to other changes while tests are failing.

This process removes judgement from the developer. A developer cannot say that the code is "good enough" without the tests confirming it. The tests are the deciders of completeness.

I believe that we want the same philosophy for code quality. Instead of allowing a developer to decide when refactoring has reached "good enough", we will instead use an automated process to make that decision.

We already have code quality tools. C and C++ have had lint for decades. Other languages have tools as well. (Wikipedia has a page for static analysis tools.) Some are commercial, others open source. Most can be tailored to meet the needs of the team, placing more weight on some issues and ignoring others. My favorite at the moment is 'Rubocop', a style-checking tool for Ruby.

I expect that Agile processes will adopt a measured approach to refactoring. By using one (or several) code assessors, a team can ensure quality of the code.

Such a change is not without ramifications. This change, like the use of automated tests, takes judgement away from the programmer. Code assessment tools can consider many things, some of which are style. They can examine indentation, names of variables or functions, the length or complexity of a function, or the length of a line of code. They can check the number of layers of 'if' statements or 'while' loops.

Deferring judgement to the style checkers will affect managers as well as programmers. If a developer must refactor code until it passes the style checker, then a manager cannot cut short the refactoring phase. Managers will probably not like this change -- it takes away some control. Yet it is necessary to maintain code quality. By ending refactoring before the code is at an acceptable quality, managers allow poor code to remain in the system, which will affect future development.

Agile is all about code quality.

Agile pushes ugliness out of the system

Agile differs from Waterfall in many ways. One significant way is that Agile handles ugliness, and Waterfall doesn't.

Agile starts by defining "ugliness" as an unmet requirement. It could be a new feature or a change to the current one. The Agile process sees the ugliness move through the system, from requirements to test to code to deployment. (Waterfall, in contrast, has the notion of requirements but not the concept of ugliness.)

Let's look at how Agile considers ugliness to be larger than just unmet requirements.

The first stage is an unmet requirement. With the Agile process, development occurs in a set of changes (sometimes called "sprints") with a small set of new requirements. Stakeholders may have a long list of unmet requirements, but a single sprint handles a small, manageable set of them. The "ugliness" is the fact that the system (as it is at the beginning of the sprint) does not perform them.

The second stage transforms the unmet requirements into tests. By creating a test -- an automated test -- the unmet requirement is documented and captured in a specific form. The "ugliness" has been captured and specified.

After capture, changes to code move the "ugliness" from a test to code. A developer changes the system to perform the necessary function, and in doing so changes the code. But the resulting code may be "ugly" -- it may duplicate other code, or it may be difficult to read.

The fourth stage (after unmet requirements, capture, and coding) is to remove the "ugliness" of the code. This is the "refactoring" stage, when code is improved without changing the functions it performs. Modifying the code to remove the ugliness is the last stage. After refactoring, the "ugliness" is gone.

The ability to handle "ugliness" is the unique capability of Agile methods. Waterfall has no concept of code quality. It can measure the number of defects, the number of requirements implemented, and even the number of lines of code, but it doesn't recognize the quality of the code. The quality of the code is simply its ability to deliver functionality. This means that ugly code can collect, and collect, and collect. There is nothing in Waterfall to address it.

Agile is different. Agile recognizes that code quality is important. That's the reason for the "refactor" phase. Agile transforms requirements into tests, then into ugly code, and finally into beautiful (or at least non-ugly) code. The result is requirements that are transformed into maintainable code.

Waterfall caused specialization; agile causes generalization

There are a number of differences between waterfall processes and agile processes. Waterfall defines one long, segmented process; agile uses a series of short iterations. Waterfall specifies a product on a specific date; agile guarantees a shippable product throughout the development process.

Another difference between waterfall and agile is the specialization of participants. Waterfall projects are divided into phases: analysis, development, testing, etc. and these phases tend to be long themselves. Agile projects have the same activities (analysis, development, testing) but on a much shorter timeframe. A waterfall project may extend for six months, or a year, or several years, and the phases of those projects may extend for months -- or possibly years.

The segmentation of a waterfall project leads to specialization of the participants. It is common to find a waterfall project staffed by analysts, developers, and testers, each a distinct team with distinct management teams. The different teams use tools specific to their tasks: databases for requirements, compilers and integrated development environments, test automation software and test case management systems.

This specialization is possible due to the long phases of waterfall projects. It is reasonable to have team "A" work on the requirements for a project and then (when the requirements are deemed complete) assign team "B" to the development phase. While team "B" develops the project, team "A" can compose the requirements for another project. Specialization allows for efficient scheduling of personnel.

Agile processes, in contrast, have short iterations of analysis, design, coding, and testing. Instead of months (or years), an iteration may be one or two weeks. With such a short period of time, it makes little sense to have multiple teams work on different aspects. The transfer of knowledge from one team to another, a small task in a multi-year project, is a large effort on a two-week iteration. Such inefficiencies are not practical for short projects, and the better approach is for a single team to perform all of the tasks. (Also, the two-week iteration is not divided into a neat linear sequence of analysis, design, development, and test. All activities occur throughout the iteration, with multiple instances of each task.)

A successful agile process needs people who can perform all of the tasks. It needs not specialists but generalists.

Years of waterfall projects have trained people and companies into thinking that specialists are more efficient than generalists. (There may be a bit of Taylorism here, too.) Such thinking is so pervasive that one finds specialization in the company's job descriptions. One can find specific job descriptions for business analysts, developers, and testers (or "QA Analysts").

The shift to agile projects will lead to a re-thinking of specialization. Generalists will be desired; specialists will find it difficult to fit in. Eventually, generalists will become the norm and specialists will become, well, special. Even the job descriptions will change, with the dominant roles being "development team members" with skills in all areas and a few specialist roles for special tasks.

Agile project cannot overrun the budget

Some recent trade rags published stories about Agile-run projects and how they overran their budgets. This seems wrong, based on what I know about Agile methods.

First, a little history. Before "agile" projects, the dominant project method was "waterfall". Waterfall is a method for running a project. All projects need a number of tasks: analysis, design, development, and testing. The waterfall process puts these in strict sequence, with each occurring before the next phase. Thus, all analysis is performed, then all design, and then all development.

The beauty of waterfall is the schedule. The waterfall method promises a specific deliverable with a specific level of quality on a specific date.

The horror or waterfall is that it rarely delivers on schedule. It is quite common for waterfall projects to overrun their schedule. (Their cost estimates, too.)

Agile is not waterfall. The agile method takes a different approach and makes a different promise. It does not promise a specific deliverable on a specific date. Instead, it makes a promise of deliverability.

An agile project consists of a number of iterations. (Some people call these iterations "sprints". I'm not too attached to either name.) Each iteration adds a small set of features. Each iteration also uses automated tests to ensure that previous features still work. The project starts with a small set of initial features and adds features on each iteration. But the important idea is that the end of each iteration is a working product. It may not have everything you want, but everything that it has does work.

The promise of agile is that you can always deliver your product.

With the agile process, one cannot overrun a budget. (You can always stop when you want.) You can, instead, underdeliver. You may get to the end of your allotted time. You may use all of your allotted funds. But you always have something to deliver. If you run over your budget or schedule, it's because you chose to spend more time or money.

Unstoppable cannon balls and immovable posts

There is an old riddle: what happens when an unstoppable cannon ball strikes an immovable post?

The unstoppable cannon ball is a hypothetical construct. It is a cannon ball that, once in motion, stays in motion. (It's ability to defy friction comes from its hypothetical nature.)

The immovable post is also a hypothetical construct. It is a post that, once placed, cannot be moved.

While the riddle is entertaining, a real-world version of it is emerging with software development methodologies. Instead of an unstoppable cannon ball and an immovable post we have the waterfall methodology and the agile methodology. Perhaps the agile method is the equivalent of the cannon ball, as both emphasize motion. It doesn't really matter, though.

What matters is that the two methodologies are incompatible. One can develop software using one methodology, or the other, but one cannot use both. For small shops that have only one project (say a start-up), this is not an issue. Larger shops (any shop with multiple projects) must choose a methodology; you cannot use waterfall for some projects and agile methods for another. (Well, you can, provided that the two sets of projects have no interaction. But that is unlikely.)

The waterfall and the agile methods make two different promises. Waterfall promises a specific deliverable, with a specific set of features, and a specific quality, on a specific date. Agile promises that the product is always shippable (that is, very high quality), and promises to implement the most important features first, but makes no promises about a complete set of features on a specific date.

It is easy to dismiss these differences if we think of these approaches as methods of developing software. It is harder to dismiss them when we realize the truth: waterfall and agile are not methods for building software but philosophies for managing projects.

Managing projects is serious business, and is the core of most IT shops. (Projects can by managed by non-IT shops, too, but I will stick with the IT world.)

One cannot run a business (a successful business) with multiple coordinated projects that use project practices which differ to the extent that waterfall and agile differ. Waterfall, with its "big design up front" and locked-in delivery dates cannot mesh with agile, with its "choose as you go" method for selecting features. The rise of agile methods -- agile project management -- means that companies must choose one or the other. Which one is important, too, and depends on how the managers of the company want to run their company.

This schism has far-reaching effects. It can be felt in the acquisition of other companies, when one merges the management of project teams. It affects the hiring of new employees: are they familiar with the practices of the hiring company? It can even affect joint projects between companies.

I don't claim that one is better than the other. My preference is for agile project management, based on my experience and projects. It may be the right methodology for you. Or waterfall may be the better methodology for you. Managers must evaluate both and consider the benefits and costs.

My advice is to learn about both and make an informed decision.

The Agile and Waterfall Impedance Mismatch

During a lunchtime conversation with a colleague, we chatted about project management and gained some insights into the Waterfall and Agile project management techniques.

Our first insight was that an organization cannot split its project management methods. It cannot run some projects with a Waterfall process and other projects with Agile processes. (At least not if the projects must coordinate deliverables.) A company that uses Waterfall processes may be tempted to run a pilot project with Agile processes -- but if that pilot project "plugs in" to other projects, the result will be failure.

The problem is that Waterfall and Agile make two different promises. Waterfall promises a specific set of functionality, with a specific level of quality, delivered on a specific date. Agile makes no such promise;  it promises to add functionality and have a product that is always ready to ship (that is a high level of quality), albeit with an unknown set of functionality. The Agile process adds small bits of functionality and waits to get them correct before adding others -- thus ensuring that everything that has been added is working as expected, but not promising to delivery everything desired by a specific date. (I am simplifying things here. Agile enthusiasts will point out that there is quite a bit more to Agile processes.)

Waterfall processes make promises that are very specific in terms of feature set, quality, and delivery time -- but are not that good at keeping them. Hence, we have a large number of projects that are late or have low quality. Agile makes promises that are specific in terms of quality, and are good at keeping them. But the promises of the Agile processes are limited to quality; they do not propose the specifics that are promised by Waterfall.

With two different promises, it is no surprise that waterfall and agile processes don't co-exist. (There are other reasons that the two methods fail to cooperate, including the "design up front" of Waterfall and the "design as you go" of Agile.)

Our second insight was that transitioning an IT shop from Waterfall to Agile methods should not be accomplished by pilot projects.

Pilot projects are suitable to introduce people to the methods -- but those pilot projects must exist in isolation from the "regular" projects. Such projects were easy to establish in the earlier age of separate systems -- "islands of automation" -- that gave each system a measure of independence. Today, it is rare to see an IT system that exists in isolation.

Rather that use pilot projects, we like the idea of introducing ideas from Agile into the standard process used within the company. Our first choice is automated testing, the unit-level automated testing that can be performed by developers. Giving each developer the ability to run automated tests introduces them to a core practice of Agile, without creating an impedance mismatch.

After automated testing, we like the idea of allowing refactoring. Often omitted from plans for Waterfall projects, refactoring is another key practice of Agile development. Once unit tests are in place, developers can refactor with confidence.

Our third insight relates to project methods and project size. We think (and this is speculation) that Agile is better suited to small projects (and small systems) and Waterfall may be better suited to large systems. Thus, if you have large, complex systems, you may want to stay with Waterfall; if you have small systems (or even small applications) then you may want to use Agile.

We think that this relationship is correlation, not causal. That is, you can pick one side of the equation and drive the other. If you have large systems, you will end up with Waterfall. But the equation works in both directions, and you don't have to start with the size of your systems. If you use Agile methods, you will end up with smaller, collaborating systems.

Now, we realize that a lot of companies have large systems. We don't believe that by switching overnight to Agile methods will lead to smaller systems overnight. A base of large systems contains a certain amount of inertia, and requires a certain effect to redesign into smaller systems.

What we do believe is that you have a choice, that you can choose to use Agile (or Waterfall) methods (carefully, avoiding impedance mismatches), and that you can change the size of your systems. You can manage the size and complexity of your systems by selecting your processes.

These are choices that managers should accept and make carefully. They should not assume that "the system is the system" and "things are big and complicated and will always be so".