The disappointing performance of the medical insurance exchange web site (the "Obamacare web site") show the dark side of current project management techniques. After an initial specification, a long wait while multiple teams worked on various parts of the system, and a hasty integration, the web site has numerous, significant problems. Now we have calls from a group I call the "Cult of Accountability". They (the cult) want to know "who is responsible" for the failure.
Big projects often work this way: A large project is assigned to a team (for government projects, the "prime contractor") along with specifications of the deliverable. That team breaks the project into smaller components and assigns components to teams, internal or external (the "sub-contractors") along with specifications for those components. When the work is complete, the work moves in the reverse direction, with the bottom layer of teams providing their components to the next higher layer, those teams assembling the components and providing the results to the next higher layer, until the top team assembles components into a finished product.
This cycle of functional decomposition and specification continues for some number of cycles. Notice that each team starts with a specification, divides the work into smaller pieces, and provides specifications to the down-stream teams.
The top-down design and project planning for many projects is a process that defines tasks, assigns resources, and specifies delivery dates up front. It locks in a deliverable of a specified functionality, a particular design, and a desired level of quality, all on an agreed date. It defines the components and assigns responsibility for each component.
The "divide and conquer" strategy works... if the top team knows everything about the desired deliverable and can divide the work into sensible components, and if the down-stream teams know everything about their particular piece. This is the case for work that has already been done, or work that is very similar to previous work. The assembly of automobiles, for example: each car is a "product" and can be assembled by following well-defined tasks. The work can be divided among multiple teams, some external to the company. The specifications for each part, each assembly, each component, are known and understood.
The "divide and conquer" strategy works poorly for projects that are not similar to previous work. Projects in "unexplored territory" contain a large number of "unknowns". Some are "known unknowns" (we know that we need to test the performance of our database with the expected level of transactions) and some are "unknown unknowns" (we didn't realize that our network bandwidth was insufficient until we went to production). "Unknowns" is another word for "surprises".
In project management, surprises are (usually) bad. You want to avoid them. You can investigate issues and resolve questions, if you know about them. (These are the "known unknowns".) But you cannot (by definition) plan for the "unknown unknowns". If you plan for them, they become "known unknowns".
Project planning must include an evaluation of unknowns, and project process must account for them. Projects with few unknowns can be run with "divide and conquer" (or "waterfall") methods. These projects have few latent surprises.
Projects with many unknowns should be managed with agile techniques. These techniques are better at exploring, performing work in small steps and using the experience from one step to guide later steps. They don't provide a specific date for delivery of all features; they provide a constantly working product with features added over time. They avoid the "big bang" at the end of a long development effort. You exchange certainty of feature set for certainty of quality.
The Cult of Accountability will never accept agile methods. They must have agreements, specific and detailed agreements, up front. In a sense, they are planning to fail -- you need the agreements only when something doesn't work and you need a "fall guy". With agile methods, your deliverable always works, so there is no "accountability hunt". There is no need for a fall guy.
Monday, October 28, 2013
Wednesday, October 23, 2013
Healthcare.gov was a "moonshot", but the Moon mission was not
Various folks have referred to the recent project to build and launch the healthcare.gov web site as a "moonshot". They are using the term to describe a project that:
We in IT seem to thrive on "moonshot" type projects.
But I will observe that the NASA Moon project (the Mercury, Gemini, and Apollo missions) was not a "moonshot". NASA ran the project more like an agile project than the typical waterfall project.
Let's compare.
The NASA Moon project was ambitious. One could even call it audacious.
The NASA Moon project involved a (relatively) large number of participants, including rocket scientists, metallurgists, electrical engineers, chemists, psychologists, biologists, and radio specialists. (And many more.)
The NASA Moon project had a fixed schedule of "by the end of the decade" assigned by President Kennedy in 1961.
The NASA Moon project consisted of a number of phases, each with specific goals and each with subprojects. The Mercury flights established the technology and skills to orbit the Earth. The Gemini missions built on Mercury to dock two vehicles in space. The Apollo missions used that experience to reach the Moon.
It's this last aspect that is very different from the healthcare.gov web site project (and also very different from many IT projects). The NASA Moon program was a series of projects, each feeding into the next. NASA started with a high-level goal and worked its way to that goal. They did not start with a "master project plan" that defined every task and intermediate deliverable. They learned as they went and made plans -- sensible plans, based on their newly-won experience -- for later flights.
The healthcare.gov web site is an ambitious project. It's launch has been difficult, and shows many defects. Could it have been built in an agile manner? Would an agile approach given us a better result?
The web site must perform several major tasks: authenticate users, verify income against government databases, and display valid plans offered by insurance companies. An agile approach would have built the web site in phases. Perhaps the first phase could be allowing people to register and create their profile, the second verifying income, and the third matching users with insurance plans. But such a "phased" release might have been received poorly ("what good is a web site that lets you register but do nothing else?") and perhaps not completed in time.
I don't know that agile methods would have made for better results at healthcare.gov.
But I do know that the Moon project was not a "moonshot".
- is ambitious in scope
- has a large number of participants
- occurs in a short and fixed time frame
- consists of a single attempt that will either succeed or fail
We in IT seem to thrive on "moonshot" type projects.
But I will observe that the NASA Moon project (the Mercury, Gemini, and Apollo missions) was not a "moonshot". NASA ran the project more like an agile project than the typical waterfall project.
Let's compare.
The NASA Moon project was ambitious. One could even call it audacious.
The NASA Moon project involved a (relatively) large number of participants, including rocket scientists, metallurgists, electrical engineers, chemists, psychologists, biologists, and radio specialists. (And many more.)
The NASA Moon project had a fixed schedule of "by the end of the decade" assigned by President Kennedy in 1961.
The NASA Moon project consisted of a number of phases, each with specific goals and each with subprojects. The Mercury flights established the technology and skills to orbit the Earth. The Gemini missions built on Mercury to dock two vehicles in space. The Apollo missions used that experience to reach the Moon.
It's this last aspect that is very different from the healthcare.gov web site project (and also very different from many IT projects). The NASA Moon program was a series of projects, each feeding into the next. NASA started with a high-level goal and worked its way to that goal. They did not start with a "master project plan" that defined every task and intermediate deliverable. They learned as they went and made plans -- sensible plans, based on their newly-won experience -- for later flights.
The healthcare.gov web site is an ambitious project. It's launch has been difficult, and shows many defects. Could it have been built in an agile manner? Would an agile approach given us a better result?
The web site must perform several major tasks: authenticate users, verify income against government databases, and display valid plans offered by insurance companies. An agile approach would have built the web site in phases. Perhaps the first phase could be allowing people to register and create their profile, the second verifying income, and the third matching users with insurance plans. But such a "phased" release might have been received poorly ("what good is a web site that lets you register but do nothing else?") and perhaps not completed in time.
I don't know that agile methods would have made for better results at healthcare.gov.
But I do know that the Moon project was not a "moonshot".
Sunday, October 20, 2013
Java and Perl are not dead yet
Are Java and Perl dead?
The past few weeks have seen members of the Java community claim that Java is not dead. They have also seen members of the Perl community claim that Perl is not dead.
Many developers may want Perl to die (or Java to die, or COBOL to die) but when members of the community claim "X is not dead" then we should review the health of X.
The industry has seen many languages come and go. Of all the languages that we have seen since the beginning of the computer age, most are dead. Don't take my word; consider the languages documented by Jean Sammet in the late 1960s. They include: A-2, A-3, ADAM, AED, AIMACO, Algol, ALTRAN, AMBIT, AMTRAN, APL, and APT (and that is just the entries under 'A'). Of these, the only language that can be said to be alive today is Algol -- and even that is pushing the definition of 'alive'.
Languages die. But not all -- popular languages live on. Long-lasting languages include COBOL, Fortran, C, C++, and Perl. One could add Java and C# to that list, depending on one's definition of "long-lasting". (I think that there is no debate about their popularity.)
Back to Java and Perl.
I think that Java and Perl are very much alive, in that many projects use them. They are also alive in that new versions are built and released by the maintainers. Perl is an open source language, supported by its community. Java is supported by Oracle.
But Java and Perl are, perhaps, not as popular as they used to be. When introduced, Java was a shiny new thing compared to the existing C++ champion. Perl's introduction was quieter and it developed a following slowly. But slow or fast, they were considered the best in programming: Java for object-oriented programming and Perl for scripting.
Today, Java and Perl have competition. Java has C#, a comparable language for object-oriented programming. It also has the JVM languages Scala and Lua (and others) that have taken the "shiny new thing" title away from Java. Perl has competition from Python and Ruby, two other capable scripting languages.
Java and Perl are no longer the clear leaders. They are no longer the "obvious best" languages for development. When starting a new project, people often pick one of the competition. I think that it is this loss of "obvious best" position that is causing the angst in their respective development communities.
Now, Java and Perl are still capable programming languages. I wouldn't abandon them. For projects that use them today, I would continue to use them.
For new projects... I would consider my options.
The past few weeks have seen members of the Java community claim that Java is not dead. They have also seen members of the Perl community claim that Perl is not dead.
Many developers may want Perl to die (or Java to die, or COBOL to die) but when members of the community claim "X is not dead" then we should review the health of X.
The industry has seen many languages come and go. Of all the languages that we have seen since the beginning of the computer age, most are dead. Don't take my word; consider the languages documented by Jean Sammet in the late 1960s. They include: A-2, A-3, ADAM, AED, AIMACO, Algol, ALTRAN, AMBIT, AMTRAN, APL, and APT (and that is just the entries under 'A'). Of these, the only language that can be said to be alive today is Algol -- and even that is pushing the definition of 'alive'.
Languages die. But not all -- popular languages live on. Long-lasting languages include COBOL, Fortran, C, C++, and Perl. One could add Java and C# to that list, depending on one's definition of "long-lasting". (I think that there is no debate about their popularity.)
Back to Java and Perl.
I think that Java and Perl are very much alive, in that many projects use them. They are also alive in that new versions are built and released by the maintainers. Perl is an open source language, supported by its community. Java is supported by Oracle.
But Java and Perl are, perhaps, not as popular as they used to be. When introduced, Java was a shiny new thing compared to the existing C++ champion. Perl's introduction was quieter and it developed a following slowly. But slow or fast, they were considered the best in programming: Java for object-oriented programming and Perl for scripting.
Today, Java and Perl have competition. Java has C#, a comparable language for object-oriented programming. It also has the JVM languages Scala and Lua (and others) that have taken the "shiny new thing" title away from Java. Perl has competition from Python and Ruby, two other capable scripting languages.
Java and Perl are no longer the clear leaders. They are no longer the "obvious best" languages for development. When starting a new project, people often pick one of the competition. I think that it is this loss of "obvious best" position that is causing the angst in their respective development communities.
Now, Java and Perl are still capable programming languages. I wouldn't abandon them. For projects that use them today, I would continue to use them.
For new projects... I would consider my options.
Thursday, October 17, 2013
For PCs, small may be the new big thing
PCs have had the same size and shape (roughly) for the past thirty years. While we have seen improvements to processors (faster), memory (more), video adapters (faster and more memory), hard disks (bigger), and communication ports (faster, more, and simpler), the general design of a PC has been stagnant. The color may have changed from IBM's beige to Compaq's brown, to Dell's white, and to HP's black-and-silver, but the PC box has remained... a box.
In the early PC days, the large, spacious box with its expansion slots made sense. In the early days, PCs needed expansion and customization. The "base" PC was not enough for corporate work. When we bought a PC, we added video cards, memory cards, serial and parallel port cards, terminal emulator cards, and network cards. We even added cards with real-time clocks. It was necessary to open the PC and add these cards.
Over the years, more and more "extra" features became "standard". The IBM PC AT came with a built-in real-time clock, which eliminated one card. Memory increased. Hard drives became larger and faster. The serial ports and parallel ports were replaced by USB ports. Today's PC has enough memory, a capable video card, a big enough hard disk, a network interface, and ample USB ports. (Apple computers have slightly different communication options, but enough.)
The one constant in the thirty years of change has been the size of the PC. The original IBM PC was about the size of today's tower PC. PCs still have the card slots and drive bays for expansion, although few corporate users need such things.
That's about to change. PCs will shrink from their current size to one of two smaller sizes: small and nothing. The small PCs will be the size of the Apple Mini: a 4-inch by 4-inch box with ports and no expansion capabilities. The "nothing" size PCs will be virtual machines, existing only in larger computers. (Let's focus on the "small" size. We can discuss virtual PCs another time.)
The small PCs have all the features of a real PC: processor, memory, storage, video, and communications. They may have some compromises, with perhaps not the fastest processors and the most capable video cards, but they are good enough. They can run Windows or Linux, and the Apple Mini Mac runs MacOS, of course. All you need is a display, a keyboard, and a network connection. (These small-form PCs often have wire network interfaces and not wireless.)
I suppose that we can give credit to Apple for the change. Apple's Mini Mac showed that there was a steady demand for smaller, non-PC-shaped PCs. Intel has their "Next Unit of Computing" or NUC device, a small 4-inch by 4-inch PC with communication ports.
Other manufacturers had built small PCs prior to Apple's Mini Mac (the Shuttle PC is a notable pioneer) but received little notice.
The Arduino, the Raspberry Pi, and the Beaglebone and also small-form devices, designed mainly for tinkerers. I expect little interest from the corporate market in these devices.
But I do expect interest in the smaller "professional" units from Apple and Intel. I also expect to see units from other manufacturers like Lenova, Asus, HP, and Dell.
Small will be the new big thing.
In the early PC days, the large, spacious box with its expansion slots made sense. In the early days, PCs needed expansion and customization. The "base" PC was not enough for corporate work. When we bought a PC, we added video cards, memory cards, serial and parallel port cards, terminal emulator cards, and network cards. We even added cards with real-time clocks. It was necessary to open the PC and add these cards.
Over the years, more and more "extra" features became "standard". The IBM PC AT came with a built-in real-time clock, which eliminated one card. Memory increased. Hard drives became larger and faster. The serial ports and parallel ports were replaced by USB ports. Today's PC has enough memory, a capable video card, a big enough hard disk, a network interface, and ample USB ports. (Apple computers have slightly different communication options, but enough.)
The one constant in the thirty years of change has been the size of the PC. The original IBM PC was about the size of today's tower PC. PCs still have the card slots and drive bays for expansion, although few corporate users need such things.
That's about to change. PCs will shrink from their current size to one of two smaller sizes: small and nothing. The small PCs will be the size of the Apple Mini: a 4-inch by 4-inch box with ports and no expansion capabilities. The "nothing" size PCs will be virtual machines, existing only in larger computers. (Let's focus on the "small" size. We can discuss virtual PCs another time.)
The small PCs have all the features of a real PC: processor, memory, storage, video, and communications. They may have some compromises, with perhaps not the fastest processors and the most capable video cards, but they are good enough. They can run Windows or Linux, and the Apple Mini Mac runs MacOS, of course. All you need is a display, a keyboard, and a network connection. (These small-form PCs often have wire network interfaces and not wireless.)
I suppose that we can give credit to Apple for the change. Apple's Mini Mac showed that there was a steady demand for smaller, non-PC-shaped PCs. Intel has their "Next Unit of Computing" or NUC device, a small 4-inch by 4-inch PC with communication ports.
Other manufacturers had built small PCs prior to Apple's Mini Mac (the Shuttle PC is a notable pioneer) but received little notice.
The Arduino, the Raspberry Pi, and the Beaglebone and also small-form devices, designed mainly for tinkerers. I expect little interest from the corporate market in these devices.
But I do expect interest in the smaller "professional" units from Apple and Intel. I also expect to see units from other manufacturers like Lenova, Asus, HP, and Dell.
Small will be the new big thing.
Monday, October 14, 2013
Executables, source code, and automated tests
People who use computers tend to think of the programs as the "real" software.
Programmers tend to have a different view. They think of the source code as the "real" software. After all, they can always create a new executable from the source code. The generative property of source code gives it priority of the mere performant property of executable code.
But that logic leads to an interesting conclusion. If source code is superior to executable code because the former can generate the latter, then how do we consider tests, especially automated tests?
Automated tests can be used to "generate" source code. One does not use tests to generate source code in the same, automated manner that a compiler converts source code to an executable, but the process is similar. Given a set of tests, a framework in which to run the tests, and the ability to write source code (and compile it for testing), one can create the source code that produces a program that conforms to the tests.
That was a bit of a circuitous route. Here's the concept in a diagram:
automated tests --> source code --> executable code
This idea has been used in a number of development techniques. There is test-driven development (TDD), extreme programming (XP), and agile methods. All use the concept of "test first, then code" in which tests (automated tests) are defined first and only then is code changed to conform to the tests.
The advantage of "test first" is that you have tests for all of your code. You are not allowed to write code "because we may need it someday". You either have a test (in which case you write code) or you don't (in which case you don't write code).
A project that follows the "test first" method has tests for all features. If the source code is lost, one can re-create it from the tests. Granted, it might take some time -- this is not a simple re-compile operation. A complex system will have thousands of tests, perhaps hundreds of thousands. Writing code to conform to all of those tests is a manual operation.
But it is possible.
A harder task is going in the other direction, that is, writing tests from the source code. It is too easy to omit cases, to skip functionality, to misunderstand the code. Given the choice, I would prefer to start with tests and write code.
Therefore, I argue that the tests are the true "source" of the system, and the entity we consider "source code" is a derived entity. If I were facing a catastrophe and had to pick one (and only one) of the tests, the source code, and the executable code, I would pick the tests -- provided that they were automated and complete.
Programmers tend to have a different view. They think of the source code as the "real" software. After all, they can always create a new executable from the source code. The generative property of source code gives it priority of the mere performant property of executable code.
But that logic leads to an interesting conclusion. If source code is superior to executable code because the former can generate the latter, then how do we consider tests, especially automated tests?
Automated tests can be used to "generate" source code. One does not use tests to generate source code in the same, automated manner that a compiler converts source code to an executable, but the process is similar. Given a set of tests, a framework in which to run the tests, and the ability to write source code (and compile it for testing), one can create the source code that produces a program that conforms to the tests.
That was a bit of a circuitous route. Here's the concept in a diagram:
automated tests --> source code --> executable code
This idea has been used in a number of development techniques. There is test-driven development (TDD), extreme programming (XP), and agile methods. All use the concept of "test first, then code" in which tests (automated tests) are defined first and only then is code changed to conform to the tests.
The advantage of "test first" is that you have tests for all of your code. You are not allowed to write code "because we may need it someday". You either have a test (in which case you write code) or you don't (in which case you don't write code).
A project that follows the "test first" method has tests for all features. If the source code is lost, one can re-create it from the tests. Granted, it might take some time -- this is not a simple re-compile operation. A complex system will have thousands of tests, perhaps hundreds of thousands. Writing code to conform to all of those tests is a manual operation.
But it is possible.
A harder task is going in the other direction, that is, writing tests from the source code. It is too easy to omit cases, to skip functionality, to misunderstand the code. Given the choice, I would prefer to start with tests and write code.
Therefore, I argue that the tests are the true "source" of the system, and the entity we consider "source code" is a derived entity. If I were facing a catastrophe and had to pick one (and only one) of the tests, the source code, and the executable code, I would pick the tests -- provided that they were automated and complete.
Sunday, October 13, 2013
Unstructured data isn't really unstructured
The introduction of NoSQL databases has brought along another concept: unstructured data. Advocates of NoSQL are quick to point out that relational databases are limited to structured data, and NoSQL data stores can handle unstructured (as well as structured) data.
I think that word does not mean what you think it means.
I've seen lots of data, and all of it has been structured. I have yet to meet unstructured data. All data has structure -- the absence of structure implies random data. Even the output of a pseudo-random number generator are structured; it is a series of numeric values.
When people say "structured data", they really mean "a series of objects, each conforming to a specific structure, known in advance". Tables in a relational database certainly follow this rule, as do records in a COBOL data declaration.
NoSQL data stores relaxes these constraints, but still requires that data be structured. If a NoSQL data store uses JSON notation, then each object must be stored in a manner consistent with JSON. The objects in a set may contain different properties, so that one object has a structure quite different from the next object, but each object must be structured.
This notion is not new. While COBOL and FORTRAN were efficient at processing homogenous records, Pascal allowed for "variant" records (it used a key field at the beginning of the record to identify the record type and layout).
What is new is that the object layout is not known in advance. Earlier languages and database systems required the design of the data up front. A COBOL program would know about customer records, for example, and a FORTRAN program would know about data observations. The structure of the data was "baked in" to the program. A new type of customer, or a new type of data set, would require a new version of the program or database schema.
NoSQL lets us create new structures without changing the program or schema. We can add new fields and create new objects for storage and processing, without changing the code.
So as I see it, it's not that data is unstructured. The idea is that we have reduced the coupling between the data from the program. Data is still structured, but the structure is not part of the code.
I think that word does not mean what you think it means.
I've seen lots of data, and all of it has been structured. I have yet to meet unstructured data. All data has structure -- the absence of structure implies random data. Even the output of a pseudo-random number generator are structured; it is a series of numeric values.
When people say "structured data", they really mean "a series of objects, each conforming to a specific structure, known in advance". Tables in a relational database certainly follow this rule, as do records in a COBOL data declaration.
NoSQL data stores relaxes these constraints, but still requires that data be structured. If a NoSQL data store uses JSON notation, then each object must be stored in a manner consistent with JSON. The objects in a set may contain different properties, so that one object has a structure quite different from the next object, but each object must be structured.
This notion is not new. While COBOL and FORTRAN were efficient at processing homogenous records, Pascal allowed for "variant" records (it used a key field at the beginning of the record to identify the record type and layout).
What is new is that the object layout is not known in advance. Earlier languages and database systems required the design of the data up front. A COBOL program would know about customer records, for example, and a FORTRAN program would know about data observations. The structure of the data was "baked in" to the program. A new type of customer, or a new type of data set, would require a new version of the program or database schema.
NoSQL lets us create new structures without changing the program or schema. We can add new fields and create new objects for storage and processing, without changing the code.
So as I see it, it's not that data is unstructured. The idea is that we have reduced the coupling between the data from the program. Data is still structured, but the structure is not part of the code.
Thursday, October 10, 2013
Hadoop shows us a possible future of computing
Computing has traditionally been processor-centric. The classic model of computing has a "central processing unit" which performs computations. The data is provided by "peripheral devices", processed by the central unit, and then routed back to peripheral devices (the same as the original devices or possibly others). Mainframes, minicomputers, and PCs all use this model. Even web applications use this model.
Hadoop changes this model. It is designed for Big Data, and the size of data requires a new model. Hadoop stores your data in segments across a number of servers -- with redundancy to prevent loss -- with each segment being 64MB to 2GB. If your data is smaller than 64MB, moving to Hadoop will gain you little. But that's not important here.
What is important is Hadoop's model. Hadoop moves away from the traditional computing model. Instead of a central processor that performs all calculations, Hadoop leverages servers that can hold data and also perform calculations.
Hadoop makes several assumptions:
With these assumptions, Hadoop builds a new model of computing. (To be fair, Hadoop may not be the only package that builds this new model of distributed processing -- or even the first. But it has a lot of interest, so I will use it as the example.)
All very interesting. But here is what I find more interesting: the distributed processing model of Hadoop can be applied to other systems. Hadoop's model makes sense for Big Data, and systems with Little (that is, not Big) data should not use Hadoop.
But perhaps smaller systems can use the model of distributed processing. Instead of moving data to the processor, we can store data with processors and move code to the data. A system could be constructed from servers holding data, connected with a network, and mobile code that can execute anywhere. The chief tasks then become identifying the need for code and moving code to the correct location.
That would give us a very different approach to system design.
Hadoop changes this model. It is designed for Big Data, and the size of data requires a new model. Hadoop stores your data in segments across a number of servers -- with redundancy to prevent loss -- with each segment being 64MB to 2GB. If your data is smaller than 64MB, moving to Hadoop will gain you little. But that's not important here.
What is important is Hadoop's model. Hadoop moves away from the traditional computing model. Instead of a central processor that performs all calculations, Hadoop leverages servers that can hold data and also perform calculations.
Hadoop makes several assumptions:
- The code is smaller than the data (or a segment of data)
- Code is transported more easily than data (because of size)
- Code can run on servers
With these assumptions, Hadoop builds a new model of computing. (To be fair, Hadoop may not be the only package that builds this new model of distributed processing -- or even the first. But it has a lot of interest, so I will use it as the example.)
All very interesting. But here is what I find more interesting: the distributed processing model of Hadoop can be applied to other systems. Hadoop's model makes sense for Big Data, and systems with Little (that is, not Big) data should not use Hadoop.
But perhaps smaller systems can use the model of distributed processing. Instead of moving data to the processor, we can store data with processors and move code to the data. A system could be constructed from servers holding data, connected with a network, and mobile code that can execute anywhere. The chief tasks then become identifying the need for code and moving code to the correct location.
That would give us a very different approach to system design.
Sunday, October 6, 2013
Transformation projects
An organization that maintains software can run several types of projects. Some are simple, others are complex. One of the most complex projects can be what I call "transformation projects", which change the nature of the software.
Examples of these transformation projects are:
All transformation projects are not equal. Upgrading a compiler (say Visual Studio 2010 to Visual Studio 2012) is a minor affair. The existing code still compiles. The existing design still works.
Compare that simple project to changing the system's database (for example, switching from MySQL to MariaDB, or possibly Microsoft SQL Server to Oracle). While SQL databases are mostly compatible, they are not exactly compatible. They use different languages for stored procedures. They have different techniques for the optimization of queries.
Transformation projects frequently add no new business features to the software. Consequently, many organizations perform them rarely, sometimes only when necessary. Such a strategy may be viewed as responsible, since by not spending time on upgrades you can focus your effort on features which bring measurable benefit to the business.
But such a strategy has a cost: you have no practice at these transformation projects, and consequently often underestimate the time and risks of such projects. Frequent (or regular) transformation projects gives you experience, which you can use to estimate the time and risk of future similar projects.
I should say "accurately estimate", since you can estimate any project, with or without experience. But experience gives us better estimates. For example, I (and I suspect many people) can accurately estimate the time needed to commute home from the office. They actually do commute home, so they have experience and can (reasonable) assume that future commutes home will take the same amount of time.
An estimate without the corresponding experience is subject to greater variance. As a counter-example to the "time to commute home" example, can you estimate the time it would take to walk to Argentina? (If walking is too long, you may substitute driving.) Such a project contains unknowns: tasks for which we have little or no experience. (In this example those would include crossing national borders, obtaining licenses to drive, and paying for gasoline in foreign countries.)
When an organization avoids transformation tasks (for whatever reason), it foregoes the experience of those projects. When it decides to perform one of those projects (because eventually you will need to perform at least one of them) it has no experience to use in the formulation of the plan. The project schedule may have wildly inaccurate estimates, or may omit important tasks.
The transition from web app to cloud app is such a transformation project. So is the transition from a desktop app to a web app, or a cloud app. If you are working on such a project and find that the project schedule is daunting, difficult to prepare, or just not working for you, look back on your experience with similar projects. If you have none (or very little), then perhaps experience with similar, smaller transformation projects can help.
Examples of these transformation projects are:
- Upgrading to a new version of the compiler or IDE
- Changing from one database to another
- Expanding to another computing platform
- Converting from one language to another
- Moving from one computing architecture to another
All transformation projects are not equal. Upgrading a compiler (say Visual Studio 2010 to Visual Studio 2012) is a minor affair. The existing code still compiles. The existing design still works.
Compare that simple project to changing the system's database (for example, switching from MySQL to MariaDB, or possibly Microsoft SQL Server to Oracle). While SQL databases are mostly compatible, they are not exactly compatible. They use different languages for stored procedures. They have different techniques for the optimization of queries.
Transformation projects frequently add no new business features to the software. Consequently, many organizations perform them rarely, sometimes only when necessary. Such a strategy may be viewed as responsible, since by not spending time on upgrades you can focus your effort on features which bring measurable benefit to the business.
But such a strategy has a cost: you have no practice at these transformation projects, and consequently often underestimate the time and risks of such projects. Frequent (or regular) transformation projects gives you experience, which you can use to estimate the time and risk of future similar projects.
I should say "accurately estimate", since you can estimate any project, with or without experience. But experience gives us better estimates. For example, I (and I suspect many people) can accurately estimate the time needed to commute home from the office. They actually do commute home, so they have experience and can (reasonable) assume that future commutes home will take the same amount of time.
An estimate without the corresponding experience is subject to greater variance. As a counter-example to the "time to commute home" example, can you estimate the time it would take to walk to Argentina? (If walking is too long, you may substitute driving.) Such a project contains unknowns: tasks for which we have little or no experience. (In this example those would include crossing national borders, obtaining licenses to drive, and paying for gasoline in foreign countries.)
When an organization avoids transformation tasks (for whatever reason), it foregoes the experience of those projects. When it decides to perform one of those projects (because eventually you will need to perform at least one of them) it has no experience to use in the formulation of the plan. The project schedule may have wildly inaccurate estimates, or may omit important tasks.
The transition from web app to cloud app is such a transformation project. So is the transition from a desktop app to a web app, or a cloud app. If you are working on such a project and find that the project schedule is daunting, difficult to prepare, or just not working for you, look back on your experience with similar projects. If you have none (or very little), then perhaps experience with similar, smaller transformation projects can help.
Subscribe to:
Posts (Atom)