Sunday, August 13, 2017

Make it go faster

I've worked on a number of projects, and a (not insignificant) number of them had a requirement (or, more accurately, a request) to improve the performance of an existing system.

A computerized system consists of hardware and software. The software is a set of instructions that perform computations, and the hardware executes those instructions.

The most common method of improving performance is to use a faster computer. Leave the software unchanged, and run it on a faster processor. (Or if the system performs a lot of I/O, a faster disk, possibly an SSD instead of a spinning hard drive.) This method is simple, with no changes to the software and therefore low risk. It is the first method of reducing run time: perform computations faster.

Another traditional method is to change your algorithm. Some algorithms are faster than others, often by using more memory. This method has higher risk, as it changes the software.

Today's technology sees cloud computing as a way to reduce computing time. If your calculations are partitionable (that is, subsets can be computed independently) then you can break a large set of computations into a group of smaller computations, assign each smaller set to its own processor, and compute them in parallel. Effectively, this is computing faster, provided that the gain in parallel processing outweighs the cost of partitioning your data and combining the multiple results.

One overlooked method is using a different compiler. (I'm assuming that you're using a compiled language such as C or C++. If you are using Python or Ruby, or even Java, you may want to change languages.) Switching from one compiler to another can make a difference in performance. The code emitted by one compiler may be fine-tuned to a specific processor; the code from another compiler may be generic and intended for all processors in a family.

Switching from one processor to another may improve performance. Often, such a change requires a different compiler, so you are changing two things, not one. But a different processor may perform the computations faster.

Fred Brooks has written about essential complexity and accidental complexity. Essential complexity is necessary and unavoidable; accidental complexity can be removed from the task. There may be an equivalent in computations, with essential computations that are unavoidable and accidental computations that can be removed (or reduced). But removing accidental complexity is merely reducing the number of computations.

To improve performance you can either perform the computations faster or you can reduce the number of computations. That's the list. You can use a faster processor, you can change you algorithms, you can change from one processor and compiler to a different processor and compiler. But there is an essential number of computations, in a specific sequence. You can't exceed that limit.

Wednesday, August 2, 2017

Agile is for startups; waterfall for established projects

The Agile method was conceived as a revolt against the Waterfall method. Agile was going to be everything that Waterfall was not: lightweight, simple, free of bureaucracy, and successful. In retrospect, Agile was successful, but that doesn't mean that Waterfall is obsolete.

It turns out that Agile is good for some situations, and Waterfall is good for others.

Agile is effective when the functionality of the completed system is not known in advance. The Agile method moves forward in small steps which explore functionality, with reviews after each step. The Agile method also allows for changes in direction after each review. Agile provides flexibility.

Waterfall is effective when the functionality and the delivery date is known in advance. The Waterfall method starts with a set of requirements and executes a plan for analysis, design, coding, testing, and deployment. It commits to delivering that functionality on the delivery date, and does not allow (in its pure form) for changes in direction. Waterfall provides predictability.

Established companies, which have an ongoing business with procedures and policies, often want the predictability that Waterfall offers. They have business partners and customers and people to whom they make commitments (such as "a new feature will be ready for the new season"). They know in detail the change they want and they know precisely when they want the change to be effective. For them, the Waterfall method is appropriate.

Start-ups, on the other hand, are building their business model and are not always certain of how it will work. Many start-ups adjust their initial vision to obtain profitability. Start-ups don't have customers and business partners, or at least not with long-standing relationships and specific expectations. Start-ups expect to make changes to their plan. While they want to start offering services and products quickly (to generate income) they don't have a specific date in mind (other than the "end of runway" date when they run out of cash). Their world is very different from the world of the established company. They need the flexibility of Agile.

The rise of Agile did not mean the death of Waterfall -- despite some intentions of the instigators. Both have strengths, and each can be used effectively. (Of course, each can be mis-used, too. It is quite easy to manage a project "into the ground" with the wrong method, or a poorly applied method.)

The moral is: know what is best for your project and use the right methods.

Tuesday, July 18, 2017

A sharp edge in Ruby

I like the Ruby programming language. I've been using it for several projects, including an interpreter for the BASIC language. (The interpreter for BASIC was an excuse to do something in Ruby and learn about the language.)

My experience with Ruby has been a good one. I find that the language lets me do what I need, and often very quickly. The included classes are well-designed and include the functions I need. From time to time I do have to add some obscure capability, but those are rare.

Yet Ruby has a sharp edge to it, an aspect that can cause trouble if you fail to pay attention.

That aspect is one of its chief features: flexibility.

Let me explain.

Ruby is an object-oriented language, which means the programmer can define classes. Each class has a name, some functions, and usually some private data. You can do quite a bit with class definitions, including class variables, class instance variables, and mix-ins to implement features in multiple classes.

You can even modify existing classes, simply by declaring the same class name and defining new functions. Ruby accepts a second definition of a class and merges it into the first definition, quietly and efficiently. And that's the sharp edge.

This "sharp edge" cut me when I wasn't expecting it. I was working on my BASIC interpreter, and had just finished a class called "Matrix", which implemented matrix operations within the language. My next enhancement was for array operations (a matrix being a two-dimensional structure and an array being a one-dimensional structure).

I defined a class called "Array" and defined some functions, including a "to_s" function. (The name "to_s" is the Ruby equivalent of "ToString()" or "to_string()" in other languages.)

And my code behaved oddly. Existing functions, having nothing to do with arrays or my Array class, broke.

Experienced Ruby programmers are probably chuckling at this description, knowing the problem.

Ruby has its own Array class, and my Array class was not a new class but a modification of the existing, built-in class named "Array". My program, in actuality, was quite different from my imagined idea. When I defined the function "to_s" in "my" Array class, I was actually overwriting the existing "to_s" function in the Ruby-supplied Array class. And that happened quietly and efficiently -- no warning, no error, no information message.

Part of this problem is my fault. I was not on my guard against such a problem. But part of the problem, I believe, is Ruby's -- specifically the design of Ruby. Letting one easily modify an existing class, with no warning, is dangerous. And I say this not simply due to my background with languages that use static checking.

My error aside, I can think of two situations in which this can be a problem. The first is when a new version of the Ruby language (and its system libraries) are released. Are there new classes defined in the libraries? Could the names of those classes duplicate any names I have used in my project? For example, will Ruby one day come with a class named "Matrix"? If it does, it will collide with my class named "Matrix". How will I know that there is a duplicate name?

The second situation is on a project with multiple developers. What happens if two developers create classes with the same name? Will they know? Or will they have to wait for something "weird" to happen?

Ruby has some mechanisms to prevent this problem. One can use namespaces within the Ruby language, to prevent such name conflicts. A simple grep of the code, looking for "class [A-Z][\w]" and then a sort will identify duplicate names. But these solutions require discipline and will -- they don't come "for free".

As I said earlier, this is a sharp edge to Ruby. Is it a defect? No, I think this is the expected behavior for the language. It's not a defect. But it is an aspect of the language, and one that may limit the practicality of Ruby on large applications.

I started this blog with the statement that I like Ruby. I still like Ruby. It has a sharp edge (like all useful tools) and I think that we should be aware of it.

Sunday, July 9, 2017

Cloud and optimizations

We all recognize that cloud computing is different.

It may be that cloud computing breaks some of our algorithms.

A colleague of mine, a long time ago, shared a story about programming early IBM mainframes. They used assembly language, because code written in assembly executed faster than code written in COBOL. (And for business applications on IBM mainframes, at the time, those were the only two options.)

Not only did they write in assembly language, they wrote code to be fast. That is, they "optimized" the code. One of the optimizations was with the "multiply" instruction.

The multiply instruction does what you think: it multiplies to numbers and stores the result. To optimize it, they wrote the code to place the larger of the two values in one register and the smaller of the two values in the other register. The multiply instruction was implemented as a "repeated addition" operation, so the second register was really a count of the number of addition operations that would be performed. By storing the smaller number in the second register, programmers reduced the number of "add" operations and improved performance.

(Technically inclined folks may balk at the notion of reducing a multiply operation to repeated additions, and observe that it works for integer values but not floating-point values. The technique was valid on early IBM equipment, because the numeric values were either integers or fixed-point values, not floating-point values.)

It was an optimization that was useful at the time, when computers were relatively slow and relatively expensive. Today's faster, cheaper computers can perform multiplication quite quickly, and we don't need to optimize it.

Over time, changes in technology make certain optimizations obsolete.

Which brings us to cloud computing.

Cloud computing is a change in technology. It makes available a variable number of processors.

Certain problems have a large number of possible outcomes, with only certain outcomes considered good. The problems could describe the travels of a salesman, or the number of items in a sack, or playing a game of checkers. We have algorithms to solve specific configurations of these problems.

One algorithm is the brute-force, search-every-possibility method, which does just what you think. While it is guaranteed to find an optimal solution, there are sometimes so many solutions (millions upon millions, or billions, or quintillions) that this method is impractical.

Faced with an impractical algorithm, we invent others. Many are iterative algorithms which start with a set of conditions and then move closer and closer to a solution by making adjustments to the starting conditions. Other algorithms discard certain possibilities ("pruning") which are known to be no better than current solutions. Both techniques reduce the number of tested possibilities and therefore reduce the time to find a solution.

But observe: The improved algorithms assume a set of sequential operations. They are designed for a single computer (or a single person), and they are designed to minimize time.

With cloud computing, we no longer have a single processor. We have multiple processors, each operating in parallel. Algorithms designed to optimize for time on a single processor may not be suitable for cloud computing.

Instead of using one processor to iteratively find a solution, it may be possible to harness thousands (millions?) of cloud-based processors, each working on a distinct configuration. Instead of examining solutions in sequence, we can examine solutions in parallel. The result may be a faster solution to the problem, in terms of "wall time" -- the time we humans are waiting for the solution.

I recognize that this approach has its costs. Cloud computing is not free, in terms of money or in terms of computing time. Money aside, there is a cost in creating the multiple configurations, sending them to respecting cloud processors, and then comparing the many results. That time is a cost, and it must be included in our evaluation.

None of these ideas are new to the folks who have been working with parallel processing. There are studies, papers, and ideas, most of which have been ignored by mainstream (sequential) computing.

Cloud computing will lead, I believe, to the re-evaluation of many of our algorithms. We may find that many of them have a built-in bias for single-processor operation. The work done in parallel computing will be pertinent to cloud computing.

Cloud computing is a very different form of computing. We're still learning about it. The application of concepts from parallel processing is one aspect of it. I won't be surprised if there are more. There may be all sorts of surprises ahead of us.

Sunday, July 2, 2017

It's not always A or B

Us folks in IT almost pride themselves on our fierce debates on technologies. And we have so many of them: emacs vs. vim, Windows vs. Mac, Windows vs. Linux, C vs. Pascal, C# vs. Java, ... the list goes on and on.

But the battles in IT are nothing compared to the fights that were held between the two different types of electricity. In the early 20th century, Edison lead the group for direct current, and Tesla lead the alternate group for, well, alternating current. The battle between these two made our disputes look like a Sunday picnic. Edison famously electrocuted an elephant -- with the "wrong" type of electricity, of course.

I think we in IT can learn from the Great Electricity War. (And its not that we should be electrocuting elephants.)

Despite all of the animosity, despite all of the propaganda, despite all of the innovation on both sides, neither format "won". Neither vanquished its opponent. We use both types of electricity.

For power generation, transmission, and large appliances, we use alternating current. (Large appliances include washing machines, dryers, refrigerators, and vacuum cleaners.)

Small appliances (personal computers, digital televisions, calculators, cell phones) use direct current. They may plug into the AC wall outlet, but the first thing they do is convert 110 VAC into lower-voltage DC.

Alternating current has advantages in certain situations and direct current has advantages in other situations. It's not that one type of electricity is better than the other, its that one type is better for a specific application.

We have a multitude of solutions in IT: multiple operating systems, multiple programming languages, multiple editors, multiple hardware platforms... lots and lots of choices. We too often pick one of many, name it our "standard", and force entire companies to use that one selection. That may be convenient for the purchasing team, and probably for the support team, but is it the best strategy for a company?

Yes, we in IT can learn a lot from electricity. And please, respect the elephants.

Monday, June 26, 2017

The humble PRINT statement

One of the first statements we learn in any language is the "print" statement. It is the core of the "Hello, world!" program. We normally learn it and then discard it, focussing our efforts on database and web service calls.

But the lowly "print" statement has its uses, as I was recently reminded.

I was working on a project with a medium-sized C++ application. We needed information to resolve several problems, information that would normally be available from the debugger and from the profiler. But the IDE's debugger was not usable (executing under the debugger would require a run time of about six hours) and the IDE did not have a profiler.

What to do?

For both cases, the PRINT statement (the "fprintf()" statement, actually, as we were using C++) was the thing we needed. A few carefully placed statements allowed use to capture the necessary information, make decisions, and resolve the problems.

The process wasn't that simple, of course. We needed several iterations, adding and removing PRINT statements in various locations. We also captured counts and timings of various functions.

The effort was worth it.

PRINT statements (or "printf()", or "print()", or "puts()", whatever you use) are useful tools. Here's how they can help:
  • They can capture values of internal variables and state when the debugger is not available.
  • They can capture lots of values of variables and state, for analysis at a level higher than the interactive level of the debugger. (Consider viewing several thousand values for trends in a debugger.)
  • They can capture performance when a profiler is not available.
  • They can extract information from the "release" version of software, because sometimes the problem doesn't occur in "debug" mode.
They may be simple, but they are useful. Keep PRINT statements in your toolbox.

* * * * *

I was uncertain about the title for this column. I considered the C/C++ form of the generic statement ('printf()'). I also considered the general form used by other languages ('print()', 'puts()', 'WriteLine()'). I settled on BASIC's form of PRINT -- all capitals, no parentheses. All popular languages have such a statement; in the end, I suspect it matters little. Use what is best for you.

Sunday, June 18, 2017

Three models of computing

Computing comes in different flavors. We're probably most familiar with personal computers and web applications. Let's look at the models used by different vendors.

Apple has the simplest model: Devices that compute. Apple has built it's empire on high-quality personal computing devices. They do not offer cloud computing services. (They do offer their "iCloud" backup service, which is an accessory to the central computing of the iMac or Macbook.) I have argued that this model is the same as personal computing in the 1970s.

Google has a different model: web-based computing. This is obvious in their Chromebook, which is a lightweight computer that can run a browser -- and nothing else. All of the "real" computing occurs on the servers in Google's data center. The same approach is visible in most of the Google Android apps -- lightweight apps that communicate with servers. In some ways, this model is an update of the 1970s minicomputer model, with terminals connected to a central processor.

Microsoft has a third model, a hybrid of the two. In Microsoft's model, some computing occurs on the personal computer and some occurs in the data center. It is the most interesting of the two, requiring communication and coordination of two components.

Microsoft did not always have their current approach. Their original model was the same as Apple's: personal computers as complete and independent computing entities. Microsoft started with implementations of the BASIC language, and then sold PC-DOS to IBM. Even early versions of Windows were for stand-alone, independent PCs.

Change to that model started with Windows for Workgroups, and became serious with Windows NT, domains, and ActiveDirectory. Those three components allowed for networked computing and distributed processing. (There were network solutions from other vendors, but the Microsoft set was a change in Microsoft's strategy.)

Today, Microsoft offers an array of services under its "Azure" mark. Azure provides servers, message queues, databases, and other services, all hosted in its cloud environment. They allow individuals and companies to create applications that can combine PC and cloud technologies. These applications perform some computing on the local PC and some computing in the Azure cloud. You can, of course, build an application that runs completely on the PC, or completely in the cloud. That you can build those applications shows the flexibility of the Microsoft platform.

I think this hybrid model, combining local computing and server-based computing, has the best potential. It is more complex, but it can handle a wider variety of applications than either the PC-only solution (Apple's) or the server-only solution (Google's). Look for Microsoft to support this model with development tools, operating systems, and communication protocols and libraries.

Looking forward, I can see Microsoft working on a "fluid" model of computing, where some processing can move from the server to the local PC (for systems with powerful local PCs) and from the PC to the server (for systems with limited PCs).

Many things in the IT realm started in a "fixed" configuration, and over time have become more flexible. I think processing is about to join them.