Tuesday, January 19, 2016

Functional programming is waiting for microservices

Functional programming has been bubbling on the edge of mainstream development for years (perhaps decades). The advent of microservices may give functional programming its big break.

Why has it taken so long for the industry to adopt functional programming? I can think of a few reasons:
  • The effort to learn the techniques of functional programming
  • The effort to convert existing programs
  • The belief that current techniques are "good enough"
Individual developers (and teams of developers) have already learned the techniques of object-oriented programming. They are comfortable with them. They know the tools and understand the issues of object-oriented development. Changing to functional programming is a big change, with new techniques and new issues. That change can make people reluctant to switch to functional programming.

Existing systems are large and complex. Converting them to functional programming is a large effort, and one that must be done completely. The switch from procedural programming to object-oriented programming could be done gradually, especially if you were moving from C to C++. Functional programming has no gradual path.

Switching from object-oriented programming to functional programming incurs a cost, and the benefits are not so clear. The "return on investment" is not obvious.

These are all reasons to not switch from object-oriented programming to functional programming. (And they're pretty good reasons.)

Microservices change the equation.

Microservices are small, independent services. Changing an existing system (probably a large, monolithic system) to one that is composed of microservices is a large significant effort. It requires the decomposition of the monolith into smaller pieces.

If one is constructing small, independent services, either as part of a re-engineering effort or a new system, one can certainly consider functional programming -- if only for some of the microservices. Writing small functional programs is much easier than writing large systems in a functional language. (A statement that holds for object-oriented programming languages, and procedural languages, and assembly language.)

For microservices, a functional language may be better than an object-oriented one. Object-oriented languages are suited to large systems; classes and namespaces are designed for organizing the data and code of large systems. Microservices are, by design, small. It is possible that functional programming will be a better match for the small code of microservices.

If you are building microservices, or just contemplating microservices, think about functional programming.

Sunday, January 10, 2016

Use the tools available

I've just completed some work on a small project. My success is due, not only to my own talent and hard work, but to the tools that were available to me.

The project was in Ruby, and the tool that assisted me was Rubocop.

Rubocop analyzes Ruby code and reports on questionable (but legal) constructs and syntax. It is, in a phrase, "lint" for Ruby.

Almost all of the major languages have syntax checkers. For C and C++, there is lint. For C# there is FXCop. For Python, PyLint. Even Perl has Perl::Critic and Perl::Lint.

Rubocop helped me, indirectly. I used it as I developed the project. I ran it on the code I was writing, and it reported that certain functions were "too long", according to its default guidelines.

Some programmers would be arrogant and refuse to heed such advice. (Myself at an earlier point in my career, for example.) But with the wisdom of experience, I chose to modify the code and reduce the size of functions. It was an investment.

When I modified the long functions, I broke them into smaller ones. This had the benefit of making duplicate code obvious, as some of the smaller functions performed identical tasks. (The large versions of the functions also performed these identical tasks, but the duplications were not apparent.)

I combined duplicate functions into single functions, and reduced the overall size of the code. I also created abstract classes to hold functions common to concrete derived classes.

The simpler version of the code was, well, simpler. That meant that subsequent changes were easier to implement. In fact, one particular feature had me worried, yet the simpler code made that feature easy to add.

Rubocop helped me simplify the code, which made it easy for me to add new features -- and get them right. Rubocop was a tool, a useful tool.

On your projects, be aware of the tools that can help you.

Which language?

When starting a project, the question you must answer is: Which language?

The answer is not simple.

If you're an Oracle shop then you are most likely comfortable with Oracle products and you should probably pick Java.

If you're a Microsoft shop then you will be comfortable with the Microsoft languages C# and F#.

If you're a Google shop, you may want to consider Google's Go language.

If you use Linux and open source tools, you may want to look at Perl, Python, Ruby, and JavaScript. Perl 6 has just been released, and may be a bit too "new" for serious enterprise projects. Python and Ruby are both mature, and JavaScript has a lot of support.

The interesting aspect here is that your choice of language is not about technology but relationships. All of these languages are capable. All of these languages have advocates, and detractors. None of these languages are perfect.

There are two other languages which you may consider. These are not connected with specific companies -- although implementations may be provided by companies. But the languages themselves are independent.

Those languages are COBOL and FORTRAN. Both are available from a number of sources, and for a number of platforms. COBOL is designed for financial transactions; FORTRAN for numerical computations. If your work falls into these categories, these languages are worth considering. (COBOL and FORTRAN are, however, not general-purpose languages and should not be considered for problems outside of their domains.)

Astute readers will note that I have omitted C and C++ from this discussion. If I were contemplating a move from Java to another language I would consider the above-listed languages before C or C++. And if I got to the point of considering C++, I would think very strongly about the STL and BOOST libraries.

All of these languages are capable. Each have advantages. A large consideration is the relationship that the language brings: Microsoft for C#, Google for Go, open source for Perl, Python, and Ruby. Don't ignore that.

Sunday, January 3, 2016

Predictions for 2016

It's the beginning of a new year, which means... predictions! Whee!

Let's start with some obvious predictions:

Mobile will be big in 2016.

Cloud will be big on 2016.

NoSQL and distributed databases will be big in 2016.

Predictions like these are easy.

Now for something a little less obvious: legacy applications.

With the continued interest in mobile, cloud, NoSQL, and distributed databases, these areas will see strong demand for architects, developers, designers, and testers. That demand will pull people away from legacy applications -- those applications built for classic, non-cloud web architectures as well as the remaining desktop applications and mainframe batch systems.

Which is unfortunate for the managers of those legacy applications, because I believe that 2016 is going to be the year that companies decide that they want to migrate those legacy applications to the cloud/mobile platform.

When the web appeared, lots of managers held back, waiting to see if the platform would prove itself. It did, and companies migrated most of their applications from desktop to web (either external or internal). Even Microsoft, stalwart of desktop applications, created a web-based version of Outlook.

Likewise, when mobile and cloud appeared, many managers held back and waited for the new technologies to prove themselves. With almost ten years of mobile and cloud, and many companies already using those technologies, its time for the holdouts to take action.

Look for renewed interest in converting existing desktop and classic web applications. The conversions have challenges. In one sense, the job is easier than the early conversions, because we now have experience with mobile/cloud systems and we understand the architecture. In other ways, this may be harder, as the easy conversions (the "low-hanging fruit") have already been done, which means that the remaining conversions are harder.

The architecture of mobile/cloud systems (with or without distributed databases) is different from classic web applications. (And very different from desktop applications.)

I think that 2016 will be the year of rude awakening, as companies look at the effort to convert their legacy systems to newer technologies.

But the rude awakening is delivered in two phases. The first is the cost and time to convert legacy applications. The second is the cost of maintaining legacy applications in their current form.

Why the cost of maintaining legacy applications, without changing them to newer technologies? Because of the demand for mobile/cloud is high. New entrants to the field will know the new technologies, and select jobs that let them use that knowledge. That means that the folks with knowledge of the older technologies will be, um, older.

The folks with knowledge about older languages (C++, Visual Basic) and older APIs (Flash) will be the senior developers. And senior developers are more expensive than junior developers.

So the owners of legacy applications have a rather unpleasant choice: migrate to mobile/cloud, which is expensive, or stay on the legacy platform, with will also be expensive.

Sunday, December 27, 2015

For the future of Java, look to Google

What is the future of Java? It is a popular language, perhaps a bit long in the tooth, yet still capable. It struggled under Sun. Now it is the property of Oracle.

Oracle is an interesting company, with a number of challenges. Its biggest challenge is the new database technologies that provide alternatives to SQL. Oracle built its fortune on the classic, ACID-based, SQL database, competing with IBM and Microsoft.

Now facing competition not only in the form of other companies but in new technologies, Oracle must perform. How will is use Java?

For the future of Java, I suggest that we look to Google and Android. Java is part of Android -- or at least the Java bytecode. Android apps are written in Java on standard PCs, compiled into Java bytecode, and then delivered to Android devices. The Android devices (phones, tablets, what-have-you) use not the standard JVM interpreter but a custom-made one named "Dalvik".

Oracle and Google have their differences. Oracle sued Google, successfully, for using the Java APIs. (A decision with which I disagree, but that is immaterial.)

Google now faces a decision: stay with Java or move to something else. Staying with Java will most likely paying Oracle a licensing fee. (Given Oracle's business practices, probably an exorbitant licensing fee.)

Moving to a different platform is equally expensive. Google will have to select a new language and make tools for developers. They will also have to assist developers with existing applications, allowing them to migrate to the new platform.

Exactly which platform Google picks isn't critical. Possibly Python; Google supports it in their App Engine. Another candidate is Google Go, which Google also supports in App Engine. (The latter would be a little more complicated, as Go compiles to executables and not bytecode, and I'm not sure that all Android devices have the same processor.)

Google's decision affects more that just Google and Android. It affects the entire market for Java. The two big segments for Java are server applications and Android applications. (Java as a teaching language is probably the third.) If Google were to move Android to another language, a full third of the Java market would disappear.

If you have a large investment in Java applications (or are considering building new Java applications), you may want to keep an eye on Google and Android.

Saturday, December 26, 2015

Servers need RAM-based SSDs, not flash-based SSDs

How to store data? How to store data on servers? In the "good old days" (prior to 2012), servers used spinning platters of metal (disk drives) to store data. The introduction of SSDs complicated the issue.

SSDs (solid state disks) eliminate the spinning metal platters and use semiconductors to store data. Today one can purchase an SSD that plugs into a PC or server and acts just like a hard disk.

SSDs provide faster read and write times, lower power consumption, and greater reliability (mostly -- more on that later). Their cost has been higher than hard disks, but that has changed. Prices for SSDs are now in the same range as "classic" hard drives.

But life with SSDs is not perfect. The solid-state disks use flash RAM, which allows data to be stored after the power is off (something we want in our storage), but flash-based RAM is not that durable. The heavy workload that servers put on their storage systems causes failures in SSDs.

I'm sure that many folks are working on methods to improve the reliability of flash-based RAM. But for servers, I think there may be another path: classic (that is, non-flash) RAM.

We've been building RAM for decades. We know how to make it, in large quantities, and reliably. Classic RAM is typically faster than flash-based RAM, and typically cheaper. So why not use it?

The big difference in classic RAM and flash-based RAM is that classic RAM, when you remove power, forgets everything. An SSD built with classic RAM would work beautifully -- until you powered the unit off. (And even then you would notice the problem only when you powered it back on.)

The impression is that such an arrangement would be useless. I'm not so sure, for two reasons.

First, while desktop PCs are powered on and off regularly, servers are not. Servers are typically installed in data centers and kept on, twenty-four hours a day, every day of the year. If the server stays on, the data stays in RAM, and everything works.

Second, it is possible to build classic RAM SSDs with small, auxiliary power supplies. These small power supplies (usually batteries) can keep the RAM active, and therefore keep the data, while main power is not available.

RAM storage units are not new. They date back to the mid-1970s, and have been used on IBM mainframes, minicomputers, and even Apple II computers.

I suspect that at some point, someone will figure out that classic RAM makes sense for servers, and build SSDs for servers. This will be another step in the divergence of desktop PCs and server computers.

Thursday, December 17, 2015

Indentation and brackets in languages

Apple has released Swift, an object-oriented language for developing applications. It may be that Swift marks the end of the C-style indentation and brace syntax.

Why is indentation and bracket style important? Maybe we should start with a definition of indentation and brace style, and go from there.

In the far past, programs were punched onto cards and the syntax of the programming language reflected that. FORTRAN and COBOL languages reserved columns 1 through 6 for line numbers, column 7 for line continuation and comments, and columns 73 to 80 for card sequencing. (The last was used to sort the card deck when it was dropped and cards spilled on the floor.)

The limitations of the punch card and the notion that one and only one statement may appear on one (or possibly more) cards had a heavy influence on the syntax of the language.

Algol introduced a number of changes. It introduced the 'begin/end' keywords that were later used by Pascal and became the braces in most modern languages. It removed the importance of newlines, allowing multiple statements on a single line, and allowing a single statement to span multiple lines without special continuation markers.

The result was the syntax we have in C, C++, Java, and C# (and a bunch of other languages). Semicolons (not newlines) terminate statements. Braces group statements. Indentation doesn't matter, statements can begin in any column we desire. All of these features come from Algol. (At the beginning of this essay I referred to it as "C-style indentation and brace syntax", but the ideas really originated in Algol.)

The "Algol revolution" threw off the shackles of column-based syntax. Programmers may not have rejoiced, but they did like the new world. They wrote programs gleefully, indenting as they liked and arguing about the "one true brace style".

Some programmers wanted this style:

function_name(params) {
    statements
}

Others wanted:

function_name(params)
{
    statements
}

And yet others wanted:

function_name(params)
    {
    statements
    }

There were some programmers who wanted to use whatever style that felt comfortable at the time, even if that meant that their code was inconsistent and difficult to read.

It turns out that the complete freedom of indentation and brace placement is not always a good thing. In the past decades, we have taken some steps in the direction of constraints on indentation and braces.

For years, programming teams have held code reviews. Some of these reviews look at the formatting of the code. Inconsistent indentation is flagged as something to be corrected. Variants of the lint program warn on inconsistent indentation.

Visual Studio, Microsoft's IDE for professionals, auto-formats code. It did some with the old Visual Basic. Today it auto-indents and auto-spaces C# code. It even aligns braces according to the style you choose.

The Python language uses indentation, not braces, to mark code blocks. It reports inconsistent indentation and refuses to run code until the indentation is consistent.

The Go language uses braces to mark code blocks and it requires a specific style of braces (the first style shown above). It won't compile programs that use other styles.

We have designed our processes, out tools, and our languages to care about indentation and brace style. We are gradually moving to language syntax that uses them, that considers them significant.

As programmers, as people who read code, we want consistent indentation. We want consistent brace style. We want these things because it makes the code easier to read.

Which gets us back to Swift.

Swift has restrictions on brace style. It uses brace placement to assist in the determination of statements, visible in the syntax for if statements and for loops (and I suspect while loops). Indentation doesn't matter. Brace style does.

We now have three popular languages (Python, Go, and Swift) that care about indentation or brace style. That, I think, shows a trend. Language designers are beginning to care about these aspects of syntax, and developers are willing to work with languages that enforce them. We will not return to the harsh constraints of FORTRAN and COBOL, but we will retreat from the complete freedom of Algol, Pascal, and C. And I think that the middle ground allow us to develop and share programs effectively.