Big data is about action, and leadership

Big data. One of the trends of the year. A new technology that brings new opportunities for businesses and organizations that use it.

Big data also brings challenges: the resources to collect and store large quantities of data, the tools to analyze and present large quantities of data, and the ability to act on that data. That last item is the most important.

Collecting data, storing it, and analyzing it are tasks that mostly consist of technology, and technology is easily available. Data storage (through SAN or NAS or cloud-based storage) is a matter of money and equipment. Collecting data may be a little harder, since you must decide on what to collect and then you must make the programming changes to perform the actual collection -- but those are not that hard.

Analyzing data is also mostly a matter of technology. We have the computing hardware and the analytic software to "slice and dice" data and serve it up in graphs and visualizations.

The hard part of big data is none of the above. The hard part of big data is deciding a course of action and executing it. Big data gives you information. It gives you insight. And I suspect that it gives you those things faster than your current systems.

It's one thing to collect the data. It's another thing to change your procedures and maybe even your business. Collecting the data is primarily a technology issue. Changing procedures is often a political one. People are (often) reluctant to change. Changes to business plans may shift the balance of power within an organization. Your co-workers may be unwilling to give up some of that power. (Of course, others may be more than happy to gain power.)

The challenge of big data is not in the technology but in the changes driven by big data and the leadership for those changes. Interpreting data, deciding on changes, executing those changes, and repeating that cycle (possibly more frequently than before) is the payoff of big data.

Software subscriptions

One of our current debates is the change from traditional, PC-installed software to web-based software.

Even Microsoft is switching. Microsoft in addition to its classic PC software "Office 2013" which is installed on your local PC, now offers the subscription package "Office 365".

It's a big change, and many folks are concerned. System admins worry about the new procedures for signing up with new software services. Managers fret that an update could change file formats, and locally stored documents in old formats may become unreadable. Ordinary users find the concept of renting, not owning, their software a bit disconcerting. A few folks are aghast to learn that their software could disappear after failing to pay for the subscription, and have railed against the increased cost and the greed of software vendors.

Yes, I know that software is usually licensed and not sold. But most folks think of current PC software as sold, regardless of the licensing agreement. It is this general understanding that I consider to be important.

Let's run a thought experiment. Suppose technology was going in the other direction. What if, instead of starting with purchased software and moving to subscriptions, we were starting with subscriptions and moving to purchased software?

In that change, people would be complaining, of course. Users would be hesitant to move from the comfort and convenience of subscription software to the strange new world of installed software. System admin types would grumble about the additional work of installing software and applying updates. Managers would fret about compatibility, fearing that some users would have old versions of software and might be unable to share files. Ordinary users might find the concept of "owning" software a bit disconcerting. I suspect that a lot of people would be aghast to learn that they would have to pay for each device they used, and rail against the increased cost and the greed of software vendors.

Such a thought experiment shows that the change from ownership to rental is big, but perhaps not a bad thing. The decision between PC-installed software and web-based software subscriptions (or mobile/cloud subscriptions) is similar to the decision to own a house or rent a condominium. Both have advantages, and drawbacks.

My advice is to experiment with this new model. Start using web-based e-mail, word processing, and spreadsheets. Try the file-sharing services of SkyDrive, Google Drive, and DropBox. Learn how they work. Learn how your organization and use them. Then you can decide which is the better method for your team.

Unexpected success

George Lucas is know for the "Star Wars" movies, some of the most successful movies of all time.

Yet I suspect it started differently.

In 1977, Lucas had the movies "THX-1138" and "American Graffiti" behind him. "THX-1138" is an obscure movie, now most famous for being one of Lucas' creations. It is a decent movie, and respected by science fiction fans, but not known outside of fandom. "American Graffiti" was a successful movie: popular in its day but now more of a fond memory. (When was the last time you watched it?) There is nothing in either movie that says "genius movie maker".

I suspect that George Lucas made "Star Wars" and was hoping for a reasonable amount of success, and that he was not expecting the movie to become the foundation of a franchise and marketing empire.

I believe that Apple, with the first iPhone, was, like George Lucas, hoping for a reasonable amount of success. I also believe that the tremendous response far surpassed Apple's expectations. (I suspect it also surpassed AT&T's expectations, which conveniently explains the difficulties encountered by so many new iPhone customers when they activated their accounts.)

Their successes were due, in part, to the gambles that each made. Lucas used computers to control the models of X-wing fighters, selected classical music, and released in the summer. Apple created an elegant design quite different from contemporary cell phones, leveraged its "easy to install" ideas for apps from Mac OSX, and built an interface that was different from the traditional Windows (and even Mac) OS.

Lucas' work stamped itself onto our culture, with "The Force" and even the quote "I've got a bad feeling about this".

Apple's work changed the course of the industry, such that Microsoft Windows and the "windows, icons, mouse, and pointer" theme is no longer the design leader. Microsoft's introduction of Windows RT and the "Modern" UI shows the effect of the iPhone success.

All of which perhaps is evidence that success is something that cannot be planned, timed, or scheduled, and that success can come from taking risks and ignoring established ideas.