Big data is about action, and leadership

Big data. One of the trends of the year. A new technology that brings new opportunities for businesses and organizations that use it.

Big data also brings challenges: the resources to collect and store large quantities of data, the tools to analyze and present large quantities of data, and the ability to act on that data. That last item is the most important.

Collecting data, storing it, and analyzing it are tasks that mostly consist of technology, and technology is easily available. Data storage (through SAN or NAS or cloud-based storage) is a matter of money and equipment. Collecting data may be a little harder, since you must decide on what to collect and then you must make the programming changes to perform the actual collection -- but those are not that hard.

Analyzing data is also mostly a matter of technology. We have the computing hardware and the analytic software to "slice and dice" data and serve it up in graphs and visualizations.

The hard part of big data is none of the above. The hard part of big data is deciding a course of action and executing it. Big data gives you information. It gives you insight. And I suspect that it gives you those things faster than your current systems.

It's one thing to collect the data. It's another thing to change your procedures and maybe even your business. Collecting the data is primarily a technology issue. Changing procedures is often a political one. People are (often) reluctant to change. Changes to business plans may shift the balance of power within an organization. Your co-workers may be unwilling to give up some of that power. (Of course, others may be more than happy to gain power.)

The challenge of big data is not in the technology but in the changes driven by big data and the leadership for those changes. Interpreting data, deciding on changes, executing those changes, and repeating that cycle (possibly more frequently than before) is the payoff of big data.