I'm the vice president of strategy at Silicon Valley Data Science. I was the founding chair of O'Reilly Strata, and founding editor of the journal Big Data. I create conferences, software and words. Current interests include data, the Internet of Things and the nature of work.

Making A Moonshot? Put Data In Your Rocket

In the rush to exploit big data, it’s not enough to have the right technologies in place. You create the conditions for success by making data itself open and available.

Larry Page, CEO of GoogleGoogle, believes in moonshots. Not just incremental thinking, but breakthrough progress that makes an order of magnitude difference in a field. At his company’s Google I/O developer conference in San Francisco, Page urged others to do the same: “I’d encourage more companies to do things that are a little bit outside of their comfort zone, because I think it gives them more scalability in what they can get done. Almost every time we have tried to do something crazy we have made progress, not all the time, but much of the time.”

Aside from Page’s belief, what has been Google’s secret? Quite simply: data. Google’s order-of-magnitude advantage in the search stakes was maintained by algorithms that gave positive returns on scale. The more people used the search engine, the better it got, thanks to deriving quality signals from user behavior—if you used the first result from a search, it’s a good result, if you have to use the seventh, then the result can be improved.

In the world of big data, we’re becoming accustomed to such progressions. Yet the data connection can also be drawn to Google’s more explicitly adventurous innovations, the autonomous car, or the wearable computer Glass. The car could not drive without the masses of data accumulated from the company’s maps product, and Glass would be useless without the social data collected by Google Plus.

In each instance, Google’s bold moves have been fueled by the vast potential of data. There’s a lesson here for us with respect to open data. Open data is not just important for transparency and accountability: it’s actually the raw stuff of economic growth.

As a community we should welcome and endorse moves to open up public data for the potential of growth in research and commerce that this data provides. We saw a useful step forward in the US this May, as President Obama signed an executive order focusing on using government open data for entrepreneurship, innovation, and scientific discovery.

Data is rocket fuel

The advent of big data technologies has caused wild excitement for its potential and also the inevitable backlash due to media overstatement. Big data stands on the legs of two important pillars: open source software and commodity computing hardware. These things have lowered the bar for accessing supercomputer-level power and vastly increased data processing throughput.

To focus on these two things alone misses an important component: data itself. Critics of big data rightly observe that most companies don’t have big data, and the IT industry’s marketing of big data tools does little to serve them. The promise offered by IT vendors only addresses a portion of the opportunity. Without data, tools have little to do. In our journey to the moon, it helps if we put fuel in our ship.

The third and most important leg of the big data platform is the data. It is the most valuable asset, being the image of the world in which we are building our businesses. The availability of open data allows business to create and identify new value. In truth, everyone has a big data opportunity, even if they’re not already sitting on tons of the stuff.

As the field of big data progresses, maturity will be shown in the focus of enthusiasm moving away from the initial excitement that we can process large data, and toward understanding the acquiring, stewarding, and sharing of our data.

Prizing data is something Google understood from the start. I doubt they understood the full implications of holding data so dear, but it proved the foundation and impetus for further invention.

If, like Larry Page, you’re making a moonshot, then having big data as rocket fuel sure helps.

(This article has been adapted from my editorial in a recent issue of Big Data.)

Post Your Comment

Post Your Reply

Forbes writers have the ability to call out member comments they find particularly interesting. Called-out comments are highlighted across the Forbes network. You'll be notified if your comment is called out.