APIs Start to Set the Big Data Pace

One the one hand Big Data is supposed to be one of the best things to happen to IT since the invention of the database. After all, managing massive amounts of data should make IT more relevant than ever. But as Big Data continues to evolve it’s starting to look like managing all that data is beyond the scope of many IT organizations. As a result line of businesses are turning to raft of emerging Big Data services in the cloud that allow them to analyze massive amounts of data by invoking a relatively simple API.

As Big Data services increasingly standard those APIs, it’s starting to look like Big Data services in the cloud will become the dominant method for accessing Big Data long before internal IT organizations can get their Hadoop applications out of pilot.

Two examples of these Big Data services are Beyond Analysis, a provider of a Big Data analytics applications used mainly in Europe that is now expanding in the U.S., and Versium, a Big Data analytics applications the organization can use to correlate email addresses with that person’s social networking activity streams to gain insights into their behavior.

According to Hunter Albright, CEO of North America and head of worldwide consulting for Beyond Analysis, rather than building Hadoop applications companies such as Tesco and Visa Europe, it’s a lot easier for organizations to load and model massive amounts of data using the Beyond Analysis platform, which can be invoked via an analytics application using a well-documented set of APIs.

Similarly, Versium CEO Chris Matty says a rich API allows organizations to not only easily query the massive database of email addresses, called the LifeData Analytics Platform, that Versium has aggregated from public web sites, they can also analyze the social attributes of the people that own those email address using an analytics application that Versium presents via series of dashboard applications.

In both instances services leveraging APIs are providing access to Big Data analytics applications that internal IT organizations don’t have the time, skills or resources to build.

As the programmable web begins to enter a new era where analytics applications are exposing massive amounts of data via APIs, the business value of the web as whole is set to increase dramatically. The only thing standing in the way of making that happen is making sure that the APIs that being used to access Big Data applications are as easy to invoke as possible.

Comments(2)

Michael - great post - at Sociocast we 100% agree with your assessment that "As Big Data services increasingly standard those APIs, itâ€™s starting to look like Big Data services in the cloud will become the dominant method for accessing Big Data long before internal IT organizations can get their Hadoop applications out of pilot."

This is something that we have commonly seen, and have our built a Predictions API that takes advantage of the current "slow" roll-out process. When internal IT departments find out about our API and Cloud Service - they are very relived and happy!