A Massive Market Opportunity Awaits In Analyzing The Internet Of Things

Author

Matt Asay

May 2, 2014

MOST READ ARTICLES

The Internet of Things has been hyped for 15 years, but until now technological realities haven’t supported technological possibilities.

Today, given the confluence of cheap semiconductors, telecom operators with excess capacity and a new generation of open source data infrastructure, IoT is not simply possible, but probable. The question is what developers will do with it. Ultimately, as I’ve written, big money awaits developers who can turn IoT’s Big Data into cross-device services.

But before we get to this point there’s plenty of money in data management and analytics. In fact, as new research from Morgan Stanley posits, IoT services depend on the industry first maturing around IoT data management and, in particular, analytics.

In The Beginning Was The Data

The Internet of Things forces developers to approach data management very differently than the old world of structured data. In the IoT world, it’s implausible to rely on a constant, “fat” connection to the Internet. As Forrester analyst Jeffrey Hammond describes:

Architects designing systems of record [e.g., ERP and CRM systems] can usually count on a local network connection with a relational database management system (RDBMS) and don’t think twice when designing for a beefy application server.

These design sensibilities don’t work when building thousands, hundreds of thousands, or millions of edge nodes to connect to systems of operation. Network bandwidth, memory, power, and compute are necessarily constrained at the edges of the connected world.

So what should a developer do? Focus on learning new data technologies like Hadoop and NoSQL, according to Hammond:

The major hurdle that traditional development shops struggle with in the [data] analysis layer is the volume of the data that edge nodes emit, and the speed with which it must be processed. As a result, we’re seeing increased deployment of NoSQL database management systems…where developers collect data and make it ready for analysis.

This move toward more modern data infrastructure is a big opportunity for vendors in the machine-to-machine data market. But it’s not just data management vendors. It’s also those who provide enabling cloud infrastructure. In particular, Amazon’s recently launched Kinesis service offers a holistic approach, given that it includes high-speed data ingestion plus connections to S3, DynamoDB and RedShift for storage and data analysis. In many ways, developers can increasingly shop for IoT infrastructure on Amazon just as they shop for books and clothes.

Seeing Into The Data

Data management is a big market. But it’s still one step removed from the end user and, hence, not the biggest market IoT will enable. For developers looking to cash in on the IoT gold rush, analytics offers an even bigger lottery. Morgan Stanley places the analytics opportunity squarely in the middle of the IoT monetization timeline:

This jibes well with Cowen & Co. analyst Peter Goldmacher, who argues that for Big Data, “The bigger category of winners are the apps and analytics vendors that abstract the complexity of working with very complicated underlying technologies into a user friendly front end.” While we have a host of companies like Palantir and Tableau that have sprung up to help users analyze and visualize their data, IoT remains relatively uncharted territory, largely because most business intelligence or analytics tools are stuck in outdated relational database models.

Splunk probably gets organizations closest to real analysis of IoT data, given that its genesis is in analysis of machine data. Splunk has already extended its technology to embrace Hadoop, offering Hadoop analytics through a product called HUNK, and will almost certainly look for other ways to extend its reach into unstructured data, which is growing at twice the rate of structured data and already accounts for 80% of all enterprise data, according to Gartner.

Opportunity In Analytics

IoT data doesn’t fit neatly into the tables and joins of traditional data management technology. Therein lies the problem. Virtually all business intelligence and analytics companies are SQL-based and rely on neat-and-tidy relational database technology. This isn’t going to work in the big data Internet of Things economy. Across the board, such tooling needs to be re-architected. Some vendors have tried to skirt this requirement by embracing ODBC-based connectors to modern data management technologies. But this results in a flattening of the richness of unstructured data and ultimately won’t work.

All of which means that there is a huge need for a new breed of business intelligence or, more particularly, IoT intelligence. We need new analytics tools capable of ingesting and processing the diverse, unstructured data that fuels IoT. Consider this your next billion-dollar startup idea. You’re welcome.