The Data Gap in Manufacturing

The amount of data the world creates and stores is staggering, and it doubles every two years. Nevertheless, enterprises, especially manufacturers, are ignoring nearly all of it to their detriment.

An IDC study sponsored by EMC shows that only 3 percent of all data created in the world each year is tagged or otherwise categorized for future use. Only 0.5 percent is ever analyzed. It isn't a surprise that the figures are low. The sheer amount of data makes it hard to figure out what is useful to analyze and what business value there is in doing so. Another issue is that the data is scattered across billions of locations and is not necessarily owned by the people who need it.

However, there is hope for making some sense of it. Though consumers create 68 percent of all new data, 80 percent of the world's data is stored by enterprises. This creates an especially interesting opportunity for manufacturers that need access to consumer data to provide the best products to their customers.

A survey of the state of data in 2012 and the near future shows opportunities and challenges for the creative use of data by the people who create, design, and sell consumer products.

Opportunity: IDC cites what it calls the opportunity a "Big Data gap," but honestly, I don't think it is about big-data so much as it is about the smart use of discrete data. Whatever you want to call it, IDC said 23 percent of the data made in 2012 would be useful to enterprises if it were tagged and analyzed. Since only 3 percent of it is tagged and less than 1 percent is analyzed, there is digital gold out there to be mined.

Challenge: Manufacturing is the least likely entity to own the kind of consumer data that would lead to smart product decisions and better designs. That data is held in social networking sites and unstructured corporate resources. One way around the problem is creating crowdsourced brainstorming sites like IdeaStorm from Dell (this site's sponsor), which invites people to share and discuss potential products and solutions. But that is not capturing naturally generated data so much as commissioning new data. Smart manufacturers will need to gain access to the data that customers create outside of formal settings. One way would involve niche social networks sponsored by products with loyal followings. We see this with Facebook pages and online forums, but enterprises are only starting to realize the value of these communities.

Opportunity: From 2005 to 2020, data will grow from 130 exabytes to 40,000 exabytes. By 2020, the world will generate 5,200 gigabytes per person each year. Most of this will be machine-generated, taggable data, and it will be useful if handled correctly.

Challenge: Though the cost of storing a gigbayte of data is dropping, the amount of data we are storing is rising faster. IDC estimates the cost of storing all the data we want to store will rise 40 percent by 2020.

Right now, any manufacturers looking to use crowd-created data for crowdsourcing new products and business opportunities find themselves in a strange situation -- water, water everywhere, but not a drop to drink. There's so much data, and enterprises are paying so much to store it, but right now, no one can make heads or tails of it.

The winners of the data revolution will be those who find a way to access the information they need from multiple sources, visualize it in ways that make sense, and store it without drowning in it or cutting into profits. CIOs, who are often overlooked at manufacturers as the ones who buy the ERP and makes the robots work, will be at the forefront of a data revolution among manufacturers -- at least at those that plan to be in business in 2020.

It sounds like if manufacturers want to compete using data they need to not only find the right data to use, which they may not have yet, but also need to sift it fast enough to only store what they really need to keep cost down. Are there any manufacturers doing a good job at this now for others to model?

@zaius- Companies do have to decide what they want. But more mportantly, they need to get the data they want in their own hands. The data they want is often stored by companies and people they don't have access to. That's why Facebook is so tantalizing to investors-- they own tons of data that people want, but they don't know how to use in a way that doesn't violate terms and privacy. If they ever good, or if that type of data could get in the hands of people who COULD use it, that's the gold mine.

@lufu- I think transient data is data that is never stored. Think about all the phone calls you've made. Unless someone is cheating you, they've never been recorded. They're lost. And frankly, that's a good thing right up untl you have an argument with your wife over what you said 5 years ago on the phone.

TransientData - IDC indicates that, "Much of the digital universe is transient - phone calls that are not recorded, digital TV images that are watched (or "consumed") that are not saved..."

Maybe this is a rhetorical question but, if this digital data isn't saved or stored, does it just disappear or keep on floating around in the Internet pipes far into perpetuity? Or is it just a virtual tree falling in the forest?

@Zaius Great point, which is another major aspect of all of this, the U.S. supposedly has a shortage of skilled analyst to perform these functions and there is still the question of whether businesses will actually pay for this expertise.

@sohaibmasood I am not too surprised every manufacturing company I have ever know was challenged in this way, even as the new generation of owners arises, they still do not understand out to effectively mine their own data.

Just goes to show that some industries are insulated from the effect of truly understanding their market. Well judging from IDC's numbers most companies are functioning this way.

The blogs and comments posted on EnterpriseEfficiency.com do not reflect the views of TechWeb, EnterpriseEfficiency.com, or its sponsors. EnterpriseEfficiency.com, TechWeb, and its sponsors do not assume responsibility for any comments, claims, or opinions made by authors and bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.

Enterprise Efficiency is looking for engaged readers to moderate the message boards on this site. Engage in high-IQ conversations with IT industry leaders; earn kudos and perks. Interested? E-mail: moderators@enterpriseefficiency.com

Now that TGen has broken new ground in genomic research by using Dell's storage, cloud, and high-performance computing solutions, the company discusses what will come next for it and for personalized medicine.

The Translational Genomics Research Institute wanted to save lives, but its efforts were hobbled by immense computing challenges related to collecting, processing, sharing, and storing enormous amounts of data.