The Supply Chain Needs Big-Data to Be Fast Data

Big-data. We've heard a lot about it recently. With the cloud, social networks, and number of devices mushrooming, the challenges associated with managing, analyzing, and executing decisions based on all this data multiply exponentially, too.

The amount of data from all sources being produced on an annual basis is already overwhelming. As this Intel video points out, global data in 2013 is predicted to grow to 2.7 zettabytes (one zettabyte equals 1 billion terabytes) -- or in clearer terms, 500 times more data than "all data ever generated prior to 2003... and it's going to grow three times bigger than that by 2015."

Many industry watchers, including Gartner Inc., predict big-data will fuel huge amounts of IT spending. Gartner notes that $28 billion of worldwide IT spending in 2012 is expected to be dog-eared for big-data, and in 2013 that number will jump to $34 billion.

Although often seen as its own market needing its own tools, big-data is not a standalone issue. Rather, it is something that affects all corporate data, practices, and software solutions, and soon there will be no distinction between big-data and regular data, according to Gartner:

"Despite the hype, big data is not a distinct, stand-alone market, it but represents an industrywide market force which must be addressed in products, practices and solution delivery," said Mark Beyer, research vice president at Gartner. "In 2011, big data formed a new driver in almost every category of IT spending. However, through 2018, big data requirements will gradually evolve from differentiation to 'table stakes' in information management practices and technology. By 2020, big data features and functionality will be non-differentiating and routinely expected from traditional enterprise vendors and part of their product offerings."

This is the key phrase: "Big data requirements will gradually evolve from differentiation to 'table stakes' in information management practices and technology." Translation: Companies that incorporate big-data solutions today will be first-movers, which leads to competitive advantages enterprise-wide but also more specifically within their supply chains.

One of the biggest challenges facing companies -- at least from a supply chain perspective -- is figuring out how to collect, aggregate, and use unstructured, big-data inputs and convert it into "fast data," or meaningful data that can be used to help make quicker decisions, allocate supply chain resources more efficiently, reduce complexity, or increase agility.

But as this Forbes article points out, "The incessantly changing positions of forecasts, orders, shipments and inventory... is complicated enough within the virtual enterprise, and becomes downright overwhelming in the context of global trading networks – with multiple tiers of partners trying to manage information changes across unique operating systems."

It's obvious, as the Forbes article notes, that all participants in an organization and the broader supply chain ecosystem "need to have access to a shared version of the truth plus the ability to act on this information in real time." Arguably, though, supply chain collaboration is only the starting point.

Many practices -- particularly those related to demand planning, inventory management, and order fulfillment -- also have to evolve. And it's just not software tools that have to be upgraded to better deal with the flow of big and fast data.

But we'd be fooling ourselves if we ignored the very human aspect involved in all this. Sure, automating supply chain decisions is effective and is probably the longer-term solution. But looking at how the supply chain team thinks about, behaves towards, and reacts to the piles of existing unanticipated, free-form data can't be underestimated either. Maybe, in fact, the big and fast data dilemma is a blessing in disguise -- something that will compel innovative supply chain thinking and create advantages not witnessed before.

@Jennifer I haven't identified any specific companies, though I did see an article about Nike and big data at http://www.smartplanet.com/blog/bulletin/how-open-source-big-data-can-improve-supply-chains/4833.

@Jennifer yes, one of the key points I've seen mentioned over and over again in articles on big data is that organziations have to set up their analytics to ask the right questions. They have to not just collect data but set it up in a way that yields what is not called "actionable" information.

Barbara - There is a definitely an unbecomin side to all of this, with privacy being one of the biggest areas of concern. Europe, at least, has some stronger privacy protection rules in places when compared to the US, but even so... maybe we're not far away from the day when device manufacturers will tap in the mall cameras and see which piece of hardware you pick up when browsing the aisles and use that info in their product planning process. Scary.

Susan, Barbara - I think the trick is going to be determining what info -- when looked at with other data -- could be turned into something "meaningful" for everyone from product development, manufacturing, and sales. I suspect we'll see a lot of trial and error, and as many different models as there are companies.

Thanks for your feedback--Ariella and Susan have good points. Part of me is scared of big data in terms of what it culls from my increasing online usage. On the other hand, I see its usefulbness and opportunity for the high-tech industry. Processing data (computer equipment and technology) and managing the data (second and third parties) will be an avenue of growth. I guess the problem I see is not yet enough customization--I get e-mail from many sites I just investigate for the heck of it.

@Barbara I belive that big data has to be taken like any technological advance. A business has to do a cost-benefit analysis to figure out if it pays to invest in it. For example, if you can use big data to improve your marketing, it still may not pay to use it if it will cost you more than it will bring in. It also is not absolutely guaranteed as it offers probabilities that, at best can offer a percentage rate of certainty in the high 90s -- not 100.

While conversations about digital supply chains are pervasive, many executives and supply chain teams are not sure how to scale their initiatives beyond piecemeal tools. The decisions made now have the potential to impact the future of business.