Some analysts claim that big data hasn’t lived up to its promise. Mind you, nothing could live up to the expectation that was generated. Ironically, now the spotlight may be turned elsewhere, big data will deliver big time.

The impact of big data was less than expected, says the September 2016 Gartner Group research note Using Algorithmic Retailing to Drive Competitive Advantage. Gartner says that future algorithms will cut costs and boost revenue for industry as the Internet of Things (IoT) takes off. I dare say they may, one day.

In the meantime, as expectation is stocked up for algorithms, the pressure is off the big data camp and they can get on with their work and start delivering the benefits. In the last few years the volumes of data being generated have boomed. According to Gartner, around 90% of all the data in the world was created in the last two years.

For once, it’s not hyperbole to use the word ‘explosion’ in the context of data, since we really are witnessing a textbook ‘sudden violent change of energy’. For proof you only need to check the data center, where servers have to be powered and processors have to be cooled in order to accommodate the mass volumes and diversities of information types being produced.

Has big data been underwhelming? It’s more fair to say the task, created by the tsunamis of intelligence flooding into servers in every industry, has been overwhelming. Pharmaceuticals companies have never been able to capture such a variety of information types. As a result of all the new, affordable types of sensors, manufacturers have never had so much intelligence about their machinery and production lines. Retailers have never had so many channels to market and have never been able to capture so much information about their customers.

Those are the scales of the big data challenge that have been created. However, cometh the hour, cometh the man or, more accurately, the plan. The ‘plan’ in this context is in-memory computing, which provided the breakthrough that makes big data analysis possible.

By smashing through the limitations of how many amorphous files and unregimented fields can be fed into the analysis machine at once, in-memory has surpassed the performance limitations in all sectors, from real-time reporting for the mobile phone operators, through retail’s omni channels, to pharmacy, to fintech’s automated trading applications.

The new uses of big data analysis (at least the ones we can write about) are often in retail. For example, in an omnichannel world, it’s no longer enough to combat fraud in silos. Omnichannel fraud management wasn’t great in 2015. An ACI Universal Payments/Forrester Consulting study unveiled that 65% of retailers believe they lack the tools to effectively manage omnichannel fraud. Just over half (54%) admitted their data analysis staff aren’t skilled in dealing with omnichannel fraud challenges. But it can be addressed now.

Social media has created a big data challenge, but it’s created a new business opportunity in social selling.

Clothing retailer, TopShelf Style, is a great example of social selling in action. TopShelf uses an Instagram/Soldsie system for a new type of ‘comment selling’ on Instagram. Instagram leads but Facebook, Twitter, and Pinterest all released buy buttons in 2015 and are investing massively to catch up in social commerce.

Meanwhile, a McKinsey report found that the uses of IoT in retail could have an economic impact of $410 billion to $1.2 trillion per year by 2025 as retailers use big data to bring all their channels together and fine tune the all-important store experience.

Today’s consumers buy goods from the same retailers through multiple channels, most of which are not integrated cohesively. These silos are proving expensive, as it’s like running several businesses at once. By analyzing the masses of online and offline data, retailers will get a better picture of their customers’ shopping journeys and will be able to smooth their paths.

Big data analysis tells a retailer like John Lewis that women who buy expensive jewellery in the store have previously browsed for it online. By examining their clients’ online history, they can make a reasonably accurate prediction of buying intentions and stock accordingly. Staff could be pre-briefed to prepare their pitches and their upselling tactics for these products. Having a complete understanding of their customers’ desires and laying metaphorical petals in their pathway to purchase will have massive benefits.

If the pressure is off big data, it may be time to start enjoying ourselves with our own brand of retail therapy!

Regional Vice President - TIBCO – UK&I / Nordics
Mark is a high energy leader, with extensive IT Sales and Marketing experience, working across Europe, Africa and North America. Having joined TIBCO in early 2016, Mark has considerable experience in Strategic Selling and Operational Excellence within Sales and Marketing and knowledge of both direct and indirect business models.
Prior to TIBCO, Mark has held roles at companies including SAP and Microsoft and has a proven track record in both large and small enterprises.