Big data and machine learning can put businesses one step ahead

The rise of big data and machine learning means businesses can shake off the past and stride confidently into the future, says Dan Kirby

Wouldn’t life be easier if we could predict the future? You could avoid that rain shower or take a different route to work. Or you could design your business to fit an emerging trend.

But predicting the future is the stuff of crackpots, fairground fortune-tellers and sci-fi movies like Minority Report, isn’t it?

Well, not when you combine big computers and even bigger amounts of data. The future then becomes an algorithm that can be predicted with near 100% accuracy.

And the winner is… Bing

There were two winners at this year’s World Cup in Brazil. Yes, the Germans – but also Microsoft Cortana, the Windows smartphone voice assistant that rivals Apple’s iPhone equivalent, Siri.

Powered by Microsoft’s Bing prediction engine – which has previously predicted various US game-show winners – it foresaw 15 out of 15 of the knockout match winners. The only one it got wrong was the third-place play-off between Netherlands and Brazil.

So, that’s 15 out of 16 total matches – better than Paul the Octopus (remember him from 2010?) and also Google’s prediction engine, which predicted 14 out of 16.

But how? By using algorithms and available cloud-based data – which included a team’s margin of victory in past matches, team records, home advantage and the influence of weather and other outside factors – Bing was able to predict winners based on hard data rather than human bias or expert opinions.

If computers can predict World Cup winners with pinpoint precision, then what else can they predict?

When truth becomes fiction

In 2013, noting the popularity of certain types of programmes, as well as directors and actors, Netflix was able to come up with a formula for tailor-made shows that would appeal to a vast audience.

The excellent political thriller House of Cards was Netflix’s first foray into original programming and it was brought about by big data. The service also reportedly personalised the show’s trailers to the individual viewer; for instance, female viewers were shown trailers that focused heavily on the show’s strong female leads. It was a formula that was highly fruitful. House of Cards was nominated for multiple Emmy awards and Netflix’s third effort, Orange is the New Black, made a similar impact on viewers. Although specific viewing numbers aren’t easy to determine, the series picked up 12 Emmy nods in 2014.

The fact that Netflix can successfully predict what we want shouldn’t come as a surprise – our browsing history has been subject to analysis for years.

Beards and business

Imagine if you could also predict the tastes of your customers before deciding on the next investment, which product to develop, which skills to train for and what markets to focus on. Malcolm Gladwell’s book The Tipping Point famously showed how New York hipsters sparked a craze for Hush Puppy shoes in the mid-1990s. The root of The Tipping Point concept is that a small, highly influential group acts as the ‘spark’ which sets a wildfire craze.

The same things is seen in the habits of hipsters in East London – have you noticed the preponderance of Victorian-style beards in your local youth? It started in Shoreditch. Imagine if you could have spotted that trend early and created an urban beard-grooming business or your own brand of beard oil (sold in ‘upcycled’ jam jars, obviously), cornering the market before the craze of the few hit the many?

You don’t need a supercomputer to be able to do this – just an alertness to the new, as well as social media accounts across Twitter, Instagram and Tumblr. By monitoring and mapping social sentiment from the influencers in your market, you can start to gauge the shape of things to come.

The predictive business

So imagine what will happen when these Bing-like tools become more readily available to us all, which inevitably they will do.

Imagine being able to better predict the stock you’ll need – based on the people who usually visit your town at this time of year – all by using weather patterns, social media sentiment and local events. For example, ladieswear shops will be able to stock up on stylish wellies because it’s due to rain during the local festival this year – and there are lots of fashion conscious, 20-something females attending.

Maybe accountants will be able to use predictions of election winners to be able to pre-emptively advise their clients on tax-planning matters. Or travel agents will be able to advertise holidays in southern Germany before we realise that in the future it’ll replace Tuscany as the holiday of choice for the British middle-class. And yes, we could have all bought shares in beard trimmers.

And if that is the case, maybe fund managers will become a thing of the past too – the trading is already done by computer.

Rapid response, fail fast

At the fingertips of agile entrepreneurs – and with rapid product prototyping – new concepts for brands or brand extensions could be piloted in focused markets, identified from social channels and data analysis (eg how many times a subject is mentioned).

With lean development plans, these could be iterated and rapidly scaled (or killed – depending on the results). The potential is enormous for us all to experiment based on these insights – testing the prediction, and then backing the true winners when they’ve been proven to win.

Today’s business is often run on the analysis of historical data but by creating insights from the increasing amounts of available data in the cloud, we can start to better predict the future.

About the Author

In recognition of the digital revolution, our tech section has undergone a radical change with the multi-talented Kirby coming on board. As if writing a regular column and running technology agency Techdept wasn’t enough, the digital entrepreneur also dabbles in a spot of photography and DJing. However, Kirby’s biggest claim to fame is in board games. Not many people can say they are responsible for the global board design of Cluedo.