The explosion of data marks the digitised world. Analysis of ‘Big Data’ – the datasets whose size is beyond the capacity of the typical database software tools to capture, store, manage and analyse – is the basis of competition.

Every area of the global economy captures data in torrential gushes. These fuel new waves of productivity growth, innovation and consumer surplus, according to a research report published by MGI and McKinsey’s Business Technology Office, originally published in 2011.

Following up on this ground breaking report on the big data revolution, the MGI, in collaboration with McKinsey Analytics, has come up with research on data and analytics as a new basis of competition. Data sciences have made rapid advances over the years and corporate competition is now based on analytics. The research aims at helping organisational leaders understand its potential impact, providing greater clarity on what the technology can do.

Advances to computational power

Five years have gone by after the MGI’s report highlighted the transformational potential of big data. The range of applications and opportunities associated with this has registered a massive growth. In a world where a fragile line differentiates the physical from the digital, the exponential growth in the volume of data is complemented by their richness and diversity. The world that constantly lives ‘online’ creates content through social media and other forms, leaving a rich data trail. There has also been an unprecedented improvement in algorithms. Advancements in computational power and storage capacity have further fueled the progress. The fastest super computer in the world (as on June 2016) is 40 times as powerful as the fastest super computer of 2010. Developments of this kind have spurred the growth of cloud-based platforms, resulting in simplified data architecture and lowered capital investments.

Leading players across industries use analytics to maximise their competitive advantages. The business giants resort to the digitisation of customer interactions to generate granular data that helps in micro-targeting new clients. Entering new markets and changing the business models have become easier, thanks to the information explosion. Data-driven decision-making is the norm.

Greatest growth in LBS

The ‘Big Data Report’ of 2011 focused on five domains: Location Based Services, US Retail, Manufacturing, US Healthcare and European Union Public Sector- to evaluate the progress towards capturing the potential value that data and analytics can deliver.

Over the five years, the greatest growth has been in Location-based services (LBS). LBS use GPS and other data to pinpoint the location of an entity. This domain has captured 50 to 60 per cent of the potential value anticipated in the 2011 Report, thanks to the omnipresence of GPS-enabled smart devices. Real-time navigation technologies are being monetised in new ways (think Uber!).In the retail sector, 40 per cent of the potential has been tapped. Data analytics has helped the leaders in the retail sector. However, the margins remain low as consumers benefit the most from the fierce competition in the retail world. Real-time information helps businesses to respond promptly to customer demand. In marketing, customer demand patterns are being identified and recommendations being given on that basis.

A handful of industry leaders has managed to capture 20 to 30 per cent of the potential anticipated in the manufacturing sector. Development of digital models of the entire production process (“Digital Factories”) has helped optimise operations in some cases, while the usage of sensor data has reduced operating costs by 5 to 15 per cent in some others. Data-driven feedback is changing the face of after-sales servicing.

Only 10 to 20 per cent of the anticipated potential for cost savings, efficiency and transparency has been tapped in the public sector, with some government entities moving more interactions online and adopting pre-filled forms.

In healthcare, the pharmaceutical industry has witnessed some growth in the use of data analytics in R&D in identifying the target population for developing drugs and in reducing the costs involved in clinical trials. The potential to launch a new era of truly personalised medicine is also alive.

Barriers: internal and external

Adopting analytics requires several changes in the organisational structure, business processes, investments in technology, infrastructure, and human talent. Human capital required for analytics is a critical constraint. The growing demand can be matched only by the available supply. While the access to data has improved, it is not always possible to share data seamlessly across various spheres. Issues like privacy, cyber-security and liability continue to challenge the private sector and policy makers alike.

In a world where it’s hard to avoid data generation, its uniqueness, end users and utility determine the value of data. Different users can use the same piece of data for entirely different purposes, making them non-rivalrous in nature.

Generation and collection of data are the starting point. If data cannot be legally shared between parties, then such data holds great value and the holders will continue to have greater opportunities to monetise them. Legal licensing may be resorted to create artificial scarcity of data. In situations of such constraints, the generators and collectors of data can capture significant value.

Data analysis, the point where data meet their use and users, is highly potent, as excellent analysis can be made of mediocre data, while poor analysis destroys high-quality data.

Data-guided decision-making has transformed the way organisations operate and create value. In areas where large-scale analytics is a necessity, but core competency is a scarcity, data collectors and data aggregators are integrating data analysis into the service they provide.

Disruptions and machine learning

Competition is being reshaped by data analtics. The economy is being shaken up by the new data and the future is open to disruptions on a wide scale.

Digital platforms that can seamlessly scale up by the increased demand (‘Hyper-scaling’) have transformed the market by matching demand and supply real time, setting off major ripples in urban transportation, retail, social networking, music... an endless list!

The population is micro-segmented by individual preferences, resulting in micro targeting. Offerings to each can now be radically personalised, with a wealth of detailed data. This has a profound impact on the way services are delivered, especially in sectors like education, healthcare, travel and hospitality and media.

Machine Learning aims at training a machine to learn without being programmed. Algorithms are now created with the ability to ‘learn from data’ and adapt to the new circumstances without explicit reprogramming. The algorithms require large volumes of data, which are termed ‘experiences,’ to learn pattern recognition, so as to develop a model. By learning from these experiences, the algorithm can refine the model and arrive at suitable solutions.

Machine learning has great application across industries in solving a variety of business problems. It helps in the personalisation of goods and services to customers and predictive analytics across industries like healthcare, pharmaceuticals, telecom, finance and media.

The world is on the brink of a massive metamorphosis with data and analytics as growing phenomena. Techniques like deep learning are paving the way for bigger changes across horizons, with machines acquiring exceptional capabilities to think, understand languages, and solve problems. The gap between the leaders and laggards is widening.

The world spells competition. The march for survival is through change. Through ‘data’-morphosis.

IE, the business magazine from south was launched in 1968 and pioneered business journalism in south. Through the 45 years IE has been focusing on well-presented and well-researched articles. When giants in the industry stumbled to keep pace with the digital revolution, IE stayed affixed embracing technology.