The decentralization of digital technology and our capacity both to generate and collect vast amounts of data have profoundly changed the way consumers interact with information. The digital world is transforming society by enabling unprecedented efficiency in our interaction with networks and markets. Open data can play a central role in this transformation, in particular helping to create new markets for economic, social, cultural and political activity. This three-part series examines the challenge of taking advantage of open data.

Data is an important commodity in our digital era and there is growing public expectation that the digital markets that both create and rely on it will shift the way we work, interact and consume goods and services.

While there has been much speculation as to how data and modern technologies will impact the economy, some governments have begun to release data in the hope that this will boost productivity and economic growth. These ‘open data’ policies are designed to provide industry and entrepreneurs with opportunities to create new data-driven products and services. The wider global initiatives such as the Open Government Partnership (OGP), of which these policies are a part, are designed to promote the growth of open data by supporting the infrastructure and resources required.

Open Data is free data that is available to the public without restrictions and, according to the McKinsey Global Institute, it has the potential to unlock as much as $3 trillion USD in additional value annually by delivering better industry insights and targeted client services. Yet since the release of this estimate in 2013 public debate has intensified as to why open data has not delivered its full potential.

While governments are recognised as key to facilitating the provision and regulation of open data and many data providers are endeavoring to follow the call to open data by default, it is clear that consumers are not fully prepared for the release of vast amounts of raw data, of varying quality.

Prolific data release, while improving government openness, may do little to address government transparency or trust if not managed and processed properly, and may create barriers to data usage if users find the data difficult to access, navigate or use. To improve open data accessibility amongst the public, standards and guidelines for data release formats and licensing have been broadly recognised and contribute towards a more consistent approach.

However, adhering to these guidelines and standards has its administration costs, and the burden often falls to data providers who may prefer to release data in bulk rather than improve its quality. Indeed, conforming to specific standards and formats at the expense of more data release may seem like shifting the focus to making the data more user-centric, but does it really benefit the user?

By limiting the supply of open data, we may not be fully considering the opportunity costs of not releasing an important dataset or the rapid technological advances in Artificial Intelligence (AI) for the purpose of cognitive automation. More data may allow AIs to learn how to best tailor data for optimal use, and they may also be able to keep up with our changing data formats as our technology evolves.

Perhaps with sufficient training and use, machines will one day be able to determine for themselves, which datasets are suitable for release and which could potentially compromise privacy. Internationally adopted AIs may even assist global anti-corruption efforts by utilizing open data more effectively in exposing bribery, tax non-compliance or corruption.

As we move towards an increasingly cognified world AI, augmented with human decision-making, may help unleash the true potential of open data, allowing for better user accessibility and interactivity. The real question we should be asking is whether the data collected and released has the capacity to revolutionise our way of life and bring about seismic changes in industry and our global market interactions.

Dr. Audrey Lobo-Pulo is a Senior Adviser in the Australian Public Service, and is an advocate for open government and open source software in government modelling. Views expressed in this article are those of the author and not necessarily those of the Australian Government, nor of the Global Digital Foundation, which does not hold corporate views.