Why clean data is essential

Procurement analytics are vital, but they only work if your organisation’s spend and supply data is accurate and up to date. Otherwise trouble looms…

Christopher Columbus was not in search of America when he discovered it. The explorer had, in fact, set out to find a western sea route to Asia from Europe. But having ignored experts who told him his information was wrong, and massively underestimating the route, it was a lucky coincidence that he came across the previously uncharted Americas. Like many war strategists, politicians – yes, think on the election polls that got it so wrong – and businesses, his action was based on information that was faulty.

Many businesses today fail to ensure the information they have is accurate and complete. And while many are looking at ‘big data’, they should be first looking at ‘clean data’, says Professor Mohan Sodhi, head of operations and supply chain management at Cass Business School. “Big and dirty data will only give false confidence,” he says. “Clean data should be a priority. But it’s something companies have not paid direct attention to yet.”

Most procurement departments are aware of the significance of strong spend analytics and up-to-date supplier information, even if they don’t yet possess it. Richie Barter, CEO of intelligent automation platform AltViz, says a number are using their move to cloud-based technology as a “once-in-a-generation opportunity to tidy up datasets associated with legacy systems”.

The challenges involved are numerous, however. Organisational culture, systems, human error, constant change and the sheer size of the task all add pressure. Earlier this year, for example, a report by think tank the Institute for Government warned British businesses that, while procuring from external suppliers amounts to a third of all public expenditure, current procurement data was poor. It said that the critical information organisations could use to make more informed spending decisions was often unavailable or incomplete.

Australia’s New South Wales government is tackling this problem head on. Last year it signed a deal with Microsoft to use technology to categorise how its Aus$30bn annual procurement budget is allocated. It hopes the data will validate what it is spending money on and confirm whether it is getting value for that money.

Across the globe, public and private organisations alike face similar challenges. Fifteen years ago the problem for many was not having enough data. Now, however, we’re all getting swamped by it. The antidote is clear processes and increased digitalisation, which requires an investment of time and money. But the prize is well worth all the effort.

“Reliable, credible data is arguably the lifeblood of extracting optimal value from your technology investments and for supporting sound decision-making,” says Indrasen Naidoo, who conducts supply chain research at Curtin University, in Perth, Western Australia.

According to Roy Anderson, CPO and digital transformation officer at global supply chain payments and marketplaces company, Tradeshift, there’s also the ability to make radical changes in areas such as recycling and the prevention of slavery, for example. “Having clean data and changing the entire world tie together because good, accurate information can be digitally transmitted faster and more easily, and so becomes more valuable.”

From this vital starting point, companies can add layers of artificial intelligence, robotic process automation and machine learning to give them more detailed, quicker and cleaner data than ever before. Tools such as artificial intelligence will tell them, for example, how many times a certain product is sold, or at what point in transit a refrigeration vehicle failed and food got spoiled.

It’s all information so vast and powerful that organisations will be able to benefit commercially within seconds, minutes or hours instead of days, weeks or months. “Machine learning will tweak things to make improvements and then start to highlight inaccuracies, which robotic process automation can then fix,” Anderson adds. “Technology will force ever cleaner data. People don’t have the bandwidth to do millions of updates; robotic process automation can do it overnight.”

But the clever stuff comes later. The first task is the essential foundation of accurate information – and that’s no mean feat.

Overcoming challenges

Vincent Toesca, VP of product management at spend management platform Coupa, says the data challenge asks three key questions: “Is the data we have valuable from a business perspective? Do we have enough of it? And is it of the right quality?”

He says paper-based invoices and documents – still the means of doing business for many – present problems. As does the manual inputting of information, which has the potential for mistakes and duplication. Finally, he warns of the fragmentation of systems, which often occurs as companies merge and grow.

Different types of data, data silos, non-standard interfaces for accessing information, quality and duplication are all issues, adds Barter at AltViz. And procurement teams experience these problems in a number of ways. “The classic example is vendor setup, where there are multiple versions of the same supplier, or different legal entities within the group structure of a supplier.”

He believes there are two ways to address the problem. The first is to clean up existing systems of record, which is typically done by the team affected, along with technology to speed things up. The second is to invest in improved technology or processes to prevent new sources of unclean data.

In 2016, Oxford University carried out line item analysis so the purchasing team could see the exact market share vendors had in each category. This meant they could offer larger volume deals in exchange for per-item cost savings. The work allowed the university to move away from adversarial supplier negotiations, and increase order volumes by consolidating suppliers and leveraging negotiations.

Anderson remembers how, in one of his past procurement roles, technology infrastructure was built internally from scratch to tidy up records. Nowadays, he says, external suppliers are better placed to help. Whether it’s vendors that are evolving, or organisations that are expanding into new territories, products or areas of expertise, supply chains are perpetually subject to change. “Companies can’t and shouldn’t reinvent the wheel,” he adds. “Let the marketplace have all the expertise, skillsets and applications you need.”

He also believes procurement should abandon ideas of cleansing old information. “The mistake people make is they feel they need to clean historical data. But you’re chasing a jet plane with a horse; you’ll never catch up, because your supply base is dynamic.”

There are three crucial elements to procurement: people, process and technology. Solving the tricky challenge of data cleansing requires all three – but it begins with process. “It’s the first place to look because with poor processes, neither systems nor people are going to be able to help,” says Professor Sodhi.

Naidoo, at Curtin University, recommends a documented, functional data strategy aligned with IT data strategy and standards of an organisation. Part of that could include sending suppliers and stakeholders an example of what information and format is required.

Getting started

Coupa’s Toesca recommends starting with the business problem you’re trying to solve and then designing processes to capture as much data as possible. “Data is the new operating system in our digital, outcome-driven economy,” he says. “There will be evolving innovations around algorithms and new artificial intelligence, but if you have the right process to extract the data that fuels them, you can future-proof your investment.”

When it comes to supplier information, Anderson suggests you put a process in place and the next time you use an existing supplier or add a new one, you subject them to it. At his company the onus on ensuring supplier information is current rests with the vendors. “Tradeshift now requires them to keep their profile up to date in the same way individuals update their own LinkedIn profiles,” he explains. This results in 80% to 95% accuracy, which is then supported by third-party audit.

In order to digitise records, investment in technology will be required. There are multiple suppliers out there who can help with spend analytics, organising and categorising data, and forecasting. Their powerful systems can consume structured and unstructured data in various forms, ranging from voice files to text and graphics, using it to provide valuable insights.

With new systems in place, organisations might attempt spend and supplier analysis across all areas of expenditure, and then subject certain categories to a deeper analysis. With clean, complete information they can learn everything about the category they’re buying in, suppliers they’re buying from and the stakeholder or department they’re buying for. This will allow them to consolidate spend and contracts, and be more efficient.

What is vital is that organisations keep it as simple as possible. This requires ongoing maintenance to ensure spend data is clean, and a concerted effort to communicate to stakeholders what is changing and how it will help. “People are like water,” says Anderson. “They will follow the easiest path to get the job done.”

There are plenty of niche players providing specific tools that can help with everything from work flow approvals and ordering supplies, to licensing and lifecycle management. “Take the iPhone as an example,” says Anderson. “It used to have four apps, and now there are about two million.” The knack will be to ensure all the information gleaned by individual tools can be fed into one place to be analysed.

He believes the ability to capture and interrogate this data will totally overhaul the future of supply chain management. “It’s going to change so radically in the next 10 years, it’s going to make the last 50 years look like kindergarten work. The data we’re going to be able to capture over the next decade will be phenomenal. And it’s going to make the world a more efficient place.”

Top tips

Be clear on what problem it is you’re trying to solve or what outcome you wish to achieve.

Understand the size of the opportunity – or the potential risk of not acting – in order to form a business case for any required change or investment.

Tackle processes first, followed by considerations of people and technology.

Digitise manual processes.

Decide on the benefit of cleaning up historic data or concentrating on only new information.

Don’t expect to do everything internally. It’s likely you will need external systems to help.

Assess the offering of potential platforms, tools and suppliers to decide which solutions will best suit your requirements.

Make any systems or processes you introduce simple to follow. Ensure users are clear about the benefits.

Make suppliers part of the solution. Consider making them responsible for ensuring their profiles and spend categories are up to date and accurate.

Consider adding layers of artificial intelligence or machine learning to make your business more sophisticated, efficient and better at forecasting and reducing risk.