Starting On A Data Transformation Journey

Digital or data transformation is a significant undertaking for any organisation. To go from an almost reluctant reliance on data (where managing and understanding data across the enterprise is viewed as necessary evil) to treating data as a primary, vitally important asset, is a transformative leap. To do so, organisations need to embrace and understand their data. And to do so on an enterprise-wide scale.

For a lot of organisations, this is a substantial challenge. The bigger or more complex the organisation, the more of a challenge this is.The hurdles to overcome are multiple:

mindset

strategy

governance

security

analysis

Mindset

For a lot of organisations, terms such as data governance and information management are stigmatised in the mind of the average employee. All too often projects are set up to tackle such subject areas. These are typically driven by initiatives such as management-driven accreditation (e.g. ISO 27001) or legislative change (such as GDPR).

Mindsets must change from managing, governing and securing your data being something “forced upon employees” and one-off initiatives to being an on-going, continual part of organisational culture. For that to happen, change must be driven from the top of the organisation. A good start is for all employees to understand this is for long term benefit of the organisation, working towards building a competitive advantage.

Strategy

It would be fair to say that the evolution to being active information asset owners has happened by accident for the majority of businesses, rather than by design. In all likelihood, there has been no data management strategy in place to manage the data and systems owned by organisations.

Without a data management strategy (and an aligned architecture reference model) system silos appear and isolated pockets of business knowledge and intelligence grow. Throw in typical pressures such as organisational mergers and competing priorities across the organisation and these problems are exacerbated and multiplied as you encounter multiple overlapping systems managing what is effectively duplicate data.

A data management strategy will provide guidance on how to surface and capture business knowledge, break down silos and provide a collective understanding of the data across the enterprises.

It also allows conscious and holistic decision making when introducing, replacing or integrating application systems.

Governance

If you have master data distributed across your organisation, you have to address the challenge of consistency to ensure your master data is being created, validated, updated and deleted by the correct members of your organisation, at the correct time and in the correct situation. Failure to do so, will mean inconsistency in your data sets. Inconsistent data is very expensive to correct, and can mean your staff and customers lose confidence in your systems and the empirical data that you are gathering. To make matters worse, they won’t outright tell you they are losing confidence in your data, they’ll just stop using it or using you as a supplier.

You can put apply all the security measures available, but if your staff and/or customers don’t trust the data, its not worth securing in the first place.

You can apply the most sophisticated algorithms available, but again if your staff and/or customers don’t trust the data, building and testing the algorithms have been an expensive waste of time.

Therefore, data governance is of primary importance when embarking on this journey and should be considered a primary step.

Security

Once you have won the hearts and minds of your organisation to understand the importance of your data, and you have proven how reliable it is through data governance, it starts becoming a valuable asset. Like any valuable asset within your organisation you have to protect it. Once you have started building a competitive advantageyou want to maintain it.

Security can be split into 3 distinct pillars:

Confidentiality – secure from external parties such as competitors, secure internally to only people with appropriate levels of authorisation

Integrity – secure that the data is correct. Incorrect data could be due to being deliberately sabotaged (hackers), lack of data validation in systems and processes or lack of data governance

Availability – Simply put, availability is ensuring that your data is available to whoever needs it, regardless of the circumstances. That means you must provide appropriate backup and failover strategies to ensure data is not lost, and is durable to hardware failure. For example, if your business relies on that data being available 24/7 then you must invest in appropriate failover and clustering. Alternatively, if it’s vitally important that the data is available in a timely manner, then you must invest in performance tuning your applications and systems.

Note, while CIA pillars (Confidentiality, Integrity & Availability) are commonplace in security parlance, there is a massive crossover with data governance, where a lot of IT professionals consider integrity and availability in particular to be classed as Governance topics. Personally, it doesn’t matter which classification they are given, providing they are met during this journey.

Analysis

Once you have secured, trusted data and your organisation is focused on using this data, you must transform this data into information. For me, information is the intelligent, analysed data that allows you to make decisions. The data that has been gathered, maintained, aggregated, shared, etc is the raw data which is distinct.

Business intelligence (such as Key Performance Indicators, trends, delta patterns, etc) are the domain of reporting and BI tools. While the data is distinct from the information, so too should the raw data be segregated from the reporting and analytical data (but of course one is extracted, copied, transformed and loaded into the other).