Messi or Ronaldo? Federer or Nadal? Some debates are pointless but fun.

This article asks the (slightly geekier) question: “which is more important, measurement or analysis?” Specifically, I’ll look at data in the context of conversion optimisation – the process of persuading more website visitors to buy – to explore the relationship between measurement and analysis.

Chicken or Egg?

The well-known framework for business operations, Lean Six Sigma model, follows a sequential workflow:

Figure 1: Lean Six Sigma Model

Obviously, measurement is a precursor to analysis. But is it so straightforward? When analysing my clients’ websites, I’ll often spot an issue that needs more investigation or additional evidence to triangulate and solidify what is the actual problem. It’s a messy, non-linear process. I’m analysing data and spot knowledge gaps that I want to fill. Many times I have started analysis and found something new that required me to go back and fetch some additional data.

Whilst the Lean Six Sigma is conceptually appealing, true optimizers have a structured framework rather than a sequential process (note: the Lean Six Sigma model is much more robust than I’ve suggested, I am just using this specific visualisation to make a point!). It can look more chaotic:

This circular nature of measurement and analysis is recognised in human-centred design principles too. Human-centred design is an iterative process that repeats steps until user requirements have been adequately met.

Figure 2: the interdependence of human-centred design activities according to ISO 9241-210 standards.

Introducing the Parallel Research Process

I have made a career of conversion optimisation precisely because that is where creative and analytical thinking is needed the most. On a website, you need to persuade visitors to take action; at the same time, you must use data to work out the best way to do this.

The most important aspect of a website optimisation programme is data in both it’s forms: measurement and analysis. Why? Because the research phase informs everything that follows thereafter. Without robust heuristic analysis, session replay analysis, web analytics analysis, technical analysis, qualitative surveys, user testing, etc. you won’t get good insights.

The symbiotic relationship of measurement and analysis has led me to develop a Parallel Research Process that distinguishes between two different types of research: foundational and directional.

The Parallel Research Process explained:

Collect foundational research. This could be a combination of Google Analytics (quantitative data) and voice of customer (qualitative data).

Develop hypothesis based on the data.

Determine the strength of the hypothesis. Is it really data-driven or is more of a guess? If the latter…

… go back and collect more (directional research)

Finalise hypotheses and validate them (e.g. A/B test)

The takeaway

High-growth companies are using data to make profitable business decisions. These days, the term ‘data-driven’ is ubiquitous but it’s critical to understand the relationship between measurement and analysis – and how they feed each other.