How do you measure change success?

Change success – it’s a thorny topic. So one of my most “popular” posts is the “70% change fails: Bollocks” – in it I argue that change is way too complex to measure by simple binary “did it succeed? – yes / no at a single point in time.

So this post follows that one with hopefully a more thoughtful exploration of what change success metrics we should use. When I talk to leaders and steering committees about change success I use three categories to understand change which can lumped in to camps – installation, benefits realisation and process of change.

Measuring the success of the “installation” of a change can occur pretty shortly after “go-live”, whereas the other two can occur prior to go live and at periodic intervals post go-live. But a consideration first of project success.

Project Success

Most of the change projects we work are deemed successful by Project Success measure (IFOTOB). But this in of itself is not sufficient, I think we can all think of projects where it looked good from a delivery perspective but user adoption was low and the benefits limited. From a change management perspective I tend to be interested in this but not ruled by it. Ultimately change in scope, time delays and cost blow outs create a lot more stakeholder engagement and expectation management. You want to contribute to a strong result here, but it really does not measure success of the change.

User Adoption

User Adoption tends to be used in technology / systems implementation, you can also consider employee compliance in the same bucket. Does anything actually change? Are people using the new system, are they behaving in a new way? Are they using new processes? Successful installation is dependent on something actually changing!

I know this sounds obvious – but it’s a real oversight in the change success discussions. Often the discussions focus on benefits realisation – and if the benefits are not achieved it is argued that the change management team did not do a good enough job and need to improve their practice! But the reasons behind low benefits realisation are many and varied – for example, the change team may have got great installation results, but the original logic for the change was flawed.

It’s this category that traditional change readiness surveys / polls come in handy. Being able to poll for “will they [make the change] and can they [make the change] pre-go live puts you in a much stronger place at time of installation. You can target your efforts with remediation or intervention to ensure a good user adoption / employee compliance measurement.

Ultimately you are looking for metrics on log ins, new processes being followed, calls to support structures decreasing (or increasing). Common sense tells you 100% adoption on day one is unlikely, but with some careful thought you can identify a baseline metric for current state and expected usage or compliance over time.

Benefits Realisation.

It is a rare change initiative that you can measure benefits realisation immediately after go live. If you can, then you were working on something that was pretty broken to start with. Benefits often take some time to start to become apparent. This can be because it takes time for new habits to form with behavioural change, proficiency to increase with systems usage, or business cycles to play out.

The time of things you are looking to measure here are tangible measures – speed to market, cost of transaction, cycle time, FTE release, speed of processes, employee engagement increase.

Change process success

Measuring how effective the change process was is critical. Nothing kills a future change faster than the legacy of poorly executed change. Some of the things that we can consider here along the way are periodic assessments of where people are at on the change curve (awareness, understanding, buy-in, commitment). Do the people feel empowered? And then the one that really makes a difference is the “campsite rule” – so for Australian readers and children of the 70/80’s, do you remember Harry Butler – In the Wild?, the wildlife conservationist? He always implored us to leave the campsite in a better state than when you got there. The same goes for change management. Really successful change management means that you have built change capability and considered sustainable change. Your organisation has a higher state of change maturity than when you got there.

Personally, I don’t think it matters as much what the metrics are that you decide on, as much as that you have this conversation with those that matter. It can be incredibly powerful to have a rich discussion on what does success look like with organisational leaders, steering committee and project teams to align expectations and education about change management. Only then when we start to move away from the puffery of “70% of change projects fail” and start to have conversations that yield better results of change. What about you? What do you use to measure successful change?

3 Comments

Great post Jen! We were just talking about this today in the office. Too many people fall into the trap of delivering to scope, time and budget (the iron triangle). But we know you can do all that and still fail. The Sydney Opera House failed on all three but is still considered an incredible achievement.
I measure change on the feedback of my most important stakeholders. If they are happy (and they are kept happy because their expectations are managed throughout the change) then the change is successful. If they are not happy (regardless of what is delivered) the change is unsuccessful. I guess the secret sauce is realising who your most important stakeholders are.
I also think quality and benefits realisation play a major role in measuring success. Thanks for sharing, it’s a great topic.