Thinking about ‘impact’ and the significance of a good mission statement

Measuring impact is the only way an organization can know whether its efforts and use of resources (often other people’s money) is doing any good. Logically, organizations that measure impact perform better and evolve faster. Furthermore, explorations around the theme of measuring impact invariably lead to continuous improvement in effectiveness and efficiency within organizations that engage in such explorations.

At the highest level of description, ‘impact’ can be defined as an improvement brought about by an intervention.

Here are five steps aimed at determining the impact and calculating the return on investment of an organization’s operations:

You can’t think about impact until you know what you’re setting out accomplish, and most mission statements offer little value in this regard. To counteract this, re-formulate (or formulate!) your mission in a phrase of 8 words or less that includes:

(1) a target population,

(2) a verb, and

(3) an ultimate outcome that identifies something to measure.

Here are some examples:

Providing households with best-value internet access

Improving the life-chances of young people

Expanding older people’s quality-of-life choices

Providing people in [location] with quality affordable homes

Generating for shareholders best-in-class sustainable returns

Facilitating public sector employees in accessing leadership development

If you can’t establish this kind of concise statement, there’s little point in going any further, either because you don’t really know what you’re trying to do or because you simply wouldn’t know if you were doing it!

2. Pick the right indicator(s)

Work out the single best indicator that would demonstrate mission accomplished. Here’s some examples relating to the missions shown above:

Increased number of households accessing the internet

Increased numbers of young people achieving a sustainable start in their working lives

Greater variety of retirement solutions being accessed

Growing base of stable tenancies

Top quartile returns on invested capital

Greater take-up of leadership development activities

Sometimes it is possible to realise a single best indicator – and this is ideal. Other times you might need to capture the outcome through a carefully chosen, and minimal, combination of indicators. The clear aim here is to be able to answer, at appropriate review points, the critical question “Are we fulfilling our mission?”

3. Get real numbers

The aim of this step is to evidence a change that has arisen from you operations. This means being able to show a change from an established baseline, having made the appropriate measurements in the appropriate way. Essentially, measuring the right thing and measuring it well.

4. Make the case for attribution

If you have real numbers that show impact, you need to make the case that it was your efforts that caused the change. This is the hardest part of measuring impact, because it asks you to be able to say what would have happened without you. When real numbers show there has been a change, a useful thing to ask is “what else could possibly explain the impact we observed?”

There are three levels – in ascending order of cost and complexity – of demonstrating attribution:

Narrative attribution: You’ve got before-and-after data showing a change and a credible story that shows that it is very unlikely that the change was from something else. This approach is vastly overused, but it can be valid when the change is big, tightly coupled with the intervention, involves few variables (factors that might have influenced the change), and you’ve got a deep knowledge of the setting.

Matched controls: At the outset of your work, you identified settings or populations similar enough to ones you work with to serve as valid comparisons. This works when there aren’t too many other variables, you can find good matches, and you can watch the process closely enough to know that significant unforeseen factors didn’t arise during the intervention period. This is rarely perfect; it’s often good enough.

Randomized controlled trials: RCT’s are the gold standard in most cases and are needed when the stakes are high and there are too many variables to be able to confidently say that your comparison groups are similar enough to show attribution.

5. Calculate return on investment

Once you’ve established your operations are having a real and measurable impact, you need to know what it cost. You can always generate impact by spending very large sums of money, but this is unlikely to be sustainable, scaleable or ‘fair’ on investors/funders.

The easiest and arguably most valid way to calculate return on investment is to divide the total money spent by the total impact. In organizations that do more than one kind of project, it is often possible to split out what they spent for their various impacts. Remember that start-ups are expensive so bear this in mind in the early stages of an operation, but do see if their projections for steady-state operations make sense and assume (as experience will tell) that such projections are usually at the optimistic end of the scale!

In the end, though, the key to evaluating real impact is an honest, curious, and constructive skepticism. A healthy dose of scepticism (not cynicism) is essential.