Value and measurement, evidence and experience

Beginning with a discussion of why and when Government intervenes in providing services or in trying to influence behaviour, Jeremy highlighted the need to provide some services for the 'public good'.

Jeremy then looked at the two main approaches available to voluntary and public sector organisations in measuring their value - using market data and willingness to pay/accept techniques. He suggested that whilst these can provide an evidence base many experiences are not open to evidence or evaluation e.g. the benefits of IT projects are often difficult to quantify and one person's benefit may have a negative impact on others.

After looking at a range of projects and value studies Jeremy concluded that it was incumbent on organisations to measure their value where possible but that cost/benefit studies could not entirely supplant the role of judgement in deciding how funding should be allocated.

The presentation slides are available for you to download in PDF format.

Measuring the economic impact of the British Library

Recognising the shortcomings in many of the traditional methods used by libraries to measure performance, in 2003 the British Library commissioned a ground-breaking study to measure its value to the UK economy.

The study - using Contingent Evaluation methodology - was conducted by Spectrum Strategy Consultants and Indepen with NOP undertaking the market research. Data for the study was provided by a series of interviews with over 2,000 Library users and members of the general public, who were asked a range of hypothetical questions, including:

How much would you be willing to pay for the Library's continued existence?

How much would you be prepared to sell your reader pass for, assuming you could not then replace it?

How much do you invest, in terms of time and money, to make use of the Library?

The results demonstrate that the Library generates value of around 4.4 times the level of its current annual public funding of £83m and show that the Library offers substantial indirect benefits to UK society as well as direct benefits to the people who use its services.

The presentation slides, questionnaires used in the study and a short overview of the study are available for you to download in PDF format.

Measuring success at the BBC

Speaker:James Thickett, Controller - Business Strategy, BBC

Following the British Library's groundbreaking economic impact research, the BBC commissioned its own Contingent Evaluation study between December 2003 and May 2004. Human Capital was commissioned to design the study, and Martin Hamblin GFK to undertake the market research. Altogether, 19 pilot studies were undertaken to arrive at a list of questions likely to lead to meaningful responses and over 3,000 interviews conducted to ascertain answers to questions such as:

How much would people be prepared to pay for the BBC - as a subscription service, or to avoid it closing down?

How much do people value different BBC services?

Which types of programme do people value most?

The study found that people place a rough value on the BBC which is double the amount they are currently paying through the licence fee. Indeed, people value the BBC more than this when they consider its role as a public organisation serving everyone. The aggregated value people attach to individual BBC services significantly exceeds the value they attach to the organisation as a whole.

The presentation slides are available for you to download in PDF format.

Measuring value: museums and galleries

Speaker:Dr Tony Travers, Director, Greater London Group, London School of Economics

The National Museum Directors' Conference (NMDC) commissioned Tony Travers and Stephen Glaister of Imperial College to produce a report on the economic, educational and broader cultural impact of the UK's national museums and galleries.

To arrive at this figure, Tony described how the study looked at the structures and activities of the NMDC institutions - examining their spending and the stimulus they provided to the economy - in order to assess the organisations' economic, educational and broader cultural impact.

To arrive at data on turnover and other impacts, information was collected using a questionnaire (looking at turnover, visitor numbers, school visits, new facilities, etc.) and through meetings with senior administrators.

Discussing the results of this 'outputs-based' study - which showed that the sector added £2 billion to the UK economy in 2003-04 - Tony emphasised the need to develop a common approach to measurement across museums and galleries.

The presentation slides are available for you to download in PDF format.The Valuing Museums report is available on the National Museum Directors' Conference (NMDC) website.

Thinking beyond economic impact

Speaker:Magnus von Wistinghausen, Senior Consultant, AEA Consulting

In a lively paper, putting the case against measurement, Magnus suggested that 1980s Thatcherism had unravelled the post-war consensus that culture should be fully funded - leading to the current quest for new valuation techniques.

He suggested that an unhealthy focus on measurable activities at the expense of, the more difficult to measure, core roles - was leading to an oversized but under-resourced sector in danger of neglecting its fundamental work.

The presentation slides are available for you to download in PDF format.