Signals from Institutions: The University of Edinburgh’s Strategic Goals, Targets and KPIs

The University of Edinburgh Strategic Plan 2012-2016

As described in a paper on What Next for Libraries? Making Sense of the Future the JISC Observatory “provides horizon-scanning of technological developments which may be of relevance to the UK’s higher and further education sectors“. The paper, available in MS Word and PDF formats, describes the systematic processes for the scanning, sense-making and synthesis activities to support this work. The paper focuses on the processes for observing technical developments. However there is also a need to observe signals of institutional interests in IT developments, especially in light of the recent announcement of Jisc’s objective to “address a number of specific priorities for universities and colleges through the development of resources, tools and supported infrastructure“.

Strategic plans published by institutions can provide a valuable starting point to help identifying areas of institutional interests. For example, Lorcan Dempsey recently drew attention to the strategic goals which have been identified by the University of Edinburgh:

The document, The University of Edinburgh Strategic Plan 2012-2016, (which is available in PDF format) is interesting not so much for the way it identifies strategic goals and the key enablers who will be needed to ensure the goals are attained, but the list of specific KPIs (Key Performance Indicators) and the associated targets.

Of particular interest is the strategic goal of excellence in research for which the KPI is listed as “Russell Group market share of research income (spend)“. The corresponding targets are:

Increase our average number of PhD students per member of academic staff to at least 2.5

Increase our score (relative to the highest scoring institution) for the citations-based measure in the THE World University Rankings to at least 94/100

The strategic goal of excellence in innovation states that the KPIs are “Knowledge exchange metrics: number of disclosures, patents, licences and new company formation“. The targets for this goal are:

Achieve at least 200 public policy impacts per annum

Increase our economic impact, measured by GVA, by at least 8%

The Importance of Metrics

It is interesting to see how the University of Edinburgh has clearly targets which are based on measurable criteria: “Increase our average number of PhD students per member of academic staff to at least 2.5“; “Increase our score … for the citations-based measure in the THE World university rankings to at least 94/100“; “Achieve at least 200 public policy impacts per annum“; “Increase our economic impact, measured by GVA, by at least 8%“; “Increase the proportion of our building condition at grades A and B on a year-on-year basis, aiming for at least 90% by 2020“; “Increase our total income per staff FTE year-on-year, aiming for an increase of at least 10% in real terms“; “Increase the level of overall satisfaction expressed in responses to the NSS, PTES and PRES student surveys to at least 88%“; “Increase the number of our students who have achieved the Edinburgh Award to at least 500“; “Create at least 800 new opportunities for our students to gain an international experience as part of their Edinburgh degree“; “Increase our headcount of non-EU international students by at least 2,000“; “Increase our research grant income from EU and other overseas sources so that we enter the Russell Group upper quartile“; “Increase our number of masters students on programmes established through our Global Academies by at least 500“; “reduce absolute CO2 emissions by 29% by 2020, against a 2007 baseline (interim target of 20% savings by 2015)” and “Increase our number of PhD students on programmes jointly awarded with international partners by at least 50%” (emphasis added).

Prior to CETI’s work in this area the importance of metrics had been identified by the JISC in 2010 when they asked UKOLN to facilitate the Evidence, Impact, Metrics activity. A series of reports on this work were published just over a year ago. As described in the document on Why the Need for this Work?:

There is a need for publicly-funded organisations, such as higher education institutions, to provide evidence of the value of the services they provide. Such accountability has always been required, but at a time of economic concerns the need to gather, analyse and publicise evidence of such value is even more pressing.

Unlike commercial organisations it is not normally possible to make use of financial evidence (e.g. profits, turnover, etc) in public sector organisations. There is therefore a need to develop other approaches which can support evidence-based accounts of the value of our services.

Which should I bother with metrics?Metrics can provide quantitative evidence of the value of aspects of project work. Metrics which indicate the success of a project can be useful in promoting the value of the work. Metrics can also be useful in helping to identify failures and limitations which may help to inform decisions on continued work in the area addressed by the metrics.

What are the benefits for funders?In addition to providing supporting evidence of the benefits of successful projects funders can also benefit by obtaining quantitative evidence from a range of projects which can be used to help identify emerging patterns of usage.

What are the benefits for projects?Metrics can inform project development work by helping to identify deviations from expected behaviours of usage patterns and inform decision-making processes.

What are the risks in using metrics?Metrics only give a partial understand and need to be interpreted careful. Metrics could lead to the publication of league tables, with risks that projects seek to maximise their metrics rather than treating metrics as a proxy indicator of value.

It will be interesting to see if other institutions emulate the University of Edinburgh in stating specific targets for their institutional strategic plans – and how pressures on staff within the institutions to achieve the targets affects operational practices.

Is anyone aware of other institutions which are taking similar approaches?