Assessing Research Impact

By

Australia's federal government will embark on the gargantuan task of measuring benefits flowing from publicly funded research and plans to use those findings to allocate a proportion of the $8.2 billion spent on grant funding each year.

The announcement by the nation's tertiary education minister, Chris Evans, is a compete about-face from current policy, which measures the quality of research, but not its impact.

The task will be to measure the social, economic and environmental benefits flowing from government-funded research in universities, the Commonwealth Scientific and Industrial Research Organization, and medical research institutes.

Evans told a research symposium in Canberra last week that "it is important that we can show this research is providing tangible benefits to the nation and all parties are getting value for money."

He said that by September 2013, he expected that "research impact measures" aligned to funding streams would be finalized.

The government's policy has been to measure quality but not impact after strident opposition from research intensive universities, which felt it would undermine their predominantly blue skies research endeavors. Such research has long time frames and no guarantee of success, but is the basic building block of all new knowledge, research university officials argued.

The government's quality audit, Excellence for Research in Australia, simplified an earlier model crafted by the government of the last prime minister, John Howard, and ditched the impact factor because it was too complex and expensive.

The government's backtrack was announced at a symposium to discuss the results of a self-funded trial of research impact by 12 universities. The trial found that 87 percent of the case studies reviewed were deemed to show that research had a "considerable" or better impact. (See related article.)

Peter Høj, vice chancellor of the University of Queensland, heralded the news as a "major step forward." He also welcomed Evans's assertion that said any measure of research impact would have a very strong business focus.

Professor Høj said research-intensive universities such as his would be more likely to accept the proposed impact factor because the government had promised to keep it separate from the audit of quality, thereby avoiding confusion between the two.

Stuart Cunningham, a distinguished professor of creative industries at Queensland, said there is often no correlation between quality and impact. Astronomy and astrophysics, the research areas of Australia's latest Nobel laureate, Brian Schmidt, have no impact at all.

He said assessment of certain arts and humanities areas were also extremely difficult to assess, and would need to be very nuanced.

For instance, the impact of the cervical cancer vaccine Gardisil was easy to measure, but the value of an Aboriginal novelist writing stories and myths for local indigenous communities about their heritage and culture was much harder to ascertain.