Monday, 9 May 2011

Impact and the funding of research

Like many others, I have watched in dismay, and have signed every petition I can find, as some of those who hold the purse-strings in humanities research have accepted that politically trendy mantras have a role in the allocation of funds. The Arts and Humanities Research Council has adopted "the Big Society", a silly slogan if ever there was one. And the management of the British Academy have negotiated a settlement on the importance of the impact of research, rather than telling politicians and civil servants that they will have nothing to do with the concept. The latest, and excellent, petition on the AHRC is here, and I encourage everyone to sign it:

Today, I came across one of the best, and scariest, comments on impact, in a letter in the 28 April edition of the London Review of Books. It is by Richard Bowring, Professor of Japanese and Master of Selwyn College, Cambridge, the college where I had the privilege of studying in the 1970s. It is available here:

He points out that while everyone knows that the concern with impact is nonsense, people will care about their institutions enough to go along with it, and will put something down on the forms in order to secure some funds. The ultimate beneficiaries will be institutions in Europe and in North America, which will welcome the really talented people from our universities with open arms. To put one of Richard Bowring's points in game-theoretic terms, our universities are in a multi-player version of the prisoner's dilemma (except that communication between players is possible). All would be better off if all resisted, but the cost of resisting when others do not is so high that all will capitulate.

And yet, some sort of control is needed. We cannot just hand over taxpayers' money, without some check that it is not being wasted. Fatuous research does get conducted, and the taxpayer should fund as little of it as feasible. It is a matter of finding the right control.

The perfect control from the point of view of saving money might appear to be one that imposed external monitoring on every research proposal. Indeed, one reason to be concerned about impact is that it would encourage precisely that approach. It is not the only supposed measurable that could do so, nor would it have to be used in that fashion. But impact has the special characteristic that it is not a quality that is naturally defined within the context of an academic discipline, unlike, for example, profundity or being wide-ranging. So the external authorities who have imposed the criterion are the natural people to say what it means, and to decide which pieces of research are likely to have it.

In fact, external monitoring of every research proposal would be a bad idea, even if the external authority was often within the institution, as when a general board monitors the proposals that a faculty board approves. Universal surveillance cramps creativity and twists research proposals. But if we are not to have universal surveillance, we must accept that some fatuous research will get funded. That is undesirable in itself, but it is a reasonable price to pay. A certain amount of money may need to be wasted, as a side-effect of properly funding worthwhile research. But we want to make sure that not much is wasted. What sort of control might do? Here are two suggestions.

The first suggestion is to monitor the achievements of our universities against those in other countries. (One reason to make comparisons with foreign universities, rather than with other domestic universities, is that it takes away a sense of competing with others for the same pot of funds, a sense that can all too easily lead to game-playing, and that can also encourage enthusiastic politicians and bureaucrats to devise games, the playing of which will increase their influence over what universities do. The assessment of impact is a game like that, whether or not that was the intention.) There is something wrong if a university does not attract a reasonable amount of private funding, or does not attract people to its conferences in the way that others do, or does not find that its professors are occasionally lured away to prestigious institutions elsewhere, or does not produce enough widely-cited publications. (It would not matter that work can take years to get recognized and widely cited, because in any one year, publications from earlier years should come to maturity and start to get cited.) These measures are chosen at random, but the principle is not. Universities that did badly in international comparisons would lose research money, while those that did well would gain it. Each university would then use its money to fund the research that it chose. It would be helpful to apply the test to each university as a whole. Then each university could develop the departments that it saw as most worth developing. The same scheme could however be applied to faculties separately.

The second suggestion is to put the decision in the hands of the faculties in each discipline. Make a central decision on how much is to be spent on philosophy, how much on history, and so on, across the country. Then ask each faculty member in each university to divide the pot among all university departments in his or her own discipline, excluding his or her own university, first supplying each person with information on the sizes of the departments. Then average the results for each discipline, possibly adjusting in some pre-defined way for the effects of more votes coming from some universities than from others, and leave each faculty to decide what research to fund with the money that it has been allocated. The point is that there should be enough collective wisdom across the country to ensure that departments that did good work, and that picked worthwhile projects to fund, would be favoured. A certain amount of discussion between people before voting would lead to some lack of independence of decision-making, so we would not get the full advantage of the wisdom of crowds. And the process would be at least slightly corrupted by mutual back-scratching. But it should not be too bad.