Howy Jacobs

For Good Measure

The Finnish universities arrange periodic assessments of their research, in line with public policy. Each such assessment is conducted separately, applying criteria deemed appropriate to local circumstances. This method is guaranteed to make the system as meaningless as it is costly: it is more a PR exercise than an objective evaluation. Virtually nobody has had the courage to make major redistribution of resources based on the findings, with low scores even considered by some as justification for continued support. Indeed, some universities have explicitly stated, in advance of such assessments, that the outcome will be for advice only, and will not influence funding allocations to different faculties, departments or research groups. Others have awarded derisory amounts of bonus funding to units deemed worthy, which hardly even match the cost of the exercise. In a word, the whole process appears to be rigged.

I believe that that this system now needs a complete overhaul, for a host of reasons. Science and scholarship are now global, and research findings are similarly applicable everywhere. In its straitened national circumstances, Finland cannot afford to pour money down the drain on uncompetitive projects that produce no benefit to itself or to the global academic community. On the other hand, as a ‘knowledge economy’, it desperately needs to capitalize on its own original research findings, so as to generate tomorrow’s prosperity. Almost everyone realises that our higher education system is expensive and inefficient, but it will not improve until there is much clearer hypothecation of funding for teaching and research. And major research infrastructures, which are increasingly expensive, must be deployed where they can be most effective, not used as a tool of regional policy.

Therefore, I believe that a meaningful assessment of research in Finnish universities must be conducted at national level (or above), applying uniform criteria and procedures to all. The units of assessment, evaluation principles and benchmarks should be based on a widely accepted international standard, such as the UK’s REF process. The rules should be comprehensive and transparent. The exercise should be steered and implemented by international adjudicators and should involve globally recognized academic leaders in each field. Scrupulously adhering to the principles of DORA (the San Francisco Declaration on Research Assessment), it must focus on the real content and impact of research, not on crude bibliometrics.

The outcome should comprise not only a national rank order in each discipline, but a comparison of both the top and bottom institutions on these national lists with their international counterparts. Units should receive not only a ranking, but detailed feedback on the overall quality and impact of their research, including guidance on what improvements could be made. Finally, the state’s financial support for research in each unit of assessment should be based to a very large extent on the outcome. Research in those units that fall well below international standards should be closed down, whilst those at the top end should see their funding enhanced considerably. Middle-ranking units should have their future funding tied to the implementation of specified improvements.