What is the REF? The REF (Research Excellence Framework) is the assessment exercise that took place in 2014, taking over for the RAE (Research Assessment Exercise of 2008). It is a national research evaluation that affects all research universities and some arts colleges.

Why does it take place? All universities are public in the UK, so research funding generally comes from government-funded agencies. The REF is therefore, indirectly, the government’s accountability exercise to assess the quality and quantity of research being subsidized with public money. (NB with the rise in student fees in 2012, the funding structure has changed, so the REF will likely change as well for 2020).

Who participates? Most departments across the country send in a submission, and most members of those departments contribute to the submission. People have to be “selected,” but everyone is expected to participate if they are “permanent” (the equivalent of TT/tenured). To sit out the process would imply not being an active researcher, and this would have salary ramifications.

What does it entail? Individuals send their “output” (publications) to their faculty research office, and the material goes out for assessment – it gets reviewed internally, and then externally by an appointed “critical friend” of the department. Much of the material has already been through peer review, so the re-assessment, sometimes by non-specialists, can be troublesome for people who do non-traditional work. The university decides which individuals “get to” participate, and individual output goes into the departmental portfolio. Then the department is assessed as a whole. The process takes 2-3 years.

How does the assessment work? Specialists (usually at the rank of full professor) from every discipline are assigned to REF sub-panels, and they are relieved of their departmental duties for a year or so while they read the complete submission for their sub-discipline (this task is prestigious but thankless), and each item (article, chapter, book) is given a ranking: 4* (world-leading), 3* (internationally excellent), 2* (nationally excellent/internationally recognized), 1* (nationally recognized), “Unclassified.” The complete departmental portfolio is then tabulated, and the department is assigned a collective ranking for output on the basis of how many 4* outputs it has, along with other criteria like “impact” (public engagement) and “environment” (facilities and resources). Departments that rank highly get a pat on the back from admin, while departments that “do poorly” are threatened with salary cuts and closure.

Do adjuncts participate? Usually not, although any candidate who has a book has leverage to upgrade their contract. Many part-timers participate, though, such as people who leave mid-year, or full professors approaching retirement who drop down to part-time.

How much material gets assessed? In 2014, active researchers were asked to submit four pieces.

That must be a mistake – only four items? Right, a newly-minted PhD working hard to build a research profile might not think that four pieces in six years is much to produce. But this applies to people at all levels, and a Professor has to put out the same amount as a Lecturer (assistant professor). It doesn’t seem like much to someone one the tenure-track, but more senior researchers might see keeping up with the cycle as a lot of work. And for people working on a book, the need to put out articles while trying to finish a manuscript can mean not finishing the book.

How do you put together a submission? Ultimately, what confuses most people is how to weigh quality against quantity. At the moment, every document counts as one item, meaning that a single-authored monograph counts the same (quantitatively) as a chapter in an edited collection. But the former would generally earn a higher rating than the latter. So status publications remain the best route to a 4* ranking – almost anyone with a single-authored monograph and three journal articles will earn top marks. It’s harder to earn 4* without a book, but it can be done by publishing in top-status journals. This means that people who do multi-media, outreach, or anything non-traditional will sometimes be penalyzed for not focusing on traditional text. On the other hand, the system of weighting allows people to game the system if they know how it works. Edited collections and conference proceedings are often used as filler to round out a submission, especially for people who have already established their reputation. This is why some British academics have dozens and dozens of publications – they write as much as they can, and leave it to the research office to sort out which pieces are worth submitting. From Karen’s “Don’t get your career at Costco” perspective, this is an inefficient use of research time.

Whom does this system favour? It depends on who you ask, but the data indicates that the system favours men in their 50s and 60s. There is research about gender imbalances – in most universities, the REF submissions slant heavily male, regardless of departmental ratios, and there is low representation of people in their 30s (and virtually nobody in their 20s, at least in the humanities). The overseers are working to fix this imbalance, but slowly and largely ineffectively: For example, an Early Career Researcher (ECR – someone who finished within the past six to ten years, depending on the metric) is entitled to a reduced submission – 1 publication if they filed in 2011, 2 if they filed in 2010, and so forth. This appears to be a benefit. However, hiring committees in 2012 and 2013 were single-mindedly focused on making hires that would raise, rather than meet their departmental rating, and an ECR with two good articles wouldn’t make much of an impact statistically. So in the years leading up to the REF, departments usually hired someone with a book, leading to short-sighted hiring practices that favour a candidate with a book in-hand, and occasionally not much else, over a candidate with a lot of demonstrated productivity and potential. Likewise, departments keep senior faculty on well into their 70s if they are still publishing books. To some ECRs, this appears to be a transfer of privilege upwards.

Does anyone like the REF? The REF is stressful and time-consuming, it strains relations with administration, it causes bullying within departments, and it’s subject to the same critiques of neoliberalism and corporatization as the job market. One thing I can say on its behalf, though, is that it forces senior/experienced colleagues to remain in active research. I have some colleagues back in the US who complain about the full professors who stopped publishing long ago and are coasting while those on the TT work like mad to get their work done. I don’t find that to be the case, at least in my department – the same expectations apply to everyone, or, everyone who’s already an insider…

If I’m looking to work in the UK, how should I plan for the REF now? As an ECR, I use Karen’s five-year plan as a good template. The same things that make a successful tenure portfolio make a good REF submission, so publishing one article in a top journal per year, plus finishing your book, will leave you in a good position.

About Karen Kelsky

I am a former tenured professor at two institutions--University of Oregon and University of Illinois at Urbana Champaign. I have trained numerous Ph.D. students, now gainfully employed in academia, and handled a number of successful tenure cases as Department Head. I've created this business, The Professor Is In, to guide graduate students and junior faculty through grad school, the job search, and tenure. I am the advisor they should already have, but probably don't.

Comments

What is the REF? A Primer on UK Academia (Guest Post) — 3 Comments

Impact! A lot of the jobs advertised which I am looking at expect you to demonstrate impactfulness- this seems to be the holy grail at the moment in UK academia. As well as that, public engagement impact is very valuable to demonstrate to funders and employers, I am more naive than the OP but I reckon this is linked to REF.

For example I recently got a research asst. job and the interview required a 10 minute presentation on how to evaluate impact, and that is just a low-level role. I used recent research which suggest that impact is best planned for from the start of projects rather than being a last-minute rush for a couple of people and suggested ways to go about gathering qualitative and quantitative impact data. I was later told that I was employed primarily because I ‘got’ the idea of impact.

“At the moment, every document counts as one item, meaning that a single-authored monograph counts the same (quantitatively) as a chapter in an edited collection.”

This isn’t strictly true – the REF guidelines gave the option for outputs to be “double-weighted”, so you could request for your book to count for two journal articles. Feedback from the assessment panels suggested that they looked very favourably on this, and very few requests for double-weighting were turned down. (At least in humanities – not sure it was common practice in other disciplines which are more journal-focussed.)