Well, here we are. The work of 52,077 academic staff submitted, 191,232 of their “outputs” assessed, 10,099 early career researchers included, 6,975 “impact case studies” reviewed. That is a lot of staff, a lot of aspiring researchers and a lot of stories. What are we to make of it all? The Research Excellence Framework 2014 results are about to hit the headlines. Buckle up.
Across the UK, academics will be nervously logging on to the REF website on Thursday to see what their peers have made of their research from 2008 to 2013. Innovation, revelation, inspiration and, yes, consternation, reduced to a grading on a four-star scale. What is this thing, and what do we do with it?
It is not just the quality of research that has been assessed (65% weighting towards the overall score), it is also the “vitality” of the departmental research environment (15%) and, controversially, an assessment of “impact” (20%), in other words whether the research made any difference in the wider world.
Some of it was surreal. We were required to identify research-inactive staff and assign them to a unit of assessment. This raises epistemological problems: what is it precisely that Dr Smith has not been researching in? Is he not researching in astrophysics or not researching in computing? I once came across someone who claimed to be not researching in genetics and simultaneously not researching in neurobiology, on the reasoning that interdisciplinary approaches are so important these days. Such is the REF bureaucracy.
Of course, we should not underestimate the capacity of the results to stimulate research endeavour in its own right. I was impressed by the advances in set theory achieved by some university PR departments in 2008 when they created new branches of mathematics by promoting groupings that contained only the top 10 elements thereof. But we should also watch out for the “Woodstock effect” in which 90% of the researchers in a department will claim credit for the 5% of work classified in the four-star “world-leading” category. They can’t all have been in there.
As is well known, the “impact” component contractually compelled all researchers to sell their souls to the devil and invent cures for baldness overnight, or risk having their research classified as worthless.
I’ve never been convinced by the rating scale “world leading”, “internationally excellent”, “recognised internationally” and “recognised nationally”. Those aren’t four points on an ill-defined scale: they’re four ill-defined points on four different scales. It’s like rating chocolate on a scale of 1 to 4, where 1 is poor value for money, 2 melts well in the mouth, 3 is nicely presented and 4 is deliciously sweet.
Allegations of selective submission have been rife. A two-star rating won’t attract funding, and this has led to the submission of only those with three-star work or better. However, keep a lookout for results which show that some well-funded universities have outputs rated as one-star or even “unclassified”. It is conceivable that some researchers have one potential Nobel prize-winning paper and just three other papers, one of which is one step above using crayons to colour in DNA sequences.
We need to have a real assessment of the return on investment in the REF. Some institutions produce extraordinary research results on levels of funding that are lower than individual faculties in other universities. Don’t get me wrong, we need to invest in our leading research departments, but we must fund excellence wherever it is found.
Yet money is eye-wateringly tight. When the autumn statement anticipates 40% post-election public spending cuts across the board, can the ringfence around the £4.6bn science research budget survive? Let’s hope it’s merely decimated. Initial allocations will be announced in April 2015, but 2016 will be when the real longer-term funding plans will be known.
Carping about league tables, value for money, impact and ratings is all good fun. But in the end, we should spend less energy aggregating ratings to determine marginal differences and instead celebrate that the UK is home to one of the world’s most vibrant research cultures. Undervalued, over-regulated, imperfect and spectacular in equal measure – but we should be proud of what we achieve. Heavy cuts would damage all this, and no amount of PR spin could fix that.
