“How do you feel about your department’s REF performance?” “Very good, as a matter of fact. We were fourth by research concentration on outputs.” “That’s odd. This table has you 27th.”
OK, I made that up. As far as I know, there is no measure of research concentration. But watch this space. After all, the results of the recent Research Excellence Framework (REF) have been ranked according to research intensity, power and strength, all weighting the same factors in different ways. Add to that grade point average and quality index and there are at least five ways in which league tables have been compiled by serious-minded folk. Some of these measures take into account the overall size of a department, some the proportion of staff that have been submitted to the exercise, and some of them try to mimic the likely cash settlement that will follow. And each is capable of generating different league tables.
That’s just for starters. Each score is a combination of three factors: outputs, environment and impact. Some pundits and bloggers have looked at rankings based on just one or two of those measures, yielding seven variations, all capable of being ranked in any of the five ways. So there are at least 35 ways of compiling the league tables. My department is second at our best but about 19th at our worst. Guess which one we’ll use?
We can go on. Each “unit of assessment” contains departments of different types. History of art, fine arts and design are all included together – as are media studies, library studies and digital humanities. With enough time on your hands, you can try to segment each league table into sub-league tables of specialisms, and then rank each sub-table according to the 35 methodologies. There is no limit to how far this can go. League table explosion.
Those who have been arguing for the abolition of the REF have pretty much got what they wanted. It has been drowned out by its own noise. In the 1960s the philosopher Herbert Marcuse introduced the idea of “repressive tolerance”. When everything is tolerated, nothing is noticed. We have achieved something close with the REF. When 25 departments can each prove that they came in the top three, ranking is over. Although many departments are crowing about their performance on their websites, very soon we are all going to learn to take it with a pinch of salt.
Is that good or bad? Why do we need rankings? In the first instance the REF is used to distribute cash to fund future research, and some of the tables attempt to predict future distributions of money, either absolutely or per capita. Beyond that, a decent league-table position boosts morale and can help win an argument about staff replacement. Not to be sniffed at. But is there anything else?
Sometimes it is argued that doing well in the REF helps student recruitment. For undergraduates, a university’s overall ranking contributes to prestige and a degree from a prestigious university is great for the CV. But I doubt that a similar effect happens at department level to any significant extent. Are prospective undergraduates really swayed by the quality of the research in a department? There might even be a negative effect: applicants could believe that the better the research, the less time the faculty is likely to devote to teaching undergraduates.
What about graduate students? Here the research strength of the department is an obvious point of attraction. But how is that to be judged? In the first instance, graduate students may hope to work with the academics whose writings they encountered and admired in their undergraduate days. And if they have any sense, they will seek advice from the teachers and other graduate students in their current university. Of course, advice can be misleading and great writers can be rotten supervisors. And so it is not impossible that a prospective graduate student, in deciding where to study, might take seriously the bold text on a departmental website that reads “4th by research concentration on outputs”. Good luck.
• Jonathan Wolff is professor of philosophy at University College London and dean of arts and humanities
