Thursday, July 12, 2007

Dump the RAE

Dump the RAE Martin Weller and Juliette White have been discussing the frustrations of the RAE for bloggers.
The RAE system has done nothing but damage and distort higher education and much original research in the UK (and waste huge sums of public money) ever since it was introduced:
The RAE has had a disastrous impact on the UK higher education system, leading to the closure of departments with strong research profiles and healthy student recruitment. It has been responsible for job losses, discriminatory practices, widespread demoralisation of staff, the narrowing of research opportunities through the over-concentration of funding and the undermining of the relationship between teaching and research (source).

But in the biological and medical sciences field at least, the progress towards open access publishing has been staggering over the last year. Recently there have been further developments such as Nature Preceedings, although this lacks the gold standard of scientific research, peer review. PLoS One does a much better job by incorporating community-based open peer review involving online annotation, discussion, and rating.

Perhaps the answer to RAE is to develop a system of peer review for bloggers. Technorati or the Google algorithms don't fully address this need, and PLoS One is based on the traditional academic paper structure for submissions rather than blogging architechture. Digg and other content voting systems are open to too much gaming (although no worse than the way universities currently gameplay the RAE system) and are not transparent.

So who's going to build PLoS One for academic bloggers?


  1. Like the image - should have found one myself. I suggested there were two complaints you might make against the RAE:
    i) that it is okay but measures the wrong things.
    ii) that it is fundamentally a pointless (and damaging) exercise.
    I'm not sure if what you are suggesting is more in the camp of i). I think I've come round to ii). Particularly if one takes an 'everything is miscellaneous' view (I know, I've been applying this to everything recently).
    What is the point of the RAE? You could argue that it gives us a reliable, standardised measure of someone's research quality. In Wienberger's analysis this is like second order order, ie separated metadata from the actual data (the researcher's output). But in a highly connected world, everything is metadata. I don't need to know how other people rank someone's research, I can find it easily and make my own assessment. It is also a system that relies on 'authority' and power, it is Brittanica in a wikipedia world. It is also hugely expensive.
    Does any of that make sense?

  2. Feel free to use the image!
    I am also in camp ii: the RAE is a damaging exercise. I'm not sure if I would go so far as to say it is is fundamentally pointless. I suspect the original intentions were honourable, but the whole exercise rapidly became perverted by Thatcherite politics and gaming by universities. (I don't blame universities for this response - what else were they supposed to do?) I like your encyclopaedia analogy. RAE is certainly assessment 1.0, but so is the practice of most universities (OU is rather exceptional in this regard).

  3. I like the idea of Philica ( which matches reviews to soundness. Unfortunately when I looked it was seeded by umm... people with a special point of view, so the ratings were meaningless. If it was used by academics writing in their own fields then it might work.