REA 2008: shiny ambiguous statistics

[Jono]

Bleh! [list=][/list]http://www.st-andrews.ac.uk/research/ResultsofRAE2008/

Feel free to poke holes.Such as the disparity between the numbers of staff surveyed between departments. Also, since when do we have a drama, dance and performing arts department?

[James Shield]

My understanding of the RAE is that we don't have answers to some of the most important questions yet. Specifically, we don't know how the newspapers will interpret the results in drawing up their league tables, and we don't yet know how much of the ~£1.5 billion pot of funding will be ours. In short, there is one major pro and one major con:

Pro: The scale of quality from 1* to 4* is non-linear. This is good for St Andrews because we're quite a high performer. If a linear scale had been used, a 4* would have twice the value of a 2*. Instead, the scale is exponential (or something like it). This is the measure that will be used to determine our funding.

Con: Volume vs density. Here's an example. Let's imagine an academic department at Edinburgh has 50 academic staff, of which 20 were submitted to RAE (because each university gets to choose who to submit). They are all rated as being excellent. Being a smaller university, the same department at St Andrews has only 25 academic staff - 20 of them are submitted and rated excellent. The problem is that even though 80% of the St Andrews department were submitted and rated excellent compared with 40% of Edinburgh's, RAE doesn't take this into account - Edinburgh comes out on top for producing a greater volume of high quality research.

The numbers above are made up, but in reality St Andrews submitted 92% of its academic staff to RAE, meaning that we're far more research intensive than other universities. This has obvious benefits for students - you're far more likely to encounter a highly-rated, research-active lecturer or tutor here than you might elsewhere. Whether or not the papers will take this into account when drawing up their league tables is yet to be seen.

Edit: The number of staff submitted is probably proportional to the size of the department. My guess is that Drama, Dance & Performing Arts refers to some of the research in the School of English (e.g. Dr Philip Parry is Senior Lecturer in Drama).

[Jono]

Is that £1.5bn for every university in britain?

[James Shield]

Yes - it's a £1.5 billion annual fund for the whole of the UK, and the purpose of the RAE is primarily to decide what proportion of that fund we receive each year. There are other sources of funding too, of course - according to the BBC, the £1.5 billion is "a third of the available research funding".

[lts2]

I was a little bit involved with organising the RAE (through the research councils) this year. Its one hell of a process..! The problem is that RAE results are very useful for the universities but the papers tend to twist the figures to say whatever they want....

Its best to read about the RAE from source rather than in the papers or even through the Uni website if you want to find out the facts.