The Student Room Group

2014 Research Excellence Framework Released!

http://www.timeshighereducation.co.uk/news/ref-2014-results-table-of-excellence/2017590.article

What does everyone think?

Universities ranked by Research Power:

1. University College London
2. University of Oxford
3. University of Cambridge
4. University of Edinburgh
5. University of Manchester
6. King's College London
7. University of Nottingham
8. Imperial College London
9. University of Bristol
10. University of Leeds
11. University of Southampton
12. University of Glasgow
13. University of Sheffield
14. University of Birmingham
15. University of Warwick
16. Newcastle University
17. Queen's University Belfast
18. Cardiff University
19. University of Liverpool
20. Durham University


Credits to LutherVan for compiling the Guardian rankings:

Here is The Guardian's (and Research Fortnight's) Research Power rankings:

1. University of Oxford
2. University College London
3. University of Cambridge
4. University of Edinburgh
5. University of Manchester
6. Imperial College London
7. King's College London
8. University of Nottingham
9. University of Bristol
10. University of Leeds
11. University of Southampton
12. University of Sheffield
13. University of Glasgow
14. University of Warwick
15. University of Birmingham
16. Newcastle University
17. Cardiff University
18. Durham University
19. Queen's University Belfast
20. Queen Mary London
(edited 9 years ago)

Scroll to see replies

Reply 1
Ranking based on "research power" isn't very sensible since it isn't normalised by institution size, and so gives the large universities (UCL/Manchester/etc) a slightly unfair advantage. It would make more sense to adjust for the university size if you were trying to measure quality, rather than quality*quantity.

For example, Imperial only being #6 is purely because they don't have many departments and so made a smaller submission than the universities ranked higher. LSE is very low for similar reasons. Both universities scored high on the units they did submit for, but obviously are going to do badly on a quality*quantity ranking since they are only submitting to something like 50-70% of the total units.
(edited 9 years ago)
Reply 2
Original post by poohat
Ranking based on "research power" isn't very sensible since it isn't normalised by institution size, and so gives the large universities (UCL/Manchester/etc) a slightly unfair advantage. It would make more sense to adjust for the university size if you were trying to measure quality, rather than quality*quantity.

For example, Imperial only being #6 is purely because they don't have many departments and so made a smaller submission than the universities ranked higher. LSE is very low for similar reasons. Both universities scored high on the units they did submit for, but obviously are going to do badly on a quality*quantity ranking since they are only submitting to something like 50-70% of the total units.


That is true, this is only one of the rankings. The other ranking ranks the institutions by GPA but it results in a really odd ranking which ends up with Institute of Cancer Research in 1st.

I don't think REF attempts to really rank it properly, it is only THE that does the ranking. REF releases the data. Its best to interpret it for yourself. % of 4* research is important but so is the # of submissions. Some schools "game" the system by submitting only their best work to make their % of 4* seem higher.
(edited 9 years ago)
Reply 3
As you say, REF isn't meant to give an overall ranking in that sense, and there is no real way to combine results across all subjects in a manner which makes obvious sense.

Also, remember that different universities had different REF strategies - some chose to optimise their (average) GPA by only submitting staff who were going to do well, while some chose to optimise their 'research power' (quality*quantity) by submitting more people. The somewhat silly staff exemption rules in REF give universities quite a bit of flexibility when it comes to artificially increasing/decreasing their score, depending on whether they value a high ranking (quality only) vs more funding (quality*quantity)

It just doesnt make any sense to amalgamate the rankings together like that, but obviously the media is going to do it since it generates click bait.
(edited 9 years ago)
Here is The Guardian's (and Research Fortnight's) Research Power rankings:

1. University of Oxford
2. University College London
3. University of Cambridge
4. University of Edinburgh
5. University of Manchester
6. Imperial College London
7. King's College London
8. University of Nottingham
9. University of Bristol
10. University of Leeds
11. University of Southampton
12. University of Sheffield
13. University of Glasgow
14. University of Warwick
15. University of Birmingham
16. Newcastle University
17. Cardiff University
18. Durham University
19. Queen's University Belfast
20. Queen Mary London
Reply 5
Good to see Oxford's engineering department do so well after spending the summer of 2013 help sort it out...
Reply 6
Original post by Okorange
That is true, this is only one of the rankings. The other ranking ranks the institutions by GPA but it results in a really odd ranking which ends up with Institute of Cancer Research in 1st.

I don't think REF attempts to really rank it properly, it is only THE that does the ranking. REF releases the data. Its best to interpret it for yourself. % of 4* research is important but so is the # of submissions. Some schools "game" the system by submitting only their best work to make their % of 4* seem higher.


It doesn't work entirely like that. Each person included has to submit a certain number of submissions and the number was only ever lower, in my experience, if they were an early career researcher or were absent for an extended period of time for medical reasons. Obviously given the set number required, the idea is to submit their absolute best work but, again from my time working on a department's submission, it was pretty difficult for the academic to know which to submit. If I recall correctly, there's even justification required for when someone who should be submitting to REF, isn't; so I don't think it's particularly easy to game the system.
Original post by Noble.
It doesn't work entirely like that. Each person included has to submit a certain number of submissions and the number was only ever lower, in my experience, if they were an early career researcher or were absent for an extended period of time for medical reasons. Obviously given the set number required, the idea is to submit their absolute best work but, again from my time working on a department's submission, it was pretty difficult for the academic to know which to submit. If I recall correctly, there's even justification required for when someone who should be submitting to REF, isn't; so I don't think it's particularly easy to game the system.


I think you insight is mainly only relevant to a university like Oxford.

Not those like Salford. They would game it.
Reply 8
Original post by Noble.
It doesn't work entirely like that. Each person included has to submit a certain number of submissions and the number was only ever lower, in my experience, if they were an early career researcher or were absent for an extended period of time for medical reasons. Obviously given the set number required, the idea is to submit their absolute best work but, again from my time working on a department's submission, it was pretty difficult for the academic to know which to submit. If I recall correctly, there's even justification required for when someone who should be submitting to REF, isn't; so I don't think it's particularly easy to game the system.


Its fairly easy to game the system. Universities can choose not to submit any staff member, for essentially any reason, essentially without justification. Typically, most places have internal trial REFs to judge which people are likely to get 3*/4* publications, and will often only submit those people (unless they are trying to maximise income without regard for ranking - you get more money by submitting more people).

You are confusing the number of publications each academic must submit (4, unless they have a reduction) with the decision about whether a particular academic will be submitted at all. Yes, if the university chooses to submit you then you will need the requisite number of papers, but they don't have to submit you at all.

All universities done this to some degree.
(edited 9 years ago)
Reply 9
Original post by poohat
Its fairly easy to game the system. Universities can choose not to submit any staff member, for essentially any reason, essentially without justification. Typically, most places have internal trial REFs to judge which people are likely to get 3*/4* publications, and will often only submit those people (unless they are trying to maximise income without regard for ranking - you get more money by submitting more people).

You are confusing the number of publications each academic must submit (4, unless they have a reduction) with the decision about whether a particular academic will be submitted at all. Yes, if the university chooses to submit you then you will need the requisite number of papers, but they don't have to submit you at all.

All universities done this to some degree.


Ah, sorry, this makes sense.

We had an odd situation where it was a real pain getting four submissions for one academic (and to get them to co-operate) and it felt as if we didn't have much choice in submitting something for them (although given what you say, this was probably a university decision). I knew the university could decide to submit someone, I only thought it went the other way - that 'key' academics/researchers had to have an excuse to not be submitted. Thanks for the clarification though. :smile:
Reply 10
Original post by Noble.
Ah, sorry, this makes sense.

We had an odd situation where it was a real pain getting four submissions for one academic (and to get them to co-operate) and it felt as if we didn't have much choice in submitting something for them (although given what you say, this was probably a university decision). I knew the university could decide to submit someone, I only thought it went the other way - that 'key' academics/researchers had to have an excuse to not be submitted. Thanks for the clarification though. :smile:

Well for example if the academic has only published 3 papers but they are all 3* or 4* then the university will probably want to submit them, so its a problem if they can't find the required fourth paper since they won't be eligible without it. In this case, they would most likely be submitted if they had (e.g.) a fourth paper that was a 2*

For most universities it was a balancing act - every paper at 3* or higher gets money for the university so on one hand you want to submit anyone who has at least one 3*, but departments also want to maximise their REF league position, which is based on the average ranking (unlike the above table, which is based on quality*quantity). So if you have a guy with (e.g.) two 3*s and two 2*s then its not obvious whether they should be submitted because on hand you get extra money for their 3*s, but their 2*s will lower your average ranking (GPA)

Realistically the exemption system is a mess and far too easy to game (and far too much time and money gets wasted on coming up with ways to game it)
Reply 11
Original post by poohat
Well for example if the academic has only published 3 papers but they are all 3* or 4* then the university will probably want to submit them, so its a problem if they can't find the required fourth paper since they won't be eligible without it. In this case, they would most likely be submitted if they had (e.g.) a fourth paper that was a 2*

For most universities it was a balancing act - every paper at 3* or higher gets money for the university so on one hand you want to submit anyone who has at least one 3*, but departments also want to maximise their REF league position, which is based on the average ranking (unlike the above table, which is based on quality*quantity). So if you have a guy with (e.g.) two 3*s and two 2*s then its not obvious whether they should be submitted because on hand you get extra money for their 3*s, but their 2*s will lower your average ranking (GPA)

Realistically the exemption system is a mess and far too easy to game (and far too much time and money gets wasted on coming up with ways to game it)


Yes, the system does seem a bit nonsensical. You would think any 'key' academic (actively researching/full-time/non-ECR - I know the definition would be a bit sketchy) would have to be submitted, unless there was a good reason.
As mentioned above, the power rankings aren’t all that helpful because of the bias toward large institutions. GPA gives a truer picture of where high quality research is being conducted. Here’s the top 10 by GPA:

1. Institute of Cancer Research
2. Imperial College
3. London School of Economics
4. Oxford University
5. Cambridge University
6. Cardiff University
7. King’s College London
8. University College London
8. University of Warwick
10. London School of Hygiene and Tropical Medicine
(edited 9 years ago)
Looking at every university in the UK boasting their world leading/global research on their homepages today is extremely entertaining (if you're a bit of a saddo).

Especially the different ways they're all trying to spin it as a major success.
Original post by War and Peace
As mentioned above, the power rankings aren’t all that helpful because of the bias toward large institutions. GPA gives a truer picture of where high quality research is being conducted. Here’s the top 10 by GPA:

1. Institute of Cancer Research
2. Imperial College
3. London School of Economics
4. Oxford University
5. Cambridge University
6. Cardiff University
7. King’s College London
8. University College London
8. University of Warwick
10. London School of Hygiene and Tropical Medicine


I don't believe so.

Otherwise a university like Cardiff, which provided roughly 30% of FTE staff compared to the FTE staff UCL provided would not be ranked above UCL.

I don't believe UCL has as much as 3 times the academic staff as Cardiff.
Original post by LutherVan
I think you insight is mainly only relevant to a university like Oxford.

Not those like Salford. They would game it.


I can confirm with you that my department at Oxford had everybody submitted, but considering how the director kept emphasising on that, I'm guessing it's not the norm everywhere, and probably not even within the university.


I think this ranking by GPA gives a better overall view. To be honest, statistics can be altered to display anything. KCL and Cardiff really have done well, in so many factors.

As expected, London and Oxbridge dominate.
(edited 9 years ago)
All these complaints about the results means nothing. The REF ranking, and the one that places Oxford on top, is the only league table that actually matters.

Universities actually get actual money for doing well on that table.
Reply 18
I think the main reason unis are "bragging" about their results is because the REF is only done once every 7 years or so and on top of that its the most "official" since its done by the UK government and has a direct effect on funding in the future. I don't even blame them for trying to manipulate the results to make them look good. Universities are almost a business these days, they want to attract the most funding, the best researchers, the best students etc so they need to maintain their public image and if that means cherry picking data, who are we to stop it.
(edited 9 years ago)
Reply 19
Using the REF to rate whole universities and try and rank them is an exercise of limited usefulness - you go somewhere to study your SUBJECT. The fact that institution X appears marginally stronger than institution Y simply because its strengths lie in disciplines other than the one you want to study seems a weird reason to make your choice (if I want to study History, for example, the fact that an institution is belting at Chemical Engineering is not really pertinent). Some places which appear modest overall may still be world-leaders and innovators in particular niche subjects.

An even if your subject scores highly in the REF at a particular institution, don't imagine that undergraduates will necessarily benefit much. Decent researchers often do decent research because they're not teaching very often! And inspiring teachers may not be cutting-edge researchers.

The purpose of the REF is to help allocate funding for research - and for expensive 'Big Science'm funding probably does need to be concentrated rather than diluted across lots of institutions (economies of scale and best value for money)- but I've never understood why that argument holds with arts and humanities subjects - or even social science.

Quick Reply