A league Table that Matters (for Engineering and Physical Sciences)

It is an unfortunate fact of academic life that league tables matter. They matter not because they are inherently good things, but because people pay attention to them. University leaders care, parents care, students care, foreign students care, employers care. They should not care so much, because most league tables are foolish time-wasting restatements of average A-level grades and little more. Essentially, universities are ranked by the undergraduate students that attend.

There is an exception. There is an excellent one produced by the bibliometricians at the University of Leiden who use a variety of measures of paper productivity and quality to create a ranking system. Although this is an excellent resource, it has limitations. The table is concerned with science so is less useful for other areas. The creators seem mostly concerned with the best universities so the tables do not really give a sense of the full range of what can be expected from rather less distinguished universities. Finally, for the UK only about a third of the universities are listed.

What to do?  Answer, get help from EPSRC (The Engineering and Physical Science Research Council). 

The EPSRC funds scientific research AND also provides data on where those funds go. The data from 2012/13 is here. This provides data for attempts to obtain funding, conversion, total funds awarded etc. This is the input side of the UK University scientific research enterprise. This can be sorted to obtain a league table relating to Engineering and Physical Sciences. 

There are two metrics considered here. There is the number of proposals funded. This is an index of ideas deemed by peer review and the research council to be worth funding. These are the ideas that will create the university's future. The other is success rate. This is an index of the competitiveness of the researchers generating new ideas at an institution. This second one is a bit tricky because at the high end it is subject to the vagaries of small sample sizes, but it is still worth contemplating.

UK universities sorted by both of these metrics are listed below (as is all the data). In the rank by number of proposals, there are not so many surprises. But in the success rate index, there are. Setting aside the few at the top which are subject to the problems of small numbers, there are a few surprises. Oxford does not make the top 20 and Cambridge barely does. What this means is that there are pockets of excellent science in some unexpected places and there are opportunities in those universities to work with excellent researchers for a student willing to take the time. 

Before finishing, an apology. One of my pet peeves is the modern obsession with the last data point (this video explains why this is a bad thing). Please accept my apologies for presenting this as a single year.

Rank by fundable ideas:

Rank	Organisation	Total number of research proposals considered	Number of research proposals funded	% Funding rate by number	Total value of research proposals considered	Value of research proposals funded	% Funding rate by value
1	University College London	177	76	43%	151,898,494	81,894,518	54%
2	Imperial College London	215	69	32%	182,858,481	59,547,192	33%
3	University of Cambridge	116	49	42%	125,168,353	67,039,995	54%
4	University of Oxford	121	43	36%	81,868,450	21,079,033	26%
5	University of Bristol	98	42	43%	80,861,747	34,955,056	43%
6	University of Warwick	109	41	38%	73,634,307	31,295,359	43%
7	The University of Manchester	125	38	30%	82,969,932	27,168,293	33%
8	University of Sheffield	113	38	34%	83,653,607	24,390,110	29%
9	University of Leeds	83	31	37%	50,109,395	18,594,362	37%
10	University of Southampton	90	31	34%	41,243,114	16,451,281	40%
11	University of Nottingham	86	30	35%	72,815,137	27,046,795	37%
12	University of Bath	94	29	31%	60,789,852	25,513,404	42%
13	University of Edinburgh	75	28	37%	57,419,900	30,524,784	53%
14	Newcastle University	55	24	44%	52,395,451	22,784,557	43%
15	University of Liverpool	63	24	38%	39,792,268	11,985,869	30%
16	University of Strathclyde	74	23	31%	51,365,657	16,929,523	33%
17	Cardiff University	53	22	42%	31,191,330	12,002,146	38%
18	Loughborough University	58	22	38%	52,364,689	20,566,754	39%
19	Durham University	43	21	49%	22,256,391	10,984,571	49%
20	University of Birmingham	67	21	31%	44,939,520	13,229,351	29%
21	Lancaster University	38	20	53%	21,388,892	8,844,983	41%
22	University of Glasgow	48	17	35%	47,219,137	23,948,483	51%
23	King's College London	48	15	31%	16,790,091	2,988,748	18%
24	University of Exeter	49	15	31%	23,875,970	7,044,042	30%
25	University of York	41	15	37%	26,186,804	9,138,981	35%
26	Brunel University	39	14	36%	22,987,936	11,949,185	52%
27	Heriot-Watt University	49	14	29%	35,628,165	11,656,495	33%
28	Queen's University of Belfast	35	12	34%	26,934,491	10,336,054	38%
29	Cranfield University	46	10	22%	25,877,431	4,511,361	17%
30	University of St Andrews	33	10	30%	18,598,220	6,660,201	36%
31	Queen Mary, University of London	43	9	21%	17,949,342	3,436,930	19%
32	University of Surrey	61	9	15%	29,291,237	3,613,891	12%
33	Royal Holloway, Univ of London	14	8	57%	6,317,224	3,598,978	57%
34	University of Aberdeen	16	7	44%	9,397,157	3,405,566	36%
35	De Montfort University	14	6	43%	6,318,695	2,406,769	38%
36	Open University	19	6	32%	7,603,128	1,300,427	17%
37	Swansea University	27	6	22%	22,023,324	4,716,297	21%
38	University of Dundee	14	6	43%	18,016,949	8,121,898	45%
39	University of East Anglia	17	5	29%	5,324,459	892,486	17%
40	University of Kent	14	5	36%	3,890,258	664,053	17%
41	University of Leicester	17	5	29%	6,286,485	1,281,111	20%
42	University of Reading	17	5	29%	6,598,284	1,755,610	27%
43	University of Ulster	11	5	45%	4,405,034	2,677,818	61%
44	Aston University	8	4	50%	2,696,568	2,239,856	83%
45	University of Plymouth	5	4	80%	1,481,365	1,157,074	78%
46	City University London	7	3	43%	3,551,957	1,522,601	43%
47	Manchester Metropolitan University	5	3	60%	1,348,569	774,385	57%
48	Northumbria University	10	3	30%	3,501,519	1,458,259	42%
49	University of Greenwich	5	3	60%	915,163	728,059	80%
50	University of Hull	10	3	30%	4,449,550	310,363	7%
51	University of Salford	10	3	30%	4,962,922	1,176,128	24%
52	University of Sussex	13	3	23%	10,756,276	3,951,483	37%
53	Aberystwyth University	5	2	40%	1,246,869	289,624	23%
54	City University	7	2	29%	2,491,376	207,330	8%
55	Glyndwr University	6	2	33%	2,535,139	569,320	22%
56	Nottingham Trent University	7	2	29%	1,336,450	156,119	12%
57	University of Brighton	7	2	29%	1,641,575	438,923	27%
58	University of Central Lancashire	7	2	29%	3,232,697	164,132	5%
59	University of Huddersfield	7	2	29%	3,357,981	653,831	19%
60	Bangor University	2	1	50%	643,556	231,548	36%
61	Coventry University	6	1	17%	2,259,288	477,591	21%
62	Edinburgh Napier University	6	1	17%	1,076,504	238,068	22%
63	Goldsmiths College	1	1	100%	130,751	130,751	100%
64	Keele University	8	1	13%	1,143,491	15,762	1%
65	London School of Economics & Pol Sci	3	1	33%	266,294	93,836	35%
66	Oxford Brookes University	6	1	17%	2,563,759	1,184,087	46%
67	University of Hertfordshire	6	1	17%	2,311,986	375,542	16%
68	University of Portsmouth	7	1	14%	1,188,675	752,949	63%
69	University of Stirling	5	1	20%	2,598,610	1,228,231	47%
70	University of the Highlands and Islands	1	1	100%	116,054	116,054	100%
71	University of the West of England	12	1	8%	7,698,034	49,730	1%
72	University of the West of Scotland	3	1	33%	301,679	99,872	33%
73	Birkbeck College	2	0	0%	1,649,467	0	0%
74	Birmingham City University	3	0	0%	444,318	0	0%
75	Bournemouth University	3	0	0%	658,830	0	0%
76	Cardiff Metropolitan University	1	0	0%	280,056	0	0%
77	Glasgow Caledonian University	4	0	0%	1,402,026	0	0%
78	Kingston University	4	0	0%	633,880	0	0%
79	Leeds Metropolitan University	1	0	0%	220,878	0	0%
80	Liverpool John Moores University	4	0	0%	1,092,417	0	0%
81	London Metropolitan University	2	0	0%	915,227	0	0%
82	London South Bank University	2	0	0%	498,243	0	0%
83	Middlesex University	5	0	0%	8,415,091	0	0%
84	Royal College of Art	1	0	0%	1,126,393	0	0%
85	Sheffield Hallam University	4	0	0%	1,792,553	0	0%
86	Southampton Solent University	1	0	0%	272,421	0	0%
87	Swansea Metropolitan University	1	0	0%	75,586	0	0%
88	The Robert Gordon University	2	0	0%	1,910,764	0	0%
89	University of Abertay Dundee	2	0	0%	430,496	0	0%
90	University of Bedfordshire	2	0	0%	311,628	0	0%
91	University of Bolton	2	0	0%	1,220,192	0	0%
92	University of Bradford	9	0	0%	1,916,361	0	0%
93	University of Derby	2	0	0%	189,508	0	0%
94	University of East London	2	0	0%	196,302	0	0%
95	University of Essex	6	0	0%	1,878,555	0	0%
96	University of Glamorgan	7	0	0%	4,342,748	0	0%
97	University of Lincoln	4	0	0%	709,562	0	0%
98	University of Northampton	4	0	0%	504,265	0	0%
99	University of Teesside	1	0	0%	268,146	0	0%
100	University of Westminster	1	0	0%	4,067,836	0	0%

Competitiveness (%funding rate by number)

Rank	Organisation	Total number of research proposals considered	Number of research proposals funded	% Funding rate by number	Total value of research proposals considered	Value of research proposals funded	% Funding rate by value
1	Goldsmiths College	1	1	100%	130,751	130,751	100%
2	University of the Highlands and Islands	1	1	100%	116,054	116,054	100%
3	University of Plymouth	5	4	80%	1,481,365	1,157,074	78%
4	Manchester Metropolitan University	5	3	60%	1,348,569	774,385	57%
5	University of Greenwich	5	3	60%	915,163	728,059	80%
6	Royal Holloway, Univ of London	14	8	57%	6,317,224	3,598,978	57%
7	Lancaster University	38	20	53%	21,388,892	8,844,983	41%
8	Aston University	8	4	50%	2,696,568	2,239,856	83%
9	Bangor University	2	1	50%	643,556	231,548	36%
10	Durham University	43	21	49%	22,256,391	10,984,571	49%
11	University of Ulster	11	5	45%	4,405,034	2,677,818	61%
12	Newcastle University	55	24	44%	52,395,451	22,784,557	43%
13	University of Aberdeen	16	7	44%	9,397,157	3,405,566	36%
14	University College London	177	76	43%	151,898,494	81,894,518	54%
15	University of Bristol	98	42	43%	80,861,747	34,955,056	43%
16	De Montfort University	14	6	43%	6,318,695	2,406,769	38%
17	University of Dundee	14	6	43%	18,016,949	8,121,898	45%
18	City University London	7	3	43%	3,551,957	1,522,601	43%
19	University of Cambridge	116	49	42%	125,168,353	67,039,995	54%
20	Cardiff University	53	22	42%	31,191,330	12,002,146	38%
21	Aberystwyth University	5	2	40%	1,246,869	289,624	23%
22	University of Warwick	109	41	38%	73,634,307	31,295,359	43%
23	University of Liverpool	63	24	38%	39,792,268	11,985,869	30%
24	Loughborough University	58	22	38%	52,364,689	20,566,754	39%
25	University of Leeds	83	31	37%	50,109,395	18,594,362	37%
26	University of Edinburgh	75	28	37%	57,419,900	30,524,784	53%
27	University of York	41	15	37%	26,186,804	9,138,981	35%
28	University of Oxford	121	43	36%	81,868,450	21,079,033	26%
29	Brunel University	39	14	36%	22,987,936	11,949,185	52%
30	University of Kent	14	5	36%	3,890,258	664,053	17%
31	University of Nottingham	86	30	35%	72,815,137	27,046,795	37%
32	University of Glasgow	48	17	35%	47,219,137	23,948,483	51%
33	University of Sheffield	113	38	34%	83,653,607	24,390,110	29%
34	University of Southampton	90	31	34%	41,243,114	16,451,281	40%
35	Queen's University of Belfast	35	12	34%	26,934,491	10,336,054	38%
36	Glyndwr University	6	2	33%	2,535,139	569,320	22%
37	London School of Economics & Pol Sci	3	1	33%	266,294	93,836	35%
38	University of the West of Scotland	3	1	33%	301,679	99,872	33%
39	Imperial College London	215	69	32%	182,858,481	59,547,192	33%
40	Open University	19	6	32%	7,603,128	1,300,427	17%
41	University of Bath	94	29	31%	60,789,852	25,513,404	42%
42	University of Strathclyde	74	23	31%	51,365,657	16,929,523	33%
43	University of Birmingham	67	21	31%	44,939,520	13,229,351	29%
44	King's College London	48	15	31%	16,790,091	2,988,748	18%
45	University of Exeter	49	15	31%	23,875,970	7,044,042	30%
46	The University of Manchester	125	38	30%	82,969,932	27,168,293	33%
47	University of St Andrews	33	10	30%	18,598,220	6,660,201	36%
48	Northumbria University	10	3	30%	3,501,519	1,458,259	42%
49	University of Hull	10	3	30%	4,449,550	310,363	7%
50	University of Salford	10	3	30%	4,962,922	1,176,128	24%
51	Heriot-Watt University	49	14	29%	35,628,165	11,656,495	33%
52	University of East Anglia	17	5	29%	5,324,459	892,486	17%
53	University of Leicester	17	5	29%	6,286,485	1,281,111	20%
54	University of Reading	17	5	29%	6,598,284	1,755,610	27%
55	City University	7	2	29%	2,491,376	207,330	8%
56	Nottingham Trent University	7	2	29%	1,336,450	156,119	12%
57	University of Brighton	7	2	29%	1,641,575	438,923	27%
58	University of Central Lancashire	7	2	29%	3,232,697	164,132	5%
59	University of Huddersfield	7	2	29%	3,357,981	653,831	19%
60	University of Sussex	13	3	23%	10,756,276	3,951,483	37%
61	Cranfield University	46	10	22%	25,877,431	4,511,361	17%
62	Swansea University	27	6	22%	22,023,324	4,716,297	21%
63	Queen Mary, University of London	43	9	21%	17,949,342	3,436,930	19%
64	University of Stirling	5	1	20%	2,598,610	1,228,231	47%
65	Coventry University	6	1	17%	2,259,288	477,591	21%
66	Edinburgh Napier University	6	1	17%	1,076,504	238,068	22%
67	Oxford Brookes University	6	1	17%	2,563,759	1,184,087	46%
68	University of Hertfordshire	6	1	17%	2,311,986	375,542	16%
69	University of Surrey	61	9	15%	29,291,237	3,613,891	12%
70	University of Portsmouth	7	1	14%	1,188,675	752,949	63%
71	Keele University	8	1	13%	1,143,491	15,762	1%
72	University of the West of England	12	1	8%	7,698,034	49,730	1%
73	Birkbeck College	2	0	0%	1,649,467	0	0%
74	Birmingham City University	3	0	0%	444,318	0	0%
75	Bournemouth University	3	0	0%	658,830	0	0%
76	Cardiff Metropolitan University	1	0	0%	280,056	0	0%
77	Glasgow Caledonian University	4	0	0%	1,402,026	0	0%
78	Kingston University	4	0	0%	633,880	0	0%
79	Leeds Metropolitan University	1	0	0%	220,878	0	0%
80	Liverpool John Moores University	4	0	0%	1,092,417	0	0%
81	London Metropolitan University	2	0	0%	915,227	0	0%
82	London South Bank University	2	0	0%	498,243	0	0%
83	Middlesex University	5	0	0%	8,415,091	0	0%
84	Royal College of Art	1	0	0%	1,126,393	0	0%
85	Sheffield Hallam University	4	0	0%	1,792,553	0	0%
86	Southampton Solent University	1	0	0%	272,421	0	0%
87	Swansea Metropolitan University	1	0	0%	75,586	0	0%
88	The Robert Gordon University	2	0	0%	1,910,764	0	0%
89	University of Abertay Dundee	2	0	0%	430,496	0	0%
90	University of Bedfordshire	2	0	0%	311,628	0	0%
91	University of Bolton	2	0	0%	1,220,192	0	0%
92	University of Bradford	9	0	0%	1,916,361	0	0%
93	University of Derby	2	0	0%	189,508	0	0%
94	University of East London	2	0	0%	196,302	0	0%
95	University of Essex	6	0	0%	1,878,555	0	0%
96	University of Glamorgan	7	0	0%	4,342,748	0	0%
97	University of Lincoln	4	0	0%	709,562	0	0%
98	University of Northampton	4	0	0%	504,265	0	0%
99	University of Teesside	1	0	0%	268,146	0	0%
100	University of Westminster	1	0	0%	4,067,836	0	0%

The Devil is in the Memes

If standards agencies were run like higher education, the meter would now be less than a yard (about 5% less).¹

If the Prime Minister (£142,500/yr) were paid like the average University VC (£235,000/yr), he would be making around £681 Million/yr.²

If cafes were like higher education, they would have more than one person standing around for everyone serving you coffee (not people standing around when they should be serving coffee but standing around to manage, administer, and provide support).³

Footnotes:

1) Between 1995 and 2013 the proportion of 2.1 and firsts increased by about 14% according to the data here: http://en.wikipedia.org/wiki/Grade_inflation. Yes, it is wikipedia...

... but this article says there may have been grade inflation in first class degrees: http://www.telegraph.co.uk/education/universityeducation/9792964/Number-of-first-class-degrees-has-tripled-since-late-90s.html.

... and this article thinks so too: http://www.independent.co.uk/student/student-life/why-are-so-many-more-firsts-being-awarded-than-ever-before-8456150.html

 

... and this article documents that at Cambridge the first class degree proportion grew from about 10% in 1960 to 22% in 2005. During that time the proportion of 2.1 degrees increased from 24% to 50% in the same period. 

There are always doubters.There are claims that when controlled for A-level grades little or no inflation has occurred (http://www.bbc.co.uk/news/education-25811702) ...

... not that there was any grade inflation in A-levels! No never...  http://www.bbc.co.uk/news/education-23695270. 

2) Assume that the average VC serves a population of 20,000 students. This means that a VC gets about £11.75/student. The Prime Minister serves a population of about 58 million people. At £11.75/person that is about £681 million a year. The PM may be underpaid. 

3) Academic staff make up 47.5% of the UK HE sectors employees. People on hourly paid and zero hours contracts were not included in that statistic. Data are here: http://www.ecu.ac.uk/publications/files/equality-in-he-stats-report-2012-staff.pdf .

Note: this is not a UK only phenomenon. See for instance: 

http://chronicle.com/article/Administrative-Bloat-How-Much/135500/
http://www.bloomberg.com/news/2012-11-14/bureaucrats-paid-250-000-feed-outcry-over-college-costs.html

Oxymorons and the Pay for Performance Myth

In case no one noticed, the Vice-Chancellors of UK Universities (that is the UK equivalent of University President in the US) got themselves large average pay increases. For the Russell group it was about 8% average increase. For all of them the average was about 5.1%. This is an inflation beating pay increase.

Like many of their counterparts in industry who fancy themselves entrepreneurial risk takers, this is not matched by performance. Before continuing let’s be clear, somewhere in the system there is a VC who deserves that raise. This is not about that VC or any individual VC. Individually, I suspect there are successes or failures to celebrate. Somewhere there is a big fish in his or her small pond who is keeping it in good order. This is not about local success or failure. It is about the lot of them.

Collectively, they are doing a terrible job. Really awful.

Those are strong words. Why would anyone say such a thing?

You might say such a thing because they have presided over the looting of students and their parents. Net cost of education to students and their families has skyrocketed. There has been a three fold increase in fees to students on the watch of most of the current VCs. If they have been in post since 2004/5 the increase is still greater. In return for a little more cash in pocket, they sold out the idea of University Education as a public good. Well done.

You might say such a thing because in recent years there has been an 11% rise in the cost of student rents – something to consider in more detail another day. This transfer of cost and increased cost is not just a financial looting, but a looting of futures. Graduates released into the world who are massively in debt do not have the same options or mind set as those with more economic freedom. They won’t be taking as many big risks, because they have already taken one and that was on their education. 

You might say such a thing because while looting students and their parents and getting paid more themselves, the VCs have presided over years of sub-inflation pay rises for their academic staff. People understand a bit of shared belt tightening. Supposing costs to students were being held down and if belt tightening were shared, no one would have too much to say. However, neither is the case.

You might say it because the rudderless boat they are pretending to be Captains of is going nowhere. Into this leaderless vacuum comes Government, the Media, the League Tables, etc. and they are defining the terms of a debate the University VCs haven’t even shown up to. As one example, over the last few years, the University VCs have allowed their institutions to be collectively defined in terms of student satisfaction in the worst possible way (Note the “satisfaction” column here). Yes, student satisfaction matters, but in the scheme of things it is not the point of what a University does. Which matters more: student achievement or student satisfaction? 

More important than any of these reasons, you might say these VCs were failures because of grade inflation. Over the last 10 years, the VCs have presided over the continuing debasement of degree standards. It just goes on and on. If the VCs do not get their act together, degree granting will be “looked at” and regulated and perhaps, with the best of intentions, standardised exams will be given to define degree class. Yes, the league table generators have much to answer for, but at the root of the problem are the people responsible: University VCs. In the last academic year alone the number of first class degrees inflated 13%. Sorry to have to break it to you Ladies and Gentlemen of the VC Class, but you are doing a terrible job. Grow a spine. What version of “this one goes to 11” will you come up with to solve this problem you have visited upon your institutions? The day the university equivalent of the A* degree is introduced is the day every VC in the UK should be told to clean out their desks and the door slammed on their backsides as they leave for the last time. That day may be sooner than anyone thinks.

That is it in a nutshell. “Academic Leadership” is an oxymoron which means charging more, delivering less, calling it better, and taking more home in pay.