1. A Comparison of Global University Rankings Erhan Erkut, PhD President, Ozyegin University December 2010 1

2. Why am I interested? A quant person Rankings, measurement, multiple criteria Using bibliometrics for quite some time Hiring, promotions, grants, partnerships, … Frustrated with the lack of transparency & accountability in the Turkish U system. The public has a right to know 2

3. Rankings matter 2005, U of Malaya #89 in THES Foreign student classification error + lower reputation rankings...2006  #169 “Shocking Global Slide” “Crisis in Malaysia's Public Universities” Vice Chancellor (Rector) was fired! The Great Brain Race Ben Wildavsky, 2010 3

4. Rankings can be manipulated QS: Alexandria #147 #4 in citations—behind only Caltech, M.I.T. and Princeton, and ahead of both Harvard and Stanford Mohamed El Naschie of Alexandria U published 320 of his own articles in a scientific journal of which he was also the editor. New York TimesNov. 14, 2010 4

5. Criticizing rankings is a popular sport among academics. All multi-criteria multi-stakeholder rankings can be criticized. Thank you for all the work you do. Please continue doing it. Keep improving it. 5

6. Global university ranking systems ARWU (Jiao Tong) Times Higher Education Quacquarelli Symonds Webometrics HEEACT Leiden SCImago 6

8. And… UK - Research Assessment Exercise US News & World Report Financial Times -- MBA rankings Gourman Report – Law Schools … EU Attempt to measure teaching and contribution to society €1.1M 2009 – multidimensional ranking system 2010 – engineering and business faculties of 150 univ. First results: 2011 May 8

9. Top 10 in the 2009 rankings 9

10. Top 10 (2009) Harvard #1 Stanford, MIT, Johns Hopkins, Washington and Michigan: 4/6 UCLA and Berkeley: 3/6 Common features Rich country Old university Large university Emphasis on grad. programs 10

11. An empirical comparison of global ranking systems Proof of concept Assumption There is a “true” ranking Each method is an approximation Method Take the top 10 from each ranking Form the union (20 univ.) Find the true ranks such that The sum of Euclidean distances between the rankings and the true rank vector is minimized 11

12. Example 12

13. Example U2 M1 M2 M3 U1 13

14. Example U2 M1 M2 M3 U1 14

15. Example U2 M1 M2 M3 U1 15

16. A defensible way of combining rankings Nonlinear optimization problem with as many variables as the number of universities Can use any distance metric (not limited to Euclidean) 16

17. With actual data…

18. Distance (similarity) Matrix 18

19. “Optimally” aggregated rankings

20. Turkish Univ? (2009) ARWU 2009 İstanbul 424 THE-QS 2009 Bilkent 360 İTÜ, İstanbul Ü, Koç 401-500 Webometrics METU 499 HEEACT (top 500) Leiden (top 250) SCImago Hacettepe 304, Ankara 363, Gazi 420, METU465, Ege 491, İstanbul 497 20

21. Factors impacting research output of Turkish universities negatively Researcher Time Financial support Infrastructure Human resources Career management Not today’s topic. More on this in the paper. 21

22. Ranking Turkish Universities Paper and citation counts Research Grants Research Prizes (Medals) Grad School Admission Exam Results URAP H-index 22

23. H-index ranking of Turkish U founded before 2000 TOP BOTTOM 23

24. The best predictor of H-index is … 24

25. 1.11 * (N)0.52 25

26. Overperformers (% Error) 26

27. Underperformers (%Error) 27

28. Problems with rankings Inputs vs. Outputs Need to measure outputs Measurement problems Citation-based errors Impact adjustments Area normalization Impact on the society PhD production Undergraduate employment Start-ups, patents Contribution to the economy 28

29. Problems with rankings A soccer team against a water polo team Emphasis on Technical Medicine Social sciences Fine arts We need area-specific rankings Business, Engineering, Law, Physics, … 29

30. 30