Urap 2014 Symposiums

1. 1
@UMultirank
U-Multirank: ideas and experiences
Prof. Dr. Frank Ziegele
Ankara, November 2014
Presented by:
www.umultirank.org

2. Part One
U-Multirank, why and how
Part Two
U-Multirank in practice
Part Three
Launching U-Multirank
Part Four
What’s to come
2
Agenda
@UMultirank

3. U-Multirank – why and how
3
@UMultirank
Agenda
Part One

4. •Rankings are here to stay
•They receive high public attention
•They have impacts on HEIs (obsession about “world class universities”)
•They have political impacts (e.g. on excellence initiatives, mergers, immigration policies; see EUA report)
•But until now there were no rankings that fulfilled certain functions
•The current rankings do not help to support decision making
•They don’t take into account diversity of stakeholders’ preferences and HEIs’ missions
@UMultirank #UMRinAthens
4
Why another ranking?

5. U-Multirank is a transparency tool for many stakeholders
•Helps students to identify study-programmes
•Assists university leaders in strategic management
•Serves policy-makers in developing higher education, research and innovation policies
•Assists business & industry in finding partnerships
•Serves society with accurate open access information
@UMultirank #UMRinAthens
5

6. 6
Teaching and learning
Research
Knowledge transfer
International orientation
Regional engagement
“University X is the fifth best in the world”
Multi-dimensional ranking: ranks on 30 individual indicators (performance measures) in five dimensions of performance
@UMultirank #UMRinAthens
A new instrument to compare
university performances

7. 7
“Indicator 1 counts for 20%, indicator 2 for 30%, etc., altogether this leads to a score of X”
No composite overall scores, transparency lost if scores are weighted / added
User-driven: user decides on areas of performance to compare (and on the kind of university to be compared) – YOUR WAY!
@UMultirank #UMRinAthens
A new instrument to compare
university performances

8. 8
ranking performances of universities as a whole
ranking performances in specific fields or disciplines
@UMultirank #UMRinAthens
A new instrument to compare
university performances

9. 9
The “global system” of higher education is characterised by an
enormous diversity in university profiles and missions
Diverse preferences and needs of students, university leaders, academics, business and industry, governments etc.
Different groups of stakeholders are interested in different types of performances by universities
But diversity is not yet transparent or understood
@UMultirank #UMRinAthens
Higher education needs this
new kind of comparison

10. @UMultirank
10
Quotes representing the
basic ideas
Dr. José Joaquín Brunner, Director Cátedra UNESCO de
Políticas Comparadas de Educación Superior, Chile,
“U-Multirank is a great instrument for students, faculty, the public and policy makers.”
Dr. Simon Marginson, Professor of International Higher Education at the
Institute of Education, The University of London,
“U-Multirank gives students and other higher education stakeholders much more information to work with in making considered decisions.”
target groups
informed decisions

11. @UMultirank
11
Quotes representing the
basic ideas
Prof. David D. Dill, Professor Emeritus of Public Policy
University of North Carolina, Chapel Hill, USA,
“U-Multirank permits students and academic staff to compare institutions of similar missions and types…”
institutional diversity, like with like
Fernando M. Galán, Vice-Chairperson of the European Students Union (ESU) “U-Multirank will be an extremely useful tool for students all over the world, being the first global ranking that includes in a serious manner the teaching and learning dimension.”
more than research

12. @UMultirank
12
Quotes representing the
basic ideas
Andreas Schleicher, Director Education and Skills and Special Advisor on Education Policy at the OECD,
“With a first-of-its-kind multi-dimensional approach to comparing institutional performance, U-Multirank is now putting students and the public into the driver’s seat of determining which universities are doing well on what and why.”
Professor Sir Howard Newby, Vice-Chancellor, University of Liverpool, UK,
“U-Multirank provides an elegant solution to a problem which undermines conventional rankings. It does not seek to weight different activities of a university but allows users to devise their own ranking according to what they believe is important."
multi-dimensional, user- driven

13. U-Multirank indicators show diversity of performance profiles
@UMultirank
Teaching and Learning
Institutional ranking
Field-based ranking
•Student-staff-ratio

•Graduation rate (BA and - separately - MA)


•Percentage of academic staff with PhD

•Percentage of students graduating within normative period (BA and –separately - MA)


•Rate of graduate employment


•Inclusion of work experience in degree programme

13

14. @UMultirank
Teaching and Learning – Student Satisfaction Indicators
Institutional ranking
Field-based ranking
Overall learning experience

Quality of courses & teaching

Organisation of the programme

Contact to teachers

Social climate

Facilities (libraries, laboratories, rooms, IT)

Research orientation of teaching /programme

Inclusion of work experience /practical elements

14
U-Multirank indicators show diversity of performance profiles

15. Research
Institutional ranking
Field-based ranking
•External research income (per fte academic staff)


•Doctorate productivity

•Total research publication output (per fte academic staff)*


•Art related output

•Field-normalised citation rate*


•Highly cited research publications *


•Interdisciplinary research publications*


•Research orientation of teaching (student survey)

•Number of post-doc positions

@UMultirank
15
U-Multirank indicators show diversity of performance profiles

16. @UMultirank
Knowledge Transfer
Institutional ranking
Field-based ranking
•Income from private sources (service contracts, consultancies, licenses, royalties, trials, etc.)


•Joint research publications with industry*


•Patents (per fte academic staff)


•Co-patents with industry (per fte academic staff)


•Number of spin-offs (average over three year period)

•Patent citations to research publications*


•Revenues from Continuous Professional Development

U-Multirank indicators show diversity of performance profiles
16

17. @UMultirank
International Orientation
Institutional ranking
Field-based ranking
•Educational programmes (BA/MA) in foreign language

•International orientation of degree programmes

•Opportunities to study abroad (student survey)

•Student mobility (composite of incoming, outgoing, joint degree students)


•Percentage of international academic staff


•Percentage of PhDs awarded to foreign students


•International joint research publications*


•International research grants


17
U-Multirank indicators show diversity of performance profiles

18. @UMultirank
Regional Engagement
Institutional ranking
Field-based ranking
•Percentage of graduates working in the region


•Student internships in regional enterprises


•Degree theses in cooperation with regional industry

•Regional joint research publications*


Income from regional sources


18
U-Multirank indicators show diversity of performance profiles

19. @UMultirank
19
and how does it really work?
Part 2: U-Multirank in practice
live presentation of web tool

20. Launching U-Multirank (findings)
20
@UMultirank
Agenda
Part Three

21. 21
universities are included with publicly available data
850
of the 879 universities have provided comprehensive data
500
faculties/departments are included in the four field based rankings (physics, electrical and mechanical engineering, business studies)
1,000
study programmes within these faculties are included
5,000
students completed the student satisfaction survey
60,000
70
countries have universities in U-Multirank
@UMultirank
U-Multirank 2014 is the most
comprehensive international data
comparison in higher education
Alex Usher, President of Higher Education Strategy Associates (HESA), Canada
“U-Multirank is the most global, most diverse and least reductive ranking ever.”

22. 22
Worldwide
coverage
In 2014, 62% of U-Multirank universities are from Europe, 17% from North America, 14% from Asia and 7% from Africa, Latin America and Oceania
PhD-awarding institutions
+
universities of applied sciences
small
+
large universities
specialised institutions
+
comprehensive universities
old (pre 1870)
+
young institutions
(after 1980)
@UMultirank
U-Multirank offers a global view
of a large variety of university
profiles

23. 23
Turkish universities in
U-Multirank – Full participation
Name of the University
Participation 2014
Bilkent U
Institutional, Business, Electrical Engineering
Hacettepe U
Institutional, Business, Electrical & Mechanical Engineering
Istanbul Aydin U
Business
Istanbul U
Institutional, Business, Electrical & Mechanical Engineering, Physics
Koç U
Institutional, Business, Electrical Engineering, Physics
Kocaeli U
Institutional, Business, Electrical & Mechanical Engineering, Physics
Mugla Sitki Kocman U
Institutional, Business, Physics
Sabanci U
Institutional, Business, Electrical & Mechanical Engineering
@UMultirank
+ 4 universities only with public data
+ 16 new registrations in 2015

24. 24
Results Institutional Ranking
2014 – Full participant
Name of the University
Nr of A’s
Teaching & Learning
Nr of A’s
Research
Nr of A’s
Knowledge
Transfer
Nr of A’s
International Orientation
Nr of A’s
Regional Engagement
Bilkent U
1 / 4
2 / 8
2 / 8
3 / 6
3 / 5
Hacettepe U
3 / 4
4 / 8
1 / 8
1 / 6
3 / 5
Istanbul U
0 / 4
1 / 8
2 / 8
1 / 6
4 / 5
Koç U
1 / 4
2 / 8
1 / 8
4 / 6
3 / 5
Kocaeli U
1 / 4
1 / 8
1 / 8
1 / 6
1 / 5
Mugla Sitki Kocman U
1 / 4
0 / 8
0 / 8
0 / 6
2 / 5
Sabanci U
3 / 4
4 / 8
3 / 8
2 / 6
1 / 5
Results: Webtool
@UMultirank

25. 25
Many univs. have specific strengths
No univ. has “across the board” “A” scores
Around 100 univs. have a wide range (>10) of “A” scores
Very few univs. with no “A” scores
On individual indicators performance differences between univs. are clearly visible
300 univs. never before seen in global rankings. Of these, 30 have more than 10 “A” scores
@UMultirank
U-Multirank shows a wide
distribution of “A” scores
“A” scores per univ.
% of univs.
0
2 %
1 to 5
43 %
6 to 10
43 %
11 to 15
10 %
16 to 22
2 %
Tom Boland, Chief Executive of the Higher Education Authority, Ireland,
“U-Multirank demonstrates very well that different universities have different kinds of strengths.”

26. 26
@UMultirank
The difference in university
profiles becomes visible
You can’t say that one university is better than the other!
You can‘t find the world‘s top 100!

27. And many special analyses…
27
Comparisons per university profile
Comparisons per dimension
Comparisons per country
Comparisons per indicator
Comparisons per field/discipline
@UMultirank
U-Multirank offers a wealth of
information and the potential
for many more analyses
Thomas D. Parker, Senior Associate at the Institute for Higher Education Policy, Washington, USA
“U-Multirank is the ‘thinking man’s ranking’. Until now, rankings have been somewhere between bad journalism and bad science, and we’re beginning to see much more serious work.”

28. U-Multirank also provides specific benefits to participating institutions
Basic benefits
•Visibility of institutions with different profiles, marketing use
•Visibility of specific profiles: tool to look for “similar” institutions
•Comparison of performance with like
•Allowing differentiated SW-analysis, evidence-based discussions both on institutional and on field level Additional analysis of own data
•Compared with the total sample
•Institutional & field-based data
•Detailed analysis of student survey Support for benchmarking processes
•Either for networks or organisations
•Multi-level: institutional and fields
@UMultirank
28

29. •cost of data collection differs largely (according to implemented data systems)
•many indicators without problems, high degrees of completeness
•difficult indicators: graduate unemployment, internships in the region, degree theses with local companies (also definition of “region”)
•some challenges with response rates to student surveys (parallel surveys, e.g. effort to use British NSS data); if response is too small we do not show data (statistical checks)
•intensive communication loops
•challenges of institutional structures (mergers, internal)
29
@UMultirank
Experiences with data
collection

30. What’s to come
30
@UMultirank
Agenda
Part Four

31. @UMultirank
31
U-Multirank will be
extended in 2015
+ 150 institutions
(at the moment >280 new regsitrations)
+ 3 new fields (psychology,
computer science and medicine)
Current sample of 879 institutions
U-Multirank 2015 release
in March 2015
next chance to join: register for 2016

32. @UMultirank
32
•For more information on U-Multirank visit us, at www.umultirank.org.
•Contact:
•info@umultirank.org
•+49 (0) 5241 9761 58
•Like/follow us:
• /U-Multirank
• @UMultirank
• @Umultirank
More information/Contact