Methodologie du classement international des institutions d enseignement superieur et de recherche elabore par l'Universite de Leiden (Pays Bas)

1. New developments in bibliometric methods:
evaluation, mapping, ranking
Ton van Raan
Atelier Bibliométrie de l’URFIST de Paris
23 mars 2012 – CNAM
Center for Science and Technology Studies (CWTS)
Leiden University

2. 2
Leiden University
oldest in the Netherlands, 1575
European League of Research Universities (LERU)
Within world top-100
12 Nobel Prizes
Leiden, historic city (2th, 11th C.), strong cultural
(arts, painting) & scientific tradition
one of the largest science parks in EU

3. Contents of this presentation:
* Bibliometric methodology:
• impact
• maps
* Leiden Ranking 2011-2012: new indicators
* Evaluation tools related to the Leiden Ranking

4. 4
WoS/Scopus sub-universe
journal articles only,> 1,000,000p/y
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” NetworkModels
due to Information Filtering
StefanoMossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI:10.1103/PhysRevLett.88.138701 PACS numbers:89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Networkstructure is critical in manycontexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the globalstructure of the networkas well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered bybeing linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely tobecome known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” NetworkModels
due to Information Filtering
StefanoMossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI:10.1103/PhysRevLett.88.138701 PACS numbers:89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Networkstructure is critical in manycontexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the globalstructure of the networkas well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered bybeing linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely tobecome known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” NetworkModels
due to Information Filtering
StefanoMossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI:10.1103/PhysRevLett.88.138701 PACS numbers:89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Networkstructure is critical in manycontexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the globalstructure of the networkas well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered bybeing linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely tobecome known.
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002
Truncation of Power Law Behavior in “Scale-Free” NetworkModels
due to Information Filtering
StefanoMossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our
model with empirical data for the World Wide Web and find agreement.
DOI:10.1103/PhysRevLett.88.138701 PACS numbers:89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
Web (WWW) [4,5] and the Internet [6]. Networkstructure is critical in manycontexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the globalstructure of the networkas well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered bybeing linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely tobecome known.
Refs > non-WoS
Total publ universe
non-WoS publ:
Books
Book chapters
Conf. proc.
Reports
Non-WoS journals

5. Aliens from Galaxy M35-
245 approach planet
Earth. They have access
to the Web of Science…..

6. pa1 pa2 pa3 pa4
pb1 pb2 pb3 pb4 pb5
pb2
pb1 pb3
pb5
pb4
pa2
pa1 pa4
pa3
Co-citation-network Bibliogr.coupl.network
Primary Citation Network
cited p <> cited p citing p <> citing p
citing p <> cited p

7. Primary network is a structure of different items
(e.g., citing publ <> cited publ; citing publ <> concepts)
In-degree primary network => received citations >
impact
Secondary network is a structure of similar items
(e.g., citing publ<> citing publ; concepts <> concepts)
Similarity in the secondary networks => co-
occurrence
map

8. 1. Bibliometric methodology: impact

9. networks leading, possibly, to different dynamics, e.g., for the initiation and spread of epidemics.
In the context of network growth, the impossibility of knowing the degrees of all the nodes comprising the network due to the filtering process—
and, hence, the inability to make the optimal, rational, choice—is not altogether unlike the “bounded rationality” concept of Simon [17].
Remarkably, it appears that, for the description of WWW growth, the preferential attachment mechanism, originally proposed by Simon [10],
must be modified along the lines of another concept also introduced by him—bounded rationality [17].
We thank R. Albert, P. Ball, A.-L. Barabási, M. Buchanan, J. Camacho, and R. Guimerà for stimulating discussions and helpful suggestions. We
are especially grateful to R. Kumar for sharing his data. We thank NIH/NCRR (P41 RR13622) and NSF for support.
[1] S. H. Strogatz, Nature (London) 410, 268 (2001).
[2] R. Albert and A.-L. Barabási, Rev. Mod. Phys. 74, 47 (2002).
[3] S. N. Dorogovtsev and J. F. F. Mendes, Adv. Phys. (to be published).
[4] R. Albert, H. Jeong, and A.-L. Barabási, Nature (London) 401, 130 (1999).
[5] B. A. Huberman and L. A. Adamic, Nature (London) 401, 131 (1999); R. Kumar et al., in Proceedings of the 25th
International Conference on
Very Large Databases (Morgan Kaufmann Publishers, San Francisco, 1999), p. 639; A. Broder et al., Comput. Netw. 33, 309 (2000); P. L.
Krapivsky, S. Redner, and F. Leyvraz, Phys. Rev. Lett. 85, 4629 (2000); S. N. Dorogovtsev, J. F. F. Mendes, and A. N. Samukhin, ibid. 85, 4633
(2000); A. Vazquez, Europhys. Lett. 54, 430 (2001).
[6] M. Faloutsos, P. Faloutsos, and C. Faloutsos, Comput. Commun. Rev. 29, 251 (1999); G. Caldarelli, R. Marchetti, and L. Pietronero,
Europhys. Lett. 52, 386 (2000); A. Medina, I. Matta, and J. Byers, Comput. Commun. Rev. 30, 18 (2000); R. Pastor-Satorras, A. Vazquez, and A.
Vespignani, arXiv:cond-mat/0105161; L. A. Adamic et al., Phys. Rev. E 64, 046135 (2001).
[7] F. B. Cohen, A Short Course on Computer Viruses (Wiley, New York, 1994); R. Pastor-Satorras and A. Vespignagni, Phys. Rev. Lett. 86,
3200 (2001); Phys. Rev. E 63, 066117 (2001).
[8] F. Liljeros, C. R. Edling, L. A. Nunes Amaral, H. E. Stanley, and Y. Åberg, Nature (London) 411, 907 (2001).
[9] A.-L. Barabási and R. Albert, Science 286, 509 (1999).
[10] Y. Ijiri and H. A. Simon, Skew Distributions and the Sizes of Business Firms (North-Holland, Amsterdam, 1977).
[11] G. Bianconi and A.-L. Barabasi, Europhys. Lett. 54, 436 (2001).
[12] A. F. J. Van Raan, Scientometrics 47, 347 (2000).
[13] We consider a modification to the network growth rule described earlier in the paper: at each time step t, the new node establishes m new
links, where m is drawn from a power law distribution with exponent gout.
[14] For n(I) = const, one recovers the scale-free model of Ref. [9].
[15] It is known [11] that, for an exponential or fat-tailed distribution of fitness, the structure of the network becomes much more complex; in
particular, the in-degree distribution is no longer a power law. Hence, we do not consider in this manuscript other shapes of the fitness
distribution.
[16] L. A. N. Amaral, A. Scala, M. Barthélémy, and H. E. Stanley, Proc. Natl. Acad. Sci. U.S.A. 97, 11 149 (2000).
[17] H. A. Simon, Models of Bounded Rationality: Empirically Grounded Economic Reason (MIT Press, Cambridge, 1997).
Citing Publications
Cited Publications
From other disciplines
From emerging fields
From research devoted to societal,
economical and technological problems
From industry
From international top-groups
These all f(t)!! > Sleeping Beauties

10. 1 2 3 4 5 6 7
a b c d e f g
Fig. 1: Citing and cited publications
Intermezzo: statistical properties of the primary network
Out-degree (all)= 1
In-degree (b) = 5

11. Large differences among research fields in citation
density
Citation counts must always be normalized to field-
specific values
and therefore:
H-index is not normalized and inconsistent

12. 500
Dept. Radiology
CPP = 6
Radiology journals
JCSm = 4
CPP/JCSm=1.5
P
2007-
2010
150,000
Radiology field
FCSm = 3
CPP/FCSm = 2
C
2007-
2010
3000 450,000
25,000
100,000

13. 13
• Two normalization mechanisms:
ci: actual number of citations of publication i
ei: expected number of citations of publication i
• Value of 1 indicates performance at world
average








 n
i i
n
i i
n
i i
n
i i
e
c
ne
nc
1
1
1
1
CPP/FCSm


n
i i
i
e
c
n 1
1
MNCS

14. Applied research, engineering,
social sciences
Basic research in natural
and medical science
high FCSm
Δ Citation density ~20 !
high FCSm, but
low JCSm
low FCSm
low FCSm, but
high JCSm
High CPP
low CPP

15. PHYSICS
0
5000
10000
15000
20000
25000
30000
35000
Cits91 Cits92 Cits93 Cits94 Cits95 Cits96 Cits97 Cits98 Cits99 Cits00 Cits01 Cits02
Publications from 1991,….1995
time lag & citation window

16. * SOCIOLOGY
0
500
1000
1500
2000
2500
Cits91 Cits92 Cits93 Cits94 Cits95 Cits96 Cits97 Cits98 Cits99 Cits00 Cits01 Cits02
Again: never use journal impact factors!!

17. University
Departments
Fields
‘bottom-up’ analysis: input data (assignment
of researchers to departments) necessary;
> Detailed research performance analysis of
a university by department
‘top-down’ analysis: field-structure is
imposed to university;
> Broad overview analysis of a university by
field

18. H = 49
H = 27
P C+sc
Dept of Radiology, UMC
1997 - 2009 1,045 20,552
1997 - 2000 196 616
1998 - 2001 213 938
1999 - 2002 243 1,151
2000 - 2003 231 1,590
2001 - 2004 221 1,576
2002 - 2005 306 2,097
2003 - 2006 358 2,516
2004 - 2007 417 3,368
2005 - 2008 492 4,254
2006 - 2009 497 3,986
CPP Pnc
16.73 17%
2.32 51%
3.43 38%
3.63 39%
5.61 36%
5.94 29%
5.72 40%
5.88 35%
6.82 35%
7.10 30%
6.42 30%
JCSm FCSm
13.63 9.84
2.16 1.92
3.30 2.60
4.00 3.00
4.78 3.14
4.34 3.03
3.94 2.79
3.71 2.65
4.37 3.03
5.08 3.73
5.08 3.52
CPP/ CPP/ JCSm/
JCSm FCSm FCSm
1.23 1.70 1.36
1.07 1.21 1.09
1.04 1.32 1.23
0.91 1.21 1.29
1.17 1.79 1.47
1.37 1.96 1.39
1.45 2.05 1.37
1.58 2.21 1.36
1.56 2.25 1.40
1.40 1.90 1.33
1.26 1.82 1.41
SelfCit
15%
26%
22%
23%
18%
17%
17%
16%
16%
18%
20%
Crown indicator

19. Impact trend of two depts. Radiology (UMC A and B)
0.00
0.50
1.00
1.50
2.00
2.50
3.00
1997 -
2000
1998 -
2001
1999 -
2002
2000 -
2003
2001 -
2004
2002 -
2005
2003 -
2006
2004 -
2007
2005 -
2008
2006 -
2009
years
CPP/FCSm
above
below international
level
P(2006-9)=561:
in which fields?

20. 2006 - 2009
Radiology UMC
RAD,NUCL MED IM
(1.63)
CARD&CARDIOV SYS
(2.24)
SURGERY (1.71)
CLIN NEUROLOGY
(2.15)
ONCOLOGY (1.55)
PERIPHL VASC DIS
(1.20)
PEDIATRICS (1.09)
MEDICINE,GEN&INT
(4.02)
NEUROSCIENCES
(2.71)
GASTROENTEROLOGY
(0.59)
FIELD
(CPP/FCSm)
Research Profile
Output (P) and Impact (CPP/FCSm)
2006 - 2009
Dept Radiology UMC

21. PUBLICATIONS AND IMPACT PER SUBFIELD
1992 - 2000
Erkrankungen des Nervensystems
0% 5% 10% 15% 20% 25% 30% 35% 40%
NEUROSCIENCES (2.28)
BIOCH & MOL BIOL (2.00)
CELL BIOLOGY (1.96)
DEVELOPMENT BIOL (1.33)
MULTIDISCIPL SC (2.84)
GENETICS & HERED (2.70)
PHYSIOLOGY (1.74)
CLIN NEUROLOGY (1.58)
PHARMACOL & PHAR (3.94)
ONCOLOGY (1.60)
BIOLOGY (2.06)
ANAT & MORPHOL (2.15)
ENDOCRIN & METAB (1.81)
PATHOLOGY (2.11)
UROLOGY & NEPHRO (1.30)
BIOPHYSICS (1.97)
IMMUNOLOGY (1.84)
MEDICINE, RES (1.88)
ZOOLOGY (4.51)
OPHTHALMOLOGY (2.21)
SUBFIELD
(CPP/FCSm)
Relative share of impact
1992 - 2000
Erkrankungen des Nervensystems
0% 10% 20% 30% 40% 50% 60%
NEUROSCIENCES (2.70)
BIOCH & MOL BIOL (1.92)
CELL BIOLOGY (2.74)
MULTIDISCIPL SC (4.33)
EVELOPMENT BIOL (1.32)
SUBFIELD
(CPP/FCSm)
Percentage of Publications
IMPACT: LOW AVERAGE HIGH
CPP/
FCSm
CPP/
FCSm
Diseases of the Neurosystem
Dept Output in Fields Dept Impact from Fields
Knowledge use by very
high impact groups
Neurosc 2.70
Biochem 1.92
Cell Biol 2.74
multidisc 4.33
Devel Biol 1.32
Neurosc 2.28
Biochem 2.00
Cell Biol 1.96
Devel Biol 1.33
multidisc 2.84
Genetics 2.70
Physiol 1.74
Clin Neurol 1.58
Pharmacol 3.94
Oncology 1.60
Biology 2.06
Anat Morph 2.15
Endocrinol 1.81
Pathology 2.11
Urology 1.30
Biophysics 1.97
Immunol 1.84
Res Medic 1.88
Zoology 4.51
Opthalmol 2.21

22. 2. Bibliometric methodology: maps

23. Major field
fields
Main field: Medical & Life Sciences
BIOMEDICAL SCIENCES
ANATOMY & MORPHOLOGY
IMMUNOLOGY
INTEGRATIVE & COMPLEMENTARY MEDICINE
MEDICAL LABORATORY TECHNOLOGY
MEDICINE, RESEARCH & EXPERIMENTAL
NEUROIMAGING
NEUROSCIENCES
PHARMACOLOGY & PHARMACY
PHYSIOLOGY
RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING
TOXICOLOGY
VIROLOGY
ACTA GENETICAE MEDICAE ET GEMELLOLOGIAE
AMERICAN JOURNAL OF HUMAN GENETICS
AMERICAN JOURNAL OF MEDICAL GENETICS
ANIMAL BLOOD GROUPS AND BIOCHEMICAL GENETICS
ANNALES DE GENETIQUE
ANNALES DE GENETIQUE ET DE SELECTION ANIMALE
ANNALS OF HUMAN GENETICS
ATTI ASSOCIAZIONE GENETICA ITALIANA
BEHAVIOR GENETICS
BIOCHEMICAL GENETICS
CANADIAN JOURNAL OF GENETICS AND CYTOLOGY
CANCER GENETICS AND CYTOGENETICS
CARYOLOGIA
CHROMOSOMA
CLINICAL GENETICS
CURRENT GENETICS
CYTOGENETICS AND CELL GENETICS
CYTOLOGIA
DEVELOPMENTAL GENETICS
ENVIRONMENTAL MUTAGENESIS
EVOLUTION
GENE
GENETICA
GENETICA POLONICA
GENETICAL RESEARCH
GENETICS
GENETIKA
HEREDITAS
HEREDITY
HUMAN BIOLOGY
HUMAN GENETICS
HUMAN HEREDITY
IMMUNOGENETICS
INDIAN JOURNAL OF GENETICS AND PLANT BREEDING
JAPANESE JOURNAL OF GENETICS
JAPANESE JOURNAL OF HUMAN GENETICS
JOURNAL DE GENETIQUE HUMAINE
JOURNAL OF HEREDITY
JOURNAL OF IMMUNOGENETICS
JOURNAL OF MEDICAL GENETICS
JOURNAL OF MENTAL DEFICIENCY RESEARCH
JOURNAL OF MOLECULAR EVOLUTION
MOLECULAR & GENERAL GENETICS
MUTATION RESEARCH
PLASMID
SILVAE GENETICA
THEORETICAL AND APPLIED GENETICS
THEORETICAL POPULATION BIOLOGY
EGYPTIAN JOURNAL OF GENETICS AND CYTOLOGY
REVISTA BRASILEIRA DE GENETICA
ANNUAL REVIEW OF GENETICS
JOURNAL OF CRANIOFACIAL GENETICS AND DEVELOPMENTAL BIOLOGY
JOURNAL OF INHERITED METABOLIC DISEASE
PRENATAL DIAGNOSIS
ADVANCES IN GENETICS INCORPORATING MOLECULAR GENETIC MEDICINE
CHEMICAL MUTAGENS-PRINCIPLES AND METHODS FOR THEIR DETECTION
DNA-A JOURNAL OF MOLECULAR & CELLULAR BIOLOGY
EVOLUTIONARY BIOLOGY
TERATOGENESIS CARCINOGENESIS AND MUTAGENESIS
TSITOLOGIYA I GENETIKA
ADVANCES IN HUMAN GENETICS
PROGRESS IN MEDICAL GENETICS
GENETICS SELECTION EVOLUTION
MOLECULAR BIOLOGY AND EVOLUTION
SOMATIC CELL AND MOLECULAR GENETICS
BIOTECHNOLOGY & GENETIC ENGINEERING REVIEWS
EXPERIMENTAL AND CLINICAL IMMUNOGENETICS
GENE ANALYSIS TECHNIQUES
JOURNAL OF MOLECULAR AND APPLIED GENETICS
JOURNAL OF NEUROGENETICS
TRENDS IN GENETICS
DISEASE MARKERS
ANIMAL GENETICS
GENETIC EPIDEMIOLOGY
JOURNAL OF GENETICS
journals
Citation relations between journals are
used to define fields of science

24. Ja1 Ja2 Ja3 Ja4
Jb1 Jb2 Jb3 Jb4 Jb5
Ja2
Ja1 Ja4
Ja3
Bibliogr.coupl.
Journal map based on citation similarity

25. Colors indicate the different subfields
Radiology, Nuclear Medicine, & Medical Imaging
111 journals, 2007-2010
Citation-based network map

26. Journal
1. Field = set of journals
Defined by WoS or Scopus
+ reference-based re-definition
(expansion) of fields
(e.g. Nature, Science)

27. Fa1 Fa2 Fa3 Fa4
Fb1 Fb2 Fb3 Fb4 Fb5
Fa2
Fa1 Fa4
Fa3
Bibliogr.coupl.
Field map based on citation similarity

28. Size: World average
Science as a structure of 200
related fields
World map
Colors indicate the different fields

29. 29
Leiden University

30. 30
Delft University of Technology

31. Instead of using citations (references) to
calculate publication-similarity and to create
citation-based maps
one can in exactly the way mathematical way
use concepts (keywords extracted from titles
and abstracts) to calculate publication-
similarity and to create
concept-based maps

32. 2.Field = clusters of concept-related
publications
own definition: new, emerging,
interdisciplinary fields,
socio-economic themes
Evaluation of universities’ ‘third
mission’:
Role of science for societal demands

33. Urban Studies 2006-2008
Concept map, n=700
Colors indicate the different subfields

34. Urban Studies 2006-2008
Concept density map
n=700
Colors indicate the different publication densities

35. Map 2005
Map 2010 Knowledge dynamics described by
density and flow
?
Map 2015

36. Serious problem in impact assessment by citations
mentioned by more and more researchers:
Normalization by an average citation density (FCSm)
of the whole field might seriously prejudice groups
working in a low density subfield
How can we measure these within-field citation
densities and determine how serious the problem is?

37. Combining concept-based maps with local
citation-density measurement

38. Clinical Neurology
2005-2009
Colors now indicate local citation density -->
Neuro-surgery has a relatively low
citation density within the field of
clinical neurology

39. Cardiovasc system 2005-2009
Concept map, n=1,500
Colors now indicate local citation density

40. 3. Leiden Ranking 2011-2012:
new indicators

41. New and innovative features!
No static tables but a user-friendly menu with
(1) selection of universities
(2) selection of performance dimension with
specific indicators
(3) selection of calculation method
for the 500 largest universities worldwide

42. Proper normalization is absolutely crucial!
Rank of all 250 universities by FCSm
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
0 50 100 150 200 250 300
r
FCSm
Ranking of the 250 largest European universities by FCSm

43. www.leidenranking.com

44. (1) selection of universities
Region:
* World
* Regions (Europe, Asia, North America, South
America, Africa, Oceania)
* Countries
By number of universities covered in the ranking:
*largest 100, 200, 300, 400, 500

45. (2) selection of performance
dimension with specific indicators
* impact: P, MCS, MNCS, PP(top10%)
* collaboration: P, PP(collab), PP(int collab),
MGCD, PP(long dist collab)
indicators with stability intervals!

46. MNCS (earlier CPP/FCSm) is an average
and this is statistically not the best measure for a
skew distribution…
So we move on to p-top10%
this measures takes the whole distribution into
account
Notice: dimension of MNCS is C/P
dimension of p-top10% is n

47. (3) selection of calculation method
* full or fractional assignment collaborative
publications
* all WoS publications or only English language

48. Observations

49. General:
* Large effect of all vs only-English publications for
D and F
* Large effect full vs fractional counting
* One-paper-explosion effect
* International collaboration

50. Stability intervals clearly show the dramatic
influence of just one publication!
Examples: Göttingen, Utrecht

51. P[2008]; C[2008-2011*] = 20,485

52. P[2009]; C[2009-2011*] = 2,856

53. The influence of Arts & Humanities on
bibliometric university rankings is
practically non-existent….

54. Leave out non-English publicationsx
Rank based on
ALL WoS
publications

55. Very
influential for
D and F univ

56. EUR-50 / language: all / counting: full
P MNCS
1 Univ Cambridge 25073 1.76
2 Univ Oxford 26161 1.71
3 ETH Zurich 15890 1.63
4 Imperial Coll London 22280 1.58
5 Univ Edinburgh 13318 1.57
6 Utrecht Univ 18221 1.56
7 Erasmus Univ Rotterdam 12788 1.54
8 Univ Bristol 12199 1.53
9 Univ Coll London 26210 1.51
10 Univ Zurich 14492 1.50
11 King's Coll London 13502 1.46
12 VU Univ Amsterdam 12732 1.46
13 Leiden Univ 12653 1.45
14 Univ Copenhagen 18654 1.43
15 Radboud Univ Nijmegen 12061 1.43
16 Univ Amsterdam 15760 1.43
17 Karolinska Inst 17054 1.41
18 Aarhus Univ 13039 1.41
19 LMUniv München 17672 1.40
20 Katholieke Univ Leuven 19673 1.38
21 Lund Univ 15558 1.37
22 Univ Manchester 18217 1.37
23 Tech Univ München 12452 1.36
24 Univ Groningen 12303 1.36
25 Univ Southampton 10861 1.36
EUR-50 / language: Engl / counting: full
P MNCS
1 Univ Cambridge 25037 1.71
2 Univ Oxford 26105 1.65
3 ETH Zurich 15708 1.60
4 Imperial Coll London 22234 1.53
5 Univ Edinburgh 13291 1.51
6 Utrecht Univ 18066 1.51
7 Univ Zurich 13840 1.50
8 LMUniv München 15827 1.48
9 Univ Bristol 12170 1.47
10 Erasmus Univ Rotterdam 12738 1.47
11 Univ Coll London 26166 1.45
12 Univ Glasgow 10144 1.41
13 VU Univ Amsterdam 12697 1.40
14 Leiden Univ 12597 1.40
15 Heidelberg Univ 14552 1.40
16 King's Coll London 13473 1.40
17 Tech Univ München 11660 1.39
18 Radboud Univ Nijmegen 12000 1.38
19 Univ Copenhagen 18633 1.37
20 Univ Amsterdam 15689 1.37
21 Univ Pierre & Marie Curie 19206 1.37
22 Karolinska Inst 17025 1.35
23 Univ Paris-Sud 11 13674 1.35
24 Aarhus Univ 13020 1.35
25 Univ Bonn 10035 1.34
26 Katholieke Univ Leuven 19540 1.34
27 Lund Univ 15528 1.33
28 Univ Manchester 18194 1.32
29 Univ Groningen 12256 1.32
30 Univ Southampton 10843 1.32
30
All <> Eng
MNCS:
D, F
UK

57. Citation Impact German and French universities
All versus English-only Publications
y = 1,08x
R2
= 0,81
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
1,60
1,80
2,00
0,00 0,20 0,40 0,60 0,80 1,00 1,20 1,40 1,60 1,80 2,00
imp-all
imp-eng
Citation Impact Ranking German and French universities
All versus English-only Publications
y = 1,02x - 38,31
R2
= 0,83
0
50
100
150
200
250
300
350
400
450
500
0 50 100 150 200 250 300 350 400 450 500
r-all
r-eng

58. EUR-50 / language: Engl / counting: full
P MNCS
1 Univ Cambridge 25037 1.71
2 Univ Oxford 26105 1.65
3 ETH Zurich 15708 1.60
4 Imperial Coll London 22234 1.53
5 Univ Edinburgh 13291 1.51
6 Utrecht Univ 18066 1.51
7 Univ Zurich 13840 1.50
8 LMUniv München 15827 1.48
9 Univ Bristol 12170 1.47
10 Erasmus Univ Rotterdam 12738 1.47
11 Univ Coll London 26166 1.45
12 Univ Glasgow 10144 1.41
13 VU Univ Amsterdam 12697 1.40
14 Leiden Univ 12597 1.40
15 Heidelberg Univ 14552 1.40
16 King's Coll London 13473 1.40
17 Tech Univ München 11660 1.39
18 Radboud Univ Nijmegen 12000 1.38
19 Univ Copenhagen 18633 1.37
20 Univ Amsterdam 15689 1.37
21 Univ Pierre & Marie Curie 19206 1.37
22 Karolinska Inst 17025 1.35
23 Univ Paris-Sud 11 13674 1.35
24 Aarhus Univ 13020 1.35
25 Univ Bonn 10035 1.34
26 Katholieke Univ Leuven 19540 1.34
27 Lund Univ 15528 1.33
28 Univ Manchester 18194 1.32
29 Univ Groningen 12256 1.32
30 Univ Southampton 10843 1.32
EUR-50 / language: Engl / counting: frac
P MNCS
1 Univ Cambridge 14065 1.57
2 ETH Zurich 8632 1.55
3 Univ Oxford 13949 1.49
4 Utrecht Univ 9710 1.49
5 Imperial Coll London 11567 1.41
6 Univ Bristol 6681 1.36
7 Univ Coll London 13745 1.36
8 Univ Edinburgh 7142 1.32
9 VU Univ Amsterdam 6494 1.31
10 Erasmus Univ Rotterdam 6765 1.31
11 Univ Zurich 7754 1.29
12 Univ Amsterdam 8104 1.29
13 King's Coll London 7249 1.28
14 Leiden Univ 6355 1.27
15 Univ Copenhagen 10157 1.25
16 Aarhus Univ 7069 1.24
17 Univ Manchester 10427 1.23
18 Radboud Univ Nijmegen 6478 1.23
19 Tech Univ München 6672 1.22
20 Univ Groningen 6967 1.22
21 Univ Nottingham 6839 1.20
22 Karolinska Inst 8573 1.20
23 Univ Southampton 6017 1.19
24 Katholieke Univ Leuven 10956 1.19
25 Univ Sheffield 6440 1.19
26 LMUniv München 9585 1.16
27 FAUniv Erlangen 5756 1.16
28 Univ Paris-Sud 11 6323 1.15
29 Univ Leeds 6627 1.15
30 Univ Pierre & Marie Curie 9309 1.14
full <>frac
MNCS:
All:
Major changes
for the smaller
universities

59. 15
29
60!
For the smaller universities
fractional counting is quite influential,
particularly for Delft!
EUR-100 / language: Engl / counting: frac
P MNCS
1 Univ Göttingen 4776 2.04
2 EPF Lausanne 4790 1.69
3 Univ Cambridge 14046 1.53
4 ETH Zurich 8507 1.53
5 Utrecht Univ 9601 1.45
6 Univ Oxford 13919 1.44
7 Tech Univ Denmark 4287 1.43
8 Imperial Coll London 11547 1.37
9 Univ Bristol 6671 1.31
10 Univ Basel 4263 1.31
11 Univ Coll London 13723 1.31
12 Univ Zurich 7293 1.30
13 Wageningen Univ 4590 1.30
14 Univ Edinburgh 7128 1.28
15 VU Univ Amsterdam 6471 1.27
16 LMUniv München 8261 1.27
17 Tech Univ München 6145 1.26
18 Erasmus Univ Rotterdam 6735 1.26
19 Univ Geneva 4971 1.25
20 Univ Amsterdam 8059 1.24
21 Delft Univ Technol 4531 1.24
22 King's Coll London 7236 1.23
23 Leiden Univ 6321 1.23
24 FAUniv Erlangen 5198 1.22
25 Univ Freiburg 4768 1.22
26 Univ Bern 4717 1.21
27 Univ Pierre & Marie Curie 8416 1.21
28 Karlsruhe Inst Technol 4362 1.21
29 Univ Copenhagen 10145 1.20
30 Aarhus Univ 7060 1.19

60. EUR-100 / language: Engl / counting: frac
P MNCS
1 Univ Göttingen 4776 2.04
2 EPF Lausanne 4790 1.69
3 Univ Cambridge 14046 1.53
4 ETH Zurich 8507 1.53
5 Utrecht Univ 9601 1.45
6 Univ Oxford 13919 1.44
7 Tech Univ Denmark 4287 1.43
8 Imperial Coll London 11547 1.37
9 Univ Bristol 6671 1.31
10 Univ Basel 4263 1.31
11 Univ Coll London 13723 1.31
12 Univ Zurich 7293 1.30
13 Wageningen Univ 4590 1.30
14 Univ Edinburgh 7128 1.28
15 VU Univ Amsterdam 6471 1.27
16
Ludwig-Maximilians-Univ
Münche 8261 1.27
17 Tech Univ München 6145 1.26
18 Erasmus Univ Rotterdam 6735 1.26
19 Univ Geneva 4971 1.25
20 Univ Amsterdam 8059 1.24
21 Delft Univ Technol 4531 1.24
22 King's Coll London 7236 1.23
23 Leiden Univ 6321 1.23
24 FAUniv Erlangen 5198 1.22
25 Univ Freiburg 4768 1.22
26 Univ Bern 4717 1.21
27 Univ Pierre & Marie Curie 8416 1.21
28 Karlsruhe Inst Technol 4362 1.21
29 Univ Copenhagen 10145 1.20
30 Aarhus Univ 7060 1.19
EUR-100 / language: Engl / counting: frac
P
PPtop
10%
1 EPF Lausanne 4790 18.8%
2 ETH Zurich 8507 17.6%
3 Univ Cambridge 14046 16.7%
4 Univ Oxford 13919 16.5%
5 Tech Univ Denmark 4287 15.9%
6 Imperial Coll London 11547 15.2%
7 Univ Coll London 13723 14.8%
8 Univ Edinburgh 7128 14.4%
9 Wageningen Univ 4590 14.3%
10 Univ Zurich 7293 14.3%
11 Univ Bristol 6671 14.2%
12 Erasmus Univ Rotterdam 6735 14.2%
13 VU Univ Amsterdam 6471 14.1%
14 Univ Basel 4263 14.1%
15 Univ Geneva 4971 13.9%
16 LMUniv München 8261 13.8%
17 Utrecht Univ 9601 13.6%
18 Tech Univ München 6145 13.5%
19 Leiden Univ 6321 13.3%
20 Univ Amsterdam 8059 13.2%
21 Univ Freiburg 4768 13.1%
22 King's Coll London 7236 13.1%
23 Delft Univ Technol 4531 12.9%
24 Aarhus Univ 7060 12.9%
25 Univ Paris-Sud 11 5866 12.9%
26 FAUniv Erlangen 5198 12.8%
27 Univ Copenhagen 10145 12.8%
28 Karlsruhe Inst Technol 4362 12.7%
29 Univ Pierre & Marie Curie 8416 12.7%
30 Univ Glasgow 5262 12.6%62 !!
MNCS > p-top10%:
Universities with rank
position dominated by
just 1 publication
change considerably in
position

61. 500 largest universities worldwide
P[2005-2009], C[2005-2010]
y = 0.14x - 0.04
R
2
= 0.97
0
0.05
0.1
0.15
0.2
0.25
0.3
0 0.5 1 1.5 2 2.5
MNCS-engl
pp(top10%)

62. EUR-50 / language: Engl / counting: frac
P PPint collab
1 ETH Zurich 8507 36.5%
2 Univ Zurich 7293 36.0%
3 Karolinska Inst 8561 35.4%
4 Univ Paris-Sud 11 5866 34.3%
5 Katholieke Univ Leuven 10872 33.9%
6 Aarhus Univ 7060 33.1%
7 Univ Pierre & Marie Curie 8416 32.6%
8 Univ Copenhagen 10145 32.2%
9 Lund Univ 8473 32.0%
10 Univ Oslo 6864 31.4%
11 Univ Helsinki 7988 31.3%
12 Univ Oxford 13919 31.1%
13 Uppsala Univ 6928 30.9%
14 Imperial Coll London 11547 30.5%
15 Leiden Univ 6321 30.1%
16 Univ Cambridge 14046 29.2%
17 VU Univ Amsterdam 6471 28.8%
18 Univ Coll London 13723 28.6%
19 Humboldt-Univ Berlin 6441 28.5%
20 Univ Amsterdam 8059 28.4%
21 Charles Univ Prague 5396 28.2%
22 Univ Edinburgh 7128 27.9%
23 Ghent Univ 8935 27.8%
24 Tech Univ München 6145 27.7%
25 LMUniv München 8261 27.6%
Collaboration and, more specifically,
international collaboration:
* no correlation with impact
* weak correlation with MGCD (see Leiden
Ranking website)

63. Correlation of p-int-collab with MGCD
for the 100 largest European universities
y = 0.01x
0.56
R
2
= 0.47
0.100
1.000
1000 10000
MGCD
p-int-collab

64. Correlation of p-int-coll with MGCD
for Australia
y = 0,00x0,92
R2
= 0,94
0,10
1,00
1000,00 10000,00
MGCD
p-int-coll

65. 4. Evaluation tools related to the Leiden
Ranking

66. Built on the basic data of the Leiden Ranking
2011:
Leiden Benchmark Analysis:
* all indicators of the ‘total’ university ranking are available for 30
main (NOWT) fields of science
* their trends in the past 10 years
* world rank by indicator and by main field
All this in comparison with 20 other universities by choice
* all indicators for each field within each main field (total 200 fields)
with distinction between fields above and below university average

67. Current and recent benchmark
projects
Manchester, Leiden, Heidelberg,
Rotterdam, Copenhagen, Zürich,
Lisbon UNL, Amsterdam UvA,
Amsterdam VU, Southampton
Gent, Antwerp, Brussels VUB,
UC London, Aarhus, Oslo

68. Output & impact compared to benchmark universities
2002 - 2007
CHEMISTRY
Zurich
UC London
Strasbourg I
Paris XI
Paris VI
Oxford
Milano
Lunds
LMU Munchen
Karolinska
KU Leuven
Helsinki
Heidelberg
Geneve
Freiburg
Edinburgh
Cambridge
Wageningen
Utrecht
Twente
TU Delft
Nijmegen
Leiden
Groningen
Eindhoven
VU Adam
UvA
0
0.5
1
1.5
2
0 500 1000 1500 2000 2500
TOTAL PUBLICATIONS
CPP/FCSm

69. Output and impact 2002 - 2007
SOCIAL SCIENCES
Zurich
UC London
Paris XI
Paris VI
Oxford
Milano
Lund
LMU Munchen
Karolinska
KU Leuven
Helsinki
Heidelberg
Geneve
Freiburg
Edinburgh
Cambridge
Wageningen
Utrecht
Twente
Tilburg
TU Delft
Nijmegen
Maastricht
Leiden
Groningen
Erasmus
Eindhoven
VU Adam
UvA
0
0,2
0,4
0,6
0,8
1
1,2
1,4
0 200 400 600 800 1000 1200 1400
TOTAL PUBLICATIONS
CPP/FCSm

70. FIGURE VII.12:
RESEARCH AND IMPACT PROFILE COMPARISON CHART
2001 - 2008
0.91
1.15
0.83
1.52
1.45
1.14
1.08
1.41
1.20
1.44
1.42
1.15
1.27
1.25
1.55
0.92
1.37
1.19
1.17
4.70
1.11
1.17
1.10
1.20
1.25
1.20
1.62
0.84
1.06
1.01
1.95
1.10
0.88
1.34
2.00
1.01
1.20
1.13
1.25
2.12
1.17
0.98
1.31
1.98
1.33
1.25
1.00
1.46
2.04
2.44
0.98
1.04
0.70
1.25
1.12
1.10
0.97
0.87
0.94
1.17
7% 5% 3% 1% 1% 3% 5% 7%
ONCOLOGY
BIOCHEM&MOL BIOL
IMMUNOLOGY
CARD&CARDIOV SYS
PUBL ENV OCC HLT
ASTRON&ASTROPH
NEUROSCIENCES
PSYCHIATRY
HEMATOLOGY
CELL BIOLOGY
MEDICINE,GEN&INT
CLIN NEUROLOGY
ENDOCRIN&METABOL
PERIPHL VASC DIS
CHEM,PHYSICAL
SURGERY
GEOSC,MULTIDISC
ECOLOGY
DENT,ORAL SURG&M
PEDIATRICS
GENETICS&HEREDIT
OBSTETRICS&GYNEC
GASTROENTEROLOGY
PHYSICS,COND MAT
PHARMACOL&PHARMA
PHYSICS,MULTIDIS
NUTRITION&DIET
RAD,NUCL MED IM
PATHOLOGY
MEDICINE,RES&EXP
Percentage of Total Publication Output
IMPACT: LOW AVERAGE HIGH
UNIV OSLO LUNDS UNIV

71. Leiden University
Major field field P(2005-9) MNCS
P>100 CLINICAL MEDICINE MEDICINE, GENERAL & INTERNAL 291.2 3.03
rank by PHYSICS AND ASTRONOMY PHYSICS, MULTIDISCIPLINARY 192.5 2.65
MNCS PHYSICS AND ASTRONOMY PHYSICS, CONDENSED MATTER 204.0 2.10
CLINICAL MEDICINE RESPIRATORY SYSTEM 122.7 1.91
CLINICAL MEDICINE RHEUMATOLOGY 349.0 1.90
CLINICAL MEDICINE CARDIAC & CARDIOVASC SYSTEMS 488.5 1.84
BIOL SCI: HUMANS MEDICINE, RESEARCH & EXPER 100.0 1.81
CLINICAL MEDICINE SURGERY 207.9 1.73
CLINICAL MEDICINE UROLOGY & NEPHROLOGY 158.0 1.67
MOLECULAR BIOL & BIOCHEM BIOTECH & APPLIED MICROBIOL 106.8 1.66
SOCIAL SC RELATED TO MED PUBLIC, ENVIRONM & OCC HEALTH 111.6 1.65
PHYSICS AND ASTRONOMY ASTRONOMY & ASTROPHYSICS 814.5 1.65
BIOL SCI: HUMANS MICROBIOLOGY 179.3 1.58
CLINICAL MEDICINE GENETICS & HEREDITY 394.9 1.57
CLINICAL MEDICINE PERIPHERAL VASCULAR DISEASE 268.0 1.51
Leiden average MNCS = 1.45

72. It appears that the medical fields benefit
more and the engineering fields less
from the full counting
This will be very influential in the relative
position of the engineering fields within
a university, particularly in performance
analyses

73. Shanghai and THES??
Use the Leiden Ranking
from now on!
What !!??

74. Thank you for your attention
more information: www.cwts.leidenuniv.nl