Journal ranking metrices new perspective in journal performance management
1. Journal Ranking Metrics
new perspective in Journal performance
management
Professor Aboul Ella
Hassanien
Cairo Univesrity
Faculty of Computers and Information
Chairman of Scientific Research Group in Egypt (SRGE)
http://www.egyptscience.net @ Aboitcairo@gmail.com
2. Using the Impact factor alone to judge
a journal is like using weight to judge
a person health
The title is a sentence taken from a report on citation
statistics prepared by the International Mathematical Union (IMU).
Another such take-home message is "Research is too important to
measure its value with only a single coarse tool." Given that citation
statistics are heavily used in assessing research and researchers,
young scientists might gain a lot from investing some time to
familiarize themselves with this subject.
3. Science is a game
Science is a game and the
best players appreciate not
only the beauty of a
discovery by a precise and
logical reasoning, but also the
taste of making a guess
There is no way to get
experience except through
experience
4. Writing a paper is …
A lot like chip
design
You need
Conceptualization
Floor planning and
layout
Interconnections
Testing
5. Why write and publish research
papers?
Ideally
to share research
findings and
discoveries with the
hope of improving
some things.
Practically –
to get funding
to get promoted
to get a job
to keep your job!
6. Academic publishing
Academic publishing describes a system that
is necessary in order for academic scholars to
peer review the work and make it available
for a wider audience
7. Multiple Trends Impact The Research
Landscape
Competition / Collaboration
Accountability /
Government policies
Improve
research
outcomes
Multidisciplinarity
High mobility
Research
Strategy
&
Research
Performance
8. What makes a good
research paper?
Good science
Good writing
Publication in good journals
9. Deadly Sins - Plagiarism
"Plagiarism is defined as the act of taking the specific substance
of another and offering it as one's own without giving credit to the
sources.
10. Major Metrics: Research Impact
Publication count
Number of publications produced by individual,
school or university
Citation count
Number of times publication is cited by other
publications
11. Major Metrics: Journal quality
Impact factor –
average number of times
published papers are cited
up to two years after
publication
Immediacy Index
average number of times
published papers are cited
during year of publication.
12. Journal Evaluation
Some challenges
I’m a publishing author in a niche
area, how can I compare with
researchers in another scientific field?
How can I get a value that reflects
citation behaviour in my research
area?
It’s 2010 – why can I only get a
ranking relevant to 2008?
Where can I find the data that have
been used to generate this number?
The Impact Factor does not cover the
journal I’m publishing in. What now?
Difficult to compare
A Not recent enough
Lack of clarity of data
origin
Lack of coverage
14. Journal Impact Factors:
Why?
Evaluate the scholarly worth of a journal
Rank journals within a discipline
Help you decide where to publish your article for
maximum impact
Evaluation for promotion / tenure / grants, or in
some countries, even government funding of an
institution
Frequently used as an evaluation source by
librarians during journal cancellations or new
purchases
15. Journal Impact Factors:
Why?
In Egypt, researchers in Cairo University are
rewarded for publishing in journals defined
by ISI as prestigious
In England, hiring panels routinely consider
impact factors
By Spanish law, researchers are rewarded for
publishing in journals defined by ISI as
prestigious
In China, scientists get cash bonuses for
publishing in high-impact journals. In some
schools, physics students must publish at least 2
articles with a combined Impact Factor of 4 to
get their PhD
16. Journal Impact Factors:
Where?
Impact factors are listed in Journal Citation
Reports JCR
You can easily get to the JCR from the Web of
Science, so let’s start there, since understanding the
Web of Science will help us better understand where
the data for the JCR comes from.
Tip: Use Web of Science for article-level
information; JCR for journal-level information. Both
the Web of Science and the JCR are based on the
same database of journal citations and cited
references
17. Web of Science®
Thomson Reuters (formerly ISI) has one, huge
database, Web of Science, that…
Indexes selected journals
> 8,000 science; > 3,000 social science journals;
> 1,800 Arts & Humanities
Tracks “cited references” and “times cited”
Sample topic/author search: “impact factor”
Activity: Search for an article in your field that
has been highly cited.
18. Journal Citation Reports
Journal Citation Reports (JCR) keeps track of citations at the
journal level (same data as in Web of Science, just a different
presentation of the data). A new edition comes out every Spring.
Tip: In addition to accessing the JCR via a Web of Science
citation, you can access the JCR from our Online Journals &
Databases resource: openurl.library.uiuc.edu/sfxlcl3/az?
Activity: Get into the JCR and then:
Choose the Science or Social Sciences edition, and year
Search by individual journal title, or by subject category
Example:
Subject: Biochemistry & Molecular Biology
Then, “View Journal Data”, sorted by Impact Factor
19. Journal Impact Factors:
What?
What is the Journal Impact Factor?
How is it calculated?
Only references in articles within the MORE than 13,000 journals
indexed in Web of Science are counted; does not include citations
that may cite the articles in Cell from book chapters, proceedings,
or other journals that are not indexed in Web of Science
Citable articles are just research articles and reviews – not news
articles, commentary, etc.
20. 20
Impact Factor =
Cites in 2009 to 2007 or 2008 papers
Papers published in 2007 or 2008
The impact factor is calculated by dividing the number of
citations in the JCR year by the total number of articles
published in the two previous years.
Impact Factor Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
21. 21
The immediacy index is calculated by dividing the number
of citations to articles published in a given year by the
number of articles published in that year.
Immediacy Index Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
22. EFFICIENCY JOURNAL IMPACT
FACTOR
2008 Impact
Factor
200820072006
Source paper – published in 2008
Cited reference – published in 2006 or
2007
Citations
All
Previous
Years
2005 2009
23. 23
Implications of Impact Factor
Tell us how frequently has the average article in a journal
been cited in a particular year.
Tell us something about a journal as a whole
e.g. the extent to which its recently published papers were
cited in a given year.
Impact factor > 1 implied a journal is frequently cited
Higher citations rate means your article has higher
chances of getting cited or read by researchers.
Tells us NOTHING concrete about any specific paper or
specific author.
24. Implications of Impact Factor
Only a limited subset of journals is indexed by ISI
Only uses the articles cited by the ~13,000 “ISI journals”
Some disciplines are especially poorly covered
Biased toward English-language journals
ISI has recently added several hundred non-English journals
Short (two year) snapshot of journal
Some disciplines use older material more or take time to cite new
research
JCR now also includes the 5-year data
Is an average; not all articles are equally well-cited
25. Implications of Impact Factor
Includes self-citations, that is articles in which the
article cites other papers in the same journal
Only includes “citable” articles in the denominator of
the equation, i.e., articles and reviews
Editors may skew IF by increasing the number of review
articles, which bring in more citations (increases the
numerator)
Or by increasing the number of “news” items (e.g., Science,
general medical journals) , which are cited (appear in
numerator) but not considered “citable” (and so aren’t in the
denominator)
It is expensive to subscribe to the JCR
http://www.slideshare.net/danrholden/bibliometrics-
12605810?next_slideshow=1
26. Publish or perish (POP)
Publish or Perish is a free analysis software that retrieves and
analyzes academic citations.
The results are available on-screen and can also be copied to the Windows clipboard
(for pasting into other applications) or saved to a variety of output formats
30. Scopus - Most comprehensive Abstracts and
Citations database
Increased
prestige,
ranking and
funding
because you . .
.
. . . improve
your
institutional
productivity . .
.
. . . by
enabling the
management
to make
better
decisions . .
.
. . . and
improving
student and
researcher
productivity
. . .
…because
Scopus (a)
enables you to
evaluate
performance
and quality
accurately . .
.
. . . (b)
offers you
the most
efficient
and
effective
search . .
.
. . . and (c)
provides
access to the
most
comprehen
sive
content . . .
. . . thereby
unlocking the
library’s full
potential
enabled by . . .
Best evaluation
of performance
and quality:
• Broadest
coverage of
peer-reviewed,
high quality
journals
• Globally as well
as locally
• More accurate
citation count,
fairer
recognition for
work
Improve
productivity of
Researcher and
Student:
• Most efficient
and effective
search
• Access to most
comprehensive
content
Secure institutional
success . . .
33. h-Index…
for Evaluating Authors (or Journals)
Proposed by JE Hirsch as an index to quantify an
individual's scientific research output
Remember, Impact Factor is just for journals, though it’s often used to evaluate
authors.
Combines an assessment of both quantity (number of
papers) and quality (impact, or citations to these papers).
h-index is automatically calculated:
Web of Science
Scopus
Publish or Perish (free download), based on data in Google Scholar
h-index can also be manually calculated for an author
based on the number of papers authored and the number
of times each paper has been cited.
See Wikipedia article for overview of h-index including
criticisms, alternatives http://en.wikipedia.org/wiki/H-
index
36. Scopus new bibliometric indicators
to measure the influence of journals
IPP (Impact per Publication)
The Impact per Publication measures the ratio of
citations per article published in the journal.
SNIP (Source Normalized Impact per Paper)
Source Normalized Impact per Paper measures
contextual citation impact by weighting citations based
on the total number of citations in a subject field
SJR (SCImago Journal Rank).
SCImago Journal Rank is a prestige metric based on the
idea that 'all citations are not created equal'. With SJR,
the subject field, quality and reputation of the journal
have a direct effect on the value of a citation. It is a
size-independent indicator and it ranks journals by their
'average prestige per article' and can be used for
journal comparisons in the scientific evaluation process
37. SJR (SCImago Journal Rank).
Developed by Professors Félix
de Moya, Research Professor at
Consejo Superior de
Investigaciones Científicas and
Vicente Guerrero Bote at
University of Extremadura,
SCImago Journal Rank (SJR) is
a prestige metric based on the
idea that 'all citations are not
created equal'.
With SJR, the subject field,
quality and reputation of the
journal have a direct effect on
the value of a citation.
SJR is a measure of
scientific influence of
scholarly journals that
accounts for both the
number of citations
received by a journal
and the importance or
prestige of the journals
where such citations
come from.
It is a variant of the
eigenvector centrality
measure used in
network theory.
Research Paper: "The SJR indicator: A new indicator of journals' scientific prestige"
38. SJR (SCImago Journal Rank).
The SCImago Journal &
Country Rank is a portal
that includes the journals
and country scientific
indicators developed from
the information contained in
the Scopus® database
These indicators can be
used to assess and
analyze scientific
domains.
This platform takes its name
from the SCImago Journal
Rank (SJR) indicator,
developed by SCImago from
the widely known
algorithmGoogle
PageRank™. This indicator
shows the visibility of the
journals contained in
the Scopus® database from
1996.
39. IPP (Impact per Publication)
The IPP measures the ratio of
citations in a year (Y) to
scholarly papers published in the
three previous years (Y-1, Y-2,
Y-3) divided by the number of
scholarly papers published in
those same years (Y-1, Y-2, Y-
3). The IPP metric is using a
citation window of three years
which is considered to be the
optimal time period to accurately
measure citations in most subject
fields.
40. SNIP (Source Normalized Impact
per Paper)
Source Normalized
Impact per Paper
measures contextual
citation impact by
weighting citations
based on the total
number of citations
in a subject field.
http://www.sciencedirect.com/science/article/pii/S17511577100000
39
41. SNIP/IPP
Source-Normalized Impact per Paper (SNIP) measures
contextual citation impact by weighting citations based
on the total number of citations in a subject field.
This unique aspect allows direct comparison of sources in
different subject fields. The impact of a single citation is
given higher value in subject areas where citations are
less likely, and vice versa.
It is a ratio, with a numerator and a denominator. SNIP's
numerator gives a journal's impact per publication (IPP).
This is simply the average number of citations received
in a particular year (e.g. 2013) by papers published in
the journal during the three preceding years (e.g. 2010,
2011 and 2012).
42. SNIP/IPP
SNIP's denominator, the Database
Citation Potential (DCP) is calculated as
follows.
We know that there are large differences in the
frequency at which authors cite papers between
various scientific subfields. In view of this, for each
journal an indicator is calculated of the citation
potential in the subject field it covers. This citation
potential is included in SNIP's denominator.
43. SNIP is IPP divided by DCP.
It aims to allow direct comparison of sources in
different subject fields. Citation potential is shown to
vary not only between journal subject categories -
groupings of journals sharing a research field - or
disciplines (e.g., journals in Mathematics,
Engineering and Social Sciences tend to have lower
values than titles in Life Sciences), but also between
journals within the same subject category. For
instance, basic journals tend to show higher citation
potentials than applied or clinical journals. Likewise,
journals covering emerging topics tend to be higher
than periodicals in classical subjects, or more general
journals.
44. SNIP
SNIP helps authors to identify which journals
are performing best within their subject field
and where to publish. Mouse over the circles
in the visualization and click on the years to
view the metrics. The size of the circles are
compared to the highest values in the 5-year
range. This highest value has a closed circle
whereas the open circles indicate the value
compared to this highest value.