This document summarizes key points from a discussion about impact metrics and altmetrics with university researchers: - Researchers had varying familiarity with traditional impact metrics like journal impact factors, with some seeing little meaning or value in them. Biology researchers saw potential issues like variability across disciplines. - Alternative metrics were newer to most researchers, with one mathematician expressing skepticism due to lack of established "truth". However, one mathematician saw potential value in altmetrics that aggregate many user-generated assessments. - Bringing it back to libraries, traditional metrics focused on research outputs libraries purchase and support. Altmetrics change the landscape by measuring broader impacts beyond citations, possibly including engagement with library resources, services, and online presences. This

1. Mindy Thuna
Science Liaison
Pam King
Scholarly Communication Liaison

2. This work is licensed under:
http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_US
You are free to:
Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material
Under the following terms:
Attribution — You must give appropriate credit, provide a link to the
license, and indicate if changes were made. You may do so in any
reasonable manner, but not in any way that suggests the licensor endorses
you or your use.
NonCommercial — You may not use the material for commercial purposes.
ShareAlike — If you remix, transform, or build upon the material, you must
distribute your contributions under the same license as the original.

3. Altmetrics in
Higher Ed Settings
metrics in context - What our researchers are saying

4. Agenda
• Impact Metrics
• Altmetrics
• What our researchers have to say
• Bringing it back to libraries

5. IMPACT METRICS

6. Impact Metrics are…
…calculations/algorithms that quantify the impact of
research or scholarly activity.

7. Research impact defined
―Impact is usually demonstrated by
pointing to a record of the active
consultation, consideration, citation,
discussion, referencing or use of a
piece of research.‖
http://blogs.lse.ac.uk/impactofsocialsciences/introduction/

8. Basic Premise
Number of
citations
Impact
Reach and
significance

9. Three main providers
• Scopus
• Web of Science
• Google Scholar

10. Three Levels
• Journal Level
• Article Level
• Author Level

11. Journal Level Metrics
Web of
Science
Impact
Factor (2 & 5
year)
Cited half-
life
Eigenfactor®
Score
Article
Influence®
Score
Scopus SCImago
Journal Rank
(SJR)
Source
Normalized
Impact per
Paper
(SNIP)
Google
Scholar
h5-index h5-median

12. Journal Impact Factor
Average number of times articles published in a two year
(or five year) period have been cited
A = total number of times ALL articles published in 2 (or 5)
year period were cited in WofS indexed journals during the
next year
B = total number of "citable items" (usually
articles, reviews, proceedings or notes; not editorials and
letters-to-the-editor) published in 2 (or 5) year period
Impact factor = A/B

13. Journal Impact Factor Debate
• Vary across disciplines
• e.g., variants in time-to-publication
• Review journals have higher impact
• fewer articles per journal, cited more
• Calculations easily skewed
• heavy editorial/letter/opinion content
• high/low percentage of review articles

14. Declared Dissent – 10,419
• ―Impact factors declared unfit for duty‖
Stephen Curry, LSE Blog
• ―Do not resuscitate: the journal impact factor
declared dead‖
Brendan Crabb, The conversation
• ―Just say no to impact factors‖
Ismael Rafols and James Wilsdon, the guardian
DORA: San Francisco Declaration on Research Assessment
(Dec. 2012)

15. Author Level Metrics
Web of
Science
h-index
Scopus h-index
Google
Scholar
h-index i10-index

16. h index is NOT perfect
Paper 1 Paper 2 Paper 3 Paper 4 Paper 5
Author A 3 cites 6 cites 100
cites
4 cites 1 cite
Author B 3 cites 6 cites 100
cites
Author C 400
cites
150
cites
3 cites 6 cites
h index = 3

17. Article Level Metrics
Citation Counts - how often a particular journal article is
being cited by other articles

18. ALTMETRICS
ALT-METRICS
ALTERNATIVE METRICS

19. Web metrics
• Webometrics - link analysis, web log file analysis, 1990s
• Bibliometric use – downloads, citations
• Google web page ranking
• Web 2.0 ->scientometrics 2.0 (Priem & Hemminger
2010)

20. Open Access
• Producing a highly complex information environment;
touches all disciplines
• Many formats, e.g. websites, nanopubs, blog posts, code,
images
• versions, e.g. preprints, postprints, ―same article different
places‖, version of record

21. Identification Systems
Authors
• International Standard Name Identifier (ISNI)
• ORCID (subset of ISNI)
• Names Project (JISC)
• Author Claim, RePEc Author Service
• arXiv Author ID
• ResearchID (Thomson Reuters)
• Scopus Author ID (Elsevier)
• Gravatars (Auttomatic)
Objects
• DOIs, PMIDs URLs, URIs

22. Altmetrics
… are calculated from newer data sources, e.g.
Wikipedia, Mendeley, Twitter, Facebook, Weibo, blogs
… report the impact of a wider range of research
outputs, e.g. presentation slides, data sets, articles, code

23. Altmetrics aggregators/tools/
services
• PLoS One Article Level Metrics (ALM)
• ImpactStory
• Altmetric
• PlumX
• and others …

24. PLoS Article Level Metrics (ALMs)

28. ImpactStory

30. 30

32. Altmetric (altmetric.com)

35. PlumX

38. The Debate
―Article-Level Metrics: An Ill-Conceived and
Meretricious Idea‖ vs ―Broaden your horizons: impact
doesn‘t need to be all about academic citations‖
(Jeffrey Beall vs Euan Adie, Aug. 2013)

39. Moving forward
―to move out of its current pilot and
proof-of-concept phase‖
• Sloan foundation grant for NISO
initiative - creation of community-
based standards; definitions,
calculations. Data classifications and
data sharing practices

40. WHAT OUR
RESEARCHERS HAVE
TO SAY

41. Our Ongoing Study
―Research Impact Metrics: Use and Understanding by Post-
Tenure Faculty‖

42. What do you think of when you
hear the term impact metrics?
―… I don't think that impact factors at
the journal level necessarily captures
the importance of the paper... I jokingly
say that out of all the things I have
published, the paper that I expect will
have the longest life is the one that's
got the lowest impact factor.‖
Biology

43. What do you think of when you
hear the term impact metrics?
―Nothing I don‘t understand what your
talking about. No clue. I understand
‗impact‘ I understand those words
separately. But not together.‖
Mathematical and Computational Sciences
―Means nothing to me – zero.‖
Political Science

44. What do you think of when you
hear the term impact metrics?
―I have a picture of that book that you find
at the Robarts library on the 4th floor that
is a collection of binders in colourful
colours and when you open it, it looks like
the phone book with all the list of all the
articles and researchers so that is my idea
of impact metrics.‖
Language Studies

45. Have you considered/do you
use any alternative metrics?
―No. No trust in them.‖
Mathematical and Computational Sciences

46. ―… I give a lot of relevance to those metrics and I will tell
you why…There was an event in the year 1900. It was
some popular festivity in England and they were running a
contest. There was a cow or a bull and people had to
guess it's weight and the person who gets closes to the
weight wins the cow, or something like that. And someone
kept a record of the answers and it is very interesting. It
was very wide-ranging so let's say the weight of the cow
was 500 kg. So the range was from 300-1000. The most
accurate answer was the average of all the answers that
people gave…So what is that telling you. It is telling you
many things…It is telling you that this phase in which we
are moving into in which content is user generated, there
is a lot of proof there. You have to remember, I am a
mathematician, I don't care what people think, we build
truth from the bottom up…‖
Mathematical and Computational Sciences

47. BRINGING IT BACK
TO LIBRARIES

48. Bringing it back to libraries
• Traditional metrics
• Libraries provide the support
• Libraries pay for the metrics
• Libraries purchase the research output
• Altmetrics change the rules
• Now what happens?
• It all depends on what libraries want to measure….
• tweets and blogs about our buildings and spaces and our
service
• Bookmarks of websites and libguides
• Likes on Facebook
• Downloads of librarian publications

49. Thoughts?