Increasingly, many aspects of scholarly communication—particularly publication, research data, and peer review—undergo scrutiny by researchers and scholars. Many of these practitioners are engaging in a variety of ways with Alternative Metrics (#altmetrics in the Twitterverse). Alternative Metrics take many forms but often focus on efforts to move beyond proprietary bibliometrics and traditional forms of peer referencing in assessing the quality and scholarly impact of published work. Join NISO for a webinar that will present several emerging aspects of Alternative Metrics.

1. http://www.niso.org/news/events/2012/nisowebinars/alternative_metrics/
NISO Webinar:
Beyond Publish or Perish: Alternative
Metrics for Scholarship
November 14, 2012
Speakers: Martin Fenner, Jason Priem,
Aalam Wassef

2. Technical Lead ALM Project
Martin Fenner
http://www.flickr.com/photos/wallyg/156079118/

3. Journal-based metrics correlate poorly
with numbers for individual article
3

4. PLOS ONE is not a second-tier journal
Combined Scopus citation counts for all PLOS
Biology and PLOS Medicine articles published in
2009, as well as top 200 PLOS ONE articles in 2009
Citation counts collected November 8, 2012
4

5. The megajournal has changed how we use the
journal as a filter
5

6. Citations are only a small fraction of how a
paper is reused and discussed
100%
22.2%
Article-Level Metrics
from November 8, 2012
for 63,771 PLOS Papers
0.3%
6

7. The PLOS Article-Level Metrics project
started in 2009 and tracks usage, citations
and social web activity for all PLOS articles
Usage Citations Social Web
PLOS Journals CrossRef Mendeley
(HTML, PDF, XML)
Scopus CiteULike
PubMed Central Web of Science Facebook
(abstract, fulltext,
PubMed Central Twitter
PDF, figures)
Research Blogging
PLOS Comments
Wikipedia
7

8. 8

9. 9

10. Why use article-level metrics?
10

11. Should article-level metrics be used to
evaluate the impact of research?
• Can numbers reflect the impact of research, across
disciplines and over time?
• Do the currently available metrics really measure impact or
something else?
• Does the use of metrics for evaluation create undesired
incentives?
• How easily can metrics be changed by self-promotion and
gaming?
11

12. What do we measure?
12

13. Gaming and Spam
13

14. Metrics as a discovery tool
14

15. Data-driven stories about the post-
publication reception of a paper
15

16. http://blogs.bbsrc.ac.uk/index.php/2012/09/summertime-reprise/
http://dx.doi.org/10.1371/journal.pcbi.1000204
16

17. 17

18. Business intelligence for a
journal, university of funder
18

19. http://bit.ly/alm36
19

20. 20

21. Tweets about Big Food Collection
Data collected August 19, 2012.
21

22. Identify Open Access articles published by PLOS,
funded by the European Commission Seventh
Framework Programme (FP7)
Match articles with grant numbers
Show the impact of these publications
PLOS Search API CORDIS
2562 Articles 1166 Articles
22

23. Author locations from the ENGAGE project
First authors (blue) and last authors (green).
European Network for Genetic and Genomic Epidemiology
http://www.openaire.eu/en/component/openaire/project_info/default/530?grant=201413
23

24. Scopus
24

25. Mendeley
25

26. Thank you
Wordle of altmetrics manifesto by AJ Cann
http://www.flickr.com/photos/ajc1/6795008004/
26

27. Altmetrics and Revolutions:
New media, new metrics, and a new
scholarly communications system.
Nov 14, NISO

28. In the beginning:
...was the letter.
• slow
• costly
• duplication
• best solution given available
technology

29. The First Revolution
In 1665, Oldenburg published The
Philisophical Transactions of the
Royal Society (now OA :).
This applied the best available
technology (the printing press) to
vastly improve dissemination.

30. The First Revolution
promoted homogeneity of
outputs.
The standardized article was born of
the need for industrial-scale replication
and interchangeability.

31. The Second Revolution
will promote diversity of
outputs.
Since publication is nearly free, it
becomes trivial to capture the missing
pieces of the scholarly record.

32. conversation
stories
analysis
data

33. Looking beneath the foliage:
Web promises new tools for conversation.
reference managers
blogs
social bookmarking
social networks

34. Examples: Mendeley
2 million user
libraries
200 million
documents
uploaded, 75M
unique (MEDLINE
has 18 million...)

35. Examples: Twitter
In one month, over 58k citations from
Twitter to scholarly articles (citwaitions?)
It is like having a jury preselect what will
probably interest you…. Occasionally there
will be something that people will link to, and
it will change what I think, or what I’m doing,
or what I’m interested in.
-study participant

36. Examples: Twitter

37. Bring on the
metrics.

38. A network of ideas: bibliometrics.
In 1961, Garfield creates the
Science Citation Index.
• replaces expert judges with
crowdsourced judgements
• based on existing patterns of use:
mining, not asking.
• And that's awesome!

39. But only part of the picture
1. Only one type of person: academics.
2. Only one kind of resource: scholarly articles.
3. Only one kind of use: using to support a
scholarly article.

40. What about all the other uses?
Reading, annotating, bookmarking, saving,
discussing, sharing, etc?
"...there are Heart of scholarly Spotting emerging
undoubtedly highly communication research fronts will
useful journals that is "visits, personal require tracking
are not cited contacts, and "formal and
frequently." letters." (Bernal, informal
(Garfield, 1972) 1944) communication"
(Kuhn, 1977)

41. What about all the other uses?
We start to confuse
“the kind of use we can
track” with use
and “citation impact”
with impact.

42. Bibliometrics mined impact
on the first scholarly Web.
altmetrics mines
impact on the next
one.

44. Why altmetrics?
• We can more fairly evaluate researchers, publications,
and institutions if we look at the whole impact picture.
• We can assess impact faster
• Etc, etc.
• Much more important: quantifying impact means we can
teach machines what’s important. This changes the
game. And the game needs to be changed.

45. Journals need an upgrade.
• First journals went hand in hand with the Scientific Revolution.
Applied the most advanced technology available to the problem
of spreading scholarship.
• Today’s journals are still the best scholarly communication
system possible using 17th-century technology.
• They’ve got some problems:
o Slow
o Restrictive format: function follows form
o Closed
o Inconsistent quality control
o Hard to innovate

46. We don’t use the Web.
• Berners-Lee created the Web as a scholarly communication tool.
• Today the Web has revolutionized everything but scholarly
communication.
• Online journals are essentially paper journals, delivered by faster
horses.
In the early days, CERN maintained a list of all the world’s Web
servers. Haha, CERN ur so krazy.
• But today, we can fit every single meaningful outlet for
scholarship (25k journals + some conferences) on one list, too.

47. But what if we did?
The Decoupled Journal article: a case study.

48. The second revolution has started.
Once we have alt-citation data, it’s too useful to
ignore; alternative filters and even certification paths
based on this data will open.
As Peter Vinkler says, citation graph data is like
Chekhov’s gun: once on stage, it has to be fired.

49. A wise man, that Chekov.

50. Questions?
Jason Priem @jasonpriem, http://jasonpriem.org
This is a living document; if a particular
version is important to you, make sure
you link to that revision.