NASIG - May 2015, Arlington, VA Todd Carpenter, NISO Executive Director Nettie Lagace, NISO Associate Director for Programs http://www.niso.org/topics/tl/altmetrics_initiative/

1. Altmetrics aren’t
so alternative anymore:
Moving new metrics
into the mainstream
Todd A. Carpenter
Executive Director, NISO
@TAC_NISO
May 28, 2015

2.  Non-profit industry trade association
accredited by the American National Standards Institute
 Mission of developing and maintaining technical
standards related to information, documentation,
discovery and distribution of published materials and
media
 Volunteer driven organization: 400+ contributors spread
out across the world
 Responsible (directly and indirectly) for standards like
ISSN, DOI, Dublin Core metadata, DAISY digital talking
books, OpenURL, MARC records, and ISBN
About
May 28, 2015 2

3. 1980s Alternative
John Taylor,
Duran Duran

4. 2010s Not-so Alternative

5. Since Jason Priem coined the term
“altmetrics”
• There have been some 1,170 articles and
posts (per Google Scholar) on the topic of
altmetrics
• There have been eight altmetrics conferences
• There are now at least four companies
focused on providing altmetrics-style analytics
May 28, 2015 5

6. When it becomes a
standard, its not alterative
any more…

7. But what’s the value in
counting Tweets and
Facebook likes?
May 28, 2015 7

8. More than just popularity
Research is pointing to the fact that there is a
modest positive correlation between
early-signal metrics (altmetrics) and
later-signal metrics (citations)
Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with
citations from a multidisciplinary perspective by Rodrigo Costas, Zohreh Zahedi, Paul Wouters
Do Altmetrics Work? Twitter and Ten Other Social Web Services by Mike Thelwall , Stefanie
Haustein, Vincent Larivière, & Cassidy R. Sugimoto
(2006) Earlier web usage statistics as predictors of later citation impact by Brody T, Harnad S &
Carr L
May 28, 2015 8

9. More importantly: Scholarship today
is much more than journals
May 28, 2015 9

10. How do we measure the
impact of these different
forms of output?
May 28, 2015 10

11. Would a researcher focus on only one data
source or methodological approach?
May 28, 2015 11Duke University - Information Initiative at Duke (IID)

12. There aren’t
metrics and “altmetrics”;
there are only metrics!
May 28, 2015 12

13. We have been using
non-citation-based
metrics for decades
May 28, 2015 13

14. May 28, 2015 14

15. May 28, 2015 15
Traverse Area District Library (TADL)

16. What has changed is our
ability to
collect and analyze data
in new ways that are
meaningful to others
May 28, 2015 16

17. What are we lacking?
=

18. TRUST
=

19. What are the elements
of metrics that we
need to build trust?

20. Granularity
Time Scale
Identificatio
n
Definitio
n
Exchange

21. I often sound like a broken record
• Defining what is to be counted = standards
• How to describe what to count = standards
• Identification of what to count = standards
• Procedures for counting or not = standards
• Aggregating counts from network = standards
• Exchange of what was counted = standards
May 28, 2015 21

24. Phase 1 Meetings
October 9, 2013 - San Francisco, CA
December 11, 2013 - Washington, DC
January 23-24 - Philadelphia, PA
Round of 1-on-1 interviews – March/Apr
Phase 1 report published in June 2014

25. Meeting Lightning Talks
• Expectations of researchers
• Exploring disciplinary differences in the use of social media in
scholarly communication
• Altmetrics as part of the services of a large university library
system
• Deriving altmetrics from annotation activity
• Altmetrics for Institutional Repositories: Are the metadata
ready?
• Snowball Metrics: Global Standards for Institutional
Benchmarking
• International Standard Name Identifier
• Altmetric.com, Plum Analytics, Mendeley reader survey
• Twitter Inconsistency
May 28, 2015 25
“Lightning" by snowpeak is licensed under CC BY 2.0

26. May 28, 2015 26

27. May 28, 2015 27

28. May 28, 2015 28

29. 30 One-on-One Interviews
May 28, 2015 29

30. Lots and lots of ideas

31. Phase 2
Presentations of Phase 1 report (June 2014)
Prioritization Effort (June - Nov, 2014)
Project approval (Nov - Dec 2014)
Working group formation (Jan - March 2015)
Consensus Development (March 2015 - Feb 2016)
Trial Use Period (Feb 15 - May 16)
Publication of final recommendations (Aug 16)

32. White Paper Released
May 28, 2015 32

33. 200 + Ideas
25 Potential Work Themes
Definitions
Application to types of research outputs
Discovery implications
Research evaluation
Data quality and gaming
Grouping, aggregating, and granularity
Context
Adoption
May 28, 2015 33

34. May 28, 2015 34
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Unimportant
Of little importance
Moderately important
Important
Very important
Community Feedback on Project Idea Themes
n=118

35. May 28, 2015 35
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Unimportant
Of little importance
Moderately important
Important
Very important
n=118
Community Feedback on Project Idea Themes

36. Standards Creation Cycle
Idea
Working
Group
Formed
and
Works!
Draft for
Public
Comment
Publication
Standing
Committee &
Maintenance
NISO
Approval:
Topic Committee
& Voting Member

37. Definitions and Principles
Balance: no single interest category constitutes a majority of
the membership / voting pool / working group: producer, user,
general interest
37
Recommended PracticeStandard
Consensus: respond to all comments and make efforts to resolve
negative votes, even if proposal is approved
Open process: allowing members and the community to
have confidence in NISO standards.
KBAR
T

40. Working Groups
• A development of definitions and descriptions of
use
• Bdefinitions for appropriate metrics and
calculation methodologies for specific output
types and promotion and facilitation of use of
persistent identifiers
• C development of strategies to improve data
quality through source data providers.

41. Steering Committee

42. Calendar
• April 2015 – Group(s) start working
• October 2015 – Draft document(s)
• Fall 2015 – Comment period(s)
• November 2015 – NISO Report to Sloan
Foundation
• Spring 2016 – Completion of final draft(s)

43. http://www.niso.org/topics/tl/altmetrics_initiative/
Thank you!
Questions?
@TAC_NISO
@abugseye