June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends Snowball Metrics: University-owned Benchmarking to Reveal Strengths within All Activities - Dr. Lisa Colledge, Snowball Metrics Program Director, Elsevier

1. Presented by Dr Lisa Colledge
Snowball Metrics Programme Director, Elsevier
18 June 2014
www.snowballmetrics.com
Snowball Metrics:
University-owned Benchmarking to Reveal
Strengths within All Activities
1

2. The origins of Snowball Metrics
Competitive award won by Imperial College
London and Elsevier to investigate the state
of research management in England
2
Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information-
management1.pdf

3. The origins of Snowball Metrics
Competitive award won by Imperial College
London and Elsevier to investigate the state
of research management in England
Clear trends were voiced:
• “Unless you have [data] you cannot make
informed decisions; you would be acting based
on opinions and hearsay.”
3
Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information-
management1.pdf

4. The origins of Snowball Metrics
Competitive award won by Imperial College
London and Elsevier to investigate the state
of research management in England
Clear trends were voiced:
• “Unless you have [data] you cannot make
informed decisions; you would be acting based
on opinions and hearsay.”
• “[There is little] thought leadership and
knowledge development around best practice.”
4
Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information-
management1.pdf

5. The origins of Snowball Metrics
Competitive award won by Imperial College
London and Elsevier to investigate the state
of research management in England
Clear trends were voiced:
• “Unless you have [data] you cannot make
informed decisions; you would be acting based
on opinions and hearsay.”
• “[There is little] thought leadership and
knowledge development around best practice.”
5
Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information-
management1.pdf
• “The principle drivers for our systems are often external… but
they shouldn’t be… a research strategy should… be developed… to respond
to our strengths and the external environment, our systems should be defined
to run our business.”

6. University recommendations
6
Universities and funders should work
more collaboratively, and develop
stronger relationships with suppliers
“Universities should work together
more to make their collective voice
heard by external agencies.”
“The lack of a long-term vision makes it hard
to… co-operate within a university let alone
across the sector.”
“Suppliers do not know what research offices
do on a daily basis.” “How educated are we
at asking suppliers the right questions?”
1

7. University recommendations
7
Universities and funders should work
more collaboratively, and develop
stronger relationships with suppliers
“Universities should work together
more to make their collective voice
heard by external agencies.”
“The lack of a long-term vision makes it hard
to… co-operate within a university let alone
across the sector.”
“Suppliers do not know what research offices
do on a daily basis.” “How educated are we
at asking suppliers the right questions?”
“Someone needs to take ownership of the process: it is
impossible to please all of the people all of the time so
somebody needs to be strong enough to stand behind
decisions and follow through.”
“It would be great if the top
five [universities] could
collaborate.”
1
2

8. Snowball Metrics are…
• Endorsed by a group of distinguished UK
universities to support their strategic
decision making
• Tried and tested methodologies that are
available free-of-charge to the higher
education sector
• Absolutely clear, unambiguous definitions
enable apples-to-apples comparisons so
universities can benchmark themselves
against their peers to judge the
excellence of their performance
8

9. Snowball Metrics are…
• Endorsed by a group of distinguished UK
universities to support their strategic
decision making
• Tried and tested methodologies that are
available free-of-charge to the higher
education sector
• Absolutely clear, unambiguous definitions
enable apples-to-apples comparisons so
universities can benchmark themselves
against their peers to judge the
excellence of their performance
Snowball Metrics are unique because:
• Universities drive this bottom up
• Academia – industry collaboration
9

10. Snowball Metrics approach
10
Vision: Snowball Metrics enable benchmarking by driving
quality and efficiency across higher education’s research and
enterprise activities, regardless of system and supplier
• Bottom-up initiative: universities define and endorse metrics to
generate a strategic dashboard. The community is their guardian
• Draw on all data: university, commercial and public
• Ensure that the metrics are system- and tool-agnostic
• Build on existing definitions and standards where possible and
sensible

11. The output of Snowball Metrics
11
www.snowballmetrics.com/metrics
“Recipes” – agreed and tested metric
methodologies – are the output of
Snowball Metrics
From Statement of Intent:
• Agreed and tested methodologies…
are and will continue to be shared
free-of-charge
• None of the project partners will
at any stage apply any charges for
the methodologies
• Any organization can use these
methodologies for their own
purposes, public service or
commercial
Statement of Intent available at http://www.snowballmetrics.com/wp-content/uploads/Snowball-Metrics-
Letter-of-Intent.pdf

12. Founding project partners
12
Snowball Metrics Project Partners

13. Main roles and responsibilities
• Everyone covers their own costs
• Universities
– Agree the metrics to be endorsed as Snowball Metrics
– Determine methodologies to generate the metrics in a commonly
understood manner to enable benchmarking, regardless of systems
• Elsevier
– Ensures that the methodologies are feasible
– Distribute the outputs using global communications networks
– Day-to-day project management of the global program
• Outside the remit of the Snowball Metrics program
– Nature and quality of data sources used to generate Snowball Metrics
– Provision of tools to enable generation and use Snowball Metrics
13

14. Snowball Metrics Vision
Snowball Metrics drive quality and efficiency across higher
education’s research and enterprise activities, regardless of
system and supplier
Key aims
• Agree, test and share metrics recipes in all areas of the
“research and enterprise landscape”
• Drive adoption of Snowball Metrics as global standards for
institutional benchmarking
– By implementation in tools, systems and reports
– By engaging with university groups in key research-intensive
nations
14

15. The Snowball Metrics Landscape
15

16. Denominators
16

17. Recipes
available
17

18. Altmetrics
Snowball Metrics
• needs to be able to aggregate counts beyond article level
• Needs to be able to share the entire recipe

19. Altmetrics
Scholarly Activity
Number of times that an institution’s output has been
posted in online tools that are typically used by
academic scholars e.g. Mendeley, CiteULike,
Academia.edu, ResearchGate
Scholarly Commentary
Number of times that an institution’s output has been
commented on in online tools that are typically used
by academic scholars e.g. science blogs, video posts,
peer reviews, post-publication comments, Wikipedia
Mass Media
Number of times that an institution’s
output has been referred to by press
clippings and news websites
Social Activity
Number of times that an institution’s
output has stimulated social media
posts e.g. Facebook, Twitter, Reddit,
LinkedIn
Snowball Metrics
• needs to be able to aggregate counts beyond article level
• Needs to be able to share the entire recipe

20. Altmetrics
Scholarly Activity
Number of times that an institution’s output has been
posted in online tools that are typically used by
academic scholars e.g. Mendeley, CiteULike,
Academia.edu, ResearchGate
Scholarly Commentary
Number of times that an institution’s output has been
commented on in online tools that are typically used
by academic scholars e.g. science blogs, video posts,
peer reviews, post-publication comments, Wikipedia
Mass Media
Number of times that an institution’s
output has been referred to by press
clippings and news websites
Social Activity
Number of times that an institution’s
output has stimulated social media
posts e.g. Facebook, Twitter, Reddit,
LinkedIn
Snowball Metrics
• needs to be able to aggregate counts beyond article level
• Needs to be able to share the entire recipe
Scholarly Activity Scholarly Activity per FTE Scholarly Activity per publication

21. Globalizing Snowball Metrics
21
US
• University of Michigan
• University of Minnesota
• Northwestern University
• University of Illinois at Urbana-
Champaign
• Arizona State University
• MD Anderson Cancer Center
• Kansas State University
Australia / New Zealand
• University of Queensland
• University of Western Australia
• University of Auckland
• University of Wollongong
• University of Tasmania
• Massey University
• The University of Canberra
• Charles Darwin University
Interest and support from:
• Japan RU11 metrics group
• Association of Pacific Rim Universities (APRU)
• European Commission for H2020
• Fundação para a Ciência e a Tecnologia (FCT) in Portugal

22. Supporting Snowball Metrics
• Feedback about how useful it would be to you to have
comparative information based on the Snowball Metrics
• Feedback on the clarity of the recipes in this book
• Call for influential bodies in the social sciences and arts and
humanities to collaborate with the initiative to agree on practical
metrics that are particularly useful for these areas
• Ask funders and suppliers to adopt Snowball Metrics. Change
requires momentum: please help us provide it
• snowballmetrics@elsevier.com
22

23. THANK YOU FOR
YOUR TIME AND ATTENTION!
l.colledge@elsevier.com
snowballmetrics@elsevier.com