American University Library's Conference for High Impact Research presentation, Measuring Scholarly and Public Impact. Given May 15th, 2017, discusses bibliometrics and altmetrics, focusing on case uses, current trends, and disciplinary considerations.
CHiR presentation measuring scholarly and public impact
1. Measuring Scholarly
and Public Impact:
Let’s Talk Metrics
RACHEL BORCHARDT MONDAY, MAY 16TH, 2017
BORCHARD@AMERICAN.EDU CONFERENCE FOR HIGH-IMPACT RESEARCH
2. Why metrics?
Quantifiable measures of research impact
Differentiate articles, journals, authors, institutions
Imperfect calculations of a fuzzy concept
3. How are metrics used?
Evaluate researchers:
Hiring
Tenure/promotion
Merit awards
Grant funding
Benchmark programs and institutions
Image from AU Faculty Manual, http://www.american.edu/loader.cfm?csModule=security/getfile&pageid=4304081
4. What are metrics
Traditional Bibliometrics
Based on citation counts
Emerging Altmetrics
Based on online tools
Image from Plum Analytics
6. Traditional bibliometrics for journals
Impact Factor and Citation Counts, created to measure
Journals and journal articles
Scholarly (journal) impact
Initially created for librarians, then largely adopted by STEM
Image from Journal Citation Reports (library database)
7. Some Impact Factor limitations
Retrospective measurement
Narrow measurement
Confined by what is indexed as a citation
Can’t be compared between disciplines
Other known issues: language, co-authorship, disproportionate article
citation impact
Image from Essential Science Indicators (library database)
8. One way to correct – disciplinary context
Image from Journal Citation Reports (library database)
9. Contextual limitations – “Political
Science” category
List from http://www.auspsa.org.au/publications
10. Other major bibliometrics
Metrics from Web of Science: Eigenfactor, Immediacy Index, Cited Half-Life
Metrics from Elsevier/Scopus: SJR, SNIP, IPP, CiteScore
Metrics from Google Scholar: H5-index, H5-median
Image from http://www.journalmetrics.com
11. SJR comparison – “Political Science
and International Relations”
13. Another alternative – citation distributions
http://www.nature.com/news/beat-it-impact-factor-publishing-elite-turns-against-controversial-metric-1.20224
14. Why altmetrics?
Measures broader scholarly outputs
Books, presentations, data sets, software code, and
more
More measures of scholarship
Views, downloads, shares, mentions in blogs, policy
papers, and more
More immediate metrics
Measures beyond scholarly impact
Policy, public, and more
https://becker.wustl.edu/impact-assessment
15. What are altmetrics?
Definition: The creation and study of new metrics based on the social web for
analyzing and informing scholarship.
Altmetrics.org
My definition: The umbrella classification of non-citation based metrics.
http://altmetrics.org/manifesto ; http://twitter.com/jasonpriem
18. How do we collect altmetrics?
Directly from the individual tools
From publishers (views, download data)
From (some) library databases
From scholarly networks
Through aggregating tools
Slideshare views
PLOS article metrics Web of Science usage ResearchGate metrics Altmetric metrics
19. “Impact” and metrics
Inherently fuzzy and subjective
Attention
Engagement
Some correlate to bibliometrics; some don’t
26. Limitations of impact: example
https://www.altmetric.com/details/2313862/policy-documents
https://www.healthypeople.gov/2020/topics-objectives/topic/Adolescent-
Health
https://www.google.com/search?q=%22Trends+in+adolescent+and+youn
g+adult+health+in+the+United+States%22+site%3A.gov&oq=%22Trends
+in+adolescent+and+young+adult+health+in+the+United+States%22+si
te%3A.gov&aqs=chrome..69i57.3775j0j8&sourceid=chrome&ie=UTF-8
27. Current uses of altmetrics
External
Applying for funding, promotion/tenure
Showcasing achievements
Internal
Better understand engagement with research
Who, why, where
28. More on grant funders
https://www.nature.com/naturejobs/science/articles/10.1038/nj7471-397a
29. Grant funders: NSF
Broader Impacts
https://www.nsf.gov/od/oia/special/broaderimpacts/
31. Putting it all together - suggestions
Develop personal impact goals
Discipline-specific, research-specific, university-specific, etc.
Demonstrate achievement of those goals
Tell your story
Bibliometrics, altmetrics, and qualitative measures
Complementary roles
Track early and often!
32. Getting help
Rachel Borchardt
borchard@american.edu
202-885-3657
(I work year-round!)
Metrics guide, http://tinyurl.com/AUmetrics
Borrow a copy of my book