1. Altmetrics:
building a broader picture of
impact
Paul Groth @pgroth
Web & Media Group
Department of Computer Science
The Network Institute
VU University Amsterdam
http://www.few.vu.nl/~pgroth
#APE2014
2.
3. Research Project Grants
Applications, awards, and success rates
NIH Data Book – (http://report.nih.gov/ndb/index.aspx)
Data provided by the Division of Information Services, Reporting Branch
4. "Outside letters basically trump
everything," says Robert Simoni,
chairman of the biology
department at Stanford University
in California.
Metrics: Do metrics matter?
Nature 2010
http://doi.org/10.1038/465860a
5.
6. ―Imagine how the academic appointment process
might change if search and review committees had
access—within an appropriately tagged or linked
online CV, for example, or via the ORCID system—
to information about the specific contributions
made by a candidate to each of his/her works,
including contributions that might not otherwise have
qualified for ‗authorship‘ status?‖
Point of view: Faculty appointments and the record of
scholarship
Amy Brand
http://dx.doi.org/10.7554/eLife.00452
7. Point of view: Faculty appointments and the record of scholarship
Amy Brand
http://dx.doi.org/10.7554/eLife.00452
Opportunities
• Individuals and institutions need better tools
for curating and networking their own record
of scholarship
• Institutions need more information about
scholarly contribution
• ALMs that reliably differentiate sources of
input (general; academic; expert; etc.) would
be more useful‖
Slide 3: http://article-level-metrics.plos.org/files/2013/10/Brand.pptx
9. Altmetrics is the study and use of scholarly
impact measures based on activity in online
tools and environments.
http://doi.org/10.1371/journal.pone.0048753
15. ―It took approximately a generation (20
years) for bibliographic citation analysis to
achieve acceptability as a measure of
academic impact."
(Vaughan and Shaw,
2003)
16. (Birkholtz et al. 2013)
(Fausto et al 2012)
http://jasonpriem.org/self-archived/5uni-poster.png
The Research is Happening
http://ploscollections.org/altmetrics
http://asis.org/Bulletin/Apr-13/AprMay13_Piwowar.html
28. Summary: a broader view
• Different research artifacts
– papers, preprints, slides, videos, code, data
• Different measures
– usage, mentions, views, sharing
• Different stories
– progress so far, workshop impact, outreach
29. Conclusion
• Altmetrics is still developing
– But useful today
• Allows to build a broader picture of impact
– Using a variety of Artifacts & Measures
• Final thought: research artifacts exist in a
network, we‘re starting to connect it
30. Thanks
Collaborators: Peter van den Besselaar, Julie
Birkholz, Frank van Harmelen, Shenghui Wang,
Rinke Hoekstra, Thomas Gurney, Mike Taylor,
Anita de Waard, Jason Priem, Dario Taraborelli,
Cameron Neylon, Ian Mulvany
Editor's Notes
This is a pain to calculate!
We believe metrics are it but policy makers don’t necessarily. What they want is evidence.
There’s more and more “stuff” – where are we
From Assistant Provost for Faculty Appointments and Information at Harvard University, where she manages the review of faculty appointments University-wide - See more at: http://elife.elifesciences.org/content/2/e00452#sthash.9shtp91n.dpuf
From Assistant Provost for Faculty Appointments and Information at Harvard University, where she manages the review of faculty appointments University-wide - See more at: http://elife.elifesciences.org/content/2/e00452#sthash.9shtp91n.dpuf
This will happen closer than 20 years
Clear correlation between F1000 recommendations and citations
Side note: From the slides:“There is no reason to condemn the incorrectly used Impact Factor and h-index. They can provide supplementary information if they are used in combination with qualitative methods, and are not used as the only decision criterion. Example:• Good practice (h-index as supporting argument): “The exceptionally high h-index of the applicant confirms his/her international standing aested to by our experts.” • Qestionable use (h-index as decision criterion):“We are inclined to support this scientist because his/her h-index distinctly exceeds that of all other applicants.”