Presentation given at 9th Conference on Open Access Scholarly Publishing - Tony Ross-Hellauer, Lisbon, Portugal 20th Sept 2017

1. Gauging researcher attitudes
to open peer review
Tony Ross-Hellauer
State and University Library Göttingen
Know-Center GmbH
COASP 2017
Lisbon, 20th September 2017
@tonyR_H
@openaire_eu

2. Human
Network
50 Partners from every EU country, and beyond
Data centers, universities, libraries, repositories, legal
experts
Digital
Networ
k
… fosters the social and technical links
that enable Open Science in Europe and
beyond
COASP 2017, Lisbon 20-09-2017 @tonyR_H

3. Transparency Accountability Inclusivity
Responsibility
Community &
Collaboration
Visibility
Rigour Equality Public good
Reproducibility Findability Accessibility
Interoperability Re-usability Innovation
Principles of Open Scholarship
CC BY

4. Open Science is more
than just Open Access
…
COASP 2017, Lisbon 20-09-2017
Opening up scientific processes and
products from all levels to everyone …
• Open Access (publications, data,
software, educational resources)
• Open Methodology (open
notebooks, study preregistration)
• Citizen Science
• Open Evaluation / Open Peer
Review
@tonyR_H

5. COASP 2017, Lisbon 20-09-2017
Waste
d effort
Traditional peer review has problems
…
CC BY Mike Licht
CC BY Mike EisenTime
Accountability & biasLack of incentives
@tonyR_H

6. DEFINING OPR
COASP 2017, Lisbon 20-09-2017 @tonyR_H

7. Peer review is usually:
Anonymous: reviewers unknown to
authors, or both authors and reviewers
unknown to each other
Opaque: neither the process nor the
reviews are made public
Selective: reviewers selected by editors
COASP 2017, Lisbon 20-09-2017 @tonyR_H

8. “Open Peer Review” encompasses diverse
constellations of many distinct aspects
• Open identities
• Open reports
• Open participation
• Open interaction
• Open pre-review
manuscripts
• Open final-version
commenting
• Open platforms
COASP 2017, Lisbon 20-09-2017
Primary
aspects
Secondar
y aspects Image CC BY AC McCann, w/ amendment (by me)
** 122 definitions collected and analysed **
** 22 distinct configurations of 7 traits identified **
See: Ross-Hellauer, 2017, "What is open peer review? A
systematic review", F1000Research (DOI:
10.12688/f1000research.11369.2)

9. Distribution of OPR traits amongst definitions
Ross-Hellauer, 2017, "What is open peer review? A systematic review",
F1000Research (DOI: 10.12688/f1000research.11369.2)

10. 22 unique configurations of OPR traits
Ross-Hellauer, 2017, "What is
open peer review? A systematic
review", F1000Research (DOI:
10.12688/f1000research.11369.2)
@tonyR_HCOASP 2017, Lisbon 20-09-2017

11. Slide, Edit Gorogh, OpenUP
• 11
COASP 2017, Lisbon 20-09-2017

12. ATTITUDES TO OPR
COASP 2017, Lisbon 20-09-2017 @tonyR_H

13. Focus groups
• Find ways to acknowledge
and reward reviewers
appropriately
• Support development of
mandates and policies
• Create a common code of
conduct, esp taking into
account the needs of young
COASP 2017, Lisbon 20-09-2017
See: Deppe, A., Hermans, E., & Ross-
Hellauer, T. (2016). Open Peer Review -
Models, Benefits and Limitations /
Workshop Report 2016. Zenodo.
http://doi.org/10.5281/zenodo.61378
OPR Workshop - Göttingen, Germany 7 June 2016
• Very heterogeneous views on what OPR is
• No one-size-fits-all: Must respect stakeholder views and
cultural differences across disciplines
@tonyR_H

14. Survey on attitudes to
OPR
• Online survey
• Open for one month (Sept – Oct 2016)
• 3062 complete responses from authors,
reviewers and editors
See: Ross-Hellauer, Deppe & Schmidt, 2017,
"OpenAIRE survey on open peer review: Attitudes
and experience amongst editors, authors and
reviewers" (preprint) (DOI: 10.5281/zenodo.570864)
COASP 2017, Lisbon 20-09-2017 @tonyR_H

15. COASP 2017, Lisbon 20-09-2017 @tonyR_H

16. COASP 2017, Lisbon 20-09-2017 @tonyR_H

17. COASP 2017, Lisbon 20-09-2017 @tonyR_H

18. COASP 2017, Lisbon 20-09-2017 @tonyR_H

19. Survey results summary
• OPR is already mainstream
• 76.2% have practical experience
• 60% believe OPR should be common
practice
• Positive reactions to most OPR traits (esp.
open interaction, reports, participation)
• However, strong rejection of open identities
(47.7% against)
COASP 2017, Lisbon 20-09-2017 @tonyR_H

20. NEXT STEPS
COASP 2017, Lisbon 20-09-2017 @tonyR_H

21. We need …
• More transparency – being clear on peer review
policies and what implications for
reviewers/authors
• More education – what OPR is, how to review
responsibly
• More incentives - make reviews count by making
them citable, discoverable, and creditable
(Crossref)
• More evidence …
COASP 2017, Lisbon 20-09-2017 @tonyR_H

22. “What is open peer review
– and should I be doing it?”
Libby Pier, July 2017
https://libbypier.com/thoughts-musings/2017/7/14/what-is-open-peer-review
COASP 2017, Lisbon 20-09-2017

23. • OPR is a very complex issue – what should be made
open, in which circumstances, at what stage, to whom?
• “The large number of possible configurations of options presents
a tool-kit for differing communities to construct open peer review
systems that reflect their own needs, preferences and goals.”
(Ross-Hellauer, 2017)
• We need more evidence to help judge!
• “[T]here is often little evidence to support or refute many of these
claims [regarding OPR]” (Ross-Hellauer, 2017)
• What I think we need
• Open up the data!
• Multi-stakeholder agreement on definitions and priorities for
research
• Cross-publisher/-journal studies on what works and what doesn’t
• See:
http://blogs.lse.ac.uk/impactofsocialsciences/2017/09/13/open-
peer-review-bringing-transparency-accountability-and-inclusivity-
to-the-peer-review-process/
COASP 2017, Lisbon 20-09-2017 @tonyR_H

24. www.openaire.eu
@openaire_eu
facebook.com/groups/openaire
linkedin.com/groups/OpenAIRE-
3893548
Thank you!
ajcross01@gmail.com
COASP 2017, Lisbon 20-09-2017 @tonyR_H