Jessica Polka talking about the future of Peer Review at the OpenUP Final Conference. Jessica Polka is Executive Director of ASAPbio, a researcher-driven non-profit working to promote innovation and transparency in life sciences communication. ASAPbio aims to accelerate cultural change in two areas: preprints and open peer review reports. She became a visiting scholar at the Whitehead Institute and a research affiliate at MIT Libraries following postdoctoral research in synthetic biology at the Harvard Medical School and a PhD in biochemistry and cell biology at the University of California, San Francisco. A few words about OpenUP Final Conference - Review | Assess | Disseminate OpenUP Final Conference is the final conference of the EU funded H2020 project OpenUP. In OpenUP Final Conference, key aspects and challenges of the currently transforming science landscape were showcased in different interactive sessions, including an Open Science Cafe and Marketplace for new and innovative tools, methods and ideas. Different Motivate and Meet sessions fostered interaction and exchange in the context of Open Science. It brought together different stakeholders who have a "stake" in the researcher lifecycle and helped them to learn about innovative methods for peer review, dissemination of research results and impact measurement, and get involved in shaping open science policies meeting their needs. More information about OpenUP Website: http://openup-h2020.eu OpenUP Hub: https://openuphub.eu Twitter: https://twitter.com/ProjectOpenUP Facebook: https://www.facebook.com/projectopenup/

1. Illustration by David Parkins
The future
of peer
review
Jessica Polka
@jessicapolka
Executive Director, ASAPbio
@ASAPbio_
These slides: goo.gl/jxTvoP

2. ● Published
● Early (preprint feedback)
● Recognized
● Transparent policies
What does the future of peer
review look like?

3. Peer review is worth publishing
“Paper Shredder” by Sh4rp_i is licensed
under CC BY 2.0

4. Why publish (anonymous) peer reviews?
● Encourages constructive reviewer behavior
● Peer review is scholarship
● Earns the trust of readers
● Makes journal decisions more transparent
● Opens a door for reviewer recognition
● Helps trainees
● Enables the study of peer review
#PublishPeerReview
View article

5. Researchers like the idea of publishing peer review
Ross-Hellauer
2017

6. What are the barriers?

7. Reviewers will pull their punches
Reviewers won’t want to review
Authors won’t want to submit
asapbio.org/pr-faq
#PublishPeerReview

8. “Weaponization”
#PublishPeerReview

9. Addressing weaponization
1. Disclaimer on peer
reviews?
2. Lay summary?
3. Interacting with
journalists, etc
#PublishPeerReview

10. Amplification of bias
link
♀ ♂ ♀ ♂ ♀ ♂
Corresponding First Last
Gender of author
%offullsubmissionsaccepted
#PublishPeerReview

11. asapbio.org/peer-review/summary
#PublishPeerReview

12. View article

13. Open letter on the publication of peer review reports
Letter
We, the undersigned journals, recognise the benefits of transparency in
the peer review process. Therefore, we enable or undertake to enable
the publication of all of the content of peer review, but not necessarily the
names of reviewers (this is also called open peer review reports) and
author responses alongside final, published articles.
We recognize that implementations of published peer review reports may
vary—with some journals mandating it for all published articles, while
others may offer authors an opt-in or opt-out option—providing an
opportunity to compare experiments across different journal policies and
fields. In the pursuit of best practices, we commit to sharing information
about community responses to varied implementations.
We encourage other journals to join us in this initiative. We hope that our
action inspires the community, including researchers, research funders,
and research institutions, to recognize the benefits of published peer
review reports for all parts of the research system.
asapbio.org/letter
#PublishPeerReview

14. 127 signatories, including some that commit to
publishing peer review in the future
Journal(s) Publisher Contact Expected implementation date
PLOS Biology PLOS Veronique Kiermer Q2 2019
PLOS Computational Biology PLOS Veronique Kiermer Q2 2019
PLOS Genetics PLOS Veronique Kiermer Q2 2019
PLOS Medicine PLOS Veronique Kiermer Q2 2019
PLOS Neglected Tropical
Diseases PLOS Veronique Kiermer Q2 2019
PLOS ONE PLOS Veronique Kiermer Q2 2019
PLOS Pathogens PLOS Veronique Kiermer Q2 2019
Journal of Cell Biology Rockefeller University Press Rebecca Alvania & Jodi Nunnari Late 2018
Proceedings of the Royal Society
B Royal Society Spencer Barrett Early 2019
Development Company of Biologists Claire Moulton Early 2019
Journal of Cell Science Company of Biologists Claire Moulton Early 2019
4open EDP Sciences Claus Roll Early 2019
#PublishPeerReview
asapbio.org/letter

15. Early

16. https://www.facebook.com/groups/853552931365745/permalink/1349684805085886/
https://osf.io/sgpe9
/
The power of early peer review
#ASAPbio

18. Preprint feedback can inform journal
decisions
“In addition, the journal reserves the right--but is not obligated--to
consider the comments made to manuscripts posted to preprint
servers and factor these comments into final decisions at any
stage of the peer review process.”
http://www.fasebj.org/site/misc/edpolicies.xhtml#Preprint_Submissions
#ASAPbio

19. Preprint servers as a marketplace for editors
#ASAPbio

20. Preprint servers as a marketplace for editors
“we now have a dedicated team of
editors who will focus on identifying
[preprints] that are potentially suitable for
publication in PLOS Genetics.” *
* Bringing PLOS Genetics Editors to Preprint
Servers
Gregory S. Barsh, Casey M. Bergman,
Christopher D. Brown, Nadia D. Singh, Gregory P.
Copenhaver
Published: December 1, 2016
https://doi.org/10.1371/journal.pgen.1006448
#ASAPbio

21. Preprint commenting venues
JMIR Preprints
Publons
F1000Prime
F1000Research and funder-controlled platforms
SciRate
Episciences
Copernicus Interactive Public Peer Review
Mendeley
Peeriodicals
PubPeer
preLights
PREreview
APPRAISE
Peer Community In...
Academic Karma
Peerage of Science
biOverlay
ScienceOpen
Self-Journal of Science
Hypothesis
More details in
this spreadsheet
#ASAPbio

22. Recognized

23. Recognizing ghost
peer reviewers
How many “individual” peer
reviewers actually represent a
hidden number of ECRs carrying
out peer review with and for them?
#ECRPeerReview
futureofresearch.org/ecrpeerreview/

24. eLife peer review survey
eLife survey
“264 researchers took
part in the survey,
including 146
postdoctoral researchers
(55% of the total), 61
group leaders (23%) and
51 PhD students (19%)”
“although 37% of PhD
students still performed
their review without the
assistance of their
advisor.”
#ECRPeerReview

25. Help us learn more about peer review ghostwriting -
survey open until September 21:
#ECRPeerReview
tinyurl.com/ECRs-in-peer-review

26. Transparent policies

27. transpose-publishing.github.io/

28. Thank you!
ASAPbio Board of Directors
Ron Vale (President)
Cynthia Wolberger (Vice President)
James Fraser (Secretary &
Treasurer)
Prachee Avasthi
Daniel Colón-Ramos
Tony Hyman
Heather Joseph
Harlan Krumholz
Harold Varmus
Dick Wilder (non-voting)
Future of Research
Gary McDowell, Becki Lijek, & BoD
2018 ASAPbio meeting coorganizers
Robert Kiley (Wellcome)
Bodo Stern & Boyana Konforti (HHMI)
ASAPbio Funders
Simons, Sloan, Arnold, Moore, Helmsley,
Wellcome, CZI, HHMI, MRC, CIHR