Laurie Goodman, the Editor -in-Chief of GigaScience discusses the issues surrounding Open Peer Review. These slides cover the concerns about open peer, and presents the current evidence supporting or rejecting the realities of these concerns. Also included are the potential benefits of open-peer review and addresses the pervasive fear of revenge by the authors for a negative review.

1. WORRIES ABOUT
OPEN PEER REVIEW
Laurie Goodman, PhD
Editor-in-Chief, GigaScience
Laurie@gigasciencejournal.com
ORCID ID: 0000-0001-9724-5976
Journal: @GigaScience Personal: @Grimhawk1

2. How We Review at GigaScience
• All reviews are signed (Mandatory)
• All reviews (and all pre-publication history) are available upon
publication (Reviews of rejected manuscripts are not available)

3. Concerns about Open Peer Review
•Reviews are Less Critical
•Is there a Difference in Review
Quality?
•Finding Reviewers Willing to be
Named will Delay the Review
Process

4. But the Biggest Concern about Open Peer
Review is—
REVENGE
If you give a negative review, the author
knows who you are and at some point may
return the ‘favor’.
I will get back to this at the end of the talk

5. Reviews are Less Critical
A named reviewer will be less likely to be
critical of a manuscript.
Presumably this stems from the fear of
reprisal from the author at some point in the
future: when the author becomes your
reviewer

6. Open Reviews are not Less Critical
There was no significant difference in the recommendation regarding publication

7. Open Peer Review Doesn’t Mean Better
Peer Review
• People often state that if the review is
open, the review will be better, because it
will be scrutinized by more people.

8. There is No Difference in Quality
There was no significant difference in quality (scored on a scale of 1 to 5)
between anonymous reviewers (3.06 (SD 0.72)) and identified reviewers (3.09
(0.68)) (P = 0.68, 95% confidence interval for difference - 0.19 to 0.12),

9. There is a Difference in Quality: It’s Better
There was a difference in the overall quality of
reports between the open and closed peer review
journals. The overall score was 5% higher under
the open model.
This higher score was primarily due to better feedback on
the methods (11% higher), constructiveness (5% higher),
and amount of evidence substantiating reviewers
comments (9% higher)
A comparison of the quality of reviewer reports from
author-suggested reviewers and editor-suggested
reviewers in journals operating on open or closed peer
review models
Kowalczuk et al http://f1000research.com/posters/1094564 Nov. 6, 2013

10. Leek JT, Taub MA, Pineda FJ (2011) Cooperation between Referees and Authors Increases Peer Review Accuracy. PLoS ONE 6(11):
e26895. doi:10.1371/journal.pone.0026895
http://journals.plos.org/plosone/article?id=info:doi/10.1371/journal.pone.0026895
There is a Difference in Quality: It’s Better

11. Open Peer Review Delays Publication
It is more difficult to find a reviewer if
they must agree to be named.
• This will substantially delay the review
process

12. Open Peer Review Can Delay Publication
From
• Reviewers who were randomized to be asked to be
identified were 12% more likely to decline to review than
reviewers randomized to remain anonymous (35% v
23%).

13. A discussion of in a Peer-to-Peer Blog raised this exact
issue, where Eric Perlman (Florida University of
Technology) stated:
“Twice in the past five years I have been asked by an
editor to serve as a referee for the referees. What I mean
by this is that a paper had received a negative report, with
which the authors had disagreed, and perhaps a second
(in one case) from a second reviewer. In one of the two
cases this had gone two rather than one round.”
http://blogs.nature.com/peer-to-peer/2006/06/perspective_the_pros_and_cons.html
The back and forth in responding to reviews due to appeals,
additional editorial involvment, and requests for an additional
Reviewer to review the reviews for fairness, can contribute
(substantially) for delays in publication
Closed Peer Review Can Delay Publication

14. But… Revenge by the author!
I’ll get to that-
Benefits First.

15. Additional Benefits of Open Peer Review
•Increased accountability
•Increased transparency
•Increased Fairness
•Credit for Reviewers
•Multiple Benefits to Everyone

16. Benefits for Reviewers
• Reviewers can get credit for their hard work.
• This is especially important when reviewers have
made a significant improvement in the manuscript
• A review can be cited
• A reviewer can provide clear and open
evidence of senior-level scholarly participation
in their for future job applications or obtaining
green cards.
• When I worked at a journal with closed peer-review, I written
many letters stating that an individual had participated in the
peer-review process for them to include as part of their
application for a job or a green card- any delay or non-
agreement on my part could impact their application.

17. Benefits for Everyone
• Greater transparency of the review
process
• Ability to assess why ‘things go wrong’ in
a review
• Not to blame the reviewer- but to see how
something was missed
• Extensive material available for training
young researchers how to carry out peer
review.

18. REVENGE!!!!

19. The Revenge of the Authors
The Primary Fear of Naming Yourself:
If you give a negative review—
The author knows who you are and may
return the ‘favor’ at some point in the future.

20. Revenge of the Authors
Anonymity does not necessarily protect reviewers:
• In small fields, there aren’t that many individuals
who could be used for review.
• In these fields anonymity by not signing your name
is an illusion.

21. Revenge of the Authors
Anonymity does not necessarily protect reviewers:
• Regardless of the field, many authors “Guess the
Reviewer” so, they can still try to get revenge
• Worse though- authors are most often WRONG in
their guess of who the reviewer is.
• Thus people who were not the reviewer can be
negatively impacted should author revenge be
commonplace…

22. Revenge of the Authors
• Beyond anecdotal evidence, there is no
data to support that this happens.
• Perhaps a study for this should be carried out
(Unsure how that could be done…)
• An alternative is that every review process be
open — reprisal would be more difficult

23. Tyranny of the Reviewers
What about the rights of the author?
• Anonymity makes it easier for an unscrupulous
reviewer to be unfairly critical of a manuscript
• Authors only recourse is appealing to the editor
• Named reviews can help the author better
understand the reviewer’s comments when the
points are unclear.
• There is the potential for direct –positive-
interaction between the author and reviewer
• Can quickly clear up misunderstandings and speed the
publication process
• Can further Improve the manuscript through
collaboration

24. Cost of Open Peer Review
The driving force against open peer-review is mainly
Fear of Revenge
This is primarily hypothetical
There is no solid non-anecdotal evidence that this occurs.
It is unlikely to be pervasive.
Anonymous reviewers (or guessed reviewers) can
still be negatively impacted
However, that doesn’t mean
fear of revenge is not worth considering
But—
Should that fear, with all the above caveats,
outweigh all the open-peer review benefits?

25. Thanks to:
Scott Edmunds, Executive Editor
Nicole Nogoy, Commissioning Editor
Peter Li, Lead Data Manager
Chris Hunter, Lead BioCurator
Rob Davidson, Data Scientist
Xiao (Jesse) Si Zhe, Database Developer
editorial@gigasciencejournal.com
database@gigasciencejournal.com
@GigaScience
facebook.com/GigaScience
blogs.openaccesscentral.com/blogs/gigablog
Contact us:
Follow us:
www.gigasciencejournal.com
www.gigadb.org