This document discusses challenges with the current peer review system and emerging alternatives. It notes increasing pressure to publish has led to more cases of scientific misconduct. It also discusses demands for recognizing reviewers and considering new peer review forms. Several new models are described, including open peer review, pre-registration, post-publication commenting, and cross-publisher review consortiums. The document advocates exploring new services and models while acknowledging traditional peer review still has value.

1. We know that the increasing pressure to publish has led to more cases of fraud and
scientific misconduct, and that there is increasing demand in the broader academic
ecosystem for rewarding or recognising reviewers, as well as considering alternative
forms of peer review.
Overview:
• Trust report
• Current challenges
• What’s new?
• Taylor & Francis initiatives, support and services
1

2. December 2013 saw the publication of a scholarly research project looking at Trust
and Authority in Scholarly Communications in the Light of the Digital Transition.
This builds on other work done by the research group CIBER about researcher
behaviour in relation to scholarly journals.
The report ranges widely, looking at citation behaviour, usage, metrics, etc, but we
are just looking at their findings on peer review:
• what researchers liked,
• didn’t like, and
• were unsure about in the peer review process.
2

3. 3

4. 4

5. 5

6. There are a number of other challenges in peer review of which we are already aware.
6

7. There are a number of new forms of peer review that are emerging, which we’ll
discuss in more detail
7

8. Methodological review, according to publication criteria – but no “filter”. No
responsibility for reviewers to contribute to improvement of content or readability.
Few rounds of revision, so quick.
8

9. The trend towards ‘openness’ is fuelling the greater demand for transparent peer
review. “Open” peer review is often allied with making data openly available as well,
but is usually used to refer to the practice of making all of the reviewing and decision
making process open, including sometimes making the reviewer’s names public.
Just to quickly mention here that T&F, together with a society partner, are working on
an exciting new open peer review journal to be launched in 2016. Look out for further
announcements on this towards the end of the year.
9

10. We’ve always had post publication peer review, in the form of Letters to the Editor,
going as far back as the Philosophical Magazine in 1799
Published comments and rejoinders.
Many journals now adding ability to comment on articles, but very few showing
multiple comments, and of those many are quite trivial notes, far from the depth and
quality of a full peer review.
10

11. In a nutshell, these are various initiatives employing different new forms of post-
publication peer review: variations of commenting on sites with some form of ranking
or rating articles.
You might think that all such post-publication commentary would be completely
“open”, and that these “reviewers” would sign their names to their comments.
However this is not always the case and their have been some instances of authors
being mobbed by anonymous hostile reviews.
T&F have asked some questions about author attitudes to peer review as part of one
of our Open access surveys and found that there’s a gender bias in how authors view
post publication peer review – female researchers are more nervous about it than
men. This also reported in an online survey conducted by BioMed Central on Views
on the peer review system of biomedical journals where female researchers held a
higher perception of unfairness in peer review:
http://www.biomedcentral.com/content/pdf/1471-2288-13-74.pdf
11

12. The idea behind pre-registration is simple: Before you conduct your experiment, you
publicly list exactly what kind of experiment you are going to conduct, how many
participants you will test, and what the predicted outcome is. Once you have done
this, you have very few degrees of freedom to tweak your results afterwards. For
example, if your results are not statistically significant, you cannot keep running
participants until you obtain the desired result (i.e. optional stopping), because you
have specified the number of participants in advance. Similarly, if you obtain results
that don’t match your hypothesis, you cannot confabulate a post-hoc hypothesis that
matches the outcome of your study.
12

13. New peer review consortium of the above publishers that builds on the existing
journal peer review process. Group of journals across publishers, all in same subject
area: Goal: By reducing the number of times a manuscript is reviewed, the
consortium aims to speed publication of research results.
Authors who submit to a participating journal in the consortium, and are not
accepted by that journal, will be able to redirect their paper, with the referee’s
reports, to any other journal in the consortium.
Referees will be given the opportunity to opt out of having their reports forwarded,
or to forward them anonymously (in all cases, the referee’s identity will be
anonymous to the author – referees will choose whether they wish to remain
anonymous to the editors of the secondary journal.
13

14. Rubriq’s model focuses on pre-publication – indeed pre-submission (to a journal)
review. Not a publisher, to provide independent reviews.
‘We provide rigorous reviews by the same qualified peers who review for journals,
but with a standardized scorecard that can be used in any publishing model. Enable
faster, more consistent reviews, and will help match papers with the right journals.’
14

15. We are constantly exploring new ways/services to improve Peer Review processes,
and I’ll just mention two interesting new services that we are in the very early stages
of investigating:
Publons’ aim is to help provide recognition and support for reviewers, which David
has referred to in his presentation. Step included to verify peer review activity, so that
there is an ‘official reviewer record’ that can be used in promotion and funding
applications.
Pre-Val is concerned with verifying and qualifying peer review.
Partnering with PRE-val would provide independent, third party verification of the
peer review process at the journal and article level, reinforcing our commitment to
peer review integrity.
15

16. At T&F we’re flexible – we’ll work with you to trial new systems and ideas and we
know that one size doesn’t fit all. Traditional peer review still holds sway but it is
encouraging and exciting to see other models developing and being supported by the
research community. Also encouraging is the impact that some of these new modes
may have on the wider dissemination and transparency of information and discovery
of unethical behaviour.
We are working with our Author Services team in hosting Peer Review focus
groups in South Africa in April which will feed into a White Paper, alongside
data from other focus groups around the world.
16

17. We also have our Editorial Resources site – specific sections on peer review, providing
information and support and also industry examples.
17

18. 18