Optionally transparent peer review: a major step forward, but which direction?

A journal of the Nature Publishing group (NPG) has recently announced “Transparent peer review at Nature Communications”.

I was dumbstruck. A journal of the notoriously elitist, secretive NPG, is opening its peer review, after going open access (OA) just one year before? Will Nature Communications be the avant-garde of open science at NPG, transforming the entire institution from the inside? Should all these OA and transparency advocating scientists, who turned their backs on established publishers in disappointment, return and start again submitting their research with the new, open NPG?

Hold your horses. The editorial announcement went on with the sub-headline: “Authors of papers submitted from January 2016 will be given the option to publish the peer review history of their paper”. This means, it is up to the authors to decide whether they want the world to see how their paper got accepted at Nature Communications. To me it looked as nothing but a pretence of openness, exactly because where the peer review history is expected to be most informative it might be unavailable. Straight away, I wrote an angry comment below the article that such “openness” is nothing more than a fig leaf, in fact a subversive move against the entire OA and science transparency movement. I even demanded that Nature Communications should instead follow the example of the EMBO Journal, which according to my knowledge published review reports of all their accepted manuscripts.

Amy Bourke-Waite, Senior Communications Manager at NPG has replied to my comment:

“We have taken care in the planning of the Nature Communications transparent peer review trial to review existing schemes used by other publishers. This system – author opt-out, names redacted – is already used successfully by a number of other publishers across several research areas”.

And in fact, Bourke-Waite was absolutely right, and I was wrong. Though I still see optionality in peer review transparency as highly counter-productive, I have to withdraw my reference to EMBO J as inappropriate. In fact, also there, transparent peer review is optional for the authors. As the Chief Editor of EMBO Press, Bernd Pulverer, put it in his email to me:

“While we encourage open peer review, where referees sign their reports, in my view this ought to remain voluntary – at least in the research area of molecular cell biology”.

Pulverer shared on behalf of his journal these thoughts on the recent Nature Communications announcement:

“We are encouraged to see a number of journals adopting a more transparent approach to the editorial process. In our view, publishing the referee reports, as well as all the relevant communication between the editors, authors and referees alongside papers in a minimally edited and redacted form is a powerful and readily implemented mechanism. It adds transparency to the process itself, but importantly it also is the best way for new referees to train themselves by analyzing the process on published papers. It adds accountability and three additional takes on a given dataset by world experts. A maximally transparent process therefore enriches every paper”

As my quick research showed, aside of the new, post-publication peer review platforms like F1000 Research and ScienceOpen, hardly any journal has been operating a truly transparent, non-optional peer review system so far (correction: I previously mentioned PeerJ is this regard, but this journal operates an optionally open traditional peer review and is not a post-publication platform. I apologise for the mistake). One such exception I found was the highly respected medical journal The BMJ, which has introduced a “fully open peer review” for all its research papers around one year ago:

“This means that every accepted research paper submitted from September 2014 onwards will have its prepublication history posted alongside it on thebmj.com.

This prepublication history comprises all previous versions of the manuscript, the study protocol […], the report from the manuscript committee meeting, the reviewers’ signed comments, and the authors’ responses to all the comments”.

At The BMJ, nobody can opt out. According to EMBO J, the participation rate is 95%, which is quite high indeed. It could also be interpreted that with remaining 5% of EMBO J publications, their authors believe the relevant peer review history is best to be kept secret. Also at the OA journal eLife, founded in 2012 by the Nobelist and former PNAS chief editor Randy Scheckman, the participation in open peer review was until now voluntarily. As I was told by the Executive Director at eLife, Mark Patterson:

“We publish the decision letter and response along with the article, subject to the authors’ approval. However, very few authors have elected not to publish the decision letter, so in the New Year we will publish the decision letter and response for all accepted articles”.

The high transparency participation rates at eLife and EMBO J may stem from their high fussiness. A major proportion of the submitted manuscripts are in fact rejected without peer review at these journals, while only one round of revision is allowed. This prevents authors from haggling and beleaguering their way in over months and even years of repeated rounds of peer review. Another common issue is that the peer review process is moderated by the handling editor. As Patterson explained, at eLife “the reviewers’ identities are open to one another”. He recommends this approach of “consultative peer review” to Nature Communications as well. At EMBO J, Pulverer mentioned the “referee-cross commenting, where referees are encouraged to discuss each other’s reports”.

However, at less elitist journals, where the peer review is done more traditionally, the participations rates at open peer review are less impressive. The journal Royal Society Open Science runs an optional peer review transparency as well. This is the table of participation rates I received from Matt Allinson, the journal’s editorial co-ordinator:

Like Pulverer and Patterson, Allinson also is in favour of the voluntary peer review:

“We allow authors to change their mind at any point from original submission up to publication as to whether or not they want to go for open peer review, so some authors at the beginning may be hesitant but then opt in, and other authors may have been keen to begin with decide after review to choose to keep the report confidential.

I feel the importance of voluntary open peer review is that it still allows the authors a degree of control over what gets published with their name on. […] Authors who receive glowing reviews like to let everyone know, but that’s not to say that people with initially negative reviews hide these either”.

Some research areas at the Royal Society Open Science show high levels of peer review transparency. Yet for genetics and whole organism biology, authors choose to opt out of transparency in case of almost every second paper, concerned about exposing the story behind their paper’s acceptance. Allinson delivers an interesting explanation:

“Biology (whole organism) features a lot of papers which get cascaded from Proceedings B, which have a lower portion of authors then opting for open peer review, hence why biology whole organism is a little lower”.

Precisely this is the potential snag for the new Nature Communications approach. Traditionally, a manuscript is rejected in the elite journals of the Nature family calibre (and top-tier journals like EMBO J and eLife as well) with the editorial advice to submit it to “a more specialized journal”. However, Nature Communications was primarily intended as such specialized journal, with the purpose of catching all these manuscripts rejected upstairs at the Nature family. The editors made perfectly clear themselves:

“Nature Communications instead provides a venue for the many comprehensive, rigorous and often elegant studies that do not have such broad appeal – the so-called ‘specialist interest’ papers”.

The rational behind this was actually more of monetary nature: if a paper is rejected for not being good enough, the elite journal retains its reputation and precious impact factor, but loses the cash from the publication fees. To avoid the customer’s dollars, euros, pounds and yen from going elsewhere, publishers introduce new journals to catch up the papers rejected at their premium outlets.

The article processing charges (APC) at Nature Communications are princely €3,700 per article, nothing to be sniffed at. Later on, as Nature Communications gained respectability, NPG has introduced Scientific Reports, another open access journal where papers previously rejected at Nature Communications are often published . The APC at Scientific Reports is not bad either: €1,165 per article. But even there, surely some manuscripts are perceived as not good enough. This thinking may have been behind NPG majority investment into the Swiss publisher Frontiers in 2013, which NPG seemed to have abandoned in favour of its later merge with Springer.  The new SpringerNature now operates the OA publishers SpringerOpen and BMC, which contain a number of small less-known journals, likely quite eager to accept papers rejected at Nature Communications and Scientific Reports. The APCs there are around €1000-1500 per article. The money then can nicely stay in the house of SpringerNature.

So what does this “manuscript transfer” business mean for the author participation in peer review transparency?

In the case of Nature Communications, potentially a lot. Seemingly sympathetic editors offer authors, after having rejected their manuscript, to get it swiftly forwarded to a “lower” journal inside the same publishing house. The advantage is tremendous, since not only the manuscript, but also its editorial history, including peer review, are transferred in this way. The receiving journal can therefore easily dismiss with another round of peer review, and accept the paper based on the reviewers’ reports from the original submission “upstairs”. This is not only convenient for the editors and the publisher, but also saves the authors lots of time and lots of stress while struggling to get their paper accepted for publication somewhere. In fact, editors even specifically recommend authors a certain “downstairs” journal for the transfer of the rejected manuscript, as a nudging hint about its almost certain immediate acceptance there.

Therefore, let us I assume I am a group leader in a major western research institution. I submitted a paper to Nature (or Nature Medicine or Nature Genetics, etc) and even after a prolonged peer review battle I failed to convince the reviewers that my experimental data was actually showing what I claim it shows, or the editors about the reviewers’ lack of competence. The editor remains adamant and issues a final rejection. However, I am offered to transfer my manuscript to Nature Communications. Assume, I take the hint and agree, since an Impact Factor of 11.5 is actually not so bad after all. Very soon, I receive a congratulatory acceptance email from Nature Communications editorial office. Without any further external peer review, the editors had a good look, and read the previous peer reviewer reports. I am now asked if I want these peer reviewer reports published openly. Do they think I am barking mad? So every Tom, Dick and Harry can read what low opinion the reviewers had of my experimental data and its interpretations? No way! Thanks to the proffered optionality, these peer reviews will remain my paper’s well hidden dirty secret. Thank you, NPG, thank you Nature Communications, for being so considerate of a scientist’s needs.

Optional peer review transparency can therefore become a counter-productive and deceitful half-measure. Richard Smith, former editor at The BMJ (which now runs fully transparent peer review) and activist for OA and transparency in medical research, commented on the new Nature Communications model in his email to me:

“I agree that this is a silly policy and a tiny step into the past not the future. As you well know, many journals have been much more transparent with their peer review. Indeed, this might be described as a cloudy rather than transparent policy in that it’s partially transparent–with authors and peer reviewers able to opt out and editors’ comments remaining confidential.

The thing that strikes me most forcefully is that a scientific publication should make all sorts of anecdotal statements with no evidence whatsoever–when lots of evidence is available, almost all of it showing the downside of peer review. Peer reviews, as I always say, is faith not evidence based.

And “half policies” can lead to problems. One journal I was responsible for released reviewers’ comments if they gave consent. In one case of a paper that was rejected a positive review was released but a negative one kept confidential”.

Indeed, what exactly is the point of such half-hearted transparency in peer review, and whose interests does it serve?