Frontiers reviewer told: don’t be strict, endorse paper, reports Giulia Liberati

Journal peer review is a mysterious black box all scientists fear. The task of the reviewers is to help authors to improve their manuscripts scientifically and to help journal editors to weed out scientifically substandard and inappropriate works. That’s the theory anyway, in practice there are good reasons why the peer review process is traditionally something to be hidden by all means from the readers of published papers. Probably to avoid occasional shock, disgust  and repulsion, similar to how the supermarket customers should by no means be made aware of the true origins of industrially raised meat. In a kind of a vicious circle, this peer review secrecy is a direct invitation to rig it even more. Editors tend to assign friendly reviewers according to authors’ eminence, while peer reviewer conflicts of interests are routinely disregarded, since no one will ever find out anyway. In the same vein, scientists who made themselves some powerful enemies will see their manuscripts destroyed by unreasonable and aggressive peer review. They often naively hope the editor was decent enough not to invite those same adversaries whom the authors specifically asked to be excluded.

There is even good money to be made from gaming the peer review process. The subscription publisher Taylor & Francis offers manuscript authors premium peer review services: 300€ per page for rapid peer review, with 5-6 weeks “submission to acceptance” and €635 per page for “fast track”, which basically guarantees you to publish your paper in only 3 weeks latest. Now imagine if you paid Taylor & Francis €2000 for a 3-page manuscript, only to get rejected in fast track process- would this make any sense at all? The system can obviously only work to everyone’s satisfaction if all pay-per-peer-review papers are accepted without any hustle or demand for extra experiments in the prepaid time.

This is why so many journals prefer to keep their peer review process “confidential”. Even progressive open access publishers like PLOS leave their readers in the dark about how any given questionable paper got to be published there. Some journals switched to publishing peer review reports, though mostly depending on authors’ approval (see my article here), and some like the Swiss open access publisher Frontiers only list the names of reviewers and editor. Which may help readers to uncover occasional lack of academic or research field qualifications, certain conflicts of interest or to learn that the handling editor was also reviewing the paper instead of recruiting independent reviewers, but this is where the buck stops. The important part, namely the peer reviewer reports, remains secret with Frontiers. As we will see below, some of those consist only of a box-ticked approval.

Frontiers editors recruit reviewers from the vast pool of review editors of a given Frontiers journal. Precisely, these reviewers are largely postdoctoral scientists, but also PhD students and even non-academics, who are led to believe they are members of the “editorial board”. Which they are most obviously not, these review “editors” have zero influence over the editorial processes at their Frontiers journal. However: although they can be reviewing for many respectable journals, where else but with Frontiers can junior scientists receive the merits of membership to a journal editorial board? The cheap trick with these phony “editorial board” credentials works therefore surprisingly well.

Until recently, these “review editors” were not allowed by Frontiers rules even to advise a rejection of a manuscript, however abysmal it was. All they could do was to withdraw, only to be replaced by more cooperative members of the review editor pool, or the editor took over the review job. Now Frontiers reviewers are permitted to recommend rejection, but: as long as at least one of them keeps endorsing publication, rejection is not possible. In fact, also the associate editors, who are charged with managing the peer review process, are not allowed to reject manuscripts after they entered peer review, only chief editors can (see details here). Chief editors are also the only ones who receive a salary from Frontiers, their main task is specified in the contract: to make sure a certain number of papers is published in their Frontiers journal per year. Aside of inviting various research topics and collections to reach that number, Frontiers chief editors have a very limited influence over their own journals. Those who attempt to set editorial and review standards which do not agree with the rules of the publisher, get sacked. This is probably why none of Frontiers chief editors I ever attempted to interact with showed any desire to face responsibility for some rather questionable works which their own journal published.

The Frontiers process is streamlined to publish as much and as quickly as possible. No wonder many reviewers and handling editors abandon all pretence at ethics and integrity to wave almost everything through, or quit. Sometimes they quit officially, like Guillaume Rousselet, sometimes it’s a passive resignation: they remain listed as editorial board members, but refuse to react to editing and review requests from the Frontiers “editorial” office.

CjuriTcWEAAkjt8 twitter

Below a guest post by one such frustrated Frontiers reviewer. Giulia Liberati is post-doctoral researcher at the Institute of Neuroscience of Université Catholique de Louvain in Belgium. She specializes in nociception and tells of her experience of reviewing for the Frontiers flagship founding journal, Frontiers in Neuroscience (where the big boss himself, Henry Markram, used to publish and where he still hold the reins). I contacted twice the responsible associate editor as well as the chief editor and Markram’s closest collaborator Idan Segev and three Neurodegeneration speciality chief editors for a comment. Since there was no reply, I can only offer here Segev’s video as reference, where he makes clear that Frontiers philosophy is about not rejecting papers of paying customers.


By Giulia Liberati, neuroscientist at Université catholique de Louvain.

In July 2014, Prof. Tibor Hortobágyi, editor of Frontiers in Neuroscience, invited me to review a Methods article by Midorikawa, Itoi and Kawamura, entitled “Detection of residual cognitive function through non-spontaneous eye-movement in a patient with advanced dementia”.

Frontiers vs Reviewer: Round 1

From the title, the article seemed to be quite close to some work I had done during my PhD studies, so the invitation made sense. My previous experience with the Frontiers group had been quite good, both as an author and as a reviewer. I personally liked the forum format of the reviews, because I thought it helped both reviewers and authors to be constructive and to the point. I had also appreciated the transparency in publishing the names of the reviewers together with the articles: everyone just had to take responsibility both for the remarks they had made during the review process, and for the manuscripts they accepted. Awesome, wasn’t it?

For these reasons, I decided to accept the invitation to review the paper. As I went through the manuscript, however, I was more and more astonished. Was this some kind of joke? The paper was not written in proper English and was difficult to understand. But what really struck me was that some of my previous work, as well as some studies from my colleagues, were completely misquoted. I knew these works well, and I could see that the authors were using these references in a completely random way. In addition, the article was supposed to be a “Methods” article, but was rather a very weak case study with a limited number of trials (eight) on a single subject, with a very questionable experimental procedure and overstated conclusions.

I listed all these problems in my review, also insisting on the language issue. Furthermore, I requested to remove the incorrect literature citations, and pointed out that the article was not a Methods article, but something closer to a “clinical case” study (though a very weak one). I was very firm, but still polite.

Frontiers vs Reviewer: Round 2

Once I submitted my comments, I could see the other reviewer’s comments in the forum. Incredibly, the other reviewer had endorsed publication of the manuscript, without comment! This was unbelievable! No scientist in good faith could have accepted such rubbish. Even wanting to agree with the manuscript’s content, it was clearly not even written in proper English! How could the other reviewer have no comments at all?

However, some days later, I was notified that some comments had been added to the review forum. To my surprise, the Editor himself had added a remark to my comments. I unfortunately cannot cite the remark verbatim, because I do not have access to the forum anymore. Anyway, the remark was something like:

“I encourage the authors to improve their paper according to the reviewer’s request. However, I would like to ask the reviewer to be less strict. Hopefully, the reviewer will be satisfied after the changes and endorse publication”.

Wow, I couldn’t believe my eyes. This was the first time an editor appeared to interfere with my review, putting pressure on me to accept a paper. My aim was not to be unreasonably strict, my aim was to be objective. Everything I had written could be directly verified. And that paper was very far from being endorsable.

Frontiers vs Reviewer: Round 3

Some weeks later, the authors updated their submission with a new version of the manuscript, this time submitted as a clinical case report. In addition, I received an e-mail from Frontiers, requesting my approval for changing the type of article from Methods to Case Report. The authors had removed some of the wrong citations, but the English was still problematic, and the paper still had major methodological problems.

I wanted to point out these issues in the forum, but I was concerned by the very short time frame I was given to complete the review, i.e., one week. Since I was going to leave for a one-week holiday, I decided to inform the editor that I would have been happy to provide feedback, but one week later. Would that still be ok?

The Editor replied that it would have been more helpful for the authors to have my feedback before I went on holiday. Could I please submit my comments just before leaving? Because there was no way I could write an accurate and helpful review in one (already busy) afternoon, I decided to ignore the editor’s request. I had been honest about my time constraints, so if he really needed an immediate review, he could ask somebody else.

As I came back from my holiday one week later, I added my comments in the forum, once more pointing out the language issue and the methodological limitations of the study. Again, the Editor intervened. This time, he wrote:

“I suggest the authors to write a few more relevant sentences to make the reviewer happy. I ask once more the reviewer not to be too strict and to finally endorse the paper”.

Knock-out

I was enraged! Why was I being asked to review the paper anyway, if my comments did not count at all? I was not simply asking for “a few more relevant sentences”. The English was just too bad, the methodological issues were just too strong. I tried to reply a couple of times to the Editor. At one point, I was sad, frustrated, and exhausted. Having understood that the paper was going to be accepted with or without my endorsement, I decided to resign from the review process.

In resigning as a reviewer, I was asked for an explanation. I still have my reply saved on a word document, so here it is:

“As I feel that I am being put under pressure to accept an article which I do not consider to live up to Frontiers in Neuroscience (neither in its current form, nor with ‘a few more relevant sentences’), it is with regret that I have made the decision to withdraw from the review process, also because I do not want my name to be listed within the reviewers who endorsed the article once it will be published”.

And the winner is: Frontiers!

The paper was eventually published, apparently, as a Methods article in the end [Frontiers charges authors for a “Methods” article more than twice as much as for a “Case Report”, $2490 vs  $1150 -LS]. The Editor is now listed also as reviewer. No third reviewer was asked to give their opinion. The Editor simply replaced me. The English in the published paper appears to be polished, and some of my suggestions were taken into account, but in general, it remains a very low quality article. After exactly two years, it was not even cited once. There is unfortunately no trace of the fact that one reviewer decided not to endorse the paper. I do not even have access to the review forum to “prove” my existence as a former reviewer. So, where is Frontiers’ claimed transparency now?

I later decided to write a complaint to the Frontiers Editorial Office (editorial.office@frontiersin.org), as they encourage researchers to write to them for general comments, suggestions, or queries. I explained the situation in detail. I never got a response. This behavior is apparently absolutely fine for them, and I don’t even deserve a reply.

Was this the best way for me to handle the situation? Probably not. The paper was accepted anyway, probably more easily after I resigned. Maybe I could have just kept on posting the same comments in the forum over and over again, ignoring the Editor’s remarks. But I was really feeling like wasting my time, and getting more and more frustrated. In the end, it was obvious that the paper was going to be accepted.

I did not officially quit the Frontiers editorial board. Although I had had a very bad experience, at that time Frontiers’ reputation was still ok (as much as I was aware of), and I did trust many of the Editors. The last paper I reviewed was for Frontiers in Human Neuroscience in autumn 2014. I don’t think I have been asked to be a reviewer since then.

10 thoughts on “Frontiers reviewer told: don’t be strict, endorse paper, reports Giulia Liberati

  1. It’s interesting that even as there are trends trying to improve the quality of science (e.g. PubPeer, this blog…), at the same time there are powerful trends in the opposite direction, tending to hollow science out even more. The money and power are on the side of the latter, but what they’re building is a hollow shell, a bubble. As with the financial crisis, huge sums of money are in play, but when confidence collapses the bubble will (presumably) burst.

    Liked by 1 person

  2. Interesting piece as usual – just a few comments.
    1.”Journal peer review is a mysterious black box all scientists fear”… sounds a bit too bleak, don’t you think?
    2.”The task of the reviewers is to help authors to improve their manuscripts scientifically”… Some would disagree, and in different journals reviewers can have different goals. Some reviewers might want just to determine whether paper is publishable or not, not to improve submitted work.
    3.”I did not officially quit the Frontiers editorial board. Although I had had a very bad experience, at that time Frontiers’ reputation was still ok (as much as I was aware of), and I did trust many of the Editor”. First, how come? Much criticism of Frontiers I hear from colleagues is that review editors get bombarded with emails. Weird that this review editor hasn’t received any invitations in two years. Perhaps she could provide more details on whether email address changed or other hypotheses. Second, Dr Liberati has submitted a paper to Frontiers after having experienced this mishandling by the editor: http://journal.frontiersin.org/article/10.3389/fnhum.2014.00865/full
    The argument “if I had known at that time…” rings hollow. In the above mentioned paper Liberati’s coauthor is the editor of the special issue, and one reviewer has Dr Liberati’s previous affiliation. This points to a problem of research topics – special issues, not only applicable to Frontiers of course. These special issues are something like family reunions. Notably, Dr Liberati seems to have taken part in one of these reunions. Not a sin, of course. And “family reunions” can be pleasant and important, of course. But she could be willing to share the reports from reviewers for her submitted paper, for instance.
    Cheers!

    Liked by 1 person

    1. I can only address point 1 and 2, obviously. Point 2 covered just as you suggested in the second part of the sentence:
      “The task of the reviewers is to help authors to improve their manuscripts scientifically and to help journal editors to weed out scientifically substandard and inappropriate works.”
      Point 1: What if the peer review process was fully transparent? Would scientist authors still fear the notorious “Reviewer Nr 3”? Probably much less so, the only downside would be the editors of elite journals losing their “gatekeeper” power. A disastrous prospect which must be avoided at all costs, of course 😉

      Like

      1. Everybody knows about the close relationships between authors and the editors of elite journals in certain specific cases. Unfortunately, these relationships have much more to do with the game of personal interests and power and very few to do with science itself.

        Liked by 1 person

    2. Dear Mark, thanks for your comment. I read your concerns also on Twitter, but it’s easier to reply here where I have more space.

      As for point 3, which is more directly addressed to me:

      I have indeed submitted two papers to Frontiers, both for special issues (which I have found quite interesting in both circumstances). The paper you are referring to required the submission of an abstract some time before the official submission date (that’s how it works for special issues), so months before the described experience. Dr. Thomas Hünefeldt was originally first author, but due to his lack of time, in the end I became first author. I did not consider retracting my submission at that time, because my previous experiences, both as an author and as a reviewer, had been very positive and constructive. My idea was – probably too naïvely – that that specific editor was problematic. Not the entire publication system.

      I also would like to add that it is quite normal for editors (and their co-authors) to take part to special issues. Of course, they do not handle their own submissions, but another editor intervenes in the process (as in the case you mentioned). What is important is that a thorough and unbiased review is carried out. As I had to undergo a normal peer-review process, I did not question this submission modality. I also understand your concern about one of the reviewers. It is true that two years before submitting that paper I was finishing my PhD in Tübingen, where Prof. Kotchoubey also works. Nevertheless, I have never had any collaboration / common project / real interaction with him. Reason why he probably did not consider having any COI in reviewing my paper without bias, two years after I had changed my affiliation.

      One problematic issue that I see in the current peer-review system is that I now basically know all my possible reviewers: from collaborations, meetings, conferences, etc. I think this happens to a lot of researchers working on very specific topics. Anybody could say there is a COI in most researchers’ submission (which can either go against or in favor of the publication…). This is why it would be ideal to have OPEN reviews. That’s what really counts! Not simply having the reviewers’ names at the end of the paper. (A system with open reviews would probably have convinced you better that I had no easy way through publishing by participating in special issues, for instance).

      Finally, regarding Frontiers mailbombing: yes, I still get it! But not for reviewing papers. Just random Frontiers stuff saying they’re the most cited journal in the world… I guess one possible reason why I don’t get invited to review papers is that I changed my field in the last years.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s