Part 2: Trolling peer review to promote its ethics

In this follow-up to my previous article, I will describe two cases of how the editorial and peer review weaknesses at a Frontiers journal were exposed or abused (depending on viewpoint) by its own academic editors.

In the first case, an established scientist, Nikolaus Kriegeskorte, research group leader at the MRC Cognition and Brain Sciences Unit in Cambridge, UK, cleverly revealed the deficiencies of Frontiers peer review, while simultaneously defending peer review ethics. By placing, with the help of his own student as handling editor, largely data-free (and scope-unrelated), yet peer-reviewed opinion articles in the journal Frontiers in Computational Neuroscience,  Kriegeskorte demonstrated that Frontiers’ official high editorial standards can easily be circumvented. In the second case, a professionally rather unrelated medical surgeon has imposed himself as associate editor of this journal, and allowed a publication of an original research paper by his senior colleague which can only be described as disastrous. Nowhere was there any protest or interference from the journals’ chief editor or from the publisher.

At Frontiers, every manuscript submission is subject to a “standardized and rigorous review”. Certain article types like editorials, book reviews or classifications can be reviewed by the associate editors alone. Otherwise, a minimum of two peer review reports is absolutely required. This applies not only to original research, methods or literature review articles, but also, somewhat puzzlingly, to Hypothesis & Theory as well as to Opinion articles. These generally contain ideas or personal viewpoints, not substantiated by actual research data or even published literature. Yet the peer reviewers are apparently invited to somehow either check the soundness of author’s thinking or to simply prompt the author to think more. As I was told by one former peer reviewer, the authors are often simply asked to further elaborate their visions. The publisher Frontiers also occasionally encourages authors to elaborate their thoughts long enough to reach the length of a proper article, which incidentally become subject to some serious article processing charges.

As the cases below exemplify, the peer review process at Frontiers occasionally went against the recommendations of the International Committee of Medical Journal Editors (ICMJE) in regard to the conflict of interest, and in fact even Frontiers own COI guidelines. Also, the editors and reviewer themselves did not always really satisfy the Frontiers own demands on their qualifications.

This is what Frontiers expects of its editors:

“Frontiers Review Editors should hold a PhD with post-doctoral experience, or an equivalent degree with several additional years of academic work, or the equivalent number of years to a recognized qualification in the relevant field of research. Review Editors should have a recognized affiliation and a proven publication record in the specialty area”.

And also:

“Frontiers Associate Editors are high impact researchers and recognized leaders in their field, with a strong publication record in international, peer-reviewed journals and with a recognized affiliation. They are typically associate professor level or higher, or an equivalent position of equal standing in their field”.

Yet, for the invited topic “Beyond open access: visions for open evaluation of scientific papers by post-publication peer review” Kriegeskorte chose as his associate editor partner his own former student, Diana Deca. According to her CV, Deca (who is still listed as Guest Associate Editor at Frontiers in Computational Neuroscience) at that time freshly finished her Master studies and may or may not have already entered her PhD scholarship. Also her list of publications at the time did not show any evidence of significant own research achievements. Not really what Frontiers originally had in mind for its peer reviewers, even less so for its associate editors.

Deca was by no means training alongside her advisor, she was actually all by herself handling manuscripts as well as reviewing them. She exposed the hollowness of the Frontiers official demands for rigorous peer review as well as for editorial academic qualifications by acting as editor and reviewer of an original research paper and once, all by herself, of an opinion article. Deca also proved that Frontiers stern demands to declare and avoid any COIs are not necessarily to be followed, by handling the manuscript of her own editor partner, co-author and former lab supervisor, Kriegeskorte. The latter publication is listed as a peer reviewed “Methods Article”, yet it contains no experimental methods or descriptions of software or technology as such. This may have been another ironic trick by Kriegeskorte, since Frontiers describes the “Methods Article” as:

“a new method or emerging technique and/or software that opens new avenues for experimental investigation of important issues in a field. Method manuscripts should include the experimental objectives, the validation of the method with example(s) of its application, the limitations of current techniques, and a detailed protocol of the new technique. In addition, there should be some data to demonstrate the power of the technique”.

In this “Methods Article”, Kriegeskorte depicts the “non-transparent evaluation process that includes secret reviews visible only to editors and authors”, the conflicts of interests of reviewers who “often have some personal stake in the paper’s publication” as well as the system which “rewards authors for suggesting reviewers known to be friendly or supportive”.  Kriegeskorte then rightly criticizes the control exercised by “private publishing companies” and laments that “profit maximization can be in conflict with what is best for science”.

In another clever move, Kriegeskorte then presents the for-profit publisher Frontiers, for which he has just with this very paper exposed its insufficient oversight over editorial and peer review processes, as a “recent positive development”. He lauds the Frontiers’ “hierarchy of journals from specialized (e.g., Frontiers in Computational Neuroscience) to general (Frontiers in Neuroscience)”, right while placing a rather out-of-scope collection in the former, very narrowly specialized, journal. Kriegeskorte then states Frontiers is “visionary and represents a substantial step in the right direction”, and suggests it is comparatively better than “the PLoS journals”.

One could convincingly argue that Frontiers rules of “rigorous review” should not apply to editorial opinion articles such as this one. Indeed, a pretense of peer review here, to satisfy the criteria of a “Method Article”, should have been avoidable. It also shouldn’t have been necessary for Misha Tsodyks, the journal’s chief editor and professor at the Weizmann Institute in Israel, to act as editor and sole reviewer for a Kriegskorte/Deca editorial.

Yet this collection also contained literature reviews and even original research, while Frontiers rules of peer review have proved rather flexible also there. Inviting editors were allowed to act simultaneously as peer reviewers, as Kriegeskorte himself demonstrated for a review and for an original research article.

After all these papers received the official seal of Frontiers peer review, Kriegeskorte listed his above mentioned editorial (handled and peer reviewed by Tsodyks alone) and the “Methods Article” on his institutional website as “peer-reviewed publications”. The two thus neatly melted into his list of “proper” research publications on the topic of basic neurosciences.

The peer review process at this out-of-scope Kriegeskorte collection was apparently less than rigorous, the classification of his extended opinion manuscript as a “method paper” somewhat incorrect. However, I often heard the argument, also by the scientists directly involved, that in this special case the peer review, scope or manuscript categorization were not really relevant issues. The collection was not about presenting original research or discussing current literature. It was designed as a forum of opinions, to promote novel concept and ethics in peer review. Indeed, and as such it surely served its cause and was widely recognized by the scientific community. Yet, circumventing peer review and editorial integrity to place even the most relevant paradigm-challenging opinions could perhaps appear unfair to other computational neuroscientists. It gives the appearance that at Frontiers some scientists are more equal than others. These less equal others are namely required to submit their actual original research, and subject it to proper peer review, to obtain a publication of a “Methods Article” at this journal.  One could argue that it may have been rather unhelpful to push through a discussion series on peer review integrity, by bending and breaking otherwise recognized rules of good editorial practice.  Because, in any case: the irony of this practical joke was lost on almost everyone; while Frontiers has earned some publication fees and extra citable papers, seemingly on the topic of computational neuroscience.

More recently, chief editor Tsodyks further tested the Frontiers quality control system by allowing in his Frontiers in Computational Neuroscience a publication of a very peculiar “Original Research Article”.  It was named “Sensing risk, fearing uncertainty: systems science approach to change” and authored by a respectable oncology surgeon from Harvard University, Ivo P Janecka. This apparently retired academic now claims to head an obscure “Foundation for Systems Research and Education” in New York. The main problem with his paper: many readers, including myself, could not understand what it was supposed to be about or actually make any sense of its content or even words. Also the figures, though colourful, made little sense, which made it rather difficult to comprehend what “original research” was supposed to be hidden in the Janecka original research study. One commenter suggested a computer algorithm was used to generate a randomly worded text, which was then passed off as serious research manuscript. Another suggested a prank on Frontiers, by someone hacking their website.

The author himself was not really helpful. In his reply Janecka described his critics as driven by “Self’s Ego”, which “invokes the toxic power of emotions eliminating any rationality and simultaneously leaving basic courtesy in any dialog behind”.

Whatever this work of original research was supposed to be about, no connection could be made to the journal’s original scope, namely computational neurosciences. It was invited as part of the research topic “Application of Nonlinear Analysis to the Study of Complex Systems in Neuroscience and Behavioral Research”, edited by Janecka’s surgeon colleague, Tobias Alecio Mattei.  The latter had enjoyed only apparently a very cursory research training during his studies of medicine at the University of São Paulo. He worked since as neurosurgeon, currently at the Kenmore Mercy Hospital in Buffalo, NY, which in turn is operated by the Catholic Church and according to its website does not seem to be running any research programs. Mattei’s academic activities show no evidence of any association with basic research, computational neuroscience-related or otherwise. He is also editorial board member with the notorious predatory publisher OMICS. Why he became associate editor at Frontiers in Computational Neuroscience in the first place and why he was invited to handle and peer review publications in this journal, is anyone’s guess. Only the chief editor Tsodyks could offer any explanation here, but he did not reply to my email.

Apparently, as in the above mentioned case with Deca, sometimes a profound layman interest in a research topic is enough to dispense with the usual requirements for field-relevant research experience or a sufficient academic track record. It seems anyone with a burning personal interest and eagerness to invite paper contributions is welcome to join this Frontiers journal as associate editor and peer reviewer.

Other reviewers of the Janecka paper were Damian Kelty-Stephen, assistant professor in psychology at Grinnel College, US, and David Kronemyer, formerly film producer in Hollywood, now postdoctoral researcher of behavioral disorders at UCLA. One would love to read their peer review reports of that mystery paper, but unfortunately, the Frontiers editorial transparency ends there. This and a number of other Frontiers publications serve as solid arguments for openness of the peer review.

Kriegeskorte and his co-authors are in my opinion fully right: science does need open and transparent post-publication peer review, simply to preserve its integrity.

I tried to engage Nikolaus Kriegeskorte into a direct discussion, also through a third party. He categorically refused to communicate with me. Prior to this, an opinion exchange (incl. myself, Diana Deca and a number of anonymous commenters) on the post-publication peer review website PubPeer was deleted by the site operators. My name-signed comments were also deleted from the thread discussing the Frontiers paper by Ivo P Janecka, my personal access to the PubPeer website was blocked.