Blatt is back: “open debate cornerstone of scientific process”

Michael Blatt, Regius Professor of Botany at the University of Glasgow and Editor-in-Chief of the journal Plant Physiology, is back into the arena, fighting against the anonymity in post-publication peer review (PPPR). I have been in regular email exchange with Blatt, as indicated in my earlier blog post about the advantages of signed PPPR.

Now the British scientist has published another editorial in his journal, titled: “When Is Science ‘Ultimately Unreliable’?”, where he addresses the arguments usually brought up in support of anonymity in PPPR. In his earlier editorial in October 2015, named “Vigilante Science”, Blatt has exposed himself to plenty of criticism, including from my side, and it seems most of it happened due to a misunderstanding.

Because for Blatt (and also for myself), it is important to separate between whistleblowing of potential misconduct on the one hand and “scientific critique” on the other hand, when talking about PPPR. Blatt specifically sets aside “the issues of policing for fraud and whistleblowing “, and declares his assent “on the need to protect the whistleblower”. But also here, he distinguishes in his new editorial between identity protection and anonymous evidence:

“Anonymity is not the answer, however, not if due process is to ensure civil society and protect the innocent from denouncement or worse”.

As journalist, I agree. Though I always take care to protect my sources when they do not wish to be named, I certainly prefer knowing who these sources are, for a number of good reasons. Yet I also occasionally take hints from those whose identity or association I do not know at all, it really depends on the nature of information they share.

This time, Blatt specifically stresses the important distinction between PPPR and accusations of misconduct, which went missing in his earlier editorial, “Vigilante Science”. He now writes:

“Questioning data and ideas is the norm in science. Unless you are making an allegation of misconduct (and we’ve set this aside for the moment), it is possible, even as a Ph.D. student or postdoctoral researcher, to question a senior scientist openly, nonanonymously, in a way that is constructive and nonthreatening”.

The Plant Physiology EiC then moves on to address several main arguments used to justify anonymous PPPR:

• Fear of reprisal by established authorities
• Levelling the field and free speech
• Focus on the message
• Failure of the traditional way of discourse

For Blatt, the latter is “not an argument for anonymity per se”, instead he fears that where scientific discussion is intended, “anonymous commenting quickly escalates de facto into trial by kangaroo court”. As the botany professor sees it, every scientific discussion is beneficial for science, if it is led in the open. He is convinced that most researchers will eagerly engage in challenging discussions and appreciate constructive critiques. In fact, though Blatt may not be aware of this, there is plenty of evidence even on PubPeer of authors responding to criticisms, even to those placed anonymously and concerning data integrity. Yet, many authors do not reply at all, and some scientists do indeed engage in “abuse, sock-puppetry, and other nefarious and conflicted behaviours” (please see my example of Jacob Hanna anonymously trolling a former colleague and current competitor). Blatt acknowledges that there are scientists “whose minds are closed”, but also suggests that “such individuals are not worth the trouble in any case”. Indeed, if you sign your paper critique, certain senior figures in science are bound to feel provoked and even turn aggressive.  In my view though, this kind of behaviour would give junior researchers a useful early warning on which potential mentors and collaborators to avoid in their career. One can only hope that Blatt is right in assuming that such close-minded scientists are indeed a minority.

Only the message counts?

Blatt has plenty to say about the often invoked argument that only the message counts, and not the messenger. First of all, he disagrees that science is “democratic”:

“Science requires substantial training; its foundations are logic and reasoning; it builds on the merits of knowledge and expertise; it is not a ‘one man, one vote’ endeavour with universal enfranchisement”.

Philip Moriarty, professor of physics at the University of Nottingham, and author of the “mostly harmful” call against anonymity in PPPR, shared with me in an email these thoughts about the “focus on the message” argument:

“This reminds me of Feynman’s famous one-sentence description of science: “Science is the belief in the ignorance of experts”. We scientists all get a warm tingly feeling when we read that but it’s, of course, not really true; no-one does science “from the bottom-up” each time they do an experiment/calculation. We rely on the literature, and track record plays a major role in the assessment of a scientist’s work at levels ranging from hiring committees to the award of national prizes. […] 

Of course, if the critique of a paper was always presented in a formal, scientific manner with objective, disinterested language and the absence of any type of hidden agenda then I’d agree – only the message counts. But that’s simply not what happens in the majority of cases. A cursory glance through many PubPeer threads clearly indicates that the language is often far from objective and disinterested.

Context is everything. And, as Mike Blatt said in his original post, anonymous critique wrong-foots the scientist who is being criticised right from the start. […] If we going to criticise someone else in public, we should have the decency and intellectual courage to do it without hiding behind anonymity”.

As I see it, both Blatt and Moriarty have a certain point here. In an academic debate, all these years of acquired knowledge and research experience in a specific field do account for something. I am quite sure none of the above implies that a professor is always right when debating an undergraduate student. A polite open discussion between these two however can only benefit from being led non-anonymously: the student will strive to formulate the criticisms with more thought and consideration, while the professor will be more careful before embarrassing him- or herself in an open public forum, and hopefully avoid dismissing valid criticisms with pompous, clever-sounding, but vacuous arguments (a scenario quite common to many privately-led academic discussions).

According to Blatt, there is also another reason for scientists to know who is asking them questions:

“Knowing who enters into a debate does matter, if only because the most appropriate response almost always demands some knowledge of the context and the background of the questioner”.

Identity protection in whistleblowing

There is even a degree of agreement between Blatt and scientists who themselves actively engage in anonymous whistleblowing. Below, two US researchers express their support for the idea that PPPR as scientific discourse should be led in the open, while remaining adamant about the need for whistleblower’s anonymity where potential misconduct is suspected.

Elisabeth Bik, research associate at the Stanford School of Medicine told to me why she comments on PubPeer anonymously, when flagging image duplications and other data integrity issues: “fear of lawsuits, retaliation, or damage to the whistleblower’s career”.  She explains:

“There have already been several cases (not just involving PubPeer, but also involving Retraction Watch, http://www.science-fraud.org) in which authors of the highlighted papers have threatened with lawsuits to either find out the identity of the commenters or to remove the comments. In addition, some of the authors of the papers featured on PubPeer are leaders in their fields. They might be in positions of power such as on journal editorial boards, grant review boards, or hiring committees, and might take revenge by rejecting manuscripts, grant proposals, or job applications written by a non-anonymous whistleblower critiquing their work. […]

Finally, yes, as a woman I feel more often criticized or undervalued than men are. Women often receive very personal and harsh feedback, criticizing their emotions or appearance rather than their work accomplishments. When I post anonymously, I remove that risk. People then can only respond to what I had to say, not to who I am or how I look like”.

Yet the microbiologist Bik nevertheless signs her comments, where she addresses the scientific aspect of publications, even when very critically:

“I have not done a lot of journal-club PPPR […], but those I will usually do under my full name. […] Those posts are mainly trying to point out new papers and I might mention some weaknesses, but the tone of those reviews (I hope!) is usually overall positive. Those I have no trouble with writing about under my full name”.

The one exception is the “pitcher plant microbiome” paper, my review of a recent Scientific Reports paper in my field that I thought was not very good. See also my comments on PubPeer. […]

It took me a lot of years to gain enough confidence to write so critically about a paper, but with this particular one I felt strongly that someone needed to say something, and I felt that my background knowledge was sufficient to give feedback on this paper. I was a bit worried that I would get negative responses myself, but I have not heard anything after writing it”.

Paul S. Brookes, professor at the Department of Anesthesiology, University of Rochester, USA, is certainly no stranger to whistleblowing. His above-mentioned website science-fraud.org had to shut down in 2013 after his anonymity was blown (by an anonymous email) and he was suddenly facing legal repercussions from those whose data manipulations he previously exposed. Brookes was also (like myself) one of the fiercest critics of Blatt’s first editorial.  With whistleblowing and PPPR finally untangled in the anonymity discussion, Brookes wrote to me:

“I would say if it’s PPPR in the broader definition (i.e. journal-club like discussions, of the type PubPeer was originally set up for) then probably there’s less need for anonymity, since there’s not a lot of malice in simply asking another scientist to clarify a point, or expressing an opinion about a paper.  I’ve done this a bit on PubPeer using my real name and the response from authors has been good.  Here’s one example.

However, when specifically flagging problem data (i.e. what PubPeer has largely become) there’s an absolute need for anonymity because there’s often (not always) the inherent implication that something was done wrongly. Even if it was a simple mistake, people can sometimes react very differently when their mistakes are pointed out in public, often with a backlash against the accuser (think Jingmai O’Connor).  Thus, anonymity is the safe choice in the current climate where we all decide each others’ fates”.

Retracted invitation from RetractionWatch

With this background, the history of Blatt’s new editorial is interesting. It was in fact originally invited by RetractionWatch, after they briefly reported about his first editorial. Blatt mentions:

“I was approached by the editors of RetractionWatch, which also accepts anonymous comments, and wrote a guest post, but the RetractionWatch editors subsequently stepped back from the debate, choosing not to publish the post”.

I asked Ivan Oransky, one of the two founders of RetractionWatch, to comment. He wrote to me in an email, and confirmed the following as his for-public statement:

“While Mike was very generous with his time, and worked with us on a few revisions, we reached a point where we did not find his arguments against anonymous comments compelling enough for a guest post. We did not agree on whether the evidence he provided supported his case”.

Useful artefacts

In his now self-published editorial, Blatt then moves on to discuss how science should respond when a paper’s main findings become invalidated by PPPR. The physiology specialist takes the highly controversial example of the notorious zombie paper about Arsenic Life (Wolfe-Simon et al., Science, 2011). Many scientists call for its retraction, after the authors erroneously claimed to have discovered a bacterium able of using arsenic instead of phosphate to generate energy. Also in my view, this is probably the most reasonable option.  Yet Blatt offers some food for thought which I think was sorely neglected in the debate:

 “True, it may be argued that the article should never have passed the review process. The study was shown to be flawed technically, and the conclusion of arsenic-based life was subsequently discounted, in part because the authors did not take account of low levels of phosphate impurities.

However, there is nothing fraudulent about the data per se. Furthermore, far from ignoring the concerns of the community, Science stepped up to the debate to facilitate community awareness. Back-to-back with the article, the journal published seven critiques of the work, the authors’ rebuttal, and a note from Editor-in-Chief Bruce Alberts in June 2011. The Science article proved ‘ultimately unreliable’ only in the hyperbole of its interpretation. It stimulated research leading to important insights about a transporter that scavenges phosphate in the presence of exceptionally high levels of the structurally similar arsenate anion (Elias et al., 2012; Erb et al., 2012; Reaves et al., 2012)”.

Indeed, it would be interesting to see the peer review history of this Arsenic Life paper, or to find out why the certain basic controls were never performed (or, who knows, even deliberately dismissed) by its NASA authors. This information might by itself make a retraction necessary. Yet, also a purely artefactual paper may still have its use (provided of course, the honest mistake was indeed honest, and corrected): simply because the artefact it erroneously described was a real phenomenon, possibly worth studying in itself. This makes such papers different from the true zombie papers, which, though since long exposed as suspicious, manipulated or even fraudulent, still freely roam the scientific literature, wreck damage and infect the brains of their heedless and unaware readers.

Blatt concludes his editorial with:

“Science does not end with publication. Publication is only the beginning of scientific debate. Progress often arises from what, in hindsight, is ‘ultimately unreliable,’ and its cornerstone is open debate”.

---

 

Update 4.4.2016: there was some interesting discussion between Michael Blatt and PubPeer managers Boris Barbour and Brendon Stell, led also publicly on PubMed Commons. Some of their email exchange has also been made public, with interspersed comments by PubPeer. The issue was a disagreement about if and how PubPeer’s reply to Blatt’s editorial should be published in Plant Physiology. Displeased with what PubPeer perceived as Blatt’s unjust restrictions (text length, topics, moderation, delayed publication in the journal or rapid publication in ASPB blog instead of journal’s homepage, etc), they withdraw their offer to debate.

I personally welcome PubPeer to rapid and unmoderated debate on my site. One subject could be their own moderation policy (which occasionally tolerates comments like this), and this public accusation against myself which PubPeer so far refused to support with evidence: