The 3rd editor and failure of ‘proper channels’

A scientist finds serious measurement errors in three publications of his former collaborators. He alerts the journals and makes his concerns public, openly under his own name. The errors would make obsolete several key observations of an established German neurophysiology lab. Indeed, one journal retracts the criticised paper, another issues a correction describing the affected results as “not reliable”. The Editor-in-Chief of the third journal however accuses the whistle-blower of unspecified conflict of interests and retracts his already published letter to editor, in the process tainting his reputation with a public insinuation of research misconduct.

Here is this story in detail.

Marco Weiergräber is tenured research group leader at the German Federal Institute for Drugs and Medical devices (BfArM), which belongs to the Ministry of Health and is located in Bonn, where the German federal government used to reside before the unification of Germany. Weiergräber specialises in neuropharmacology, he obtained this specialisation by working for many years as MD/PhD student and postdoc at the Institute for Neurophysiology at the University of Cologne, which is led by Jürgen Hescheler. Disclaimer: I myself did my MSc (Diploma) thesis in 2002-2003 at the same institute, apparently when Weiergräber had been working in the neighbouring lab.

Weiergräber’s collaborator was Toni Schneider (no relation to me), professor and group leader at this Institute for Neurophysiology in Cologne. Schneider’s research focuses on the voltage gated calcium ion channels, primarily in the heart and the brain.  His lab studies the role of such ion channels in epilepsy, where they perform electrophysiological measurements on certain transgenic mouse models, using transmitters implanted into the mouse brains and an electroencephalogram (EEG) detection apparatus. While reading Schneider’s papers, Weiergräber noticed that some of these measurements were performed inappropriately: crucial EEG readings were taken far outside of the range in which the EEG device was physically able to detect. As Weiergräber told me:

“In early 2015 my group and I became aware of the systematic use of a radiofrequency transmitter out of its technical specifications in three publications, i.e. Dibue et al., 2013 (Epilepsia), Kamp et al., 2014 (PLOS One) and Dibue et al., 2014 (Epilepsy Research). The transmitter with a nominal sampling rate of 250 Hz was used in these publications for EEG analysis up to 500 Hz meaning a violation of the Nyquist-Shannon limit of 125 Hz for this type of transmitter.
Based on the systematic violation of the Nyquist-Shannon limit in the aforementioned publications we did seek for generous clarification particularly as the effects described in the publications were in a frequency range where the frequencies actually cannot be reconstructed using this type of transmitter. Thus, we sent letters to the EiCs of Epilepsia, PLOS One and Epilepsy Research”.

With these communications taking their time, Weiergräber additionally voiced his concerns on PubPeer. He chose to sign his post-publication peer review (PPPR) with his name, a brave move only few of his peers choose to make (I reported on this topic of signed PPPR previously, see also debate by Mike Blatt, EiC of Plant Physiology). Eventually, Weiergräber also published a general description of these technical caveats of EEG in a paper in Journal of Neuroscience Methods (Weiergräber e al, 2016), without directly discussing the relevant research of Schneider’s lab. Its summarised ‘highlights’ are:

• “The nominal sampling rate is a central technical specification in EEG systems.
• The nominal sampling rate determines the Nyquist limit.
• Frequency reconstruction above the Nyquist limit causes invalid results.
• Approaching the Nyquist limit from below results in data of decreasing validity”.

This is how journals eventually reacted.

Journal 1, Epilepsia: retraction

The editors of the journal Epilepsia invited Schneider and the first author Dibue to submit a reply to Weiergräber’s criticisms, which he again countered in another reply. Both letters are however paywalled, the comment by Weiergräber and his postdoctoral scientist Anna Papazoglou is available on ResearchGate. Schneider and Dibue admitted the technical implausibility of their measurements, but used the strange argument that others do the same (which in fact, is the truth, they even quoted literature):

“Dr. Weiergräber correctly states that the actual bandwidth of the transmitter is 1–50 Hz and that accuracy in this range is greatest. Indeed, beyond 50 Hz, readable amplitudes decrease resulting in a decreased signal-to-noise ratio. However, because high-frequency oscillations are often read as low-amplitude currents, despite lower accuracy, some investigators including ourselves choose to include the nominal bandwidth (fivefold or greater) when using these transmitters to gain insight into higher frequency bands and to sample at a higher rate to avoid aliasing”.

Therefore, an experimentally meaningless read-out, obtained from incorrect use of the technology with disregard to basic physics, become meaningful and correct through the simple fact that other researchers succeeded publishing this before. Apparently, it is not the scientific publication which describes the natural world, but the natural world which has to submit itself to what scientists publish about it.

However, the Epilepsia editors disagreed with this view and found the issues serious enough to announce on November 3^rd 2015 their intent to retract the Dibue et al, 2013, paper. The somewhat hidden retraction note appeared some months later, on March 21^st 2016, stating:

“The retraction is due to the unintentional inclusion of erroneous data due to the limitations of the recording system used. The system demonstrated greater than a 99% loss of amplitude of the signal, and a non-linear response of the recording system to changes in amplitude of an input signal. Because the signal reported in the published manuscript is within the range of noise in the system, and validity of the signal cannot be assured, the editors have concluded that the paper should be retracted”.

Weiergräber commented to me in this regard that it was not at all a technical problem, caused by the device, but a fault of its human operators:

“The Dibue et al. (2013) retraction notice erroneously stated that the limitations of the system are responsible for the retraction. It is essential to know for all telemetry users that it is not a system related failure as explained above”.

Update 01.06.2016. I asked Astrid Nehlig, one of the Editors-in-Chief  of Epilepsia, why the retraction notice is not linked to the online paper, why it lacks information about which data exactly was affected and why no evidence of the retraction is available on PubMed, after more than three months. Nehlig told me:

“apparently there are a few legal issues that still need to be solved but no worries this article is withdrawn and ASAP it will appear clearly on the different links. This is the wish of everybody”.

Journal 2, PLOS One: correction

This criticised publication Kamp et al, 2014, is a collaborative effort of Schneider’s lab and that of Daniel Hänggi, neurosurgery professor at the University Clinic Düsseldorf. PLOS One issued the following correction on the same day the Dibue et al paper in Epilepsia was retracted:

“The article reports the recording of telemetric data from implanted TL11M2-F20-EET transmitters. These results are described in Figures 4 and 5.

The TL11M2-F20-EET transmitter has a reported transmission bandwidth of 1-50Hz and a sampling frequency of 250 Hz. The maximum identifiable signal for the device is 125 Hz.

The article reports experiments measuring signals of up to 500Hz. Given the limitations of the transmitter employed, the results reported for frequencies beyond 100Hz are not reliable. We are thus issuing this notification to alert readers that the results in Figure 4 and Figure 5C reporting signals over 100Hz are not reliable”

 

Here, Weiergräber acknowledges that this PLOS ONE correction notice “did not blame the system but correctly stated that the Nyquist-Shannon limit had been violated in Kamp et al., 2014 resulting in invalid data”.

Journal 3, Epilepsy Research: wrath of the Editor-in-Chief

What happened with the third journal can only be described as an unworthy farce, and under certain viewpoint, as an example of blatant editorial misconduct.

Weiergräber sent in February 2015 a letter of concern to this journal, describing the problem they found in the Dibue et al., 2014 publication. According to Weiergräber’s chronology listed on ResearchGate, his letter was assigned on February 14^th  to the EiC, David Treiman, who is director of the epilepsy research at Barrow Neurological Institute, as well as a neurology professor at three universities, namely Arizona State University, University of Arizona College of Medicine and Creighton University School of Medicine. On April 10^th, Treiman wrote to Weiergräber announcing the acceptation of his letter:

“I am pleased to confirm that your letter “EEG Radiotelemetry in Mice – Transmitter Bandwidth, Sampling Rate and Experimental Pifalls” has been accepted for publication in Epilepsy Research. I apologize for the delay in notifying you.  We had been holding off on a response to you until we received confirmation from Dibué and colleagues that they would like to respond to your letter.  They are now preparing a response and we anticipate publishing your letter and their response together.

Thank you for submitting your letter. The type of dialogue letters such as yours should generate is good for the whole epilepsy research community”.

On April 21^st, Weiergräber’s letter went online, where it survived for only 1 day. It was “withdrawn”, without any official explanation on April 22^nd. Instead, there is a reference to ‘Elsevier Policy on Article Withdrawal’, which explains that this withdrawal is

“Only used for Articles in Press which represent early versions of articles and sometimes contain errors, or may have been accidentally submitted twice. Occasionally, but less frequently, the articles may represent infringements of professional ethical codes, such as multiple submission, bogus claims of authorship, plagiarism, fraudulent use of data or the like”.

Of all the listed justifications none fits Weiergräber’s letter of concern. Unless that is, Treiman meant to publicly denounce him on some kind of “infringements of professional ethical codes”. Which Treiman in fact did, in his email to Weiergräber from April 22^nd, accusing him of unspecified conflicts of interest:

“I am sorry to inform you that I have to rescind my acceptance of your recent letter to the editor.  It has come to my attention that you have a conflict of interest with regard to the Dibué et al. paper.  Because of this conflict of interest it is not appropriate for us to publish your critique of the Dibué paper in Epilepsy Research.  It is essential that scientific criticism be conducted with objectivity and be completely free from personal conflicts or other issues.

Thank you in advance for understanding the journal’s position in this matter”.

Weiergräber had nothing at all to gain from his scientific criticism of Schneider’s measurements, except a number of powerful enemies. He therefore wrote back to Treiman asking what exactly this alleged conflict of interest was supposed to be. To which Treiman, who used to serve in the United States Navy, replied rather harshly:

“With regard to the Dibué et al. paper I think at this stage it would be best to let the matter rest, without further comment from you, your post-doctoral fellow [Papazoglou, -LS] , or other colleagues”.

Tretman’s follow-up and final email to Weiergräber was even less polite. Meanwhile, two journals acknowledged that the concerns Weiergräber voiced about the implausibility of EEG measurements were correct and made the published results of Schneider lab invalid. Treiman however remained true to his word. Regardless of Epilepsia retraction and PLOS One correction, there was no further action from the side of Epilepsy Research, the Dibue et al 2014 paper remains pristine.

Weiergräber told me he and his colleagues now regret “that our justified concerns had been withdrawn by Epilepsy Research while confirmed errors remain unaltered“.

The letter to Treiman and his editor colleagues has been purged from the journal’s website, only its scant note of withdrawal remained. Allegedly, it was already interpreted by at least one science journalist as a retraction of a research paper to Weiergräber’s name. Maybe such public misconduct insinuation was to a degree the purpose of this questionable editorial action by Treiman.

To help averting this, the original letter is now available from my website.

I wrote to Treiman, together with his associate editors at Epilepsy Research. One of them was Merab Kokaia, professor for experimental epilepsy at the Lund University in Sweden, who was originally also involved into the editorial handling of Weiergräber’s letter. My emails were only met with silence.

Here, writing to editor and other traditional ‘proper channels’ in academia have failed disastrously. If nothing else can be achieved in this case, it is a good example of how badly transparency and open PPPR are needed in science.