A scientist finds serious measurement errors in three publications of his former collaborators. He alerts the journals and makes his concerns public, openly under his own name. The errors would make obsolete several key observations of an established German neurophysiology lab. Indeed, one journal retracts the criticised paper, another issues a correction describing the affected results as “not reliable”. The Editor-in-Chief of the third journal however accuses the whistle-blower of unspecified conflict of interests and retracts his already published letter to editor, in the process tainting his reputation with a public insinuation of research misconduct.
Here is this story in detail.
Marco Weiergräber is tenured research group leader at the German Federal Institute for Drugs and Medical devices (BfArM), which belongs to the Ministry of Health and is located in Bonn, where the German federal government used to reside before the unification of Germany. Weiergräber specialises in neuropharmacology, he obtained this specialisation by working for many years as MD/PhD student and postdoc at the Institute for Neurophysiology at the University of Cologne, which is led by Jürgen Hescheler. Disclaimer: I myself did my MSc (Diploma) thesis in 2002-2003 at the same institute, apparently when Weiergräber had been working in the neighbouring lab.
Weiergräber’s collaborator was Toni Schneider (no relation to me), professor and group leader at this Institute for Neurophysiology in Cologne. Schneider’s research focuses on the voltage gated calcium ion channels, primarily in the heart and the brain. His lab studies the role of such ion channels in epilepsy, where they perform electrophysiological measurements on certain transgenic mouse models, using transmitters implanted into the mouse brains and an electroencephalogram (EEG) detection apparatus. While reading Schneider’s papers, Weiergräber noticed that some of these measurements were performed inappropriately: crucial EEG readings were taken far outside of the range in which the EEG device was physically able to detect. As Weiergräber told me:
“In early 2015 my group and I became aware of the systematic use of a radiofrequency transmitter out of its technical specifications in three publications, i.e. Dibue et al., 2013 (Epilepsia), Kamp et al., 2014 (PLOS One) and Dibue et al., 2014 (Epilepsy Research). The transmitter with a nominal sampling rate of 250 Hz was used in these publications for EEG analysis up to 500 Hz meaning a violation of the Nyquist-Shannon limit of 125 Hz for this type of transmitter.
Based on the systematic violation of the Nyquist-Shannon limit in the aforementioned publications we did seek for generous clarification particularly as the effects described in the publications were in a frequency range where the frequencies actually cannot be reconstructed using this type of transmitter. Thus, we sent letters to the EiCs of Epilepsia, PLOS One and Epilepsy Research”.
With these communications taking their time, Weiergräber additionally voiced his concerns on PubPeer. He chose to sign his post-publication peer review (PPPR) with his name, a brave move only few of his peers choose to make (I reported on this topic of signed PPPR previously, see also debate by Mike Blatt, EiC of Plant Physiology). Eventually, Weiergräber also published a general description of these technical caveats of EEG in a paper in Journal of Neuroscience Methods (Weiergräber e al, 2016), without directly discussing the relevant research of Schneider’s lab. Its summarised ‘highlights’ are:
- “The nominal sampling rate is a central technical specification in EEG systems.
- The nominal sampling rate determines the Nyquist limit.
- Frequency reconstruction above the Nyquist limit causes invalid results.
- Approaching the Nyquist limit from below results in data of decreasing validity”.
This is how journals eventually reacted.
Journal 1, Epilepsia: retraction
The editors of the journal Epilepsia invited Schneider and the first author Dibue to submit a reply to Weiergräber’s criticisms, which he again countered in another reply. Both letters are however paywalled, the comment by Weiergräber and his postdoctoral scientist Anna Papazoglou is available on ResearchGate. Schneider and Dibue admitted the technical implausibility of their measurements, but used the strange argument that others do the same (which in fact, is the truth, they even quoted literature):
“Dr. Weiergräber correctly states that the actual bandwidth of the transmitter is 1–50 Hz and that accuracy in this range is greatest. Indeed, beyond 50 Hz, readable amplitudes decrease resulting in a decreased signal-to-noise ratio. However, because high-frequency oscillations are often read as low-amplitude currents, despite lower accuracy, some investigators including ourselves choose to include the nominal bandwidth (fivefold or greater) when using these transmitters to gain insight into higher frequency bands and to sample at a higher rate to avoid aliasing”.
Therefore, an experimentally meaningless read-out, obtained from incorrect use of the technology with disregard to basic physics, become meaningful and correct through the simple fact that other researchers succeeded publishing this before. Apparently, it is not the scientific publication which describes the natural world, but the natural world which has to submit itself to what scientists publish about it.
However, the Epilepsia editors disagreed with this view and found the issues serious enough to announce on November 3rd 2015 their intent to retract the Dibue et al, 2013, paper. The somewhat hidden retraction note appeared some months later, on March 21st 2016, stating:
“The retraction is due to the unintentional inclusion of erroneous data due to the limitations of the recording system used. The system demonstrated greater than a 99% loss of amplitude of the signal, and a non-linear response of the recording system to changes in amplitude of an input signal. Because the signal reported in the published manuscript is within the range of noise in the system, and validity of the signal cannot be assured, the editors have concluded that the paper should be retracted”.
Weiergräber commented to me in this regard that it was not at all a technical problem, caused by the device, but a fault of its human operators:
“The Dibue et al. (2013) retraction notice erroneously stated that the limitations of the system are responsible for the retraction. It is essential to know for all telemetry users that it is not a system related failure as explained above”.
Update 01.06.2016. I asked Astrid Nehlig, one of the Editors-in-Chief of Epilepsia, why the retraction notice is not linked to the online paper, why it lacks information about which data exactly was affected and why no evidence of the retraction is available on PubMed, after more than three months. Nehlig told me:
“apparently there are a few legal issues that still need to be solved but no worries this article is withdrawn and ASAP it will appear clearly on the different links. This is the wish of everybody”.
Journal 2, PLOS One: correction
This criticised publication Kamp et al, 2014, is a collaborative effort of Schneider’s lab and that of Daniel Hänggi, neurosurgery professor at the University Clinic Düsseldorf. PLOS One issued the following correction on the same day the Dibue et al paper in Epilepsia was retracted:
“The article reports the recording of telemetric data from implanted TL11M2-F20-EET transmitters. These results are described in Figures 4 and 5.
The TL11M2-F20-EET transmitter has a reported transmission bandwidth of 1-50Hz and a sampling frequency of 250 Hz. The maximum identifiable signal for the device is 125 Hz.
The article reports experiments measuring signals of up to 500Hz. Given the limitations of the transmitter employed, the results reported for frequencies beyond 100Hz are not reliable. We are thus issuing this notification to alert readers that the results in Figure 4 and Figure 5C reporting signals over 100Hz are not reliable”
Here, Weiergräber acknowledges that this PLOS ONE correction notice “did not blame the system but correctly stated that the Nyquist-Shannon limit had been violated in Kamp et al., 2014 resulting in invalid data”.
Journal 3, Epilepsy Research: wrath of the Editor-in-Chief
What happened with the third journal can only be described as an unworthy farce, and under certain viewpoint, as an example of blatant editorial misconduct.
Weiergräber sent in February 2015 a letter of concern to this journal, describing the problem they found in the Dibue et al., 2014 publication. According to Weiergräber’s chronology listed on ResearchGate, his letter was assigned on February 14th to the EiC, David Treiman, who is director of the epilepsy research at Barrow Neurological Institute, as well as a neurology professor at three universities, namely Arizona State University, University of Arizona College of Medicine and Creighton University School of Medicine. On April 10th, Treiman wrote to Weiergräber announcing the acceptation of his letter:
“I am pleased to confirm that your letter “EEG Radiotelemetry in Mice – Transmitter Bandwidth, Sampling Rate and Experimental Pifalls” has been accepted for publication in Epilepsy Research. I apologize for the delay in notifying you. We had been holding off on a response to you until we received confirmation from Dibué and colleagues that they would like to respond to your letter. They are now preparing a response and we anticipate publishing your letter and their response together.
Thank you for submitting your letter. The type of dialogue letters such as yours should generate is good for the whole epilepsy research community”.
On April 21st, Weiergräber’s letter went online, where it survived for only 1 day. It was “withdrawn”, without any official explanation on April 22nd. Instead, there is a reference to ‘Elsevier Policy on Article Withdrawal’, which explains that this withdrawal is
“Only used for Articles in Press which represent early versions of articles and sometimes contain errors, or may have been accidentally submitted twice. Occasionally, but less frequently, the articles may represent infringements of professional ethical codes, such as multiple submission, bogus claims of authorship, plagiarism, fraudulent use of data or the like”.
Of all the listed justifications none fits Weiergräber’s letter of concern. Unless that is, Treiman meant to publicly denounce him on some kind of “infringements of professional ethical codes”. Which Treiman in fact did, in his email to Weiergräber from April 22nd, accusing him of unspecified conflicts of interest:
“I am sorry to inform you that I have to rescind my acceptance of your recent letter to the editor. It has come to my attention that you have a conflict of interest with regard to the Dibué et al. paper. Because of this conflict of interest it is not appropriate for us to publish your critique of the Dibué paper in Epilepsy Research. It is essential that scientific criticism be conducted with objectivity and be completely free from personal conflicts or other issues.
Thank you in advance for understanding the journal’s position in this matter”.
Weiergräber had nothing at all to gain from his scientific criticism of Schneider’s measurements, except a number of powerful enemies. He therefore wrote back to Treiman asking what exactly this alleged conflict of interest was supposed to be. To which Treiman, who used to serve in the United States Navy, replied rather harshly:
“With regard to the Dibué et al. paper I think at this stage it would be best to let the matter rest, without further comment from you, your post-doctoral fellow [Papazoglou, -LS] , or other colleagues”.
Tretman’s follow-up and final email to Weiergräber was even less polite. Meanwhile, two journals acknowledged that the concerns Weiergräber voiced about the implausibility of EEG measurements were correct and made the published results of Schneider lab invalid. Treiman however remained true to his word. Regardless of Epilepsia retraction and PLOS One correction, there was no further action from the side of Epilepsy Research, the Dibue et al 2014 paper remains pristine.
Weiergräber told me he and his colleagues now regret “that our justified concerns had been withdrawn by Epilepsy Research while confirmed errors remain unaltered“.
The letter to Treiman and his editor colleagues has been purged from the journal’s website, only its scant note of withdrawal remained. Allegedly, it was already interpreted by at least one science journalist as a retraction of a research paper to Weiergräber’s name. Maybe such public misconduct insinuation was to a degree the purpose of this questionable editorial action by Treiman.
To help averting this, the original letter is now available from my website.
I wrote to Treiman, together with his associate editors at Epilepsy Research. One of them was Merab Kokaia, professor for experimental epilepsy at the Lund University in Sweden, who was originally also involved into the editorial handling of Weiergräber’s letter. My emails were only met with silence.
Here, writing to editor and other traditional ‘proper channels’ in academia have failed disastrously. If nothing else can be achieved in this case, it is a good example of how badly transparency and open PPPR are needed in science.
Here it is a good example why PPPR must be anonymous
Epilepsy Research says “It is essential that scientific criticism be conducted with objectivity and be completely free from personal conflicts or other issues.”
On the contrary, if scientific criticism is valid, then the motivations etc. of the critics are irrelevant. And in this case, the criticism (you can’t beat the Nyquist frequency!) is clearly valid, and no-one has even tried to rebut it.
A good way of keeping objectivity is doing everything anonymously
Most likely Schneider sent some letter to editors accusing his opponent of unknown conflicts. If that is the case, he likely sent similar letters to other two journals. In principle, that is most common method of defence : to substitute discussion of subject of problemes with subject of relations. Instead of discussing what is the problem they start to discuss why it was revealed. Much too often this line of defence works well.
I assume you meant Toni Schneider, not me…. 😉
LikeLiked by 1 person
Another good reason for anonymous PPPR….
Weiergräber’s outcome wasn’t so bad! 2/3. In my case, there were seemingly endless problems over several years with many Taylor & Francis journals, and their editors. Ultimately, T&F management got fed-up and banned me from all journals, squashing all editorial independence:
Nothing feels more frustrating that dealing with editors and publishers who fail to deal with the substantive issues. Requests for T&F and COPE to provide the web-sites where rules pertaining to claims and reports, and what is excessive and what is not, have never been provided.
It’s a shameful situation when scientists get banned for being critical. In fact, it seems to go against COPE’s Virginia Barbour’s principles of scholarly communication:
Philip E. Bourne, Virginia Barbour (2011) Ten Simple Rules for Building and Maintaining a Scientific Reputation. PLoS Comput Biol 7(6): e1002108. doi:10.1371/journal.pcbi.1002108
“Rule 2: Do Not Ignore Criticism
Whether in your eyes, criticism is deserved or not, do not ignore it, but respond with the knowledge of Rule 1. Failure to respond to criticism is perceived either as an acknowledgement of that criticism or as a lack of respect for the critic. Neither is good.”
Sadly, written promises made by Dr. Barbour to hold her profitable COPE member accountable have never come to fruition. The crisis in trust in science begins with such “ethics” organizations and two-faced publishers like Taylor & Francis who are quick to grab copyrights, but then equally quick to shove critics down the drain.
Shouldn’t Toni Schneider himself retract that paper in Epilepsy Research??? The results are kind of invalidated by the other two publications retracted… What’s the point of keeping it there?
Also pretty interesting how people that are exposed in this pages never raise their voice to provide explanations, to defend themselves… How should we interpret their silence?
actually, Toni Schneider approached me by email on June 14th, but declined engaging in public discussion, in order to avoid “leaving the level of scientific rationality” and “presenting ad hominem accusations”. He spoke again of „conflicts of interest“, which Weiergräber allegedly failed to declare, without specifying what these were.
The issue is that there is nothing to discuss that could result in “leaving the level of scientific rationality”.
In three publications mentioned above the Nyquist-Shannon limit had been violated resulting in invalid data- full stop.
Any questioning of this basic physical law means that any level of scientific rationality had already been left long time ago.
At the end of the day, the authors of the three publications and the EiC of Epilepsy Research are fully responsible for their scientific behavior.
I vigorously contradict any ongoing insinuation from T. Schneider of a conflict of interest from my site to distract from their mistakes. I had no personal contact with T. Schneider or his group since 6 years, no cooperations with his group and my group is working in a totally different field (Alzheimer’s disease).
Even if you did have a conflict of interest, this would be irrelevant, the facts are the facts!
Elsevier has now published an updated version of my (originally withdrawn) letter to the Editor in Epilepsy Research (Epilepsy Res. 2016 Dec 8. pii: S0920-1211(16)30377-1. doi: 0.1016/j.eplepsyres.2016.12.003. [Epub ahead of print]) in which I pointed to the violation of the Nyquist-Shannon limit in Dibue et al. (2014).
The withdrawal notice of my original letter from April 2015 is said to be removed by Elsevier.
There is also again a response from the senior author of Dibue et al. (2014) (DOI: http://dx.doi.org/10.1016/j.eplepsyres.2016.12.004).
As the violation of the Nyquist-Shannon limit and the use of the transmitters out of their technical specifications in Dibue et al., (2013), Kamp et al., (2014) and Dibue et al., (2014) have been confirmed independently several times now (i.a. Epilepsia, Plos One), it is incomprehensible why the response letter once again refers to EEG power values for f > 125 Hz for the F20EET transmitter as all related power values used for argumentation or in the figures of the response letter are invalid. It is senseless to further comment on what is mentioned in the response letter.
Pingback: Risultati segreti = risultati negativi - Ocasapiens - Blog - Repubblica.it
I actually believe that very few people in the EEG field have even heard of aliasing. I once reviewed a paper in which the data were essentially meaningless because the Nyquist frequency had been violated. I recommended rejection, taking the time to explain the problem in detail in hopes of educating both author and editor. The result was that the author simply changed the number in the Methods section of the paper to the minimum I’d said in my referee report would be a valid sampling rate — no apology, comment, or attempt to explain away the original number as a typo — and the paper was published. I’m sorry to report that I didn’t bother to complain, for the same reason that I’m not signing this comment. I now take essentially all of the EEG literature with a grain of salt.
I just came along this: the retracted paper in Epilepsia was published with the same data in another Journal (see below).
So NeuroDoubleSceptic might be right…the Nyquist issue seems to be too sophisticated for most people acting in the field…
Dibué-Adjei M, Kamp MA, Alpdogan S, Tevoufouet EE, Neiss WF, Hescheler J, Schneider T. Cav2.3 (R-Type) Calcium Channels are Critical for Mediating Anticonvulsive and Neuroprotective Properties of Lamotrigine In Vivo. Cell Physiol Biochem. 2017 Nov 24;44(3):935-947. doi: 10.1159/000485361.
Dibué M, Kamp MA, Alpdogan S, Tevoufouet EE, Neiss WF, Hescheler J, Schneider T. Cav 2.3 (R-type) calcium channels are critical for mediating anticonvulsive and neuroprotective properties of lamotrigine in vivo. Epilepsia. 2013 Sep;54(9):1542-50. doi: 10.1111/epi.12250. Epub 2013 Jun 17. Erratum in: Epilepsia. 2016 Nov;57(11):1947. Retraction in: Epilepsia. 2016 Nov;57(11):1947.