Lawyering-up Research integrity

Spiderman’s lawyer is having you for dinner tonight

Spider researcher Jonathan Pruitt is accused by his coauthors of data manipulations, after 3 retractions they demand more. A lawyer's letter was supposed to stop that, but Pruitt tells me: "I'm happy for folks to engage in public discourse about my data integrity""

The Canadian shooting star of behavioural ecology Jonathan Pruitt used to be everyone’s best friend. Up until his co-authors accused the spider researcher of data manipulation and achieved retractions for 3 of his papers. His reputation in tatters, his research record facing even more retractions, his elite funding as one of Canada’s “150 Research Chairs” and prestigious job at McMaster University in danger, Pruitt did what every red-blooded academic in his position does: he deployed lawyers against his critics.

As Science reported, the lawyer’s letter was so scary that some journal editors panicked and hid under their tables, waving a white flag of surrender. No more Pruitt retractions unless specifically requested by the ongoing investigation at McMaster, no matter what the evidence submitted by co-authors proves.

The original recipients however, all of them former Pruitt co-authors, are not complying, they continue vetting Pruitt’s publications for further irregularities. As I learned, nobody signed anything or announced to adhere to any of lawyer’s demands. And why should they be afraid, the lawyer’s letter is not even really threatening anyone. It is a cack-handed attempt in intimidation which even Pruitt now apparently distances himself from.

All the letter from Millard and Company does, is to educate its recipient that nobody is supposed to be scrutinising Pruitt’s papers but his employer McMaster University, and that every retraction must be agreed by the Committee on Publication Ethics (COPE, a scholarly publisher lobbyist), whose advisory guidelines to its member journals are presented by Pruitt’s lawyers as a kind of a law for breaking of which you might end up in prison.

The letter is labelled “PRIVATE & CONFIDENTIAL WITHOUT PREJUDICE”, which is exactly why I publish it here in full, below. I hope the Millard lawyers are happy with this arrangement.

The Pruitt case made international news, it even got its own hashtag n Twitter: #PruittData. The communal effort to uncover data irregularities began with Kate Laskowski, now assistant professor at UC Davis in USA. Laskowski told the story on her blog, and it might explain the success recipe of Pruitt: when he met some scientist with an unproven pet theory, he provided them with the perfect experimental data. Which now proved to have been too good to be true.

In science, many beautifully designed theories collapse on experimental reality. Biology is very messy, complicated and rarely does as told, even and especially when the theory which biology is expected to adhere to is clever, beautiful and elegant, of the kind the editors in elite journals love and the research funders cherish. Which is exactly why so many scientists resort to research fraud, to force scientific reality to adhere to their theories.

Laskowski, then a PhD student, had a theory about animal behaviour in social context. She met at a conference the young shooting star Pruitt, who works with social spiders. Pruitt generously offered his collaboration, and soon perfect data arrived which perfectly proved Laskowski’s theory right. Pruitt’s datasets served as template for 3 papers in respectable journals, until someone contacted Laskowski with concerns. That colleague was Niels Dingemanse, professor at the LMU Munich in Germany, and he noticed there were statistically too many duplicate numbers in the paper’s publicly available original data file.

Pruitt told Laskowski he measured 4 spiders simultaneously, hence identical values. But how can spider act so synchronously and into 1/100 of a second?

Thus alerted, Laskowski started to go through Pruitt’s Excel files, and indeed found many repetitive numbers. Pruitt was quick with an explanation, which sounded a bit weird, but it kind of sufficed to consider a correction. But then Laskowski analysed the files in more details and found entire strings of numbers repeating themselves, with the only logical explanation that they were made up. Once Laskowski removed all repetitive numbers, the remaining ones showed no significant difference in spider behaviour between experimental treatments. Her theory collapsed, so Laskowski had her own three papers retracted, one by one.

Laskowski: “Each line is an individual spider. What I noticed is that the last two pre-treatment values, seemed to be identical to the first two post-treatment values in many spiders in this treatment. I colored those values red here. I also noticed that many values were very similar, but off by one digit in the hundreds column, e.g. 77.65 became 477.56. Those values I colored in yellow.”

In another blogpost on the topic, Dan Bolnick, editor of American Naturalist and professor at University of Connecticut, explains that the journals actually very much can be expected to take responsibility and to perform data integrity investigations, which his journal did. Bolnick even used to curate a publicly viewable (but not editable) Google Docs file to track the reliability of Pruitt co-authored papers. Some papers actually proved to contain no evidence of irregularities. The list was removed now, after Pruitt lawyered-up. Bolnick’s commendable behaviour stands in stark contrast to certain other editors who now hide behind Pruitt’s lawyer’s letter and the McMaster investigation.

Laskowski: “as opposed to being formatted in columns as on Sheet 1, these values were transposed – so that the columns became rows. […] Here on Sheet 2, the duplicate sequences were not just sequences of 4 numbers (as they were on Sheet 1), but rather I could now see that in fact there were whole blocks of duplicated sequences.”

Unlike his lawyer claims, Pruitt even agreed to the first retraction, which happened in American Naturalist. Laskowski quotes him with: “it is well that the paper is being retracted”. But eventually Pruitt changed his mind. Things got out of his control namely. After that retraction, other Pruitt coauthors reached out to Laskowski, the search for data manipulations became a communal effort. Many went on to publish their concerns on PubPeer, where other Pruitt’s coauthors first defended Pruitt, until some switched sides and joined the investigative group effort, like the associate professor Noa Pinter-Wollman at UC Los Angeles. Did these scientists also discover their own beautiful theories, once verified by Pruitt’s data, now in lying in tatters? Possibly.

One former Pruitt collaborator explained why his co-authors initially defended the accused:

It’s worth remember that lots of us considered ourselves to be good to very close friends with him. You just don’t want to believe something like this.

And then they looked into their own papers with Pruitt. A co-author shared this about the raw data behind a publication:

literally 1/2 the data were sequence repeats 10-14 units long but sometimes with select manipulation

That source also described the communal #PruittData investigation:

There are 28 channels, on the slack channel where people are looking at this, each of which is dedicated to a paper. […] The sequence repeat pops up a ton. Formulas are pretty common. One person found that a formula was used but it was like + 20.3 or something to better hide the pattern.

Rows K-O in the Pruitt raw data file “Black_Widow_Data” list independent experimental values for “Darkness”. And yet, values in row O are simply values in M +1.5. The author sloppily even left inside the formula with which the values were fabricated.

Pruitt’s manipulative approach over his former friends does not work anymore. If anything, people are afraid of his revenge now, rightly or not, time will show. As a source wrote to me:

Jonathan has a typical response when confronted.  He restates the issue, gives excuses that don’t directly address the question, often will say we should remove the questionable data and reanalyze, and then finally will agree to retract if the results don’t hold. This was the pattern until lawyers got involved that is.  

[…] When I was early in my career I was also just enamored with him and trusted him.  So when he comes dangling a dataset that would turn into a high level paper, you gladly take it and write it up.  Later it seemed like what would happen is that he’s send off just chunks of datasets to different people so they don’t see the whole thing.  Or he would talk to say a postdoc about an idea, the six months later the data would appear in that postdocs inbox….  so they don’t ever see the stuff being collected, but are thrilled to have the data and to get a famous person to work with.” 

Pruitt raw data file “Spider_StateBasedPersonality”, sheet “Anelosimus.2” Colour coded are repetitive number sequences

Much of #PruittData problems came out because Pruitt used to share his research data openly. Until he got caught that is:

Dryad is the data repository for open source stuff. Many journals have started requiring us to upload our data. Jonathan would do that sometimes, but not others. Or he would upload only a portion of the data. This started when one of his own grad students acted as a whistleblower and told another lab about some issues. That lab looked into a bunch of his data on Dryad and found all these issues.

But now, the lawyer’s letter. Which even Pruitt himself admits is not suitable to scare anyone. He namely wrote to me via Twitter direct message, obviously with reference to the Science article:

I realize the report made it sound like I legally carpet bombed people; that’s not the case. I know folks have interpreted the action as a flimsy fear-induced gag order. That’s not the case either. I’m happy for folks to engage in public discourse about my data integrity


PRIVATE & CONFIDENTIAL
WITHOUT PREJUDICE

[….]

Please be advised that our firm has been retained by Jonathan Pruitt with respect to the investigation (if any) being conducted by […] into the paper “[…]” by […] in 20[…]. I understand you have been contacted by Dr […] one of the coauthors, seeking a retraction.

Over the last number of weeks, there has been an online campaign directed at Dr. Pruitt and his work which has come to be known as “#pruittdata” and “#pruittgate”. As part of #pruittgate, a number of academics have organized an extra-institutional online forum for posting, reviewing, and criticizing the data integrity of various of Dr. Pruitt’s research studies and papers. It appears that Dr. […]’s concerns arise from that project.

Dr. Pruitt disagrees that there are any grounds for this paper’s retraction. Any decision about retraction must be conducted fairly and in keeping with professional norms as articulated by the Committee on Publication Ethics (“COPE”) Guidelines on Retractions. It does not appear that this paper meets any of the criteria for retraction as set out in the COPE Guidelines. In particular, the COPE Guidelines do not support retraction if “the main findings of the work are still reliable” or if “an editor has inconclusive evidence to support retraction, or is awaiting additional information such as from an institutional investigation.”

In addition, I am concerned that the procedural fairness of any investigation has been hampered by the manner in which Dr. Pruitt’s work is currently being criticized online. The social media campaign and online forums which spurred this complaint break with the normative practices of the field and very likely prevent a fair assessment of Dr. Pruitt’s work.

The controversy over this particular paper highlights the problems with using an unmediated online forum to attack the data integrity of a body of work. In this case, it appears that the wrong electronic file was uploaded to the forum. The final published paper does not rely on the impugned data.

If […] there is sufficient grounds to retract this paper, it must first notify Dr. Pruitt and provide him with a chance to respond. A premature retraction without Dr. Pruitt’s agreement is likely to further harm his reputation and negatively impact his career in an irreversible manner.

Please confirm that […] does not intend to take further steps at this time
with respect to the above noted paper. Otherwise, if the journal intends to take any further steps, please provide me with correspondence (1) detailing the journal’s concerns, including what evidence it has showing that the paper’s findings are no longer reliable, and (2) outlining the process the journal intends to follow prior to retraction, including when and how Dr. Pruitt will be permitted to respond to any allegations against him.
I am happy to discuss this matter further if you have any questions or concerns. If you would like to speak to me, please arrange a time through my assistant Lori Leblond (lori@millardco.ca, 1-416-920-3123).
Yours truly,

Marcus McCann

Update 19.05.2020. Seems Pruitt’s lawyer was successful. The Pruitt et al Proc Roy Soc B: Biol Sci 2016 paper, previously retracted on 26 February 2020, was stealthily de-retracted by the publisher The Royal Society in March. Instead, an Expression of Concern was published on 29. April 2020, announcing “An investigation into these aspects is under way“. This is the deleted retraction:


Donate!

If you are interested to support my work, you can leave here a small tip of $5. Or several of small tips, just increase the amount as you like (2x=€10; 5x=€25). Your generous patronage of my journalism will be most appreciated!

€5.00

30 comments on “Spiderman’s lawyer is having you for dinner tonight

  1. Klaas van Dijk

    Another update about Jonathan Pruitt at https://twitter.com/GrinstedLena/status/1357791220425195520 (posted on 5 February 2021): “Never thought I’d be so happy to get a paper retracted! But it’s been a horrendous fight for nearly a year, and finally I got the (only acceptable) final decision from Proc B today to retract our paper that linked social spider personality with task differentiation. #pruittdata”

    See https://pubpeer.com/publications/11ED07D6DCDE43FFC88DFEA189874B

    Also an update in a short thread at https://twitter.com/RRoyaute/status/1357730477638094849 (also posted on 5 February 2021):

    “An #ExpressionofConcern for my 2015 paper in @ESAEcology https://esajournals.onlinelibrary.wiley.com/doi/10.1002/ecy.3292 We found anomalous patterns of duplicated values in several of the data spreadsheets (issues summarized here: https://github.com/rroyaute/Royaute-Pruitt-Ecology-2015-Data-and-Code/blob/master/SequenceSniffing/Summary ). I know the editorial team @ESAEcology is working very hard to determine what may have caused these duplications and will update on the situation as soon as possible”.

    See https://pubpeer.com/publications/C6ACC9E8B7FE9CE4DB3AC705B27468

    Like

  2. Klaas van Dijk

    Copy/pasted from https://link.springer.com/article/10.1007/s00265-021-02997-3 : “Retraction Note to: Behav Ecol Sociobiol (2011) 65:1987–1995 https://doi.org/10.1007/s00265-011-1208-0 . This article (Pruitt et al. 2011) has been retracted by the Editors-in-Chief at the request of the authors. After publication, the authors found anomalies in the raw data that were collected. The conclusions presented based on these data are therefore unreliable. Nicholas DiRienzo, Simona Karlj-Fiser, J. Chadwick Johnson, and Andrew Sih agree with this retraction. Jonathan Pruitt has not explicitly stated whether he agrees with this retraction. Published: 11 March 2021.”

    See for backgrounds https://pubpeer.com/publications/2BE2F156C5AC7C0C33939A612FDD9C and a recent thread at https://twitter.com/Niku_DiRienzo/status/1370007020976308227

    Like

  3. Klaas van Dijk

    Copy/pasted from https://dynamicecology.wordpress.com/2021/03/12/friday-links-284/ :

    (….) “The latest #pruittdata retraction just dropped. It’s Lichtenstein et al. 2016 Behav Ecol Sociobiol. It’s for anomalous strings of repeated sequences of observations, and formulaically generated observations, that Jonathan Pruitt couldn’t explain. Pruitt didn’t agree to the retraction, because of course he didn’t. Lead author James Lichtenstein and co-author Nick DiRienzo each spent something like 150-200 hours investigating the data, writing reports on it, and evaluating Pruitt’s purported explanations for the anomalies. Here’s Nick DiRienzo’s detailed PubPeer writeup of the anomalies https://pubpeer.com/publications/1084FEB633DD7021D4D5A954AACFBA Kudos to all the co-authors for doing the right thing; it sucks that you had to do it.

    “And the hits just keep on coming.” https://www.youtube.com/watch?v=WQZqJ_-WAO8 You said it, Tom. Grinsted et al. 2013 Proc B also was retracted this week. You should click the link https://royalsocietypublishing.org/doi/full/10.1098/rspb.2021.0335?af=R and read the retraction notice, it’s exceptionally detailed. Kudos to co-authors Lena Grinsted, Virginia Settepani and Trine Bild for bringing the issues with the data to the journal’s attention and requesting the retraction. Pruitt is now up to a dozen retractions. That doesn’t count the various corrections and Expressions of Concern, or any of the many papers still under investigation. Protip: try not to do things that result in your Wikipedia page looking like that last link.

    Whoops, make that 13 retractions. It’s hard to keep up… Pruitt et al. 2011 Behav Ecol Sociobiol just bit the dust. Kudos to co-authors Nick DiRienzo, Simona Kralj-Fišer, J. Chadwick Johnson, and Andy Sih for fighting to get this paper retracted as soon as they discovered the data were full of anomalies. Want to know exactly what the anomalies were, how Jonathan Pruitt tried to explain them (totally unconvincingly), and what it’s like to go through a back-and-forth with him? Here’s co-author Nick DiRienzo with the blow-by-blow. Amazingly, Pruitt still has a ways to go before his name shows up here https://retractionwatch.com/the-retraction-watch-leaderboard/ (Note that this paragraph was updated a few hours after posting to include Andy Sih, whom I inadvertently left out originally. My bad.)” (…..)

    Like

  4. Klaas van Dijk

    Copy/pasted from https://www.nature.com/articles/nature13811 (Site-specific group selection drives locally adapted group compositions, Jonathan N. Pruitt & Charles J. Goodnight, Nature volume 514, pages 359–362, 2014):

    “26 February 2021. Editor’s Note: Readers are alerted that the reliability of some of the data presented in this Article is currently in question and is being investigated by the editors. A further editorial response will follow when the issues are resolved.”

    It seems to me that this “Editor’s Note” has to be interpreted as an Expression of Concern.

    See also https://pubpeer.com/publications/77C43A85588D997CA3DE730344AE3B (with a whole bunch of new questions, posted on 17 April 2021 by “Atheropla decaspila”).

    Like

  5. Klaas van Dijk

    Copy/pasted from https://dynamicecology.wordpress.com/2021/04/23/friday-links-289/ :

    “(….) The #pruittdata saga started with an Am Nat paper (see here for background). Now, #pruittdata is (hopefully!) over as far as Am Nat is concerned https://twitter.com/DanielBolnick/status/1384100738570194948 . Pruitt et al. 2012 Am Nat https://www.journals.uchicago.edu/doi/10.1086/663680 has received a correction https://www.journals.uchicago.edu/doi/10.1086/714866 and an unusually detailed Expression of Concern https://www.journals.uchicago.edu/doi/10.1086/714867 . Which means that all of Jonathan Pruitt’s Am Nat papers have now been retracted, corrected, and/or subjected to EoCs. As someone who’s been heavily involved with Am Nat’s investigations, boy do I wish I could say the saga is over for everyone else too. But it’s not. Not even close. There are still investigations ongoing at various other journals (see below). The McMaster University investigation is still ongoing (or it was as of a couple of months ago, anyway). Is Tennessee still investigating?

    Speaking of ongoing #pruittdata investigations…I think I missed this at the time, but back in late Feb. 2021, Nature attached an Expression of Concern to Pruitt & Goodnight 2014. So, if you’re scoring at home, it took Nature ~13 months to get around to publishing an EoC that merely says (in so many words) ‘concerns have been raised; we’re investigating’. Even though they have in fact been investigating (or at least, have said that they’re investigating) that whole time. That’s Nature for you. Bob Trivers had to fight them for years and self-publish an entire book to get them to retract his own paper https://www.nature.com/news/symmetry-study-deemed-a-fraud-1.12932 .

    One more #pruittdata link: I second Flo Débarre’s comments https://twitter.com/flodebarre/status/1384166692280958984 . Investigating #pruittdata for Am Nat was an enormous amount of work, but it needed doing and it needed to be done right–thoroughly, promptly, and fairly. I’m proud to have so many colleagues who care so much about getting it right. I hope we’ve set an example that others will be inspired to follow if–god forbid–they’re ever faced with a similar situation in future. (…)”

    Sorbus torminalis, an anonymous Pubpeer commenter, is one of the 8 authors of this EoC. It is towards my opinion highly unusual that efforts of an (anonymous) commenter at Pubpeer results in being a co-author of an EoC (or of a Retraction Note).

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: