Lawyering-up Research integrity

Spiderman’s lawyer is having you for dinner tonight

Spider researcher Jonathan Pruitt is accused by his coauthors of data manipulations, after 3 retractions they demand more. A lawyer's letter was supposed to stop that, but Pruitt tells me: "I'm happy for folks to engage in public discourse about my data integrity""

The Canadian shooting star of behavioural ecology Jonathan Pruitt used to be everyone’s best friend. Up until his co-authors accused the spider researcher of data manipulation and achieved retractions for 3 of his papers. His reputation in tatters, his research record facing even more retractions, his elite funding as one of Canada’s “150 Research Chairs” and prestigious job at McMaster University in danger, Pruitt did what every red-blooded academic in his position does: he deployed lawyers against his critics.

As Science reported, the lawyer’s letter was so scary that some journal editors panicked and hid under their tables, waving a white flag of surrender. No more Pruitt retractions unless specifically requested by the ongoing investigation at McMaster, no matter what the evidence submitted by co-authors proves.

The original recipients however, all of them former Pruitt co-authors, are not complying, they continue vetting Pruitt’s publications for further irregularities. As I learned, nobody signed anything or announced to adhere to any of lawyer’s demands. And why should they be afraid, the lawyer’s letter is not even really threatening anyone. It is a cack-handed attempt in intimidation which even Pruitt now apparently distances himself from.

All the letter from Millard and Company does, is to educate its recipient that nobody is supposed to be scrutinising Pruitt’s papers but his employer McMaster University, and that every retraction must be agreed by the Committee on Publication Ethics (COPE, a scholarly publisher lobbyist), whose advisory guidelines to its member journals are presented by Pruitt’s lawyers as a kind of a law for breaking of which you might end up in prison.

The letter is labelled “PRIVATE & CONFIDENTIAL WITHOUT PREJUDICE”, which is exactly why I publish it here in full, below. I hope the Millard lawyers are happy with this arrangement.

The Pruitt case made international news, it even got its own hashtag n Twitter: #PruittData. The communal effort to uncover data irregularities began with Kate Laskowski, now assistant professor at UC Davis in USA. Laskowski told the story on her blog, and it might explain the success recipe of Pruitt: when he met some scientist with an unproven pet theory, he provided them with the perfect experimental data. Which now proved to have been too good to be true.

In science, many beautifully designed theories collapse on experimental reality. Biology is very messy, complicated and rarely does as told, even and especially when the theory which biology is expected to adhere to is clever, beautiful and elegant, of the kind the editors in elite journals love and the research funders cherish. Which is exactly why so many scientists resort to research fraud, to force scientific reality to adhere to their theories.

Laskowski, then a PhD student, had a theory about animal behaviour in social context. She met at a conference the young shooting star Pruitt, who works with social spiders. Pruitt generously offered his collaboration, and soon perfect data arrived which perfectly proved Laskowski’s theory right. Pruitt’s datasets served as template for 3 papers in respectable journals, until someone contacted Laskowski with concerns. That colleague was Niels Dingemanse, professor at the LMU Munich in Germany, and he noticed there were statistically too many duplicate numbers in the paper’s publicly available original data file.

Pruitt told Laskowski he measured 4 spiders simultaneously, hence identical values. But how can spider act so synchronously and into 1/100 of a second?

Thus alerted, Laskowski started to go through Pruitt’s Excel files, and indeed found many repetitive numbers. Pruitt was quick with an explanation, which sounded a bit weird, but it kind of sufficed to consider a correction. But then Laskowski analysed the files in more details and found entire strings of numbers repeating themselves, with the only logical explanation that they were made up. Once Laskowski removed all repetitive numbers, the remaining ones showed no significant difference in spider behaviour between experimental treatments. Her theory collapsed, so Laskowski had her own three papers retracted, one by one.

Laskowski: “Each line is an individual spider. What I noticed is that the last two pre-treatment values, seemed to be identical to the first two post-treatment values in many spiders in this treatment. I colored those values red here. I also noticed that many values were very similar, but off by one digit in the hundreds column, e.g. 77.65 became 477.56. Those values I colored in yellow.”

In another blogpost on the topic, Dan Bolnick, editor of American Naturalist and professor at University of Connecticut, explains that the journals actually very much can be expected to take responsibility and to perform data integrity investigations, which his journal did. Bolnick even used to curate a publicly viewable (but not editable) Google Docs file to track the reliability of Pruitt co-authored papers. Some papers actually proved to contain no evidence of irregularities. The list was removed now, after Pruitt lawyered-up. Bolnick’s commendable behaviour stands in stark contrast to certain other editors who now hide behind Pruitt’s lawyer’s letter and the McMaster investigation.

Laskowski: “as opposed to being formatted in columns as on Sheet 1, these values were transposed – so that the columns became rows. […] Here on Sheet 2, the duplicate sequences were not just sequences of 4 numbers (as they were on Sheet 1), but rather I could now see that in fact there were whole blocks of duplicated sequences.”

Unlike his lawyer claims, Pruitt even agreed to the first retraction, which happened in American Naturalist. Laskowski quotes him with: “it is well that the paper is being retracted”. But eventually Pruitt changed his mind. Things got out of his control namely. After that retraction, other Pruitt coauthors reached out to Laskowski, the search for data manipulations became a communal effort. Many went on to publish their concerns on PubPeer, where other Pruitt’s coauthors first defended Pruitt, until some switched sides and joined the investigative group effort, like the associate professor Noa Pinter-Wollman at UC Los Angeles. Did these scientists also discover their own beautiful theories, once verified by Pruitt’s data, now in lying in tatters? Possibly.

One former Pruitt collaborator explained why his co-authors initially defended the accused:

It’s worth remember that lots of us considered ourselves to be good to very close friends with him. You just don’t want to believe something like this.

And then they looked into their own papers with Pruitt. A co-author shared this about the raw data behind a publication:

literally 1/2 the data were sequence repeats 10-14 units long but sometimes with select manipulation

That source also described the communal #PruittData investigation:

There are 28 channels, on the slack channel where people are looking at this, each of which is dedicated to a paper. […] The sequence repeat pops up a ton. Formulas are pretty common. One person found that a formula was used but it was like + 20.3 or something to better hide the pattern.

Rows K-O in the Pruitt raw data file “Black_Widow_Data” list independent experimental values for “Darkness”. And yet, values in row O are simply values in M +1.5. The author sloppily even left inside the formula with which the values were fabricated.

Pruitt’s manipulative approach over his former friends does not work anymore. If anything, people are afraid of his revenge now, rightly or not, time will show. As a source wrote to me:

Jonathan has a typical response when confronted.  He restates the issue, gives excuses that don’t directly address the question, often will say we should remove the questionable data and reanalyze, and then finally will agree to retract if the results don’t hold. This was the pattern until lawyers got involved that is.  

[…] When I was early in my career I was also just enamored with him and trusted him.  So when he comes dangling a dataset that would turn into a high level paper, you gladly take it and write it up.  Later it seemed like what would happen is that he’s send off just chunks of datasets to different people so they don’t see the whole thing.  Or he would talk to say a postdoc about an idea, the six months later the data would appear in that postdocs inbox….  so they don’t ever see the stuff being collected, but are thrilled to have the data and to get a famous person to work with.” 

Pruitt raw data file “Spider_StateBasedPersonality”, sheet “Anelosimus.2” Colour coded are repetitive number sequences

Much of #PruittData problems came out because Pruitt used to share his research data openly. Until he got caught that is:

Dryad is the data repository for open source stuff. Many journals have started requiring us to upload our data. Jonathan would do that sometimes, but not others. Or he would upload only a portion of the data. This started when one of his own grad students acted as a whistleblower and told another lab about some issues. That lab looked into a bunch of his data on Dryad and found all these issues.

But now, the lawyer’s letter. Which even Pruitt himself admits is not suitable to scare anyone. He namely wrote to me via Twitter direct message, obviously with reference to the Science article:

I realize the report made it sound like I legally carpet bombed people; that’s not the case. I know folks have interpreted the action as a flimsy fear-induced gag order. That’s not the case either. I’m happy for folks to engage in public discourse about my data integrity


PRIVATE & CONFIDENTIAL
WITHOUT PREJUDICE

[….]

Please be advised that our firm has been retained by Jonathan Pruitt with respect to the investigation (if any) being conducted by […] into the paper “[…]” by […] in 20[…]. I understand you have been contacted by Dr […] one of the coauthors, seeking a retraction.

Over the last number of weeks, there has been an online campaign directed at Dr. Pruitt and his work which has come to be known as “#pruittdata” and “#pruittgate”. As part of #pruittgate, a number of academics have organized an extra-institutional online forum for posting, reviewing, and criticizing the data integrity of various of Dr. Pruitt’s research studies and papers. It appears that Dr. […]’s concerns arise from that project.

Dr. Pruitt disagrees that there are any grounds for this paper’s retraction. Any decision about retraction must be conducted fairly and in keeping with professional norms as articulated by the Committee on Publication Ethics (“COPE”) Guidelines on Retractions. It does not appear that this paper meets any of the criteria for retraction as set out in the COPE Guidelines. In particular, the COPE Guidelines do not support retraction if “the main findings of the work are still reliable” or if “an editor has inconclusive evidence to support retraction, or is awaiting additional information such as from an institutional investigation.”

In addition, I am concerned that the procedural fairness of any investigation has been hampered by the manner in which Dr. Pruitt’s work is currently being criticized online. The social media campaign and online forums which spurred this complaint break with the normative practices of the field and very likely prevent a fair assessment of Dr. Pruitt’s work.

The controversy over this particular paper highlights the problems with using an unmediated online forum to attack the data integrity of a body of work. In this case, it appears that the wrong electronic file was uploaded to the forum. The final published paper does not rely on the impugned data.

If […] there is sufficient grounds to retract this paper, it must first notify Dr. Pruitt and provide him with a chance to respond. A premature retraction without Dr. Pruitt’s agreement is likely to further harm his reputation and negatively impact his career in an irreversible manner.

Please confirm that […] does not intend to take further steps at this time
with respect to the above noted paper. Otherwise, if the journal intends to take any further steps, please provide me with correspondence (1) detailing the journal’s concerns, including what evidence it has showing that the paper’s findings are no longer reliable, and (2) outlining the process the journal intends to follow prior to retraction, including when and how Dr. Pruitt will be permitted to respond to any allegations against him.
I am happy to discuss this matter further if you have any questions or concerns. If you would like to speak to me, please arrange a time through my assistant Lori Leblond (lori@millardco.ca, 1-416-920-3123).
Yours truly,

Marcus McCann


Donate!

If you are interested to support my work, you can leave here a small tip of $5. Or several of small tips, just increase the amount as you like (2x=€10; 5x=€25). Your generous patronage of my journalism will be most appreciated!

€5.00

9 comments on “Spiderman’s lawyer is having you for dinner tonight

  1. Somebody should inform spiderboy’s lawyers:

    Like

  2. Pingback: Here’s the letter Jonathan Pruitt’s lawyers have been sending to journals and his former collaborators #pruittgate #pruittdata | Dynamic Ecology

  3. Though Laskowski is completely innocent, it is not unreasonable to speculate that she got a faculty job from his questionable data, from which she got three published papers from, and she may have pivoted off that for more work. This gave her an advantage over other candidates that did not have her great publication record. I dont, however, seeing her rescind her faculty position! Why didn’t she collect her own data? Most of my ideas are wrong from the data that I collect myself, and I kind of wish the same thing happened to her to equal the playing field. But then again, maybe if this had happened, a full fledged cheater who fudged data could have then gotten the position.

    Like

  4. Pruitt’s co-author Nicholas DiRienzo now commented on PubPeer
    Pruitt et al Behavioral Ecology and Sociobiology (2011)
    https://pubpeer.com/publications/2BE2F156C5AC7C0C33939A612FDD9C#1
    https://drive.google.com/uc?id=1grJs92VKblsNro0rKGRKNRETuRakZUz7
    Pruitt explained it as

    “Lingering formula are present in some versions of data files when there contained missing values. Excel-based stats and other older stats programs don’t always contend with them well. It’s possible I uploaded a version with one of those present. […] If they change the results in an important way that changes the interpretation, then ax the paper/s. If not, then consider an addendum. It’s important not to throw the baby out with the bath water.”

    DiRienzo concludes

    “Myself and the other coauthors (excluding Dr. Pruitt) deem the data associated with this paper as being untrustworthy. There is no reason that such a formula should ever exist in a dataset that is supposed to contain individual behavioral measures, and I have yet to receive a satisfactory explanation as to why it was in there. Additionally, the discrepancies between the values contained in the Small and Large datasets cannot be explained by any simple excel or data input error. We have requested a retraction from the journal and it is currently being reviewed by the editors.”

    Like

  5. Looks like Pruitt’s lawyer letter may have had a chilling effect after all:

    https://pubpeer.com/publications/F62202DB4DB0804CD2E65C25764549#8

    It seems reasonable to draw a connection between the lawyer letter and the apparent posting and removal of a retraction notice.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: