In early 2020, I had the honour to publish on my site a grand investigation into Chinese paper mill industry, a curious business model created by the centrally imposed system of promotions and rewards for healthcare employees in China. Almost 600 papers were eventually traced back to one single mysterious factory of fraudulent biomedical research papers, all of these completely fake, fabricated by ghost-writers from libraries of random graphic fragments, like microscopy pictures, flow cytometry plots and individual western blot bands. Yet all published in allegedly respectable journals by all the well-known publishers.

The investigative work was done by a group of experts. Only one of these paper mill industry investigators is known to the general public: the US-based research integrity consultant and image duplication sleuth Elisabeth Bik. The others chose to remain anonymous, using only pseudonyms, because unfortunately academia is not anywhere near as tolerant towards whistleblowers as it is towards perpetrators of research misconduct. But this doesn’t mean their efforts should go unrecognised, in fact the team still continues working together exposing paper mill activities, without earning a cent from their efforts. Their impressive output can be seen daily on the website PubPeer.com, where individual papers can be commented upon anonymously.  

In particular two of these dedicated sleuths have been extremely productive, their joint work uncovered hundreds of paper mill fabrications which ended up published as peer reviewed research papers, many are retracted now (here you can find them all). One journal took extreme measures to prevent this from happening again. Unfortunately, many other affected journals and their internationally recognised publishers chose to do little to nothing so far. Some are even an active part of the problem.
Below, I interview these two amazing and pseudonymous experts: Smut Clyde (an academic in New Zealand with a unique talent for spotting image duplications) and Tiger BB8 (a US-based clinical researcher, originally from China). Their real identities are known to me

---

The Chinese Paper Mill Industry: Interview with Smut Clyde and Tiger BB8

LS: Dear Tiger BB8, dear Smut Clyde, many thanks for agreeing to this interview. Let us start with an explainer: how would you define a paper mill? How do you imagine it to operate, which services does it offer? What does one need to set up a paper mill? 

TigerBB8: I define a paper mill as producing papers for profit (obviously), the papers they produce do not have any real data, they use templates to do a batch production, they have careful plans to avoid being detected by journals, e.g. submitting a set of very similar manuscripts to multiple journals almost at the same time.

Everything needed for publishing a paper can be a service a paper mill provides. Now we know, that even includes responding on Pubpeer.

A small paper mill needs a couple of people with basic understanding of the components of a manuscript. All they do is to copy sections from published papers and piece them together into a new manuscript. There also needs to be someone who has expertise to supervise them to make sure at least the products look like they are making sense.

A much more sophisticated paper mill can be in the format of a consulting company providing service to researchers on almost everything. They can have legitimate business projects, e.g. taking care of animal experiments, sample analysis, data analysis, manuscript draft assistance, etc. However, they are also running high level paper mills, using the collection of data and other materials from their small or big legitimate service.

Smut Clyde: I suspect (without evidence, only intuition) that many papermills are larger, industrial-scale studios. I think of them as a medieval scriptorium, basing my mental picture on “The Name of the Rose”, with a room full of monks at their separate desks, working on separate tasks.

There are grades of quality, and (I imagine) a corresponding range of prices… related to the calibre of the journal where a paper appears, and how much effort the mill must invest in individualising the product to hide its assembly-line origin. As many people have already pointed out, the ones we detect as fraudulent are at the cheap end of the market. There must be more sophisticated papers that are never suspected of shenanigans because the fakers take more pride in what they do.

When authors are asked in PubPeer about the fake or repurposed material in their papers, they often explain that they outsourced some (or all) of the experiments to an outside biomedical laboratory, making them the real victim. It would add credibility if they named the company who cheated them to warn others against the same mistake of using its services. Even so, I think there’s an element of truth here, and that some papermills specialise in creating results and weaving a narrative around material contributed by the customer, so they can advertise themselves as a “paper-finishing service”.

A commenter on Leonid’s blog gave a first-hand account of small-scale, one-person papermilling as a cottage industry.

Using their own information, James Heathers and Otto Kalliokoski reconstructed the daily routine of a similar operation.

LS: You both investigate research papers for image duplications and other irregularities. How did you arrive at your paper mill investigations? Did you suspect there must be mills operating, or did you arrive at that conclusion seeing a string of similar papers? At which point did you start to see patterns, and how did you decide these papers must be from the same paper mill? 

Smut Clyde: My introduction to this slippery slope was a post at PubPeer about miraculously multiplied mice. I wondered if there were additional manifestations. Then I wondered if it was easier to search for distinctive verbal passages. The papermill phenomenon had already been documented by then, but it was my introduction to the ecosystem.

The ‘tadpole’ papermill [read here and here, -LS] came to attention because contributor ‘Indigofera’ commented at PubPeer about fake Transwell images in a couple of papers, and the fakes were constructed from the same motifs… that is, they were constructed by the same people. That inspired a search for other examples, and attention broadened to the stylised form of the Western Blots, and the next thing we knew, the spreadsheet had stretched to nearly 600 entries. Byrne & Christopher had independently zeroed in on the same constellation of defining features, which was reassuring. Other investigations followed the same trajectory. 

The unifying feature could be a style of synthetic Western Blot bands, or a narrow universe of blots providing the same Figures again and again. Or flow-cytometry scatterplots, either recycled or obviously hand-drawn. PubPeer contributor “Xylocampa Areola” started the ball rolling for another papermill with a comment about repeating flaw cytometry, and I became obsessive. 

LS: Which criteria do you use to point at a research paper and say: that was fabricated by a paper mill?

Tiger BB8: Usually I would wonder if there’s a paper mill behind it if I start to see some pattern. For example, when two unrelated papers had identical language, figure arrangement, language errors, etc, which is very unlikely. Then I’d log this suspicious pattern in my mind and be vigilant for this pattern to appear again. If I am convinced that this pattern is real, I’ll share my thoughts with colleagues, asking if anyone else has seen that same/similar pattern, and lots of time, I received confirmatory input. Further clues are situations where people very unlikely to collaborate end up as co-authors, or the authors don’t appear to have the expertise needed for that study.

I also check the authors’ affiliations. E.g., if the author’s affiliation is a regional hospital in China, e.g a hospital for an administrative district below the level of a province, it is suspicious. Unless in a very affluent area, the possibility for such a hospital to support cellular or molecular level research or even animal research is very minimal.  Certain kinds of hospitals are unlikely to be able to provide the facilities and the qualified technicians to support experimental research.

When concerns about the paper are posted on PubPeer, I observe  the author’s response or lack of response there, especially when I additionally reported these PubPeer cases on Weibo, the Chinese social media. When the authors still totally ignore the PubPeer critiques after I posted them on Weibo, it is very likely that they purchased paper mill products. Sometimes however you suddenly see a couple of very similarly worded PubPeer responses, in these cases it’s very likely the paper mill case manager is responding for their customers.

Smut Clyde: I can’t offer any helpful advice there. Papermill products are only apparent in retrospect, in the context of a family of papers from unrelated authors, using the same image (or the same image-generation methods).

What tends to happen, each time one of these ‘mill styles’ is identified, we find that some of its output had already been called out at PubPeer. But called out for internal image overlaps perhaps, or visible manipulation, NOT for being mass-produced, even though the papers had already attracted skeptical attention. Papermills tend to be sloppy… the customers aren’t paying a premium for perfectionism. So their products have an elevated risk of shenanigans.

There are a lot of papers out there that came off a production line, with (say) Western Blots or flow-cytometry plots that are variations on a theme. But I can’t see a way to prove it, and editors are reluctant to retract papers simply on the basis of “Anonymous PubPeer Contributor’s Intuition”.

LS:  You speak of regional hospitals as suspicious affiliations. Who are these typical paper mill customers and why do they resort to buying such products?

Tiger BB8: Doctors in regional hospitals are the primary customers of Chinese paper mills. They need papers to achieve a promotion but their hospitals do not have the facility and technical support to allow any meaningful research. On the other hand, most of Chinese doctors do not receive good research training. When they are required to present something that they were not trained for and they have no ways to do experiment-based research to produce a paper, they turn to paper mills to buy ready products. Most importantly, they do not want to reduce their clinical time (that’s where they get paid) to invest into research.

Smut Clyde: The economic niche for papermills is a product of a policy that forces clinicians to publish an academic paper as a requirement for promotion along a non-research career. As far as I know, that only exists in China. If the Chinese govt. dropped this requirement, the incentive for papermilling would disappear and we could go back to looking for old-fashioned home-made fraud, and looking for images copy-pasted within the output of single laboratories rather than dispersed across the entire literature.

Papermills have a customer base of people who need to publish once, As with dissertation mills and essay mills, if someone is aiming for a full-time research career, they’ll do actual research (or handle their own fabrication, or they’ll have students to fake results for them). That said, every papermill does have a couple of repeat customers, but they’re a minority.

LS:  So you say it is mostly one customer per paper mill product? One small doctor in a provincial hospital buying a paper to get a promotion and pay rise?

I must qualify that. There is one papermill that seems to have faculty from Jilin university and its affiliated hospitals as its only customers (which is not to say that fakes from that institution all come from one place; Jilin Uni accounted for about one-sixth of the production of the “tadpole papermill”). A second papermill sells about half its production to Central South University. In terms of repeat custom, CSU has professors with five, six, seven papers, all with the hallmarks of coming from the same papermill. They’re pikers compared to Jilin where professors have signed nine, 11, 12, even 15 papers from a single (different) source. Whole programs of research, across over a dozen departments, are just fairy-tales.

I am advised that Jilin University and CSU are established, important universities that became medical teaching institutions by way of mergers with medical schools. They don’t have a tradition of biomed research, and perhaps they don’t give the medical curriculum a high priority. Each is surrounded by a little empire of affiliated hospitals, so there are also lots of clinicians, all needing a publication or two. I begin to suspect that people in central administration realised how much money was spent on papermill fees, so they turned “making stuff up” into official policy and set up their own fabrication studios to save money and increase staff productivity. 

About 80% of the faked papers from CSU appeared in journals from a single publisher. The Jilin studio preferred the same publisher to a lesser extent, using those journals for 45% of their output – so the bond wasn’t such a tight symbiosis, and they also worked with other low-end publishers from the “grey literature”. In the last few years they widened their scope to target “Biomedicine & Pharmacotherapy”, from Elsevier. 

LS: In this regard, where do you think paper mills originally come from? Are the owners former or current scientists? Are there further clues that active academics in actual universities are running a paper mill from the inside? 

Tiger BB8: Yes, I believe many paper mill owners are current or former scientists. I do believe that some paper mills are run by actual active academics. I can give you one example. Years ago, I found 8-9 papers reusing data from a lab in a Chinese university. I announced it on Weibo as “Professor so-and-so, there are papers out there using your lab’s data and it made your own papers being questioned on PubPeer. Please be vigilant in protecting your data”. I posted this at least 2-3 times, receiving no reaction. Then, my Weibo followers told me that they personally relayed my warning message to this professor. But again, no response. Why? Later someone suggested the possibility that those data were sold to others by the lab. That’s the only plausible explanation that makes sense.

Smut Clyde: I am also convinced that some papermills are based at laboratories and run by actual researchers to supplement their incomes. They can draw on the lab’s image archives, using the ‘out-takes’ that weren’t needed for their own publications to fabricate manuscripts for customers in need of authorship. Qin He’s laboratory is an example. 

In the field of geophysics, a plague of papers with fake high-energy X-ray spectroscopy could be traced to one person, though he managed to leave his customers with the blame when it all became a scandal.

I don’t know if the trade in purchased co-authorship counts as true papermilling. Elisabeth Bik blogged about Fatih Sen in Turkey [read also here, -LS], and his lab’ industrious output of nanotechnology forgery, with versatile hard-working TEM images; it seemed that authorship on his confections was available for a price. Then there was Guoqiang Zhao, whose papers often boasted co-authors who weren’t students or linked to his laboratory in any obvious way.

LS: We spoke about the paper mill providers and its customers. But there is of course another party in this game. What is the typical journal falling prey to a paper mill? What were the worst editorial practices you encountered in this regard?

Tiger BB8: Journals must meet the following conditions to easily be the target of a paper mill: be for profit or even profit-only (like the e-Century journals), with a journal impact factor below 10 (mostly about 1-5), with low quality peer-review or even lack of peer-review, and of course, predatory journals.

Based on my observations, Open Access journals are the most common victims of a paper mill. However, some seemingly prestigious paywalled journals are also infiltrated by paper mill products.

Smut Clyde: There is no typical journal! No-one is safe!

That said, some publishers undeniably went into business to help Chinese authors meet their ‘international publication’ requirement (and to profit while doing so), or they drifted into that niche, so in effect they’re stovepipes for papermills (though sometimes naive scholars send genuine research there by mistake). I also have in mind the e-Century journals; plus European Review for Medical and Pharmacological Sciences (ERMPS); Journal of the Balkan Union of Oncology (J.BUOn); a few others. Papers in J.BUOn might cite ones in ERMPS or the American Journal of Translational Research (published by e-Century), and vice versa, as they’re all from the same source. 

J.BUOn is the most egregious example, where the working relationship between editors and papermills is especially close, and the editors react to warnings about bogus papers by quietly revising those papers to remove the evidence that exposed them.

Together these low-end journals form a kind of penumbra, a parallel literature of fantasy, parasitical upon the real-world literature of experiment-based results. These are ‘write-only journals’, not intended to be read, but the Abstracts do show up in literature searches (because they’re PubMed-listed), so lazy authors of systematic / synthetic reviews cite them anyway. I am glad that I’m not working in these fields so I don’t have to worry about distinguishing wheat from chaff.

Other publishers escape the label ‘predatory’ because they are large enough and long-enough established, but of course there are journals from Elsevier and Wiley and Taylor & Francis and Karger that are also stovepipes.

LS: How many different mills have you found, with how many papers each, and how are their products different? 

Smut Clyde: Without claiming undivided credit for detecting them, I set up Google-document spreadsheets for over a dozen styles of fabrication (we might as well call them ‘separate papermills’). The ‘tadpole’ style has 600 examples. Others are less prolific, but it might be that I just haven’t looked in the right journals. 

One papermill has a small repertoire of flow-cytometry scatterplots, and a penchant for Western Blots that resemble a line of processionary caterpillars; I had recorded 50 cases, with the journal Oncology Research as the largest stovepipe. Then the news broke that the editors of RSC Advances were retracting 70 papers or so. Several turn out to be from this “travelling tumor” papermill, but I had never looked in RSC Advances!

LS: Do you see a qualitative difference and are there clues about the value for money aspect?

Tiger BB8: I do see a difference in quality of paper mill products.

The price of papermill products varies considerably, but almost always tied to the Impact Factor of the targeted journal. As I said earlier, journals with IF in the vicinity of 5 are the easy prey for papermills. I asked on Weibo about the price tag. Some followers forwarded me the ads they received and I also obtained from some other ways. The price tag I have seen is anywhere between RMB 10,000 – 25,000 [€1,300 – 3,200, -LS] per point of the IF, with much higher price for expedited order, and discount on orders of multiple papers. 

LS: How do Chinese authorities react to your findings? Are paper mill customers sanctioned in any way? Are there signs of a crackdown on the paper mill industry?

Smut Clyde: I am not aware of any signs of a crackdown. It’s my impression that the Chinese authorities make noises deploring research fraud, but their equivalent of the Office of Research Integrity is indeed hopelessly underfunded. They’re not reversing the policies that make papermilling inevitable. Repeat fraudsters are sometimes reprimanded as an example to others, but this doesn’t really extend to the papermill customers (how could they be sanctioned? Nothing would be accomplished by banning a non-researcher from future research).

Tiger BB8: First, the Chinese authorities are really mad at the discovery and exposure of Chinese paper mills. However, like always, they like to silence the person discovering the problem rather than working on solving the problem itself. As far as I learned, they are very mad at us, especially at Dr. Bik. They ignore the fact that Dr. Bik has been exposing research fraud from many countries e.g. USA, India, Brazil, Turkey, etc., in addition to those from China. 

Second, they do take every of our reports on paper mills quite seriously. They went through our list of papers in each of our reports on paper mill and, of course, picked those less influential authors to impose some sort of sanctions. This is why some paper mill customers have been sanctioned one way or another, with different degrees of punishment.

Yet so far, zero signs on any crack down on the paper mill industry. I believe this has something to do with lack of resources. Rumors say that the people working on research misconduct at the Ministry of Science and Technology (MOST) are only a handful. No way for them to catch up with the report of paper mills from us. 

I have been warned many times by various parties from China, not necessarily regarding the paper mill industry, but broadly for exposing research misconduct by Chinese authors. My Weibo account has been suspended for exposing a big fish, my Weibo posts have been deleted by the administration per demands and threats from angry authors. But these attacks mostly were not from paper mill customers or business owners.

LS: Finally, your advice for the scientific and publishing community. What would you suggest should journals, their editors and reviewers do in order not to become prey to paper mills?

Tiger BB8: I have the following suggestions: do actual peer-review, require original images and data, retract all paper mill products, regularly browse PubPeer, and hire research integrity sleuths.

Smut Clyde: Hire me! 

Otherwise: I won’t suggest “choose better peer-reviewers”, because peer reviewers are already over-worked volunteers, and detecting deliberate fraud would be a big ask to make of them. But someone in the editorial process needs to take an adversarial stance, and ask of every submission “Could this be from a papermill? How would it look different if it was?” Here is more advice: 

• Check the IP addresses used for manuscript submission in case there is a pattern of multiple papers streaming in through the same channel.
• Be prepared to retract papers, when it becomes obvious in retrospect that they were cobbled together to pad out someone’s CV. Embrace the retractions! Boast about them!
• Made-up papers become a real problem when people accept them as fact, and expect their students to replicate the results or extend them, and cite them as part of their own research program. Perhaps we need software that scans incoming manuscripts, and informs the authors (and reviewers) if they cited work that’s been retracted or earned an Expression of Concern.
• A Wiki devoted to the styles of known papermills could be good, too. Currently there are discussions at PubPeer when a papermill comes to light, but mixed in with other conversations, and that is not PubPeer’s primary function. Nor is it the core activity of Retraction Watch, nor of For Better Science.

LS: Dear Tiger BB8, dear Smut Clyde, many thanks for this interview, and especially for all the excellent work you do!

---

Note: The interview was originally invited by a peer reviewed journal and submitted there in early March 2021. After delays and disagreements about requests for certain extra references and extensive editing of the interviewees’ replies, the original interview is published here instead, but in an updated form. 

---