The pseudonymous PubPeer user Parashorea tomentella continues their investigation of Chinese papermills. Again, their special focus is on Hindawi, an Open Access publisher which was founded in 1997 by the Egyptian physicist Ahmed Hindawi and his wife, sprouted countless journals publishing countless papers, yet with a quite decent research integrity record: papers reported as fraudulent were generally retracted. But in January 2021 Hindawi was sold to the international publishing giant Wiley for almost $300 million, and the old research integrity staff was let go. Just then, Hindawi became a total free-for-all bonanza for papermills which probably earned a lot of money for Wiley.

But the party soured thanks to the work of sleuths like Parashorea, Smut Clyde and others: almost 6000 Hindawi papers published in 2021-2022 were determined to be of papermill origin. In September 2022, Wiley announced exclusively in Retraction Watch to retract 511 papermill products, never crediting the sleuths but blaming individual special issue guest editors. As it turned out, many of these retracted papers weren’t even flagged on PubPeer before, so far so nice of Wiley. But what about the rest, the remaining five and a half thousand of papermill products? Is anyone at Wiley and Hindawi planning to retract those? Or are they now officially vetted and certified as quality science by rigorous peer review?

Parashorea spent a lot of time digging. It looks really bad. Sometimes there as students running special issues as academic guest editors, on any random and unrelated topic. Sometimes the papermillers openly admit that the data was entirely made up. Sometimes there is even Marxism to be found!

---

Hindawi Garbage Sorting System, Based on Citations

By Parashorea tomentella 

This article is based on a post by Smut Clyde from September 2022, and I suggest you experience the shock of reading that post before reading mine. It will give you a better understanding of the symbiosis of paper mills that exists in special issues of Hindawi journals based on our review of thousands of papers. 

Sorting the Garbage

The spreadsheet “Hindawi” was first published as part of Smut Clyde’s September 2022 article with “only” 155 entries. It then grew to a monument to papermill abundance with 5,467 entries from special issues of 50 journals. It records comments posted by PubPeer users, with contributors including like Rhipidura albiventris, Smut Clyde, Alexander Magazinov, Dorothy V.M. Bishop, N. H. Wise, Sylvain Bernès, Guillaume Cabanac, John A Loadsman, Lonni Besançon, Schizomeris leibleinii, Macrostomum lignano, Tulipa fosteriana, Mycosphaerella arachidis, Cyrtomium chingianum, Saxifraga bergenioides, Scleria hilsenbergii, Papilio homothoas, Sorbus vexans, Parabagrotis sulinaris, Ctenium brevispicatum, Indigofera tanganyikensis and me, and perhaps others not mentioned. To this day, it is still impossible to estimate how many papermill products were published in Hindawi journals through the manipulated publication process, mostly because of the large number of special issues up for post-publication peer review. Quite often the papermill fraud spills over into regular issues of Hindawi, for example where academic editorial board members agree with the papermillers that indeed “No data were used to support this study” (as Indigofera Tanganyikensis discovered):

Special issues in Hindawi I found to be very efficient. A considerable number of papers were accepted within one month, and most papers were accepted within two months. Most accepted manuscripts were published quickly, a few were delayed for a long time.

In the absence of editorial oversight, the Hindawi spreadsheet entries cover a wide variety of topics and concerns raised in PubPeer comments. I think the only reason for this is that out-of-control special issues simultaneously receive both papermill products and self-made forgeries from actual authors. For example, 10 such entries have been identified thanks to the “Vickers Curse“, including Alqahtani 2022 and El Maissi et al 2022, their origins a mystery. 

The records in the spreadsheet come from more than 428 special issues, handled by more than 425 guest editors. Some are guest editors in multiple special issues of multiple journals. For example, Hangjun Che, who served as guest editor for five special issues in five journals. Another example is Kaifa Zhao of the Hong Kong Polytechnic University, who only served as guest editor for two 2022 special issues in different journals, in Computational Intelligence and Neuroscience and in Journal of Environmental and Public Health, yet both are quite impressive. The title of many special issues hints at their broad scope, such as Machine Learning and Network Methods for Biology and Medicine 2020. You might wonder why there is 2020 in the title, but this is because the guest editorial team led by Lei Chen decided to launch Machine Learning and Network Methods for Biology and Medicine 2021 and Machine Learning and Network Methods for Biology and Medicine 2022, most likely with similar plans for 2023. Other guest editors were invited to host subdivide one special issues into many, for example, in Disease Markers, the guest editorial team led by Zhongjie Shi also launched Imaging Disease Markers as a Diagnostic, Prognostic, and Educational Tool series.  

The individual guest editors are of little interest to me – they are interchangeable and replaceable – but I do care about the papermills’ relationship with the publishers. This means I need to find the common characteristics of products that may have come from the same paper mill, and where they were published. Many reported irrelevant citations, and I noted in my early investigations that some papermill products were often found as irrelevant references. Once enough records had been accumulated, I found that probably less than 0.3% of the references were cited more than 10 times, but most of the top 50 were clearly from a handful of authors. I found three clusters by citations (Cluster Huang, Cluster Zhang, and Cluster Editor Li), and then clustered papers with the topic of ideological and political education as the fourth cluster (Cluster Marxism) and papers with the bizarre data availability statement of “The simulation experiment data used to …” as the fifth cluster (Cluster Simulation). These five clusters were placed as five fields in the Hindawi spreadsheet. They contain 1,717 records from 40 journals, and a Venn diagram shows that 1,644 of these records exist in only one cluster. 

The following are the definitions of the three citation-based clusters: 

• Cluster Huang: papers cited Ruihang Huang / Xin Liu / P. Ajay / Balakrishnan Nagaraj / Ashutosh Sharma. 
• Cluster Zhang: papers cited Chunjiong Zhang / Osamah Ibrahim Khalaf / Shahid Mumtaz / Rahman Ali. 
• Cluster Editor Li: papers cited references published on Science Progress and Research (ISSN 2635-0955) / International Journal of Research in Engineering and Innovation (ISSN 2456-6934) / Systematic reviews in pharmacy (ISSN 0976-2779), all which have in common that Zihan Li is on the editorial board. 

The Five Clusters 

Cluster Huang 

In 2019, Xiaoming Yang, the adviser and co-author of Ruihang Huang, wrote in Public Communication of Science & Technology (ISSN 1674-6708, sponsored by the Chinese Society for Science and Technology Journalism):

“In some ways, sometimes the audience’s distrust of the scientific community for giving opinions and conclusions is not their distrust of scientific knowledge points after expert analysis, but an emotional distrust.” 

I read few papers on this topic, but Yang’s exposition struck me deeply because he actually described how I felt when I saw the papers citing his PhD student, Ruihang Huang. 

References to Huang’s Framework for a smart adult education environment from 2015 supported many things, such as the protective factors of hypertension (Wu et al 2022), application of laparoscope (Xu et al 2022), and absorption peak of solar cells (Ma 2022). His two papers on textiles published in IET Collaborative Intelligent Manufacturing (Huang et al 2021a and Huang et al 2021b) were similarly used to support any claim entirely unrelated to textiles. For these two studies, there are many referencing records in the spreadsheet, none of these citing papers concerned with the textile industry. Almost all these citations were from special issues of Hindawi journals, while a few were from Nonlinear Engineering, a De Gruyter journal that has also saw its publishing process hijacked. 

Huang is special, not only because his papers were heavily cited, but also because he is cited for papers that cover different stages of his education. This pattern is most often found with authors who cite themselves, or force others to do so. Publicly available information shows that Huang began his Bachelor program of Education at the South China Normal University in 2010 and continued in the Master program of Educational Technology there in 2014. His exceptional publishing experience began his freshman year and resulted in at least 29 published papers by 2016. Although most of these pedagogical essays are published in journals that most likely do not have a peer review process, it is still not an easy feat to achieve. The exceptionally gifted pedagogy student also participated in the Mathematical Contest in Modeling (MCM) in 2014. 

After a gap, Huang started his PhD program in the history of textile science and technology at Donghua University in 2020. Soon after his admission, he committed himself to publishing of special issues. In 2021 and 2022, he served together with a certain Ashutosh Sharma (a postdoc at the Southern Federal University in Russia) as guest editor for two special issues of the Slovenian research agency journal Informatica (Applications of Artificial Intelligence in Evolution of Smart Cities and Societies and Recent Trends and Advances of Informatica in E-Commerce: Opportunities, Challenges and Solutions). The former featured an editorial that was cited about 100 times by special issues of Hindawi journals. Huang also joined Sharma’s guest editor team at the International Journal of Agricultural and Environmental Information Systems by the US-based publisher IGI Global. Huang also participated in the publication of thriving special issues of Hindawi journals, this time not as a guest editor, but as an author. In 2022, he published four problematic articles with a certain P. Ajay (an active guest editor affiliated with Anna University in India) in four journals: Scanning, Contrast Media & Molecular Imaging, Journal of Control Science and Engineering, and Security and Communication Networks.

Hindawi’s business model of recruiting guest editors may explain why Huang and Ajay were willing to contribute to different special issues on different research areas in a short period of time. Hindawi provide a gift for guest editors: 

“The Guest Editor team as a whole receives three APC (Article processing charge) waivers which they can use to submit a maximum of three research and/or review manuscripts to their own Special Issue. The team can also use the APC waivers to commission research or review articles from experts in the field for their own Special Issue.”  

This may be quite generous, and the offer of “commission” seems to be encouraging guest editors to find more enticing gains on their own.

Huang’s papers are often cited together with those by P. Ajay, Sharma, Xin Liu (Donghua University) and Balakrishnan Nagaraj (Rathinam Technical Campus). Dr. Liu is not a co-author of Huang; the other three are, but the citations they received came not exclusively from papers co-authored by Huang.

Cluster Zhang 

At the beginning, this cluster was actually the “periphery of Cluster Huang”, until I realized that Chunjiong Zhang was almost as prolific as Huang, and I thought he deserved his own cluster. Zhang’s research interests are in machine learning, wireless sensor networks, and distributed robust optimization. He is about the same age as Huang, also Zhang’s papers from different stages of his education (MSc in Chongqing Three Gorges University and PhD program in Tongji University) have been heavily cited in special issues of Hindawi journals. Also his papers (three articles in three Hindawi journals) were caught as irrelevant references. While Huang never cited himself, Zhang did: in the article Cao et al 2021 that has been cited dozens of times by special issues of Hindawi journals, he cites himself four times, yet once where it is clearly irrelevant. He serves as lead guest editor of special issues of Security and Communication Networks and Journal of Healthcare Engineering, but so far no papers have been published in these issues. 

This cluster also includes Osamah Ibrahim Khalaf (Al-Nahrain University, Iraq), Rahman Ali (University of Peshawar, Pakistan), and Shahid Mumtaz (Instituto de Telecomunicações, Portugal), all of whom are guest editors at Hindawi. It is interesting to note that most of the citations they receive do not come from the special issues for which they are guest editors. 

In this cluster, some articles (like Chu 2022) were marked for publication in a special issue that have been deleted for some reason, without any explanation from Hindawi. 

Cluster Editor Li 

Elsewhere, no single author seems to be cited in a large number of papers. Some papers by Hanlie Cheng (affiliated with COSL-EXPRO Testing Services (Tianjin) Co., Ltd. in China) and Malik Bader Alazzam (a guest editor affiliated with Amman Arab University in Jordan) were cited dozens of times as irrelevant references, e.g. by Zhen 2022 and Shdefat et al 2022. There is also a mystery as to why a Mody & Bhoosreddy 1995 case report on multiple odontogenic keratocysts was cited multiple times. This case report on oral surgery was not only used to support research on preschool education or martial arts virtual reality, some authors discussing data warehouse technology even added abrupt sentences in order to cite it. It seems difficult to explain why a papermill would use that old case report. I subsequently found that it was always co-cited with different papers published in Science Progress and Research (SPR), a journal by a British company UK Zhende Publishing that has not yet established its reputation. I noticed that on the editorial board, a certain managing editor named Zihan Li was the only person with a Han Chinese name. She is one of directors of UK Zhende Publishing (together with two other Chinese nationals named Li), as well as managing editor of Journal of Recent Trends in Cancer Research (another journal owned by Zhende Publishing), and on the editorial boards of other shady journals: Global Journal of Civil and Environmental Engineering, International Journal of Research in Engineering and Innovation (IJREI), and Systematic reviews in pharmacy (SysRP). Finalyl, she is also working with some Dubai-based scam called “International Scientific Indexing” where she uses a false affiliation of “Environmetal (sic!) Science University of Glasgow, UK“.

All these journals are recorded in citation databases under a variety of confusing titles, this is why finding papers that cite these journals is not easy. Rather, the output of these obscure journals is found as irrelevant references in other journals. As Hoya camphorifolia mentioned for Hindawi’s Computational Intelligence and Neuroscience: 

“An obscure new “Science Progress & Research” journal seems to be receiving intensive citational promotion from papers in Hindawi journals, including this one. It wasn’t necessary for these citations to be in any way relevant.” 

In addition, there are some papers published in Soft Computing (e.g., Su 2022), Ethiopian Journal of Health Development and Frontiers in Public Health which cited papers published in SPR as irrelevant references (Ren 2022, Zhao 2022 and Yaling Li 2022, respectively).  

Cluster Marxism and Cluster Simulation

The publication of manuscripts on the ideological and political education of the Chinese Communist Party was in itself proof of madness, but I found even stronger evidence that editorial oversight in some Hindawi journals does not exist at all, and that guest editors can do whatever they want.  

The only relief is that contrary to what the authors claim, most of the papers in this Cluster Marxism do not actually deal with ideological and political education or Marxism. They are just some “AI Gobbledegook”, as Prof. Dorothy Bishop put it. Of course, true Marxism looks worse. 

Cluster Simulation is characterized by strange data availability statements, like:

“The simulation experiment data used to support the findings of this study are available from the corresponding author upon request.”

Obviously the data on offer is made up, as the authors freely admit. Nick Wise commented on PubPeer: 

Retraction and Retreat 

Beginning November 16, 2022 (the International Day for Tolerance), Hindawi retractions began to appear, which were previously promised by Wiley’s Senior Vice President Liz Ferguson. By the time when over 200 retraction statements were published, Clarivate announced on the Web of Science Master Journal List that they would re-evaluate whether six Hindawi journals meet their journal selection criteria. Strangely, most retracted publications were not commented on PubPeer or other post-publication peer review platform, except for a few cases. Additional, I included in the public spreadsheet many papers whose topics did not match the scope of the special issue in the absence of PubPeer comments, but these rarely ended up retracted.

It seemed, Wiley was determined to right the wrongs which took place at Hindawi after the former publisher bought the latter, while publicly advocating for editorial ethics. In May 2022, Hindawi held a webinar on the topic of “What makes an exceptional peer review?”. In September, they discussed the “Foundations of peer review”. In November, following the release of Ferguson’s statement, Hindawi hosted a webinar titled “Becoming a Journal Editor”, inviting researchers to become guest editors. 

But if they are so concerned, why didn’t Hindawi retract all those papers we exposed? Like this alleged clinical study of pregnant women, found because it referenced Huang:

Ying Liu , Xiaolin Hou , Mei Yu , Jin Zhou Clinical Analysis of Echocardiography and Serum IL-6 and TNF-α Changes in Pregnant Women with Hypertension Scanning (2022) doi: 10.1155/2022/9299746

Hoya camphorifolia commented on PubPeer:

“It comes as a shock to discover that the authors dissected their pregnant-women participants.

“Blood samples for TNF-α and IL-6 were collected from the heart and stored at -80° C.“

“(1) Right Ventricular Systolic Blood Pressure Detection. Pentohexidine was administered intravenously (30 mg/kg) was injected intravenously, and the trachea was intubated and placed in a well-ventilated environment. The thoracic cavity was opened, a PE-50 polyethylene tube was inserted into the right ventricle, and the right ventricular systolic pressure was collected by energy-Lab data collection.”

“(2) Extraction of Total RNA. Use TRIzol to extract total RNA from lung tissue, and the specific operation steps are as follows:“

Based on an emotional distrust, I decided to explore whether Hindawi was covering up something. 

Publisher-level manipulation of the publishing process 

Hindawi issued a batch of retraction statements using the same template in the following style:

“Applied Bionics and Biomechanics has retracted the article titled “Event Scene Method of Legal Domain Knowledge Map Based on Neural Network Hybrid Model” [1] due to concerns that the peer review process has been compromised. 

Following an investigation conducted by the Hindawi Research Integrity team [2], significant concerns were identified with the peer reviewers assigned to this article; the investigation has concluded that the peer review process was compromised. We therefore can no longer trust the peer review process and the article is being retracted with the agreement of the Chief Editor.” 

In contrast, I prefer Springer Nature’s template for such retractions:

“The Editor-in-Chief and the publisher have retracted this article. The article was submitted to be part of a guest-edited issue. An investigation by the publisher found a number of articles, including this one, with a number of concerns, including but not limited to compromised editorial handling and peer review process, inappropriate or irrelevant references or not being in scope of the journal or guest-edited issue. Based on the investigation’s findings the Editor-in-Chief therefore no longer has confidence in the results and conclusions of this article.”

"The Editor-in-Chief and the publisher have retracted this article. The article was submitted to be part of a guest-edited issue." https://t.co/vedHBS8xVx@springernature ramping up the retractions from corrupt special issues in 2022. Suspect there will be more in 2023. pic.twitter.com/qDLLAjZu7J

— Nick Wise (@nickwizzo) January 2, 2023

Hindawi’s story is that in specific journals, specific special issues, specific evil guest editors took advantage of negligent editorial oversight and assigned specific questionable reviewers, thereby manipulating the peer review of specific manuscripts. By December 31, 2022, Hindawi had issued 306 retractions, and only 31 retracted papers fell into the five clusters I described above (1 in Cluster Huang, 23 in Cluster Zhang, 5 in Cluster Editor Li, 2 in Cluster Marxism, and 1 in Cluster Simulation). I propose a hypothesis that different clusters are published due to different manipulations of the publishing process.

Here is what I noticed:

• Papers from a cluster can be published in the same journal via different guest editors. 

For example, Cluster Huang appears in the Journal of Environmental and Public Health with Kaifa Zhao and with Fu-Sheng Tsai as guest editors. 

• Papers from a cluster can be published with the same guest editor in different journals. 

For example, Cluster Huang with K. K. Aruna as a guest editor in Wireless Communications and Mobile Computing and in Journal of Chemistry. 

• A guest editor can work with different clusters in the same special issue. 

For example, N. Rajesh is the guest editor of special issue Ubiquitous Artificial Intelligence Enabled Next Generation Wireless Systems in Computational Intelligence and Neuroscience, where we find Cluster Zhang and Cluster Editor Li papers. 

I think it’s a systematic manipulation of the publishing process. Hindawi provides a platform where guest editors can receive manuscripts according to their preferences and are free to work with papermills. Retracting more papers with retraction notices blaming solely to a few bad guest editors, and even then only those that totally don’t match the topic of the special issue, is probably meant to mislead people into thinking that there is no systematic manipulation. To be honest, I never expected Hindawi to issue 6,000 retractions, as it would necessarily go against their business interests. But I believe that recognition of a systemic manipulation is a necessary step toward reconciliation. 

Citation stacking? 

The other problem Hindawi Research Integrity team should be addressing is citation stacking. For some reason, Hindawi does not want their special issues to be special and hides this information on papers’ origin from databases and even the Hindawi XML corpus.

The first concern is publisher self-citation. The 5,346 papers in the spreadsheet cited 126 Hindawi journals 4,655 times, with the top ten journals were cited at least 3,334 times. There are 78 Hindawi journals received at least 1,906 citations, with the top ten receiving at least 1,554 citations from 1,694 papers in the five clusters. However, there was no evidence to suggest what the expected number of publisher self-citations should be.

Even more worrisome are the special issue-to-special issue citations. Yajun Chen’s paper on English teaching evaluation, published in special issue of Journal of Sensors in 2021 needed to be cited in 2022 and 2023 to raise the Journal Impact Factor. It was cited 18 times in 2022, 15 of which were from special issues of Hindawi journals, all with negative comments in PubPeer. The reason for Chen, an author of three papers in special issues of Hindawi Journals, to get these citations could be that he is a repeat customer. He also published two articles on Overseas English and Shanxi Youth during 2021 to 2022. I believe he will soon receive a promotion. 

The second concern is citations to three journals in Clarivate’s Emerging Sources Citation Index (ESCI). They are Nonlinear Engineering (ISSN 2192-8010), IET Collaborative Intelligent Manufacturing (ISSN 2516-8398), and International Journal of Emerging Technologies in Learning (ISSN 1863-0383). They received at least 434, 316, and 277 citations, respectively, from the records in the spreadsheet. Papers in the five clusters contributed at least 422, 309 and 112 citations, respectively. 

According to Clarivate, ESCI “contains quality publications, selected by our expert in-house editors for editorial rigor and best practice at a journal level.” Concretely, ESCI indexes many journals that have a compliant publication process but are published in small numbers and have few citations. ESCI is considered a wait-list for Clarivate’s advanced indexes, and hundreds of citations are already enough to improve a journal’s ranking and give them a better chance of being indexed by SCIE (Science Citation Index Expanded), SSCI (Social Sciences Citation Index) or AHCI (Arts & Humanities Citation Index). 

Unfortunately, these are the three journals that have been found to be problematic. De Gruyter’s Nonlinear Engineering surrendered its publishing process in regular issues (see Lv &Yu 2022) to the same papermill as some special issues of Hindawi journals did. IET Collaborative Intelligent Manufacturing, a journal launched by Wiley in collaboration with Huazhong University of Science and Technology in 2019, invited Pradeep Kumar Singh to edit a special issue in 2021, and it was this special issue that published two of Huang’s highly cited articles. This special issue is special because unlike the journals by Hindawi, Frontiers and MDPI, which can publish special issues on demand, the Chinese government only allows journals registered in China to publish two (any kind of) supplemental issues per year. Singh edited another even more gibberish-full special issue for Hindawi in 2022. International Journal of Emerging Technologies in Learning has at least acknowledged that their publishing process was being disrupted by paper mills during 2020 to 2021. By December 31, 2022, it seems that none of the papers in the spreadsheet that cited the garbage from Nonlinear Engineering or IET Collaborative Intelligent Manufacturing were retracted by Hindawi.

Spillover 

Smut Clyde, Alexander Magazinov, Rhipidura albiventris and others have been for some time reporting many articles published in other journals that have patterns and absurdities similar to those published in special issues of Hindawi journals. I found in many journals some kind of chimeras on the topic of ideological and political education, some published in special issues and some not. These papers appeared in Journal of Interconnection Networks and Journal of Information & Knowledge Management by the predatory publisher World Scientific, Ingenta’s Tobacco Regulatory Science, Springer Nature’s Progress in Artificial Intelligence, Soft Computing and Wireless Networks, Elsevier’s Computers and Electrical Engineering, IGI Global’s International Journal of Distributed Systems and Technologies, IOS Press’ Journal of Intelligent & Fuzzy Systems, Sciendo’s Applied Mathematics and Nonlinear Sciences, Frontiers’ Frontiers in Psychology and Frontiers in Public Health, and Fresenius Environmental Bulletin, the latter owned and published by Harun Parlar, a professor at the TU Munich in Germany. A landfill like Fresenius Environmental Bulletin could be both the testing ground and the ultimate destiny for products from papermills. 

For example, thanks to Smut Clyde we know that from 2016 to 2022, Latin American Journal of Pharmacy has been publishing research on the medical applications of coordination polymers. The journal was still receiving such manuscripts from a papermill until August of 2022. 

If the papermills’ business continues to grow, we will see their products in other journals, like we did with the prolific coordination-polymer papermill. The manipulation of the publishing process has been in place at Hindawi for at least three years, and we don’t know if it really stopped. In any case, there is no doubt that the paper mills will move to new platforms.  

---