C Ronald Kahn and The Problem of Irreproducible Bioscience Research

Jeffrey Flier, endocrinology professor and former Dean of the Harvard Medical School, is very interested in scientific ethics. He has just published yet another analysis on the topic of research integrity and irreproducibility in biomedicine.

Problem is that Flier’s fellow Harvard professor, the mighty giant of diabetes research in USA, but also his closest friend and neighbour of almost five decades, Carl Ronald Kahn, is the proverbial biblical beam in Flier’s own eye. I tried to talk to Flier about Kahn before, but it only made Flier angry at me. So I now decided to illustrate Flier’s recent opinion piece with the trash science of his dear friend.

My paper, "The Problem of Irreproducible Bioscience Research" is now live & open access. I examine meaning of the term, its prevalence, causal factors and potential interventions – all based on a career of observing & dealing with this issue. Do read it. https://t.co/fdbqKpvcxq

— Jeffrey Flier (@jflier) August 23, 2022

Flier’s new paper is titled “The Problem of Irreproducible Bioscience Research“, and his friend Kahn has 3 retractions and a bunch of corrections, issued in 2016-2017. Which suggests there must have been an institutional investigation at Harvard and probably also at NIH which probably determined everything to be innocent mistakes, conclusions unaffected, whistleblowers swiftly silenced and dealt with most ruthlessly.

I wrote about Kahn’s bad science before, back then with the focus on his Spanish collaborators. Kahn’s papers with Manuel Benito and Angela Valverde are ridiculously fraudulent, but nobody bothered so far to retract those. Certainly not Kahn, who, even when he was forced to retract some papers, still insisted that all conclusions remained unaffected and all results totally reproducible.

Flier and Kahn are best friends for already five decades, in 2019 Flier wrote a celebratory article in the Journal of Clinical Investigation about Kahn’s family history and scientific achievements, with many family photos of their time together. We learn about Kahn that:

“One of his legacies was outstanding mentorship of women scientists, one of whom was my wife, Terry Maratos-Flier”

In that article, Flier shares this photo, which I think is the perfect illustration for the following story:

Flier was of course joking. Kahn was born in 1944, even Harvard professors can’t time-travel. The porn-bar moustachioed Kahn was crudely pasted into an archive picture of Banting and Best, a hack-job of bad Photoshop. He even lacks the right foot and the left lower leg. But then again: younger generations of scientists might not look closely, also not at the dates, they will see the picture only on their small smartphone screen, and take home the message that the great Kahn indeed discovered insulin all these many years ago. Like with Kahn’s falsified research figures, it’s enough to briefly fool the editors and reviewers, and if anyone wants to complain later, they won’t be heard.

Maybe it was an insider joke among buddies, of the kind: Ron, I know how you achieved your scientific success, and I not only don’t mind, I think it’s hilarious even. Let’s all have a laugh at these failed scientists losers who never had your chutzpah.

While researching my presentation of the AAP Kober Medal to Professor C. Ronald Kahn, I discovered this long lost 1921 photo in the archives of the Banting and Best Institute in Toronto. Will now have to rethink our understanding of the discovery of insulin. pic.twitter.com/Uv3y2YxbTt

— Jeffrey Flier (@jflier) April 8, 2019

Maybe these data fudgery antics of his dear friend of almost 50 years is what drove Flier to use his status to appoint himself as the top authority on research reproducibility? Is this another practical joke of Flier’s, or is he trying to control the debate so his friends like Kahn won’t suffer too much exposure and shame?

Here is the abstract of Flier’s recent paper:

“Over recent decades, progress in bioscience research has been remarkable, but alongside the many transformative advances is a growing concern that a surprisingly high fraction of published research cannot be reproduced by the scientific community. Though experimental and interpretive errors are unavoidable features of the scientific process, recent evidence suggests that irreproducibility is a serious issue requiring analysis, understanding, and remediation. This article reviews the meaning of research reproducibility, examines ongoing efforts to estimate its prevalence, and considers the factors that contribute to it. Two recent case studies illustrate the disparate responses that researchers may take when facing serious claims that a high-profile research finding is irreproducible and may be false. Finally, the article examines potential interventions to counter the current level of irreproducibility, aimed at increasing the efficiency and impact of society’s substantial and critically important investment in bioscience research.”

Happy New Year Twitter friends! Since 1981 been cooking NY Eve dinner with my buddy Ron Kahn. This year we’re joined by our friend, former mentor and former NIDDK director Phil Gorden. Celebrating halcyon days at NIH. All day cooking. Menu later. pic.twitter.com/uoEakh3Bxu

— Jeffrey Flier (@jflier) December 31, 2018

Let us begin then.

“Do scientists generally believe the research published by other scientists to be true? Since the scientific method is rooted in rationality, objectivity, and the pursuit of truth, and progress requires building on the work of others, one might expect they do. […] Early in my career, in the 1970s, I believed most research papers I read generally reflected reality, and if I scrupulously followed the authors’ published methods, I’d most [End Page 374] often be able to reproduce their most important results, at least to a substantial degree, sufficient to convince me that they were more right than wrong. Of course, I knew scientists were fallible, that researchers sometimes made errors requiring correction, and that some scientists were more skilled, judicious, and honest than others. As one of my mentors used to say: “In God I trust. Everyone else show me data.””

J. Flier

Here is Ron’s data, have a look, Jeff. Do YOU trust it?

Kohjiro Ueki , Tatsuya Kondo , C. Ronald Kahn Suppressor of cytokine signaling 1 (SOCS-1) and SOCS-3 cause insulin resistance through inhibition of tyrosine phosphorylation of insulin receptor substrate proteins by discrete mechanisms Molecular and Cellular Biology (2004) doi: 10.1128/mcb.24.12.5434-5446.2004

The first author and Kahn’s former postdoc Kohjiro Ueki became power-wise basically his mentor’s counterpart in Japan, being a director of the national diabetes centre and the Chair of the Board of Directors of the Japan Diabetes Society.

Link to a truly inspiring lecture on diabetes, insulin and insulin resistance by Ron Kahn, towering figure in the field, & friend for 48 years. Starts w history, then gets technical. One of the very best medical science lectures I have heard. https://t.co/vZR6udbLwe https://t.co/D2r19F4Z0A

— Jeffrey Flier (@jflier) April 1, 2022

“Because we know that science is inherently subject to error (both false positives—type 1 errors—that receive the greatest attention, as well as false negatives—type 2 errors), we must consider what level of reproducibility might be optimal for scientific progress as a whole, or for specific disciplines where this might vary. It seems intuitively likely that an “excessive” demand for reproducibility (and punishment for its absence) would slow scientific progress and deter discovery and publication of true and useful results. This negative outcome would need to be balanced against the opposite type of negative outcome: wherein untrue claims induce unproductive and ultimately wasteful efforts to repeat or build upon them.”

J. Flier

Here is a couple of of Kahn’s papers, never even corrected, are those type 1 or type 2 errors which led to the untrue claims therein?

Yu-Hua Tseng , Kristina M. Kriauciunas , Efi Kokkotou , C. Ronald Kahn Differential roles of insulin receptor substrates in brown adipocyte differentiation Molecular and Cellular Biology (2004) doi: 10.1128/mcb.24.5.1918-1929.2004

Yu-Hua Tseng, Atul J. Butte , Efi Kokkotou , Vijay K. Yechoor , Cullen M. Taniguchi , Kristina M. Kriauciunas, Aaron M. Cypess, Michio Niinobe , Kazuaki Yoshikawa , Mary Elizabeth Patti, C. Ronald Kahn Prediction of preadipocyte differentiation by gene expression reveals role of insulin receptor substrates and necdin Nature Cell Biology (2005) doi: 10.1038/ncb1259 

“Some claims of discovery receive special scrutiny from the scientific community, however. Consider a finding that purports to have major implications for a field, is carried out by a highly respected scientist, and is published in an elite, selective journal. Even more, consider one that is picked up by the popular press and social media, which often hype its potential health implications. Findings like this are likely important to other scientists, whose research plans may be altered if the “transformative” new observations are indeed true. Such claims attract great scrutiny, as interested labs race to assess their truth.”

J. Flier

Here a paper by Kahn and his former mentees – postdoc Ueki and a certain PhD student Cullen Taniguchi (now professor at MD Anderson, where else), which did attract great scrutiny, but to no avail:

Cullen M. Taniguchi , José O Aleman , Kohjiro Ueki , Ji Luo , Tomoichiro Asano , Hideaki Kaneto , Gregory Stephanopoulos , Lewis C. Cantley, C. Ronald Kahn The p85alpha regulatory subunit of phosphoinositide 3-kinase potentiates c-Jun N-terminal kinase-mediated insulin resistance Molecular and Cellular Biology (2007) doi: 10.1128/mcb.00079-07 

“While many such findings are reproducible, a surprising number are not. And when a widely reported and celebrated finding is not confirmed or is suggested to be false, the consequences can have substantial impact on the field. This is true when the disconfirmed finding is eventually formally retracted, but more so if the finding remains uncorrected in the literature, as is often the case, causing confusion and slowing or derailing progress in the field (Tatsioni, Bonitsis, and Ioannidis 2007).”

J. Flier

Because some journals do care (or in this case, used to care), Kahn and Taniguchi previously retracted a paper, which I discussed here:

It was this diabetes breakthrough research outrageous fraud, retracted in 2016:

Amelia J. Entingh , Cullen M. Taniguchi , C. Ronald Kahn Bi-directional regulation of brown fat adipogenesis by the insulin receptor The Journal of biological chemistry (2003) doi: 10.1074/jbc.m303056200

“It would be nice if the problems with reproducibility had a simple, unifying explanation, which would facilitate implementation of effective remedies. Unfortunately, the causes are complex (Flier 2017a, 2017b; NASEM 2019). As I became interested in this problem, the more I sought explanations for irreproducibility, the more I found them. These include inadequate training, oversight and mentorship; deficiencies in experimental design and reagents; and perhaps as an important root cause, misaligned incentives from our approach to funding, publishing, and recognizing research accomplishments. Each of these is real and important and requires assessment, though their interaction within the prevailing [End Page 380] institutional ecosystem and culture of research complicates the design and implementation of potential remedies.”

J. Flier

Here another retracted Kahn paper, I wonder if Flier would explain it with Kahn’s inadequate training, oversight and mentorship of Ueki? Or with deficiencies in experimental design and reagents?

Kohjiro Ueki , David A. Fruman , Claudine M. Yballe , Mathias Fasshauer , Johannes Klein , Tomoichiro Asano , Lewis C. Cantley, C. Ronald Kahn Positive and negative roles of p85 alpha and p85 beta regulatory subunits of phosphoinositide 3-kinase in insulin signaling The Journal of biological chemistry (2003) doi: 10.1074/jbc.m305602200

You may have noticed that also that paper was co-authored by Lew Cantley, formerly at Harvard, now at Weill Cornell, another problematic but very influential scientist. I mentioned Cantley’s problematic research here:

“Another key methodological issue is observer bias. Whether aware of it or not (most certainly are), scientists prefer their experimental hypotheses to be true, especially when their grant proposals are justified by claims that they are. Since there are numerous ways in which confirmation bias can inappropriately influence experimental results, experimenters should wherever possible be blinded to experimental details, such as the identity of treated vs. untreated groups—whether these be molecules, cells, animals, or people.”

J. Flier

More conformation bias from Kahn, Cantley and Tanaguchi, in a Cell Press journal founded by Kahn’s dear friend Flier:

Cullen M. Taniguchi , Tatsuya Kondo , Mini Sajan , Ji Luo , Roderick Bronson , Tomoichiro Asano , Robert Farese , Lewis C. Cantley, C. Ronald Kahn Divergent regulation of hepatic glucose and lipid metabolism by phosphoinositide 3-kinase via Akt and PKClambda/zeta Cell Metabolism (2006) doi: 10.1016/j.cmet.2006.04.005 

I was (am) a founding editor of @Cell_Metabolism. My colleagues Ron Kahn and Bruce Spiegelman and I pitched the idea of a high end metabolism journal to Science, Nature and Cell Press in 2004, and only @CellPressNews was interested. The rest is history! https://t.co/NTS7PzzMPV

— Jeffrey Flier (@jflier) September 23, 2021

Oh dear, Flier is also friends with Bruce Spiegelman (another bigwig of diabetes research, also at Harvard). So I guess that irisin fraud is also perfectly reproducible? Read here:

“Another cause of irreproducibility are research reagents that are unreliable or otherwise problematic. Many reagents are employed in lab-based research, ranging from chemicals, solutions, nucleic acid derivatives, diverse proteins and antibodies, and cell lines, to stocks of “model organisms” ranging from fruit flies to mice. Some are created by scientists to enable their own research, but many are obtained from other scientists or purchased from commercial suppliers. Needless to say, the reproducibility and reliability of research is heavily dependent on the quality of these reagents, and there are many instances where they are insufficient to the task (Bradbury and Pluckthun 2015).”

J. Flier

Here is Kahn’s third retraction, must have been faulty reagents, what else, I mean do you have any other clues?

Cullen M. Taniguchi , Kohjiro Ueki , Ronald Kahn Complementary roles of IRS-1 and IRS-2 in the hepatic regulation of metabolism Journal of Clinical Investigation (2005) doi: 10.1172/jci23187 

“Mice are a staple of modern bioscience research. Most lines of genetically modified mice are created by scientists and eventually provided as commodities by commercial purveyors to whom they are transferred. There are hundreds of lines of genetically modified mice in which specific genes are deleted, overexpressed, or modified to test innumerable biological hypotheses in the living organism. It was initially a surprise when different labs conducting experiments on the “same” lines of mice observed different findings. This may be due to misidentifying the mice, exposure to different diets, or different house environments with respect to ambient temperature, noise levels, or other factors not [End Page 382] previously thought to be highly consequential (Bailoo et al. 2020).”

J. Flier

Maybe the mice are indeed the real problem. Yes, let’s blame those. Furry buggers, sneak into the labs of Flier’s friends at night to nag and damage perfectly reliable datasets. They must have caused this Franken-Blot which Kahn then “contributed” to PNAS:

Brice Emanuelli , Delphine Eberlé , Ryo Suzuki , C. Ronald Kahn Overexpression of the dual-specificity phosphatase MKP-4/DUSP-9 protects against stress-induced insulin resistance Proceedings of the National Academy of Sciences of the United States of America (2008) doi: 10.1073/pnas.0712275105

“In addition to issues of training, experimental design, statistics, and reagents, another major cause of irreproducibility results from scientists’ unfortunate responses to incentives and disincentives prevalent in todays’ research environment (Flier 2019; Nosek, Spies, and Motyl 2012). When scientists face highly consequential grant deadlines, publication decisions, and impending promotions, some forgo best practices. Instead, they cut corners when selecting data to “present a story” that in the end varies from the truth and will not be reproduced by others. The most selective journals frequently require results to be packaged as excessively tidy stories that claim to lack ambiguity or uncertainty, often promising to transform the field and lead to therapeutic outcomes. Despite many insightful and potentially important findings, most papers don’t deliver these outcomes.”

J. Flier

Indeed. Poor scientists in Kahn’s lab, cut some corners while being harassed by deadlines and impending promotions, forced to publish in the most selective journals.

Jonathon N Winnay, Jeremie Boucher , Marcelo A Mori , Kohjiro Ueki , C Ronald Kahn A regulatory subunit of phosphoinositide 3-kinase increases the nuclear accumulation of X-box–binding protein-1 to modulate the unfolded protein response Nature Medicine (2010) doi: 10.1038/nm.2121

“I believe many decisions to select and present data in questionable ways are made innocently, the scientists believing that they really do capture the underlying reality. But this isn’t always the case, and considerable effort has gone into distinguishing what are referred to as “questionable research practices” from research misconduct, the latter formally composed of plagiarism, fabrication, and falsification (Steneck 2006). Misconduct as formally defined accounts for a small fraction of irreproducibility, but the prevalence of both misconduct and questionable research practices are too high (de Vrieze 2021; Fanelli 2009). Here is the key moral question: at what point do sloppiness, wishful thinking, and morally innocent (if inappropriate) selectivity in data presentation transition into a more serious realm, falsification—an accepted hallmark of research misconduct—which, if judged to be present, may end a scientific career?”

J. Flier

Unlike for all these male cheaters Kahn innocently trained in questionable practices, Amelia Entigh‘s scientific career did indeed end. Which in her case is certainly not a loss, but still, fact remains that research fraud is sanctioned gender-specifically.

Amelia Entingh-Pearsall , C. Ronald Kahn Differential roles of the insulin and insulin-like growth factor-I (IGF-I) receptors in response to insulin and IGF-I The Journal of biological chemistry (2004) doi: 10.1074/jbc.m313201200

“In order to address these problems, we need to require better training and mentoring on best practices, in areas ranging from experimental design and statistics to the use of reagents to developing manuscripts and negotiating their acceptance. The National Institutes of Health (NIH) have taken note of this problem, and the NIH now requires students and postdocs they fund to take courses on the “responsible conduct of research”; most institutions comply by developing and running live or online modules (NIH 2020). Many trainees fail to complete these modules, and it’s unclear whether those who do acquire the necessary skills. Courses and lectures vary in quality and can only achieve so much.”

J. Flier

But who will train Flier’s dear friend Kahn on those best practices? Maybe one former dean can give a private course during the next joint New Year Eve’s dinner?

Franck Mauvais-Jarvis , Kohjiro Ueki , David A. Fruman , Michael F. Hirshman , Kei Sakamoto , Laurie J. Goodyear , Matteo Iannacone , Domenico Accili , Lewis C. Cantley , C. Ronald Kahn Reduced expression of the murine p85alpha subunit of phosphoinositide 3-kinase improves insulin signaling and ameliorates diabetes Journal of Clinical Investigation (2002) doi: 10.1172/jci13305

“In fact, the greatest influence on trainee behavior almost certainly derives not from formal curricula, but from daily observations of colleagues, mentors, and the broader community in which they work. This “hidden curriculum” is far more impactful in shaping behavior than exposure to slides listing dos and don’ts. Unfortunately, many mentors are themselves poorly trained in these areas. Few take the opportunity to enhance these skills as their careers progress, and it seems no one is responsible for asking them to do so.”

J. Flier

Poor poorly trained Kahn….

Zhen Y. Jiang , Zhiheng He , Benjamin L. King , Tatsuya Kuroki , Darren M. Opland , Kiyoshi Suzuma , Izumi Suzuma , Kohjiro Ueki , Rohit N. Kulkarni , C. Ronald Kahn , George L. King Characterization of multiple signaling pathways of insulin in the regulation of vascular endothelial growth factor expression in vascular cells and angiogenesis The Journal of biological chemistry (2003) doi: 10.1074/jbc.m303314200

“The impact on trainees of observing and adopting their mentors’ questionable approaches was assessed by Smaldino and McElreath in a 2016 paper entitled “The Natural Selection of Bad Science.” Their thesis goes like this: poor experimental design and data analysis promote false positive findings, publication of which is incentivized by the requirements of publishing for career advancement, and many errors in that published work are never identified. Too many scientists succeed despite (periodically) publishing irreproducible science. Successful labs produce more researchers, and some of them will mimic their mentors’ methods when running their own labs. And so, this undesirable trait is propagated through a form of adverse social evolution (Grimes, Bauch, and Ioannidis 2018).”

J. Flier

Social evolution of Kahn’s undesirable traits, propagated to Ueki and Taniguchi, right? Who probably are propagating it to their own mentees as you read this?

Cullen M. Taniguchi , Thien T. Tran , Tatsuya Kondo , Ji Luo , Kohjiro Ueki , Lewis C. Cantley , C. Ronald Kahn Phosphoinositide 3-kinase regulatory subunit p85alpha suppresses insulin action via positive regulation of PTEN Proceedings of the National Academy of Sciences of the United States of America (2006) doi: 10.1073/pnas.0604628103 

Somewhere in his article, Flier discussed statistics:

“Unfortunately, too many scientists have a rudimentary command of statistics, often employing it robotically to claim that findings are “statistically significant” through so-called “t-tests,” hoping for values of p<.05, thought to indicate that results are unlikely to be due to chance alone. Statistics are often misapplied and misinterpreted, despite statistical input being required at every stage of the process, from initial design to interpretation. “

J. Flier

Well, after the presumable investigation in 2016, Kahn sure learned his lesson. No more cloned gel bands, no sir. But cloned numbers, that’s a different story.

Thomas Thomou , Marcelo A. Mori , Jonathan M. Dreyfuss , Masahiro Konishi , Masaji Sakaguchi , Christian Wolfrum, Tata Nageswara Rao , Jonathon N. Winnay , Ruben Garcia-Martin , Steven K. Grinspoon , Phillip Gorden , C. Ronald Kahn Adipose-derived circulating miRNAs regulate gene expression in other tissues Nature (2017) doi: 10.1038/nature21365

At one point Flier even seems to talk about me!

“Another concern, which I share, is that public attention to irreproducible research could be weaponized by anti-science activists to reduce funding and tarnish the reputation of the research community. This is one reason why many academic leaders are hesitant to take up this issue, preferring (not surprisingly) to highlight the quality and impact of the research at their institutions, which they surely believe, rather than alerting others, including funders, donors, and the general public, of systemic flaws in the ecosystem. But I believe that failing to identify and take responsibility for this problem poses greater risks. In fact, failing to recognize a quite evident problem provides ammunition for zealots to pursue their irrational anti-science agenda.”

Yes, it’s anti-science zealots with an agenda who are the real problem, not scientists like Kahn. Some of these zealots post such things on PubPeer and others write stories like this one.

Flier ends his treatise with:

“The crisis of reproducibility that we face today involves errors that far too often go unacknowledged and uncorrected. Too many scientists and academic institutions sidestep questions about reproducibility rather than address them head on. When other scientists mistakenly accept their validity during their own search for the truth, these errors generate wasted effort. The rate of scientific progress is slowed, and, not surprisingly, mistrust—sometimes justified and sometimes not—of the scientific enterprise grows. To reverse this unfortunate assault on the integrity of biomedical science and to enhance the efficiency of our incredible scientific ecosystem, the culture of irreproducibility must be acknowledged and addressed by participants at every level of the enterprise.”

Obviously this crisis of reproducibility has nothing to do with his friend Kahn.

My buddies Ron Kahn and Dick Eastman and I walked the Halls of NIH in 70's as "yellow berets", doing research and patient care. 40+ yrs later, shared great memories, food and guitar music – as good as it gets. pic.twitter.com/l5ZnuHR7Ji

— Jeffrey Flier (@jflier) July 16, 2022

PS: a reader just told me that Kahn also trained Mario Saad and Licio Velloso. It’s true, the Brazilian fraudster duo was in Kahn’s lab forging data long before Photoshop. What a brood Kahn spawned in those decades of bad science, each of these crooks a professor somewhere, training the next generation of fraudsters.

---