Human Brain Project: bureaucratic success despite scientific failure

The EU €1-Billion-Flagship Human Brain Project (HBP) has passed its midterm evaluation with flying colours. Noone knows exactly what the objectives of this bombastic project is, as members of the evaluation panel indicated to me, while others refused to answer this question. The HBP leadership sure keeps the exact definition of these objectives secret, or maybe they don’t know them themselves. Which is easy to understand, because given the leniency HBP keeps receiving from those supposed to evaluate it, its real objective becomes perfectly clear: to secure the public funding. There, HBP succeeded indeed, the €1 Billion seems rather safe. It is none of the public’s business where the money will go, but it can rest assured it will certainly go somewhere. The public should also not expect any deliverables or return on its research investment, this the HBP leadership already made perfectly clear. I am showing below what a farce the recent HBP evaluations were, while the positive outcome was much hailed as evidence for excellent scientific performance. 

We published in Neuron!

The HBP Flagship will go down in history by demonstrating the EU’s capability to give a mind-boggling amount of money to a bunch of engineers and computational neuroscientists without any plan or even pretence of a concept (see my earlier report here). The idea of simulating the brain and its diseases in a supercomputer is dropped under the table whenever any serious experts start asking questions, and is brought forth again when politicians, funders or consortium members themselves are to be duped.

To make it all even more confusing, the original Brain Simulation objective became just one of 12 sub-platforms, probably because the task of making a “brain-in-a-box” was deemed as too trivial, so HBP sought another 11 challenges. What exactly any one of these branches has achieved scientifically so far, is none of our business. The official goal of HBP, as described in the much trumpeted paper by Amunts et al 2016, published not somewhere, but “in the prestigious journal Neuron” of Cell Press (impressed, aren’t you? When did you last publish in Neuron? See?)  is:

“The HBP is developing toward a European research infrastructure advancing brain research, medicine, and brain-inspired information technology”.

This infrastructure will apparently primarily focus on securing its EU funding, while pursuing a rather generally described goal: “ to decode the human brain”.  What does this “decode” mean? Don’t ask.

Simulants at work

That a brain is not a giant supercomputer, and that as a biological system inseparably integrated into a living organism it operates utterly differently from our digital computers, is seen inside the HBP cult as a deranged heresy of incompetent looneys who rightfully shall never get any proper funding. Regardless of HBP position, brains do not compute in a way we understand computing, just as hearts and livers do not compute. No human computer ever simulated the neural system of a snail, and it most likely never will, but so what? Who cares about stupid snails? To HBP, even a mouse brain is just a small supercomputer (named Blue Brain, allegedly as good as up and running in its virtual cage in the Swiss federal university EPFL, the digital pet’s owner is the dethroned HBP emperor Henry Markram). You only need to simulate a small bunch of neurons in a computer (or pretend that you did), upscale it million-fold, and there you have a functioning in silico brain of a rodent. Upscale it even higher, using bigger supercomputers, and it will suddenly develop consciousness of a human (though mice and other animals have their kinds of consciousness as well, but HBP simply decreed they don’t).

In this HBP cult thinking, a monkey’s brain is a slightly bigger supercomputer than that of a rodent (no consciousness there, obviously), and the mighty human brain is the biggest one there is and costs exactly €1 Billion to build (or, in newspeak, to “decode”). Or maybe more, and it will be all politicians’ fault if HBP never delivers, because they failed to cough up another half a billion or so when they will be asked for it in around 2020.

Just like the HBP cult members, the gullible EU politicians seem to follow the notion that in order to “decode” and simulate a human brain, you only need to give enough money to some very self-important scientists and voila, pardon Viola:

Secret objective

I once attempted to corner one of the heads of HBP, Wolfgang Marquardt, at a journalist conference in Bremen and get him to divulge to me in public what the objectives of HBP are. Officially, Marquart is thought to have nothing to do with HBP, aside of being the mediator of the past Markram-mutiny. In fact, even the current Flagship evaluation report refers to his mediation as “independent”. Except that he is representing Germany in HBP’s stakeholder board and the Forschungszentrum Jülich which he heads is now in control of HBP’s scientific orientation (see my past report). The following Twitter thread is the pathetic outcome of my Marquardt questioning. The objectives of HBP are none of public’s business. And yes, he doesn’t promise to deliver anything at all but will need more money.

Now that you understood that HBP is more similar to a cult of research funding worshippers than to an actual research project, you will also understand that it cannot be assessed scientifically. No matter what HBP does or does not do, it will always be evaluated positively, because its success was already predetermined by the EU when they chose Marquardt’s biased mediation over honest admittance of error, dissolution of HBP and redistribution of its funding. All EU evaluations of HBP and the Flagship concept in general are now nothing but smokescreens you are allowed to peek at through a keyhole. The last one was certainly exactly that.

The clueless Panel

As Maria Chiara Carrozza, Chair of the Flagship evaluation commission and rector at Scuola Superiore Sant’Anna (SSSA) in Pisa, Italy, told me:

“as it is written in the Report, we have based our evaluation on the available documents including the technical reports of the independent reviewers of the two on going Flagships. We didn’t perform a scientific review but we acquired the existing official documents”.

That earlier scientific evaluation report of HBP, which was performed in June-July 2016 by a panel of unnamed “18 independent experts”, is secret (unlike the previous one, for 2013-2015, which is public). The EU only published in September 2016 a very confusing summary which reads like someone desperately tried to turn failures and lack of progress into something positive, and didn’t make a good job there either. This is what according to that highlights summary the HBP’s brain simulation platform achieved, namely exactly nothing at all but some obscure theory:

“The Brain Simulation Platform demonstrated transdisciplinary research that integrates modelling with experimental data-sets.  It  enables  a  close  interaction  of  simulation,  experimental,  and  data  science  that  lead  to  theory  and simulation-driven experiments, which is of great value”.

Apparently also Carrozza did not know what the objectives of HBP are, since she replied to me in this regard:

“For the objectives of Human Brain you can contact the project coordinator”.

Why didn’t she ask him that at her evaluation meetings? Was it irrelevant?

Anyway, this is the official report of the Carrozza commission, published in February 2017 (I made my own backup copies, just in case). It evaluated both HBP and its sibling Graphene, which apparently ran into trouble of its own by failing to raise enough industrial interest, against all expectations:

“At the moment, some companies are only present with a watching brief rather than engaging in deeper cooperation. This may be because the Flagship [Graphene, -LS] is still in an early research phase, exploring various directions, which may make it appear to be an unattractive partnering proposition”.

Which economic impact?

In regard to HBP, the Flagship evaluation panel found (highlight mine):

“Yet there are also some differences in the achievements of the two Flagships. This stems from the difficulties that the Human Brain Project experienced in the early part of the ramp-up phase, which necessitated the need to form a mediation panel (see Appendix 2 for details). This has certainly slowed down the project. But the expert reviewers also note that there is a need for the project to achieve a more uniform level of research excellence than has been achieved thus far.

In particular the project needs to create stronger interactions among some of the sub-projects and to prove the value of research infrastructures to all Human Brain Project communities. The Human Brain Project’s problems in its early stages illustrate that while Flagships have demonstrated their contribution to delivering excellent science, there are factors that can create difficulties and hinder the achievement of scientific excellence”.

In plain speak, HBP sucks excellence-wise, certainly at its current stage, even the evaluation panel could not hide this well enough. The only section of HBP worth anything is apparently its Neuromorphic Computing platform:

“Developments such as the neuromorphic computing architectures have scope for high economic impact”.

It is yet another question what brain simulation has to do with neuromorphic computer chip technology. The real challenge with the latter is to try and translate the basic function principles of a simple biological neuronal networks to a computer chip, to escape the technical limitations and noise susceptibility of the digital 1/0 computation. Because this is complicated enough, with feasibility unclear, the focus was moved to a more prestigious model to emulate: the mammalian brain. Sure, why not. If one fails here, it won’t be for the lack of ambition.

Regardless of all this, the Panel advised (highlights mine):

“The continuing strategic relevance of the Flagship instrument for Europe’s research and innovation is confirmed, with a strong endorsement of the thinking underlying the Flagship concept. The funding of the Flagships instrument represents good value for money in terms of the quality of the research and its potential for innovation. It is thus recommended that the Flagship initiative be continued, and new Flagships launched in fields where the concept is relevant.”

Conclusions made ahead of the report

Mark Humphries, research fellow at the Division of Neuroscience & Experimental Psychology of University of Manchester, commented to me after reading the Panel report:

“It is hard to take the report seriously when it makes the astonishing claim that research on graphene and an IT project about brains are essential to Europe’s future and well-being. The report is consistently critical of the two existing Flagships, yet enthusiastically recommends continuing to use the Flagship instrument. It reads as though the conclusion was made ahead of the report.”

In fact, it was indeed. HBP was asked at a meeting on February 1^st 2016 to submit a self-assessment report (meeting minutes here). The HBP coordinator Philippe Gillet (as well as his Graphene colleague Jari Kinaret) were asked to

“provide a self-assessment of the two Flagships on lessons learnt so far and challenges they face”

Guess what? HBP could not be bothered, they delivered no such thing, as evidenced by the meeting minutes from July 8^th :

“The HBP Flagship has not provided the self-assessment requested at the February 1st meeting”.

Nobody in the Flagship evaluation commission ever asked for this self-assessment ever again, this I was told by an inside source. Did Gillet and his HBP colleagues already know they will be positively evaluated no matter what? It certainly seems so.

Now remember when the scientific evaluation of HBP was performed: June-July 2016. Which means, its results were not available to the Carrozza Commission at their July 8^th conference. The next meeting of her Panel was on December 9^th 2016, and according to the meeting minutes, the report was basically finalised by then:

“The whole meeting was devoted to discussing the draft version of the Panel’s final report”.

Thus, the Carrozza Panel certainly never discussed or likely never even read the scientific evaluation report of HBP, but in their own final report they claimed something else:

“The Panel also received the technical reports of the two scientific project reviews, undertaken by independent experts, of the on-going Flagship initiatives. These reports provided insights into the views of the technical experts assessing the excellence of the two Flagships. This included their technical assessment of progress to date. In the Panel’s report, however, scientific matters and achievements are only referenced as they relate to the purpose of the Flagships as an instrument, and how the instrument itself serves to support the achievement of purpose. The Panel also received from these review experts, responses to more strategic questions posed to them by the Panel. These questions concerned the role and effectiveness of the Flagship instrument as evidenced by the progress they have observed”.

There is nothing at all in the minutes of the three Panel meetings from February, July or December to indicate that any scientific review experts were invited to give their insights. Given the overlapping time frames, their reports could never have served as basis for the Flagship interim evaluation. We all have been duped here.

Bring your own Senior Advisor

Who were these people doing the EU Flagship evaluation? In the February 2017 report, they are named, and for the first time we know the names of the reviewers. Maybe this is because the committee’s composition was made public elsewhere long ago, e.g. here (the Flagship offices nevertheless refused to confirm to me the exact identities and affiliations of the Panel members).

The Panel Chair was Maria Chiara Carrozza, who is professor at The BioRobotics Institute in Pisa, Italy. By a freak coincident her colleague and co-author on several papers Cecilia Laschi is member of HBP in its SP10 Neurorobotics Platform. By another freak coincident, exactly this Neurorobotics Platform was explicitly criticised in the scientific evaluation report as seriously under-performing:

“The Neurorobotics Platform will have to show better how it can contribute to perform robotic experiments that address neuroscience research.”

Well, now thanks to Carrozza we know that all HBP platforms work and deliver sufficiently good. Carrozza herself explained to me:

“I am not aware of what Prof. Laschi is doing in Human Brain, I know only that she is part of it, and because I was part of the Panel, I never discussed with her or with other members of the Project about the evaluation.  I also know personally other scientists in Human Brain and in Graphene but I did not discuss with them”.

Other Panel members were: Michal Kleiber, Head of the Computational Science Department at the Polish Academy of Sciences; Matthias Kleiner, President of the Leibniz Association (a German research society which comprises a large network of research institutes); Charlotte Brogren, Director General of the Swedish Governmental Agency for Innovation Systems, VINNOVA (Update 28.02.2017: see Brogren’s insights into HBP evaluation below); and Ruth McKernan, General Director of the United Kingdom Government Executive Agency for innovation (Innovate UK). Both Brogren and McKernan came to their current jobs from the positions of senior industry executives, the former with the robotics division of the electrical company ABB and the latter with the pharma giants Pfizer and Merck.

They were joined by the science enthusiast and book author Paul T Kidd (who describes himself as “Scriptovisualist”) as Rapporteur. Kidd was employed by the EU solely to write the Panel’s report, while following the discussions of the Panel members and bringing their profound thoughts on HBP and Graphene to paper. Thus, he was not part of the evaluation committee, but employed for his professional protocol writing service which he performed already several times for the EU.

The most embarrassing bit of smokescreening is about the so-called “Senior Advisors”. You might expect some well-respected international academic scientists to be listed here, whose expert advice the panel members sought when evaluating the Flagships. Nothing is further from the truth. The Panel members simply brought their personal assistants with them, to interview the Flagship executives, to keep track of the paperwork and schedule and probably also in charge of fetching coffee and biscuits. In fact, in the meeting minutes those “Senior Advisors” were merely referred to as “panel research assistants”.

Carrozza brought in tow Calogero M Oddo, her student protégée and now assistant professor under her at The BioRobotics Institute. The Leibniz director Kleiner arrived with his head secretary, Caroline Lodemann. The funding chief executives Brogren and McKernan were assisted by their agencies’ respective EU-liaison officers, namely  Johan Lindberg and Sivasegaram Manimaaran. Only Kleiber did not bring anyone to assist him. I asked him why, but he did not reply. Was the Polish professor forced to get his own coffee then?

The Art of Workshopping

The HBP meanwhile are celebrating their tremendous success in being positively evaluated, just when the report appeared they organised a workshop in the Spanish beach city of Malaga:

In fact, workshopping is where HBP really succeeded. For their “ramp-up” phase 2013-2016 they promised to EU fifty workshops, and performed already 73, even though they recruited less potential participants (PhD students and postdocs) than the rather modest number scheduled. The EU was apparently warned not to expect any patents at all on this €1 Billion investment, so as a pleasant surprise, HBP patented something already (one patent exactly). Another reassuring advance: instead of just 4 (yes, just four!) scheduled academic or industry collaborations, HBP already established 124! Whoo-hoo! The trick is to set your official goals really low, everything becomes a success then.

There were unfortunately less “mentions in public media” than projected, and HBP for some reason did not bother to categorize those into positive and negative. I wonder if my own article was included there.

Meanwhile, the EU already announced the next €1 billion Flagship project: The Quantum Technology Flagship. The announcement came in May 2016, long before this Flagship evaluation was performed, which now recommended to start new Flagships. It seems in the EU all research funding decisions are first made, and then justified and evaluated retrospectively.

---