This is a guest post by Future of Research policy activist, Adriana Bankston.
There are “internal problems rooted in the culture of science,” write Arturo Casadevall and Ferric C. Fang in a 2017 post for The Baltimore Sun entitled “Is science in crisis?” Arturo Casadevall, professor and chair of the Department of Molecular Microbiology and Immunology at the Johns Hopkins Bloomberg School of Public Health, highlighted some of these problems in his talk entitled “Crisis in biomedical science: time for a change?” given at the Microbiology & Immunology Department seminar at the University of Louisville on February 2, 2017.
Innovation, science and technology drive economic growth. And yet, many different types of problems exist within the scientific enterprise, and these are both external and internal. Some notable external problems are inadequate funding and workforce imbalances, whereas major internal problems are disproportionate rewards to winners, an obsession with the impact factor, and poor communication with the public. Proposed solutions are political, societal and scientific reforms.
From this list, Dr. Casadevall focused on three major issues with the culture of science during his talk – namely funding science, reproducibility issues and the impact factor distortion.
Science funding is a chronic structural problem, complicated by the inability of reviewers to discriminate the future success of grants between “good enough” grants – scoring 2% vs. 20% for example in study section, making the funding process more like random chance according to Casadevall. This leads to the question of whether we are funding the best work. In a controversial article for the Wall Street Journal published in 2014, Casadevall and Fang state the following: “Winning a government grant is already a crapshoot. Making it official by running a lottery would be an improvement.” They propose the use of a lottery system amongst these top grant proposals, for several reasons which appear to be immediate advantages.
A major problem within science is lack of reproducibility, and this leads to less public confidence in science. One shocking fact is that the number of retractions is growing faster than the number of publications! Further data show, according to an mBio study published by Fang, Bennett and Casadevall in 2013, that scientific misconduct is higher in men than women and can occur at all career stages. Their study also found that, surprisingly, higher retractions do not occur with papers that have the highest impact factors (such as Cell, Nature, Science) – instead, most retractions occur as a result of scientific misconduct. In 2016, Bik, Casadevall and Fang published a paper showing their findings as a result of their analysis of images from over 20,000 published papers. They found that 1 in 25 papers contains a problematic (i.e. inappropriately duplicated) image, which is very high. Interestingly, this problem appears to be a 21st century phenomenon, potentially coincidental with the increased usage by scientists of various image manipulation tools when making figures. As a potential solution, the authors suggest creating a teaching set which all reviewers of publications can utilize as guidelines. Casadevall offered another useful tip to help with this problem, and that is having someone in the lab make the figure who is not also responsible for generating the data, thereby eliminating some bias.
The third important issue brought up by Casadevall is the issue of impact factor distortion. The impact factor, according to Casadevall, is based on the false premise that ‘exclusivity = quality.’ However, data show that for example Nature papers are actually not cited more than those in other journals, as an example of this concept. Also, this type of premise is really damaging to the communal enterprise of science! This is because the norms of science are out of sync with the rewards, a conclusion from a study published by Bowen and Casadevall in PNAS in 2015. Here, they argue that we are expecting reproducibility, but are rewarding impact. And if we are rewarding impact, which is not the same as importance, we may be missing out on actually doing the most important scientific work! He listed several initiatives addressing some of these critical issues in science, such as the Cancer Biology Reproducibility Project and the San Francisco Declaration on Research Assessment (DORA).
In his concluding slides, Casadevall stressed that more data are necessary on these important issues in science, in order to convince both scientists and the public of their existence and need for action. I am a big fan of this idea, and wholeheartedly agree with his statement that “the study of science is a science that we need to develop.”
To end, Casadevall showed an excerpt of a report published in the October 2014 issue of the ASM Microbe Magazine magazine together with Fang, in which they suggest humor as a way to deal with some of these issues. Their paper humorously showing unrecognized medical conditions among scientists has resonated well with the scientific community. Since then, the paper has been translated into a graphic which is now being widely-shared and displayed in labs as shown below.
Note: all images of slides shown here are taken from Dr. Casadevall’s presentation at the University of Louisville on February 2, 2017.