Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices

Navigating the Science System: Research Integrity and Academic Survival Strategies

Research Integrity (RI) is high on the agenda of both institutions and science policy. The European Union as well as national ministries of science have launched ambitious initiatives to combat misconduct and breaches of research integrity. Often, such initiatives entail attempts to regulate scientific behavior through guidelines that institutions and academic communities can use to more easily identify and deal with cases of misconduct. Rather than framing misconduct as a result of an information deficit, we instead conceptualize Questionable Research Practices (QRPs) as attempts by researchers to reconcile epistemic and social forms of uncertainty in knowledge production. Drawing on previous literature, we define epistemic uncertainty as the inherent intellectual unpredictability of scientific inquiry, while social uncertainty arises from the human-made conditions for scientific work. Our core argument-developed on the basis of 30 focus group interviews with researchers across different fields and European countries-is that breaches of research integrity can be understood as attempts to loosen overly tight coupling between the two forms of uncertainty. Our analytical approach is not meant to relativize or excuse misconduct, but rather to offer a more fine-grained perspective on what exactly it is that researchers want to accomplish by engaging in it. Based on the analysis, we conclude by proposing some concrete ways in which institutions and academic communities could try to reconcile epistemic and social uncertainties on a more collective level, thereby reducing incentives for researchers to engage in misconduct.

How Competition for Funding Impacts Scientific Practice: Building Pre-fab Houses but no Cathedrals

In the research integrity literature, funding plays two different roles: it is thought to elevate questionable research practices (QRPs) due to perverse incentives, and it is a potential actor to incentivize research integrity standards. Recent studies, asking funders, have emphasized the importance of the latter. However, the perspective of active researchers on the impact of competitive research funding on science has not been explored yet. Here, I address this issue by conducting a series of group sessions with researchers in two different countries with different degrees of competition for funding, from three scientific fields (medical sciences, natural sciences, humanities), and in two different career stages (permanent versus temporary employment). Researchers across all groups experienced that competition for funding shapes science, with many unintended negative consequences. Intriguingly, these consequences had little to do with the type of QRPs typically being presented in the research integrity literature. Instead, the researchers pointed out that funding could result in predictable, fashionable, short-sighted, and overpromising science. This was seen as highly problematic: scientists experienced that the 'projectification' of science makes it more and more difficult to do any science of real importance: plunging into the unknown or addressing big issues that need a long-term horizon to mature. They also problematized unintended negative effects from collaboration and strategizing. I suggest it may be time to move away from a focus on QRPs in connection with funding, and rather address the real problems. Such a shift may then call for entirely different types of policy actions.

Factors influencing the promotion and implementation of research integrity in research performing and research funding organizations: A scoping review

Promoting and implementing research integrity is considered the joint responsibility and effort of multiple stakeholders in the research community. We conducted a scoping review and analyzed 236 research articles and gray literature publications from biomedical sciences, social sciences, natural sciences (including engineering), and humanities that dealt with the factors that may positively or negatively impact the promotion and implementation of research integrity. Critical appraisal of evidence was performed for studies describing interventions aimed at research integrity promotion in order to provide insight into the effectiveness of these interventions. The results of this scoping review provide a comprehensive taxonomy of factors with positive or negative impact and their relatedness to individual researchers, research performing and funding organizations, and the system of science. Moreover, the results show that efforts for fostering and promoting research integrity should be implemented at all three levels (researcher, institution, system) simultaneously to deliver greater adherence and implementation of research integrity practices. Although various educational interventions aiming at research integrity promotion exist, we were not able to conclude on the effectiveness of explored interventions due to the methodological quality issues in the studies.

Is N-Hacking Ever OK? The consequences of collecting more data in pursuit of statistical significance

Upon completion of an experiment, if a trend is observed that is "not quite significant," it can be tempting to collect more data in an effort to achieve statistical significance. Such sample augmentation or "N-hacking" is condemned because it can lead to an excess of false positives, which can reduce the reproducibility of results. However, the scenarios used to prove this rule tend to be unrealistic, assuming the addition of unlimited extra samples to achieve statistical significance, or doing so when results are not even close to significant; an unlikely situation for most experiments involving patient samples, cultured cells, or live animals. If we were to examine some more realistic scenarios, could there be any situations where N-hacking might be an acceptable practice? This Essay aims to address this question, using simulations to demonstrate how N-hacking causes false positives and to investigate whether this increase is still relevant when using parameters based on real-life experimental settings.

Fifty years of research on questionable research practises in science: quantitative analysis of co-citation patterns

Questionable research practises (QRPs) have been the focus of the scientific community amid greater scrutiny and evidence highlighting issues with replicability across many fields of science. To capture the most impactful publications and the main thematic domains in the literature on QRPs, this study uses a document co-citation analysis. The analysis was conducted on a sample of 341 documents that covered the past 50 years of research in QRPs. Nine major thematic clusters emerged. Statistical reporting and statistical power emerged as key areas of research, where systemic-level factors in how research is conducted are consistently raised as the precipitating factors for QRPs. There is also an encouraging shift in the focus of research into open science practises designed to address engagement in QRPs. Such a shift is indicative of the growing momentum of the open science movement, and more research can be conducted on how these practises are employed on the ground and how their uptake by researchers can be further promoted. However, the results suggest that, while pre-registration and registered reports receive the most research interest, less attention has been paid to other open science practises (e.g. data sharing).

Principal Investigators' Priorities and Perceived Barriers and Facilitators When Making Decisions About Conducting Essential Research in the COVID-19 Pandemic

At the onset of the COVID-19 pandemic in the United States, stay-at-home orders disrupted normal research operations. Principal investigators (PIs) had to make decisions about conducting and staffing essential research under unprecedented, rapidly changing conditions. These decisions also had to be made amid other substantial work and life stressors, like pressures to be productive and staying healthy. Using survey methods, we asked PIs funded by the National Institutes of Health and the National Science Foundation (N = 930) to rate how they prioritized different considerations, such as personal risks, risks to research personnel, and career consequences, when making decisions. They also reported how difficult they found these choices and associated symptoms of stress. Using a checklist, PIs indicated those factors in their research environments that made their decisions easier (i.e., facilitators) or more difficult (i.e., barriers) to make. Finally, PIs also indicated how satisfied they were with their decisions and management of research during the disruption. Descriptive statistics summarize PIs' responses and inferential tests explore whether responses varied by academic rank or gender. PIs overall reported prioritizing the well-being and perspectives of research personnel, and they perceived more facilitators than barriers. Early-career faculty, however, rated concerns about their careers and productivity as higher priorities compared to their senior counterparts. Early-career faculty also perceived greater difficulty and stress, more barriers, fewer facilitators, and had less satisfaction with their decisions. Women rated several interpersonal concerns about their research personnel more highly than men and reported greater stress. The experience and perceptions of researchers during the COVID-19 pandemic can inform policies and practices when planning for future crises and recovering from the pandemic.

Quality Output Checklist and Content Assessment (QuOCCA): a new tool for assessing research quality and reproducibility

Research must be well designed, properly conducted and clearly and transparently reported. Our independent medical research institute wanted a simple, generic tool to assess the quality of the research conducted by its researchers, with the goal of identifying areas that could be improved through targeted educational activities. Unfortunately, none was available, thus we devised our own. Here, we report development of the Quality Output Checklist and Content Assessment (QuOCCA), and its application to publications from our institute's scientists. Following consensus meetings and external review by statistical and methodological experts, 11 items were selected for the final version of the QuOCCA: research transparency (items 1-3), research design and analysis (items 4-6) and research reporting practices (items 7-11). Five pairs of raters assessed all 231 articles published in 2017 and 221 in 2018 by researchers at our institute. Overall, the results were similar between years and revealed limited engagement with several recommended practices highlighted in the QuOCCA. These results will be useful to guide educational initiatives and their effectiveness. The QuOCCA is brief and focuses on broadly applicable and relevant concepts to open, high-quality, reproducible and well-reported science. Thus, the QuOCCA could be used by other biomedical institutions and individual researchers to evaluate research publications, assess changes in research practice over time and guide the discussion about high-quality, open science. Given its generic nature, the QuOCCA may also be useful in other research disciplines.

An Ethical Exploration of Increased Average Number of Authors Per Publication

This article explores the impact of an Increase in the average Number of Authors per Publication (INAP) on known ethical issues of authorship. For this purpose, the ten most common ethical issues associated with scholarly authorship are used to set up a taxonomy of existing issues and raise awareness among the community to take precautionary measures and adopt best practices to minimize the negative impact of INAP. We confirm that intense international, interdisciplinary and complex collaborations are necessary, and INAP is an expression of this trend. However, perverse incentives aimed to increase institutional and personal publication counts and egregious instances of guest or honorary authorship are problematic. We argue that whether INAP is due to increased complexity and scale of science, perverse incentives or undeserved authorship, it could negatively affect known ethical issues of authorship at some level. In the long run, INAP depreciates the value of authorship status and may disproportionately impact junior researchers and those who contribute to technical and routine tasks. We provide two suggestions that could reduce the long-term impact of INAP on the reward system of science. First, we suggest further refinement of the CRediT taxonomy including better integration into current systems of attribution and acknowledgement, and better harmony with major authorship guidelines such as those suggested by the ICMJE. Second, we propose adjustments to the academic recognition and promotion systems at an institutional level as well as the introduction of best practices.

Grant writing and grant peer review as questionable research practices

A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.

Grant writing and grant peer review as questionable research practices

A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.

Survey on the Research Misconduct and Questionable Research Practices of Medical Students, PhD Students, and Supervisors at the Zagreb School of Medicine in Croatia

This cross-sectional study evaluates the knowledge, attitudes, experiences, and behavior of final year medical students, PhD students, and supervisors at the School of Medicine of the University of Zagreb in relation to research misconduct, questionable research practices, and the research environment. The overall response rate was 36.4% (68%-100% for the paper survey and 8%-15% for the online surveys). The analysis reveals statistically significant differences in attitude scores between PhD students and supervisors, the latter having attitudes more in concordance with accepted norms. The results overall show a nonnegligible incidence of self-reported misconduct and questionable research practices, as well as some problematic attitudes towards misconduct and questionable research practices. The incidence of problematic authorship practices was particularly high. The research environment was evaluated as being mostly supportive of research integrity.

Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study

To foster research integrity (RI), it is necessary to address the institutional and system-of-science factors that influence researchers' behavior. Consequently, research performing and research funding organizations (RPOs and RFOs) could develop comprehensive RI policies outlining the concrete steps they will take to foster RI. So far, there is no consensus on which topics are important to address in RI policies. Therefore, we conducted a three round Delphi survey study to explore which RI topics to address in institutional RI policies by seeking consensus from research policy experts and institutional leaders. A total of 68 RPO and 52 RFO experts, representing different disciplines, countries and genders, completed one, two or all rounds of the study. There was consensus among the experts on the importance of 12 RI topics for RPOs and 11 for RFOs. The topics that ranked highest for RPOs concerned education and training, supervision and mentoring, dealing with RI breaches, and supporting a responsible research process (e.g. through quality assurance). The highest ranked RFO topics concerned dealing with breaches of RI, conflicts of interest, and setting expectations on RPOs (e.g. about educating researchers about RI). Together with the research policy experts and institutional leaders, we developed a comprehensive overview of topics important for inclusion in the RI policies of RPOs and RFOs. The topics reflect preference for a preventative approach to RI, coupled with procedures for dealing with RI breaches. RPOs and RFOs should address each of these topics in order to support researchers in conducting responsible research.

Views on ethical issues in research labs: A university-wide survey

In this article, we summarize the key findings of an exploratory study in which students and faculty completed a survey that sought to identify the most important ethical issues in STEM fields, how often these issues are discussed in research groups, and how often these ethical issues come up in the daily practice of research. Participants answered a series of open-ended and Likert-scale questions to provide a detailed look at the current ethical landscape at a private research university in the Midwest. The survey also looked at potential differences between faculty and undergraduate and graduate students' perceptions in answering these questions. The results indicate that while all community members tended to view issues that can be classified as research misconduct as the most important activities to avoid in STEM-related research, the level of discussion and actual witnessing of these practices was relatively low. The study points to a consensus among students and faculty about the important ethical issues in STEM and the need for more discussion and attention to be paid to communication, collaboration, and interpersonal relationships in the research environment.