Fifty years of research on questionable research practises in science: quantitative analysis of co-citation patterns

Questionable research practises (QRPs) have been the focus of the scientific community amid greater scrutiny and evidence highlighting issues with replicability across many fields of science. To capture the most impactful publications and the main thematic domains in the literature on QRPs, this study uses a document co-citation analysis. The analysis was conducted on a sample of 341 documents that covered the past 50 years of research in QRPs. Nine major thematic clusters emerged. Statistical reporting and statistical power emerged as key areas of research, where systemic-level factors in how research is conducted are consistently raised as the precipitating factors for QRPs. There is also an encouraging shift in the focus of research into open science practises designed to address engagement in QRPs. Such a shift is indicative of the growing momentum of the open science movement, and more research can be conducted on how these practises are employed on the ground and how their uptake by researchers can be further promoted. However, the results suggest that, while pre-registration and registered reports receive the most research interest, less attention has been paid to other open science practises (e.g. data sharing).

A network of change: united action on research integrity

The last decade has seen renewed concern within the scientific community over the reproducibility and transparency of research findings. This paper outlines some of the various responsibilities of stakeholders in addressing the systemic issues that contribute to this concern. In particular, this paper asserts that a united, joined-up approach is needed, in which all stakeholders, including researchers, universities, funders, publishers, and governments, work together to set standards of research integrity and engender scientific progress and innovation. Using two developments as examples: the adoption of Registered Reports as a discrete initiative, and the use of open data as an ongoing norm change, we discuss the importance of collaboration across stakeholders.

Preliminary Evidence for an Association between Journal Submission Requirements and Reproducibility of Published Findings: A Pilot Study

Systemic efforts have been employed to improve the reproducibility of published findings in psychology. To date, little research has been conducted evaluating how well these efforts work. In an effort to bridge this gap, the current study looked at journal submission requirements intended to encourage authors to engage in best practices for facilitating reproducible science and offers preliminary evidence for their potential efficacy. We calculated reproducibility indices (p-curves) for three randomly selected empirical studies published in each of 23 psychology journals in 2019 and correlated quantitative results from those analyzes with the number of submission requirements for each journal that intended to ensure compliance with best reporting practices. Results indicated a greater number of submission requirements at a given outlet was associated with indices indicating greater likelihood of reproducibility of findings. We frame findings as impetus for future, more extensive, research to identify causal links between submission requirements and reproducibility.

Prevalence of Research Misconduct and Questionable Research Practices: A Systematic Review and Meta-Analysis

Irresponsible research practices damaging the value of science has been an increasing concern among researchers, but previous work failed to estimate the prevalence of all forms of irresponsible research behavior. Additionally, these analyses have not included articles published in the last decade from 2011 to 2020. This meta-analysis provides an updated meta-analysis that calculates the pooled estimates of research misconduct (RM) and questionable research practices (QRPs), and explores the factors associated with the prevalence of these issues. The estimates, committing RM concern at least 1 of FFP (falsification, fabrication, plagiarism) and (unspecified) QRPs concern 1 or more QRPs, were 2.9% (95% CI 2.1-3.8%) and 12.5% (95% CI 10.5-14.7%), respectively. In addition, 15.5% (95% CI 12.4-19.2%) of researchers witnessed others who had committed at least 1 RM, while 39.7% (95% CI 35.6-44.0%) were aware of others who had used at least 1 QRP. The results document that response proportion, limited recall period, career level, disciplinary background and locations all affect significantly the prevalence of these issues. This meta-analysis addresses a gap in existing meta-analyses and estimates the prevalence of all forms of RM and QRPs, thus providing a better understanding of irresponsible research behaviors.

Spanish Scientists' Opinion about Science and Researcher Behavior

We surveyed 348 Psychology and Education researchers within Spain, on issues such as their perception of a crisis in Science, their confidence in the quality of published results, and the use of questionable research practices (QRP). Their perceptions regarding pressure to publish and academic competition were also collected. The results indicate that a large proportion of the sample of Spanish academics think there is a crisis in Science, mainly due to a lack of economic investment, and doubts the quality of published findings. They also feel strong pressure to publish in high impact factor journals and a highly competitive work climate.

Research Misconduct in the Fields of Ethics and Philosophy: Researchers' Perceptions in Spain

Empirical studies have revealed a disturbing prevalence of research misconduct in a wide variety of disciplines, although not, to date, in the areas of ethics and philosophy. This study aims to provide empirical evidence on perceptions of how serious a problem research misconduct is in these two disciplines in Spain, particularly regarding the effects that the model used to evaluate academics' research performance may have on their ethical behaviour. The methodological triangulation applied in the study combines a questionnaire, a debate at the annual meeting of scientific association, and in-depth interviews. Of the 541 questionnaires sent out, 201 responses were obtained (37.1% of the total sample), with a significant difference in the participation of researchers in philosophy (30.5%) and in ethics (52.8%); 26 researchers took part in the debate and 14 interviews were conducted. The questionnaire results reveal that 91.5% of the respondents considered research misconduct to be on the rise; 63.2% considered at least three of the fraudulent practices referred to in the study to be commonplace, and 84.1% identified two or more such practices. The researchers perceived a high prevalence of duplicate publication (66.5%) and self-plagiarism (59.0%), use of personal influence (57.5%) and citation manipulation (44.0%), in contrast to a low perceived incidence of data falsification or fabrication (10.0%). The debate and the interviews corroborated these data. Researchers associated the spread of these misconducts with the research evaluation model applied in Spain.

The myth of 'older LGBT+' people: Research shortcomings and policy/practice implications for health/care provision

This article explores the implications of research which takes a collectivised approach to lesbian, gay, bisexual and trans+ (LGBT+) ageing and which engages in Questionable Research Practices (QRPs) in doing so. Collectivised approaches to heterogenous identity-based groups address commonalities but often fail to address internal diversity, i.e. the differences between and among older LGBT+ people. This article explores six key problems associated with collectivised research: (1) Homogenising language and phrases; (2) Uneven numerical representation of sub-groups; (3) Thematic over-representation of sexuality; (4) Non-intersectional analyses; (5) Thematic under-representation of gender; and (6) Inaccurate reporting of data. Research which does not differentiate between 'older LGBT+' sub-populations, can provide policy-makers and practitioners with inaccurate and/or misleading information, resulting in services which meet the needs of some, but not all, older LGBT+ people. This article discusses how research can become more inclusive, intersectional and reliable.

Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices

As concern over the use of questionable research practices (QRPs) in academic science has increased over the last couple of decades, some reforms have been implemented and many others have been debated and recommended. While many of these proposals have merit, efforts to improve scientific practices are more likely to succeed when they are responsive to the prevailing views and concerns of scientists themselves. To date, there have been few efforts to solicit wide-ranging input from researchers on the topic of needed reforms. This article is a qualitative report of responses from federally funded scientists to the question of what should be done to address the problem of QRPs in their disciplines. Overall, participants were concerned about how institutional and career-oriented incentives encourage the use of QRPs. Compared to previous recommendations, participants had surprisingly little confidence in the ability of ethics training to improve research integrity.

Skin in the Game: Personal Accountability and Journal Peer Review

Two preregistered studies explored the likelihood paper reviewers would request clarification from authors regarding potential questionable research practices (QRPs). Study 1 participants were instructed to imagine reviewing a journal manuscript as either a coauthor or peer reviewer and rate the extent to which they would request clarification from the author when encountering potential QRPs. Participants reported greater likelihood of requesting clarification when assigned to the coauthor relative to the peer reviewer role. Study 2 participants were assigned to either an anonymous or open-review condition and rated the extent to which they would seek clarification from an author regarding potential QRPs. Men (but not women) in the open review condition reported greater likelihood of seeking clarification about potential QRPs than men in the blind review condition. Results provide tentative evidence that motivational factors influence the peer review process, and suggestions are made for improving peer review practices.