Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands

A registered report survey of open research practices in psychology departments in the UK and Ireland

Open research practices seek to enhance the transparency and reproducibility of research. While there is evidence of increased uptake in these practices, such as study preregistration and open data, facilitated by new infrastructure and policies, little research has assessed general uptake of such practices across psychology university researchers. The current study estimates psychologists' level of engagement in open research practices across universities in the United Kingdom and Ireland, while also assessing possible explanatory factors that may impact their engagement. Data were collected from 602 psychology researchers in the United Kingdom and Ireland on the extent to which they have implemented various practices (e.g., use of preprints, preregistration, open data, open materials). Here we present the summarized descriptive results, as well as considering differences between various categories of researcher (e.g., career stage, subdiscipline, methodology), and examining the relationship between researcher's practices and their self-reported capability, opportunity, and motivation (COM-B) to engage in open research practices. Results show that while there is considerable variability in engagement of open research practices, differences across career stage and subdiscipline of psychology are small by comparison. We observed consistent differences according to respondent's research methodology and based on the presence of institutional support for open research. COM-B dimensions were collectively significant predictors of engagement in open research, with automatic motivation emerging as a consistently strong predictor. We discuss these findings, outline some of the challenges experienced in this study, and offer suggestions and recommendations for future research. Estimating the prevalence of responsible research practices is important to assess sustained behaviour change in research reform, tailor educational training initiatives, and to understand potential factors that might impact engagement.

Impact and Assessment of Research Integrity Teaching: A Systematic Literature Review

Presented here is a systematic literature review of what the academic literature asserts about: (1) the stages of the ethical decision-making process (i.e. awareness, reasoning, motivation, and action) that are claimed to be improved or not improved by RI teaching and whether these claims are supported by evidence; (2) the measurements used to determine the effectiveness of RI teaching; and (3) the stage/s of the ethical decision-making process that are difficult to assess. Regarding (1), awareness was the stage most claimed to be amenable to improvement following RI teaching, and with motivation being the stage that is rarely addressed in the academic literature. While few, some sources claimed RI teaching cannot improve specific stages. With behaviour (action) being the stage referenced most, albeit in only 9% of the total sources, for not being amenable to improvement following RI teaching. Finally, most claims were supported by empirical evidence. Regarding (2), measures most frequently used are custom in-house surveys and some validated measures. Additionally, there is much debate in the literature regarding the adequacy of current assessment measures in RI teaching, and even their absence. Such debate warrants caution when we are considering the empirical evidence supplied to support that RI teaching does or does not improve a specific stage of the decision-making process. Regarding (3), only behaviour was discussed as being difficult to assess, if not impossible. In our discussion section we contextualise these results, and following this we derive some recommendations for relevant stakeholders in RI teaching.

The punishment intensity for research misconduct and its related factors: An exploratory study on hospitals in Mainland China

Previous studies have found that factors such as gender and academic positions do not influence the severity of administrative actions taken by institutions. However, this study provides partly inconsistent evidence. It focuses on incidents of research misconduct in hospitals across Mainland China and explores factors related to punishment using a large cross-sectional dataset (N = 815). Regression analysis revealed a significant correlation between authorship order and the punishment intensity (p < 0.05). Under specific conditions, there was a significant correlation between the professional title (senior) and punishment intensity (p = 0.001), and an interaction between professional title and types of research misbehavior. Further analysis of simple effects showed that, in cases of fabrication and falsification, and combinations of multiple research misbehavior, researchers with senior titles received significantly lighter punishments compared to those with junior, intermediate, and associate senior titles (p < 0.05). The study unveils the potential accountability patterns (collective accountability and tiered punishment) that may be adopted by hospitals in Mainland China, as well as the challenges faced in ensuring fairness, emphasizing the importance of independent investigative bodies for incidents of research misconduct, and advocating for fairness as a priority in governance of research misconduct.

[International Multi-stakeholder Consensus Statement on Clinical Trial Integrity]

BACKGROUND

Science integrity initiatives require specific recommendations for randomised clinical trials (RCT).

OBJECTIVE

To prepare a set of statements for RCT integrity through an international multi-stakeholder consensus.

METHODS

The consensus was developed via multi-country multidisciplinary stakeholder group composition and engagement; evidence synthesis of 55 systematic reviews concerning RCT integrity; anonymised two-round modified Delphi survey with consensus threshold based on the average percent of majority opinions; and, a final consensus development meeting.

RESULTS

There were 30 stakeholders representing 15 countries from 5 continents including trialists, ethicists, methodologists, statisticians, consumer representative, industry representative, systematic reviewers, funding body panel members, regulatory experts, authors, journal editors, peer-reviewers and advisors for resolving integrity concerns. Delphi survey response rate was 86.7% (26/30 stakeholders). There were 111 statements (73 stakeholder-provided, 46 systematic review-generated, 8 supported by both) in the initial long list, with 8 additional statements provided during the consensus rounds. Through consensus the final set consolidated 81 statements (49 stakeholder-provided, 41 systematic review-generated, 9 supported by both). The entire RCT life cycle was covered by the set of statements including general aspects (n=6), design and approval (n=11), conduct and monitoring (n=19), reporting of protocols and findings (n=20), post-publication concerns (n=12), and future research and development (n=13).

CONCLUSION

Implementation of this multi-stakeholder consensus statement is expected to enhance RCT integrity.

'Special issue-ization' as a growth and revenue strategy: Reproduction by the "big five" and the risks for research integrity

The exponential growth of MDPI and Frontiers over the last decade has been powered by their extensive use of special issues. The "special issue-ization" of journal publishing has been particularly associated with new publishers and seen as potentially "questionable." Through an extended case-study analysis of three journals owned by one of the "big five" commercial publishers, this paper explores the risks that this growing use of special issues presents to research integrity. All three case-study journals show sudden and marked changes in their publication patterns. An analysis of special issue editorials and retraction notes was used to determine the specifics of special issues and reasons for retractions. Descriptive statistics were used to analyse data. Findings suggest that these commercial publishers are also promoting special issues and that article retractions are often connected to guest editor manipulation. This underlies the threat that "special issue-ization" presents to research integrity. It highlights the risks posed by the guest editor model, and the importance of extending this analysis to long-existing commercial publishers. The paper emphasizes the need for an in-depth examination of the underlying structures and political economy of science, and a discussion of the rise of gaming and manipulation within higher education systems.

How to fight fake papers: a review on important information sources and steps towards solution of the problem

Scientific fake papers, containing manipulated or completely fabricated data, are a problem that has reached dramatic dimensions. Companies known as paper mills (or more bluntly as "criminal science publishing gangs") produce and sell such fake papers on a large scale. The main drivers of the fake paper flood are the pressure in academic systems and (monetary) incentives to publish in respected scientific journals and sometimes the personal desire for increased "prestige." Published fake papers cause substantial scientific, economic, and social damage. There are numerous information sources that deal with this topic from different points of view. This review aims to provide an overview of these information sources until June 2024. Much more original research with larger datasets is needed, for example on the extent and impact of the fake paper problem and especially on how to detect them, as many findings are based more on small datasets, anecdotal evidence, and assumptions. A long-term solution would be to overcome the mantra of publication metrics for evaluating scientists in academia.

Verification of nucleotide sequence reagent identities in original publications in high impact factor cancer research journals

Human gene research studies that describe wrongly identified nucleotide sequence reagents have been mostly identified in journals of low to moderate impact factor, where unreliable findings could be considered to have limited influence on future research. This study examined whether papers describing wrongly identified nucleotide sequences are also published in high-impact-factor cancer research journals. We manually verified nucleotide sequence identities in original Molecular Cancer articles published in 2014, 2016, 2018, and 2020, including nucleotide sequence reagents that were claimed to target circRNAs. Using keywords identified in some 2018 and 2020 Molecular Cancer papers, we also verified nucleotide sequence identities in 2020 Oncogene papers that studied miRNA(s) and/or circRNA(s). Overall, 3.8% (251/6647) and 4.0% (47/1165) nucleotide sequences that were verified in Molecular Cancer and Oncogene papers, respectively, were found to be wrongly identified. Wrongly identified nucleotide sequences were distributed across 18% (91/500) original Molecular Cancer papers, including 38% (31/82) Molecular Cancer papers from 2020, and 40% (21/52) selected Oncogene papers from 2020. Original papers with wrongly identified nucleotide sequences were therefore unexpectedly frequent in two high-impact-factor cancer research journals, highlighting the risks of employing journal impact factors or citations as proxies for research quality.

Research culture in biomedicine: what we learned, and what we would like to do about it

This comment reports on work done at Harvard Medical School to identify areas for improvement in research rigor, reproducibility, and responsibility in pursuit of continued research excellence.

Aligning the principles and practice of research integrity and research fairness in global health: a mixed-methods study

INTRODUCTION

In the past decade, global health research has seen a growing emphasis on research integrity and fairness. The concept of research integrity emerged in response to the reproducibility crisis in science during the late 2000s. Research fairness initiatives aim to enhance ownership and inclusivity in research involving partners with varying powers, decision-making roles and resource capacities, ultimately prioritising local health research needs. Despite extensive academic discussions, empirical data on these aspects, especially in the context of global health, remain limited.

METHODS

To address this gap, we conducted a mixed-methods study focusing on research integrity and fairness. The study included an online frequency survey and in-depth key informant interviews with researchers from international research networks. The dual objectives were to quantify the frequency of practices related to research integrity and fairness and explore the determinants influencing these practices in global health.

RESULTS

Out of 145 participants in the quantitative survey (8.4% response rate), findings indicate that global health researchers generally adhere to principles of research integrity and fairness, with variations in reported behaviours. The study identified structural, institutional and individual factors influencing these patterns, including donor landscape rigidity, institutional investments in relationship building, guidelines, mentoring and power differentials among researchers.

CONCLUSION

This research highlights that, despite some variations, there is a substantial alignment between research integrity and fairness, with both sharing similar determinants and the overarching goal of enhancing research quality and societal benefits. The study emphasises the potential to explicitly recognise and leverage these synergies, aligning both agendas to further advance global health research.

Mismatch in perceptions of the quality of supervision and research data management as an area of concern: Results from a university-wide survey of the research integrity culture at a Belgian university

Researchers of KU Leuven, a large Belgian university, were invited to complete a bespoke questionnaire assessing their attitudes toward research integrity and the local research culture, with specific emphasis on the supervision of junior researchers. A total of 7,353 invitations were sent via e-mail and 1,866 responses were collected (25.3% response rate), of which 1,723 responses are reported upon here. Some of the findings are relevant to the broader research community. Whereas supervisors evaluated their supervision of junior researchers almost unanimously as positive, fewer supervisees evaluated it as such. Data management emerged as an area of concern, both in terms of reviewing raw data and of data storage. More female than male professors emphasized open communication and supported their supervisees' professional development and personal well-being. At the same time, fewer female professors felt safe to speak up than male professors. Finally, researchers who obtained their master's degree outside Europe evaluated their supervision and KU Leuven's research culture more positively than researchers with a master's degree from KU Leuven. The results of the survey were fed back to the university's board and several bodies and served as input to update the university's research policy. Faculties and departments received a detailed report.

How Competition for Funding Impacts Scientific Practice: Building Pre-fab Houses but no Cathedrals

In the research integrity literature, funding plays two different roles: it is thought to elevate questionable research practices (QRPs) due to perverse incentives, and it is a potential actor to incentivize research integrity standards. Recent studies, asking funders, have emphasized the importance of the latter. However, the perspective of active researchers on the impact of competitive research funding on science has not been explored yet. Here, I address this issue by conducting a series of group sessions with researchers in two different countries with different degrees of competition for funding, from three scientific fields (medical sciences, natural sciences, humanities), and in two different career stages (permanent versus temporary employment). Researchers across all groups experienced that competition for funding shapes science, with many unintended negative consequences. Intriguingly, these consequences had little to do with the type of QRPs typically being presented in the research integrity literature. Instead, the researchers pointed out that funding could result in predictable, fashionable, short-sighted, and overpromising science. This was seen as highly problematic: scientists experienced that the 'projectification' of science makes it more and more difficult to do any science of real importance: plunging into the unknown or addressing big issues that need a long-term horizon to mature. They also problematized unintended negative effects from collaboration and strategizing. I suggest it may be time to move away from a focus on QRPs in connection with funding, and rather address the real problems. Such a shift may then call for entirely different types of policy actions.

Bounded research ethicality: researchers rate themselves and their field as better than others at following good research practice

Bounded ethicality refers to people's limited capacity to consistently behave in line with their ethical standards. Here, we present results from a pre-registered, large-scale (N = 11,050) survey of researchers in Sweden, suggesting that researchers too are boundedly ethical. Specifically, researchers on average rated themselves as better than other researchers in their field at following good research practice, and rated researchers in their own field as better than researchers in other fields at following good research practice. These effects were stable across all academic fields, but strongest among researchers in the medical sciences. Taken together, our findings illustrate inflated self-righteous beliefs among researchers and research disciplines when it comes to research ethics, which may contribute to academic polarization and moral blindspots regarding one's own and one's colleagues' use of questionable research practices.

Editorial Perspective: Misaligned incentives in mental health research - the case for Registered Reports

Current incentive structures reward mental health researchers for producing positive, novel, and clean results. This can promote questionable research practices which contribute to a distorted evidence base, in turn limiting progress in mental health research. Registered Reports (RRs) offer a solution to realign the incentives towards conducting high-quality, rigorous, and accurate studies, by preventing publication and reporting biases. However, the uptake of RRs in mental health research has so far been limited. This editorial perspective highlights the advantages of RRs for mental health research, before discussing potential challenges and how they can be addressed. Greater uptake of RRs in mental health research could help to promote a fairer research culture, limit publication bias and questionable research practices, and ultimately, improve understanding of mental health.

Inappropriate image duplications in rhinology research publications

KEY POINTS

Duplicated images in research articles erode integrity and credibility of biomedical science. Forensic software is necessary to detect figures with inappropriately duplicated images. This analysis reveals a significant issue of inappropriate image duplication in our field.

The landscape of open science in behavioral addiction research: Current practices and future directions

Open science refers to a set of practices that aim to make scientific research more transparent, accessible, and reproducible, including pre-registration of study protocols, sharing of data and materials, the use of transparent research methods, and open access publishing. In this commentary, we describe and evaluate the current state of open science practices in behavioral addiction research. We highlight the specific value of open science practices for the field; discuss recent field-specific meta-scientific reviews that show the adoption of such practices remains in its infancy; address the challenges to engaging with open science; and make recommendations for how researchers, journals, and scientific institutions can work to overcome these challenges and promote high-quality, transparently reported behavioral addiction research. By collaboratively promoting open science practices, the field can create a more sustainable and productive research environment that benefits both the scientific community and society as a whole.

Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment

The integrity of randomized clinical trials (RCT) has become a concern owing to a recent rise in the number of retractions and the repercussions this has for evidence-based patient care. However, there is little research on the subject of RCT integrity assessment. Recent literature reviews have revealed that journals' authors' instructions concerning integrity and their investigation policies concerning allegations of misconduct are heterogeneous. The judicious use of integrity tests applied to RCT manuscripts is hampered by an absence of data concerning misconduct prevalence (pre-test probability), a failure to evaluate test performance (validity) and a lack of consensus over a gold standard (against which test accuracy can be evaluated). These deficiencies hinder the post-publication correction of RCT records, the integrity evaluations in systematic reviews of RCTs and the prospective application of preventive solutions in RCT peer-review and preprint assessment. Dealing with the current controversy about trustworthiness of RCT evidence requires a strong investment in research, reform and education concerning research integrity. The purpose of this review article is to highlight the current limitations in dealing with trial integrity-related concerns and to propose solutions to some of these issues.

Questionable research practices in competitive grant funding: A survey

There has been a surge of interest in research integrity over the last decade, with a wide range of studies investigating the prevalence of questionable research practices (QRPs). However, nearly all these studies focus on research design, data collection and analysis, and hardly any empirical research has been done on the occurrence of QRPs in the context of research funding. To fill this gap, we conducted a cross-sectional pre-registered survey of applicants, reviewers and panel members from the Research Foundation-Flanders (FWO), one of the main funding agencies in Belgium. We developed a bespoke survey and further refined it through feedback from experienced researchers and a pilot study. We asked how often respondents had engaged in a series of QRPs over the last ten years. A total of 1748 emails were sent, inviting recipients to participate in the survey, complemented by featuring the survey in the FWO newsletter. This resulted in 704 complete responses. Our results indicate that such QRPs are remarkably prevalent. Of the 496 participants who answered both the applicant and reviewer track, more than 60% responded that they engaged regularly in at least one of such practices, and around 40% indicated that they engaged at least occasionally in half of the QRPs queried. Only 12% reported not to have engaged in any of the QRPs. Contrary to our hypotheses, male respondents did not self-report to engage in the QRPs more often than female respondents, nor was there an association between the prevalence of QRPs and self-reported success rate in grant funding. Furthermore, half of the respondents indicated that they doubted the reliability of the grant peer review process more often than not. These results suggest that preventive action is needed, and provide new reasons to reconsider the practice of allocating research money through grant peer review.

Open Science Standards at Journals that Inform Evidence-Based Policy

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

Evidence of questionable research practices in clinical prediction models

BACKGROUND

Clinical prediction models are widely used in health and medical research. The area under the receiver operating characteristic curve (AUC) is a frequently used estimate to describe the discriminatory ability of a clinical prediction model. The AUC is often interpreted relative to thresholds, with "good" or "excellent" models defined at 0.7, 0.8 or 0.9. These thresholds may create targets that result in "hacking", where researchers are motivated to re-analyse their data until they achieve a "good" result.

METHODS

We extracted AUC values from PubMed abstracts to look for evidence of hacking. We used histograms of the AUC values in bins of size 0.01 and compared the observed distribution to a smooth distribution from a spline.

RESULTS

The distribution of 306,888 AUC values showed clear excesses above the thresholds of 0.7, 0.8 and 0.9 and shortfalls below the thresholds.

CONCLUSIONS

The AUCs for some models are over-inflated, which risks exposing patients to sub-optimal clinical decision-making. Greater modelling transparency is needed, including published protocols, and data and code sharing.

Empirical evidence of widespread exaggeration bias and selective reporting in ecology

In many scientific disciplines, common research practices have led to unreliable and exaggerated evidence about scientific phenomena. Here we describe some of these practices and quantify their pervasiveness in recent ecology publications in five popular journals. In an analysis of over 350 studies published between 2018 and 2020, we detect empirical evidence of exaggeration bias and selective reporting of statistically significant results. This evidence implies that the published effect sizes in ecology journals exaggerate the importance of the ecological relationships that they aim to quantify. An exaggerated evidence base hinders the ability of empirical ecology to reliably contribute to science, policy, and management. To increase the credibility of ecology research, we describe a set of actions that ecologists should take, including changes to scientific norms about what high-quality ecology looks like and expectations about what high-quality studies can deliver.

Research integrity in randomized clinical trials: A scoping umbrella review

BACKGROUND

Randomized clinical trials (RCTs) are experiencing a crisis of confidence in their trustworthiness. Although a comprehensive literature search yielded several reviews on RCT integrity, an overarching overview is lacking.

OBJECTIVES

The authors undertook a scoping umbrella review of the research integrity literature concerning RCTs.

SEARCH STRATEGY AND SELECTION CRITERIA

Following prospective registration (https://osf.io/3ursn), two reviewers independently searched PubMed, Scopus, The Cochrane Library, and Google Scholar, without language or time restrictions, until November 2021. The authors included systematic reviews covering any aspect of research integrity throughout the RCT lifecycle.

DATA COLLECTION AND ANALYSIS

The authors assessed methodological quality using a modified AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) tool and collated the main findings.

MAIN RESULTS

A total of 55 relevant reviews, summarizing 6001 studies (median per review, 63; range, 8-1106) from 1964 to 2021, had an overall critically low quality of 96% (53 reviews). Topics covered included general aspects (15%), design and approval (22%), conduct and monitoring (11%), reporting (38%), postpublication concerns (2%), and future research (13%). The most common integrity issues covered were ethics (18%) and transparency (18%).

CONCLUSIONS

Low-quality reviews identified various integrity issues across the RCT lifecycle, emphasizing the importance of high ethical standards and professionalism while highlighting gaps in the integrity landscape. Multistakeholder consensus is needed to develop specific RCT integrity standards.

Reproducibility in Neuroimaging Analysis: Challenges and Solutions

Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience, and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research-one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish 3 main types of reproducibility, discussing each in turn. Analytical reproducibility is the ability to reproduce findings using the same data and methods. Replicability is the ability to find an effect in new datasets, using the same or similar methods. Finally, robustness to analytical variability refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in more reproducible, replicable, and robust psychological and brain research and a stronger scientific foundation across fields of inquiry.

Post-publication Peer Review with an Intention to Uncover Data/Result Irregularities and Potential Research Misconduct in Scientific Research: Vigilantism or Volunteerism?

Irregularities in data/results of scientific research might be spotted pre-publication by co-workers and reviewers, or post-publication by readers typically with vested interest. The latter might consist of fellow researchers in the same subject area who would naturally pay closer attention to a published paper. However, it is increasingly apparent that there are readers who interrogate papers in detail with a primary intention to identify potential problems with the work. Here, we consider post-publication peer review (PPPR) by individuals, or groups of individuals, who perform PPPRs with a perceptible intention to actively identify irregularities in published data/results and to expose potential research fraud or misconduct, or intentional misconduct exposing (IME)-PPPR. On one hand, such activities, when done anonymously or pseudonymously with no formal discourse, have been deemed as lacking in accountability, or perceived to incur some degree of maleficence, and have been labelled as vigilantism. On the other, these voluntary works have unravelled many instances of research misconduct and have helped to correct the literature. We explore the tangible benefits of IME-PPPR in detecting errors in published papers and from the perspectives of moral permissibility, research ethics, and the sociological perspective of science. We posit that the benefits of IME-PPPR activities that uncover clear evidence of misconduct, even when performed anonymously or pseudonymously, outweigh their perceived deficiencies. These activities contribute to a vigilant research culture that manifests the self-correcting nature of science, and are in line with the Mertonian norms of scientific ethos.

Assessing the Integrity of Clinical Trials Included in Evidence Syntheses

Evidence syntheses of randomized clinical trials (RCTs) offer the highest level of scientific evidence for informing clinical practice and policy. The value of evidence synthesis itself depends on the trustworthiness of the included RCTs. The rising number of retractions and expressions of concern about the authenticity of RCTs has raised awareness about the existence of problematic studies, sometimes called "zombie" trials. Research integrity, i.e., adherence to ethical and professional standards, is a multi-dimensional concept that is incompletely evaluated for the RCTs included in current evidence syntheses. Systematic reviewers tend to rely on the editorial and peer-review system established by journals as custodians of integrity of the RCTs they synthesize. It is now well established that falsified and fabricated RCTs are slipping through. Thus, RCT integrity assessment becomes a necessary step in systematic reviews going forward, in particular because RCTs with data-related integrity concerns remain available for use in evidence syntheses. There is a need for validated tools for systematic reviewers to proactively deploy in the assessment of integrity deviations without having to wait for RCTs to be retracted by journals or expressions of concern issued. This article analyzes the issues and challenges in conducting evidence syntheses where the literature contains RCTs with possible integrity deficits. The way forward in the form of formal RCT integrity assessments in systematic reviews is proposed, and implications of this new initiative are discussed. Future directions include emphasizing ethical and professional standards, providing tailored integrity-specific training, and creating systems to promote research integrity, as improvements in RCT integrity will benefit evidence syntheses.

Responsible research practices could be more strongly endorsed by Australian university codes of research conduct

BACKGROUND

This study aimed to investigate how strongly Australian university codes of research conduct endorse responsible research practices.

METHODS

Codes of research conduct from 25 Australian universities active in health and medical research were obtained from public websites, and audited against 19 questions to assess how strongly they (1) defined research integrity, research quality, and research misconduct, (2) required research to be approved by an appropriate ethics committee, (3) endorsed 9 responsible research practices, and (4) discouraged 5 questionable research practices.

RESULTS

Overall, a median of 10 (IQR 9 to 12) of 19 practices covered in the questions were mentioned, weakly endorsed, or strongly endorsed. Five to 8 of 9 responsible research practices were mentioned, weakly, or strongly endorsed, and 3 questionable research practices were discouraged. Results are stratified by Group of Eight (n = 8) and other (n = 17) universities. Specifically, (1) 6 (75%) Group of Eight and 11 (65%) other codes of research conduct defined research integrity, 4 (50%) and 8 (47%) defined research quality, and 7 (88%) and 16 (94%) defined research misconduct. (2) All codes required ethics approval for human and animal research. (3) All codes required conflicts of interest to be declared, but there was variability in how strongly other research practices were endorsed. The most commonly endorsed practices were ensuring researcher training in research integrity [8 (100%) and 16 (94%)] and making study data publicly available [6 (75%) and 12 (71%)]. The least commonly endorsed practices were making analysis code publicly available [0 (0%) and 0 (0%)] and registering analysis protocols [0 (0%) and 1 (6%)]. (4) Most codes discouraged fabricating data [5 (63%) and 15 (88%)], selectively deleting or modifying data [5 (63%) and 15 (88%)], and selective reporting of results [3 (38%) and 15 (88%)]. No codes discouraged p-hacking or hypothesising after results are known.

CONCLUSIONS

Responsible research practices could be more strongly endorsed by Australian university codes of research conduct. Our findings may not be generalisable to smaller universities, or those not active in health and medical research.

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.

Biomedical supervisors' role modeling of open science practices

Supervision is one important way to socialize Ph.D. candidates into open and responsible research. We hypothesized that one should be more likely to identify open science practices (here publishing open access and sharing data) in empirical publications that were part of a Ph.D. thesis when the Ph.D. candidates' supervisors engaged in these practices compared to those whose supervisors did not or less often did. Departing from thesis repositories at four Dutch University Medical centers, we included 211 pairs of supervisors and Ph.D. candidates, resulting in a sample of 2062 publications. We determined open access status using UnpaywallR and Open Data using Oddpub, where we also manually screened publications with potential open data statements. Eighty-three percent of our sample was published openly, and 9% had open data statements. Having a supervisor who published open access more often than the national average was associated with an odds of 1.99 to publish open access. However, this effect became nonsignificant when correcting for institutions. Having a supervisor who shared data was associated with 2.22 (CI:1.19-4.12) times the odds to share data compared to having a supervisor that did not. This odds ratio increased to 4.6 (CI:1.86-11.35) after removing false positives. The prevalence of open data in our sample was comparable to international studies; open access rates were higher. Whilst Ph.D. candidates spearhead initiatives to promote open science, this study adds value by investigating the role of supervisors in promoting open science.

Which factors are associated with fraud in medical imaging research?

PURPOSE

To investigate the determinants of fraud in medical imaging research.

METHOD

This study analyzed aggregated survey data on scientific integrity completed by 877 corresponding authors who published in imaging journals in 2021. Multivariate regression analyses were performed to determine the association of scientific fraud with the following variables: survey participants' age (<18, 18-24, 25-34, 35-44, 45-54, 55-64, or > 65 years), gender (male, female, or other), Corruption Perceptions Index (CPI) of their country of work (linear 0-100 scale), academic degree (medical doctor or other), academic position (none, fellow/resident, instructor/ lecturer, assistant professor, associate professor, full professor, or other), and years of research experience (<5, 5-10, or > 10 years).

RESULTS

Thirty-seven survey participants (4.2%) indicated they had committed scientific fraud in the past 5 years, and 223 (25.4%) indicated they had witnessed or suspected scientific fraud by departmental colleagues in the past 5 years. Instructors/lecturers were significantly more likely (P = 0.029) and fellows/residents were nearly significantly more likely (P = 0.050) to have committed scientific fraud, with odds ratios (ORs) of 4.954 and 5.156, respectively (Nagelkerke R² of 0.114). Survey participants > 65 years of age and survey participants working in less corrupt countries were significantly less likely (P = 0.022 and P = 0.044, respectively) to have witnessed or suspected scientific fraud committed by their departmental colleagues, with ORs of 0.412 and 0.988 (per unit increase in CPI), respectively (Nagelkerke R² of 0.064).

CONCLUSIONS

Fraud in medical imaging research appears to be more common among junior faculty and in more corrupt countries.

International multi-stakeholder consensus statement on clinical trial integrity

OBJECTIVE

To prepare a set of statements for randomised clinical trials (RCT) integrity through an international multi-stakeholder consensus.

METHODS

The consensus was developed via: multi-country multidisciplinary stakeholder group composition and engagement; evidence synthesis of 55 systematic reviews concerning RCT integrity; anonymised two-round modified Delphi survey with consensus threshold based on the average percentage of majority opinions; and, a final consensus development meeting. Prospective registrations: (https://osf.io/bhncy, https://osf.io/3ursn).

RESULTS

There were 30 stakeholders representing 15 countries from five continents including triallists, ethicists, methodologists, statisticians, consumer representatives, industry representatives, systematic reviewers, funding body panel members, regulatory experts, authors, journal editors, peer-reviewers and advisors for resolving integrity concerns. Delphi survey response rate was 86.7% (26/30 stakeholders). There were 111 statements (73 stakeholder-provided, 46 systematic review-generated, 8 supported by both) in the initial long list, with eight additional statements provided during the consensus rounds. Through consensus the final set consolidated 81 statements (49 stakeholder-provided, 41 systematic review-generated, 9 supported by both). The entire RCT life cycle was covered by the set of statements including general aspects (n = 6), design and approval (n = 11), conduct and monitoring (n = 19), reporting of protocols and findings (n = 20), post-publication concerns (n = 12), and future research and development (n = 13).

CONCLUSION

Implementation of this multi-stakeholder consensus statement is expected to enhance RCT integrity.

NSF Fellows' perceptions about incentives, research misconduct, and scientific integrity in STEM academia

There is increased concern about perverse incentives, quantitative performance metrics, and hyper-competition for funding and faculty positions in US academia. Recipients of the prestigious National Science Foundation Graduate Research Fellowships (n = 244) from Civil and Environmental Engineering (45.5%) and Computer Science and Engineering (54.5%) were anonymously surveyed to create a baseline snapshot of their perceptions, behaviors and experiences. NSF Fellows ranked scientific advancement as the top metric for evaluating academics followed by publishing in high-impact journals, social impact of research, and publication/citation counts. The self-reported rate of academic cheating was 16.7% and of research misconduct was 3.7%. Thirty-one percent of fellows reported direct knowledge of graduate peers cheating, and 11.9% had knowledge of research misconduct by colleagues. Only 30.7% said they would report suspected misconduct. A majority of fellows (55.3%) felt that mandatory ethics trainings left them unprepared for dealing with ethical issues. Fellows stated academic freedom, flexible schedules and opportunity to mentor students were the most positive aspects of academia, whereas pressures for funding, publication, and tenure were cited as the most negative aspects. These data may be useful in considering how to better prepare STEM graduate trainees for academic careers.

How to combine rules and commitment in fostering research integrity?

Research integrity (RI) is crucial for trustworthy research. Rules are important in setting RI standards and improving research practice, but they can lead to increased bureaucracy; without commensurate commitment amongst researchers toward RI, they are unlikely to improve research practices. In this paper, we explore how to combine rules and commitment in fostering RI. Research institutions can govern RI using markets (using incentives), bureaucracies (using rules), and network processes (through commitment and agreements). Based on Habermas' Theory of Communicative Action, we argue that network processes, as part of the lifeworld, can legitimize systems - that is, market or bureaucratic governance modes. This can regulate and support RI practices in an efficient way. Systems can also become dominant and repress consensus processes. Fostering RI requires a balance between network, market and bureaucratic governance modes. We analyze the institutional response to a serious RI case to illustrate how network processes can be combined with bureaucratic rules. Specifically, we analyze how the Science Committee established at Tilburg University in 2012 has navigated different governance modes, resulting in a normatively grounded and efficient approach to fostering RI. Based on this case, we formulate recommendations to research institutions on how to combine rules and commitment.

Why research integrity matters and how it can be improved

Scholars need to be able to trust each other, because otherwise they cannot collaborate and use each other's findings. Similarly trust is essential for research to be applied for individuals, society or the natural environment. The trustworthiness is threatened when researchers engage in questionable research practices or worse. By adopting open science practices, research becomes transparent and accountable. Only then it is possible to verify whether trust in research findings is justified. The magnitude of the issue is substantial with a prevalence of four percent for both fabrication and falsification, and more than 50% for questionable research practices. This implies that researchers regularly engage in behaviors that harm the validity and trustworthiness of their work. What is good for the quality and reliability of research is not always good for a scholarly career. Navigating this dilemma depends on how virtuous the researcher at issue is, but also on the local research climate and the perverse incentives in the way the research system functions. Research institutes, funding agencies and scholarly journals can do a lot to foster research integrity, first and foremost by improving the quality of peer review and reforming researcher assessment.

Research misconduct and questionable research practices form a continuum

Research data mismanagement (RDMM) is a serious threat to accountability, reproducibility, and re-use of data. In a recent article in this journal, it was argued that RDMM can take two forms: intentional research misconduct or unintentional questionable research practice (QRP). I disagree because the scale for severity of consequences of research misbehavior is not bimodal. Furthermore, intentionality is difficult to prove beyond doubt and is only one of many criteria that should be taken into account when deciding on the severity of a breach of research integrity and whether a sanction is justified. Making a distinction between RDMM that is research misconduct and RDMM which not puts too much emphasis on intentionality and sanctioning. The focus should rather be on improving data management practices by preventive actions, in which research institutions should take a leading role.

Researchers on research integrity: a survey of European and American researchers

BACKGROUND

Reports of questionable or detrimental research practices (QRPs) call into question the reliability of scientific evidence and the trustworthiness of research. A critical component of the research ecosystem is the organization within which research takes place. We conducted a survey to explore the attitudes and beliefs of European and American researchers about the organisations in which they work, their own research practices and their attitudes towards research integrity and research integrity policies.

METHODS

We administered an online survey (International Research Integrity Survey (IRIS)) to 2,300 active researchers based in the US and 45,000 in Europe (including UK, Norway, Iceland and Switzerland).  We employed a stratified probability sample of the authors of research articles published between 2016 and 2020 included in Clarivate's Web of Science citation database. Coverage includes researchers in the humanities, social sciences, natural sciences and medical sciences, who hold at least a master's level degree.

RESULTS

In comparison to researchers in the US, European researchers admit to more QRPs and are less confident in maintaining high research integrity (RI) standards. In the US and Europe, many researchers judge their organization to fall short of best RI practice. All researchers recognize the benefits of RI, reliable knowledge and the trust of colleagues and the public, and there is support for RI training particularly among Europeans.

CONCLUSION

To create and maintain a culture of integrity in scientific research, a collective commitment from researchers, their institutions and funders is needed. Researchers rely on many channels of communication about research integrity and thus the involvement of many different participants in the research system is required to make improvements. Policies must be developed to reinforce best practice rather than being seen as an irrelevance to the real business of research.

Big little lies: a compendium and simulation of p-hacking strategies

In many research fields, the widespread use of questionable research practices has jeopardized the credibility of scientific results. One of the most prominent questionable research practices is p-hacking. Typically, p-hacking is defined as a compound of strategies targeted at rendering non-significant hypothesis testing results significant. However, a comprehensive overview of these p-hacking strategies is missing, and current meta-scientific research often ignores the heterogeneity of strategies. Here, we compile a list of 12 p-hacking strategies based on an extensive literature review, identify factors that control their level of severity, and demonstrate their impact on false-positive rates using simulation studies. We also use our simulation results to evaluate several approaches that have been proposed to mitigate the influence of questionable research practices. Our results show that investigating p-hacking at the level of strategies can provide a better understanding of the process of p-hacking, as well as a broader basis for developing effective countermeasures. By making our analyses available through a Shiny app and R package, we facilitate future meta-scientific research aimed at investigating the ramifications of p-hacking across multiple strategies, and we hope to start a broader discussion about different manifestations of p-hacking in practice.

[Can we save biomedical research from the managerial model?]

This article reflects on the disastrous effects that the managerial model had on contemporary biomedical research. It is argued that the time has come to reinvent an "antibureaucratic laboratory" which favors decision-making based on the intrinsic values of the researcher rather than the need to please external committees. Towards achieving this goal, governmental agencies should rely less on peer review and adopt new funding mechanisms. It is also predicted that the advent of the antibureaucratic laboratory" will only come at the cost of a revolution in the mentalities of all those involved in research.

The Embassy of Good Science - a community driven initiative to promote ethics and integrity in research

The Embassy of Good Science ( https://www.embassy.science) aims to improve research integrity and research ethics by offering an online, open, 'go-to' platform, which brings together information on research integrity and research ethics and makes that information accessible, understandable, and appealing. It effectively organizes and describes research integrity and research ethics guidelines, educational materials, cases, and scenarios. The Embassy is wiki-based, allowing users to add -- when logged in with their ORCID researcher id -- new information, and update and refine existing information. The platform also makes the research integrity and research ethics community visible and more accessible in pages dedicated to relevant initiatives, news and events. Therefore, the Embassy enables researchers to find useful guidance, rules and tools to conduct research responsibly. The platform empowers researchers through increased knowledge and awareness, and through the support of the research integrity and research ethics community. In this article we will discuss the background of this new platform, the way in which it is organized, and how users can contribute.

Research Integrity Supervision Practices and Institutional Support: A Qualitative Study

Scientific malpractice is not just due to researchers having bad intentions, but also due to a lack of education concerning research integrity practices. Besides the importance of institutionalised trainings on research integrity, research supervisors play an important role in translating what doctoral students learn during research integrity formal sessions. Supervision practices and role modelling influence directly and indirectly supervisees' attitudes and behaviour toward responsible research. Research supervisors can not be left alone in this effort. Research institutions are responsible for supporting supervisors in being more aware of their RI function, and in supporting responsible supervision practices to have a positive cascading effect on supervisees' research practices. We interviewed 22 European research supervisors to investigate how they perceive their role as research integrity trainers and their real-life supervision practices. Moreover, we investigated their points of view concerning the role of research institutions in supporting supervision practices. Although there are different commonalities in supervisors' perception of their research integrity-related role, differences are emphasised depending on the supervisors' characteristics such as academic domain, seniority, working country and gender. In addition, supervisors' way of mentoring depend also on supervisees' learning curve. Overall, all supervisors agreed on institutions playing an important role in support their supervision effort and practices. This study aims to be a starting point for better understanding research integrity supervision practices and the role of institutions in supporting them. Moreover, it puts the basis to further investigate differences in supervision practices depending on supervisors' characteristics.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10805-022-09468-y.

Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field

Known methodological issues such as publication bias, questionable research practices and studies with underpowered designs are known to decrease the replicability of study findings. The presence of such issues has been widely established across different research fields, especially in psychology. Their presence raised the first concerns that the replicability of study findings could be low and led researchers to conduct large replication projects. These replication projects revealed that a significant portion of original study findings could not be replicated, giving rise to the conceptualization of the replication crisis. Although previous research in the field of sports and exercise science has identified the first warning signs, such as an overwhelming proportion of significant findings, small sample sizes and lack of data availability, their possible consequences for the replicability of our field have been overlooked. We discuss the consequences of the above issues on the replicability of our field and offer potential solutions to improve replicability.

Publication games: In the web of reciprocity

The present processes of research assessment, i.e. focusing on one or a few, related, scientometrics, foster questionable authorship practices, like gifting authorship to non-contributing people. An especially harmful one of these unethical practices is the formation of publication cartels, where authors offer gift authorship to each other reciprocally. Here, by developing a simple model and a simulation of the publication process I investigate how beneficial cartels can be and what measure can be used to restrict them. My results indicate that publication cartels can significantly boost members' productivity even if paper counts are weighted by the inverse of author number (the 1/n rule). Nevertheless, applying the 1/n rule generates conflicts of interest both among cartel members themselves and between cartel members and non-members which might lead to the self-purification of the academic publishing industry.

Leaving academia: PhD attrition and unhealthy research environments

This study investigates PhD candidates’ (N = 391) perceptions about their research environment at a Dutch university in terms of the research climate, (un)ethical supervisory practices, and questionable research practices. We assessed whether their perceptions are related to career considerations. We gathered quantitative self-report estimations of the perceptions of PhD candidates using an online survey tool and then conducted descriptive and within-subject correlation analysis of the results. While most PhD candidates experience fair evaluation processes, openness, integrity, trust, and freedom in their research climate, many report lack of time and support, insufficient supervision, and witness questionable research practices. Results based on Spearman correlations indicate that those who experience a less healthy research environment (including experiences with unethical supervision, questionable practices, and barriers to responsible research), more often consider leaving academia and their current PhD position.

Research integrity in clinical trials: innocent errors and spin versus scientific misconduct

PURPOSE OF REVIEW

High-quality research underpins the best healthcare practice. This article focuses on analyzing the current literature to promote research integrity across clinical trials.

RECENT FINDINGS

Recent admissions of questionable practices by researchers have undermined practitioner and public confidence. There is limited evidence specifically for ethical and professional standards in clinical trials to guide researchers and institutions to embed integrity into research practice.

SUMMARY

Unintentional errors and spin in research are not uncommon as training in design and conduct of clinical trials is not part of health education for medical and allied health professions. There is unfamiliarity with procedures, such as prospective registration, a priori documentation of statistical analysis plans, openness in data sharing, and so forth. This, combined with the academic culture of secrecy, has led to an environment where scientific suspicion, instead of trust, is the norm. Existing science integrity documents are devoid of specific recommendations about how to translate any guidance into clinical trial practice. There is a need for constructive, supportive and multidisciplinary approaches based on open dialogue and continuous training, targeting the research environment. Research integrity now needs to take centre stage to re-instill confidence in randomized trial evidence to inform clinical practice.