Identical summary statistics were uncommon in randomized trials and cohort studies

Statistical techniques to assess publication integrity in groups of randomized trials: a narrative review

OBJECTIVES

To describe statistical tools available for assessing publication integrity of groups of randomized controlled trials (RCTs).

STUDY DESIGN AND SETTING

Narrative review.

RESULTS

Freely available statistical tools have been developed that compare the observed distributions of baseline variables with the expected distributions that would occur if successful randomization occurred. For continuous variables, the tools assess baseline means, baseline P values, and the occurrence of identical means and/or standard deviation. For categorical variables, they assess baseline P values, frequency counts for individual or all variables, numbers of trial participants randomized or withdrawing, and compare reported with independently calculated P values. The tools have been used to identify publication integrity concerns in RCTs from individual groups, and performed at an acceptable level in discriminating intentionally fabricated baseline summary data from genuine RCTs. The tools can be used when concerns have been raised about RCT(s) from an individual/group and when the whole body of their work is being examined, when conducting systematic reviews, and could be adapted to aid screening of RCTs at journal submission.

CONCLUSION

Statistical tools are useful for the assessment of publication integrity of groups of RCTs.

Post-publication research integrity concerns in randomized clinical trials: A scoping review of the literature

BACKGROUND

Post-publication handling of integrity concerns in randomized clinical trials (RCTs) is a contentious matter.

OBJECTIVES

We undertook a scoping systematic review to map the literature regarding post-publication integrity issues in RCTs.

SEARCH STRATEGY AND SELECTION CRITERIA

Following prospective registration (https://osf.io/pgxd8) we initially searched PubMed and Scopus but subsequently extended it to include the Cochrane Library, and Google Scholar databases without language, article type or publication time restriction until November 2022. Reviewers independently selected published articles covering any aspect of post-publication research integrity concerns in RCTs.

DATA COLLECTION AND ANALYSIS

The study findings grouped within domains relating to issues concerning post-publication integrity were extracted in duplicate, verified by a third reviewer, and then tabulated.

MAIN RESULTS

The initial search captured 3159 citations, of which 89 studies were included in the review. Cross-sectional studies constituted the majority of included studies (n = 34, 38.2%), followed by systematic reviews (n = 10, 11.2%), methodology reviews/studies (n = 9, 10.1%) and other types of descriptive studies (n = 8, 9.0%). A total of 21 articles (23.6%) covered the domain on general issues, 25 (28.1%) in the journal's instructions and policies domain, eight (9.0%) in the editorial and peer review domain, one (1.1%) in the correspondence and complaints (post-publication peer review) domain, 12 (13.5%) in the investigation for concerns domain, six (6.7%) in the post-investigation decisions and sanctions domain, none in the critical appraisal guidance domain, five (5.6%) in the integrity assessment in systematic reviews domain, and 26 (29.2%) in the recommendations for future research domain. A total of 12 of the selected articles (13.5%) covered two (n = 9) or three (n = 3) different domains.

CONCLUSIONS

Various research integrity domains and issues covering post-publication aspects of RCT integrity were captured and gaps were identified, mostly related with the necessary implications for all stakeholders to improve research transparency. There is an urgent need for a multistakeholder consensus towards creating specific statements for addressing post-publication integrity concerns in RCTs.

Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment

The integrity of randomized clinical trials (RCT) has become a concern owing to a recent rise in the number of retractions and the repercussions this has for evidence-based patient care. However, there is little research on the subject of RCT integrity assessment. Recent literature reviews have revealed that journals' authors' instructions concerning integrity and their investigation policies concerning allegations of misconduct are heterogeneous. The judicious use of integrity tests applied to RCT manuscripts is hampered by an absence of data concerning misconduct prevalence (pre-test probability), a failure to evaluate test performance (validity) and a lack of consensus over a gold standard (against which test accuracy can be evaluated). These deficiencies hinder the post-publication correction of RCT records, the integrity evaluations in systematic reviews of RCTs and the prospective application of preventive solutions in RCT peer-review and preprint assessment. Dealing with the current controversy about trustworthiness of RCT evidence requires a strong investment in research, reform and education concerning research integrity. The purpose of this review article is to highlight the current limitations in dealing with trial integrity-related concerns and to propose solutions to some of these issues.

Distributions of baseline categorical variables were different from the expected distributions in randomized trials with integrity concerns

BACKGROUND AND OBJECTIVES

Comparing observed and expected distributions of baseline continuous variables in randomized controlled trials (RCTs) can be used to assess publication integrity. We explored whether baseline categorical variables could also be used.

METHODS

The observed and expected (binomial) distribution of all baseline categorical variables were compared in four sets of RCTs: two controls, and two with publication integrity concerns. We also compared baseline calculated and reported P-values.

RESULTS

The observed and expected distributions of baseline categorical variables were similar in the control datasets, both for frequency counts (and percentages) and for between-group differences in frequency counts. However, in both sets of RCTs with publication integrity concerns, about twice as many variables as expected had between-group differences in frequency counts of one or 2, and far fewer variables than expected had between-group differences of >4 (P < 0.001 for both datasets). Furthermore, about one in six reported P-values for baseline categorial variables differed by > 0.1 from the calculated P-value in trials with publication integrity concerns.

CONCLUSION

Comparing the observed and expected distributions and reported and calculated P-values of baseline categorical variables may help in the assessment of publication integrity of a body of RCTs.