Prospective clinical trial registration: not sufficient, but always necessary

Prospective registration of randomized clinical trials for total shoulder arthroplasty is low: a systematic review

BACKGROUND

Prospective trial registration has become an important means of improving the transparency and reproducibility of randomized controlled trials (RCTs) and is recommended by the Journal of Shoulder and Elbow Surgery (JSES) per the Consolidated Standards of Reporting Trials (CONSORT) guidelines. Herein, we performed a cross-sectional evaluation of RCTs published in JSES from 2010 to present to determine the prevalence of trial registration and consistency of outcome reporting.

METHODS

The electronic database PubMed was searched to identify all RCTs on total shoulder arthroplasty (TSA) published in JSES from 2010 to 2022 using the search terms "randomized controlled trial" AND "shoulder" AND "arthroplasty OR replacement." RCTs were considered to be registered if they provided a registration number. For articles that were registered, authors also extracted the registry name, registration date, date of first enrollment, date of last enrollment, and if the primary outcomes reported in the registry were either (1) omitted, (2) newly introduced in the publication, (3) reported as a secondary outcome or vice versa, or (4) varied in timing of assessment compared to the publication. "Early" RCTs were considered those published from 2010 to 2016, whereas "later" RCTs were from 2017 to 2022.

RESULTS

Fifty-eight RCTs met inclusion criteria. There were 16 early RCTs and 42 later RCTs. Twenty-three of the 58 (39.7%) studies were registered, with 9 of 22 with an available registry (40.9%) of those being enrolled prior to patient enrollment. Nineteen of the registered studies (82.6%) provided the name of the registry and a registration number. The proportion of later RCTs that were registered was not significantly different from the early RCTs (45.2% vs. 25.0%, P = .232). Seven RCTs (31.8%) had at least 1 inconsistency compared with the registry. The most common discrepancy was the timing of the assessment (ie, follow-up period) reported in the publication vs. the registry.

DISCUSSION

Although JSES recommends prospective trial registration, less than half of shoulder arthroplasty RCTs are registered and more than 30% registered trials have at least 1 inconsistency with their registry record. More rigorous review of trial registration and accuracy is necessary to limit bias in published shoulder arthroplasty RCTs.

The Fragility Index of randomized controlled trials in pediatric anesthesiology

PURPOSE

The P value is a widely used measure of statistical importance but has many drawbacks and limitations, one being that it does not reflect the robustness of the results of a clinical trial. The Fragility Index (FI) was developed as a measure of how many outcome events would need to change to nonevents to render a significant P value nonsignificant (P ≥ 0.05). The FI of trials from other medical specialties is typically < 5. We aimed to determine the FI of pediatric anesthesiology randomized controlled trials (RCT) and to test for association with various characteristics of the included trials.

METHODS

We conducted a comprehensive systematic search of high-impact anesthesia, surgical, and medical journals from the last 25 years for trials comparing an intervention between two groups with a statistically significant P value (< 0.05) for a dichotomous outcome. We also compared FI values for variables that reflect the quality and importance of a trial.

RESULTS

The median [interquartile range] FI was 3 [1-7] and correlated positively with the number of participants (rS = 0.41; P < 0.001) and events (rS = 0.42; P < 0.001), and negatively with the P value (rPB = -0.36; P < 0.001). Other measures of trial quality and impact or importance were not strongly associated with the FI.

CONCLUSIONS

The FI of published trials in pediatric anesthesiology is similarly low as in other medical specialties. Larger trials with more events and P values ≤ 0.01 were associated with a higher FI.

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

Inconsistent Outcome Reporting in Heart Failure Randomized Controlled Trials

BACKGROUND

Randomized controlled trials (RCTs) may report outcomes different from those prespecified on trial-registration websites, protocols and statistical analysis plans (SAPs). This study sought to investigate the prevalence and characteristics of heart failure (HF) RCTs that report outcomes different from those prespecified.

METHODS AND RESULTS

MEDLINE via PubMed was searched to include phase II-IV HF RCTs in 9 high-impact journals from 2010 to 2020. Outcomes reported in trial publications were compared with prespecified outcomes in protocols, registration websites and SAPs. We used the χ² or Fisher exact test to analyze correlations between trial characteristics and inconsistencies. Among 216 trials, 32 inconsistencies were observed in 28 trials (13.0%). Among 32 inconsistencies, 2 (6.3%) pertained to omission of prespecified primary outcomes, 4 (12.5%) to omission of prespecified secondary outcomes, 2 (6.3%) to changing prespecified primary outcomes to secondary outcomes, and 2 (6.3%) to changing prespecified secondary outcomes to primary outcomes. Of the inconsistencies, 3 (9.4%) pertained to addition of new primary outcomes, 17 (53.1%) to addition of new secondary outcomes, and 2 (6.3%,) to changes in the timing of assessment of primary outcomes. The majority of the inconsistencies favored statistically significant findings; 78 (36.1%) were registered retrospectively. Single-center recruitment was associated with outcome inconsistencies (β = -0.14; 95% CI, -0.22 - -0.01; P = 0.035).

CONCLUSIONS

More than 1 in 10 trials reported outcomes inconsistent with those specified in trial registration websites, SAPs and protocols. An action plan is warranted to minimize selective reporting and improve transparency.

A comparison of registered and published primary outcomes in clinical trials of opioid use disorder: ACTTION review and recommendations

BACKGROUND AND AIMS

Prospective trial registration can increase research integrity. This Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) review was designed to compare the primary outcomes (PO) reported in registries with associated publications for opioid use disorder (OUD) clinical trials.

DESIGN

The World Health Organization's International Clinical Trials Registry Platform (ICTRP) was searched for completed trials (2010 through 2019). Associated publications were identified and paired with trial registry data based on the publication date.

MEASUREMENTS

Reviewers independently rated the occurrence of discrepancies between the POs in the registry compared to the publication. An analysis of prospective versus retrospective registration was also completed.

FINDINGS

One-hundred and forty trials were identified in the search, and 43 registry-publication pairs evaluated. Only 34 of the 43 pairs could be examined for discrepancies because nine did not report a PO in registry and publication. Of the 34 pairs, only four met rigorous criteria for prospective trial registration and had an exact match of POs. In contrast, the majority of the 34 trials, or 80%, had inconsistent POs (e.g., registered secondary outcomes published as primary; the timing of PO not specified) and/or were retrospectively registered.

CONCLUSIONS

Many clinical trials focused on OUD have not met the standards of trial registration, such as consistent reporting of POs and prospective registration. Failure to properly register trial characteristics undermines the validity of research findings and can delay the development of life-saving treatments. Recommendations for improving prospective trial reporting practices are provided.

Benefit-risk assessment and reporting in clinical trials of chronic pain treatments: IMMPACT recommendations

ABSTRACT

Chronic pain clinical trials have historically assessed benefit and risk outcomes separately. However, a growing body of research suggests that a composite metric that accounts for benefit and risk in relation to each other can provide valuable insights into the effects of different treatments. Researchers and regulators have developed a variety of benefit-risk composite metrics, although the extent to which these methods apply to randomized clinical trials (RCTs) of chronic pain has not been evaluated in the published literature. This article was motivated by an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials consensus meeting and is based on the expert opinion of those who attended. In addition, a review of the benefit-risk assessment tools used in published chronic pain RCTs or highlighted by key professional organizations (ie, Cochrane, European Medicines Agency, Outcome Measures in Rheumatology, and U.S. Food and Drug Administration) was completed. Overall, the review found that benefit-risk metrics are not commonly used in RCTs of chronic pain despite the availability of published methods. A primary recommendation is that composite metrics of benefit-risk should be combined at the level of the individual patient, when possible, in addition to the benefit-risk assessment at the treatment group level. Both levels of analysis (individual and group) can provide valuable insights into the relationship between benefits and risks associated with specific treatments across different patient subpopulations. The systematic assessment of benefit-risk in clinical trials has the potential to enhance the clinical meaningfulness of RCT results.

High impact nutrition and dietetics journals' use of publication procedures to increase research transparency

BACKGROUND

The rigor and integrity of the published research in nutrition studies has come into serious question in recent years. Concerns focus on the use of flexible data analysis practices and selective reporting and the failure of peer review journals to identify and correct these practices. In response, it has been proposed that journals employ editorial procedures designed to improve the transparency of published research.

OBJECTIVE

The present study examines the adoption of editorial procedures designed to improve the reporting of empirical studies in the field of nutrition and dietetics research.

DESIGN

The instructions for authors of 43 journals included in Quartiles 1 and 2 of the Clarivate Analytics' 2018 Journal Citation Report category Nutrition and Dietetics were reviewed. For journals that published original research, conflict of interest disclosure, recommendation of reporting guidelines, registration of clinical trials, registration of other types of studies, encouraging data sharing, and use of the Registered Reports were assessed. For journals that only published reviews, all of the procedures except clinical trial registration were assessed.

RESULTS

Thirty-three journals published original research and 10 published only reviews. Conflict of interest disclosure was required by all 33 original research journals. Use of guidelines, trial registration and encouragement of data sharing were mentioned by 30, 27 and 25 journals, respectively. Registration of other studies was required by eight and none offered Registered Reports as a publication option at the time of the review. All 10 review journals required conflict of interest disclosure, four recommended data sharing and three the use of guidelines. None mentioned the other two procedures.

CONCLUSIONS

While nutrition journals have adopted a number of procedures designed to improve the reporting of research findings, their limited effects likely result from the mechanisms through which they influence analytic flexibility and selective reporting and the extent to which they are properly implemented and enforced by journals.

Characteristics associated with Publication of Randomized Controlled Trials in the Journal of Cardiothoracic and Vascular Anesthesia: A 15-Year Analysis, 2004-2018

Randomized controlled trials (RCTs) provide important data to guide clinical decisions. Publication bias may limit the applicability of RCTs because many clinical investigators prefer to submit and journals more selectively accept studies with positive results. The authors tested the hypothesis that positive RCTs published in the Journal of Cardiothoracic and Vascular Anesthesia were more likely to be associated with factors known to predict publication of positive versus negative RCTs in other journals. This observational study was an internet analysis of all issues of Journal of Cardiothoracic and Vascular Anesthesia from 2004-2018. Each issue was searched to identify human RCTs. The numbers of centers and enrolled patients in each RCT were tabulated. The corresponding author determined the country of origin (United States v international). A trial was "positive" or "negative" based on rejection or confirmation of the null hypothesis, respectively, for the primary outcome variable or the majority of measured outcomes if a primary outcome was not identified. The presence or absence of a hypothesis, randomization methodology, sample size calculation, and blinded research design was recorded. Registration in a public database, Consolidated Statements of Reporting Trials (CONSORT) guideline compliance, and the source of funding also were determined. The number of citations for each RCT was determined by using Google Scholar; the citation rate was calculated as the ratio of the number of total citations and the duration in years since the trial's original publication. A total of 296 RCTs were identified, of which 58.8% reported positive results. Most RCTs were single center, relatively small, and international in origin. Total citations/RCT decreased over time, but citations/year did not. The percentage of RCTs that identified a randomization method, were registered, or followed CONSORT guidelines increased in a time-dependent manner. No differences in any factors associated with publication of RCTs were observed when positive and negative trials were compared. The Journal of Cardiothoracic and Vascular Anesthesia publishes more positive than negative RCTs, but factors that have been previously associated with RCT publication in other journals were similar between groups.