Prevalence of Multiplicity and Appropriate Adjustments Among Cardiovascular Randomized Clinical Trials Published in Major Medical Journals

Vaginal Progesterone for Pregnancy Prolongation After Arrested Preterm Labor: A Randomized Clinical Trial

Importance

Women with arrested preterm labor (APTL) are at very high risk for spontaneous preterm delivery (SPTD), the leading cause of neonatal mortality and morbidity. To date, no maintenance therapy has been found to be effective for pregnancy prolongation. A few clinical trials with considerable methodological limitations have demonstrated some efficacy for 400 mg vaginal micronized progesterone (VMP) in women with APTL.

Objective

To investigate the effectiveness of daily 400 mg VMP for the prolongation of pregnancy after APTL.

Design, Setting, and Participants

This randomized clinical trial was conducted between December 19, 2018, and February 27, 2023, in 3 university-affiliated medical centers in Israel. Participants included women with singleton and twin pregnancies after APTL following tocolysis at 24 weeks 0 days to 34 weeks 0 days' gestation. Women with a history of preterm delivery or asymptomatic cervical shortening in the current pregnancy were excluded.

Interventions

Participants were randomly allocated to receive VMP 200 mg twice a day or no treatment until 36 weeks 6 days' gestation.

Main Outcomes and Measures

The primary end points were mean number of days from study enrollment to delivery and the rate of SPTD prior to 37 weeks' gestation.

Results

A total of 129 participants were enrolled (65 in the VMP group and 64 in the no-treatment group). Mean (SD) age was 27.6 (5.1) years. Between the VMP and no-treatment groups, there was no difference in pregnancy prolongation (mean [SD], 40.0 [17.8] vs 37.4 [20.3] days; P = .44) and the rate of SPTD (16 [25%] vs 19 [30%]; relative risk, 0.8; 95% CI, 0.5-1.5; P = .52). In twin pregnancies, including 12 and 15 pairs in the VMP and no-treatment groups, respectively, VMP prolonged pregnancy (mean [SD], 43.7 [18.1] vs 26.1 [15.2] days; P = .02), postponed the delivery week (36.5 [1.4] vs 34.7 [2.2] weeks; P = .01), shortened the length of stay in the neonatal intensive care unit (4.9 [10.6] vs 13.2 [18.5] days; P = .03) and overall hospital stay (8.3 [9.6] vs 15.1 [17.2] days; P = .03), and was associated with a higher birth weight (2444 [528] vs 2018 [430] g; P = .01).

Conclusions and Relevance

These findings show that VMP given in a dosage of 200 mg twice a day following APTL is not an effective treatment to prolong pregnancy or prevent SPTD. However, VMP demonstrated beneficial effects in twin pregnancies, warranting further investigation.

Trial Registration

ClinicalTrials.gov Identifier: NCT02430233.

Evidence-based medicine or statistically manipulated medicine? Are we slaves to the P-value?

First popularized almost a century ago in epidemiologic research by Ronald Fisher and Jerzy Neyman, the P-value has become perhaps the most misunderstood and even misused statistical value or descriptor. Indeed, modern clinical research has now come to be centered around and guided by an arbitrary P-value of <0.05 as a magical threshold for significance, so much so that experimental design, reporting of experimental findings, and interpretation and adoption of such findings have become largely dependent on this "significant" P-value. This has given rise to multiple biases in the overall body of biomedical literature that threatens the very validity of clinical research. Ultimately, a drive toward reporting a "significant" P-value (by various statistical manipulations) risks creating a falsely positive body of science, leading to (i) wasted resources in pursuing fruitless research and (ii) futile or even harmful policies/therapeutic recommendations. This article reviews the history of the P-value, the conceptual basis of P-value in the context of hypothesis testing and challenges in critically appraising clinical evidence vis-à-vis the P-value. This review is aimed at raising awareness of the pitfalls of this rigid observation of the threshold of statistical significance when evaluating clinical trials and to generate discussion regarding whether the scientific body needs a rethink about how we decide clinical significance.

Differential Treatment Effects of Subgroup Analyses in Phase 3 Oncology Trials From 2004 to 2020

Importance

Subgroup analyses are often performed in oncology to investigate differential treatment effects and may even constitute the basis for regulatory approvals. Current understanding of the features, results, and quality of subgroup analyses is limited.

Objective

To evaluate forest plot interpretability and credibility of differential treatment effect claims among oncology trials.

Design, Setting, and Participants

This cross-sectional study included randomized phase 3 clinical oncology trials published prior to 2021. Trials were screened from ClinicalTrials.gov.

Main Outcomes and Measures

Missing visual elements in forest plots were defined as a missing point estimate or use of a linear x-axis scale for hazard and odds ratios. Multiplicity of testing control was recorded. Differential treatment effect claims were rated using the Instrument for Assessing the Credibility of Effect Modification Analyses. Linear and logistic regressions evaluated associations with outcomes.

Results

Among 785 trials, 379 studies (48%) enrolling 331 653 patients reported a subgroup analysis. The forest plots of 43% of trials (156 of 363) were missing visual elements impeding interpretability. While 4148 subgroup effects were evaluated, only 1 trial (0.3%) controlled for multiple testing. On average, trials that did not meet the primary end point conducted 2 more subgroup effect tests compared with trials meeting the primary end point (95% CI, 0.59-3.43 tests; P = .006). A total of 101 differential treatment effects were claimed across 15% of trials (55 of 379). Interaction testing was missing in 53% of trials (29 of 55) claiming differential treatment effects. Trials not meeting the primary end point were associated with greater odds of no interaction testing (odds ratio, 4.47; 95% CI, 1.42-15.55, P = .01). The credibility of differential treatment effect claims was rated as low or very low in 93% of cases (94 of 101).

Conclusions and Relevance

In this cross-sectional study of phase 3 oncology trials, nearly half of trials presented a subgroup analysis in their primary publication. However, forest plots of these subgroup analyses largely lacked essential features for interpretation, and most differential treatment effect claims were not supported. Oncology subgroup analyses should be interpreted with caution, and improvements to the quality of subgroup analyses are needed.

Constrained randomization and statistical inference for multi‐arm parallel cluster randomized controlled trials

A practical limitation of cluster randomized controlled trials (cRCTs) is that the number of available clusters may be small, resulting in an increased risk of baseline imbalance under simple randomization. Constrained randomization overcomes this issue by restricting the allocation to a subset of randomization schemes where sufficient overall covariate balance across comparison arms is achieved. However, for multi-arm cRCTs, several design and analysis issues pertaining to constrained randomization have not been fully investigated. Motivated by an ongoing multi-arm cRCT, we elaborate the method of constrained randomization and provide a comprehensive evaluation of the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs in multi-arm cRCTs, with varying combinations of design and analysis-based covariate adjustment strategies. In particular, as randomization-based tests have not been extensively studied in multi-arm cRCTs, we additionally develop most-powerful randomization tests under the linear mixed model framework for our comparisons. Our results indicate that under constrained randomization, both model-based and randomization-based analyses could gain power while preserving nominal type I error rate, given proper analysis-based adjustment for the baseline covariates. Randomization-based analyses, however, are more robust against violations of distributional assumptions. The choice of balance metrics and candidate set sizes and their implications on the testing of the pairwise and global hypotheses are also discussed. Finally, we caution against the design and analysis of multi-arm cRCTs with an extremely small number of clusters, due to insufficient degrees of freedom and the tendency to obtain an overly restricted randomization space.

Multiple secondary outcome analyses: precise interpretation is important

Analysis of multiple secondary outcomes in a clinical trial leads to an increased probability of at least one false significant result among all secondary outcomes studied. In this paper, we question the notion that that if no multiplicity adjustment has been applied to multiple secondary outcome analyses in a clinical trial, then they must necessarily be regarded as exploratory. Instead, we argue that if individual secondary outcome results are interpreted carefully and precisely, there is no need to downgrade our interpretation to exploratory. This is because the probability of a false significant result for each comparison, the per-comparison wise error rate, does not increase with multiple testing. Strong effects on secondary outcomes should always be taken seriously and must not be dismissed purely on the basis of multiplicity concerns.

Review of pragmatic trials found that multiple primary outcomes are common but so too are discrepancies between protocols and final reports

OBJECTIVES

To describe prevalence of multiple primary outcomes, changes in primary outcomes and target sample sizes between protocols and final reports, and how issues of multiplicity are addressed in pragmatic trials.

STUDY DESIGN AND SETTING

Individually randomised trials labelled as pragmatic, published 2014-2019 in MEDLINE and registered with ClinicalTrials.gov.

RESULTS

We identified 262 final reports and located protocols for 159 (61%); primary outcomes were clearly reported in 145 (91%) protocols and 256 (98%) final reports. Thirty (19%) protocols and 38 (15%) final reports had multiple primary outcomes. Primary outcomes were present and identical in 128 (81%) matched protocol-final reports. Among 140 pairs with target sample sizes reported, 28 (20.0%) reduced their target sample size (mean 543 fewer participants per trial) and 16 (11.4%) increased it (mean 192 more participants per trial). Thirteen (29.5%) provided an explanation. Only 2/30 (7%) protocols and 4/38 (11%) final reports with co-primary outcomes explained how results would be interpreted in light of multiplicity; 21/30 (70%) protocols and 20/38 (53%) final reports accounted for co-primary outcomes in power calculations.

CONCLUSION

Co-primary outcomes are common in pragmatic trials; improved transparency around design and analysis decisions involving co-primary outcomes is required.

Characteristics of Randomized Clinical Trials in Surgery From 2008 to 2020: A Systematic Review

IMPORTANCE

Randomized clinical trials (RCTs) provide the highest level of evidence to evaluate 2 or more surgical interventions. Surgical RCTs, however, face unique challenges in design and implementation.

OBJECTIVE

To evaluate the design, conduct, and reporting of contemporary surgical RCTs.

EVIDENCE REVIEW

A literature search performed in the 2 journals with the highest impact factor in general medicine as well as 6 key surgical specialties was conducted to identify RCTs published between 2008 and 2020. All RCTs describing a surgical intervention in both experimental and control arms were included. The quality of included data was assessed by establishing an a priori protocol containing all the details to extract. Trial characteristics, fragility index, risk of bias (Cochrane Risk of Bias 2 Tool), pragmatism (Pragmatic Explanatory Continuum Indicator Summary 2 [PRECIS-2]), and reporting bias were assessed.

FINDINGS

A total of 388 trials were identified. Of them, 242 (62.4%) were registered; discrepancies with the published protocol were identified in 81 (33.5%). Most trials used superiority design (329 [84.8%]), and intention-to-treat as primary analysis (221 [56.9%]) and were designed to detect a large treatment effect (50.0%; interquartile range [IQR], 24.7%-63.3%). Only 123 trials (31.7%) used major clinical events as the primary outcome. Most trials (303 [78.1%]) did not control for surgeon experience; only 17 trials (4.4%) assessed the quality of the intervention. The median sample size was 122 patients (IQR, 70-245 patients). The median follow-up was 24 months (IQR, 12.0-32.0 months). Most trials (211 [54.4%]) had some concern of bias and 91 (23.5%) had high risk of bias. The mean (SD) PRECIS-2 score was 3.52 (0.65) and increased significantly over the study period. Most trials (212 [54.6%]) reported a neutral result; reporting bias was identified in 109 of 211 (51.7%). The median fragility index was 3.0 (IQR, 1.0-6.0). Multiplicity was detected in 175 trials (45.1%), and only 35 (20.0%) adjusted for multiple comparisons.

CONCLUSIONS AND RELEVANCE

In this systematic review, the size of contemporary surgical trials was small and the focus was on minor clinical events. Trial registration remained suboptimal and discrepancies with the published protocol and reporting bias were frequent. Few trials controlled for surgeon experience or assessed the quality of the intervention.