Evaluation of dental trials comparing baseline differences using p values

Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy

BACKGROUND

The 'Table 1 Fallacy' refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials.

METHODS

From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy.

RESULTS

A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18-0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19-0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03-0.13; multiplicity-corrected P < 0.0001).

CONCLUSIONS

This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.

Evaluation of randomised controlled trials published in Indian specialty dental journals for statistical testing of baseline differences: A meta-epidemiological study

Background

In randomised controlled trials (RCTs), the application of a test of significance to compare the baseline differences between the intervention groups is a common practice, though it has been condemned by many researchers.

Objective

This study aimed to assess the proportion of RCTs on human participants comparing the baseline differences between intervention groups using the test of significance in nine dental specialty journals published in India and to estimate the proportion of studies reporting baseline demographic and clinical characteristics in a table.

Materials and Methods

RCTs published in nine dental journals published by dental specialty associations of India were screened. A literature search was limited to the time duration of five years from 2017 to 2021.

Results

The authors analysed 326 RCTs. Of 326 RCTs published, 237 RCTs (72.7%) did not report the baseline demographic and clinical characteristics table. Tests of significance were applied to compare baseline differences between the intervention arms in 148 (45.4%) RCTs published.

Conclusions

Although criticised by the Consolidated Standards of Reporting Trials (CONSORT) statement, the majority of the trials published in dental specialty journals failed to avoid comparison of baseline differences with significance test and failed to report baseline characteristic table.

The prevalence of and factors influencing statistical testing of baseline characteristics in randomized controlled trials published in high-impact orthodontic journals

INTRODUCTION

When proper randomization has been undertaken, statistical testing of baseline characteristics between participants in trial arms in randomized controlled trials (RCTs) is not required. This investigation aimed to assess the prevalence of statistical testing of baseline differences in orthodontic RCTs. Factors influencing the undertaking of this analysis were explored.

METHODS

Orthodontic RCTs published between January 1, 2017 and December 31, 2021 in 5 orthodontic journals were identified. To determine if statistical testing of baseline differences had been undertaken, each article was reviewed in detail to identify the reporting of P values and the term "significant difference" in the table of characteristics, the table legends, and the results section of each included RCT. Trial characteristics at the RCT level were extracted. Frequency distributions were calculated for the included trial characteristics. Significant predictors from the univariate analysis were used to construct a multivariable Bayesian logistic regression model.

RESULTS

One hundred and thirty-two RCTs were analyzed. Significance testing of baseline characteristics was undertaken in 50% (66/132) of RCTs. At a journal level, significance testing at baseline was infrequently undertaken in RCTs published in the American Journal of Orthodontics and Dentofacial Orthopedics (AJODO). Compared with 2017, RCTs published in 2018 (odds ratio [OR], 5.57; 95% credible interval [CrI], 1.33-25.69), 2019 (OR, 17.82; 95% CrI, 4.41-82.11), 2020 (OR, 6.48; 95% CrI, 1.72-27.12), and 2021 (OR, 3.24, 95% CrI, 0.81-14.01) had higher odds of significance testing at baseline. RCTs published in the European Journal of Orthodontics (OR, 5.31; 95% CrI, 1.79-17.04), Progress in Orthodontics (OR, 5.00; 95% CrI, 0.97-28.43), Orthodontics and Craniofacial Research (OR, 6.49; 95% CrI, 1.04-46.10), and Angle Orthodontist (OR, 12.30; 95% CrI, 3.27-51.44) had higher odds of significant testing at baseline testing compared with AJODO.

CONCLUSIONS

Statistical testing of baseline differences is common in orthodontic RCTs. Trials published in AJODO had the lowest incidence of statistical testing of baseline differences. RCTs published between 2018-2021 had higher odds of significance testing at baseline than in 2017. Per the consolidated standards of reporting trials guidelines, this practice should be discouraged as it can be misleading and unnecessary.

Annual review of selected scientific literature: A report of the Committee on Scientific Investigation of the American Academy of Restorative Dentistry

This comprehensive review of the 2019 restorative dental literature is offered to inform busy dentists regarding remarkable publications and noteworthy progress made in the profession. Developed by the Scientific Investigation Committee of the American Academy of Restorative Dentistry, each author brings discipline-specific expertise to 1 of 8 sections of the report: (1) prosthodontics; (2) periodontics, alveolar bone, and peri-implant tissues; (3) implant dentistry; (4) dental materials and therapeutics; (5) occlusion and temporomandibular disorders; (6) sleep-related breathing disorders; (7) oral medicine and oral and maxillofacial surgery; and (8) dental caries and cariology. The report targets important information likely to influence day-to-day dental treatment decisions. Each review is not intended to stand alone but to update interested readers so that they may visit source material when greater detail is desired. As the profession moves toward evidence-based clinical decision-making, an incredible volume of potentially valuable dental literature continues to increase. It is the intention of this review and its authors to provide assistance in negotiating the extensive dental literature published in 2019. It is our hope that readers find this work useful in the clinical management of dental patients.