Non-inferiority studies: is ‘better’ the enemy of ‘good enough’?

Assessment of Noninferiority Margins in Cardiovascular Medicine Trials

Background

Noninferiority trials are increasingly common in cardiovascular medicine, but their reporting and interpretation are challenging, particularly when an absolute risk difference is used as noninferiority margin.

Objectives

This study aimed to investigate the effect of using absolute rather than relative noninferiority margins in cardiovascular trials.

Methods

We reviewed noninferiority trials presented at major cardiovascular conferences from 2015 to 2022 and published within the same period. Based on the actual versus anticipated event rates in the control group, we recalculated the absolute noninferiority margin and re-assessed the trial results. The primary outcome of interest was the proportion of trials with a different interpretation after recalculation. Additionally, we analyzed the conclusion statements of these trials to determine if cautionary notes for the interpretation of study results were included.

Results

We analyzed a total of 768 trials, of which 88 had a noninferiority design and 66 used an absolute noninferiority margin. Of 48 comparisons from 45 trials qualifying for the analysis, 11 (22.9%) had divergent results after recalculation of the absolute noninferiority margin based on the observed rather than anticipated event rate. Ten trials originally claiming noninferiority, did not meet it after the margin recalculation. All of them did not include statements suggesting cautionary interpretation of the study results in the conclusion section. Compared with the other trials, these displayed a larger median difference between anticipated and recalculated noninferiority margins (44.7% [IQR: 38.6%-56.7%] vs 15.3% [IQR: -1.5% to 28.9%]; P < 0.001).

Conclusions

Recalculating noninferiority margins based on actual event rates, rather than anticipated ones, led to different outcomes in approximately 1 out of 4 cardiovascular trials, with most divergent trials lacking cautionary interpretation. These findings emphasize the importance of using or supplementing the relative noninferiority margin, particularly in studies with significant deviations between observed and expected event rates. This underscores the critical need for enhanced methodological and reporting standards in noninferiority trials, especially those employing absolute margins.

Interpreting the results of clinical trials, embracing uncertainty: A Bayesian approach

Most clinical trials use null hypothesis significance testing with frequentist statistical inference to report P values and confidence intervals for effect estimates. This method leads to a dichotomisation of results as 'significant' or 'non-significant'. A more nuanced interpretation may often be considered and in particular when the majority of the confidence interval for the effect estimate suggests benefit or harm. In contrast to the frequentist dichotomised approach based on a P value, the application of Bayesian statistics allocates credibility to a continuous spectrum of possibilities and for this reason a Bayesian approach to inference is often warranted as it will incorporate uncertainty when updating our current belief with information from a new trial. The use of Bayesian statistics is introduced in this paper for a hypothetical sepsis trial with worked examples in the R language for Statistical Computing environment and the open-source statistical software JASP. It is hoped that this general introduction to Bayesian inference stimulates some interest and confidence among clinicians to consider applying these methods to the interpretation of new evidence for interventions relevant to anaesthesia and intensive care medicine.