Noninferiority margins exceed superiority effect estimates for mortality in cardiovascular trials in high-impact journals

OBJECTIVES

To determine, in a sample of noninferiority trials, the noninferiority margins (NIMs) and their relation to effect estimates in superiority trials, the rationale being that in general NIMs should be no larger than effects declared important in superiority trials.

STUDY DESIGN AND SETTING

To identify cardiovascular trials published in high-impact journals with a statistically significant primary outcome involving mortality, we systematically searched PubMed, Embase, and MEDLINE databases (January 2015-July 2020). We documented the NIMs and determined the proportion of trials with NIMs larger than the median effect estimates across superiority trials.

RESULTS

From 1,477 screened titles, 65 (39 noninferiority, 26 superiority) trials proved eligible. The NIMs ranged from risk difference of 0.54-10%. The effect estimate across superiority trials was a median risk difference of 2.1% (interquartile range 1.5-4.9); the NIM was larger than 2.1% in 28 (71.8%) noninferiority trials and larger than 1.5%, which was the lower bound of the interquartile range in 32 (82.1%) noninferiority trials.

CONCLUSION

The very wide range of noninferiority margins and the proportion above a threshold that most would consider an important mortality reduction suggest that clinicians and guideline panels should focus on study results, paying little attention to authors' noninferiority margins.

6 versus 12 months of adjuvant trastuzumab in HER2+ early breast cancer: A systematic review and meta-analysis

BACKGROUND

Adjuvant trastuzumab improves survival outcomes of human epidermal receptor 2 positive early breast cancer patients. Currently, administration of 12 months adjuvant trastuzumab is the standard therapy. However, whether 6 months treatment is non-inferior to the standard 12 months treatment remains controversial.

METHODS

Relevant records were searched in PubMed, Cochrane Library, Web of Science, and EMBASE through Jan 14, 2020. Pooled hazard ratios (HRs) and 95% confidence intervals (CIs) for disease-free survival (DFS) and overall survival (OS) were meta-analyzed. The primary endpoint was DFS with a non-inferiority hazard margin of 1.2 and the second was OS with 1.43.

RESULTS

Three randomized clinical studies met the inclusion criteria, including 3974 patients in 6 months group and 3976 in 12 months group. HR for DFS was 1.18 (95% CI 0.97-1.44, P = .09), with the non-inferiority margin comprised in the 95% CI. HR for OS was 1.14 (95% CI 0.98-1.32, P= .08), whereas the upper limit of 95% CI did not exceed the non-inferiority hazard margin.

CONCLUSION

Our analysis failed to show that 6 months treatment was non-inferior to 12 months treatment in improving the DFS. Although the non-inferiority of the 6-month adjuvant trastuzumab treatment was found for OS, considering that breast cancer patients should receive additional systematic therapies when disease progression or relapse happens, we suggest that 12 months adjuvant trastuzumab treatment should remain the standard therapeutic strategy for patients with early human epidermal receptor 2 positive breast cancer.

Use of Mortality as an Endpoint in Noninferiority Trials May Lead to Ethically Problematic Conclusions

BACKGROUND

Noninferiority trials are becoming more common. Their design often requires investigators to "trade" a secondary benefit for efficacy. Use of mortality as an outcome of interest leads to important ethical conflicts whereby researchers must establish a minimal clinically important difference for mortality, a process which has the potential to result in problematic conclusions.

OBJECTIVE

We sought to investigate the frequency of the use of mortality as an outcome in noninferiority trials, as well as to determine the average pre-specified noninferiority ("delta") values.

DESIGN

We searched MEDLINE for reports of parallel-group randomized controlled noninferiority trials published in five high-impact general medical journals.

MAIN OUTCOME MEASURES

Data abstracted from articles including trial design parameters, results, and interpretation of results based on CONSORT recommendations.

RESULTS

One hundred seventy-three manuscripts reporting 196 noninferiority comparisons were included in our analysis. Of these, over a third (67 trials) used mortality either as their sole endpoint (11 trials) or as part of a composite endpoint (56 trials). Nine trials were consort A, 21 trials consort B, 19 trials consort C, 12 were consort F, 4 consort G, and 2 were consort H. Four analyses showed statistically significant more deaths in the new treatment arm, while meeting consort criteria as "inconclusive" (consort G), (Behringer et al. in Lancet. 385(9976):1418-1427, 2015; Kaul et al. in N Engl J Med. 373(18):1709-1719, 2015; Bwakura-Dangarembizi et al. in N Engl J Med. 370(1):41-53, 2014) and thirteen trials utilizing mortality as an endpoint and had an absolute increase of > 3%, and six had an absolute increase of > 5%.

CONCLUSIONS

The use of mortality as an outcome in noninferiority trials is not rare and scenarios where the new treatment is statistically worse, but a conclusion of noninferiority or inconclusive do occur. We highlight these issues and propose simple steps to reduce the risk of ethically dubious conclusions.

Some issues for the evaluation of noninferiority trials

Although published noninferiority trials (NITs) generally conclude that the experimental intervention being studied is noninferior compared with standard therapy or active control, NIT quality is often not satisfactory. We have proposed 14 questions to assist in evaluating the clinical evidence of the experimental versus standard therapy. The aim of these questions is to critically appraise NITs and support proper interpretation of study results. Readers should not only consider whether the confidence interval of the primary effect measure falls within the prespecified noninferiority margin (thus concluding noninferiority), but also assess the similarities between primary and secondary outcomes for the experimental and standard therapy. To conclude noninferiority conceptually is to synthesize evidence from both the current NIT comparing experimental therapy with standard therapy and historical data comparing standard therapy with placebo control. Therefore, readers should use external data sources (e.g., historical data) to validate the study design (e.g., selection of standard therapy, effect measure and the noninferiority margin), and assess the uncertainty of findings due to differences between the observed and expected incidence rates, follow-up time, effects of adjuvant therapy and the secondary outcomes of therapies. Following an explanation of the 14 questions, we then apply the questions to a NIT on intraoperative radiation therapy for early stage breast cancer, as an example.

The Impact of the Underlying Risk in Control Group and Effect Measures in Non-Inferiority Trials With Time-to-Event Data: A Simulation Study

Background

We designed a simulation study to assess how the conclusions of a non-inferiority trial (NIT) will change if the observed risk is different from the expected risk.

Methods

We simulated Weibull distribution time-to-event data with a true hazard ratio (HR) being equal or close to 1. The empirical margins and sample size of a hypothetical trial were chosen based on a systematic review. Setting the significance level at 5% for the two-sided confidence interval (CI), we examined the statistical power (i.e., the probabilities of the upper limit of the 95% CI falling within the margin) of using two measures at various underlying risk in the control group.

Results

Using the empirical margins, HRs of 1.2, 1.35 or 1.5, the statistical power is lower than 0.22 when the underlying risk in the control group is less than 10%, but the power increases along with the higher underlying risk. The predicted upper limit of the 95% CI of the difference in two Kaplan-Meier estimators (DTKME) is low when risk is low (< 20%) or high (> 80%), but reaches the highest value when risk is around 50%. When the underlying risk in the control group is lower than 10%, measures of DTKME resulted in much higher power than HR.

Conclusions

When HR is the effect measure, the probability of concluding non-inferiority will increase as the underlying risk in the control group increases. When DTKME is the effect measure, the probability of concluding non-inferiority will decrease as the underlying risk in the control increases. In this case, the probability of concluding non-inferiority is at a minimum when the control risk reaches about 50%. When the risk in the control arm is less than 10%, the conclusion of an NIT is sensitive to the choice of effect measure.

Empirical Consequences of Current Recommendations for the Design and Interpretation of Noninferiority Trials

BACKGROUND

Noninferiority trials are increasingly common, though they have less standardized designs and their interpretation is less familiar to clinicians than superiority trials.

OBJECTIVE

To empirically evaluate a cohort of noninferiority trials to determine 1) their interpretation as recommended by CONSORT, 2) choice of alpha threshold and its sidedness, and 3) differences between methods of analysis such as intention-to-treat and per-protocol.

DESIGN

We searched MEDLINE for parallel-group randomized controlled noninferiority trials published in the five highest-impact general medical journals between 2011 and 2016.

MAIN MEASURES

Data abstracted included trial design parameters, results, and interpretation of results based on CONSORT recommendations.

KEY RESULTS

One hundred sixty-three trials and 182 noninferiority comparisons were included in our analysis. Based on CONSORT-recommended interpretation, 79% of experimental therapies met criteria for noninferiority, 13% met criteria for superiority, 20% were declared inconclusive, and 2% met criteria for inferiority. However, for 12% of trials, the experimental therapy was statistically significantly worse than the active control, but CONSORT recommended an interpretation of inconclusive or noninferior. A two-sided alpha equivalent of greater than 0.05 was used in 34% of the trials, and in five of these trials, the use of a standard two-sided alpha of 0.05 led to changes in the interpretation of results that disfavored the experimental therapy. In four of the five comparisons where different methods of analysis (e.g., intention-to-treat and per-protocol) yielded different results, the intention-to-treat analysis was the more conservative. In 11% of trials, a secondary advantage of the new therapy was neither reported nor could it be inferred by reviewers.

CONCLUSIONS

In this cohort, the design and interpretation of noninferiority trials led to significant and systematic bias in favor of the experimental therapy. Clinicians should exercise caution when interpreting these trials. Future trials may be more reliable if design parameters are standardized.

Antibiotic treatment for Clostridium difficile-associated diarrhoea in adults

BACKGROUND

Clostridium difficile (C. difficile) is recognized as a frequent cause of antibiotic-associated diarrhoea and colitis. This review is an update of a previously published Cochrane review.

OBJECTIVES

The aim of this review is to investigate the efficacy and safety of antibiotic therapy for C. difficile-associated diarrhoea (CDAD), or C. difficile infection (CDI), being synonymous terms.

SEARCH METHODS

We searched MEDLINE, EMBASE, CENTRAL and the Cochrane IBD Group Specialized Trials Register from inception to 26 January 2017. We also searched clinicaltrials.gov and clinicaltrialsregister.eu for ongoing trials.

SELECTION CRITERIA

Only randomised controlled trials assessing antibiotic treatment for CDI were included in the review.

DATA COLLECTION AND ANALYSIS

Three authors independently assessed abstracts and full text articles for inclusion and extracted data. The risk of bias was independently rated by two authors. For dichotomous outcomes, we calculated the risk ratio (RR) and corresponding 95% confidence interval (95% CI). We pooled data using a fixed-effect model, except where significant heterogeneity was detected, at which time a random-effects model was used. The following outcomes were sought: sustained symptomatic cure (defined as initial symptomatic response and no recurrence of CDI), sustained bacteriologic cure, adverse reactions to the intervention, death and cost.

MAIN RESULTS

Twenty-two studies (3215 participants) were included. The majority of studies enrolled patients with mild to moderate CDI who could tolerate oral antibiotics. Sixteen of the included studies excluded patients with severe CDI and few patients with severe CDI were included in the other six studies. Twelve different antibiotics were investigated: vancomycin, metronidazole, fusidic acid, nitazoxanide, teicoplanin, rifampin, rifaximin, bacitracin, cadazolid, LFF517, surotomycin and fidaxomicin. Most of the studies were active comparator studies comparing vancomycin with other antibiotics. One small study compared vancomycin to placebo. There were no other studies that compared antibiotic treatment to a placebo or a 'no treatment' control group. The risk of bias was rated as high for 17 of 22 included studies. Vancomycin was found to be more effective than metronidazole for achieving symptomatic cure. Seventy-two per cent (318/444) of metronidazole patients achieved symptomatic cure compared to 79% (339/428) of vancomycin patients (RR 0.90, 95% CI 0.84 to 0.97; moderate quality evidence). Fidaxomicin was found to be more effective than vancomycin for achieving symptomatic cure. Seventy-one per cent (407/572) of fidaxomicin patients achieved symptomatic cure compared to 61% (361/592) of vancomycin patients (RR 1.17, 95% CI 1.04 to 1.31; moderate quality evidence). Teicoplanin may be more effective than vancomycin for achieving a symptomatic cure. Eightly-seven per cent (48/55) of teicoplanin patients achieved symptomatic cure compared to 73% (40/55) of vancomycin patients (RR 1.21, 95% CI 1.00 to 1.46; very low quality evidence). For other comparisons including the one placebo-controlled study the quality of evidence was low or very low due to imprecision and in many cases high risk of bias because of attrition and lack of blinding. One hundred and forty deaths were reported in the studies, all of which were attributed by study authors to the co-morbidities of the participants that lead to acquiring CDI. Although many other adverse events were reported during therapy, these were attributed to the participants' co-morbidities. The only adverse events directly attributed to study medication were rare nausea and transient elevation of liver enzymes. Recent cost data (July 2016) for a 10 day course of treatment shows that metronidazole 500 mg is the least expensive antibiotic with a cost of USD 13 (Health Warehouse). Vancomycin 125 mg costs USD 1779 (Walgreens for 56 tablets) compared to fidaxomicin 200 mg at USD 3453.83 or more (Optimer Pharmaceuticals) and teicoplanin at approximately USD 83.67 (GBP 71.40, British National Formulary).

AUTHORS' CONCLUSIONS

No firm conclusions can be drawn regarding the efficacy of antibiotic treatment in severe CDI as most studies excluded patients with severe disease. The lack of any 'no treatment' control studies does not allow for any conclusions regarding the need for antibiotic treatment in patients with mild CDI beyond withdrawal of the initiating antibiotic. Nonetheless, moderate quality evidence suggests that vancomycin is superior to metronidazole and fidaxomicin is superior to vancomycin. The differences in effectiveness between these antibiotics were not too large and the advantage of metronidazole is its far lower cost compared to the other two antibiotics. The quality of evidence for teicoplanin is very low. Adequately powered studies are needed to determine if teicoplanin performs as well as the other antibiotics. A trial comparing the two cheapest antibiotics, metronidazole and teicoplanin, would be of interest.

Sample Size Estimation for Non-Inferiority Trials: Frequentist Approach versus Decision Theory Approach

BACKGROUND

Non-inferiority trials are performed when the main therapeutic effect of the new therapy is expected to be not unacceptably worse than that of the standard therapy, and the new therapy is expected to have advantages over the standard therapy in costs or other (health) consequences. These advantages however are not included in the classic frequentist approach of sample size calculation for non-inferiority trials. In contrast, the decision theory approach of sample size calculation does include these factors. The objective of this study is to compare the conceptual and practical aspects of the frequentist approach and decision theory approach of sample size calculation for non-inferiority trials, thereby demonstrating that the decision theory approach is more appropriate for sample size calculation of non-inferiority trials.

METHODS

The frequentist approach and decision theory approach of sample size calculation for non-inferiority trials are compared and applied to a case of a non-inferiority trial on individually tailored duration of elastic compression stocking therapy compared to two years elastic compression stocking therapy for the prevention of post thrombotic syndrome after deep vein thrombosis.

RESULTS

The two approaches differ substantially in conceptual background, analytical approach, and input requirements. The sample size calculated according to the frequentist approach yielded 788 patients, using a power of 80% and a one-sided significance level of 5%. The decision theory approach indicated that the optimal sample size was 500 patients, with a net value of €92 million.

CONCLUSIONS

This study demonstrates and explains the differences between the classic frequentist approach and the decision theory approach of sample size calculation for non-inferiority trials. We argue that the decision theory approach of sample size estimation is most suitable for sample size calculation of non-inferiority trials.

Comparing vaccines: a systematic review of the use of the non-inferiority margin in vaccine trials

BACKGROUND

Non-inferiority (NI) randomized controlled trials (RCTs) aim to demonstrate that a new treatment is no worse than a comparator that has already shown its efficacy over placebo within a pre-specified margin. However, clear guidelines on how the NI margin should be determined are lacking for vaccine trials. A difference (seroprevalence/risk) of 10% or a geometric mean titre/concentration (GMT) ratio of 1.5 or 2.0 in antibody levels is implicitly recommended for vaccine trials. We aimed to explore which NI margins were used in vaccine RCTs and how they were determined.

METHODS

A systematic search for NI vaccine RCTs yielded 177 eligible articles. Data were extracted from these articles using a standardized form and included general characteristics and characteristics specific for NI trials. Relations between the study characteristics and the NI margin used were explored.

RESULTS

Among the 143 studies using an NI margin based on difference (n=136 on immunogenicity, n=2 on efficacy and n=5 on safety), 66% used a margin of 10%, 23% used margins lower than 10% (range 1-7.5%) and 11% used margins larger than 10% (range 11.5-25%). Of the 103 studies using a NI margin based on the GMT ratio, 50% used a margin of 0.67/1.5 and 49% used 0.5/2.0. As observed, 85% of the studies did not discuss the method of margin determination; and 19% of the studies lacked a confidence interval or p-value for non-inferiority.

CONCLUSION

Most NI vaccine RCTs used an NI margin of 10% for difference or a GMT ratio of 1.5 or 2.0 without a clear rationale. Most articles presented enough information for the reader to make a judgement about the NI margin used and the conclusions. The reporting on the design, margins used and results of NI vaccine trials could be improved; more explicit guidelines may help to achieve this end.

Comparison of noninferiority margins reported in protocols and publications showed incomplete and inconsistent reporting

OBJECTIVES

To compare noninferiority margins defined in study protocols and trial registry records with margins reported in subsequent publications.

STUDY DESIGN AND SETTING

Comparison of protocols of noninferiority trials submitted 2001 to 2005 to ethics committees in Switzerland and The Netherlands with corresponding publications and registry records. We searched MEDLINE via PubMed, the Cochrane Controlled Trials Register (Cochrane Library issue 01/2012), and Google Scholar in September 2013 to identify published reports, and the International Clinical Trials Registry Platform of the World Health Organization in March 2013 to identify registry records. Two readers recorded the noninferiority margin and other data using a standardized data-abstraction form.

RESULTS

The margin was identical in study protocol and publication in 43 (80%) of 54 pairs of study protocols and articles. In the remaining pairs, reporting was inconsistent (five pairs, 9%), or the noninferiority margin was either not reported in the publication (five pairs, 9%) or not defined in the study protocol (one pair). The confidence interval or the exact P-value required to judge whether the result was compatible with noninferior, inferior, or superior efficacy was reported in 43 (80%) publications. Complete and consistent reporting of both noninferiority margin and confidence interval (or exact P-value) was present in 39 (72%) protocol-publication pairs. Twenty-nine trials (54%) were registered in trial registries, but only one registry record included the noninferiority margin.

CONCLUSION

The reporting of noninferiority margins was incomplete and inconsistent with study protocols in a substantial proportion of published trials, and margins were rarely reported in trial registries.

Choice of non-inferiority (NI) margins does not protect against degradation of treatment effects on an average--an observational study of registered and published NI trials

OBJECTIVE

NI margins have to be chosen appropriately to control the risk of degradation of treatment effects in non-inferiority (NI) trials. We aimed to study whether the current choice of NI margins protects sufficiently against a degradation of treatment effect on an average.

STUDY DESIGN AND SETTING

NI trials reflecting current practice were assembled and for each trial, the NI margin was translated into a likelihood of degradation. The likelihood of degradation was calculated as the conditional probability of a treatment being harmful given that it is declared non-inferior in the trial, using simulation. Its distribution among the NI trials was then studied to assess the potential risk of degradation.

RESULTS

The median (lower/upper quartile) NI margin among 112 binary outcome NI trials corresponded to an odds ratio of 0.57(0.45, 0.66), while among 38 NI trials with continuous outcome, to a Cohen's d of -0.42(-0.54, -0.31) and a hazard ratio of 0.82(0.73, 0.86) among 24 survival outcome NI trials. Overall, the median likelihood of degradation was 56% (45%, 62%).

CONCLUSION

Only two fifths of the current NI trials had a likelihood of degradation lower than 50%, suggesting that, in majority of the NI trials, there is no sufficient protection against degradation on an average. We suggest a third hurdle for the choice of NI margins, thus contributing a sufficient degree of protection.

Assessing additional benefit in noninferiority trials

A method for simultaneously assessing noninferiority with respect to efficacy and superiority with respect to another endpoint in two-arm noninferiority trials is presented. The procedure controls both the average type I error rate for the intersection-union test problem and the frequentist type I error rate for the noninferiority test by α while allowing an increased level for the superiority test. For normally distributed outcomes, two methods are presented to deal with the uncertainty about the correlation between the endpoints which defines the adjusted levels. The operating characteristics of these procedures are investigated. Furthermore, the sample size required when applying the proposed method is compared with that of alternative procedures. Application of the method in the situation of binary endpoints and mixed normal and binary endpoints, respectively, is sketched. An illustrative example is provided demonstrating implementation of the proposed approach in a clinical trial.

Ciclesonide versus other inhaled corticosteroids for chronic asthma in children

BACKGROUND

Inhaled corticosteroids (ICS) are the cornerstone of asthma maintenance treatment in children. Particularly among parents, there is concern about the safety of ICS as studies in children have shown reduced growth. Small-particle-size ICS targeting the smaller airways have improved lung deposition and effective asthma control might be achieved at lower daily doses.Ciclesonide is a relatively new ICS. This small-particle ICS is a pro-drug that is converted in the airways to an active metabolite and therefore with potentially less local (throat infection) and systemic (reduced growth) side effects. It can be inhaled once daily, thereby possibly improving adherence.

OBJECTIVES

To assess the efficacy and adverse effects of ciclesonide compared to other ICS in the management of chronic asthma in children.

SEARCH METHODS

We searched the Cochrane Airways Group Register of trials with pre-defined terms. Additional searches of MEDLINE (via PubMed), EMBASE and Clinical study results.org were undertaken. Searches are up to date to 7 November 2012.

SELECTION CRITERIA

Randomised controlled parallel or cross-over studies were eligible for the review. We included studies comparing ciclesonide with other corticosteroids both at nominally equivalent doses or lower doses of ciclesonide.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed trial quality and extracted data. Study authors were contacted for additional information. Adverse effects information was collected from the trials.

MAIN RESULTS

Six studies were included in this review (3256 children, 4 to 17 years of age). Two studies were published as conference abstracts only. Ciclesonide was compared to budesonide and fluticasone.Ciclesonide compared to budesonide (dose ratio 1:2): asthma symptoms and adverse effect were similar in both groups. Pooled results showed no significant difference in children who experience an exacerbation (risk ratio (RR) 2.20, 95% confidence interval (CI) 0.75 to 6.43). Both studies reported that 24-hour urine cortisol levels showed a statistically significant decrease in the budesonide group compared to the ciclesonide group.Ciclesonide compared to fluticasone (dose ratio 1:1): no significant differences were found for the outcome asthma symptoms. Pooled results showed no significant differences in number of patients with exacerbations (RR 1.37, 95% CI 0.58 to 3.21) and data from a study that could not be pooled in the meta-analysis reported similar numbers of patients with exacerbations in both groups. None of the studies found a difference in adverse effects. No significant difference was found for 24-hour urine cortisol levels between the groups (mean difference 0.54 nmol/mmol, 95% CI -5.92 to 7.00).Ciclesonide versus fluticasone (dose ratio 1:2) was assessed in one study and showed similar results between the two corticosteroids for asthma symptoms. The number of children with exacerbations was significantly higher in the ciclesonide group (RR 3.57, 95% CI 1.35 to 9.47). No significant differences were found in adverse effects (RR 0.98, 95% CI 0.81 to 1.14) and 24-hour urine cortisol levels (mean difference 1.15 nmol/mmol, 95% CI 0.07 to 2.23).The quality of evidence was judged 'low' for the outcomes asthma symptoms and adverse events and 'very low' for the outcome exacerbations for ciclesonide versus budesonide (dose ratio 1:1). The quality of evidence was graded 'moderate' for the outcome asthma symptoms, 'very low' for the outcome exacerbations and 'low' for the outcome adverse events for ciclesonide versus fluticasone (dose ratio 1:1). For ciclesonide versus fluticasone (dose ratio 1:2) the quality was rated 'low' for the outcome asthma symptoms and 'very low' for exacerbations and adverse events (dose ratio 1:2).

AUTHORS' CONCLUSIONS

An improvement in asthma symptoms, exacerbations and side effects of ciclesonide versus budesonide and fluticasone could be neither demonstrated nor refuted and the trade-off between benefits and harms of using ciclesonide instead of budesonide or fluticasone is unclear. The resource use or costs of different ICS should therefore also be considered in final decision making. Longer-term superiority trials are needed to identify the usefulness and safety of ciclesonide compared to other ICS. Additionally these studies should be powered for patient relevant outcomes (exacerbations, asthma symptoms, quality of life and side effects). There is a need for studies comparing ciclesonide once daily with other ICS twice daily to assess the advantages of ciclesonide being a pro-drug that can be administered once daily with possibly increased adherence leading to increased control of asthma and fewer side effects.

About half of the noninferiority trials tested superior treatments: a trial-register based study

OBJECTIVES

A concern that noninferiority (NI) trials pose a risk of degradation of the treatment effects is prevalent. Thus, we aimed to determine the fraction of positive true effects (superiority rate) and the average true effect of current NI trials based on data from registered NI trials.

STUDY DESIGN AND SETTING

All NI trials carried out between 2000 and 2007 analyzing the NI of efficacy as the primary objective and registered in one of the two major clinical trials registers were studied. Having retrieved results from these trials, random effects modeling of the effect estimates was performed to determine the distribution of true effects.

RESULTS

Effect estimates were available for 79 of 99 eligible trials identified. For trials with binary outcome, we estimated a superiority rate of 49% (95% confidence interval = 27-70%) and a mean true log odds ratio of -0.005 (-0.112, 0.102). For trials with continuous outcome, the superiority rate was 58% (41-74%) and the mean true effect as Cohen's d of 0.06 (-0.064, 0.192).

CONCLUSIONS

The unanticipated finding of a positive average true effect and superiority of the new treatment in most NI trials suggest that the current practice of choosing NI designs in clinical trials makes degradation on average unlikely. However, the distribution of true treatment effects demonstrates that, in some NI trials, the new treatment is distinctly inferior.

Paradoxical interpretations of noninferiority studies: violating the excluded middle

Background: The noninferiority of a novel therapy compared with a standard of care is customarily defined by a noninferiority margin derived from an assessment of what would constitute a clinically relevant decrement in efficacy while preserving some of the treatment effect over placebo. Conundrum: If the one-sided 97.5% CI around the difference in the point estimates of efficacy between the two treatments (investigational drug minus comparator drug) does not extend below the prespecified threshold, noninferiority of the new agent to the comparator is typically concluded. In some cases, the corresponding two-sided 95% CI will fall entirely between zero and the noninferiority delta, technically implying inferiority and noninferiority concurrently. Solution: Stipulating that the upper bound of the two-sided confidence interval reach or exceed zero (as well as fall entirely above the noninferiority limit) to establish statistical noninferiority versus the comparator would avoid paradoxical interpretations.

Non-inferiority trials in breast and non-small cell lung cancer: choice of non-inferiority margins and other statistical aspects

BACKGROUND

Determining the non-inferiority margin is an essential step in the design and interpretation of non-inferiority trials, and this margin should be preferably justified on clinical and statistical grounds.

METHODS

After a PubMed search for phase III trials in advanced breast cancer (BC) or non-small cell lung cancer (NSCLC) published between January 1998 and December 2009 in 11 leading journals, non-inferiority trials were selected by manual search of the full papers.

RESULTS

Twenty-four of 195 trials had a primary non-inferiority hypothesis. When the two six-year study periods were compared, there were time trends within BC and NSCLC, with most non-inferiority trials in BC reported in the first six-year period, and vice-versa for NSCLC. The median sample size was larger for non-inferiority than superiority trials (p < 0.01). The choice of a non-inferiority margin was reportedly justified in only five cases. Non-inferiority trials were more likely than superiority trials to yield positive results (p < 0.001), as were trials in breast cancer (p = 0.02).

CONCLUSIONS

Non-inferiority margins for cancer trials appear to be chosen mostly on historical grounds. Since nearly three-quarters of non-inferiority trials achieve their primary objective, the extent to which the choice of margins has influence on trial results remains to be determined.

Effectiveness of protease inhibitor monotherapy versus combination antiretroviral maintenance therapy: a meta-analysis

BACKGROUND

The unparalleled success of combination antiretroviral therapy (cART) is based on the combination of three drugs from two classes. There is insufficient evidence whether simplification to ritonavir boosted protease inhibitor (PI/r) monotherapy in virologically suppressed HIV-infected patients is effective and safe to reduce cART side effects and costs.

METHODS

We systematically searched Medline, Embase, the Cochrane Library, conference proceedings and trial registries to identify all randomised controlled trials comparing PI/r monotherapy to cART in suppressed patients. We calculated in an intention to treat (loss-of follow-up, discontinuation of assigned drugs equals failure) and per-protocol analysis (exclusion of protocol violators following randomisation) and based on three different definitions for virological failure pooled risk ratios for remaining virologically suppressed.

FINDINGS

We identified 10 trials comparing 3 different PIs with cART based on a PI/r plus 2 reverse transcriptase inhibitors in 1189 patients. With the most conservative approach (viral load <50 copies/ml on two consecutive measurements), the risk ratios for viral suppression at 48 weeks of PI/r monotherapy compared to cART were in the ITT analysis 0.94 8 (95% CI 0.89 to 1.00) p = 0.06; risk difference -0.06 (95%CI -0.11 to 0) p = 0.05, p for heterogeneity  = 0.08, I(2) = 43.1%) and in the PP analysis 0.93 ((95%CI 0.90 to 0.97) p<0.001; risk difference -0.07 (95%CI -0.10 to -0.03) p<0.001, p for heterogeneity  = 0.44, I(2) = 0%). Reintroduction of cART in 44 patients with virological failure led in 93% to de-novo viral suppression.

INTERPRETATION

Virologically well suppressed HIV-infected patients have a lower chance to maintain viral suppression when switching from cART to PI/r monotherapy. Failing patients achieve high rates of de-novo viral suppression following reintroduction of reverse transcriptase inhibitors.

Reporting of noninferiority trials was incomplete in trial registries

OBJECTIVE

To examine the registration of noninferiority trials, with a focus on the reporting of study design and noninferiority margins.

STUDY DESIGN AND SETTING

Cross-sectional study of registry records of noninferiority trials published from 2005 to 2009 and records of noninferiority trials in the International Standard Randomized Controlled Trial Number (ISRCTN) or ClinicalTrials.gov trial registries. The main outcome was the proportion of records that reported the noninferiority design and margin.

RESULTS

We analyzed 87 registry records of published noninferiority trials and 149 registry records describing noninferiority trials. Thirty-five (40%) of 87 records from published trials described the trial as a noninferiority trial; only two (2%) reported the noninferiority margin. Reporting of the noninferiority design was more frequent in the ISRCTN registry (13 of 18 records, 72%) compared with ClinicalTrials.gov (22 of 69 records, 32%; P=0.002). Among the 149 records identified in the registries, 13 (9%) reported the noninferiority margin. Only one of the industry-sponsored trial compared with 11 of the publicly funded trials reported the margin (P=0.001).

CONCLUSION

Most registry records of noninferiority trials do not mention the noninferiority design and do not include the noninferiority margin. The registration of noninferiority trials is unsatisfactory and must be improved.