Preservation of effect and the regulatory approval of new treatments on the basis of non-inferiority trials

A confirmatory basket design considering non-inferiority and superiority testing

For multiple rare diseases as defined by a common biomarker signature, or a disease with multiple disease subtypes of low frequency, it is often possible to provide confirmatory evidence for these disease or subtypes (baskets) as a combined group. A novel drug, as a second generation, may have marginal improvement in efficacy overall but superior efficacy in some baskets. In this situation, it is appealing to test hypotheses of both non-inferiority overall and superiority on certain baskets. The challenge is designing a confirmatory study efficient to address multiple questions in one trial. A two-stage adaptive design is proposed to test the non-inferiority hypothesis at the interim stage, followed by pruning and pooling before testing a superiority hypothesis at the final stage. Such a design enables an efficient and novel registration pathway, including an early claim of non-inferiority followed by a potential label extension with superiority on certain baskets and an improved benefit-risk profile demonstrated by longer term efficacy and safety data. Operating characteristics of this design are examined by simulation studies, and its appealing features make it ready for use in a confirmatory setting, especially in emerging markets, where both the need and the possibility for efficient use of resources may be the greatest.

Interpretation of active-control randomised trials: the case for a new analytical perspective involving averted events

Active-control trials, where an experimental treatment is compared with an established treatment, are performed when the inclusion of a placebo control group is deemed to be unethical. For time-to-event outcomes, the primary estimand is usually the rate ratio, or the closely-related hazard ratio, comparing the experimental group with the control group. In this article we describe major problems in the interpretation of this estimand, using examples from COVID-19 vaccine and HIV pre-exposure prophylaxis trials. In particular, when the control treatment is highly effective, the rate ratio may indicate that the experimental treatment is clearly statistically inferior even when it is worthwhile from a public health perspective. We argue that it is crucially important to consider averted events as well as observed events in the interpretation of active-control trials. An alternative metric that incorporates this information, the averted events ratio, is proposed and exemplified. Its interpretation is simple and conceptually appealing, namely the proportion of events that would be averted by using the experimental treatment rather than the control treatment. The averted events ratio cannot be directly estimated from the active-control trial, and requires an additional assumption about either: (a) the incidence that would have been observed in a hypothetical placebo arm (the counterfactual incidence) or (b) the efficacy of the control treatment (relative to no treatment) that pertained in the active-control trial. Although estimation of these parameters is not straightforward, this must be attempted in order to draw rational inferences. To date, this method has been applied only within HIV prevention research, but has wider applicability to treatment trials and other disease areas.

Evaluating a novel, low-cost technique for cervical-spine immobilization for application in resource-limited LMICs: a non-inferiority trial

STUDY DESIGN

Non-inferiority trial.

OBJECTIVE

Limited cervical spinal (c-spine) immobilization in resource-limited settings of LMICs suggests alternatives are necessary for patients with traumatic injuries. We propose a novel method of c-spine immobilization using folded towels.

SETTING

Washington University in St. Louis.

METHODS

Using non-inferiority trial design, thirty healthy patients (median age = 22) were enrolled to test the efficacy of folded towels in comparison with rigid cervical collars, foam neck braces, and no immobilization. We measured cervical range of motion (CROM) in six cardinal directions in seated and supine positions. A weighted composite score (CS) was generated to compare immobilization methods. A preserved fraction of 75% was determined for non-inferiority, corresponding to the difference between the median values for CROM between control (no immobilization) and c-collar states.

RESULTS

C-collars reduce median CROM in six cardinal directions in seated and supine positions by an average of -36.83° seated (-17.75° supine) vs. no immobilization. Folded towels and foam neck braces reduced CROM by -27° seated (-16.75° supine) and -14.25° seated (-9.5° supine), respectively. Compared to a 25% non-inferiority margin (permitting an average 9.21° of cervical movement across six cardinal directions), the CS determined folded towels are non-inferior (CS_seated = 0.89, CS_supine = 0.47). Foam neck braces are inferior (CS_seated = 2.35, CS_supine = 2.10). CS > 1 surpassed the non-inferiority margin and were deemed inferior.

CONCLUSIONS

Folded towels are a non-inferior means of immobilizing c-spine in extension and rotation, but not flexion, vs. c-collars. We propose folded towels could be trialed in combination with backboards to deliver affordable and effective prehospital TSCI management in resource-limited settings.

Myths and Methodologies: The use of equivalence and non-inferiority tests for interventional studies in exercise physiology and sport science

NEW FINDINGS

What is the topic of this review? The traditional null-hypothesis test allows making informed decisions about whether an experimental intervention is superior to a control while controlling the Type I and Type II error rates. Equivalence and non-inferiority tests allow determining whether two interventions are similar in efficacy or whether one is not unacceptably worse than the other. What advances does it highlight? Equivalence and non-inferiority designs may better fit the research questions of exercise physiologists and sport scientists when investigating new interventions that have better cost-effectiveness, are safer and easier to implement, or are less demanding than the standard.

ABSTRACT

Exercise physiology and sport science have traditionally made use of the null hypothesis of no difference to make decisions about experimental interventions. This article aims to review current statistical approaches typically used by exercise physiologists and sport scientists for the design and analysis of experimental interventions and to highlight the importance of including equivalence and non-inferiority studies, which address different research questions than deciding whether an effect is present. Firstly, we briefly describe the most common approaches, along with their rationale, to investigate the effects of different interventions. We then discuss the main steps involved in the design and analysis of equivalence and non-inferiority studies, commonly performed in other research fields, with worked examples from exercise physiology and sport science scenarios. Finally, we provide recommendations to exercise physiologists and sport scientists who would like to apply the different approaches in future research. We hope this work will promote the correct use of equivalence and non-inferiority designs in exercise physiology and sport science whenever the research context, conditions, applications, researchers' interests, or reasonable beliefs, justify these approaches. This article is protected by copyright. All rights reserved.

Comparison of Chloroprocaine Versus Lidocaine With Epinephrine, Sodium Bicarbonate, and Fentanyl for Epidural Extension Anesthesia in Elective Cesarean Delivery: A Randomized, Triple-Blind, Noninferiority Study

BACKGROUND

For emergent intrapartum cesarean delivery (CD), the literature does not support the use of any particular local anesthetic solution to extend epidural analgesia to cesarean anesthesia. We hypothesized that 3% chloroprocaine (CP) would be noninferior to a mixture of 2% lidocaine, 150 µg of epinephrine, 2 mL of 8.4% bicarbonate, and 100 µg of fentanyl (LEBF) in terms of onset time to surgical anesthesia.

METHODS

In this single-center randomized noninferiority trial, adult healthy women undergoing CD were randomly assigned to epidural anesthesia with either CP or LEBF. Sensory blockade (pinprick) to T10 was established before operating room (OR) entry for elective CD. On arrival to the OR, participants received the epidural study medications in a standardized manner to simulate the conversion of "epidural labor analgesia to surgical anesthesia." The primary outcome was the time to loss of touch sensation at the T7 level. A noninferiority margin was set at 3 minutes. The secondary outcome was the need for intraoperative analgesia supplementation.

RESULTS

In total, 70 women were enrolled in the study. The mean onset time to achieve a bilateral sensory block to touch at the T7 dermatome level was 655 (standard deviation [SD] = 258) seconds for group CP and 558 (269) seconds for group LEBF, a difference in means of 97 seconds (90% confidence interval [CI], SD = -10.6 to 204; P = .10 for noninferiority). The upper limit of the 90% CI for the mean difference exceeded the prespecified 3-minute noninferiority margin. There was no meaningful difference in the requirement for intraoperative analgesia between the 2 groups.

CONCLUSION

Both anesthetic solutions have a rapid onset of anesthesia when used to extend low-dose epidural sensory block to surgical anesthesia. Data from the current study provide insufficient evidence to confirm that CP is noninferior to LEBF for rapid epidural extension anesthesia for CD, and further research is required to determine noninferiority.

Confidence limits for the averted infections ratio estimated via the counterfactual placebo incidence rate

Objectives

The averted infections ratio (AIR) is a novel measure for quantifying the preservation-of-effect in active-control non-inferiority clinical trials with a time-to-event outcome. In the main formulation, the AIR requires an estimate of the counterfactual placebo incidence rate. We describe two approaches for calculating confidence limits for the AIR given a point estimate of this parameter, a closed-form solution based on a Taylor series expansion (delta method) and an iterative method based on the profile-likelihood.

Methods

For each approach, exact coverage probabilities for the lower and upper confidence limits were computed over a grid of values of (1) the true value of the AIR (2) the expected number of counterfactual events (3) the effectiveness of the active-control treatment.

Results

Focussing on the lower confidence limit, which determines whether non-inferiority can be declared, the coverage achieved by the delta method is either less than or greater than the nominal coverage, depending on the true value of the AIR. In contrast, the coverage achieved by the profile-likelihood method is consistently accurate.

Conclusions

The profile-likelihood method is preferred because of better coverage properties, but the simpler delta method is valid when the experimental treatment is no less effective than the control treatment. A complementary Bayesian approach, which can be applied when the counterfactual incidence rate can be represented as a prior distribution, is also outlined.

Mosaic effectiveness: measuring the impact of novel PrEP methods

Various ongoing trials seek to evaluate long-acting pre-exposure prophylaxis (PrEP) agents by showing that they are non-inferior to daily oral tenofovir disoproxil fumarate and emtricitabine. Trials comparing oral PrEP to new methods examine effectiveness in a setting where only one or the other is provided; however, a new product will probably be delivered in a context where oral PrEP is also available. The effectiveness of a new PrEP product is best measured by its potential effect in a context that also includes oral tenofovir disoproxil fumarate and emtricitabine as an option. We offer an alternative standard for long-acting products-a measure of the effectiveness of the new product in addition to oral tenofovir disoproxil fumarate and emtricitabine as compared with oral PrEP alone. We term this measure mosaic effectiveness. We illustrate scenarios where a novel product can fail to show non-inferiority but show substantial mosaic effectiveness, thus implying the public health value of the novel product even if it is less effective than oral PrEP. Regulatory standards should consider mosaic effectiveness, not just comparative effectiveness. We assert that measurements that combine rigor with public health relevance can accelerate progress against the HIV epidemic.

The Connection between the Averted Infections Ratio and the Rate Ratio in Active-control Trials of Pre-exposure Prophylaxis Agents

The design and analysis of active-control trials to evaluate experimental HIV pre-exposure prophylaxis (PrEP) agents pose serious statistical challenges. We recently proposed a new outcome measure, the averted infections ratio (AIR) – the proportion of infections that would be averted by using the experimental agent rather than the control agent (compared to no intervention). The main aim of the current paper is to examine the mathematical connection between AIR and the HIV incidence rate ratio, the standard outcome measure. We also consider the sample size implications of the choice of primary outcome measure and explore the connection between effectiveness and efficacy under a simplified model of adherence.

Advancing Novel PrEP Products - Alternatives to Non-Inferiority

With the scale-up of HIV pre-exposure prophylaxis (PrEP) with tenofovir (TDF) with or without emtricitabine (FTC), we have entered an era of highly effective HIV prevention with a growing pipeline of potential products to be studied. These studies are likely to be randomized trials with an oral TDF/FTC control arm. These studies require comparison of incident infections and can be time and resource intensive. Conventional approaches for design and analysis active controlled trial can lead to very large sample sizes. We demonstrate the important of assumptions about background infections for interpreting trial results and suggest alternative criteria for demonstrating the efficacy and effectiveness of potential PrEP agents.

Superiority and non-inferiority: two sides of the same coin?

Background

The classification of phase 3 trials as superiority or non-inferiority has become routine, and it is widely accepted that there are important differences between the two types of trial in their design, analysis and interpretation.

Main text

There is a clear rationale for the superiority/non-inferiority framework in the context of regulatory trials. The focus of our article is non-regulatory trials with a public health objective. First, using two examples from infectious disease research, we show that the classification of superiority or non-inferiority trials is not always straightforward. Second, we show that several arguments for different approaches to the design, analysis and interpretation of superiority and non-inferiority trials are unconvincing when examined in detail. We consider, in particular, the calculation of sample size (and the choice of delta or the non-inferiority margin), intention-to-treat versus per-protocol analyses, and one-sided versus two-sided confidence intervals. We argue that the superiority/non-inferiority framework is not just unnecessary but can have a detrimental effect, being a barrier to clear scientific thought and communication. In particular, it places undue emphasis on tests for significance or non-inferiority at the expense of estimation. We emphasise that these concerns apply to phase 3 non-regulatory trials in general, not just to those where the classification of the trial as superiority or non-inferiority is ambiguous.

Conclusions

Guidelines and statistical practice should abandon the sharp division between superiority and non-inferiority phase 3 non-regulatory trials and be more closely aligned to the clinical and public health questions that motivate the trial.

The averted infections ratio: a novel measure of effectiveness of experimental HIV pre-exposure prophylaxis agents

Tenofovir disoproxil fumarate combined with emtricitabine is a highly effective oral pre-exposure prophylaxis (PrEP) agent for preventing the acquisition of HIV. This effectiveness has consequences for the design and analysis of trials assessing experimental PrEP regimens, which now generally include an active-control tenofovir disoproxil fumarate plus emtricitabine group, rather than a placebo group, as a comparator. Herein, we describe major problems in the interpretation of the primary measure of effectiveness proposed for these trials, namely the ratio of HIV incidence in the experimental agent group to that in the active-control group. We argue that valid interpretation requires an assumption about one of two parameters: either the incidence among trial participants had they not received PrEP or the effectiveness of tenofovir disoproxil fumarate plus emtricitabine within the trial. However, neither parameter is directly observed because of the absence of a no-treatment group, thus requiring the use of external evidence or subjective judgment. We propose an alternative measure of effectiveness based on the concept of averted infections, which incorporates one of these parameters. The measure is simple to interpret, has clinical and public health relevance, and is a natural preservation-of-effect criterion for assessing statistical non-inferiority. Its adoption could also allow the use of smaller sample sizes, currently a major barrier to the assessment of experimental PrEP regimens.

Defining the noninferiority margin and analysing noninferiority: An overview

Noninferiority trials are used to assess whether the effect of a new drug is not worse than an active comparator by more than a noninferiority margin. If the difference between the new drug and the active comparator does not exceed this prespecified margin, noninferiority can be concluded. This margin must be specified based on clinical and statistical reasoning; however, it is considered as one of the most challenging steps in the design of noninferiority trials. Regulators recommend that the margin should be defined based on the historical evidence of the active comparator (the latter is often the well-established standard treatment of the disease), which can be performed by different approaches. There are several factors and assumptions that need to be accounted for during the process of defining the margin and during the analysis of noninferiority. Three methods are commonly used to analyse noninferiority trials: the fixed-margin method; the point-estimate method; and the synthesis method. This article provides an overview of analysing noninferiority and choosing the noninferiority margin.

Methods of defining the non-inferiority margin in randomized, double-blind controlled trials: a systematic review

BACKGROUND

There is no consensus on the preferred method for defining the non-inferiority margin in non-inferiority trials, and previous studies showed that the rationale for its choice is often not reported. This study investigated how the non-inferiority margin is defined in the published literature, and whether its reporting has changed over time.

METHODS

A systematic PubMed search was conducted for all published randomized, double-blind, non-inferiority trials from January 1, 1966, to February 6, 2015. The primary outcome was the number of margins that were defined by methods other than the historical evidence of the active comparator. This was evaluated for a time trend. We also assessed the under-reporting of the methods of defining the margin as a secondary outcome, and whether this changed over time. Both outcomes were analyzed using a Poisson log-linear model. Predictors for better reporting of the methods, and the use of the fixed-margin method (one of the historical evidence methods) were also analyzed using logistic regression.

RESULTS

Two hundred seventy-three articles were included, which account for 273 non-inferiority margins. There was no statistically significant difference in the number of margins that were defined by other methods compared to those defined based on the historical evidence (ratio 2.17, 95% CI 0.86 to 5.82, p = 0.11), and this did not change over time. The number of margins for which methods were unreported was similar to those with reported methods (ratio 1.35, 95% CI 0.76 to 2.43, p = 0.31), with no change over time. The method of defining the margin was less often reported in journals with low-impact factors compared to journals with high-impact factors (OR 0.20; 95% CI 0.10 to 0.37, p < 0.0001). The publication of the FDA draft guidance in 2010 was associated with increased reporting of the fixed-margin method (after versus before 2010) (OR 3.54; 95% CI 1.12 to 13.35, p = 0.04).

CONCLUSIONS

Non-inferiority margins are not commonly defined based on the historical evidence of the active comparator, and they are poorly reported. Authors, reviewers, and editors need to take notice of reporting this critical information to allow for better judgment of non-inferiority trials.

Bayesian methods for the design and analysis of noninferiority trials

The gold standard for evaluating treatment efficacy of a medical product is a placebo-controlled trial. However, when the use of placebo is considered to be unethical or impractical, a viable alternative for evaluating treatment efficacy is through a noninferiority (NI) study where a test treatment is compared to an active control treatment. The minimal objective of such a study is to determine whether the test treatment is superior to placebo. An assumption is made that if the active control treatment remains efficacious, as was observed when it was compared against placebo, then a test treatment that has comparable efficacy with the active control, within a certain range, must also be superior to placebo. Because of this assumption, the design, implementation, and analysis of NI trials present challenges for sponsors and regulators. In designing and analyzing NI trials, substantial historical data are often required on the active control treatment and placebo. Bayesian approaches provide a natural framework for synthesizing the historical data in the form of prior distributions that can effectively be used in design and analysis of a NI clinical trial. Despite a flurry of recent research activities in the area of Bayesian approaches in medical product development, there are still substantial gaps in recognition and acceptance of Bayesian approaches in NI trial design and analysis. The Bayesian Scientific Working Group of the Drug Information Association provides a coordinated effort to target the education and implementation issues on Bayesian approaches for NI trials. In this article, we provide a review of both frequentist and Bayesian approaches in NI trials, and elaborate on the implementation for two common Bayesian methods including hierarchical prior method and meta-analytic-predictive approach. Simulations are conducted to investigate the properties of the Bayesian methods, and some real clinical trial examples are presented for illustration.

Edoxaban in the evolving scenario of non vitamin K antagonist oral anticoagulants imputed placebo analysis and multiple treatment comparisons

Background

Edoxaban recently proved non-inferior to warfarin for prevention of thromboembolism in patients with non-valvular atrial fibrillation (AF). We conducted an imputed-placebo analysis with estimates of the proportion of warfarin effect preserved by each non vitamin K antagonist oral anticoagulant (NOAC) and indirect comparisons between edoxaban and different NOACs.

Methods and Findings

We performed a literature search (up to January 2014), clinical trials registers, conference proceedings, and websites of regulatory agencies. We selected non-inferiority randomised controlled phase III trials of dabigatran, rivaroxaban, apixaban and edoxaban compared with adjusted-dose warfarin in non-valvular AF. Compared to imputed placebo, all NOACs reduced the risk of stroke (ORs between 0.24 and 0.42, all p<0.001) and all-cause mortality (ORs between 0.55 and 0.59, all p<0.05). Edoxaban 30 mg and 60 mg preserved 87% and 112%, respectively, of the protective effect of warfarin on stroke, and 133% and 121%, respectively, of the protective effect of warfarin on all-cause mortality. The risk of primary outcome (stroke/systemic embolism), all strokes and ischemic strokes was significantly higher with edoxaban 30 mg than dabigatran 150 mg and apixaban. There were no significant differences between edoxaban 60 mg and other NOACs for all efficacy outcomes except stroke, which was higher with edoxaban 60 mg than dabigatran 150 mg. The risk of major bleedings was lower with edoxaban 30 mg than any other NOAC, odds ratios (ORs) ranging between 0.45 and 0.67 (all p<0.001).

Conclusions

This study suggests that all NOACs preserve a substantial or even larger proportion of the protective warfarin effect on stroke and all-cause mortality. Edoxaban 30 mg is associated with a definitely lower risk of major bleedings than other NOACs. This is counterbalanced by a lower efficacy in the prevention of thromboembolism, although with a final benefit on all-cause mortality.

Adjusting for unknown bias in non-inferiority clinical trials

Evaluation of non-inferiority is based on ruling out a threshold for what would constitute unacceptable loss of efficacy of an experimental treatment relative to an active comparator "Standard". This threshold, the "non-inferiority margin", is often based on preservation of a percentage of Standard's effect. To obtain an estimate of this effect to be used in the development of the "non-inferiority margin", data are needed from earlier trials comparing Standard to placebo if the non-inferiority trial does not have a placebo arm. This approach often provides a biased over-estimate of Standard's true effect in the setting of the current non-inferiority study. We describe two commonly used non-inferiority margin methods that adjust for this bias, the two-confidence interval (95-95) and the Synthesis margins. However, the added 'variance inflation' adjustment made by 95-95 margin diminishes with increasing information from historical trial(s), and the Synthesis margin is based on a strong assumption that the relative bias is known. We introduce an alternative "Bias-adjusted" margin addressing vulnerabilities of each by attenuating the estimate and by accounting for uncertainty in the true level of bias. Examples and asymptotic estimates of non-inferiority hypothesis rejection rates in the proportional hazards setting are used to compare methods.

Designing group sequential randomized clinical trials with time to event end points using a R function

A major and difficult task is the design of clinical trials with a time to event endpoint. In fact, it is necessary to compute the number of events and in a second step the required number of patients. Several commercial software packages are available for computing sample size in clinical trials with sequential designs and time to event endpoints, but there are a few R functions implemented. The purpose of this paper is to describe features and use of the R function. plansurvct.func, which is an add-on function to the package gsDesign which permits in one run of the program to calculate the number of events, and required sample size but also boundaries and corresponding p-values for a group sequential design. The use of the function plansurvct.func is illustrated by several examples and validated using East software.

A valid formulation of the analysis of noninferiority trials under random effects meta-analysis

A noninferiority (NI) trial is sometimes employed to show efficacy of a new treatment when it is unethical to randomize current patients to placebo because of the established efficacy of a standard treatment. Under this framework, if the NI trial determines that the treatment advantage of the standard to the new drug (i.e. S-N) is less than the historic advantage of the standard to placebo (S-P), then the efficacy of the new treatment (N-P) is established indirectly. We explicitly combine information from the NI trial with estimates from a random effects model, allowing study-to-study variability in k historic trials. Existing methods under random effects, such as the synthesis method, fail to account for the variability of the true standard versus placebo effect in the NI trial. Our method effectively uses a prediction interval for the missing standard versus placebo effect rather than a confidence interval of the mean. The consequences are to increase the variance of the synthesis method by incorporating a prediction variance term and to approximate the null distribution of the new statistic with a t with k-1 degrees of freedom instead of the standard normal. Thus, it is harder to conclude NI of the new to (predicted) placebo, compared with traditional methods, especially when k is small or when between study variability is large. When the between study variances are nonzero, we demonstrate substantial Type I error rate inflation with conventional approaches; simulations suggest that the new procedure has only modest inflation, and it is very conservative when between study variances are zero. An example is used to illustrate practical issues.

The network meta-analytic-predictive approach to non-inferiority trials

In non-inferiority clinical trials, a test treatment is compared to an active-control rather than to placebo. Such designs are considered when placebo is unethical or not feasible. The critical question is whether the test treatment would have been superior to placebo, had placebo been used in the non-inferiority trial. This question can only be addressed indirectly, based on information from relevant historical trials with data on active-control and placebo. The network meta-analytic-predictive approach to non-inferiority trials is based on a network meta-analysis of the data from the historical trials and the non-inferiority trial, and the prediction of the putative test vs. placebo effect in the non-inferiority trial. The approach extends previous work by incorporating between-trial variability for all relevant parameters and focusing on the parameters in the non-inferiority trial rather than on population means. Two prominent examples with binary outcomes are used to illustrate the approach.

Some essential considerations in the design and conduct of non-inferiority trials

BACKGROUND

Suppose a standard therapy (Standard) has been established to provide a clinically important reduction in risk of irreversible morbidity or mortality. In that setting, the safety and efficacy of an experimental intervention likely would be assessed in a clinical trial providing a comparison with Standard rather than a placebo arm. Such a trial often is designed to assess whether the efficacy of the experimental intervention is not unacceptably worse than that of Standard, and is called a non-inferiority trial. Formally, the non-inferiority trial usually is designed to rule out a non-inferiority margin, defined as the minimum threshold for what would constitute an unacceptable loss of efficacy.

PURPOSE

Even though the literature has many important articles identifying various approaches to the design and conduct of non-inferiority trials, confusion remains especially regarding key considerations for selecting the non-inferiority margin. The purpose of this article is to provide improved clarity regarding these considerations.

METHODS

We present scientific insights into many factors that should be addressed in the design and conduct of non-inferiority trials to enhance their integrity and reliability, and provide motivation for key considerations that guide the selection of non-inferiority margins. We also provide illustrations and insights from recent experiences.

RESULTS

Two considerations are essential, and should be addressed in separate steps, in the formulation of the non-inferiority margin. First, the margin should be formulated using adjustments to account for bias or lack of reliability in the estimate of the effect of Standard in the non-inferiority trial setting. Second, the non-inferiority margin should be formulated to achieve preservation of an appropriate percentage of the effect of Standard.

LIMITATIONS

The considerations, in particular regarding the importance of preservation of effect, might not apply to settings where it would be ethical as well as clinically relevant to include both Standard and placebo arms in the trial for direct comparisons with the experimental intervention arm.

CONCLUSIONS

Non-inferiority trials with non-rigorous margins allow substantial risk for accepting inadequately effective experimental regimens, leading to the risk of erosion in quality of health care. The design and conduct of non-inferiority trials, including selection of non-inferiority margins, should account for many factors that can induce bias in the estimated effect of Standard in the non-inferiority trial and thus lead to bias in the estimated effect of the experimental treatment, for the need to ensure the experimental treatment preserves a clinically acceptable fraction of Standard's effect, and for the particular vulnerability of the integrity of a non-inferiority trial to the irregularities in trial conduct. Due to the inherent uncertainties in non-inferiority trials, alternative designs should be pursued whenever possible.

Understanding the PRoFESS Study for Secondary Stroke Prevention

The Prevention Regimen for Effectively Avoiding Second Strokes (PRoFESS) trial is the largest secondary stroke prevention study completed to date. It compared extended-release dipyridamole plus aspirin (eDYP-ASA) versus clopidogrel and telmisartan versus antihypertensive regimens excluding angiotensin receptor blockers (ARBs). No statistical differences were found in either arm for the primary outcome of fatal or nonfatal stroke or prespecified secondary end points. eDYP-ASA also was associated with increases in major hemorrhagic events but not with statistical increases in combined rates of stroke recurrence or hemorrhage. Despite PRoFESS, the role of ARBs post stroke remains unclear, as concomitant angiotensin-converting enzyme inhibitor use in PRoFESS obscured whether just blood pressure lowering or renin-angiotensin system blockade is important. The resulting interpretation that eDYP-ASA is "not noninferior" has raised questions about how to interpret noninferiority analyses. Also, although the PRoFESS editorialists suggested that aspirin, the historical bystander control, was the "winner," a review of prior antiplatelet studies suggests that the benefits of aspirin, either as combination or monotherapy, are outweighed by its bleeding hazards. The benefits of clopidogrel or eDYP-ASA, compared with aspirin, are small but real, and both remain preferred agents in secondary stroke prevention.

Through the looking glass: understanding non-inferiority

Non-inferiority trials test whether a new product is not unacceptably worse than a product already in use. This paper introduces concepts related to non-inferiority, and discusses the regulatory views of both the European Medicines Agency and the United States Food and Drug Administration.

Sample size for equivalence trials: a case study from a vaccine lot consistency trial

For some trials, simple but subtle assumptions can have a profound impact on the size of the trial. A case in point is a vaccine lot consistency (or equivalence) trial. Standard sample size formulas used for designing lot consistency trials rely on only one component of variation, namely, the variation in antibody titers within lots. The other component, the variation in the means of titers between lots, is assumed to be equal to zero. In reality, some amount of variation between lots, however small, will be present even under the best manufacturing practices. Using data from a published lot consistency trial, we demonstrate that when the between-lot variation is only 0.5 per cent of the total variation, the increase in the sample size is nearly 300 per cent when compared with the size assuming that the lots are identical. The increase in the sample size is so pronounced that in order to maintain power one is led to consider a less stringent criterion for demonstration of lot consistency. The appropriate sample size formula that is a function of both components of variation is provided. We also discuss the increase in the sample size due to correlated comparisons arising from three pairs of lots as a function of the between-lot variance.