Testing superiority and non-inferiority hypotheses in active controlled clinical trials

A biomechanical comparison of the modified Bunnell pullout and Teo intraosseous suture techniques for attachment of tendon to bone

We compared the biomechanical properties of the Teo intraosseous suture technique with the modified Bunnell pullout technique in a cadaver model after a tendon to bone repair. Thirty-six fresh-frozen cadaveric fingers were assigned randomly to three groups (Teo, Bunnell and control groups). They were loaded cyclically from 2 to 15 N at 25 mm/min, for 500 cycles. Gap formation at the repair site was assessed every 100 cycles and then specimens were tested to failure. The Teo group had an approximately 30% smaller gap every 100 cycles and needed 30% more energy to obtain a 2 mm gap than the modified Bunnell group. Displacement after 500 cyclic loads was significantly lower in the Teo group than in the Bunnell group. For the failure of the Teo suture, it was necessary to apply 31% more load than the Bunnell technique.

Non-inferiority versus superiority drug claims: the (not so) subtle distinction

BACKGROUND

Current regulatory guidance and practice of non-inferiority trials are asymmetric in favor of the test treatment (Test) over the reference treatment (Control). These trials are designed to compare the relative efficacy of Test to Control by reference to a clinically important margin, M.

MAIN TEXT

Non-inferiority trials allow for the conclusion of: (a) non-inferiority of Test to Control if Test is slightly worse than Control but by no more than M; and (b) superiority if Test is slightly better than Control even if it is by less than M. From Control's perspective, (b) should lead to a conclusion of non-inferiority of Control to Test. The logical interpretation ought to be that, while Test is statistically better, it is not clinically superior to Control (since Control should be able to claim non-inferiority to Test). This article makes a distinction between statistical and clinical significance, providing for symmetry in the interpretation of results. Statistical superiority and clinical superiority are achieved, respectively, when the null and the non-inferiority margins are exceeded. We discuss a similar modification to placebo-controlled trials.

CONCLUSION

Rules for interpretation should not favor one treatment over another. Claims of statistical or clinical superiority should depend on whether or not the null margin or the clinically relevant margin is exceeded.

Testing of non-inferiority and superiority for three-arm clinical studies with multiple experimental treatments

The purpose of a non-inferiority trial is to assert the efficacy of an experimental treatment compared with a reference treatment by showing that the experimental treatment retains a substantial proportion of the efficacy of the reference treatment. Statistical methods have been developed to test multiple experimental treatments in three-arm non-inferiority trials. In this paper, we report the development of procedures that simultaneously test the non-inferiority and the superiority of experimental treatments after the assay sensitivity has been established. The advantage of the proposed test procedures is the additional ability to identify superior treatments while retaining an non-inferiority testing power comparable to that of existing testing procedures. Single-step and stepwise procedures are derived and then compared with each other to determine their relative testing power and testing error in a simulation study. Finally, the suggested procedures are illustrated with two clinical examples.

Comparing the response rates for superiority, noninferiority and equivalence testing with multiple-to-one matched binary data

Paired and multiple-to-one matched data have often been collected in clinical trials and epidemiological safety studies. When the response is binary, the sample size and power are determined by the discordant pairs or matched sets. Fixed sample size determination in assessing response rate difference of superiority, noninferiority, and equivalence tests of paired binary data has been discussed by Lu and Bean (1995)Nam (1997), and Liu et al. (2002). We extend the results of Lu and Bean (1995)Nam (1997), and Liu et al. (2002) to more general cases of multiple-to-one matched binary data. We further examine two issues regarding such tests. First, we examine the issue of simultaneous test and two-stage test for both superiority and noninferiority/equivalence hypotheses. Although, as discussed in Nam (1997) and Liu et al. (2002), the standard errors restricted to null hypothesis are different between superiority and noninferiority test, the monotonic property of the two tests makes the simultaneous testing and switching between the hypotheses valid. Second, in practice, the joint distribution of matched responses is often unknown, and thus determining the sample size using only the background information of the control group could be inefficient. Furthermore, for noninferiority or equivalence tests, the sample sizes are often determined using the unrealistic alternative hypothesis that the response rates of both treatments are identical. We propose to use a two-stage adaptive design strategy for sample size reestimation that uses the interim information to improve the efficiency.

A more powerful test based on ratio distribution for retention noninferiority hypothesis

Rothmann et al. ( 2003 ) proposed a method for the statistical inference of fraction retention noninferiority (NI) hypothesis. A fraction retention hypothesis is defined as a ratio of the new treatment effect verse the control effect in the context of a time to event endpoint. One of the major concerns using this method in the design of an NI trial is that with a limited sample size, the power of the study is usually very low. This makes an NI trial not applicable particularly when using time to event endpoint. To improve power, Wang et al. ( 2006 ) proposed a ratio test based on asymptotic normality theory. Under a strong assumption (equal variance of the NI test statistic under null and alternative hypotheses), the sample size using Wang's test was much smaller than that using Rothmann's test. However, in practice, the assumption of equal variance is generally questionable for an NI trial design. This assumption is removed in the ratio test proposed in this article, which is derived directly from a Cauchy-like ratio distribution. In addition, using this method, the fundamental assumption used in Rothmann's test, that the observed control effect is always positive, that is, the observed hazard ratio for placebo over the control is greater than 1, is no longer necessary. Without assuming equal variance under null and alternative hypotheses, the sample size required for an NI trial can be significantly reduced if using the proposed ratio test for a fraction retention NI hypothesis.

Weighted evidence approach of bridging study

The ICH E5 Guidance facilitates the registration of medicine among ICH regions by recommending a framework for evaluating the impact of ethnic factors upon a medicine's effect. It further describes the use of bridging studies, when necessary, to allow extrapolation of foreign clinical data to a new region. Bridging studies are performed in a new region for medicines already approved in the original region. The conventional noninferiority criterion requires the treatment effect (adjusted for placebo) attained in the new region preserves a prespecified proportion of the treatment effect attained in the original region. Such a bridging criterion, however, is often impractical. Hsiao et al. (2007) proposed a Bayesian approach that borrows the strength of the original trial to establish the treatment effect in the bridging region through using a weighted prior distribution. The weight, however, is often difficult to prespecify. In this presentation, we consider the overall treatment effect by combining the weighted effects attained in the original and bridging regions. The maximum weight allowed to be placed on the estimate of bridging region in order to show a significant overall treatment effect represents the strength of the treatment effect in the bridging region. Regional approval will be evaluated either by comparing the weight estimate with the prespecified limit or by benefit-risk evaluation of the medicine. Sample size requirements for the approaches are derived. The simulation results of type I error rate and power for the proposed methods are given. An example illustrates the application of the proposed procedures.

A hybrid Bayesian-frequentist approach to evaluate clinical trial designs for tests of superiority and non-inferiority

Specification of the study objective of superiority or non-inferiority at the design stage of a phase III clinical trial can sometimes be very difficult due to the uncertainty that surrounds the efficacy level of the experimental treatment. This uncertainty makes it tempting for investigators to design a trial that would allow testing of both superiority and non-inferiority hypotheses. However, when a conventional single-stage design is used to test both hypotheses, the sample size is based on the chosen primary objective of either superiority or non-inferiority. In this situation, the power of the test for the secondary objective can be low, which may lead to a large loss of resources. Potentially low reproducibility is another major concern for the single-stage design in phase III trials, because significant findings of confirmatory trials are required to be reproducible. In this paper, we propose a hybrid Bayesian-frequentist approach to evaluate reproducibility and power in single-stage designs for phase III trials to test both superiority and non-inferiority. The essence of the proposed approach is to express the uncertainty that surrounds the efficacy of the experimental treatment as a probability distribution. Then one can use Bayes formula with simple graphical techniques to evaluate reproducibility and power adequacy.

Simultaneous test for superiority and noninferiority hypotheses in active-controlled clinical trials

Two stage switching between testing for superiority (SUP) and noninferiority (NI) has been an important statistical issue in the design and analysis of the active-controlled clinical trials. Tsong and Zhang (2005) has shown that the Type I error rates do not change when switching between SUP and NI with the traditional generalized historical control (GHC) approach, however, they may change when switching with the cross-trial comparison (X-trial) approach. Tsong and Zhang (2005) further proposed a simultaneous test for both hypotheses to avoid the problem. The procedure was based on Fieller's confidence interval proposed by Hauschke et al. (1999). Since with the X-trial approach, using the simultaneous test, superiority is tested using all four treatment arms (current test and active control arms, active control and placebo arms in historical trials), the Type I error rate and power are expected to be somewhat different from the conventional superiority test (using the current test and active control arms only). Through a simulation study, we demonstrate that the Type I error rate and power between simultaneous test and the conventional superiority test are compatible. We also examine the impact of the assumption of equal variances of the current trial and the historical trial.

Choice of δ Noninferiority Margin and Dependency of the Noninferiority Trials

For a two-arm active control clinical trial designed to test for noninferiority of the test treatment to the active control standard treatment, data of historical studies were often used. For example, with a cross-trial comparison approach (also called synthetic approach or lambda-margin approach), the trial is conducted to test the hypothesis that the mean difference or the ratio between the current test product and the active control is no larger than a certain portion of the mean difference or no smaller that a certain portion of the ratio of the active control and placebo obtained in the historical data when the positive response indicates treatment effective. For a generalized historical control approach (also known as confidence interval approach or delta -margin approach), the historical data is often used to determine a fixed value noninferiority margin delta for all trials involving the active control treatment. The regulatory agency usually requires that the clinical trials of two different test treatments need to be independent and in most regular cases, it also requires to have two independent positive trials of the same test treatment in order to provide confirmatory evidence of the efficacy of the test product. Because of the nature of information (historical data) shared in active-controlled trials, the independency assumption of the trials is not satisfied in general. The correlation between two noninferiority tests has been examined which showed that it is an increasing function of (1 - lambda ) when the response variable is normally distributed. In this article, we examine the relationship between the correlation of the two test statistics and the choice of the noninferiority margin, delta as well as the sample sizes and variances under the normality assumption. We showed that when delta is determined by the lower limit of the confidence interval of the adjusted effect size of the active control treatment (muC - muP) using data from historical studies, dependency of the two noninferiority tests can be very high. In order to control the correlation under 15%, the overall sample size of the historical studies needs to be at least five times of the current active control trial.

Noninferiority Testing Beyond Simple Two-Sample Comparison

In order to fulfill the requirement of a new drug application, a sponsor often need to conduct multiple clinical trials. Often these trials are of designs more complicated than a randomized two-sample single-factor study. For example, these trials could be designed with multiple centers, multiple factors, covariates, group sequential and/or adaptive scheme, etc. When an active standard treatment used as the control treatment in a two-arm clinical trial, the efficacy of the test treatment is often established by performing a noninferiority test through comparison of the test treatment and the active standard treatment. Typically, the noninferiority trials are designed with either a generalized historical control approach (i.e., noninferiority margin approach or delta-margin approach) or a cross-trial comparison approach (i.e., synthesis approach or lambda-margin approach). Many of the statistical properties of the approaches discussed in the literature were focused on testing in a simple two sample comparison form. We studied the limitations of the two approaches for the consideration of switching between superiority and noninferiority testing, feasibility to be applied with group sequential design, constancy assumption requirements, test dependency in multiple trials, analysis of homogeneity of efficacy among centers in a multi-center trial, data transformation and changing analysis method from the historical studies. Our evaluation shows that the cross-trial comparison approach is more restricted to simple two sample comparison with normal approximation test because of its poor properties with more complicated design and analysis. On the other hand, the generalized historical control comparison approach may have more flexible properties when the variability of the margin delta is indeed negligibly small.

Flexible Designs by Adaptive Plans of Generalized Pocock- and O'Brien-Fleming-Type and by Self-Designing Clinical Trials

Flexible designs are provided by adaptive planning of sample sizes as well as by introducing the weighted inverse normal combining method and the generalized inverse chi-square combining method in the context of conducting trials consecutively step by step. These general combining methods allow quite different weighting of sequential study parts, also in a completely adaptive way, based on full information from unblinded data in previously performed stages. So, in reviewing some basic developments of flexible designing, we consider a generalizing approach to group sequentially performed clinical trials of Pocock-type, of O'Brien-Fleming-type, and of Self-designing-type. A clinical trial may be originally planned either to show non-inferiority or superiority. The proposed flexible designs, however, allow in each interim analysis to change the planning from showing non-inferiority to showing superiority and vice versa. Several examples of clinical trials with normal and binary outcomes are worked out in detail. We demonstrate the practicable performance of the discussed approaches, confirmed in an extensive simulation study. Our flexible designing is a useful tool, provided that a priori information about parameters involved in the trial is not available or subject to uncertainty.

Repeated Confidence Intervals in Self-Designing Clinical Trials and Switching between Noninferiority and Superiority

In self-designing clinical trials, repeated confidence intervals are derived for the parameter of interest where the results of the independent study stages are combined using the generalized inverse chi-square-method. The confidence intervals can be calculated at each interim analysis and always hold the predefined overall nominal confidence level. Moreover, the confidence intervals calculated during the course of the trial are nested in the sense that a calculated interval is completely contained in all the previously calculated intervals. During the course of the self-designing trial the sample sizes as well as the number of study stages can be determined simultaneously in a completely adaptive way. The adaptive procedure allows an early stop for significance. The clinical trial may be originally designed either to show noninferiority or superiority. However, in each interim analysis, it is possible to change the planning from showing superiority to showing noninferiority or vice versa. Since the repeated confidence intervals are nested, there is no risk to loose the noninferiority once showed when, after an interim analysis, the trial is continued in an attempt to reach superiority. A simulation study investigates the behavior of the considered confidence intervals. The performance of the derived nested repeated confidence intervals is also demonstrated in examples showing both kinds of switching during an ongoing trial.

Testing superiority and non-inferiority hypotheses in active controlled clinical trials

Switching between testing for superiority and non-inferiority has been an important statistical issue in the design and analysis of active controlled clinical trial. In practice, it is often conducted with a two-stage testing procedure. It has been assumed that there is no type I error rate adjustment required when either switching to test for non-inferiority once the data fail to support the superiority claim or switching to test for superiority once the null hypothesis of non-inferiority is rejected with a pre-specified non-inferiority margin in a generalized historical control approach. However, when using a cross-trial comparison approach for non-inferiority testing, controlling the type I error rate sometimes becomes an issue with the conventional two-stage procedure. We propose to adopt a single-stage simultaneous testing concept as proposed by Ng (2003) to test both non-inferiority and superiority hypotheses simultaneously. The proposed procedure is based on Fieller's confidence interval procedure as proposed by Hauschke et al. (1999).