TACT method for non-inferiority testing in active controlled trials

A reproducibility probability-based bias-adjustment approach on the specification of non-inferiority margin using historical data

The effect of reference treatment over placebo, known as M1, is essential in the development of non-inferiority margin. We proposed a M1 adjustment approach to reduce the selection bias for collected data of historical trials. A quantitative illustration of selection bias of historical data is also defined. Simulation study shows that the proposed approaches would significantly reduce the bias when the proportion of positive studies in historical data is noticeably larger than the power of studies include in historical data. When historical data are constituted by only positive studies, the performance of the proposed method is also appreciable. However, when the proportion of positive studies is close to the power of studies included or the number of studies included is too small, the performance of the proposed approach may not be reliable. A real-data application is also presented. The proposed bias-adjustment approach is a reasonable method to reduce the over-estimate of effect size in the specification of non-inferiority margin. It could also be applied in most non-inferiority margin specification methods or be cooperate used with other bias-adjustment approaches.

Some issues for the evaluation of noninferiority trials

Although published noninferiority trials (NITs) generally conclude that the experimental intervention being studied is noninferior compared with standard therapy or active control, NIT quality is often not satisfactory. We have proposed 14 questions to assist in evaluating the clinical evidence of the experimental versus standard therapy. The aim of these questions is to critically appraise NITs and support proper interpretation of study results. Readers should not only consider whether the confidence interval of the primary effect measure falls within the prespecified noninferiority margin (thus concluding noninferiority), but also assess the similarities between primary and secondary outcomes for the experimental and standard therapy. To conclude noninferiority conceptually is to synthesize evidence from both the current NIT comparing experimental therapy with standard therapy and historical data comparing standard therapy with placebo control. Therefore, readers should use external data sources (e.g., historical data) to validate the study design (e.g., selection of standard therapy, effect measure and the noninferiority margin), and assess the uncertainty of findings due to differences between the observed and expected incidence rates, follow-up time, effects of adjuvant therapy and the secondary outcomes of therapies. Following an explanation of the 14 questions, we then apply the questions to a NIT on intraoperative radiation therapy for early stage breast cancer, as an example.

Incorporating a companion test into the noninferiority design of medical device trials

Noninferiority trials are commonly utilized to evaluate the safety and effectiveness of medical devices. It could happen that the noninferiority hypothesis is rejected while the performance of the active control is clinically not satisfactory. This may pose a great challenge when making a regulatory decision. To avoid such a difficult situation, we propose to conduct a companion test to assess the performance of the active control when testing the main noninferiority hypothesis and to incorporate such a test into the study design. Under our proposal, the noninferiority of the investigational device to the active control can only be claimed when both hypotheses are rejected. The operating characteristics of the proposed study design based on these two tests can be fully evaluated at the design stage. This proposed approach is aimed to facilitate regulatory decision making in a more transparent manner.

The Impact of the Underlying Risk in Control Group and Effect Measures in Non-Inferiority Trials With Time-to-Event Data: A Simulation Study

Background

We designed a simulation study to assess how the conclusions of a non-inferiority trial (NIT) will change if the observed risk is different from the expected risk.

Methods

We simulated Weibull distribution time-to-event data with a true hazard ratio (HR) being equal or close to 1. The empirical margins and sample size of a hypothetical trial were chosen based on a systematic review. Setting the significance level at 5% for the two-sided confidence interval (CI), we examined the statistical power (i.e., the probabilities of the upper limit of the 95% CI falling within the margin) of using two measures at various underlying risk in the control group.

Results

Using the empirical margins, HRs of 1.2, 1.35 or 1.5, the statistical power is lower than 0.22 when the underlying risk in the control group is less than 10%, but the power increases along with the higher underlying risk. The predicted upper limit of the 95% CI of the difference in two Kaplan-Meier estimators (DTKME) is low when risk is low (< 20%) or high (> 80%), but reaches the highest value when risk is around 50%. When the underlying risk in the control group is lower than 10%, measures of DTKME resulted in much higher power than HR.

Conclusions

When HR is the effect measure, the probability of concluding non-inferiority will increase as the underlying risk in the control group increases. When DTKME is the effect measure, the probability of concluding non-inferiority will decrease as the underlying risk in the control increases. In this case, the probability of concluding non-inferiority is at a minimum when the control risk reaches about 50%. When the risk in the control arm is less than 10%, the conclusion of an NIT is sensitive to the choice of effect measure.

Detecting and accounting for violations of the constancy assumption in non-inferiority clinical trials

Randomized, placebo-controlled clinical trials are the gold standard for evaluating a novel therapeutic agent. In some instances, it may not be considered ethical or desirable to complete a placebo-controlled clinical trial and, instead, the placebo is replaced by an active comparator with the objective of showing either superiority or non-inferiority to the active comparator. In a non-inferiority trial, the experimental treatment is considered non-inferior if it retains a pre-specified proportion of the effect of the active comparator as represented by the non-inferiority margin. A key assumption required for valid inference in the non-inferiority setting is the constancy assumption, which requires that the effect of the active comparator in the non-inferiority trial is consistent with the effect that was observed in previous trials. It has been shown that violations of the constancy assumption can result in a dramatic increase in the rate of incorrectly concluding non-inferiority in the presence of ineffective or even harmful treatment. In this paper, we illustrate how Bayesian hierarchical modeling can be used to facilitate multi-source smoothing of the data from the current trial with the data from historical studies, enabling direct probabilistic evaluation of the constancy assumption. We then show how this result can be used to adapt the non-inferiority margin when the constancy assumption is violated and present simulation results illustrating that our method controls the type-I error rate when the constancy assumption is violated, while retaining the power of the standard approach when the constancy assumption holds. We illustrate our adaptive procedure using a non-inferiority trial of raltegravir, an antiretroviral drug for the treatment of HIV.

Covariate effect on constancy assumption in noninferiority clinical trials

Noninferiority (NI) clinical trials are getting a lot of attention of late due to their direct application in biosimilar studies. Because of the missing placebo arm, NI is an indirect approach to demonstrate efficacy of a test treatment. One of the key assumptions in the NI test is the constancy assumption, that is, that the effect of the reference treatment is the same in current NI trials as in historical superiority trials. However, if a covariate interacts with the treatment arms, then changes in distribution of this covariate will likely result in violation of constancy assumption. In this article, we propose four new NI methods and compare them with two existing methods to evaluate the change of background constancy assumption on the performance of these six methods. To achieve this goal, we study the impact of three elements-(1) strength of covariate, (2) degree of interaction between covariate and treatment, and (3) differences in distribution of the covariate between historical and current trials-on both the type I error rate and power using three different measures of association: difference, log relative risk, and log odds ratio. Based on this research, we recommend using a modified covariate-adjustment fixed margin method.

An evidential approach to non-inferiority clinical trials

We present likelihood methods for defining the non-inferiority margin and measuring the strength of evidence in non-inferiority trials using the 'fixed-margin' framework. Likelihood methods are used to (1) evaluate and combine the evidence from historical trials to define the non-inferiority margin, (2) assess and report the smallest non-inferiority margin supported by the data, and (3) assess potential violations of the constancy assumption. Data from six aspirin-controlled trials for acute coronary syndrome and data from an active-controlled trial for acute coronary syndrome, Organisation to Assess Strategies for Ischemic Syndromes (OASIS-2) trial, are used for illustration. The likelihood framework offers important theoretical and practical advantages when measuring the strength of evidence in non-inferiority trials. Besides eliminating the influence of sample spaces and prior probabilities on the 'strength of evidence in the data', the likelihood approach maintains good frequentist properties. Violations of the constancy assumption can be assessed in the likelihood framework when it is appropriate to assume a unifying regression model for trial data and a constant control effect including a control rate parameter and a placebo rate parameter across historical placebo controlled trials and the non-inferiority trial. In situations where the statistical non-inferiority margin is data driven, lower likelihood support interval limits provide plausibly conservative candidate margins.

Bio-creep in non-inferiority clinical trials

After a non-inferiority clinical trial, a new therapy may be accepted as effective, even if its treatment effect is slightly smaller than the current standard. It is therefore possible that, after a series of trials where the new therapy is slightly worse than the preceding drugs, an ineffective or harmful therapy might be incorrectly declared efficacious; this is known as 'bio-creep'. Several factors may influence the rate at which bio-creep occurs, including the distribution of the effects of the new agents being tested and how that changes over time, the choice of active comparator, the method used to account for the variability of the estimate of the effect of the active comparator, and changes in the effect of the active comparator from one trial to the next (violations of the constancy assumption). We performed a simulation study to examine which of these factors might lead to bio-creep and found that bio-creep was rare, except when the constancy assumption was violated.

A covariate-adjustment regression model approach to noninferiority margin definition

To maintain the interpretability of the effect of experimental treatment (EXP) obtained from a noninferiority trial, current statistical approaches often require the constancy assumption. This assumption typically requires that the control treatment effect in the population of the active control trial is the same as its effect presented in the population of the historical trial. To prevent constancy assumption violation, clinical trial sponsors were recommended to make sure that the design of the active control trial is as close to the design of the historical trial as possible. However, these rigorous requirements are rarely fulfilled in practice. The inevitable discrepancies between the historical trial and the active control trial have led to debates on many controversial issues. Without support from a well-developed quantitative method to determine the impact of the discrepancies on the constancy assumption violation, a correct judgment seems difficult. In this paper, we present a covariate-adjustment generalized linear regression model approach to achieve two goals: (1) to quantify the impact of population difference between the historical trial and the active control trial on the degree of constancy assumption violation and (2) to redefine the active control treatment effect in the active control trial population if the quantification suggests an unacceptable violation. Through achieving goal (1), we examine whether or not a population difference leads to an unacceptable violation. Through achieving goal (2), we redefine the noninferiority margin if the violation is unacceptable. This approach allows us to correctly determine the effect of EXP in the noninferiority trial population when constancy assumption is violated due to the population difference. We illustrate the covariate-adjustment approach through a case study.

Aspirin and extended-release dipyridamole versus clopidogrel for recurrent stroke

BACKGROUND

Recurrent stroke is a frequent, disabling event after ischemic stroke. This study compared the efficacy and safety of two antiplatelet regimens--aspirin plus extended-release dipyridamole (ASA-ERDP) versus clopidogrel.

METHODS

In this double-blind, 2-by-2 factorial trial, we randomly assigned patients to receive 25 mg of aspirin plus 200 mg of extended-release dipyridamole twice daily or to receive 75 mg of clopidogrel daily. The primary outcome was first recurrence of stroke. The secondary outcome was a composite of stroke, myocardial infarction, or death from vascular causes. Sequential statistical testing of noninferiority (margin of 1.075), followed by superiority testing, was planned.

RESULTS

A total of 20,332 patients were followed for a mean of 2.5 years. Recurrent stroke occurred in 916 patients (9.0%) receiving ASA-ERDP and in 898 patients (8.8%) receiving clopidogrel (hazard ratio, 1.01; 95% confidence interval [CI], 0.92 to 1.11). The secondary outcome occurred in 1333 patients (13.1%) in each group (hazard ratio for ASA-ERDP, 0.99; 95% CI, 0.92 to 1.07). There were more major hemorrhagic events among ASA-ERDP recipients (419 [4.1%]) than among clopidogrel recipients (365 [3.6%]) (hazard ratio, 1.15; 95% CI, 1.00 to 1.32), including intracranial hemorrhage (hazard ratio, 1.42; 95% CI, 1.11 to 1.83). The net risk of recurrent stroke or major hemorrhagic event was similar in the two groups (1194 ASA-ERDP recipients [11.7%], vs. 1156 clopidogrel recipients [11.4%]; hazard ratio, 1.03; 95% CI, 0.95 to 1.11).

CONCLUSIONS

The trial did not meet the predefined criteria for noninferiority but showed similar rates of recurrent stroke with ASA-ERDP and with clopidogrel. There is no evidence that either of the two treatments was superior to the other in the prevention of recurrent stroke. (ClinicalTrials.gov number, NCT00153062.)

Assessing non-inferiority: a combination approach

Non-inferiority designs are growing in importance as a strategy for comparing new drugs with established therapies. Because it is not possible to show that a new drug and the established therapy have identical efficacy profiles, non-inferiority trials are designed to demonstrate that the new drug is not inferior to an established drug (the 'control') relative to a prespecified 'non-inferiority margin'. No objective principle guides the choice of the non-inferiority margin, and controversies about the margin have, in some cases, had important consequences for drug development. We argue that some of these controversies have arisen because non-inferiority trials must achieve two objectives. They must demonstrate not only that the new drug is not inferior to the control drug by the non-inferiority margin, but also that the new drug is superior to placebo. When the second objective is not considered explicitly, it can distort the choice of the non-inferiority margin. Some methods designed to address both objectives through the choice of the non-inferiority margin lead to overly stringent non-inferiority criteria. We describe an approach to non-inferiority analysis that combines two tests, a traditional test for non-inferiority and a test for superiority based on a synthetic estimate of the effect of the new treatment relative to placebo. The synthetic estimate may be 'discounted' to address concerns about assay inconstancy. We discuss power and sample size considerations for the proposed procedure.

Controlling the type 1 error rate in non-inferiority trials

Two different approaches have been proposed for establishing the efficacy of an experimental therapy through a non-inferiority trial: The fixed-margin approach involves first defining a non-inferiority margin and then demonstrating that the experimental therapy is not worse than the control by more than this amount, and the synthesis approach involves combining the data from the non-inferiority trial with the data from historical trials evaluating the effect of the control. In this paper, we introduce a unified approach that has both these approaches as special cases and show how the parameters of this approach can be selected to control the unconditional type 1 error rate in the presence of departures from the assumptions of assay sensitivity and constancy. It is shown that the fixed-margin approach can be extremely inefficient and that it is always possible to achieve equivalent control of the unconditional type 1 error rate, with higher power, by using an appropriately chosen synthesis method.

Preservation of effect and the regulatory approval of new treatments on the basis of non-inferiority trials

The criteria for approval of an experimental treatment on the basis of active-controlled non-inferiority trials often include demonstration of 'preservation of effect' relative to the active control. While this appears on its surface to be a reasonable criterion, on closer inspection it can be shown to lead to serious logical inconsistencies. In particular, an experimental treatment may have clinical trial results that are superior to those for the standard treatment and yet fail to meet this criterion for approval. In this paper we propose a set of principles that will help avoid these logical inconsistencies, and we argue that the qualities of an experimental treatment that are required for approval should be consistent regardless of the presence or absence of existing treatments and the type of study design used to evaluate the treatment.

[Non-inferiority trial used in venous thromboembolic disease. A warily interpretation is necessary!]

INTRODUCTION

Equivalence trials are actually frequently used to prove non-inferiority in anticoagulant therapy. Equivalence trials consist to demonstrate that two treatments are not too much different. This difference has to be under a margin previously determined. The margin corresponds to an efficacy loss that is defined to be acceptable, in accordance to the advantages due to the new treatment. The aim of this work is to explore the equivalence trial published in the thromboembolic disease by focus on the non-inferiority margin used.

METHODS

We identified published equivalence trials in the venous thromboembolic disease, by a systematic search in Medline. We calculated the efficacy loss by reference with the value of the smallest effect size of the standard treatment compared to placebo.

RESULTS

We found 9 equivalence trials used in venous thromboembolic disease. The mean value of the efficacy loss was 434%, and the median value was 357%. Eighty-five percent of the values of the efficacy loss were above 100%.

DISCUSSION

Eighty-five percent of the equivalence trials conclude to equivalence despite a complete efficacy loss of the effect of the standard treatment compared to placebo. The results of equivalence trials should be interpreted warily. The corresponding non-inferiority margin should be chosen more rigorously and by reference with the value of the smallest effect size of the standard treatment compared to placebo.

A ratio test in active control non-inferiority trials with a time-to-event endpoint

There are essentially two kinds of non-inferiority hypotheses in an active control trial: fixed margin and ratio hypotheses. In a fixed margin hypothesis, the margin is a prespecified constant and the hypothesis is defined in terms of a single parameter that represents the effect of the active treatment relative to the control. The statistical inference for a fixed margin hypothesis is straightforward. The outstanding issue for a fixed margin non-inferiority hypothesis is how to select the margin, a task that may not be as simple as it appears. The selection of a fixed non-inferiority margin has been discussed in a few articles (Chi et al., 2003; Hung et al., 2003; Ng, 1993). In a ratio hypothesis, the control effect is also considered as an unknown parameter, and the noninferiority hypothesis is then formulated as a ratio in terms of these two parameters, the treatment effect and the control effect. This type of non-inferiority hypothesis has also been called the fraction retention hypothesis because the ratio hypothesis can be interpreted as a retention of certain fraction of the control effect. Rothmann et al. (2003) formulated a ratio non-inferiority hypothesis in terms of log hazards in the time-to-event setting. To circumvent the complexity of having to deal with a ratio test statistic, the ratio hypothesis was linearized to an equivalent hypothesis under the assumption that the control effect is positive. An associated test statistic for this linearized hypothesis was developed. However, there are three important issues that are not addressed by this method. First, the retention fraction being defined in terms of log hazard is difficult to interpret. Second, in order to linearize the ratio hypothesis, Rothmann's method has to assume that the true control effect is positive. Third, the test statistic is not powerful and thus requires a huge sample size, which renders the method impractical. In this paper, a ratio hypothesis is defined directly in terms of the hazard. A natural ratio test statistic can be defined and is shown to have the desired asymptotic normality. The demand on sample size is much reduced. In most commonly encountered situations, the sample size required is less than half of those needed by either the fixed margin approach or Rothmann's method.

Testing superiority and non-inferiority hypotheses in active controlled clinical trials

Switching between testing for superiority and non-inferiority has been an important statistical issue in the design and analysis of active controlled clinical trial. In practice, it is often conducted with a two-stage testing procedure. It has been assumed that there is no type I error rate adjustment required when either switching to test for non-inferiority once the data fail to support the superiority claim or switching to test for superiority once the null hypothesis of non-inferiority is rejected with a pre-specified non-inferiority margin in a generalized historical control approach. However, when using a cross-trial comparison approach for non-inferiority testing, controlling the type I error rate sometimes becomes an issue with the conventional two-stage procedure. We propose to adopt a single-stage simultaneous testing concept as proposed by Ng (2003) to test both non-inferiority and superiority hypotheses simultaneously. The proposed procedure is based on Fieller's confidence interval procedure as proposed by Hauschke et al. (1999).

A regulatory perspective on choice of margin and statistical inference issue in non-inferiority trials

Without a placebo arm, any non-inferiority inference involving assessment of the placebo effect under the active control trial setting is difficult. The statistical risk for falsely concluding non-inferiority cannot be evaluated unless the constancy assumption approximately holds that the effect of the active control under the historical trial setting where the control effect can be assessed carries to the noninferiority trial setting. The constancy assumption cannot be checked because of missing the placebo arm in the non-inferiority trial. Depending on how serious the violation of the assumption is thought to be, one may need to seek an alternative design strategy that includes a cushion for a very conservative non-inferiority analysis or shows superiority of the experimental treatment over the control. Determination of the non-inferiority margin depends on what objective the non-inferiority analysis is intended to achieve. The margin can be a fixed margin or a margin functionally defined. Between-trial differences always exist and need to be properly considered.

Active-controlled, non-inferiority trials in oncology: arbitrary limits, infeasible sample sizes and uninformative data analysis. Is there another way?

In oncology, it may not always be possible to evaluate the efficacy of new medicines in placebo-controlled trials. Furthermore, while some newer, biologically targeted anti-cancer treatments may be expected to deliver therapeutic benefit in terms of better tolerability or improved symptom control, they may not always be expected to provide increased efficacy relative to existing therapies. This naturally leads to the use of active-control, non-inferiority trials to evaluate such treatments. In recent evaluations of anti-cancer treatments, the non-inferiority margin has often been defined in terms of demonstrating that at least 50% of the active control effect has been retained by the new drug using methods such as those described by Rothmann et al., Statistics in Medicine 2003; 22:239-264 and Wang and Hung Controlled Clinical Trials 2003; 24:147-155. However, this approach can lead to prohibitively large clinical trials and results in a tendency to dichotomize trial outcome as either 'success' or 'failure' and thus oversimplifies interpretation. With relatively modest modification, these methods can be used to define a stepwise approach to design and analysis. In the first design step, the trial is sized to show indirectly that the new drug would have beaten placebo; in the second analysis step, the probability that the new drug is superior to placebo is assessed and, if sufficiently high in the third and final step, the relative efficacy of the new drug to control is assessed on a continuum of effect retention via an 'effect retention likelihood plot'. This stepwise approach is likely to provide a more complete assessment of relative efficacy so that the value of new treatments can be better judged.

Enoxaparin in the Treatment of Deep Vein Thrombosis With or Without Pulmonary Embolism

STUDY OBJECTIVES

Low-molecular-weight heparins have been compared with unfractionated heparin (UFH) for treatment of deep vein thrombosis (DVT). However, a comparison of their efficacy in the presence or absence of pulmonary embolism (PE) has not been studied. We estimated the efficacy and safety of enoxaparin vs UFH in patients with proximal DVT with/without symptomatic PE using a meta-analysis of individual data from randomized controlled trials.

DESIGN AND SETTING

Randomized controlled trials were identified from MEDLINE, EMBASE, abstracts from international meetings on venous thromboembolism (VTE), previous meta-analyses, and trial data provided by the sponsor.

PARTICIPANTS

For inclusion, randomized controlled trials had to be properly randomized; include patients with objectively diagnosed DVT; compare enoxaparin twice daily with UFH; use objective methods to assess recurrent symptomatic VTE, major bleeding, and death at 3 months; and include blind evaluation of clinical events.

MEASUREMENTS

A meta-analysis was performed using the logarithm of the relative risk (RR) method. Enoxaparin in DVT treatment with/without symptomatic PE was considered noninferior to UFH for preventing VTE at 3 months if the upper limit of the 95% confidence interval (CI) of the RR (enoxaparin/UFH) was lower than a prespecified noninferiority margin (1.61). No increase in major bleeding or mortality should be observed.

RESULTS

The meta-analysis included individual data from three randomized controlled trials (749 patients and 754 patients in the enoxaparin and UFH groups, respectively). The observed RR (enoxaparin/UFH) of VTE was 0.81 (95% CI, 0.52 to 1.26) for the intention-to-treat population (RR, 0.70; 95% CI, 0.43 to 1.13; for per-protocol analysis). Results did not differ for patients with clinical PE (235 patients; RR, 0.84) and without clinical PE (1,268 patients; RR, 0.71), with a nonsignificant heterogeneity test between groups (p = 0.76). A trend in favor of enoxaparin was observed for reduced mortality and major bleeding.

CONCLUSIONS

The efficacy and safety of enoxaparin vs UFH for DVT treatment is not modified by the presence of symptomatic PE.

Bridging bioequivalence studies

In some new regions, an innovative drug of the original region was not marketed. However, after the patent of the innovative drug is expired, a generic copy of the innovative drug from the original region was introduced and approved for marketing in the new region. Another generic copy manufactured by the local sponsor of the new region is seeking for approval in the new region. Despite unavailability of the innovative drug, the regulatory authority of the new region still wants to approve the local generic copy based on assessment of bioequivalence between the local generic drug and the innovative drug. Following the bridging concept suggested by the ICH E5 guidance, we propose a method to evaluate average bioequivalence between the generic copy of the new region and the innovative drug of the original region using the generic copy of the original region as the bridging reference formulation. Sample size required by the bioequivalence study in the new region is also provided. Numerical examples illustrate the proposed method.