Comparison of conditional bias-adjusted estimators for interim analysis in clinical trials with survival data

Point estimation for adaptive trial designs II: Practical considerations and guidance

In adaptive clinical trials, the conventional end-of-trial point estimate of a treatment effect is prone to bias, that is, a systematic tendency to deviate from its true value. As stated in recent FDA guidance on adaptive designs, it is desirable to report estimates of treatment effects that reduce or remove this bias. However, it may be unclear which of the available estimators are preferable, and their use remains rare in practice. This article is the second in a two-part series that studies the issue of bias in point estimation for adaptive trials. Part I provided a methodological review of approaches to remove or reduce the potential bias in point estimation for adaptive designs. In part II, we discuss how bias can affect standard estimators and assess the negative impact this can have. We review current practice for reporting point estimates and illustrate the computation of different estimators using a real adaptive trial example (including code), which we use as a basis for a simulation study. We show that while on average the values of these estimators can be similar, for a particular trial realization they can give noticeably different values for the estimated treatment effect. Finally, we propose guidelines for researchers around the choice of estimators and the reporting of estimates following an adaptive design. The issue of bias should be considered throughout the whole lifecycle of an adaptive design, with the estimation strategy prespecified in the statistical analysis plan. When available, unbiased or bias-reduced estimates are to be preferred.

Point estimation following a two-stage group sequential trial

Repeated testing in a group sequential trial can result in bias in the maximum likelihood estimate of the unknown parameter of interest. Many authors have therefore proposed adjusted point estimation procedures, which attempt to reduce such bias. Here, we describe nine possible point estimators within a common general framework for a two-stage group sequential trial. We then contrast their performance in five example trial settings, examining their conditional and marginal biases and residual mean square error. By focusing on the case of a trial with a single interim analysis, additional new results aiding the determination of the estimators are given. Our findings demonstrate that the uniform minimum variance unbiased estimator, whilst being marginally unbiased, often has large conditional bias and residual mean square error. If one is concerned solely about inference on progression to the second trial stage, the conditional uniform minimum variance unbiased estimator may be preferred. Two estimators, termed mean adjusted estimators, which attempt to reduce the marginal bias, arguably perform best in terms of the marginal residual mean square error. In all, one should choose an estimator accounting for its conditional and marginal biases and residual mean square error; the most suitable estimator will depend on relative desires to minimise each of these factors. If one cares solely about the conditional and marginal biases, the conditional maximum likelihood estimate may be preferred provided lower and upper stopping boundaries are included. If the conditional and marginal residual mean square error are also of concern, two mean adjusted estimators perform well.

Point estimation for adaptive trial designs I: A methodological review

Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value," and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realized trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This article is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this article can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.

Estimation of the treatment effect following a clinical trial that stopped early for benefit

When a clinical trial stops early for benefit, the maximum likelihood estimate (MLE) of the treatment effect may be subject to overestimation bias. Several authors have proposed adjusting for this bias using the conditional MLE, which is obtained by conditioning on early stopping. However, this approach has a fundamental problem in that the adjusted estimate may not be in the direction of benefit, even though the study has stopped early due to benefit. In this paper, we address this problem by embedding both the MLE and the conditional MLE within a broader class of penalised likelihood estimates, and choosing a member of the class that is a favourable compromise between the two. This penalised MLE, and its associated confidence interval, always lie in the direction of benefit when the study stops early for benefit. We study its properties using both simulations and analyses of the ENZAMET trial in metastatic prostate cancer. Conditional on stopping early for benefit, the method is found to have good unbiasedness and coverage properties, along with very favourable efficiency at earlier interim analyses. We recommend the penalised MLE as a supplementary analysis to a conventional primary analysis when a clinical trial stops early for benefit.

Assessment of Hazard Ratios in Oncology Clinical Trials Terminated Early for Superiority: A Systematic Review

IMPORTANCE

Group sequential designs allow potential early trial termination at the interim analysis, before study completion. Traditional maximum likelihood estimate is commonly used to quantify the treatment effect in group sequential design trials; however, in published clinical trials, a bias-adjusted estimator has rarely been reported.

OBJECTIVE

To emphasize the need for considering overestimation of treatment effect by applying 2 bias-adjusted estimators to previously published, early-terminated oncology clinical trials.

EVIDENCE REVIEW

Trials published from 2013 to 2017 were identified by searching MEDLINE and Embase on February 23, 2018. This review was restricted to oncology clinical trials using group sequential designs with a single preplanned interim analysis as well as 2-arm randomized clinical trials that were subsequently stopped for efficacy reasons. Each article was independently reviewed by 3 biostatisticians during text screening, and differences in opinion were resolved by discussion. This report presents the unadjusted hazard ratio (HR) of an experimental arm to a reference arm and 2 bias-adjusted HRs calculated by using the conditional mean-adjusted estimator (CMAE) and weighted CMAE (WCMAE).

FINDINGS

In total, 198 abstracts were screened for eligibility, of which, 19 eligible clinical trials were identified as applicable to the bias-adjusted estimators. Unadjusted HRs ranged from 0.203 (95% CI, 0.150-0.276) to 0.71 (95% CI, 0.60-0.84), number of events at the interim analysis from 58 to 540, and information time from 48% to 82%. In each study, the HRs adjusted by CMAE and WCMAE were higher than the unadjusted HR. Bias-adjusted estimates in large trials (243 and 414 events at the interim analysis) were similar to the unadjusted HR. However, in small trials (eg, with 58 events at the interim analysis), bias-adjusted estimates were highly disparate from the unadjusted HR. In trials with large treatment effects (eg, HRs of 0.20 and 0.22), the difference between unadjusted and bias-adjusted HRs was small even though the number of events at the interim analysis was small; larger differences were observed when the unadjusted HR was greater than 0.5.

CONCLUSIONS AND RELEVANCE

In this systematic review of oncology clinical trials that were stopped for efficacy at the interim analysis, relatively large differences were noted between the unadjusted and adjusted HRs when the number of events at the interim analysis was small or when the unadjusted HR was close to the boundaries. These findings suggest presenting the 2 bias-adjusted HRs along with the unadjusted HR in the data monitoring committee meeting.

Reducing overestimation of the treatment effect by interim analysis when designing clinical trials

WHAT IS KNOWN AND OBJECTIVE

Several researchers in the statistical and medical communities have noted the overestimation of the treatment effect when a trial is stopped early in the interim analysis for efficacy; however, methods to reduce this overestimation are rarely used because the overestimation mechanisms are not well understood by many in clinical trial practice. A trial design that leads to less overestimation is needed.

METHODS

A computer simulation of hypothetical clinical trials is used to visually explain why the overestimation occurs. A quantitative evaluation of the magnitude of the overestimation is made according to the characteristics of the trial design, such as the total number of events, number of events in the interim analysis, proportion of the number of events to total events and the type of α-spending function.

RESULTS AND DISCUSSION

When the total number of events was more than or equal to 300 and the proportion of the interim events was larger than 50%, the overestimation was acceptable. Moreover, even if the total number of events was 150, the overestimation was sufficiently small when the proportion of the interim events was >70% and a Pocock type α-spending function was used. The overestimation decreased when the total number of events and the proportion of the number of events in the interim analysis increased. In addition, the overestimation of the Pocock type α-spending function was smaller in comparison with that of the O'Brien-Fleming type, which is widely accepted for confirmatory trials.

WHAT IS NEW AND CONCLUSION

We recommend setting the proportion of events in the interim analysis at 50% when the O'Brien-Fleming type α-spending function is used in confirmatory trials to reduce the risk of overestimation. In contrast, the Pocock type boundary could be used in explanatory trials.

The reverse metabolic syndrome in the elderly: Is it a "catabolic" syndrome?

Traditional risk factors of cardiovascular death in the general population, including body mass index (BMI), serum cholesterol, and blood pressure are also found to relate to outcomes in the geriatric population, but in a differing direction. A higher body mass index, hypercholesterolemia and hypertension are not harmful but even permit better survival at advancing age. This phenomenon is called "reverse epidemiology" or "risk factor paradox" and is also detected in a variety of chronic disease states such as chronic heart failure. Accordingly, a low BMI, blood pressure and cholesterol values are associated with a worse prognosis. Several possible causes are hypothesized to explain this elderly paradox, but this phenomenon remains controversial and its underlying reasons are poorly understood. The aim of this review is to recognize the factors behind this intriguing phenomenon and analyse the consequences that it can bring in the management of the cardiovascular therapy in elderly patient. Finally, a new phenotype identified as "catabolic syndrome" has been postulated.

Conditional estimation using prior information in 2-stage group sequential designs assuming asymptotic normality when the trial terminated early

Two-stage designs are widely used to determine whether a clinical trial should be terminated early. In such trials, a maximum likelihood estimate is often adopted to describe the difference in efficacy between the experimental and reference treatments; however, this method is known to display conditional bias. To reduce such bias, a conditional mean-adjusted estimator (CMAE) has been proposed, although the remaining bias may be nonnegligible when a trial is stopped for efficacy at the interim analysis. We propose a new estimator for adjusting the conditional bias of the treatment effect by extending the idea of the CMAE. This estimator is calculated by weighting the maximum likelihood estimate obtained at the interim analysis and the effect size prespecified when calculating the sample size. We evaluate the performance of the proposed estimator through analytical and simulation studies in various settings in which a trial is stopped for efficacy or futility at the interim analysis. We find that the conditional bias of the proposed estimator is smaller than that of the CMAE when the information time at the interim analysis is small. In addition, the mean-squared error of the proposed estimator is also smaller than that of the CMAE. In conclusion, we recommend the use of the proposed estimator for trials that are terminated early for efficacy or futility.