Precision of maximum likelihood estimation in adaptive designs

Point estimation for adaptive trial designs II: Practical considerations and guidance

In adaptive clinical trials, the conventional end-of-trial point estimate of a treatment effect is prone to bias, that is, a systematic tendency to deviate from its true value. As stated in recent FDA guidance on adaptive designs, it is desirable to report estimates of treatment effects that reduce or remove this bias. However, it may be unclear which of the available estimators are preferable, and their use remains rare in practice. This article is the second in a two-part series that studies the issue of bias in point estimation for adaptive trials. Part I provided a methodological review of approaches to remove or reduce the potential bias in point estimation for adaptive designs. In part II, we discuss how bias can affect standard estimators and assess the negative impact this can have. We review current practice for reporting point estimates and illustrate the computation of different estimators using a real adaptive trial example (including code), which we use as a basis for a simulation study. We show that while on average the values of these estimators can be similar, for a particular trial realization they can give noticeably different values for the estimated treatment effect. Finally, we propose guidelines for researchers around the choice of estimators and the reporting of estimates following an adaptive design. The issue of bias should be considered throughout the whole lifecycle of an adaptive design, with the estimation strategy prespecified in the statistical analysis plan. When available, unbiased or bias-reduced estimates are to be preferred.

Distribution Theory Following Blinded and Unblinded Sample Size Re-estimation under Parametric Models

Asymptotic distribution theory for maximum likelihood estimators under fixed alternative hypotheses is reported in the literature even though the power of any realistic test converges to one under fixed alternatives. Under fixed alternatives, authors have established that nuisance parameter estimates are inconsistent when sample size re-estimation (SSR) follows blinded randomization. These results have helped to inhibit the use of SSR. In this paper, we argue for local alternatives to be used instead of fixed alternatives. Motivated by Gould and Shih (1998), we treat unavailable treatment assignments in blinded experiments as missing data and rely on single imputation from marginal distributions to fill in for missing data. With local alternatives, it is sufficient to proceed only with the first step of the EM algorithm mimicking imputation under the null hypothesis. Then, we show that blinded and unblinded estimates of the nuisance parameter σ θ 2 are consistent, and re-estimated sample sizes converge to their locally asymptotically optimal values. This theoretical finding is confirmed through Monte-Carlo simulation studies. Practical utility is illustrated through a multiple logistic regression example. We conclude that, for hypothesis testing with a predetermined minimally clinically relevant local effect size, both blinded and unblinded SSR procedures lead to similar sample sizes and power.

Asymptotic properties of maximum likelihood estimators with sample size recalculation

Consider an experiment in which the primary objective is to determine the significance of a treatment effect at a predetermined type I error and statistical power. Assume that the sample size required to maintain these type I error and power will be re-estimated at an interim analysis. A secondary objective is to estimate the treatment effect. Our main finding is that the asymptotic distributions of standardized statistics are random mixtures of distributions, which are non-normal except under certain model choices for sample size re-estimation (SSR). Monte-Carlo simulation studies and an illustrative example highlight the fact that asymptotic distributions of estimators with SSR may differ from the asymptotic distribution of the same estimators without SSR.

Bias induced by adaptive dose-finding designs

There is a long literature on bias in maximum likelihood estimators. Here we demonstrate that adaptive dose-finding procedures (such as Continual Reassessment Methods, Up-and-Down and Interval Designs) themselves induce bias. In particular, with Bernoulli responses and dose assignments that depend on prior responses, we provide an explicit formula for the bias of observed response rates. We illustrate the patterns of bias for designs that aim to concentrate dose allocations around a target dose, which represents a specific quantile of a cumulative response-threshold distribution. For such designs, bias tends to be positive above the target dose and negative below it. To our knowledge, this property of dose-finding designs has not previously been recognized by design developers. We discuss the implications of this bias and suggest a simple shrinkage mitigation formula that improves estimation at doses away from the target.

Conditional estimation in two-stage adaptive designs

We consider conditional estimation in two-stage sample size adjustable designs and the consequent bias. More specifically, we consider a design which permits raising the sample size when interim results look rather promising, and which retains the originally planned sample size when results look very promising. The estimation procedures reported comprise the unconditional maximum likelihood, the conditionally unbiased Rao-Blackwell estimator, the conditional median unbiased estimator, and the conditional maximum likelihood with and without bias correction. We compare these estimators based on analytical results and a simulation study. We show how they can be applied in a real clinical trial setting.

Precision of maximum likelihood estimation in adaptive designs

There has been increasing interest in trials that allow for design adaptations like sample size reassessment or treatment selection at an interim analysis. Ignoring the adaptive and multiplicity issues in such designs leads to an inflation of the type 1 error rate, and treatment effect estimates based on the maximum likelihood principle become biased. Whereas the methodological issues concerning hypothesis testing are well understood, it is not clear how to deal with parameter estimation in designs were adaptation rules are not fixed in advanced so that, in practice, the maximum likelihood estimate (MLE) is used. It is therefore important to understand the behavior of the MLE in such designs. The investigation of Bias and mean squared error (MSE) is complicated by the fact that the adaptation rules need not be fully specified in advance and, hence, are usually unknown. To investigate Bias and MSE under such circumstances, we search for the sample size reassessment and selection rules that lead to the maximum Bias or maximum MSE. Generally, this leads to an overestimation of Bias and MSE, which can be reduced by imposing realistic constraints on the rules like, for example, a maximum sample size. We consider designs that start with k treatment groups and a common control and where selection of a single treatment and control is performed at the interim analysis with the possibility to reassess each of the sample sizes. We consider the case of unlimited sample size reassessments as well as several realistically restricted sample size reassessment rules. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.