Quantifying the cost in power of ignoring continuous covariate imbalances in clinical trial randomization

Simulated annealing for balancing covariates

Covariate balance is one of the fundamental issues in designing experiments for treatment comparisons, especially in randomized clinical trials. In this article, we introduce a new class of covariate-adaptive procedures based on the Simulated Annealing algorithm aimed at balancing the allocations of two competing treatments across a set of pre-specified covariates. Due to the nature of the simulated annealing, these designs are intrinsically randomized, thus completely unpredictable, and very flexible: they can manage both quantitative and qualitative factors and be implemented in a static version as well as sequentially. The properties of the suggested proposal are described, showing a significant improvement in terms of covariate balance and inferential accuracy with respect to all the other procedures proposed in the literature. An illustrative example based on real data is also discussed.

Multi-arm covariate-adaptive randomization

Simultaneously investigating multiple treatments in a single study achieves considerable efficiency in contrast to the traditional two-arm trials. Balancing treatment allocation for influential covariates has become increasingly important in today's clinical trials. The multi-arm covariate-adaptive randomized clinical trial is one of the most powerful tools to incorporate covariate information and multiple treatments in a single study. Pocock and Simon's procedure has been extended to the multi-arm case. However, the theoretical properties of multi-arm covariate-adaptive randomization have remained largely elusive for decades. In this paper, we propose a general framework for multi-arm covariate-adaptive designs which also includes the two-arm case, and establish the corresponding theory under widely satisfied conditions. The theoretical results provide new insights into the balance properties of covariate-adaptive randomization procedures and make foundations for most existing statistical inferences under two-arm covariate-adaptive randomization. Furthermore, these open a door to study the theoretical properties of statistical inferences for clinical trials based on multi-arm covariate-adaptive randomization procedures.

Bryostatin Placebo-Controlled Trials Indicate Cognitive Restoration Above Baseline for Advanced Alzheimer’s Disease in the Absence of Memantine1

Background: In pre-clinical studies of Alzheimer’s disease (AD) transgenic mice, bryostatin restored synaptic connections, prevented neuronal death, reduced amyloid plaques, and reduced neurofibrillary tangles. Objective: Within pre-specified cohorts of advanced AD patients in two double-blind placebo-controlled bryostatin Phase II trials, to conduct exploratory statistical analyses of patients with identical conditions of enrollment and treatment. Methods: Severe Impairment Battery (SIB) scores above baseline at 5, 9, and 13 weeks were analyzed initially in the complete cases, with multiple imputation methods based on an iterative Markov chain Monte Carlo algorithm used for missing SIB scores. To mitigate confounding by a chance imbalance of 4.9 SIB baseline scores (Study #203), each patient was used as their own control with differences in 13-week SIB from baseline in single trial and pooled analyses to measure benefit at 13 weeks using general estimating equations (GEE) modeling. Results: Patients treated with bryostatin pre-specified at Mini-Mental State Examination scores 10–14, without memantine, showed baseline balance, complete safety, and SIB improvements at 13 weeks with multiple imputation analysis: Study #203 = 4.1 SIB points above baseline (p = 0.005), and Study #202 = 4.2 SIB points above baseline (p = 0.016). An increased power (N = 95) “pooled analysis” showed an increased SIB over time and a higher mean SIB at 13 weeks in the bryostatin treatment group (p <  0.001) but not significant (NS) for the placebo patients. Conclusion: Pre-specified exploratory analyses for the individual trials and the pooled trials confirmed significant bryostatin-induced improvement over baseline (treatment p <  0.001, placebo NS).

Impact of minimal sufficient balance, minimization, and stratified permuted blocks on bias and power in the estimation of treatment effect in sequential clinical trials with a binary endpoint

Minimization is among the most common methods for controlling baseline covariate imbalance at the randomization phase of clinical trials. Previous studies have found that minimization does not preserve allocation randomness as well as other methods, such as minimal sufficient balance, making it more vulnerable to allocation predictability and selection bias. Additionally, minimization has been shown in simulation studies to inadequately control serious covariate imbalances when modest biased coin probabilities (≤0.65) are used. This current study extends the investigation of randomization methods to the analysis phase, comparing the impact of treatment allocation methods on power and bias in estimating treatment effects on a binary outcome using logistic regression. Power and bias in the estimation of treatment effect was found to be comparable across complete randomization, minimization, and minimal sufficient balance in unadjusted analyses. Further, minimal sufficient balance was found to have the most modest impact on power and the least bias in covariate-adjusted analyses. The minimal sufficient balance method is recommended for use in clinical trials as an alternative to minimization when covariate-adaptive subject randomization takes place.

Incorporating covariates information in adaptive clinical trials for precision medicine

Precision medicine is the systematic use of information that pertains to an individual patient to select or optimize that patient's preventative and therapeutic care. Recent studies have classified biomarkers into predictive and prognostic biomarkers based on their roles in clinical studies. To design a clinical trial for precision medicine, predictive biomarkers and prognostic biomarkers should both be included. In statistical analysis, biomarkers are mathematically treated as covariates. We first classify covariates into predictive and prognostic covariates according to their roles. We then provide a brief review of recent advances in adaptive designs that incorporate covariates. However, the literature includes no designs that incorporate both prognostic covariates and predictive covariates simultaneously. In this paper, we propose a new family of covariate-adjusted response-adaptive (CARA) designs that incorporate both prognostic and predictive covariates and the responses. It is important to note that the predictive biomarkers and prognostic biomarkers play different roles in the new designs. The advantages of the proposed methods are demonstrated via numerical studies, and some further statistical issues are also discussed.

Design of healthy hearts in the heartland (H3): A practice-randomized, comparative effectiveness study

BACKGROUND

The Healthy Hearts in the Heartland (H3) study is part of a nationwide effort, EvidenceNOW, seeking to better understand the ability of small primary care practices to improve "ABCS" clinical quality measures: appropriate Aspirin therapy, Blood pressure control, Cholesterol management, and Smoking cessation. H3 aimed to assess feasibility of implementing Point-of-Care (POC) or POC plus Population Management (POC + PM) quality improvement (QI) strategies to improve ABCS at practices in Illinois, Indiana, and Wisconsin. We describe the design and randomization of the H3 study.

METHODS

We conducted a two-arm (1:1, POC:POC + PM), practice-randomized, comparative effectiveness study in 226 primary care practices across four "waves" of randomization with a 12-month intervention period, followed by a six-month sustainability period. Randomization controlled imbalance in nine baseline variables through a modified constrained algorithm. Among others, we used initial, unverified estimates of baseline ABCS values.

RESULTS

We randomized 112 and 114 practices to POC and POC + PM arms, respectively. Randomization ensured baseline comparability for all nine key variables, including the ABCS measures indicating proportion of patients at the practice level meeting each quality measure. Median(Inner Quartile Range) values were A: 0.78(0.66-0.86) in POC arm vs. 0.77(0.63-0.86) in POC + PM arm, B: 0.64(0.53-0.73) vs. 0.64(0.53-0.75), C: 0.78(0.63-0.86) vs. 0.75(0.64-0.81), S: 0.80(0.65-0.81) vs. 0.79(0.61-0.91).

DISCUSSION

Surrogate estimates for the true ABCS at baseline coupled with the unique randomization logic achieved adequate baseline balance on these outcomes. Similar practice- or cluster-randomized trials may consider adaptations of this design. Final analyses on 12- and 18-month ABCS outcomes for the H3 study are forthcoming.

TRIAL REGISTRATION

This trial is registered on ClinicalTrials.gov (Initial post: 11/05/2015; identifier: NCT02598284; https://clinicaltrials.gov/ct2/show/NCT02598284?term=NCT02598284&rank=1).

Testing hypotheses under adaptive randomization with continuous covariates in clinical trials

Covariate-adaptive designs are widely used to balance covariates and maintain randomization in clinical trials. Adaptive designs for discrete covariates and their asymptotic properties have been well studied in the literature. However, important continuous covariates are often involved in clinical studies. Simply discretizing or categorizing continuous covariates can result in loss of information. The current understanding of adaptive designs with continuous covariates lacks a theoretical foundation as the existing works are entirely based on simulations. Consequently, conventional hypothesis testing in clinical trials using continuous covariates is still not well understood. In this paper, we establish a theoretical framework for hypothesis testing on adaptive designs with continuous covariates based on linear models. For testing treatment effects and significance of covariates, we obtain the asymptotic distributions of the test statistic under null and alternative hypotheses. Simulation studies are conducted under a class of covariate-adaptive designs, including the p-value-based method, the Su’s percentile method, the empirical cumulative-distribution method, the Kullback–Leibler divergence method, and the kernel-density method. Key findings about adaptive designs with independent covariates based on linear models are (1) hypothesis testing that compares treatment effects are conservative in terms of smaller type I error, (2) hypothesis testing using adaptive designs outperforms complete randomization method in terms of power, and (3) testing on significance of covariates is still valid.

Minimal sufficient balance-a new strategy to balance baseline covariates and preserve randomness of treatment allocation

In many clinical trials, baseline covariates could affect the primary outcome. Commonly used strategies to balance baseline covariates include stratified constrained randomization and minimization. Stratification is limited to few categorical covariates. Minimization lacks the randomness of treatment allocation. Both apply only to categorical covariates. As a result, serious imbalances could occur in important baseline covariates not included in the randomization algorithm. Furthermore, randomness of treatment allocation could be significantly compromised because of the high proportion of deterministic assignments associated with stratified block randomization and minimization, potentially resulting in selection bias. Serious baseline covariate imbalances and selection biases often contribute to controversial interpretation of the trial results. The National Institute of Neurological Disorders and Stroke recombinant tissue plasminogen activator Stroke Trial and the Captopril Prevention Project are two examples. In this article, we propose a new randomization strategy, termed the minimal sufficient balance randomization, which will dually prevent serious imbalances in all important baseline covariates, including both categorical and continuous types, and preserve the randomness of treatment allocation. Computer simulations are conducted using the data from the National Institute of Neurological Disorders and Stroke recombinant tissue plasminogen activator Stroke Trial. Serious imbalances in four continuous and one categorical covariate are prevented with a small cost in treatment allocation randomness. A scenario of simultaneously balancing 11 baseline covariates is explored with similar promising results. The proposed minimal sufficient balance randomization algorithm can be easily implemented in computerized central randomization systems for large multicenter trials.

Covariate imbalance and adjustment for logistic regression analysis of clinical trial data

In logistic regression analysis for binary clinical trial data, adjusted treatment effect estimates are often not equivalent to unadjusted estimates in the presence of influential covariates. This article uses simulation to quantify the benefit of covariate adjustment in logistic regression. However, International Conference on Harmonization guidelines suggest that covariate adjustment be prespecified. Unplanned adjusted analyses should be considered secondary. Results suggest that if adjustment is not possible or unplanned in a logistic setting, balance in continuous covariates can alleviate some (but never all) of the shortcomings of unadjusted analyses. The case of log binomial regression is also explored.

Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study

Background

Analysis of variance (ANOVA), change-score analysis (CSA) and analysis of covariance (ANCOVA) respond differently to baseline imbalance in randomized controlled trials. However, no empirical studies appear to have quantified the differential bias and precision of estimates derived from these methods of analysis, and their relative statistical power, in relation to combinations of levels of key trial characteristics. This simulation study therefore examined the relative bias, precision and statistical power of these three analyses using simulated trial data.

Methods

126 hypothetical trial scenarios were evaluated (126 000 datasets), each with continuous data simulated by using a combination of levels of: treatment effect; pretest-posttest correlation; direction and magnitude of baseline imbalance. The bias, precision and power of each method of analysis were calculated for each scenario.

Results

Compared to the unbiased estimates produced by ANCOVA, both ANOVA and CSA are subject to bias, in relation to pretest-posttest correlation and the direction of baseline imbalance. Additionally, ANOVA and CSA are less precise than ANCOVA, especially when pretest-posttest correlation ≥ 0.3. When groups are balanced at baseline, ANCOVA is at least as powerful as the other analyses. Apparently greater power of ANOVA and CSA at certain imbalances is achieved in respect of a biased treatment effect.

Conclusions

Across a range of correlations between pre- and post-treatment scores and at varying levels and direction of baseline imbalance, ANCOVA remains the optimum statistical method for the analysis of continuous outcomes in RCTs, in terms of bias, precision and statistical power.

Continuous covariate imbalance and conditional power for clinical trial interim analyses

Oftentimes valid statistical analyses for clinical trials involve adjustment for known influential covariates, regardless of imbalance observed in these covariates at baseline across treatment groups. Thus, it must be the case that valid interim analyses also properly adjust for these covariates. There are situations, however, in which covariate adjustment is not possible, not planned, or simply carries less merit as it makes inferences less generalizable and less intuitive. In this case, covariate imbalance between treatment groups can have a substantial effect on both interim and final primary outcome analyses. This paper illustrates the effect of influential continuous baseline covariate imbalance on unadjusted conditional power (CP), and thus, on trial decisions based on futility stopping bounds. The robustness of the relationship is illustrated for normal, skewed, and bimodal continuous baseline covariates that are related to a normally distributed primary outcome. Results suggest that unadjusted CP calculations in the presence of influential covariate imbalance require careful interpretation and evaluation.

Assessment and implication of prognostic imbalance in randomized controlled trials with a binary outcome--a simulation study

BACKGROUND

Chance imbalance in baseline prognosis of a randomized controlled trial can lead to over or underestimation of treatment effects, particularly in trials with small sample sizes. Our study aimed to (1) evaluate the probability of imbalance in a binary prognostic factor (PF) between two treatment arms, (2) investigate the impact of prognostic imbalance on the estimation of a treatment effect, and (3) examine the effect of sample size (n) in relation to the first two objectives.

METHODS

We simulated data from parallel-group trials evaluating a binary outcome by varying the risk of the outcome, effect of the treatment, power and prevalence of the PF, and n. Logistic regression models with and without adjustment for the PF were compared in terms of bias, standard error, coverage of confidence interval and statistical power.

RESULTS

For a PF with a prevalence of 0.5, the probability of a difference in the frequency of the PF≥5% reaches 0.42 with 125/arm. Ignoring a strong PF (relative risk = 5) leads to underestimating the strength of a moderate treatment effect, and the underestimate is independent of n when n is >50/arm. Adjusting for such PF increases statistical power. If the PF is weak (RR = 2), adjustment makes little difference in statistical inference. Conditional on a 5% imbalance of a powerful PF, adjustment reduces the likelihood of large bias. If an absolute measure of imbalance ≥5% is deemed important, including 1000 patients/arm provides sufficient protection against such an imbalance. Two thousand patients/arm may provide an adequate control against large random deviations in treatment effect estimation in the presence of a powerful PF.

CONCLUSIONS

The probability of prognostic imbalance in small trials can be substantial. Covariate adjustment improves estimation accuracy and statistical power, and hence should be performed when strong PFs are observed.

Balancing Treatment Allocation over Continuous Covariates: A New Imbalance Measure for Minimization

In many clinical trials, it is important to balance treatment allocation over covariates. Although a great many papers have been published on balancing over discrete covariates, the procedures for continuous covariates have been less well studied. Traditionally, a continuous covariate usually needs to be transformed to a discrete one by splitting its range into several categories. Such practice may lead to loss of information and is susceptible to misspecification of covariate distribution. The more recent papers seek to define an imbalance measure that preserves the nature of continuous covariates and set the allocation rule in order to minimize that measure. We propose a new design, which defines the imbalance measure by the maximum assignment difference when all possible divisions of the covariate range are considered. This measure depends only on ranks of the covariate values and is therefore free of covariate distribution. In addition, we developed an efficient algorithm to implement the new procedure. By simulation studies we show that the new procedure is able to keep good balance properties in comparison with other popular designs.

Measuring continuous baseline covariate imbalances in clinical trial data

This paper presents and compares several methods of measuring continuous baseline covariate imbalance in clinical trial data. Simulations illustrate that though the t-test is an inappropriate method of assessing continuous baseline covariate imbalance, the test statistic itself is a robust measure in capturing imbalance in continuous covariate distributions. Guidelines to assess effects of imbalance on bias, type I error rate and power for hypothesis test for treatment effect on continuous outcomes are presented, and the benefit of covariate-adjusted analysis (ANCOVA) is also illustrated.

Balancing multiple baseline characteristics in randomized clinical trials

It is of vital importance to the success of a randomized clinical trial to maintain the balance of baseline characteristics (covariates) that could potentially confound the outcomes of the trial. Various randomization methods have been proposed to increase the likelihood of having balanced covariates at the end of a trial, most of which only apply to categorical covariates. An optimization approach to maintaining the balance of multiple covariates in randomized clinical trials is proposed, which is applicable to both continuous and categorical covariates and allows the covariates to be ranked according to their clinical importance as perceived by the clinical trial practitioners. Simulation results show that the proposed algorithm significantly outperforms the standard randomization approach via flipping an unbiased coin. The proposed randomization method can be easily implemented and generalized for cases where there are multiple treatment arms with equal or unequal randomization probabilities.