Choosing an imbalance metric for covariate-constrained randomization in multiple-arm cluster-randomized trials

Covariate-constrained randomization in cluster randomized 2 × 2 factorial trials: application to a diabetes prevention study

BACKGROUND

Cluster randomized trials (CRTs) are randomized trials where randomization takes place at an administrative level (e.g., hospitals, clinics, or schools) rather than at the individual level. When the number of available clusters is small, researchers may not be able to rely on simple randomization to achieve balance on cluster-level covariates across treatment conditions. If these cluster-level covariates are predictive of the outcome, covariate imbalance may distort treatment effects, threaten internal validity, lead to a loss of power, and increase the variability of treatment effects. Covariate-constrained randomization (CR) is a randomization strategy designed to reduce the risk of imbalance in cluster-level covariates when performing a CRT. Existing methods for CR have been developed and evaluated for two- and multi-arm CRTs but not for factorial CRTs.

METHODS

Motivated by the BEGIN study-a CRT for weight loss among patients with pre-diabetes-we develop methods for performing CR in 2 × 2 factorial cluster randomized trials with a continuous outcome and continuous cluster-level covariates. We apply our methods to the BEGIN study and use simulation to assess the performance of CR versus simple randomization for estimating treatment effects by varying the number of clusters, the degree to which clusters are associated with the outcome, the distribution of cluster level covariates, the size of the constrained randomization space, and analysis strategies.

RESULTS

Compared to simple randomization of clusters, CR in the factorial setting is effective at achieving balance across cluster-level covariates between treatment conditions and provides more precise inferences. When cluster-level covariates are included in the analyses model, CR also results in greater power to detect treatment effects, but power is low compared to unadjusted analyses when the number of clusters is small.

CONCLUSIONS

CR should be used instead of simple randomization when performing factorial CRTs to avoid highly imbalanced designs and to obtain more precise inferences. Except when there are a small number of clusters, cluster-level covariates should be included in the analysis model to increase power and maintain coverage and type 1 error rates at their nominal levels.

Cervical pessary to prevent preterm birth and poor neonatal outcome: An integrity meta-analysis of randomized controlled trials focusing on adherence to the European Medical Device Regulation

BACKGROUND

Findings from randomized trials (RCTs) on cervical pessary treatment to prevent spontaneous preterm birth are inconsistent.

OBJECTIVES

Our hypothesis suggests that adhering to the European Medical Device Regulation (MDR) and following the instructions for use are essential prerequisites for successful therapy. Conversely, the non-adherence to these guidelines will probably contribute to its failure.

SEARCH STRATEGY AND SELECTION CRITERIA

Based on validated criteria from integrity assessments we performed a systematic review identifying 14 RCTs evaluating the effect of cervical pessaries.

DATA COLLECTION AND ANALYSIS

We analyzed the implications of 14 criteria each accounting for 0-2 points of a score reflecting the clinical evaluation plan (CEP) as proposed by the MDR to evaluate the risk-benefit ratio of medical devices.

MAIN RESULTS

Seven RCTs in each singleton and twin pregnancies (5193 "cases") were included, detecting a high heterogeneity within control groups (I2 = 85% and 87%, respectively, P < 0.01). The CEP score varied from 11 to 26 points for all studies. The most common reasons for low scores and potential data compromise were poor recruitment rates, no (completed) power analysis, and no pre-registration, but mainly non-adherence to technical, biological, and clinical equivalence to the instructions for use as required by the MDR. All trials with score values greater than 20 had applied audit procedures. Within this group we found significantly reduced rates of spontaneous preterm birth at less than 34 weeks within the pessary group in singleton (odds ratio 0.28; 95% confidence interval 0.12-0.65) and twin pregnancies (odds ratio 0.30; 95% confidence interval 0.13-0.67). Similarly, there was a significant reduction in the composite poor neonatal outcome in singleton (odds ratio 0.25; 95% confidence interval 0.10-0.61) and twin pregnancies (odds ratio 0.54; 95% confidence interval 0.35-0.82) after a pessary as compared with controls.

CONCLUSION

Non-audited RCTs and meta-analyses mixing studies of different clinical quality as pre-defined by a CEP and the MDR pose the risk for erroneous conclusions.

Minimisation for the design of parallel cluster-randomised trials: An evaluation of balance in cluster-level covariates and numbers of clusters allocated to each arm

BACKGROUND

Cluster-randomised trials often use some form of restricted randomisation, such as stratified- or covariate-constrained randomisation. Minimisation has the potential to balance on more covariates than blocked stratification and can be implemented sequentially unlike covariate-constrained randomisation. Yet, unlike stratification, minimisation has no inbuilt guard to maintain close to a 1:1 allocation. A departure from a 1:1 allocation can be unappealing in a setting with a small number of allocation units such as cluster randomisation which typically include about 30 clusters.

METHODS

Using simulation (10,000 per scenario), we evaluate the performance of a range of minimisation procedures on the likelihood of a 1:1 allocation of clusters (10-80 clusters) to treatment arms, along with its performance on covariate imbalance. The range of minimisation procedures includes varying: the proportion of clusters allocated to the least imbalanced arm (known as the stochastic element) - between 0.7 and 1, percentage of first clusters allocated completely at random (known as the bed-in period) - between 0% and 20% and adding 'number of clusters allocated to each arm' as a covariate in the minimisation algorithm. We additionally include a comparison of stratifying and then minimising within key strata (such as country within a multi country cluster trial) as a potential aid to increasing balance.

RESULTS

Minimisation is unlikely to result in an exact 1:1 allocation unless the stochastic element is set higher than 0.9. For example, with 20 clusters, 2 binary covariates and setting the stochastic element to 0.7: only 41% of the possible randomisations over the 10,000 simulations achieved a 1:1 allocation. While typical sizes of imbalance were small (a difference of two clusters per arm), allocations as extreme as of 10:10 were observed. Adding the 'number of clusters' into the minimisation algorithm reduces this risk slightly, but covariate imbalance increases slightly. Stratifying and then minimising within key strata improve balance within strata but increase imbalance across all clusters, both on the number of clusters and covariate imbalance.

CONCLUSION

In cluster trials, where there are typically about 30 allocation units, when using minimisation, unless the stochastic element is set very high, there is a high risk of not achieving a 1:1 allocation, and a small but nonetheless real risk of an extreme departure from a 1:1 allocation. Stratification with minimisation within key strata (such as country) improves the balance within strata although compromises overall balance.

SHARING Choices: Design and rationale for a pragmatic trial of an advance care planning intervention for older adults with and without dementia in primary care

BACKGROUND

Advance care planning (ACP) and involving family are particularly important in dementia, and primary care is a key setting. The purpose of this trial is to examine the impact and implementation of SHARING Choices, an intervention to improve communication for older adults with and without dementia through proactively supporting ACP and family engagement in primary care.

METHODS

We cluster-randomized 55 diverse primary care practices across two health systems to the intervention or usual care. SHARING Choices is a multicomponent intervention that aims to improve communication through patient and family engagement in ACP, agenda setting, and shared access to the patient portal for all patients over 65 years of age. The primary outcomes include documentation of an advance directive or medical orders for life-sustaining treatment in the electronic health record (EHR) at 12 months for all patients and receipt of potentially burdensome care within 6 months of death for the subgroup of patients with serious illness. We plan a priori sub-analysis for patients with dementia. Data sources include the health system EHRs and the Maryland health information exchange. We use a mixed-methods approach to evaluate uptake, fidelity and adaptation of the intervention and implementation facilitators and barriers.

CONCLUSIONS

This cluster-randomized pragmatic trial examines ACP with a focus on the key population of those with dementia, implementation in diverse settings and innovative approaches to trial design and outcome abstraction. Mixed-methods approaches enable understanding of intervention delivery and facilitators and barriers to implementation in rapidly changing health care systems.

CLINICALTRIALS

gov Identifier: NCT04819191.

Constrained randomization and statistical inference for multi‐arm parallel cluster randomized controlled trials

A practical limitation of cluster randomized controlled trials (cRCTs) is that the number of available clusters may be small, resulting in an increased risk of baseline imbalance under simple randomization. Constrained randomization overcomes this issue by restricting the allocation to a subset of randomization schemes where sufficient overall covariate balance across comparison arms is achieved. However, for multi-arm cRCTs, several design and analysis issues pertaining to constrained randomization have not been fully investigated. Motivated by an ongoing multi-arm cRCT, we elaborate the method of constrained randomization and provide a comprehensive evaluation of the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs in multi-arm cRCTs, with varying combinations of design and analysis-based covariate adjustment strategies. In particular, as randomization-based tests have not been extensively studied in multi-arm cRCTs, we additionally develop most-powerful randomization tests under the linear mixed model framework for our comparisons. Our results indicate that under constrained randomization, both model-based and randomization-based analyses could gain power while preserving nominal type I error rate, given proper analysis-based adjustment for the baseline covariates. Randomization-based analyses, however, are more robust against violations of distributional assumptions. The choice of balance metrics and candidate set sizes and their implications on the testing of the pairwise and global hypotheses are also discussed. Finally, we caution against the design and analysis of multi-arm cRCTs with an extremely small number of clusters, due to insufficient degrees of freedom and the tendency to obtain an overly restricted randomization space.

Comparing the effectiveness of home visiting paraprofessionals and mental health professionals delivering a postpartum depression preventive intervention: a cluster-randomized non-inferiority clinical trial

To determine whether pregnant women receiving the Mothers and Babies group-based intervention exhibited greater depressive symptom reductions and fewer new cases of major depression than women receiving usual community-based services, and to examine whether groups run by paraprofessional home visitors and mental health professionals yielded similar depressive symptom reductions and prevention of major depression. Using a cluster-randomized design, 37 home visiting programs were randomized to usual home visiting, Mothers and Babies delivered via home visiting paraprofessionals, or Mothers and Babies delivered via mental health professionals. Baseline assessments were conducted prenatally with follow-up extending to 24 weeks postpartum. Eligibility criteria were ≥ 16 years old, ≤ 33 gestation upon referral, and Spanish/English speaking. Depressive symptoms at 24 weeks postpartum was the primary outcome. Eight hundred seventy-four women were enrolled. Neither intervention arm was superior to usual care in decreasing depressive symptoms across the sample (p = 0.401 home visiting paraprofessional vs. control; p = 0.430 mental health professional vs. control). Post hoc analyses suggest a positive intervention effect for women exhibiting mild depressive symptoms at baseline. We have evidence of non-inferiority, as the model-estimated mean difference in depressive symptoms between intervention arms (0.01 points, 95% CI: -0.79, 0.78) did not surpass our pre-specified margin of non-inferiority of two points. Although we did not find statistically significant differences between intervention and control arms, non-inferiority analyses found paraprofessional home visitors generated similar reductions in depressive symptoms as mental health professionals. Additionally, Mothers and Babies appears to reduce depressive symptoms among women with mild depressive symptoms when delivered by mental health professionals. This trial is registered on ClinicalTrials.gov (initial post: December 1, 2016; identifier: NCT02979444).

Using the half normal distribution to quantify covariate balance in cluster-randomized pragmatic trials

Background

Pragmatic trials often consist of cluster-randomized controlled trials (C-RCTs), where staff of existing clinics or sites deliver interventions and randomization occurs at the site level. Covariate-constrained randomization (CCR) methods are often recommended to minimize imbalance on important site characteristics across intervention and control arms because sizable imbalances can occur by chance in simple randomizations when the number of units to be randomized is relatively small. CCR methods involve multiple random assignments initially, an assessment of balance achieved on site-level covariates from each randomization, and the final selection of an allocation that produces acceptable balance. However, no clear consensus exists on how to assess imbalance or identify allocations with sufficient balance. In this article, we describe an overall imbalance index (I) that is based on the mean of the absolute value of the standardized differences in means on the site characteristics.

Methods

We derive the theoretical distribution of I, then conduct simulation studies to examine its empirical properties under the varying covariate distributions and inter-correlations.

Results

I has an expected value of 0.798 and, assuming independent site characteristics, a variance of 0.363/k, where k is the number of site characteristics being balanced. Simulations indicated that the properties of I are robust under varying covariate circumstances as long as k is greater than 3 and the covariates are not too highly inter-correlated.

Conclusions

We recommend that values of I below the 10th percentile indicate sufficient overall site balance in CCRs. Definitions of acceptable randomizations might also include individual covariate criteria specified in advance, in addition to overall balance criteria.

Design and analysis of three-arm parallel cluster randomized trials with small numbers of clusters

In this article, we review and evaluate a number of methods used in the design and analysis of small three-arm parallel cluster randomized trials. We conduct a simulation-based study to evaluate restricted randomization methods including covariate-constrained randomization and a novel method for matched-group cluster randomization. We also evaluate the appropriate modelling of the data and small sample inferential methods for a variety of treatment effects relevant to three-arm trials. Our results indicate that small-sample corrections are required for high (0.05) but not low (0.001) values of the intraclass correlation coefficient and their performance can depend on trial design, number of clusters, and the nature of the hypothesis being tested. The Satterthwaite correction generally performed best at an ICC of 0.05 with a nominal type I error rate for single-period trials, and in trials with repeated measures type I error rates were between 0.04 and 0.06. Restricted randomization methods produce little benefit in trials with repeated measures but in trials with single post-intervention design can provide relatively large gains in power when compared to the most unbalanced possible allocations. Matched-group randomization improves power but is not as effective as covariate-constrained randomization. For model-based analysis, adjusting for fewer covariates than were used in a restricted randomization process under any design can produce non-nominal type I error rates and reductions in power. Where comparisons to two-arm cluster trials are possible, the performance of the methods is qualitatively very similar.

Application of randomization techniques for balancing site covariates in the adult day service plus pragmatic cluster-randomized trial

Cluster-randomized trials (CRTs) are increasingly common in pragmatic trials of interventions for older adults, where staff of existing clinics or service agencies deliver interventions. The Adult Day Service (ADS) Plus intervention is delivered by trained staff at adult day service facilities to assist older adults with cognitive impairments and their family caregivers. Because sizable imbalances on important site characteristics might emerge from a simple randomization, we implemented a 3-stage constrained randomization approach to limit imbalance between intervention and usual care control conditions on 5 site characteristics: capacity; % of minority clients; % of clients with dementia; urban, rural or suburban location; and private or public ownership. In stage 1, the Balance Match Weighted (BMW) re-randomization procedure was used to assign 30 sites to ADS Plus or control arms based on the best randomization out of 20 total randomizations for minimizing site imbalance. In stage 2, propensity scores from the BMW logistic regression analysis for reserve sites were used to determine substitutions for randomized sites that opted out of the CRT prior to implementation. In stage 3, a minimization approach was used to add 20 more sites to the trial. A standardized metric based on the half-normal distribution of the absolute value of mean differences was used to assess site imbalance. After stage 3, the remaining imbalance for the 49 enrolled sites was reduced by 75% from what would have been expected from a simple randomization. Optimized randomization procedures with similar imbalance metrics should be used more routinely in pragmatic CRTs.