Design and analysis of three-arm parallel cluster randomized trials with small numbers of clusters

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Optimal study designs for cluster randomised trials: An overview of methods and results

There are multiple possible cluster randomised trial designs that vary in when the clusters cross between control and intervention states, when observations are made within clusters, and how many observations are made at each time point. Identifying the most efficient study design is complex though, owing to the correlation between observations within clusters and over time. In this article, we present a review of statistical and computational methods for identifying optimal cluster randomised trial designs. We also adapt methods from the experimental design literature for experimental designs with correlated observations to the cluster trial context. We identify three broad classes of methods: using exact formulae for the treatment effect estimator variance for specific models to derive algorithms or weights for cluster sequences; generalised methods for estimating weights for experimental units; and, combinatorial optimisation algorithms to select an optimal subset of experimental units. We also discuss methods for rounding experimental weights, extensions to non-Gaussian models, and robust optimality. We present results from multiple cluster trial examples that compare the different methods, including determination of the optimal allocation of clusters across a set of cluster sequences and selecting the optimal number of single observations to make in each cluster-period for both Gaussian and non-Gaussian models, and including exchangeable and exponential decay covariance structures.

Permutation-based multiple testing corrections for P $$ P $$ -values and confidence intervals for cluster randomized trials

In this article, we derive and compare methods to derive P-values and sets of confidence intervals with strong control of the family-wise error rates and coverage for estimates of treatment effects in cluster randomized trials with multiple outcomes. There are few methods for P-value corrections and deriving confidence intervals, limiting their application in this setting. We discuss the methods of Bonferroni, Holm, and Romano and Wolf and adapt them to cluster randomized trial inference using permutation-based methods with different test statistics. We develop a novel search procedure for confidence set limits using permutation tests to produce a set of confidence intervals under each method of correction. We conduct a simulation-based study to compare family-wise error rates, coverage of confidence sets, and the efficiency of each procedure in comparison to no correction using both model-based standard errors and permutation tests. We show that the Romano-Wolf type procedure has nominal error rates and coverage under non-independent correlation structures and is more efficient than the other methods in a simulation-based study. We also compare results from the analysis of a real-world trial.

Evaluation of an organisational-level monetary incentive to promote the health and wellbeing of workers in small and medium-sized enterprises: A mixed-methods cluster randomised controlled trial

We conducted an independent evaluation on the effectiveness of an organisational-level monetary incentive to encourage small and medium-sized enterprises (SMEs) to improve employees' health and wellbeing. This was A mixed-methods cluster randomised trial with four arms: high monetary incentive, low monetary incentive, and two no monetary incentive controls (with or without baseline measurements to examine 'reactivity' The consequence of particpant awareness of being studied, and potential impact on participant behavior effects). SMEs with 10-250 staff based in West Midlands, England were eligible. We randomly selected up to 15 employees at baseline and 11 months post-intervention. We elicited employee perceptions of employers' actions to improve health and wellbeing; and employees' self-reported health behaviours and wellbeing. We also interviewed employers and obtained qualitative data. One hundred and fifty-two SMEs were recruited. Baseline assessments were conducted in 85 SMEs in three arms, and endline assessments in 100 SMEs across all four arms. The percentage of employees perceiving "positive action" by their employer increased after intervention (5 percentage points, pp [95% Credible Interval -3, 21] and 3pp [-9, 17], in models for high and low incentive groups). Across six secondary questions about specific issues the results were strongly and consistently positive, especially for the high incentive. This was consistent with qualitative data and quantitative employer interviews. However, there was no evidence of any impact on employee health behaviour or wellbeing outcomes, nor evidence of 'reactivity'. An organisational intervention (a monetary incentive) changed employee perceptions of employer behaviour but did not translate into changes in employees' self-reports of their own health behaviours or wellbeing. Trial registration: AEARCTR-0003420, registration date: 17.10.2018, retrospectively registered (delays in contracts and identfying a suitable trial registry). The authors confirm that there are no ongoing and related trials for this intervention.

Models for cluster randomized designs using ranked set sampling

Cluster randomized designs (CRD) provide a rigorous development for randomization principles for studies where treatments are allocated to cluster units rather than the individual subjects within clusters. It is known that CRDs are less efficient than completely randomized designs since the randomization of treatment allocation is applied to the cluster units. To mitigate this problem, we embed a ranked set sampling design from survey sampling studies into CRD for the selection of both cluster and subsampling units. We show that ranking groups in ranked set sampling act like a covariate, reduce the expected mean squared cluster error, and increase the precision of the sampling design. We provide an optimality result to determine the sample sizes at cluster and sub-sample level. We apply the proposed sampling design to a dental study on human tooth size, and to a longitudinal study from an education intervention program.

Minimisation for the design of parallel cluster-randomised trials: An evaluation of balance in cluster-level covariates and numbers of clusters allocated to each arm

BACKGROUND

Cluster-randomised trials often use some form of restricted randomisation, such as stratified- or covariate-constrained randomisation. Minimisation has the potential to balance on more covariates than blocked stratification and can be implemented sequentially unlike covariate-constrained randomisation. Yet, unlike stratification, minimisation has no inbuilt guard to maintain close to a 1:1 allocation. A departure from a 1:1 allocation can be unappealing in a setting with a small number of allocation units such as cluster randomisation which typically include about 30 clusters.

METHODS

Using simulation (10,000 per scenario), we evaluate the performance of a range of minimisation procedures on the likelihood of a 1:1 allocation of clusters (10-80 clusters) to treatment arms, along with its performance on covariate imbalance. The range of minimisation procedures includes varying: the proportion of clusters allocated to the least imbalanced arm (known as the stochastic element) - between 0.7 and 1, percentage of first clusters allocated completely at random (known as the bed-in period) - between 0% and 20% and adding 'number of clusters allocated to each arm' as a covariate in the minimisation algorithm. We additionally include a comparison of stratifying and then minimising within key strata (such as country within a multi country cluster trial) as a potential aid to increasing balance.

RESULTS

Minimisation is unlikely to result in an exact 1:1 allocation unless the stochastic element is set higher than 0.9. For example, with 20 clusters, 2 binary covariates and setting the stochastic element to 0.7: only 41% of the possible randomisations over the 10,000 simulations achieved a 1:1 allocation. While typical sizes of imbalance were small (a difference of two clusters per arm), allocations as extreme as of 10:10 were observed. Adding the 'number of clusters' into the minimisation algorithm reduces this risk slightly, but covariate imbalance increases slightly. Stratifying and then minimising within key strata improve balance within strata but increase imbalance across all clusters, both on the number of clusters and covariate imbalance.

CONCLUSION

In cluster trials, where there are typically about 30 allocation units, when using minimisation, unless the stochastic element is set very high, there is a high risk of not achieving a 1:1 allocation, and a small but nonetheless real risk of an extreme departure from a 1:1 allocation. Stratification with minimisation within key strata (such as country) improves the balance within strata although compromises overall balance.

Mind the gap: covariate constrained randomisation can protect against substantial power loss in parallel cluster randomised trials

Background

Cluster randomised trials often randomise a small number of units, putting them at risk of poor balance of covariates across treatment arms. Covariate constrained randomisation aims to reduce this risk by removing the worst balanced allocations from consideration. This is known to provide only a small gain in power over that averaged under simple randomisation and is likely influenced by the number and prognostic effect of the covariates.

We investigated the performance of covariate constrained randomisation in comparison to the worst balanced allocations, and considered the impact on the power of the prognostic effect and number of covariates adjusted for in the analysis.

Methods

Using simulation, we examined the Monte Carlo type I error rate and power of cross-sectional, two-arm parallel cluster-randomised trials with a continuous outcome and four binary cluster-level covariates, using either simple or covariate constrained randomisation. Data were analysed using a small sample corrected linear mixed-effects model, adjusted for some or all of the binary covariates. We varied the number of clusters, intra-cluster correlation, number and prognostic effect of covariates balanced in the randomisation and adjusted in the analysis, and the size of the candidate set from which the allocation was selected. For each scenario, 20,000 simulations were conducted.

Results

When compared to the worst balanced allocations, covariate constrained randomisation with an adjusted analysis provided gains in power of up to 20 percentage points. Even with analysis-based adjustment for those covariates balanced in the randomisation, the type I error rate was not maintained when the intracluster correlation is very small (0.001). Generally, greater power was achieved when more prognostic covariates are restricted in the randomisation and as the size of the candidate set decreases. However, adjustment for weakly prognostic covariates lead to a loss in power of up to 20 percentage points.

Conclusions

When compared to the worst balanced allocations, covariate constrained randomisation provides moderate to substantial improvements in power. However, the prognostic effect of the covariates should be carefully considered when selecting them for inclusion in the randomisation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01588-8.

Methods for dealing with unequal cluster sizes in cluster randomized trials: A scoping review

In a cluster-randomized trial (CRT), the number of participants enrolled often varies across clusters. This variation should be considered during both trial design and data analysis to ensure statistical performance goals are achieved. Most methodological literature on the CRT design has assumed equal cluster sizes. This scoping review focuses on methodology for unequal cluster size CRTs. EMBASE, Medline, Google Scholar, MathSciNet and Web of Science databases were searched to identify English-language articles reporting on methodology for unequal cluster size CRTs published until March 2021. We extracted data on the focus of the paper (power calculation, Type I error etc.), the type of CRT, the type and the range of parameter values investigated (number of clusters, mean cluster size, cluster size coefficient of variation, intra-cluster correlation coefficient, etc.), and the main conclusions. Seventy-nine of 5032 identified papers met the inclusion criteria. Papers primarily focused on the parallel-arm CRT (p-CRT, n = 60, 76%) and the stepped-wedge CRT (n = 14, 18%). Roughly 75% of the papers addressed trial design issues (sample size/power calculation) while 25% focused on analysis considerations (Type I error, bias, etc.). The ranges of parameter values explored varied substantially across different studies. Methods for accounting for unequal cluster sizes in the p-CRT have been investigated extensively for Gaussian and binary outcomes. Synthesizing the findings of these works is difficult as the magnitude of impact of the unequal cluster sizes varies substantially across the combinations and ranges of input parameters. Limited investigations have been done for other combinations of a CRT design by outcome type, particularly methodology involving binary outcomes-the most commonly used type of primary outcome in trials. The paucity of methodological papers outside of the p-CRT with Gaussian or binary outcomes highlights the need for further methodological development to fill the gaps.