Unequal cluster sizes in stepped-wedge cluster randomised trials: a systematic review

Crown adjustment and chairside efficiency of single-unit restorations fabricated from immediate and staged impressions using a digital workflow for posterior implants

PURPOSE

This is a clinical study to compare immediate and staged impression methods in a complete digital workflow for single-unit implants in the posterior area.

MATERIALS AND METHODS

Sixty patients requiring single-unit implant crowns were enrolled. Forty patients were assigned to the test group, immediate digital impression after implant surgery with crown delivery 4 months later. The remaining 20 patients were assigned to the control group, staged digital impressions 4 months after implant surgery, and crown delivery 1 month later. Both workflows involved free-model CAD-CAM crown fabrications. The crowns were scanned before and after clinical adjustment using an intraoral scanner (TRIOS Color; 3Shape). Two 3D digital models were trimmed and superimposed to evaluate the dimensional changes using Geomagic Control software. Chairside times for the entire workflow were recorded. Kruskal-Wallis was performed to compare crown adjustments between two groups, while One-way ANOVA was used to compare chairside time durations between the test and control groups.

RESULTS

All crowns were delivered without refabrication. The average maximum occlusion adjustment of crowns was -353.2 ± 207.1 μm in the test group and -212.7 ± 150.5 μm in the control group (p = 0.02). The average area of occlusal adjustment, measured as an area of deviation larger than 100 μm, was 14.8 ± 15.3 and 8.4 ± 8.1 mm2 in the test and control groups, respectively (p = 0.056). There were no significant differences in the mesial and distal contact adjustment amounts, or the maximum deviations of the proximal area, between the two groups. The mean chair-side time was 50.25 ± 13.48 and 51.20 ± 5.34 min in the test and control groups, respectively (p = 0.763).

CONCLUSIONS

The immediate impression method in the digital workflow for single-unit implants required more occlusal adjustments of crowns but showed similar chairside times compared to the staged impression method.

Bayesian optimal stepped wedge design

Recently, there has been a growing interest in designing cluster trials using stepped wedge design (SWD). An SWD is a type of cluster-crossover design in which clusters of individuals are randomized unidirectional from a control to an intervention at certain time points. The intraclass correlation coefficient (ICC) that measures the dependency of subject within a cluster plays an important role in design and analysis of stepped wedge trials. In this paper, we discuss a Bayesian approach to address the dependency of SWD on the ICC and robust Bayesian SWDs are proposed. Bayesian design is shown to be more robust against the misspecification of the parameter values compared to the locally optimal design. Designs are obtained for the various choices of priors assigned to the ICC. A detailed sensitivity analysis is performed to assess the robustness of proposed optimal designs. The power superiority of Bayesian design against the commonly used balanced design is demonstrated numerically using hypothetical as well as real scenarios.

Statistical Analysis Plan for the INTEnsive Care Bundle with Blood Pressure Reduction in Acute Cerebral Hemorrhage Trial: A Stepped-Wedge Cluster Randomized Controlled Trial

The third INTEnsive care bundle with blood pressure Reduction in Acute Cerebral hemorrhage Trial (INTERACT3) is an international, multicenter, stepped-wedge (4 phases/3 steps) cluster randomized trial involving 110 hospitals in mainly low- and middle-income countries during 2017-2022. The aim is to determine the effectiveness of a goal-directed care bundle of intensive blood pressure (BP) lowering, glycemic control, antipyrexia, and anticoagulation reversal treatment versus usual standard of care, in patients with acute intracerebral hemorrhage (ICH). After a "usual care" period, hospitals were randomly allocated to implementing care-bundle protocols for control targets (systolic BP <140 mm Hg; glucose 6.1-7.8/7.8-10.0 mmol/L according to diabetes mellitus status; temperature ≤37.5°C; normalization of anticoagulation). A sample size of 8,360 patients (mean 19 per phase per site) provides 90% power (α = 0.05) for a 5.6% absolute improvement in the primary outcome of scores on the modified Rankin scale at 6 months, analyzed by ordinal logistic regression. A detailed statistical analysis plan (SAP) was developed to prespecify the method of analysis for all outcomes and key variables collected in the trial. The primary analysis will use ordinal logistic regression adjusted for the stepped-wedge design. The SAP also includes planned sensitivity analyses, including covariate adjustments, missing data imputations, and subgroup analysis. This SAP allows transparent, verifiable, and prespecified analyses in consideration of the challenges in conducting the study during the COVID pandemic. It also avoids analysis bias arising from prior knowledge of the findings in determining the benefits and harms of a care bundle in acute ICH.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

A comparative study of R functions for clustered data analysis

Background

Clustered or correlated outcome data is common in medical research studies, such as the analysis of national or international disease registries, or cluster-randomized trials, where groups of trial participants, instead of each trial participant, are randomized to interventions. Within-group correlation in studies with clustered data requires the use of specific statistical methods, such as generalized estimating equations and mixed-effects models, to account for this correlation and support unbiased statistical inference.

Methods

We compare different approaches to estimating generalized estimating equations and mixed effects models for a continuous outcome in R through a simulation study and a data example. The methods are implemented through four popular functions of the statistical software R, “geese”, “gls”, “lme”, and “lmer”. In the simulation study, we compare the mean squared error of estimating all the model parameters and compare the coverage proportion of the 95% confidence intervals. In the data analysis, we compare estimation of the intervention effect and the intra-class correlation.

Results

In the simulation study, the function “lme” takes the least computation time. There is no difference in the mean squared error of the four functions. The “lmer” function provides better coverage of the fixed effects when the number of clusters is small as 10. The function “gls” produces close to nominal scale confidence intervals of the intra-class correlation. In the data analysis and the “gls” function yields a positive estimate of the intra-class correlation while the “geese” function gives a negative estimate. Neither of the confidence intervals contains the value zero.

Conclusions

The “gls” function efficiently produces an estimate of the intra-class correlation with a confidence interval. When the within-group correlation is as high as 0.5, the confidence interval is not always obtainable.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13063-021-05900-7).

Ratio estimators of intervention effects on event rates in cluster randomized trials

We consider five asymptotically unbiased estimators of intervention effects on event rates in non‐matched and matched‐pair cluster randomized trials, including ratio of mean counts r1, ratio of mean cluster‐level event rates r2, ratio of event rates r3, double ratio of counts r4, and double ratio of event rates r5. In the absence of an indirect effect, they all estimate the direct effect of the intervention. Otherwise, r1, r2, and r3 estimate the total effect, which comprises the direct and indirect effects, whereas r4 and r5 estimate the direct effect only. We derive the conditions under which each estimator is more precise or powerful than its alternatives. To control bias in studies with a small number of clusters, we propose a set of approximately unbiased estimators. We evaluate their properties by simulation and apply the methods to a trial of seasonal malaria chemoprevention. The approximately unbiased estimators are practically unbiased and their confidence intervals usually have coverage probability close to the nominal level; the asymptotically unbiased estimators perform well when the number of clusters is approximately 32 or more per trial arm. Despite its simplicity, r1 performs comparably with r2 and r3 in trials with a large but realistic number of clusters. When the variability of baseline event rate is large and there is no indirect effect, r4 and r5 tend to offer higher power than r1, r2, and r3. We discuss the implications of these findings to the planning and analysis of cluster randomized trials.

Methods for dealing with unequal cluster sizes in cluster randomized trials: A scoping review

In a cluster-randomized trial (CRT), the number of participants enrolled often varies across clusters. This variation should be considered during both trial design and data analysis to ensure statistical performance goals are achieved. Most methodological literature on the CRT design has assumed equal cluster sizes. This scoping review focuses on methodology for unequal cluster size CRTs. EMBASE, Medline, Google Scholar, MathSciNet and Web of Science databases were searched to identify English-language articles reporting on methodology for unequal cluster size CRTs published until March 2021. We extracted data on the focus of the paper (power calculation, Type I error etc.), the type of CRT, the type and the range of parameter values investigated (number of clusters, mean cluster size, cluster size coefficient of variation, intra-cluster correlation coefficient, etc.), and the main conclusions. Seventy-nine of 5032 identified papers met the inclusion criteria. Papers primarily focused on the parallel-arm CRT (p-CRT, n = 60, 76%) and the stepped-wedge CRT (n = 14, 18%). Roughly 75% of the papers addressed trial design issues (sample size/power calculation) while 25% focused on analysis considerations (Type I error, bias, etc.). The ranges of parameter values explored varied substantially across different studies. Methods for accounting for unequal cluster sizes in the p-CRT have been investigated extensively for Gaussian and binary outcomes. Synthesizing the findings of these works is difficult as the magnitude of impact of the unequal cluster sizes varies substantially across the combinations and ranges of input parameters. Limited investigations have been done for other combinations of a CRT design by outcome type, particularly methodology involving binary outcomes-the most commonly used type of primary outcome in trials. The paucity of methodological papers outside of the p-CRT with Gaussian or binary outcomes highlights the need for further methodological development to fill the gaps.

Comparison of small-sample standard-error corrections for generalised estimating equations in stepped wedge cluster randomised trials with a binary outcome: A simulation study

Generalised estimating equations with the sandwich standard-error estimator provide a promising method of analysis for stepped wedge cluster randomised trials. However, they have inflated type-one error when used with a small number of clusters, which is common for stepped wedge cluster randomised trials. We present a large simulation study of binary outcomes comparing bias-corrected standard errors from Fay and Graubard; Mancl and DeRouen; Kauermann and Carroll; Morel, Bokossa, and Neerchal; and Mackinnon and White with an independent and exchangeable working correlation matrix. We constructed 95% confidence intervals using a t-distribution with degrees of freedom including clusters minus parameters (DFC-P), cluster periods minus parameters, and estimators from Fay and Graubard (DFFG), and Pan and Wall. Fay and Graubard and an approximation to Kauermann and Carroll (with simpler matrix inversion) were unbiased in a wide range of scenarios with an independent working correlation matrix and more than 12 clusters. They gave confidence intervals with close to 95% coverage with DFFG with 12 or more clusters, and DFC-P with 18 or more clusters. Both standard errors were conservative with fewer clusters. With an exchangeable working correlation matrix, approximated Kauermann and Carroll and Fay and Graubard had a small degree of under-coverage.

Information content of stepped wedge designs with unequal cluster-period sizes in linear mixed models: Informing incomplete designs

In practice, stepped wedge trials frequently include clusters of differing sizes. However, investigations into the theoretical aspects of stepped wedge designs have, until recently, typically assumed equal numbers of subjects in each cluster and in each period. The information content of the cluster-period cells, clusters, and periods of stepped wedge designs has previously been investigated assuming equal cluster-period sizes, and has shown that incomplete stepped wedge designs may be efficient alternatives to the full stepped wedge. How this changes when cluster-period sizes are not equal is unknown, and we investigate this here. Working within the linear mixed model framework, we show that the information contributed by design components (clusters, sequences, and periods) does depend on the sizes of each cluster-period. Using a particular trial that assessed the impact of an individual education intervention on log-length of stay in rehabilitation units, we demonstrate how strongly the efficiency of incomplete designs depends on which cells are excluded: smaller incomplete designs may be more powerful than alternative incomplete designs that include a greater total number of participants. This also serves to demonstrate how the pattern of information content can be used to inform a set of incomplete designs to be considered as alternatives to the complete stepped wedge design. Our theoretical results for the information content can be extended to a broad class of longitudinal (ie, multiple period) cluster randomized trial designs.

Variance formulae for multiphase stepped wedge cluster randomized trial

In a multiphase stepped wedge cluster randomized trial (MSW-CRT), more than one intervention will be initiated on each sequence in a fixed order. Hence, with the MSW-CRT design, the effect of the first intervention can be evaluated when compared to control, as well as the added-on effects of the subsequent interventions. Studies that use MSW-CRT have been proposed, but properties of this design have not been explicitly studied. We derive closed-form variance formulae to test the interventions' effects, which can be readily used for sample size and power calculation. Additionally, we provide relationships between variances to test the interventions' effects and design parameters. Under special conditions, some important properties include: (i) the variances to test different interventions' effects (ie, the first intervention effect and the second intervention effect) may be same; (ii) as the cluster-period mean autocorrelation increases, the variance to test an intervention effect may first increase and then decrease; (iii) as the amount of periods between the initiations of two interventions (ie, lag) increases, the variance to test an intervention effect may remain unchanged. We illustrate the relationships between power and design parameters using the variance formulae. From a few illustrative examples, we observe that the statistical test that uses data only relevant to a specific intervention has inferior power (relative power loss <15%) compared to the test when using all the study data. Also, power is reduced when both the total number of periods and lag are decreased simultaneously (relative power loss <20%).

Variations in stepped-wedge cluster randomized trial design: Insights from the Patient-Centered Care Transitions in Heart Failure trial

The stepped-wedge (SW) cluster randomized controlled trial, in which clusters cross over in a randomized sequence from control to intervention, is ideal for the implementation and testing of complex health service interventions. In certain cases however, implementation of the intervention may pose logistical challenges, and variations in SW design may be required. We examine the logistical and statistical implications of variations in SW design using the optimization of the Patient-Centered Care Transitions in Heart Failure trial for illustration. We review the following complete SW design variations: a typical SW design; an SW design with multiple clusters crossing over per period to achieve balanced cluster sizes at each step; hierarchical randomization to account for higher-level clustering effects; nested substudies to measure outcomes requiring a smaller sample size than the primary outcomes; and hybrid SW design, which combines parallel cluster with SW design to improve efficiency. We also reviewed 3 incomplete SW design variations in which data are collected in some but not all steps to ease measurement burden. These include designs with a learning period that improve fidelity to the intervention, designs with reduced measurements to minimize collection burden, and designs with early and late blocks to accommodate cluster readiness. Variations in SW design offer pragmatic solutions to logistical challenges but have implications to statistical power. Advantages and disadvantages of each variation should be considered before finalizing the design of an SW randomized controlled trial.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Power calculation for cross-sectional stepped wedge cluster randomized trials with variable cluster sizes

Standard sample size calculation formulas for stepped wedge cluster randomized trials (SW-CRTs) assume that cluster sizes are equal. When cluster sizes vary substantially, ignoring this variation may lead to an under-powered study. We investigate the relative efficiency of a SW-CRT with varying cluster sizes to equal cluster sizes, and derive variance estimators for the intervention effect that account for this variation under a mixed effects model-a commonly used approach for analyzing data from cluster randomized trials. When cluster sizes vary, the power of a SW-CRT depends on the order in which clusters receive the intervention, which is determined through randomization. We first derive a variance formula that corresponds to any particular realization of the randomized sequence and propose efficient algorithms to identify upper and lower bounds of the power. We then obtain an "expected" power based on a first-order approximation to the variance formula, where the expectation is taken with respect to all possible randomization sequences. Finally, we provide a variance formula for more general settings where only the cluster size arithmetic mean and coefficient of variation, instead of exact cluster sizes, are known in the design stage. We evaluate our methods through simulations and illustrate that the average power of a SW-CRT decreases as the variation in cluster sizes increases, and the impact is largest when the number of clusters is small.

Effect of a novel vital sign device on maternal mortality and morbidity in low-resource settings: a pragmatic, stepped-wedge, cluster-randomised controlled trial

BACKGROUND

In 2015, an estimated 303 000 women died in pregnancy and childbirth. Obstetric haemorrhage, sepsis, and hypertensive disorders of pregnancy account for more than 50% of maternal deaths worldwide. There are effective treatments for these pregnancy complications, but they require early detection by measurement of vital signs and timely administration to save lives. The primary aim of this trial was to determine whether implementation of the CRADLE Vital Sign Alert and an education package into community and facility maternity care in low-resource settings could reduce a composite of all-cause maternal mortality or major morbidity (eclampsia and hysterectomy).

METHODS

We did a pragmatic, stepped-wedge, cluster-randomised controlled trial in ten clusters across Africa, India, and Haiti, introducing the device into routine maternity care. Each cluster contained at least one secondary or tertiary hospital and their main referral facilities. Clusters crossed over from existing routine care to the CRADLE intervention in one of nine steps at 2-monthly intervals, with CRADLE devices replacing existing equipment at the randomly allocated timepoint. A computer-generated randomly allocated sequence determined the order in which the clusters received the intervention. Because of the nature of the intervention, this trial was not masked. Data were gathered monthly, with 20 time periods of 1 month. The primary composite outcome was at least one of eclampsia, emergency hysterectomy, and maternal death. This study is registered with the ISRCTN registry, number ISRCTN41244132.

FINDINGS

Between April 1, 2016, and Nov 30, 2017, among 536 223 deliveries, the primary outcome occurred in 4067 women, with 998 maternal deaths, 2692 eclampsia cases, and 681 hysterectomies. There was an 8% decrease in the primary outcome from 79·4 per 10 000 deliveries pre-intervention to 72·8 per 10 000 deliveries post-intervention (odds ratio [OR] 0·92, 95% CI 0·86-0·97; p=0·0056). After planned adjustments for variation in event rates between and within clusters over time, the unexpected degree of variability meant we were unable to judge the benefit or harms of the intervention (OR 1·22, 95% CI 0·73-2·06; p=0·45).

INTERPRETATION

There was an absolute 8% reduction in primary outcome during the trial, with no change in resources or staffing, but this reduction could not be directly attributed to the intervention due to variability. We encountered unanticipated methodological challenges with this trial design, which can provide valuable learning for future research and inform the trial design of future international stepped-wedge trials.

FUNDING

Newton Fund Global Research Programme: UK Medical Research Council; Department of Biotechnology, Ministry of Science & Technology, Government of India; and UK Department of International Development.

The current use of feasibility studies in the assessment of feasibility for stepped-wedge cluster randomised trials: a systematic review

Background

Stepped-wedge cluster randomised trials (SW-CRTs) are a pragmatic trial design, providing an unprecedented opportunity to increase the robustness of evidence underpinning implementation and quality improvement interventions. Given the complexity of the SW-CRT, the likelihood of trials not delivering on their objectives will be mitigated if a feasibility study precedes the definitive trial. It is not currently known if feasibility studies are being conducted for SW-CRTs nor what the objectives of these studies are.

Methods

Searches were conducted of several databases to identify published feasibility studies which were designed to inform a future SW-CRT. For each eligible study, data were extracted on the characteristics of and rationale for the feasibility study; the process for determining progression to the main trial; how the feasibility study informed the main trial; and whether the main trial went ahead. A narrative synthesis and descriptive analysis are presented.

Results

Eleven feasibility studies were identified, which included eight completed study reports and three protocols. Three studies used a stepped-wedge design and these were the only studies to be randomised. Studies were predominantly of a mixed-methods design. Only one study assessed specific features related to the feasibility of using a SW-CRT and one investigated the time taken to complete the study procedures. The other studies were mostly assessing the feasibility and acceptability of the intervention.

Conclusion

Published feasibility studies for SW-CRTs are scarce and those that are being reported do not investigate issues specific to the complexities of the trial design. When conducting feasibility studies in advance of a definitive SW-CRT, researchers should consider assessing the feasibility of study procedures, particularly those specific to the SW-CRT design, and ensure that the findings are published for the benefit of other researchers.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0658-3) contains supplementary material, which is available to authorized users.