Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters

A review of current practice in the design and analysis of extremely small stepped-wedge cluster randomized trials

BACKGROUND/AIMS

Stepped-wedge cluster randomized trials tend to require fewer clusters than standard parallel-arm designs due to the switches between control and intervention conditions, but there are no recommendations for the minimum number of clusters. Trials randomizing an extremely small number of clusters are not uncommon, but the justification for small numbers of clusters is often unclear and appropriate analysis is often lacking. In addition, stepped-wedge cluster randomized trials are methodologically more complex due to their longitudinal correlation structure, and ignoring the distinct within- and between-period intracluster correlations can underestimate the sample size in small stepped-wedge cluster randomized trials. We conducted a review of published small stepped-wedge cluster randomized trials to understand how and why they are used, and to characterize approaches used in their design and analysis.

METHODS

Electronic searches were used to identify primary reports of full-scale stepped-wedge cluster randomized trials published during the period 2016-2022; the subset that randomized two to six clusters was identified. Two reviewers independently extracted information from each report and any available protocol. Disagreements were resolved through discussion.

RESULTS

We identified 61 stepped-wedge cluster randomized trials that randomized two to six clusters: median sample size (Q1-Q3) 1426 (420-7553) participants. Twelve (19.7%) gave some indication that the evaluation was considered a "preliminary" evaluation and 16 (26.2%) recognized the small number of clusters as a limitation. Sixteen (26.2%) provided an explanation for the limited number of clusters: the need to minimize contamination (e.g. by merging adjacent units), limited availability of clusters, and logistical considerations were common explanations. Majority (51, 83.6%) presented sample size or power calculations, but only one assumed distinct within- and between-period intracluster correlations. Few (10, 16.4%) utilized restricted randomization methods; more than half (34, 55.7%) identified baseline imbalances. The most common statistical method for analysis was the generalized linear mixed model (44, 72.1%). Only four trials (6.6%) reported statistical analyses considering small numbers of clusters: one used generalized estimating equations with small-sample correction, two used generalized linear mixed model with small-sample correction, and one used Bayesian analysis. Another eight (13.1%) used fixed-effects regression, the performance of which requires further evaluation under stepped-wedge cluster randomized trials with small numbers of clusters. None used permutation tests or cluster-period level analysis.

CONCLUSION

Methods appropriate for the design and analysis of small stepped-wedge cluster randomized trials have not been widely adopted in practice. Greater awareness is required that the use of standard sample size calculation methods can provide spuriously low numbers of required clusters. Methods such as generalized estimating equations or generalized linear mixed models with small-sample corrections, Bayesian approaches, and permutation tests may be more appropriate for the analysis of small stepped-wedge cluster randomized trials. Future research is needed to establish best practices for stepped-wedge cluster randomized trials with a small number of clusters.

Statistical analysis plan for the NU IMPACT stepped-wedge cluster randomized trial

BACKGROUND

As part of the IMPACT Consortium of three effectiveness-implementation trials, the NU IMPACT trial was designed to evaluate implementation and effectiveness outcomes for an electronic health record (EHR)-embedded symptom monitoring and management program for outpatient cancer care. NU IMPACT uses a unique stepped-wedge cluster randomized design, involving six clusters of 26 clinics, for evaluation of implementation outcomes with an embedded patient-level randomized trial to evaluate effectiveness outcomes. Collaborative, consortium-wide efforts to ensure use of the most robust and recent analytic methodologies for stepped-wedge trials motivated updates to the statistical analysis plan for implementation outcomes in the NU IMPACT trial.

METHODS

In the updated statistical analysis plan for NU IMPACT, the primary implementation outcome patient adoption, as measured by clinic-level monthly proportions of patient engagement with the EHR-based cancer symptom monitoring system, will be analyzed using generalized least squares linear regression with auto-regressive errors and adjustment for cluster and time effects (underlying secular trends). A similar strategy will be used for secondary patient and provider implementation outcomes.

DISCUSSION

The analytic updates described here resulted from highly iterative, collaborative efforts among statisticians, implementation scientists, and trial leads in the IMPACT Consortium. This updated statistical analysis plan will serve as the a priori specified approach for analyzing implementation outcomes for the NU IMPACT trial.

A Bayesian adaptive design approach for stepped-wedge cluster randomized trials

BACKGROUND

The Bayesian group sequential design has been applied widely in clinical studies, especially in Phase II and III studies. It allows early termination based on accumulating interim data. However, to date, there lacks development in its application to stepped-wedge cluster randomized trials, which are gaining popularity in pragmatic trials conducted by clinical and health care delivery researchers.

METHODS

We propose a Bayesian adaptive design approach for stepped-wedge cluster randomized trials, which makes adaptive decisions based on the predictive probability of declaring the intervention effective at the end of study given interim data. The Bayesian models and the algorithms for posterior inference and trial conduct are presented.

RESULTS

We present how to determine design parameters through extensive simulations to achieve desired operational characteristics. We further evaluate how various design factors, such as the number of steps, cluster size, random variability in cluster size, and correlation structures, impact trial properties, including power, type I error, and the probability of early stopping. An application example is presented.

CONCLUSION

This study presents the incorporation of Bayesian adaptive strategies into stepped-wedge cluster randomized trials design. The proposed approach provides the flexibility to stop the trial early if substantial evidence of efficacy or futility is observed, improving the flexibility and efficiency of stepped-wedge cluster randomized trials.

Delineating maternal influence in regulation of variance in major economic traits of White Leghorns: Bayesian insights

Proper variance partitioning and estimation of genetic parameters at appropriate time interval is crucial for understanding the dynamics of trait variance and genetic correlations and for deciding the future breeding strategy of the population. This study was conducted on the same premise to estimate genetic parameters of major economic traits in a White Leghorn strain IWH using Bayesian approach and to identify the role of maternal effects in the regulation of trait variance. Three different models incorporating the direct additive effect (Model 1), direct additive and maternal genetic effect (Model 2) and direct additive, maternal genetic and maternal permanent environmental effects (Model 3) were tried to estimate the genetic parameters for body weight traits (birth weight, body weight at 16, 20, 40 and 52 weeks), Age at sexual maturity (ASM), egg production traits (egg production up to 24, 28, 40, 52, 64 and 72 weeks) and egg weight traits (egg weight at 28, 40 and 52 weeks). Model 2 and Model 3 with maternal effects were found to be the best having the highest accuracy for almost all the traits. The direct additive genetic heritability was moderate for ASM, moderate to high for body weight traits and egg weight traits and low to moderate for egg production traits. Though the maternal heritability (h2mat) and permanent environmental effect (c2mpe) was low (<0.1) for most of the traits, they formed an important component of trait variance. Traits like egg weight at 28 weeks (0.14±0.06) and egg production at 72 weeks (0.13±0.07) reported comparatively higher values for c2mpe and h2mat respectively. Additive genetic correlation was high and positive between body weight traits, between egg weight traits, between consecutive egg production traits and between body weight and egg weight traits. However, a negative genetic correlation existed between egg production and egg weight traits, egg production and body weight traits, ASM and early egg production traits. Overall, a moderate positive genetic correlation was estimated between ASM and body weight traits and ASM and egg weight traits. Based on our findings, we can deduce that maternal effects constitute an important source of variation for all the major economic traits in White Leghorn and should be necessarily considered in genetic evaluation programs.

Increased risk of hearing loss associated with macrolide use: a systematic review and meta-analysis

The increased risk of hearing loss with macrolides remains controversial. We aimed to systematically review and meta-analyze data on the clinical risk of hearing loss, tinnitus, and ototoxicity following macrolide use. A systematic search was conducted across PubMed, MEDLINE, Cochrane, and Embase databases from database inception to May 2023. Medical Subject Heading (MeSH) terms and text keywords were utilized, without any language restrictions. In addition to the electronic databases, two authors manually and independently searched for relevant studies in the US and European clinical trial registries and Google Scholar. Studies that involved (1) patients who had hearing loss, tinnitus, or ototoxicity after macrolide use, (2) intervention of use of macrolides such as azithromycin, clarithromycin, erythromycin, fidaxomicin, roxithromycin, spiramycin, and/or telithromycin, (3) comparisons with specified placebos or other antibiotics, (4) outcomes measured as odds ratio (OR), relative risk (RR), hazard ratio (HR), and mean difference for ototoxicity symptoms using randomized control trial (RCT)s and observational studies (case-control, cross-section, and cohort studies) were included. Data extraction was performed independently by two extractors, and a crosscheck was performed to identify any errors. ORs along with their corresponding 95% confidence intervals (CIs) were estimated using random-effects models. The Preferred Reporting Items for Systematic Reviews and Meta-analyses reporting guidelines for RCTs and Meta-Analysis of Observational Studies in Epidemiology guidelines for observational studies were followed. We assessed the hearing loss risk after macrolide use versus controls (placebos and other antibiotics). Based on data from 13 studies including 1,142,021 patients (n = 267,546 for macrolide and n = 875,089 for controls), the overall pooled OR was 1.25 (95% CI 1.07-1.47). In subgroup analysis by study design, the ORs were 1.37 (95% CI 1.08-1.73) for RCTs and 1.33 (95% CI 1.24-1.43) for case-control studies, indicating that RCT and case-control study designs showed a statistically significant higher risk of hearing loss. The group with underlying diseases such as multiple infectious etiologies (OR, 1.16 [95% CI 0.96-1.41]) had a statistically significant lower risk than the group without (OR, 1.53 [95% CI 1.38-1.70] P = .013). The findings from this systematic review and meta-analysis suggest that macrolide antibiotics increase the risk of hearing loss and that healthcare professionals should carefully consider this factor while prescribing macrolides.