Review of Recent Methodological Developments in Group-Randomized Trials: Part 2-Analysis

A Hybrid Pragmatic and Factorial Cluster Randomized Controlled Trial for an Anti-racist, Multilevel Intervention to Improve Mental Health Equity in High Schools

Systemic racism is pervasive in US society and disproportionately limits opportunities for education, work, and health for historically marginalized and minoritized racial and ethnic groups, making it an urgent issue of social justice. Because systemic racism is a social determinant of health prevalent across multiple social and institutional structures, it requires multilevel intervention approaches using effective designs and analytic methods to measure and evaluate outcomes. Racism is a fundamental cause of poor health outcomes, including mental health outcomes; thus, mental health services and programs that address racism and discrimination are key to promoting positive mental health of racial and ethnic minority youth. While multilevel interventions are well-suited for improving outcomes like youth mental health disparities, their evaluation poses unique methodological challenges, requiring specialized design and analytic approaches. There has been limited methodological guidance provided to researchers on how to test multilevel interventions using approaches that balance methodological rigor, practicality, and acceptability across stakeholder groups, especially within communities most affected by systemic racism. This paper addresses this gap by providing an example of how to rigorously evaluate a hypothetical, theoretically based, multilevel intervention promoting mental health equity in three US school systems using an anti-racist approach intervening at the macro- (i.e., school system), meso- (i.e., school), and micro- (i.e., family and student) levels to improve mental health in adolescents. We describe the design, sample size considerations, and analytic methods to comprehensively evaluate its effectiveness while exploring the extent to which the components interact synergistically to improve outcomes. The methodological approach proposed can be adapted to other multilevel interventions that include strategies addressing macro-, meso-, and micro-levels of influence.

Sample Size Requirements to Test Subgroup-Specific Treatment Effects in Cluster-Randomized Trials

Cluster-randomized trials (CRTs) often allocate intact clusters of participants to treatment or control conditions and are increasingly used to evaluate healthcare delivery interventions. While previous studies have developed sample size methods for testing confirmatory hypotheses of treatment effect heterogeneity in CRTs (i.e., targeting the difference between subgroup-specific treatment effects), sample size methods for testing the subgroup-specific treatment effects themselves have not received adequate attention-despite a rising interest in health equity considerations in CRTs. In this article, we develop formal methods for sample size and power analyses for testing subgroup-specific treatment effects in parallel-arm CRTs with a continuous outcome and a binary subgroup variable. We point out that the variances of the subgroup-specific treatment effect estimators and their covariance are given by weighted averages of the variance of the overall average treatment effect estimator and the variance of the heterogeneous treatment effect estimator. This analytical insight facilitates an explicit characterization of the requirements for both the omnibus test and the intersection-union test to achieve the desired level of power. Generalizations to allow for subgroup-specific variance structures are also discussed. We report on a simulation study to validate the proposed sample size methods and demonstrate that the empirical power corresponds well with the predicted power for both tests. The design and setting of the Umea Dementia and Exercise (UMDEX) CRT in older adults are used to illustrate our sample size methods.

Application of a Heuristic Framework for Multilevel Interventions to Eliminate the Impact of Unjust Social Processes and Other Harmful Social Determinants of Health

There is consensus about the importance of developing a strong cadre of effective multilevel interventions to eliminate the impacts of unjust social processes, such as structural racism and other harmful social determinants of health (SDOH), on health inequities in the USA. However, the available cadre of rigorously evaluated evidence-based interventions for SDOH mitigation remains underdeveloped relative to the magnitude of historic and current health inequities. The proposed manuscript addresses this gap in two ways: first, by introducing a heuristic framework to inform decisions in multilevel intervention development, study design, and selection of analytic methods and, second, by providing a roadmap for future applications of the framework in multilevel intervention research through an exemplar application using the ongoing NIH-funded evaluation study of the Nurse-Community-Family Partnership (NCFP) intervention. NCFP leverages individual, family, institutional, and system factors to shape COVID-19 mitigation outcomes at the individual and household levels. NCFP takes an approach informed by the heuristic framework to addressing and mitigating unjust social processes and other harmful SDOH. We discuss the application of a two-arm parallel explanatory group randomized trial to evaluate the efficacy of NCFP in improving the primary (COVID-19 testing uptake) and secondary (adoption of COVID-19 control measures, COVID-19 vaccine uptake, mutual aid capacity, etc.) outcomes at the individual and household levels. The analysis approach relies on random-intercept models, and we calculate the variance partitioning coefficient to estimate the extent to which household- and individual-level variables contribute to the outcome, allowing examination of NCFP effects at multiple levels.

Demystifying estimands in cluster-randomised trials

Estimands can help clarify the interpretation of treatment effects and ensure that estimators are aligned with the study's objectives. Cluster-randomised trials require additional attributes to be defined within the estimand compared to individually randomised trials, including whether treatment effects are marginal or cluster-specific, and whether they are participant- or cluster-average. In this paper, we provide formal definitions of estimands encompassing both these attributes using potential outcomes notation and describe differences between them. We then provide an overview of estimators for each estimand, describe their assumptions, and show consistency (i.e. asymptotically unbiased estimation) for a series of analyses based on cluster-level summaries. Then, through a re-analysis of a published cluster-randomised trial, we demonstrate that the choice of both estimand and estimator can affect interpretation. For instance, the estimated odds ratio ranged from 1.38 (p = 0.17) to 1.83 (p = 0.03) depending on the target estimand, and for some estimands, the choice of estimator affected the conclusions by leading to smaller treatment effect estimates. We conclude that careful specification of the estimand, along with an appropriate choice of estimator, is essential to ensuring that cluster-randomised trials address the right question.

Assessing treatment effect heterogeneity in the presence of missing effect modifier data in cluster-randomized trials

Understanding whether and how treatment effects vary across subgroups is crucial to inform clinical practice and recommendations. Accordingly, the assessment of heterogeneous treatment effects based on pre-specified potential effect modifiers has become a common goal in modern randomized trials. However, when one or more potential effect modifiers are missing, complete-case analysis may lead to bias and under-coverage. While statistical methods for handling missing data have been proposed and compared for individually randomized trials with missing effect modifier data, few guidelines exist for the cluster-randomized setting, where intracluster correlations in the effect modifiers, outcomes, or even missingness mechanisms may introduce further threats to accurate assessment of heterogeneous treatment effect. In this article, the performance of several missing data methods are compared through a simulation study of cluster-randomized trials with continuous outcome and missing binary effect modifier data, and further illustrated using real data from the Work, Family, and Health Study. Our results suggest that multilevel multiple imputation and Bayesian multilevel multiple imputation have better performance than other available methods, and that Bayesian multilevel multiple imputation has lower bias and closer to nominal coverage than standard multilevel multiple imputation when there are model specification or compatibility issues.

Reconsidering stepped wedge cluster randomized trial designs with implementation periods: Fewer sequences or the parallel-group design with baseline and implementation periods are potentially more efficient

BACKGROUND/AIMS

When designing a cluster randomized trial, advantages and disadvantages of tentative designs must be weighed. The stepped wedge design is popular for multiple reasons, including its potential to increase power via improved efficiency relative to a parallel-group design. In many realistic settings, it will take time for clusters to fully implement the intervention. When designing the HEALing (Helping to End Addiction Long-termSM) Communities Study, implementation time was a major consideration, and we examined the efficiency and practicality of three designs. Specifically, a three-sequence stepped wedge design with implementation periods, a corresponding two-sequence modified design that is created by removing the middle sequence, and a parallel-group design with baseline and implementation periods. In this article, we study the relative efficiencies of these specific designs. More generally, we study the relative efficiencies of modified designs when the stepped wedge design with implementation periods has three or more sequences. We also consider different correlation structures.

METHODS

We compare efficiencies of stepped wedge designs with implementation periods consisting of three to nine sequences with a variety of corresponding designs. The three-sequence design is compared to the two-sequence modified design and to the parallel-group design with baseline and implementation periods analysed via analysis of covariance. Stepped wedge designs with implementation periods consisting of four or more sequences are compared to modified designs that remove all or a subset of 'middle' sequences. Efficiencies are based on the use of linear mixed effects models.

RESULTS

In the studied settings, the modified design is more efficient than the three-sequence stepped wedge design with implementation periods. The parallel-group design with baseline and implementation periods with analysis of covariance-based analysis is often more efficient than the three-sequence design. With respect to stepped wedge designs with implementation periods that are comprised of more sequences, there are often corresponding modified designs that improve efficiency. However, use of only the first and last sequences has the potential to be either relatively efficient or inefficient. Relative efficiency is impacted by the strength of the statistical correlation among outcomes from the same cluster; for example, the relative efficiencies of modified designs tend to be greater for smaller cluster auto-correlation values.

CONCLUSION

If a three-sequence stepped wedge design with implementation periods is being considered for a future cluster randomized trial, then a corresponding modified design using only the first and last sequences should be considered if sole focus is on efficiency. However, a parallel-group design with baseline and implementation periods and analysis of covariance-based analysis can be a practical, efficient alternative. For stepped wedge designs with implementation periods and a larger number of sequences, modified versions that remove 'middle' sequences should be considered. Due to the potential sensitivity of design efficiencies, statistical correlation should be carefully considered.

A mixed model approach to estimate the survivor average causal effect in cluster-randomized trials

In many medical studies, the outcome measure (such as quality of life, QOL) for some study participants becomes informatively truncated (censored, missing, or unobserved) due to death or other forms of dropout, creating a nonignorable missing data problem. In such cases, the use of a composite outcome or imputation methods that fill in unmeasurable QOL values for those who died rely on strong and untestable assumptions and may be conceptually unappealing to certain stakeholders when estimating a treatment effect. The survivor average causal effect (SACE) is an alternative causal estimand that surmounts some of these issues. While principal stratification has been applied to estimate the SACE in individually randomized trials, methods for estimating the SACE in cluster-randomized trials are currently limited. To address this gap, we develop a mixed model approach along with an expectation-maximization algorithm to estimate the SACE in cluster-randomized trials. We model the continuous outcome measure with a random intercept to account for intracluster correlations due to cluster-level randomization, and model the principal strata membership both with and without a random intercept. In simulations, we compare the performance of our approaches with an existing fixed-effects approach to illustrate the importance of accounting for clustering in cluster-randomized trials. The methodology is then illustrated using a cluster-randomized trial of telecare and assistive technology on health-related QOL in the elderly.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Analysing cluster randomised controlled trials using GLMM, GEE1, GEE2, and QIF: results from four case studies

BACKGROUND

Using four case studies, we aim to provide practical guidance and recommendations for the analysis of cluster randomised controlled trials.

METHODS

Four modelling approaches (Generalized Linear Mixed Models with parameters estimated by maximum likelihood/restricted maximum likelihood; Generalized Linear Models with parameters estimated by Generalized Estimating Equations (1st order or second order) and Quadratic Inference Function, for analysing correlated individual participant level outcomes in cluster randomised controlled trials were identified after we reviewed the literature. We systematically searched the online bibliography databases of MEDLINE, EMBASE, PsycINFO (via OVID), CINAHL (via EBSCO), and SCOPUS. We identified the above-mentioned four statistical analytical approaches and applied them to four case studies of cluster randomised controlled trials with the number of clusters ranging from 10 to 100, and individual participants ranging from 748 to 9,207. Results were obtained for both continuous and binary outcomes using R and SAS statistical packages.

RESULTS

The intracluster correlation coefficient (ICC) estimates for the case studies were less than 0.05 and are consistent with the observed ICC values commonly reported in primary care and community-based cluster randomised controlled trials. In most cases, the four methods produced similar results. However, in a few analyses, quadratic inference function produced different results compared to the generalized linear mixed model, first-order generalized estimating equations, and second-order generalized estimating equations, especially in trials with small to moderate numbers of clusters.

CONCLUSION

This paper demonstrates the analysis of cluster randomised controlled trials with four modelling approaches. The results obtained were similar in most cases, however, for trials with few clusters we do recommend that the quadratic inference function should be used with caution, and where possible a small sample correction should be used. The generalisability of our results is limited to studies with similar features to our case studies, for example, studies with a similar-sized ICC. It is important to conduct simulation studies to comprehensively evaluate the performance of the four modelling approaches.

Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial

BACKGROUND

Recent work has shown that cluster-randomised trials can estimate two distinct estimands: the participant-average and cluster-average treatment effects. These can differ when participant outcomes or the treatment effect depends on the cluster size (termed informative cluster size). In this case, estimators that target one estimand (such as the analysis of unweighted cluster-level summaries, which targets the cluster-average effect) may be biased for the other. Furthermore, commonly used estimators such as mixed-effects models or generalised estimating equations with an exchangeable correlation structure can be biased for both estimands. However, there has been little empirical research into whether informative cluster size is likely to occur in practice.

METHOD

We re-analysed a cluster-randomised trial comparing two different thresholds for red blood cell transfusion in patients with acute upper gastrointestinal bleeding to explore whether estimates for the participant- and cluster-average effects differed, to provide empirical evidence for whether informative cluster size may be present. For each outcome, we first estimated a participant-average effect using independence estimating equations, which are unbiased under informative cluster size. We then compared this to two further methods: (1) a cluster-average effect estimated using either weighted independence estimating equations or unweighted cluster-level summaries, and (2) estimates from a mixed-effects model or generalised estimating equations with an exchangeable correlation structure. We then performed a small simulation study to evaluate whether observed differences between cluster- and participant-average estimates were likely to occur even if no informative cluster size was present.

RESULTS

For most outcomes, treatment effect estimates from different methods were similar. However, differences of >10% occurred between participant- and cluster-average estimates for 5 of 17 outcomes (29%). We also observed several notable differences between estimates from mixed-effects models or generalised estimating equations with an exchangeable correlation structure and those based on independence estimating equations. For example, for the EQ-5D VAS score, the independence estimating equation estimate of the participant-average difference was 4.15 (95% confidence interval: -3.37 to 11.66), compared with 2.84 (95% confidence interval: -7.37 to 13.04) for the cluster-average independence estimating equation estimate, and 3.23 (95% confidence interval: -6.70 to 13.16) from a mixed-effects model. Similarly, for thromboembolic/ischaemic events, the independence estimating equation estimate for the participant-average odds ratio was 0.43 (95% confidence interval: 0.07 to 2.48), compared with 0.33 (95% confidence interval: 0.06 to 1.77) from the cluster-average estimator.

CONCLUSION

In this re-analysis, we found that estimates from the various approaches could differ, which may be due to the presence of informative cluster size. Careful consideration of the estimand and the plausibility of assumptions underpinning each estimator can help ensure an appropriate analysis methods are used. Independence estimating equations and the analysis of cluster-level summaries (with appropriate weighting for each to correspond to either the participant-average or cluster-average treatment effect) are a desirable choice when informative cluster size is deemed possible, due to their unbiasedness in this setting.

i-MoMCARE: Innovative Mobile Technology for Maternal and Child Health Care in Cambodia-study protocol of a cluster randomized controlled trial

BACKGROUND

The Government of Cambodia established the village health support groups (VHSGs) in 2003 to facilitate primary healthcare activities, including maternal and child health (MCH) services. However, VHSGs face several challenges that hinder them from performing optimally, including a lack of regular structured training and remuneration and limited and inconsistent support and supervision from the health centers (HCs). This implementation research aims to develop, implement, and evaluate a digital health intervention to improve the performance of VHSGs through better support and supervision and increase the MCH service coverage in rural Cambodia.

METHODS

i-MoMCARE, a two-arm cluster randomized controlled trial, will be conducted between 2022 and 2025. Five operational districts (ODs) have been randomized to an intervention arm and the other five ODs to the control arm. The intervention will last for 24 months. Around 200 VHSGs in the intervention arm will be equipped with a mobile application as a job aid and 20 HC staff with a web interface to improve support and supervision of VHSGs. The potential beneficiaries will include pregnant women, mothers, and children under 2 years old. We will measure the outcomes at baseline and endline. The primary outcomes will consist of a composite MCH index constructed from maternal and newborn care indicators, child immunization, and treatment of under-two children. Secondary outcomes will include coverage of selected MCH services. We will conduct the intention-to-treat and per-protocol analyses. We will conduct qualitative interviews with selected beneficiaries and stakeholders to evaluate the intervention's acceptability, feasibility, and scalability. We will also conduct a cost-effective analysis using decision-analytic modeling incorporating a societal perspective that explores different time horizons, intervention effects, and when scaled up to the national level.

DISCUSSION

i-MoMCARE is expected to increase MCH service access and coverage in rural Cambodia. It will contribute to advancing digital health use in primary healthcare interventions, which remains in its infancy in the country. Furthermore, the study findings will be a valuable addition to a growing body of literature on the effectiveness and feasibility of mobile health to improve coverage of MCH services in rural low- and middle-income country settings.

TRIAL REGISTRATION

ClinicalTrial.gov NCT05639595. Registered on 06 December 2022.

Evaluation of the Strengthening a Palliative Approach in Long Term Care (SPA-LTC) programme: a protocol of a cluster randomised control trial

INTRODUCTION

Despite the high mortality rates in long-term care (LTC) homes, most do not have a formalised palliative programme. Hence, our research team has developed the Strengthening a Palliative Approach in Long Term Care (SPA-LTC) programme. The goal of the proposed study is to examine the implementation and effectiveness of the SPA-LTC programme.

METHODS AND ANALYSIS

A cross-jurisdictional, effectiveness-implementation type II hybrid cluster randomised control trial design will be used to assess the SPA-LTC programme for 18 LTC homes (six homes within each of three provinces). Randomisation will occur at the level of the LTC home within each province, using a 1:1 ratio (three homes in the intervention and control groups). Baseline staff surveys will take place over a 3-month period at the beginning for both the intervention and control groups. The intervention group will then receive facilitated training and education for staff, and residents and their family members will participate in the SPA-LTC programme. Postintervention data collection will be conducted in a similar manner as in the baseline period for both groups. The overall target sample size will be 594 (297 per arm, 33 resident/family member participants per home, 18 homes). Data collection and analysis will involve organisational, staff, resident and family measures. The primary outcome will be a binary measure capturing any emergency department use in the last 6 months of life (resident); with secondary outcomes including location of death (resident), satisfaction and decisional conflict (family), knowledge and confidence implementing a palliative approach (staff), along with implementation outcomes (ie, feasibility, reach, fidelity and perceived sustainability of the SPA-LTC programme). The primary outcome will be analysed via multivariable logistic regression using generalised estimating equations. Intention-to-treat principles will be used in the analysis.

ETHICS AND DISSEMINATION

The study has received ethical approval. Results will be disseminated at various presentations and feedback sessions; at provincial, national and international conferences, and in a series of manuscripts that will be submitted to peer-reviewed, open access journals.

TRIAL REGISTRATION NUMBER

NCT039359.

Key considerations for designing, conducting and analysing a cluster randomized trial

Not only do cluster randomized trials require a larger sample size than individually randomized trials, they also face many additional complexities. The potential for contamination is the most commonly used justification for using cluster randomization, but the risk of contamination should be carefully weighed against the more serious problem of questionable scientific validity in settings with post-randomization identification or recruitment of participants unblinded to the treatment allocation. In this paper we provide some simple guidelines to help researchers conduct cluster trials in a way that minimizes potential biases and maximizes statistical efficiency. The overarching theme of this guidance is that methods that apply to individually randomized trials rarely apply to cluster randomized trials. We recommend that cluster randomization be only used when necessary-balancing the benefits of cluster randomization with its increased risks of bias and increased sample size. Researchers should also randomize at the lowest possible level-balancing the risks of contamination with ensuring an adequate number of randomization units-as well as exploring other options for statistically efficient designs. Clustering should always be allowed for in the sample size calculation; and the use of restricted randomization (and adjustment in the analysis for covariates used in the randomization) should be considered. Where possible, participants should be recruited before randomizing clusters and, when recruiting (or identifying) participants post-randomization, recruiters should be masked to the allocation. In the analysis, the target of inference should align with the research question, and adjustment for clustering and small sample corrections should be used when the trial includes less than about 40 clusters.

Maximin optimal cluster randomized designs for assessing treatment effect heterogeneity

Cluster randomized trials (CRTs) are studies where treatment is randomized at the cluster level but outcomes are typically collected at the individual level. When CRTs are employed in pragmatic settings, baseline population characteristics may moderate treatment effects, leading to what is known as heterogeneous treatment effects (HTEs). Pre-specified, hypothesis-driven HTE analyses in CRTs can enable an understanding of how interventions may impact subpopulation outcomes. While closed-form sample size formulas have recently been proposed, assuming known intracluster correlation coefficients (ICCs) for both the covariate and outcome, guidance on optimal cluster randomized designs to ensure maximum power with pre-specified HTE analyses has not yet been developed. We derive new design formulas to determine the cluster size and number of clusters to achieve the locally optimal design (LOD) that minimizes variance for estimating the HTE parameter given a budget constraint. Given the LODs are based on covariate and outcome-ICC values that are usually unknown, we further develop the maximin design for assessing HTE, identifying the combination of design resources that maximize the relative efficiency of the HTE analysis in the worst case scenario. In addition, given the analysis of the average treatment effect is often of primary interest, we also establish optimal designs to accommodate multiple objectives by combining considerations for studying both the average and heterogeneous treatment effects. We illustrate our methods using the context of the Kerala Diabetes Prevention Program CRT, and provide an R Shiny app to facilitate calculation of optimal designs under a wide range of design parameters.

ORTH.Ord: An R package for analyzing correlated ordinal outcomes using alternating logistic regressions with orthogonalized residuals

BACKGROUND AND OBJECTIVES

Marginal models with generalized estimating equations (GEE) are usually recommended for analyzing correlated ordinal outcomes which are commonly seen in a longitudinal study or clustered randomized trial (CRT). Within-cluster association is often of interest in longitudinal studies or CRTs, and can be estimated with paired estimating equations. However, the estimators for within-cluster association parameters and variances may be subject to finite-sample biases when the number of clusters is small. The objective of this article is to introduce a newly developed R package ORTH.Ord for analyzing correlated ordinal outcomes using GEE models with finite-sample bias corrections.

METHODS

The R package ORTH.Ord implements a modified version of alternating logistic regressions with estimation based on orthogonalized residuals (ORTH), which use paired estimating equations to jointly estimate parameters in marginal mean and association models. The within-cluster association between ordinal responses is modeled by global pairwise odds ratios (POR). The R package also provides a finite-sample bias correction to POR parameter estimates based on matrix multiplicative adjusted orthogonalized residuals (MMORTH) for correcting estimating equations, and bias-corrected sandwich estimators with different options for covariance estimation.

RESULTS

A simulation study shows that MMORTH provides less biased global POR estimates and coverage of their 95% confidence intervals closer to the nominal level than uncorrected ORTH. An analysis of patient-reported outcomes from an orthognathic surgery clinical trial illustrates features of ORTH.Ord.

CONCLUSIONS

This article provides an overview of the ORTH method with bias-correction on both estimating equations and sandwich estimators for analyzing correlated ordinal data, describes the features of the ORTH.Ord R package, evaluates the performance of the package using a simulation study, and finally illustrates its application in an analysis of a clinical trial.

Models for cluster randomized designs using ranked set sampling

Cluster randomized designs (CRD) provide a rigorous development for randomization principles for studies where treatments are allocated to cluster units rather than the individual subjects within clusters. It is known that CRDs are less efficient than completely randomized designs since the randomization of treatment allocation is applied to the cluster units. To mitigate this problem, we embed a ranked set sampling design from survey sampling studies into CRD for the selection of both cluster and subsampling units. We show that ranking groups in ranked set sampling act like a covariate, reduce the expected mean squared cluster error, and increase the precision of the sampling design. We provide an optimality result to determine the sample sizes at cluster and sub-sample level. We apply the proposed sampling design to a dental study on human tooth size, and to a longitudinal study from an education intervention program.

Estimands in cluster-randomized trials: choosing analyses that answer the right question

BACKGROUND

Cluster-randomized trials (CRTs) involve randomizing groups of individuals (e.g. hospitals, schools or villages) to different interventions. Various approaches exist for analysing CRTs but there has been little discussion around the treatment effects (estimands) targeted by each.

METHODS

We describe the different estimands that can be addressed through CRTs and demonstrate how choices between different analytic approaches can impact the interpretation of results by fundamentally changing the question being asked, or, equivalently, the target estimand.

RESULTS

CRTs can address either the participant-average treatment effect (the average treatment effect across participants) or the cluster-average treatment effect (the average treatment effect across clusters). These two estimands can differ when participant outcomes or the treatment effect depends on the cluster size (referred to as 'informative cluster size'), which can occur for reasons such as differences in staffing levels or types of participants between small and large clusters. Furthermore, common estimators, such as mixed-effects models or generalized estimating equations with an exchangeable working correlation structure, can produce biased estimates for both the participant-average and cluster-average treatment effects when cluster size is informative. We describe alternative estimators (independence estimating equations and cluster-level analyses) that are unbiased for CRTs even when informative cluster size is present.

CONCLUSION

We conclude that careful specification of the estimand at the outset can ensure that the study question being addressed is clear and relevant, and, in turn, that the selected estimator provides an unbiased estimate of the desired quantity.

Simulating time-to-event data subject to competing risks and clustering: A review and synthesis

Simulation studies play an important role in evaluating the performance of statistical models developed for analyzing complex survival data such as those with competing risks and clustering. This article aims to provide researchers with a basic understanding of competing risks data generation, techniques for inducing cluster-level correlation, and ways to combine them together in simulation studies, in the context of randomized clinical trials with a binary exposure or treatment. We review data generation with competing and semi-competing risks and three approaches of inducing cluster-level correlation for time-to-event data: the frailty model framework, the probability transform, and Moran's algorithm. Using exponentially distributed event times as an example, we discuss how to introduce cluster-level correlation into generating complex survival outcomes, and illustrate multiple ways of combining these methods to simulate clustered, competing and semi-competing risks data with pre-specified correlation values or degree of clustering.

power swgee: GEE-based power calculations in stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials are increasingly being used to evaluate interventions in medical, public health, educational, and social science contexts. With the longitudinal and crossover nature of a SW-CRT, complex analysis techniques are often needed which makes appropriately powering SW-CRTs challenging. In this paper, we introduce a newly-developed SW-CRT power calculator, embedded within the power command in Stata. The power calculator assumes a marginal model (i.e., generalized estimating equations [GEE]) for the primary analysis of SW-CRTs, for which other currently available SW-CRT power calculators may not be suitable. The program accommodates complete cross-sectional and closed-cohort designs, and includes multilevel correlation structures appropriate for such designs. We discuss the methods and formulae underlying our SW-CRT calculator, and provide illustrative examples of the use of power swgee. We provide suggestions about the choice of parameters in power swgee, and conclude by discussing areas of future research which may improve the program.

Design and analysis of cluster randomized trials with time-to-event outcomes under the additive hazards mixed model

A primary focus of current methods for cluster randomized trials (CRTs) has been for continuous, binary, and count outcomes, with relatively less attention given to right-censored, time-to-event outcomes. In this article, we detail considerations for sample size requirement and statistical inference in CRTs with time-to-event outcomes when the intervention effect parameter is specified through the additive hazards mixed model (AHMM), which includes a frailty term to explicitly account for the dependency between the failure times. First, we discuss improved inference for the treatment effect parameter via bias-corrected sandwich variance estimators and randomization-based test under AHMM, addressing potential small-sample biases in CRTs. Next, we derive a new sample size formula for AHMM analysis of CRTs accommodating both equal and unequal cluster sizes. When the cluster sizes vary, our sample size formula depends on the mean and coefficient of variation of cluster sizes, based on which we articulate the impact of cluster size variation in CRTs with time-to-event outcomes. Furthermore, we obtain the insight that the classical variance inflation factor for CRTs with a non-censored outcome can in fact apply to CRTs with a time-to-event outcome, providing that an appropriate definition of the intraclass correlation coefficient is considered under AHMM. Simulation studies are carried out to illustrate key design and analysis considerations in CRTs with a small to moderate number of clusters. The proposed sample size procedure and analytical methods are further illustrated using the context of the STrategies to Reduce Injuries and Develop Confidence in Elders CRT.

Applying dissemination and implementation research methods to translate a school-based asthma program

Our school-based asthma program has reduced asthma exacerbations for youth with health disparities in the Denver metropolitan area, due partly to addressing social determinants of health, such as access to health care and medications. Dissemination and implementation (D&I) science approaches accelerate the translation of evidence-based programs into routine practice. D&I methods are being applied more commonly to improve health equity. The purpose of this publication was to give an overview of D&I research methods, using our school-based asthma program as an example. To successfully scale out our program across the state of Colorado, we are applying a D&I framework that guides the adaptation of our existing implementation approach to better meet our stakeholders' local context-the Exploration, Preparation, Implementation, Sustainment framework. In a pragmatic trial design, we will evaluate the outcomes of implementing the program across 5 Colorado regions, with attention to health equity, using a second commonly used D&I framework-Reach, Effectiveness, Adoption, Implementation, and Maintenance. Our central hypothesis is that our program will have broad and equitable reach to eligible students (primary outcome) and will reduce asthma attacks and symptoms. This D&I approach accelerates dissemination of our program and is an applicable process for translating other effective allergy/asthma programs to address asthma and allergy-related disparities.

Couples-based interventions and postpartum contraceptive uptake: A systematic review

OBJECTIVE

Systematically review the existing evidence about couples-based interventions and postpartum contraceptive uptake and generate recommendations for future research.

DATA SOURCES

PubMed, Web of Science, PsycINFO, Embase, and CINAHL through June 7, 2021.

STUDY SELECTION AND DATA EXTRACTION

Studies with a couples-based intervention assessing postpartum contraceptive uptake. Two independent reviewers screened studies, extracted data, and assessed risk of bias with RoB-2 (Cochrane Risk of Bias 2) for randomized and ROBINS-I (Risk of Bias in Non-Randomized Studies - Interventions) for observational studies. Data were synthesized in tables, figures, and a narrative review.

RESULTS

A total of 925 papers were identified, 66 underwent full text review, and 17 articles, which included 18 studies - 16 randomized, 2 observational - were included. The lack of intervention and outcome homogeneity precluded meta-analysis and isolating the effect of partner involvement. Four studies were partner-required, where partner involvement was a required component of the intervention, and 14 were partner-optional. Unadjusted risk differences ranged from 0.01 to 0.51 in favor of couples-based interventions increasing postpartum contraceptive uptake versus standard of care. Bias assessment of the 16 randomized studies classified 8, 3, and 5 studies as at a high, some concern, and low risk of bias. Common sources of bias included intervention non-adherence and missing outcome data. One observational study was at a high and the other at a low risk of bias.

CONCLUSIONS

Future studies that assess couples-based interventions must clearly define and measure how partners are involved in the intervention and assess how intervention adherence impacts postpartum contraceptive uptake.

Marginal modeling of cluster-period means and intraclass correlations in stepped wedge designs with binary outcomes

Stepped wedge cluster randomized trials (SW-CRTs) with binary outcomes are increasingly used in prevention and implementation studies. Marginal models represent a flexible tool for analyzing SW-CRTs with population-averaged interpretations, but the joint estimation of the mean and intraclass correlation coefficients (ICCs) can be computationally intensive due to large cluster-period sizes. Motivated by the need for marginal inference in SW-CRTs, we propose a simple and efficient estimating equations approach to analyze cluster-period means. We show that the quasi-score for the marginal mean defined from individual-level observations can be reformulated as the quasi-score for the same marginal mean defined from the cluster-period means. An additional mapping of the individual-level ICCs into correlations for the cluster-period means further provides a rigorous justification for the cluster-period approach. The proposed approach addresses a long-recognized computational burden associated with estimating equations defined based on individual-level observations, and enables fast point and interval estimation of the intervention effect and correlations. We further propose matrix-adjusted estimating equations to improve the finite-sample inference for ICCs. By providing a valid approach to estimate ICCs within the class of generalized linear models for correlated binary outcomes, this article operationalizes key recommendations from the CONSORT extension to SW-CRTs, including the reporting of ICCs.

A comparison of analytical strategies for cluster randomized trials with survival outcomes in the presence of competing risks

While statistical methods for analyzing cluster randomized trials with continuous and binary outcomes have been extensively studied and compared, little comparative evidence has been provided for analyzing cluster randomized trials with survival outcomes in the presence of competing risks. Motivated by the Strategies to Reduce Injuries and Develop Confidence in Elders trial, we carried out a simulation study to compare the operating characteristics of several existing population-averaged survival models, including the marginal Cox, marginal Fine and Gray, and marginal multi-state models. For each model, we found that adjusting for the intraclass correlations through the sandwich variance estimator effectively maintained the type I error rate when the number of clusters is large. With no more than 30 clusters, however, the sandwich variance estimator can exhibit notable negative bias, and a permutation test provides better control of type I error inflation. Under the alternative, the power for each model is differentially affected by two types of intraclass correlations-the within-individual and between-individual correlations. Furthermore, the marginal Fine and Gray model occasionally leads to higher power than the marginal Cox model or the marginal multi-state model, especially when the competing event rate is high. Finally, we provide an illustrative analysis of Strategies to Reduce Injuries and Develop Confidence in Elders trial using each analytical strategy considered.

Partisan differences in the effects of economic evidence and local data on legislator engagement with dissemination materials about behavioral health: a dissemination trial

BACKGROUND

State legislators make policy decisions that influence children's exposure to adverse childhood experiences (ACEs), such as child maltreatment, and their effects on behavioral health. Effective dissemination of scientific research can increase the likelihood that legislators' decisions are aligned with evidence to prevent ACEs and their consequences, and effective dissemination requires legislators to engage with dissemination materials. Informed by the elaboration likelihood model of persuasive communication and Brownson's Model of Dissemination Research, we tested the hypothesis that inclusion of economic evidence and local data would increase legislator engagement with dissemination materials about evidence-supported policies related to ACEs and behavioral health.

METHODS

A three-arm randomized dissemination trial was conducted. A university researcher e-mailed dissemination materials which contained evidence about ACEs and behavioral health problems to state legislators (two e-mails sent 2 weeks apart, 12,662 e-mails delivered to 6509 legislators). The e-mail subject lines, text, and policy brief content were manipulated across the study arms. The intervention condition received state-tailored data about rates of ACEs and state-tailored economic evidence about the costs of ACEs for public systems, the enhanced control condition received state-tailored data and not economic evidence, and the control condition received national data and not economic evidence. Outcomes were rates of e-mail views, policy brief link clicks, requests for researcher consultation, and mentions of child maltreatment terms in legislators' social media posts.

RESULTS

For the first e-mail, the e-mail view rate was 42.6% higher in the intervention than in the enhanced control condition (22.8% vs. 14.8%) and 20.8% higher than in the control condition (22.8% vs. 18.5%) (both p < .0001). Similar results were observed for the second e-mail. These differences remained significant after adjustment for demographic differences across study conditions in individual-level models, but not multilevel models. There was a significant interaction between the experimental condition and political party (p < .0001) in which the intervention increased e-mail view rates among Democrats but not Republicans. The intervention had no effect on policy brief link clicks or requests for consultation and a mixed effect on social media posts.

CONCLUSIONS

Inclusion of state-tailored economic evidence in dissemination materials can increase engagement with research evidence among Democrat, but not Republican, legislators. Dissemination strategies tailored for legislators' political party affiliation may be needed.

Implementation strategy in collaboration with people with lived experience of mental illness to reduce stigma among primary care providers in Nepal (RESHAPE): protocol for a type 3 hybrid implementation effectiveness cluster randomized controlled trial

BACKGROUND

There are increasing efforts for the integration of mental health services into primary care settings in low- and middle-income countries. However, commonly used approaches to train primary care providers (PCPs) may not achieve the expected outcomes for improved service delivery, as evidenced by low detection rates of mental illnesses after training. One contributor to this shortcoming is the stigma among PCPs. Implementation strategies for training PCPs that reduce stigma have the potential to improve the quality of services.

DESIGN

In Nepal, a type 3 hybrid implementation-effectiveness cluster randomized controlled trial will evaluate the implementation-as-usual training for PCPs compared to an alternative implementation strategy to train PCPs, entitled Reducing Stigma among Healthcare Providers (RESHAPE). In implementation-as-usual, PCPs are trained on the World Health Organization Mental Health Gap Action Program Intervention Guide (mhGAP-IG) with trainings conducted by mental health specialists. In RESHAPE, mhGAP-IG training includes the added component of facilitation by people with lived experience of mental illness (PWLE) and their caregivers using PhotoVoice, as well as aspirational figures. The duration of PCP training is the same in both arms. Co-primary outcomes of the study are stigma among PCPs, as measured with the Social Distance Scale at 6 months post-training, and reach, a domain from the RE-AIM implementation science framework. Reach is operationalized as the accuracy of detection of mental illness in primary care facilities and will be determined by psychiatrists at 3 months after PCPs diagnose the patients. Stigma will be evaluated as a mediator of reach. Cost-effectiveness and other RE-AIM outcomes will be assessed. Twenty-four municipalities, the unit of clustering, will be randomized to either mhGAP-IG implementation-as-usual or RESHAPE arms, with approximately 76 health facilities and 216 PCPs divided equally between arms. An estimated 1100 patients will be enrolled for the evaluation of accurate diagnosis of depression, generalized anxiety disorder, psychosis, or alcohol use disorder. Masking will include PCPs, patients, and psychiatrists.

DISCUSSION

This study will advance the knowledge of stigma reduction for training PCPs in partnership with PWLE. This collaborative approach to training has the potential to improve diagnostic competencies. If successful, this implementation strategy could be scaled up throughout low-resource settings to reduce the global treatment gap for mental illness.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04282915 . Date of registration: February 25, 2020.

Stepped Wedge Cluster Randomized Trials: A Methodological Overview

BACKGROUND

Stepped wedge cluster randomized trials enable rigorous evaluations of health intervention programs in pragmatic settings. In the present study, we aimed to update neurosurgeon scientists on the design of stepped wedge randomized trials.

METHODS

We have presented an overview of recent methodological developments for stepped wedge designs and included an update on the newer associated methodological tools to aid with future study designs.

RESULTS

We defined the stepped wedge trial design and reviewed the indications for the design in depth. In addition, key considerations, including mainstream methods of analysis and sample size determination, were discussed.

CONCLUSIONS

Stepped wedge designs can be attractive for study intervention programs aiming to improve the delivery of patient care, especially when examining a small number of heterogeneous clusters.

Marginal modeling in community randomized trials with rare events: Utilization of the negative binomial regression model

BACKGROUND/AIMS

This work is motivated by the HEALing Communities Study, which is a post-test only cluster randomized trial in which communities are randomized to two different trial arms. The primary interest is in reducing opioid overdose fatalities, which will be collected as a count outcome at the community level. Communities range in size from thousands to over one million residents, and fatalities are expected to be rare. Traditional marginal modeling approaches in the cluster randomized trial literature include the use of generalized estimating equations with an exchangeable correlation structure when utilizing subject-level data, or analogously quasi-likelihood based on an over-dispersed binomial variance when utilizing community-level data. These approaches account for and estimate the intra-cluster correlation coefficient, which should be provided in the results from a cluster randomized trial. Alternatively, the coefficient of variation or R coefficient could be reported. In this article, we show that negative binomial regression can also be utilized when communities are large and events are rare. The objectives of this article are (1) to show that the negative binomial regression approach targets the same marginal regression parameter(s) as an over-dispersed binomial model and to explain why the estimates may differ; (2) to derive formulas relating the negative binomial overdispersion parameter k with the intra-cluster correlation coefficient, coefficient of variation, and R coefficient; and (3) analyze pre-intervention data from the HEALing Communities Study to demonstrate and contrast models and to show how to report the intra-cluster correlation coefficient, coefficient of variation, and R coefficient when utilizing negative binomial regression.

METHODS

Negative binomial and over-dispersed binomial regression modeling are contrasted in terms of model setup, regression parameter estimation, and formulation of the overdispersion parameter. Three specific models are used to illustrate concepts and address the third objective.

RESULTS

The negative binomial regression approach targets the same marginal regression parameter(s) as an over-dispersed binomial model, although estimates may differ. Practical differences arise in regard to how overdispersion, and hence the intra-cluster correlation coefficient is modeled. The negative binomial overdispersion parameter is approximately equal to the ratio of the intra-cluster correlation coefficient and marginal probability, the square of the coefficient of variation, and the R coefficient minus 1. As a result, estimates corresponding to all four of these different types of overdispersion parameterizations can be reported when utilizing negative binomial regression.

CONCLUSION

Negative binomial regression provides a valid, practical, alternative approach to the analysis of count data, and corresponding reporting of overdispersion parameters, from community randomized trials in which communities are large and events are rare.

Social network-based group intervention to promote HIV prevention in Uganda: study protocol for a cluster randomized controlled trial of Game Changers

INTRODUCTION

Innovative strategies are needed to disseminate HIV prevention messages across communities efficiently, as well as reduce HIV stigma while promoting HIV prevention. This randomized controlled trial will evaluate the efficacy of a social network-based group intervention, Game Changers, which trains persons living with HIV (PLWH) to encourage members of their social network to use HIV protective behaviors METHODS: PLWH in HIV care for at least 1 year will be randomly assigned to receive the 8-session group advocacy training intervention or no-intervention control group. Each enrolled PLWH (index participant) will be asked to recruit up to four social network members (alter participant). Assessments will be administered at baseline and months 6, 12, and 18 to both index and alter participants. The primary outcomes are HIV testing and condom use among alter participants; secondary outcomes are engagement in HIV prevention advocacy and internalized HIV stigma among index participants. Repeated-measures multivariable regression analyses will be conducted to compare outcomes between the intervention and control arms, in addition to a cost-effectiveness evaluation.

DISCUSSION

This social network-based approach to HIV prevention is particularly timely in the era of biomedical interventions, which require widespread penetration of effective HIV prevention and care messaging into communities. Positioning PLWH as central to the solution for controlling (vs. causing) the HIV epidemic has the potential to reduce HIV stigma and improve prevention outcomes at the individual and network levels.

TRIAL REGISTRATION

ClinicalTrials.gov NIH Clinical Trial Registry NCT05098015. Registered on October 18, 2021.

Finite-sample adjustments in variance estimators for clustered competing risks regression

The marginal Fine-Gray proportional subdistribution hazards model is a popular approach to directly study the association between covariates and the cumulative incidence function with clustered competing risks data, which often arise in multicenter randomized trials or multilevel observational studies. To account for the within-cluster correlations between failure times, the uncertainty of the regression parameters estimators is quantified by the robust sandwich variance estimator, which may have unsatisfactory performance with a limited number of clusters. To overcome this limitation, we propose four bias-corrected variance estimators to reduce the negative bias of the usual sandwich variance estimator, extending the bias-correction techniques from generalized estimating equations with noncensored exponential family outcomes to clustered competing risks outcomes. We further compare their finite-sample operating characteristics through simulations and two real data examples. In particular, we found the Mancl and DeRouen (MD) type sandwich variance estimator generally has the smallest bias. Furthermore, with a small number of clusters, the Wald t -confidence interval with the MD sandwich variance estimator carries close to nominal coverage for the cluster-level effect parameter. The t -confidence intervals based on the sandwich variance estimator with any one of the three types of multiplicative bias correction or the z -confidence interval with the Morel, Bokossa and Neerchal (MBN) type sandwich variance estimator have close to nominal coverage for the individual-level effect parameter. Finally, we develop a user-friendly R package crrcbcv implementing the proposed sandwich variance estimators to assist practical applications.

Practical considerations in utilizing cluster randomized controlled trials conducted in biopharmaceutical industry

Cluster randomized controlled trials (cluster RCTs), also known as parallel-arm group-randomized trials, are trials in which the randomized units are groups of participants, as opposed to individual participants. These trials have largely been implemented to address broad public health issues, but with the growing interest in use of real-world data in the regulatory setting, this design may be increasingly considered for industry trials. The key difference between cluster RCTs and traditional RCTs is the intraclass correlation coefficient (ICC) that needs to be considered in cluster RCTs. In this article, we discuss some key practical considerations that are related to ICC in the design, conduct, analysis, and report stages of a cluster RCT. These key considerations related to ICC can lead to improvement in how we translate research findings from cluster RCTs into practices in the biopharmaceutical industry.

Sample size calculation in hierarchical 2×2 factorial trials with unequal cluster sizes

Motivated by a suicide prevention trial with hierarchical treatment allocation (cluster-level and individual-level treatments), we address the sample size requirements for testing the treatment effects as well as their interaction. We assume a linear mixed model, within which two types of treatment effect estimands (controlled effect and marginal effect) are defined. For each null hypothesis corresponding to an estimand, we derive sample size formulas based on large-sample z-approximation, and provide finite-sample modifications based on a t-approximation. We relax the equal cluster size assumption and express the sample size formulas as functions of the mean and coefficient of variation of cluster sizes. We show that the sample size requirement for testing the controlled effect of the cluster-level treatment is more sensitive to cluster size variability than that for testing the controlled effect of the individual-level treatment; the same observation holds for testing the marginal effects. In addition, we show that the sample size for testing the interaction effect is proportional to that for testing the controlled or the marginal effect of the individual-level treatment. We conduct extensive simulations to validate the proposed sample size formulas, and find the empirical power agrees well with the predicted power for each test. Furthermore, the t-approximations often provide better control of type I error rate with a small number of clusters. Finally, we illustrate our sample size formulas to design the motivating suicide prevention factorial trial. The proposed methods are implemented in the R package H2x2Factorial.

Constrained randomization and statistical inference for multi‐arm parallel cluster randomized controlled trials

A practical limitation of cluster randomized controlled trials (cRCTs) is that the number of available clusters may be small, resulting in an increased risk of baseline imbalance under simple randomization. Constrained randomization overcomes this issue by restricting the allocation to a subset of randomization schemes where sufficient overall covariate balance across comparison arms is achieved. However, for multi-arm cRCTs, several design and analysis issues pertaining to constrained randomization have not been fully investigated. Motivated by an ongoing multi-arm cRCT, we elaborate the method of constrained randomization and provide a comprehensive evaluation of the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs in multi-arm cRCTs, with varying combinations of design and analysis-based covariate adjustment strategies. In particular, as randomization-based tests have not been extensively studied in multi-arm cRCTs, we additionally develop most-powerful randomization tests under the linear mixed model framework for our comparisons. Our results indicate that under constrained randomization, both model-based and randomization-based analyses could gain power while preserving nominal type I error rate, given proper analysis-based adjustment for the baseline covariates. Randomization-based analyses, however, are more robust against violations of distributional assumptions. The choice of balance metrics and candidate set sizes and their implications on the testing of the pairwise and global hypotheses are also discussed. Finally, we caution against the design and analysis of multi-arm cRCTs with an extremely small number of clusters, due to insufficient degrees of freedom and the tendency to obtain an overly restricted randomization space.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

A note on the estimation and inference with quadratic inference functions for correlated outcomes

The quadratic inference function approach is a popular method in the analysis of correlated data. The quadratic inference function is formulated based on multiple sets of score equations (or extended score equations) that over-identify the regression parameters of interest, and improves efficiency over the generalized estimating equations under correlation misspecification. In this note, we provide an alternative solution to the quadratic inference function by separately solving each set of score equations and combining the solutions. We provide an insight that an optimally weighted combination of estimators obtained separately from the distinct sets of score equations is asymptotically equivalent to the estimator obtained via the quadratic inference function. We further establish results on inference for the optimally weighted estimator and extend these insights to the general setting with over-identified estimating equations. A simulation study is carried out to confirm the analytical insights and connections in finite samples.

A comparative study of R functions for clustered data analysis

Background

Clustered or correlated outcome data is common in medical research studies, such as the analysis of national or international disease registries, or cluster-randomized trials, where groups of trial participants, instead of each trial participant, are randomized to interventions. Within-group correlation in studies with clustered data requires the use of specific statistical methods, such as generalized estimating equations and mixed-effects models, to account for this correlation and support unbiased statistical inference.

Methods

We compare different approaches to estimating generalized estimating equations and mixed effects models for a continuous outcome in R through a simulation study and a data example. The methods are implemented through four popular functions of the statistical software R, “geese”, “gls”, “lme”, and “lmer”. In the simulation study, we compare the mean squared error of estimating all the model parameters and compare the coverage proportion of the 95% confidence intervals. In the data analysis, we compare estimation of the intervention effect and the intra-class correlation.

Results

In the simulation study, the function “lme” takes the least computation time. There is no difference in the mean squared error of the four functions. The “lmer” function provides better coverage of the fixed effects when the number of clusters is small as 10. The function “gls” produces close to nominal scale confidence intervals of the intra-class correlation. In the data analysis and the “gls” function yields a positive estimate of the intra-class correlation while the “geese” function gives a negative estimate. Neither of the confidence intervals contains the value zero.

Conclusions

The “gls” function efficiently produces an estimate of the intra-class correlation with a confidence interval. When the within-group correlation is as high as 0.5, the confidence interval is not always obtainable.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13063-021-05900-7).

Two-Stage TMLE to reduce bias and improve efficiency in cluster randomized trials

Cluster randomized trials (CRTs) randomly assign an intervention to groups of individuals (e.g., clinics or communities) and measure outcomes on individuals in those groups. While offering many advantages, this experimental design introduces challenges that are only partially addressed by existing analytic approaches. First, outcomes are often missing for some individuals within clusters. Failing to appropriately adjust for differential outcome measurement can result in biased estimates and inference. Second, CRTs often randomize limited numbers of clusters, resulting in chance imbalances on baseline outcome predictors between arms. Failing to adaptively adjust for these imbalances and other predictive covariates can result in efficiency losses. To address these methodological gaps, we propose and evaluate a novel two-stage targeted minimum loss-based estimator to adjust for baseline covariates in a manner that optimizes precision, after controlling for baseline and postbaseline causes of missing outcomes. Finite sample simulations illustrate that our approach can nearly eliminate bias due to differential outcome measurement, while existing CRT estimators yield misleading results and inferences. Application to real data from the SEARCH community randomized trial demonstrates the gains in efficiency afforded through adaptive adjustment for baseline covariates, after controlling for missingness on individual-level outcomes.

Outcomes of Team-Centered, Occupational Adaptation-Based Versus Traditional Dementia Workforce Training : Résultats comparés d'une formation axée sur l'équipe et l'adaptation occupationnelle et de l'approche de formation traditionnelle pour le personnel travaillant auprès des personnes atteintes de démence

Background. Dementia workforce training aligned with Occupational Adaptation (OA) theory may facilitate teams resolving challenges in the care environment more than traditional skills-based (SB) training, although comparisons are needed. Purpose. This pilot study compared effectiveness of an OA and SB program on relative mastery and team development for dementia care teams at a continuing care retirement community. Method. In a quasi-experimental study, employees underwent nine sessions in an OA or SB program. Relative Mastery Measurement Scale and Team Development Measure results were collected pre-, mid-, post-intervention. A 3 × 2 ANOVA determined differences in group score changes across time. Findings. Data from 28 employees (14/group) showed group-by-time interaction reached statistical significance for both relative mastery (F = 3.17, df = 2, p = .05) and team development (F = 8.38, df = 2, p = .001). Implications. OA-based training may improve dementia care teams' collaborative mastery over real-world challenges. While preliminary findings inform program developers, further research must explore clinical effectiveness.

Cluster randomized trials of individual-level interventions were at high risk of bias

OBJECTIVES

To describe the prevalence of risks of bias in cluster-randomized trials of individual-level interventions, according to the Cochrane Risk of Bias tool.

STUDY DESIGN AND SETTING

Review undertaken in duplicate of a random sample of 40 primary reports of cluster-randomized trials of individual-level interventions.

RESULTS

The most common reported reasons for adopting cluster randomization were the need to avoid contamination (17, 42.5%) and practical considerations (14, 35%). Of the 40 trials all but one was assessed as being at risk of bias. A majority (27, 67.5%) were assessed as at risk due to the timing of identification and recruitment of participants; many (21, 52.5%) due to an apparent lack of adequate allocation concealment; and many due to selectively reported results (22, 55%), arising from a mixture of reasons including lack of documentation of primary outcome. Other risks mostly occurred infrequently.

CONCLUSION

Many cluster-randomized trials evaluating individual-level interventions appear to be at risk of bias, mostly due to identification and recruitment biases. We recommend that investigators carefully consider the need for cluster randomization; follow recommended procedures to mitigate risks of identification and recruitment bias; and adhere to good reporting practices including clear documentation of primary outcome and allocation concealment methods.

Using cluster-robust standard errors when analyzing group-randomized trials with few clusters

Accounting for dependent observations in cluster-randomized trials (CRTs) using nested data is necessary in order to avoid misestimated standard errors resulting in questionable inferential statistics. Cluster-robust standard errors (CRSEs) are often used to address this issue. However, CRSEs are still well-known to underestimate standard errors for group-level variables when the number of clusters is low (e.g., < 50) and with CRTs, a small number of clusters, due to logistical or financial considerations, is the norm rather than the exception. Using a simulation with various conditions, we investigate the use of a small sample correction (i.e., CR2 estimator) proposed by Bell and McCaffrey (2002) together with empirically derived degrees of freedom estimates (dof_BM). Findings indicate that even with as few as 10 clusters, the CR2 estimator used with dof_BM yields generally unbiased results with acceptable type I error and coverage rates. Results show that coverage and type I error rates can be largely influenced by the choice of dof, not just the standard error adjustments. An applied example is provided together with R syntax to conduct the analysis. To facilitate the use of different CRSEs, a free graphical, menu-driven SPSS add-on to compute the various cluster-robust variance estimates can be downloaded from https://github.com/flh3/CR2/tree/master/SPSS .

Randomized controlled trial of the "WISER" intervention to reduce healthcare worker burnout

OBJECTIVE

Test web-based implementation for the science of enhancing resilience (WISER) intervention efficacy in reducing healthcare worker (HCW) burnout.

DESIGN

RCT using two cohorts of HCWs of four NICUs each, to improve HCW well-being (primary outcome: burnout). Cohort 1 received WISER while Cohort 2 acted as a waitlist control.

RESULTS

Cohorts were similar, mostly female (83%) and nurses (62%). In Cohorts 1 and 2 respectively, 182 and 299 initiated WISER, 100 and 176 completed 1-month follow-up, and 78 and 146 completed 6-month follow-up. Relative to control, WISER decreased burnout (-5.27 (95% CI: -10.44, -0.10), p = 0.046). Combined adjusted cohort results at 1-month showed that the percentage of HCWs reporting concerning outcomes was significantly decreased for burnout (-6.3% (95%CI: -11.6%, -1.0%); p = 0.008), and secondary outcomes depression (-5.2% (95%CI: -10.8, -0.4); p = 0.022) and work-life integration (-11.8% (95%CI: -17.9, -6.1); p < 0.001). Improvements endured at 6 months.

CONCLUSION

WISER appears to durably improve HCW well-being.

CLINICAL TRIALS NUMBER

NCT02603133; https://clinicaltrials.gov/ct2/show/NCT02603133.

Methods for dealing with unequal cluster sizes in cluster randomized trials: A scoping review

In a cluster-randomized trial (CRT), the number of participants enrolled often varies across clusters. This variation should be considered during both trial design and data analysis to ensure statistical performance goals are achieved. Most methodological literature on the CRT design has assumed equal cluster sizes. This scoping review focuses on methodology for unequal cluster size CRTs. EMBASE, Medline, Google Scholar, MathSciNet and Web of Science databases were searched to identify English-language articles reporting on methodology for unequal cluster size CRTs published until March 2021. We extracted data on the focus of the paper (power calculation, Type I error etc.), the type of CRT, the type and the range of parameter values investigated (number of clusters, mean cluster size, cluster size coefficient of variation, intra-cluster correlation coefficient, etc.), and the main conclusions. Seventy-nine of 5032 identified papers met the inclusion criteria. Papers primarily focused on the parallel-arm CRT (p-CRT, n = 60, 76%) and the stepped-wedge CRT (n = 14, 18%). Roughly 75% of the papers addressed trial design issues (sample size/power calculation) while 25% focused on analysis considerations (Type I error, bias, etc.). The ranges of parameter values explored varied substantially across different studies. Methods for accounting for unequal cluster sizes in the p-CRT have been investigated extensively for Gaussian and binary outcomes. Synthesizing the findings of these works is difficult as the magnitude of impact of the unequal cluster sizes varies substantially across the combinations and ranges of input parameters. Limited investigations have been done for other combinations of a CRT design by outcome type, particularly methodology involving binary outcomes-the most commonly used type of primary outcome in trials. The paucity of methodological papers outside of the p-CRT with Gaussian or binary outcomes highlights the need for further methodological development to fill the gaps.

Effects of Mindfulness-Based Stress Reduction on Experimental Pain Sensitivity and Cortisol Responses in Women With Early Life Abuse: A Randomized Controlled Trial

OBJECTIVE

Early life abuse (ELAb) initiates pathophysiological cascades resulting in long-term maladaptive stress responsivity, hyperalgesia, and an increased risk of psychopathology. Mindfulness-based stress reduction (MBSR) is effective in modifying psychological and somatic symptoms; thus, we predicted that MBSR would be particularly efficacious for women with ELAb.

METHOD

Medically healthy women (mean age = 31 years) with or without a history of early (≤13 years) physical or sexual abuse provided self-report measures and were tested in the laboratory before and after randomization to standard MBSR (n = 52) or social support (SSG) (n = 60) for 8 weeks. The laboratory procedure involved pain testing using the cold pressor and temporal summation of heat pain (indexing central sensitization) procedures, and exposure to the Trier Social Stress Test. Plasma cortisol in response to the experimental protocol was assessed as area under the curve (AUC).

RESULTS

The interventions differentially impacted pain sensitivity and cortisol AUC for women with ELAb, as MBSR increased the temporal summation of heat pain intensity ratings (p = .024) and reduced cortisol AUC (p = .004). For women without ELAb, MBSR decreased cold pressor tolerance (p = .045) and decreased the temporal summation of heat pain intensity ratings relative to SSG (p = .024). Both MBSR and SSG improved depression symptoms and emotion regulation abilities (p values < .001); however, MBSR was associated with greater benefits in describing emotions (p = .008) and impulse control (p = .017) for women with ELAb.

CONCLUSIONS

Women with ELAb benefited from MBSR-specific improvements in central sensitization, mindfulness skills, and emotion regulation abilities. This is the first study to examine the efficacy of MBSR in modifying affective and somatic symptoms based on ELAb status and provides evidence for considering ELAb in tailoring treatment approaches.Trial Registration: ClinicalTrials.gov Identifier: NCT01995916; https://clinicaltrials.gov/ct2/show/NCT01995916.