The 2 × 2 cluster randomized controlled factorial trial design is mainly used for efficiency and to explore intervention interactions: a systematic review

A Hybrid Pragmatic and Factorial Cluster Randomized Controlled Trial for an Anti-racist, Multilevel Intervention to Improve Mental Health Equity in High Schools

Systemic racism is pervasive in US society and disproportionately limits opportunities for education, work, and health for historically marginalized and minoritized racial and ethnic groups, making it an urgent issue of social justice. Because systemic racism is a social determinant of health prevalent across multiple social and institutional structures, it requires multilevel intervention approaches using effective designs and analytic methods to measure and evaluate outcomes. Racism is a fundamental cause of poor health outcomes, including mental health outcomes; thus, mental health services and programs that address racism and discrimination are key to promoting positive mental health of racial and ethnic minority youth. While multilevel interventions are well-suited for improving outcomes like youth mental health disparities, their evaluation poses unique methodological challenges, requiring specialized design and analytic approaches. There has been limited methodological guidance provided to researchers on how to test multilevel interventions using approaches that balance methodological rigor, practicality, and acceptability across stakeholder groups, especially within communities most affected by systemic racism. This paper addresses this gap by providing an example of how to rigorously evaluate a hypothetical, theoretically based, multilevel intervention promoting mental health equity in three US school systems using an anti-racist approach intervening at the macro- (i.e., school system), meso- (i.e., school), and micro- (i.e., family and student) levels to improve mental health in adolescents. We describe the design, sample size considerations, and analytic methods to comprehensively evaluate its effectiveness while exploring the extent to which the components interact synergistically to improve outcomes. The methodological approach proposed can be adapted to other multilevel interventions that include strategies addressing macro-, meso-, and micro-levels of influence.

Development of a conceptual framework for defining trial efficiency

BACKGROUND

Globally, there is a growing focus on efficient trials, yet numerous interpretations have emerged, suggesting a significant heterogeneity in understanding "efficiency" within the trial context. Therefore in this study, we aimed to dissect the multifaceted nature of trial efficiency by establishing a comprehensive conceptual framework for its definition.

OBJECTIVES

To collate diverse perspectives regarding trial efficiency and to achieve consensus on a conceptual framework for defining trial efficiency.

METHODS

From July 2022 to July 2023, we undertook a literature review to identify various terms that have been used to define trial efficiency. We then conducted a modified e-Delphi study, comprising an exploratory open round and a subsequent scoring round to refine and validate the identified items. We recruited a wide range of experts in the global trial community including trialists, funders, sponsors, journal editors and members of the public. Consensus was defined as items rated "without disagreement", measured by the inter-percentile range adjusted for symmetry through the UCLA/RAND approach.

RESULTS

Seventy-eight studies were identified from a literature review, from which we extracted nine terms related to trial efficiency. We then used review findings as exemplars in the Delphi open round. Forty-nine international experts were recruited to the e-Delphi panel. Open round responses resulted in the refinement of the initial nine terms, which were consequently included in the scoring round. We obtained consensus on all nine items: 1) four constructs that collectively define trial efficiency containing scientific efficiency, operational efficiency, statistical efficiency and economic efficiency; and 2) five essential building blocks for efficient trial comprising trial design, trial process, infrastructure, superstructure, and stakeholders.

CONCLUSIONS

This is the first attempt to dissect the concept of trial efficiency into theoretical constructs. Having an agreed definition will allow better trial implementation and facilitate effective communication and decision-making across stakeholders. We also identified essential building blocks that are the cornerstones of an efficient trial. In this pursuit of understanding, we are not only unravelling the complexities of trial efficiency but also laying the groundwork for evaluating the efficiency of an individual trial or a trial system in the future.

Reporting of Factorial Randomized Trials: Extension of the CONSORT 2010 Statement

Importance

Transparent reporting of randomized trials is essential to facilitate critical appraisal and interpretation of results. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, reporting of factorial trials is suboptimal.

Objective

To develop a consensus-based extension to the Consolidated Standards of Reporting Trials (CONSORT) 2010 Statement for factorial trials.

Design

Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the CONSORT extension for factorial trials was developed by (1) generating a list of reporting recommendations for factorial trials using a scoping review of methodological articles identified using a MEDLINE search (from inception to May 2019) and supplemented with relevant articles from the personal collections of the authors; (2) a 3-round Delphi survey between January and June 2022 to identify additional items and assess the importance of each item, completed by 104 panelists from 14 countries; and (3) a hybrid consensus meeting attended by 15 panelists to finalize the selection and wording of items for the checklist.

Findings

This CONSORT extension for factorial trials modifies 16 of the 37 items in the CONSORT 2010 checklist and adds 1 new item. The rationale for the importance of each item is provided. Key recommendations are (1) the reason for using a factorial design should be reported, including whether an interaction is hypothesized, (2) the treatment groups that form the main comparisons should be clearly identified, and (3) for each main comparison, the estimated interaction effect and its precision should be reported.

Conclusions and Relevance

This extension of the CONSORT 2010 Statement provides guidance on the reporting of factorial randomized trials and should facilitate greater understanding of and transparency in their reporting.

Consensus Statement for Protocols of Factorial Randomized Trials: Extension of the SPIRIT 2013 Statement

Importance

Trial protocols outline a trial's objectives as well as the methods (design, conduct, and analysis) that will be used to meet those objectives, and transparent reporting of trial protocols ensures objectives are clear and facilitates appraisal regarding the suitability of study methods. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, no extension of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 Statement, which provides guidance on reporting of trial protocols, for factorial trials is available.

Objective

To develop a consensus-based extension to the SPIRIT 2013 Statement for factorial trials.

Evidence Review

The SPIRIT extension for factorial trials was developed using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework. First, a list of reporting recommendations was generated using a scoping review of methodological articles identified using a MEDLINE search (inception to May 2019), which was supplemented with relevant articles from the personal collections of the authors. Second, a 3-round Delphi survey (January to June 2022, completed by 104 panelists from 14 countries) was conducted to assess the importance of items and identify additional recommendations. Third, a hybrid consensus meeting was held, attended by 15 panelists to finalize selection and wording of the checklist.

Findings

This SPIRIT extension for factorial trials modified 9 of the 33 items in the SPIRIT 2013 checklist. Key reporting recommendations were that the rationale for using a factorial design should be provided, including whether an interaction is hypothesized; the treatment groups that will form the main comparisons should be identified; and statistical methods for each main comparison should be provided, including how interactions will be assessed.

Conclusions and Relevance

In this consensus statement, 9 factorial-specific items were provided that should be addressed in all protocols of factorial trials to increase the trial's utility and transparency.

Four 2×2 factorial trials of smartphone CBT to reduce subthreshold depression and to prevent new depressive episodes among adults in the community-RESiLIENT trial (Resilience Enhancement with Smartphone in LIving ENvironmenTs): a master protocol

INTRODUCTION

The health burden due to depression is ever increasing in the world. Prevention is a key to reducing this burden. Guided internet cognitive-behavioural therapies (iCBT) appear promising but there is room for improvement because we do not yet know which of various iCBT skills are more efficacious than others, and for whom. In addition, there has been no platform for iCBT that can accommodate ongoing evolution of internet technologies.

METHODS AND ANALYSIS

Based on our decade-long experiences in developing smartphone CBT apps and examining them in randomised controlled trials, we have developed the Resilience Training App Version 2. This app now covers five CBT skills: cognitive restructuring, behavioural activation, problem-solving, assertion training and behaviour therapy for insomnia. The current study is designed as a master protocol including four 2×2 factorial trials using this app (1) to elucidate specific efficacies of each CBT skill, (2) to identify participants' characteristics that enable matching between skills and individuals, and (3) to allow future inclusion of new skills. We will recruit 3520 participants with subthreshold depression and ca 1700 participants without subthreshold depression, to examine the short-term efficacies of CBT skills to reduce depressive symptoms in the former and to explore the long-term efficacies in preventing depression in the total sample. The primary outcome for the short-term efficacies is the change in depressive symptoms as measured with the Patient Health Questionnaire-9 at week 6, and that for the long-term efficacies is the incidence of major depressive episodes as assessed by the computerised Composite International Diagnostic Interview by week 50.

ETHICS AND DISSEMINATION

The trial has been approved by the Ethics Committee of Kyoto University Graduate School of Medicine (C1556).

TRIAL REGISTRATION NUMBER

UMIN000047124.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Sample size calculation in hierarchical 2×2 factorial trials with unequal cluster sizes

Motivated by a suicide prevention trial with hierarchical treatment allocation (cluster-level and individual-level treatments), we address the sample size requirements for testing the treatment effects as well as their interaction. We assume a linear mixed model, within which two types of treatment effect estimands (controlled effect and marginal effect) are defined. For each null hypothesis corresponding to an estimand, we derive sample size formulas based on large-sample z-approximation, and provide finite-sample modifications based on a t-approximation. We relax the equal cluster size assumption and express the sample size formulas as functions of the mean and coefficient of variation of cluster sizes. We show that the sample size requirement for testing the controlled effect of the cluster-level treatment is more sensitive to cluster size variability than that for testing the controlled effect of the individual-level treatment; the same observation holds for testing the marginal effects. In addition, we show that the sample size for testing the interaction effect is proportional to that for testing the controlled or the marginal effect of the individual-level treatment. We conduct extensive simulations to validate the proposed sample size formulas, and find the empirical power agrees well with the predicted power for each test. Furthermore, the t-approximations often provide better control of type I error rate with a small number of clusters. Finally, we illustrate our sample size formulas to design the motivating suicide prevention factorial trial. The proposed methods are implemented in the R package H2x2Factorial.

A factorial randomized controlled trial to examine separate and combined effects of a simulation-based empathy enhancement program and a lecture-based education program on family caregivers of people with dementia

OBJECTIVES

The present study was a 2 × 2 factorial randomized controlled trial that examined if a simulation-based empathy enhancement program and a lecture-based education program, together or separately, improved outcomes of family caregivers of people with dementia.

METHOD

A total of 101 participants were randomly assigned to simulation-based education only, lecture-based education only, simulation-based education plus lecture-based education, or treatment as usual. Data were analyzed using a two-way analysis of covariance while controlling for pretest results, kinship, and gender.

RESULTS

Statistically significant interaction effects between the simulation program and the lecture program were found in the levels of well-being and helplessness. The lecture program accompanied by the simulation program led to higher level of well-being in terms of happiness and lower level of helplessness than the lecture program alone. Caregivers with the lecture program provided led to less frequent use of dysfunctional coping strategies than those with no lecture program provided. Caregivers with the simulation program provided led to more frequent use of emotion-focused coping strategies than those with no simulation program provided.

CONCLUSIONS

Findings of the present study support benefits of combined of and separate simulation-based and lecture-based programs on family caregivers of people with dementia in important outcomes affecting quality of care and quality of lives in families of people with dementia. Further studies are needed to identify intervention components that can improve empathy of family caregivers of people with dementia and be embedded into a multicomponent program tailored better to families in different needs.

Impact of nonclinical factors on intensive care unit admission decisions: a vignette-based randomized trial (V-TRIAGE)

OBJECTIVE

To assess the impact of intensive care unit bed availability, distractors and choice framing on intensive care unit admission decisions.

METHODS

This study was a randomized factorial trial using patient-based vignettes. The vignettes were deemed archetypical for intensive care unit admission or refusal, as judged by a group of experts. Intensive care unit physicians were randomized to 1) an increased distraction (intervention) or a control group, 2) an intensive care unit bed scarcity or nonscarcity (availability) setting, and 3) a multiple-choice or omission (status quo) vignette scenario. The primary outcome was the proportion of appropriate intensive care unit allocations, defined as concordance with the allocation decision made by the group of experts.

RESULTS

We analyzed 125 physicians. Overall, distractors had no impact on the outcome; however, there was a differential drop-out rate, with fewer physicians in the intervention arm completing the questionnaire. Intensive care unit bed availability was associated with an inappropriate allocation of vignettes deemed inappropriate for intensive care unit admission (OR = 2.47; 95%CI 1.19 - 5.11) but not of vignettes appropriate for intensive care unit admission. There was a significant interaction with the presence of distractors (p = 0.007), with intensive care unit bed availability being associated with increased intensive care unit admission of vignettes inappropriate for intensive care unit admission in the distractor (intervention) arm (OR = 9.82; 95%CI 2.68 - 25.93) but not in the control group (OR = 1.02; 95%CI 0.38 - 2.72). Multiple choices were associated with increased inappropriate allocation in comparison to the omission group (OR = 5.18; 95%CI 1.37 - 19.61).

CONCLUSION

Intensive care unit bed availability and cognitive biases were associated with inappropriate intensive care unit allocation decisions. These findings may have implications for intensive care unit admission policies.

Contamination: How much can an individually randomized trial tolerate?

Cluster randomization results in an increase in sample size compared to individual randomization, referred to as an efficiency loss. This efficiency loss is typically presented under an assumption of no contamination in the individually randomized trial. An alternative comparator is the sample size needed under individual randomization to detect the attenuated treatment effect due to contamination. A general framework is provided for determining the extent of contamination that can be tolerated in an individually randomized trial before a cluster randomized design yields a larger sample size. Results are presented for a variety of cluster trial designs including parallel arm, stepped-wedge and cluster crossover trials. Results reinforce what is expected: individually randomized trials can tolerate a surprisingly large amount of contamination before they become less efficient than cluster designs. We determine the point at which the contamination means an individual randomized design to detect an attenuated effect requires a larger sample size than cluster randomization under a nonattenuated effect. This critical rate is a simple function of the design effect for clustering and the design effect for multiple periods as well as design effects for stratification or repeated measures under individual randomization. These findings are important for pragmatic comparisons between a novel treatment and usual care as any bias due to contamination will only attenuate the true treatment effect. This is a bias that operates in a predictable direction. Yet, cluster randomized designs with post-randomization recruitment without blinding, are at high risk of bias due to the differential recruitment across treatment arms. This sort of bias operates in an unpredictable direction. Thus, with knowledge that cluster randomized trials are generally at a greater risk of biases that can operate in a nonpredictable direction, results presented here suggest that even in situations where there is a risk of contamination, individual randomization might still be the design of choice.

The PrEPARE Pretoria Project: protocol for a cluster-randomized factorial-design trial to prevent HIV with PrEP among adolescent girls and young women in Tshwane, South Africa

BACKGROUND

Despite increased prevention efforts, HIV remains the leading cause of death among adolescent girls and young women in South Africa. Although research indicates important determinants of HIV acquisition at the individual and interpersonal levels, structural-level stigma and discrimination continue to be critical barriers to reaching and retaining this key population for HIV prevention and sexual and reproductive health services. Innovative and multilevel interventions are needed that can address the intersectional structural and gender issues that young women face, including stigma, alcohol and drug use, gender-based violence, and other risk factors when seeking health services. Oral pre-exposure prophylaxis (PrEP) taken daily has been found to be an effective biomedical HIV prevention tool. Testing a comprehensive gender-focused biobehavioral HIV prevention intervention that is inclusive of social ecological determinants, such as stigma and discrimination reduction in clinics, is critical for reducing HIV among adolescent girls and young women.

METHODS

This project involves both a Community Collaborative Board and a Youth Advisory Board in helping to adapt the Young Women's Health CoOp intervention and the Health Policy Project (HPP) Stigma and Discrimination (S&D) reduction training curriculum to the setting and population. This study uses a two-by-two factorial design with stratified randomization of 12 clinics, each with distinct catchment areas. The Young Women's Health CoOp addresses substance use, sexual risk, violence prevention and sexual negotiation, condom demonstration, and problem solving with the following additions: knowledge of PrEP, the importance of PrEP adherence, and sexual and reproductive health. Adolescent girls and young women will be assessed with behavioral and biological measures at baseline, 3-, 6- and 9-month follow-up. The S&D reduction training is provided for all staff in the clinics randomized to this condition. Clinic staff will be surveyed at baseline, 4- and 8-month follow-up. We will recruit 900 AGYW from communities in the 12 clinic catchment areas.

DISCUSSION

The study findings, if efficacious across the outcomes, will be incorporated into the gender-focused HIV prevention intervention toolkit and disseminated to inform multilevel prevention approaches.

TRIAL REGISTRATION

ClinicalTrials.gov. Identifier: NCT04048551 (Recruiting). Registered: August 7, 2019 (Retrospectively registered).

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Reducing sexual risk and promoting acceptance of men who have sex with men living with HIV in India: Outcomes and process evaluation of a pilot randomised multi-level intervention

HIV-positive men who have sex with men (HIV+MSM) in India need culturally-relevant interventions to promote safer sex. We tested a multi-level intervention among HIV+MSM that targeted individual, interpersonal, and community factors, based on the Social-Personal and Social Ecological Models. We conducted a 2 × 2 factorial RCT with 119 HIV+MSM randomised to receive either an individual-level intervention (ILI) using motivational interviewing to promote safer sex, a community-level intervention (CLI) to strengthen community norms toward safer sex and reduce stigma among MSM communities, a multi-level intervention combining the individual- and community-level interventions (ILI + CLI), or standard-of-care control. Participants completed pre- and post-intervention assessments of a composite sexual risk score and a process evaluation to assess fidelity and satisfaction. Out of the 119 HIV+MSM, 106 (89.0%) completed pre- and post-intervention assessments. Generalised Estimating Equation models showed that both CLI (Incidence Rate Ratio [IRR] = .67, 95% CI .47 to .96) and ILI + CLI (IRR = .66, 95% CI .48 to .91) groups had a statistically significant decrease in sexual risk compared to the standard-of-care. The interventions had high levels of fidelity and satisfaction. This pilot RCT demonstrated feasibility and potential effectiveness of a multi-level intervention that addresses individual, interpersonal and community-level contributors of sexual risk among HIV+MSM.

Acupuncture and clomiphene for Chinese women with polycystic ovary syndrome (PCOSAct): statistical analysis approach with the revision and explanation

BACKGROUND

Polycystic ovary syndrome (PCOS) is the most common endocrinopathy of reproductive-aged women. Clomiphene is regarded as the first-line medical treatment for ovulation induction in PCOS patients and acupuncture is often used as an alternative and complementary treatment for fertility issues such as those associated with PCOS. The efficacy of acupuncture alone or combined with clomiphene still lacks strong supporting evidence. Factorial 2 × 2 designs can be used for the evaluations of two treatments within a single study, to test the main effects of acupuncture and clomiphene and their interactions.

METHODS

PCOSAct was designed to test the effect of clomiphene and acupuncture by three two-group comparisons in the original protocol. However, the trial was designed as a standard factorial trial and the factorial analysis approach for analyzing the data that were actually obtained during the trial was found to be more appropriate and more powerful than the three two-group comparisons described in the original protocol, so the statistical analysis approach and different datasets of PCOSAct in the primary publication were accordingly changed.

DISCUSSION

Although the statistical analysis approach used in the primary publication deviated from the statistical analysis planned in the study protocol, focusing on the main effects of the two interventions and their interactions was a more standard approach to a factorial trial and proved to be more suitable and consistent with the characteristics of the trial data. Statistically, the revision is more powerful and precise and should be more useful to the journal and the readers.

TRIAL REGISTRATION

Chinese clinical trial registry, ChiCTR-TRC-12002081 . Registered on 20 March 2012. Clinicaltrials.gov, NCT01573858 . Registered on 4 April 2012.

The split-plot design was useful for evaluating complex, multilevel interventions, but there is need for improvement in its design and report

OBJECTIVES

To describe the sample size calculation, analysis and reporting of split-plot (S-P) randomized controlled trials in health care (trials that use two units of randomization: one at a cluster-level and one at a level lower than the cluster).

STUDY DESIGN AND SETTING

We carried out a comprehensive search in the EMBASE database from 1946 to 2016. Health care trials with a S-P design in human subjects were included. Three authors screened and assessed the studies, and the data were extracted on methodology and reporting standards based on CONSORT.

RESULTS

Eighteen S-P studies were included, with authors using nine different designations to describe them. Units of randomization were unclear in nine abstracts. Explicit rationale for choosing the design was not given. Ten studies presented a sample size calculation accounting for clustering; the analyses were coherent with that. Flow of participant diagrams was presented but was incomplete in 14 articles.

CONCLUSION

S-P designs can be useful complex designs but challenging to report. Researchers need to clearly describe the rationale, sample size calculation, and participant flow. We provide a suggested CONSORT style participant flow diagram to aid reporting. There is need for more research regarding sample size calculation for S-P.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 1-Design

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have highlighted the developments of the past 13 years in design with a companion article to focus on developments in analysis. As a pair, these articles update the 2004 review. We have discussed developments in the topics of the earlier review (e.g., clustering, matching, and individually randomized group-treatment trials) and in new topics, including constrained randomization and a range of randomized designs that are alternatives to the standard parallel-arm GRT. These include the stepped-wedge GRT, the pseudocluster randomized trial, and the network-randomized GRT, which, like the parallel-arm GRT, require clustering to be accounted for in both their design and analysis.

Improved Designs for Cluster Randomized Trials

Studies in which clusters of individuals are randomized to conditions are increasingly common in public health research. However, the designs utilized for such studies are often suboptimal and inefficient. We review strategies to improve the design of cluster randomized trials. We discuss both older but effective design concepts that are underutilized, such as stratification and factorial designs, as well as emergent ideas including fractional factorial designs and cluster randomized crossover studies. We draw examples from the recent literature and provide resources for sample size and power planning. Given the inherent inefficiencies of cluster randomized trials, these design strategies merit wider consideration and can lead to studies that are more cost-effective and potentially more rigorous than traditional approaches.