Estimands in cluster-randomized trials: choosing analyses that answer the right question

Statistical analysis plan for the NU IMPACT stepped-wedge cluster randomized trial

BACKGROUND

As part of the IMPACT Consortium of three effectiveness-implementation trials, the NU IMPACT trial was designed to evaluate implementation and effectiveness outcomes for an electronic health record (EHR)-embedded symptom monitoring and management program for outpatient cancer care. NU IMPACT uses a unique stepped-wedge cluster randomized design, involving six clusters of 26 clinics, for evaluation of implementation outcomes with an embedded patient-level randomized trial to evaluate effectiveness outcomes. Collaborative, consortium-wide efforts to ensure use of the most robust and recent analytic methodologies for stepped-wedge trials motivated updates to the statistical analysis plan for implementation outcomes in the NU IMPACT trial.

METHODS

In the updated statistical analysis plan for NU IMPACT, the primary implementation outcome patient adoption, as measured by clinic-level monthly proportions of patient engagement with the EHR-based cancer symptom monitoring system, will be analyzed using generalized least squares linear regression with auto-regressive errors and adjustment for cluster and time effects (underlying secular trends). A similar strategy will be used for secondary patient and provider implementation outcomes.

DISCUSSION

The analytic updates described here resulted from highly iterative, collaborative efforts among statisticians, implementation scientists, and trial leads in the IMPACT Consortium. This updated statistical analysis plan will serve as the a priori specified approach for analyzing implementation outcomes for the NU IMPACT trial.

Robust analysis of stepped wedge trials using composite likelihood models

Stepped wedge trials (SWTs) are a type of cluster randomized trial that involve repeated measures on clusters and design-induced confounding between time and treatment. Although mixed models are commonly used to analyze SWTs, they are susceptible to misspecification particularly for cluster-longitudinal designs such as SWTs. Mixed model estimation leverages both "horizontal" or within-cluster information and "vertical" or between-cluster information. To use horizontal information in a mixed model, both the mean model and correlation structure must be correctly specified or accounted for, since time is confounded with treatment and measurements are likely correlated within clusters. Alternative non-parametric methods have been proposed that use only vertical information; these are more robust because between-cluster comparisons in a SWT preserve randomization, but these non-parametric methods are not very efficient. We propose a composite likelihood method that focuses on vertical information, but has the flexibility to recover efficiency by using additional horizontal information. We compare the properties and performance of various methods, using simulations based on COVID-19 data and a demonstration of application to the LIRE trial. We found that a vertical composite likelihood model that leverages baseline data is more robust than traditional methods, and more efficient than methods that use only vertical information. We hope that these results demonstrate the potential value of model-based vertical methods for SWTs with a large number of clusters, and that these new tools are useful to researchers who are concerned about misspecification of traditional models.

Multilevel Intervention Stepped Wedge Designs (MLI-SWDs)

Multilevel interventions (MLIs) hold promise for reducing health inequities by intervening at multiple types of social determinants of health consistent with the socioecological model of health. In spite of their potential, methodological challenges related to study design compounded by a lack of tools for sample size calculation inhibit their development. We help address this gap by proposing the Multilevel Intervention Stepped Wedge Design (MLI-SWD), a hybrid experimental design which combines cluster-level (CL) randomization using a Stepped Wedge design (SWD) with independent individual-level (IL) randomization. The MLI-SWD is suitable for MLIs where the IL intervention has a low risk of interference between individuals in the same cluster, and it enables estimation of the component IL and CL treatment effects, their interaction, and the combined intervention effect. The MLI-SWD accommodates cross-sectional and cohort designs as well as both incomplete (clusters are not observed in every study period) and complete observation patterns. We adapt recent work using generalized estimating equations for SWD sample size calculation to the multilevel setting and provide an R package for power and sample size calculation. Furthermore, motivated by our experiences with the ongoing NC Works 4 Health study, we consider how to apply the MLI-SWD when individuals join clusters over the course of the study. This situation arises when unemployment MLIs include IL interventions that are delivered while the individual is unemployed. This extension requires carefully considering whether the study interventions will satisfy additional causal assumptions but could permit randomization in new settings.

Demystifying estimands in cluster-randomised trials

Estimands can help clarify the interpretation of treatment effects and ensure that estimators are aligned with the study's objectives. Cluster-randomised trials require additional attributes to be defined within the estimand compared to individually randomised trials, including whether treatment effects are marginal or cluster-specific, and whether they are participant- or cluster-average. In this paper, we provide formal definitions of estimands encompassing both these attributes using potential outcomes notation and describe differences between them. We then provide an overview of estimators for each estimand, describe their assumptions, and show consistency (i.e. asymptotically unbiased estimation) for a series of analyses based on cluster-level summaries. Then, through a re-analysis of a published cluster-randomised trial, we demonstrate that the choice of both estimand and estimator can affect interpretation. For instance, the estimated odds ratio ranged from 1.38 (p = 0.17) to 1.83 (p = 0.03) depending on the target estimand, and for some estimands, the choice of estimator affected the conclusions by leading to smaller treatment effect estimates. We conclude that careful specification of the estimand, along with an appropriate choice of estimator, is essential to ensuring that cluster-randomised trials address the right question.

Bayesian semi-parametric inference for clustered recurrent events with zero inflation and a terminal event

Recurrent events are common in clinical studies and are often subject to terminal events. In pragmatic trials, participants are often nested in clinics and can be susceptible or structurally unsusceptible to the recurrent events. We develop a Bayesian shared random effects model to accommodate this complex data structure. To achieve robustness, we consider the Dirichlet processes to model the residual of the accelerated failure time model for the survival process as well as the cluster-specific shared frailty distribution, along with an efficient sampling algorithm for posterior inference. Our method is applied to a recent cluster randomized trial on fall injury prevention.

The Ottawa Statement implementation guidance document for cluster randomized trials in the hemodialysis setting

Research teams are increasingly interested in using cluster randomized trial (CRT) designs to generate practice-guiding evidence for in-center maintenance hemodialysis. However, CRTs raise complex ethical issues. The Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials, published in 2012, provides 15 recommendations to address ethical issues arising within 7 domains: justifying the CRT design, research ethics committee review, identifying research participants, obtaining informed consent, gatekeepers, assessing benefits and harms, and protecting vulnerable participants. But applying the Ottawa Statement recommendations to CRTs in the hemodialysis setting is complicated by the unique features of the setting and population. Here, with the help of content experts and patient partners, we co-developed this implementation guidance document to provide research teams, research ethics committees, and other stakeholders with detailed guidance on how to apply the Ottawa Statement recommendations to CRTs in the hemodialysis setting, the result of a 4-year research project. Thus, our work demonstrates how the voices of patients, caregivers, and all stakeholders may be included in the development of research ethics guidance.

Assessing treatment effect heterogeneity in the presence of missing effect modifier data in cluster-randomized trials

Understanding whether and how treatment effects vary across subgroups is crucial to inform clinical practice and recommendations. Accordingly, the assessment of heterogeneous treatment effects based on pre-specified potential effect modifiers has become a common goal in modern randomized trials. However, when one or more potential effect modifiers are missing, complete-case analysis may lead to bias and under-coverage. While statistical methods for handling missing data have been proposed and compared for individually randomized trials with missing effect modifier data, few guidelines exist for the cluster-randomized setting, where intracluster correlations in the effect modifiers, outcomes, or even missingness mechanisms may introduce further threats to accurate assessment of heterogeneous treatment effect. In this article, the performance of several missing data methods are compared through a simulation study of cluster-randomized trials with continuous outcome and missing binary effect modifier data, and further illustrated using real data from the Work, Family, and Health Study. Our results suggest that multilevel multiple imputation and Bayesian multilevel multiple imputation have better performance than other available methods, and that Bayesian multilevel multiple imputation has lower bias and closer to nominal coverage than standard multilevel multiple imputation when there are model specification or compatibility issues.

Current developments of the estimand concept

Since the introduction of the estimand in therapeutical studies, several adaptions have been developed. This short article highlights the important aspects of the estimand concept. A literature research was conducted to identify different extensions to this framework. Different modified strategies for intercurrent events are presented, as well as examples of methods to implement the estimand in clinical studies. The article reflects that the estimand is an ongoing research field with further exploration.

Equity-Centered Postdischarge Support for Medicaid-Insured People: Protocol for a Type 1 Hybrid Effectiveness-Implementation Stepped Wedge Cluster Randomized Controlled Trial

BACKGROUND

Disparities in posthospitalization outcomes for people with chronic medical conditions and insured by Medicaid are well documented, yet interventions that mitigate them are lacking. Prevailing transitional care interventions narrowly target people aged 65 years and older, with specific disease processes, or limitedly focus on individual-level behavioral change such as self-care or symptom management, thus failing to adequately provide a holistic approach to ensure an optimal posthospital care continuum. This study evaluates the implementation of THRIVE-an evidence-based, equity-focused clinical pathway that supports Medicaid-insured individuals with multiple chronic conditions transitioning from hospital to home by focusing on the social determinants of health and systemic and structural barriers in health care delivery. THRIVE services include coordinating care, standardizing interdisciplinary communication, and addressing unmet clinical and social needs following hospital discharge.

OBJECTIVE

The study's objectives are to (1) examine referral patterns, 30-day readmission, and emergency department use for participants who receive THRIVE support services compared to those receiving usual care and (2) evaluate the implementation of the THRIVE clinical pathway, including fidelity, feasibility, appropriateness, and acceptability.

METHODS

We will perform a sequential randomized rollout of THRIVE to case managers at the study hospital in 3 steps (4 in the first group, 4 in the second, and 5 in the third), and data collection will occur over 18 months. Inclusion criteria for THRIVE participation include (1) being Medicaid insured, dually enrolled in Medicaid and Medicare, or Medicaid eligible; (2) residing in Philadelphia; (3) having experienced a hospitalization at the study hospital for more than 24 hours with a planned discharge to home; (4) agreeing to home care at partner home care settings; and (5) being aged 18 years or older. Qualitative data will include interviews with clinicians involved in THRIVE, and quantitative data on health service use (ie, 30-day readmission, emergency department use, and primary and specialty care) will be derived from the electronic health record.

RESULTS

This project was funded in January 2023 and approved by the institutional review board on March 10, 2023. Data collection will occur from March 2023 to July 2024. Results are expected to be published in 2025.

CONCLUSIONS

The THRIVE clinical pathway aims to reduce disparities and improve postdischarge care transitions for Medicaid-insured patients through a system-level intervention that is acceptable for THRIVE participants, clinicians, and their teams in hospitals and home care settings. By using our equity-focused case management services and leveraging the power of the electronic medical record, THRIVE creates efficiencies by identifying high-need patients, improving communication across acute and community-based sectors, and driving evidence-based care coordination. This study will add important findings about how the infusion of equity-focused principles in the design and evaluation of evidence-based interventions contributes to both implementation and effectiveness outcomes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/54211.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05714605; https://clinicaltrials.gov/ct2/show/NCT05714605.

Household Air Pollution and Child Lung Function: The Ghana Randomized Air Pollution and Health Study

Rationale: The impact of a household air pollution (HAP) stove intervention on child lung function has been poorly described. Objectives: To assess the effect of a HAP stove intervention for infants prenatally to age 1 on, and exposure-response associations with, lung function at child age 4. Methods: The Ghana Randomized Air Pollution and Health Study randomized pregnant women to liquefied petroleum gas (LPG), improved biomass, or open-fire (control) stove conditions through child age 1. We quantified HAP exposure by repeated maternal and child personal carbon monoxide (CO) exposure measurements. Children performed oscillometry, an effort-independent lung function measurement, at age 4. We examined associations between Ghana Randomized Air Pollution and Health Study stove assignment and prenatal and infant CO measurements and oscillometry using generalized linear regression models. We used reverse distributed lag models to examine time-varying associations between prenatal CO and oscillometry. Measurements and Main Results: The primary oscillometry measure was reactance at 5 Hz, X5, a measure of elastic and inertial lung properties. Secondary measures included total, large airway, and small airway resistance at 5 Hz, 20 Hz, and the difference in resistance at 5 Hz and 20 Hz (R5, R20, and R5-20, respectively); area of reactance (AX); and resonant frequency. Of the 683 children who attended the lung function visit, 567 (83%) performed acceptable oscillometry. A total of 221, 106, and 240 children were from the LPG, improved biomass, and control arms, respectively. Compared with control, the improved biomass stove condition was associated with lower reactance at 5 Hz (X5 z-score: β = -0.25; 95% confidence interval [CI] = -0.39, -0.11), higher large airway resistance (R20 z-score: β = 0.34; 95% CI = 0.23, 0.44), and higher AX (AX z-score: β = 0.16; 95% CI = 0.06, 0.26), which is suggestive of overall worse lung function. The LPG stove condition was associated with higher X5 (X5 score: β = 0.16; 95% CI = 0.01, 0.31) and lower small airway resistance (R5-20 z-score: β = -0.15; 95% CI = -0.30, 0.0), which is suggestive of better small airway function. Higher average prenatal CO exposure was associated with higher R5 and R20, and distributed lag models identified sensitive windows of exposure between CO and X5, R5, R20, and R5-20. Conclusions: These data support the importance of prenatal HAP exposure on child lung function. Clinical trial registered with www.clinicaltrials.gov (NCT01335490).

Peer Comparison or Guideline-Based Feedback and Postsurgery Opioid Prescriptions: A Randomized Clinical Trial

Importance

Excess opioid prescribing after surgery can result in prolonged use and diversion. Email feedback based on social norms may reduce the number of pills prescribed.

Objective

To assess the effectiveness of 2 social norm-based interventions on reducing guideline-discordant opioid prescribing after surgery.

Design, Setting, and Participants

This cluster randomized clinical trial conducted at a large health care delivery system in northern California between October 2021 and October 2022 included general, obstetric/gynecologic, and orthopedic surgeons with patients aged 18 years or older discharged to home with an oral opioid prescription.

Interventions

In 19 hospitals, 3 surgical specialties (general, orthopedic, and obstetric/gynecologic) were randomly assigned to a control group or 1 of 2 interventions. The guidelines intervention provided email feedback to surgeons on opioid prescribing relative to institutionally endorsed guidelines; the peer comparison intervention provided email feedback on opioid prescribing relative to that of peer surgeons. Emails were sent to surgeons with at least 2 guideline-discordant prescriptions in the previous month. The control group had no intervention.

Main Outcome and Measures

The probability that a discharged patient was prescribed a quantity of opioids above the guideline for the respective procedure during the 12 intervention months.

Results

There were 38 235 patients discharged from 640 surgeons during the 12-month intervention period. Control-group surgeons prescribed above guidelines 36.8% of the time during the intervention period compared with 27.5% and 25.4% among surgeons in the peer comparison and guidelines arms, respectively. In adjusted models, the peer comparison intervention reduced guideline-discordant prescribing by 5.8 percentage points (95% CI, -10.5 to -1.1; P = .03) and the guidelines intervention reduced it by 4.7 percentage points (95% CI, -9.4 to -0.1; P = .05). Effects were driven by surgeons who performed more surgeries and had more guideline-discordant prescribing at baseline. There was no significant difference between interventions.

Conclusions and Relevance

In this cluster randomized clinical trial, email feedback based on either guidelines or peer comparison reduced opioid prescribing after surgery. Guideline-based feedback was as effective as peer comparison-based feedback. These interventions are simple, low-cost, and scalable, and may reduce downstream opioid misuse.

Trial Registration

ClinicalTrials.gov NCT05070338.

Designing individually randomized group treatment trials with repeated outcome measurements using generalized estimating equations

Individually randomized group treatment (IRGT) trials, in which the clustering of outcome is induced by group-based treatment delivery, are increasingly popular in public health research. IRGT trials frequently incorporate longitudinal measurements, of which the proper sample size calculations should account for correlation structures reflecting both the treatment-induced clustering and repeated outcome measurements. Given the relatively sparse literature on designing longitudinal IRGT trials, we propose sample size procedures for continuous and binary outcomes based on the generalized estimating equations approach, employing the block exchangeable correlation structures with different correlation parameters for the treatment arm and for the control arm, and surveying five marginal mean models with different assumptions of time effect: no-time constant treatment effect, linear-time constant treatment effect, categorical-time constant treatment effect, linear time by treatment interaction, and categorical time by treatment interaction. Closed-form sample size formulas are derived for continuous outcomes, which depends on the eigenvalues of the correlation matrices; detailed numerical sample size procedures are proposed for binary outcomes. Through simulations, we demonstrate that the empirical power agrees well with the predicted power, for as few as eight groups formed in the treatment arm, when data are analyzed using the matrix-adjusted estimating equations for the correlation parameters with a bias-corrected sandwich variance estimator.

Statistical analysis plan for a cluster randomised controlled trial to compare screening, feedback and intervention for child anxiety problems to usual school practice: identifying Child Anxiety Through Schools-identification to intervention (iCATS-i2i)

BACKGROUND

The Identifying Child Anxiety Through Schools-identification to intervention (iCATS-i2i) trial is being conducted to establish whether 'screening and intervention', consisting of usual school practice plus a pathway comprising screening, feedback and a brief parent-led online intervention (OSI: Online Support and Intervention for child anxiety), bring clinical and health economic benefits compared to usual school practice and assessment only - 'usual school practice', for children aged 8-9 years in the following: (1) the 'target population', who initially screen positive for anxiety problems according to a two-item parent-report child anxiety questionnaire - iCATS-2, and (2) the 'total population', comprising all children in participating classes. This article describes the detailed statistical analysis plan for the trial.

METHODS AND DESIGN

iCATS-i2i is a definitive, superiority, pragmatic, school-based cluster randomised controlled trial (with internal pilot), with two parallel groups. Schools are randomised 1:1 to receive either screening and intervention or usual school practice. This article describes the following: trial objectives and outcomes; statistical analysis principles, including detailed estimand information necessary for aligning trial objectives, conduct, analyses and interpretation when there are different analysis populations and outcome measures to be considered; and planned main analyses, sensitivity and additional analyses.

TRIAL REGISTRATION

ClinicalTrials.gov ISRCTN76119074. Registered on 4 January 2022.

A mixed model approach to estimate the survivor average causal effect in cluster-randomized trials

In many medical studies, the outcome measure (such as quality of life, QOL) for some study participants becomes informatively truncated (censored, missing, or unobserved) due to death or other forms of dropout, creating a nonignorable missing data problem. In such cases, the use of a composite outcome or imputation methods that fill in unmeasurable QOL values for those who died rely on strong and untestable assumptions and may be conceptually unappealing to certain stakeholders when estimating a treatment effect. The survivor average causal effect (SACE) is an alternative causal estimand that surmounts some of these issues. While principal stratification has been applied to estimate the SACE in individually randomized trials, methods for estimating the SACE in cluster-randomized trials are currently limited. To address this gap, we develop a mixed model approach along with an expectation-maximization algorithm to estimate the SACE in cluster-randomized trials. We model the continuous outcome measure with a random intercept to account for intracluster correlations due to cluster-level randomization, and model the principal strata membership both with and without a random intercept. In simulations, we compare the performance of our approaches with an existing fixed-effects approach to illustrate the importance of accounting for clustering in cluster-randomized trials. The methodology is then illustrated using a cluster-randomized trial of telecare and assistive technology on health-related QOL in the elderly.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Accounting for Competing Events When Evaluating Long-Term Outcomes in Survivors of Critical Illness

The clinical trajectory of survivors of critical illness after hospital discharge can be complex and highly unpredictable. Assessing long-term outcomes after critical illness can be challenging because of possible competing events, such as all-cause death during follow-up (which precludes the occurrence of an event of particular interest). In this perspective, we explore challenges and methodological implications of competing events during the assessment of long-term outcomes in survivors of critical illness. In the absence of competing events, researchers evaluating long-term outcomes commonly use the Kaplan-Meier method and the Cox proportional hazards model to analyze time-to-event (survival) data. However, traditional analytical and modeling techniques can yield biased estimates in the presence of competing events. We present different estimands of interest and the use of different analytical approaches, including changes to the outcome of interest, Fine and Gray regression models, cause-specific Cox proportional hazards models, and generalized methods (such as inverse probability weighting). Finally, we provide code and a simulated dataset to exemplify the application of the different analytical strategies in addition to overall reporting recommendations.

Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial

BACKGROUND

Recent work has shown that cluster-randomised trials can estimate two distinct estimands: the participant-average and cluster-average treatment effects. These can differ when participant outcomes or the treatment effect depends on the cluster size (termed informative cluster size). In this case, estimators that target one estimand (such as the analysis of unweighted cluster-level summaries, which targets the cluster-average effect) may be biased for the other. Furthermore, commonly used estimators such as mixed-effects models or generalised estimating equations with an exchangeable correlation structure can be biased for both estimands. However, there has been little empirical research into whether informative cluster size is likely to occur in practice.

METHOD

We re-analysed a cluster-randomised trial comparing two different thresholds for red blood cell transfusion in patients with acute upper gastrointestinal bleeding to explore whether estimates for the participant- and cluster-average effects differed, to provide empirical evidence for whether informative cluster size may be present. For each outcome, we first estimated a participant-average effect using independence estimating equations, which are unbiased under informative cluster size. We then compared this to two further methods: (1) a cluster-average effect estimated using either weighted independence estimating equations or unweighted cluster-level summaries, and (2) estimates from a mixed-effects model or generalised estimating equations with an exchangeable correlation structure. We then performed a small simulation study to evaluate whether observed differences between cluster- and participant-average estimates were likely to occur even if no informative cluster size was present.

RESULTS

For most outcomes, treatment effect estimates from different methods were similar. However, differences of >10% occurred between participant- and cluster-average estimates for 5 of 17 outcomes (29%). We also observed several notable differences between estimates from mixed-effects models or generalised estimating equations with an exchangeable correlation structure and those based on independence estimating equations. For example, for the EQ-5D VAS score, the independence estimating equation estimate of the participant-average difference was 4.15 (95% confidence interval: -3.37 to 11.66), compared with 2.84 (95% confidence interval: -7.37 to 13.04) for the cluster-average independence estimating equation estimate, and 3.23 (95% confidence interval: -6.70 to 13.16) from a mixed-effects model. Similarly, for thromboembolic/ischaemic events, the independence estimating equation estimate for the participant-average odds ratio was 0.43 (95% confidence interval: 0.07 to 2.48), compared with 0.33 (95% confidence interval: 0.06 to 1.77) from the cluster-average estimator.

CONCLUSION

In this re-analysis, we found that estimates from the various approaches could differ, which may be due to the presence of informative cluster size. Careful consideration of the estimand and the plausibility of assumptions underpinning each estimator can help ensure an appropriate analysis methods are used. Independence estimating equations and the analysis of cluster-level summaries (with appropriate weighting for each to correspond to either the participant-average or cluster-average treatment effect) are a desirable choice when informative cluster size is deemed possible, due to their unbiasedness in this setting.

A Hospital-Wide Open-Label Cluster Crossover Pragmatic Comparative Effectiveness Randomized Trial Comparing Normal Saline to Ringer's Lactate: Protocol and Statistical Analysis Plan of The FLUID Trial

BACKGROUND

Normal saline (NS) and Ringer's lactate (RL) are the most common crystalloids given to hospitalized patients. Despite concern about possible harm associated with NS (eg, hyperchloremic metabolic acidosis, impaired kidney function, and death), few large multicenter randomized trials focused on critically ill patients have compared these fluids. Uncertainty exists about the effects of these fluids on clinically important outcomes across all hospitalized patients.

OBJECTIVE

The FLUID trial is a pragmatic, multicenter, 2×2 cluster crossover comparative effectiveness randomized trial that aims to evaluate the effectiveness of a hospital-wide policy that stocks either NS or RL as the main crystalloid fluid in 16 hospitals across Ontario, Canada.

METHODS

All hospitalized adult and pediatric patients (anticipated sample size 144,000 patients) with an incident admission to the hospital over the course of each study period will be included. Either NS or RL will be preferentially stocked throughout the hospital for 12 weeks before crossing to the alternate fluid for the subsequent 12 weeks. The primary outcome is a composite of death and hospital readmission within 90 days of hospitalization. Secondary outcomes include death, hospital readmission, dialysis, reoperation, postoperative reintubation, length of hospital stay, emergency department visits, and discharge to a facility other than home. All outcomes will be obtained from health administrative data, eliminating the need for individual case reports. The primary analysis will use cluster-level summaries to estimate cluster-average treatment effects.

RESULTS

The statistical analysis plan has been prepared "a priori" in advance of receipt of the trial data set from ICES and any analyses.

CONCLUSIONS

We describe the protocol and statistical analysis plan for the evaluation of primary and secondary outcomes for the FLUID trial.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04512950; https://classic.clinicaltrials.gov/ct2/show/NCT04512950.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/51783.

Key considerations for designing, conducting and analysing a cluster randomized trial

Not only do cluster randomized trials require a larger sample size than individually randomized trials, they also face many additional complexities. The potential for contamination is the most commonly used justification for using cluster randomization, but the risk of contamination should be carefully weighed against the more serious problem of questionable scientific validity in settings with post-randomization identification or recruitment of participants unblinded to the treatment allocation. In this paper we provide some simple guidelines to help researchers conduct cluster trials in a way that minimizes potential biases and maximizes statistical efficiency. The overarching theme of this guidance is that methods that apply to individually randomized trials rarely apply to cluster randomized trials. We recommend that cluster randomization be only used when necessary-balancing the benefits of cluster randomization with its increased risks of bias and increased sample size. Researchers should also randomize at the lowest possible level-balancing the risks of contamination with ensuring an adequate number of randomization units-as well as exploring other options for statistically efficient designs. Clustering should always be allowed for in the sample size calculation; and the use of restricted randomization (and adjustment in the analysis for covariates used in the randomization) should be considered. Where possible, participants should be recruited before randomizing clusters and, when recruiting (or identifying) participants post-randomization, recruiters should be masked to the allocation. In the analysis, the target of inference should align with the research question, and adjustment for clustering and small sample corrections should be used when the trial includes less than about 40 clusters.

Impact of differentiated service delivery models on 12-month retention in HIV treatment in Mozambique: an interrupted time-series analysis

BACKGROUND

HIV treatment has been available in Mozambique since 2004, but coverage of, and retention in, antiretroviral therapy (ART) remain suboptimal. Therefore, to increase health system efficiency and reduce HIV-associated mortality, in November, 2018, the Ministry of Health launched national guidelines on implementing eight differentiated service delivery models (DSDMs) for HIV treatment. We assessed the effect of this implementation on retention in ART 12 months after initiation, and explored the associated effects of COVID-19.

METHODS

In this uncontrolled interrupted time-series analysis, data were extracted from the Mozambique ART database, which contains data on individuals in ART care from 1455 health facilities providing ART in Mozambique. We included individual-level data from facilities that were providing ART at the beginning of the study period (Jan 1, 2016) and at the start of DSDM implementation (Dec 1, 2018). We compared the proportion of individuals retained in ART 12 months after initiation between the periods before (Jan 1, 2017, to Nov 30, 2018) and after (Dec 1, 2019, to June 30, 2021) implementation of the DSDMs, overall and stratified by sex and age. We applied a generalised estimating equation model with a working independence correlation and cluster-robust standard errors to account for clustering at the facility level. In a secondary analysis, we assessed the effect of COVID-19 response measures during the post-intervention period on ART retention.

FINDINGS

The study included 613 facilities and 1 131 118 individuals who started ART during the inclusion period up to June 30, 2020, of whom 79 178 (7·0%) were children (age ≤14 years), 226 224 (20·0%) were adolescents and young adults (age 15-24 years), and 825 716 (73·0%) were adults (age ≥25 years). 731 623 (64·7%) were female and 399 495 (35·3%) were male. Introduction of the DSDMs was associated with an estimated increase of 24·5 percentage points (95% CI 21·1 to 28·0) in 12-month ART retention by the end of the study period, compared with the counterfactual scenario without DSDM implementation. By age, the smallest effect was estimated in children (6·1 percentage points, 1·3 to 10·9) and the largest effect in adolescents and young adults (28·8 percentage points, 24·2 to 33·4); by sex, a larger effect was estimated in males (29·7 percentage points, 25·6 to 33·7). Our analysis showed that COVID-19 had an overall negative effect on 12-month retention in ART compared with a counterfactual scenario based on the post-intervention period without COVID-19 (-10·0 percentage points, -18·2 to -1·8).

INTERPRETATION

The implementation of eight DSDMs for HIV treatment had a positive impact on 12-month retention in ART. COVID-19 negatively influenced this outcome.

FUNDING

None.

TRANSLATION

For the Portuguese translation of the abstract see Supplementary Materials section.

Decreasing consumption of sugar-sweetened beverages and Raising tap water consumption through Interventions based on Nutrition and sustainability for Kids: study protocol of the "DRINK" cluster randomised controlled trial

BACKGROUND

Effectiveness of actions to reduce sugar-sweetened beverage (SB) consumption in children still needs to be improved. Furthermore, the growing concern about sustainable food systems encourages to develop sustainability-based interventions. The objective of this cluster randomised controlled trial is to evaluate the long-term effectiveness of nutrition- and environmental sustainability-based interventions on the reduction in SB intake and on the increase in tap water consumption in 3rd to 6th grade primary school children (8 to 11 years of age).

METHODS

Forty-eight French-speaking Belgian primary schools (equivalent to around 3500 pupils involved in the evaluation) are randomised using a factorial plan: (i) control, (ii) nutrition-based intervention, (iii) sustainability-based intervention, and (iv) both. The interventions (encouragement of water breaks; provision of posters, leaflets, reusable cups, and glass bottles; website; meetings at school) were undertaken from February 2022 to June 2023. Evaluation includes questionnaires for the children and their parents on various determinants of dietary behaviour, a 4-day diary to collect information on the child's beverage consumption, and audits at schools. The first evaluation was conducted in Spring 2021 before any intervention, with the two post-intervention evaluations being held in 2022 and 2023. The main quantitative judgement criterion will be the change over time in the mean SB consumption (in ml/day) in the intervention groups compared with the control group. Given the context of the research (school), the safety of the intervention, and the content of data collection, a consent was acknowledged as unnecessary by the Ethical Committee of the Faculty of Psychology (ULB; n°073/2021), but children and parents are explicitly informed of their right to refuse to fill in the questionnaires.

DISCUSSION

Multicomponent interventions based on nutrition and on environmental sustainability, alone or mixed, will provide an original and topical insight into health promotion at school around dietary behaviours. The dissemination plan will enable to widely inform stakeholders, school staff, and families, in addition to the scientific community through the usual medium (articles, conferences), about the research findings in 2024-2025.

TRIAL REGISTRATION

ISRCTN Registry ISRCTN99843102. Retrospectively registered on 25 May 2021.

Joint evaluation of placebo and treatment effects in cluster randomized trials by causal inference models

The term placebo effect refers to the psychobiological effect of a patient's knowledge or belief of being treated. A placebo effect is patient-driven, which makes it fundamentally different from the usual treatment effect resulting from external actions. In modern clinical research, the presence of a placebo effect is often treated as a nuisance issue, something to be "adjusted away" in estimating a treatment effect of primary interest. However, from a patient-centered perspective, we believe that a possible placebo produces substantial improvements in patient-centered outcomes. Understanding placebo effects is therefore an important part of patient-centered outcomes research. The available methods for estimating placebo effects are designed for individually randomized trials and are not directly applicable to cluster randomized trials (CRTs). There are several challenges in estimating placebo effects in CRTs. A major challenge is the possible presence of interference within clusters, in the sense that a subject's outcome may depend on the beliefs subjects in the same cluster about treatment assignment (mentality) and therefore possible correlation in outcome and mentality among subjects exists in the same cluster. In this article, we extend the previously developed causal inference framework to also encompass CRTs, using the G-Computation and inverse probability weighting (IPW) approaches. We also develop methodologies and further extend the G-Computation and IPW approaches to handle missingness for jointly evaluating placebo effect and treatment-specific effect, specifically in the context of CRTs. The proposed methods are demonstrated in simulation studies and a cluster randomized trial on effect of fermented dairy drink.

Costs of fall injuries in the STRIDE study: an economic evaluation of healthcare system heterogeneity and heterogeneity of treatment effect

OBJECTIVES

The Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) Study cluster-randomized 86 primary care practices in 10 healthcare systems to a patient-centered multifactorial fall injury prevention intervention or enhanced usual care, enrolling 5451 participants. We estimated total healthcare costs from participant-reported fall injuries receiving medical attention (FIMA) that were averted by the STRIDE intervention and tested for healthcare-system-level heterogeneity and heterogeneity of treatment effect (HTE).

METHODS

Participants were community-dwelling adults age ≥ 70 at increased fall injury risk. We estimated practice-level total costs per person-year of follow-up (PYF), assigning unit costs to FIMA with and without an overnight hospital stay. Using independent variables for treatment arm, healthcare system, and their interaction, we fit a generalized linear model with log link, log follow-up time offset, and Tweedie error distribution.

RESULTS

Unadjusted total costs per PYF were $2,034 (intervention) and $2,289 (control). The adjusted (intervention minus control) cost difference per PYF was -$167 (95% confidence interval (CI), -$491, $216). Cost heterogeneity by healthcare system was present (p = 0.035), as well as HTE (p = 0.090). Adjusted total costs per PYF in control practices varied from $1,529 to $3,684 for individual healthcare systems; one system with mean intervention minus control costs of -$2092 (95% CI, -$3,686 to -$944) per PYF accounted for HTE, but not healthcare system cost heterogeneity.

CONCLUSIONS

We observed substantial heterogeneity of healthcare system costs in the STRIDE study, with small reductions in healthcare costs for FIMA in the STRIDE intervention accounted for by a single healthcare system.

TRIAL REGISTRATION

Clinicaltrials.gov (NCT02475850).

Evaluating the effectiveness of a multifaceted intervention to reduce low-value care in adults hospitalized following trauma: a protocol for a pragmatic cluster randomized controlled trial

BACKGROUND

While simple Audit & Feedback (A&F) has shown modest effectiveness in reducing low-value care, there is a knowledge gap on the effectiveness of multifaceted interventions to support de-implementation efforts. Given the need to make rapid decisions in a context of multiple diagnostic and therapeutic options, trauma is a high-risk setting for low-value care. Furthermore, trauma systems are a favorable setting for de-implementation interventions as they have quality improvement teams with medical leadership, routinely collected clinical data, and performance-linked to accreditation. We aim to evaluate the effectiveness of a multifaceted intervention for reducing low-value clinical practices in acute adult trauma care.

METHODS

We will conduct a pragmatic cluster randomized controlled trial (cRCT) embedded in a Canadian provincial quality assurance program. Level I-III trauma centers (n = 30) will be randomized (1:1) to receive simple A&F (control) or a multifaceted intervention (intervention). The intervention, developed using extensive background work and UK Medical Research Council guidelines, includes an A&F report, educational meetings, and facilitation visits. The primary outcome will be the use of low-value initial diagnostic imaging, assessed at the patient level using routinely collected trauma registry data. Secondary outcomes will be low-value specialist consultation, low-value repeat imaging after a patient transfer, unintended consequences, determinants for successful implementation, and incremental cost-effectiveness ratios.

DISCUSSION

On completion of the cRCT, if the intervention is effective and cost-effective, the multifaceted intervention will be integrated into trauma systems across Canada. Medium and long-term benefits may include a reduction in adverse events for patients and an increase in resource availability. The proposed intervention targets a problem identified by stakeholders, is based on extensive background work, was developed using a partnership approach, is low-cost, and is linked to accreditation. There will be no attrition, identification, or recruitment bias as the intervention is mandatory in line with trauma center designation requirements, and all outcomes will be assessed with routinely collected data. However, investigators cannot be blinded to group allocation and there is a possibility of contamination bias that will be minimized by conducting intervention refinement only with participants in the intervention arm.

TRIAL REGISTRATION

This protocol has been registered on ClinicalTrials.gov (February 24, 2023, # NCT05744154 ).

Eliminating Ambiguous Treatment Effects Using Estimands

Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This is because most authors do not attempt to describe what the treatment effect represents, and instead require readers to deduce this based on the reported statistical methods. However, this approach is challenging, because many methods provide counterintuitive results. For example, some methods include data from all patients, yet the resulting treatment effect applies only to a subset of patients, whereas other methods will exclude certain patients while results will apply to everyone. Additionally, some analyses provide estimates pertaining to hypothetical settings in which patients never die or discontinue treatment. Herein we introduce estimands as a solution to the aforementioned problem. An estimand is a clear description of what the treatment effect represents, thus saving readers the necessity of trying to infer this from study methods and potentially getting it wrong. We provide examples of how estimands can remove ambiguity from reported treatment effects and describe their current use in practice. The crux of our argument is that readers should not have to infer what investigators are estimating; they should be told explicitly.

Effect of community health worker home visits on antenatal care and institutional delivery: an analysis of secondary outcomes from a cluster randomised trial in Mali

INTRODUCTION

Though community health workers (CHWs) have improved access to antenatal care (ANC) and institutional delivery in different settings, it is unclear what package and delivery strategy maximises impact.

METHODS

This study reports a secondary aim of the Proactive Community Case Management cluster randomised trial, conducted between December 2016 and April 2020 in Mali. It evaluated whether proactive home visits can improve ANC access at a population level compared with passive site-based care. 137 unique village clusters, covering the entire study area, were stratified by health catchment area and distance to the nearest primary health centre. Within each stratum, clusters were randomly assigned to intervention or control arm. CHWs in intervention clusters proactively visited all homes to provide care. In the control clusters, CHWs provided the same services at their fixed community health post to care-seeking patients. Pregnant women 15-49 years old were enrolled in a series of community-based and facility-based visits. We analysed individual-level annual survey data from baseline and 24-month and 36-month follow-up for the secondary outcomes of ANC and institutional delivery, complemented with CHW monitoring data during the trial period. We compared outcomes between: (1) the intervention and control arms, and (2) the intervention period and baseline.

RESULTS

With 2576 and 2536 pregnancies from 66 and 65 clusters in the intervention and control arms, respectively, the estimated risk ratios for receiving any ANC was 1.05 (95% CI 1.02 to 1.07), four or more ANC visits was 1.25 (95% CI 1.08 to 1.43) and ANC initiated in the first trimester was 1.11 (95% CI 1.02 to 1.19), relative to the controls; no differences in institutional delivery were found. However, both arms achieved large improvements in institutional delivery, compared with baseline. Monitoring data show that 19% and 2% of registered pregnancies received at least eight ANC contacts in the intervention and control arms, respectively. Six clusters, three from each arm had to be dropped in the last 2 years of the trial.

CONCLUSIONS

Proactive home visits increased ANC and the number of antenatal contacts at the clinic and community levels. ANC and institutional delivery can be increased when provided without fees from professional CHWs in upgraded primary care clinics.

TRIAL REGISTRATION NUMBER

NCT02694055.

Methods for Assessing Spillover in Network-Based Studies of HIV/AIDS Prevention among People Who Use Drugs

Human Immunodeficiency Virus (HIV) interventions among people who use drugs (PWUD) often have spillover, also known as interference or dissemination, which occurs when one participant's exposure affects another participant's outcome. PWUD are often members of networks defined by social, sexual, and drug-use partnerships and their receipt of interventions can affect other members in their network. For example, HIV interventions with possible spillover include educational training about HIV risk reduction, pre-exposure prophylaxis, or treatment as prevention. In turn, intervention effects frequently depend on the network structure, and intervention coverage levels and spillover can occur even if not measured in a study, possibly resulting in an underestimation of intervention effects. Recent methodological approaches were developed to assess spillover in the context of network-based studies. This tutorial provides an overview of different study designs for network-based studies and related methodological approaches for assessing spillover in each design. We also provide an overview of other important methodological issues in network studies, including causal influence in networks and missing data. Finally, we highlight applications of different designs and methods from studies of PWUD and conclude with an illustrative example from the Transmission Reduction Intervention Project (TRIP) in Athens, Greece.

Randomization, design and analysis for interdependency in aging research: no person or mouse is an island

Investigators traditionally use randomized designs and corresponding analysis procedures to make causal inferences about the effects of interventions, assuming independence between an individual's outcome and treatment assignment and the outcomes of other individuals in the study. Often, such independence may not hold. We provide examples of interdependency in model organism studies and human trials and group effects in aging research and then discuss methodologic issues and solutions. We group methodologic issues as they pertain to (1) single-stage individually randomized trials; (2) cluster-randomized controlled trials; (3) pseudo-cluster-randomized trials; (4) individually randomized group treatment; and (5) two-stage randomized designs. Although we present possible strategies for design and analysis to improve the rigor, accuracy and reproducibility of the science, we also acknowledge real-world constraints. Consequences of nonadherence, differential attrition or missing data, unintended exposure to multiple treatments and other practical realities can be reduced with careful planning, proper study designs and best practices.