Assessing heterogeneity of treatment effect analyses in health-related cluster randomized trials: A systematic review

Refining, implementing, and evaluating an anesthesia choice conversation aid for older adults with hip fracture: protocol for a stepped wedge cluster randomized trial

BACKGROUND

Hip fracture surgery under general or spinal anesthesia is a common procedure for older adults in the United States (US). Although spinal or general anesthesia can be appropriate for many patients, and the choice between anesthesia types is preference-sensitive, shared decision-making is not consistently used by anesthesiologists counseling patients on anesthesia for this procedure. We designed an Option Grid™-style conversation aid, My Anesthesia Choice─Hip Fracture, to promote shared decision making in this interaction. This study will refine the aid and evaluate its implementation and effectiveness in clinical practice.

METHODS

The study will be conducted over 2 phases: qualitative interviews with relevant clinicians and patients to refine the aid, followed by a stepped wedge cluster randomized trial of the intervention at 6 settings in the US. Primary outcomes will include the percentage of eligible patients who receive the intervention (intervention reach) and the change in quality of patient/clinician communication (intervention effectiveness). Secondary outcomes addressing other RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) domains will also be collected. Outcomes will be compared between baseline data and an active implementation period and then compared between the active implementation period and a sustainment period. Implementation strategies are guided by three constructs from the Practical, Robust Implementation and Sustainability Model (PRISM): intervention, recipients, and implementation and sustainability infrastructure.

DISCUSSION

This is a novel, large-scale trial evaluating and implementing a shared decision-making conversation aid for anesthesia choices. Strong buy-in from site leads and expert advisors will support both the success of implementation and the future dissemination of results and the intervention. Results from this study will inform the broader implementation of this aid for patients with hip fractures and can lead to the development and implementation of similar conversation aids for other anesthesia choices.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT06438640.

Sample Size Requirements to Test Subgroup-Specific Treatment Effects in Cluster-Randomized Trials

Cluster-randomized trials (CRTs) often allocate intact clusters of participants to treatment or control conditions and are increasingly used to evaluate healthcare delivery interventions. While previous studies have developed sample size methods for testing confirmatory hypotheses of treatment effect heterogeneity in CRTs (i.e., targeting the difference between subgroup-specific treatment effects), sample size methods for testing the subgroup-specific treatment effects themselves have not received adequate attention-despite a rising interest in health equity considerations in CRTs. In this article, we develop formal methods for sample size and power analyses for testing subgroup-specific treatment effects in parallel-arm CRTs with a continuous outcome and a binary subgroup variable. We point out that the variances of the subgroup-specific treatment effect estimators and their covariance are given by weighted averages of the variance of the overall average treatment effect estimator and the variance of the heterogeneous treatment effect estimator. This analytical insight facilitates an explicit characterization of the requirements for both the omnibus test and the intersection-union test to achieve the desired level of power. Generalizations to allow for subgroup-specific variance structures are also discussed. We report on a simulation study to validate the proposed sample size methods and demonstrate that the empirical power corresponds well with the predicted power for both tests. The design and setting of the Umea Dementia and Exercise (UMDEX) CRT in older adults are used to illustrate our sample size methods.

Sample size and power calculation for testing treatment effect heterogeneity in cluster randomized crossover designs

The cluster randomized crossover design has been proposed to improve efficiency over the traditional parallel-arm cluster randomized design. While statistical methods have been developed for designing cluster randomized crossover trials, they have exclusively focused on testing the overall average treatment effect, with little attention to differential treatment effects across subpopulations. Recently, interest has grown in understanding whether treatment effects may vary across pre-specified patient subpopulations, such as those defined by demographic or clinical characteristics. In this article, we consider the two-treatment two-period cluster randomized crossover design under either a cross-sectional or closed-cohort sampling scheme, where it is of interest to detect the heterogeneity of treatment effect via an interaction test. Assuming a patterned correlation structure for both the covariate and the outcome, we derive new sample size formulas for testing the heterogeneity of treatment effect with continuous outcomes based on linear mixed models. Our formulas also address unequal cluster sizes and therefore allow us to analytically assess the impact of unequal cluster sizes on the power of the interaction test in cluster randomized crossover designs. We conduct simulations to confirm the accuracy of the proposed methods, and illustrate their application in two real cluster randomized crossover trials.

Assessing treatment effect heterogeneity in the presence of missing effect modifier data in cluster-randomized trials

Understanding whether and how treatment effects vary across subgroups is crucial to inform clinical practice and recommendations. Accordingly, the assessment of heterogeneous treatment effects based on pre-specified potential effect modifiers has become a common goal in modern randomized trials. However, when one or more potential effect modifiers are missing, complete-case analysis may lead to bias and under-coverage. While statistical methods for handling missing data have been proposed and compared for individually randomized trials with missing effect modifier data, few guidelines exist for the cluster-randomized setting, where intracluster correlations in the effect modifiers, outcomes, or even missingness mechanisms may introduce further threats to accurate assessment of heterogeneous treatment effect. In this article, the performance of several missing data methods are compared through a simulation study of cluster-randomized trials with continuous outcome and missing binary effect modifier data, and further illustrated using real data from the Work, Family, and Health Study. Our results suggest that multilevel multiple imputation and Bayesian multilevel multiple imputation have better performance than other available methods, and that Bayesian multilevel multiple imputation has lower bias and closer to nominal coverage than standard multilevel multiple imputation when there are model specification or compatibility issues.

Sample size requirements for testing treatment effect heterogeneity in cluster randomized trials with binary outcomes

Cluster randomized trials (CRTs) refer to a popular class of experiments in which randomization is carried out at the group level. While methods have been developed for planning CRTs to study the average treatment effect, and more recently, to study the heterogeneous treatment effect, the development for the latter objective has currently been limited to a continuous outcome. Despite the prevalence of binary outcomes in CRTs, determining the necessary sample size and statistical power for detecting differential treatment effects in CRTs with a binary outcome remain unclear. To address this methodological gap, we develop sample size procedures for testing treatment effect heterogeneity in two-level CRTs under a generalized linear mixed model. Closed-form sample size expressions are derived for a binary effect modifier, and in addition, a computationally efficient Monte Carlo approach is developed for a continuous effect modifier. Extensions to multiple effect modifiers are also discussed. We conduct simulations to examine the accuracy of the proposed sample size methods. We present several numerical illustrations to elucidate features of the proposed formulas and to compare our method to the approximate sample size calculation under a linear mixed model. Finally, we use data from the Strategies and Opportunities to Stop Colon Cancer in Priority Populations (STOP CRC) CRT to illustrate the proposed sample size procedure for testing treatment effect heterogeneity.

Designing three-level cluster randomized trials to assess treatment effect heterogeneity

Cluster randomized trials often exhibit a three-level structure with participants nested in subclusters such as health care providers, and subclusters nested in clusters such as clinics. While the average treatment effect has been the primary focus in planning three-level randomized trials, interest is growing in understanding whether the treatment effect varies among prespecified patient subpopulations, such as those defined by demographics or baseline clinical characteristics. In this article, we derive novel analytical design formulas based on the asymptotic covariance matrix for powering confirmatory analyses of treatment effect heterogeneity in three-level trials, that are broadly applicable to the evaluation of cluster-level, subcluster-level, and participant-level effect modifiers and to designs where randomization can be carried out at any level. We characterize a nested exchangeable correlation structure for both the effect modifier and the outcome conditional on the effect modifier, and generate new insights from a study design perspective for conducting analyses of treatment effect heterogeneity based on a linear mixed analysis of covariance model. A simulation study is conducted to validate our new methods and two real-world trial examples are used for illustrations.

A quantitative assessment of the frequency and magnitude of heterogeneous treatment effects in studies of the health effects of social policies

Substantial heterogeneity in effects of social policies on health across subgroups may be common, but has not been systematically characterized. Using a sample of 55 contemporary studies on health effects of social policies, we recorded how often heterogeneous treatment effects (HTEs) were assessed, for what subgroups (e.g., male, female), and the subgroup-specific effect estimates expressed as Standardized Mean Differences (SMDs). For each study, outcome, and dimension (e.g., gender), we fit a random-effects meta-analysis. We characterized the magnitude of heterogeneity in policy effects using the standard deviation of the subgroup-specific effect estimates (τ). Among the 44% of studies reporting subgroup-specific estimates, policy effects were generally small (<0.1 SMDs) with mixed impacts on health (67% beneficial) and disparities (50% implied narrowing of disparities). Across study-outcome-dimensions, 54% indicated any heterogeneity in effects, and 20% had τ > 0.1 SMDs. For 26% of study-outcome-dimensions, the magnitude of τ indicated that effects of opposite signs were plausible across subgroups. Heterogeneity was more common in policy effects not specified a priori. Our findings suggest social policies commonly have heterogeneous effects on health of different populations; these HTEs may substantially impact disparities. Studies of social policies and health should routinely evaluate HTEs.

Accounting for expected attrition in the planning of cluster randomized trials for assessing treatment effect heterogeneity

BACKGROUND

Detecting treatment effect heterogeneity is an important objective in cluster randomized trials and implementation research. While sample size procedures for testing the average treatment effect accounting for participant attrition assuming missing completely at random or missing at random have been previously developed, the impact of attrition on the power for detecting heterogeneous treatment effects in cluster randomized trials remains unknown.

METHODS

We provide a sample size formula for testing for a heterogeneous treatment effect assuming the outcome is missing completely at random. We also propose an efficient Monte Carlo sample size procedure for assessing heterogeneous treatment effect assuming covariate-dependent outcome missingness (missing at random). We compare our sample size methods with the direct inflation method that divides the estimated sample size by the mean follow-up rate. We also evaluate our methods through simulation studies and illustrate them with a real-world example.

RESULTS

Simulation results show that our proposed sample size methods under both missing completely at random and missing at random provide sufficient power for assessing heterogeneous treatment effect. The proposed sample size methods lead to more accurate sample size estimates than the direct inflation method when the missingness rate is high (e.g., ≥ 30%). Moreover, sample size estimation under both missing completely at random and missing at random is sensitive to the missingness rate, but not sensitive to the intracluster correlation coefficient among the missingness indicators.

CONCLUSION

Our new sample size methods can assist in planning cluster randomized trials that plan to assess a heterogeneous treatment effect and participant attrition is expected to occur.

Impact of interventions for tuberculosis prevention and care in South Africa - a systematic review of mathematical modelling studies

BACKGROUND

Substantial additional efforts are needed to prevent, find and successfully treat tuberculosis (TB) in South Africa (SA). In thepast decade, an increasing body of mathematical modelling research has investigated the population-level impact of TB prevention and careinterventions. To date, this evidence has not been assessed in the SA context.

OBJECTIVE

To systematically review mathematical modelling studies that estimated the impact of interventions towards the World HealthOrganization's End TB Strategy targets for TB incidence, TB deaths and catastrophic costs due to TB in SA.

METHODS

We searched the PubMed, Web of Science and Scopus databases for studies that used transmission-dynamic models of TB in SAand reported on at least one of the End TB Strategy targets at population level. We described study populations, type of interventions andtheir target groups, and estimates of impact and other key findings. For studies of country-level interventions, we estimated average annualpercentage declines (AAPDs) in TB incidence and mortality attributable to the intervention.

RESULTS

We identified 29 studies that met our inclusion criteria, of which 7 modelled TB preventive interventions (vaccination,antiretroviral treatment (ART) for HIV, TB preventive treatment (TPT)), 12 considered interventions along the care cascade for TB(screening/case finding, reducing initial loss to follow-up, diagnostic and treatment interventions), and 10 modelled combinationsof preventive and care-cascade interventions. Only one study focused on reducing catastrophic costs due to TB. The highest impactof a single intervention was estimated in studies of TB vaccination, TPT among people living with HIV, and scale-up of ART. Forpreventive interventions, AAPDs for TB incidence varied between 0.06% and 7.07%, and for care-cascade interventions between 0.05%and 3.27%.

CONCLUSION

We describe a body of mathematical modelling research with a focus on TB prevention and care in SA. We found higherestimates of impact reported in studies of preventive interventions, highlighting the need to invest in TB prevention in SA. However, studyheterogeneity and inconsistent baseline scenarios limit the ability to compare impact estimates between studies. Combinations, rather thansingle interventions, are likely needed to reach the End TB Strategy targets in SA.

Heterogeneous treatment effects in social policy studies: An assessment of contemporary articles in the health and social sciences

PURPOSE

. Social policies are important determinants of population health but may have varying effects on subgroups of people. Evaluating heterogeneous treatment effects (HTEs) of social policies is critical to determine how social policies will affect health inequities. Methods for evaluating HTEs are not standardized. Little is known about how often and by what methods HTEs are assessed in social policy and health research.

METHODS

. A sample of 55 articles from 2019 on the health effects of social policies were evaluated for frequency of reporting HTEs; for what subgroupings HTEs were reported; frequency of a priori specification of intent to assess HTEs; and methods used for assessing HTEs.

RESULTS

. 24 (44%) studies described some form of HTE assessment, including by age, gender, education, race/ethnicity, and/or geography. Among studies assessing HTEs, 63% specified HTE assessment a priori, and most (71%) used descriptive methods such as stratification; 21% used statistical tests (e.g., interaction terms in a regression); and no studies used data-driven algorithms.

CONCLUSIONS

. Although understanding HTEs could enhance policy and practice-based efforts to reduce inequities, it is not routine research practice. Increased evaluation of HTEs across relevant subgroups is needed.

Biased Survival Predictions When Appraising Health Technologies in Heterogeneous Populations

INTRODUCTION

Time-to-event data from clinical trials are routinely extrapolated using parametric models to estimate the cost effectiveness of novel therapies, but how this approach performs in the presence of heterogeneous populations remains unknown.

METHODS

We performed a simulation study of seven scenarios with varying exponential distributions modelling treatment and prognostic effects across subgroup and complement populations, with follow-up typical of clinical trials used to appraise the cost effectiveness of therapies by agencies such as the UK National Institute for Health and Care Excellence (NICE). We compared established and emerging methods of estimating population life-years (LYs) using parametric models. We also proved analytically that an exponential model fitted to censored heterogeneous survival times sampled from two distinct exponential distributions will produce a biased estimate of the hazard rate and LYs.

RESULTS

LYs are underestimated by the methods in the presence of heterogeneity, resulting in either under- or overestimation of the incremental benefit. In scenarios where the overestimation of benefit is likely, which is of interest to the healthcare provider, the method of taking the average LYs from all plausible models has the least bias. LY estimates from complete Kaplan-Meier curves have high variation, suggesting mature data may not be a reliable solution. We explore the effect of increasing trial sample size and accounting for detected treatment-subgroup interactions.

CONCLUSIONS

The bias associated with heterogeneous populations suggests that NICE may need to be more cautious when appraising therapies and to consider model averaging or the separate modelling of subgroups when heterogeneity is suspected or detected.

Accounting for unequal cluster sizes in designing cluster randomized trials to detect treatment effect heterogeneity

Unequal cluster sizes are common in cluster randomized trials (CRTs). While there are a number of previous investigations studying the impact of unequal cluster sizes on the power for testing the average treatment effect in CRTs, little is known about the impact of unequal cluster sizes on the power for testing the heterogeneous treatment effect (HTE) in CRTs. In this work, we expand the sample size procedures for studying HTE in CRTs to accommodate cluster size variation under the linear mixed model framework. Through analytical derivation and graphical exploration, we show that the sample size for the HTE with an individual-level effect modifier is less affected by unequal cluster sizes than with a cluster-level effect modifier. The impact of cluster size variability jointly depends on the mean and coefficient of variation of cluster sizes, covariate intraclass correlation coefficient (ICC) and the conditional outcome ICC. In addition, we demonstrate that the HTE-motivated analysis of covariance framework can be used for analyzing the average treatment effect, and offer a more efficient sample size procedure for studying the average treatment effect adjusting for the effect modifier. We use simulations to confirm the accuracy of the proposed sample size procedures for both the average treatment effect and HTE in CRTs. Extensions to multivariate effect modifiers are provided and our procedure is illustrated in the context of the Strategies to Reduce Injuries and Develop Confidence in Elders trial.

Sample size requirements for detecting treatment effect heterogeneity in cluster randomized trials

Cluster randomized trials (CRTs) refer to experiments with randomization carried out at the cluster or the group level. While numerous statistical methods have been developed for the design and analysis of CRTs, most of the existing methods focused on testing the overall treatment effect across the population characteristics, with few discussions on the differential treatment effect among subpopulations. In addition, the sample size and power requirements for detecting differential treatment effect in CRTs remain unclear, but are helpful for studies planned with such an objective. In this article, we develop a new sample size formula for detecting treatment effect heterogeneity in two-level CRTs for continuous outcomes, continuous or binary covariates measured at cluster or individual level. We also investigate the roles of two intraclass correlation coefficients (ICCs): the adjusted ICC for the outcome of interest and the marginal ICC for the covariate of interest. We further derive a closed-form design effect formula to facilitate the application of the proposed method, and provide extensions to accommodate multiple covariates. Extensive simulations are carried out to validate the proposed formula in finite samples. We find that the empirical power agrees well with the prediction across a range of parameter constellations, when data are analyzed by a linear mixed effects model with a treatment-by-covariate interaction. Finally, we use data from the HF-ACTION study to illustrate the proposed sample size procedure for detecting heterogeneous treatment effects.