Marginal modeling in community randomized trials with rare events: Utilization of the negative binomial regression model

Community-Based Cluster-Randomized Trial to Reduce Opioid Overdose Deaths

BACKGROUND

Evidence-based practices for reducing opioid-related overdose deaths include overdose education and naloxone distribution, the use of medications for the treatment of opioid use disorder, and prescription opioid safety. Data are needed on the effectiveness of a community-engaged intervention to reduce opioid-related overdose deaths through enhanced uptake of these practices.

METHODS

In this community-level, cluster-randomized trial, we randomly assigned 67 communities in Kentucky, Massachusetts, New York, and Ohio to receive the intervention (34 communities) or a wait-list control (33 communities), stratified according to state. The trial was conducted within the context of both the coronavirus disease 2019 (Covid-19) pandemic and a national surge in the number of fentanyl-related overdose deaths. The trial groups were balanced within states according to urban or rural classification, previous overdose rate, and community population. The primary outcome was the number of opioid-related overdose deaths among community adults.

RESULTS

During the comparison period from July 2021 through June 2022, the population-averaged rates of opioid-related overdose deaths were similar in the intervention group and the control group (47.2 deaths per 100,000 population vs. 51.7 per 100,000 population), for an adjusted rate ratio of 0.91 (95% confidence interval, 0.76 to 1.09; P = 0.30). The effect of the intervention on the rate of opioid-related overdose deaths did not differ appreciably according to state, urban or rural category, age, sex, or race or ethnic group. Intervention communities implemented 615 evidence-based practice strategies from the 806 strategies selected by communities (254 involving overdose education and naloxone distribution, 256 involving the use of medications for opioid use disorder, and 105 involving prescription opioid safety). Of these evidence-based practice strategies, only 235 (38%) had been initiated by the start of the comparison year.

CONCLUSIONS

In this 12-month multimodal intervention trial involving community coalitions in the deployment of evidence-based practices to reduce opioid overdose deaths, death rates were similar in the intervention group and the control group in the context of the Covid-19 pandemic and the fentanyl-related overdose epidemic. (Funded by the National Institutes of Health; HCS ClinicalTrials.gov number, NCT04111939.).

Effect of the Communities that HEAL intervention on receipt of behavioral therapies for opioid use disorder: A cluster randomized wait-list controlled trial

BACKGROUND

The U.S. opioid overdose crisis persists. Outpatient behavioral health services (BHS) are essential components of a comprehensive response to opioid use disorder and overdose fatalities. The Helping to End Addiction Long-Term® (HEALing) Communities Study developed the Communities That HEAL (CTH) intervention to reduce opioid overdose deaths in 67 communities in Kentucky, Ohio, New York, and Massachusetts through the implementation of evidence-based practices (EBPs), including BHS. This paper compares the rate of individuals receiving outpatient BHS in Wave 1 intervention communities (n = 34) to waitlisted Wave 2 communities (n = 33).

METHODS

Medicaid data included individuals ≥18 years of age receiving any of five BHS categories: intensive outpatient, outpatient, case management, peer support, and case management or peer support. Negative binomial regression models estimated the rate of receiving each BHS for Wave 1 and Wave 2. Effect modification analyses evaluated changes in the effect of the CTH intervention between Wave 1 and Wave 2 by research site, rurality, age, sex, and race/ethnicity.

RESULTS

No significant differences were detected between intervention and waitlisted communities in the rate of individuals receiving any of the five BHS categories. None of the interaction effects used to test the effect modification were significant.

CONCLUSIONS

Several factors should be considered when interpreting results-no significant intervention effects were observed through Medicaid claims data, the best available data source but limited in terms of capturing individuals reached by the intervention. Also, the 12-month evaluation window may have been too brief to see improved outcomes considering the time required to stand-up BHS.

TRIAL REGISTRATION

Clinical Trials.gov http://www.

CLINICALTRIALS

gov: Identifier: NCT04111939.

Reconsidering stepped wedge cluster randomized trial designs with implementation periods: Fewer sequences or the parallel-group design with baseline and implementation periods are potentially more efficient

BACKGROUND/AIMS

When designing a cluster randomized trial, advantages and disadvantages of tentative designs must be weighed. The stepped wedge design is popular for multiple reasons, including its potential to increase power via improved efficiency relative to a parallel-group design. In many realistic settings, it will take time for clusters to fully implement the intervention. When designing the HEALing (Helping to End Addiction Long-termSM) Communities Study, implementation time was a major consideration, and we examined the efficiency and practicality of three designs. Specifically, a three-sequence stepped wedge design with implementation periods, a corresponding two-sequence modified design that is created by removing the middle sequence, and a parallel-group design with baseline and implementation periods. In this article, we study the relative efficiencies of these specific designs. More generally, we study the relative efficiencies of modified designs when the stepped wedge design with implementation periods has three or more sequences. We also consider different correlation structures.

METHODS

We compare efficiencies of stepped wedge designs with implementation periods consisting of three to nine sequences with a variety of corresponding designs. The three-sequence design is compared to the two-sequence modified design and to the parallel-group design with baseline and implementation periods analysed via analysis of covariance. Stepped wedge designs with implementation periods consisting of four or more sequences are compared to modified designs that remove all or a subset of 'middle' sequences. Efficiencies are based on the use of linear mixed effects models.

RESULTS

In the studied settings, the modified design is more efficient than the three-sequence stepped wedge design with implementation periods. The parallel-group design with baseline and implementation periods with analysis of covariance-based analysis is often more efficient than the three-sequence design. With respect to stepped wedge designs with implementation periods that are comprised of more sequences, there are often corresponding modified designs that improve efficiency. However, use of only the first and last sequences has the potential to be either relatively efficient or inefficient. Relative efficiency is impacted by the strength of the statistical correlation among outcomes from the same cluster; for example, the relative efficiencies of modified designs tend to be greater for smaller cluster auto-correlation values.

CONCLUSION

If a three-sequence stepped wedge design with implementation periods is being considered for a future cluster randomized trial, then a corresponding modified design using only the first and last sequences should be considered if sole focus is on efficiency. However, a parallel-group design with baseline and implementation periods and analysis of covariance-based analysis can be a practical, efficient alternative. For stepped wedge designs with implementation periods and a larger number of sequences, modified versions that remove 'middle' sequences should be considered. Due to the potential sensitivity of design efficiencies, statistical correlation should be carefully considered.

Impact of the Communities That HEAL Intervention on Buprenorphine-Waivered Practitioners and Buprenorphine Prescribing: A Prespecified Secondary Analysis of the HCS Randomized Clinical Trial

Importance

Buprenorphine significantly reduces opioid-related overdose mortality. From 2002 to 2022, the Drug Addiction Treatment Act of 2000 (DATA 2000) required qualified practitioners to receive a waiver from the Drug Enforcement Agency to prescribe buprenorphine for treatment of opioid use disorder. During this period, waiver uptake among practitioners was modest; subsequent changes need to be examined.

Objective

To determine whether the Communities That HEAL (CTH) intervention increased the rate of practitioners with DATA 2000 waivers and buprenorphine prescribing.

Design, Setting, and Participants

This prespecified secondary analysis of the HEALing Communities Study, a multisite, 2-arm, parallel, community-level, cluster randomized, open, wait-list-controlled comparison clinical trial was designed to assess the effectiveness of the CTH intervention and was conducted between January 1, 2020, to December 31, 2023, in 67 communities in Kentucky, Massachusetts, New York, and Ohio, accounting for approximately 8.2 million adults. The participants in this trial were communities consisting of counties (n = 48) and municipalities (n = 19). Trial arm randomization was conducted using a covariate constrained randomization procedure stratified by state. Each state was balanced by community characteristics including urban/rural classification, fatal opioid overdose rate, and community population. Thirty-four communities were randomized to the intervention and 33 to wait-list control arms. Data analysis was conducted between March 20 and September 29, 2023, with a focus on the comparison period from July 1, 2021, to June 30, 2022.

Intervention

Waiver trainings and other educational trainings were offered or supported by the HEALing Communities Study research sites in each state to help build practitioner capacity.

Main Outcomes and Measures

The rate of practitioners with a DATA 2000 waiver (overall, and stratified by 30-, 100-, and 275-patient limits) per 100 000 adult residents aged 18 years or older during July 1, 2021, to June 30, 2022, were compared between the intervention and wait-list control communities. The rate of buprenorphine prescribing among those waivered practitioners was also compared between the intervention and wait-list control communities. Intention-to-treat and per-protocol analyses were performed.

Results

A total of 8 166 963 individuals aged 18 years or older were residents of the 67 communities studied. There was no evidence of an effect of the CTH intervention on the adjusted rate of practitioners with a DATA 2000 waiver (adjusted relative rate [ARR], 1.04; 95% CI, 0.94-1.14) or the adjusted rate of practitioners with a DATA 2000 waiver who actively prescribed buprenorphine (ARR, 0.97; 95% CI, 0.86-1.10).

Conclusions and Relevance

In this randomized clinical trial, the CTH intervention was not associated with increases in the rate of practitioners with a DATA 2000 waiver or buprenorphine prescribing among those waivered practitioners. Supporting practitioners to prescribe buprenorphine remains a critical yet challenging step in the continuum of care to treat opioid use disorder.

Trial Registration

ClinicalTrials.gov Identifier: NCT04111939.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Protocol for community-driven selection of strategies to implement evidence-based practices to reduce opioid overdoses in the HEALing Communities Study: a trial to evaluate a community-engaged intervention in Kentucky, Massachusetts, New York and Ohio

INTRODUCTION

Opioid-involved overdose deaths continue to surge in many communities, despite numerous evidence-based practices (EBPs) that exist to prevent them. The HEALing Communities Study (HCS) was launched to develop and test an intervention (ie, Communities That HEAL (CTH)) that supports communities in expanding uptake of EBPs to reduce opioid-involved overdose deaths. This paper describes a protocol for a process foundational to the CTH intervention through which community coalitions select strategies to implement EBPs locally.

METHODS AND ANALYSIS

The CTH is being implemented in 67 communities (randomised to receive the intervention) in four states in partnership with coalitions (one per community). Coalitions must select at least five strategies, including one to implement each of the following EBPs: (a) overdose education and naloxone distribution; expanded (b) access to medications for opioid use disorder (MOUD), (c) linkage to MOUD, (d) retention in MOUD and (e) safer opioid prescribing/dispensing. Facilitated by decision aid tools, the community action planning process includes (1) data-driven goal setting, (2) discussion and prioritisation of EBP strategies, (3) selection of EBP strategies and (4) identification of next steps. Following review of epidemiologic data and information on existing local services, coalitions set goals and discuss, score and/or rank EBP strategies based on feasibility, appropriateness within the community context and potential impact on reducing opioid-involved overdose deaths with a focus on three key sectors (healthcare, behavioural health and criminal justice) and high-risk/vulnerable populations. Coalitions then select EBP strategies through consensus or majority vote and, subsequently, suggest or choose agencies with which to partner for implementation.

ETHICS AND DISSEMINATION

The HCS protocol was approved by a central Institutional Review Board (Advarra). Results of the action planning process will be disseminated in academic conferences and peer-reviewed journals, online and print media, and in meetings with community stakeholders.

TRIAL REGISTRATION NUMBER

NCT04111939.

Performance of model-based vs. permutation tests in the HEALing (Helping to End Addiction Long-termSM) Communities Study, a covariate-constrained cluster randomized trial

BACKGROUND

The HEALing (Helping to End Addiction Long-term^SM) Communities Study (HCS) is a multi-site parallel group cluster randomized wait-list comparison trial designed to evaluate the effect of the Communities That Heal (CTH) intervention compared to usual care on opioid overdose deaths. Covariate-constrained randomization (CCR) was applied to balance the community-level baseline covariates in the HCS. The purpose of this paper is to evaluate the performance of model-based tests and permutation tests in the HCS setting. We conducted a simulation study to evaluate type I error rates and power for model-based and permutation tests for the multi-site HCS as well as for a subgroup analysis of a single state (Massachusetts). We also investigated whether the maximum degree of imbalance in the CCR design has an impact on the performance of the tests.

METHODS

The primary outcome, the number of opioid overdose deaths, is count data assessed at the community level that will be analyzed using a negative binomial regression model. We conducted a simulation study to evaluate the type I error rates and power for 3 tests: (1) Wald-type t-test with small-sample corrected empirical standard error estimates, (2) Wald-type z-test with model-based standard error estimates, and (3) permutation test with test statistics calculated by the difference in average residuals for the two groups.

RESULTS

Our simulation results demonstrated that Wald-type t-tests with small-sample corrected empirical standard error estimates from the negative binomial regression model maintained proper type I error. Wald-type z-tests with model-based standard error estimates were anti-conservative. Permutation tests preserved type I error rates if the constrained space was not too small. For all tests, the power was high to detect the hypothesized 40% reduction in opioid overdose deaths for the intervention vs. comparison group both for the overall HCS and the subgroup analysis of Massachusetts (MA).

CONCLUSIONS

Based on the results of our simulation study, the Wald-type t-test with small-sample corrected empirical standard error estimates from a negative binomial regression model is a valid and appropriate approach for analyzing cluster-level count data from the HEALing Communities Study.

TRIAL REGISTRATION

ClinicalTrials.gov http://www.

CLINICALTRIALS

gov ; Identifier: NCT04111939.

Cluster randomised trials with a binary outcome and a small number of clusters: comparison of individual and cluster level analysis method

BACKGROUND

Cluster randomised trials (CRTs) are often designed with a small number of clusters, but it is not clear which analysis methods are optimal when the outcome is binary. This simulation study aimed to determine (i) whether cluster-level analysis (CL), generalised linear mixed models (GLMM), and generalised estimating equations with sandwich variance (GEE) approaches maintain acceptable type-one error including the impact of non-normality of cluster effects and low prevalence, and if so (ii) which methods have the greatest power. We simulated CRTs with 8-30 clusters, altering the cluster-size, outcome prevalence, intracluster correlation coefficient, and cluster effect distribution. We analysed each dataset with weighted and unweighted CL; GLMM with adaptive quadrature and restricted pseudolikelihood; GEE with Kauermann-and-Carroll and Fay-and-Graubard sandwich variance using independent and exchangeable working correlation matrices. P-values were from a t-distribution with degrees of freedom (DoF) as clusters minus cluster-level parameters; GLMM pseudolikelihood also used Satterthwaite and Kenward-Roger DoF.

RESULTS

Unweighted CL, GLMM pseudolikelihood, and Fay-and-Graubard GEE with independent or exchangeable working correlation matrix controlled type-one error in > 97% scenarios with clusters minus parameters DoF. Cluster-effect distribution and prevalence of outcome did not usually affect analysis method performance. GEE had the least power. With 20-30 clusters, GLMM had greater power than CL with varying cluster-size but similar power otherwise; with fewer clusters, GLMM had lower power with common cluster-size, similar power with medium variation, and greater power with large variation in cluster-size.

CONCLUSION

We recommend that CRTs with ≤ 30 clusters and a binary outcome use an unweighted CL or restricted pseudolikelihood GLMM both with DoF clusters minus cluster-level parameters.