Improved standard error estimator for maintaining the validity of inference in cluster randomized trials with a small number of clusters

Reducing perceived barriers to scaling up overdose education and naloxone distribution and medications for opioid use disorder in the United States in the HEALing (Helping End Addiction Long-Term®) communities study

BACKGROUND

Scaling up overdose education and naloxone distribution (OEND) and medications for opioid use disorder (MOUD) is needed to reduce opioid overdose deaths, but barriers are pervasive. This study examines whether the Communities That HEAL (CTH) intervention reduced perceived barriers to expanding OEND and MOUD in healthcare/behavioral health, criminal-legal, and other/non-traditional venues.

METHODS

The HEALing (Helping End Addiction Long-Term®) Communities Study is a parallel, wait-list, cluster randomized trial testing the CTH intervention in 67 communities in the United States. Surveys administered to coalition members and key stakeholders measured the magnitude of perceived barriers to scaling up OEND and MOUD in November 2019-January 2020, May-June 2021, and May-June 2022. Multilevel linear mixed models compared Wave 1 (intervention) and Wave 2 (wait-list control) respondents. Interactions by rural/urban status and research site were tested.

RESULTS

Wave 1 respondents reported significantly greater reductions in mean scores for three outcomes: perceived barriers to scaling up OEND in Healthcare/Behavioral Health Venues (-0.26, 95% confidence interval, CI: -0.48, -0.05, p = 0.015), OEND in Other/Non-traditional Venues (-0.53, 95% CI: - 0.84, -0.22, p = 0.001) and MOUD in Other/Non-traditional Venues (-0.34, 95% CI: -0.62, -0.05, p = 0.020). There were significant interactions by research site for perceived barriers to scaling up OEND and MOUD in Criminal-Legal Venues. There were no significant interactions by rural/urban status.

DISCUSSION

The CTH Intervention reduced perceived barriers to scaling up OEND and MOUD in certain venues, with no difference in effectiveness between rural and urban communities. More research is needed to understand facilitators and barriers in different venues.

The effect of number of clusters and magnitude of within-cluster homogeneity in outcomes on the performance of four variance estimators for a marginal multivariable Cox regression model fit to clustered data in the context of observational research

Researchers often estimate the association between the hazard of a time-to-event outcome and the characteristics of individuals and the clusters in which individuals are nested. Lin and Wei's robust variance estimator is often used with a Cox regression model fit to clustered data. Recently, alternative variance estimators have been proposed: the Fay-Graubard estimator, the Kauermann-Carroll estimator, and the Mancl-DeRouen estimator. Using Monte Carlo simulations, we found that, when fitting a marginal Cox regression model with both individual-level and cluster-level covariates: (i) in the presence of weak to moderate within-cluster homogeneity of outcomes, the Lin-Wei variance estimator can result in estimates of the SE with moderate bias when the number of clusters is fewer than 20-30, while in the presence of strong within-cluster homogeneity, it can result in biased estimation even when the number of clusters is as large as 100; (ii) when the number of clusters was less than approximately 20, the Fay-Graubard variance estimator tended to result in estimates of SE with the lowest bias; (iii) when the number of clusters exceeded approximately 20, the Mancl-DeRouen estimator tended to result in estimated standard errors with the lowest bias; (iv) the Mancl-DeRouen estimator used with a t-distribution tended to result in 95% confidence that had the best performance of the estimators; (v) when the magnitude of within-cluster homogeneity in outcomes was strong or very strong, all methods resulted in confidence intervals with lower than advertised coverage rates even when the number of clusters was very large.

Explicit inference: A meta-replication of SEER cancer registry research evaluating the Affordable Care Act's Medicaid expansion

OBJECTIVES

Among the provisions within the Affordable Care Act (ACA), expanding Medicaid was arguably the greatest contributor to increasing access to care. For over a decade, researchers have investigated how Medicaid expansion impacted cancer outcomes. Over this same decade, statistical theory illuminated how state-based policy research could be compromised by invalid inference. After reviewing the literature to identify the inference strategies of state-based cancer registry Medicaid expansion research, this study aimed to assess how inference decisions could change the interpretation of Medicaid expansion's impact on staging, treatment, and mortality in cancer patients.

DATA SOURCES

Cancer case data (2000-2019) was obtained from the Surveillance, Epidemiology, End Results (SEER) programme. Cases included all cancer sites combined, top 10 cancer sites combined, and three screening amenable cancers (colorectal, female breast, female cervical).

STUDY DESIGN

A Difference-in-Differences design estimated the association between Medicaid expansion and four binary outcomes: distant stage, initiating treatment >1 month after diagnosis, no surgery recommendation, and death. Three inference techniques were compared: (1) traditional, (2) cluster, and (3) Wild Cluster Bootstrap.

DATA COLLECTION

Data was accessed via SEER*Stat.

PRINCIPAL FINDINGS

Estimating standard errors via traditional inference would suggest that Medicaid expansion was associated with delayed treatment initiation and surgery recommendations. Traditional and clustered inference also suggested that Medicaid expansion reduced mortality. Inference using Wild Cluster Bootstrap techniques never rejected the null hypotheses.

CONCLUSIONS

This study reiterates the importance of explicit inference. Future state-based, cancer policy research can be improved by incorporating emerging techniques. These findings warrant caution when interpreting prior SEER research reporting significant effects of Medicaid expansion on cancer outcomes, especially studies that did not explicitly define their inference strategy.

Community-Based Cluster-Randomized Trial to Reduce Opioid Overdose Deaths

BACKGROUND

Evidence-based practices for reducing opioid-related overdose deaths include overdose education and naloxone distribution, the use of medications for the treatment of opioid use disorder, and prescription opioid safety. Data are needed on the effectiveness of a community-engaged intervention to reduce opioid-related overdose deaths through enhanced uptake of these practices.

METHODS

In this community-level, cluster-randomized trial, we randomly assigned 67 communities in Kentucky, Massachusetts, New York, and Ohio to receive the intervention (34 communities) or a wait-list control (33 communities), stratified according to state. The trial was conducted within the context of both the coronavirus disease 2019 (Covid-19) pandemic and a national surge in the number of fentanyl-related overdose deaths. The trial groups were balanced within states according to urban or rural classification, previous overdose rate, and community population. The primary outcome was the number of opioid-related overdose deaths among community adults.

RESULTS

During the comparison period from July 2021 through June 2022, the population-averaged rates of opioid-related overdose deaths were similar in the intervention group and the control group (47.2 deaths per 100,000 population vs. 51.7 per 100,000 population), for an adjusted rate ratio of 0.91 (95% confidence interval, 0.76 to 1.09; P = 0.30). The effect of the intervention on the rate of opioid-related overdose deaths did not differ appreciably according to state, urban or rural category, age, sex, or race or ethnic group. Intervention communities implemented 615 evidence-based practice strategies from the 806 strategies selected by communities (254 involving overdose education and naloxone distribution, 256 involving the use of medications for opioid use disorder, and 105 involving prescription opioid safety). Of these evidence-based practice strategies, only 235 (38%) had been initiated by the start of the comparison year.

CONCLUSIONS

In this 12-month multimodal intervention trial involving community coalitions in the deployment of evidence-based practices to reduce opioid overdose deaths, death rates were similar in the intervention group and the control group in the context of the Covid-19 pandemic and the fentanyl-related overdose epidemic. (Funded by the National Institutes of Health; HCS ClinicalTrials.gov number, NCT04111939.).

Compare the marginal effects for environmental exposure and biomonitoring data with repeated measurements and values below the limit of detection

BACKGROUND

Environmental exposure and biomonitoring data with repeated measurements from environmental and occupational studies are commonly right-skewed and in the presence of limits of detection (LOD). However, existing model has not been discussed for small-sample properties and highly skewed data with non-detects and repeated measurements.

OBJECTIVE

Marginal modeling provides an alternative to analyzing longitudinal and cluster data, in which the parameter interpretations are with respect to marginal or population-averaged means.

METHODS

We outlined the theories of three marginal models, i.e., generalized estimating equations (GEE), quadratic inference functions (QIF), and generalized method of moments (GMM). With these approaches, we proposed to incorporate the fill-in methods, including single and multiple value imputation techniques, such that any measurements less than the limit of detection are assigned values.

RESULTS

We demonstrated that the GEE method works well in terms of estimating the regression parameters in small sample sizes, while the QIF and GMM outperform in large-sample settings, as parameter estimates are consistent and have relatively smaller mean squared error. No specific fill-in method can be deemed superior as each has its own merits.

IMPACT

Marginal modeling is firstly employed to analyze repeated measures data with non-detects, in which only the mean structure needs to be correctly provided to obtain consistent parameter estimates. After replacing non-detects through substitution methods and utilizing small-sample bias corrections, in a simulation study we found that the estimating approaches used in the marginal models have corresponding advantages under a wide range of sample sizes. We also applied the models to longitudinal and cluster working examples.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Improving sandwich variance estimation for marginal Cox analysis of cluster randomized trials

Cluster randomized trials (CRTs) frequently recruit a small number of clusters, therefore necessitating the application of small-sample corrections for valid inference. A recent systematic review indicated that CRTs reporting right-censored, time-to-event outcomes are not uncommon and that the marginal Cox proportional hazards model is one of the common approaches used for primary analysis. While small-sample corrections have been studied under marginal models with continuous, binary, and count outcomes, no prior research has been devoted to the development and evaluation of bias-corrected sandwich variance estimators when clustered time-to-event outcomes are analyzed by the marginal Cox model. To improve current practice, we propose nine bias-corrected sandwich variance estimators for the analysis of CRTs using the marginal Cox model and report on a simulation study to evaluate their small-sample properties. Our results indicate that the optimal choice of bias-corrected sandwich variance estimator for CRTs with survival outcomes can depend on the variability of cluster sizes and can also slightly differ whether it is evaluated according to relative bias or type I error rate. Finally, we illustrate the new variance estimators in a real-world CRT where the conclusion about intervention effectiveness differs depending on the use of small-sample bias corrections. The proposed sandwich variance estimators are implemented in an R package CoxBcv.

power swgee: GEE-based power calculations in stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials are increasingly being used to evaluate interventions in medical, public health, educational, and social science contexts. With the longitudinal and crossover nature of a SW-CRT, complex analysis techniques are often needed which makes appropriately powering SW-CRTs challenging. In this paper, we introduce a newly-developed SW-CRT power calculator, embedded within the power command in Stata. The power calculator assumes a marginal model (i.e., generalized estimating equations [GEE]) for the primary analysis of SW-CRTs, for which other currently available SW-CRT power calculators may not be suitable. The program accommodates complete cross-sectional and closed-cohort designs, and includes multilevel correlation structures appropriate for such designs. We discuss the methods and formulae underlying our SW-CRT calculator, and provide illustrative examples of the use of power swgee. We provide suggestions about the choice of parameters in power swgee, and conclude by discussing areas of future research which may improve the program.

Design and analysis of cluster randomized trials with time-to-event outcomes under the additive hazards mixed model

A primary focus of current methods for cluster randomized trials (CRTs) has been for continuous, binary, and count outcomes, with relatively less attention given to right-censored, time-to-event outcomes. In this article, we detail considerations for sample size requirement and statistical inference in CRTs with time-to-event outcomes when the intervention effect parameter is specified through the additive hazards mixed model (AHMM), which includes a frailty term to explicitly account for the dependency between the failure times. First, we discuss improved inference for the treatment effect parameter via bias-corrected sandwich variance estimators and randomization-based test under AHMM, addressing potential small-sample biases in CRTs. Next, we derive a new sample size formula for AHMM analysis of CRTs accommodating both equal and unequal cluster sizes. When the cluster sizes vary, our sample size formula depends on the mean and coefficient of variation of cluster sizes, based on which we articulate the impact of cluster size variation in CRTs with time-to-event outcomes. Furthermore, we obtain the insight that the classical variance inflation factor for CRTs with a non-censored outcome can in fact apply to CRTs with a time-to-event outcome, providing that an appropriate definition of the intraclass correlation coefficient is considered under AHMM. Simulation studies are carried out to illustrate key design and analysis considerations in CRTs with a small to moderate number of clusters. The proposed sample size procedure and analytical methods are further illustrated using the context of the STrategies to Reduce Injuries and Develop Confidence in Elders CRT.

Performance of model-based vs. permutation tests in the HEALing (Helping to End Addiction Long-termSM) Communities Study, a covariate-constrained cluster randomized trial

BACKGROUND

The HEALing (Helping to End Addiction Long-term^SM) Communities Study (HCS) is a multi-site parallel group cluster randomized wait-list comparison trial designed to evaluate the effect of the Communities That Heal (CTH) intervention compared to usual care on opioid overdose deaths. Covariate-constrained randomization (CCR) was applied to balance the community-level baseline covariates in the HCS. The purpose of this paper is to evaluate the performance of model-based tests and permutation tests in the HCS setting. We conducted a simulation study to evaluate type I error rates and power for model-based and permutation tests for the multi-site HCS as well as for a subgroup analysis of a single state (Massachusetts). We also investigated whether the maximum degree of imbalance in the CCR design has an impact on the performance of the tests.

METHODS

The primary outcome, the number of opioid overdose deaths, is count data assessed at the community level that will be analyzed using a negative binomial regression model. We conducted a simulation study to evaluate the type I error rates and power for 3 tests: (1) Wald-type t-test with small-sample corrected empirical standard error estimates, (2) Wald-type z-test with model-based standard error estimates, and (3) permutation test with test statistics calculated by the difference in average residuals for the two groups.

RESULTS

Our simulation results demonstrated that Wald-type t-tests with small-sample corrected empirical standard error estimates from the negative binomial regression model maintained proper type I error. Wald-type z-tests with model-based standard error estimates were anti-conservative. Permutation tests preserved type I error rates if the constrained space was not too small. For all tests, the power was high to detect the hypothesized 40% reduction in opioid overdose deaths for the intervention vs. comparison group both for the overall HCS and the subgroup analysis of Massachusetts (MA).

CONCLUSIONS

Based on the results of our simulation study, the Wald-type t-test with small-sample corrected empirical standard error estimates from a negative binomial regression model is a valid and appropriate approach for analyzing cluster-level count data from the HEALing Communities Study.

TRIAL REGISTRATION

ClinicalTrials.gov http://www.

CLINICALTRIALS

gov ; Identifier: NCT04111939.

Cluster randomised trials with a binary outcome and a small number of clusters: comparison of individual and cluster level analysis method

BACKGROUND

Cluster randomised trials (CRTs) are often designed with a small number of clusters, but it is not clear which analysis methods are optimal when the outcome is binary. This simulation study aimed to determine (i) whether cluster-level analysis (CL), generalised linear mixed models (GLMM), and generalised estimating equations with sandwich variance (GEE) approaches maintain acceptable type-one error including the impact of non-normality of cluster effects and low prevalence, and if so (ii) which methods have the greatest power. We simulated CRTs with 8-30 clusters, altering the cluster-size, outcome prevalence, intracluster correlation coefficient, and cluster effect distribution. We analysed each dataset with weighted and unweighted CL; GLMM with adaptive quadrature and restricted pseudolikelihood; GEE with Kauermann-and-Carroll and Fay-and-Graubard sandwich variance using independent and exchangeable working correlation matrices. P-values were from a t-distribution with degrees of freedom (DoF) as clusters minus cluster-level parameters; GLMM pseudolikelihood also used Satterthwaite and Kenward-Roger DoF.

RESULTS

Unweighted CL, GLMM pseudolikelihood, and Fay-and-Graubard GEE with independent or exchangeable working correlation matrix controlled type-one error in > 97% scenarios with clusters minus parameters DoF. Cluster-effect distribution and prevalence of outcome did not usually affect analysis method performance. GEE had the least power. With 20-30 clusters, GLMM had greater power than CL with varying cluster-size but similar power otherwise; with fewer clusters, GLMM had lower power with common cluster-size, similar power with medium variation, and greater power with large variation in cluster-size.

CONCLUSION

We recommend that CRTs with ≤ 30 clusters and a binary outcome use an unweighted CL or restricted pseudolikelihood GLMM both with DoF clusters minus cluster-level parameters.

Clustered restricted mean survival time regression

For multicenter randomized trials or multilevel observational studies, the Cox regression model has long been the primary approach to study the effects of covariates on time-to-event outcomes. A critical assumption of the Cox model is the proportionality of the hazard functions for modeled covariates, violations of which can result in ambiguous interpretations of the hazard ratio estimates. To address this issue, the restricted mean survival time (RMST), defined as the mean survival time up to a fixed time in a target population, has been recommended as a model-free target parameter. In this article, we generalize the RMST regression model to clustered data by directly modeling the RMST as a continuous function of restriction times with covariates while properly accounting for within-cluster correlations to achieve valid inference. The proposed method estimates regression coefficients via weighted generalized estimating equations, coupled with a cluster-robust sandwich variance estimator to achieve asymptotically valid inference with a sufficient number of clusters. In small-sample scenarios where a limited number of clusters are available, however, the proposed sandwich variance estimator can exhibit negative bias in capturing the variability of regression coefficient estimates. To overcome this limitation, we further propose and examine bias-corrected sandwich variance estimators to reduce the negative bias of the cluster-robust sandwich variance estimator. We study the finite-sample operating characteristics of proposed methods through simulations and reanalyze two multicenter randomized trials.

Finite-sample adjustments in variance estimators for clustered competing risks regression

The marginal Fine-Gray proportional subdistribution hazards model is a popular approach to directly study the association between covariates and the cumulative incidence function with clustered competing risks data, which often arise in multicenter randomized trials or multilevel observational studies. To account for the within-cluster correlations between failure times, the uncertainty of the regression parameters estimators is quantified by the robust sandwich variance estimator, which may have unsatisfactory performance with a limited number of clusters. To overcome this limitation, we propose four bias-corrected variance estimators to reduce the negative bias of the usual sandwich variance estimator, extending the bias-correction techniques from generalized estimating equations with noncensored exponential family outcomes to clustered competing risks outcomes. We further compare their finite-sample operating characteristics through simulations and two real data examples. In particular, we found the Mancl and DeRouen (MD) type sandwich variance estimator generally has the smallest bias. Furthermore, with a small number of clusters, the Wald t -confidence interval with the MD sandwich variance estimator carries close to nominal coverage for the cluster-level effect parameter. The t -confidence intervals based on the sandwich variance estimator with any one of the three types of multiplicative bias correction or the z -confidence interval with the Morel, Bokossa and Neerchal (MBN) type sandwich variance estimator have close to nominal coverage for the individual-level effect parameter. Finally, we develop a user-friendly R package crrcbcv implementing the proposed sandwich variance estimators to assist practical applications.

Power considerations for generalized estimating equations analyses of four-level cluster randomized trials

In this article, we develop methods for sample size and power calculations in four-level intervention studies when intervention assignment is carried out at any level, with a particular focus on cluster randomized trials (CRTs). CRTs involving four levels are becoming popular in healthcare research, where the effects are measured, for example, from evaluations (level 1) within participants (level 2) in divisions (level 3) that are nested in clusters (level 4). In such multilevel CRTs, we consider three types of intraclass correlations between different evaluations to account for such clustering: that of the same participant, that of different participants from the same division, and that of different participants from different divisions in the same cluster. Assuming arbitrary link and variance functions, with the proposed correlation structure as the true correlation structure, closed-form sample size formulas for randomization carried out at any level (including individually randomized trials within a four-level clustered structure) are derived based on the generalized estimating equations approach using the model-based variance and using the sandwich variance with an independence working correlation matrix. We demonstrate that empirical power corresponds well with that predicted by the proposed method for as few as eight clusters, when data are analyzed using the matrix-adjusted estimating equations for the correlation parameters with a bias-corrected sandwich variance estimator, under both balanced and unbalanced designs.

Randomization-based inference for a marginal treatment effect in stepped wedge cluster randomized trials

In a cross-sectional stepped wedge cluster randomized trial (SWT), clusters are randomized to crossover from control to intervention at different time periods and outcomes are assessed for a different set of individuals in each cluster-period. Randomization-based inference is an attractive analysis strategy for SWTs because it does not require full parametric specification of the outcome distribution or correlation structure and its validity does not rely on having a large number of clusters. Existing randomization-based approaches for SWTs, however, either focus on hypothesis testing and omit technical details on confidence interval (CI) calculation with noncontinuous outcomes, or employ weighted cluster-period summary statistics for p-value and CI calculation, which can result in suboptimal efficiency if weights do not incorporate information on varying cluster-period sizes. In this article, we propose a framework for calculating randomization-based p-values and CIs for a marginal treatment effect in SWTs by using test statistics derived from individual-level generalized linear models. We also investigate how study design features, such as stratified randomization, subsequently impact various SWT analysis methods including the proposed approach. Data from the XpertMTB/RIF tuberculosis trial are reanalyzed to illustrate our method and compare it to alternatives.

Sample size estimation for modified Poisson analysis of cluster randomized trials with a binary outcome

The modified Poisson regression coupled with a robust sandwich variance has become a viable alternative to log-binomial regression for estimating the marginal relative risk in cluster randomized trials. However, a corresponding sample size formula for relative risk regression via the modified Poisson model is currently not available for cluster randomized trials. Through analytical derivations, we show that there is no loss of asymptotic efficiency for estimating the marginal relative risk via the modified Poisson regression relative to the log-binomial regression. This finding holds both under the independence working correlation and under the exchangeable working correlation provided a simple modification is used to obtain the consistent intraclass correlation coefficient estimate. Therefore, the sample size formulas developed for log-binomial regression naturally apply to the modified Poisson regression in cluster randomized trials. We further extend the sample size formulas to accommodate variable cluster sizes. An extensive Monte Carlo simulation study is carried out to validate the proposed formulas. We find that the proposed formulas have satisfactory performance across a range of cluster size variability, as long as suitable finite-sample corrections are applied to the sandwich variance estimator and the number of clusters is at least 10. Our findings also suggest that the sample size estimate under the exchangeable working correlation is more robust to cluster size variability, and recommend the use of an exchangeable working correlation over an independence working correlation for both design and analysis. The proposed sample size formulas are illustrated using the Stop Colorectal Cancer (STOP CRC) trial.

Sample size and power considerations for cluster randomized trials with count outcomes subject to right truncation

Cluster randomized trials (CRTs) are widely used in epidemiological and public health studies assessing population-level effect of group-based interventions. One important application of CRTs is the control of vector-borne disease, such as malaria. However, a particular challenge for designing these trials is that the primary outcome involves counts of episodes that are subject to right truncation. While sample size formulas have been developed for CRTs with clustered counts, they are not directly applicable when the counts are right truncated. To address this limitation, we discuss two marginal modeling approaches for the analysis of CRTs with truncated counts and develop two corresponding closed-form sample size formulas to facilitate the design of such trials. The proposed sample size formulas allow investigators to explore the power under a large number of scenarios without computationally intensive simulations. The proposed formulas are validated in extensive simulations. We further explore the implication of right truncation on power and apply the proposed formulas to illustrate the power calculation for a malaria control CRT where the primary outcome is subject to right truncation.

The HEALing (Helping to End Addiction Long-term SM) Communities Study: Protocol for a cluster randomized trial at the community level to reduce opioid overdose deaths through implementation of an integrated set of evidence-based practices

BACKGROUND

Opioid overdose deaths remain high in the U.S. Despite having effective interventions to prevent overdose deaths, there are numerous barriers that impede their adoption. The primary aim of the HEALing Communities Study (HCS) is to determine the impact of an intervention consisting of community-engaged, data-driven selection, and implementation of an integrated set of evidence-based practices (EBPs) on reducing opioid overdose deaths.

METHODS

The HCS is a four year multi-site, parallel-group, cluster randomized wait-list controlled trial. Communities (n = 67) in Kentucky, Massachusetts, New York and Ohio are randomized to active intervention (Wave 1), which starts the intervention in Year 1 or the wait-list control (Wave 2), which starts the intervention in Year 3. The HCS will test a conceptually driven framework to assist communities in selecting and adopting EBPs with three components: 1) a community engagement strategy with local coalitions to guide and implement the intervention; 2) a compendium of EBPs coupled with technical assistance; and 3) a series of communication campaigns to increase awareness and demand for EBPs and reduce stigma. An implementation science framework guides the intervention and allows for examination of the multilevel contexts that promote or impede adoption and expansion of EBPs. The primary outcome, number of opioid overdose deaths, will be compared between Wave 1 and Wave 2 communities during Year 2 of the intervention for Wave 1. Numerous secondary outcomes will be examined.

DISCUSSION

The HCS is the largest community-based implementation study in the field of addiction with an ambitious goal of significantly reducing fatal opioid overdoses.

Systematic review of endoscopic-assisted surgical management for congenital nasolacrimal duct obstruction

OBJECTIVES

Epiphora is a common presenting complaint in infants affecting up to 6% of infants in the United States. It is most frequently due to congenital anomalies of the nasolacrimal duct system, termed congenital nasolacrimal duct obstruction (CNLDO). Nasolacrimal duct probing is widely accepted as the primary surgical intervention in cases that fail conservative management. Recently, nasal endoscopy has been combined with traditional probing to improve success rates and outcomes. Several studies have been conducted, but the results have been inconclusive overall. The goal of this systematic review is to examine the existing literature on the role of nasal endoscopy during nasolacrimal duct probing and evaluate patient outcomes.

METHODS

A systematic search was performed in PubMed, MEDLINE, The Cochrane Library, ClinicalTrials.gov, LILAC, and EMBASE to identify peer-reviewed research. Eligible studies were those containing original peer-reviewed research in English addressing nasolacrimal duct probing for congenital nasolacrimal duct obstruction in conjunction with nasal endoscopy. Two investigators independently reviewed all articles and extracted data. Bias was assessed using the Cochrane Collaboration's tool, the Newcastle-Ottawa Assessment scale, and the NIH Quality Assessment scale.

RESULTS

A literature review ultimately yielded 11 articles that were selected and included in our analysis. Of the seven case series, the success rate was 92.5% for a total of 492 eyes. Of the four cohort/case-control studies, the success rate in conventional probings was 75.3% for a total of 293 eyes. In probings with nasal endoscopy, the success rate was 95.3% for 162 eyes. Overall, there was a high risk of bias and variation in data reporting amongst studies.

CONCLUSION

Infants with congenital nasolacrimal duct obstruction that does not resolve with conservative treatment often require surgical intervention. Based on this systematic review of the current literature, nasal endoscopy is a useful adjunct for nasolacrimal duct probing.

Assessment of triphenyl phosphate (TPhP) exposure to nail salon workers by air, hand wipe, and urine analysis

Triphenyl phosphate (TPP or TPhP) is commonly used as an additive plasticizer or organophosphate flame retardant (OPFR) in consumer products including nail polish. We evaluated exposure to TPhP from 12 nail salon technicians working at four nail salons located in California over a period of two work days. Bulk samples of 15 nail polishes and other nail products were collected. Study participants also provided two personal air samples, two hand wipe samples (pre- and post-shift on day two), and two urine samples (pre-shift day one and post-shift day two). The geometric mean (GM) of TPhP air sampling concentrations was 7.39 ng/m³. Post-shift TPhP hand wipe concentrations (GM 1.35 μg/sample) were significantly higher (p = 0.024) than pre-shift hand wipe concentrations (GM 0.29 μg/sample). Diphenyl phosphate (DPP or DPhP), a urinary metabolite of TPhP used in this study as a biomarker of exposure, was detected in all post-shift urine samples and 75% of urine pre-shift samples. DPhP post-shift concentrations (GM 1.35 μg/g creatinine) were significantly higher than pre-shift concentrations (GM 0.84 μg/g creatinine; p = 0.012). In addition, DPhP post-shift concentrations were correlated with TPhP post-shift hand wipe concentrations, suggesting dermal contact may be a relevant exposure pathway for nail salon workers.

Maintaining the validity of inference in small-sample stepped wedge cluster randomized trials with binary outcomes when using generalized estimating equations

Stepped wedge cluster trials are an increasingly popular alternative to traditional parallel cluster randomized trials. Such trials often utilize a small number of clusters and numerous time intervals, and these components must be considered when choosing an analysis method. A generalized linear mixed model containing a random intercept and fixed time and intervention covariates is the most common analysis approach. However, the sole use of a random intercept applies a constant intraclass correlation coefficient structure, which is an assumption that is likely to be violated given stepped wedge trials (SWTs) have multiple time intervals. Alternatively, generalized estimating equations (GEE) are robust to the misspecification of the working correlation structure, although it has been shown that small-sample adjustments to standard error estimates and the use of appropriate degrees of freedom are required to maintain the validity of inference when the number of clusters is small. In this article, we show, using an extensive simulation study based on a motivating example and a more general design, the use of GEE can maintain the validity of inference in small-sample SWTs with binary outcomes. Furthermore, we show which combinations of bias corrections to standard error estimates and degrees of freedom work best in terms of attaining nominal type I error rates.

xtgeebcv: A command for bias-corrected sandwich variance estimation for GEE analyses of cluster randomized trials

Cluster randomized trials, where clusters (for example, schools or clinics) are randomized to comparison arms but measurements are taken on individuals, are commonly used to evaluate interventions in public health, education, and the social sciences. Analysis is often conducted on individual-level outcomes, and such analysis methods must consider that outcomes for members of the same cluster tend to be more similar than outcomes for members of other clusters. A popular individual-level analysis technique is generalized estimating equations (GEE). However, it is common to randomize a small number of clusters (for example, 30 or fewer), and in this case, the GEE standard errors obtained from the sandwich variance estimator will be biased, leading to inflated type I errors. Some bias-corrected standard errors have been proposed and studied to account for this finite-sample bias, but none has yet been implemented in Stata. In this article, we describe several popular bias corrections to the robust sandwich variance. We then introduce our newly created command, xtgeebcv, which will allow Stata users to easily apply finite-sample corrections to standard errors obtained from GEE models. We then provide examples to demonstrate the use of xtgeebcv. Finally, we discuss suggestions about which finite-sample corrections to use in which situations and consider areas of future research that may improve xtgeebcv.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Design and analysis considerations for cohort stepped wedge cluster randomized trials with a decay correlation structure

A stepped wedge cluster randomized trial is a type of longitudinal cluster design that sequentially switches clusters to intervention over time until all clusters are treated. While the traditional posttest-only parallel design requires adjustment for a single intraclass correlation coefficient, the stepped wedge design allows multiple outcome measurements from the same cluster and so additional correlation parameters are necessary to characterize the within-cluster correlation structure. Although a number of studies have differentiated between the concepts of within-period and between-period correlations, only a few studies have allowed the between-period correlation to decay over time. In this article, we consider the proportional decay correlation structure for a cohort stepped wedge design, and provide a matrix-adjusted quasi-least squares approach to accurately estimate the correlation parameters along with the marginal intervention effect. We further develop the sample size and power procedures accounting for the correlation decay, and investigate the accuracy of the power procedure with continuous outcomes in a simulation study. We show that the empirical power agrees well with the prediction even with as few as nine clusters, when data are analyzed with matrix-adjusted quasi-least squares concurrently with a suitable bias-corrected sandwich variance. Two trial examples are provided to illustrate the new sample size procedure.

Effect Partitioning in Cross-Sectionally Clustered Data Without Multilevel Models

Effect partitioning is almost exclusively performed with multilevel models (MLMs) - so much so that some have considered the two to be synonymous. MLMs are able to provide estimates with desirable statistical properties when data come from a hierarchical structure; but the random effects included in MLMs are not always integral to the analysis. As a result, other methods with relaxed assumptions are viable options in many cases. Through empirical examples and simulations, we show how generalized estimating equations (GEEs) can be used to effectively partition effects without random effects. We show that more onerous steps of MLMs such as determining the number of random effects and the structure for their covariance can be bypassed with GEEs while still obtaining identical or near-identical results. Additionally, violations of distributional assumptions adversely affect estimates with MLMs but have no effect on GEEs because no such assumptions are made. This makes GEEs a flexible alternative to MLMs with minimal assumptions that may warrant consideration. Limitations of GEEs for partitioning effects are also discussed.

A readily available improvement over method of moments for intra-cluster correlation estimation in the context of cluster randomized trials and fitting a GEE-type marginal model for binary outcomes

BACKGROUND/AIMS

Cluster randomized trials are popular in health-related research due to the need or desire to randomize clusters of subjects to different trial arms as opposed to randomizing each subject individually. As outcomes from subjects within the same cluster tend to be more alike than outcomes from subjects within other clusters, an exchangeable correlation arises that is measured via the intra-cluster correlation coefficient. Intra-cluster correlation coefficient estimation is especially important due to the increasing awareness of the need to publish such values from studies in order to help guide the design of future cluster randomized trials. Therefore, numerous methods have been proposed to accurately estimate the intra-cluster correlation coefficient, with much attention given to binary outcomes. As marginal models are often of interest, we focus on intra-cluster correlation coefficient estimation in the context of fitting such a model with binary outcomes using generalized estimating equations. Traditionally, intra-cluster correlation coefficient estimation with generalized estimating equations has been based on the method of moments, although such estimators can be negatively biased. Furthermore, alternative estimators that work well, such as the analysis of variance estimator, are not as readily applicable in the context of practical data analyses with generalized estimating equations. Therefore, in this article we assess, in terms of bias, the readily available residual pseudo-likelihood approach to intra-cluster correlation coefficient estimation with the GLIMMIX procedure of SAS (SAS Institute, Cary, NC). Furthermore, we study a possible corresponding approach to confidence interval construction for the intra-cluster correlation coefficient.

METHODS

We utilize a simulation study and application example to assess bias in intra-cluster correlation coefficient estimates obtained from GLIMMIX using residual pseudo-likelihood. This estimator is contrasted with method of moments and analysis of variance estimators which are standards of comparison. The approach to confidence interval construction is assessed by examining coverage probabilities.

RESULTS

Overall, the residual pseudo-likelihood estimator performs very well. It has considerably less bias than moment estimators, which are its competitor for general generalized estimating equation-based analyses, and therefore, it is a major improvement in practice. Furthermore, it works almost as well as analysis of variance estimators when they are applicable. Confidence intervals have near-nominal coverage when the intra-cluster correlation coefficient estimate has negligible bias.

CONCLUSION

Our results show that the residual pseudo-likelihood estimator is a good option for intra-cluster correlation coefficient estimation when conducting a generalized estimating equation-based analysis of binary outcome data arising from cluster randomized trials. The estimator is practical in that it is simply a result from fitting a marginal model with GLIMMIX, and a confidence interval can be easily obtained. An additional advantage is that, unlike most other options for performing generalized estimating equation-based analyses, GLIMMIX provides analysts the option to utilize small-sample adjustments that ensure valid inference.

Design and analysis of group-randomized trials in cancer: A review of current practices

The purpose of this paper is to summarize current practices for the design and analysis of group-randomized trials involving cancer-related risk factors or outcomes and to offer recommendations to improve future trials. We searched for group-randomized trials involving cancer-related risk factors or outcomes that were published or online in peer-reviewed journals in 2011-15. During 2016-17, in Bethesda MD, we reviewed 123 articles from 76 journals to characterize their design and their methods for sample size estimation and data analysis. Only 66 (53.7%) of the articles reported appropriate methods for sample size estimation. Only 63 (51.2%) reported exclusively appropriate methods for analysis. These findings suggest that many investigators do not adequately attend to the methodological challenges inherent in group-randomized trials. These practices can lead to underpowered studies, to an inflated type 1 error rate, and to inferences that mislead readers. Investigators should work with biostatisticians or other methodologists familiar with these issues. Funders and editors should ensure careful methodological review of applications and manuscripts. Reviewers should ensure that studies are properly planned and analyzed. These steps are needed to improve the rigor and reproducibility of group-randomized trials. The Office of Disease Prevention (ODP) at the National Institutes of Health (NIH) has taken several steps to address these issues. ODP offers an online course on the design and analysis of group-randomized trials. ODP is working to increase the number of methodologists who serve on grant review panels. ODP has developed standard language for the Application Guide and the Review Criteria to draw investigators' attention to these issues. Finally, ODP has created a new Research Methods Resources website to help investigators, reviewers, and NIH staff better understand these issues.

An evaluation of constrained randomization for the design and analysis of group-randomized trials with binary outcomes

Group-randomized trials are randomized studies that allocate intact groups of individuals to different comparison arms. A frequent practical limitation to adopting such research designs is that only a limited number of groups may be available, and therefore, simple randomization is unable to adequately balance multiple group-level covariates between arms. Therefore, covariate-based constrained randomization was proposed as an allocation technique to achieve balance. Constrained randomization involves generating a large number of possible allocation schemes, calculating a balance score that assesses covariate imbalance, limiting the randomization space to a prespecified percentage of candidate allocations, and randomly selecting one scheme to implement. When the outcome is binary, a number of statistical issues arise regarding the potential advantages of such designs in making inference. In particular, properties found for continuous outcomes may not directly apply, and additional variations on statistical tests are available. Motivated by two recent trials, we conduct a series of Monte Carlo simulations to evaluate the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs, with varying degrees of analysis-based covariate adjustment. Our results indicate that constrained randomization improves the power of the linearization F-test, the KC-corrected GEE t-test (Kauermann and Carroll, 2001, Journal of the American Statistical Association 96, 1387-1396), and two permutation tests when the prognostic group-level variables are controlled for in the analysis and the size of randomization space is reasonably small. We also demonstrate that constrained randomization reduces power loss from redundant analysis-based adjustment for non-prognostic covariates. Design considerations such as the choice of the balance metric and the size of randomization space are discussed.