Using machine learning to advance disparities research: Subgroup analyses of access to opioid treatment

Identifying the Heterogeneity in the Association between Workforce Diversity and Retention in Opioid Treatment among Black clients

Background

This study investigates the impact of workforce diversity, specifically staff identified as Black/African American, on retention in opioid use disorder (OUD) treatment, aiming to enhance patient outcomes. Employing a novel machine learning technique known as 'causal forest,' we explore heterogeneous treatment effects on retention.

Methods

We relied on four waves of the National Drug Abuse Treatment System Survey (NDATSS), a nationally representative longitudinal dataset of treatment programs. We analyzed OUD program data from the years 2000, 2005, 2014 and 2017 (n = 627). Employing the 'causal forest' method, we analyzed the heterogeneity in the relationship between workforce diversity and retention in OUD treatment. Interviews with program directors and clinical supervisors provided the data for this study.

Results

The results reveal diversity-related variations in the association with retention across 61 out of 627 OUD treatment programs (less than 10%). These programs, associated with positive impacts of workforce diversity, were more likely private-for-profit, newer, had lower percentages of Black and Latino clients, lower staff-to-client ratios, higher proportions of staff with graduate degrees, and lower percentages of unemployed clients.

Conclusions

While workforce diversity is crucial, our findings underscore that it alone is insufficient for improving retention in addiction health services research. Programs with characteristics typically linked to positive outcomes are better positioned to maximize the benefits of a diverse workforce in client retention. This research has implications for policy and program design, guiding decisions on resource allocation and workforce diversity to enhance retention rates among Black clients with OUDs.

A Tutorial for Propensity Score Weighting for Moderation Analysis With Categorical Variables: An Application Examining Smoking Disparities Among Sexual Minority Adults

OBJECTIVE

To provide step-by-step guidance and STATA and R code for using propensity score (PS) weighting to estimate moderation effects with categorical variables.

RESEARCH DESIGN

Tutorial illustrating the key steps for estimating and testing moderation using observational data. Steps include: (1) examining covariate overlap across treatment groups within levels of the moderator; (2) estimating the PS weights; (3) evaluating whether PS weights improved covariate balance; (4) estimating moderated treatment effects; and (5) assessing the sensitivity of findings to unobserved confounding. Our illustrative case study uses data from 41,832 adults from the 2019 National Survey on Drug Use and Health to examine if gender moderates the association between sexual minority status (eg, lesbian, gay, or bisexual [LGB] identity) and adult smoking prevalence.

RESULTS

For our case study, there were no noted concerns about covariate overlap, and we were able to successfully estimate the PS weights within each level of the moderator. Moreover, balance criteria indicated that PS weights successfully achieved covariate balance for both moderator groups. PS-weighted results indicated there was significant evidence of moderation for the case study, and sensitivity analyses demonstrated that results were highly robust for one level of the moderator but not the other.

CONCLUSIONS

When conducting moderation analyses, covariate imbalances across levels of the moderator can cause biased estimates. As demonstrated in this tutorial, PS weighting within each level of the moderator can improve the estimated moderation effects by minimizing bias from imbalance within the moderator subgroups.

Determinants of outpatient substance use disorder treatment length-of-stay and completion: the case of a treatment program in the southeast U.S

Successful outcomes of outpatient substance use disorder treatment result from many factors for clients-including intersections between individual characteristics, choices made, and social determinants. However, prioritizing which of these and in what combination, to address and provide support for remains an open and complex question. Therefore, we ask: What factors are associated with outpatient substance use disorder clients remaining in treatment for > 90 days and successfully completing treatment? To answer this question, we apply a virtual twins machine learning (ML) model to de-identified data for a census of clients who received outpatient substance use disorder treatment services from 2018 to 2021 from one treatment program in the Southeast U.S. We find that primary predictors of outcome success are: (1) attending self-help groups while in treatment, and (2) setting goals for treatment. Secondary predictors are: (1) being linked to a primary care provider (PCP) during treatment, (2) being linked to supplemental nutrition assistance program (SNAP), and (3) attending 6 or more self-help group sessions during treatment. These findings can help treatment programs guide client choice making and help set priorities for social determinant support. Further, the ML method applied can explain intersections between individual and social predictors, as well as outcome heterogeneity associated with subgroup differences.

Development and validation of explainable machine-learning models for carotid atherosclerosis early screening

BACKGROUND

Carotid atherosclerosis (CAS), an important factor in the development of stroke, is a major public health concern. The aim of this study was to establish and validate machine learning (ML) models for early screening of CAS using routine health check-up indicators in northeast China.

METHODS

A total of 69,601 health check-up records from the health examination center of the First Hospital of China Medical University (Shenyang, China) were collected between 2018 and 2019. For the 2019 records, 80% were assigned to the training set and 20% to the testing set. The 2018 records were used as the external validation dataset. Ten ML algorithms, including decision tree (DT), K-nearest neighbors (KNN), logistic regression (LR), naive Bayes (NB), random forest (RF), multiplayer perceptron (MLP), extreme gradient boosting machine (XGB), gradient boosting decision tree (GBDT), linear support vector machine (SVM-linear), and non-linear support vector machine (SVM-nonlinear), were used to construct CAS screening models. The area under the receiver operating characteristic curve (auROC) and precision-recall curve (auPR) were used as measures of model performance. The SHapley Additive exPlanations (SHAP) method was used to demonstrate the interpretability of the optimal model.

RESULTS

A total of 6315 records of patients undergoing carotid ultrasonography were collected; of these, 1632, 407, and 1141 patients were diagnosed with CAS in the training, internal validation, and external validation datasets, respectively. The GBDT model achieved the highest performance metrics with auROC of 0.860 (95% CI 0.839-0.880) in the internal validation dataset and 0.851 (95% CI 0.837-0.863) in the external validation dataset. Individuals with diabetes or those over 65 years of age showed low negative predictive value. In the interpretability analysis, age was the most important factor influencing the performance of the GBDT model, followed by sex and non-high-density lipoprotein cholesterol.

CONCLUSIONS

The ML models developed could provide good performance for CAS identification using routine health check-up indicators and could hopefully be applied in scenarios without ethnic and geographic heterogeneity for CAS prevention.

Predicting Patient Wait Times by Using Highly Deidentified Data in Mental Health Care: Enhanced Machine Learning Approach

BACKGROUND

Wait times impact patient satisfaction, treatment effectiveness, and the efficiency of care that the patients receive. Wait time prediction in mental health is a complex task and is affected by the difficulty in predicting the required number of treatment sessions for outpatients, high no-show rates, and the possibility of using group treatment sessions. The task of wait time analysis becomes even more challenging if the input data has low utility, which happens when the data is highly deidentified by removing both direct and quasi identifiers.

OBJECTIVE

The first aim of this study was to develop machine learning models to predict the wait time from referral to the first appointment for psychiatric outpatients by using real-time data. The second aim was to enhance the performance of these predictive models by utilizing the system's knowledge while the input data were highly deidentified. The third aim was to identify the factors that drove long wait times, and the fourth aim was to build these models such that they were practical and easy-to-implement (and therefore, attractive to care providers).

METHODS

We analyzed retrospective highly deidentified administrative data from 8 outpatient clinics at Ontario Shores Centre for Mental Health Sciences in Canada by using 6 machine learning methods to predict the first appointment wait time for new outpatients. We used the system's knowledge to mitigate the low utility of our data. The data included 4187 patients who received care through 30,342 appointments.

RESULTS

The average wait time varied widely between different types of mental health clinics. For more than half of the clinics, the average wait time was longer than 3 months. The number of scheduled appointments and the rate of no-shows varied widely among clinics. Despite these variations, the random forest method provided the minimum root mean square error values for 4 of the 8 clinics, and the second minimum root mean square error for the other 4 clinics. Utilizing the system's knowledge increased the utility of our highly deidentified data and improved the predictive power of the models.

CONCLUSIONS

The random forest method, enhanced with the system's knowledge, provided reliable wait time predictions for new outpatients, regardless of low utility of the highly deidentified input data and the high variation in wait times across different clinics and patient types. The priority system was identified as a factor that contributed to long wait times, and a fast-track system was suggested as a potential solution.