Using administrative data to predict cessation risk and identify novel predictors among new entrants to opioid agonist treatment

Predictability of buprenorphine-naloxone treatment retention: A multi-site analysis combining electronic health records and machine learning

BACKGROUND AND AIMS

Opioid use disorder (OUD) and opioid dependence lead to significant morbidity and mortality, yet treatment retention, crucial for the effectiveness of medications like buprenorphine-naloxone, remains unpredictable. Our objective was to determine the predictability of 6-month retention in buprenorphine-naloxone treatment using electronic health record (EHR) data from diverse clinical settings and to identify key predictors.

DESIGN

This retrospective observational study developed and validated machine learning-based clinical risk prediction models using EHR data.

SETTING AND CASES

Data were sourced from Stanford University's healthcare system and Holmusk's NeuroBlu database, reflecting a wide range of healthcare settings. The study analyzed 1800 Stanford and 7957 NeuroBlu treatment encounters from 2008 to 2023 and from 2003 to 2023, respectively.

MEASUREMENTS

Predict continuous prescription of buprenorphine-naloxone for at least 6 months, without a gap of more than 30 days. The performance of machine learning prediction models was assessed by area under receiver operating characteristic (ROC-AUC) analysis as well as precision, recall and calibration. To further validate our approach's clinical applicability, we conducted two secondary analyses: a time-to-event analysis on a single site to estimate the duration of buprenorphine-naloxone treatment continuity evaluated by the C-index and a comparative evaluation against predictions made by three human clinical experts.

FINDINGS

Attrition rates at 6 months were 58% (NeuroBlu) and 61% (Stanford). Prediction models trained and internally validated on NeuroBlu data achieved ROC-AUCs up to 75.8 (95% confidence interval [CI] = 73.6-78.0). Addiction medicine specialists' predictions show a ROC-AUC of 67.8 (95% CI = 50.4-85.2). Time-to-event analysis on Stanford data indicated a median treatment retention time of 65 days, with random survival forest model achieving an average C-index of 65.9. The top predictor of treatment retention identified included the diagnosis of opioid dependence.

CONCLUSIONS

US patients with opioid use disorder or opioid dependence treated with buprenorphine-naloxone prescriptions appear to have a high (∼60%) treatment attrition by 6 months. Machine learning models trained on diverse electronic health record datasets appear to be able to predict treatment continuity with accuracy comparable to that of clinical experts.

Estimating effects of time-varying exposures on mortality risk

Administrative databases have become an increasingly popular data source for population-based health research. We explore how mortality risk is associated with some health service utilization process via linked administrative data. A generalized Cox regression model is proposed using a time-dependent stratification variable to summarize lifetime service utilization. Recognizing the service utilization over time as an internal covariate in the survival analysis, conventional likelihood methods are inapplicable. We present an estimating function based procedure for estimating model parameters, and provide a testing procedure for updating the stratification levels. The proposed approach is examined both asymptotically and numerically via simulation. We motivate and illustrate the proposed approach using an on-going program pertaining to opioid agonist treatment (OAT) management for individuals identified with opioid use disorders. Our analysis of the OAT data indicates that the OAT effect on mortality risk decreases in successive OAT attempts, in which two risk classes based on an individual's treatment episode number are established: one with 1-3 OAT episodes, and the other with 4+ OAT episodes.

Optimizing Digital Tools for the Field of Substance Use and Substance Use Disorders: Backcasting Exercise

BACKGROUND

Substance use trends are complex; they often rapidly evolve and necessitate an intersectional approach in research, service, and policy making. Current and emerging digital tools related to substance use are promising but also create a range of challenges and opportunities.

OBJECTIVE

This paper reports on a backcasting exercise aimed at the development of a roadmap that identifies values, challenges, facilitators, and milestones to achieve optimal use of digital tools in the substance use field by 2030.

METHODS

A backcasting exercise method was adopted, wherein the core elements are identifying key values, challenges, facilitators, milestones, cornerstones and a current, desired, and future scenario. A structured approach was used by means of (1) an Open Science Framework page as a web-based collaborative working space and (2) key stakeholders' collaborative engagement during the 2022 Lisbon Addiction Conference.

RESULTS

The identified key values were digital rights, evidence-based tools, user-friendliness, accessibility and availability, and person-centeredness. The key challenges identified were ethical funding, regulations, commercialization, best practice models, digital literacy, and access or reach. The key facilitators identified were scientific research, interoperable infrastructure and a culture of innovation, expertise, ethical funding, user-friendly designs, and digital rights and regulations. A range of milestones were identified. The overarching identified cornerstones consisted of creating ethical frameworks, increasing access to digital tools, and continuous trend analysis.

CONCLUSIONS

The use of digital tools in the field of substance use is linked to a range of risks and opportunities that need to be managed. The current trajectories of the use of such tools are heavily influenced by large multinational for-profit companies with relatively little involvement of key stakeholders such as people who use drugs, service providers, and researchers. The current funding models are problematic and lack the necessary flexibility associated with best practice business approaches such as lean and agile principles to design and execute customer discovery methods. Accessibility and availability, digital rights, user-friendly design, and person-focused approaches should be at the forefront in the further development of digital tools. Global legislative and technical infrastructures by means of a global action plan and strategy are necessary and should include ethical frameworks, accessibility of digital tools for substance use, and continuous trend analysis as cornerstones.

Using Machine Learning to Predict Treatment Adherence in Patients on Medication for Opioid Use Disorder

OBJECTIVE

Patients receiving medication for opioid use disorder (MOUD) may continue using nonprescribed drugs or have trouble with medication adherence, and it is difficult to predict which patients will continue to do so. In this study, we develop and validate an automated risk-modeling framework to predict opioid abstinence and medication adherence at a patient's next attended appointment and evaluate the predictive performance of machine-learning algorithms versus logistic regression.

METHODS

Urine drug screen and attendance records from 40,005 appointments drawn from 2742 patients at a multilocation office-based MOUD program were used to train logistic regression, logistic ridge regression, and XGBoost models to predict a composite indicator of treatment adherence (opioid-negative and norbuprenorphine-positive urine, no evidence of urine adulteration) at next attended appointment.

RESULTS

The XGBoost model had similar accuracy and discriminative ability (accuracy, 88%; area under the receiver operating curve, 0.87) to the two logistic regression models (accuracy, 88%; area under the receiver operating curve, 0.87). The XGBoost model had nearly perfect calibration in independent validation data; the logistic and ridge regression models slightly overestimated adherence likelihood. Historical treatment adherence, attendance rate, and fentanyl-positive urine at current appointment were the strongest contributors to treatment adherence at next attended appointment.

DISCUSSION

There is a need for risk prediction tools to improve delivery of MOUD. This study presents an automated and portable risk-modeling framework to predict treatment adherence at each patient's next attended appointment. The XGBoost algorithm appears to provide similar classification accuracy to logistic regression models; however, XGBoost may offer improved calibration of risk estimates compared with logistic regression.

Measuring treatment attrition at various stages of engagement in Opioid Agonist Treatment in Ontario Canada using a cascade of care framework

Background

The cascade of care framework is an effective way to measure attrition at various stages of engagement in Opioid Agonist Treatment (OAT). The primary objective of the study was to describe the cascade of care for patients who have accessed OAT from a network of specialized addiction clinics in Ontario, Canada. The secondary objectives were to evaluate correlates associated with retention in OAT at various stages and the impact of patients’ location of the residence on retention in OAT.

Design

A multi-clinic retrospective cohort study was conducted using electronic medical record (EMR) data from the largest network of OAT clinics in Canada (70 clinics) from 2014 to 2020. Study participants included all patients who received OAT from the network of clinics during the study period.

Measurements

In this study, four stages of the cascade of care framework were operationalized to identify treatment engagement patterns, including patients retained within 90 days, 90 to 365 days, one to 2 years, and more than 2 years. Correlates associated with OAT retention for 90 days, 90 to 365 days, 1 to 2 years, and more than 2 years were also evaluated and compared across rural and urban areas in northern and southern Ontario.

Results

A total of 32,487 patients were included in the study. Compared to patients who were retained in OAT for 90 days, patients who were retained for 90 to 365 days, 1 to 2 years, or more than 2 years were more likely to have a higher number of treatment attempts, a higher number of average monthly urine drug screening and a lower proportion of positive urine drug screening results for other drug use.

Conclusion

Distinct sociodemographic and clinical factors are likely to influence treatment retention at various stages of engagement along the OAT continuum. Research is required to determine if tailored strategies specific to people at different stages of retention have the potential to improve outcomes of OAT.