Predicting patients who will drop out of out-patient psychotherapy using machine learning algorithms

Experience in psychological counseling supported by artificial intelligence technology

BACKGROUND

In recent years, artificial intelligence (AI) technology has been continuously advancing and finding extensive applications, with one of its core technologies, machine learning, being increasingly utilized in the field of healthcare.

OBJECTIVE

This research aims to explore the role of Artificial Intelligence (AI) technology in psychological counseling and utilize machine learning algorithms to predict counseling outcomes.

METHODS

Firstly, by employing natural language processing techniques to analyze user conversations with AI chatbots, researchers can gain insights into the psychological states and needs of users during the counseling process. This involves detailed analysis using text analysis, sentiment analysis, and other relevant techniques. Subsequently, machine learning algorithms are used to establish predictive models that forecast counseling outcomes and user satisfaction based on data such as user language, emotions, and behavior. These predictive results can assist counselors or AI chatbots in adjusting counseling strategies, thereby enhancing counseling effectiveness and user experience. Additionally, this study explores the potential and prospects of AI technology in the field of psychological counseling.

RESULTS

The research findings indicate that the designed machine learning models achieve an accuracy rate of approximately 89% in analyzing psychological conditions. This demonstrates significant innovation and breakthroughs in AI technology. Consequently, AI technology will gradually become a highly important tool and method in the field of psychological counseling.

CONCLUSION

In the future, AI chatbots will become more intelligent and personalized, providing users with precise, efficient, and convenient psychological counseling services. The results of this research provide valuable technical insights for further improving AI-supported psychological counseling, contributing positively to the application and development of AI technology.

Leveraging Novel Technologies and Artificial Intelligence to Advance Practice-Oriented Research

Mental health services are experiencing notable transformations as innovative technologies and artificial intelligence (AI) are increasingly utilized in a growing number of studies and services.These cutting-edge technologies carry the promise of substantial improvements in the field of mental health. Nevertheless, questions emerge about the alignment of novel technologies and AI systems with human needs, especially in the context of vulnerable populations receiving mental healthcare. The practice-oriented research (POR) model is pivotal in seamlessly integrating these emerging technologies into clinical research and practice. It underscores the importance of tight collaboration between clinicians and researchers, all driven by the central goal of ensuring and elevating client well-being. This paper focuses on how novel technologies can enhance the POR model and highlights its pivotal role in integrating these technologies into clinical research and practice. We discuss two key phases: pre-treatment, and during treatment. For each phase, we describe the challenges, present the major technological innovations, describe recent studies exemplifying technology use, and suggest future directions. Ethical concerns and the importance of aligning humans and technology are also considered, in addition to implications for practice and training.

Using classification trees to identify psychotherapy patients at risk for poor treatment adherence

To determine the relative importance of a wide variety of personality and psychopathology variables in influencing patients' adherence to psychotherapy treatment.

Two classification trees were trained to predict patients' (1) treatment utilization (i.e., their likelihood of missing a given appointment) and (2) termination status (i.e., their likelihood of dropping out of therapy prematurely). Each tree was then validated in an external dataset to examine performance accuracy.

Patients' social detachment was most influential in predicting their treatment utilization, followed by affective instability and activity/energy levels. Patients' interpersonal warmth was most influential in predicting their termination status, followed by levels of disordered thought and resentment. The overall accuracy rating for the tree for termination status was 71.4%, while the tree for treatment utilization had a 38.7% accuracy rating.

Classification trees are a practical tool for clinicians to determine patients at risk of premature termination. More research is needed to develop trees that predict treatment utilization with high accuracy across different types of patients and settings.

What if we wait? Using synthetic waiting lists to estimate treatment effects in routine outcome data

Objective: Due to the lack of randomization, pre-post routine outcome data precludes causal conclusions. We propose the "synthetic waiting list" (SWL) control group to overcome this limitation.

Method: First, a step-by-step introduction illustrates this novel approach. Then, this approach is demonstrated using an empirical example with data from an outpatient cognitive-behavioral therapy (CBT) clinic (N = 139). We trained an ensemble machine learning model ("Super Learner") on a data set of patients waiting for treatment (N = 311) to make counterfactual predictions of symptom change during this hypothetical period.

Results: The between-group treatment effect was estimated to be d = 0.42. Of the patients who received CBT, 43.88% achieved reliable and clinically significant change, while this probability was estimated to be 14.54% in the SWL group. Counterfactual estimates suggest a clear net benefit of psychotherapy for 41% of patients. In 32%, the benefit was unclear, and 27% would have improved similarly without receiving CBT.

Conclusions: The SWL is a viable new approach that provides between-group outcome estimates similar to those reported in the literature comparing psychotherapy with high-intensity control interventions. It holds the potential to mitigate common limitations of routine outcome data analysis.

Predicting Youth and Young Adult Treatment Engagement in a Transdiagnostic Remote Intensive Outpatient Program: Latent Profile Analysis

BACKGROUND

The youth mental health crisis in the United States continues to worsen, and research has shown poor mental health treatment engagement. Despite the need for personalized engagement strategies, there is a lack of research involving youth. Due to complex youth developmental milestones, there is a need to better understand clinical presentation and factors associated with treatment engagement to effectively identify and tailor beneficial treatments.

OBJECTIVE

This quality improvement investigation sought to identify subgroups of clients attending a remote intensive outpatient program (IOP) based on clinical acuity data at intake, to determine the factors associated with engagement outcomes for clients who present in complex developmental periods and with cooccurring conditions. The identification of these subgroups was used to inform programmatic decisions within this remote IOP system.

METHODS

Data were collected as part of ongoing quality improvement initiatives at a remote IOP for youth and young adults. Participants included clients (N=2924) discharged between July 2021 and February 2023. A latent profile analysis was conducted using 5 indicators of clinical acuity at treatment entry, and the resulting profiles were assessed for associations with demographic factors and treatment engagement outcomes.

RESULTS

Among the 2924 participants, 4 profiles of clinical acuity were identified: a low-acuity profile (n=943, 32.25%), characterized by minimal anxiety, depression, and self-harm, and 3 high-acuity profiles defined by moderately severe depression and anxiety but differentiated by rates of self-harm (high acuity+low self-harm: n=1452, 49.66%; high acuity+moderate self-harm: n=203, 6.94%; high acuity+high self-harm: n=326, 11.15%). Age, gender, transgender identity, and sexual orientation were significantly associated with profile membership. Clients identified as sexually and gender-marginalized populations were more likely to be classified into high-acuity profiles than into the low-acuity profile (eg, for clients who identified as transgender, high acuity+low self-harm: odds ratio [OR] 2.07, 95% CI 1.35-3.18; P<.001; high acuity+moderate self-harm: OR 2.85, 95% CI 1.66-4.90; P<.001; high acuity+high self-harm: OR 3.67, 95% CI 2.45-5.51; P<.001). Race was unrelated to the profile membership. Profile membership was significantly associated with treatment engagement: youth and young adults in the low-acuity and high-acuity+low-self-harm profiles attended an average of 4 fewer treatment sessions compared with youth in the high-acuity+moderate-self-harm and high-acuity+high-self-harm profiles (ꭓ23=27.6, P<.001). Individuals in the high-acuity+low-self-harm profile completed treatment at a significantly lower rate relative to the other 2 high-acuity profiles (ꭓ23=13.4, P=.004). Finally, those in the high-acuity+high-self-harm profile were significantly less likely to disengage early relative to youth in all other profiles (ꭓ23=71.12, P<.001).

CONCLUSIONS

This investigation represents a novel application for identifying subgroups of adolescents and young adults based on clinical acuity data at intake to identify patterns in treatment engagement outcomes. Identifying subgroups that differentially engage in treatment is a critical first step toward targeting engagement strategies for complex populations.

Unwanted outcomes in cognitive behavior therapy for pathological health anxiety: a systematic review and a secondary original study of two randomized controlled trials

INTRODUCTION

Cognitive behavior therapy (CBT) is effective for pathological health anxiety, but little is known about unwanted outcomes.

AREAS COVERED

We investigated unwanted outcomes in the form of adverse events, overall symptom deterioration, and dropouts in CBT for pathological health anxiety based on a systematic review of 19 randomized controlled trials (PubMed, PsycInfo, and OATD; last updated 2 June 2023; pooled N = 2188), and then a secondary original study of two randomized controlled trials (pooled N = 336). In the systematic review, 10% of participants in CBT reported at least one adverse event and 17% dropped out. Heterogeneity was substantial. In the original investigation, 17% reported at least one adverse event, 0-10% met criteria for overall symptom deterioration, and 10-19% dropped out. In guided Internet-delivered CBT, dropouts were more common with lower education and lower credibility/expectancy ratings. Higher adherence was associated with a larger reduction in health anxiety.

EXPERT OPINION

Unwanted effects are routinely seen in CBT for pathological health anxiety, but, under typical circumstances, appear to be acceptable in light of the treatment's efficacy. There is a need for more consistent methods to improve our understanding adverse events, dropouts, and overall symptom deterioration, and how these outcomes can be prevented.

Personalization strategies in digital mental health interventions: a systematic review and conceptual framework for depressive symptoms

Introduction

Personalization is a much-discussed approach to improve adherence and outcomes for Digital Mental Health interventions (DMHIs). Yet, major questions remain open, such as (1) what personalization is, (2) how prevalent it is in practice, and (3) what benefits it truly has.

Methods

We address this gap by performing a systematic literature review identifying all empirical studies on DMHIs targeting depressive symptoms in adults from 2015 to September 2022. The search in Pubmed, SCOPUS and Psycinfo led to the inclusion of 138 articles, describing 94 distinct DMHIs provided to an overall sample of approximately 24,300 individuals.

Results

Our investigation results in the conceptualization of personalization as purposefully designed variation between individuals in an intervention's therapeutic elements or its structure. We propose to further differentiate personalization by what is personalized (i.e., intervention content, content order, level of guidance or communication) and the underlying mechanism [i.e., user choice, provider choice, decision rules, and machine-learning (ML) based approaches]. Applying this concept, we identified personalization in 66% of the interventions for depressive symptoms, with personalized intervention content (32% of interventions) and communication with the user (30%) being particularly popular. Personalization via decision rules (48%) and user choice (36%) were the most used mechanisms, while the utilization of ML was rare (3%). Two-thirds of personalized interventions only tailored one dimension of the intervention.

Discussion

We conclude that future interventions could provide even more personalized experiences and especially benefit from using ML models. Finally, empirical evidence for personalization was scarce and inconclusive, making further evidence for the benefits of personalization highly needed.

Systematic Review Registration

Identifier: CRD42022357408.

Predicting early dropout in online versus face-to-face guided self-help: A machine learning approach

BACKGROUND

Early dropout hinders the effective adoption of brief psychological interventions and is associated with poor treatment outcomes. This study examined if attendance and depression treatment outcomes could be improved by matching patients to either face-to-face or computerized low-intensity psychological interventions.

METHODS

Archival clinical records were analysed for 85,664 patients who accessed face-to-face or computerized guided self-help (GSH). The primary outcome was early dropout (attending ≤3 sessions). Supervised machine learning analyses were applied in a training sample (n = 55,529). The trained algorithm was cross-validated in an independent test sample (n = 30,135). The clinical utility of the model was evaluated using logistic regression, chi-square tests, and sensitivity analyses in a balanced subsample.

RESULTS

Patients who received their model-indicated treatment modality were 12% more likely to receive an adequate dose of treatment OR = 1.12 (95% CI = 1.02 to 1.24), p = .02, and the strength of this effect was larger in the balanced subsample (OR = 2.10, 95% CI = 1.65 to 2.68, p < .001). Patients had better treatment outcomes when matched to their model-indicated treatment modality.

CONCLUSIONS

Machine learning approaches may enable services to optimally match patients to the treatment modality that maximizes attendance.

Implementation of evidence-based treatment for PTSD in Norway: clinical outcomes and impact of probable complex PTSD

Background: Posttraumatic stress disorder (PTSD) is a long-lasting and debilitating psychological disorder that affects a large portion of the population. Treatments such as Cognitive therapy for PTSD (CT-PTSD) and Eye movement desensitization and reprocessing (EMDR) have been shown to be effective and cost-efficient in clinical trials, but uptake and evidence of positive outcomes in real-world clinical services are limited. Implementation efforts have been hampered by providers' concerns about the feasibility of trauma-focused treatments in more complex presentations (i.e. Complex PTSD). Objective: To evaluate the effectiveness of CT-PTSD and EMDR in a real-world setting, as implemented in Norwegian outpatient mental health clinics for adults, and investigate the impact of probable Complex PTSD status on treatment outcomes. Methods: Clinicians from 15 different outpatient clinics received training and supervision in EMDR or CT-PTSD as part of a national implementation project. 104 clinicians recruited and treated 196 participants with PTSD. Symptoms of PTSD, depression and anxiety were assessed session-by-session and used to estimate pre-post effect sizes. Mixed-models were employed to investigate the impact of complex PTSD. Results: Both EMDR and CT-PTSD were associated with significant reductions in PTSD symptoms, with large effect sizes. Probable Complex PTSD was associated with higher levels of symptoms before and after treatment but did not significantly impact the effectiveness of treatment. Conclusion: The use of evidence-based treatments for PTSD in routine clinical service is associated with good treatment outcomes, also for patients with Complex PTSD.