Personalized treatment selection in routine care: Integrating machine learning and statistical algorithms to recommend cognitive behavioral or psychodynamic therapy

Which symptom to address in psychological treatment for cancer survivors when fear of cancer recurrence, depressive symptoms, and cancer-related fatigue co-occur? Exploring the level of agreement between three systematic approaches to select the focus of treatment

PURPOSE

To investigate the extent to which three systematic approaches for prioritizing symptoms lead to similar treatment advices in cancer survivors with co-occurring fear of cancer recurrence, depressive symptoms, and/or cancer-related fatigue.

METHODS

Psychological treatment advices were was based on three approaches: patient preference, symptom severity, and temporal precedence of symptoms based on ecological momentary assessments. The level of agreement was calculated according to the Kappa statistic.

RESULTS

Overall, we found limited agreement between the three approaches. Pairwise comparison showed moderate agreement between patient preference and symptom severity. Most patients preferred treatment for fatigue. Treatment for fear of cancer recurrence was mostly indicated when based on symptom severity. Agreement between temporal precedence and the other approaches was slight. A clear treatment advice based on temporal precedence was possible in 57% of cases. In cases where it was possible, all symptoms were about equally likely to be indicated.

CONCLUSIONS

The three approaches lead to different treatment advices. Future research should determine how the approaches are related to treatment outcome. We propose to discuss the results of each approach in a shared decision-making process to make a well-informed and personalized decision with regard to which symptom to target in psychological treatment.

IMPLICATIONS FOR CANCER SURVIVORS

This study contributes to the development of systematic approaches for selecting the focus of psychological treatment in cancer survivors with co-occurring symptoms by providing and comparing three different systematic approaches for prioritizing symptoms.

Applying precision methods to treatment selection for moderate/severe depression in person-centered experiential therapy or cognitive behavioral therapy

OBJECTIVE

To develop two prediction algorithms recommending person-centered experiential therapy (PCET) or cognitive-behavioral therapy (CBT) for patients with depression: (1) a full data model using multiple trial-based and routine variables, and (2) a routine data model using only variables available in the English NHS Talking Therapies program.

METHOD

Data was used from the PRaCTICED trial comparing PCET vs. CBT for 255 patients meeting a diagnosis of moderate or severe depression. Separate full and routine data models were derived and the latter tested in an external data sample.

RESULTS

The full data model provided the better prediction, yielding a significant difference in outcome between patients receiving their optimal vs. non-optimal treatment at 6- (Cohen's d = .65 [.40, .91]) and 12 months (d = .85 [.59, 1.10]) post-randomization. The routine data model performed similarly in the training and test samples with non-significant effect sizes, d = .19 [-.05, .44] and d = .21 [-.00, .43], respectively. For patients with the strongest treatment matching (d ≥ 0.3), the resulting effect size was significant, d = .38 [.11, 64].

CONCLUSION

A treatment selection algorithm might be used to recommend PCET or CBT. Although the overall effects were small, targeted matching yielded somewhat larger effects.

Predicting optimal treatment allocation for cognitive analytic-guided self-help versus cognitive behavioural-guided self-help

OBJECTIVES

Given the ubiquity in routine services of low-intensity guided self-help (GSH) psychological interventions, better patient selection for these brief interventions would be organizationally efficient. This study therefore sought to define who would respond best to two different types of GSH for anxiety to enable better future treatment matching.

METHODS

The study used outcome data from a patient preference trial (N = 209) comparing cognitive analytic therapy-guided self-help (CAT-GSH) with cognitive behavioural therapy-guided self-help (CBT-GSH). Elastic Net regularization and Boruta random forest variable selection methods were applied. Regression models calculated the patient advantage index (PAI) to designate which GSH was likely the most effective for each patient. Outcomes were compared for those receiving their PAI-indicated optimal and non-optimal GSH.

RESULTS

Lower baseline depression and anxiety severity predicted better outcomes for both types of GSH. Patient preference status was not associated with outcome during either GSH. Sixty-three % received their model indicating optimal GSH and these had significantly higher rates of reliable and clinically significant reductions in anxiety at both post-treatment (35.9% vs. 16.6%) and follow-up (36.6% vs. 19.2%). No single patient with a large PAI had a reliable and clinically significant reduction in anxiety at post-treatment or follow-up when they did not receive their optimal GSH.

CONCLUSIONS

Treatment matching algorithms have the potential to support evidenced-based treatment selection for GSH. Treatment selection and supporting patient choice needs to be integrated. Future research needs to investigate the use of the PAI for GSH treatment matching, but with larger and more balanced samples.

Prediction of childbearing tendency in women on the verge of marriage using machine learning techniques

The declining fertility rate and increasing marriage age among girls pose challenges for policymakers, leading to issues such as population decline, higher social and economic costs, and reduced labor productivity. Using machine learning (ML) techniques to predict the desire to have children can offer a promising solution to address these challenges. Therefore, this study aimed to predict the childbearing tendency in women on the verge of marriage using ML techniques. Data from 252 participants (203 expressing a "desire to have children" and 49 indicating "reluctance to have children") in Abadan, and Khorramshahr cities (Khuzestan Province, Iran) was analyzed. Seven ML algorithms, including multilayer perceptron (MLP), support vector machine (SVM), logistic regression (LR), random forest (RF), J48 decision tree, Naive Bayes (NB), and K-nearest neighbors (KNN), were employed. The performance of these algorithms was assessed using metrics derived from the confusion matrix. The RF algorithm showed superior performance, with the highest sensitivity (99.5%), specificity (95.6%), and receiver operating characteristic curve (90.1%) values. Meanwhile, MLP emerged as the top-performing algorithm, showcasing the best overall performance in accuracy (77.75%) and precision (81.8%) compared to other algorithms. Factors such as age of marriage, place of residence, and strength of the family center with the birth of a child were the most effective predictors of a woman's desire to have children. Conversely, the number of daughters, the wife's ethnicity, and the spouse's ownership of assets such as cars and houses were among the least important factors in predicting this desire. ML algorithms exhibit excellent predictive capabilities for childbearing tendencies in women on the verge of marriage, highlighting their remarkable effectiveness. This capacity to offer accurate prognoses holds significant promise for advancing research in this field.

Using the 'Leapfrog' Design as a Simple Form of Adaptive Platform Trial to Develop, Test, and Implement Treatment Personalization Methods in Routine Practice

The route for the development, evaluation and dissemination of personalized psychological therapies is complex and challenging. In particular, the large sample sizes needed to provide adequately powered trials of newly-developed personalization approaches means that the traditional treatment development route is extremely inefficient. This paper outlines the promise of adaptive platform trials (APT) embedded within routine practice as a method to streamline development and testing of personalized psychological therapies, and close the gap to implementation in real-world settings. It focuses in particular on a recently-developed simplified APT design, the 'leapfrog' trial, illustrating via simulation how such a trial may proceed and the advantages it can bring, for example in terms of reduced sample sizes. Finally it discusses models of how such trials could be implemented in routine practice, including potential challenges and caveats, alongside a longer-term perspective on the development of personalized psychological treatments.

Precision Mental Health and Data-Informed Decision Support in Psychological Therapy: An Example

Outcome measurement including data-informed decision support for therapists in psychological therapy has developed impressively over the past two decades. New technological developments such as computerized data assessment, and feedback tools have facilitated advanced implementation in several seetings. Recent developments try to improve the clinical decision-making process by connecting clinical practice better with empirical data. For example, psychometric data can be used by clinicians to personalize the selection of therapeutic programs, strategies or modules and to monitor a patient's response to therapy in real time. Furthermore, clinical support tools can be used to improve the treatment for patients at risk for a negative outcome. Therefore, measurement-based care can be seen as an important and integral part of clinical competence, practice, and training. This is comparable to many other areas in the healthcare system, where continuous monitoring of health indicators is common in day-to-day clinical practice (e.g., fever, blood pressure). In this paper, we present the basic concepts of a data-informed decision support system for tailoring individual psychological interventions to specific patient needs, and discuss the implications for implementing this form of precision mental health in clinical practice.

A Qualitative Analysis of Stakeholder Attitudes Regarding Personalized Provider Selection and Patient-Therapist Matching

This study explored mental health care patients and therapists' perspectives on using therapists' measurement-based and problem-specific effectiveness data to inform case assignments - a type of treatment personalization that has been shown to outperform non-measurement-based case assignment as usual (Constantino et al., 2021). We conducted semi-structured qualitative interviews with 8 patients (75% women; M age = 33.75 years) and 8 therapists (75% women; M age = 47.50 years). The interview protocols were unique to stakeholder group. Recorded responses were transcribed and qualitatively analyzed by four judges using a blend of consensual qualitative research and grounded theory methods. Derived patient domains included preferred characteristics of a provider, and experiences and suggestions regarding provider selection. Within the domains, most patients expressed an interest in accessing more specific provider information online. Additionally, most patients indicated that both provider outcome track records and personal preference information (e.g., therapist characteristics) should be considered in the therapist selection process. All patients endorsed being comfortable with having the ability to select a provider based on a list of empirically well-matched recommendations. Derived therapist domains included using routine outcomes monitoring for patient-provider matching, referral source and direct patient use of preferred provider lists, and improvements to the provider selection process. Within the domains, all therapists remarked that outcome data would be useful for matching patients to providers; however, most also indicated that outcome data should not be the only factor used in provider selection. All therapists expressed a willingness to be included in preferred provider lists that incorporate track record data. Overall, both patients and therapists held generally positive views toward using therapist effectiveness data to help personalize mental health care. Yet, both stakeholder groups acknowledged that other personalization factors should be considered alongside these data. Based on these results, our team is in the process of implementing patient-therapist match strategies in larger and more diverse mental health care contexts.

A Tutorial Introduction to Heterogeneous Treatment Effect Estimation with Meta-learners

Psychotherapy has been proven to be effective on average, though patients respond very differently to treatment. Understanding which characteristics are associated with treatment effect heterogeneity can help to customize therapy to the individual patient. In this tutorial, we describe different meta-learners, which are flexible algorithms that can be used to estimate personalized treatment effects. More specifically, meta-learners decompose treatment effect estimation into multiple prediction tasks, each of which can be solved by any machine learning model. We begin by reviewing necessary assumptions for interpreting the estimated treatment effects as causal, and then give an overview over key concepts of machine learning. Throughout the article, we use an illustrative data example to show how the different meta-learners can be implemented in R. We also point out how current popular practices in psychotherapy research fit into the meta-learning framework. Finally, we show how heterogeneous treatment effects can be analyzed, and point out some challenges in the implementation of meta-learners.

Is personalization of psychological pain treatments necessary? Evidence from a Bayesian variance ratio meta-analysis

ABSTRACT

This is the first study to empirically determine the potential for data-driven personalization in the context of chronic primary pain (CPP). Effect sizes of psychological treatments for individuals with CPP are small to moderate on average. Aiming for better treatment outcomes for the individual patient, the call to personalize CPP treatment increased over time. However, empirical evidence that personalization of psychological treatments can optimize treatment outcomes in CPP is needed. This study seeks to estimate heterogeneity of treatment effect for cognitive behavioral therapy (CBT) as the psychological treatment approach for CPP with the greatest evidence base. For this purpose, a Bayesian variance ratio meta-regression is conducted using updated data from 2 recently published meta-analyses with randomized controlled trials comparing CBT delivered face-to-face to treatment-as-usual or waiting list controls. Heterogeneity in patients with CPP would be reflected by a larger overall variance in the post-treatment score compared with the control group. We found first evidence for an individual treatment effect in CBT compared with the control group. The estimate for the intercept was 0.06, indicating a 6% higher variance of end point values in the intervention groups. However, this result warrants careful consideration. Further research is needed to shed light on the heterogeneity of psychological treatment studies and thus to uncover the full potential of data-driven personalized psychotherapy for patients with CPP.A Bayesian variance ratio meta-regression indicates empirical evidence that data-driven personalized psychotherapy for patients with chronic primary pain could increase effects of cognitive behavioral therapy.

Data-driven personalized medicine approaches to cognitive-behavioral therapy allocation in a large sample: A reanalysis of the ENRICHED study

BACKGROUND

Although effective treatments for common mental health problems are available, individual responses to treatments are difficult to predict. Treatment efficacy could be optimized by targeting interventions using individual predictions of treatment outcomes. The aim of this study was to develop a prediction algorithm using data from one of the largest randomized controlled trials on psychological interventions for common mental health problems.

METHODS

This is a secondary analysis of the Enhancing Recovery in Coronary Heart Disease study investigating the effectiveness of cognitive behavioral therapy (CBT) and care as usual (CAU) for depression and low perceived social support following acute myocardial infarction. 2481 participants were randomly assigned to CBT and CAU. Baseline social-demographics, depression characteristics, comorbid symptoms, and stress and adversity measures were used to build an algorithm predicting post-treatment depression severity using elastic net regularization. Performance and generalizability of this algorithm were determined in a hold-out sample (n = 1203).

RESULTS

Treatment matching based on predictions in the hold-out sample resulted in inconsistent and small effects (d = 0.15), that were more pronounced for individuals matched to CBT (d = 0.22). We identified a small subgroup of individuals for which CBT did not appear more efficacious than CAU.

LIMITATIONS

Limitations are a poorly defined CAU condition, a low-severity sample, specific exclusion criteria and unavailability of certain baseline variables.

CONCLUSIONS

Small matching effects are likely a realistic representation of the performance and generalizability of multivariable prediction algorithms based on clinical measures. Results indicate that future work and new approaches are needed.

Heterogeneity of Treatment Effects in Internet- and Mobile-Based Interventions for Depression: A Systematic Review and Meta-Analysis

Importance

While the effects of internet- and mobile-based interventions (IMIs) for depression have been extensively studied, no systematic evidence is available regarding the heterogeneity of treatment effects (HTEs), indicating to what extent patient-by-treatment interactions exist and personalized treatment models might be necessary.

Objective

To investigate the HTEs in IMIs for depression as well as their efficacy and effectiveness.

Data Sources

A systematic search in Embase, MEDLINE, Central, and PsycINFO for randomized clinical trials and supplementary reference searches was conducted on October 13, 2019, and updated March 25, 2022. The search string included various terms related to digital psychotherapy, depression, and randomized clinical trials.

Study Selection

Titles, abstracts, and full texts were reviewed by 2 independent researchers. Studies of all populations with at least 1 intervention group receiving an IMI for depression and at least 1 control group were eligible, if they assessed depression severity as a primary outcome and followed a randomized clinical trial (RCT) design.

Data Extraction and Synthesis

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines. Risk of bias was evaluated using the Cochrane Risk of Bias Tool. HTE was investigated using logarithmic variance ratios (lnVR) and effect sizes using Hedges g. Three-level bayesian meta-regressions were conducted.

Main Outcomes and Measures

Heterogeneity of treatment effects was the primary outcome of this study; magnitudes of treatment effect sizes were the secondary outcome. Depression severity was measured by different self-report and clinician-rated scales in the included RCTs.

Results

The systematic review of 102 trials included 19 758 participants (mean [SD] age, 39.9 [10.58] years) with moderate depression severity (mean [SD] in Patient Health Questionnaire-9 score, 12.81 [2.93]). No evidence for HTE in IMIs was found (lnVR = -0.02; 95% credible interval [CrI], -0.07 to 0.03). However, HTE was higher in more severe depression levels (β̂ = 0.04; 95% CrI, 0.01 to 0.07). The effect size of IMI was medium (g = -0.56; 95% CrI, -0.46 to -0.66). An interaction effect between guidance and baseline severity was found (β̂ = -0.24, 95% CrI, -0.03 to -0.46).

Conclusions and Relevance

In this systematic review and meta-analysis of RCTs, no evidence for increased patient-by-treatment interaction in IMIs among patients with subthreshold to mild depression was found. Guidance did not increase effect sizes in this subgroup. However, the association of baseline severity with HTE and its interaction with guidance indicates a more sensitive, guided, digital precision approach would benefit individuals with more severe symptoms. Future research in this population is needed to explore personalization strategies and fully exploit the potential of IMI.

Individual treatment selection for patients with post-traumatic stress disorder: External validation of a personalised advantage index

OBJECTIVE

To test the predictive accuracy and generalisability of a personalised advantage index (PAI) model designed to support treatment selection for Post-Traumatic Stress Disorder (PTSD).

METHOD

A PAI model developed by Deisenhofer et al. (2018) was used to predict treatment outcomes in a statistically independent dataset including archival records for N = 152 patients with PSTD who accessed either trauma-focussed cognitive behavioural therapy or eye movement desensitisation and reprocessing in routine care. Outcomes were compared between patients who received their PAI-indicated optimal treatment versus those who received their suboptimal treatment.

RESULTS

The model did not yield treatment specific predictions and patients who had received their PAI-indicated optimal treatment did not have better treatment outcomes in this external validation sample.

CONCLUSION

This PAI model did not generalise to an external validation sample.

Leveraging Novel Technologies and Artificial Intelligence to Advance Practice-Oriented Research

Mental health services are experiencing notable transformations as innovative technologies and artificial intelligence (AI) are increasingly utilized in a growing number of studies and services.These cutting-edge technologies carry the promise of substantial improvements in the field of mental health. Nevertheless, questions emerge about the alignment of novel technologies and AI systems with human needs, especially in the context of vulnerable populations receiving mental healthcare. The practice-oriented research (POR) model is pivotal in seamlessly integrating these emerging technologies into clinical research and practice. It underscores the importance of tight collaboration between clinicians and researchers, all driven by the central goal of ensuring and elevating client well-being. This paper focuses on how novel technologies can enhance the POR model and highlights its pivotal role in integrating these technologies into clinical research and practice. We discuss two key phases: pre-treatment, and during treatment. For each phase, we describe the challenges, present the major technological innovations, describe recent studies exemplifying technology use, and suggest future directions. Ethical concerns and the importance of aligning humans and technology are also considered, in addition to implications for practice and training.

Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

Anticipating interpersonal sensitivity: A predictive model for early intervention in psychological disorders in college students

Psychological disorders, notably social anxiety and depression, exert detrimental effects on university students, impeding academic achievement and overall development. Timely identification of interpersonal sensitivity becomes imperative to implement targeted support and interventions. This study selected 958 freshmen from higher education institutions in Zhejiang province as the research sample. Utilizing the runge-kutta search and elite levy spreading enhanced moth-flame optimization (MFO) in conjunction with the kernel extreme learning machine (KELM), we propose an efficient intelligent prediction model, namely bREMFO-KELM, for predicting the interpersonal sensitivity of college students. IEEE CEC 2017 benchmark functions and the interpersonal sensitivity dataset were employed as the basis for detailed comparisons with peer-reviewed studies and well-known machine learning models. The experimental results demonstrate the outstanding performance of the bREMFO-KELM model in predicting the sensitivity of interpersonal relationships in college students, achieving an impressive accuracy rate of 97.186%. In-depth analysis reveals that the prediction of interpersonal sensitivity in college students is closely associated with multiple features, including easily hurt in relationships, shy and uneasy with the opposite sex, feeling inferior to others, discomfort when observed or discussed, and blame and criticize others. These features are not only crucial for the accuracy of the prediction model but also provide valuable information for a deeper understanding of the sensitivity of college students' interpersonal relationships. In conclusion, the bREMFO-KELM model excels not only in performance but also possesses a high degree of interpretability, providing robust support for predicting the sensitivity of interpersonal relationships in college students.

Exploring mechanisms of change in the metacognitive training for depression

The Metacognitive Training for Depression (D-MCT) is a highly structured group therapy that has been shown to be effective in reducing depressive symptoms. First evidence suggests that need for control represents a mechanism of change. However, more research is needed to evaluate the mode of action of each module and identify predictors of treatment response. Two sequential studies (one naturalistic pilot study [study I, N = 45] and one randomized controlled trial [study II, N = 32]) were conducted to evaluate the session-specific effects and predictors of D-MCT in patients with depression. The D-MCT was conducted over eight weeks, and patients answered a questionnaire on dysfunctional beliefs (e.g., negative filter) and depressive symptoms (e.g., lack of energy, self-esteem) before and after each session. Linear mixed-effects models showed that several dysfunctional beliefs and symptoms improved over the course of the treatment; three modules were able to evoke within-session effects, but no between-session effects were found. The improvement in lack of energy in one module was identified as a relevant predictor in study I via lasso regression but was not replicated in study II. Exploratory analyses revealed further predictors that warrant replication in future studies. The identified predictors were inconclusive when the two studies were compared, which may be explained by the different instruments administered. Even so, the results may be used to revise questionnaires and improve the intervention.

How well can U.S. military veterans' suicidal ideation be predicted from static and change-based indicators of their psychosocial well-being as they adapt to civilian life?

BACKGROUND

Identifying predictors of suicidal ideation (SI) is important to inform suicide prevention efforts, particularly among high-risk populations like military veterans. Although many studies have examined the contribution of psychopathology to veterans' SI, fewer studies have examined whether experiencing good psychosocial well-being with regard to multiple aspects of life can protect veterans from SI or evaluated whether SI risk prediction can be enhanced by considering change in life circumstances along with static factors.

METHODS

The study drew from a longitudinal population-based sample of 7141 U.S. veterans assessed throughout the first three years after leaving military service. Machine learning methods (cross-validated random forests) were applied to examine the predictive utility of static and change-based well-being indicators to veterans' SI, as compared to psychopathology predictors.

RESULTS

Although psychopathology models performed better, the full set of well-being predictors demonstrated acceptable discrimination in predicting new-onset SI and accounted for approximately two-thirds of cases of SI in the top strata (quintile) of predicted risk. Greater engagement in health promoting behavior and social well-being were most important in predicting reduced SI risk, with several change-based predictors of SI identified but stronger associations observed for static as compared to change-based indicator sets as a whole.

CONCLUSIONS

Findings support the value of considering veterans' broader well-being in identifying individuals at risk for suicidal ideation and suggest the possibility that well-being promotion efforts may be useful in reducing suicide risk. Findings also highlight the need for additional attention to change-based predictors to better understand their potential value in identifying individuals at risk for SI.

[How Cooperation Instead of Coexistence in Psychotherapy Research can Improve Science, Practice and Continuing Education]

Die kontinuierliche Erhebung psychometrischer Daten vor, während und nach einer psychotherapeutischen Behandlung kann als Qualitätssicherungsmaßnahme Therapeut:innen in ihrer klinischen Arbeit unterstützen und zugleich eine belastbare Datengrundlage für die Psychotherapieforschung schaffen. Im Rahmen der Qualitätssicherung können die erhobenen Daten als zusätzliche Informationsquelle den klinischen Eindruck der Therapeut:innen erweitern und zur Evaluation der Behandlung am Einzelfall aber auch auf der Ebene des Versorgungssystems herangezogen werden. Darüber hinaus können prognostische Vorhersagen von Therapieergebnissen und Abbruchwahrscheinlichkeiten, Behandlungsempfehlungen sowie adaptive Behandlungsanpassungen während der Behandlung auf ihnen aufgebaut werden, die Therapeut:innen in ihren klinischen Entscheidungen unterstützen 1. Eine solche daten-gestützte und evidenzbasierte psychologische Psychotherapie kann die wissenschaftliche Fundierung der therapeutischen Herangehensweise und die Wirksamkeit der Behandlung verbessern. Dazu bedarf es umfangreicher Datenerhebungen, die verlässliche und aussagekräftige Forschungsbefunde ermöglichen 2.

Predicting optimal treatment outcomes in phase-based treatment and direct trauma-focused treatment among patients with posttraumatic stress disorder stemming from childhood abuse

Research over the last few decades has demonstrated the effectiveness of various treatments for posttraumatic stress disorder (PTSD). However, the question of which treatment works best remains, especially for patients with PTSD stemming from childhood abuse. Using the Personalized Advantage Index (PAI), we explored which patients benefit more from phase-based treatment and which benefit more from direct trauma-focused treatment. Data were obtained from a multicenter randomized controlled trial (RCT) comparing a phase-based treatment condition (i.e., eye-movement desensitization and reprocessing [EMDR] therapy preceded by Skills Training in Affect and Interpersonal Regulation [STAIR]; n = 57) and a direct trauma-focused treatment (EMDR only; n = 64) among individuals with PTSD related to childhood abuse. Machine learning techniques were used to examine all pretreatment variables included in the trial as potential predictors and moderators, with selected variables combined to build the PAI model. The utility of the PAI was tested by comparing actual posttreatment outcomes of individuals who received PAI-indicated treatment with those allocated to a non-PAI-indicated treatment. Although eight pretreatment variables between PTSD treatment outcome and treatment condition were selected as moderators, there was no significant difference between participants assigned to their PAI-indicated treatment and those randomized to a non-PAI-indicated treatment, d = 0.25, p = .213. Hence, the results of this study do not support the need for personalized medicine for patients with PTSD and a history of childhood abuse. Further research with larger sample sizes and external validation is warranted.

What if we wait? Using synthetic waiting lists to estimate treatment effects in routine outcome data

Objective: Due to the lack of randomization, pre-post routine outcome data precludes causal conclusions. We propose the "synthetic waiting list" (SWL) control group to overcome this limitation.

Method: First, a step-by-step introduction illustrates this novel approach. Then, this approach is demonstrated using an empirical example with data from an outpatient cognitive-behavioral therapy (CBT) clinic (N = 139). We trained an ensemble machine learning model ("Super Learner") on a data set of patients waiting for treatment (N = 311) to make counterfactual predictions of symptom change during this hypothetical period.

Results: The between-group treatment effect was estimated to be d = 0.42. Of the patients who received CBT, 43.88% achieved reliable and clinically significant change, while this probability was estimated to be 14.54% in the SWL group. Counterfactual estimates suggest a clear net benefit of psychotherapy for 41% of patients. In 32%, the benefit was unclear, and 27% would have improved similarly without receiving CBT.

Conclusions: The SWL is a viable new approach that provides between-group outcome estimates similar to those reported in the literature comparing psychotherapy with high-intensity control interventions. It holds the potential to mitigate common limitations of routine outcome data analysis.

[Digitization and Machine Learning in Psychotherapy Research and Clinical Practice - Potentials and Problems]

Im Zuge des weltweiten Anstieges der Bedeutung von psychischen Störungen 1, werden frühzeitige Interventionen und wirksame psychotherapeutische Behandlungen für ein funktionierendes Gesundheitssystem immer wichtiger. Der aktuelle Stand der Psychotherapieforschung zeigt jedoch, dass nicht alle Patient:innen gleichermaßen von Psychotherapie profitieren, sondern dass die meisten Patient:innen (70–80%) zwar deutliche Verbesserung zeigen, während andere nur geringe oder keine Fortschritte erzielen oder sogar Verschlechterungen erfahren 2. Dies impliziert eine stärkere Berücksichtigung von individuellen Unterschieden von Patient:innen und deren Therapieverlauf in der Psychotherapieforschung, sowie eine stärkere Refokussierung auf ungünstige Therapieverläufe und Abkehr von der Frage nach durchschnittlichen Unterschieden zwischen den verschiedenen Therapieverfahren oder Therapieschulen.

Predicting non-improvement of symptoms in daily mental healthcare practice using routinely collected patient-level data: a machine learning approach

Objectives

Anxiety and mood disorders greatly affect the quality of life for individuals worldwide. A substantial proportion of patients do not sufficiently improve during evidence-based treatments in mental healthcare. It remains challenging to predict which patients will or will not benefit. Moreover, the limited research available on predictors of treatment outcomes comes from efficacy RCTs with strict selection criteria which may limit generalizability to a real-world context. The current study evaluates the performance of different machine learning (ML) models in predicting non-improvement in an observational sample of patients treated in routine specialized mental healthcare.

Methods

In the current longitudinal exploratory prediction study diagnosis-related, sociodemographic, clinical and routinely collected patient-reported quantitative outcome measures were acquired during treatment as usual of 755 patients with a primary anxiety, depressive, obsessive compulsive or trauma-related disorder in a specialized outpatient mental healthcare center. ML algorithms were trained to predict non-response (< 0.5 standard deviation improvement) in symptomatic distress 6 months after baseline. Different models were trained, including models with and without early change scores in psychopathology and well-being and models with a trimmed set of predictor variables. Performance of trained models was evaluated in a hold-out sample (30%) as a proxy for unseen data.

Results

ML models without early change scores performed poorly in predicting six-month non-response in the hold-out sample with Area Under the Curves (AUCs) < 0.63. Including early change scores slightly improved the models' performance (AUC range: 0.68-0.73). Computationally-intensive ML models did not significantly outperform logistic regression (AUC: 0.69). Reduced prediction models performed similar to the full prediction models in both the models without (AUC: 0.58-0.62 vs. 0.58-0.63) and models with early change scores (AUC: 0.69-0.73 vs. 0.68-0.71). Across different ML algorithms, early change scores in psychopathology and well-being consistently emerged as important predictors for non-improvement.

Conclusion

Accurately predicting treatment outcomes in a mental healthcare context remains challenging. While advanced ML algorithms offer flexibility, they showed limited additional value compared to traditional logistic regression in this study. The current study confirmed the importance of taking early change scores in both psychopathology and well-being into account for predicting longer-term outcomes in symptomatic distress.

How to assess and analyse session-specific effects and predictors: An example with the Metacognitive Training for Obsessive-Compulsive Disorder intervention

It is essential to understand the effects of specific therapy elements (i.e., mechanisms of change) to optimize the efficacy of available treatments. There are, however, existing challenges in the assessment and analysis of constructs of interest. The present study aims to improve research on the effects of specific therapy elements using the example of the Metacognitive Training for Obsessive-Compulsive Disorder (MCT-OCD) intervention. Specifically, we introduce an innovative analytical method to identify predictors of treatment outcome and expand the assessment of common factors (e.g., coping expectations). A sample of 50 day- and inpatients with OCD was assessed before and after participation in an 8-week MCT-OCD programme. We investigated within-session change in scores on revised questionnaires administered before and after each session. Linear mixed models (for session-effects) and lasso regression (for prediction analyses) were used to analyse data. The revised assessments and data analyses showed greater improvement in dysfunctional (meta-)cognitive beliefs over the time of the intervention and within sessions compared to previous MCT-OCD studies. Some predictors, for example, improvement in coping expectation after the module on overestimation of threat for treatment outcome, were identified. The present study contributed to a better understanding of how to assess and analyse data of a modular intervention and demonstrated the strengths and weaknesses of different analytic approaches. Moreover, the analyses provided a deeper understanding of the specific effects and mechanisms of change of MCT-OCD modules, which can be refined and examined in future studies.

Gradient Tree Boosting for Hierarchical Data

Gradient tree boosting is a powerful machine learning technique that has shown good performance in predicting a variety of outcomes. However, when applied to hierarchical (e.g., longitudinal or clustered) data, the predictive performance of gradient tree boosting may be harmed by ignoring the hierarchical structure, and may be improved by accounting for it. Tree-based methods such as regression trees and random forests have already been extended to hierarchical data settings by combining them with the linear mixed effects model (MEM). In the present article, we add to this literature by proposing two algorithms to estimate a combination of the MEM and gradient tree boosting. We report on two simulation studies that (i) investigate the predictive performance of the two MEM boosting algorithms and (ii) compare them to standard gradient tree boosting, standard random forest, and other existing methods for hierarchical data (MEM, MEM random forests, model-based boosting, Bayesian additive regression trees [BART]). We found substantial improvements in the predictive performance of our MEM boosting algorithms over standard boosting when the random effects were non-negligible. MEM boosting as well as BART showed a predictive performance similar to the correctly specified MEM (i.e., the benchmark model), and overall outperformed the model-based boosting and random forest approaches.

Who benefits from guided internet-based interventions? A systematic review of predictors and moderators of treatment outcome

To our knowledge, no systematic review has been conducted on predictors or moderators of treatment outcome across diagnoses in guided internet-based interventions (IBIs) for adults. To identify who benefits from this specific format and therein inform future research on improving patient-treatment fit, we aimed to aggregate results of relevant studies. 2100 articles, identified by searching the databases PsycInfo, Ovid Medline, and Pubmed and through snowballing, were screened in April/May 2021 and October 2022. Risk of bias and intra- and interrater reliability were assessed. Variables were grouped by predictor category, then synthesized using vote counting based on direction of effect. N = 60 articles were included in the review. Grouping resulted in 88 predictors/moderators, of which adherence, baseline symptoms, education, age, and gender were most frequently assessed. Better adherence, treatment credibility, and working alliance emerged as conclusive predictors/moderators for better outcome, whereas higher baseline scores predicted more reliable change but higher post-treatment symptoms. Results of all other predictors/moderators were inconclusive or lacked data. Our review highlights that it is currently difficult to predict, across diagnoses, who will benefit from guided IBIs. Further rigorous research is needed to identify predictors and moderators based on a sufficient number of studies. PROSPERO registration: CRD42021242305.

Associations between specific depressive symptoms and psychosocial functioning in psychotherapy

BACKGROUND

Psychotherapy for depression aims to reduce symptoms and to improve psychosocial functioning. We examined whether some symptoms are more important than others in the association between depression and functioning over the course of psychotherapy treatment.

METHODS

We studied associations between specific symptoms of depression (PHQ-9) and change in social and occupational functioning (SOFAS), both with structural equation models (considering liabilities of depression and each specific symptom) and with logistic regression models (considering the risk for individual patients). The study sample consisted of adult patients (n = 771) from the Finnish Psychotherapy Quality Registry (FPQR) who completed psychotherapy treatment between September 2018 and September 2021.

RESULTS

Based on our results of logistic regression analyses and SEM models, the baseline measures of depression symptoms were not associated with changes in functioning. Changes in depressed mood or hopelessness, problems with sleep, feeling tired, and feeling little interest or pleasure were associated with improved functioning during psychotherapy. The strongest evidence for symptom-specific effects was found for the symptom of depressed mood or hopelessness.

LIMITATIONS

Due to our naturalistic study design containing only two measurement points, we were unable to study the causal relationship between symptoms and functioning.

CONCLUSIONS

Changes in certain symptoms during psychotherapy may affect functioning independently of underlying depression. Knowledge about the dynamics between symptoms and functioning could be used in treatment planning or implementation. Depressed mood or hopelessness appears to have a role in the dynamic relationship between depression and functioning.

Developing Transdiagnostic Internalizing Disorder Prognostic Indices for Outpatient Cognitive Behavioral Therapy

A growing literature is devoted to understanding and predicting heterogeneity in response to cognitive behavioral therapy (CBT), including using supervised machine learning to develop prognostic models that could be used to inform treatment planning. The current study developed CBT prognostic models using data from a broad dimensionally oriented pretreatment assessment (324 predictors) of 1,210 outpatients with internalizing psychopathology. Super learning was implemented to develop prognostic indices for three outcomes assessed at 12-month follow-up: principal diagnosis improvement (attained by 65.8% of patients), principal diagnosis remission (56.8%), and transdiagnostic full remission (14.3%). The models for principal diagnosis remission and transdiagnostic remission performed best (AUROCs = 0.71-0.73). Calibration was modest for all three models. Three-quarters (77.3%) of patients in the top tertile of the predicted probability distribution achieved principal diagnosis remission, compared to 35.0% in the bottom tertile. One-third (35.3%) of patients in the top two deciles of predicted probabilities for transdiagnostic complete remission achieved this outcome, compared to 2.7% in the bottom tertile. Key predictors included principal diagnosis severity, social anxiety diagnosis/severity, hopelessness, temperament, and global impairment. While additional work is needed to improve performance, integration of CBT prognostic models ultimately could lead to more effective and efficient treatment of patients with internalizing psychopathology.

Predicting dropout from psychological treatment using different machine learning algorithms, resampling methods, and sample sizes

OBJECTIVE

The occurrence of dropout from psychological interventions is associated with poor treatment outcome and high health, societal and economic costs. Recently, machine learning (ML) algorithms have been tested in psychotherapy outcome research. Dropout predictions are usually limited by imbalanced datasets and the size of the sample. This paper aims to improve dropout prediction by comparing ML algorithms, sample sizes and resampling methods.

METHOD

Twenty ML algorithms were examined in twelve subsamples (drawn from a sample of N = 49,602) using four resampling methods in comparison to the absence of resampling and to each other. Prediction accuracy was evaluated in an independent holdout dataset using the F₁-Measure.

RESULTS

Resampling methods improved the performance of ML algorithms and down-sampling can be recommended, as it was the fastest method and as accurate as the other methods. For the highest mean F₁-Score of .51 a minimum sample size of N = 300 was necessary. No specific algorithm or algorithm group can be recommended.

CONCLUSION

Resampling methods could improve the accuracy of predicting dropout in psychological interventions. Down-sampling is recommended as it is the least computationally taxing method. The training sample should contain at least 300 cases.

Trans-diagnostic determinants of psychotherapeutic treatment response: The pressing need and new opportunities for a more systematic way of selecting psychotherapeutic treatment in the age of virtual service delivery

Numerous forms of psychotherapy have demonstrated effectiveness for individuals with specific mental disorders. It is, therefore, the task of the clinician to choose the most appropriate therapeutic approach for any given client to maximize effectiveness. This can prove to be a difficult task due to at least three considerations: (1) there is no treatment approach, method or model that works well on all patients, even within a particular diagnostic class; (2) several treatments are equally efficacious (i.e., more likely to be effective than no treatment at all) when considered only in terms of the patient's diagnosis; and (3) effectiveness in the real-world therapeutic setting is determined by a host of non-diagnostic factors. Typically, consideration of these latter, trans-diagnostic factors is unmethodical or altogether excluded from treatment planning - often resulting in suboptimal patient care, inappropriate clinic resource utilization, patient dissatisfaction with care, patient demoralization/hopelessness, and treatment failure. In this perspective article, we argue that a more systematic research on and clinical consideration of trans-diagnostic factors determining psychotherapeutic treatment outcome (i.e., treatment moderators) would be beneficial and - with the seismic shift toward online service delivery - is more feasible than it used to be. Such a transition toward more client-centered care - systematically considering variables such as sociodemographic characteristics, patient motivation for change, self-efficacy, illness acuity, character pathology, trauma history when making treatment choices - would result in not only decreased symptom burden and improved quality of life but also better resource utilization in mental health care and improved staff morale reducing staff burnout and turnover.

A Scoping Review of the Use of Machine Learning in Health Economics and Outcomes Research: Part 2-Data From Nonwearables

OBJECTIVES

Despite the increasing interest in applying machine learning (ML) methods in health economics and outcomes research (HEOR), stakeholders face uncertainties in when and how ML can be used. We reviewed the recent applications of ML in HEOR.

METHODS

We searched PubMed for studies published between January 2020 and March 2021 and randomly chose 20% of the identified studies for the sake of manageability. Studies that were in HEOR and applied an ML technique were included. Studies related to wearable devices were excluded. We abstracted information on the ML applications, data types, and ML methods and analyzed it using descriptive statistics.

RESULTS

We retrieved 805 articles, of which 161 (20%) were randomly chosen. Ninety-two of the random sample met the eligibility criteria. We found that ML was primarily used for predicting future events (86%) rather than current events (14%). The most common response variables were clinical events or disease incidence (42%) and treatment outcomes (22%). ML was less used to predict economic outcomes such as health resource utilization (16%) or costs (3%). Although electronic medical records (35%) were frequently used for model development, claims data were used less frequently (9%). Tree-based methods (eg, random forests and boosting) were the most commonly used ML methods (31%).

CONCLUSIONS

The use of ML techniques in HEOR is growing rapidly, but there remain opportunities to apply them to predict economic outcomes, especially using claims databases, which could inform the development of cost-effectiveness models.

Personalized Prediction of Response to Smartphone-Delivered Meditation Training: Randomized Controlled Trial

BACKGROUND

Meditation apps have surged in popularity in recent years, with an increasing number of individuals turning to these apps to cope with stress, including during the COVID-19 pandemic. Meditation apps are the most commonly used mental health apps for depression and anxiety. However, little is known about who is well suited to these apps.

OBJECTIVE

This study aimed to develop and test a data-driven algorithm to predict which individuals are most likely to benefit from app-based meditation training.

METHODS

Using randomized controlled trial data comparing a 4-week meditation app (Healthy Minds Program [HMP]) with an assessment-only control condition in school system employees (n=662), we developed an algorithm to predict who is most likely to benefit from HMP. Baseline clinical and demographic characteristics were submitted to a machine learning model to develop a "Personalized Advantage Index" (PAI) reflecting an individual's expected reduction in distress (primary outcome) from HMP versus control.

RESULTS

A significant group × PAI interaction emerged (t658=3.30; P=.001), indicating that PAI scores moderated group differences in outcomes. A regression model that included repetitive negative thinking as the sole baseline predictor performed comparably well. Finally, we demonstrate the translation of a predictive model into personalized recommendations of expected benefit.

CONCLUSIONS

Overall, the results revealed the potential of a data-driven algorithm to inform which individuals are most likely to benefit from a meditation app. Such an algorithm could be used to objectively communicate expected benefits to individuals, allowing them to make more informed decisions about whether a meditation app is appropriate for them.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04426318; https://clinicaltrials.gov/ct2/show/NCT04426318.

Is it worth it to personalize the treatment of PTSD? - A variance-ratio meta-analysis and estimation of treatment effect heterogeneity in RCTs of PTSD

Several evidence-based treatments for posttraumatic stress disorder (PTSD) are recommended by international guidelines (e.g., APA, NICE). While their average effects are in general high, non-response rates indicate differential treatment effects. Here, we used a large database of RCTs on psychotherapy for PTSD to determine a reliable estimate of this heterogeneity in treatment effects (HTE) by applying Bayesian variance ratio meta-analysis. In total, 66 studies with a total of 8803 patients were included in our study. HTE was found for all psychological treatments, with varying degrees of certainty, only slight differences between psychological treatments, and active control groups yielding a smaller variance ratio compared to waiting list control groups. Across all psychological treatment and control group types, the estimate for the intercept was 0.12, indicating a 12% higher variance of posttreatment values in the intervention groups after controlling for differences in treatment outcomes. This study is the first to determine the maximum increase in treatment effects of psychological treatments for PTSD by personalization. The results indicate that there is comparatively high heterogeneity in treatment effects across all psychological treatment and control groups, which in turn allow personalizing psychological treatments by using treatment selection approaches.

Praxisorientierte Forschung in der Psychotherapie

Zusammenfassung. In den letzten Jahrzehnten hat sich durch randomisiert-kontrollierte Studien (RCTs) eine breite Evidenzbasis von Psychotherapie mit mittleren bis großen Effekten für verschiedene psychische Störungen gebildet. Neben der Bestimmung dieser Wirksamkeit („Efficacy“) ebneten Studien zur Wirksamkeit unter alltäglichen Routinebedingungen („Effectiveness“) historisch den Weg zur Entwicklung eines praxisorientierten Forschungsparadigmas. Im Beitrag wird argumentiert, dass im Rahmen dieses Paradigmas praxisbasierte Studien eine wertvolle Ergänzung zu RCTs darstellen, da sie existierende Probleme in der Psychotherapieforschung adressieren können. In der gegenwärtigen praxisorientierten Forschung liefern dabei neue Ansätze aus der personalisierten Medizin und Methoden aus der ‚Computational Psychiatry‘ wichtige Anhaltspunkte zur Optimierung von Effekten in der Psychotherapie. Im Kontext der Personalisierung werden bspw. klinische multivariable Prädiktionsmodelle entwickelt, welche durch Rückmeldeschleifen an Praktiker_innen kurzfristig ein evidenzbasiertes Outcome-Monitoring ermöglicht und langfristig das Praxis-Forschungsnetzwerk in Deutschland stärkt. Am Ende des Beitrags werden zukünftige Richtungen für die praxisorientierte Forschung im Sinne des ‘Precision Mental Health Care’ -Paradigmas abgeleitet und diskutiert.

Movement-based patient-therapist attunement in psychological therapy and its association with early change

Objective

Attunement is a novel measure of nonverbal synchrony reflecting the duration of the present moment shared by two interaction partners. This study examined its association with early change in outpatient psychotherapy.

Methods

Automated video analysis based on motion energy analysis (MEA) and cross-correlation of the movement time-series of patient and therapist was conducted to calculate movement synchrony for N = 161 outpatients. Movement-based attunement was defined as the range of connected time lags with significant synchrony. Latent change classes in the HSCL-11 were identified with growth mixture modeling (GMM) and predicted by pre-treatment covariates and attunement using multilevel multinomial regression.

Results

GMM identified four latent classes: high impairment, no change (Class 1); high impairment, early response (Class 2); moderate impairment (Class 3); and low impairment (Class 4). Class 2 showed the strongest attunement, the largest early response, and the best outcome. Stronger attunement was associated with a higher likelihood of membership in Class 2 (b = 0.313, p = .007), Class 3 (b = 0.251, p = .033), and Class 4 (b = 0.275, p = .043) compared to Class 1. For highly impaired patients, the probability of no early change (Class 1) decreased and the probability of early response (Class 2) increased as a function of attunement.

Conclusions

Among patients with high impairment, stronger patient-therapist attunement was associated with early response, which predicted a better treatment outcome. Video-based assessment of attunement might provide new information for therapists not available from self-report questionnaires and support therapists in their clinical decision-making.

Accelerating the impact of artificial intelligence in mental healthcare through implementation science

Background

The implementation of artificial intelligence (AI) in mental healthcare offers a potential solution to some of the problems associated with the availability, attractiveness, and accessibility of mental healthcare services. However, there are many knowledge gaps regarding how to implement and best use AI to add value to mental healthcare services, providers, and consumers. The aim of this paper is to identify challenges and opportunities for AI use in mental healthcare and to describe key insights from implementation science of potential relevance to understand and facilitate AI implementation in mental healthcare.

Methods

The paper is based on a selective review of articles concerning AI in mental healthcare and implementation science.

Results

Research in implementation science has established the importance of considering and planning for implementation from the start, the progression of implementation through different stages, and the appreciation of determinants at multiple levels. Determinant frameworks and implementation theories have been developed to understand and explain how different determinants impact on implementation. AI research should explore the relevance of these determinants for AI implementation. Implementation strategies to support AI implementation must address determinants specific to AI implementation in mental health. There might also be a need to develop new theoretical approaches or augment and recontextualize existing ones. Implementation outcomes may have to be adapted to be relevant in an AI implementation context.

Conclusion

Knowledge derived from implementation science could provide an important starting point for research on implementation of AI in mental healthcare. This field has generated many insights and provides a broad range of theories, frameworks, and concepts that are likely relevant for this research. However, when taking advantage of the existing knowledge basis, it is important to also be explorative and study AI implementation in health and mental healthcare as a new phenomenon in its own right since implementing AI may differ in various ways from implementing evidence-based practices in terms of what implementation determinants, strategies, and outcomes are most relevant. Plain Language Summary: The implementation of artificial intelligence (AI) in mental healthcare offers a potential solution to some of the problems associated with the availability, attractiveness, and accessibility of mental healthcare services. However, there are many knowledge gaps concerning how to implement and best use AI to add value to mental healthcare services, providers, and consumers. This paper is based on a selective review of articles concerning AI in mental healthcare and implementation science, with the aim to identify challenges and opportunities for the use of AI in mental healthcare and describe key insights from implementation science of potential relevance to understand and facilitate AI implementation in mental healthcare. AI offers opportunities for identifying the patients most in need of care or the interventions that might be most appropriate for a given population or individual. AI also offers opportunities for supporting a more reliable diagnosis of psychiatric disorders and ongoing monitoring and tailoring during the course of treatment. However, AI implementation challenges exist at organizational/policy, individual, and technical levels, making it relevant to draw on implementation science knowledge for understanding and facilitating implementation of AI in mental healthcare. Knowledge derived from implementation science could provide an important starting point for research on AI implementation in mental healthcare. This field has generated many insights and provides a broad range of theories, frameworks, and concepts that are likely relevant for this research.

Measurement-Based and Data-Informed Psychological Therapy

Outcome measurement in the field of psychotherapy has developed considerably in the last decade. This review discusses key issues related to outcome measurement, modeling, and implementation of data-informed and measurement-based psychological therapy. First, an overview is provided, covering the rationale of outcome measurement by acknowledging some of the limitations of clinical judgment. Second, different models of outcome measurement are discussed, including pre-post, session-by-session, and higher-resolution intensive outcome assessments. Third, important concepts related to modeling patterns of change are addressed, including early response, dose-response, and nonlinear change. Furthermore, rational and empirical decision tools are discussed as the foundation for measurement-based therapy. Fourth, examples of clinical applications are presented, which show great promise to support the personalization of therapy and to prevent treatment failure. Finally, we build on continuous outcome measurement as the basis for a broader understanding of clinical concepts and data-driven clinical practice in the future. Expected final online publication date for the Annual Review of Clinical Psychology, Volume 18 is May 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Predicting therapy outcome in a digital mental health intervention for depression and anxiety: A machine learning approach

Objective

Predicting the outcomes of individual participants for treatment interventions appears central to making mental healthcare more tailored and effective. However, little work has been done to investigate the performance of machine learning-based predictions within digital mental health interventions. Therefore, this study evaluates the performance of machine learning in predicting treatment response in a digital mental health intervention designed for treating depression and anxiety.

Methods

Several algorithms were trained based on the data of 970 participants to predict a significant reduction in depression and anxiety symptoms using clinical and sociodemographic variables. As a random forest classifier performed best over cross-validation, it was used to predict the outcomes of 279 new participants.

Results

The random forest achieved an accuracy of 0.71 for the test set (base rate: 0.67, area under curve (AUC): 0.60, p = 0.001, balanced accuracy: 0.60). Additionally, predicted non-responders showed less average reduction of their Patient Health Questionnaire-9 (PHQ-9) (-2.7, p = 0.004) and General Anxiety Disorder Screener-7 values (-3.7, p < 0.001) compared to responders. Besides pre-treatment Patient Health Questionnaire-9 and General Anxiety Disorder Screener-7 values, the self-reported motivation, type of referral into the programme (self vs. healthcare provider) as well as Work Productivity and Activity Impairment Questionnaire items contributed most to the predictions.

Conclusions

This study provides evidence that social-demographic and clinical variables can be used for machine learning to predict therapy outcomes within the context of a therapist-supported digital mental health intervention. Despite the overall moderate performance, this appears promising as these predictions can potentially improve the outcomes of non-responders by monitoring their progress or by offering alternative or additional treatment.

Mental health trajectories of individuals and families following the COVID-19 pandemic: Study protocol of a longitudinal investigation and prevention program

Introduction

Many adults, adolescents and children are suffering from persistent stress symptoms in the face of the COVID-19 pandemic. This study aims to characterize long-term trajectories of mental health and to reduce the transition to manifest mental disorders by means of a stepped care program for indicated prevention.

Methods and analysis

Using a prospective-longitudinal design, we will assess the mental strain of the pandemic using the Patient Health Questionnaire, Strength and Difficulties Questionnaire and Spence Child Anxiety Scale. Hair samples will be collected to assess cortisol as a biological stress marker of the previous months. Additionally, we will implement a stepped-care program with online- and face-to-face-interventions for adults, adolescents, and children. After that we will assess long-term trajectories of mental health at 6, 12, and 24 months follow-up. The primary outcome will be psychological distress (depression, anxiety and somatoform symptoms). Data will be analyzed with general linear model and machine learning. This study will contribute to the understanding of the impact of the COVID-19 pandemic on mental health. The evaluation of the stepped-care program and longitudinal investigation will inform clinicians and mental health stakeholders on populations at risk, disease trajectories and the sufficiency of indicated prevention to ameliorate the mental strain of the pandemic.

Ethics and dissemination

The study is performed according to the Declaration of Helsinki and was approved by the Ethics Committee of the Department of Psychology at the Humboldt Universität zu Berlin (no. 2020-35).

Trial registration number

DRKS00023220.

Personalization of Treatment for Patients with Childhood-Abuse-Related Posttraumatic Stress Disorder

BACKGROUND

Differences in effectiveness among treatments for posttraumatic stress disorder (PTSD) are typically small. Given the variation between patients in treatment response, personalization offers a new way to improve treatment outcomes. The aim of this study was to identify predictors of psychotherapy outcome in PTSD and to combine these into a personalized advantage index (PAI).

METHODS

We used data from a recent randomized controlled trial comparing prolonged exposure (PE; n = 48), intensified PE (iPE; n = 51), and skills training (STAIR), followed by PE (n = 50) in 149 patients with childhood-abuse-related PTSD (CA-PTSD). Outcome measures were clinician-assessed and self-reported PTSD symptoms. Predictors were identified in the exposure therapies (PE and iPE) and STAIR+PE separately using random forests and subsequent bootstrap procedures. Next, these predictors were used to calculate PAI and to retrospectively determine optimal and suboptimal treatment in a leave-one-out cross-validation approach.

RESULTS

More depressive symptoms, less social support, more axis-1 diagnoses, and higher severity of childhood sexual abuse were predictors of worse treatment outcomes in PE and iPE. More emotion regulation difficulties, lower general health status, and higher baseline PTSD symptoms were predictors of worse treatment outcomes in STAIR+PE. Randomization to optimal treatment based on these predictors resulted in more improvement than suboptimal treatment in clinician assessed (Cohens' d = 0.55) and self-reported PTSD symptoms (Cohens' d = 0.47).

CONCLUSION

Personalization based on PAI is a promising tool to improve therapy outcomes in patients with CA-PTSD. Further studies are needed to replicate findings in prospective studies.

The promise of machine learning in predicting treatment outcomes in psychiatry

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

For whom should psychotherapy focus on problem coping? A machine learning algorithm for treatment personalization

OBJECTIVE

We aimed to develop and test an algorithm for individual patient predictions of problem coping experiences (PCE) (i.e., patients' understanding and ability to deal with their problems) effects in cognitive-behavioral therapy. Method: In an outpatient sample with a variety of diagnoses (n=1010), we conducted Dynamic Structural Equation Modelling to estimate within-patient cross-lagged PCE effects on outcome during the first ten sessions. In a randomly selected training sample (2/3 of the cases), we tried different machine learning algorithms (i.e., ridge regression, LASSO, elastic net, and random forest) to predict PCE effects (i.e., the degree to which PCE was a time-lagged predictor of symptoms), using baseline demographic, diagnostic, and clinically-relevant patient features. Then, we validated the best algorithm on a test sample (1/3 of the cases).

RESULTS

The random forest algorithm performed best, explaining 14.7% of PCE effects variance in the training set. The results remained stable in the test set, explaining 15.4% of PCE effects variance.

CONCLUSIONS

The results show the suitability to perform individual predictions of process effects, based on patients' initial information. If the results are replicated, the algorithm might have the potential to be implemented in clinical practice by integrating it into monitoring and therapist feedback systems.

Predicting optimal treatment outcomes using the Personalized Advantage Index for patients with persistent somatic symptoms

Because individual patients with persistent somatic symptoms (PSS) respond differently to treatments, a better understanding of the factors that predict therapy outcomes are of high importance. Aggregating a wide selection of information into the treatment-decision process is a challenge for clinicians. Using the Personalized Advantage Index (PAI) this study aims to deal with this. Methods: Data from a multicentre RCT comparing CBT (N = 128) versus CBT enriched with emotion regulation training (ENCERT) (N = 126) for patients diagnosed with somatic symptom disorder were used to identify based on two machine learning approaches predictors of therapy outcomes. The identified predictors were used to calculate the PAI. Results: Five treatment unspecific predictors (pre-treatment somatic symptom severity, depression, symptom disability, health-related quality of life, age) and five treatment specific moderators (global functioning, early childhood traumatic events, gender, health anxiety, emotion regulation skills) were identified. Individuals assigned to their PAI-indicated optimal treatment had significantly lower somatic symptom severity at the end of therapy compared to those randomised to their non-optimal condition. Conclusion: Allowing patients to choose a personalised treatment seems to be meaningful. This could help to improve outcomes for PSS and reduce its high costs to the health care system.

Process-outcome associations in an interdisciplinary treatment for chronic pain and comorbid mental disorders based on Acceptance and Commitment Therapy

OBJECTIVE

Numerous studies support the effectiveness of Acceptance and Commitment Therapy (ACT) for chronic pain, yet little research has been conducted about its underlying mechanisms of change, especially regarding patients with comorbid mental disorders. The present investigation addressed this issue by examining associations of processes targeted by ACT (pain acceptance, mindfulness, psychological flexibility) and clinical outcomes (pain intensity, somatic symptoms, physical health, mental health, depression, general anxiety).

SUBJECTS

Participants were 109 patients who attended an ACT-based interdisciplinary treatment program for chronic pain and comorbid mental disorders in a routine care psychiatric day hospital.

METHODS

Pre- to post-treatment differences in processes and outcomes were examined with Wilcoxon signed-rank tests and effect size r. Associations between changes in processes and changes in outcomes were analyzed with correlation and multiple regression analyses.

RESULTS

Pre- to post-treatment effect sizes were mostly moderate to large (r between |0.21| and |0.62|). Associations between changes in processes and changes in outcomes were moderate to large for both, bivariate correlations (r between |0.30| and |0.54|) and shared variances accounting for all three processes combined (R2 between 0.21 and 0.29).

CONCLUSION

The present investigation suggests that changes in pain acceptance, mindfulness and psychological flexibility are meaningfully associated with changes in clinical outcomes. It provides evidence on particular process-outcome associations that had not been investigated in this way before. The focus on comorbid mental disorders informs clinicians about a population of chronic pain patients that often has a severe course of illness and has seldom been studied.

The Use of Digitally Assessed Stress Levels to Model Change Processes in CBT - A Feasibility Study on Seven Case Examples

In psychotherapy research, the measurement of treatment processes and outcome are predominantly based on self-reports. However, given new technological developments, other potential sources can be considered to improve measurements. In a feasibility study, we examined whether Ecological Momentary Assessments (EMA) using digital phenotyping (stress level) can be a valuable tool to investigate change processes during cognitive behavioral therapy (CBT). Seven outpatients undergoing psychological treatment were assessed using EMA. Continuous stress levels (heart rate variability) were assessed via fitness trackers (Garmin) every 3 min over a 2-week time period (6,720 measurements per patient). Time-varying change point autoregressive (TVCP-AR) models were employed to detect both gradual and abrupt changes in stress levels. Results for seven case examples indicate differential patterns of change processes in stress. More precisely, inertia of stress level changed gradually over time in one of the participants, whereas the other participants showed both gradual and abrupt changes. This feasibility study demonstrates that intensive longitudinal assessments enriched by digitally assessed stress levels have the potential to investigate intra- and interindividual differences in treatment change processes and their relations to treatment outcome. Further, implementation issues and implications for future research and developments using digital phenotyping are discussed.

A scoping review of machine learning in psychotherapy research

Machine learning (ML) offers robust statistical and probabilistic techniques that can help to make sense of large amounts of data. This scoping review paper aims to broadly explore the nature of research activity using ML in the context of psychological talk therapies, highlighting the scope of current methods and considerations for clinical practice and directions for future research. Using a systematic search methodology, fifty-one studies were identified. A narrative synthesis indicates two types of studies, those who developed and tested an ML model (k=44), and those who reported on the feasibility of a particular treatment tool that uses an ML algorithm (k=7). Most model development studies used supervised learning techniques to classify or predict labeled treatment process or outcome data, whereas others used unsupervised techniques to identify clusters in the unlabeled patient or treatment data. Overall, the current applications of ML in psychotherapy research demonstrated a range of possible benefits for indications of treatment process, adherence, therapist skills and treatment response prediction, as well as ways to accelerate research through automated behavioral or linguistic process coding. Given the novelty and potential of this research field, these proof-of-concept studies are encouraging, however, do not necessarily translate to improved clinical practice (yet).