1
|
Hilbert K, Böhnlein J, Meinke C, Chavanne AV, Langhammer T, Stumpe L, Winter N, Leenings R, Adolph D, Arolt V, Bischoff S, Cwik JC, Deckert J, Domschke K, Fydrich T, Gathmann B, Hamm AO, Heinig I, Herrmann MJ, Hollandt M, Hoyer J, Junghöfer M, Kircher T, Koelkebeck K, Lotze M, Margraf J, Mumm JLM, Neudeck P, Pauli P, Pittig A, Plag J, Richter J, Ridderbusch IC, Rief W, Schneider S, Schwarzmeier H, Seeger FR, Siminski N, Straube B, Straube T, Ströhle A, Wittchen HU, Wroblewski A, Yang Y, Roesmann K, Leehr EJ, Dannlowski U, Lueken U. Lack of evidence for predictive utility from resting state fMRI data for individual exposure-based cognitive behavioral therapy outcomes: A machine learning study in two large multi-site samples in anxiety disorders. Neuroimage 2024; 295:120639. [PMID: 38796977 DOI: 10.1016/j.neuroimage.2024.120639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2024] [Revised: 05/03/2024] [Accepted: 05/06/2024] [Indexed: 05/29/2024] Open
Abstract
Data-based predictions of individual Cognitive Behavioral Therapy (CBT) treatment response are a fundamental step towards precision medicine. Past studies demonstrated only moderate prediction accuracy (i.e. ability to discriminate between responders and non-responders of a given treatment) when using clinical routine data such as demographic and questionnaire data, while neuroimaging data achieved superior prediction accuracy. However, these studies may be considerably biased due to very limited sample sizes and bias-prone methodology. Adequately powered and cross-validated samples are a prerequisite to evaluate predictive performance and to identify the most promising predictors. We therefore analyzed resting state functional magnet resonance imaging (rs-fMRI) data from two large clinical trials to test whether functional neuroimaging data continues to provide good prediction accuracy in much larger samples. Data came from two distinct German multicenter studies on exposure-based CBT for anxiety disorders, the Protect-AD and SpiderVR studies. We separately and independently preprocessed baseline rs-fMRI data from n = 220 patients (Protect-AD) and n = 190 patients (SpiderVR) and extracted a variety of features, including ROI-to-ROI and edge-functional connectivity, sliding-windows, and graph measures. Including these features in sophisticated machine learning pipelines, we found that predictions of individual outcomes never significantly differed from chance level, even when conducting a range of exploratory post-hoc analyses. Moreover, resting state data never provided prediction accuracy beyond the sociodemographic and clinical data. The analyses were independent of each other in terms of selecting methods to process resting state data for prediction input as well as in the used parameters of the machine learning pipelines, corroborating the external validity of the results. These similar findings in two independent studies, analyzed separately, urge caution regarding the interpretation of promising prediction results based on neuroimaging data from small samples and emphasizes that some of the prediction accuracies from previous studies may result from overestimation due to homogeneous data and weak cross-validation schemes. The promise of resting-state neuroimaging data to play an important role in the prediction of CBT treatment outcomes in patients with anxiety disorders remains yet to be delivered.
Collapse
Affiliation(s)
- Kevin Hilbert
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; Department of Psychology, HMU Health and Medical University Erfurt, Erfurt, Germany
| | - Joscha Böhnlein
- Institute for Translational Psychiatry, University of Münster, Germany.
| | - Charlotte Meinke
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Alice V Chavanne
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; Université Paris-Saclay, INSERM U1299 "Trajectoires développementales et psychiatrie", CNRS UMR 9010 Centre Borelli, Ecole Normale Supérieure Paris-Saclay, France
| | - Till Langhammer
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Lara Stumpe
- Institute for Translational Psychiatry, University of Münster, Germany
| | - Nils Winter
- Institute for Translational Psychiatry, University of Münster, Germany
| | - Ramona Leenings
- Institute for Translational Psychiatry, University of Münster, Germany
| | - Dirk Adolph
- Mental Health Research and Treatment Center, Faculty of Psychology, Ruhr-Universität Bochum, Bochum, Germany
| | - Volker Arolt
- Institute for Translational Psychiatry, University of Münster, Germany
| | - Sophie Bischoff
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Jan C Cwik
- Department of Clinical Psychology and Psychotherapy, Faculty of Human Sciences, Universität zu Köln, Germany
| | - Jürgen Deckert
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
| | - Katharina Domschke
- Department of Psychiatry and Psychotherapy, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Thomas Fydrich
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Bettina Gathmann
- Institute of Medical Psychology and Systems Neuroscience, University of Münster, Germany
| | - Alfons O Hamm
- Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany
| | - Ingmar Heinig
- Institute of Clinical Psychology & Psychotherapy, Technische Universität Dresden, Dresden, Germany
| | - Martin J Herrmann
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
| | - Maike Hollandt
- Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany
| | - Jürgen Hoyer
- Institute of Clinical Psychology & Psychotherapy, Technische Universität Dresden, Dresden, Germany
| | - Markus Junghöfer
- Institute for Biomagnetism and Biosignalanalysis, University of Münster, Germany
| | - Tilo Kircher
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
| | - Katja Koelkebeck
- LVR-University-Hospital Essen, Department of Psychiatry and Psychotherapy, University of Duisburg-Essen, Essen, Germany
| | - Martin Lotze
- Functional Imaging Unit. Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, Greifswald, Germany
| | - Jürgen Margraf
- Mental Health Research and Treatment Center, Faculty of Psychology, Ruhr-Universität Bochum, Bochum, Germany
| | - Jennifer L M Mumm
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Peter Neudeck
- Protect-AD Study Site Cologne, Cologne, Germany; Institut für Klinische Psychologie und Psychotherapie, TU Chemnitz, Germany
| | - Paul Pauli
- Department of Psychology, University of Würzburg, Würzburg, Germany
| | - Andre Pittig
- Translational Psychotherapy, Institute of Psychology, University of Göttingen, Germany
| | - Jens Plag
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, Berlin, Germany; Department of Psychiatry, Psychotherapy and Psychosomatics, Alexianer Krankenhaus Hedwigshoehe, St. Hedwig Kliniken, Berlin, Germany
| | - Jan Richter
- Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany; Department of Experimental Psychopathology, University of Hildesheim, Hildesheim, Germany
| | | | - Winfried Rief
- Department of Clinical Psychology and Psychotherapy, Faculty of Psychology & Center for Mind, Brain and Behavior - CMBB, Philipps-University of Marburg, Marburg, Germany
| | - Silvia Schneider
- Faculty of Psychology, Clinical Child and Adolescent Psychology, Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany
| | - Hanna Schwarzmeier
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
| | - Fabian R Seeger
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
| | - Niklas Siminski
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
| | - Benjamin Straube
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
| | - Thomas Straube
- Institute of Psychology, Unit of Clinical Psychology and Psychotherapy in Childhood and Adolescence, University of Osnabrueck, Osnabruck, Germany
| | - Andreas Ströhle
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | | | - Adrian Wroblewski
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
| | - Yunbo Yang
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
| | - Kati Roesmann
- Institute for Biomagnetism and Biosignalanalysis, University of Münster, Germany; Institute of Psychology, Unit of Clinical Psychology and Psychotherapy in Childhood and Adolescence, University of Osnabrueck, Osnabruck, Germany
| | - Elisabeth J Leehr
- Institute for Translational Psychiatry, University of Münster, Germany
| | - Udo Dannlowski
- Institute for Translational Psychiatry, University of Münster, Germany
| | - Ulrike Lueken
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; German Center for Mental Health (DZPG), partner site Berlin/Potsdam, Germany
| |
Collapse
|
2
|
Ping Y. Experience in psychological counseling supported by artificial intelligence technology. Technol Health Care 2024:THC230809. [PMID: 38968060 DOI: 10.3233/thc-230809] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/07/2024]
Abstract
BACKGROUND In recent years, artificial intelligence (AI) technology has been continuously advancing and finding extensive applications, with one of its core technologies, machine learning, being increasingly utilized in the field of healthcare. OBJECTIVE This research aims to explore the role of Artificial Intelligence (AI) technology in psychological counseling and utilize machine learning algorithms to predict counseling outcomes. METHODS Firstly, by employing natural language processing techniques to analyze user conversations with AI chatbots, researchers can gain insights into the psychological states and needs of users during the counseling process. This involves detailed analysis using text analysis, sentiment analysis, and other relevant techniques. Subsequently, machine learning algorithms are used to establish predictive models that forecast counseling outcomes and user satisfaction based on data such as user language, emotions, and behavior. These predictive results can assist counselors or AI chatbots in adjusting counseling strategies, thereby enhancing counseling effectiveness and user experience. Additionally, this study explores the potential and prospects of AI technology in the field of psychological counseling. RESULTS The research findings indicate that the designed machine learning models achieve an accuracy rate of approximately 89% in analyzing psychological conditions. This demonstrates significant innovation and breakthroughs in AI technology. Consequently, AI technology will gradually become a highly important tool and method in the field of psychological counseling. CONCLUSION In the future, AI chatbots will become more intelligent and personalized, providing users with precise, efficient, and convenient psychological counseling services. The results of this research provide valuable technical insights for further improving AI-supported psychological counseling, contributing positively to the application and development of AI technology.
Collapse
|
3
|
Jankowsky K, Krakau L, Schroeders U, Zwerenz R, Beutel ME. Predicting treatment response using machine learning: A registered report. BRITISH JOURNAL OF CLINICAL PSYCHOLOGY 2024; 63:137-155. [PMID: 38111213 DOI: 10.1111/bjc.12452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 11/27/2023] [Indexed: 12/20/2023]
Abstract
OBJECTIVE Previous research on psychotherapy treatment response has mainly focused on outpatients or clinical trial data which may have low ecological validity regarding naturalistic inpatient samples. To reduce treatment failures by proactively screening for patients at risk of low treatment response, gain more knowledge about risk factors and to evaluate treatments, accurate insights about predictors of treatment response in naturalistic inpatient samples are needed. METHODS We compared the performance of different machine learning algorithms in predicting treatment response, operationalized as a substantial reduction in symptom severity as expressed in the Patient Health Questionnaire Anxiety and Depression Scale. To achieve this goal, we used different sets of variables-(a) demographics, (b) physical indicators, (c) psychological indicators and (d) treatment-related variables-in a naturalistic inpatient sample (N = 723) to specify their joint and unique contribution to treatment success. RESULTS There was a strong link between symptom severity at baseline and post-treatment (R2 = .32). When using all available variables, both machine learning algorithms outperformed the linear regressions and led to an increment in predictive performance of R2 = .12. Treatment-related variables were the most predictive, followed psychological indicators. Physical indicators and demographics were negligible. CONCLUSIONS Treatment response in naturalistic inpatient settings can be predicted to a considerable degree by using baseline indicators. Regularization via machine learning algorithms leads to higher predictive performances as opposed to including nonlinear and interaction effects. Heterogenous aspects of mental health have incremental predictive value and should be considered as prognostic markers when modelling treatment processes.
Collapse
Affiliation(s)
| | - Lina Krakau
- Department of Psychosomatic Medicine and Psychotherapy, University Medical Center Mainz, Mainz, Germany
| | | | - Rüdiger Zwerenz
- Department of Psychosomatic Medicine and Psychotherapy, University Medical Center Mainz, Mainz, Germany
| | - Manfred E Beutel
- Department of Psychosomatic Medicine and Psychotherapy, University Medical Center Mainz, Mainz, Germany
| |
Collapse
|
4
|
Hammelrath L, Hilbert K, Heinrich M, Zagorscak P, Knaevelsrud C. Select or adjust? How information from early treatment stages boosts the prediction of non-response in internet-based depression treatment. Psychol Med 2024; 54:1641-1650. [PMID: 38087867 DOI: 10.1017/s0033291723003537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
BACKGROUND Internet-based interventions produce comparable effectiveness rates as face-to-face therapy in treating depression. Still, more than half of patients do not respond to treatment. Machine learning (ML) methods could help to overcome these low response rates by predicting therapy outcomes on an individual level and tailoring treatment accordingly. Few studies implemented ML algorithms in internet-based depression treatment using baseline self-report data, but differing results hinder inferences on clinical practicability. This work compares algorithms using features gathered at baseline or early in treatment in their capability to predict non-response to a 6-week online program targeting depression. METHODS Our training and test sample encompassed 1270 and 318 individuals, respectively. We trained random forest algorithms on self-report and process features gathered at baseline and after 2 weeks of treatment. Non-responders were defined as participants not fulfilling the criteria for reliable and clinically significant change on PHQ-9 post-treatment. Our benchmark models were logistic regressions trained on baseline PHQ-9 sum or PHQ-9 early change, using 100 iterations of randomly sampled 80/20 train-test-splits. RESULTS Best performances were reached by our models involving early treatment characteristics (recall: 0.75-0.76; AUC: 0.71-0.77). Therapeutic alliance and early symptom change constituted the most important predictors. Models trained on baseline data were not significantly better than our benchmark. CONCLUSIONS Fair accuracies were only attainable by involving information from early treatment stages. In-treatment adaptation, instead of a priori selection, might constitute a more feasible approach for improving response when relying on easily accessible self-report features. Implementation trials are needed to determine clinical usefulness.
Collapse
Affiliation(s)
- Leona Hammelrath
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | - Kevin Hilbert
- Department of Psychology, Health and Medical University Erfurt, Erfurt, Germany
| | - Manuel Heinrich
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | - Pavle Zagorscak
- Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany
| | | |
Collapse
|
5
|
Haque UM, Kabir E, Khanam R. Early detection of paediatric and adolescent obsessive-compulsive, separation anxiety and attention deficit hyperactivity disorder using machine learning algorithms. Health Inf Sci Syst 2023; 11:31. [PMID: 37489154 PMCID: PMC10363094 DOI: 10.1007/s13755-023-00232-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Accepted: 07/04/2023] [Indexed: 07/26/2023] Open
Abstract
Purpose Mental health issues of young minds are at the threshold of all development and possibilities. Obsessive-compulsive disorder (OCD), separation anxiety disorder (SAD), and attention deficit hyperactivity disorder (ADHD) are three of the most common mental illness affecting children and adolescents. Several studies have been conducted on approaches for recognising OCD, SAD and ADHD, but their accuracy is inadequate due to limited features and participants. Therefore, the purpose of this study is to investigate the approach using machine learning (ML) algorithms with 1474 features from Australia's nationally representative mental health survey of children and adolescents. Methods Based on the internal cross-validation (CV) score of the Tree-based Pipeline Optimization Tool (TPOTClassifier), the dataset has been examined using three of the most optimal algorithms, including Random Forest (RF), Decision Tree (DT), and Gaussian Naïve Bayes (GaussianNB). Results GaussianNB performs well in classifying OCD with 91% accuracy, 76% precision, and 96% specificity as well as in detecting SAD with 79% accuracy, 62% precision, 91% specificity. RF outperformed all other methods in identifying ADHD with 91% accuracy, 94% precision, and 99% specificity. Conclusion Using Streamlit and Python a web application was developed based on the findings of the analysis. The application will assist parents/guardians and school officials in detecting mental illnesses early in their children and adolescents using signs and symptoms to start the treatment at the earliest convenience.
Collapse
Affiliation(s)
- Umme Marzia Haque
- School of Mathematics, Physics and Computing, University of Southern Queensland, Toowoomba, Australia
| | - Enamul Kabir
- School of Mathematics, Physics and Computing, University of Southern Queensland, Toowoomba, Australia
| | - Rasheda Khanam
- School of Business, University of Southern Queensland, Toowoomba, Australia
| |
Collapse
|
6
|
Franken K, ten Klooster P, Bohlmeijer E, Westerhof G, Kraiss J. Predicting non-improvement of symptoms in daily mental healthcare practice using routinely collected patient-level data: a machine learning approach. Front Psychiatry 2023; 14:1236551. [PMID: 37817829 PMCID: PMC10560743 DOI: 10.3389/fpsyt.2023.1236551] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/11/2023] [Accepted: 09/11/2023] [Indexed: 10/12/2023] Open
Abstract
Objectives Anxiety and mood disorders greatly affect the quality of life for individuals worldwide. A substantial proportion of patients do not sufficiently improve during evidence-based treatments in mental healthcare. It remains challenging to predict which patients will or will not benefit. Moreover, the limited research available on predictors of treatment outcomes comes from efficacy RCTs with strict selection criteria which may limit generalizability to a real-world context. The current study evaluates the performance of different machine learning (ML) models in predicting non-improvement in an observational sample of patients treated in routine specialized mental healthcare. Methods In the current longitudinal exploratory prediction study diagnosis-related, sociodemographic, clinical and routinely collected patient-reported quantitative outcome measures were acquired during treatment as usual of 755 patients with a primary anxiety, depressive, obsessive compulsive or trauma-related disorder in a specialized outpatient mental healthcare center. ML algorithms were trained to predict non-response (< 0.5 standard deviation improvement) in symptomatic distress 6 months after baseline. Different models were trained, including models with and without early change scores in psychopathology and well-being and models with a trimmed set of predictor variables. Performance of trained models was evaluated in a hold-out sample (30%) as a proxy for unseen data. Results ML models without early change scores performed poorly in predicting six-month non-response in the hold-out sample with Area Under the Curves (AUCs) < 0.63. Including early change scores slightly improved the models' performance (AUC range: 0.68-0.73). Computationally-intensive ML models did not significantly outperform logistic regression (AUC: 0.69). Reduced prediction models performed similar to the full prediction models in both the models without (AUC: 0.58-0.62 vs. 0.58-0.63) and models with early change scores (AUC: 0.69-0.73 vs. 0.68-0.71). Across different ML algorithms, early change scores in psychopathology and well-being consistently emerged as important predictors for non-improvement. Conclusion Accurately predicting treatment outcomes in a mental healthcare context remains challenging. While advanced ML algorithms offer flexibility, they showed limited additional value compared to traditional logistic regression in this study. The current study confirmed the importance of taking early change scores in both psychopathology and well-being into account for predicting longer-term outcomes in symptomatic distress.
Collapse
Affiliation(s)
- Katinka Franken
- Department of Psychology, Health and Technology, Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, Netherlands
| | | | | | | | | |
Collapse
|
7
|
Rollmann I, Gebhardt N, Stahl-Toyota S, Simon J, Sutcliffe M, Friederich HC, Nikendei C. Systematic review of machine learning utilization within outpatient psychodynamic psychotherapy research. Front Psychiatry 2023; 14:1055868. [PMID: 37229386 PMCID: PMC10203389 DOI: 10.3389/fpsyt.2023.1055868] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 04/17/2023] [Indexed: 05/27/2023] Open
Abstract
Introduction Although outpatient psychodynamic psychotherapy is effective, there has been no improvement in treatment success in recent years. One way to improve psychodynamic treatment could be the use of machine learning to design treatments tailored to the individual patient's needs. In the context of psychotherapy, machine learning refers mainly to various statistical methods, which aim to predict outcomes (e.g., drop-out) of future patients as accurately as possible. We therefore searched various literature for all studies using machine learning in outpatient psychodynamic psychotherapy research to identify current trends and objectives. Methods For this systematic review, we applied the Preferred Reporting Items for systematic Reviews and Meta-Analyses Guidelines. Results In total, we found four studies that used machine learning in outpatient psychodynamic psychotherapy research. Three of these studies were published between 2019 and 2021. Discussion We conclude that machine learning has only recently made its way into outpatient psychodynamic psychotherapy research and researchers might not yet be aware of its possible uses. Therefore, we have listed a variety of perspectives on how machine learning could be used to increase treatment success of psychodynamic psychotherapies. In doing so, we hope to give new impetus to outpatient psychodynamic psychotherapy research on how to use machine learning to address previously unsolved problems.
Collapse
|
8
|
Rosellini AJ, Andrea AM, Galiano CS, Hwang I, Brown TA, Luedtke A, Kessler RC. Developing Transdiagnostic Internalizing Disorder Prognostic Indices for Outpatient Cognitive Behavioral Therapy. Behav Ther 2023; 54:461-475. [PMID: 37088504 PMCID: PMC10126479 DOI: 10.1016/j.beth.2022.11.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/17/2022] [Revised: 11/03/2022] [Accepted: 11/29/2022] [Indexed: 12/12/2022]
Abstract
A growing literature is devoted to understanding and predicting heterogeneity in response to cognitive behavioral therapy (CBT), including using supervised machine learning to develop prognostic models that could be used to inform treatment planning. The current study developed CBT prognostic models using data from a broad dimensionally oriented pretreatment assessment (324 predictors) of 1,210 outpatients with internalizing psychopathology. Super learning was implemented to develop prognostic indices for three outcomes assessed at 12-month follow-up: principal diagnosis improvement (attained by 65.8% of patients), principal diagnosis remission (56.8%), and transdiagnostic full remission (14.3%). The models for principal diagnosis remission and transdiagnostic remission performed best (AUROCs = 0.71-0.73). Calibration was modest for all three models. Three-quarters (77.3%) of patients in the top tertile of the predicted probability distribution achieved principal diagnosis remission, compared to 35.0% in the bottom tertile. One-third (35.3%) of patients in the top two deciles of predicted probabilities for transdiagnostic complete remission achieved this outcome, compared to 2.7% in the bottom tertile. Key predictors included principal diagnosis severity, social anxiety diagnosis/severity, hopelessness, temperament, and global impairment. While additional work is needed to improve performance, integration of CBT prognostic models ultimately could lead to more effective and efficient treatment of patients with internalizing psychopathology.
Collapse
|
9
|
Huang FF, Wang PC, Yang XY, Luo J, Yang XJ, Li ZJ. Predicting responses to cognitive behavioral therapy in obsessive-compulsive disorder based on multilevel indices of rs-fMRI. J Affect Disord 2023; 323:345-353. [PMID: 36470552 DOI: 10.1016/j.jad.2022.11.073] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Revised: 10/28/2022] [Accepted: 11/20/2022] [Indexed: 12/12/2022]
Abstract
OBJECTIVE This study aimed to identify neuroimaging predictors to predict the response of cognitive behavioral therapy (CBT) in patients with obsessive-compulsive disorder (OCD) based on indices of resting-state functional magnetic resonance imaging (rs-fMRI). METHODS Fifty patients with OCD were enrolled and allocated to either high or low responder groups after CBT using a 50 % response rate as the delineator. The pre-treatment amplitude of low-frequency fluctuation (ALFF), fractional ALFF (fALFF), regional homogeneity (ReHo), and degree centrality (DC) in each cerebrum region, defined by automated anatomical labeling atlas, were extracted. Least absolute shrinkage and selection operator and logistic regression were used to select features and establish models. RESULTS The combination of multilevel rs-fMRI indices achieved the best performance, with a cross-validation area under the receiver operating characteristic curve (AUC) of 0.900. In this combined model, an increase of interquartile range (IQR) in fALFF of right inferior orbital frontal gyrus (IOFG), and ReHo of left hippocampus and superior occipital gyrus (SOG) corresponded to a 26.52 %, 38.67 % and 24.38 % increase in the possibility to be high responders of CBT, respectively. ALFF of left thalamus and ReHo of left putamen were negatively associated with the response to CBT, with a 14.30 % and 19.91 % decrease per IQR increase of the index value. CONCLUSION The combination of ALFF, fALFF and ReHo achieved a better predictive performance than separate index. Pre-treatment ALFF of the left thalamus, fALFF of the right IOFG, ReHo of the left hippocampus, SOG and putamen can be used as predictors of CBT response.
Collapse
Affiliation(s)
- Fang-Fang Huang
- Department of Clinical Psychology, The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China; Department of Preventive Medicine, School of Basic Medical Sciences, Henan University of Science and Technology, Henan, China
| | - Peng-Chong Wang
- Department of Clinical Psychology, The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Xiang-Yun Yang
- Department of Clinical Psychology, The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Jia Luo
- Department of Clinical Psychology, The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Xiao-Jie Yang
- Department of Clinical Psychology, The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Zhan-Jiang Li
- Department of Clinical Psychology, The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China.
| |
Collapse
|
10
|
Castle D, Feusner J, Laposa JM, Richter PMA, Hossain R, Lusicic A, Drummond LM. Psychotherapies and digital interventions for OCD in adults: What do we know, what do we need still to explore? Compr Psychiatry 2023; 120:152357. [PMID: 36410261 PMCID: PMC10848818 DOI: 10.1016/j.comppsych.2022.152357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 08/07/2022] [Accepted: 11/14/2022] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND Despite significant advances in the understanding and treatment of obsessive compulsive disorder (OCD), current treatment options are limited in terms of efficacy for symptom remission. Thus, assessing the potential role of iterative or alternate psychotherapies is important. Also, the potential role of digital technologies to enhance the accessibility of these therapies, should not be underestimated. We also need to embrace the idea of a more personalized treatment choice, being cognisant of clinical, genetic and neuroimaging predictors of treatment response. PROCEDURES Non-systematic review of current literature on emerging psychological and digital therapies for OCD, as well as of potential biomarkers of treatment response. FINDINGS A number of 'third wave' therapies (e.g., Acceptance and Commitment Therapy, Mindfulness-Based Cognitive Therapy) have an emerging and encouraging evidence base in OCD. Other approaches entail employment of elements of other psychotherapies such as Dialectical Behaviour Therapy; or trauma-focussed therapies such as Eye Movement Desensitisation and Reprocessing, and Imagery Rescripting and Narrative Therapy. Further strategies include Danger Ideation Reduction Therapy and Habit Reversal. For these latter approaches, large-scale randomised controlled trials are largely lacking, and the precise role of these therapies in treating people with OCD, remains to be clarified. A concentrated 4-day program (the Bergen program) has shown promising short- and long-term results. Exercise, music, and art therapy have not been adequately tested in people with OCD, but may have an adjunctive role. Digital technologies are being actively investigated for enhancing reach and efficacy of psychological therapies for OCD. Biomarkers, including genetic and neuroimaging, are starting to point to a future with more 'personalised medicine informed' treatment strategizing for OCD. CONCLUSIONS There are a number of potential psychological options for the treatment of people with OCD who do not respond adequately to exposure/response prevention or cognitive behaviour therapy. Adjunctive exercise, music, and art therapy might be useful, albeit the evidence base for these is very small. Consideration should be given to different ways of delivering such interventions, including group-based, concentrated, inpatient, or with outreach, where appropriate. Digital technologies are an emerging field with a number of potential applications for aiding the treatment of OCD. Biomarkers for treatment response determination have much potential capacity and deserve further empirical testing.
Collapse
Affiliation(s)
- David Castle
- Centre for Addiction and Mental Health, 60 White Squirrel Way, Toronto, Ontario M6J 1H4, Canada; Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada.
| | - Jamie Feusner
- Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada; Centre for Addiction and Mental Health, 250 College Street, Toronto, Ontario M5T 1RB, Canada
| | - Judith M Laposa
- Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada; Campbell Family Mental Health Research Institute, Center for Addiction and Mental Health, 100 Stokes St., Toronto, Ontario M6J 1H4, Canada
| | - Peggy M A Richter
- Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada; Frederick W Thompson Anxiety Disorders Centre, Sunnybrook Health Sciences Centre, 2075 Bayview, Toronto, Ontario M4N 3M5, Canada
| | - Rahat Hossain
- Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada
| | - Ana Lusicic
- Centre for Addiction and Mental Health, 60 White Squirrel Way, Toronto, Ontario M6J 1H4, Canada; Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada
| | - Lynne M Drummond
- Service for OCD/ BDD, South-West London and St George's NHS Trust, Glenburnie Road, London SW17 7DJ, United Kingdom
| |
Collapse
|
11
|
Lee W, Schwartz N, Bansal A, Khor S, Hammarlund N, Basu A, Devine B. A Scoping Review of the Use of Machine Learning in Health Economics and Outcomes Research: Part 2-Data From Nonwearables. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:2053-2061. [PMID: 35989154 DOI: 10.1016/j.jval.2022.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 06/10/2022] [Accepted: 07/10/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVES Despite the increasing interest in applying machine learning (ML) methods in health economics and outcomes research (HEOR), stakeholders face uncertainties in when and how ML can be used. We reviewed the recent applications of ML in HEOR. METHODS We searched PubMed for studies published between January 2020 and March 2021 and randomly chose 20% of the identified studies for the sake of manageability. Studies that were in HEOR and applied an ML technique were included. Studies related to wearable devices were excluded. We abstracted information on the ML applications, data types, and ML methods and analyzed it using descriptive statistics. RESULTS We retrieved 805 articles, of which 161 (20%) were randomly chosen. Ninety-two of the random sample met the eligibility criteria. We found that ML was primarily used for predicting future events (86%) rather than current events (14%). The most common response variables were clinical events or disease incidence (42%) and treatment outcomes (22%). ML was less used to predict economic outcomes such as health resource utilization (16%) or costs (3%). Although electronic medical records (35%) were frequently used for model development, claims data were used less frequently (9%). Tree-based methods (eg, random forests and boosting) were the most commonly used ML methods (31%). CONCLUSIONS The use of ML techniques in HEOR is growing rapidly, but there remain opportunities to apply them to predict economic outcomes, especially using claims databases, which could inform the development of cost-effectiveness models.
Collapse
Affiliation(s)
- Woojung Lee
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA.
| | - Naomi Schwartz
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Aasthaa Bansal
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Sara Khor
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Noah Hammarlund
- Department of Health Services Research, Management & Policy, University of Florida, Gainesville, FL, USA
| | - Anirban Basu
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Beth Devine
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| |
Collapse
|
12
|
Grützmann R, Klawohn J, Elsner B, Reuter B, Kaufmann C, Riesel A, Bey K, Heinzel S, Kathmann N. Error-related activity of the sensorimotor network contributes to the prediction of response to cognitive-behavioral therapy in obsessive-compulsive disorder. Neuroimage Clin 2022; 36:103216. [PMID: 36208547 PMCID: PMC9668595 DOI: 10.1016/j.nicl.2022.103216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 09/28/2022] [Accepted: 09/29/2022] [Indexed: 12/14/2022]
Abstract
BACKGROUND Although cognitive behavioral therapy is a highly effective treatment for obsessive-compulsive disorder (OCD), yielding large symptom reductions on the group level, individual treatment response varies considerably. Identification of treatment response predictors may provide important information for maximizing individual treatment response and thus achieving efficient treatment resource allocation. Here, we investigated the predictive value of previously identified biomarkers of OCD, namely the error-related activity of the supplementary motor area (SMA) and the sensorimotor network (SMN, postcentral gyrus/precuneus). METHODS Seventy-two participants with a primary diagnosis of OCD underwent functional magnetic resonance imaging (fMRI) scanning while performing a flanker task prior to receiving routine-care CBT. RESULTS Error-related BOLD response of the SMN significantly contributed to the prediction of treatment response beyond the variance accounted for by clinical and sociodemographic variables. Stronger error-related SMN activity at baseline was associated with a higher likelihood of treatment response. CONCLUSIONS The present results illustrate that the inclusion of error-related SMN activity can significantly increase treatment response prediction quality in OCD. Stronger error-related activity of the SMN may reflect the ability to activate symptom-relevant processing networks and may thus facilitate response to exposure-based CBT interventions.
Collapse
Affiliation(s)
- Rosa Grützmann
- Humboldt-Universität zu Berlin, Department of Psychology, Germany; MSB Medical School Berlin, Department of Psychology, Germany.
| | - Julia Klawohn
- Humboldt-Universität zu Berlin, Department of Psychology, Germany; MSB Medical School Berlin, Department of Medicine, Germany
| | - Björn Elsner
- Humboldt-Universität zu Berlin, Department of Psychology, Germany
| | - Benedikt Reuter
- Humboldt-Universität zu Berlin, Department of Psychology, Germany; MSB Medical School Berlin, Department of Medicine, Germany
| | | | - Anja Riesel
- Humboldt-Universität zu Berlin, Department of Psychology, Germany; Universität Hamburg, Department of Psychology, Germany
| | - Katharina Bey
- University Hospital Bonn, Department of Psychiatry and Psychotherapy, Germany
| | - Stephan Heinzel
- Humboldt-Universität zu Berlin, Department of Psychology, Germany; Freie Universität Berlin, Department of Education and Psychology, Germany
| | - Norbert Kathmann
- Humboldt-Universität zu Berlin, Department of Psychology, Germany
| |
Collapse
|
13
|
Kuckertz JM, McNally RJ, Riemann BC, Van Borkulo C, Bellet BW, Krompinger JW, Van Kirk N, Falkenstein MJ. Does the network structure of obsessive-compulsive symptoms at treatment admission identify patients at risk for non-response? Behav Res Ther 2022; 156:104151. [PMID: 35728274 PMCID: PMC9810266 DOI: 10.1016/j.brat.2022.104151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Revised: 05/12/2022] [Accepted: 06/10/2022] [Indexed: 01/05/2023]
Abstract
Exposure and response prevention is the gold-standard treatment for obsessive compulsive disorder (OCD), yet up to half of patients do not adequately respond. Thus, different approaches to identifying and intervening with non-responders are badly needed. One approach would be to better understand the functional connections among aspects of OCD symptoms and, ultimately, how to target those associations in treatment. In a large sample of patients who completed intensive treatment for OCD and related disorders (N = 1343), we examined whether differences in network structure of OCD symptom aspects existed at baseline between treatment responders versus non-responders. A network comparison test indicated a significant difference between OCD network structure for responders versus non-responders (M = 0.19, p = .02). Consistent differences emerged between responders and non-responders in how they responded to emotional distress. This pattern of associations suggests that non-responders may have been more reactive to their distress by performing compulsions, thereby worsening their functioning. By examining the association between baseline distress intolerance with other symptom aspects that presumably maintain the disorder (e.g., ritualizing), clinicians can more effectively target those associations in treatment.
Collapse
Affiliation(s)
- Jennie M Kuckertz
- Department of Psychiatry, Harvard Medical School, USA; Obsessive Compulsive Disorder Institute, McLean Hospital, USA.
| | | | | | - Claudia Van Borkulo
- Psychological Methods Department, University of Amsterdam, the Netherlands; Centre for Urban Mental Health, University of Amsterdam, the Netherlands
| | | | - Jason W Krompinger
- Department of Psychiatry, Harvard Medical School, USA; Obsessive Compulsive Disorder Institute, McLean Hospital, USA
| | - Nathaniel Van Kirk
- Department of Psychiatry, Harvard Medical School, USA; Obsessive Compulsive Disorder Institute, McLean Hospital, USA
| | - Martha J Falkenstein
- Department of Psychiatry, Harvard Medical School, USA; Obsessive Compulsive Disorder Institute, McLean Hospital, USA
| |
Collapse
|
14
|
Can we predict who will benefit from cognitive-behavioural therapy? A systematic review and meta-analysis of machine learning studies. Clin Psychol Rev 2022; 97:102193. [DOI: 10.1016/j.cpr.2022.102193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 06/29/2022] [Accepted: 08/04/2022] [Indexed: 11/23/2022]
|
15
|
Sarter L, Heider J, Witthöft M, Rief W, Kleinstäuber M. Using clinical patient characteristics to predict treatment outcome of cognitive behavior therapies for individuals with medically unexplained symptoms: A systematic review and meta-analysis. Gen Hosp Psychiatry 2022; 77:11-20. [PMID: 35390568 DOI: 10.1016/j.genhosppsych.2022.03.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Revised: 02/27/2022] [Accepted: 03/01/2022] [Indexed: 11/17/2022]
Abstract
OBJECTIVE For individuals with medically unexplained symptoms (MUS), cognitive behavioral therapy (CBT) is the best-evaluated treatment. This systematic review and meta-analyses identify clinical patient characteristics associated with the treatment outcome of CBT for MUS. METHODS A systematic literature search (PubMed, PsycInfo, Web of Science) resulted in 53 eligible studies; of these 32 studies could be included in meta-analyses. Pooled correlation coefficients between predictors and treatment outcome were calculated with a random-effects model. Moderator analyses were conducted to examine differences between subgroups of MUS and different levels of methodological study quality. RESULTS Meta-analyses demonstrated that individuals with higher symptom intensity (r = 0.38; p < 0.001), lower physical functioning (r = -0.29; p < 0.001), lower emotional and social functioning (r = -0.37; p < 0.001), more potential symptom-related incentives (r = -0.15; p = 0.001), or longer symptom duration (r = 0.10; p = 0.033) at the beginning of treatment reported less change of symptom severity until the end of therapy or higher end-of-treatment symptom severity. The pooled effect sizes did not differ between certain subgroups of MUS or between different levels of methodological quality. CONCLUSION Our findings indicated that clinical characteristics of MUS patients are associated with treatment outcome of CBT. We discuss how the results can be used to optimize and personalize future treatments for MUS.
Collapse
Affiliation(s)
- Lena Sarter
- Philipps-University Marburg, Department of Clinical Psychology and Psychotherapy, Gutenbergstraße 18, 35037 Marburg, Germany.
| | - Jens Heider
- University Koblenz-Landau, Department of Clinical Psychology and Psychotherapy, Ostbahnstraße 10, 76829 Landau, Germany.
| | - Michael Witthöft
- Johannes Gutenberg-University Mainz, Department of Clinical Psychology, Psychotherapy and Experimental Psychopathology, Wallstraße 3, 55122 Mainz, Germany.
| | - Winfried Rief
- Philipps-University Marburg, Department of Clinical Psychology and Psychotherapy, Gutenbergstraße 18, 35037 Marburg, Germany.
| | - Maria Kleinstäuber
- Department of Psychology, Emma Eccles Jones College of Education and Human Services, Utah State University, Logan, USA.
| |
Collapse
|
16
|
Neural correlates of emotional reactivity predict response to cognitive-behavioral therapy in obsessive-compulsive disorder. J Affect Disord 2022; 308:398-406. [PMID: 35427712 DOI: 10.1016/j.jad.2022.04.024] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Revised: 04/05/2022] [Accepted: 04/08/2022] [Indexed: 11/21/2022]
Abstract
BACKGROUND Examining predictive biomarkers to identify individuals who will likely benefit from a specific treatment is important for the development of targeted interventions. The late positive potential (LPP) is a neural marker of attention and elaborated stimulus processing, and increased LPP responses to negative stimuli are characteristic of pathological anxiety. The present study investigated whether LPP reactivity would prospectively predict response to cognitive-behavioral therapy (CBT), the first-line treatment for obsessive-compulsive disorder (OCD). METHODS To this end, the LPP in response to negative as compared to neutral pictures was examined in 45 patients with OCD, who underwent CBT in a naturalistic outpatient setting. LPP amplitudes were used as predictors of symptom reduction after CBT. RESULTS We found that higher LPP amplitudes to negative relative to neutral stimuli were predictive of lower self-reported OCD symptoms after completion of CBT, controlling for pre-treatment symptoms. Further, LPP reactivity was negatively correlated with self-reported habitual use of suppression in everyday life. LIMITATIONS Some participants had already begun treatment at the time of study participation. Overall, results need further replication in larger samples and standardized therapy settings. CONCLUSIONS The current findings suggest that patients with increased emotional reactivity benefit more from CBT, possibly through less avoidance of anxiety-provoking stimuli during exposure with response prevention, a crucial component in CBT for OCD. Although its clinical utility still needs to be evaluated further, the LPP constitutes a promising candidate as a prognostic marker for CBT response in OCD.
Collapse
|
17
|
Hornstein S, Forman-Hoffman V, Nazander A, Ranta K, Hilbert K. Predicting therapy outcome in a digital mental health intervention for depression and anxiety: A machine learning approach. Digit Health 2021; 7:20552076211060659. [PMID: 34868624 PMCID: PMC8637697 DOI: 10.1177/20552076211060659] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Accepted: 10/30/2021] [Indexed: 01/19/2023] Open
Abstract
Objective Predicting the outcomes of individual participants for treatment interventions appears central to making mental healthcare more tailored and effective. However, little work has been done to investigate the performance of machine learning-based predictions within digital mental health interventions. Therefore, this study evaluates the performance of machine learning in predicting treatment response in a digital mental health intervention designed for treating depression and anxiety. Methods Several algorithms were trained based on the data of 970 participants to predict a significant reduction in depression and anxiety symptoms using clinical and sociodemographic variables. As a random forest classifier performed best over cross-validation, it was used to predict the outcomes of 279 new participants. Results The random forest achieved an accuracy of 0.71 for the test set (base rate: 0.67, area under curve (AUC): 0.60, p = 0.001, balanced accuracy: 0.60). Additionally, predicted non-responders showed less average reduction of their Patient Health Questionnaire-9 (PHQ-9) (-2.7, p = 0.004) and General Anxiety Disorder Screener-7 values (-3.7, p < 0.001) compared to responders. Besides pre-treatment Patient Health Questionnaire-9 and General Anxiety Disorder Screener-7 values, the self-reported motivation, type of referral into the programme (self vs. healthcare provider) as well as Work Productivity and Activity Impairment Questionnaire items contributed most to the predictions. Conclusions This study provides evidence that social-demographic and clinical variables can be used for machine learning to predict therapy outcomes within the context of a therapist-supported digital mental health intervention. Despite the overall moderate performance, this appears promising as these predictions can potentially improve the outcomes of non-responders by monitoring their progress or by offering alternative or additional treatment.
Collapse
Affiliation(s)
- Silvan Hornstein
- Meru Health Inc, Palo Alto, CA, USA.,Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | | | | | | | - Kevin Hilbert
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
18
|
Clinical predictors of treatment response towards exposure therapy in virtuo in spider phobia: A machine learning and external cross-validation approach. J Anxiety Disord 2021; 83:102448. [PMID: 34298236 DOI: 10.1016/j.janxdis.2021.102448] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Revised: 04/07/2021] [Accepted: 07/06/2021] [Indexed: 12/29/2022]
Abstract
While being highly effective on average, exposure-based treatments are not equally effective in all patients. The a priori identification of patients with a poor prognosis may enable the application of more personalized psychotherapeutic interventions. We aimed at identifying sociodemographic and clinical pre-treatment predictors for treatment response in spider phobia (SP). N = 174 patients with SP underwent a highly standardized virtual reality exposure therapy (VRET) at two independent sites. Analyses on group-level were used to test the efficacy. We applied a state-of-the-art machine learning protocol (Random Forests) to evaluate the predictive utility of clinical and sociodemographic predictors for a priori identification of individual treatment response assessed directly after treatment and at 6-month follow-up. The reliability and generalizability of predictive models was tested via external cross-validation. Our study shows that one session of VRET is highly effective on a group-level and is among the first to reveal long-term stability of this treatment effect. Individual short-term symptom reductions could be predicted above chance, but accuracies dropped to non-significance in our between-site prediction and for predictions of long-term outcomes. With performance metrics hardly exceeding chance level and the lack of generalizability in the employed between-site replication approach, our study suggests limited clinical utility of clinical and sociodemographic predictors. Predictive models including multimodal predictors may be more promising.
Collapse
|