1
|
Delamain H, Buckman JEJ, O'Driscoll C, Suh JW, Stott J, Singh S, Naqvi SA, Leibowitz J, Pilling S, Saunders R. Predicting post-treatment symptom severity for adults receiving psychological therapy in routine care for generalised anxiety disorder: a machine learning approach. Psychiatry Res 2024; 336:115910. [PMID: 38608539 DOI: 10.1016/j.psychres.2024.115910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 04/03/2024] [Accepted: 04/08/2024] [Indexed: 04/14/2024]
Abstract
Approximately half of generalised anxiety disorder (GAD) patients do not recover from first-line treatments, and no validated prediction models exist to inform individuals or clinicians of potential treatment benefits. This study aimed to develop and validate an accurate and explainable prediction model of post-treatment GAD symptom severity. Data from adults receiving treatment for GAD in eight Improving Access to Psychological Therapies (IAPT) services (n=15,859) were separated into training, validation and holdout datasets. Thirteen machine learning algorithms were compared using 10-fold cross-validation, against two simple clinically relevant comparison models. The best-performing model was tested on the holdout dataset and model-specific explainability measures identified the most important predictors. A Bayesian Additive Regression Trees model out-performed all comparison models (MSE=16.54 [95 % CI=15.58; 17.51]; MAE=3.19; R²=0.33, including a single predictor linear regression model: MSE=20.70 [95 % CI=19.58; 21.82]; MAE=3.94; R²=0.14). The five most important predictors were: PHQ-9 anhedonia, GAD-7 annoyance/irritability, restlessness and fear items, then the referral-assessment waiting time. The best-performing model accurately predicted post-treatment GAD symptom severity using only pre-treatment data, outperforming comparison models that approximated clinical judgement and remaining within the GAD-7 error of measurement and minimal clinically important differences. This model could inform treatment decision-making and provide desired information to clinicians and patients receiving treatment for GAD.
Collapse
Affiliation(s)
- H Delamain
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom.
| | - J E J Buckman
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom; iCope - Camden and Islington Psychological Therapies Services, Camden & Islington NHS Foundation Trust, London, United Kingdom
| | - C O'Driscoll
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| | - J W Suh
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| | - J Stott
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| | - S Singh
- Waltham Forest Talking Therapies, North East London NHS Foundation Trust, London, United Kingdom
| | - S A Naqvi
- Barking and Dagenham and Havering IAPT Services, North East London NHS Foundation Trust, London, United Kingdom
| | - J Leibowitz
- iCope - Camden and Islington Psychological Therapies Services, Camden & Islington NHS Foundation Trust, London, United Kingdom
| | - S Pilling
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom; Camden and Islington NHS Foundation Trust, London, United Kingdom
| | - R Saunders
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| |
Collapse
|
2
|
Chekroud AM, Hawrilenko M, Loho H, Bondar J, Gueorguieva R, Hasan A, Kambeitz J, Corlett PR, Koutsouleris N, Krumholz HM, Krystal JH, Paulus M. Illusory generalizability of clinical prediction models. Science 2024; 383:164-167. [PMID: 38207039 DOI: 10.1126/science.adg8538] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 11/10/2023] [Indexed: 01/13/2024]
Abstract
It is widely hoped that statistical models can improve decision-making related to medical treatments. Because of the cost and scarcity of medical outcomes data, this hope is typically based on investigators observing a model's success in one or two datasets or clinical contexts. We scrutinized this optimism by examining how well a machine learning model performed across several independent clinical trials of antipsychotic medication for schizophrenia. Models predicted patient outcomes with high accuracy within the trial in which the model was developed but performed no better than chance when applied out-of-sample. Pooling data across trials to predict outcomes in the trial left out did not improve predictions. These results suggest that models predicting treatment outcomes in schizophrenia are highly context-dependent and may have limited generalizability.
Collapse
Affiliation(s)
- Adam M Chekroud
- Spring Health, New York City, NY 10010, USA
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA
| | | | - Hieronimus Loho
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA
| | | | | | - Alkomiet Hasan
- Department of Psychiatry, Psychotherapy and Psychosomatics, University Augsburg, 86159 Augsburg, Germany
| | - Joseph Kambeitz
- Department of Psychiatry and Psychotherapy, University of Cologne, Faculty of Medicine and University Hospital of Cologne, Cologne, Germany
| | - Philip R Corlett
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA
| | - Nikolaos Koutsouleris
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilians-University, Munich, Germany
| | - Harlan M Krumholz
- Center for Outcomes Research and Evaluation, Yale New Haven Hospital, New Haven, CT 06520, USA
| | - John H Krystal
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06520, USA
| | - Martin Paulus
- Laureate Institute for Brain Research, Tulsa, OK 74136, USA
| |
Collapse
|
3
|
Jimenez-Mesa C, Arco JE, Martinez-Murcia FJ, Suckling J, Ramirez J, Gorriz JM. Applications of machine learning and deep learning in SPECT and PET imaging: General overview, challenges and future prospects. Pharmacol Res 2023; 197:106984. [PMID: 37940064 DOI: 10.1016/j.phrs.2023.106984] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 10/04/2023] [Accepted: 11/04/2023] [Indexed: 11/10/2023]
Abstract
The integration of positron emission tomography (PET) and single-photon emission computed tomography (SPECT) imaging techniques with machine learning (ML) algorithms, including deep learning (DL) models, is a promising approach. This integration enhances the precision and efficiency of current diagnostic and treatment strategies while offering invaluable insights into disease mechanisms. In this comprehensive review, we delve into the transformative impact of ML and DL in this domain. Firstly, a brief analysis is provided of how these algorithms have evolved and which are the most widely applied in this domain. Their different potential applications in nuclear imaging are then discussed, such as optimization of image adquisition or reconstruction, biomarkers identification, multimodal fusion and the development of diagnostic, prognostic, and disease progression evaluation systems. This is because they are able to analyse complex patterns and relationships within imaging data, as well as extracting quantitative and objective measures. Furthermore, we discuss the challenges in implementation, such as data standardization and limited sample sizes, and explore the clinical opportunities and future horizons, including data augmentation and explainable AI. Together, these factors are propelling the continuous advancement of more robust, transparent, and reliable systems.
Collapse
Affiliation(s)
- Carmen Jimenez-Mesa
- Department of Signal Theory, Networking and Communications, University of Granada, 18010, Spain
| | - Juan E Arco
- Department of Signal Theory, Networking and Communications, University of Granada, 18010, Spain; Department of Communications Engineering, University of Malaga, 29010, Spain
| | | | - John Suckling
- Department of Psychiatry, University of Cambridge, Cambridge CB21TN, UK
| | - Javier Ramirez
- Department of Signal Theory, Networking and Communications, University of Granada, 18010, Spain
| | - Juan Manuel Gorriz
- Department of Signal Theory, Networking and Communications, University of Granada, 18010, Spain; Department of Psychiatry, University of Cambridge, Cambridge CB21TN, UK.
| |
Collapse
|
4
|
Horien C, Greene AS, Shen X, Fortes D, Brennan-Wydra E, Banarjee C, Foster R, Donthireddy V, Butler M, Powell K, Vernetti A, Mandino F, O’Connor D, Lake EMR, McPartland JC, Volkmar FR, Chun M, Chawarska K, Rosenberg MD, Scheinost D, Constable RT. A generalizable connectome-based marker of in-scan sustained attention in neurodiverse youth. Cereb Cortex 2023; 33:6320-6334. [PMID: 36573438 PMCID: PMC10183743 DOI: 10.1093/cercor/bhac506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 09/15/2022] [Accepted: 09/16/2022] [Indexed: 12/29/2022] Open
Abstract
Difficulty with attention is an important symptom in many conditions in psychiatry, including neurodiverse conditions such as autism. There is a need to better understand the neurobiological correlates of attention and leverage these findings in healthcare settings. Nevertheless, it remains unclear if it is possible to build dimensional predictive models of attentional state in a sample that includes participants with neurodiverse conditions. Here, we use 5 datasets to identify and validate functional connectome-based markers of attention. In dataset 1, we use connectome-based predictive modeling and observe successful prediction of performance on an in-scan sustained attention task in a sample of youth, including participants with a neurodiverse condition. The predictions are not driven by confounds, such as head motion. In dataset 2, we find that the attention network model defined in dataset 1 generalizes to predict in-scan attention in a separate sample of neurotypical participants performing the same attention task. In datasets 3-5, we use connectome-based identification and longitudinal scans to probe the stability of the attention network across months to years in individual participants. Our results help elucidate the brain correlates of attentional state in youth and support the further development of predictive dimensional models of other clinically relevant phenotypes.
Collapse
Affiliation(s)
- Corey Horien
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, CT, United States
- MD-PhD Program, Yale School of Medicine, New Haven, CT, United States
| | - Abigail S Greene
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, CT, United States
- MD-PhD Program, Yale School of Medicine, New Haven, CT, United States
| | - Xilin Shen
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - Diogo Fortes
- Yale Child Study Center, New Haven, CT, United States
| | | | | | - Rachel Foster
- Yale Child Study Center, New Haven, CT, United States
| | | | | | - Kelly Powell
- Yale Child Study Center, New Haven, CT, United States
| | | | - Francesca Mandino
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - David O’Connor
- Department of Biomedical Engineering, Yale University, New Haven, CT, United States
| | - Evelyn M R Lake
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
| | - James C McPartland
- Yale Child Study Center, New Haven, CT, United States
- Department of Psychology, Yale University, New Haven, CT, United States
| | - Fred R Volkmar
- Yale Child Study Center, New Haven, CT, United States
- Department of Psychology, Yale University, New Haven, CT, United States
| | - Marvin Chun
- Department of Psychology, Yale University, New Haven, CT, United States
| | - Katarzyna Chawarska
- Yale Child Study Center, New Haven, CT, United States
- Department of Statistics and Data Science, Yale University, New Haven, CT, United States
- Department of Pediatrics, Yale School of Medicine, New Haven, CT, United States
| | - Monica D Rosenberg
- Department of Psychology, University of Chicago, Chicago, IL, United States
- Neuroscience Institute, University of Chicago, Chicago, IL, United States
| | - Dustin Scheinost
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, CT, United States
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
- Yale Child Study Center, New Haven, CT, United States
- Department of Statistics and Data Science, Yale University, New Haven, CT, United States
| | - R Todd Constable
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, CT, United States
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States
- Department of Neurosurgery, Yale School of Medicine, New Haven, CT, United States
| |
Collapse
|
5
|
Horien C, Floris DL, Greene AS, Noble S, Rolison M, Tejavibulya L, O'Connor D, McPartland JC, Scheinost D, Chawarska K, Lake EMR, Constable RT. Functional Connectome-Based Predictive Modeling in Autism. Biol Psychiatry 2022; 92:626-642. [PMID: 35690495 PMCID: PMC10948028 DOI: 10.1016/j.biopsych.2022.04.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Revised: 04/14/2022] [Accepted: 04/17/2022] [Indexed: 01/08/2023]
Abstract
Autism is a heterogeneous neurodevelopmental condition, and functional magnetic resonance imaging-based studies have helped advance our understanding of its effects on brain network activity. We review how predictive modeling, using measures of functional connectivity and symptoms, has helped reveal key insights into this condition. We discuss how different prediction frameworks can further our understanding of the brain-based features that underlie complex autism symptomatology and consider how predictive models may be used in clinical settings. Throughout, we highlight aspects of study interpretation, such as data decay and sampling biases, that require consideration within the context of this condition. We close by suggesting exciting future directions for predictive modeling in autism.
Collapse
Affiliation(s)
- Corey Horien
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, Connecticut; MD-PhD Program, Yale School of Medicine, New Haven, Connecticut.
| | - Dorothea L Floris
- Methods of Plasticity Research, Department of Psychology, University of Zürich, Zurich, Switzerland; Donders Center for Brain, Cognition and Behavior, Radboud University Nijmegen, Nijmegen, The Netherlands
| | - Abigail S Greene
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, Connecticut; MD-PhD Program, Yale School of Medicine, New Haven, Connecticut
| | - Stephanie Noble
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut
| | - Max Rolison
- Yale Child Study Center, New Haven, Connecticut
| | - Link Tejavibulya
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, Connecticut
| | - David O'Connor
- Department of Biomedical Engineering, Yale University, New Haven, Connecticut
| | - James C McPartland
- Department of Psychology, Yale University, New Haven, Connecticut; Yale Child Study Center, New Haven, Connecticut
| | - Dustin Scheinost
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, Connecticut; Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut; Department of Biomedical Engineering, Yale University, New Haven, Connecticut; Department of Statistics and Data Science, Yale University, New Haven, Connecticut; Yale Child Study Center, New Haven, Connecticut
| | - Katarzyna Chawarska
- Department of Pediatrics, Yale School of Medicine, New Haven, Connecticut; Department of Statistics and Data Science, Yale University, New Haven, Connecticut; Yale Child Study Center, New Haven, Connecticut
| | - Evelyn M R Lake
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut
| | - R Todd Constable
- Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, Connecticut; Department of Neurosurgery, Yale School of Medicine, New Haven, Connecticut; Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, Connecticut; Department of Biomedical Engineering, Yale University, New Haven, Connecticut.
| |
Collapse
|
6
|
The PSYchiatric clinical outcome prediction (PSYCOP) cohort: leveraging the potential of electronic health records in the treatment of mental disorders. Acta Neuropsychiatr 2021; 33:323-330. [PMID: 34369330 DOI: 10.1017/neu.2021.22] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
BACKGROUND The quality of life and lifespan are greatly reduced among individuals with mental illness. To improve prognosis, the nascent field of precision psychiatry aims to provide personalised predictions for the course of illness and response to treatment. Unfortunately, the results of precision psychiatry studies are rarely externally validated, almost never implemented in clinical practice, and tend to focus on a few selected outcomes. To overcome these challenges, we have established the PSYchiatric Clinical Outcome Prediction (PSYCOP) cohort, which will form the basis for extensive studies in the upcoming years. METHODS PSYCOP is a retrospective cohort study that includes all patients with at least one contact with the psychiatric services of the Central Denmark Region in the period from January 1, 2011, to October 28, 2020 (n = 119 291). All data from the electronic health records (EHR) are included, spanning diagnoses, information on treatments, clinical notes, discharge summaries, laboratory tests, etc. Based on these data, machine learning methods will be used to make prediction models for a range of clinical outcomes, such as diagnostic shifts, treatment response, medical comorbidity, and premature mortality, with an explicit focus on clinical feasibility and implementation. DISCUSSIONS We expect that studies based on the PSYCOP cohort will advance the field of precision psychiatry through the use of state-of-the-art machine learning methods on a large and representative data set. Implementation of prediction models in clinical psychiatry will likely improve treatment and, hopefully, increase the quality of life and lifespan of those with mental illness.
Collapse
|
7
|
Chekroud AM, Bondar J, Delgadillo J, Doherty G, Wasil A, Fokkema M, Cohen Z, Belgrave D, DeRubeis R, Iniesta R, Dwyer D, Choi K. The promise of machine learning in predicting treatment outcomes in psychiatry. World Psychiatry 2021; 20:154-170. [PMID: 34002503 PMCID: PMC8129866 DOI: 10.1002/wps.20882] [Citation(s) in RCA: 150] [Impact Index Per Article: 50.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.
Collapse
Affiliation(s)
- Adam M Chekroud
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
- Spring Health, New York City, NY, USA
| | | | - Jaime Delgadillo
- Clinical Psychology Unit, Department of Psychology, University of Sheffield, Sheffield, UK
| | - Gavin Doherty
- School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland
| | - Akash Wasil
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| | - Marjolein Fokkema
- Department of Methods and Statistics, Institute of Psychology, Leiden University, Leiden, The Netherlands
| | - Zachary Cohen
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
| | | | - Robert DeRubeis
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| | - Raquel Iniesta
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neurosciences, King's College London, London, UK
| | - Dominic Dwyer
- Department of Psychiatry and Psychotherapy, Section for Neurodiagnostic Applications, Ludwig-Maximilian University, Munich, Germany
| | - Karmel Choi
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
8
|
Vatansever S, Schlessinger A, Wacker D, Kaniskan HÜ, Jin J, Zhou M, Zhang B. Artificial intelligence and machine learning-aided drug discovery in central nervous system diseases: State-of-the-arts and future directions. Med Res Rev 2021; 41:1427-1473. [PMID: 33295676 PMCID: PMC8043990 DOI: 10.1002/med.21764] [Citation(s) in RCA: 95] [Impact Index Per Article: 31.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Revised: 10/30/2020] [Accepted: 11/20/2020] [Indexed: 01/11/2023]
Abstract
Neurological disorders significantly outnumber diseases in other therapeutic areas. However, developing drugs for central nervous system (CNS) disorders remains the most challenging area in drug discovery, accompanied with the long timelines and high attrition rates. With the rapid growth of biomedical data enabled by advanced experimental technologies, artificial intelligence (AI) and machine learning (ML) have emerged as an indispensable tool to draw meaningful insights and improve decision making in drug discovery. Thanks to the advancements in AI and ML algorithms, now the AI/ML-driven solutions have an unprecedented potential to accelerate the process of CNS drug discovery with better success rate. In this review, we comprehensively summarize AI/ML-powered pharmaceutical discovery efforts and their implementations in the CNS area. After introducing the AI/ML models as well as the conceptualization and data preparation, we outline the applications of AI/ML technologies to several key procedures in drug discovery, including target identification, compound screening, hit/lead generation and optimization, drug response and synergy prediction, de novo drug design, and drug repurposing. We review the current state-of-the-art of AI/ML-guided CNS drug discovery, focusing on blood-brain barrier permeability prediction and implementation into therapeutic discovery for neurological diseases. Finally, we discuss the major challenges and limitations of current approaches and possible future directions that may provide resolutions to these difficulties.
Collapse
Affiliation(s)
- Sezen Vatansever
- Department of Genetics and Genomic SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Mount Sinai Center for Transformative Disease ModelingIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Icahn Institute for Data Science and Genomic TechnologyIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| | - Avner Schlessinger
- Department of Pharmacological SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Mount Sinai Center for Therapeutics DiscoveryIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| | - Daniel Wacker
- Department of Pharmacological SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Mount Sinai Center for Therapeutics DiscoveryIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Department of NeuroscienceIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| | - H. Ümit Kaniskan
- Department of Pharmacological SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Mount Sinai Center for Therapeutics DiscoveryIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Department of Oncological Sciences, Tisch Cancer InstituteIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| | - Jian Jin
- Department of Pharmacological SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Mount Sinai Center for Therapeutics DiscoveryIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Department of Oncological Sciences, Tisch Cancer InstituteIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| | - Ming‐Ming Zhou
- Department of Pharmacological SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Department of Oncological Sciences, Tisch Cancer InstituteIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| | - Bin Zhang
- Department of Genetics and Genomic SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Mount Sinai Center for Transformative Disease ModelingIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Icahn Institute for Data Science and Genomic TechnologyIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
- Department of Pharmacological SciencesIcahn School of Medicine at Mount SinaiNew YorkNew YorkUSA
| |
Collapse
|
9
|
Jacobs M, Pradier MF, McCoy TH, Perlis RH, Doshi-Velez F, Gajos KZ. How machine-learning recommendations influence clinician treatment selections: the example of the antidepressant selection. Transl Psychiatry 2021; 11:108. [PMID: 33542191 PMCID: PMC7862671 DOI: 10.1038/s41398-021-01224-x] [Citation(s) in RCA: 42] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/02/2020] [Revised: 01/11/2021] [Accepted: 01/18/2021] [Indexed: 02/06/2023] Open
Abstract
Decision support systems embodying machine learning models offer the promise of an improved standard of care for major depressive disorder, but little is known about how clinicians' treatment decisions will be influenced by machine learning recommendations and explanations. We used a within-subject factorial experiment to present 220 clinicians with patient vignettes, each with or without a machine-learning (ML) recommendation and one of the multiple forms of explanation. We found that interacting with ML recommendations did not significantly improve clinicians' treatment selection accuracy, assessed as concordance with expert psychopharmacologist consensus, compared to baseline scenarios in which clinicians made treatment decisions independently. Interacting with incorrect recommendations paired with explanations that included limited but easily interpretable information did lead to a significant reduction in treatment selection accuracy compared to baseline questions. These results suggest that incorrect ML recommendations may adversely impact clinician treatment selections and that explanations are insufficient for addressing overreliance on imperfect ML algorithms. More generally, our findings challenge the common assumption that clinicians interacting with ML tools will perform better than either clinicians or ML algorithms individually.
Collapse
Affiliation(s)
- Maia Jacobs
- Department of Computer Science, Harvard University, 29 Oxford Street, Cambridge, MA, 02138, USA
| | - Melanie F Pradier
- Department of Computer Science, Harvard University, 29 Oxford Street, Cambridge, MA, 02138, USA
| | - Thomas H McCoy
- Center for Quantitative Health, Massachusetts General Hospital, 185 Cambridge Street, Boston, MA, 02114, USA
- Harvard Medical School, 25 Shattuck Street, Boston, MA, 02115, USA
| | - Roy H Perlis
- Center for Quantitative Health, Massachusetts General Hospital, 185 Cambridge Street, Boston, MA, 02114, USA
- Harvard Medical School, 25 Shattuck Street, Boston, MA, 02115, USA
| | - Finale Doshi-Velez
- Department of Computer Science, Harvard University, 29 Oxford Street, Cambridge, MA, 02138, USA
| | - Krzysztof Z Gajos
- Department of Computer Science, Harvard University, 29 Oxford Street, Cambridge, MA, 02138, USA.
| |
Collapse
|
10
|
Oliver D, Spada G, Colling C, Broadbent M, Baldwin H, Patel R, Stewart R, Stahl D, Dobson R, McGuire P, Fusar-Poli P. Real-world implementation of precision psychiatry: Transdiagnostic risk calculator for the automatic detection of individuals at-risk of psychosis. Schizophr Res 2021; 227:52-60. [PMID: 32571619 PMCID: PMC7875179 DOI: 10.1016/j.schres.2020.05.007] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/01/2020] [Revised: 05/01/2020] [Accepted: 05/04/2020] [Indexed: 12/31/2022]
Abstract
BACKGROUND Risk estimation models integrated into Electronic Health Records (EHRs) can deliver innovative approaches in psychiatry, but clinicians' endorsement and their real-world usability are unknown. This study aimed to investigate the real-world feasibility of implementing an individualised, transdiagnostic risk calculator to automatically screen EHRs and detect individuals at-risk for psychosis. METHODS Feasibility implementation study encompassing an in-vitro phase (March 2018 to May 2018) and in-vivo phase (May 2018 to April 2019). The in-vitro phase addressed implementation barriers and embedded the risk calculator (predictors: age, gender, ethnicity, index cluster diagnosis, age*gender) into the local EHR. The in-vivo phase investigated the real-world feasibility of screening individuals accessing secondary mental healthcare at the South London and Maudsley NHS Trust. The primary outcome was adherence of clinicians to automatic EHR screening, defined by the proportion of clinicians who responded to alerts from the risk calculator, over those contacted. RESULTS In-vitro phase: implementation barriers were identified/overcome with clinician and service user engagement, and the calculator was successfully integrated into the local EHR through the CogStack platform. In-vivo phase: 3722 individuals were automatically screened and 115 were detected. Clinician adherence was 74% without outreach and 85% with outreach. One-third of clinicians responded to the first email (37.1%) or phone calls (33.7%). Among those detected, cumulative risk of developing psychosis was 12% at six-month follow-up. CONCLUSION This is the first implementation study suggesting that combining precision psychiatry and EHR methods to improve detection of individuals with emerging psychosis is feasible. Future psychiatric implementation research is urgently needed.
Collapse
Affiliation(s)
- Dominic Oliver
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom
| | - Giulia Spada
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom
| | - Craig Colling
- National Institute for Health Research, Maudesley Biomedical Research Centre, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Matthew Broadbent
- National Institute for Health Research, Maudesley Biomedical Research Centre, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Helen Baldwin
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom; National Institute for Health Research, Maudesley Biomedical Research Centre, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Rashmi Patel
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom
| | - Robert Stewart
- Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom; South London and Maudsley Foundation Trust, London, United Kingdom
| | - Daniel Stahl
- Department of Biostatistics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, United Kingdom
| | - Richard Dobson
- National Institute for Health Research, Maudesley Biomedical Research Centre, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom; Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom; Institute of Health Informatics Research, University College London, London, United Kingdom; Health Data Research UK London, University College London, London, United Kingdom
| | - Philip McGuire
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom; OASIS Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom; National Institute for Health Research, Maudesley Biomedical Research Centre, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom; OASIS Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom; Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy.
| |
Collapse
|
11
|
Salazar de Pablo G, Studerus E, Vaquerizo-Serrano J, Irving J, Catalan A, Oliver D, Baldwin H, Danese A, Fazel S, Steyerberg EW, Stahl D, Fusar-Poli P. Implementing Precision Psychiatry: A Systematic Review of Individualized Prediction Models for Clinical Practice. Schizophr Bull 2020; 47:284-297. [PMID: 32914178 PMCID: PMC7965077 DOI: 10.1093/schbul/sbaa120] [Citation(s) in RCA: 86] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
BACKGROUND The impact of precision psychiatry for clinical practice has not been systematically appraised. This study aims to provide a comprehensive review of validated prediction models to estimate the individual risk of being affected with a condition (diagnostic), developing outcomes (prognostic), or responding to treatments (predictive) in mental disorders. METHODS PRISMA/RIGHT/CHARMS-compliant systematic review of the Web of Science, Cochrane Central Register of Reviews, and Ovid/PsycINFO databases from inception until July 21, 2019 (PROSPERO CRD42019155713) to identify diagnostic/prognostic/predictive prediction studies that reported individualized estimates in psychiatry and that were internally or externally validated or implemented. Random effect meta-regression analyses addressed the impact of several factors on the accuracy of prediction models. FINDINGS Literature search identified 584 prediction modeling studies, of which 89 were included. 10.4% of the total studies included prediction models internally validated (n = 61), 4.6% models externally validated (n = 27), and 0.2% (n = 1) models considered for implementation. Across validated prediction modeling studies (n = 88), 18.2% were diagnostic, 68.2% prognostic, and 13.6% predictive. The most frequently investigated condition was psychosis (36.4%), and the most frequently employed predictors clinical (69.5%). Unimodal compared to multimodal models (β = .29, P = .03) and diagnostic compared to prognostic (β = .84, p < .0001) and predictive (β = .87, P = .002) models were associated with increased accuracy. INTERPRETATION To date, several validated prediction models are available to support the diagnosis and prognosis of psychiatric conditions, in particular, psychosis, or to predict treatment response. Advancements of knowledge are limited by the lack of implementation research in real-world clinical practice. A new generation of implementation research is required to address this translational gap.
Collapse
Affiliation(s)
- Gonzalo Salazar de Pablo
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,Institute of Psychiatry and Mental Health, Department of Child and Adolescent Psychiatry, Hospital General Universitario Gregorio Marañón School of Medicine, Universidad Complutense, Instituto de Investigación Sanitaria Gregorio Marañón, CIBERSAM, Madrid, Spain,Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
| | - Erich Studerus
- Division of Personality and Developmental Psychology, Department of Psychology, University of Basel, Basel, Switzerland
| | - Julio Vaquerizo-Serrano
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,Institute of Psychiatry and Mental Health, Department of Child and Adolescent Psychiatry, Hospital General Universitario Gregorio Marañón School of Medicine, Universidad Complutense, Instituto de Investigación Sanitaria Gregorio Marañón, CIBERSAM, Madrid, Spain,Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
| | - Jessica Irving
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK
| | - Ana Catalan
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,Department of Psychiatry, Basurto University Hospital, Bilbao, Spain,Mental Health Group, BioCruces Health Research Institute, Bizkaia, Spain,Neuroscience Department, University of the Basque Country UPV/EHU, Leioa, Spain
| | - Dominic Oliver
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK
| | - Helen Baldwin
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK
| | - Andrea Danese
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK,Social, Genetic and Developmental Psychiatry Centre, King’s College London, London, UK,National and Specialist CAMHS Clinic for Trauma, Anxiety, and Depression, South London and Maudsley NHS Foundation Trust, London, UK
| | - Seena Fazel
- Department of Psychiatry, University of Oxford, Oxford, UK
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands,Department of Public Health, Erasmus MC, Rotterdam, the Netherlands
| | - Daniel Stahl
- Biostatistics Department, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
| | - Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,OASIS Service, South London and Maudsley NHS Foundation Trust, London, UK,Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy,National Institute for Health Research, Maudsley Biomedical Research Centre, South London and Maudsley NHS Foundation Trust, London, UK,To whom correspondence should be addressed; tel: +44-0-20-7848-0900, fax:+44-0-20-7848-0976, e-mail:
| |
Collapse
|
12
|
Ermers NJ, Hagoort K, Scheepers FE. The Predictive Validity of Machine Learning Models in the Classification and Treatment of Major Depressive Disorder: State of the Art and Future Directions. Front Psychiatry 2020; 11:472. [PMID: 32523557 PMCID: PMC7261928 DOI: 10.3389/fpsyt.2020.00472] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/18/2019] [Accepted: 05/07/2020] [Indexed: 11/13/2022] Open
Abstract
Major depressive disorder imposes a substantial disease burden worldwide, ranking as the third leading contributor to global disability. In spite of its ubiquity, classifying and treating depression has proven troublesome. One argument put forward to explain this predicament is the heterogeneity of patients diagnosed with the disorder. Recently, many areas of daily life have witnessed the surge of machine learning techniques, computational approaches to elucidate complex patterns in large datasets, which can be employed to make predictions and detect relevant clusters. Due to the multidimensionality at play in the pathogenesis of depression, it is suggested that machine learning could contribute to improving classification and treatment. In this paper, we investigated literature focusing on the use of machine learning models on datasets with clinical variables of patients diagnosed with depression to predict treatment outcomes or find more homogeneous subgroups. Identified studies based on best practices in the field are evaluated. We found 16 studies predicting outcomes (such as remission) and identifying clusters in patients with depression. The identified studies are mostly still in proof-of-concept phase, with small datasets, lack of external validation, and providing single performance metrics. Larger datasets, and models with similar variables present across these datasets, are needed to develop accurate and generalizable models. We hypothesize that harnessing natural language processing to obtain data 'hidden' in clinical texts might prove useful in improving prediction models. Besides, researchers will need to focus on the conditions to feasibly implement these models to support psychiatrists and patients in their decision-making in practice. Only then we can enter the realm of precision psychiatry.
Collapse
Affiliation(s)
- Nick J. Ermers
- Department of Psychiatry, University Medical Center Utrecht, Utrecht, Netherlands
| | | | | |
Collapse
|
13
|
Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res 2020; 121:159-172. [PMID: 31830722 DOI: 10.1016/j.jpsychires.2019.12.001] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/29/2019] [Revised: 11/22/2019] [Accepted: 12/05/2019] [Indexed: 12/27/2022]
Abstract
Establishing the diagnosis of trauma-related disorders such as Acute Stress Disorder (ASD) and Posttraumatic Stress Disorder (PTSD) have always been a challenge in clinical practice and in academic research, due to clinical and biological heterogeneity. Machine learning (ML) techniques can be applied to improve classification of disorders, to predict outcomes or to determine person-specific treatment selection. We aim to review the existing literature on the use of machine learning techniques in the assessment of subjects with ASD or PTSD. We systematically searched PubMed, Embase and Web of Science for articles published in any language up to May 2019. We found 806 abstracts and included 49 studies in our review. Most of the included studies used multiple levels of biological data to predict risk factors or to identify early symptoms related to PTSD. Other studies used ML classification techniques to distinguish individuals with ASD or PTSD from other psychiatric disorder or from trauma-exposed and healthy controls. We also found studies that attempted to define outcome profiles using clustering techniques and studies that assessed the relationship among symptoms using network analysis. Finally, we proposed a quality assessment in this review, evaluating methodological and technical features on machine learning studies. We concluded that etiologic and clinical heterogeneity of ASD/PTSD patients is suitable to machine learning techniques and a major challenge for the future is to use it in clinical practice for the benefit of patients in an individual level.
Collapse
Affiliation(s)
- Luis Francisco Ramos-Lima
- Post-graduate Program in Psychiatry and Behavioral Sciences, Federal University at Rio Grande do Sul, Porto Alegre, Brazil; Psychological Trauma Research and Treatment Program (NET-Trauma), Clinical Hospital of Porto Alegre, Porto Alegre, Brazil.
| | - Vitoria Waikamp
- Post-graduate Program in Psychiatry and Behavioral Sciences, Federal University at Rio Grande do Sul, Porto Alegre, Brazil; Psychological Trauma Research and Treatment Program (NET-Trauma), Clinical Hospital of Porto Alegre, Porto Alegre, Brazil
| | - Thyago Antonelli-Salgado
- Bipolar Disorder Program, Laboratory of Molecular Psychiatry, Clinical Hospital of Porto Alegre, Porto Alegre, Brazil
| | - Ives Cavalcante Passos
- Post-graduate Program in Psychiatry and Behavioral Sciences, Federal University at Rio Grande do Sul, Porto Alegre, Brazil; Bipolar Disorder Program, Laboratory of Molecular Psychiatry, Clinical Hospital of Porto Alegre, Porto Alegre, Brazil
| | - Lucia Helena Machado Freitas
- Post-graduate Program in Psychiatry and Behavioral Sciences, Federal University at Rio Grande do Sul, Porto Alegre, Brazil; Psychological Trauma Research and Treatment Program (NET-Trauma), Clinical Hospital of Porto Alegre, Porto Alegre, Brazil
| |
Collapse
|
14
|
Studerus E, Beck K, Fusar-Poli P, Riecher-Rössler A. Development and Validation of a Dynamic Risk Prediction Model to Forecast Psychosis Onset in Patients at Clinical High Risk. Schizophr Bull 2020; 46:252-260. [PMID: 31355885 PMCID: PMC7442327 DOI: 10.1093/schbul/sbz059] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
The prediction of outcomes in patients at Clinical High Risk for Psychosis (CHR-P) almost exclusively relies on static data obtained at a single snapshot in time (ie, baseline data). Although the CHR-P symptoms are intrinsically evolving over time, available prediction models cannot be dynamically updated to reflect these changes. Hence, the aim of this study was to develop and internally validate a dynamic risk prediction model (joint model) and to implement this model in a user-friendly online risk calculator. Furthermore, we aimed to explore the prognostic performance of extended dynamic risk prediction models and to compare static with dynamic prediction. One hundred ninety-six CHR-P patients were recruited as part of the "Basel Früherkennung von Psychosen" (FePsy) study. Psychopathology and transition to psychosis was assessed at regular intervals for up to 5 years using the Brief Psychiatric Rating Scale-Expanded (BPRS-E). Various specifications of joint models were compared with regard to their cross-validated prognostic performance. We developed and internally validated a joint model that predicts psychosis onset from BPRS-E disorganization and years of education at baseline and BPRS-E positive symptoms during the follow-up with good prognostic performance. The model was implemented as online risk calculator (http://www.fepsy.ch/DPRP/). The use of extended joint models slightly increased the prognostic accuracy compared to basic joint models, and dynamic models showed a higher prognostic accuracy than static models. Our results confirm that extended joint modeling could improve the prediction of psychosis in CHR-P patients. We implemented the first online risk calculator that can dynamically update psychosis risk prediction.
Collapse
Affiliation(s)
- Erich Studerus
- Center for Gender Research and Early Detection, University of Basel Psychiatric Hospital, Basel, Switzerland,To whom correspondence should be addressed; tel: +41-61-325-59-95, e-mail:
| | - Katharina Beck
- Center for Gender Research and Early Detection, University of Basel Psychiatric Hospital, Basel, Switzerland,Department of Psychology, Division of Clinical Psychology and Epidemiology, University of Basel, Basel, Switzerland
| | - Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK,OASIS Service, South London and Maudsley National Health Service Foundation Trust, London, UK,Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy,National Institute of Health Research—Mental Health—Translational Research Collaboration—Early Psychosis Workstream, London, UK
| | - Anita Riecher-Rössler
- Center for Gender Research and Early Detection, University of Basel Psychiatric Hospital, Basel, Switzerland
| |
Collapse
|
15
|
Schmidt A, Borgwardt S. Implementing MR Imaging into Clinical Routine Screening in Patients with Psychosis? Neuroimaging Clin N Am 2020; 30:65-72. [DOI: 10.1016/j.nic.2019.09.004] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
16
|
Saunders R, Buckman JEJ, Pilling S. Latent variable mixture modelling and individual treatment prediction. Behav Res Ther 2020; 124:103505. [PMID: 31841709 PMCID: PMC7417810 DOI: 10.1016/j.brat.2019.103505] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2019] [Revised: 10/08/2019] [Accepted: 10/25/2019] [Indexed: 01/10/2023]
Abstract
Understanding which groups of patients are more or less likely to benefit from specific treatments has important implications for healthcare. Many personalised medicine approaches in mental health employ variable-centred approaches to predicting treatment response, yet person-centred approaches that identify clinical profiles of patients can provide information on the likelihood of a range of important outcomes. In this paper, we discuss the use of latent variable mixture modelling and demonstrate its use in the application of a patient profiling algorithm using routinely collected patient data to predict outcomes from psychological treatments. This validation study analysed data from two services, which included n = 44,905 patients entering treatment. There were different patterns of reliable recovery, improvement and clinical deterioration from therapy, across the eight profiles which were consistent over time. Outcomes varied between different types of therapy within the profiles: there were significantly higher odds of reliable recovery with High Intensity therapies in two profiles (32.5% of patients) and of reliable improvement in three profiles (32.2% of patients) compared with Low Intensity treatments. In three profiles (37.4% of patients) reliable recovery was significantly more likely if patients had CBT vs Counselling. The developments and potential application of latent variable mixture approaches are further discussed.
Collapse
Affiliation(s)
- Rob Saunders
- Research Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London, WC1E 7HB, UK
| | - Joshua E J Buckman
- Research Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London, WC1E 7HB, UK
| | - Stephen Pilling
- Research Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London, WC1E 7HB, UK.
| |
Collapse
|
17
|
Passos IC, Ballester PL, Barros RC, Librenza-Garcia D, Mwangi B, Birmaher B, Brietzke E, Hajek T, Lopez Jaramillo C, Mansur RB, Alda M, Haarman BCM, Isometsa E, Lam RW, McIntyre RS, Minuzzi L, Kessing LV, Yatham LN, Duffy A, Kapczinski F. Machine learning and big data analytics in bipolar disorder: A position paper from the International Society for Bipolar Disorders Big Data Task Force. Bipolar Disord 2019; 21:582-594. [PMID: 31465619 DOI: 10.1111/bdi.12828] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
OBJECTIVES The International Society for Bipolar Disorders Big Data Task Force assembled leading researchers in the field of bipolar disorder (BD), machine learning, and big data with extensive experience to evaluate the rationale of machine learning and big data analytics strategies for BD. METHOD A task force was convened to examine and integrate findings from the scientific literature related to machine learning and big data based studies to clarify terminology and to describe challenges and potential applications in the field of BD. We also systematically searched PubMed, Embase, and Web of Science for articles published up to January 2019 that used machine learning in BD. RESULTS The results suggested that big data analytics has the potential to provide risk calculators to aid in treatment decisions and predict clinical prognosis, including suicidality, for individual patients. This approach can advance diagnosis by enabling discovery of more relevant data-driven phenotypes, as well as by predicting transition to the disorder in high-risk unaffected subjects. We also discuss the most frequent challenges that big data analytics applications can face, such as heterogeneity, lack of external validation and replication of some studies, cost and non-stationary distribution of the data, and lack of appropriate funding. CONCLUSION Machine learning-based studies, including atheoretical data-driven big data approaches, provide an opportunity to more accurately detect those who are at risk, parse-relevant phenotypes as well as inform treatment selection and prognosis. However, several methodological challenges need to be addressed in order to translate research findings to clinical settings.
Collapse
Affiliation(s)
- Ives C Passos
- Laboratory of Molecular Psychiatry and Bipolar Disorder Program, Programa de Pós-Graduação em Psiquiatria e Ciências do Comportamento, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Pedro L Ballester
- School of Technology, Pontifícia Universidade Católica do Rio Grande do Sul, Rio Grande do Sul, Brazil
| | - Rodrigo C Barros
- School of Technology, Pontifícia Universidade Católica do Rio Grande do Sul, Rio Grande do Sul, Brazil
| | - Diego Librenza-Garcia
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Benson Mwangi
- Department of Psychiatry and Behavioral Sciences, UT Center of Excellence on Mood Disorders, McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Boris Birmaher
- Department of Psychiatry, Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | - Elisa Brietzke
- Department of Psychiatry, Queen's University School of Medicine, Kingston, ON, Canada
| | - Tomas Hajek
- Department of Psychiatry, Dalhousie University, Halifax, NS, Canada.,National Institute of Mental Health, Klecany, Czech Republic
| | - Carlos Lopez Jaramillo
- Research Group in Psychiatry, Department of Psychiatry, Faculty of Medicine, University of Antioquia, Medellín, Colombia.,Mood Disorders Program, Hospital Universitario San Vicente Fundación, Medellín, Colombia
| | - Rodrigo B Mansur
- Mood Disorders Psychopharmacology Unit (MDPU), University Health Network, University of Toronto, Toronto, ON, Canada
| | - Martin Alda
- Department of Psychiatry, Dalhousie University, Halifax, NS, Canada
| | - Bartholomeus C M Haarman
- Department of Psychiatry, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Erkki Isometsa
- Department of Psychiatry, University of Helsinki and Helsinki University Central Hospital, Helsinki, Finland
| | - Raymond W Lam
- Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada
| | - Roger S McIntyre
- Department of Psychiatry, University of Toronto, Toronto, ON, Canada
| | - Luciano Minuzzi
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Lars V Kessing
- Copenhagen Affective Disorder Research Center (CADIC), Psychiatric Center Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark
| | - Lakshmi N Yatham
- Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada
| | - Anne Duffy
- Department of Psychiatry, Queen's University School of Medicine, Kingston, ON, Canada
| | - Flavio Kapczinski
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
18
|
Toward clinically useful models for individualised prognostication in psychosis. Lancet Digit Health 2019; 1:e244-e245. [DOI: 10.1016/s2589-7500(19)30122-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2019] [Accepted: 09/09/2019] [Indexed: 12/23/2022]
|
19
|
Rutledge RB, Chekroud AM, Huys QJ. Machine learning and big data in psychiatry: toward clinical applications. Curr Opin Neurobiol 2019; 55:152-159. [PMID: 30999271 DOI: 10.1016/j.conb.2019.02.006] [Citation(s) in RCA: 79] [Impact Index Per Article: 15.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2018] [Revised: 01/29/2019] [Accepted: 02/07/2019] [Indexed: 12/21/2022]
Abstract
Psychiatry is a medical field concerned with the treatment of mental illness. Psychiatric disorders broadly relate to higher functions of the brain, and as such are richly intertwined with social, cultural, and experiential factors. This makes them exquisitely complex phenomena that depend on and interact with a large number of variables. Computational psychiatry provides two ways of approaching this complexity. Theory-driven computational approaches employ mechanistic models to make explicit hypotheses at multiple levels of analysis. Data-driven machine-learning approaches can make predictions from high-dimensional data and are generally agnostic as to the underlying mechanisms. Here, we review recent advances in the use of big data and machine-learning approaches toward the aim of alleviating the suffering that arises from psychiatric disorders.
Collapse
Affiliation(s)
- Robb B Rutledge
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, England, United Kingdom; Wellcome Centre for Human Neuroimaging, University College London, London, England, United Kingdom
| | - Adam M Chekroud
- Department of Psychiatry, Yale University, New Haven, CT, United States; Spring Health, New York, NY, United States
| | - Quentin Jm Huys
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, England, United Kingdom; Division of Psychiatry, University College London, London, England, United Kingdom; Camden and Islington NHS Foundation Trust, London, England, United Kingdom.
| |
Collapse
|
20
|
Abstract
IMPORTANCE Prognosis is a venerable component of medical knowledge introduced by Hippocrates (460-377 BC). This educational review presents a contemporary evidence-based approach for how to incorporate clinical risk prediction models in modern psychiatry. The article is organized around key methodological themes most relevant for the science of prognosis in psychiatry. Within each theme, the article highlights key challenges and makes pragmatic recommendations to improve scientific understanding of prognosis in psychiatry. OBSERVATIONS The initial step to building clinical risk prediction models that can affect psychiatric care involves designing the model: preparation of the protocol and definition of the outcomes and of the statistical methods (theme 1). Further initial steps involve carefully selecting the predictors, preparing the data, and developing the model in these data. A subsequent step is the validation of the model to accurately test its generalizability (theme 2). The next consideration is that the accuracy of the clinical prediction model is affected by the incidence of the psychiatric condition under investigation (theme 3). Eventually, clinical prediction models need to be implemented in real-world clinical routine, and this is usually the most challenging step (theme 4). Advanced methods such as machine learning approaches can overcome some problems that undermine the previous steps (theme 5). The relevance of each of these themes to current clinical risk prediction modeling in psychiatry is discussed and recommendations are given. CONCLUSIONS AND RELEVANCE Together, these perspectives intend to contribute to an integrative, evidence-based science of prognosis in psychiatry. By focusing on the outcome of the individuals, rather than on the disease, clinical risk prediction modeling can become the cornerstone for a scientific and personalized psychiatry.
Collapse
Affiliation(s)
- Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom.,OASIS Service, South London and Maudsley National Health Service Foundation Trust, London, United Kingdom.,Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
| | - Ziad Hijazi
- Department of Medical Sciences, Cardiology, and Uppsala Clinical Research Center, Uppsala University, Uppsala University Hospital, Uppsala, Sweden
| | - Daniel Stahl
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Medical Statistics and Medical Decision Making, Leiden University Medical Center, Leiden, the Netherlands
| |
Collapse
|
21
|
Koutsouleris N, Kambeitz-Ilankovic L, Ruhrmann S, Rosen M, Ruef A, Dwyer DB, Paolini M, Chisholm K, Kambeitz J, Haidl T, Schmidt A, Gillam J, Schultze-Lutter F, Falkai P, Reiser M, Riecher-Rössler A, Upthegrove R, Hietala J, Salokangas RKR, Pantelis C, Meisenzahl E, Wood SJ, Beque D, Brambilla P, Borgwardt S. Prediction Models of Functional Outcomes for Individuals in the Clinical High-Risk State for Psychosis or With Recent-Onset Depression: A Multimodal, Multisite Machine Learning Analysis. JAMA Psychiatry 2018; 75:1156-1172. [PMID: 30267047 PMCID: PMC6248111 DOI: 10.1001/jamapsychiatry.2018.2165] [Citation(s) in RCA: 210] [Impact Index Per Article: 35.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
IMPORTANCE Social and occupational impairments contribute to the burden of psychosis and depression. There is a need for risk stratification tools to inform personalized functional-disability preventive strategies for individuals in at-risk and early phases of these illnesses. OBJECTIVE To determine whether predictors associated with social and role functioning can be identified in patients in clinical high-risk (CHR) states for psychosis or with recent-onset depression (ROD) using clinical, imaging-based, and combined machine learning; assess the geographic, transdiagnostic, and prognostic generalizability of machine learning and compare it with human prognostication; and explore sequential prognosis encompassing clinical and combined machine learning. DESIGN, SETTING, AND PARTICIPANTS This multisite naturalistic study followed up patients in CHR states, with ROD, and with recent-onset psychosis, and healthy control participants for 18 months in 7 academic early-recognition services in 5 European countries. Participants were recruited between February 2014 and May 2016, and data were analyzed from April 2017 to January 2018. AIN OUTCOMES AND MEASURES Performance and generalizability of prognostic models. RESULTS A total of 116 individuals in CHR states (mean [SD] age, 24.0 [5.1] years; 58 [50.0%] female) and 120 patients with ROD (mean [SD] age, 26.1 [6.1] years; 65 [54.2%] female) were followed up for a mean (SD) of 329 (142) days. Machine learning predicted the 1-year social-functioning outcomes with a balanced accuracy of 76.9% of patients in CHR states and 66.2% of patients with ROD using clinical baseline data. Balanced accuracy in models using structural neuroimaging was 76.2% in patients in CHR states and 65.0% in patients with ROD, and in combined models, it was 82.7% for CHR states and 70.3% for ROD. Lower functioning before study entry was a transdiagnostic predictor. Medial prefrontal and temporo-parieto-occipital gray matter volume (GMV) reductions and cerebellar and dorsolateral prefrontal GMV increments had predictive value in the CHR group; reduced mediotemporal and increased prefrontal-perisylvian GMV had predictive value in patients with ROD. Poor prognoses were associated with increased risk of psychotic, depressive, and anxiety disorders at follow-up in patients in the CHR state but not ones with ROD. Machine learning outperformed expert prognostication. Adding neuroimaging machine learning to clinical machine learning provided a 1.9-fold increase of prognostic certainty in uncertain cases of patients in CHR states, and a 10.5-fold increase of prognostic certainty for patients with ROD. CONCLUSIONS AND RELEVANCE Precision medicine tools could augment effective therapeutic strategies aiming at the prevention of social functioning impairments in patients with CHR states or with ROD.
Collapse
Affiliation(s)
- Nikolaos Koutsouleris
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
| | | | - Stephan Ruhrmann
- Department of Psychiatry and Psychotherapy, University of Cologne, Cologne, Germany
| | - Marlene Rosen
- Department of Psychiatry and Psychotherapy, University of Cologne, Cologne, Germany
| | - Anne Ruef
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
| | - Dominic B. Dwyer
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
| | - Marco Paolini
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
| | | | - Joseph Kambeitz
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
| | - Theresa Haidl
- Department of Psychiatry and Psychotherapy, University of Cologne, Cologne, Germany
| | - André Schmidt
- Department of Psychiatry, University Psychiatric Clinic, Psychiatric University Hospital, University of Basel, Basel, Switzerland
| | - John Gillam
- Orygen, the National Centre of Excellence for Youth Mental Health, Melbourne, Australia,Centre for Youth Mental Health, University of Melbourne, Melbourne, Australia
| | - Frauke Schultze-Lutter
- Department of Psychiatry and Psychotherapy, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany
| | - Peter Falkai
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
| | - Maximilian Reiser
- Department of Radiology, Ludwig-Maximilian-University, Munich, Germany
| | - Anita Riecher-Rössler
- Department of Psychiatry, University Psychiatric Clinic, Psychiatric University Hospital, University of Basel, Basel, Switzerland
| | - Rachel Upthegrove
- Institute of Mental Health, University of Birmingham, Birmingham, United Kingdom,School of Psychology, University of Birmingham, United Kingdom
| | - Jarmo Hietala
- Department of Psychiatry, University of Turku, Turku, Finland
| | | | - Christos Pantelis
- Melbourne Neuropsychiatry Centre, University of Melbourne, Melbourne, Australia ,Melbourne Health, Melbourne, Australia
| | - Eva Meisenzahl
- Department of Psychiatry and Psychotherapy, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany
| | - Stephen J. Wood
- School of Psychology, University of Birmingham, United Kingdom,Orygen, the National Centre of Excellence for Youth Mental Health, Melbourne, Australia,Centre for Youth Mental Health, University of Melbourne, Melbourne, Australia
| | - Dirk Beque
- Corporate Global Research, GE Corporation, Munich, Germany
| | - Paolo Brambilla
- Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, University of Milan, Milan, Italy
| | - Stefan Borgwardt
- Department of Psychiatry, University Psychiatric Clinic, Psychiatric University Hospital, University of Basel, Basel, Switzerland
| | | |
Collapse
|
22
|
Rosenberg MD, Casey BJ, Holmes AJ. Prediction complements explanation in understanding the developing brain. Nat Commun 2018; 9:589. [PMID: 29467408 PMCID: PMC5821815 DOI: 10.1038/s41467-018-02887-9] [Citation(s) in RCA: 90] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2017] [Accepted: 01/05/2018] [Indexed: 11/08/2022] Open
Abstract
A central aim of human neuroscience is understanding the neurobiology of cognition and behavior. Although we have made significant progress towards this goal, reliance on group-level studies of the developed adult brain has limited our ability to explain population variability and developmental changes in neural circuitry and behavior. In this review, we suggest that predictive modeling, a method for predicting individual differences in behavior from brain features, can complement descriptive approaches and provide new ways to account for this variability. Highlighting the outsized scientific and clinical benefits of prediction in developmental populations including adolescence, we show that predictive brain-based models are already providing new insights on adolescent-specific risk-related behaviors. Together with large-scale developmental neuroimaging datasets and complementary analytic approaches, predictive modeling affords us the opportunity and obligation to identify novel treatment targets and individually tailor the course of interventions for developmental psychopathologies that impact so many young people today.
Collapse
Affiliation(s)
| | - B J Casey
- Department of Psychology, Yale University, New Haven, CT, 06520, USA
| | - Avram J Holmes
- Department of Psychology, Yale University, New Haven, CT, 06520, USA
- Department of Psychiatry, Yale University, New Haven, CT, 06511, USA
| |
Collapse
|