1
|
Ostojic D, Lalousis PA, Donohoe G, Morris DW. The challenges of using machine learning models in psychiatric research and clinical practice. Eur Neuropsychopharmacol 2024; 88:53-65. [PMID: 39232341 DOI: 10.1016/j.euroneuro.2024.08.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 08/06/2024] [Accepted: 08/12/2024] [Indexed: 09/06/2024]
Abstract
To understand the complex nature of heterogeneous psychiatric disorders, scientists and clinicians are required to employ a wide range of clinical, endophenotypic, neuroimaging, genomic, and environmental data to understand the biological mechanisms of psychiatric illness before this knowledge is applied into clinical setting. Machine learning (ML) is an automated process that can detect patterns from large multidimensional datasets and can supersede conventional statistical methods as it can detect both linear and non-linear relationships. Due to this advantage, ML has potential to enhance our understanding, improve diagnosis, prognosis and treatment of psychiatric disorders. The current review provides an in-depth examination of, and offers practical guidance for, the challenges encountered in the application of ML models in psychiatric research and clinical practice. These challenges include the curse of dimensionality, data quality, the 'black box' problem, hyperparameter tuning, external validation, class imbalance, and data representativeness. These challenges are particularly critical in the context of psychiatry as it is expected that researchers will encounter them during the stages of ML model development and deployment. We detail practical solutions and best practices to effectively mitigate the outlined challenges. These recommendations have the potential to improve reliability and interpretability of ML models in psychiatry.
Collapse
Affiliation(s)
- Dijana Ostojic
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland
| | - Paris Alexandros Lalousis
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom; Section for Precision Psychiatry, Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University Munich, Munich, Germany
| | - Gary Donohoe
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland
| | - Derek W Morris
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland.
| |
Collapse
|
2
|
Ciharova M, Amarti K, van Breda W, Peng X, Lorente-Català R, Funk B, Hoogendoorn M, Koutsouleris N, Fusar-Poli P, Karyotaki E, Cuijpers P, Riper H. Use of Machine Learning Algorithms Based on Text, Audio, and Video Data in the Prediction of Anxiety and Posttraumatic Stress in General and Clinical Populations: A Systematic Review. Biol Psychiatry 2024; 96:519-531. [PMID: 38866173 DOI: 10.1016/j.biopsych.2024.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 05/24/2024] [Accepted: 06/04/2024] [Indexed: 06/14/2024]
Abstract
Research in machine learning (ML) algorithms using natural behavior (i.e., text, audio, and video data) suggests that these techniques could contribute to personalization in psychology and psychiatry. However, a systematic review of the current state of the art is missing. Moreover, individual studies often target ML experts who may overlook potential clinical implications of their findings. In a narrative accessible to mental health professionals, we present a systematic review conducted in 5 psychology and 2 computer science databases. We included 128 studies that assessed the predictive power of ML algorithms using text, audio, and/or video data in the prediction of anxiety and posttraumatic stress disorder. Most studies (n = 87) were aimed at predicting anxiety, while the remainder (n = 41) focused on posttraumatic stress disorder. They were mostly published since 2019 in computer science journals and tested algorithms using text (n = 72) as opposed to audio or video. Studies focused mainly on general populations (n = 92) and less on laboratory experiments (n = 23) or clinical populations (n = 13). Methodological quality varied, as did reported metrics of the predictive power, hampering comparison across studies. Two-thirds of studies, which focused on both disorders, reported acceptable to very good predictive power (including high-quality studies only). The results of 33 studies were uninterpretable, mainly due to missing information. Research into ML algorithms using natural behavior is in its infancy but shows potential to contribute to diagnostics of mental disorders, such as anxiety and posttraumatic stress disorder, in the future if standardization of methods, reporting of results, and research in clinical populations are improved.
Collapse
Affiliation(s)
- Marketa Ciharova
- Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands; Black Dog Institute, University of New South Wales, Sydney, New South Wales, Australia.
| | - Khadicha Amarti
- Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Ward van Breda
- Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Xianhua Peng
- Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands; Department of Methodology and Statistics, Tilburg School of Social and Behavioral Sciences, Tilburg University, Tilburg, the Netherlands
| | - Rosa Lorente-Català
- Department of Basic and Clinical Psychology and Psychobiology, Universitat Jaume I, Castellon, Spain
| | - Burkhardt Funk
- Institute of Information Systems, Leuphana University, Lüneburg, Germany
| | - Mark Hoogendoorn
- Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Nikolaos Koutsouleris
- Artificial Intelligence in Mental Health Group, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom; Precision Psychiatry Group, Max Planck Institute, Munich, Germany; Section for Precision Psychiatry, Department of Psychiatry and Psychotherapy, University Medical Center, Ludwig-Maximilians-University Munich, Munich, Germany
| | - Paolo Fusar-Poli
- Section for Precision Psychiatry, Department of Psychiatry and Psychotherapy, University Medical Center, Ludwig-Maximilians-University Munich, Munich, Germany; Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom; Department of Brain and Behavioural Sciences, University of Pavia, Pavia, Italy; OASIS Service, South London and the Maudsley National Health Service Foundation Trust, London, United Kingdom
| | - Eirini Karyotaki
- Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands; WHO Collaborating Center for Research and Dissemination of Psychological Interventions, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Pim Cuijpers
- Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands; WHO Collaborating Center for Research and Dissemination of Psychological Interventions, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands; Babeș-Bolyai University, International Institute for Psychotherapy, Cluj-Napoca, Romania
| | - Heleen Riper
- Department of Clinical, Neuro, and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands; Department of Psychiatry, Amsterdam Public Health Research Institute, Amsterdam University Medical Centre, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
3
|
Perna G, Caldirola D, Schatzberg AF, Nemeroff CB. Advancements, challenges and future horizons in personalized psychiatry. World Psychiatry 2024; 23:460-461. [PMID: 39279367 PMCID: PMC11403164 DOI: 10.1002/wps.21257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/18/2024] Open
Affiliation(s)
- Giampaolo Perna
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Personalized Medicine Center for Anxiety and Panic Disorders, Humanitas San Pio X, Milan, Italy
- Department of Clinical Neurosciences, Villa S. Benedetto Menni Hospital, Albese con Cassano, Italy
| | - Daniela Caldirola
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy
- Personalized Medicine Center for Anxiety and Panic Disorders, Humanitas San Pio X, Milan, Italy
- Department of Clinical Neurosciences, Villa S. Benedetto Menni Hospital, Albese con Cassano, Italy
| | - Alan F Schatzberg
- Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA
| | - Charles B Nemeroff
- Department of Psychiatry and Behavioral Sciences, University of Texas-Austin, Dell Medical School, Austin, TX, USA
| |
Collapse
|
4
|
Kretzer S, Lawrence AJ, Pollard R, Ma X, Chen PJ, Amasi-Hartoonian N, Pariante C, Vallée C, Meaney M, Dazzan P. The Dynamic Interplay Between Puberty and Structural Brain Development as a Predictor of Mental Health Difficulties in Adolescence: A Systematic Review. Biol Psychiatry 2024; 96:585-603. [PMID: 38925264 DOI: 10.1016/j.biopsych.2024.06.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 06/16/2024] [Accepted: 06/19/2024] [Indexed: 06/28/2024]
Abstract
Puberty is a time of intense reorganization of brain structure and a high-risk period for the onset of mental health problems, with variations in pubertal timing and tempo intensifying this risk. We conducted 2 systematic reviews of articles published up to February 1, 2024, focusing on 1) the role of brain structure in the relationship between puberty and mental health, and 2) precision psychiatry research evaluating the utility of puberty in making individualized predictions of mental health outcomes in young people. The first review provides inconsistent evidence about whether and how pubertal and psychopathological processes may interact in relation to brain development. While most studies found an association between early puberty and mental health difficulties in adolescents, evidence on whether brain structure mediates this relationship is mixed. The pituitary gland was found to be associated with mental health status during this time, possibly through its central role in regulating puberty and its function in the hypothalamic-pituitary-gonadal and hypothalamic-pituitary-adrenal axes. In the second review, the design of studies that have explored puberty in predictive models did not allow for a quantification of its predictive power. However, when puberty was evaluated through physically observable characteristics rather than hormonal measures, it was more commonly identified as a predictor of depression, anxiety, and suicidality in adolescence. Social processes may be more relevant than biological ones to the link between puberty and mental health problems and represent an important target for educational strategies.
Collapse
Affiliation(s)
- Svenja Kretzer
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom; Singapore Institute for Clinical Sciences, Agency for Science, Technology & Research (A∗STAR) Singapore, Republic of Singapore.
| | - Andrew J Lawrence
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom
| | - Rebecca Pollard
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom
| | - Xuemei Ma
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom
| | - Pei Jung Chen
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom; Department of Psychiatry, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Nare Amasi-Hartoonian
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom; NIHR Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London, London, United Kingdom
| | - Carmine Pariante
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom
| | - Corentin Vallée
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom
| | - Michael Meaney
- Singapore Institute for Clinical Sciences, Agency for Science, Technology & Research (A∗STAR) Singapore, Republic of Singapore; Douglas Hospital Research Centre, Department of Psychiatry, McGill University, Montreal, Quebec, Canada
| | - Paola Dazzan
- Department of Psychological Medicine, Institute of Psychiatry, Psychology, and Neuroscience, King's College London, London, United Kingdom; NIHR Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London, London, United Kingdom.
| |
Collapse
|
5
|
Gargari OK, Fatehi F, Mohammadi I, Firouzabadi SR, Shafiee A, Habibi G. Diagnostic accuracy of large language models in psychiatry. Asian J Psychiatr 2024; 100:104168. [PMID: 39111087 DOI: 10.1016/j.ajp.2024.104168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/27/2024] [Revised: 07/20/2024] [Accepted: 07/22/2024] [Indexed: 09/13/2024]
Abstract
INTRODUCTION Medical decision-making is crucial for effective treatment, especially in psychiatry where diagnosis often relies on subjective patient reports and a lack of high-specificity symptoms. Artificial intelligence (AI), particularly Large Language Models (LLMs) like GPT, has emerged as a promising tool to enhance diagnostic accuracy in psychiatry. This comparative study explores the diagnostic capabilities of several AI models, including Aya, GPT-3.5, GPT-4, GPT-3.5 clinical assistant (CA), Nemotron, and Nemotron CA, using clinical cases from the DSM-5. METHODS We curated 20 clinical cases from the DSM-5 Clinical Cases book, covering a wide range of psychiatric diagnoses. Four advanced AI models (GPT-3.5 Turbo, GPT-4, Aya, Nemotron) were tested using prompts to elicit detailed diagnoses and reasoning. The models' performances were evaluated based on accuracy and quality of reasoning, with additional analysis using the Retrieval Augmented Generation (RAG) methodology for models accessing the DSM-5 text. RESULTS The AI models showed varied diagnostic accuracy, with GPT-3.5 and GPT-4 performing notably better than Aya and Nemotron in terms of both accuracy and reasoning quality. While models struggled with specific disorders such as cyclothymic and disruptive mood dysregulation disorders, others excelled, particularly in diagnosing psychotic and bipolar disorders. Statistical analysis highlighted significant differences in accuracy and reasoning, emphasizing the superiority of the GPT models. DISCUSSION The application of AI in psychiatry offers potential improvements in diagnostic accuracy. The superior performance of the GPT models can be attributed to their advanced natural language processing capabilities and extensive training on diverse text data, enabling more effective interpretation of psychiatric language. However, models like Aya and Nemotron showed limitations in reasoning, indicating a need for further refinement in their training and application. CONCLUSION AI holds significant promise for enhancing psychiatric diagnostics, with certain models demonstrating high potential in interpreting complex clinical descriptions accurately. Future research should focus on expanding the dataset and integrating multimodal data to further enhance the diagnostic capabilities of AI in psychiatry.
Collapse
Affiliation(s)
- Omid Kohandel Gargari
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Farhad Fatehi
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Australia; School of Psychological Sciences, Monash University, Melbourne, Australia
| | - Ida Mohammadi
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Shahryar Rajai Firouzabadi
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Arman Shafiee
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Gholamreza Habibi
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran.
| |
Collapse
|
6
|
Efthimiou O, Seo M, Chalkou K, Debray T, Egger M, Salanti G. Developing clinical prediction models: a step-by-step guide. BMJ 2024; 386:e078276. [PMID: 39227063 PMCID: PMC11369751 DOI: 10.1136/bmj-2023-078276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 06/12/2024] [Indexed: 09/05/2024]
Affiliation(s)
- Orestis Efthimiou
- Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Michael Seo
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | | | - Thomas Debray
- Smart Data Analysis and Statistics B V, Utrecht, The Netherlands
| | - Matthias Egger
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Georgia Salanti
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| |
Collapse
|
7
|
Xiang Q, Xiong XY, Liu S, Zhang MJ, Li YJ, Wang HW, Wu R, Chen L. Risk prediction model for in-stent restenosis following PCI: a systematic review. Front Cardiovasc Med 2024; 11:1445076. [PMID: 39267809 PMCID: PMC11390508 DOI: 10.3389/fcvm.2024.1445076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Accepted: 08/19/2024] [Indexed: 09/15/2024] Open
Abstract
Introduction The morbidity and mortality rates of coronary heart disease are significant, with PCI being the primary treatment. The high incidence of ISR following PCI poses a challenge to its effectiveness. Currently, there are numerous studies on ISR risk prediction models after PCI, but the quality varies and there is still a lack of systematic evaluation and analysis. Methods To systematically retrieve and evaluate the risk prediction models for ISR after PCI. A comprehensive search was conducted across 9 databases from inception to March 1, 2024. The screening of literature and extraction of data were independently carried out by two investigators, utilizing the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS). Additionally, the risk of bias and applicability were evaluated using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Results A total of 17 studies with 29 models were included, with a sample size of 175-10,004 cases, and the incidence of outcome events was 5.79%-58.86%. The area under the receiver operating characteristic curve was 0.530-0.953. The top 5 predictors with high frequency were diabetes, number of diseased vessels, age, LDL-C and stent diameter. Bias risk assessment into the research of the risk of higher bias the applicability of the four study better. Discussion The overall risk of bias in the current ISR risk prediction model post-PCI is deemed high. Moving forward, it is imperative to enhance study design and specify the reporting process, optimize and validate the model, and enhance its performance.
Collapse
Affiliation(s)
- Qin Xiang
- Department of Nursing, The 2nd Affiliated Hospital, Jiangxi Medical College, Nanchang University, Nanchang, China
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Xiao-Yun Xiong
- Department of Nursing, The 2nd Affiliated Hospital, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Si Liu
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Mei-Jun Zhang
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Ying-Jie Li
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Hui-Wen Wang
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Rui Wu
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| | - Lu Chen
- School of Nursing, Jiangxi Medical College, Nanchang University, Nanchang, China
| |
Collapse
|
8
|
Farooq S, Hattle M, Kingstone T, Ajnakina O, Dazzan P, Demjaha A, Murray RM, Di Forti M, Jones PB, Doody GA, Shiers D, Andrews G, Milner A, Nettis MA, Lawrence AJ, van der Windt DA, Riley RD. Development and initial evaluation of a clinical prediction model for risk of treatment resistance in first-episode psychosis: Schizophrenia Prediction of Resistance to Treatment (SPIRIT). Br J Psychiatry 2024:1-10. [PMID: 39101211 DOI: 10.1192/bjp.2024.101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/06/2024]
Abstract
BACKGROUND A clinical tool to estimate the risk of treatment-resistant schizophrenia (TRS) in people with first-episode psychosis (FEP) would inform early detection of TRS and overcome the delay of up to 5 years in starting TRS medication. AIMS To develop and evaluate a model that could predict the risk of TRS in routine clinical practice. METHOD We used data from two UK-based FEP cohorts (GAP and AESOP-10) to develop and internally validate a prognostic model that supports identification of patients at high-risk of TRS soon after FEP diagnosis. Using sociodemographic and clinical predictors, a model for predicting risk of TRS was developed based on penalised logistic regression, with missing data handled using multiple imputation. Internal validation was undertaken via bootstrapping, obtaining optimism-adjusted estimates of the model's performance. Interviews and focus groups with clinicians were conducted to establish clinically relevant risk thresholds and understand the acceptability and perceived utility of the model. RESULTS We included seven factors in the prediction model that are predominantly assessed in clinical practice in patients with FEP. The model predicted treatment resistance among the 1081 patients with reasonable accuracy; the model's C-statistic was 0.727 (95% CI 0.723-0.732) prior to shrinkage and 0.687 after adjustment for optimism. Calibration was good (expected/observed ratio: 0.999; calibration-in-the-large: 0.000584) after adjustment for optimism. CONCLUSIONS We developed and internally validated a prediction model with reasonably good predictive metrics. Clinicians, patients and carers were involved in the development process. External validation of the tool is needed followed by co-design methodology to support implementation in early intervention services.
Collapse
Affiliation(s)
- Saeed Farooq
- School of Medicine, Keele University, Newcastle-under-Lyme, UK; National Institute for Health and Care Research (NIHR), UK; and St George's Hospital, Midlands Partnership University NHS Foundation Trust, Stafford, UK
| | - Miriam Hattle
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; and National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Tom Kingstone
- School of Medicine, Keele University, Newcastle-under-Lyme, UK; National Institute for Health and Care Research (NIHR), UK; and St George's Hospital, Midlands Partnership University NHS Foundation Trust, Stafford, UK
| | - Olesya Ajnakina
- Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK; and Department of Behavioural Science and Health, Institute of Epidemiology and Health Care, University College London, London, UK
| | - Paola Dazzan
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Arsime Demjaha
- Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Robin M Murray
- Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK; and Department of Psychiatry, Experimental Biomedicine and Clinical Neuroscience, University of Palermo, Palermo, Italy
| | - Marta Di Forti
- Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Peter B Jones
- Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - Gillian A Doody
- Division of Psychiatry and Applied Psychology, University of Nottingham, Nottingham, UK
| | - David Shiers
- School of Medicine, Keele University, Newcastle-under-Lyme, UK; Psychosis Research Unit, Greater Manchester Mental Health NHS Trust, Manchester, UK; and University of Manchester, Manchester, UK
| | - Gabrielle Andrews
- St George's Hospital, Midlands Partnership University NHS Foundation Trust, Stafford, UK
| | - Abbie Milner
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; and National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Maria Antonietta Nettis
- South London and Maudsley NHS Foundation Trust, London, UK; and Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Andrew J Lawrence
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Danielle A van der Windt
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; and National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; and National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| |
Collapse
|
9
|
Cortese S. The long road to precision pediatric sleep medicine. Sleep Med Rev 2024; 76:101971. [PMID: 38917682 DOI: 10.1016/j.smrv.2024.101971] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2024] [Accepted: 06/18/2024] [Indexed: 06/27/2024]
Affiliation(s)
- Samuele Cortese
- EPI (Evidence synthesis, Prediction, Implementation) developmental lab, Centre for Innovation in Mental Health, School of Psychology, Faculty of Environmental and Life Sciences, University of Southampton, Southampton, UK; Clinical and Experimental Sciences, Faculty of Medicine, University of Southampton, Southampton, UK; Solent NHS Trust, Southampton, UK; Hassenfeld Children's Hospital at NYU Langone, New York University Child Study Center, New York, NY, USA; DiMePRe-J-Department of Precision and Regenerative Medicine-Jonic Area, University of Bari "Aldo Moro", Bari, Italy.
| |
Collapse
|
10
|
Zainal NH, Bossarte RM, Gildea SM, Hwang I, Kennedy CJ, Liu H, Luedtke A, Marx BP, Petukhova MV, Post EP, Ross EL, Sampson NA, Sverdrup E, Turner B, Wager S, Kessler RC. Developing an individualized treatment rule for Veterans with major depressive disorder using electronic health records. Mol Psychiatry 2024; 29:2335-2345. [PMID: 38486050 PMCID: PMC11399319 DOI: 10.1038/s41380-024-02500-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 02/23/2024] [Accepted: 02/27/2024] [Indexed: 09/16/2024]
Abstract
Efforts to develop an individualized treatment rule (ITR) to optimize major depressive disorder (MDD) treatment with antidepressant medication (ADM), psychotherapy, or combined ADM-psychotherapy have been hampered by small samples, small predictor sets, and suboptimal analysis methods. Analyses of large administrative databases designed to approximate experiments followed iteratively by pragmatic trials hold promise for resolving these problems. The current report presents a proof-of-concept study using electronic health records (EHR) of n = 43,470 outpatients beginning MDD treatment in Veterans Health Administration Primary Care Mental Health Integration (PC-MHI) clinics, which offer access not only to ADMs but also psychotherapy and combined ADM-psychotherapy. EHR and geospatial databases were used to generate an extensive baseline predictor set (5,865 variables). The outcome was a composite measure of at least one serious negative event (suicide attempt, psychiatric emergency department visit, psychiatric hospitalization, suicide death) over the next 12 months. Best-practices methods were used to adjust for nonrandom treatment assignment and to estimate a preliminary ITR in a 70% training sample and to evaluate the ITR in the 30% test sample. Statistically significant aggregate variation was found in overall probability of the outcome related to baseline predictors (AU-ROC = 0.68, S.E. = 0.01), with test sample outcome prevalence of 32.6% among the 5% of patients having highest predicted risk compared to 7.1% in the remainder of the test sample. The ITR found that psychotherapy-only was the optimal treatment for 56.0% of patients (roughly 20% lower risk of the outcome than if receiving one of the other treatments) and that treatment type was unrelated to outcome risk among other patients. Change in aggregate treatment costs of implementing this ITR would be negligible, as 16.1% fewer patients would be prescribed ADMs and 2.9% more would receive psychotherapy. A pragmatic trial would be needed to confirm the accuracy of the ITR.
Collapse
Affiliation(s)
- Nur Hani Zainal
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| | - Robert M Bossarte
- Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, USA
| | - Sarah M Gildea
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| | - Irving Hwang
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| | - Chris J Kennedy
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
| | - Howard Liu
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
- Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, USA
| | - Alex Luedtke
- Department of Statistics, University of Washington, Seattle, WA, USA
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, USA
| | - Brian P Marx
- National Center for PTSD, VA Boston Healthcare System, Boston, MA, USA
- Department of Psychiatry, Boston University School of Medicine, Boston, MA, USA
| | - Maria V Petukhova
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| | - Edward P Post
- Center for Clinical Management Research, VA Ann Arbor Health Care System, Ann Arbor, MI, USA
- Department of Medicine, University of Michigan Medical School, Ann Arbor, MI, USA
| | - Eric L Ross
- Department of Psychiatry, Larner College of Medicine, University of Vermont, Burlington, VT, USA
| | - Nancy A Sampson
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| | - Erik Sverdrup
- Graduate School of Business, Stanford University, Stanford, CA, USA
| | - Brett Turner
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Stefan Wager
- Graduate School of Business, Stanford University, Stanford, CA, USA
| | - Ronald C Kessler
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
11
|
Tait J, Kellett S, Saxon D, Deisenhofer AK, Lutz W, Barkham M, Delgadillo J. Individual treatment selection for patients with post-traumatic stress disorder: External validation of a personalised advantage index. Psychother Res 2024:1-14. [PMID: 38862129 DOI: 10.1080/10503307.2024.2360449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Accepted: 05/21/2024] [Indexed: 06/13/2024] Open
Abstract
OBJECTIVE To test the predictive accuracy and generalisability of a personalised advantage index (PAI) model designed to support treatment selection for Post-Traumatic Stress Disorder (PTSD). METHOD A PAI model developed by Deisenhofer et al. (2018) was used to predict treatment outcomes in a statistically independent dataset including archival records for N = 152 patients with PSTD who accessed either trauma-focussed cognitive behavioural therapy or eye movement desensitisation and reprocessing in routine care. Outcomes were compared between patients who received their PAI-indicated optimal treatment versus those who received their suboptimal treatment. RESULTS The model did not yield treatment specific predictions and patients who had received their PAI-indicated optimal treatment did not have better treatment outcomes in this external validation sample. CONCLUSION This PAI model did not generalise to an external validation sample.
Collapse
Affiliation(s)
- James Tait
- School of Psychology, University of Sheffield, ICOSS Building, 219 Portobello, Sheffield, S1 4DP, United Kingdom
| | - Stephen Kellett
- Grounded Research, RDaSH NHS Foundation Trust, Doncaster, United Kingdom
| | - David Saxon
- Clinical and Applied Psychology Unit, School of Psychology, University of Sheffield, Sheffield, United Kingdom
| | | | - Wolfgang Lutz
- Clinical Psychology and Psychotherapy, Department of Psychology, University of Trier, Trier, Germany
| | - Michael Barkham
- Clinical and Applied Psychology Unit, School of Psychology, University of Sheffield, Sheffield, United Kingdom
| | - Jaime Delgadillo
- Clinical and Applied Psychology Unit, School of Psychology, University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|
12
|
van Dellen E. Precision psychiatry: predicting predictability. Psychol Med 2024; 54:1500-1509. [PMID: 38497091 DOI: 10.1017/s0033291724000370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Precision psychiatry is an emerging field that aims to provide individualized approaches to mental health care. An important strategy to achieve this precision is to reduce uncertainty about prognosis and treatment response. Multivariate analysis and machine learning are used to create outcome prediction models based on clinical data such as demographics, symptom assessments, genetic information, and brain imaging. While much emphasis has been placed on technical innovation, the complex and varied nature of mental health presents significant challenges to the successful implementation of these models. From this perspective, I review ten challenges in the field of precision psychiatry, including the need for studies on real-world populations and realistic clinical outcome definitions, and consideration of treatment-related factors such as placebo effects and non-adherence to prescriptions. Fairness, prospective validation in comparison to current practice and implementation studies of prediction models are other key issues that are currently understudied. A shift is proposed from retrospective studies based on linear and static concepts of disease towards prospective research that considers the importance of contextual factors and the dynamic and complex nature of mental health.
Collapse
Affiliation(s)
- Edwin van Dellen
- Department of Psychiatry and University Medical Center Utrecht Brain Center, Utrecht University, Utrecht, the Netherlands
- Department of Neurology, UZ Brussel and Vrije Universiteit Brussel, Brussels, Belgium
| |
Collapse
|
13
|
Piccin J, Viduani A, Buchweitz C, Pereira RB, Zimerman A, Amando GR, Cosenza V, Ferreira LZ, McMahon NA, Melo RF, Richter D, Reckziegel FD, Rohrsetzer F, Souza L, Tonon AC, Costa-Valle MT, Zajkowska Z, Araújo RM, Hauser TU, van Heerden A, Hidalgo MP, Kohrt BA, Mondelli V, Swartz JR, Fisher HL, Kieling C. Prospective Follow-Up of Adolescents With and at Risk for Depression: Protocol and Methods of the Identifying Depression Early in Adolescence Risk Stratified Cohort Longitudinal Assessments. JAACAP OPEN 2024; 2:145-159. [PMID: 38863682 PMCID: PMC11163476 DOI: 10.1016/j.jaacop.2023.11.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 12/07/2023] [Indexed: 06/13/2024]
Abstract
Objective To present the protocol and methods for the prospective longitudinal assessments-including clinical and digital phenotyping approaches-of the Identifying Depression Early in Adolescence Risk Stratified Cohort (IDEA-RiSCo) study, which comprises Brazilian adolescents stratified at baseline by risk of developing depression or presence of depression. Method Of 7,720 screened adolescents aged 14 to 16 years, we recruited 150 participants (75 boys, 75 girls) based on a composite risk score: 50 with low risk for developing depression (LR), 50 with high risk for developing depression (HR), and 50 with an active untreated major depressive episode (MDD). Three annual follow-up assessments were conducted, involving clinical measures (parent- and adolescent-reported questionnaires and psychiatrist assessments), active and passive data sensing via smartphones, and neurobiological measures (neuroimaging and biological material samples). Retention rates were 96% (Wave 1), 94% (Wave 2), and 88% (Wave 3), with no significant differences by sex or group (p > .05). Participants highlighted their familiarity with the research team and assessment process as a motivator for sustained engagement. Discussion This protocol relied on novel aspects, such as the use of a WhatsApp bot, which is particularly pertinent for low- to-middle-income countries, and the collection of information from diverse sources in a longitudinal design, encompassing clinical data, self-reports, parental reports, Global Positioning System (GPS) data, and ecological momentary assessments. The study engaged adolescents over an extensive period and demonstrated the feasibility of conducting a prospective follow-up study with a risk-enriched cohort of adolescents in a middle-income country, integrating mobile technology with traditional methodologies to enhance longitudinal data collection.
Collapse
Affiliation(s)
- Jader Piccin
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Anna Viduani
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Claudia Buchweitz
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Rivka B. Pereira
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Aline Zimerman
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Guilherme R. Amando
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Laboratório de Cronobiologia e Sono, Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil
| | - Victor Cosenza
- Universidade Federal de Pelotas (UFPEL), Pelotas, Brazil
| | | | - Natália A.G. McMahon
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | | | - Danyella Richter
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Frederico D.S. Reckziegel
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Fernanda Rohrsetzer
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Laila Souza
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - André C. Tonon
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Laboratório de Cronobiologia e Sono, Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil
| | - Marina Tuerlinckx Costa-Valle
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| | - Zuzanna Zajkowska
- King’s College London, Institute of Psychiatry, Psychology & Neuroscience, London, United Kingdom
| | | | - Tobias U. Hauser
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom, Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom and with Eberhard Karls University of Tübingen, Tübingen, Germany
| | - Alastair van Heerden
- Human and Social Development, Human Sciences Research Council, Pietermaritzburg, South Africa and Medical Research Council/Wits Developmental Pathways for Health Research Unit, University of the Witwatersrand, Johannesburg, South Africa
| | - Maria Paz Hidalgo
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Laboratório de Cronobiologia e Sono, Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil
| | | | - Valeria Mondelli
- King’s College London, Institute of Psychiatry, Psychology & Neuroscience, London, United Kingdom
- National Institute for Health and Care Research Maudsley Mental Health Biomedical Research Centre, South London and Maudsley NHS Foundation Trust, London, United Kingdom
| | | | - Helen L. Fisher
- King’s College London, Institute of Psychiatry, Psychology & Neuroscience, London, United Kingdom
- ESRC Centre for Society and Mental Health, King’s College London, London, United Kingdom
| | - Christian Kieling
- Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil
- Prodia - Child & Adolescent Depression Program, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, Brazil
| |
Collapse
|
14
|
Newby D, Taylor N, Joyce DW, Winchester LM. Optimising the use of electronic medical records for large scale research in psychiatry. Transl Psychiatry 2024; 14:232. [PMID: 38824136 PMCID: PMC11144247 DOI: 10.1038/s41398-024-02911-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 04/13/2024] [Accepted: 04/15/2024] [Indexed: 06/03/2024] Open
Abstract
The explosion and abundance of digital data could facilitate large-scale research for psychiatry and mental health. Research using so-called "real world data"-such as electronic medical/health records-can be resource-efficient, facilitate rapid hypothesis generation and testing, complement existing evidence (e.g. from trials and evidence-synthesis) and may enable a route to translate evidence into clinically effective, outcomes-driven care for patient populations that may be under-represented. However, the interpretation and processing of real-world data sources is complex because the clinically important 'signal' is often contained in both structured and unstructured (narrative or "free-text") data. Techniques for extracting meaningful information (signal) from unstructured text exist and have advanced the re-use of routinely collected clinical data, but these techniques require cautious evaluation. In this paper, we survey the opportunities, risks and progress made in the use of electronic medical record (real-world) data for psychiatric research.
Collapse
Affiliation(s)
- Danielle Newby
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Centre for Statistics in Medicine, University of Oxford, Oxford, UK
| | - Niall Taylor
- Department of Psychiatry, University of Oxford, Oxford, UK
| | - Dan W Joyce
- Department of Primary Care and Mental Health and Civic Health, Innovation Labs, Institute of Population Health, University of Liverpool, Liverpool, UK
| | | |
Collapse
|
15
|
Delamain H, Buckman JEJ, O'Driscoll C, Suh JW, Stott J, Singh S, Naqvi SA, Leibowitz J, Pilling S, Saunders R. Predicting post-treatment symptom severity for adults receiving psychological therapy in routine care for generalised anxiety disorder: a machine learning approach. Psychiatry Res 2024; 336:115910. [PMID: 38608539 DOI: 10.1016/j.psychres.2024.115910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 04/03/2024] [Accepted: 04/08/2024] [Indexed: 04/14/2024]
Abstract
Approximately half of generalised anxiety disorder (GAD) patients do not recover from first-line treatments, and no validated prediction models exist to inform individuals or clinicians of potential treatment benefits. This study aimed to develop and validate an accurate and explainable prediction model of post-treatment GAD symptom severity. Data from adults receiving treatment for GAD in eight Improving Access to Psychological Therapies (IAPT) services (n=15,859) were separated into training, validation and holdout datasets. Thirteen machine learning algorithms were compared using 10-fold cross-validation, against two simple clinically relevant comparison models. The best-performing model was tested on the holdout dataset and model-specific explainability measures identified the most important predictors. A Bayesian Additive Regression Trees model out-performed all comparison models (MSE=16.54 [95 % CI=15.58; 17.51]; MAE=3.19; R²=0.33, including a single predictor linear regression model: MSE=20.70 [95 % CI=19.58; 21.82]; MAE=3.94; R²=0.14). The five most important predictors were: PHQ-9 anhedonia, GAD-7 annoyance/irritability, restlessness and fear items, then the referral-assessment waiting time. The best-performing model accurately predicted post-treatment GAD symptom severity using only pre-treatment data, outperforming comparison models that approximated clinical judgement and remaining within the GAD-7 error of measurement and minimal clinically important differences. This model could inform treatment decision-making and provide desired information to clinicians and patients receiving treatment for GAD.
Collapse
Affiliation(s)
- H Delamain
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom.
| | - J E J Buckman
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom; iCope - Camden and Islington Psychological Therapies Services, Camden & Islington NHS Foundation Trust, London, United Kingdom
| | - C O'Driscoll
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| | - J W Suh
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| | - J Stott
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| | - S Singh
- Waltham Forest Talking Therapies, North East London NHS Foundation Trust, London, United Kingdom
| | - S A Naqvi
- Barking and Dagenham and Havering IAPT Services, North East London NHS Foundation Trust, London, United Kingdom
| | - J Leibowitz
- iCope - Camden and Islington Psychological Therapies Services, Camden & Islington NHS Foundation Trust, London, United Kingdom
| | - S Pilling
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom; Camden and Islington NHS Foundation Trust, London, United Kingdom
| | - R Saunders
- CORE Data Lab, Centre for Outcomes Research and Effectiveness (CORE), Research Department of Clinical, Educational and Health Psychology, UCL, London, United Kingdom
| |
Collapse
|
16
|
Guinart D, Fagiolini A, Fusar-Poli P, Giordano GM, Leucht S, Moreno C, Correll CU. On the Road to Individualizing Pharmacotherapy for Adolescents and Adults with Schizophrenia - Results from an Expert Consensus Following the Delphi Method. Neuropsychiatr Dis Treat 2024; 20:1139-1152. [PMID: 38812809 PMCID: PMC11133879 DOI: 10.2147/ndt.s456163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Accepted: 04/27/2024] [Indexed: 05/31/2024] Open
Abstract
Introduction Schizophrenia is a severe mental illness that usually begins in late adolescence or early adulthood. Current pharmacological treatments, while acceptably effective for many patients, are rarely clinically tailored or individualized. The lack of sufficient etiopathological knowledge of the disease, together with overall comparable effect sizes for efficacy between available antipsychotics and the absence of clinically actionable biomarkers, has hindered the advance of individualized medicine in the treatment of schizophrenia. Nevertheless, some degree of stratification based on clinical markers could guide treatment choices and help clinicians move toward individualized psychiatry. To this end, a panel of experts met to formally discuss the current approach to individualized treatment in schizophrenia and to define how treatment individualization could help improve clinical outcomes. Methods A task force of seven experts iteratively developed, evaluated, and refined questionnaire items, which were then evaluated using the Delphi method. Descriptive statistics were used to summarize and rank expert responses. Expert discussion, informed by the results of a scoping review on personalizing the pharmacologic treatment of adults and adolescents with schizophrenia, ultimately generated recommendations to guide individualized pharmacologic treatment in this population. Results There was substantial agreement among the expert group members, resulting in the following recommendations: 1) individualization of treatment requires consideration of the patient's diagnosis, clinical presentation, comorbidities, previous treatment response, drug tolerability, adherence patterns, and social factors; 2) patient preferences should be considered in a shared decision-making approach; 3) identified barriers to personalized care that need to be overcome include the lack of actionable biomarkers and mechanistic similarities between available treatments, but digital tools should be increasingly used to enhance individualized treatment. Conclusion Individualized care can help provide effective, tailored treatments based on an individual's clinical characteristics, disease trajectory, family and social environment, and goals and preferences.
Collapse
Affiliation(s)
- Daniel Guinart
- Institut de Salut Mental, Parc de Salut Mar, Barcelona, Spain
- Hospital Del Mar Research Institute, CIBERSAM, Barcelona, Spain
- Department of Psychiatry, The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA
| | - Andrea Fagiolini
- Department of Molecular Medicine, University of Siena School of Medicine, Siena, Italy
| | - Paolo Fusar-Poli
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
- Department of Psychosis Studies, King’s College London, London, UK
- Outreach and Support in South-London (OASIS) Service, South London and Maudsley (Slam) NHS Foundation Trust, London, UK
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University Munich, Munich, Germany
| | | | - Stefan Leucht
- Technical University of Munich, TUM School of Medicine and Health, Department of Psychiatry and Psychotherapy, Munich, Germany
| | - Carmen Moreno
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry and Mental Health, Hospital General Universitario Gregorio Marañón, Instituto de Investigación Sanitaria Gregorio Marañón (LISGM), Madrid, Spain
- Centro de Investigación Biomedica en Red (CIBERSAM), ISCIII, Madrid, Spain
- School of Medicine, Universidad Complutense, Madrid, Spain
| | - Christoph U Correll
- Department of Psychiatry, The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA
- Department of Psychiatry, The Zucker Hillside Hospital, New York, NY, USA
- Department of Child and Adolescent Psychiatry, Charité Universitatsmedizin, Berlin, Germany
- German Center for Mental Health (DZPG), Partner Site, Berlin, Germany
| |
Collapse
|
17
|
Benster L, Weissman C, Suprani F, Toney K, Afshar H, Stapper N, Tello V, Stolz L, Poorganji M, Daskalakis Z, Appelbaum L, Kohn J. Predictive modeling of response to repetitive transcranial magnetic stimulation in treatment-resistant depression. RESEARCH SQUARE 2024:rs.3.rs-4396926. [PMID: 38826302 PMCID: PMC11142351 DOI: 10.21203/rs.3.rs-4396926/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
Identifying predictors of treatment response to repetitive transcranial magnetic stimulation (rTMS) remain elusive in treatment-resistant depression (TRD). Leveraging electronic medical records (EMR), this retrospective cohort study applied supervised machine learning (ML) to sociodemographic, clinical, and treatment-related data to predict depressive symptom response (>50% reduction on PHQ-9) and remission (PHQ-9 < 5) following rTMS in 232 patients with TRD (mean age: 54.5, 63.4% women) treated at the University of California, San Diego Interventional Psychiatry Program between 2017 and 2023. ML models were internally validated using nested cross-validation and Shapley values were calculated to quantify contributions of each feature to response prediction. The best-fit models proved reasonably accurate at discriminating treatment responders (Area under the curve (AUC): 0.689 [0.638, 0.740], p < 0.01) and remitters (AUC 0.745 [0.692, 0.797], p < 0.01), though only the response model was well-calibrated. Both models were associated with significant net benefits, indicating their potential utility for clinical decision-making. Shapley values revealed that patients with comorbid anxiety, obesity, concurrent psychiatric medication use, and more chronic TRD were less likely to respond or remit following rTMS. Patients with trauma and former tobacco users were more likely to respond. Furthermore, delivery of intermittent theta burst stimulation and more rTMS sessions were associated with superior outcomes. These findings highlight the potential of ML-guided techniques to guide clinical decision-making for rTMS treatment in patients with TRD to optimize therapeutic outcomes.
Collapse
|
18
|
Salazar de Pablo G, Iniesta R, Bellato A, Caye A, Dobrosavljevic M, Parlatini V, Garcia-Argibay M, Li L, Cabras A, Haider Ali M, Archer L, Meehan AJ, Suleiman H, Solmi M, Fusar-Poli P, Chang Z, Faraone SV, Larsson H, Cortese S. Individualized prediction models in ADHD: a systematic review and meta-regression. Mol Psychiatry 2024:10.1038/s41380-024-02606-5. [PMID: 38783054 DOI: 10.1038/s41380-024-02606-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 04/30/2024] [Accepted: 05/09/2024] [Indexed: 05/25/2024]
Abstract
There have been increasing efforts to develop prediction models supporting personalised detection, prediction, or treatment of ADHD. We overviewed the current status of prediction science in ADHD by: (1) systematically reviewing and appraising available prediction models; (2) quantitatively assessing factors impacting the performance of published models. We did a PRISMA/CHARMS/TRIPOD-compliant systematic review (PROSPERO: CRD42023387502), searching, until 20/12/2023, studies reporting internally and/or externally validated diagnostic/prognostic/treatment-response prediction models in ADHD. Using meta-regressions, we explored the impact of factors affecting the area under the curve (AUC) of the models. We assessed the study risk of bias with the Prediction Model Risk of Bias Assessment Tool (PROBAST). From 7764 identified records, 100 prediction models were included (88% diagnostic, 5% prognostic, and 7% treatment-response). Of these, 96% and 7% were internally and externally validated, respectively. None was implemented in clinical practice. Only 8% of the models were deemed at low risk of bias; 67% were considered at high risk of bias. Clinical, neuroimaging, and cognitive predictors were used in 35%, 31%, and 27% of the studies, respectively. The performance of ADHD prediction models was increased in those models including, compared to those models not including, clinical predictors (β = 6.54, p = 0.007). Type of validation, age range, type of model, number of predictors, study quality, and other type of predictors did not alter the AUC. Several prediction models have been developed to support the diagnosis of ADHD. However, efforts to predict outcomes or treatment response have been limited, and none of the available models is ready for implementation into clinical practice. The use of clinical predictors, which may be combined with other type of predictors, seems to improve the performance of the models. A new generation of research should address these gaps by conducting high quality, replicable, and externally validated models, followed by implementation research.
Collapse
Affiliation(s)
- Gonzalo Salazar de Pablo
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
- Child and Adolescent Mental Health Services, South London and Maudsley NHS Foundation Trust, London, UK
- Institute of Psychiatry and Mental Health. Department of Child and Adolescent Psychiatry, Hospital General Universitario Gregorio Marañón School of Medicine, Universidad Complutense, Instituto de Investigación Sanitaria Gregorio Marañón (IiSGM), CIBERSAM, Madrid, Spain
| | - Raquel Iniesta
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neurosciences, King's College London, London, UK
- King's Institute for Artificial Intelligence, King's College London, London, UK
| | - Alessio Bellato
- School of Psychology, University of Nottingham, Nottingham, Malaysia
- Centre for Innovation in Mental Health-Developmental Lab, School of Psychology, University of Southampton, Southampton, UK
- School of Psychology, University of Southampton, Southampton, UK
| | - Arthur Caye
- Post-Graduate Program of Psychiatry, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
- National Center for Research and Innovation (CISM), University of São Paulo, São Paulo, Brazil
- ADHD Outpatient Program, Hospital de Clínicas de Porto Alegre, Porto Alegre, Brazil
| | - Maja Dobrosavljevic
- School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro, Sweden
| | - Valeria Parlatini
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
- Centre for Innovation in Mental Health-Developmental Lab, School of Psychology, University of Southampton, Southampton, UK
- School of Psychology, University of Southampton, Southampton, UK
- Solent NHS Trust, Southampton, UK
| | - Miguel Garcia-Argibay
- Centre for Innovation in Mental Health-Developmental Lab, School of Psychology, University of Southampton, Southampton, UK
- School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro, Sweden
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Lin Li
- School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro, Sweden
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Anna Cabras
- Department of Neurology and Psychiatry, University of Rome La Sapienza, Rome, Italy
| | - Mian Haider Ali
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neurosciences, King's College London, London, UK
| | - Lucinda Archer
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR), Birmingham Biomedical Research Centre, Birmingham, UK
| | - Alan J Meehan
- Department of Psychology, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
- Yale Child Study Center, Yale School of Medicine, New Haven, CT, USA
| | - Halima Suleiman
- Departments of Psychiatry and of Neuroscience and Physiology, SUNY Upstate Medical University, Syracuse, Syracuse, NY, USA
| | - Marco Solmi
- Centre for Innovation in Mental Health-Developmental Lab, School of Psychology, University of Southampton, Southampton, UK
- Department of Psychiatry, University of Ottawa, Ottawa, ON, Canada
- Department of Mental Health, The Ottawa Hospital, Ottawa, ON, Canada
- Hospital Research Institute (OHRI) Clinical Epidemiology Program University of Ottawa, Ontario, ON, Canada
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
- Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany
| | - Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
- Outreach and Support in South-London (OASIS) service, South London and Maudsley NHS Foundation Trust, London, UK
- Department of Psychiatry and Psychotherapy, University Hospital, Ludwig-Maximilian-University (LMU), Munich, Germany
| | - Zheng Chang
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Stephen V Faraone
- Departments of Psychiatry and of Neuroscience and Physiology, SUNY Upstate Medical University, Syracuse, Syracuse, NY, USA
| | - Henrik Larsson
- School of Psychology, University of Southampton, Southampton, UK
- School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro, Sweden
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Samuele Cortese
- Centre for Innovation in Mental Health-Developmental Lab, School of Psychology, University of Southampton, Southampton, UK.
- Solent NHS Trust, Southampton, UK.
- Clinical and Experimental Sciences (CNS and Psychiatry), Faculty of Medicine, University of Southampton, Southampton, UK.
- Hassenfeld Children's Hospital at NYU Langone, New York University Child Study Center, New York City, NY, USA.
- DiMePRe-J-Department of Precision and Rigenerative Medicine-Jonic Area, University of Bari "Aldo Moro", Bari, Italy.
| |
Collapse
|
19
|
Bouter DC, Ravensbergen SJ, de Neve-Enthoven NGM, Zarchev M, Mulder CL, Hoogendijk WJG, Roza SJ, Grootendorst-van Mil NH. Five-year follow-up of the iBerry Study: screening in early adolescence to identify those at risk of psychopathology in emerging adulthood. Eur Child Adolesc Psychiatry 2024:10.1007/s00787-024-02462-2. [PMID: 38772966 DOI: 10.1007/s00787-024-02462-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 04/30/2024] [Indexed: 05/23/2024]
Abstract
The iBerry Study, a Dutch population-based high-risk cohort (n = 1022) examines the transition from subclinical symptoms to psychiatric disorders in adolescents. Here, we present the first follow-up measurement, approximately 3 years after baseline assessment and 5 years after the screening based on self-reported emotional and behavioral problems (SDQ-Y). We give an update on the data collection, details on the (non)response, and the results on psychopathology outcomes. The first follow-up (2019-2022) had a response rate of 79% (n = 807). Our results at baseline (mean age 15.0 years) have shown the effectiveness of using the SDQ-Y to select a cohort oversampled for the risk of psychopathology. At first follow-up (mean age 18.1 years), the previously administered SDQ-Y remains predictive for selecting adolescents at risk. At follow-up, 47% of the high-risk adolescents showed significant mental health problems based on self- and parent reports and 46% of the high-risk adolescents met the criteria for multiple DSM-5 diagnoses. Compared to low-risk adolescents, high-risk adolescents had a sevenfold higher odds of significant emotional and behavioral problems at follow-up. Comprehensive assessment on psychopathology, substance abuse, psychotic symptoms, suicidality, nonsuicidal self-injury, addiction to social media and/or video gaming, and delinquency, as well as social development, and the utilization of healthcare and social services were conducted. This wave, as well as the ones to follow, track these adolescents into their young adulthood to identify risk factors, elucidate causal mechanisms, and discern pathways leading to both common and severe mental disorders. Results from the iBerry Study will provide leads for preventive interventions.
Collapse
Affiliation(s)
- D C Bouter
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands
| | - S J Ravensbergen
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands
| | - N G M de Neve-Enthoven
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands
| | - M Zarchev
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands
| | - C L Mulder
- Epidemiological and Social Psychiatric Research Institute (ESPRi), Department of Psychiatry, Erasmus MC University Medical Center, Rotterdam, The Netherlands
- Parnassia Psychiatric Institute, Rotterdam, The Netherlands
| | - W J G Hoogendijk
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands
| | - S J Roza
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands
| | - N H Grootendorst-van Mil
- Department of Psychiatry, Erasmus MC, University Medical Center Rotterdam, P.O. box 2040, 3000 CA, Rotterdam, The Netherlands.
- Epidemiological and Social Psychiatric Research Institute (ESPRi), Department of Psychiatry, Erasmus MC University Medical Center, Rotterdam, The Netherlands.
| |
Collapse
|
20
|
Meinke C, Lueken U, Walter H, Hilbert K. Predicting treatment outcome based on resting-state functional connectivity in internalizing mental disorders: A systematic review and meta-analysis. Neurosci Biobehav Rev 2024; 160:105640. [PMID: 38548002 DOI: 10.1016/j.neubiorev.2024.105640] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 02/29/2024] [Accepted: 03/21/2024] [Indexed: 04/07/2024]
Abstract
Predicting treatment outcome in internalizing mental disorders prior to treatment initiation is pivotal for precision mental healthcare. In this regard, resting-state functional connectivity (rs-FC) and machine learning have often shown promising prediction accuracies. This systematic review and meta-analysis evaluates these studies, considering their risk of bias through the Prediction Model Study Risk of Bias Assessment Tool (PROBAST). We examined the predictive performance of features derived from rs-FC, identified features with the highest predictive value, and assessed the employed machine learning pipelines. We searched the electronic databases Scopus, PubMed and PsycINFO on the 12th of December 2022, which resulted in 13 included studies. The mean balanced accuracy for predicting treatment outcome was 77% (95% CI: [72%- 83%]). rs-FC of the dorsolateral prefrontal cortex had high predictive value in most studies. However, a high risk of bias was identified in all studies, compromising interpretability. Methodological recommendations are provided based on a comprehensive exploration of the studies' machine learning pipelines, and potential fruitful developments are discussed.
Collapse
Affiliation(s)
- Charlotte Meinke
- Department of Psychology, Humboldt-Universität zu Berlin, Germany.
| | - Ulrike Lueken
- Department of Psychology, Humboldt-Universität zu Berlin, Germany; German Center for Mental Health (DZPG), partner site Berlin/Potsdam, Germany.
| | - Henrik Walter
- Charité Universtätsmedizin Berlin, corporate member of FU Berlin and Humboldt Universität zu Berlin, Department of Psychiatrie and Psychotherapy, CCM, Germany.
| | - Kevin Hilbert
- Department of Psychology, Humboldt-Universität zu Berlin, Germany; Department of Psychology, Health and Medical University Erfurt, Germany.
| |
Collapse
|
21
|
Byrne JF, Healy C, Föcking M, Susai SR, Mongan D, Wynne K, Kodosaki E, Heurich M, de Haan L, Hickie IB, Smesny S, Thompson A, Markulev C, Young AR, Schäfer MR, Riecher-Rössler A, Mossaheb N, Berger G, Schlögelhofer M, Nordentoft M, Chen EYH, Verma S, Nieman DH, Woods SW, Cornblatt BA, Stone WS, Mathalon DH, Bearden CE, Cadenhead KS, Addington J, Walker EF, Cannon TD, Cannon M, McGorry P, Amminger P, Cagney G, Nelson B, Jeffries C, Perkins D, Cotter DR. Proteomic Biomarkers for the Prediction of Transition to Psychosis in Individuals at Clinical High Risk: A Multi-cohort Model Development Study. Schizophr Bull 2024; 50:579-588. [PMID: 38243809 PMCID: PMC11059811 DOI: 10.1093/schbul/sbad184] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/22/2024]
Abstract
Psychosis risk prediction is one of the leading challenges in psychiatry. Previous investigations have suggested that plasma proteomic data may be useful in accurately predicting transition to psychosis in individuals at clinical high risk (CHR). We hypothesized that an a priori-specified proteomic prediction model would have strong predictive accuracy for psychosis risk and aimed to replicate longitudinal associations between plasma proteins and transition to psychosis. This study used plasma samples from participants in 3 CHR cohorts: the North American Prodrome Longitudinal Studies 2 and 3, and the NEURAPRO randomized control trial (total n = 754). Plasma proteomic data were quantified using mass spectrometry. The primary outcome was transition to psychosis over the study follow-up period. Logistic regression models were internally validated, and optimism-corrected performance metrics derived with a bootstrap procedure. In the overall sample of CHR participants (age: 18.5, SD: 3.9; 51.9% male), 20.4% (n = 154) developed psychosis within 4.4 years. The a priori-specified model showed poor risk-prediction accuracy for the development of psychosis (C-statistic: 0.51 [95% CI: 0.50, 0.59], calibration slope: 0.45). At a group level, Complement C8B, C4B, C5, and leucine-rich α-2 glycoprotein 1 (LRG1) were associated with transition to psychosis but did not surpass correction for multiple comparisons. This study did not confirm the findings from a previous proteomic prediction model of transition from CHR to psychosis. Certain complement proteins may be weakly associated with transition at a group level. Previous findings, derived from small samples, should be interpreted with caution.
Collapse
Affiliation(s)
- Jonah F Byrne
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Colm Healy
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- Department of Psychology, Royal College of Surgeons in Ireland, Dublin 2, Ireland
| | - Melanie Föcking
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Subash Raj Susai
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - David Mongan
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- Centre for Public Health, Queen’s University Belfast, Belfast, UK
| | - Kieran Wynne
- School of Biomolecular and Biomedical Science, Conway Institute, University College Dublin, Dublin, Ireland
| | - Eleftheria Kodosaki
- School of Pharmacy and Pharmaceutical Sciences, Cardiff University, Wales, UK
| | - Meike Heurich
- School of Pharmacy and Pharmaceutical Sciences, Cardiff University, Wales, UK
| | - Lieuwe de Haan
- Department of Psychiatry, Academic Medical Center, Amsterdam, The Netherlands
| | - Ian B Hickie
- Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia
| | - Stefan Smesny
- Department of Psychiatry, Jena University Hospital, Jena, Germany
| | - Andrew Thompson
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, VIC, Australia
| | - Connie Markulev
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, VIC, Australia
| | - Alison Ruth Young
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Institute for Mental and Physical Health and Clinical Translation (IMPACT), Deakin University, Geelong, VIC, Australia
- School of Health Sciences, University of Manchester, Manchester, UK
| | - Miriam R Schäfer
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, VIC, Australia
| | | | - Nilufar Mossaheb
- Department of Psychiatry and Psychotherapy, Clinical Division of Social Psychiatry, Medical University of Vienna, Vienna, Austria
| | - Gregor Berger
- Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric Hospital, University of Zurich, Zurich, Switzerland
| | - Monika Schlögelhofer
- BioPsyC—Biopsychosocial Corporation, Non-profit Association for Research Funding Ltd, Vienna, Austria
| | - Merete Nordentoft
- Mental Health Center Copenhagen, Research Unit (CORE), Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Eric Y H Chen
- Department of Psychiatry, School of Clinical Medicine, LKS Faculty of Medicine, The University of Hong Kong, 2/F New Clinical Building, Queen Mary Hospital, Pok Fu Lam, Hong Kong
- The State Key Laboratory of Brain and Cognitive Sciences, The University of Hong Kong, Pok Fu Lam, Hong Kong
| | - Swapna Verma
- Office of Education, Duke-NUS Graduate Medical School, Singapore, Singapore
- Department of Psychosis & East Region, Institute of Mental Health, Singapore, Singapore
| | - Dorien H Nieman
- Department of Psychiatry, Academic Medical Center, Amsterdam, The Netherlands
| | - Scott W Woods
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA
| | | | - William S Stone
- Department of Psychiatry, Harvard Medical School at Beth Israel Deaconess Medical Center and Massachusetts General Hospital, Boston, MA, USA
| | - Daniel H Mathalon
- Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, San Francisco, CA, USA
- Mental Health Service 116d, Veterans Affairs San Francisco Health Care System, San Francisco, CA, USA
| | - Carrie E Bearden
- Semel Institute for Neuroscience and Human Behavior, Departments of Psychiatry and Biobehavioral Sciences and Psychology, University of California, Los Angeles, CA, USA
| | | | - Jean Addington
- Department of Psychiatry, Hotchkiss Brain Institute, University of Calgary, Calgary, Canada
| | - Elaine F Walker
- Department of Psychology, Emory University, Atlanta, GA, USA
- Department of Psychiatry, Emory University, Atlanta, GA, USA
| | - Tyrone D Cannon
- Department of Psychology, Yale University, New Haven, CT, USA
- Department of Psychiatry, Yale University, New Haven, CT, USA
| | - Mary Cannon
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- Department of Psychiatry, Beaumont Hospital, Dublin 9, Ireland
| | - Pat McGorry
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, VIC, Australia
| | - Paul Amminger
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, VIC, Australia
| | - Gerard Cagney
- School of Biomolecular and Biomedical Science, Conway Institute, University College Dublin, Dublin, Ireland
| | - Barnaby Nelson
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia
- Orygen, The National Centre of Excellence in Youth Mental Health, Melbourne, VIC, Australia
| | - Clark Jeffries
- Renaissance Computing Institute, University of North Carolina, Chapel Hill, NC, USA
| | - Diana Perkins
- Department of Psychiatry, University of North Carolina, Chapel Hill, NC, USA
| | - David R Cotter
- Department of Psychiatry, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
- Department of Psychiatry, Beaumont Hospital, Dublin 9, Ireland
| |
Collapse
|
22
|
López Steinmetz LC, Sison M, Zhumagambetov R, Godoy JC, Haufe S. Machine learning models predict the emergence of depression in Argentinean college students during periods of COVID-19 quarantine. Front Psychiatry 2024; 15:1376784. [PMID: 38690202 PMCID: PMC11059062 DOI: 10.3389/fpsyt.2024.1376784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2024] [Accepted: 03/29/2024] [Indexed: 05/02/2024] Open
Abstract
Introduction The COVID-19 pandemic has exacerbated mental health challenges, particularly depression among college students. Detecting at-risk students early is crucial but remains challenging, particularly in developing countries. Utilizing data-driven predictive models presents a viable solution to address this pressing need. Aims 1) To develop and compare machine learning (ML) models for predicting depression in Argentinean students during the pandemic. 2) To assess the performance of classification and regression models using appropriate metrics. 3) To identify key features driving depression prediction. Methods A longitudinal dataset (N = 1492 college students) captured T1 and T2 measurements during the Argentinean COVID-19 quarantine. ML models, including linear logistic regression classifiers/ridge regression (LogReg/RR), random forest classifiers/regressors, and support vector machines/regressors (SVM/SVR), are employed. Assessed features encompass depression and anxiety scores (at T1), mental disorder/suicidal behavior history, quarantine sub-period information, sex, and age. For classification, models' performance on test data is evaluated using Area Under the Precision-Recall Curve (AUPRC), Area Under the Receiver Operating Characteristic curve, Balanced Accuracy, F1 score, and Brier loss. For regression, R-squared (R2), Mean Absolute Error, and Mean Squared Error are assessed. Univariate analyses are conducted to assess the predictive strength of each individual feature with respect to the target variable. The performance of multi- vs univariate models is compared using the mean AUPRC score for classifiers and the R2 score for regressors. Results The highest performance is achieved by SVM and LogReg (e.g., AUPRC: 0.76, 95% CI: 0.69, 0.81) and SVR and RR models (e.g., R2 for SVR and RR: 0.56, 95% CI: 0.45, 0.64 and 0.45, 0.63, respectively). Univariate models, particularly LogReg and SVM using depression (AUPRC: 0.72, 95% CI: 0.64, 0.79) or anxiety scores (AUPRC: 0.71, 95% CI: 0.64, 0.78) and RR using depression scores (R2: 0.48, 95% CI: 0.39, 0.57) exhibit performance levels close to those of the multivariate models, which include all features. Discussion These findings highlight the relevance of pre-existing depression and anxiety conditions in predicting depression during quarantine, underscoring their comorbidity. ML models, particularly SVM/SVR and LogReg/RR, demonstrate potential in the timely detection of at-risk students. However, further studies are needed before clinical implementation.
Collapse
Affiliation(s)
- Lorena Cecilia López Steinmetz
- Inverse Modeling and Machine Learning, Chair of Uncertainty, Institute of Software Engineering and Theoretical Computer Science, Faculty IV Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
- Instituto de Investigaciones Psicológicas (IIPsi), Facultad de Psicología, Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Universidad Nacional de Córdoba (UNC), Córdoba, Argentina
| | - Margarita Sison
- Berlin Center for Advanced Neuroimaging (BCAN), Charité – Universitätsmedizin Berlin, Berlin, Germany
| | - Rustam Zhumagambetov
- Working Group 8.44 Machine Learning and Uncertainty, Mathematical Modelling and Data Analysis Department, Physikalisch-Technische Bundesanstalt Braunschweig und Berlin, Berlin, Germany
| | - Juan Carlos Godoy
- Instituto de Investigaciones Psicológicas (IIPsi), Facultad de Psicología, Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Universidad Nacional de Córdoba (UNC), Córdoba, Argentina
| | - Stefan Haufe
- Inverse Modeling and Machine Learning, Chair of Uncertainty, Institute of Software Engineering and Theoretical Computer Science, Faculty IV Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
- Berlin Center for Advanced Neuroimaging (BCAN), Charité – Universitätsmedizin Berlin, Berlin, Germany
- Working Group 8.44 Machine Learning and Uncertainty, Mathematical Modelling and Data Analysis Department, Physikalisch-Technische Bundesanstalt Braunschweig und Berlin, Berlin, Germany
- Institute for Medical Informatics, Charité – Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
23
|
Winter NR, Blanke J, Leenings R, Ernsting J, Fisch L, Sarink K, Barkhau C, Emden D, Thiel K, Flinkenflügel K, Winter A, Goltermann J, Meinert S, Dohm K, Repple J, Gruber M, Leehr EJ, Opel N, Grotegerd D, Redlich R, Nitsch R, Bauer J, Heindel W, Gross J, Risse B, Andlauer TFM, Forstner AJ, Nöthen MM, Rietschel M, Hofmann SG, Pfarr JK, Teutenberg L, Usemann P, Thomas-Odenthal F, Wroblewski A, Brosch K, Stein F, Jansen A, Jamalabadi H, Alexander N, Straube B, Nenadić I, Kircher T, Dannlowski U, Hahn T. A Systematic Evaluation of Machine Learning-Based Biomarkers for Major Depressive Disorder. JAMA Psychiatry 2024; 81:386-395. [PMID: 38198165 PMCID: PMC10782379 DOI: 10.1001/jamapsychiatry.2023.5083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 11/05/2023] [Indexed: 01/11/2024]
Abstract
Importance Biological psychiatry aims to understand mental disorders in terms of altered neurobiological pathways. However, for one of the most prevalent and disabling mental disorders, major depressive disorder (MDD), no informative biomarkers have been identified. Objective To evaluate whether machine learning (ML) can identify a multivariate biomarker for MDD. Design, Setting, and Participants This study used data from the Marburg-Münster Affective Disorders Cohort Study, a case-control clinical neuroimaging study. Patients with acute or lifetime MDD and healthy controls aged 18 to 65 years were recruited from primary care and the general population in Münster and Marburg, Germany, from September 11, 2014, to September 26, 2018. The Münster Neuroimaging Cohort (MNC) was used as an independent partial replication sample. Data were analyzed from April 2022 to June 2023. Exposure Patients with MDD and healthy controls. Main Outcome and Measure Diagnostic classification accuracy was quantified on an individual level using an extensive ML-based multivariate approach across a comprehensive range of neuroimaging modalities, including structural and functional magnetic resonance imaging and diffusion tensor imaging as well as a polygenic risk score for depression. Results Of 1801 included participants, 1162 (64.5%) were female, and the mean (SD) age was 36.1 (13.1) years. There were a total of 856 patients with MDD (47.5%) and 945 healthy controls (52.5%). The MNC replication sample included 1198 individuals (362 with MDD [30.1%] and 836 healthy controls [69.9%]). Training and testing a total of 4 million ML models, mean (SD) accuracies for diagnostic classification ranged between 48.1% (3.6%) and 62.0% (4.8%). Integrating neuroimaging modalities and stratifying individuals based on age, sex, treatment, or remission status does not enhance model performance. Findings were replicated within study sites and also observed in structural magnetic resonance imaging within MNC. Under simulated conditions of perfect reliability, performance did not significantly improve. Analyzing model errors suggests that symptom severity could be a potential focus for identifying MDD subgroups. Conclusion and Relevance Despite the improved predictive capability of multivariate compared with univariate neuroimaging markers, no informative individual-level MDD biomarker-even under extensive ML optimization in a large sample of diagnosed patients-could be identified.
Collapse
Affiliation(s)
- Nils R. Winter
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| | - Julian Blanke
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Ramona Leenings
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Faculty of Mathematics and Computer Science, University of Münster, Münster, Germany
| | - Jan Ernsting
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Faculty of Mathematics and Computer Science, University of Münster, Münster, Germany
- Institute for Geoinformatics, University of Münster, Münster, Germany
| | - Lukas Fisch
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Kelvin Sarink
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Carlotta Barkhau
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Daniel Emden
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Katharina Thiel
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Kira Flinkenflügel
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Alexandra Winter
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Janik Goltermann
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Susanne Meinert
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Institute for Translational Neuroscience, University of Münster, Münster, Germany
| | - Katharina Dohm
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Jonathan Repple
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Department of Psychiatry, Psychosomatic Medicine and Psychotherapy, University Hospital Frankfurt, Goethe University, Frankfurt am Main, Germany
| | - Marius Gruber
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Department of Psychiatry, Psychosomatic Medicine and Psychotherapy, University Hospital Frankfurt, Goethe University, Frankfurt am Main, Germany
| | - Elisabeth J. Leehr
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Nils Opel
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Department of Psychiatry and Psychotherapy, University Hospital Jena, Jena, Germany
- Center for Intervention and Research on Adaptive and Maladaptive Brain Circuits Underlying Mental Health, Jena, Germany
- German Center for Mental Health (DZPG), Jena, Germany
| | - Dominik Grotegerd
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
| | - Ronny Redlich
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Center for Intervention and Research on Adaptive and Maladaptive Brain Circuits Underlying Mental Health, Jena, Germany
- Department of Psychology, University of Halle, Halle, Germany
- German Center for Mental Health (DZPG), Halle, Germany
| | - Robert Nitsch
- Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
- Institute for Translational Neuroscience, University of Münster, Münster, Germany
| | - Jochen Bauer
- Clinic for Radiology, University of Münster, University Hospital Münster, Münster, Germany
| | - Walter Heindel
- Clinic for Radiology, University of Münster, University Hospital Münster, Münster, Germany
| | - Joachim Gross
- Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
- Institute for Biomagnetism and Biosignalanalysis, University of Münster, Münster, Germany
| | - Benjamin Risse
- Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
- Faculty of Mathematics and Computer Science, University of Münster, Münster, Germany
- Institute for Geoinformatics, University of Münster, Münster, Germany
| | - Till F. M. Andlauer
- Department of Neurology, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany
| | - Andreas J. Forstner
- Institute of Human Genetics, University of Bonn, School of Medicine and University Hospital Bonn, Bonn, Germany
- Institute of Neuroscience and Medicine (INM-1), Research Centre Jülich, Jülich, Germany
| | - Markus M. Nöthen
- Institute of Human Genetics, University of Bonn, School of Medicine and University Hospital Bonn, Bonn, Germany
| | - Marcella Rietschel
- Department of Genetic Epidemiology, Central Institute of Mental Health, Faculty of Medicine Mannheim, University of Heidelberg, Mannheim, Germany
| | - Stefan G. Hofmann
- Department of Clinical Psychology, Philipps-University Marburg, Marburg, Germany
| | - Julia-Katharina Pfarr
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Lea Teutenberg
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Paula Usemann
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Florian Thomas-Odenthal
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Adrian Wroblewski
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Katharina Brosch
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Frederike Stein
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Andreas Jansen
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
- Core Facility Brain Imaging, Faculty of Medicine, Philipps-University Marburg, Marburg, Germany
| | - Hamidreza Jamalabadi
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
| | - Nina Alexander
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Benjamin Straube
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Igor Nenadić
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Tilo Kircher
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
- Center for Mind, Brain and Behavior (CMBB), Marburg, Germany
| | - Udo Dannlowski
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| | - Tim Hahn
- Institute for Translational Psychiatry, University of Münster, Münster, Germany
- Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| |
Collapse
|
24
|
Fiedorowicz JG, Merranko JA, Goldstein TR, Hower H, Iyengar S, Hafeman DM, Hunt JI, Strober M, Keller MB, Goldstein BI, Diler RS, Siddiqi S, Birmaher B. Validation of a youth suicide risk calculator in an adult sample with bipolar disorder. J Affect Disord 2024; 347:278-284. [PMID: 38007103 PMCID: PMC11022308 DOI: 10.1016/j.jad.2023.11.066] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Revised: 11/08/2023] [Accepted: 11/18/2023] [Indexed: 11/27/2023]
Abstract
BACKGROUND Bipolar disorder (BD) conveys the highest risk of suicide of all mental disorders. We sought to externally validate a risk calculator (RC) of suicide attempts developed in youth with BD from the Course and Outcome of Bipolar Youth (COBY) study in an adult sample. METHODS A prospective cohort of adults with BD from the National Institute of Mental Health Collaborative Depression Study (CDS; N = 427; mean (+/- SD) age at intake (36 +/- 13 years)) was secondarily analyzed to validate the COBY RC for one-year risk of suicide attempts/deaths. Nine of the ten predictor variables from the COBY RC were available in the CDS and used: age, age of mood disorder onset, first and second (partial) degree family history of suicide, history of psychotic symptoms, substance use disorder, prior suicide attempt, socioeconomic status, and non-suicidal self-injury (prospectively, incompletely at baseline). RESULTS Over a mean (SD) follow-up of 19 (10) years, 29 % of the CDS sample attempted suicide. The RC predicted suicide attempts/deaths over one-year follow-up with an area under the receiver operating characteristic curve (AUC) of 0.78 (95 % CI 0.75-0.80). The RC performed slightly better in those with a younger age of mood disorder onset. LIMITATIONS Clinical samples may limit generalizability; the RC does not assess more acute suicide risk. CONCLUSIONS One-year risk of suicide attempts/deaths can be predicted with acceptable accuracy in youth and adults with BD, comparable to commonly used RCs to predict cardiovascular risk. This RC may help identify higher-risk individuals with BD for personalized treatment and research. https://cobysuicideattemptsrc.shinyapps.io/Shiny.
Collapse
Affiliation(s)
- Jess G Fiedorowicz
- Departments of Psychiatry and Epidemiology, The University of Ottawa, 75 Laurier Ave. East, Ottawa, ON K1N 6N5, Canada.
| | - John A Merranko
- Department of Psychiatry, Western Psychiatric Hospital, University of Pittsburgh School of Medicine, 3811 O'Hara St., Pittsburgh, PA 15213, USA
| | - Tina R Goldstein
- Department of Psychiatry, Western Psychiatric Hospital, University of Pittsburgh School of Medicine, 3811 O'Hara St., Pittsburgh, PA 15213, USA
| | - Heather Hower
- Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Box G-BH, Providence, RI 02912, USA; Department of Health Services, Policy, and Practice, Brown University School of Public Health, 121 South Main Street, Providence, RI 02903, USA; Department of Psychiatry, University of California San Diego, 4510 Executive Drive, Suite 315, San Diego, CA 92121, USA
| | - Satish Iyengar
- Department of Statistics, University of Pittsburgh, 230 S. Bouquet St., Pittsburgh, PA 15213, USA
| | - Danella M Hafeman
- Department of Psychiatry, Western Psychiatric Hospital, University of Pittsburgh School of Medicine, 3811 O'Hara St., Pittsburgh, PA 15213, USA
| | - Jeffrey I Hunt
- Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Box G-BH, Providence, RI 02912, USA; Department of Psychiatry, Bradley Hospital, 1011 Veterans Memorial Parkway, East Providence, RI 02915, USA
| | - Michael Strober
- Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Martin B Keller
- Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Box G-BH, Providence, RI 02912, USA; Department of Psychiatry, University of Miami, 1120 NW 14(th) St., Miami, FL 33136, USA
| | - Benjamin I Goldstein
- Department of Psychiatry, CAMH, University of Toronto Faculty of Medicine, 2075 Bayview Ave., FG-53, Toronto, ON M4N-3M5, Canada
| | - Rasim S Diler
- Department of Psychiatry, Western Psychiatric Hospital, University of Pittsburgh School of Medicine, 3811 O'Hara St., Pittsburgh, PA 15213, USA
| | - Sara Siddiqi
- Departments of Psychiatry and Epidemiology, The University of Ottawa, 75 Laurier Ave. East, Ottawa, ON K1N 6N5, Canada
| | - Boris Birmaher
- Department of Psychiatry, Western Psychiatric Hospital, University of Pittsburgh School of Medicine, 3811 O'Hara St., Pittsburgh, PA 15213, USA
| |
Collapse
|
25
|
Zhou B, Wang A, Cao H. Risk prediction models for postoperative delirium in elderly patients with fragility hip fracture: A systematic review and critical appraisal. Int J Orthop Trauma Nurs 2024; 52:101077. [PMID: 38096619 DOI: 10.1016/j.ijotn.2023.101077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 12/03/2023] [Accepted: 12/06/2023] [Indexed: 03/06/2024]
Abstract
BACKGROUND Elderly patients with fragility hip fracture continue to experience a high rate of postoperative delirium (POD), which has a significant negative impact on prognosis and imposes a huge economic burden. A number of risk prediction models have been developed to detect POD early. However, the risk of bias and clinical applicability of the models remain unclear. The aim of this study was to systematically evaluate risk prediction models for POD. METHODS CNKI, WanFang DATA, Vip Database, SinoMed, PubMed, Web of Science, Embase, and the Cochrane Library were searched for studies published by July 2023. The literature was screened independently by two investigators. The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) and the Prediction Model Risk of Bias Assessment Tool (PROBAST) were respectively used for data extraction, risk of bias, and applicability assessment. RESULTS A total of 16 studies on the construction of POD risk prediction models were included. The area under the ROC curve of the models ranges from 0.670 to 0.957. The most common predictors of POD included age, history of dementia, history of delirium, ASA classification, preoperative waiting time, and preoperative albumin level. All models had a high risk of bias, mainly due to inadequate sample size, inappropriate handling of missing data, a lack of model performance evaluation, and overfitting of the models. CONCLUSIONS Overall, risk prediction models for POD in fragility hip fracture patients are still in the development stage. The majority of POD prediction models have substantial bias risks, attributable primarily to poor reporting of analysis and evaluation of model performance. In future research, it is recommended to conduct validation of the models or develop localized prediction models with demonstrated high performance, with the aim of benefiting POD screening. SYSTEMATIC REVIEW REGISTRATION PROSPERO CRD42023449153.
Collapse
Affiliation(s)
- Bingqian Zhou
- Tianjin University of Traditional Chinese Medicine, 301610, Tianjin, China
| | - Ai Wang
- Tianjin University of Traditional Chinese Medicine, 301610, Tianjin, China
| | - Hong Cao
- Tianjin Hospital Trauma Upper Extremity Ward I, 300211, Tianjin, China.
| |
Collapse
|
26
|
Zugman A, Ringlein GV, Finn ES, Lewis KM, Berman E, Silverman WK, Lebowitz ER, Pine DS, Winkler AM. Brain Functional Connectivity and Anatomical Features as Predictors of Cognitive Behavioral Therapy Outcome for Anxiety in Youths. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.01.29.24301959. [PMID: 38352528 PMCID: PMC10862993 DOI: 10.1101/2024.01.29.24301959] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/25/2024]
Abstract
Background Because pediatric anxiety disorders precede the onset of many other problems, successful prediction of response to the first-line treatment, cognitive-behavioral therapy (CBT), could have major impact. However, existing clinical models are weakly predictive. The current study evaluates whether structural and resting-state functional magnetic resonance imaging can predict post-CBT anxiety symptoms. Methods Two datasets were studied: (A) one consisted of n=54 subjects with an anxiety diagnosis, who received 12 weeks of CBT, and (B) one consisted of n=15 subjects treated for 8 weeks. Connectome Predictive Modeling (CPM) was used to predict treatment response, as assessed with the PARS; additionally we investigated models using anatomical features, instead of functional connectivity. The main analysis included network edges positively correlated with treatment outcome, and age, sex, and baseline anxiety severity as predictors. Results from alternative models and analyses also are presented. Model assessments utilized 1000 bootstraps, resulting in a 95% CI for R2, r and mean absolute error (MAE). Outcomes The main model showed a mean absolute error of approximately 3.5 (95%CI: [3.1-3.8]) points a R2 of 0.08 [-0.14 - 0.26] and r of 0.38 [0.24 - 0.511]. When testing this model in the left-out sample (B) the results were similar, with a MAE of 3.4 [2.8 - 4.7], R2-0.65 [-2.29 - 0.16] and r of 0.4 [0.24 - 0.54]. The anatomical metrics showed a similar pattern, where models rendered overall low R2. Interpretation The analysis showed that models based on earlier promising results failed to predict clinical outcomes. Despite the small sample size, the current study does not support extensive use of CPM to predict outcome in pediatric anxiety.
Collapse
Affiliation(s)
- Andre Zugman
- Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD, 20892, USA
| | - Grace V. Ringlein
- Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD, 20892, USA
| | - Emily S. Finn
- Psychological and Brain Sciences, Dartmouth College, 3 Maynard St, Hanover, NH, 03755, USA
| | - Krystal M. Lewis
- Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD, 20892, USA
| | - Erin Berman
- Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD, 20892, USA
| | - Wendy K. Silverman
- Child Study Center, Yale University, 230 South Frontage Rd., New Haven, CT 06520, USA
| | - Eli R. Lebowitz
- Child Study Center, Yale University, 230 South Frontage Rd., New Haven, CT 06520, USA
| | - Daniel S. Pine
- Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health, 9000 Rockville Pike, Bethesda, MD, 20892, USA
| | - Anderson M. Winkler
- Division of Human Genetics, School of Medicine, University of Texas Rio Grande Valley, 1 West University Blvd, Brownsville, TX 78520, USA
| |
Collapse
|
27
|
Naddaf M. Medical AI falters when assessing patients it hasn't seen. Nature 2024:10.1038/d41586-024-00094-9. [PMID: 38212616 DOI: 10.1038/d41586-024-00094-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2024]
|
28
|
Deisenhofer AK, Barkham M, Beierl ET, Schwartz B, Aafjes-van Doorn K, Beevers CG, Berwian IM, Blackwell SE, Bockting CL, Brakemeier EL, Brown G, Buckman JEJ, Castonguay LG, Cusack CE, Dalgleish T, de Jong K, Delgadillo J, DeRubeis RJ, Driessen E, Ehrenreich-May J, Fisher AJ, Fried EI, Fritz J, Furukawa TA, Gillan CM, Gómez Penedo JM, Hitchcock PF, Hofmann SG, Hollon SD, Jacobson NC, Karlin DR, Lee CT, Levinson CA, Lorenzo-Luaces L, McDanal R, Moggia D, Ng MY, Norris LA, Patel V, Piccirillo ML, Pilling S, Rubel JA, Salazar-de-Pablo G, Schleider JL, Schnurr PP, Schueller SM, Siegle GJ, Uher R, Watkins E, Webb CA, Wiltsey Stirman S, Wynants L, Youn SJ, Zilcha-Mano S, Lutz W, Cohen ZD. Implementing precision methods in personalizing psychological therapies: Barriers and possible ways forward. Behav Res Ther 2024; 172:104443. [PMID: 38086157 DOI: 10.1016/j.brat.2023.104443] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 11/21/2023] [Accepted: 11/27/2023] [Indexed: 12/26/2023]
Affiliation(s)
| | | | | | | | | | | | | | | | - Claudi L Bockting
- AmsterdamUMC, Department of Psychiatry, Research Program Amsterdam Public Health and Centre for Urban Mental Health, University of Amsterdam, the Netherlands
| | | | | | | | | | | | | | - Kim de Jong
- Leiden University, Institute of Psychology, USA
| | | | | | | | | | | | | | - Jessica Fritz
- University of Cambridge, UK; Philipps University of Marburg, Germany
| | | | - Claire M Gillan
- School of Psychology, Trinity College Institute for Neuroscience, And Global Brain Health Institute, Trinity College Dublin, USA
| | | | | | | | | | | | | | | | | | | | | | | | - Mei Yi Ng
- Florida International University, USA
| | | | | | | | | | | | | | - Jessica L Schleider
- Stony Brook University and Feinberg School of Medicine Northwestern University, USA
| | - Paula P Schnurr
- National Center for PTSD and Geisel School of Medicine at Dartmouth, USA
| | | | | | | | | | | | | | | | - Soo Jeong Youn
- Reliant Medical Group, OptumCare and Harvard Medical School, USA
| | | | | | - Zachary D Cohen
- University of California, Los Angeles and University of Arizona, USA.
| |
Collapse
|
29
|
Cullen AE, Labad J, Oliver D, Al-Diwani A, Minichino A, Fusar-Poli P. The Translational Future of Stress Neurobiology and Psychosis Vulnerability: A Review of the Evidence. Curr Neuropharmacol 2024; 22:350-377. [PMID: 36946486 PMCID: PMC10845079 DOI: 10.2174/1570159x21666230322145049] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2022] [Revised: 12/17/2022] [Accepted: 12/27/2022] [Indexed: 03/23/2023] Open
Abstract
Psychosocial stress is a well-established risk factor for psychosis, yet the neurobiological mechanisms underlying this relationship have yet to be fully elucidated. Much of the research in this field has investigated hypothalamic-pituitary-adrenal (HPA) axis function and immuno-inflammatory processes among individuals with established psychotic disorders. However, as such studies are limited in their ability to provide knowledge that can be used to develop preventative interventions, it is important to shift the focus to individuals with increased vulnerability for psychosis (i.e., high-risk groups). In the present article, we provide an overview of the current methods for identifying individuals at high-risk for psychosis and review the psychosocial stressors that have been most consistently associated with psychosis risk. We then describe a network of interacting physiological systems that are hypothesised to mediate the relationship between psychosocial stress and the manifestation of psychotic illness and critically review evidence that abnormalities within these systems characterise highrisk populations. We found that studies of high-risk groups have yielded highly variable findings, likely due to (i) the heterogeneity both within and across high-risk samples, (ii) the diversity of psychosocial stressors implicated in psychosis, and (iii) that most studies examine single markers of isolated neurobiological systems. We propose that to move the field forward, we require well-designed, largescale translational studies that integrate multi-domain, putative stress-related biomarkers to determine their prognostic value in high-risk samples. We advocate that such investigations are highly warranted, given that psychosocial stress is undoubtedly a relevant risk factor for psychotic disorders.
Collapse
Affiliation(s)
- Alexis E. Cullen
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, United Kingdom
- Department of Clinical Neuroscience, Division of Insurance Medicine, Karolinska Institutet, Solna, Sweden
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, United Kingdom
| | - Javier Labad
- CIBERSAM, Sabadell, Barcelona, Spain
- Department of Mental Health and Addictions, Consorci Sanitari del Maresme, Mataró, Spain
| | - Dominic Oliver
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, United Kingdom
- Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
| | - Adam Al-Diwani
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, United Kingdom
| | - Amedeo Minichino
- Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
| | - Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
- Department of Brain and Behavioural Sciences, University of Pavia, Pavia, Italy
- OASIS Service, South London and Maudsley NHS Foundation Trust, London, United Kingdom
- National Institute of Health Research Maudsley Biomedical Research Centre, South London and Maudsley NHS Foundation Trust, London, UK
| |
Collapse
|
30
|
Grzenda A, Widge AS. Electronic health records and stratified psychiatry: bridge to precision treatment? Neuropsychopharmacology 2024; 49:285-290. [PMID: 37667021 PMCID: PMC10700348 DOI: 10.1038/s41386-023-01724-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Revised: 08/24/2023] [Accepted: 08/27/2023] [Indexed: 09/06/2023]
Abstract
The use of a stratified psychiatry approach that combines electronic health records (EHR) data with machine learning (ML) is one potentially fruitful path toward rapidly improving precision treatment in clinical practice. This strategy, however, requires confronting pervasive methodological flaws as well as deficiencies in transparency and reporting in the current conduct of ML-based studies for treatment prediction. EHR data shares many of the same data quality issues as other types of data used in ML prediction, plus some unique challenges. To fully leverage EHR data's power for patient stratification, increased attention to data quality and collection of patient-reported outcome data is needed.
Collapse
Affiliation(s)
- Adrienne Grzenda
- Department of Psychiatry & Biobehavioral Sciences, David Geffen School of Medicine, University of California-Los Angeles, Los Angeles, CA, USA.
- Olive View-UCLA Medical Center, Sylmar, CA, USA.
| | - Alik S Widge
- Department of Psychiatry & Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
31
|
Nguyen TML, Jollant F, Tritschler L, Colle R, Corruble E, Gardier AM. [Ketamine and suicidal behavior: Contribution of animal models of aggression-impulsivity to understanding its mechanism of action]. ANNALES PHARMACEUTIQUES FRANÇAISES 2024; 82:3-14. [PMID: 37890717 DOI: 10.1016/j.pharma.2023.10.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Revised: 10/16/2023] [Accepted: 10/19/2023] [Indexed: 10/29/2023]
Abstract
More than two-thirds of suicides occur during a major depressive episode. Acting out prevention measures and therapeutic options to manage the suicidal crisis are limited. The impulsive-aggressive dimensions are vulnerability factors associated with suicide in patients suffering from a characterized depressive episode: this can be a dimension involved in animals. Impulsive and aggressive rodent models can help analyze, at least in part, the neurobiology of suicide and the beneficial effects of treatments. Ketamine, a glutamatergic antagonist, by rapidly improving the symptoms of depressive episodes, would help reduce suicidal thoughts in the short term. Animal models share with humans impulsive and aggressive endophenotypes modulated by the serotonergic system (5-HTB receptor, MAO-A enzyme), neuroinflammation or the hypothalamic-pituitary-adrenal axis and stress. Significant effects of ketamine on these endophenotypes remain to be demonstrated.
Collapse
Affiliation(s)
- Thi Mai Loan Nguyen
- Inserm CESP/UMR 1018, équipe MOODS, faculté de pharmacie, université Paris-Saclay, 91400 Orsay, France
| | - Fabrice Jollant
- Inserm CESP/UMR 1018, équipe MOODS, faculté de médecine, université Paris-Saclay, 94270 Le Kremin-Bicêtre, France; Service hospitalo-universitaire de psychiatrie, hôpital de Bicêtre, hôpitaux universitaires Paris-Saclay, Assistance publique-Hôpitaux de Paris (AP-HP), 94275 Le Kremlin-Bicêtre, France; Pôle de psychiatrie, CHU de Nîmes, Nîmes, France; Département de psychiatrie, Université McGill et Groupe McGill d'études sur le suicide, Montréal, Canada
| | - Laurent Tritschler
- Inserm CESP/UMR 1018, équipe MOODS, faculté de pharmacie, université Paris-Saclay, 91400 Orsay, France
| | - Romain Colle
- Inserm CESP/UMR 1018, équipe MOODS, faculté de médecine, université Paris-Saclay, 94270 Le Kremin-Bicêtre, France; Service hospitalo-universitaire de psychiatrie, hôpital de Bicêtre, hôpitaux universitaires Paris-Saclay, Assistance publique-Hôpitaux de Paris (AP-HP), 94275 Le Kremlin-Bicêtre, France
| | - Emmanuelle Corruble
- Inserm CESP/UMR 1018, équipe MOODS, faculté de médecine, université Paris-Saclay, 94270 Le Kremin-Bicêtre, France; Service hospitalo-universitaire de psychiatrie, hôpital de Bicêtre, hôpitaux universitaires Paris-Saclay, Assistance publique-Hôpitaux de Paris (AP-HP), 94275 Le Kremlin-Bicêtre, France
| | - Alain M Gardier
- Inserm CESP/UMR 1018, équipe MOODS, faculté de pharmacie, université Paris-Saclay, 91400 Orsay, France.
| |
Collapse
|
32
|
Byrne JF, Mongan D, Murphy J, Healy C, Fӧcking M, Cannon M, Cotter DR. Prognostic models predicting transition to psychotic disorder using blood-based biomarkers: a systematic review and critical appraisal. Transl Psychiatry 2023; 13:333. [PMID: 37898606 PMCID: PMC10613280 DOI: 10.1038/s41398-023-02623-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Revised: 09/15/2023] [Accepted: 10/06/2023] [Indexed: 10/30/2023] Open
Abstract
Accumulating evidence suggests individuals with psychotic disorder show abnormalities in metabolic and inflammatory processes. Recently, several studies have employed blood-based predictors in models predicting transition to psychotic disorder in risk-enriched populations. A systematic review of the performance and methodology of prognostic models using blood-based biomarkers in the prediction of psychotic disorder from risk-enriched populations is warranted. Databases (PubMed, EMBASE and PsycINFO) were searched for eligible texts from 1998 to 15/05/2023, which detailed model development or validation studies. The checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was used to guide data extraction from eligible texts and the Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias and applicability of the studies. A narrative synthesis of the included studies was performed. Seventeen eligible studies were identified: 16 eligible model development studies and one eligible model validation study. A wide range of biomarkers were assessed, including nucleic acids, proteins, metabolites, and lipids. The range of C-index (area under the curve) estimates reported for the models was 0.67-1.00. No studies assessed model calibration. According to PROBAST criteria, all studies were at high risk of bias in the analysis domain. While a wide range of potentially predictive biomarkers were identified in the included studies, most studies did not account for overfitting in model performance estimates, no studies assessed calibration, and all models were at high risk of bias according to PROBAST criteria. External validation of the models is needed to provide more accurate estimates of their performance. Future studies which follow the latest available methodological and reporting guidelines and adopt strategies to accommodate required sample sizes for model development or validation will clarify the value of including blood-based biomarkers in models predicting psychosis.
Collapse
Affiliation(s)
- Jonah F Byrne
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland.
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, Dublin, Ireland.
| | - David Mongan
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
- Centre for Public Health, Queen's University Belfast, Belfast, United Kingdom
| | - Jennifer Murphy
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Colm Healy
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Melanie Fӧcking
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Mary Cannon
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - David R Cotter
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, Dublin, Ireland
| |
Collapse
|
33
|
Rodrigues de Aguiar K, Braga Montezano B, Gabriel Feiten J, Watts D, Zimerman A, Campos Mondin T, Azevedo da Silva R, Dias de Mattos Souza L, Kapczinski F, de Azevedo Cardoso T, Jansen K, Cavalcante Passos I. Predicting functional impairment in euthymic patients with mood disorder: A 5-year follow-up. Psychiatry Res 2023; 328:115404. [PMID: 37748239 DOI: 10.1016/j.psychres.2023.115404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 07/31/2023] [Accepted: 08/05/2023] [Indexed: 09/27/2023]
Abstract
Major Depressive Disorder and Bipolar Disorder are psychiatric disorders associated with psychosocial impairment. Despite clinical improvement, functional complaints usually remain, mainly impairing occupational and cognitive performance. The aim of this study was to use machine learning techniques to predict functional impairment in patients with mood disorders. For that, analyzes were performed using a population-based cohort study. Participants diagnosed with a mood disorder at baseline and reassessed were considered (n = 282). Random forest (RF) with previous recursive feature selection and LASSO algorithms were applied to a training set with imputed data by bagged trees resulting in two main models. Following recursive feature selection, 25 variables were retained. The RF model had the best performance compared to LASSO. The most important variables in predicting functional impairment were sexual abuse, severity of depressive, anxiety, and somatic symptoms, physical neglect, emotional abuse, and physical abuse. The model demonstrated acceptable performance to predict functional impairment. However, our sample is composed of young participants and the model may not generalize to older individuals with mood disorders. More studies are needed in this direction. The presented calculator has clinical, sociodemographic, and environmental data, demonstrating that it is possible to use such information to predict functional performance.
Collapse
Affiliation(s)
- Kyara Rodrigues de Aguiar
- Laboratory of Molecular Psychiatry, Centro de Pesquisa Experimental (CPE) and Centro de Pesquisa Clínica (CPC), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil; Universidade Federal do Rio Grande do Sul, School of Medicine, Graduate Program in Psychiatry and Behavioral Sciences, Department of Psychiatry, Porto Alegre, RS, Brazil; Instituto Nacional de Ciência e Tecnologia Translacional em Medicina (INCT-TM), Porto Alegre, RS, Brazil
| | - Bruno Braga Montezano
- Laboratory of Molecular Psychiatry, Centro de Pesquisa Experimental (CPE) and Centro de Pesquisa Clínica (CPC), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil; Universidade Federal do Rio Grande do Sul, School of Medicine, Graduate Program in Psychiatry and Behavioral Sciences, Department of Psychiatry, Porto Alegre, RS, Brazil; Instituto Nacional de Ciência e Tecnologia Translacional em Medicina (INCT-TM), Porto Alegre, RS, Brazil
| | - Jacson Gabriel Feiten
- Laboratory of Molecular Psychiatry, Centro de Pesquisa Experimental (CPE) and Centro de Pesquisa Clínica (CPC), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil; Universidade Federal do Rio Grande do Sul, School of Medicine, Graduate Program in Psychiatry and Behavioral Sciences, Department of Psychiatry, Porto Alegre, RS, Brazil; Instituto Nacional de Ciência e Tecnologia Translacional em Medicina (INCT-TM), Porto Alegre, RS, Brazil
| | - Devon Watts
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Aline Zimerman
- Laboratory of Molecular Psychiatry, Centro de Pesquisa Experimental (CPE) and Centro de Pesquisa Clínica (CPC), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil; Universidade Federal do Rio Grande do Sul, School of Medicine, Graduate Program in Psychiatry and Behavioral Sciences, Department of Psychiatry, Porto Alegre, RS, Brazil; Instituto Nacional de Ciência e Tecnologia Translacional em Medicina (INCT-TM), Porto Alegre, RS, Brazil
| | - Thaíse Campos Mondin
- Programa de Pós-Graduação em Saúde e Comportamento, Universidade Católica de Pelotas (UCPel), Rua Gonçalves Chaves, 373, sala 424 C, Pelotas, RS 96015-560, Brazil; Universidade Federal de Pelotas (UFPel), Pró-Reitoria de Assuntos Estudantis, Pelotas, RS, Brazil
| | - Ricardo Azevedo da Silva
- Programa de Pós-Graduação em Saúde e Comportamento, Universidade Católica de Pelotas (UCPel), Rua Gonçalves Chaves, 373, sala 424 C, Pelotas, RS 96015-560, Brazil
| | - Luciano Dias de Mattos Souza
- Programa de Pós-Graduação em Saúde e Comportamento, Universidade Católica de Pelotas (UCPel), Rua Gonçalves Chaves, 373, sala 424 C, Pelotas, RS 96015-560, Brazil
| | - Flávio Kapczinski
- Laboratory of Molecular Psychiatry, Centro de Pesquisa Experimental (CPE) and Centro de Pesquisa Clínica (CPC), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil; Universidade Federal do Rio Grande do Sul, School of Medicine, Graduate Program in Psychiatry and Behavioral Sciences, Department of Psychiatry, Porto Alegre, RS, Brazil; Instituto Nacional de Ciência e Tecnologia Translacional em Medicina (INCT-TM), Porto Alegre, RS, Brazil; Center for Precision Psychiatry, MGH Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
| | - Taiane de Azevedo Cardoso
- Center for Precision Psychiatry, MGH Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
| | - Karen Jansen
- Programa de Pós-Graduação em Saúde e Comportamento, Universidade Católica de Pelotas (UCPel), Rua Gonçalves Chaves, 373, sala 424 C, Pelotas, RS 96015-560, Brazil
| | - Ives Cavalcante Passos
- Laboratory of Molecular Psychiatry, Centro de Pesquisa Experimental (CPE) and Centro de Pesquisa Clínica (CPC), Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil; Universidade Federal do Rio Grande do Sul, School of Medicine, Graduate Program in Psychiatry and Behavioral Sciences, Department of Psychiatry, Porto Alegre, RS, Brazil; Instituto Nacional de Ciência e Tecnologia Translacional em Medicina (INCT-TM), Porto Alegre, RS, Brazil.
| |
Collapse
|
34
|
Vulpius GM, Köhler-Forsberg K, Ozenne B, Larsen SV, Nasser A, Svarer C, Gillings N, Keller SH, Jørgensen MB, Knudsen GM, Frokjaer VG. Stress Hormone Dynamics Are Coupled to Brain Serotonin 4 Receptor Availability in Unmedicated Patients With Major Depressive Disorder: A NeuroPharm Study. Int J Neuropsychopharmacol 2023; 26:639-648. [PMID: 37542733 PMCID: PMC10519814 DOI: 10.1093/ijnp/pyad041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Accepted: 08/03/2023] [Indexed: 08/07/2023] Open
Abstract
BACKGROUND A prominent finding in major depressive disorder (MDD) is distorted stress hormone dynamics, which is regulated by serotonergic brain signaling. An interesting feature of the cerebral serotonin system is the serotonin 4 receptor (5-HT4R), which is lower in depressed relative to healthy individuals and also has been highlighted as a promising novel antidepressant target. Here, we test the novel hypothesis that brain 5-HT4R availability in untreated patients with MDD is correlated with cortisol dynamics, indexed by the cortisol awakening response (CAR). Further, we evaluate if CAR changes with antidepressant treatment, including a selective serotonin reuptake inhibitor, and if pretreatment CAR can predict treatment outcome. METHODS Sixty-six patients (76% women) with a moderate to severe depressive episode underwent positron emission tomography imaging with [11C]SB207145 for quantification of brain 5-HT4R binding using BPND as outcome. Serial home sampling of saliva in the first hour from awakening was performed to assess CAR before and after 8 weeks of antidepressant treatment. Treatment outcome was measured by change in Hamilton Depression Rating Scale 6 items. RESULTS In the unmedicated depressed state, prefrontal and anterior cingulate cortices 5-HT4R binding was positively associated with CAR. CAR remained unaltered after 8 weeks of antidepressant treatment, and pretreatment CAR did not significantly predict treatment outcome. CONCLUSIONS Our findings highlight a link between serotonergic disturbances in MDD and cortisol dynamics, which likely is involved in disease and treatment mechanisms. Further, our data support 5-HT4R agonism as a promising precision target in patients with MDD and disturbed stress hormone dynamics.
Collapse
Affiliation(s)
- Gunild M Vulpius
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Psychiatric Center Copenhagen, Denmark
| | - Kristin Köhler-Forsberg
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Psychiatric Center Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Denmark
| | - Brice Ozenne
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Department of Public Health, Section of Biostatistics, University of Copenhagen, Denmark
| | - Søren V Larsen
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Department of Clinical Medicine, University of Copenhagen, Denmark
| | - Arafat Nasser
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
| | - Claus Svarer
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
| | - Nic Gillings
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital Rigshospitalet, Denmark
| | - Sune H Keller
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital Rigshospitalet, Denmark
| | - Martin B Jørgensen
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Psychiatric Center Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Denmark
| | - Gitte M Knudsen
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Department of Clinical Medicine, University of Copenhagen, Denmark
| | - Vibe G Frokjaer
- Neurobiology Research Unit, Copenhagen University Hospital Rigshospitalet, Denmark
- Psychiatric Center Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Denmark
| |
Collapse
|
35
|
White N, Parsons R, Collins G, Barnett A. Evidence of questionable research practices in clinical prediction models. BMC Med 2023; 21:339. [PMID: 37667344 PMCID: PMC10478406 DOI: 10.1186/s12916-023-03048-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 08/24/2023] [Indexed: 09/06/2023] Open
Abstract
BACKGROUND Clinical prediction models are widely used in health and medical research. The area under the receiver operating characteristic curve (AUC) is a frequently used estimate to describe the discriminatory ability of a clinical prediction model. The AUC is often interpreted relative to thresholds, with "good" or "excellent" models defined at 0.7, 0.8 or 0.9. These thresholds may create targets that result in "hacking", where researchers are motivated to re-analyse their data until they achieve a "good" result. METHODS We extracted AUC values from PubMed abstracts to look for evidence of hacking. We used histograms of the AUC values in bins of size 0.01 and compared the observed distribution to a smooth distribution from a spline. RESULTS The distribution of 306,888 AUC values showed clear excesses above the thresholds of 0.7, 0.8 and 0.9 and shortfalls below the thresholds. CONCLUSIONS The AUCs for some models are over-inflated, which risks exposing patients to sub-optimal clinical decision-making. Greater modelling transparency is needed, including published protocols, and data and code sharing.
Collapse
Affiliation(s)
- Nicole White
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia
| | - Rex Parsons
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia
| | - Gary Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Adrian Barnett
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia.
| |
Collapse
|
36
|
Solmi M, Cortese S, Vita G, De Prisco M, Radua J, Dragioti E, Köhler-Forsberg O, Madsen NM, Rohde C, Eudave L, Aymerich C, Pedruzo B, Rodriguez V, Rosson S, Sabé M, Hojlund M, Catalan A, de Luca B, Fornaro M, Ostuzzi G, Barbui C, Salazar-de-Pablo G, Fusar-Poli P, Correll CU. An umbrella review of candidate predictors of response, remission, recovery, and relapse across mental disorders. Mol Psychiatry 2023; 28:3671-3687. [PMID: 37957292 PMCID: PMC10730397 DOI: 10.1038/s41380-023-02298-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 09/21/2023] [Accepted: 10/06/2023] [Indexed: 11/15/2023]
Abstract
We aimed to identify diagnosis-specific/transdiagnostic/transoutcome multivariable candidate predictors (MCPs) of key outcomes in mental disorders. We conducted an umbrella review (protocol link ), searching MEDLINE/Embase (19/07/2022), including systematic reviews of studies reporting on MCPs of response, remission, recovery, or relapse, in DSM/ICD-defined mental disorders. From published predictors, we filtered MCPs, validating MCP criteria. AMSTAR2/PROBAST measured quality/risk of bias of systematic reviews/individual studies. We included 117 systematic reviews, 403 studies, 299,888 individuals with mental disorders, testing 796 prediction models. Only 4.3%/1.2% of the systematic reviews/individual studies were at low risk of bias. The most frequently targeted outcome was remission (36.9%), the least frequent was recovery (2.5%). Studies mainly focused on depressive (39.4%), substance-use (17.9%), and schizophrenia-spectrum (11.9%) disorders. We identified numerous MCPs within disorders for response, remission and relapse, but none for recovery. Transdiagnostic MCPs of remission included lower disease-specific symptoms (disorders = 5), female sex/higher education (disorders = 3), and quality of life/functioning (disorders = 2). Transdiagnostic MCPs of relapse included higher disease-specific symptoms (disorders = 5), higher depressive symptoms (disorders = 3), and younger age/higher anxiety symptoms/global illness severity/ number of previous episodes/negative life events (disorders = 2). Finally, positive trans-outcome MCPs for depression included less negative life events/depressive symptoms (response, remission, less relapse), female sex (response, remission) and better functioning (response, less relapse); for schizophrenia, less positive symptoms/higher depressive symptoms (remission, less relapse); for substance use disorder, marital status/higher education (remission, less relapse). Male sex, younger age, more clinical symptoms and comorbid mental/physical symptoms/disorders were poor prognostic factors, while positive factors included social contacts and employment, absent negative life events, higher education, early access/intervention, lower disease-specific and comorbid mental and physical symptoms/conditions, across mental disorders. Current data limitations include high risk of bias of studies and extraction of single predictors from multivariable models. Identified MCPs can inform future development, validation or refinement of prediction models of key outcomes in mental disorders.
Collapse
Affiliation(s)
- Marco Solmi
- Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin Berlin, Berlin, Germany
- Department of Psychiatry, University of Ottawa, Ottawa, ON, Canada
- On Track: The Champlain First Episode Psychosis Program, Department of Mental Health, The Ottawa Hospital, Ottawa, ON, Canada
- Ottawa Hospital Research Institute (OHRI) Clinical Epidemiology Program University of Ottawa, Ottawa, ON, Canada
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Samuele Cortese
- Centre for Innovation in Mental Health, School of Psychology, Faculty of Environmental and Life Sciences, University of Southampton, Southampton, UK
- Clinical and Experimental Sciences (CNS and Psychiatry), Faculty of Medicine, University of Southampton, Southampton, UK
- Solent NHS Trust, Southampton, UK
- Hassenfeld Children's Hospital at NYU Langone, New York University Child Study Center, New York, NY, USA
- Division of Psychiatry and Applied Psychology, School of Medicine, University of Nottingham, Nottingham, UK
- DiMePRe-J-Department of Precision and Regenerative Medicine-Jonic Area, University of Bari "Aldo Moro", Bari, Italy
| | - Giovanni Vita
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine, and Movement Sciences, Section of Psychiatry, University of Verona, Verona, Italy
| | - Michele De Prisco
- Bipolar and Depressive Disorders Unit, Hospìtal Clinic de Barcelona, c. Villarroel, 170, 08036, Barcelona, Spain
- Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), c. Villarroel, 170, 08036, Barcelona, Spain
- Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Instituto de Salud Carlos III, Madrid, Spain
| | - Joaquim Radua
- Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Imaging of Mood- and Anxiety-Related Disorders (IMARD), CIBERSAM, University of Barcelona, Barcelona, Spain
| | - Elena Dragioti
- University of Ioannina, Research Laboratory Psychology of Patients, Families & Health Professionals, Department of Nursing, School of Health Sciences, Ioannina, Greece
- Linköping University, Pain and Rehabilitation Centre and Department of Health, Medicine and Caring Sciences, Linköping, Sweden
| | - Ole Köhler-Forsberg
- Psychosis Research Unit, Aarhus University Hospital Psychiatry, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Nanna M Madsen
- Psychosis Research Unit, Aarhus University Hospital Psychiatry, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Christopher Rohde
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Affective Disorders, Aarhus University Hospital - Psychiatry, Aarhus, Denmark
| | - Luis Eudave
- Faculty of Education and Psychology, University of Navarra, Pamplona, Spain
| | - Claudia Aymerich
- Biobizkaia Health Research Institute, Basurto University Hospital, OSI Bilbao-Basurto. University of the Basque Country UPV/EHU. Centro de Investigación en Red de Salud Mental. (CIBERSAM), Instituto de Salud Carlos III. Plaza de Cruces 12, 48903, Barakaldo, Bizkaia, Spain
| | - Borja Pedruzo
- Psychiatry Department, Basurto University Hospital, Bilbao, Spain
| | | | - Stella Rosson
- Mental Health Department, Local Health Unit ULSS3 Serenissima, Venice, Italy
| | - Michel Sabé
- Division of Adult Psychiatry, Department of Psychiatry, University Hospitals of Geneva, 2, Chemin du Petit-Bel-Air, CH-1226, Thonex, Switzerland
| | - Mikkel Hojlund
- Department of Psychiatry Aabenraa, Mental Health Services Region of Southern Denmark, Aabenraa, Denmark
- Clinical Pharmacology, Pharmacy, and Environmental Medicine, Department of Public Health, University of Southern Denmark, Odense, Denmark
- Child and Adolescent Mental Health Centre, Mental Health Services Capital Region of Denmark, Copenhagen, Denmark
| | - Ana Catalan
- Biobizkaia Health Research Institute, Basurto University Hospital, OSI Bilbao-Basurto. University of the Basque Country UPV/EHU. Centro de Investigación en Red de Salud Mental. (CIBERSAM), Instituto de Salud Carlos III. Plaza de Cruces 12, 48903, Barakaldo, Bizkaia, Spain
| | - Beatrice de Luca
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine, and Movement Sciences, Section of Psychiatry, University of Verona, Verona, Italy
| | - Michele Fornaro
- Department of Psychiatry, Federico II of Naples, Naples, Italy
| | - Giovanni Ostuzzi
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine, and Movement Sciences, Section of Psychiatry, University of Verona, Verona, Italy
| | - Corrado Barbui
- WHO Collaborating Centre for Research and Training in Mental Health and Service Evaluation, Department of Neuroscience, Biomedicine, and Movement Sciences, Section of Psychiatry, University of Verona, Verona, Italy
| | - Gonzalo Salazar-de-Pablo
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
- Child and Adolescent Mental Health Services, South London and Maudsley NHS Foundation Trust, London, UK
- Institute of Psychiatry and Mental Health. Department of Child and Adolescent Psychiatry, Hospital General Universitario Gregorio Marañón School of Medicine, Universidad Complutense, Instituto de Investigación Sanitaria Gregorio Marañón (IiSGM), CIBERSAM, Madrid, Spain
| | - Paolo Fusar-Poli
- Department of Psychosis Studies, King's College London, London, UK
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
- Outreach and Support in South London (OASIS) service, NHS South London and Maudsley Foundation Trust, London, UK
- Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University Munich, Munich, Germany
| | - Christoph U Correll
- Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin Berlin, Berlin, Germany.
- The Zucker Hillside Hospital, Northwell Health, New York, NY, USA.
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, New York, NY, USA.
- The Feinstein Institute for Medical Research, Center for Psychiatric Neuroscience, Manhasset, NY, USA.
| |
Collapse
|
37
|
Dhiman P, Ma J, Qi C, Bullock G, Sergeant JC, Riley RD, Collins GS. Sample size requirements are not being considered in studies developing prediction models for binary outcomes: a systematic review. BMC Med Res Methodol 2023; 23:188. [PMID: 37598153 PMCID: PMC10439652 DOI: 10.1186/s12874-023-02008-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 08/04/2023] [Indexed: 08/21/2023] Open
Abstract
BACKGROUND Having an appropriate sample size is important when developing a clinical prediction model. We aimed to review how sample size is considered in studies developing a prediction model for a binary outcome. METHODS We searched PubMed for studies published between 01/07/2020 and 30/07/2020 and reviewed the sample size calculations used to develop the prediction models. Using the available information, we calculated the minimum sample size that would be needed to estimate overall risk and minimise overfitting in each study and summarised the difference between the calculated and used sample size. RESULTS A total of 119 studies were included, of which nine studies provided sample size justification (8%). The recommended minimum sample size could be calculated for 94 studies: 73% (95% CI: 63-82%) used sample sizes lower than required to estimate overall risk and minimise overfitting including 26% studies that used sample sizes lower than required to estimate overall risk only. A similar number of studies did not meet the ≥ 10EPV criteria (75%, 95% CI: 66-84%). The median deficit of the number of events used to develop a model was 75 [IQR: 234 lower to 7 higher]) which reduced to 63 if the total available data (before any data splitting) was used [IQR:225 lower to 7 higher]. Studies that met the minimum required sample size had a median c-statistic of 0.84 (IQR:0.80 to 0.9) and studies where the minimum sample size was not met had a median c-statistic of 0.83 (IQR: 0.75 to 0.9). Studies that met the ≥ 10 EPP criteria had a median c-statistic of 0.80 (IQR: 0.73 to 0.84). CONCLUSIONS Prediction models are often developed with no sample size calculation, as a consequence many are too small to precisely estimate the overall risk. We encourage researchers to justify, perform and report sample size calculations when developing a prediction model.
Collapse
Affiliation(s)
- Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK.
| | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK
| | - Cathy Qi
- Population Data Science, Faculty of Medicine, Health and Life Science, Swansea University Medical School, Swansea University, Singleton Park, Swansea, SA2 8PP, UK
| | - Garrett Bullock
- Department of Orthopaedic Surgery, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Centre for Sport, Exercise and Osteoarthritis Research Versus Arthritis, University of Oxford, Oxford, UK
| | - Jamie C Sergeant
- Centre for Biostatistics, University of Manchester, Manchester Academic Health Science Centre, Manchester, M13 9PL, UK
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre, Manchester, M13 9PT, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, B15 2TT, Birmingham, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK
| |
Collapse
|
38
|
Qi W, Wang Y, Li C, He K, Wang Y, Huang S, Li C, Guo Q, Hu J. Predictive models for predicting the risk of maternal postpartum depression: A systematic review and evaluation. J Affect Disord 2023; 333:107-120. [PMID: 37084958 DOI: 10.1016/j.jad.2023.04.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/10/2022] [Revised: 03/21/2023] [Accepted: 04/14/2023] [Indexed: 04/23/2023]
Abstract
OBJECTIVES Clinical prediction models have been widely used to screen and diagnose postpartum depression (PPD). This study systematically reviews and evaluates the risk of bias and the applicability of PPD prediction models. METHODS A systematic search was performed in eight databases from inception to June 1, 2022. The literature was independently screened, and data were extracted by two investigators using the checklist for critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS). The risk of bias and applicability was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). RESULTS After the screening, 12 studies of PPD risk prediction models were included, with the area under the ROC curve of the models ranging from 0.611 to 0.937. The most-reported predictors of PPD included several aspects, including prenatal mood disorders, endocrine and hormonal influences, psychosocial aspects, the influence of family factors, and somatic illness factors. The applicability of all studies was good. However, there was some bias, mainly due to inadequate outcome events, missing data not appropriately handled, lack of model performance assessment, and overfitting of the models. CONCLUSIONS This systematic review and evaluation indicate that most present PPD prediction models have a high risk of bias during development and validation. Despite some models' predictive solid performance, the models' clinical practice rate is low. Therefore, future research should develop predictive models with excellent performance in all aspects and clinical applicability to better inform maternal medical decisions.
Collapse
Affiliation(s)
- Weijing Qi
- Department of Clinical Humanistic Care and Nursing Research Center, School of Nursing, Hebei Medical University, 361 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Yongjian Wang
- Department of Clinical Humanistic Care and Nursing Research Center, School of Nursing, Hebei Medical University, 361 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Caixia Li
- Shijiazhuang Obstetrics and Gynecology Hospital, 206 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Ke He
- Shijiazhuang Obstetrics and Gynecology Hospital, 206 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Yipeng Wang
- Department of Clinical Humanistic Care and Nursing Research Center, School of Nursing, Hebei Medical University, 361 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Sha Huang
- Department of Clinical Humanistic Care and Nursing Research Center, School of Nursing, Hebei Medical University, 361 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Cong Li
- Department of Clinical Humanistic Care and Nursing Research Center, School of Nursing, Hebei Medical University, 361 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China
| | - Qing Guo
- Shijiazhuang Obstetrics and Gynecology Hospital, 206 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China.
| | - Jie Hu
- Department of Clinical Humanistic Care and Nursing Research Center, School of Nursing, Hebei Medical University, 361 East Zhongshan Road, Shijiazhuang 050017, Hebei Province, China.
| |
Collapse
|
39
|
Chen Z, Hu B, Liu X, Becker B, Eickhoff SB, Miao K, Gu X, Tang Y, Dai X, Li C, Leonov A, Xiao Z, Feng Z, Chen J, Chuan-Peng H. Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry. BMC Med 2023; 21:241. [PMID: 37400814 DOI: 10.1186/s12916-023-02941-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Accepted: 06/13/2023] [Indexed: 07/05/2023] Open
Abstract
BACKGROUND The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation. METHODS Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses. RESULTS A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (β = - 2.75, p < .001, R2adj = 0.40; r = - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF10 > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance. CONCLUSIONS Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.
Collapse
Affiliation(s)
- Zhiyi Chen
- Experimental Research Center for Medical and Psychological Science (ERC-MPS), School of Psychology, Third Military Medical University, Chongqing, China.
- Faculty of Psychology, Southwest University, Chongqing, China.
| | - Bowen Hu
- Faculty of Psychology, Southwest University, Chongqing, China
| | - Xuerong Liu
- Experimental Research Center for Medical and Psychological Science (ERC-MPS), School of Psychology, Third Military Medical University, Chongqing, China
| | - Benjamin Becker
- The Center of Psychosomatic Medicine, Sichuan Provincial Center for Mental Health, Sichuan Provincial People's Hospital, Chengdu, China
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Laboratory for Neuroinformation, University of Electronic Science and Technology of China, Chengdu, China
| | - Simon B Eickhoff
- Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Kuan Miao
- Experimental Research Center for Medical and Psychological Science (ERC-MPS), School of Psychology, Third Military Medical University, Chongqing, China
| | - Xingmei Gu
- Experimental Research Center for Medical and Psychological Science (ERC-MPS), School of Psychology, Third Military Medical University, Chongqing, China
| | - Yancheng Tang
- School of Business and Management, Shanghai International Studies University, Shanghai, China
| | - Xin Dai
- Faculty of Psychology, Southwest University, Chongqing, China
| | - Chao Li
- Department of Radiology, The Third Affiliated Hospital, Sun Yat-Sen University, Guangdong, China
| | - Artemiy Leonov
- School of Psychology, Clark University, Worcester, MA, USA
| | - Zhibing Xiao
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Zhengzhi Feng
- Experimental Research Center for Medical and Psychological Science (ERC-MPS), School of Psychology, Third Military Medical University, Chongqing, China
| | - Ji Chen
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou, China.
- Department of Psychiatry, The Fourth Affiliated Hospital, Zhejiang University School of Medicine, Yiwu, Zhejiang, China.
| | - Hu Chuan-Peng
- School of Psychology, Nanjing Normal University, Nanjing, China
| |
Collapse
|
40
|
Abi-Dargham A, Moeller SJ, Ali F, DeLorenzo C, Domschke K, Horga G, Jutla A, Kotov R, Paulus MP, Rubio JM, Sanacora G, Veenstra-VanderWeele J, Krystal JH. Candidate biomarkers in psychiatric disorders: state of the field. World Psychiatry 2023; 22:236-262. [PMID: 37159365 PMCID: PMC10168176 DOI: 10.1002/wps.21078] [Citation(s) in RCA: 71] [Impact Index Per Article: 71.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/08/2023] [Indexed: 05/11/2023] Open
Abstract
The field of psychiatry is hampered by a lack of robust, reliable and valid biomarkers that can aid in objectively diagnosing patients and providing individualized treatment recommendations. Here we review and critically evaluate the evidence for the most promising biomarkers in the psychiatric neuroscience literature for autism spectrum disorder, schizophrenia, anxiety disorders and post-traumatic stress disorder, major depression and bipolar disorder, and substance use disorders. Candidate biomarkers reviewed include various neuroimaging, genetic, molecular and peripheral assays, for the purposes of determining susceptibility or presence of illness, and predicting treatment response or safety. This review highlights a critical gap in the biomarker validation process. An enormous societal investment over the past 50 years has identified numerous candidate biomarkers. However, to date, the overwhelming majority of these measures have not been proven sufficiently reliable, valid and useful to be adopted clinically. It is time to consider whether strategic investments might break this impasse, focusing on a limited number of promising candidates to advance through a process of definitive testing for a specific indication. Some promising candidates for definitive testing include the N170 signal, an event-related brain potential measured using electroencephalography, for subgroup identification within autism spectrum disorder; striatal resting-state functional magnetic resonance imaging (fMRI) measures, such as the striatal connectivity index (SCI) and the functional striatal abnormalities (FSA) index, for prediction of treatment response in schizophrenia; error-related negativity (ERN), an electrophysiological index, for prediction of first onset of generalized anxiety disorder, and resting-state and structural brain connectomic measures for prediction of treatment response in social anxiety disorder. Alternate forms of classification may be useful for conceptualizing and testing potential biomarkers. Collaborative efforts allowing the inclusion of biosystems beyond genetics and neuroimaging are needed, and online remote acquisition of selected measures in a naturalistic setting using mobile health tools may significantly advance the field. Setting specific benchmarks for well-defined target application, along with development of appropriate funding and partnership mechanisms, would also be crucial. Finally, it should never be forgotten that, for a biomarker to be actionable, it will need to be clinically predictive at the individual level and viable in clinical settings.
Collapse
Affiliation(s)
- Anissa Abi-Dargham
- Renaissance School of Medicine at Stony Brook University, Stony Brook, NY, USA
| | - Scott J Moeller
- Renaissance School of Medicine at Stony Brook University, Stony Brook, NY, USA
| | - Farzana Ali
- Renaissance School of Medicine at Stony Brook University, Stony Brook, NY, USA
| | - Christine DeLorenzo
- Renaissance School of Medicine at Stony Brook University, Stony Brook, NY, USA
| | - Katharina Domschke
- Department of Psychiatry and Psychotherapy, Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Centre for Basics in Neuromodulation, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Guillermo Horga
- Department of Psychiatry, Columbia University, New York, NY, USA
- New York State Psychiatric Institute, New York, NY, USA
| | - Amandeep Jutla
- Department of Psychiatry, Columbia University, New York, NY, USA
- New York State Psychiatric Institute, New York, NY, USA
| | - Roman Kotov
- Renaissance School of Medicine at Stony Brook University, Stony Brook, NY, USA
| | | | - Jose M Rubio
- Zucker School of Medicine at Hofstra-Northwell, Hempstead, NY, USA
- Feinstein Institute for Medical Research - Northwell, Manhasset, NY, USA
- Zucker Hillside Hospital - Northwell Health, Glen Oaks, NY, USA
| | - Gerard Sanacora
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
| | - Jeremy Veenstra-VanderWeele
- Department of Psychiatry, Columbia University, New York, NY, USA
- New York State Psychiatric Institute, New York, NY, USA
| | - John H Krystal
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
| |
Collapse
|
41
|
Donnelly N, Cunningham A, Salas SM, Bracher-Smith M, Chawner S, Stochl J, Ford T, Raymond FL, Escott-Price V, van den Bree MBM. Identifying the neurodevelopmental and psychiatric signatures of genomic disorders associated with intellectual disability: a machine learning approach. Mol Autism 2023; 14:19. [PMID: 37221545 PMCID: PMC10207854 DOI: 10.1186/s13229-023-00549-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Accepted: 04/16/2023] [Indexed: 05/25/2023] Open
Abstract
BACKGROUND Genomic conditions can be associated with developmental delay, intellectual disability, autism spectrum disorder, and physical and mental health symptoms. They are individually rare and highly variable in presentation, which limits the use of standard clinical guidelines for diagnosis and treatment. A simple screening tool to identify young people with genomic conditions associated with neurodevelopmental disorders (ND-GCs) who could benefit from further support would be of considerable value. We used machine learning approaches to address this question. METHOD A total of 493 individuals were included: 389 with a ND-GC, mean age = 9.01, 66% male) and 104 siblings without known genomic conditions (controls, mean age = 10.23, 53% male). Primary carers completed assessments of behavioural, neurodevelopmental and psychiatric symptoms and physical health and development. Machine learning techniques (penalised logistic regression, random forests, support vector machines and artificial neural networks) were used to develop classifiers of ND-GC status and identified limited sets of variables that gave the best classification performance. Exploratory graph analysis was used to understand associations within the final variable set. RESULTS All machine learning methods identified variable sets giving high classification accuracy (AUROC between 0.883 and 0.915). We identified a subset of 30 variables best discriminating between individuals with ND-GCs and controls which formed 5 dimensions: conduct, separation anxiety, situational anxiety, communication and motor development. LIMITATIONS This study used cross-sectional data from a cohort study which was imbalanced with respect to ND-GC status. Our model requires validation in independent datasets and with longitudinal follow-up data for validation before clinical application. CONCLUSIONS In this study, we developed models that identified a compact set of psychiatric and physical health measures that differentiate individuals with a ND-GC from controls and highlight higher-order structure within these measures. This work is a step towards developing a screening instrument to identify young people with ND-GCs who might benefit from further specialist assessment.
Collapse
Affiliation(s)
- Nicholas Donnelly
- Centre for Academic Mental Health, Population Health Sciences, University of Bristol, Bristol, UK
- MRC Integrative Epidemiology Unit, Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Adam Cunningham
- Division of Psychological Medicine and Clinical Neurosciences, Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK
| | - Sergio Marco Salas
- Division of Psychological Medicine and Clinical Neurosciences, Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK
| | - Matthew Bracher-Smith
- Division of Psychological Medicine and Clinical Neurosciences, Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK
| | - Samuel Chawner
- Division of Psychological Medicine and Clinical Neurosciences, Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK
| | - Jan Stochl
- Department of Psychiatry, University of Cambridge, Cambridge, UK
- Department of Kinanthropology, Charles University, Prague, Czechia
| | - Tamsin Ford
- Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - F Lucy Raymond
- Department of Medical Genetics, University of Cambridge, Cambridge, UK
| | - Valentina Escott-Price
- Division of Psychological Medicine and Clinical Neurosciences, Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK
| | - Marianne B M van den Bree
- Division of Psychological Medicine and Clinical Neurosciences, Centre for Neuropsychiatric Genetics and Genomics, Institute of Psychological Medicine and Clinical Neurosciences, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK.
| |
Collapse
|
42
|
Nguyen TML, Jollant F, Tritschler L, Colle R, Corruble E, Gardier AM. Pharmacological Mechanism of Ketamine in Suicidal Behavior Based on Animal Models of Aggressiveness and Impulsivity: A Narrative Review. Pharmaceuticals (Basel) 2023; 16:ph16040634. [PMID: 37111391 PMCID: PMC10146327 DOI: 10.3390/ph16040634] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Revised: 04/12/2023] [Accepted: 04/14/2023] [Indexed: 04/29/2023] Open
Abstract
Around 700,000 people die from suicide each year in the world. Approximately 90% of suicides have a history of mental illness, and more than two-thirds occur during a major depressive episode. Specific therapeutic options to manage the suicidal crisis are limited and measures to prevent acting out also remain limited. Drugs shown to reduce the risk of suicide (antidepressants, lithium, or clozapine) necessitate a long delay of onset. To date, no treatment is indicated for the treatment of suicidality. Ketamine, a glutamate NMDA receptor antagonist, is a fast-acting antidepressant with significant effects on suicidal ideation in the short term, while its effects on suicidal acts still need to be demonstrated. In the present article, we reviewed the literature on preclinical studies in order to identify the potential anti-suicidal pharmacological targets of ketamine. Impulsive-aggressive traits are one of the vulnerability factors common to suicide in patients with unipolar and bipolar depression. Preclinical studies in rodent models with impulsivity, aggressiveness, and anhedonia may help to analyze, at least in part, suicide neurobiology, as well as the beneficial effects of ketamine/esketamine on reducing suicidal ideations and preventing suicidal acts. The present review focuses on disruptions in the serotonergic system (5-HTB receptor, MAO-A enzyme), neuroinflammation, and/or the HPA axis in rodent models with an impulsive/aggressive phenotype, because these traits are critical risk factors for suicide in humans. Ketamine can modulate these endophenotypes of suicide in human as well as in animal models. The main pharmacological properties of ketamine are then summarized. Finally, numerous questions arose regarding the mechanisms by which ketamine may prevent an impulsive-aggressive phenotype in rodents and suicidal ideations in humans. Animal models of anxiety/depression are important tools to better understand the pathophysiology of depressed patients, and in helping develop novel and fast antidepressant drugs with anti-suicidal properties and clinical utility.
Collapse
Affiliation(s)
- Thi Mai Loan Nguyen
- Université Paris-Saclay, Faculté de Pharmacie, Inserm CESP/UMR 1018, MOODS Team, F-91400 Orsay, France
| | - Fabrice Jollant
- Université Paris-Saclay, Faculté de Médecine, Inserm CESP/UMR 1018, MOODS Team, F-94270 Le Kremin-Bicêtre, France
- Service Hospitalo-Universitaire de Psychiatrie, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpitaux Universitaires Paris-Saclay, Hôpital de Bicêtre, F-94275 Le Kremlin Bicêtre, France
- Pôle de Psychiatrie, CHU Nîmes, 30900 Nîmes, France
- Department of Psychiatry, McGill University and McGill Group for Suicide Studies, Montréal, QC H3A 0G4, Canada
| | - Laurent Tritschler
- Université Paris-Saclay, Faculté de Pharmacie, Inserm CESP/UMR 1018, MOODS Team, F-91400 Orsay, France
| | - Romain Colle
- Université Paris-Saclay, Faculté de Médecine, Inserm CESP/UMR 1018, MOODS Team, F-94270 Le Kremin-Bicêtre, France
- Service Hospitalo-Universitaire de Psychiatrie, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpitaux Universitaires Paris-Saclay, Hôpital de Bicêtre, F-94275 Le Kremlin Bicêtre, France
| | - Emmanuelle Corruble
- Université Paris-Saclay, Faculté de Médecine, Inserm CESP/UMR 1018, MOODS Team, F-94270 Le Kremin-Bicêtre, France
- Service Hospitalo-Universitaire de Psychiatrie, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpitaux Universitaires Paris-Saclay, Hôpital de Bicêtre, F-94275 Le Kremlin Bicêtre, France
| | - Alain M Gardier
- Université Paris-Saclay, Faculté de Pharmacie, Inserm CESP/UMR 1018, MOODS Team, F-91400 Orsay, France
| |
Collapse
|
43
|
Yasrebi-de Kom IAR, Dongelmans DA, de Keizer NF, Jager KJ, Schut MC, Abu-Hanna A, Klopotowska JE. Electronic health record-based prediction models for in-hospital adverse drug event diagnosis or prognosis: a systematic review. J Am Med Inform Assoc 2023; 30:978-988. [PMID: 36805926 PMCID: PMC10114128 DOI: 10.1093/jamia/ocad014] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 01/13/2023] [Accepted: 02/01/2023] [Indexed: 02/22/2023] Open
Abstract
OBJECTIVE We conducted a systematic review to characterize and critically appraise developed prediction models based on structured electronic health record (EHR) data for adverse drug event (ADE) diagnosis and prognosis in adult hospitalized patients. MATERIALS AND METHODS We searched the Embase and Medline databases (from January 1, 1999, to July 4, 2022) for articles utilizing structured EHR data to develop ADE prediction models for adult inpatients. For our systematic evidence synthesis and critical appraisal, we applied the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS). RESULTS Twenty-five articles were included. Studies often did not report crucial information such as patient characteristics or the method for handling missing data. In addition, studies frequently applied inappropriate methods, such as univariable screening for predictor selection. Furthermore, the majority of the studies utilized ADE labels that only described an adverse symptom while not assessing causality or utilizing a causal model. None of the models were externally validated. CONCLUSIONS Several challenges should be addressed before the models can be widely implemented, including the adherence to reporting standards and the adoption of best practice methods for model development and validation. In addition, we propose a reorientation of the ADE prediction modeling domain to include causality as a fundamental challenge that needs to be addressed in future studies, either through acquiring ADE labels via formal causality assessments or the usage of adverse event labels in combination with causal prediction modeling.
Collapse
Affiliation(s)
- Izak A R Yasrebi-de Kom
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| | - Dave A Dongelmans
- Amsterdam Public Health, Amsterdam, The Netherlands
- Amsterdam UMC location University of Amsterdam, Department of Intensive Care Medicine, Amsterdam, The Netherlands
| | - Nicolette F de Keizer
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| | - Kitty J Jager
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
- Amsterdam Cardiovascular Sciences, Pulmonary Hypertension & Thrombosis, Amsterdam, The Netherlands
| | - Martijn C Schut
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
- Amsterdam UMC location Vrije Universiteit Amsterdam, Department of Clinical Chemistry, Amsterdam, The Netherlands
| | - Ameen Abu-Hanna
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| | - Joanna E Klopotowska
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| |
Collapse
|
44
|
Sariaslan A, Fanshawe T, Pitkänen J, Cipriani A, Martikainen P, Fazel S. Predicting suicide risk in 137,112 people with severe mental illness in Finland: external validation of the Oxford Mental Illness and Suicide tool (OxMIS). Transl Psychiatry 2023; 13:126. [PMID: 37072392 PMCID: PMC10113231 DOI: 10.1038/s41398-023-02422-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/13/2022] [Revised: 03/27/2023] [Accepted: 03/29/2023] [Indexed: 04/20/2023] Open
Abstract
Oxford Mental Illness and Suicide tool (OxMIS) is a standardised, scalable, and transparent instrument for suicide risk assessment in people with severe mental illness (SMI) based on 17 sociodemographic, criminal history, familial, and clinical risk factors. However, alongside most prediction models in psychiatry, external validations are currently lacking. We utilised a Finnish population sample of all persons diagnosed by mental health services with SMI (schizophrenia-spectrum and bipolar disorders) between 1996 and 2017 (n = 137,112). To evaluate the performance of OxMIS, we initially calculated the predicted 12-month suicide risk for each individual by weighting risk factors by effect sizes reported in the original OxMIS prediction model and converted to a probability. This probability was then used to assess the discrimination and calibration of the OxMIS model in this external sample. Within a year of assessment, 1.1% of people with SMI (n = 1475) had died by suicide. The overall discrimination of the tool was good, with an area under the curve of 0.70 (95% confidence interval: 0.69-0.71). The model initially overestimated suicide risks in those with elevated predicted risks of >5% over 12 months (Harrell's Emax = 0.114), which applied to 1.3% (n = 1780) of the cohort. However, when we used a 5% maximum predicted suicide risk threshold as is recommended clinically, the calibration was excellent (ICI = 0.002; Emax = 0.005). Validating clinical prediction tools using routinely collected data can address research gaps in prediction psychiatry and is a necessary step to translating such models into clinical practice.
Collapse
Affiliation(s)
- Amir Sariaslan
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK.
| | - Thomas Fanshawe
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Joonas Pitkänen
- Population Research Unit, Faculty of Social Sciences, University of Helsinki, Helsinki, Finland
| | - Andrea Cipriani
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK
| | - Pekka Martikainen
- Population Research Unit, Faculty of Social Sciences, University of Helsinki, Helsinki, Finland
- Centre for Health Equity Studies (CHESS), Stockholm University and Karolinska Institutet, Stockholm, Sweden
- Max Planck Institute for Demographic Research, Rostock, Germany
| | - Seena Fazel
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK.
| |
Collapse
|
45
|
Validation of the Collaborative Outcomes study on Health and Functioning during Infection Times (COH-FIT) questionnaire for adults. J Affect Disord 2023; 326:249-261. [PMID: 36586617 PMCID: PMC9794522 DOI: 10.1016/j.jad.2022.12.022] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 12/06/2022] [Accepted: 12/07/2022] [Indexed: 12/29/2022]
Abstract
BACKGROUND The Collaborative Outcome study on Health and Functioning during Infection Times (COH-FIT; www.coh-fit.com) is an anonymous and global online survey measuring health and functioning during the COVID-19 pandemic. The aim of this study was to test concurrently the validity of COH-FIT items and the internal validity of the co-primary outcome, a composite psychopathology "P-score". METHODS The COH-FIT survey has been translated into 30 languages (two blind forward-translations, consensus, one independent English back-translation, final harmonization). To measure mental health, 1-4 items ("COH-FIT items") were extracted from validated questionnaires (e.g. Patient Health Questionnaire 9). COH-FIT items measured anxiety, depressive, post-traumatic, obsessive-compulsive, bipolar and psychotic symptoms, as well as stress, sleep and concentration. COH-FIT Items which correlated r ≥ 0.5 with validated companion questionnaires, were initially retained. A P-score factor structure was then identified from these items using exploratory factor analysis (EFA) and confirmatory factor analyses (CFA) on data split into training and validation sets. Consistency of results across languages, gender and age was assessed. RESULTS From >150,000 adult responses by May 6th, 2022, a subset of 22,456 completed both COH-FIT items and validated questionnaires. Concurrent validity was consistently demonstrated across different languages for COH-FIT items. CFA confirmed EFA results of five first-order factors (anxiety, depression, post-traumatic, psychotic, psychophysiologic symptoms) and revealed a single second-order factor P-score, with high internal reliability (ω = 0.95). Factor structure was consistent across age and sex. CONCLUSIONS COH-FIT is a valid instrument to globally measure mental health during infection times. The P-score is a valid measure of multidimensional mental health.
Collapse
|
46
|
Jensen KHR, Dam VH, Ganz M, Fisher PM, Ip CT, Sankar A, Marstrand-Joergensen MR, Ozenne B, Osler M, Penninx BWJH, Pinborg LH, Frokjaer VG, Knudsen GM, Jørgensen MB. Deep phenotyping towards precision psychiatry of first-episode depression - the Brain Drugs-Depression cohort. BMC Psychiatry 2023; 23:151. [PMID: 36894940 PMCID: PMC9999625 DOI: 10.1186/s12888-023-04618-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/12/2022] [Accepted: 02/19/2023] [Indexed: 03/11/2023] Open
Abstract
BACKGROUND Major Depressive Disorder (MDD) is a heterogenous brain disorder, with potentially multiple psychosocial and biological disease mechanisms. This is also a plausible explanation for why patients do not respond equally well to treatment with first- or second-line antidepressants, i.e., one-third to one-half of patients do not remit in response to first- or second-line treatment. To map MDD heterogeneity and markers of treatment response to enable a precision medicine approach, we will acquire several possible predictive markers across several domains, e.g., psychosocial, biochemical, and neuroimaging. METHODS All patients are examined before receiving a standardised treatment package for adults aged 18-65 with first-episode depression in six public outpatient clinics in the Capital Region of Denmark. From this population, we will recruit a cohort of 800 patients for whom we will acquire clinical, cognitive, psychometric, and biological data. A subgroup (subcohort I, n = 600) will additionally provide neuroimaging data, i.e., Magnetic Resonance Imaging, and Electroencephalogram, and a subgroup of patients from subcohort I unmedicated at inclusion (subcohort II, n = 60) will also undergo a brain Positron Emission Tomography with the [11C]-UCB-J tracer binding to the presynaptic glycoprotein-SV2A. Subcohort allocation is based on eligibility and willingness to participate. The treatment package typically lasts six months. Depression severity is assessed with the Quick Inventory of Depressive Symptomatology (QIDS) at baseline, and 6, 12 and 18 months after treatment initiation. The primary outcome is remission (QIDS ≤ 5) and clinical improvement (≥ 50% reduction in QIDS) after 6 months. Secondary endpoints include remission at 12 and 18 months and %-change in QIDS, 10-item Symptom Checklist, 5-item WHO Well-Being Index, and modified Disability Scale from baseline through follow-up. We also assess psychotherapy and medication side-effects. We will use machine learning to determine a combination of characteristics that best predict treatment outcomes and statistical models to investigate the association between individual measures and clinical outcomes. We will assess associations between patient characteristics, treatment choices, and clinical outcomes using path analysis, enabling us to estimate the effect of treatment choices and timing on the clinical outcome. DISCUSSION The BrainDrugs-Depression study is a real-world deep-phenotyping clinical cohort study of first-episode MDD patients. TRIAL REGISTRATION Registered at clinicaltrials.gov November 15th, 2022 (NCT05616559).
Collapse
Affiliation(s)
- Kristian Høj Reveles Jensen
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.,Psychiatric Centre Copenhagen, Copenhagen, Denmark
| | - Vibeke H Dam
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Melanie Ganz
- Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | - Patrick MacDonald Fisher
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Cheng-Teng Ip
- Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Center for Cognitive and Brain Sciences, University of Macau, Taipa, Macau SAR, China
| | - Anjali Sankar
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Maja Rou Marstrand-Joergensen
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.,Department of Neurology, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
| | - Brice Ozenne
- Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Public Health, Section of Biostatistics, University of Copenhagen, Copenhagen, Denmark
| | - Merete Osler
- Center for Clinical Research and Prevention, Bispebjerg & Frederiksberg Hospitals, Copenhagen, Denmark.,Department of Public Health, Section of Epidemiology, University of Copenhagen, Copenhagen, Denmark
| | - Brenda W J H Penninx
- Department of Psychiatry, Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands
| | - Lars H Pinborg
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.,Department of Neurology, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
| | - Vibe Gedsø Frokjaer
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.,Psychiatric Centre Copenhagen, Copenhagen, Denmark
| | - Gitte Moos Knudsen
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark.,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.,Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.,Department of Neurology, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
| | - Martin Balslev Jørgensen
- BrainDrugs, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark. .,Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark. .,Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark. .,Psychiatric Centre Copenhagen, Copenhagen, Denmark.
| |
Collapse
|
47
|
Garcia-Argibay M, Zhang-James Y, Cortese S, Lichtenstein P, Larsson H, Faraone SV. Predicting childhood and adolescent attention-deficit/hyperactivity disorder onset: a nationwide deep learning approach. Mol Psychiatry 2023; 28:1232-1239. [PMID: 36536075 PMCID: PMC10005952 DOI: 10.1038/s41380-022-01918-8] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 12/05/2022] [Accepted: 12/09/2022] [Indexed: 12/23/2022]
Abstract
Attention-deficit/hyperactivity disorder (ADHD) is a heterogeneous disorder with a high degree of psychiatric and physical comorbidity, which complicates its diagnosis in childhood and adolescence. We analyzed registry data from 238,696 persons born and living in Sweden between 1995 and 1999. Several machine learning techniques were used to assess the ability of registry data to inform the diagnosis of ADHD in childhood and adolescence: logistic regression, random Forest, gradient boosting, XGBoost, penalized logistic regression, deep neural network (DNN), and ensemble models. The best fitting model was the DNN, achieving an area under the receiver operating characteristic curve of 0.75, 95% CI (0.74-0.76) and balanced accuracy of 0.69. At the 0.45 probability threshold, sensitivity was 71.66% and specificity was 65.0%. There was an overall agreement in the feature importance among all models (τ > .5). The top 5 features contributing to classification were having a parent with criminal convictions, male sex, having a relative with ADHD, number of academic subjects failed, and speech/learning disabilities. A DNN model predicting childhood and adolescent ADHD trained exclusively on Swedish register data achieved good discrimination. If replicated and validated in an external sample, and proven to be cost-effective, this model could be used to alert clinicians to individuals who ought to be screened for ADHD and to aid clinicians' decision-making with the goal of decreasing misdiagnoses. Further research is needed to validate results in different populations and to incorporate new predictors.
Collapse
Affiliation(s)
- Miguel Garcia-Argibay
- School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro, Sweden.
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden.
| | - Yanli Zhang-James
- Departments of Psychiatry and of Neuroscience and Physiology, SUNY Upstate Medical University, Syracuse, NY, USA
| | - Samuele Cortese
- School of Psychology, University of Southampton, Southampton, UK
- Clinical and Experimental Sciences (CNS and Psychiatry), Faculty of Medicine, University of Southampton, Southampton, UK
- Solent NHS Trust, Southampton, UK
- Hassenfeld Children's Hospital at NYU Langone, New York University Child Study Center, New York City, New York, NY, USA
- Division of Psychiatry and Applied Psychology, School of Medicine, University of Nottingham, Nottingham, UK
| | - Paul Lichtenstein
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Henrik Larsson
- School of Medical Sciences, Faculty of Medicine and Health, Örebro University, Örebro, Sweden
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Stephen V Faraone
- Departments of Psychiatry and of Neuroscience and Physiology, SUNY Upstate Medical University, Syracuse, NY, USA
| |
Collapse
|
48
|
Chen Z, Liu X, Yang Q, Wang YJ, Miao K, Gong Z, Yu Y, Leonov A, Liu C, Feng Z, Chuan-Peng H. Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis: A Systematic Review. JAMA Netw Open 2023; 6:e231671. [PMID: 36877519 PMCID: PMC9989906 DOI: 10.1001/jamanetworkopen.2023.1671] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/07/2023] Open
Abstract
IMPORTANCE Neuroimaging-based artificial intelligence (AI) diagnostic models have proliferated in psychiatry. However, their clinical applicability and reporting quality (ie, feasibility) for clinical practice have not been systematically evaluated. OBJECTIVE To systematically assess the risk of bias (ROB) and reporting quality of neuroimaging-based AI models for psychiatric diagnosis. EVIDENCE REVIEW PubMed was searched for peer-reviewed, full-length articles published between January 1, 1990, and March 16, 2022. Studies aimed at developing or validating neuroimaging-based AI models for clinical diagnosis of psychiatric disorders were included. Reference lists were further searched for suitable original studies. Data extraction followed the CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. A closed-loop cross-sequential design was used for quality control. The PROBAST (Prediction Model Risk of Bias Assessment Tool) and modified CLEAR (Checklist for Evaluation of Image-Based Artificial Intelligence Reports) benchmarks were used to systematically evaluate ROB and reporting quality. FINDINGS A total of 517 studies presenting 555 AI models were included and evaluated. Of these models, 461 (83.1%; 95% CI, 80.0%-86.2%) were rated as having a high overall ROB based on the PROBAST. The ROB was particular high in the analysis domain, including inadequate sample size (398 of 555 models [71.7%; 95% CI, 68.0%-75.6%]), poor model performance examination (with 100% of models lacking calibration examination), and lack of handling data complexity (550 of 555 models [99.1%; 95% CI, 98.3%-99.9%]). None of the AI models was perceived to be applicable to clinical practices. Overall reporting completeness (ie, number of reported items/number of total items) for the AI models was 61.2% (95% CI, 60.6%-61.8%), and the completeness was poorest for the technical assessment domain with 39.9% (95% CI, 38.8%-41.1%). CONCLUSIONS AND RELEVANCE This systematic review found that the clinical applicability and feasibility of neuroimaging-based AI models for psychiatric diagnosis were challenged by a high ROB and poor reporting quality. Particularly in the analysis domain, ROB in AI diagnostic models should be addressed before clinical application.
Collapse
Affiliation(s)
- Zhiyi Chen
- School of Psychology, Third Military Medical University, Chongqing, China
- Experimental Research Center for Medical and Psychological Science, Third Military Medical University, Chongqing, China
| | - Xuerong Liu
- School of Psychology, Third Military Medical University, Chongqing, China
- Experimental Research Center for Medical and Psychological Science, Third Military Medical University, Chongqing, China
| | - Qingwu Yang
- Department of Neurology, Daping Hospital, Third Military Medical University, Chongqing, China
| | - Yan-Jiang Wang
- Department of Neurology, Daping Hospital, Third Military Medical University, Chongqing, China
| | - Kuan Miao
- School of Psychology, Third Military Medical University, Chongqing, China
- Experimental Research Center for Medical and Psychological Science, Third Military Medical University, Chongqing, China
| | - Zheng Gong
- School of Psychology, Third Military Medical University, Chongqing, China
- Experimental Research Center for Medical and Psychological Science, Third Military Medical University, Chongqing, China
| | - Yang Yu
- School of Psychology, Third Military Medical University, Chongqing, China
| | - Artemiy Leonov
- Department of Psychology, Clark University, Worcester, Massachusetts
| | - Chunlei Liu
- School of Psychology, Qufu Normal University, Qufu, China
| | - Zhengzhi Feng
- School of Psychology, Third Military Medical University, Chongqing, China
- Experimental Research Center for Medical and Psychological Science, Third Military Medical University, Chongqing, China
| | - Hu Chuan-Peng
- School of Psychology, Nanjing Normal University, Nanjing, China
| |
Collapse
|
49
|
Allesøe RL, Thompson WK, Bybjerg-Grauholm J, Hougaard DM, Nordentoft M, Werge T, Rasmussen S, Benros ME. Deep Learning for Cross-Diagnostic Prediction of Mental Disorder Diagnosis and Prognosis Using Danish Nationwide Register and Genetic Data. JAMA Psychiatry 2023; 80:146-155. [PMID: 36477816 PMCID: PMC9857190 DOI: 10.1001/jamapsychiatry.2022.4076] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Importance Diagnoses and treatment of mental disorders are hampered by the current lack of objective markers needed to provide a more precise diagnosis and treatment strategy. Objective To develop deep learning models to predict mental disorder diagnosis and severity spanning multiple diagnoses using nationwide register data, family and patient-specific diagnostic history, birth-related measurement, and genetics. Design, Setting, and Participants This study was conducted from May 1, 1981, to December 31, 2016. For the analysis, which used a Danish population-based case-cohort sample of individuals born between 1981 and 2005, genotype data and matched longitudinal health register data were taken from the longitudinal Danish population-based Integrative Psychiatric Research Consortium 2012 case-cohort study. Included were individuals with mental disorders (attention-deficit/hyperactivity disorder [ADHD]), autism spectrum disorder (ASD), major depressive disorder (MDD), bipolar disorder (BD), schizophrenia spectrum disorders (SCZ), and population controls. Data were analyzed from February 1, 2021, to January 24, 2022. Exposure At least 1 hospital contact with diagnosis of ADHD, ASD, MDD, BD, or SCZ. Main Outcomes and Measures The predictability of (1) mental disorder diagnosis and (2) severity trajectories (measured by future outpatient hospital contacts, admissions, and suicide attempts) were investigated using both a cross-diagnostic and single-disorder setup. Predictive power was measured by AUC, accuracy, and Matthews correlation coefficient (MCC), including an estimate of feature importance. Results A total of 63 535 individuals (mean [SD] age, 23 [7] years; 34 944 male [55%]; 28 591 female [45%]) were included in the model. Based on data prior to diagnosis, the specific diagnosis was predicted in a multidiagnostic prediction model including the background population with an overall area under the curve (AUC) of 0.81 and MCC of 0.28, whereas the single-disorder models gave AUCs/MCCs of 0.84/0.54 for SCZ, 0.79/0.41 for BD, 0.77/0.39 for ASD, 0.74/0.38, for ADHD, and 0.74/0.38 for MDD. The most important data sets for multidiagnostic prediction were previous mental disorders and age (11%-23% reduction in prediction accuracy when removed) followed by family diagnoses, birth-related measurements, and genetic data (3%-5% reduction in prediction accuracy when removed). Furthermore, when predicting subsequent disease trajectories of the disorder, the most severe cases were the most easily predictable, with an AUC of 0.72. Conclusions and Relevance Results of this diagnostic study suggest the possibility of combining genetics and registry data to predict both mental disorder diagnosis and disorder progression in a clinically relevant, cross-diagnostic setting prior to clinical assessment.
Collapse
Affiliation(s)
- Rosa Lundbye Allesøe
- Copenhagen Research Centre for Mental Health, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark,Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Wesley K. Thompson
- Division of Biostatistics and Department of Radiology, Population Neuroscience and Genetics Lab, University of California, San Diego, La Jolla
| | - Jonas Bybjerg-Grauholm
- iPSYCH, The Lundbeck Foundation Initiative for Integrative Psychiatric Research, Department of Immunology and Microbiology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark,Center for Neonatal Screening, Department for Congenital Disorders, Statens Serum Institut, Copenhagen, Denmark
| | - David M. Hougaard
- iPSYCH, The Lundbeck Foundation Initiative for Integrative Psychiatric Research, Department of Immunology and Microbiology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark,Center for Neonatal Screening, Department for Congenital Disorders, Statens Serum Institut, Copenhagen, Denmark
| | - Merete Nordentoft
- Copenhagen Research Centre for Mental Health, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark,iPSYCH, The Lundbeck Foundation Initiative for Integrative Psychiatric Research, Department of Immunology and Microbiology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Thomas Werge
- iPSYCH, The Lundbeck Foundation Initiative for Integrative Psychiatric Research, Department of Immunology and Microbiology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark,Institute of Biological Psychiatry, Mental Health Centre Sct Hans, Mental Health Services Copenhagen, Roskilde, Denmark
| | - Simon Rasmussen
- Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Michael Eriksen Benros
- Copenhagen Research Centre for Mental Health, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark,Department of Immunology and Microbiology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
50
|
Sathyanarayanan A, Mueller TT, Ali Moni M, Schueler K, Baune BT, Lio P, Mehta D, Baune BT, Dierssen M, Ebert B, Fabbri C, Fusar-Poli P, Gennarelli M, Harmer C, Howes OD, Janzing JGE, Lio P, Maron E, Mehta D, Minelli A, Nonell L, Pisanu C, Potier MC, Rybakowski F, Serretti A, Squassina A, Stacey D, van Westrhenen R, Xicota L. Multi-omics data integration methods and their applications in psychiatric disorders. Eur Neuropsychopharmacol 2023; 69:26-46. [PMID: 36706689 DOI: 10.1016/j.euroneuro.2023.01.001] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Revised: 11/22/2022] [Accepted: 01/02/2023] [Indexed: 01/27/2023]
Abstract
To study mental illness and health, in the past researchers have often broken down their complexity into individual subsystems (e.g., genomics, transcriptomics, proteomics, clinical data) and explored the components independently. Technological advancements and decreasing costs of high throughput sequencing has led to an unprecedented increase in data generation. Furthermore, over the years it has become increasingly clear that these subsystems do not act in isolation but instead interact with each other to drive mental illness and health. Consequently, individual subsystems are now analysed jointly to promote a holistic understanding of the underlying biological complexity of health and disease. Complementing the increasing data availability, current research is geared towards developing novel methods that can efficiently combine the information rich multi-omics data to discover biologically meaningful biomarkers for diagnosis, treatment, and prognosis. However, clinical translation of the research is still challenging. In this review, we summarise conventional and state-of-the-art statistical and machine learning approaches for discovery of biomarker, diagnosis, as well as outcome and treatment response prediction through integrating multi-omics and clinical data. In addition, we describe the role of biological model systems and in silico multi-omics model designs in clinical translation of psychiatric research from bench to bedside. Finally, we discuss the current challenges and explore the application of multi-omics integration in future psychiatric research. The review provides a structured overview and latest updates in the field of multi-omics in psychiatry.
Collapse
Affiliation(s)
- Anita Sathyanarayanan
- Queensland University of Technology, Centre for Genomics and Personalised Health, School of Biomedical Sciences, Faculty of Health, Kelvin Grove, Queensland 4059, Australia
| | - Tamara T Mueller
- Institute for Artificial Intelligence and Informatics in Medicine, TU Munich, 80333 Munich, Germany
| | - Mohammad Ali Moni
- Artificial Intelligence and Digital Health Data Science, School of Health and Rehabilitation Sciences, Faculty of Health and Behavioural Sciences, The University of Queensland, St Lucia, QLD, 4072, Australia
| | - Katja Schueler
- Clinic for Psychosomatics, Hospital zum Heiligen Geist, Frankfurt am Main, Germany; Frankfurt Psychoanalytic Institute, Frankfurt am Main, Germany
| | - Bernhard T Baune
- Department of Psychiatry and Psychotherapy, University of Münster, Germany; Department of Psychiatry, Melbourne Medical School, University of Melbourne, Australia; The Florey Institute of Neuroscience and Mental Health, The University of Melbourne, Australia
| | - Pietro Lio
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Divya Mehta
- Queensland University of Technology, Centre for Genomics and Personalised Health, School of Biomedical Sciences, Faculty of Health, Kelvin Grove, Queensland 4059, Australia.
| | | | - Bernhard T Baune
- Department of Psychiatry and Psychotherapy, University of Münster, Germany; Department of Psychiatry, Melbourne Medical School, University of Melbourne, Australia; The Florey Institute of Neuroscience and Mental Health, The University of Melbourne, Australia
| | - Mara Dierssen
- Center for Genomic Regulation (CRG), The Barcelona Institute of Science and Technology; Universitat Pompeu Fabra (UPF), Barcelona, Spain
| | - Bjarke Ebert
- Medical Strategy & Communication, H. Lundbeck A/S, Valby, Denmark
| | - Chiara Fabbri
- Department of Biomedical and NeuroMotor Sciences, University of Bologna, Bologna, Italy; Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom
| | - Paolo Fusar-Poli
- Early Psychosis: Intervention and Clinical-detection (EPIC) Lab, Department of Psychosis Studies, King's College London, United Kingdom; Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
| | - Massimo Gennarelli
- Department of Molecular and Translational Medicine, University of Brescia; Genetics Unit, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | | | - Oliver D Howes
- Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, United Kingdom; Psychiatric Imaging, Medical Research Council Clinical Sciences Centre, Imperial College London, Hammersmith Hospital Campus, London, United Kingdom
| | | | - Pietro Lio
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Eduard Maron
- Department of Psychiatry, University of Tartu, Tartu, Estonia; Centre for Neuropsychopharmacology, Division of Brain Sciences, Imperial College London, London, United Kingdom; Documental Ltd, Tallin, Estonia; West Tallinn Central Hospital, Tallinn, Estonia
| | - Divya Mehta
- Queensland University of Technology, Centre for Genomics and Personalised Health, School of Biomedical Sciences, Faculty of Health, Kelvin Grove, Queensland 4059, Australia
| | - Alessandra Minelli
- Department of Molecular and Translational Medicine, University of Brescia; Genetics Unit, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | - Lara Nonell
- MARGenomics, IMIM (Hospital del Mar Research Institute), Barcelona, Spain
| | - Claudia Pisanu
- Department of Biomedical Sciences, Section of Neuroscience and Clinical Pharmacology, University of Cagliari, Cagliari, Italy
| | | | - Filip Rybakowski
- Department of Psychiatry, Poznan University of Medical Sciences, Poznan, Poland
| | - Alessandro Serretti
- Department of Biomedical and NeuroMotor Sciences, University of Bologna, Bologna, Italy
| | - Alessio Squassina
- Department of Biomedical Sciences, Section of Neuroscience and Clinical Pharmacology, University of Cagliari, Cagliari, Italy
| | - David Stacey
- British Heart Foundation Cardiovascular Epidemiology Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, United Kingdom
| | - Roos van Westrhenen
- Parnassia Psychiatric Institute, Amsterdam, the Netherlands; Department of Psychiatry and Neuropsychology, Faculty of Health and Sciences, Maastricht University, Maastricht, the Netherlands; Institute of Psychiatry, Psychology & Neuroscience (IoPPN) King's College London, United Kingdom
| | - Laura Xicota
- Paris Brain Institute ICM, Salpetriere Hospital, Paris, France
| |
Collapse
|