1
|
Ostojic D, Lalousis PA, Donohoe G, Morris DW. The challenges of using machine learning models in psychiatric research and clinical practice. Eur Neuropsychopharmacol 2024; 88:53-65. [PMID: 39232341 DOI: 10.1016/j.euroneuro.2024.08.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 08/06/2024] [Accepted: 08/12/2024] [Indexed: 09/06/2024]
Abstract
To understand the complex nature of heterogeneous psychiatric disorders, scientists and clinicians are required to employ a wide range of clinical, endophenotypic, neuroimaging, genomic, and environmental data to understand the biological mechanisms of psychiatric illness before this knowledge is applied into clinical setting. Machine learning (ML) is an automated process that can detect patterns from large multidimensional datasets and can supersede conventional statistical methods as it can detect both linear and non-linear relationships. Due to this advantage, ML has potential to enhance our understanding, improve diagnosis, prognosis and treatment of psychiatric disorders. The current review provides an in-depth examination of, and offers practical guidance for, the challenges encountered in the application of ML models in psychiatric research and clinical practice. These challenges include the curse of dimensionality, data quality, the 'black box' problem, hyperparameter tuning, external validation, class imbalance, and data representativeness. These challenges are particularly critical in the context of psychiatry as it is expected that researchers will encounter them during the stages of ML model development and deployment. We detail practical solutions and best practices to effectively mitigate the outlined challenges. These recommendations have the potential to improve reliability and interpretability of ML models in psychiatry.
Collapse
Affiliation(s)
- Dijana Ostojic
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland
| | - Paris Alexandros Lalousis
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom; Section for Precision Psychiatry, Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University Munich, Munich, Germany
| | - Gary Donohoe
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland
| | - Derek W Morris
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland.
| |
Collapse
|
2
|
Jan Ben S, Dörner M, Günther MP, von Känel R, Euler S. Proof of concept: Predicting distress in cancer patients using back propagation neural network (BPNN). Heliyon 2023; 9:e18328. [PMID: 37576295 PMCID: PMC10412887 DOI: 10.1016/j.heliyon.2023.e18328] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2022] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/15/2023] Open
Abstract
Background Research findings suggest that a significant proportion of individuals diagnosed with cancer, ranging from 25% to 60%, experience distress and require access to psycho-oncological services. Until now, only contemporary approaches, such as logistic regression, have been used to determine predictors of distress in oncological patients. To improve individual prediction accuracy, novel approaches are required. We aimed to establish a prediction model for distress in cancer patients based on a back propagation neural network (BPNN). Methods Retrospective data was gathered from a cohort of 3063 oncological patients who received diagnoses and treatment spanning the years 2011-2019. The distress thermometer (DT) has been used as screening instrument. Potential predictors of distress were identified using logistic regression. Subsequently, a prediction model for distress was developed using BPNN. Results Logistic regression identified 13 significant independent variables as predictors of distress, including emotional, physical and practical problems. Through repetitive data simulation processes, it was determined that a 3-layer BPNN with 8 neurons in the hidden layer demonstrates the highest level of accuracy as a prediction model. This model exhibits a sensitivity of 79.0%, specificity of 71.8%, positive predictive value of 78.9%, negative predictive value of 71.9%, and an overall coincidence rate of 75.9%. Conclusion The final BPNN model serves as a compelling proof of concept for leveraging artificial intelligence in predicting distress and its associated risk factors in cancer patients. The final model exhibits a remarkable level of discrimination and feasibility, underscoring its potential for identifying patients vulnerable to distress.
Collapse
Affiliation(s)
- Schulze Jan Ben
- Department of Consultation-Liaison-Psychiatry and Psychosomatic Medicine, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Marc Dörner
- Department of Consultation-Liaison-Psychiatry and Psychosomatic Medicine, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Moritz Philipp Günther
- Department of Consultation-Liaison-Psychiatry and Psychosomatic Medicine, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Roland von Känel
- Department of Consultation-Liaison-Psychiatry and Psychosomatic Medicine, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Sebastian Euler
- Department of Consultation-Liaison-Psychiatry and Psychosomatic Medicine, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| |
Collapse
|
3
|
Reading Wishes from the Lips: Cancer Patients' Need for Psycho-Oncological Support during Inpatient and Outpatient Treatment. Diagnostics (Basel) 2022; 12:diagnostics12102440. [PMID: 36292128 PMCID: PMC9600894 DOI: 10.3390/diagnostics12102440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 09/23/2022] [Accepted: 10/07/2022] [Indexed: 11/24/2022] Open
Abstract
Background: Psycho-oncological support (PO) is an effective measure to reduce distress and improve the quality of life in patients with cancer. Currently, there are only a few studies investigating the (expressed) wish for PO. The aim of this study was to evaluate the number of patients who request PO and to identify predictors for the wish for PO. Methods: Data from 3063 cancer patients who had been diagnosed and treated at a Comprehensive Cancer Center between 2011 and 2019 were analyzed retrospectively. Potential predictors for the wish for PO were identified using logistic regression. As a novelty, a Back Propagation Neural Network (BPNN) was applied to establish a prediction model for the wish for PO. Results: In total, 1752 patients (57.19%) had a distress score above the cut-off and 14.59% expressed the wish for PO. Patients’ requests for pastoral care (OR = 13.1) and social services support (OR = 5.4) were the strongest predictors of the wish for PO. Patients of the female sex or who had a current psychiatric diagnosis, opioid treatment and malignant neoplasms of the skin and the hematopoietic system also predicted the wish for PO, while malignant neoplasms of digestive organs and older age negatively predicted the wish for PO. These nine significant predictors were used as input variables for the BPNN model. BPNN computations indicated that a three-layer network with eight neurons in the hidden layer is the most precise prediction model. Discussion: Our results suggest that the identification of predictors for the wish for PO might foster PO referrals and help cancer patients reduce barriers to expressing their wish for PO. Furthermore, the final BPNN prediction model demonstrates a high level of discrimination and might be easily implemented in the hospital information system.
Collapse
|
4
|
Yang JP, Zhao H, Du YZ, Ma HW, Zhao Q, Li C, Zhang Y, Li B, Guo HX, Ban HP, Lin HP, Gu WL, Meng XG, Song Q, Jin XX, Jiang T, Du X, Dong YX, Jiang HL, Wu NF, Liu W, Rao C, Tong YJ, Li Y, Liu JY. Study on quantitative diagnosis model of TCM syndromes of post-stroke depression based on combination of disease and syndrome. Medicine (Baltimore) 2021; 100:e25041. [PMID: 33761663 PMCID: PMC9281908 DOI: 10.1097/md.0000000000025041] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 02/12/2021] [Indexed: 01/05/2023] Open
Abstract
BACKGROUND Post-stroke depression (PSD) is one of the most common stroke complications with high morbidity. Researchers have done much clinical research on Traditional Chinese Medicine (TCM) treatment, but very little research on diagnosis. Based on the thought of combination of disease and syndrome, this study will establish a unified and objective quantitative diagnosis model of TCM syndromes of PSD, so as to improve the clinical diagnosis and treatment of PSD. OBJECTIVE First: To establish a unified and objective quantitative diagnosis model of TCM syndromes in PSD under different disease courses, and identify the corresponding main, secondary, and concurrent symptoms, which are based on the weighting factor of each TCM symptom. Second: To find out the relationship between different stages of PSD and TCM syndromes. Clarify the main syndrome types of PSD under different stages of disease. Reveal the evolution and progression mechanism of TCM syndromes of PSD. METHODS AND ANALYSIS This is a retrospective study of PSD TCM diagnosis. Three hundred patients who were hospitalized in the First Teaching Hospital of Tianjin University of TCM with complete cases from January 2014 to January 2019 are planned to be recruited. The study will mainly collect the diagnostic information from the cases, find the related indicators of TCM and Western medicine in PSD, and clarify the relationship between different disease stages and TCM syndromes. Finally, the PSD TCM syndrome quantitative diagnosis model will be established based on the operation principle of Back Propagation (BP) artificial neural network. CONCLUSION To collect sufficient medical records and establish models to speed up the process of TCM diagnosis.
Collapse
Affiliation(s)
- Ji-Peng Yang
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Hong Zhao
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Yu-Zheng Du
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Hong-Wen Ma
- Nankai University Affiliated Hospital, Tianjin
| | - Qi Zhao
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Chen Li
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Yi Zhang
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Bo Li
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Hong-Xia Guo
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Hai-Peng Ban
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Hai-Ping Lin
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Wen-Long Gu
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Xiang-Gang Meng
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Qian Song
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Xiao-Xian Jin
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Tao Jiang
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Xin Du
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | | | - Hai-Lun Jiang
- Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Nan-Fang Wu
- Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Wei Liu
- Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Chang Rao
- Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Yan-Jie Tong
- Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Yu Li
- First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion
| | - Jing-Ying Liu
- Tianjin University of Traditional Chinese Medicine, Tianjin, China
| |
Collapse
|
5
|
Chattopadhyay S, Kaur P, Rabhi F, Acharya UR. Neural network approaches to grade adult depression. J Med Syst 2012; 36:2803-2815. [PMID: 21833604 DOI: 10.1007/s10916-011-9759-1] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2011] [Accepted: 07/07/2011] [Indexed: 02/08/2023]
Abstract
Depression is a common but worrying psychological disorder that adversely affects one's quality of life. It is more ominous to note that its incidence is increasing. On the other hand, screening and grading of depression is still a manual and time consuming process that might be biased. In addition, grades of depression are often determined in continuous ranges, e.g., 'mild to moderate' and 'moderate to severe' instead of making them more discrete as 'mild', 'moderate', and 'severe'. Grading as a continuous range is confusing to the doctors and thus affecting the management plan at large. Given this practical issue, the present paper attempts to differentiate depression grades more accurately using two neural net learning approaches-'supervised', i.e., classification with Back propagation neural network (BPNN) and Adaptive Network-based Fuzzy Inference System (ANFIS) classifiers, and 'unsupervised', i.e., 'clustering' technique with Self-organizing map (SOM), built in MATLAB 7. The reason for using the supervised and unsupervised learning approaches is that, supervised learning depends exclusively on domain knowledge. Supervision may induce biasness and subjectivities related to the decision-making. Finally, the performance of BPNN and ANFIS are compared and discussed. It was observed that ANFIS, being a hybrid system, performed much better compared to the BPNN classifier. On the other hand, SOM-based clustering technique is also found useful in constructing three distinct clusters. It also assists visualizing the multidimensional data clusters into 2-D.
Collapse
Affiliation(s)
- Subhagata Chattopadhyay
- Dept. of Computer Science and Engineering, National Institute of Science and Technology, Berhampur, Orissa, India.
| | | | | | | |
Collapse
|
6
|
Designing a decision support system for distinguishing ADHD from similar children behavioral disorders. J Med Syst 2010; 36:1335-43. [PMID: 20878211 DOI: 10.1007/s10916-010-9594-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2010] [Accepted: 09/06/2010] [Indexed: 10/19/2022]
Abstract
In this study, a decision support system was designed to distinguish children with ADHD from other similar children behavioral disorders such as depression, anxiety, comorbid depression and anxiety and conduct disorder based on the signs and symptoms. Accuracy of classifying with Radial basis function and multilayer neural networks were compared. Finally, the average accuracy of the networks in classification reached to 95.50% and 96.62% by multilayer and radial basis function networks respectively. Our results indicate that a decision support system, especially RBF, may be a good preliminary assistant for psychiatrists in diagnosing high risk behavioral disorders of children.
Collapse
|
7
|
Rădulescu AR, Mujica-Parodi LR. A principal component network analysis of prefrontal-limbic functional magnetic resonance imaging time series in schizophrenia patients and healthy controls. Psychiatry Res 2009; 174:184-94. [PMID: 19880294 PMCID: PMC2788080 DOI: 10.1016/j.pscychresns.2009.04.017] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2008] [Revised: 04/29/2009] [Accepted: 04/29/2009] [Indexed: 10/20/2022]
Abstract
We investigated neural regulation of emotional arousal. We hypothesized that the interactions between the components of the prefrontal-limbic system determine the global trajectories of the individual's brain activation, with the strengths and modulations of these interactions being potentially key components underlying the differences between healthy individuals and those with schizophrenia. Using affect-valent facial stimuli presented to 11 medicated schizophrenia patients and 65 healthy controls, we activated neural regions associated with the emotional arousal response during functional magnetic resonance imaging (fMRI). Performing first a random effects analysis of the fMRI data to identify activated regions, we obtained 352 data-point time series for six brain regions: bilateral amygdala, hippocampus and two prefrontal regions (Brodmann Areas 9 and 45). Since standard statistical methods are not designed to capture system features and evolution, we used principal component analyses on two types of pre-processed data: contrasts and group averages. We captured an important characteristic of the evolution of our six-dimensional brain network: all subject trajectories are almost embedded in a two-dimensional plane. Moreover, the direction of the largest principal component was a significant differentiator between the control and patient populations: the left and right amygdala coefficients were substantially higher in the case of patients, and the coefficients of Brodmann Area 9 were, to a lesser extent, higher in controls. These results are evidence that modulations between the regions of interest are the important determinant factors for the system's dynamical behavior. We place our results within the context of other principal component analyses used in neuroimaging, as well as of our existing theoretical model of prefrontal-limbic dysregulation.
Collapse
Affiliation(s)
- Anca R Rădulescu
- Department of Applied Mathematics, UCB 526, University of Colorado at Boulder, Boulder, CO 80309-0526, USA.
| | | |
Collapse
|
8
|
Chattopadhyay S, Pratihar D, De Sarkar S. Fuzzy-Logic-Based Screening and Prediction of Adult Psychoses: A Novel Approach. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS - PART A: SYSTEMS AND HUMANS 2009; 39:381-387. [DOI: 10.1109/tsmca.2008.2010138] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
|
9
|
Tam SF, Cheing GLY, Hui-Chan CWY. Predicting osteoarthritic knee rehabilitation outcome by using a prediction model developed by data mining techniques. Int J Rehabil Res 2004; 27:65-9. [PMID: 15097172 DOI: 10.1097/00004356-200403000-00009] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
Artificial neural networks (ANN) have been applied to assist in clinical decision-making and prediction. While we consider possible effective treatments for patients with osteoarthritic knee such as Transcutaneous Electrical Nerve Stimulation (TENS), exercise, and TENS with exercise respectively, we have to select a treatment protocol for patients such that they would gain the best improvements according to their clinical conditions. To facilitate this functionality with the existing patient assessment, we hope to apply the ANN programming techniques to develop a computerized prediction system. A preliminary validation was performed to test the validity of the newly developed prediction protocol on knee rehabilitation. We input the key clinical attributes of 62 patients who have undergone the three above-mentioned knee treatments to the protocol. The expected pain improvement of each patient as predicted by the protocol was obtained. Spearman rank-order correlation was used to identify whether there was a significant correlation between the rankings of the observed and expected pain improvement. We found that the Spearman's rho was 0.424, which is statistically significant at p < 0.001. From this preliminary analysis, we are confident that this newly developed prediction protocol will be useful when deciding which treatment regime best suits a patient.
Collapse
Affiliation(s)
- Sing-Fai Tam
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong.
| | | | | |
Collapse
|
10
|
Abstract
This paper proposes the use of multilayer perceptron for brain dysfunction diagnosis. The performance of MLP was better than that of Discriminant Analysis and Decision Tree classifiers, with an 85% accuracy rate in an experimental test involving 332 subjects. In addition, the neural network employing Bayesian learning was able to identify the most important input variable. These two results demonstrate that the neural network can be effectively used in the diagnosis of children with brain dysfunction.
Collapse
Affiliation(s)
- S Cho
- Department of Industrial Engineering, Seoul National University, San 56-1 Shinrimdong, Seoul, 151-744, Korea.
| | | |
Collapse
|
11
|
Affiliation(s)
- P J Drew
- University of Hull Academic Surgical Unit, Castle Hill Hospital, United Kingdom
| | | |
Collapse
|
12
|
Price RK, Spitznagel EL, Downey TJ, Meyer DJ, Risk NK, El-Ghazzawy OG. Applying artificial neural network models to clinical decision making. Psychol Assess 2000. [DOI: 10.1037/1040-3590.12.1.40] [Citation(s) in RCA: 28] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
13
|
Davis GE, Lowell WE, Davis GL. Determining the number of state psychiatric hospital beds by measuring quality of care with artificial neural networks. Am J Med Qual 1998; 13:13-24. [PMID: 9509590 DOI: 10.1177/106286069801300103] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
This study uses a new paradigm to calculate the minimum and the optimum number of involuntary psychiatric beds at a state hospital in Maine with 5538 admissions over a 7-year period. The method measures quality of care (Q) based upon the accuracy of prediction of length-of-stay for the hospital, and of community length-of-stay for the community, each corrected for the severity of illness of the average patient. When Q in the hospital equals Q in the community, there is no net movement of patients from one phase of care to the other, analogous to a zero electromotive force, and the census at that point is the minimum number of beds (22 beds/100,000 population). When patients in the community were least ill, relative to the hospital then hospital bed census is at its optimum (31 beds/100,000) given current resources and technology. In studying specific diagnosis groups with the same methodology the authors found that patients with schizophrenia having the benefit of clozapine for most of the study period had a Q averaged over 7 years that was nearly equal in both hospital and community settings. This explains the perception that tertiary psychiatric hospitals comprised mostly of patients with schizophrenia can downsize significantly. However, affective disorders and "borderline" personality disorders clearly benefit from structured hospital care with specialized experienced staff. We make arguments for the role of the state hospital as a homeostat for the mental health care delivery system.
Collapse
Affiliation(s)
- G E Davis
- Augusta Mental Health Institute, ME 04332, USA
| | | | | |
Collapse
|
14
|
|