1
|
Gholi Zadeh Kharrat F, Gagne C, Lesage A, Gariépy G, Pelletier JF, Brousseau-Paradis C, Rochette L, Pelletier E, Lévesque P, Mohammed M, Wang J. Explainable artificial intelligence models for predicting risk of suicide using health administrative data in Quebec. PLoS One 2024; 19:e0301117. [PMID: 38568987 PMCID: PMC10990247 DOI: 10.1371/journal.pone.0301117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 03/11/2024] [Indexed: 04/05/2024] Open
Abstract
Suicide is a complex, multidimensional event, and a significant challenge for prevention globally. Artificial intelligence (AI) and machine learning (ML) have emerged to harness large-scale datasets to enhance risk detection. In order to trust and act upon the predictions made with ML, more intuitive user interfaces must be validated. Thus, Interpretable AI is one of the crucial directions which could allow policy and decision makers to make reasonable and data-driven decisions that can ultimately lead to better mental health services planning and suicide prevention. This research aimed to develop sex-specific ML models for predicting the population risk of suicide and to interpret the models. Data were from the Quebec Integrated Chronic Disease Surveillance System (QICDSS), covering up to 98% of the population in the province of Quebec and containing data for over 20,000 suicides between 2002 and 2019. We employed a case-control study design. Individuals were considered cases if they were aged 15+ and had died from suicide between January 1st, 2002, and December 31st, 2019 (n = 18339). Controls were a random sample of 1% of the Quebec population aged 15+ of each year, who were alive on December 31st of each year, from 2002 to 2019 (n = 1,307,370). We included 103 features, including individual, programmatic, systemic, and community factors, measured up to five years prior to the suicide events. We trained and then validated the sex-specific predictive risk model using supervised ML algorithms, including Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost) and Multilayer perceptron (MLP). We computed operating characteristics, including sensitivity, specificity, and Positive Predictive Value (PPV). We then generated receiver operating characteristic (ROC) curves to predict suicides and calibration measures. For interpretability, Shapley Additive Explanations (SHAP) was used with the global explanation to determine how much the input features contribute to the models' output and the largest absolute coefficients. The best sensitivity was 0.38 with logistic regression for males and 0.47 with MLP for females; the XGBoost Classifier with 0.25 for males and 0.19 for females had the best precision (PPV). This study demonstrated the useful potential of explainable AI models as tools for decision-making and population-level suicide prevention actions. The ML models included individual, programmatic, systemic, and community levels variables available routinely to decision makers and planners in a public managed care system. Caution shall be exercised in the interpretation of variables associated in a predictive model since they are not causal, and other designs are required to establish the value of individual treatments. The next steps are to produce an intuitive user interface for decision makers, planners and other stakeholders like clinicians or representatives of families and people with live experience of suicidal behaviors or death by suicide. For example, how variations in the quality of local area primary care programs for depression or substance use disorders or increased in regional mental health and addiction budgets would lower suicide rates.
Collapse
Affiliation(s)
- Fatemeh Gholi Zadeh Kharrat
- Institut Intelligence et Données (IID), Université Laval, Québec, Québec, Canada
- Institut National de Santé Publique du Québec (INSPQ), Québec, Québec, Canada
| | - Christian Gagne
- Institut Intelligence et Données (IID), Université Laval, Québec, Québec, Canada
| | - Alain Lesage
- Department of Psychiatry and Addiction, Université de Montréal, Montreal, QC, Canada
- Centre de Recherche de l’Institut Universitaire en Santé Mentale de Montréal, Québec, Canada
| | - Geneviève Gariépy
- Centre for Surveillance and Applied Research, Health Promotion and Chronic Disease Prevention Branch, Public Health Agency of Canada, Ottawa, Canada
- Department of Social and Preventive Medicine, School of Public Health, University of Montreal, Montreal, Canada
- Montreal Mental Health University Institute Research Center, Montreal, Canada
| | - Jean-François Pelletier
- Department of Psychiatry and Addiction, Université de Montréal, Montreal, QC, Canada
- Centre de Recherche de l’Institut Universitaire en Santé Mentale de Montréal, Québec, Canada
| | - Camille Brousseau-Paradis
- Department of Psychiatry and Addiction, Université de Montréal, Montreal, QC, Canada
- Centre de Recherche de l’Institut Universitaire en Santé Mentale de Montréal, Québec, Canada
| | - Louis Rochette
- Institut National de Santé Publique du Québec (INSPQ), Québec, Québec, Canada
| | - Eric Pelletier
- Institut National de Santé Publique du Québec (INSPQ), Québec, Québec, Canada
| | - Pascale Lévesque
- Institut National de Santé Publique du Québec (INSPQ), Québec, Québec, Canada
| | - Mada Mohammed
- Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Halifax, Canada
| | - JianLi Wang
- Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Halifax, Canada
| |
Collapse
|
2
|
Sara SS, Rahman MA, Rahman R, Talukder A. Prediction of suicidal ideation with associated risk factors among university students in the southern part of Bangladesh: Machine learning approach. J Affect Disord 2024; 349:502-508. [PMID: 38218257 DOI: 10.1016/j.jad.2024.01.092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/27/2023] [Revised: 11/09/2023] [Accepted: 01/07/2024] [Indexed: 01/15/2024]
Abstract
BACKGROUND The prevalence of suicidal ideation has become an urgent issue, particularly among adolescents. The primary objective of this research is to determine the prevalence of suicidal ideation among students in the southern region of Bangladesh and to predict this phenomenon using machine learning (ML) models. METHODS The data collection process involved using a simple random sampling technique to gather information from university students located in the southern region of Bangladesh during the period spreading from April 2022 to June 2022. Upon accounting for missing values and non-response rates, the ultimate sample size was determined to be 584, with 51.5 % of participants identifying as male and 48.5 % female. RESULTS A significant proportion of students, precisely 19.9 %, reported experiencing suicidal ideation. Most participants were female (77 %) and unmarried (78 %). Within the machine learning (ML) framework, KNN exhibited the highest accuracy score of 91.45 %. In addition, the Random Forest (RF), and Categorical Boosting (CatBoost) algorithms exhibited comparable levels of accuracy, achieving scores of 90.60 and 90.59 respectively. LIMITATIONS Using a cross-sectional design in research limits the ability to establish causal relationships. CONCLUSION Mental health practitioners can employ the KNN model alongside patients' medical histories to detect those who may be at a higher risk of attempting suicide. This approach enables healthcare professionals to take appropriate measures, such as counselling, encouraging regular sleep patterns, and addressing depression and anxiety, to prevent suicide attempts.
Collapse
Affiliation(s)
- Sabiha Shirin Sara
- Statistics Discipline, Science Engineering and Technology School, Khulna University, Khulna 9208, Bangladesh
| | - Md Asikur Rahman
- Statistics Discipline, Science Engineering and Technology School, Khulna University, Khulna 9208, Bangladesh
| | - Riaz Rahman
- Statistics Discipline, Science Engineering and Technology School, Khulna University, Khulna 9208, Bangladesh
| | - Ashis Talukder
- Statistics Discipline, Science Engineering and Technology School, Khulna University, Khulna 9208, Bangladesh; National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT, 2600, Australia.
| |
Collapse
|
3
|
Somé NH, Noormohammadpour P, Lange S. The use of machine learning on administrative and survey data to predict suicidal thoughts and behaviors: a systematic review. Front Psychiatry 2024; 15:1291362. [PMID: 38501090 PMCID: PMC10944962 DOI: 10.3389/fpsyt.2024.1291362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/09/2023] [Accepted: 02/12/2024] [Indexed: 03/20/2024] Open
Abstract
Background Machine learning is a promising tool in the area of suicide prevention due to its ability to combine the effects of multiple risk factors and complex interactions. The power of machine learning has led to an influx of studies on suicide prediction, as well as a few recent reviews. Our study distinguished between data sources and reported the most important predictors of suicide outcomes identified in the literature. Objective Our study aimed to identify studies that applied machine learning techniques to administrative and survey data, summarize performance metrics reported in those studies, and enumerate the important risk factors of suicidal thoughts and behaviors identified. Methods A systematic literature search of PubMed, Medline, Embase, PsycINFO, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Allied and Complementary Medicine Database (AMED) to identify all studies that have used machine learning to predict suicidal thoughts and behaviors using administrative and survey data was performed. The search was conducted for articles published between January 1, 2019 and May 11, 2022. In addition, all articles identified in three recently published systematic reviews (the last of which included studies up until January 1, 2019) were retained if they met our inclusion criteria. The predictive power of machine learning methods in predicting suicidal thoughts and behaviors was explored using box plots to summarize the distribution of the area under the receiver operating characteristic curve (AUC) values by machine learning method and suicide outcome (i.e., suicidal thoughts, suicide attempt, and death by suicide). Mean AUCs with 95% confidence intervals (CIs) were computed for each suicide outcome by study design, data source, total sample size, sample size of cases, and machine learning methods employed. The most important risk factors were listed. Results The search strategy identified 2,200 unique records, of which 104 articles met the inclusion criteria. Machine learning algorithms achieved good prediction of suicidal thoughts and behaviors (i.e., an AUC between 0.80 and 0.89); however, their predictive power appears to differ across suicide outcomes. The boosting algorithms achieved good prediction of suicidal thoughts, death by suicide, and all suicide outcomes combined, while neural network algorithms achieved good prediction of suicide attempts. The risk factors for suicidal thoughts and behaviors differed depending on the data source and the population under study. Conclusion The predictive utility of machine learning for suicidal thoughts and behaviors largely depends on the approach used. The findings of the current review should prove helpful in preparing future machine learning models using administrative and survey data. Systematic review registration https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022333454 identifier CRD42022333454.
Collapse
Affiliation(s)
- Nibene H. Somé
- Institute for Mental Health Policy Research, Centre for Addiction and Mental Health, Toronto, ON, Canada
- Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, ON, Canada
- Department of Epidemiology and Biostatistics, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Pardis Noormohammadpour
- Institute for Mental Health Policy Research, Centre for Addiction and Mental Health, Toronto, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Shannon Lange
- Institute for Mental Health Policy Research, Centre for Addiction and Mental Health, Toronto, ON, Canada
- Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, ON, Canada
- Department of Psychiatry, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
4
|
Kim EY, Jeong YW, Lim J, Kang DR. Factors Related to Suicidal Ideation by Gender and Age Group in Korean Adults. J Korean Med Sci 2023; 38:e376. [PMID: 38111279 PMCID: PMC10727917 DOI: 10.3346/jkms.2023.38.e376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 09/14/2023] [Indexed: 12/20/2023] Open
Abstract
BACKGROUND According to the results of the age-standardized suicide rate (suicide deaths per 100,000 population) published by Organization for Economic Cooperation and Development countries from 2003 to 2019, the suicide rate in Korea is ranked first almost every year, so it should be managed at the national level. To reduce the suicide rate, many scholars are studying suicide. The suicidal process begins with suicidal ideation, progresses through suicide attempts, and ends with suicide. Among them, the frequency of suicidal thoughts was found to be highly correlated with suicide plans and attempts, and it is said that 60% of those who change from suicidal ideation to attempts appear within one year. Therefore, research related to suicidal ideation to lower the suicide rate will contribute to preventing suicide at an early stage. METHODS This study used data from the Korea National Health and Nutrition Examination Survey (KNHANES). Among them, data from 2013, 2015, 2017, and 2019 surveyed adults were compared for suicidal ideation among 18,339 adults. Considering the characteristics of the KNHANES data, complex sample analysis was performed considering the primary extraction unit (region), stratification variables, and weights. The Rao-Scott χ² test was stratified by age group to confirm the distribution of suicidal ideation according to general characteristics. Univariate and multiple logistic regression analyses were performed to understand the effect on suicidal ideation. RESULTS Among all subjects, the rate of suicidal ideation was 4.75%, and among the age groups, the rate of suicidal thoughts was the highest among those over 80 years old (10.39%). Compared to those who were 50s, those in their 30s had the lowest suicidal ideation (odds ratio [OR], 0.52; 95% confidence interval [CI], 0.39-0.70), and those in their 80s had the highest rate of suicidal ideation (OR, 1.97; 95% CI, 1.38-2.82). Except for those in their 20s, the rate of suicidal ideation tends to increase with age. Overall, the lower 25% of the Euro Quality of Life-5 Dimensions (EQ-5D) index, depression lasting more than two weeks, and subjects with poor subjective health or high stress were more likely to have suicidal ideation. In particular, EQ-5D Index (OR, 5.86; 95% CI, 3.73-9.20), depressive symptoms (OR, 19.65; 95% CI, 9.94-38.83) in 20s, and stress in 80s (OR, 10.87; 95% CI, 5.63-20.96) was the highest, and those had the good subjective health perception in 30s (OR, 0.10; 95% CI, 0.05-0.20) was lowest. Participants in their 30s, 40s, and 60s who were divorced or widowed and those in their 50s and 60s who had never married or lived together were more likely to consider suicide ideation. Except for those in their 20s and 80s, the rate of suicidal ideation tends to increase with lower household income levels. CONCLUSIONS In the results of this study, it can be seen that a combination of factors such as a person's personality traits, stress or coping ability to various events, social support, and mental or physical limitations influence suicidal behavior. Suicidal ideation is not simply affected by a single factor but may be influenced by multiple factors.
Collapse
Affiliation(s)
- Eun Young Kim
- Department of Biomedical Laboratory Science, Songho University, Hoengseong, Korea
| | - Yong Whi Jeong
- Department of Medical Informatics and Biostatistics, Graduate School, Yonsei University, Seoul, Korea
| | - Jihye Lim
- Department of Medical Informatics and Biostatistics, Graduate School, Yonsei University, Seoul, Korea
| | - Dae Ryong Kang
- Department of Precision Medicine, Wonju College of Medicine, Yonsei University, Wonju, Korea.
| |
Collapse
|
5
|
Nielsen SD, Christensen RHB, Madsen T, Karstoft KI, Clemmensen L, Benros ME. Prediction models of suicide and non-fatal suicide attempt after discharge from a psychiatric inpatient stay: A machine learning approach on nationwide Danish registers. Acta Psychiatr Scand 2023; 148:525-537. [PMID: 37961014 DOI: 10.1111/acps.13629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 10/06/2023] [Accepted: 10/13/2023] [Indexed: 11/15/2023]
Abstract
INTRODUCTION To develop machine learning models capable of predicting suicide and non-fatal suicide attempt as separate outcomes in the first 30 days after discharge from a psychiatric inpatient stay. METHODS Prospective cohort study using nationwide Danish registry data. We included individuals who were 18 years or older, and all discharges from psychiatric hospitalizations in Denmark from 1995 to 2018. We trained predictive models using 10-fold cross validation on 80% of the data and did testing on the remaining 20%. RESULTS The best model for predicting non-fatal suicide attempt was an ensemble of predictions from gradient boosting (XGBoost) and categorical boosting (catBoost). The ROC-AUC for predicting suicide attempt was 0.85 (95% CI: 0.84-0.85). At a risk threshold of 4.36%, positive predictive value (PPV) was 11.0% and sensitivity was 47.2%. The best model for predicting suicide was an ensemble of predictions from random forest, XGBoost and catBoost. For suicide, the ROC-AUC was 0.71 (95% CI: 0.70-0.73). At a risk threshold of 0.15%, PPV was 0.34% and sensitivity was 56.0%. The most contributing predictors differed when predicting suicide and suicide attempt, indicating that separate models are needed. The ensemble model was fair across sex and age, and more so than the penalized logistic regression model. CONCLUSIONS We achieved good performance for predicting suicide attempts and demonstrated a clinical application of ensemble models. Our results indicate a difference in predictive performance for models predicting suicide and suicide attempt, respectively. Thus, we recommend that suicide and suicide attempt are treated as two separate endpoints, in particular for clinical application. We demonstrated that the ensemble model is fairer across sex and age compared with a penalized logistic regression, and therefore we recommend the use of well-tested ensembles despite a more complex explainability.
Collapse
Affiliation(s)
- Sara Dorthea Nielsen
- Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kgs. Lyngby, Denmark
- Copenhagen Research Center for Biological and Precision Psychiatry, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark
| | - Rune H B Christensen
- Copenhagen Research Center for Biological and Precision Psychiatry, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark
| | - Trine Madsen
- Danish Research Institute of Suicide Prevention, Mental Health Center Copenhagen, Copenhagen, Denmark
- Section of Epidemiology, University of Copenhagen, Faculty of Health and Medical Sciences, Copenhagen, Denmark
| | - Karen-Inge Karstoft
- Department of Psychology, Faculty of Social Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Line Clemmensen
- Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kgs. Lyngby, Denmark
| | - Michael E Benros
- Copenhagen Research Center for Biological and Precision Psychiatry, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
6
|
Su R, John JR, Lin PI. Machine learning-based prediction for self-harm and suicide attempts in adolescents. Psychiatry Res 2023; 328:115446. [PMID: 37683319 DOI: 10.1016/j.psychres.2023.115446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Revised: 08/24/2023] [Accepted: 08/27/2023] [Indexed: 09/10/2023]
Abstract
This study aimed to use machine learning (ML) models to predict the risk of self-harm and suicide attempts in adolescents. We conducted secondary analysis of cross-sectional data from the Longitudinal Study of Australian Children dataset. Several key variables at the age of 14-15 years were used to predict self-harm or suicide attempt at 16-17 years. Random forest classification models were used to select the optimal subset of predictors and subsequently make predictions. Among 2809 participants, 296 (10.54%) reported an act of self-harm and 145 (5.16%) reported attempting suicide at least once in the past 12 months. The area under the receiver operating curve was fair for self-harm (0.7397) and suicide attempt (0.7220), which outperformed the prediction strategy solely based on prior suicide or self-harm attempt (AUC: 0.6). The most important factors identified were similar, and included depressed feelings, strengths and difficulties questionnaire scores, perceptions of self, and school- and parent-related factors. The random forest classification algorithm, an ML technique, can effectively select the optimal subset of predictors from hundreds of variables to forecast the risks of suicide and self-harm among adolescents. Further research is needed to validate the utility and scalability of ML techniques in mental health research.
Collapse
Affiliation(s)
- Raymond Su
- School of Clinical Medicine, University of New South Wales, Sydney, NSW, Australia
| | - James Rufus John
- School of Clinical Medicine, University of New South Wales, Sydney, NSW, Australia; Ingham Institute of Applied Medical Research, Liverpool, NSW, Australia
| | - Ping-I Lin
- School of Clinical Medicine, University of New South Wales, Sydney, NSW, Australia; Academic Unit of Child Psychiatry Services, South Western Sydney Local Health District, Liverpool, NSW, Australia; Department of Mental Health, School of Medicine, Western Sydney University, Penrith, NSW, Australia.
| |
Collapse
|
7
|
Shin S, Kim K. Prediction of suicidal ideation in children and adolescents using machine learning and deep learning algorithm: A case study in South Korea where suicide is the leading cause of death. Asian J Psychiatr 2023; 88:103725. [PMID: 37595385 DOI: 10.1016/j.ajp.2023.103725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Revised: 08/01/2023] [Accepted: 08/04/2023] [Indexed: 08/20/2023]
Abstract
BACKGROUND Korea has the highest suicide rate among Organisation for Economic Co-operation and Development (OECD) countries. Consequently, central and local governments and private organizations in Korea cooperate in promoting various suicide prevention projects to actively respond to suicide problems. Machine learning has been used to predict suicidal ideation in the fields of health and medicine but not from a social science perspective. OBJECTIVE Since suicidal ideation is a major predictor of suicide attempts, being able to anticipate and mitigate it helps prevent suicide. Therefore, this study presents a data-based analysis method for predicting suicidal thoughts quickly and effectively and suggests countermeasures against the causes of suicidal thoughts. PARTICIPANTS AND METHODS To predict early signs of suicidal ideation in children and adolescents, big data collected for approximately 4 years (from 2017 to 2020) from the Korea Youth Policy Institute (NYPI) were used. To accurately predict suicidal ideation, supervised ma- chine learning classification algorithms such as logistic regression, random forest, XGBoost, multilayer perceptron (MLP), and convolutional neural network (CNN) were used. RESULTS Using CNN, suicidal ideation was predicted with an accuracy of approximately 90 %. The logistic regression results showed that sadness and depression increased suicidal thoughts by more than 25 times, and anxiety, loneliness, and experience of abusive language increased suicidal thoughts by more than three times. CONCLUSIONS Machine learning and deep learning approaches have the potential to predict and respond to suicidal thoughts in children, adolescents, and the general population, as well as help respond to the suicide crisis by preemptively identifying the cause.
Collapse
Affiliation(s)
- Soomin Shin
- Department of Health and Welfare, Yuhan University, Bucheon 14780, the Republic of Korea
| | - Kyungwon Kim
- School of International Trade and Business, Incheon National University, Incheon 22012, the Republic of Korea.
| |
Collapse
|
8
|
Kister K, Laskowski J, Makarewicz A, Tarkowski J. Application of artificial intelligence tools in diagnosis and treatmentof mental disorders. CURRENT PROBLEMS OF PSYCHIATRY 2023. [DOI: 10.12923/2353-8627/2023-0001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023] Open
Abstract
Introduction: Artificial intelligence research is increasing its application in mental health services. Machine learning, deep learning, semantic analysis in the form of transcriptions of patients' statements enable early diagnosis of psychotic disorders, ADHD, anorexia nervosa. Of great importance are the so-called digital therapists. This paper aims to show the use of AI tools in diagnosing, treating, the benefits and limitations associated with mental disorders.
Material and methodS: This literature review was conducted by searching scientific articles from 2015 to 2022. The basis were PubMED, OpenKnowledge, Web of Science, using the following keywords: artificial intelligence, digital therapy, psychiatry, machine learning.
Results: A review indicates the widespread use of AI tools in screening for mental disorders. These tools advance the clinical diagnosis medical specialists make up for several years. They impact solving medical staff shortages, lack of access to medical facilities and leveling patient resistance to treatment. The benefits are ultra-fast analysis of large sets of information, effective screening of people in need of specialized psychiatric care, reduction of doctors' duties and maximization of their work efficiency. During the current COVID 19 pandemic, robots in the form of digital psychotherapists are playing a special role.
Conclusions: The need for further research, testing and clarification of regulations related to the use of AI tools is indicated. Ethical and social problems need to be resolved. The tools should not form the basis of autonomous therapy without the supervision of highly trained professionals. Human beings should be at the center of analysis just as their health and well-being.
Keywords: artificial intelligence, digital therapy, psychiatry, machine learning
Collapse
Affiliation(s)
- Klaudia Kister
- I Departmentof Psychiatry, Psychoterapy and Early Intervention of Medical University in Lublin, Poland - Students Research Group
| | - Jakub Laskowski
- Department of Paediatrician Oncology, Transplantology and Haematology of Medical University in Lublin, Medical University in Lublin, Poland - Students Research Group
| | - Agata Makarewicz
- I Department of Psychiatry, Psychoterapy and Early Intervention of Medical University in Lublin, Poland
| | | |
Collapse
|
9
|
Nordin N, Zainol Z, Mohd Noor MH, Chan LF. An explainable predictive model for suicide attempt risk using an ensemble learning and Shapley Additive Explanations (SHAP) approach. Asian J Psychiatr 2023; 79:103316. [PMID: 36395702 DOI: 10.1016/j.ajp.2022.103316] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Revised: 10/20/2022] [Accepted: 11/04/2022] [Indexed: 11/09/2022]
Abstract
Machine learning approaches have been used to develop suicide attempt predictive models recently and have been shown to have a good performance. However, those proposed models have difficulty interpreting and understanding why an individual has suicidal attempts. To overcome this issue, the identification of features such as risk factors in predicting suicide attempts is important for clinicians to make decisions. Therefore, the aim of this study is to propose an explainable predictive model to predict and analyse the importance of features for suicide attempts. This model can also provide explanations to improve the clinical understanding of suicide attempts. Two complex ensemble learning models, namely Random Forest and Gradient Boosting with an explanatory model (SHapley Additive exPlanations (SHAP)) have been constructed. The models are used for predictive interpretation and understanding of the importance of the features. The experiment shows that both models with SHAP are able to interpret and understand the nature of an individual's predictions with suicide attempts. However, compared with Random Forest, the results show that Gradient Boosting with SHAP achieves higher accuracy and the analyses found that history of suicide attempts, suicidal ideation, and ethnicity as the main predictors for suicide attempts.
Collapse
Affiliation(s)
- Noratikah Nordin
- School of Computer Sciences, Universiti Sains Malaysia, 11800 USM, Pulau Pinang, Malaysia.
| | - Zurinahni Zainol
- School of Computer Sciences, Universiti Sains Malaysia, 11800 USM, Pulau Pinang, Malaysia.
| | - Mohd Halim Mohd Noor
- School of Computer Sciences, Universiti Sains Malaysia, 11800 USM, Pulau Pinang, Malaysia.
| | - Lai Fong Chan
- Department of Psychiatry, Faculty of Medicine, National University of Malaysia (UKM), 56000 Cheras, Wilayah Persekutuan Kuala Lumpur, Malaysia.
| |
Collapse
|
10
|
Aldhyani THH, Alsubari SN, Alshebami AS, Alkahtani H, Ahmed ZAT. Detecting and Analyzing Suicidal Ideation on Social Media Using Deep Learning and Machine Learning Models. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:12635. [PMID: 36231935 PMCID: PMC9565132 DOI: 10.3390/ijerph191912635] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Revised: 09/28/2022] [Accepted: 09/30/2022] [Indexed: 06/16/2023]
Abstract
Individuals who suffer from suicidal ideation frequently express their views and ideas on social media. Thus, several studies found that people who are contemplating suicide can be identified by analyzing social media posts. However, finding and comprehending patterns of suicidal ideation represent a challenging task. Therefore, it is essential to develop a machine learning system for automated early detection of suicidal ideation or any abrupt changes in a user's behavior by analyzing his or her posts on social media. In this paper, we propose a methodology based on experimental research for building a suicidal ideation detection system using publicly available Reddit datasets, word-embedding approaches, such as TF-IDF and Word2Vec, for text representation, and hybrid deep learning and machine learning algorithms for classification. A convolutional neural network and Bidirectional long short-term memory (CNN-BiLSTM) model and the machine learning XGBoost model were used to classify social posts as suicidal or non-suicidal using textual and LIWC-22-based features by conducting two experiments. To assess the models' performance, we used the standard metrics of accuracy, precision, recall, and F1-scores. A comparison of the test results showed that when using textual features, the CNN-BiLSTM model outperformed the XGBoost model, achieving 95% suicidal ideation detection accuracy, compared with the latter's 91.5% accuracy. Conversely, when using LIWC features, XGBoost showed better performance than CNN-BiLSTM.
Collapse
Affiliation(s)
- Theyazn H. H. Aldhyani
- Applied College in Abqaiq, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia
| | - Saleh Nagi Alsubari
- Department of Computer Science, Dr. Babasaheb Ambedkar Marathwada University, Aurangabad 431004, India
| | - Ali Saleh Alshebami
- Applied College in Abqaiq, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia
| | - Hasan Alkahtani
- College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia
| | - Zeyad A. T. Ahmed
- Department of Computer Science, Dr. Babasaheb Ambedkar Marathwada University, Aurangabad 431004, India
| |
Collapse
|
11
|
Nordin N, Zainol Z, Mohd Noor MH, Chan LF. Suicidal behaviour prediction models using machine learning techniques: A systematic review. Artif Intell Med 2022; 132:102395. [DOI: 10.1016/j.artmed.2022.102395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2022] [Revised: 08/12/2022] [Accepted: 08/29/2022] [Indexed: 11/02/2022]
|
12
|
Lee J, Pak TY. Machine learning prediction of suicidal ideation, planning, and attempt among Korean adults: A population-based study. SSM Popul Health 2022; 19:101231. [PMID: 36263295 PMCID: PMC9573904 DOI: 10.1016/j.ssmph.2022.101231] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Revised: 09/09/2022] [Accepted: 09/13/2022] [Indexed: 11/18/2022] Open
Abstract
Background Suicide remains the leading cause of premature death in South Korea. This study aims to develop machine learning algorithms for screening Korean adults at risk for suicidal ideation and suicide planning or attempt. Methods Two sets of balanced data for Korean adults aged 19–64 years were drawn from the 2012–2019 waves of the Korea Welfare Panel Study using the random down-sampling method (N = 3292 for the prediction of suicidal ideation, N = 488 for the prediction of suicide planning or attempt). Demographic, socioeconomic, and psychosocial characteristics were used to predict suicidal ideation and suicide planning or attempt. Four machine-learning classifiers (logistic regression, random forest, support vector machine, and extreme gradient boosting) were tuned and cross-validated. Results All four algorithms demonstrated satisfactory classification performance in predicting suicidal ideation (sensitivity 0.808–0.853, accuracy 0.843–0.863) and suicide planning or attempt (sensitivity 0.814–0.861, accuracy 0.864–0.884). Extreme gradient boosting was the best-performing algorithm for predicting both suicidal outcomes. The most important predictors were depressive symptoms, self-esteem, income, consumption, and life satisfaction. The algorithms trained with the top two predictors, depressive symptoms and self-esteem, showed comparable classification performance in predicting suicidal ideation (sensitivity 0.801–0.839, accuracy 0.841–0.846) and suicide planning or attempt (sensitivity 0.814–0.837, accuracy 0.874–0.884). Limitations Suicidal ideation and behaviors may be under-reported due to social desirability bias. Causality is not established. Discussion More than 80% of individuals at risk for suicidal ideation and suicide planning or attempt could be predicted by a number of mental and socioeconomic characteristics of respondents. This finding suggests the potential of developing a quick screening tool based on the known risk factors and applying it to primary care or community settings for early intervention. This study develops machine learning models to predict suicidal ideation and behaviors. Logistic regression, random forest, support vector machine, and extreme gradient boosting are used. The algorithms correctly identifyed 80–90% of suicidal cases. The algorithms with the top two predictors (depressive symptoms and self-esteem) could achieve comparable accuracy. Our findings can be used to design a quick screening tool for use in primary care or community settings.
Collapse
Affiliation(s)
- Jeongyoon Lee
- Convergence Program for Social Innovation, Sungkyunkwan University, Seoul, South Korea
| | - Tae-Young Pak
- Department of Consumer Science and Convergence Program for Social Innovation, Sungkyunkwan University, Seoul, South Korea
- Corresponding author.
| |
Collapse
|
13
|
Machine learning for suicidal ideation identification: A systematic literature review. COMPUTERS IN HUMAN BEHAVIOR 2022. [DOI: 10.1016/j.chb.2021.107095] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
14
|
Colic S, He JC, Richardson JD, Cyr KS, Reilly JP, Hasey GM. A machine learning approach to identification of self-harm and suicidal ideation among military and police Veterans. JOURNAL OF MILITARY, VETERAN AND FAMILY HEALTH 2022. [DOI: 10.3138/jmvfh-2021-0035] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
LAY SUMMARY Combat Veterans are vulnerable to suicidal thoughts and behaviour. Many who die by suicide deny having suicidal ideation (SI). Typically, researchers try to find variables indicating the presence of SI using traditional statistical approaches. These approaches do not possess the capacity to detect highly complex multivariable interactions. In contrast, machine learning (ML) is designed to detect such patterns and can consequently yield much higher predictive accuracy. In this study, the authors trained ML algorithms using 192 variables extracted from questionnaires administered to 738 Veterans and serving personnel to detect the presence of self-harm and SI (SHSI). Using the 10 most predictive non-suicide-related items, the ML algorithms could detect SHSI with 75.3% accuracy. Most of these items reflect psychological phenomena that can change quickly over time, allowing repeated risk reassessment from day to day. The study’s findings suggest that ML methods may play an important role in the discovery, within a large data set, of predictive patterns that might be useful in suicide risk assessment.
Collapse
Affiliation(s)
- Sinisa Colic
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada
| | - Jiang Chen He
- Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada
| | - J. Don Richardson
- St. Joseph’s Operational Stress Injury Clinic, St. Joseph’s Health Care London, London, Ontario, Canada
| | - Kate St. Cyr
- MacDonald Franklin Operational Stress Injury Research Centre, St. Joseph’s Health Care London, London, Ontario, Canada
| | - James P. Reilly
- Department of Electrical and Computer Engineering, McMaster University, Hamilton, Ontario, Canada
| | - Gary M. Hasey
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
15
|
Na KS, Geem ZW, Cho SE. The Development of a Suicidal Ideation Predictive Model for Community-Dwelling Elderly Aged >55 Years. Neuropsychiatr Dis Treat 2022; 18:163-172. [PMID: 35140466 PMCID: PMC8819701 DOI: 10.2147/ndt.s336947] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Accepted: 01/20/2022] [Indexed: 11/23/2022] Open
Abstract
PURPOSE Suicide is an important health and social concern worldwide. Both suicidal ideation and suicide rates are higher in the elderly population than in other age groups; thus, more careful attention and targeted interventions are required. Therefore, we have developed a model to predict suicidal ideation in the community-dwelling elderly aged of >55 years. PATIENTS AND METHODS A random forest algorithm was applied to those who participated in the Korea Welfare Panel. We used a total of 26 variables as potential predictors. To resolve the imbalance in the dataset resulting from the low frequency of suicidal ideation, training was performed by applying the synthetic minority oversampling technique. The performance index was calculated by applying the predictive model to the test set, which was not included in the training process. RESULTS A total of 6410 elderly Korean aged of >55 (mean, 71.48; standard deviation, 9.56) years were included in the analysis, of which 2.7% had suicidal ideation. The results for predicting suicidal ideation using the 26 chosen variables showed an AUC of 0.879, accuracy of 0.871, sensitivity of 0.750, and specificity of 0.874. The most significant variable in the predictive model was the severity of depression, followed by life satisfaction and self-esteem factors. Basic demographic variables such as age and gender demonstrated a relatively small effect. CONCLUSION Machine learning can be used to create algorithms for predicting suicidal ideation in community-dwelling elderly. However, there are limitations to predicting future suicidal ideation. A predictive model that includes both biological and cognitive indicators should be created in the future.
Collapse
Affiliation(s)
- Kyoung-Sae Na
- Department of Psychiatry, Gachon University College of Medicine, Incheon, 21565, Republic of Korea
| | - Zong Woo Geem
- College of IT Convergence, Gachon University, Seongnam, 13120, Republic of Korea
| | - Seo-Eun Cho
- Department of Psychiatry, Gil Medical Center, Incheon, 21565, Republic of Korea
| |
Collapse
|
16
|
Chou PH, Wang SC, Wu CS, Horikoshi M, Ito M. A machine-learning model to predict suicide risk in Japan based on national survey data. Front Psychiatry 2022; 13:918667. [PMID: 35990064 PMCID: PMC9387201 DOI: 10.3389/fpsyt.2022.918667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 07/11/2022] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE Several prognostic models of suicide risk have been published; however, few have been implemented in Japan using longitudinal cohort data. The aim of this study was to identify suicide risk factors for suicidal ideation in the Japanese population and to develop a machine-learning model to predict suicide risk in Japan. MATERIALS AND METHODS Data was obtained from Wave1 Time 1 (November 2016) and Time 2 (March 2017) of the National Survey for Stress and Health in Japan, were incorporated into a suicide risk prediction machine-learning model, trained using 65 items related to trauma and stress. The study included 3,090 and 2,163 survey respondents >18 years old at Time 1 and Time 2, respectively. The mean (standard deviation, SD) age was 44.9 (10.9) years at Time 1 and 46.0 (10.7) years at Time 2. We analyzed the participants with increased suicide risk at Time 2 survey. Model performance, including the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, were also analyzed. RESULTS The model showed a good performance (AUC = 0.830, 95% confidence interval = 0.795-0.866). Overall, the model achieved an accuracy of 78.8%, sensitivity of 75.4%, specificity of 80.4%, positive predictive value of 63.4%, and negative predictive value of 87.9%. The most important risk factor for suicide risk was the participants' Suicidal Ideation Attributes Scale score, followed by the Sheehan Disability Scale score, Patient Health Questionnaire-9 scores, Cross-Cutting Symptom Measure (CCSM-suicidal ideation domain, Dissociation Experience Scale score, history of self-harm, Generalized Anxiety Disorder-7 score, Post-Traumatic Stress Disorder check list-5 score, CCSM-dissociation domain, and Impact of Event Scale-Revised scores at Time 1. CONCLUSIONS This prognostic study suggests the ability to identify patients at a high risk of suicide using an online survey method. In addition to confirming several well-known risk factors of suicide, new risk measures related to trauma and trauma-related experiences were also identified, which may help guide future clinical assessments and early intervention approaches.
Collapse
Affiliation(s)
- Po-Han Chou
- Department of Psychiatry, China Medical University Hsinchu Hospital, China Medical University, Hsinchu, Taiwan.,Department of Psychiatry, China Medical University Hospital, China Medical University, Taichung, Taiwan
| | - Shao-Cheng Wang
- Department of Psychiatry, Taoyuan General Hospital, Ministry of Health and Welfare, Taoyuan, Taiwan.,Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States.,Department of Medical Laboratory Science and Biotechnology, Chung Hwa University of Medical Technology, Tainan, Taiwan
| | - Chi-Shin Wu
- National Center for Geriatrics and Welfare Research, National Health Research Institutes, Zhunan Town, Yunlin County, Taiwan.,Department of Psychiatry, National Taiwan University Hospital, Douliu, Taiwan
| | - Masaru Horikoshi
- National Center for Cognitive-Behavior Therapy and Research, National Center of Neurology and Psychiatry, Tokyo, Japan
| | - Masaya Ito
- National Center for Cognitive-Behavior Therapy and Research, National Center of Neurology and Psychiatry, Tokyo, Japan
| |
Collapse
|
17
|
Comparing machine learning to a rule-based approach for predicting suicidal behavior among adolescents: Results from a longitudinal population-based survey. J Affect Disord 2021; 295:1415-1420. [PMID: 34620490 DOI: 10.1016/j.jad.2021.09.018] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 07/22/2021] [Accepted: 09/12/2021] [Indexed: 11/21/2022]
Abstract
INTRODUCTION Suicidal thoughts and suicide attempts are one of the most prominent public health concerns in adolescents and therefore early detection is important to initiate preventive interventions and closer monitoring. METHOD We examined whether the Machine Learning models Random Forest and Lasso Regression better predict future suicidal behavior than a simple decision rule that classifies every adolescent with history of suicide ideation at baseline as at risk (current practice). We used data from a general population of students in second and fourth year of secondary education in Amsterdam, the Netherlands. RESULTS Both the Random Forest and the Lasso Regression resulted in slightly better prediction. The AUC of the Random Forest (0.79) and Lasso regression (0.76) were both higher than the AUC of the decision rule (0.64). The Random Forest achieved slightly (but non-significantly) higher sensitivity than the decision rule (0.37 versus 0.34), with the same specificity (0.94). With Lasso Regression the sensitivity increased significantly (0.52), but at the expense of the specificity (0.85). LIMITATIONS The loss of cases after merging the data, the use of self-reported data, confidential data collection and the use of only four questions to measure suicidal behavior. CONCLUSIONS This is the first study applying Machine Learning techniques to predict future suicidal behavior on survey data collected in a general population of adolescents. Our study showed that integrating machine learning techniques in screening practice will result in a small improvement in the ability to predict suicide. The models need to be further optimized to improve accuracy.
Collapse
|
18
|
Nordin N, Zainol Z, Mohd Noor MH, Lai Fong C. A comparative study of machine learning techniques for suicide attempts predictive model. Health Informatics J 2021; 27:1460458221989395. [PMID: 33745355 DOI: 10.1177/1460458221989395] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Current suicide risk assessments for predicting suicide attempts are time consuming, of low predictive value and have inadequate reliability. This paper aims to develop a predictive model for suicide attempts among patients with depression using machine learning algorithms as well as presents a comparative study on single predictive models with ensemble predictive models for differentiating depressed patients with suicide attempts from non-suicide attempters. We applied and trained eight different machine learning algorithms using a dataset that consists of 75 patients diagnosed with a depressive disorder. A recursive feature elimination was used to reduce the features via three-fold cross validation. An ensemble predictive models outperformed the single predictive models. Voting and bagging revealed the highest accuracy of 92% compared to other machine learning algorithms. Our findings indicate that history of suicide attempt, religion, race, suicide ideation and severity of clinical depression are useful factors for prediction of suicide attempts.
Collapse
Affiliation(s)
| | | | | | - Chan Lai Fong
- National University of Malaysia Medical Centre, Malaysia
| |
Collapse
|
19
|
Kim S, Lee HK, Lee K. Detecting suicidal risk using MMPI-2 based on machine learning algorithm. Sci Rep 2021; 11:15310. [PMID: 34321546 PMCID: PMC8319391 DOI: 10.1038/s41598-021-94839-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2021] [Accepted: 07/13/2021] [Indexed: 12/25/2022] Open
Abstract
Minnesota Multiphasic Personality Inventory-2 (MMPI-2) is a widely used tool for early detection of psychological maladjustment and assessing the level of adaptation for a large group in clinical settings, schools, and corporations. This study aims to evaluate the utility of MMPI-2 in assessing suicidal risk using the results of MMPI-2 and suicidal risk evaluation. A total of 7,824 datasets collected from college students were analyzed. The MMPI-2-Resturcutred Clinical Scales (MMPI-2-RF) and the response results for each question of the Mini International Neuropsychiatric Interview (MINI) suicidality module were used. For statistical analysis, random forest and K-Nearest Neighbors (KNN) techniques were used with suicidal ideation and suicide attempt as dependent variables and 50 MMPI-2 scale scores as predictors. On applying the random forest method to suicidal ideation and suicidal attempts, the accuracy was 92.9% and 95%, respectively, and the Area Under the Curves (AUCs) were 0.844 and 0.851, respectively. When the KNN method was applied, the accuracy was 91.6% and 94.7%, respectively, and the AUCs were 0.722 and 0.639, respectively. The study confirmed that machine learning using MMPI-2 for a large group provides reliable accuracy in classifying and predicting the subject's suicidal ideation and past suicidal attempts.
Collapse
Affiliation(s)
- Sunhae Kim
- Department of Psychiatry, Hanyang University Medical Center, 222-1, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Republic of Korea
| | - Hye-Kyung Lee
- Department of Nursing, College of Nursing and Health, Kongju National University, Gongju, Republic of Korea
| | - Kounseok Lee
- Department of Psychiatry, Hanyang University Medical Center, 222-1, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Republic of Korea.
| |
Collapse
|
20
|
Which PHQ-9 Items Can Effectively Screen for Suicide? Machine Learning Approaches. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18073339. [PMID: 33804879 PMCID: PMC8036742 DOI: 10.3390/ijerph18073339] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/21/2021] [Revised: 03/21/2021] [Accepted: 03/22/2021] [Indexed: 12/21/2022]
Abstract
(1) Background: The Patient Health Questionnaire-9 (PHQ-9) is a tool that screens patients for depression in primary care settings. In this study, we evaluated the efficacy of PHQ-9 in evaluating suicidal ideation (2) Methods: A total of 8760 completed questionnaires collected from college students were analyzed. The PHQ-9 was scored in combination with and evaluated against four categories (PHQ-2, PHQ-8, PHQ-9, and PHQ-10). Suicidal ideations were evaluated using the Mini-International Neuropsychiatric Interview suicidality module. Analyses used suicide ideation as the dependent variable, and machine learning (ML) algorithms, k-nearest neighbors, linear discriminant analysis (LDA), and random forest. (3) Results: Random forest application using the nine items of the PHQ-9 revealed an excellent area under the curve with a value of 0.841, with 94.3% accuracy. The positive and negative predictive values were 84.95% (95% CI = 76.03–91.52) and 95.54% (95% CI = 94.42–96.48), respectively. (4) Conclusion: This study confirmed that ML algorithms using PHQ-9 in the primary care field are reliably accurate in screening individuals with suicidal ideation.
Collapse
|
21
|
Mansourian M, Khademi S, Marateb HR. A Comprehensive Review of Computer-Aided Diagnosis of Major Mental and Neurological Disorders and Suicide: A Biostatistical Perspective on Data Mining. Diagnostics (Basel) 2021; 11:393. [PMID: 33669114 PMCID: PMC7996506 DOI: 10.3390/diagnostics11030393] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Revised: 02/13/2021] [Accepted: 02/17/2021] [Indexed: 02/07/2023] Open
Abstract
The World Health Organization (WHO) suggests that mental disorders, neurological disorders, and suicide are growing causes of morbidity. Depressive disorders, schizophrenia, bipolar disorder, Alzheimer's disease, and other dementias account for 1.84%, 0.60%, 0.33%, and 1.00% of total Disability Adjusted Life Years (DALYs). Furthermore, suicide, the 15th leading cause of death worldwide, could be linked to mental disorders. More than 68 computer-aided diagnosis (CAD) methods published in peer-reviewed journals from 2016 to 2021 were analyzed, among which 75% were published in the year 2018 or later. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol was adopted to select the relevant studies. In addition to the gold standard, the sample size, neuroimaging techniques or biomarkers, validation frameworks, the classifiers, and the performance indices were analyzed. We further discussed how various performance indices are essential based on the biostatistical and data mining perspective. Moreover, critical information related to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines was analyzed. We discussed how balancing the dataset and not using external validation could hinder the generalization of the CAD methods. We provided the list of the critical issues to consider in such studies.
Collapse
Affiliation(s)
- Mahsa Mansourian
- Department of Medical Physics, School of Medicine, Isfahan University of Medical Sciences, Isfahan 81746-73461, Iran;
| | - Sadaf Khademi
- Biomedical Engineering Department, Faculty of Engineering, University of Isfahan, Isfahan 8174-67344, Iran;
| | - Hamid Reza Marateb
- Biomedical Engineering Department, Faculty of Engineering, University of Isfahan, Isfahan 8174-67344, Iran;
| |
Collapse
|
22
|
Kim S, Lee K. Screening for Depression in Mobile Devices Using Patient Health Questionnaire-9 (PHQ-9) Data: A Diagnostic Meta-Analysis via Machine Learning Methods. Neuropsychiatr Dis Treat 2021; 17:3415-3430. [PMID: 34848962 PMCID: PMC8612669 DOI: 10.2147/ndt.s339412] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Accepted: 11/02/2021] [Indexed: 12/25/2022] Open
Abstract
PURPOSE Depression is a symptom commonly encountered in primary care; however, it is often not detected by doctors. Recently, disease diagnosis and treatment approaches have been attempted using smart devices. In this study, instrumental effectiveness was confirmed with the diagnostic meta-analysis of studies that demonstrated the diagnostic effectiveness of PHQ-9 for depression using mobile devices. PATIENTS AND METHODS We found all published and unpublished studies through EMBASE, MEDLINE, MEDLINE In-Process, and PsychINFO up to March 26, 2021. We performed a meta-analysis by including 1099 subjects in four studies. We performed a diagnostic meta-analysis according to the PHQ-9 cut-off score and machine learning algorithm techniques. Quality assessment was conducted using the QUADAS-2 tool. Data on the sensitivity and specificity of the studies included in the meta-analysis were extracted in a standardized format. Bivariate and summary receiver operating characteristic (SROC) curve were constructed using the metandi, midas, metabias, and metareg functions of the Stata algorithm meta-analysis words. RESULTS Using four studies out of the 5476 papers searched, a diagnostic meta-analysis of the PHQ-9 scores of 1099 people diagnosed with depression was performed. The pooled sensitivity and specificity were 0.797 (95% CI = 0.642-0.895) and 0.85 (95% CI = 0.780-0.900), respectively. The diagnostic odds ratio was 22.16 (95% CI = 7.273-67.499). Overall, a good balance was maintained, and no heterogeneity or publication bias was presented. CONCLUSION Through various machine learning algorithm techniques, it was possible to confirm that PHQ-9 depression screening in mobiles is an effective diagnostic tool when integrated into a diagnostic meta-analysis.
Collapse
Affiliation(s)
- Sunhae Kim
- Department of Psychiatry, Hanyang University Medical Center, Seoul, Korea
| | - Kounseok Lee
- Department of Psychiatry, Hanyang University Medical Center, Seoul, Korea
| |
Collapse
|
23
|
Desai S, Tanguay-Sela M, Benrimoh D, Fratila R, Brown E, Perlman K, John A, DelPozo-Banos M, Low N, Israel S, Palladini L, Turecki G. Identification of Suicidal Ideation in the Canadian Community Health Survey-Mental Health Component Using Deep Learning. Front Artif Intell 2021; 4:561528. [PMID: 34250463 PMCID: PMC8264793 DOI: 10.3389/frai.2021.561528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Accepted: 05/25/2021] [Indexed: 11/13/2022] Open
Abstract
Introduction: Suicidal ideation (SI) is prevalent in the general population, and is a risk factor for suicide. Predicting which patients are likely to have SI remains challenging. Deep Learning (DL) may be a useful tool in this context, as it can be used to find patterns in complex, heterogeneous, and incomplete datasets. An automated screening system for SI could help prompt clinicians to be more attentive to patients at risk for suicide. Methods: Using the Canadian Community Health Survey-Mental Health Component, we trained a DL model based on 23,859 survey responses to classify patients with and without SI. Models were created to classify both lifetime SI and SI over the last 12 months. From 582 possible parameters we produced 96- and 21-feature versions of the models. Models were trained using an undersampling procedure that balanced the training set between SI and non-SI; validation was done on held-out data. Results: For lifetime SI, the 96 feature model had an Area under the receiver operating curve (AUC) of 0.79 and the 21 feature model had an AUC of 0.77. For SI in the last 12 months the 96 feature model had an AUC of 0.71 and the 21 feature model had an AUC of 0.68. In addition, sensitivity analyses demonstrated feature relationships in line with existing literature. Discussion: Although further study is required to ensure clinical relevance and sample generalizability, this study is an initial proof of concept for the use of DL to improve identification of SI. Sensitivity analyses can help improve the interpretability of DL models. This kind of model would help start conversations with patients which could lead to improved care and a reduction in suicidal behavior.
Collapse
Affiliation(s)
- Sneha Desai
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
- Aifred Health Inc., Montreal, QC, Canada
| | - Myriam Tanguay-Sela
- Aifred Health Inc., Montreal, QC, Canada
- Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - David Benrimoh
- Aifred Health Inc., Montreal, QC, Canada
- Department of Psychiatry, McGill University, Montreal, QC, Canada
- Faculty of Medicine, McGill University, Montreal, QC, Canada
- *Correspondence: David Benrimoh,
| | | | - Eleanor Brown
- Aifred Health Inc., Montreal, QC, Canada
- Department of Psychological and Brain Sciences, Boston University, Boston, MA, United States
| | - Kelly Perlman
- Aifred Health Inc., Montreal, QC, Canada
- Douglas Mental Health University Institute, Montrea, QC, Canada
| | - Ann John
- Swansea University, Swansea, United Kingdom
| | | | - Nancy Low
- Department of Psychiatry, McGill University, Montreal, QC, Canada
| | | | - Lisa Palladini
- Department of Psychiatry, McGill University, Montreal, QC, Canada
- Faculty of Medicine, McGill University, Montreal, QC, Canada
| | - Gustavo Turecki
- Department of Psychiatry, McGill University, Montreal, QC, Canada
- Douglas Mental Health University Institute, Montrea, QC, Canada
| |
Collapse
|
24
|
Taylor AK, Steeg S, Quinlivan L, Gunnell D, Hawton K, Kapur N. Accuracy of individual and combined risk-scale items in the prediction of repetition of self-harm: multicentre prospective cohort study. BJPsych Open 2020; 7:e2. [PMID: 33261707 PMCID: PMC7791570 DOI: 10.1192/bjo.2020.123] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Revised: 09/18/2020] [Accepted: 09/29/2020] [Indexed: 11/25/2022] Open
Abstract
BACKGROUND Individuals attending emergency departments following self-harm have increased risks of future self-harm. Despite the common use of risk scales in self-harm assessment, there is growing evidence that combinations of risk factors do not accurately identify those at greatest risk of further self-harm and suicide. AIMS To evaluate and compare predictive accuracy in prediction of repeat self-harm from clinician and patient ratings of risk, individual risk-scale items and a scale constructed with top-performing items. METHOD We conducted secondary analysis of data from a five-hospital multicentre prospective cohort study of participants referred to psychiatric liaison services following self-harm. We tested predictive utility of items from five risk scales: Manchester Self-Harm Rule, ReACT Self-Harm Rule, SAD PERSONS, Modified SAD PERSONS, Barratt Impulsiveness Scale and clinician and patient risk estimates. Area under the curve (AUC), sensitivity, specificity, predictive values and likelihood ratios were used to evaluate predictive accuracy, with sensitivity analyses using classification-tree regression. RESULTS A total of 483 self-harm episodes were included, and 145 (30%) were followed by a repeat presentation within 6 months. AUC of individual items ranged from 0.43-0.65. Combining best performing items resulted in an AUC of 0.56. Some individual items outperformed the scale they originated from; no items were superior to clinician or patient risk estimations. CONCLUSIONS No individual or combination of items outperformed patients' or clinicians' ratings. This suggests there are limitations to combining risk factors to predict risk of self-harm repetition. Risk scales should have little role in the management of people who have self-harmed.
Collapse
Affiliation(s)
- Anna Kathryn Taylor
- Centre for Mental Health and Safety, Manchester Academic Health Science Centre, University of Manchester, UK
| | - Sarah Steeg
- Centre for Mental Health and Safety, Manchester Academic Health Science Centre, University of Manchester, UK
| | - Leah Quinlivan
- Centre for Mental Health and Safety, Manchester Academic Health Science Centre, University of Manchester, UK; and NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, UK
| | - David Gunnell
- Department of Population Health Sciences, University of Bristol, UK
| | - Keith Hawton
- Centre for Suicide Research University Department of Psychiatry, Warneford Hospital, UK; and Oxford Health NHS Foundation Trust, Warneford Hospital, UK
| | - Nav Kapur
- Centre for Mental Health and Safety, Manchester Academic Health Science Centre, University of Manchester, UK; and NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, UK
| |
Collapse
|
25
|
Salazar de Pablo G, Studerus E, Vaquerizo-Serrano J, Irving J, Catalan A, Oliver D, Baldwin H, Danese A, Fazel S, Steyerberg EW, Stahl D, Fusar-Poli P. Implementing Precision Psychiatry: A Systematic Review of Individualized Prediction Models for Clinical Practice. Schizophr Bull 2020; 47:284-297. [PMID: 32914178 PMCID: PMC7965077 DOI: 10.1093/schbul/sbaa120] [Citation(s) in RCA: 86] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
BACKGROUND The impact of precision psychiatry for clinical practice has not been systematically appraised. This study aims to provide a comprehensive review of validated prediction models to estimate the individual risk of being affected with a condition (diagnostic), developing outcomes (prognostic), or responding to treatments (predictive) in mental disorders. METHODS PRISMA/RIGHT/CHARMS-compliant systematic review of the Web of Science, Cochrane Central Register of Reviews, and Ovid/PsycINFO databases from inception until July 21, 2019 (PROSPERO CRD42019155713) to identify diagnostic/prognostic/predictive prediction studies that reported individualized estimates in psychiatry and that were internally or externally validated or implemented. Random effect meta-regression analyses addressed the impact of several factors on the accuracy of prediction models. FINDINGS Literature search identified 584 prediction modeling studies, of which 89 were included. 10.4% of the total studies included prediction models internally validated (n = 61), 4.6% models externally validated (n = 27), and 0.2% (n = 1) models considered for implementation. Across validated prediction modeling studies (n = 88), 18.2% were diagnostic, 68.2% prognostic, and 13.6% predictive. The most frequently investigated condition was psychosis (36.4%), and the most frequently employed predictors clinical (69.5%). Unimodal compared to multimodal models (β = .29, P = .03) and diagnostic compared to prognostic (β = .84, p < .0001) and predictive (β = .87, P = .002) models were associated with increased accuracy. INTERPRETATION To date, several validated prediction models are available to support the diagnosis and prognosis of psychiatric conditions, in particular, psychosis, or to predict treatment response. Advancements of knowledge are limited by the lack of implementation research in real-world clinical practice. A new generation of implementation research is required to address this translational gap.
Collapse
Affiliation(s)
- Gonzalo Salazar de Pablo
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,Institute of Psychiatry and Mental Health, Department of Child and Adolescent Psychiatry, Hospital General Universitario Gregorio Marañón School of Medicine, Universidad Complutense, Instituto de Investigación Sanitaria Gregorio Marañón, CIBERSAM, Madrid, Spain,Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
| | - Erich Studerus
- Division of Personality and Developmental Psychology, Department of Psychology, University of Basel, Basel, Switzerland
| | - Julio Vaquerizo-Serrano
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,Institute of Psychiatry and Mental Health, Department of Child and Adolescent Psychiatry, Hospital General Universitario Gregorio Marañón School of Medicine, Universidad Complutense, Instituto de Investigación Sanitaria Gregorio Marañón, CIBERSAM, Madrid, Spain,Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
| | - Jessica Irving
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK
| | - Ana Catalan
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,Department of Psychiatry, Basurto University Hospital, Bilbao, Spain,Mental Health Group, BioCruces Health Research Institute, Bizkaia, Spain,Neuroscience Department, University of the Basque Country UPV/EHU, Leioa, Spain
| | - Dominic Oliver
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK
| | - Helen Baldwin
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK
| | - Andrea Danese
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK,Social, Genetic and Developmental Psychiatry Centre, King’s College London, London, UK,National and Specialist CAMHS Clinic for Trauma, Anxiety, and Depression, South London and Maudsley NHS Foundation Trust, London, UK
| | - Seena Fazel
- Department of Psychiatry, University of Oxford, Oxford, UK
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands,Department of Public Health, Erasmus MC, Rotterdam, the Netherlands
| | - Daniel Stahl
- Biostatistics Department, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
| | - Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-detection Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, 16 De Crespigny Park, London, UK,OASIS Service, South London and Maudsley NHS Foundation Trust, London, UK,Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy,National Institute for Health Research, Maudsley Biomedical Research Centre, South London and Maudsley NHS Foundation Trust, London, UK,To whom correspondence should be addressed; tel: +44-0-20-7848-0900, fax:+44-0-20-7848-0976, e-mail:
| |
Collapse
|
26
|
Bernert RA, Hilberg AM, Melia R, Kim JP, Shah NH, Abnousi F. Artificial Intelligence and Suicide Prevention: A Systematic Review of Machine Learning Investigations. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:E5929. [PMID: 32824149 PMCID: PMC7460360 DOI: 10.3390/ijerph17165929] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Accepted: 07/28/2020] [Indexed: 12/12/2022]
Abstract
Suicide is a leading cause of death that defies prediction and challenges prevention efforts worldwide. Artificial intelligence (AI) and machine learning (ML) have emerged as a means of investigating large datasets to enhance risk detection. A systematic review of ML investigations evaluating suicidal behaviors was conducted using PubMed/MEDLINE, PsychInfo, Web-of-Science, and EMBASE, employing search strings and MeSH terms relevant to suicide and AI. Databases were supplemented by hand-search techniques and Google Scholar. Inclusion criteria: (1) journal article, available in English, (2) original investigation, (3) employment of AI/ML, (4) evaluation of a suicide risk outcome. N = 594 records were identified based on abstract search, and 25 hand-searched reports. N = 461 reports remained after duplicates were removed, n = 316 were excluded after abstract screening. Of n = 149 full-text articles assessed for eligibility, n = 87 were included for quantitative synthesis, grouped according to suicide behavior outcome. Reports varied widely in methodology and outcomes. Results suggest high levels of risk classification accuracy (>90%) and Area Under the Curve (AUC) in the prediction of suicidal behaviors. We report key findings and central limitations in the use of AI/ML frameworks to guide additional research, which hold the potential to impact suicide on broad scale.
Collapse
Affiliation(s)
- Rebecca A. Bernert
- Stanford Suicide Prevention Research Laboratory, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94304, USA
| | - Amanda M. Hilberg
- Stanford Suicide Prevention Research Laboratory, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94304, USA
| | - Ruth Melia
- Stanford Suicide Prevention Research Laboratory, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94304, USA
- Department of Psychology, National University of Ireland, Galway, Ireland
| | - Jane Paik Kim
- Stanford Suicide Prevention Research Laboratory, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94304, USA
| | - Nigam H. Shah
- Department of Medicine, Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, CA 94304, USA
- Informatics, Stanford Center for Clinical and Translational Research, and Education (Spectrum), Stanford University, Stanford CA 94304, USA
| | - Freddy Abnousi
- Facebook, Menlo Park, CA 94025, USA
- Yale University School of Medicine, New Haven, CT 06510, USA
| |
Collapse
|
27
|
Liu D, Yu M, Duncan J, Fondario A, Kharrazi H, Nestadt PS. Discovering the Unclassified Suicide Cases Among Undetermined Drug Overdose Deaths Using Machine Learning Techniques. Suicide Life Threat Behav 2020; 50:333-344. [PMID: 31536175 DOI: 10.1111/sltb.12591] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Accepted: 06/24/2019] [Indexed: 11/30/2022]
Abstract
OBJECTIVE The Centers for Disease Control and Prevention (CDC) monitor accidental and intentional deaths to answer questions that are critical for the development of effective prevention and resource allocation. CDC's National Violent Death Reporting System (NVDRS) is a major innovation in surveillance linking individual-level data from multiple sources. However, suicide underreporting is common, particularly from drug overdose deaths. This study sought to assess machine learning (ML) techniques in quantifying drug overdose suicide underreporting rates. METHODS Clinical, sociodemographic, toxicological, and proximal stressor data on overdose decedents (n = 2,665) were extracted from Utah's NVDRS from 2012 to 2015. The existing well-determined cases were used to train and test our ML models. We assessed and compared multiple machine learning methods including Logistic Regression, Random Forest Classifier, Support Vector Machines, and Artificial Neural Networks. We applied a majority voting methodology to classify undetermined drug overdose deaths. RESULTS Overdose suicide rates were estimated to be underreported by 33% across all years, increasing yearly from 29% in 2012 to 37% in 2015. The overall test accuracies for all models ranged from 92.3% to 94.6%. CONCLUSIONS This research identifies a cost-effective, replicable, and expandable ML-based methodology to estimate the true rates of suicide which may be partially masked during the opioid epidemic.
Collapse
Affiliation(s)
- Daphne Liu
- West High School, Salt Lake City, UT, USA
| | - Mia Yu
- The College, University of Chicago, Chicago, IL, USA
| | | | | | - Hadi Kharrazi
- Center for Population Health IT, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.,Division of Health Sciences Informatics, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Paul S Nestadt
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins School of Medicine, Baltimore, MD, USA.,Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| |
Collapse
|
28
|
Early-life stressful events and suicide attempt in schizophrenia: Machine learning models. Schizophr Res 2020; 218:329-331. [PMID: 31973994 DOI: 10.1016/j.schres.2019.11.061] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Revised: 10/10/2019] [Accepted: 11/30/2019] [Indexed: 11/20/2022]
Abstract
Childhood abuse and neglect predicts suicide attempt. Furthermore, other early-life stressful events may predict lifetime suicide attempt in psychiatric disorders. We assessed 189 schizophrenics for suicide attempt and stressful life events. Early-life stressful events were used as predictors of lifetime suicide attempt in three machine learning models. In our sample, 38% of the schizophrenics had at least one suicide attempt lifetime. The machine learning models provided an overall significant prediction (accuracy range: 62-69%). Childhood sexual molestation and mental illness were important predictors of suicide attempt. Early-life stressful events should be included in models aiming at predicting suicide attempt in schizophrenia.
Collapse
|
29
|
Zheng L, Wang O, Hao S, Ye C, Liu M, Xia M, Sabo AN, Markovic L, Stearns F, Kanov L, Sylvester KG, Widen E, McElhinney DB, Zhang W, Liao J, Ling XB. Development of an early-warning system for high-risk patients for suicide attempt using deep learning and electronic health records. Transl Psychiatry 2020; 10:72. [PMID: 32080165 PMCID: PMC7033212 DOI: 10.1038/s41398-020-0684-2] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/21/2019] [Revised: 11/12/2019] [Accepted: 11/22/2019] [Indexed: 02/05/2023] Open
Abstract
Suicide is the tenth leading cause of death in the United States (US). An early-warning system (EWS) for suicide attempt could prove valuable for identifying those at risk of suicide attempts, and analyzing the contribution of repeated attempts to the risk of eventual death by suicide. In this study we sought to develop an EWS for high-risk suicide attempt patients through the development of a population-based risk stratification surveillance system. Advanced machine-learning algorithms and deep neural networks were utilized to build models with the data from electronic health records (EHRs). A final risk score was calculated for each individual and calibrated to indicate the probability of a suicide attempt in the following 1-year time period. Risk scores were subjected to individual-level analysis in order to aid in the interpretation of the results for health-care providers managing the at-risk cohorts. The 1-year suicide attempt risk model attained an area under the curve (AUC ROC) of 0.792 and 0.769 in the retrospective and prospective cohorts, respectively. The suicide attempt rate in the "very high risk" category was 60 times greater than the population baseline when tested in the prospective cohorts. Mental health disorders including depression, bipolar disorders and anxiety, along with substance abuse, impulse control disorders, clinical utilization indicators, and socioeconomic determinants were recognized as significant features associated with incident suicide attempt.
Collapse
Affiliation(s)
- Le Zheng
- Department of Cardiothoracic Surgery, Stanford University, Stanford, CA, USA
- Clinical and Translational Research Program, Betty Irene Moore Children's Heart Center, Lucile Packard Children's Hospital, Palo Alto, CA, USA
| | | | - Shiying Hao
- Department of Cardiothoracic Surgery, Stanford University, Stanford, CA, USA
- Clinical and Translational Research Program, Betty Irene Moore Children's Heart Center, Lucile Packard Children's Hospital, Palo Alto, CA, USA
| | - Chengyin Ye
- Department of Health Management, Hangzhou Normal University, Hangzhou, China
| | - Modi Liu
- HBI Solutions Inc, Palo Alto, CA, USA
| | | | - Alex N Sabo
- Department of Psychiatry and Behavioral Sciences, Berkshire Medical Center, Pittsfield, MA, USA
- Department of Psychiatry and Behavioral Sciences, University of Massachusetts Medical School, Worcester, MA, USA
| | - Liliana Markovic
- Department of Psychiatry and Behavioral Sciences, Berkshire Medical Center, Pittsfield, MA, USA
- Department of Psychiatry and Behavioral Sciences, University of Massachusetts Medical School, Worcester, MA, USA
| | | | | | | | | | - Doff B McElhinney
- Department of Cardiothoracic Surgery, Stanford University, Stanford, CA, USA
- Clinical and Translational Research Program, Betty Irene Moore Children's Heart Center, Lucile Packard Children's Hospital, Palo Alto, CA, USA
| | - Wei Zhang
- Department of Psychiatry, West China Hospital of Sichuan University, Chengdu, China
| | - Jiayu Liao
- Department of Bioengineering, Bourns College of Engineering, University of California at Riverside, Riverside, CA, USA
- West China-California Center for Predictive Intervention Medicine, West China Hospital, Sichuan University, Chengdu, Sichuan, China
| | - Xuefeng B Ling
- Clinical and Translational Research Program, Betty Irene Moore Children's Heart Center, Lucile Packard Children's Hospital, Palo Alto, CA, USA.
- Department of Surgery, Stanford University, Stanford, CA, USA.
| |
Collapse
|
30
|
Affiliation(s)
- Heon-Jeong Lee
- Department of Psychiatry, Korea University College of Medicine, Seoul, Republic of Korea.,Chronobiology Institute, Korea University, Seoul, Republic of Korea
| |
Collapse
|
31
|
Fonseka TM, Bhat V, Kennedy SH. The utility of artificial intelligence in suicide risk prediction and the management of suicidal behaviors. Aust N Z J Psychiatry 2019; 53:954-964. [PMID: 31347389 DOI: 10.1177/0004867419864428] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
OBJECTIVE Suicide is a growing public health concern with a global prevalence of approximately 800,000 deaths per year. The current process of evaluating suicide risk is highly subjective, which can limit the efficacy and accuracy of prediction efforts. Consequently, suicide detection strategies are shifting toward artificial intelligence platforms that can identify patterns within 'big data' to generate risk algorithms that can determine the effects of risk (and protective) factors on suicide outcomes, predict suicide outbreaks and identify at-risk individuals or populations. In this review, we summarize the role of artificial intelligence in optimizing suicide risk prediction and behavior management. METHODS This paper provides a general review of the literature. A literature search was conducted in OVID Medline, EMBASE and PsycINFO databases with coverage from January 1990 to June 2019. Results were restricted to peer-reviewed, English-language articles. Conference and dissertation proceedings, case reports, protocol papers and opinion pieces were excluded. Reference lists were also examined for additional articles of relevance. RESULTS At the individual level, prediction analytics help to identify individuals in crisis to intervene with emotional support, crisis and psychoeducational resources, and alerts for emergency assistance. At the population level, algorithms can identify at-risk groups or suicide hotspots, which help inform resource mobilization, policy reform and advocacy efforts. Artificial intelligence has also been used to support the clinical management of suicide across diagnostics and evaluation, medication management and behavioral therapy delivery. There could be several advantages of incorporating artificial intelligence into suicide care, which includes a time- and resource-effective alternative to clinician-based strategies, adaptability to various settings and demographics, and suitability for use in remote locations with limited access to mental healthcare supports. CONCLUSION Based on the observed benefits to date, artificial intelligence has a demonstrated utility within suicide prediction and clinical management efforts and will continue to advance mental healthcare forward.
Collapse
Affiliation(s)
- Trehani M Fonseka
- Centre for Mental Health and Krembil Research Centre, University Health Network, Toronto, ON, Canada.,Centre for Depression and Suicide Studies, St. Michael's Hospital, Toronto, ON, Canada.,School of Social Work, King's University College, Western University, London, ON, Canada
| | - Venkat Bhat
- Centre for Mental Health and Krembil Research Centre, University Health Network, Toronto, ON, Canada.,Centre for Depression and Suicide Studies, St. Michael's Hospital, Toronto, ON, Canada.,Department of Psychiatry, University of Toronto, Toronto, ON, Canada
| | - Sidney H Kennedy
- Centre for Mental Health and Krembil Research Centre, University Health Network, Toronto, ON, Canada.,Centre for Depression and Suicide Studies, St. Michael's Hospital, Toronto, ON, Canada.,Department of Psychiatry, University of Toronto, Toronto, ON, Canada.,Keenan Research Centre for Biomedical Science, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada
| |
Collapse
|
32
|
Senanayake S, White N, Graves N, Healy H, Baboolal K, Kularatna S. Machine learning in predicting graft failure following kidney transplantation: A systematic review of published predictive models. Int J Med Inform 2019; 130:103957. [PMID: 31472443 DOI: 10.1016/j.ijmedinf.2019.103957] [Citation(s) in RCA: 44] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2019] [Revised: 07/15/2019] [Accepted: 08/21/2019] [Indexed: 01/11/2023]
Abstract
INTRODUCTION Machine learning has been increasingly used to develop predictive models to diagnose different disease conditions. The heterogeneity of the kidney transplant population makes predicting graft outcomes extremely challenging. Several kidney graft outcome prediction models have been developed using machine learning, and are available in the literature. However, a systematic review of machine learning based prediction methods applied to kidney transplant has not been done to date. The main aim of our study was to perform an in-depth systematic analysis of different machine learning methods used to predict graft outcomes among kidney transplant patients, and assess their usefulness as an aid to decision-making. METHODS A systemic review of machine learning methods used to predict graft outcomes among kidney transplant patients was carried out using a search of the Medline, the Cumulative Index to Nursing and Allied Health Literature, EMBASE, PsycINFO and Cochrane databases. RESULTS A total of 295 articles were identified and extracted. Of these, 18 met the inclusion criteria. Most of the studies were published in the United States after 2010. The population size used to develop the models varied from 80 to 92,844, and the number of features in the models ranged from 6 to 71. The most common machine learning methods used were artificial neural networks, decision trees and Bayesian belief networks. Most of the machine learning based predictive models predicted graft failure with high sensitivity and specificity. Only one machine learning based prediction model had modelled time-to-event (survival) information. Seven studies compared the predictive performance of machine learning models with traditional regression methods and the performance of machine learning methods was found to be mixed, when compared with traditional regression methods. CONCLUSION There was a wide variation in the size of the study population and the input variables used. However, the prediction accuracy provided mixed results when machine learning and traditional predictive methods are compared. Based on reported gains in predictive performance, machine learning has the potential to improve kidney transplant outcome prediction and aid medical decision making.
Collapse
Affiliation(s)
- Sameera Senanayake
- Australian Centre for Health Service Innovation, Queensland University of Technology, Australia.
| | - Nicole White
- Australian Centre for Health Service Innovation, Queensland University of Technology, Australia
| | - Nicholas Graves
- Australian Centre for Health Service Innovation, Queensland University of Technology, Australia
| | - Helen Healy
- Royal Brisbane Hospital for Women, Brisbane, Australia; School of Medicine, University of Queensland, Australia
| | - Keshwar Baboolal
- Royal Brisbane Hospital for Women, Brisbane, Australia; School of Medicine, University of Queensland, Australia
| | - Sanjeewa Kularatna
- Australian Centre for Health Service Innovation, Queensland University of Technology, Australia
| |
Collapse
|
33
|
Ryu S, Lee H, Lee DK, Kim SW, Kim CE. Detection of Suicide Attempters among Suicide Ideators Using Machine Learning. Psychiatry Investig 2019; 16:588-593. [PMID: 31446686 PMCID: PMC6710424 DOI: 10.30773/pi.2019.06.19] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/17/2019] [Accepted: 06/19/2019] [Indexed: 01/10/2023] Open
Abstract
OBJECTIVE We aimed to develop predictive models to identify suicide attempters among individuals with suicide ideation using a machine learning algorithm. METHODS Among 35,116 individuals aged over 19 years from the Korea National Health & Nutrition Examination Survey, we selected 5,773 subjects who reported experiencing suicide ideation and had answered a survey question about suicide attempts. Then, we performed resampling with the Synthetic Minority Over-sampling TEchnique (SMOTE) to obtain data corresponding to 1,324 suicide attempters and 1,330 non-suicide attempters. We randomly assigned the samples to a training set (n=1,858) and a test set (n=796). In the training set, random forest models were trained with features selected through recursive feature elimination with 10-fold cross validation. Subsequently, the fitted model was used to predict suicide attempters in the test set. RESULTS In the test set, the prediction model achieved very good performance [area under receiver operating characteristic curve (AUC)=0.947] with an accuracy of 88.9%. CONCLUSION Our results suggest that a machine learning approach can enable the prediction of individuals at high risk of suicide through the integrated analysis of various suicide risk factors.
Collapse
Affiliation(s)
- Seunghyong Ryu
- Department of Mental Health Research, National Center for Mental Health, Seoul, Republic of Korea
| | - Hyeongrae Lee
- Department of Mental Health Research, National Center for Mental Health, Seoul, Republic of Korea
| | - Dong-Kyun Lee
- Department of Mental Health Research, National Center for Mental Health, Seoul, Republic of Korea
| | - Sung-Wan Kim
- Department of Psychiatry, Chonnam National University Medical School, Gwangju, Republic of Korea
| | - Chul-Eung Kim
- Mental Health Research Institute, National Center for Mental Health, Seoul, Republic of Korea
| |
Collapse
|