Use of a Machine Learning Algorithm to Predict Individuals with Suicide Ideation in the General Population

Explainable artificial intelligence models for predicting risk of suicide using health administrative data in Quebec

Suicide is a complex, multidimensional event, and a significant challenge for prevention globally. Artificial intelligence (AI) and machine learning (ML) have emerged to harness large-scale datasets to enhance risk detection. In order to trust and act upon the predictions made with ML, more intuitive user interfaces must be validated. Thus, Interpretable AI is one of the crucial directions which could allow policy and decision makers to make reasonable and data-driven decisions that can ultimately lead to better mental health services planning and suicide prevention. This research aimed to develop sex-specific ML models for predicting the population risk of suicide and to interpret the models. Data were from the Quebec Integrated Chronic Disease Surveillance System (QICDSS), covering up to 98% of the population in the province of Quebec and containing data for over 20,000 suicides between 2002 and 2019. We employed a case-control study design. Individuals were considered cases if they were aged 15+ and had died from suicide between January 1st, 2002, and December 31st, 2019 (n = 18339). Controls were a random sample of 1% of the Quebec population aged 15+ of each year, who were alive on December 31st of each year, from 2002 to 2019 (n = 1,307,370). We included 103 features, including individual, programmatic, systemic, and community factors, measured up to five years prior to the suicide events. We trained and then validated the sex-specific predictive risk model using supervised ML algorithms, including Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost) and Multilayer perceptron (MLP). We computed operating characteristics, including sensitivity, specificity, and Positive Predictive Value (PPV). We then generated receiver operating characteristic (ROC) curves to predict suicides and calibration measures. For interpretability, Shapley Additive Explanations (SHAP) was used with the global explanation to determine how much the input features contribute to the models' output and the largest absolute coefficients. The best sensitivity was 0.38 with logistic regression for males and 0.47 with MLP for females; the XGBoost Classifier with 0.25 for males and 0.19 for females had the best precision (PPV). This study demonstrated the useful potential of explainable AI models as tools for decision-making and population-level suicide prevention actions. The ML models included individual, programmatic, systemic, and community levels variables available routinely to decision makers and planners in a public managed care system. Caution shall be exercised in the interpretation of variables associated in a predictive model since they are not causal, and other designs are required to establish the value of individual treatments. The next steps are to produce an intuitive user interface for decision makers, planners and other stakeholders like clinicians or representatives of families and people with live experience of suicidal behaviors or death by suicide. For example, how variations in the quality of local area primary care programs for depression or substance use disorders or increased in regional mental health and addiction budgets would lower suicide rates.

Prediction of suicidal ideation with associated risk factors among university students in the southern part of Bangladesh: Machine learning approach

BACKGROUND

The prevalence of suicidal ideation has become an urgent issue, particularly among adolescents. The primary objective of this research is to determine the prevalence of suicidal ideation among students in the southern region of Bangladesh and to predict this phenomenon using machine learning (ML) models.

METHODS

The data collection process involved using a simple random sampling technique to gather information from university students located in the southern region of Bangladesh during the period spreading from April 2022 to June 2022. Upon accounting for missing values and non-response rates, the ultimate sample size was determined to be 584, with 51.5 % of participants identifying as male and 48.5 % female.

RESULTS

A significant proportion of students, precisely 19.9 %, reported experiencing suicidal ideation. Most participants were female (77 %) and unmarried (78 %). Within the machine learning (ML) framework, KNN exhibited the highest accuracy score of 91.45 %. In addition, the Random Forest (RF), and Categorical Boosting (CatBoost) algorithms exhibited comparable levels of accuracy, achieving scores of 90.60 and 90.59 respectively.

LIMITATIONS

Using a cross-sectional design in research limits the ability to establish causal relationships.

CONCLUSION

Mental health practitioners can employ the KNN model alongside patients' medical histories to detect those who may be at a higher risk of attempting suicide. This approach enables healthcare professionals to take appropriate measures, such as counselling, encouraging regular sleep patterns, and addressing depression and anxiety, to prevent suicide attempts.

The use of machine learning on administrative and survey data to predict suicidal thoughts and behaviors: a systematic review

Background

Machine learning is a promising tool in the area of suicide prevention due to its ability to combine the effects of multiple risk factors and complex interactions. The power of machine learning has led to an influx of studies on suicide prediction, as well as a few recent reviews. Our study distinguished between data sources and reported the most important predictors of suicide outcomes identified in the literature.

Objective

Our study aimed to identify studies that applied machine learning techniques to administrative and survey data, summarize performance metrics reported in those studies, and enumerate the important risk factors of suicidal thoughts and behaviors identified.

Methods

A systematic literature search of PubMed, Medline, Embase, PsycINFO, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Allied and Complementary Medicine Database (AMED) to identify all studies that have used machine learning to predict suicidal thoughts and behaviors using administrative and survey data was performed. The search was conducted for articles published between January 1, 2019 and May 11, 2022. In addition, all articles identified in three recently published systematic reviews (the last of which included studies up until January 1, 2019) were retained if they met our inclusion criteria. The predictive power of machine learning methods in predicting suicidal thoughts and behaviors was explored using box plots to summarize the distribution of the area under the receiver operating characteristic curve (AUC) values by machine learning method and suicide outcome (i.e., suicidal thoughts, suicide attempt, and death by suicide). Mean AUCs with 95% confidence intervals (CIs) were computed for each suicide outcome by study design, data source, total sample size, sample size of cases, and machine learning methods employed. The most important risk factors were listed.

Results

The search strategy identified 2,200 unique records, of which 104 articles met the inclusion criteria. Machine learning algorithms achieved good prediction of suicidal thoughts and behaviors (i.e., an AUC between 0.80 and 0.89); however, their predictive power appears to differ across suicide outcomes. The boosting algorithms achieved good prediction of suicidal thoughts, death by suicide, and all suicide outcomes combined, while neural network algorithms achieved good prediction of suicide attempts. The risk factors for suicidal thoughts and behaviors differed depending on the data source and the population under study.

Conclusion

The predictive utility of machine learning for suicidal thoughts and behaviors largely depends on the approach used. The findings of the current review should prove helpful in preparing future machine learning models using administrative and survey data.

Systematic review registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022333454 identifier CRD42022333454.

Factors Related to Suicidal Ideation by Gender and Age Group in Korean Adults

BACKGROUND

According to the results of the age-standardized suicide rate (suicide deaths per 100,000 population) published by Organization for Economic Cooperation and Development countries from 2003 to 2019, the suicide rate in Korea is ranked first almost every year, so it should be managed at the national level. To reduce the suicide rate, many scholars are studying suicide. The suicidal process begins with suicidal ideation, progresses through suicide attempts, and ends with suicide. Among them, the frequency of suicidal thoughts was found to be highly correlated with suicide plans and attempts, and it is said that 60% of those who change from suicidal ideation to attempts appear within one year. Therefore, research related to suicidal ideation to lower the suicide rate will contribute to preventing suicide at an early stage.

METHODS

This study used data from the Korea National Health and Nutrition Examination Survey (KNHANES). Among them, data from 2013, 2015, 2017, and 2019 surveyed adults were compared for suicidal ideation among 18,339 adults. Considering the characteristics of the KNHANES data, complex sample analysis was performed considering the primary extraction unit (region), stratification variables, and weights. The Rao-Scott χ² test was stratified by age group to confirm the distribution of suicidal ideation according to general characteristics. Univariate and multiple logistic regression analyses were performed to understand the effect on suicidal ideation.

RESULTS

Among all subjects, the rate of suicidal ideation was 4.75%, and among the age groups, the rate of suicidal thoughts was the highest among those over 80 years old (10.39%). Compared to those who were 50s, those in their 30s had the lowest suicidal ideation (odds ratio [OR], 0.52; 95% confidence interval [CI], 0.39-0.70), and those in their 80s had the highest rate of suicidal ideation (OR, 1.97; 95% CI, 1.38-2.82). Except for those in their 20s, the rate of suicidal ideation tends to increase with age. Overall, the lower 25% of the Euro Quality of Life-5 Dimensions (EQ-5D) index, depression lasting more than two weeks, and subjects with poor subjective health or high stress were more likely to have suicidal ideation. In particular, EQ-5D Index (OR, 5.86; 95% CI, 3.73-9.20), depressive symptoms (OR, 19.65; 95% CI, 9.94-38.83) in 20s, and stress in 80s (OR, 10.87; 95% CI, 5.63-20.96) was the highest, and those had the good subjective health perception in 30s (OR, 0.10; 95% CI, 0.05-0.20) was lowest. Participants in their 30s, 40s, and 60s who were divorced or widowed and those in their 50s and 60s who had never married or lived together were more likely to consider suicide ideation. Except for those in their 20s and 80s, the rate of suicidal ideation tends to increase with lower household income levels.

CONCLUSIONS

In the results of this study, it can be seen that a combination of factors such as a person's personality traits, stress or coping ability to various events, social support, and mental or physical limitations influence suicidal behavior. Suicidal ideation is not simply affected by a single factor but may be influenced by multiple factors.

Prediction models of suicide and non-fatal suicide attempt after discharge from a psychiatric inpatient stay: A machine learning approach on nationwide Danish registers

INTRODUCTION

To develop machine learning models capable of predicting suicide and non-fatal suicide attempt as separate outcomes in the first 30 days after discharge from a psychiatric inpatient stay.

METHODS

Prospective cohort study using nationwide Danish registry data. We included individuals who were 18 years or older, and all discharges from psychiatric hospitalizations in Denmark from 1995 to 2018. We trained predictive models using 10-fold cross validation on 80% of the data and did testing on the remaining 20%.

RESULTS

The best model for predicting non-fatal suicide attempt was an ensemble of predictions from gradient boosting (XGBoost) and categorical boosting (catBoost). The ROC-AUC for predicting suicide attempt was 0.85 (95% CI: 0.84-0.85). At a risk threshold of 4.36%, positive predictive value (PPV) was 11.0% and sensitivity was 47.2%. The best model for predicting suicide was an ensemble of predictions from random forest, XGBoost and catBoost. For suicide, the ROC-AUC was 0.71 (95% CI: 0.70-0.73). At a risk threshold of 0.15%, PPV was 0.34% and sensitivity was 56.0%. The most contributing predictors differed when predicting suicide and suicide attempt, indicating that separate models are needed. The ensemble model was fair across sex and age, and more so than the penalized logistic regression model.

CONCLUSIONS

We achieved good performance for predicting suicide attempts and demonstrated a clinical application of ensemble models. Our results indicate a difference in predictive performance for models predicting suicide and suicide attempt, respectively. Thus, we recommend that suicide and suicide attempt are treated as two separate endpoints, in particular for clinical application. We demonstrated that the ensemble model is fairer across sex and age compared with a penalized logistic regression, and therefore we recommend the use of well-tested ensembles despite a more complex explainability.

Machine learning-based prediction for self-harm and suicide attempts in adolescents

This study aimed to use machine learning (ML) models to predict the risk of self-harm and suicide attempts in adolescents. We conducted secondary analysis of cross-sectional data from the Longitudinal Study of Australian Children dataset. Several key variables at the age of 14-15 years were used to predict self-harm or suicide attempt at 16-17 years. Random forest classification models were used to select the optimal subset of predictors and subsequently make predictions. Among 2809 participants, 296 (10.54%) reported an act of self-harm and 145 (5.16%) reported attempting suicide at least once in the past 12 months. The area under the receiver operating curve was fair for self-harm (0.7397) and suicide attempt (0.7220), which outperformed the prediction strategy solely based on prior suicide or self-harm attempt (AUC: 0.6). The most important factors identified were similar, and included depressed feelings, strengths and difficulties questionnaire scores, perceptions of self, and school- and parent-related factors. The random forest classification algorithm, an ML technique, can effectively select the optimal subset of predictors from hundreds of variables to forecast the risks of suicide and self-harm among adolescents. Further research is needed to validate the utility and scalability of ML techniques in mental health research.

Prediction of suicidal ideation in children and adolescents using machine learning and deep learning algorithm: A case study in South Korea where suicide is the leading cause of death

BACKGROUND

Korea has the highest suicide rate among Organisation for Economic Co-operation and Development (OECD) countries. Consequently, central and local governments and private organizations in Korea cooperate in promoting various suicide prevention projects to actively respond to suicide problems. Machine learning has been used to predict suicidal ideation in the fields of health and medicine but not from a social science perspective.

OBJECTIVE

Since suicidal ideation is a major predictor of suicide attempts, being able to anticipate and mitigate it helps prevent suicide. Therefore, this study presents a data-based analysis method for predicting suicidal thoughts quickly and effectively and suggests countermeasures against the causes of suicidal thoughts.

PARTICIPANTS AND METHODS

To predict early signs of suicidal ideation in children and adolescents, big data collected for approximately 4 years (from 2017 to 2020) from the Korea Youth Policy Institute (NYPI) were used. To accurately predict suicidal ideation, supervised ma- chine learning classification algorithms such as logistic regression, random forest, XGBoost, multilayer perceptron (MLP), and convolutional neural network (CNN) were used.

RESULTS

Using CNN, suicidal ideation was predicted with an accuracy of approximately 90 %. The logistic regression results showed that sadness and depression increased suicidal thoughts by more than 25 times, and anxiety, loneliness, and experience of abusive language increased suicidal thoughts by more than three times.

CONCLUSIONS

Machine learning and deep learning approaches have the potential to predict and respond to suicidal thoughts in children, adolescents, and the general population, as well as help respond to the suicide crisis by preemptively identifying the cause.

Application of artificial intelligence tools in diagnosis and treatmentof mental disorders

Introduction: Artificial intelligence research is increasing its application in mental health services. Machine learning, deep learning, semantic analysis in the form of transcriptions of patients' statements enable early diagnosis of psychotic disorders, ADHD, anorexia nervosa. Of great importance are the so-called digital therapists. This paper aims to show the use of AI tools in diagnosing, treating, the benefits and limitations associated with mental disorders. Material and methodS: This literature review was conducted by searching scientific articles from 2015 to 2022. The basis were PubMED, OpenKnowledge, Web of Science, using the following keywords: artificial intelligence, digital therapy, psychiatry, machine learning. Results: A review indicates the widespread use of AI tools in screening for mental disorders. These tools advance the clinical diagnosis medical specialists make up for several years. They impact solving medical staff shortages, lack of access to medical facilities and leveling patient resistance to treatment. The benefits are ultra-fast analysis of large sets of information, effective screening of people in need of specialized psychiatric care, reduction of doctors' duties and maximization of their work efficiency. During the current COVID 19 pandemic, robots in the form of digital psychotherapists are playing a special role. Conclusions: The need for further research, testing and clarification of regulations related to the use of AI tools is indicated. Ethical and social problems need to be resolved. The tools should not form the basis of autonomous therapy without the supervision of highly trained professionals. Human beings should be at the center of analysis just as their health and well-being. Keywords: artificial intelligence, digital therapy, psychiatry, machine learning

An explainable predictive model for suicide attempt risk using an ensemble learning and Shapley Additive Explanations (SHAP) approach

Machine learning approaches have been used to develop suicide attempt predictive models recently and have been shown to have a good performance. However, those proposed models have difficulty interpreting and understanding why an individual has suicidal attempts. To overcome this issue, the identification of features such as risk factors in predicting suicide attempts is important for clinicians to make decisions. Therefore, the aim of this study is to propose an explainable predictive model to predict and analyse the importance of features for suicide attempts. This model can also provide explanations to improve the clinical understanding of suicide attempts. Two complex ensemble learning models, namely Random Forest and Gradient Boosting with an explanatory model (SHapley Additive exPlanations (SHAP)) have been constructed. The models are used for predictive interpretation and understanding of the importance of the features. The experiment shows that both models with SHAP are able to interpret and understand the nature of an individual's predictions with suicide attempts. However, compared with Random Forest, the results show that Gradient Boosting with SHAP achieves higher accuracy and the analyses found that history of suicide attempts, suicidal ideation, and ethnicity as the main predictors for suicide attempts.

Detecting and Analyzing Suicidal Ideation on Social Media Using Deep Learning and Machine Learning Models

Individuals who suffer from suicidal ideation frequently express their views and ideas on social media. Thus, several studies found that people who are contemplating suicide can be identified by analyzing social media posts. However, finding and comprehending patterns of suicidal ideation represent a challenging task. Therefore, it is essential to develop a machine learning system for automated early detection of suicidal ideation or any abrupt changes in a user's behavior by analyzing his or her posts on social media. In this paper, we propose a methodology based on experimental research for building a suicidal ideation detection system using publicly available Reddit datasets, word-embedding approaches, such as TF-IDF and Word2Vec, for text representation, and hybrid deep learning and machine learning algorithms for classification. A convolutional neural network and Bidirectional long short-term memory (CNN-BiLSTM) model and the machine learning XGBoost model were used to classify social posts as suicidal or non-suicidal using textual and LIWC-22-based features by conducting two experiments. To assess the models' performance, we used the standard metrics of accuracy, precision, recall, and F1-scores. A comparison of the test results showed that when using textual features, the CNN-BiLSTM model outperformed the XGBoost model, achieving 95% suicidal ideation detection accuracy, compared with the latter's 91.5% accuracy. Conversely, when using LIWC features, XGBoost showed better performance than CNN-BiLSTM.

Machine learning prediction of suicidal ideation, planning, and attempt among Korean adults: A population-based study

Background

Suicide remains the leading cause of premature death in South Korea. This study aims to develop machine learning algorithms for screening Korean adults at risk for suicidal ideation and suicide planning or attempt.

Methods

Two sets of balanced data for Korean adults aged 19–64 years were drawn from the 2012–2019 waves of the Korea Welfare Panel Study using the random down-sampling method (N = 3292 for the prediction of suicidal ideation, N = 488 for the prediction of suicide planning or attempt). Demographic, socioeconomic, and psychosocial characteristics were used to predict suicidal ideation and suicide planning or attempt. Four machine-learning classifiers (logistic regression, random forest, support vector machine, and extreme gradient boosting) were tuned and cross-validated.

Results

All four algorithms demonstrated satisfactory classification performance in predicting suicidal ideation (sensitivity 0.808–0.853, accuracy 0.843–0.863) and suicide planning or attempt (sensitivity 0.814–0.861, accuracy 0.864–0.884). Extreme gradient boosting was the best-performing algorithm for predicting both suicidal outcomes. The most important predictors were depressive symptoms, self-esteem, income, consumption, and life satisfaction. The algorithms trained with the top two predictors, depressive symptoms and self-esteem, showed comparable classification performance in predicting suicidal ideation (sensitivity 0.801–0.839, accuracy 0.841–0.846) and suicide planning or attempt (sensitivity 0.814–0.837, accuracy 0.874–0.884).

Limitations

Suicidal ideation and behaviors may be under-reported due to social desirability bias. Causality is not established.

Discussion

More than 80% of individuals at risk for suicidal ideation and suicide planning or attempt could be predicted by a number of mental and socioeconomic characteristics of respondents. This finding suggests the potential of developing a quick screening tool based on the known risk factors and applying it to primary care or community settings for early intervention.

•

This study develops machine learning models to predict suicidal ideation and behaviors.

•

Logistic regression, random forest, support vector machine, and extreme gradient boosting are used.

•

The algorithms correctly identifyed 80–90% of suicidal cases.

•

The algorithms with the top two predictors (depressive symptoms and self-esteem) could achieve comparable accuracy.

•

Our findings can be used to design a quick screening tool for use in primary care or community settings.

A machine learning approach to identification of self-harm and suicidal ideation among military and police Veterans

LAY SUMMARY

Combat Veterans are vulnerable to suicidal thoughts and behaviour. Many who die by suicide deny having suicidal ideation (SI). Typically, researchers try to find variables indicating the presence of SI using traditional statistical approaches. These approaches do not possess the capacity to detect highly complex multivariable interactions. In contrast, machine learning (ML) is designed to detect such patterns and can consequently yield much higher predictive accuracy. In this study, the authors trained ML algorithms using 192 variables extracted from questionnaires administered to 738 Veterans and serving personnel to detect the presence of self-harm and SI (SHSI). Using the 10 most predictive non-suicide-related items, the ML algorithms could detect SHSI with 75.3% accuracy. Most of these items reflect psychological phenomena that can change quickly over time, allowing repeated risk reassessment from day to day. The study’s findings suggest that ML methods may play an important role in the discovery, within a large data set, of predictive patterns that might be useful in suicide risk assessment.

The Development of a Suicidal Ideation Predictive Model for Community-Dwelling Elderly Aged >55 Years

PURPOSE

Suicide is an important health and social concern worldwide. Both suicidal ideation and suicide rates are higher in the elderly population than in other age groups; thus, more careful attention and targeted interventions are required. Therefore, we have developed a model to predict suicidal ideation in the community-dwelling elderly aged of >55 years.

PATIENTS AND METHODS

A random forest algorithm was applied to those who participated in the Korea Welfare Panel. We used a total of 26 variables as potential predictors. To resolve the imbalance in the dataset resulting from the low frequency of suicidal ideation, training was performed by applying the synthetic minority oversampling technique. The performance index was calculated by applying the predictive model to the test set, which was not included in the training process.

RESULTS

A total of 6410 elderly Korean aged of >55 (mean, 71.48; standard deviation, 9.56) years were included in the analysis, of which 2.7% had suicidal ideation. The results for predicting suicidal ideation using the 26 chosen variables showed an AUC of 0.879, accuracy of 0.871, sensitivity of 0.750, and specificity of 0.874. The most significant variable in the predictive model was the severity of depression, followed by life satisfaction and self-esteem factors. Basic demographic variables such as age and gender demonstrated a relatively small effect.

CONCLUSION

Machine learning can be used to create algorithms for predicting suicidal ideation in community-dwelling elderly. However, there are limitations to predicting future suicidal ideation. A predictive model that includes both biological and cognitive indicators should be created in the future.

A machine-learning model to predict suicide risk in Japan based on national survey data

OBJECTIVE

Several prognostic models of suicide risk have been published; however, few have been implemented in Japan using longitudinal cohort data. The aim of this study was to identify suicide risk factors for suicidal ideation in the Japanese population and to develop a machine-learning model to predict suicide risk in Japan.

MATERIALS AND METHODS

Data was obtained from Wave1 Time 1 (November 2016) and Time 2 (March 2017) of the National Survey for Stress and Health in Japan, were incorporated into a suicide risk prediction machine-learning model, trained using 65 items related to trauma and stress. The study included 3,090 and 2,163 survey respondents >18 years old at Time 1 and Time 2, respectively. The mean (standard deviation, SD) age was 44.9 (10.9) years at Time 1 and 46.0 (10.7) years at Time 2. We analyzed the participants with increased suicide risk at Time 2 survey. Model performance, including the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, were also analyzed.

RESULTS

The model showed a good performance (AUC = 0.830, 95% confidence interval = 0.795-0.866). Overall, the model achieved an accuracy of 78.8%, sensitivity of 75.4%, specificity of 80.4%, positive predictive value of 63.4%, and negative predictive value of 87.9%. The most important risk factor for suicide risk was the participants' Suicidal Ideation Attributes Scale score, followed by the Sheehan Disability Scale score, Patient Health Questionnaire-9 scores, Cross-Cutting Symptom Measure (CCSM-suicidal ideation domain, Dissociation Experience Scale score, history of self-harm, Generalized Anxiety Disorder-7 score, Post-Traumatic Stress Disorder check list-5 score, CCSM-dissociation domain, and Impact of Event Scale-Revised scores at Time 1.

CONCLUSIONS

This prognostic study suggests the ability to identify patients at a high risk of suicide using an online survey method. In addition to confirming several well-known risk factors of suicide, new risk measures related to trauma and trauma-related experiences were also identified, which may help guide future clinical assessments and early intervention approaches.

Comparing machine learning to a rule-based approach for predicting suicidal behavior among adolescents: Results from a longitudinal population-based survey

INTRODUCTION

Suicidal thoughts and suicide attempts are one of the most prominent public health concerns in adolescents and therefore early detection is important to initiate preventive interventions and closer monitoring.

METHOD

We examined whether the Machine Learning models Random Forest and Lasso Regression better predict future suicidal behavior than a simple decision rule that classifies every adolescent with history of suicide ideation at baseline as at risk (current practice). We used data from a general population of students in second and fourth year of secondary education in Amsterdam, the Netherlands.

RESULTS

Both the Random Forest and the Lasso Regression resulted in slightly better prediction. The AUC of the Random Forest (0.79) and Lasso regression (0.76) were both higher than the AUC of the decision rule (0.64). The Random Forest achieved slightly (but non-significantly) higher sensitivity than the decision rule (0.37 versus 0.34), with the same specificity (0.94). With Lasso Regression the sensitivity increased significantly (0.52), but at the expense of the specificity (0.85).

LIMITATIONS

The loss of cases after merging the data, the use of self-reported data, confidential data collection and the use of only four questions to measure suicidal behavior.

CONCLUSIONS

This is the first study applying Machine Learning techniques to predict future suicidal behavior on survey data collected in a general population of adolescents. Our study showed that integrating machine learning techniques in screening practice will result in a small improvement in the ability to predict suicide. The models need to be further optimized to improve accuracy.

A comparative study of machine learning techniques for suicide attempts predictive model

Current suicide risk assessments for predicting suicide attempts are time consuming, of low predictive value and have inadequate reliability. This paper aims to develop a predictive model for suicide attempts among patients with depression using machine learning algorithms as well as presents a comparative study on single predictive models with ensemble predictive models for differentiating depressed patients with suicide attempts from non-suicide attempters. We applied and trained eight different machine learning algorithms using a dataset that consists of 75 patients diagnosed with a depressive disorder. A recursive feature elimination was used to reduce the features via three-fold cross validation. An ensemble predictive models outperformed the single predictive models. Voting and bagging revealed the highest accuracy of 92% compared to other machine learning algorithms. Our findings indicate that history of suicide attempt, religion, race, suicide ideation and severity of clinical depression are useful factors for prediction of suicide attempts.

Detecting suicidal risk using MMPI-2 based on machine learning algorithm

Minnesota Multiphasic Personality Inventory-2 (MMPI-2) is a widely used tool for early detection of psychological maladjustment and assessing the level of adaptation for a large group in clinical settings, schools, and corporations. This study aims to evaluate the utility of MMPI-2 in assessing suicidal risk using the results of MMPI-2 and suicidal risk evaluation. A total of 7,824 datasets collected from college students were analyzed. The MMPI-2-Resturcutred Clinical Scales (MMPI-2-RF) and the response results for each question of the Mini International Neuropsychiatric Interview (MINI) suicidality module were used. For statistical analysis, random forest and K-Nearest Neighbors (KNN) techniques were used with suicidal ideation and suicide attempt as dependent variables and 50 MMPI-2 scale scores as predictors. On applying the random forest method to suicidal ideation and suicidal attempts, the accuracy was 92.9% and 95%, respectively, and the Area Under the Curves (AUCs) were 0.844 and 0.851, respectively. When the KNN method was applied, the accuracy was 91.6% and 94.7%, respectively, and the AUCs were 0.722 and 0.639, respectively. The study confirmed that machine learning using MMPI-2 for a large group provides reliable accuracy in classifying and predicting the subject's suicidal ideation and past suicidal attempts.

Which PHQ-9 Items Can Effectively Screen for Suicide? Machine Learning Approaches

(1) Background: The Patient Health Questionnaire-9 (PHQ-9) is a tool that screens patients for depression in primary care settings. In this study, we evaluated the efficacy of PHQ-9 in evaluating suicidal ideation (2) Methods: A total of 8760 completed questionnaires collected from college students were analyzed. The PHQ-9 was scored in combination with and evaluated against four categories (PHQ-2, PHQ-8, PHQ-9, and PHQ-10). Suicidal ideations were evaluated using the Mini-International Neuropsychiatric Interview suicidality module. Analyses used suicide ideation as the dependent variable, and machine learning (ML) algorithms, k-nearest neighbors, linear discriminant analysis (LDA), and random forest. (3) Results: Random forest application using the nine items of the PHQ-9 revealed an excellent area under the curve with a value of 0.841, with 94.3% accuracy. The positive and negative predictive values were 84.95% (95% CI = 76.03–91.52) and 95.54% (95% CI = 94.42–96.48), respectively. (4) Conclusion: This study confirmed that ML algorithms using PHQ-9 in the primary care field are reliably accurate in screening individuals with suicidal ideation.

A Comprehensive Review of Computer-Aided Diagnosis of Major Mental and Neurological Disorders and Suicide: A Biostatistical Perspective on Data Mining

The World Health Organization (WHO) suggests that mental disorders, neurological disorders, and suicide are growing causes of morbidity. Depressive disorders, schizophrenia, bipolar disorder, Alzheimer's disease, and other dementias account for 1.84%, 0.60%, 0.33%, and 1.00% of total Disability Adjusted Life Years (DALYs). Furthermore, suicide, the 15th leading cause of death worldwide, could be linked to mental disorders. More than 68 computer-aided diagnosis (CAD) methods published in peer-reviewed journals from 2016 to 2021 were analyzed, among which 75% were published in the year 2018 or later. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol was adopted to select the relevant studies. In addition to the gold standard, the sample size, neuroimaging techniques or biomarkers, validation frameworks, the classifiers, and the performance indices were analyzed. We further discussed how various performance indices are essential based on the biostatistical and data mining perspective. Moreover, critical information related to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines was analyzed. We discussed how balancing the dataset and not using external validation could hinder the generalization of the CAD methods. We provided the list of the critical issues to consider in such studies.

Screening for Depression in Mobile Devices Using Patient Health Questionnaire-9 (PHQ-9) Data: A Diagnostic Meta-Analysis via Machine Learning Methods

PURPOSE

Depression is a symptom commonly encountered in primary care; however, it is often not detected by doctors. Recently, disease diagnosis and treatment approaches have been attempted using smart devices. In this study, instrumental effectiveness was confirmed with the diagnostic meta-analysis of studies that demonstrated the diagnostic effectiveness of PHQ-9 for depression using mobile devices.

PATIENTS AND METHODS

We found all published and unpublished studies through EMBASE, MEDLINE, MEDLINE In-Process, and PsychINFO up to March 26, 2021. We performed a meta-analysis by including 1099 subjects in four studies. We performed a diagnostic meta-analysis according to the PHQ-9 cut-off score and machine learning algorithm techniques. Quality assessment was conducted using the QUADAS-2 tool. Data on the sensitivity and specificity of the studies included in the meta-analysis were extracted in a standardized format. Bivariate and summary receiver operating characteristic (SROC) curve were constructed using the metandi, midas, metabias, and metareg functions of the Stata algorithm meta-analysis words.

RESULTS

Using four studies out of the 5476 papers searched, a diagnostic meta-analysis of the PHQ-9 scores of 1099 people diagnosed with depression was performed. The pooled sensitivity and specificity were 0.797 (95% CI = 0.642-0.895) and 0.85 (95% CI = 0.780-0.900), respectively. The diagnostic odds ratio was 22.16 (95% CI = 7.273-67.499). Overall, a good balance was maintained, and no heterogeneity or publication bias was presented.

CONCLUSION

Through various machine learning algorithm techniques, it was possible to confirm that PHQ-9 depression screening in mobiles is an effective diagnostic tool when integrated into a diagnostic meta-analysis.

Identification of Suicidal Ideation in the Canadian Community Health Survey-Mental Health Component Using Deep Learning

Introduction: Suicidal ideation (SI) is prevalent in the general population, and is a risk factor for suicide. Predicting which patients are likely to have SI remains challenging. Deep Learning (DL) may be a useful tool in this context, as it can be used to find patterns in complex, heterogeneous, and incomplete datasets. An automated screening system for SI could help prompt clinicians to be more attentive to patients at risk for suicide. Methods: Using the Canadian Community Health Survey-Mental Health Component, we trained a DL model based on 23,859 survey responses to classify patients with and without SI. Models were created to classify both lifetime SI and SI over the last 12 months. From 582 possible parameters we produced 96- and 21-feature versions of the models. Models were trained using an undersampling procedure that balanced the training set between SI and non-SI; validation was done on held-out data. Results: For lifetime SI, the 96 feature model had an Area under the receiver operating curve (AUC) of 0.79 and the 21 feature model had an AUC of 0.77. For SI in the last 12 months the 96 feature model had an AUC of 0.71 and the 21 feature model had an AUC of 0.68. In addition, sensitivity analyses demonstrated feature relationships in line with existing literature. Discussion: Although further study is required to ensure clinical relevance and sample generalizability, this study is an initial proof of concept for the use of DL to improve identification of SI. Sensitivity analyses can help improve the interpretability of DL models. This kind of model would help start conversations with patients which could lead to improved care and a reduction in suicidal behavior.

Accuracy of individual and combined risk-scale items in the prediction of repetition of self-harm: multicentre prospective cohort study

BACKGROUND

Individuals attending emergency departments following self-harm have increased risks of future self-harm. Despite the common use of risk scales in self-harm assessment, there is growing evidence that combinations of risk factors do not accurately identify those at greatest risk of further self-harm and suicide.

AIMS

To evaluate and compare predictive accuracy in prediction of repeat self-harm from clinician and patient ratings of risk, individual risk-scale items and a scale constructed with top-performing items.

METHOD

We conducted secondary analysis of data from a five-hospital multicentre prospective cohort study of participants referred to psychiatric liaison services following self-harm. We tested predictive utility of items from five risk scales: Manchester Self-Harm Rule, ReACT Self-Harm Rule, SAD PERSONS, Modified SAD PERSONS, Barratt Impulsiveness Scale and clinician and patient risk estimates. Area under the curve (AUC), sensitivity, specificity, predictive values and likelihood ratios were used to evaluate predictive accuracy, with sensitivity analyses using classification-tree regression.

RESULTS

A total of 483 self-harm episodes were included, and 145 (30%) were followed by a repeat presentation within 6 months. AUC of individual items ranged from 0.43-0.65. Combining best performing items resulted in an AUC of 0.56. Some individual items outperformed the scale they originated from; no items were superior to clinician or patient risk estimations.

CONCLUSIONS

No individual or combination of items outperformed patients' or clinicians' ratings. This suggests there are limitations to combining risk factors to predict risk of self-harm repetition. Risk scales should have little role in the management of people who have self-harmed.

Implementing Precision Psychiatry: A Systematic Review of Individualized Prediction Models for Clinical Practice

BACKGROUND

The impact of precision psychiatry for clinical practice has not been systematically appraised. This study aims to provide a comprehensive review of validated prediction models to estimate the individual risk of being affected with a condition (diagnostic), developing outcomes (prognostic), or responding to treatments (predictive) in mental disorders.

METHODS

PRISMA/RIGHT/CHARMS-compliant systematic review of the Web of Science, Cochrane Central Register of Reviews, and Ovid/PsycINFO databases from inception until July 21, 2019 (PROSPERO CRD42019155713) to identify diagnostic/prognostic/predictive prediction studies that reported individualized estimates in psychiatry and that were internally or externally validated or implemented. Random effect meta-regression analyses addressed the impact of several factors on the accuracy of prediction models.

FINDINGS

Literature search identified 584 prediction modeling studies, of which 89 were included. 10.4% of the total studies included prediction models internally validated (n = 61), 4.6% models externally validated (n = 27), and 0.2% (n = 1) models considered for implementation. Across validated prediction modeling studies (n = 88), 18.2% were diagnostic, 68.2% prognostic, and 13.6% predictive. The most frequently investigated condition was psychosis (36.4%), and the most frequently employed predictors clinical (69.5%). Unimodal compared to multimodal models (β = .29, P = .03) and diagnostic compared to prognostic (β = .84, p < .0001) and predictive (β = .87, P = .002) models were associated with increased accuracy.

INTERPRETATION

To date, several validated prediction models are available to support the diagnosis and prognosis of psychiatric conditions, in particular, psychosis, or to predict treatment response. Advancements of knowledge are limited by the lack of implementation research in real-world clinical practice. A new generation of implementation research is required to address this translational gap.

Artificial Intelligence and Suicide Prevention: A Systematic Review of Machine Learning Investigations

Suicide is a leading cause of death that defies prediction and challenges prevention efforts worldwide. Artificial intelligence (AI) and machine learning (ML) have emerged as a means of investigating large datasets to enhance risk detection. A systematic review of ML investigations evaluating suicidal behaviors was conducted using PubMed/MEDLINE, PsychInfo, Web-of-Science, and EMBASE, employing search strings and MeSH terms relevant to suicide and AI. Databases were supplemented by hand-search techniques and Google Scholar. Inclusion criteria: (1) journal article, available in English, (2) original investigation, (3) employment of AI/ML, (4) evaluation of a suicide risk outcome. N = 594 records were identified based on abstract search, and 25 hand-searched reports. N = 461 reports remained after duplicates were removed, n = 316 were excluded after abstract screening. Of n = 149 full-text articles assessed for eligibility, n = 87 were included for quantitative synthesis, grouped according to suicide behavior outcome. Reports varied widely in methodology and outcomes. Results suggest high levels of risk classification accuracy (>90%) and Area Under the Curve (AUC) in the prediction of suicidal behaviors. We report key findings and central limitations in the use of AI/ML frameworks to guide additional research, which hold the potential to impact suicide on broad scale.

Discovering the Unclassified Suicide Cases Among Undetermined Drug Overdose Deaths Using Machine Learning Techniques

OBJECTIVE

The Centers for Disease Control and Prevention (CDC) monitor accidental and intentional deaths to answer questions that are critical for the development of effective prevention and resource allocation. CDC's National Violent Death Reporting System (NVDRS) is a major innovation in surveillance linking individual-level data from multiple sources. However, suicide underreporting is common, particularly from drug overdose deaths. This study sought to assess machine learning (ML) techniques in quantifying drug overdose suicide underreporting rates.

METHODS

Clinical, sociodemographic, toxicological, and proximal stressor data on overdose decedents (n = 2,665) were extracted from Utah's NVDRS from 2012 to 2015. The existing well-determined cases were used to train and test our ML models. We assessed and compared multiple machine learning methods including Logistic Regression, Random Forest Classifier, Support Vector Machines, and Artificial Neural Networks. We applied a majority voting methodology to classify undetermined drug overdose deaths.

RESULTS

Overdose suicide rates were estimated to be underreported by 33% across all years, increasing yearly from 29% in 2012 to 37% in 2015. The overall test accuracies for all models ranged from 92.3% to 94.6%.

CONCLUSIONS

This research identifies a cost-effective, replicable, and expandable ML-based methodology to estimate the true rates of suicide which may be partially masked during the opioid epidemic.

Early-life stressful events and suicide attempt in schizophrenia: Machine learning models

Childhood abuse and neglect predicts suicide attempt. Furthermore, other early-life stressful events may predict lifetime suicide attempt in psychiatric disorders. We assessed 189 schizophrenics for suicide attempt and stressful life events. Early-life stressful events were used as predictors of lifetime suicide attempt in three machine learning models. In our sample, 38% of the schizophrenics had at least one suicide attempt lifetime. The machine learning models provided an overall significant prediction (accuracy range: 62-69%). Childhood sexual molestation and mental illness were important predictors of suicide attempt. Early-life stressful events should be included in models aiming at predicting suicide attempt in schizophrenia.

Development of an early-warning system for high-risk patients for suicide attempt using deep learning and electronic health records

Suicide is the tenth leading cause of death in the United States (US). An early-warning system (EWS) for suicide attempt could prove valuable for identifying those at risk of suicide attempts, and analyzing the contribution of repeated attempts to the risk of eventual death by suicide. In this study we sought to develop an EWS for high-risk suicide attempt patients through the development of a population-based risk stratification surveillance system. Advanced machine-learning algorithms and deep neural networks were utilized to build models with the data from electronic health records (EHRs). A final risk score was calculated for each individual and calibrated to indicate the probability of a suicide attempt in the following 1-year time period. Risk scores were subjected to individual-level analysis in order to aid in the interpretation of the results for health-care providers managing the at-risk cohorts. The 1-year suicide attempt risk model attained an area under the curve (AUC ROC) of 0.792 and 0.769 in the retrospective and prospective cohorts, respectively. The suicide attempt rate in the "very high risk" category was 60 times greater than the population baseline when tested in the prospective cohorts. Mental health disorders including depression, bipolar disorders and anxiety, along with substance abuse, impulse control disorders, clinical utilization indicators, and socioeconomic determinants were recognized as significant features associated with incident suicide attempt.

The utility of artificial intelligence in suicide risk prediction and the management of suicidal behaviors

OBJECTIVE

Suicide is a growing public health concern with a global prevalence of approximately 800,000 deaths per year. The current process of evaluating suicide risk is highly subjective, which can limit the efficacy and accuracy of prediction efforts. Consequently, suicide detection strategies are shifting toward artificial intelligence platforms that can identify patterns within 'big data' to generate risk algorithms that can determine the effects of risk (and protective) factors on suicide outcomes, predict suicide outbreaks and identify at-risk individuals or populations. In this review, we summarize the role of artificial intelligence in optimizing suicide risk prediction and behavior management.

METHODS

This paper provides a general review of the literature. A literature search was conducted in OVID Medline, EMBASE and PsycINFO databases with coverage from January 1990 to June 2019. Results were restricted to peer-reviewed, English-language articles. Conference and dissertation proceedings, case reports, protocol papers and opinion pieces were excluded. Reference lists were also examined for additional articles of relevance.

RESULTS

At the individual level, prediction analytics help to identify individuals in crisis to intervene with emotional support, crisis and psychoeducational resources, and alerts for emergency assistance. At the population level, algorithms can identify at-risk groups or suicide hotspots, which help inform resource mobilization, policy reform and advocacy efforts. Artificial intelligence has also been used to support the clinical management of suicide across diagnostics and evaluation, medication management and behavioral therapy delivery. There could be several advantages of incorporating artificial intelligence into suicide care, which includes a time- and resource-effective alternative to clinician-based strategies, adaptability to various settings and demographics, and suitability for use in remote locations with limited access to mental healthcare supports.

CONCLUSION

Based on the observed benefits to date, artificial intelligence has a demonstrated utility within suicide prediction and clinical management efforts and will continue to advance mental healthcare forward.

Machine learning in predicting graft failure following kidney transplantation: A systematic review of published predictive models

INTRODUCTION

Machine learning has been increasingly used to develop predictive models to diagnose different disease conditions. The heterogeneity of the kidney transplant population makes predicting graft outcomes extremely challenging. Several kidney graft outcome prediction models have been developed using machine learning, and are available in the literature. However, a systematic review of machine learning based prediction methods applied to kidney transplant has not been done to date. The main aim of our study was to perform an in-depth systematic analysis of different machine learning methods used to predict graft outcomes among kidney transplant patients, and assess their usefulness as an aid to decision-making.

METHODS

A systemic review of machine learning methods used to predict graft outcomes among kidney transplant patients was carried out using a search of the Medline, the Cumulative Index to Nursing and Allied Health Literature, EMBASE, PsycINFO and Cochrane databases.

RESULTS

A total of 295 articles were identified and extracted. Of these, 18 met the inclusion criteria. Most of the studies were published in the United States after 2010. The population size used to develop the models varied from 80 to 92,844, and the number of features in the models ranged from 6 to 71. The most common machine learning methods used were artificial neural networks, decision trees and Bayesian belief networks. Most of the machine learning based predictive models predicted graft failure with high sensitivity and specificity. Only one machine learning based prediction model had modelled time-to-event (survival) information. Seven studies compared the predictive performance of machine learning models with traditional regression methods and the performance of machine learning methods was found to be mixed, when compared with traditional regression methods.

CONCLUSION

There was a wide variation in the size of the study population and the input variables used. However, the prediction accuracy provided mixed results when machine learning and traditional predictive methods are compared. Based on reported gains in predictive performance, machine learning has the potential to improve kidney transplant outcome prediction and aid medical decision making.

Detection of Suicide Attempters among Suicide Ideators Using Machine Learning

OBJECTIVE

We aimed to develop predictive models to identify suicide attempters among individuals with suicide ideation using a machine learning algorithm.

METHODS

Among 35,116 individuals aged over 19 years from the Korea National Health & Nutrition Examination Survey, we selected 5,773 subjects who reported experiencing suicide ideation and had answered a survey question about suicide attempts. Then, we performed resampling with the Synthetic Minority Over-sampling TEchnique (SMOTE) to obtain data corresponding to 1,324 suicide attempters and 1,330 non-suicide attempters. We randomly assigned the samples to a training set (n=1,858) and a test set (n=796). In the training set, random forest models were trained with features selected through recursive feature elimination with 10-fold cross validation. Subsequently, the fitted model was used to predict suicide attempters in the test set.

RESULTS

In the test set, the prediction model achieved very good performance [area under receiver operating characteristic curve (AUC)=0.947] with an accuracy of 88.9%.

CONCLUSION

Our results suggest that a machine learning approach can enable the prediction of individuals at high risk of suicide through the integrated analysis of various suicide risk factors.