Reaching Those at Highest Risk for Suicide: Development of a Model Using Machine Learning Methods for use With Native American Communities

Automatically extracting social determinants of health for suicide: a narrative literature review

Suicide is a complex phenomenon that is often not preceded by a diagnosed mental health condition, therefore making it difficult to study and mitigate. Artificial Intelligence has increasingly been used to better understand Social Determinants of Health factors that influence suicide outcomes. In this review we find that many studies use limited SDoH information and minority groups are often underrepresented, thereby omitting important factors that could influence risk of suicide.

Developing a suicide risk model for use in the Indian Health Service

We developed and evaluated an electronic health record (EHR)-based model for suicide risk specific to an American Indian patient population. Using EHR data for all patients over 18 with a visit between 1/1/2017 and 10/2/2021, we developed a model for the risk of a suicide attempt or death in the 90 days following a visit. Features included demographics, medications, diagnoses, and scores from relevant screening tools. We compared the predictive performance of logistic regression and random forest models against existing suicide screening, which was augmented to include the history of previous attempts or ideation. During the study, 16,835 patients had 331,588 visits, with 490 attempts and 37 deaths by suicide. The logistic regression and random forest models (area under the ROC (AUROC) 0.83 [0.80-0.86]; both models) performed better than enhanced screening (AUROC 0.64 [0.61-0.67]). These results suggest that an EHR-based suicide risk model can add value to existing practices at Indian Health Service clinics.

Machine learning applications in studying mental health among immigrants and racial and ethnic minorities: an exploratory scoping review

BACKGROUND

The use of machine learning (ML) in mental health (MH) research is increasing, especially as new, more complex data types become available to analyze. By examining the published literature, this review aims to explore the current applications of ML in MH research, with a particular focus on its use in studying diverse and vulnerable populations, including immigrants, refugees, migrants, and racial and ethnic minorities.

METHODS

From October 2022 to March 2024, Google Scholar, EMBASE, and PubMed were queried. ML-related, MH-related, and population-of-focus search terms were strung together with Boolean operators. Backward reference searching was also conducted. Included peer-reviewed studies reported using a method or application of ML in an MH context and focused on the populations of interest. We did not have date cutoffs. Publications were excluded if they were narrative or did not exclusively focus on a minority population from the respective country. Data including study context, the focus of mental healthcare, sample, data type, type of ML algorithm used, and algorithm performance were extracted from each.

RESULTS

Ultimately, 13 peer-reviewed publications were included. All the articles were published within the last 6 years, and over half of them studied populations within the US. Most reviewed studies used supervised learning to explain or predict MH outcomes. Some publications used up to 16 models to determine the best predictive power. Almost half of the included publications did not discuss their cross-validation method.

CONCLUSIONS

The included studies provide proof-of-concept for the potential use of ML algorithms to address MH concerns in these special populations, few as they may be. Our review finds that the clinical application of these models for classifying and predicting MH disorders is still under development.

Performance of Machine Learning Suicide Risk Models in an American Indian Population

Importance

Few suicide risk identification tools have been developed specifically for American Indian and Alaska Native populations, even though these populations face the starkest suicide-related inequities.

Objective

To examine the accuracy of existing machine learning models in a majority American Indian population.

Design, Setting, and Participants

This prognostic study used secondary data analysis of electronic health record data collected from January 1, 2017, to December 31, 2021. Existing models from the Mental Health Research Network (MHRN) and Vanderbilt University (VU) were fitted. Models were compared with an augmented screening indicator that included any previous attempt, recent suicidal ideation, or a recent positive suicide risk screen result. The comparison was based on the area under the receiver operating characteristic curve (AUROC). The study was performed in partnership with a tribe and local Indian Health Service (IHS) in the Southwest. All patients were 18 years or older with at least 1 encounter with the IHS unit during the study period. Data were analyzed between October 6, 2022, and July 29, 2024.

Exposures

Suicide attempts or deaths within 90 days.

Main Outcomes and Measures

Model performance was compared based on the ability to distinguish between those with a suicide attempt or death within 90 days of their last IHS visit with those without this outcome.

Results

Of 16 835 patients (mean [SD] age, 40.0 [17.5] years; 8660 [51.4%] female; 14 251 [84.7%] American Indian), 324 patients (1.9%) had at least 1 suicide attempt, and 37 patients (0.2%) died by suicide. The MHRN model had an AUROC value of 0.81 (95% CI, 0.77-0.85) for 90-day suicide attempts, whereas the VU model had an AUROC value of 0.68 (95% CI, 0.64-0.72), and the augmented screening indicator had an AUROC value of 0.66 (95% CI, 0.63-0.70). Calibration was poor for both models but improved after recalibration.

Conclusion and Relevance

This prognostic study found that existing risk identification models for suicide prevention held promise when applied to new contexts and performed better than relying on a combined indictor of a positive suicide risk screen result, history of attempt, and recent suicidal ideation.

The use of machine learning on administrative and survey data to predict suicidal thoughts and behaviors: a systematic review

Background

Machine learning is a promising tool in the area of suicide prevention due to its ability to combine the effects of multiple risk factors and complex interactions. The power of machine learning has led to an influx of studies on suicide prediction, as well as a few recent reviews. Our study distinguished between data sources and reported the most important predictors of suicide outcomes identified in the literature.

Objective

Our study aimed to identify studies that applied machine learning techniques to administrative and survey data, summarize performance metrics reported in those studies, and enumerate the important risk factors of suicidal thoughts and behaviors identified.

Methods

A systematic literature search of PubMed, Medline, Embase, PsycINFO, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Allied and Complementary Medicine Database (AMED) to identify all studies that have used machine learning to predict suicidal thoughts and behaviors using administrative and survey data was performed. The search was conducted for articles published between January 1, 2019 and May 11, 2022. In addition, all articles identified in three recently published systematic reviews (the last of which included studies up until January 1, 2019) were retained if they met our inclusion criteria. The predictive power of machine learning methods in predicting suicidal thoughts and behaviors was explored using box plots to summarize the distribution of the area under the receiver operating characteristic curve (AUC) values by machine learning method and suicide outcome (i.e., suicidal thoughts, suicide attempt, and death by suicide). Mean AUCs with 95% confidence intervals (CIs) were computed for each suicide outcome by study design, data source, total sample size, sample size of cases, and machine learning methods employed. The most important risk factors were listed.

Results

The search strategy identified 2,200 unique records, of which 104 articles met the inclusion criteria. Machine learning algorithms achieved good prediction of suicidal thoughts and behaviors (i.e., an AUC between 0.80 and 0.89); however, their predictive power appears to differ across suicide outcomes. The boosting algorithms achieved good prediction of suicidal thoughts, death by suicide, and all suicide outcomes combined, while neural network algorithms achieved good prediction of suicide attempts. The risk factors for suicidal thoughts and behaviors differed depending on the data source and the population under study.

Conclusion

The predictive utility of machine learning for suicidal thoughts and behaviors largely depends on the approach used. The findings of the current review should prove helpful in preparing future machine learning models using administrative and survey data.

Systematic review registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022333454 identifier CRD42022333454.

Factors Associated With Suicide in Four West African Countries Among Adolescent Students: An Analysis Using the Global School-Based Student Health Survey

PURPOSE

Globally, suicide is a leading cause of death among adolescents, with the highest burden of suicide occurring in Africa. Despite this, little is known about the epidemiology of suicide among adolescents in West Africa. In this study, we explore suicidality among West African adolescents.

METHODS

Using pooled data from the Global School-Based Student Health Survey in four West African countries (Ghana, Benin, Liberia, and Sierra Leone), we investigated the prevalence of suicidal ideation and suicide attempt and examined associations with 15 covariates using univariate and multivariable logistic regression.

RESULTS

Overall, 18.6% of adolescents in the pooled sample (N = 9,726) had considered suicide, while 24.7% reported attempting suicide. Significant correlates of suicide attempt included older age (16+ years; odds ratio [OR]: 1.70, confidence interval [CI]: 1.09-2.63), difficulty sleeping due to worry (OR: 1.27, CI: 1.04-1.56), loneliness (OR: 1.65, CI: 1.39-1.96), truancy (OR: 1.38. CI: 1.05-1.82), being a target of bullying (OR: 1.53, CI: 1.26-1.85), getting physically attacked (OR: 1.73, CI: 1.42-2.11), physical fighting (OR: 1.47, CI: 1.21-1.79), current cigarette use (OR: 2.71, CI: 1.88-3.89), and initiation of drug use (OR: 2.19, CI: 1.71-2.81). Conversely, having close friends was associated with lower odds of suicide attempt (OR: 0.67, CI: 0.48-0.93). Several covariates were also significantly associated with suicidal ideation.

DISCUSSION

Suicidal ideation and attempts are highly prevalent among school-going adolescents in these West African countries. Multiple modifiable risk and protective factors were identified. Programs, interventions, and policies aimed at addressing these factors may play a significant role in preventing suicides in these countries.

Validating a predictive algorithm for suicide risk with Alaska Native populations

INTRODUCTION

The American Indian/Alaska Native (AI/AN) suicide rate in Alaska is twice the state rate and four times the U.S. rate. Healthcare systems need innovative methods of suicide risk detection. The Mental Health Research Network (MHRN) developed suicide risk prediction algorithms in a general U.S.

PATIENT POPULATION

METHODS

We applied MHRN predictors and regression coefficients to electronic health records of AI/AN patients aged ≥13 years with behavioral health diagnoses and primary care visits between October 1, 2016, and March 30, 2018. Logistic regression assessed model accuracy for predicting and stratifying risk for suicide attempt within 90 days after a visit. We compared expected to observed risk and assessed model performance characteristics.

RESULTS

10,864 patients made 47,413 primary care visits. Suicide attempt occurred after 589 (1.2%) visits. Visits in the top 5% of predicted risk accounted for 40% of actual attempts. Among visits in the top 0.5% of predicted risk, 25.1% were followed by suicide attempt. The best fitting model had an AUC of 0.826 (95% CI: 0.809-0.843).

CONCLUSIONS

The MHRN model accurately predicted suicide attempts among AI/AN patients. Future work should develop clinical and operational guidance for effective implementation of the model with this population.

Artificial intelligence and suicide prevention: a systematic review

Background

Suicide is one of the main preventable causes of death. Artificial intelligence (AI) could improve methods for assessing suicide risk. The objective of this review is to assess the potential of AI in identifying patients who are at risk of attempting suicide.

Methods

A systematic review of the literature was conducted on PubMed, EMBASE, and SCOPUS databases, using relevant keywords.

Results

Thanks to this research, 296 studies were identified. Seventeen studies, published between 2014 and 2020 and matching inclusion criteria, were selected as relevant. Included studies aimed at predicting individual suicide risk or identifying at-risk individuals in a specific population. The AI performance was overall good, although variable across different algorithms and application settings.

Conclusions

AI appears to have a high potential for identifying patients at risk of suicide. The precise use of these algorithms in clinical situations, as well as the ethical issues it raises, remain to be clarified.

Leveraging data science to enhance suicide prevention research: a literature review

OBJECTIVE

The purpose of this research is to identify how data science is applied in suicide prevention literature, describe the current landscape of this literature and highlight areas where data science may be useful for future injury prevention research.

DESIGN

We conducted a literature review of injury prevention and data science in April 2020 and January 2021 in three databases.

METHODS

For the included 99 articles, we extracted the following: (1) author(s) and year; (2) title; (3) study approach (4) reason for applying data science method; (5) data science method type; (6) study description; (7) data source and (8) focus on a disproportionately affected population.

RESULTS

Results showed the literature on data science and suicide more than doubled from 2019 to 2020, with articles with individual-level approaches more prevalent than population-level approaches. Most population-level articles applied data science methods to describe (n=10) outcomes, while most individual-level articles identified risk factors (n=27). Machine learning was the most common data science method applied in the studies (n=48). A wide array of data sources was used for suicide research, with most articles (n=45) using social media and web-based behaviour data. Eleven studies demonstrated the value of applying data science to suicide prevention literature for disproportionately affected groups.

CONCLUSION

Data science techniques proved to be effective tools in describing suicidal thoughts or behaviour, identifying individual risk factors and predicting outcomes. Future research should focus on identifying how data science can be applied in other injury-related topics.

Comparing the predictive value of screening to the use of electronic health record data for detecting future suicidal thoughts and behavior in an urban pediatric emergency department: A preliminary analysis

AIM

Brief screening and predictive modeling have garnered attention for utility at identifying individuals at risk of suicide. Although previous research has investigated these methods, little is known about how these methods compare against each other or work in combination in the pediatric population.

METHODS

Patients were aged 8-18 years old who presented from January 1, 2017, to June 30, 2019, to a Pediatric Emergency Department (PED). All patients were screened with the Ask Suicide Questionnaire (ASQ) as part of a universal screening approach. For all models, we used 5-fold cross-validation. We compared four models: Model 1 only included the ASQ; Model 2 included the ASQ and EHR data gathered at the time of ED visit (EHR data); Model 3 only included EHR data; and Model 4 included EHR data and a single item from the ASQ that asked about a lifetime history of suicide attempt. The main outcome was subsequent PED visit with suicide-related presenting problem within a 3-month follow-up period.

RESULTS

Of the N = 13,420 individuals, n = 141 had a subsequent suicide-related PED visit. Approximately 63% identified as Black. Results showed that a model based only on EHR data (Model 3) had an area under the curve (AUC) of 0.775 compared to the ASQ alone (Model 1), which had an AUC of 0.754. Combining screening and EHR data (Model 4) resulted in a 17.4% (absolute difference = 3.6%) improvement in sensitivity and 13.4% increase in AUC (absolute difference = 6.6%) compared to screening alone (Model 1).

CONCLUSION

Our findings show that predictive modeling based on EHR data is helpful either in the absence or as an addition to brief suicide screening. This is the first study to compare brief suicide screening to EHR-based predictive modeling and adds to our understanding of how best to identify youth at risk of suicidal thoughts and behaviors in clinical care settings.

Designing a Clinical Decision Support Tool That Leverages Machine Learning for Suicide Risk Prediction: Development Study in Partnership With Native American Care Providers

Background

Machine learning algorithms for suicide risk prediction have been developed with notable improvements in accuracy. Implementing these algorithms to enhance clinical care and reduce suicide has not been well studied.

Objective

This study aims to design a clinical decision support tool and appropriate care pathways for community-based suicide surveillance and case management systems operating on Native American reservations.

Methods

Participants included Native American case managers and supervisors (N=9) who worked on suicide surveillance and case management programs on 2 Native American reservations. We used in-depth interviews to understand how case managers think about and respond to suicide risk. The results from interviews informed a draft clinical decision support tool, which was then reviewed with supervisors and combined with appropriate care pathways.

Results

Case managers reported acceptance of risk flags based on a predictive algorithm in their surveillance system tools, particularly if the information was available in a timely manner and used in conjunction with their clinical judgment. Implementation of risk flags needed to be programmed on a dichotomous basis, so the algorithm could produce output indicating high versus low risk. To dichotomize the continuous predicted probabilities, we developed a cutoff point that favored specificity, with the understanding that case managers’ clinical judgment would help increase sensitivity.

Conclusions

Suicide risk prediction algorithms show promise, but implementation to guide clinical care remains relatively elusive. Our study demonstrates the utility of working with partners to develop and guide the operationalization of risk prediction algorithms to enhance clinical care in a community setting.

Suicide After Stroke in the United States Veteran Health Administration Population

OBJECTIVE

To evaluate risk for suicide among veterans with a history of stroke, seeking care within the Veterans Health Administration (VHA), we analyzed existing clinical data.

DESIGN

This retrospective cohort study was approved and performed in accordance with the local Institutional Review Board. Veterans were identified via the VHA's Corporate Data Warehouse. Initial eligibility criteria included confirmed veteran status and at least 90 days of VHA utilization between fiscal years 2001-2015. Cox proportional hazards models were used to assess the association between history of stroke and suicide. Among those veterans who died by suicide, the association between history of stroke and method of suicide was also investigated.

SETTING

VHA.

PARTICIPANTS

Veterans with at least 90 days of VHA utilization between fiscal years 2001-2015 (N=1,647,671). Data from these 1,647,671 veterans were analyzed (1,405,762 without stroke and 241,909 with stroke).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Suicide and method of suicide.

RESULTS

The fully adjusted model, which controlled for age, sex, mental health diagnoses, mild traumatic brain injury, and modified Charlson/Deyo Index (stroke-related diagnoses excluded), demonstrated a hazard ratio of 1.13 (95% confidence interval, 1.02-1.25; P=.02). The majority of suicides in both cohorts was by firearm, and a significantly larger proportion of suicides occurred by firearm in the group with stroke than the cohort without (81.2% vs 76.6%).

CONCLUSIONS

Findings suggest that veterans with a history of stroke are at increased risk for suicide, specifically by firearm, compared with veterans without a history of stroke. Increased efforts are needed to address the mental health needs and lethal means safety of veterans with a history of stroke, with the goal of improving function and decreasing negative psychiatric outcomes, such as suicide.

A Comprehensive Review of Computer-Aided Diagnosis of Major Mental and Neurological Disorders and Suicide: A Biostatistical Perspective on Data Mining

The World Health Organization (WHO) suggests that mental disorders, neurological disorders, and suicide are growing causes of morbidity. Depressive disorders, schizophrenia, bipolar disorder, Alzheimer's disease, and other dementias account for 1.84%, 0.60%, 0.33%, and 1.00% of total Disability Adjusted Life Years (DALYs). Furthermore, suicide, the 15th leading cause of death worldwide, could be linked to mental disorders. More than 68 computer-aided diagnosis (CAD) methods published in peer-reviewed journals from 2016 to 2021 were analyzed, among which 75% were published in the year 2018 or later. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol was adopted to select the relevant studies. In addition to the gold standard, the sample size, neuroimaging techniques or biomarkers, validation frameworks, the classifiers, and the performance indices were analyzed. We further discussed how various performance indices are essential based on the biostatistical and data mining perspective. Moreover, critical information related to the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines was analyzed. We discussed how balancing the dataset and not using external validation could hinder the generalization of the CAD methods. We provided the list of the critical issues to consider in such studies.

Sustaining suicide prevention programs in American Indian and Alaska Native communities and Tribal health centers

Background

Research on sustaining community-based interventions is limited. This is particularly true for suicide prevention programs and in American Indian and Alaska Native (AIAN) settings. Aiming to inform research in this area, this paper sought to identify factors and strategies that are key to sustain suicide prevention efforts in AIAN communities.

Methods

We used a modified Nominal Group Technique with a purposeful sample of N = 35 suicide prevention research experts, program implementors and AIAN community leaders to develop a list of prioritized factors and sustainability strategies. We then compared this list with the Public Health Program Capacity for Sustainability Framework (PHPCSF) to examine the extent the factors identified aligned with the existing literature.

Results

Major factors identified included cultural fit of intervention approaches, buy in from local communities, importance of leadership and policy making, and demonstrated program success. Strategies to promote these factors included partnership building, continuous growth of leadership, policy development, and ongoing strategic planning and advocacy. All domains of the PHPCF were representative, but additional factors and strategies were identified that emerged as important in AIAN settings.

Conclusions

Sustaining effective and culturally informed suicide prevention efforts is of paramount importance to prevent suicide and save lives. Future research will focus on generating empirical evidence of these strategies and their effectiveness at promoting program sustainability in AIAN communities.

Suicide in the pediatric population: screening, risk assessment and treatment

The number of children and adolescents dying by suicide is increasing over time. Patterns for who is at risk are also changing, leading to a need to review clinical suicide prevention progress and identify limitations with existing practices and research that can help us further address this growing problem. This paper aims to synthesise the literature on paediatric suicide screening, risk assessment and treatment to inform clinical practice and suicide prevention efforts. Our review shows that universal screening is strongly recommended, feasible and acceptable, and that there are screening tools that have been validated with youth. However, screening may not accurately identify those at risk of dying due to the relative rarity of suicide death and the associated research and clinical challenges in studying such a rare event and predicting future behaviour. Similarly, while risk assessments have been developed and tested in some populations, there is limited research on their validity and challenges with their implementation. Several promising suicide-specific treatments have been developed for youth, but overall there is an insufficient number of randomised trials. Despite great need, the research evidence to support screening, risk assessment and treatment is still limited. As suicide rates increase for children and adolescents, continued research in all three domains is needed to reverse this trend.