Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans health Administration

Age-stratified predictions of suicide attempts using machine learning in middle and late adolescence

BACKGROUND

Prevalence of suicidal behaviour increases rapidly in middle to late adolescence. Predicting suicide attempts across different ages would enhance our understanding of how suicidal behaviour manifests in this period of rapid development. This study aimed to develop separate models to predict suicide attempts within a cohort at middle and late adolescence. It also sought to examine differences between the models derived across both developmental stages.

METHODS

This study used data from the nationally representative Longitudinal Study of Australian Children (N = 2266). We selected over 700 potential suicide attempt predictors measured via self-report questionnaires, and linked healthcare and education administrative datasets. Logistic regression, random forests, and gradient boosting algorithms were developed to predict suicide attempts across two stages (mid-adolescence: 14-15 years; late adolescence: 18-19 years) using predictors sampled two years prior (mid-adolescence: 12-13 years; late adolescence: 16-17 years).

RESULTS

The late adolescence models (AUROC = 0.77-0.88, F1-score = 0.22-0.28, Sensitivity = 0.54-0.64) performed better than the mid-adolescence models (AUROC = 0.70-0.76, F1-score = 0.12-0.19, Sensitivity = 0.40-0.64). The most important features for predicting suicide attempts in mid-adolescence were mostly school-related, while the most important features in late adolescence included measures of prior suicidality, psychosocial health, and future plans.

CONCLUSIONS

To date, this is the first study to use machine learning models to predict suicide attempts at different ages. Our findings suggest that the optimal suicide risk prediction model differs by stage of adolescence. Future research and interventions should consider that risk presentations can change rapidly during adolescence.

Using natural language processing to evaluate temporal patterns in suicide risk variation among high-risk Veterans

Measuring suicide risk fluctuation remains difficult, especially for high-suicide risk patients. Our study addressed this issue by leveraging Dynamic Topic Modeling, a natural language processing method that evaluates topic changes over time, to analyze high-suicide risk Veterans Affairs patients' unstructured electronic health records. Our sample included all high-risk patients that died (cases) or did not (controls) by suicide in 2017 and 2018. Cases and controls shared the same risk, location, and treatment intervals and received nine months of mental health care during the year before the relevant end date. Each case was matched with five controls. We analyzed case records from diagnosis until death and control records from diagnosis until matched case's death date. Our final sample included 218 cases and 943 controls. We analyzed the corpus using a Python-based Dynamic Topic Modeling algorithm. We identified five distinct topics, "Medication," "Intervention," "Treatment Goals," "Suicide," and "Treatment Focus." We observed divergent change patterns over time, with pathology-focused care increasing for cases and supportive care increasing for controls. The case topics tended to fluctuate more than the control topics, suggesting the importance of monitoring lability. Our study provides a method for monitoring risk fluctuation and strengthens the groundwork for time-sensitive risk measurement.

Evaluating accuracy and fairness of clinical decision support algorithms when health care resources are limited

OBJECTIVE

Guidance on how to evaluate accuracy and algorithmic fairness across subgroups is missing for clinical models that flag patients for an intervention but when health care resources to administer that intervention are limited. We aimed to propose a framework of metrics that would fit this specific use case.

METHODS

We evaluated the following metrics and applied them to a Veterans Health Administration clinical model that flags patients for intervention who are at risk of overdose or a suicidal event among outpatients who were prescribed opioids (N = 405,817): Receiver - Operating Characteristic and area under the curve, precision - recall curve, calibration - reliability curve, false positive rate, false negative rate, and false omission rate. In addition, we developed a new approach to visualize false positives and false negatives that we named 'per true positive bars.' We demonstrate the utility of these metrics to our use case for three cohorts of patients at the highest risk (top 0.5 %, 1.0 %, and 5.0 %) by evaluating algorithmic fairness across the following age groups: <=30, 31-50, 51-65, and >65 years old.

RESULTS

Metrics that allowed us to assess group differences more clearly were the false positive rate, false negative rate, false omission rate, and the new 'per true positive bars'. Metrics with limited utility to our use case were the Receiver - Operating Characteristic and area under the curve, the calibration - reliability curve, and the precision - recall curve.

CONCLUSION

There is no "one size fits all" approach to model performance monitoring and bias analysis. Our work informs future researchers and clinicians who seek to evaluate accuracy and fairness of predictive models that identify patients to intervene on in the context of limited health care resources. In terms of ease of interpretation and utility for our use case, the new 'per true positive bars' may be the most intuitive to a range of stakeholders and facilitates choosing a threshold that allows weighing false positives against false negatives, which is especially important when predicting severe adverse events.

Impact of Implementation Facilitation on the REACH VET Clinical Program for Veterans at Risk for Suicide

OBJECTIVE

In 2017, the Veterans Health Administration (VHA) implemented a national suicide prevention program, called Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment (REACH VET), that uses a predictive algorithm to identify, attempt to reach, assess, and care for patients at the highest risk for suicide. The authors aimed to evaluate whether facilitation enhanced implementation of REACH VET at VHA facilities not meeting target completion rates.

METHODS

In this hybrid effectiveness-implementation type 2 program evaluation, a quasi-experimental pre-post design was used to assess changes in implementation outcome measures evaluated 6 months before and 6 months after onset of facilitation of REACH VET implementation at 23 VHA facilities. Measures included percentages of patients with documented coordinator and provider acknowledgment of receipt, care evaluation, and outreach attempt. Generalized estimating equations were used to compare differences in REACH VET outcome measures before and after facilitation. Qualitative interviews were conducted with personnel and were explored via template analysis.

RESULTS

Time had a significant effect in all outcomes models (p<0.001). An effect of facilitation was significant only for the outcome of attempted outreach. Patients identified by REACH VET had significantly higher odds of having a documented outreach attempt after facilitation of REACH VET implementation, compared with before facilitation. Site personnel felt supported and reported that the external facilitators were helpful and responsive.

CONCLUSIONS

Facilitation of REACH VET implementation was associated with an improvement in outreach attempts to veterans identified as being at increased risk for suicide. Outreach is critical for engaging veterans in care.

Suicide Death Prediction Using the Maryland Suicide Data Warehouse: A Sensitivity Analysis

OBJECTIVE

Nearly 50,000 Americans die each year from suicide, despite suicide death being a rare event in the context of health risk assessment and modeling. Prior research has underscored the need for contextualizing suicide risk models in terms of their potential uses and generalizability. This sensitivity analysis makes use of the Maryland Suicide Data Warehouse (MSDW) and illustrates how results inform clinical decision support.

METHOD

A cohort of 1 million living control patients were extracted from the MSDW in addition to 1,667 patients who had died by suicide between the years 2016 and 2019 according to the Maryland Office of the Medical Examiner (OCME). Data were extracted and aggregated as part of a 4-year retrospective design. Binary logistic and two penalized regression models were deployed in a repeated fivefold cross-validation. Model performances were evaluated using sensitivity, positive predictive value (PPV), and F1, and model coefficients were ranked according to coefficient size.

RESULTS

Several features were significantly associated with patients having died by suicide, including male sex, depressive and anxiety disorder diagnoses, social needs, and prior suicidal ideation and suicide attempt. Cross-validated binary logistic regression outperformed either ridge or LASSO (least absolute shrinkage and selection operator) models but generally achieved low-to-moderate PPV and sensitivity across most thresholds and a peak F1 of 0.323.

CONCLUSIONS

Suicide death prediction is constrained by the context of use, which determines the best balance of precision and recall. Predictive models must be evaluated close to the level of intervention. They may not hold up to different needs at different levels of care.

Predicting inmate suicidal behavior with an interpretable ensemble machine learning approach in smart prisons

The convergence of smart technologies and predictive modelling in prisons presents an exciting opportunity to revolutionize the monitoring of inmate behaviour, allowing for the early detection of signs of distress and the effective mitigation of suicide risks. While machine learning algorithms have been extensively employed in predicting suicidal behaviour, a critical aspect that has often been overlooked is the interoperability of these models. Most of the work done on model interpretations for suicide predictions often limits itself to feature reduction and highlighting important contributing features only. To address this research gap, we used Anchor explanations for creating human-readable statements based on simple rules, which, to our knowledge, have never been used before for suicide prediction models. We also overcome the limitation of anchor explanations, which create weak rules on high-dimensionality datasets, by first reducing data features with the help of SHapley Additive exPlanations (SHAP). We further reduce data features through anchor interpretations for the final ensemble model of XGBoost and random forest. Our results indicate significant improvement when compared with state-of-the-art models, having an accuracy and precision of 98.6% and 98.9%, respectively. The F1-score for the best suicide ideation model appeared to be 96.7%.

Prediction of suicide attempts among persons with depression: a population-based case cohort study

Studies have highlighted the potential importance of modeling interactions for suicide attempt prediction. This case-cohort study identified risk factors for suicide attempts among persons with depression in Denmark using statistical approaches that do (random forests) or do not (least absolute shrinkage and selection operator regression [LASSO]) model interactions. Cases made a nonfatal suicide attempt (n = 6032) between 1995 and 2015. The comparison subcohort was a 5% random sample of all persons in Denmark on January 1, 1995 (n = 11 963). We used random forests and LASSO for sex-stratified prediction of suicide attempts from demographic variables, psychiatric and somatic diagnoses, and treatments. Poisonings, psychiatric disorders, and medications were important predictors for both sexes. Area under the receiver-operating characteristic curve (AUC) values were higher in LASSO models (in men, 0.85, 95% CI, 0.84-0.86; in women, 0.89, 95% C, 0.88-0.90) than random forests (in men, 0.76, 95% CI, 0.74-0.78; in women, 0.79, 95% CI = 0.78-0.81). Automatic detection of interactions via random forests did not result in better model performance than LASSO models that did not model interactions. Due to the complex nature of psychiatric comorbidity and suicide, modeling interactions may not always be the optimal statistical approach to enhancing suicide attempt prediction in high-risk samples. This article is part of a Special Collection on Mental Health.

Developing a clinical decision support system software prototype that assists in the management of patients with self-harm in the emergency department: protocol of the PERMANENS project

BACKGROUND

Self-harm presents a significant public health challenge. Emergency departments (EDs) are crucial healthcare settings in managing self-harm, but clinician uncertainty in risk assessment may contribute to ineffective care. Clinical Decision Support Systems (CDSSs) show promise in enhancing care processes, but their effective implementation in self-harm management remains unexplored.

METHODS

PERMANENS comprises a combination of methodologies and study designs aimed at developing a CDSS prototype that assists clinicians in the personalized assessment and management of ED patients presenting with self-harm. Ensemble prediction models will be constructed by applying machine learning techniques on electronic registry data from four sites, i.e., Catalonia (Spain), Ireland, Norway, and Sweden. These models will predict key adverse outcomes including self-harm repetition, suicide, premature death, and lack of post-discharge care. Available registry data include routinely collected electronic health record data, mortality data, and administrative data, and will be harmonized using the OMOP Common Data Model, ensuring consistency in terminologies, vocabularies and coding schemes. A clinical knowledge base of effective suicide prevention interventions will be developed rooted in a systematic review of clinical practice guidelines, including quality assessment of guidelines using the AGREE II tool. The CDSS software prototype will include a backend that integrates the prediction models and the clinical knowledge base to enable accurate patient risk stratification and subsequent intervention allocation. The CDSS frontend will enable personalized risk assessment and will provide tailored treatment plans, following a tiered evidence-based approach. Implementation research will ensure the CDSS' practical functionality and feasibility, and will include periodic meetings with user-advisory groups, mixed-methods research to identify currently unmet needs in self-harm risk assessment, and small-scale usability testing of the CDSS prototype software.

DISCUSSION

Through the development of the proposed CDSS software prototype, PERMANENS aims to standardize care, enhance clinician confidence, improve patient satisfaction, and increase treatment compliance. The routine integration of CDSS for self-harm risk assessment within healthcare systems holds significant potential in effectively reducing suicide mortality rates by facilitating personalized and timely delivery of effective interventions on a large scale for individuals at risk of suicide.

Association between cognitive functioning and lifetime suicidal ideation among Chinese older adults: the mediating effect of depression

PURPOSE

Existing evidence indicates an association between cognitive functioning and both geriatric depression and suicidality, with mixed evidence regarding the direction of the relationship between cognitive functioning and aspects of geriatric lifetime suicidal ideation. This study aims to examine the relationship between cognitive functioning, depression, and suicide ideation and to explore the intermediary role of depression between cognitive functioning and suicidal ideation in the older adults.

METHODS

A multi-stage random cluster sampling method was used to collect a sample of 3896 individuals aged 60 and above. Descriptive statistics of the sample data were analyzed using one-way ANOVA, and then the correlation between variables was obtained by binary logistic regression analysis. SPSS macro program PROCESS V3.5 was used to test the mediating role of depression in the relationship between cognitive function and lifetime suicidal ideation.

RESULTS

The prevalence of lifetime suicidal ideation among older adults was 3.9%. Lifetime suicidal ideation was associated with depression (OR = 1.308, P < 0.001) but was not significantly correlated with cognitive function (OR = 0.972, P > 0.05). The relationship between cognitive function and depression was also supported in this study (β = - 0.0841, P < 0.001). Depression completely mediated the relationship between cognitive function and lifetime suicidal ideation.

CONCLUSION

There was no significant correlation between cognitive impairment in older adults and a heightened risk of lifetime suicidal ideation. However, this relationship was completely mediated by depression. It is crucial to prevent the onset of depression among older adults with cognitive impairment, as depression is strongly linked to lifetime suicidal ideation.

Stability of Suicide Risk Prediction Models During Changes in Health Care Delivery

OBJECTIVE

The authors aimed to use health records data to examine how the accuracy of statistical models predicting self-harm or suicide changed between 2015 and 2019, as health systems implemented suicide prevention programs.

METHODS

Data from four large health systems were used to identify specialty mental health visits by patients ages ≥11 years, assess 311 potential predictors of self-harm (including demographic characteristics, historical risk factors, and index visit characteristics), and ascertain fatal or nonfatal self-harm events over 90 days after each visit. New prediction models were developed with logistic regression with LASSO (least absolute shrinkage and selection operator) in random samples of visits (65%) from each calendar year and were validated in the remaining portion of the sample (35%).

RESULTS

A model developed for visits from 2009 to mid-2015 showed similar classification performance and calibration accuracy in a new sample of about 13.1 million visits from late 2015 to 2019. Area under the receiver operating characteristic curve (AUC) ranged from 0.840 to 0.849 in the new sample, compared with 0.851 in the original sample. New models developed for each year for 2015-2019 had classification performance (AUC range 0.790-0.853), sensitivity, and positive predictive value similar to those of the previously developed model. Models selected similar predictors from 2015 to 2019, except for more frequent selection of depression questionnaire data in later years, when questionnaires were more frequently recorded.

CONCLUSIONS

A self-harm prediction model developed with 2009-2015 visit data performed similarly when applied to 2015-2019 visits. New models did not yield superior performance or identify different predictors.

Development of a 3-Step theory of suicide ontology to facilitate 3ST factor extraction from clinical progress notes

OBJECTIVE

Suicide risk prediction algorithms at the Veterans Health Administration (VHA) do not include predictors based on the 3-Step Theory of suicide (3ST), which builds on hopelessness, psychological pain, connectedness, and capacity for suicide. These four factors are not available from structured fields in VHA electronic health records, but they are found in unstructured clinical text. An ontology and controlled vocabulary that maps psychosocial and behavioral terms to these factors does not exist. The objectives of this study were 1) to develop an ontology with a controlled vocabulary of terms that map onto classes that represent the 3ST factors as identified within electronic clinical progress notes, and 2) to determine the accuracy of automated extractions based on terms in the controlled vocabulary.

METHODS

A team of four annotators did linguistic annotation of 30,000 clinical progress notes from 231 Veterans in VHA electronic health records who attempted suicide or who died by suicide for terms relating to the 3ST factors. Annotation involved manually assigning a label to words or phrases that indicated presence or absence of the factor (polarity). These words and phrases were entered into a controlled vocabulary that was then used by our computational system to tag 14 million clinical progress notes from Veterans who attempted or died by suicide after 2013. Tagged text was extracted and machine-labelled for presence or absence of the 3ST factors. Accuracy of these machine-labels was determined for 1000 randomly selected extractions for each factor against a ground truth created by our annotators.

RESULTS

Linguistic annotation identified 8486 terms that related to 33 subclasses across the four factors and polarities. Precision of machine-labeled extractions ranged from 0.73 to 1.00 for most factor-polarity combinations, whereas recall was somewhat lower 0.65-0.91.

CONCLUSION

The ontology that was developed consists of classes that represent each of the four 3ST factors, subclasses, relationships, and terms that map onto those classes which are stored in a controlled vocabulary (https://bioportal.bioontology.org/ontologies/THREE-ST). The use case that we present shows how scores based on clinical notes tagged for terms in the controlled vocabulary capture meaningful change in the 3ST factors during weeks preceding a suicidal event.

High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of [Formula: see text] 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.

Artificial intelligence for dementia prevention

INTRODUCTION

A wide range of modifiable risk factors for dementia have been identified. Considerable debate remains about these risk factors, possible interactions between them or with genetic risk, and causality, and how they can help in clinical trial recruitment and drug development. Artificial intelligence (AI) and machine learning (ML) may refine understanding.

METHODS

ML approaches are being developed in dementia prevention. We discuss exemplar uses and evaluate the current applications and limitations in the dementia prevention field.

RESULTS

Risk-profiling tools may help identify high-risk populations for clinical trials; however, their performance needs improvement. New risk-profiling and trial-recruitment tools underpinned by ML models may be effective in reducing costs and improving future trials. ML can inform drug-repurposing efforts and prioritization of disease-modifying therapeutics.

DISCUSSION

ML is not yet widely used but has considerable potential to enhance precision in dementia prevention.

HIGHLIGHTS

Artificial intelligence (AI) is not widely used in the dementia prevention field. Risk-profiling tools are not used in clinical practice. Causal insights are needed to understand risk factors over the lifespan. AI will help personalize risk-management tools for dementia prevention. AI could target specific patient groups that will benefit most for clinical trials.

Identifying suicide documentation in clinical notes through zero-shot learning

Background and Aims

In deep learning, a major difficulty in identifying suicidality and its risk factors in clinical notes is the lack of training samples given the small number of true positive instances among the number of patients screened. This paper describes a novel methodology that identifies suicidality in clinical notes by addressing this data sparsity issue through zero-shot learning. Our general aim was to develop a tool that leveraged zero-shot learning to effectively identify suicidality documentation in all types of clinical notes.

Methods

US Veterans Affairs clinical notes served as data. The training data set label was determined using diagnostic codes of suicide attempt and self-harm. We used a base string associated with the target label of suicidality to provide auxiliary information by narrowing the positive training cases to those containing the base string. We trained a deep neural network by mapping the training documents' contents to a semantic space. For comparison, we trained another deep neural network using the identical training data set labels, and bag-of-words features.

Results

The zero-shot learning model outperformed the baseline model in terms of area under the curve, sensitivity, specificity, and positive predictive value at multiple probability thresholds. In applying a 0.90 probability threshold, the methodology identified notes documenting suicidality but not associated with a relevant ICD-10-CM code, with 94% accuracy.

Conclusion

This method can effectively identify suicidality without manual annotation.

Derivation and validation of risk prediction for posttraumatic stress symptoms following trauma exposure

BACKGROUND

Posttraumatic stress symptoms (PTSS) are common following traumatic stress exposure (TSE). Identification of individuals with PTSS risk in the early aftermath of TSE is important to enable targeted administration of preventive interventions. In this study, we used baseline survey data from two prospective cohort studies to identify the most influential predictors of substantial PTSS.

METHODS

Self-identifying black and white American women and men (n = 1546) presenting to one of 16 emergency departments (EDs) within 24 h of motor vehicle collision (MVC) TSE were enrolled. Individuals with substantial PTSS (⩾33, Impact of Events Scale - Revised) 6 months after MVC were identified via follow-up questionnaire. Sociodemographic, pain, general health, event, and psychological/cognitive characteristics were collected in the ED and used in prediction modeling. Ensemble learning methods and Monte Carlo cross-validation were used for feature selection and to determine prediction accuracy. External validation was performed on a hold-out sample (30% of total sample).

RESULTS

Twenty-five percent (n = 394) of individuals reported PTSS 6 months following MVC. Regularized linear regression was the top performing learning method. The top 30 factors together showed good reliability in predicting PTSS in the external sample (Area under the curve = 0.79 ± 0.002). Top predictors included acute pain severity, recovery expectations, socioeconomic status, self-reported race, and psychological symptoms.

CONCLUSIONS

These analyses add to a growing literature indicating that influential predictors of PTSS can be identified and risk for future PTSS estimated from characteristics easily available/assessable at the time of ED presentation following TSE.

Intentional Self-Harm Among US Veterans With Traumatic Brain Injury or Posttraumatic Stress Disorder: Retrospective Cohort Study From 2008 to 2017

BACKGROUND

Veterans with a history of traumatic brain injury (TBI) and/or posttraumatic stress disorder (PTSD) may be at increased risk of suicide attempts and other forms of intentional self-harm as compared to veterans without TBI or PTSD.

OBJECTIVE

Using administrative data from the US Veterans Health Administration (VHA), we studied associations between TBI and PTSD diagnoses, and subsequent diagnoses of intentional self-harm among US veterans who used VHA health care between 2008 and 2017.

METHODS

All veterans with encounters or hospitalizations for intentional self-harm were assigned "index dates" corresponding to the date of the first related visit; among those without intentional self-harm, we randomly selected a date from among the veteran's health care encounters to match the distribution of case index dates over the 10-year period. We then examined the prevalence of TBI and PTSD diagnoses within the 5-year period prior to veterans' index dates. TBI, PTSD, and intentional self-harm were identified using International Classification of Diseases diagnosis and external cause of injury codes from inpatient and outpatient VHA encounters. We stratified analyses by veterans' average yearly VHA utilization in the 5-year period before their index date (low, medium, or high). Variations in prevalence and odds of intentional self-harm diagnoses were compared by veterans' prior TBI and PTSD diagnosis status (TBI only, PTSD only, and comorbid TBI/PTSD) for each VHA utilization stratum. Multivariable models adjusted for age, sex, race, ethnicity, marital status, Department of Veterans Affairs service-connection status, and Charlson Comorbidity Index scores.

RESULTS

About 6.7 million veterans with at least two VHA visits in the 5-year period before their index dates were included in the analyses; 86,644 had at least one intentional self-harm diagnosis during the study period. During the periods prior to veterans' index dates, 93,866 were diagnosed with TBI only; 892,420 with PTSD only; and 102,549 with comorbid TBI/PTSD. Across all three VHA utilization strata, the prevalence of intentional self-harm diagnoses was higher among veterans diagnosed with TBI, PTSD, or TBI/PTSD than among veterans with neither diagnosis. The observed difference was most pronounced among veterans in the high VHA utilization stratum. The prevalence of intentional self-harm was six times higher among those with comorbid TBI/PTSD (6778/58,295, 11.63%) than among veterans with neither TBI nor PTSD (21,979/1,144,991, 1.92%). Adjusted odds ratios suggested that, after accounting for potential confounders, veterans with TBI, PTSD, or comorbid TBI/PTSD had higher odds of self-harm compared to veterans without these diagnoses. Among veterans with high VHA utilization, those with comorbid TBI/PTSD were 4.26 (95% CI 4.15-4.38) times more likely to receive diagnoses for intentional self-harm than veterans with neither diagnosis. This pattern was similar for veterans with low and medium VHA utilization.

CONCLUSIONS

Veterans with TBI and/or PTSD diagnoses, compared to those with neither diagnosis, were substantially more likely to be subsequently diagnosed with intentional self-harm between 2008 and 2017. These associations were most pronounced among veterans who used VHA health care most frequently. These findings suggest a need for suicide prevention efforts targeted at veterans with these diagnoses.

Dynamic suicide topic modelling: Deriving population-specific, psychosocial and time-sensitive suicide risk variables from Electronic Health Record psychotherapy notes

In the machine learning subfield of natural language processing, a topic model is a type of unsupervised method that is used to uncover abstract topics within a corpus of text. Dynamic topic modelling (DTM) is used for capturing change in these topics over time. The study deploys DTM on corpus of electronic health record psychotherapy notes. This retrospective study examines whether DTM helps distinguish closely matched patients that did and did not die by suicide. Cohort consists of United States Department of Veterans Affairs (VA) patients diagnosed with Posttraumatic Stress Disorder (PTSD) between 2004 and 2013. Each case (those who died by suicide during the year following diagnosis) was matched with five controls (those who remained alive) that shared psychotherapists and had similar suicide risk based on VA's suicide prediction algorithm. Cohort was restricted to patients who received psychotherapy for 9+ months after initial PTSD diagnoses (cases = 77; controls = 362). For cases, psychotherapy notes from diagnosis until death were examined. For controls, psychotherapy notes from diagnosis until matched case's death date were examined. A Python-based DTM algorithm was utilized. Derived topics identified population-specific themes, including PTSD, psychotherapy, medication, communication and relationships. Control topics changed significantly more over time than case topics. Topic differences highlighted engagement, expressivity and therapeutic alliance. This study strengthens groundwork for deriving population-specific, psychosocial and time-sensitive suicide risk variables.

Leveraging Natural Language Processing to Improve Electronic Health Record Suicide Risk Prediction for Veterans Health Administration Users

Background: Suicide risk prediction models frequently rely on structured electronic health record (EHR) data, including patient demographics and health care usage variables. Unstructured EHR data, such as clinical notes, may improve predictive accuracy by allowing access to detailed information that does not exist in structured data fields. To assess comparative benefits of including unstructured data, we developed a large case-control dataset matched on a state-of-the-art structured EHR suicide risk algorithm, utilized natural language processing (NLP) to derive a clinical note predictive model, and evaluated to what extent this model provided predictive accuracy over and above existing predictive thresholds.

Methods: We developed a matched case-control sample of Veterans Health Administration (VHA) patients in 2017 and 2018. Each case (all patients that died by suicide in that interval, n = 4,584) was matched with 5 controls (patients who remained alive during treatment year) who shared the same suicide risk percentile. All sample EHR notes were selected and abstracted using NLP methods. We applied machine-learning classification algorithms to NLP output to develop predictive models. We calculated area under the curve (AUC) and suicide risk concentration to evaluate predictive accuracy overall and for high-risk patients.

Results: The best performing NLP-derived models provided 19% overall additional predictive accuracy (AUC = 0.69; 95% CI, 0.67, 0.72) and 6-fold additional risk concentration for patients at the highest risk tier (top 0.1%), relative to the structured EHR model.

Conclusions: The NLP-supplemented predictive models provided considerable benefit when compared to conventional structured EHR models. Results support future structured and unstructured EHR risk model integrations.

Moral injury and peri- and post-military suicide attempts among post-9/11 veterans

BACKGROUND

Our goal was to examine the association between moral injury, mental health, and suicide attempts during military service and after separation by gender in post-9/11 veterans.

METHODS

A nationally representative sample of 14057 veterans completed a cross-sectional survey. To examine associations of exposure to potentially morally injurious events (PMIEs; witnessing, perpetrating, and betrayal) and suicidal self-directed violence, we estimated two series of multivariable logistic regressions stratified by gender, with peri- and post-military suicide attempt as the dependent variables.

RESULTS

PMIE exposure accounted for additional risk of suicide attempt during and after military service after controlling for demographic and military characteristics, current mental health status, and pre-military history of suicidal ideation and attempt. Men who endorsed PMIE exposure by perpetration were 50% more likely to attempt suicide during service and twice as likely to attempt suicide after separating from service. Men who endorsed betrayal were nearly twice as likely to attempt suicide during service; however, this association attenuated to non-significance after separation in the fully adjusted models. In contrast, women who endorsed betrayal were over 50% more likely to attempt suicide during service and after separation; PMIE exposure by perpetration did not significantly predict suicide attempts before or after service among women in the fully adjusted models.

CONCLUSIONS

Our findings indicate that suicide assessment and prevention programs should consider the impact of moral injury and attend to gender differences in this risk factor in order to provide the most comprehensive care.

Complex modeling with detailed temporal predictors does not improve health records-based suicide risk prediction

Suicide risk prediction models can identify individuals for targeted intervention. Discussions of transparency, explainability, and transportability in machine learning presume complex prediction models with many variables outperform simpler models. We compared random forest, artificial neural network, and ensemble models with 1500 temporally defined predictors to logistic regression models. Data from 25,800,888 mental health visits made by 3,081,420 individuals in 7 health systems were used to train and evaluate suicidal behavior prediction models. Model performance was compared across several measures. All models performed well (area under the receiver operating curve [AUC]: 0.794-0.858). Ensemble models performed best, but improvements over a regression model with 100 predictors were minimal (AUC improvements: 0.006-0.020). Results are consistent across performance metrics and subgroups defined by race, ethnicity, and sex. Our results suggest simpler parametric models, which are easier to implement as part of routine clinical practice, perform comparably to more complex machine learning methods.

The Veterans Health Administration REACH VET Program: Suicide Predictive Modeling in Practice

The U.S. Veterans Health Administration developed a suicide prediction statistical model and implemented a novel clinical program, Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment (REACH VET). This high-value suicide prevention program aims to efficiently identify patients at risk and connect them with care. Starting in April 2017, national REACH VET metric data were collected from electronic health records to evaluate required task completion. By October 2020, 98% of veterans identified (N=6,579) were contacted by providers and had their care evaluated. In the nation's largest health care system, it was feasible to implement a clinical program based on a suicide prediction model.

Comparing risk prediction models aimed at predicting hospitalizations for adverse drug events in community dwelling older adults: a protocol paper

Background

The objective of this paper is to describe the creation, validation, and comparison of two risk prediction modeling approaches for community-dwelling older adults to identify individuals at highest risk for adverse drug event-related hospitalizations. One approach will use traditional statistical methods, the second will use a machine learning approach.

Methods

We will construct medication, clinical, health care utilization, and other variables known to be associated with adverse drug event-related hospitalizations. To create the cohort, we will include older adults (≥ 65 years of age) empaneled to a primary care physician within the Cedars-Sinai Health System primary care clinics with polypharmacy (≥ 5 medications) or at least 1 medication commonly implicated in ADEs (certain oral hypoglycemics, anti-coagulants, anti-platelets, and insulins). We will use a Fine-Gray Cox proportional hazards model for one risk modeling approach and DataRobot, a data science and analytics platform, to run and compare several widely used supervised machine learning algorithms, including Random Forest, Support Vector Machine, Extreme Gradient Boosting (XGBoost), Decision Tree, Naïve Bayes, and K-Nearest Neighbors. We will use a variety of metrics to compare model performance and to assess the risk of algorithmic bias.

Discussion

In conclusion, we hope to develop a pragmatic model that can be implemented in the primary care setting to risk stratify older adults to further optimize medication management.

Clinical implementation of suicide risk prediction models in healthcare: a qualitative study

BACKGROUND

Suicide risk prediction models derived from electronic health records (EHR) are a novel innovation in suicide prevention but there is little evidence to guide their implementation.

METHODS

In this qualitative study, 30 clinicians and 10 health care administrators were interviewed from one health system anticipating implementation of an automated EHR-derived suicide risk prediction model and two health systems piloting different implementation approaches. Site-tailored interview guides focused on respondents' expectations for and experiences with suicide risk prediction models in clinical practice, and suggestions for improving implementation. Interview prompts and content analysis were guided by Consolidated Framework for Implementation Research (CFIR) constructs.

RESULTS

Administrators and clinicians found use of the suicide risk prediction model and the two implementation approaches acceptable. Clinicians desired opportunities for early buy-in, implementation decision-making, and feedback. They wanted to better understand how this manner of risk identification enhanced existing suicide prevention efforts. They also wanted additional training to understand how the model determined risk, particularly after patients they expected to see identified by the model were not flagged at-risk and patients they did not expect to see identified were. Clinicians were concerned about having enough suicide prevention resources for potentially increased demand and about their personal liability; they wanted clear procedures for situations when they could not reach patients or when patients remained at-risk over a sustained period. Suggestions for making risk model workflows more efficient and less burdensome included consolidating suicide risk information in a dedicated module in the EHR and populating risk assessment scores and text in clinical notes.

CONCLUSION

Health systems considering suicide risk model implementation should engage clinicians early in the process to ensure they understand how risk models estimate risk and add value to existing workflows, clarify clinician role expectations, and summarize risk information in a convenient place in the EHR to support high-quality patient care.

Comparing mental and physical health of U.S. veterans by VA healthcare use: implications for generalizability of research in the VA electronic health records

OBJECTIVE

The Department of Veterans Affairs' (VA) electronic health records (EHR) offer a rich source of big data to study medical and health care questions, but patient eligibility and preferences may limit generalizability of findings. We therefore examined the representativeness of VA veterans by comparing veterans using VA healthcare services to those who do not.

METHODS

We analyzed data on 3051 veteran participants age ≥ 18 years in the 2019 National Health Interview Survey. Weighted logistic regression was used to model participant characteristics, health conditions, pain, and self-reported health by past year VA healthcare use and generate predicted marginal prevalences, which were used to calculate Cohen's d of group differences in absolute risk by past-year VA healthcare use.

RESULTS

Among veterans, 30.4% had past-year VA healthcare use. Veterans with lower income and members of racial/ethnic minority groups were more likely to report past-year VA healthcare use. Health conditions overrepresented in past-year VA healthcare users included chronic medical conditions (80.6% vs. 69.4%, d = 0.36), pain (78.9% vs. 65.9%; d = 0.35), mental distress (11.6% vs. 5.9%; d = 0.47), anxiety (10.8% vs. 4.1%; d = 0.67), and fair/poor self-reported health (27.9% vs. 18.0%; d = 0.40).

CONCLUSIONS

Heterogeneity in veteran sociodemographic and health characteristics was observed by past-year VA healthcare use. Researchers working with VA EHR data should consider how the patient selection process may relate to the exposures and outcomes under study. Statistical reweighting may be needed to generalize risk estimates from the VA EHR data to the overall veteran population.

Frequency and correlates of suicidal ideation and behaviors in treatment-seeking Post-9/11 Veterans

OBJECTIVE

Post-9/11 U.S. veterans and servicemembers are at increased risk for suicide, indicating an important need to identify and mitigate suicidal ideation and behaviors in this population.

METHOD

Using data modeling techniques, we examined correlates of suicidal ideation and behavior at intake in 261 Post-9/11 veterans and servicemembers seeking mental health treatment.

RESULTS

Our sample endorsed high rates of suicidal ideation and behavior. Approximately 40% of our sample scored in a range on the Suicide Behaviors Questionnaire-Revised (SBQ-R), indicating high clinical risk for suicide. Results from multivariate analyses indicate that greater state and/or trait depression severity, greater anger and anger expression, less impulse control, and lower rank were consistently associated with suicidal ideation and behavior across our models. Negative posttraumatic thoughts about the self, gender, and military branch of service were also significantly associated with suicidal ideation and behavior.

CONCLUSIONS

Suicidal ideation and behaviors are common in veterans seeking mental health treatment. State and/or trait depression, anger and impulse control were predictors of increased risk for suicidal ideation and behavior across models. Consistencies and differences across models as well as limitations and practical implications for the findings are discussed.

Risk factors for suicide one year after discharge from hospitalization for physical illness in Denmark

While suicide risk following psychiatric hospitalization has been studied extensively, risk following hospitalization for physical illness is less well understood. We used random forests to examine risk factors for suicide in the year following physical illness hospitalization in Denmark. In this case-cohort study, suicide cases were all individuals who died by suicide within one year of a hospitalization for a physical illness (n = 4563) and the comparison subcohort was a 5% random sample of individuals living in Denmark on January 1, 1995 who had a hospitalization for a physical illness between January 1, 1995 and December 31, 2015 (n = 177,664). We used random forests to examine identify the most important predictors of suicide stratified by sex. For women, the top 10 most important variables for random forest prediction were all related to psychiatric diagnoses. For men, many physical health conditions also appeared important to suicide prediction. Among the top 10 variables in the variable importance plot for men were influenza, injuries to the head, nervous system surgeries, and cerebrovascular diseases. Suicide prediction after a physical illness hospitalization requires comprehensive consideration of different and multiple factors for each sex.

The performance of machine learning models in predicting suicidal ideation, attempts, and deaths: A meta-analysis and systematic review

Research has posited that machine learning could improve suicide risk prediction models, which have traditionally performed poorly. This systematic review and meta-analysis evaluated the performance of machine learning models in predicting longitudinal outcomes of suicide-related outcomes of ideation, attempt, and death and examines outcome, data, and model types as potential covariates of model performance. Studies were extracted from PubMed, Web of Science, Embase, and PsycINFO. A bivariate mixed effects meta-analysis and meta-regression analyses were performed for studies using machine learning to predict future events of suicidal ideation, attempts, and/or deaths. Risk of bias was assessed for each study using an adaptation of the Prediction model Risk Of Bias Assessment Tool. Narrative review included 56 studies, and analyses examined 54 models from 35 studies. The models achieved a very good pooled AUC of 0.86, sensitivity of 0.66 (95% CI [0.60, 0.72)], and specificity of 0.87 (95% CI [0.84, 0.90]). Pooled AUCs for ideation, attempt, and death were similar at 0.88, 0.87, and 0.84 respectively. Model performance was highly varied; however, meta-regressions did not provide evidence that performance varied by outcome, data, or model types. Findings suggest that machine learning has the potential to improve suicide risk detection, with pooled estimates of machine learning performance comparing favourably to performance of traditional suicide prediction models. However, more studies with lower risk of bias are necessary to improve the application of machine learning in suicidology.

Mapping of machine learning approaches for description, prediction, and causal inference in the social and health sciences

Machine learning (ML) methodology used in the social and health sciences needs to fit the intended research purposes of description, prediction, or causal inference. This paper provides a comprehensive, systematic meta-mapping of research questions in the social and health sciences to appropriate ML approaches by incorporating the necessary requirements to statistical analysis in these disciplines. We map the established classification into description, prediction, counterfactual prediction, and causal structural learning to common research goals, such as estimating prevalence of adverse social or health outcomes, predicting the risk of an event, and identifying risk factors or causes of adverse outcomes, and explain common ML performance metrics. Such mapping may help to fully exploit the benefits of ML while considering domain-specific aspects relevant to the social and health sciences and hopefully contribute to the acceleration of the uptake of ML applications to advance both basic and applied social and health sciences research.

Utilizing digital predictive biomarkers to identify Veteran suicide risk

Veteran suicide is one of the most complex and pressing health issues in the United States. According to the 2020 National Veteran Suicide Prevention Annual Report, since 2018 an average of 17.2 Veterans died by suicide each day. Veteran suicide risk screening is currently limited to suicide hotlines, patient reporting, patient visits, and family or friend reporting. As a result of these limitations, innovative approaches in suicide screening are increasingly garnering attention. An essential feature of these innovative methods includes better incorporation of risk factors that might indicate higher risk for tracking suicidal ideation based on personal behavior. Digital technologies create a means through which measuring these risk factors more reliably, with higher fidelity, and more frequently throughout daily life is possible, with the capacity to identify potentially telling behavior patterns. In this review, digital predictive biomarkers are discussed as they pertain to suicide risk, such as sleep vital signs, sleep disturbance, sleep quality, and speech pattern recognition. Various digital predictive biomarkers are reviewed and evaluated as well as their potential utility in predicting and diagnosing Veteran suicidal ideation in real time. In the future, these digital biomarkers could be combined to generate further suicide screening for diagnosis and severity assessments, allowing healthcare providers and healthcare teams to intervene more optimally.

Leveraging unstructured electronic medical record notes to derive population-specific suicide risk models

Electronic medical record (EMR)-based suicide risk prediction methods typically rely on analysis of structured variables such as demographics, visit history, and prescription data. Leveraging unstructured EMR notes may improve predictive accuracy by allowing access to nuanced clinical information. We utilized natural language processing (NLP) to analyze a large EMR note corpus to develop a data-driven suicide risk prediction model. We developed a matched case-control sample of U.S. Department of Veterans Affairs (VA) patients in 2015 and 2016. We randomly matched each case (all patients that died by suicide in that interval, n = 5029) with five controls (patients that remained alive). We processed note corpus using NLP methods and applied machine-learning classification algorithms to output. We calculated area under the curve (AUC) and risk tiers to determine predictive accuracy. NLP-derived models demonstrated strong predictive accuracy. Patients that scored within top 10% of risk model accounted for up to 29% of suicide decedents. NLP-derived model compares positively to other leading prediction methods. Our approach is highly implementable, only requiring access to text data and open-source software. Additional studies should evaluate ensemble models incorporating NLP-derived information alongside more typical structured variables.

Suicide theory-guided natural language processing of clinical progress notes to improve prediction of veteran suicide risk: protocol for a mixed-method study

INTRODUCTION

The state-of-the-art 3-step Theory of Suicide (3ST) describes why people consider suicide and who will act on their suicidal thoughts and attempt suicide. The central concepts of 3ST-psychological pain, hopelessness, connectedness, and capacity for suicide-are among the most important drivers of suicidal behaviour but they are missing from clinical suicide risk prediction models in use at the US Veterans Health Administration (VHA). These four concepts are not systematically recorded in structured fields of VHA's electronic healthcare records. Therefore, this study will develop a domain-specific ontology that will enable automated extraction of these concepts from clinical progress notes using natural language processing (NLP), and test whether NLP-based predictors for these concepts improve accuracy of existing VHA suicide risk prediction models.

METHODS AND ANALYSIS

Our mixed-method study has an exploratory sequential design where a qualitative component (aim 1) will inform quantitative analyses (aims 2 and 3). For aim 1, subject matter experts will manually annotate progress notes of clinical encounters with veterans who attempted or died by suicide to develop a domain-specific ontology for the 3ST concepts. During aim 2, we will use NLP to machine-annotate clinical progress notes and derive longitudinal representations for each patient with respect to the presence and intensity of hopelessness, psychological pain, connectedness and capacity for suicide in temporal proximity of suicide attempts and deaths by suicide. These longitudinal representations will be evaluated during aim 3 for their ability to improve existing VHA prediction models of suicide and suicide attempts, STORM (Stratification Tool for Opioid Risk Mitigation) and REACHVET (Recovery Engagement and Coordination for Health - Veterans Enhanced Treatment).

ETHICS AND DISSEMINATION

Ethics approval for this study was granted by the Stanford University Institutional Review Board and the Research and Development Committee of the VA Palo Alto Health Care System. Results of the study will be disseminated through several outlets, including peer-reviewed publications and presentations at national conferences.

Patient expectations of and experiences with a suicide risk identification algorithm in clinical practice

BACKGROUND

Suicide risk prediction models derived from electronic health records (EHR) and insurance claims are a novel innovation in suicide prevention but patient perspectives on their use have been understudied.

METHODS

In this qualitative study, between March and November 2020, 62 patients were interviewed from three health systems: one anticipating implementation of an EHR-derived suicide risk prediction model and two others piloting different implementation approaches. Site-tailored interview guides focused on patients' perceptions of this technology, concerns, and preferences for and experiences with suicide risk prediction model implementation in clinical practice. A constant comparative analytic approach was used to derive themes.

RESULTS

Interview participants were generally supportive of suicide risk prediction models derived from EHR data. Concerns included apprehension about inducing anxiety and suicidal thoughts, or triggering coercive treatment, particularly among those who reported prior negative experiences seeking mental health care. Participants who were engaged in mental health care or case management expected to be asked about their suicide risk and largely appreciated suicide risk conversations, particularly by clinicians comfortable discussing suicidality.

CONCLUSION

Most patients approved of suicide risk models that use EHR data to identify patients at-risk for suicide. As health systems proceed to implement such models, patient-centered care would involve dialogue initiated by clinicians experienced with assessing suicide risk during virtual or in person care encounters. Health systems should proactively monitor for negative consequences that result from risk model implementation to protect patient trust.

Preventing Suicide Among Women Veterans: Gender-Sensitive, Trauma-Informed Conceptualization

Purpose of Review

There is growing concern regarding suicide among women veterans, who have experienced an increase in suicide rates that has exceeded that reported for other US adult populations. Recent research has bolstered understanding of correlates of suicide risk specific to women veterans. Yet most existing suicide prevention initiatives take a gender-neutral, rather than gender-sensitive, approach. We offer clinical considerations and suggestions for suicide prevention tailored to the needs, preferences, and experiences of women veterans. Discussion is framed around the White House strategy for preventing suicide among military service members and veterans.

Recent Findings

Considering high rates of trauma exposure among women veterans, we propose that a trauma-informed lens is essential for taking a gender-sensitive approach to suicide prevention with this population. Nonetheless, research to inform evidence-based assessment and intervention remains largely focused on veteran men or gender-neutral. Integral next steps for research are posited.

Summary

Extant research provides an initial foundation for beginning to understand and address suicide among women veterans in a gender-sensitive, trauma-informed manner. Additional research that is specific to women veterans or that examines gender differences is critical to ensure women veterans receive optimal, evidence-based care to prevent suicide.

Predictors of nonfatal suicide attempts within 30 days of discharge from psychiatric hospitalization: Sex-specific models developed using population-based registries

BACKGROUND

Risk for nonfatal suicide attempts is heightened in the month after psychiatric hospitalization discharge. Investigations of factors associated with such attempts are limited.

METHODS

We conducted a case-subcohort study using data from Danish medical, administrative, and social registries to develop sex-specific risk models using two machine learning methods: classification trees and random forests. Cases included individuals who received a diagnostic code for a nonfatal suicide attempt within 30 days of discharge following a psychiatric hospitalization between January 1, 1995 and December 31, 2015 (n = 3166, 56.5% female). The comparison subcohort consisted of a 5% random sample of individuals living in Denmark (n = 24,559, 51.3% female) on January 1, 1995 who had a psychiatric hospitalization during the study period.

RESULTS

Histories of self-poisoning, substance-related disorders, and eating disorders were important predictors of nonfatal suicide attempt among women, with notable interactions observed between age, self-poisoning history, and other characteristics (e.g., medication use). Self-poisoning, substance-related disorders, and severe stress reactions were among the most important variables for men, with key interactions noted between self-poisoning history, age, major depressive disorder diagnosis, and prescription classes.

LIMITATIONS

Findings are based on Danish administrative data, which may be subject to inaccuracies, missingness, etc. It is unclear whether results would generalize to other populations.

CONCLUSIONS

Markers of behavioral dysregulation were important predictors of nonfatal suicide attempts in the 30 days after psychiatric hospitalization discharge for both sexes. Examining risk markers for nonfatal suicide attempt following discharge is important to enhance support for this vulnerable population.

Developing and Implementing Predictive Models in a Learning Healthcare System: Traditional and Artificial Intelligence Approaches in the Veterans Health Administration

Predicting clinical risk is an important part of healthcare and can inform decisions about treatments, preventive interventions, and provision of extra services. The field of predictive models has been revolutionized over the past two decades by electronic health record data; the ability to link such data with other demographic, socioeconomic, and geographic information; the availability of high-capacity computing; and new machine learning and artificial intelligence methods for extracting insights from complex datasets. These advances have produced a new generation of computerized predictive models, but debate continues about their development, reporting, validation, evaluation, and implementation. In this review we reflect on more than 10 years of experience at the Veterans Health Administration, the largest integrated healthcare system in the United States, in developing, testing, and implementing such models at scale. We report lessons from the implementation of national risk prediction models and suggest an agenda for research. Expected final online publication date for the Annual Review of Biomedical Data Science, Volume 5 is August 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Characteristics of suicide attempts associated with lethality and method: A latent class analysis of the Military Suicide Research Consortium

While suicide prevention is a national priority, particularly among service members and veterans (SMVs), understanding of suicide-related outcomes remains poor. Person-centered approaches (e.g., latent class analysis) have promise to identify unique risk profiles and subgroups in the larger population. The current study identified latent subgroups characterized by prior self-directed violence history and proximal risk factors for suicide among suicide attempt survivors, and compared subgroups on demographics and most-lethal attempt characteristics. Participants included civilians and SMVs reporting lifetime suicide attempt(s) (n = 2643) from the Military Suicide Research Consortium. Two classes emerged from Common Data Elements: suicide attempt and non-suicidal self-injury frequency, suicide attempt method, perceived likelihood of future suicide, suicide disclosure, suicide intent, and perceived and actual lethality of attempt. A Higher-Risk History class was characterized by greater intent to die, certainty about attempt fatality and method lethality, belief injury would be medically unfixable, and likelihood of prior non-suicidal self-injury. A Lower-Risk History class was characterized by greater ambivalence toward death and methods. Higher-Risk class members were more likely to be male, older, SMVs, have less formal education, use firearms as most-lethal attempt method, and require a higher degree of medical attention. Lower-Risk class members were more likely to be female, civilian, use cutting as most-lethal attempt method, and require less medical attention for attempts. Findings have implications for risk assessments and highlight the importance of subjective perceptions about suicidal behavior. Further investigation of real-time individual-level is necessary, especially for SMVs who may be at greatest risk for potentially lethal suicidal behavior.

Local implementation evaluation of a suicide prevention predictive model at a large VA health care system

BACKGROUND

The Veterans Health Administration (VHA) implemented REACH VET, which analyzes health records to identify veterans at statistically elevated risk for suicide and other adverse outcomes compared to other veterans in VHA. This project evaluated REACH VET program implementation at a large VA health care system by examining program fidelity and treatment engagement, receipt of suicide prevention interventions, and suicide-related behaviors in the 6 months following identification.

METHODS

Over a 12-month period, 218 unique cases were identified by REACH VET within a local VA system. Data were extracted from the VA's electronic medical records.

RESULTS

Protocol adherence for required clinical and administrative steps was 94% and above. After identification, 88% received outpatient mental health treatment, 21% had a psychiatric hospitalization, and 83% engaged in Safety Planning around the time of identification or in the following six months. Twenty-six percent of cases were identified by another existing method for identifying high-risk veterans. Five percent had a medically documented suicide attempt, and none were known to die by suicide in the following 6 months.

CONCLUSIONS

Local evaluation suggested high protocol fidelity and high engagement in mental health and suicide prevention services following identification among veterans who remained at elevated risk in the 6 months that followed.

Implementation strategy to increase clinicians' use of the caring letters suicide prevention intervention

Caring Letters is recommended in multiple best practice guidelines; however, the Caring Letters intervention has not been widely implemented. The process of tracking, scheduling, and mailing letters for multiple patients over many months may represent a significant barrier for busy clinicians. This evaluation examined whether the use of centralized administrative support (Centralized Caring Letters; CCL) was associated with increased utilization of the intervention. These procedures were tested in the Department of Veterans Affairs (VA) Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment (REACH VET) program. In REACH VET, VA clinicians are routinely asked to consider Caring Letters as one option for veterans identified as at-risk. In this evaluation, clinicians at two VA facilities were offered assistance in the tracking, preparation, mailing, and documentation of Caring Letters for veterans they chose to enroll in CCL. The utilization of Caring Letters increased more than 14-fold after CCL was implemented. In the year that preceded CCL, 3% of REACH VET veterans were sent Caring Letters at the two sites; this increased to 43% of cases after the implementation of CCL (45% at Site 1 and 41% at Site 2). In qualitative interviews with providers, clinicians described Caring Letters as beneficial and stated that the centralized features of the program were helpful. Caring Letters were discontinued for 30% of enrolled veterans, often because of a bad address (9% of enrolled) or relocation (8% of enrolled). Although there are barriers for the use of Caring Letters, CCL was associated with a very large increase in the use of Caring Letters. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Implementing Machine Learning Models for Suicide Risk Prediction in Clinical Practice: Focus Group Study With Hospital Providers

Background

Interest in developing machine learning models that use electronic health record data to predict patients’ risk of suicidal behavior has recently proliferated. However, whether and how such models might be implemented and useful in clinical practice remain unknown. To ultimately make automated suicide risk–prediction models useful in practice, and thus better prevent patient suicides, it is critical to partner with key stakeholders, including the frontline providers who will be using such tools, at each stage of the implementation process.

Objective

The aim of this focus group study is to inform ongoing and future efforts to deploy suicide risk–prediction models in clinical practice. The specific goals are to better understand hospital providers’ current practices for assessing and managing suicide risk; determine providers’ perspectives on using automated suicide risk–prediction models in practice; and identify barriers, facilitators, recommendations, and factors to consider.

Methods

We conducted 10 two-hour focus groups with a total of 40 providers from psychiatry, internal medicine and primary care, emergency medicine, and obstetrics and gynecology departments within an urban academic medical center. Audio recordings of open-ended group discussions were transcribed and coded for relevant and recurrent themes by 2 independent study staff members. All coded text was reviewed and discrepancies were resolved in consensus meetings with doctoral-level staff.

Results

Although most providers reported using standardized suicide risk assessment tools in their clinical practices, existing tools were commonly described as unhelpful and providers indicated dissatisfaction with current suicide risk assessment methods. Overall, providers’ general attitudes toward the practical use of automated suicide risk–prediction models and corresponding clinical decision support tools were positive. Providers were especially interested in the potential to identify high-risk patients who might be missed by traditional screening methods. Some expressed skepticism about the potential usefulness of these models in routine care; specific barriers included concerns about liability, alert fatigue, and increased demand on the health care system. Key facilitators included presenting specific patient-level features contributing to risk scores, emphasizing changes in risk over time, and developing systematic clinical workflows and provider training. Participants also recommended considering risk-prediction windows, timing of alerts, who will have access to model predictions, and variability across treatment settings.

Conclusions

Providers were dissatisfied with current suicide risk assessment methods and were open to the use of a machine learning–based risk-prediction system to inform clinical decision-making. They also raised multiple concerns about potential barriers to the usefulness of this approach and suggested several possible facilitators. Future efforts in this area will benefit from incorporating systematic qualitative feedback from providers, patients, administrators, and payers on the use of these new approaches in routine care, especially given the complex, sensitive, and unfortunately still stigmatized nature of suicide risk.

Predicting suicide attempts among U.S. Army soldiers after leaving active duty using information available before leaving active duty: results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS)

Suicide risk is elevated among military service members who recently transitioned to civilian life. Identifying high-risk service members before this transition could facilitate provision of targeted preventive interventions. We investigated the feasibility of doing this by attempting to develop a prediction model for self-reported suicide attempts (SAs) after leaving or being released from active duty in the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS). This study included two self-report panel surveys (LS1: 2016-2018, LS2: 2018-2019) administered to respondents who previously participated while on active duty in one of three Army STARRS 2011-2014 baseline self-report surveys. We focus on respondents who left active duty >12 months before their LS survey (n = 8899). An ensemble machine learning model using predictors available prior to leaving active duty was developed in a 70% training sample and validated in a 30% test sample. The 12-month self-reported SA prevalence (SE) was 1.0% (0.1). Test sample AUC (SE) was 0.74 (0.06). The 15% of respondents with highest predicted risk included nearly two-thirds of 12-month SAs and over 80% of medically serious 12-month SAs. These results show that it is possible to identify soldiers at high post-transition self-report SA risk before the transition. Future model development is needed to examine prediction of SAs assessed by administrative data and using surveys administered closer to the time of leaving active duty.

Suicide Mortality Among Veterans Health Administration Care Recipients With Suicide Risk Record Flags

OBJECTIVE

In 2008, the Veterans Health Administration (VHA) established a suicide high-risk flag (HRF) for patient records. To inform ongoing suicide prevention activities as part of operations and quality improvement work in the U.S. Department of Veterans Affairs, the authors evaluated suicide risk following HRF activations and inactivations.

METHODS

For annual cohorts of VHA users, HRF receipt and demographic and clinical care contexts in the 30 days before HRF activations were examined for 2014-2016 (N=7,450,831). Veterans were included if they had VHA inpatient or outpatient encounters during the index or previous year. Suicide rates in the 12 months after HRF activations and inactivations were assessed. Using multivariable Cox proportional hazards regression, the authors compared suicide risk following HRF activation and inactivation with veterans without HRFs, adjusted for age, gender, and race-ethnicity.

RESULTS

HRF activation (N=47,015) was commonly preceded within 30 days by a documented suicide attempt (39.5%) or inpatient mental health admission (40.1%). Suicide risk was elevated in the 12 months after flag activation (crude suicide rate=682 per 100,000 person-years, adjusted hazard ratio [HR]=21.00, 95% confidence interval [CI]=18.55-23.72) compared with risk among VHA users without HRF activity. Risk after HRF inactivation (N=41,251) was also elevated (crude suicide rate=408 per 100,000 person-years, adjusted HR=12.43, 95% CI=10.57-14.63) compared with risk among VHA users without HRF activity.

CONCLUSIONS

Suicide risk after HRF activation was substantially elevated and also high after HRF inactivation. Findings suggest the importance of comprehensive suicide risk mitigation and support recent VHA process enhancements to formalize inactivation criteria and support veterans after HRF inactivation.

Translating promise into practice: a review of machine learning in suicide research and prevention

In ever more pressured health-care systems, technological solutions offering scalability of care and better resource targeting are appealing. Research on machine learning as a technique for identifying individuals at risk of suicidal ideation, suicide attempts, and death has grown rapidly. This research often places great emphasis on the promise of machine learning for preventing suicide, but overlooks the practical, clinical implementation issues that might preclude delivering on such a promise. In this Review, we synthesise the broad empirical and review literature on electronic health record-based machine learning in suicide research, and focus on matters of crucial importance for implementation of machine learning in clinical practice. The challenge of preventing statistically rare outcomes is well known; progress requires tackling data quality, transparency, and ethical issues. In the future, machine learning models might be explored as methods to enable targeting of interventions to specific individuals depending upon their level of need-ie, for precision medicine. Primarily, however, the promise of machine learning for suicide prevention is limited by the scarcity of high-quality scalable interventions available to individuals identified by machine learning as being at risk of suicide.

Predictors of Intentional Self -Harm Among Medicaid Mental Health Clinic Clients In New York

BACKGROUND

Behavioral health outpatients are at risk for self-harm. Identifying individuals or combination of risk factors could discriminate those at elevated risk for self-harm.

METHODS

The study population (N = 248,491) included New York State Medicaid-enrolled individuals aged 10 to 64 with mental health clinic services between November 1, 2015 to November 1, 2016. Self-harm episodes were defined using ICD-10 codes from emergency department and inpatient visits. Multi-predictor logistic regression models were fit on a subsample of the data and compared to a testing sample based on discrimination performance (Area Under the Curve or AUC).

RESULTS

Of N = 248,491 patients, 4,224 (1.70%) had an episode of intentional self-harm. Factors associated with increased self-harm risk were age 17-25, being female and having recent diagnoses of depression (AOR=4.3, 95%CI: 3.6-5.0), personality disorder (AOR=4.2, 95%CI: 2.9-6.1), or substance use disorder (AOR=3.4, 95%CI: 2.7-4.3) within the last month. A multi-predictor logistic regression model including demographics and new psychiatric diagnoses within 90 days prior to index date had good discrimination and outperformed competitor models on a testing sample (AUC=0.86, 95%CI:0.85-0.87).

LIMITATIONS

New York State Medicaid data may not be generalizable to the entire U.S population. ICD-10 codes do not allow distinction between self-harm with and without intent to die.

CONCLUSIONS

Our results highlight the usefulness of recency of new psychiatric diagnoses, in predicting the magnitude and timing of intentional self-harm risk. An algorithm based on this finding could enhance clinical assessments support screening, intervention and outreach programs that are at the heart of a Zero Suicide prevention model.

Artificial intelligence and suicide prevention: a systematic review

Background

Suicide is one of the main preventable causes of death. Artificial intelligence (AI) could improve methods for assessing suicide risk. The objective of this review is to assess the potential of AI in identifying patients who are at risk of attempting suicide.

Methods

A systematic review of the literature was conducted on PubMed, EMBASE, and SCOPUS databases, using relevant keywords.

Results

Thanks to this research, 296 studies were identified. Seventeen studies, published between 2014 and 2020 and matching inclusion criteria, were selected as relevant. Included studies aimed at predicting individual suicide risk or identifying at-risk individuals in a specific population. The AI performance was overall good, although variable across different algorithms and application settings.

Conclusions

AI appears to have a high potential for identifying patients at risk of suicide. The precise use of these algorithms in clinical situations, as well as the ethical issues it raises, remain to be clarified.

Suicide typologies among Medicaid beneficiaries, North Carolina 2014-2017

BACKGROUND

There is a well-established need for population-based screening strategies to identify people at risk of suicide. Because only about half of suicide decedents are ever diagnosed with a behavioral health condition, it may be necessary for providers to consider life circumstances that may also put individuals at risk. This study described the alignment of medical diagnoses with life circumstances by identifying suicide typologies among decedents. Demographics, stressful life events, suicidal behavior, perceived and diagnosed health problems, and suicide method contributed to the typologies.

METHODS

This study linked North Carolina Medicaid and North Carolina Violent Death Reporting System (NC-VDRS) data for analysis in 2020. For suicide decedents from 2014 to 2017 aged 25-54 years, we analyzed 12 indicators of life circumstances from NC-VDRS and 6 indicators from Medicaid claims, using a latent class model. Separate models were developed for men and women.

RESULTS

Most decedents were White (88.3%), with a median age of 41, and over 70% had a health care visit in the 90 days prior to suicide. Two typologies were identified in both males (n = 175) and females (n = 153). Both typologies had similar profiles of life circumstances, but one had high probabilities of diagnosed behavioral health conditions (45% of men, 71% of women), compared to low probabilities in the other (55% of men, 29% of women). Black beneficiaries and men who died by firearm were over-represented in the less-diagnosed class, though estimates were imprecise (odds ratio for Black men: 3.1, 95% confidence interval: 0.8, 12.4; odds ratio for Black women: 5.0, 95% confidence interval: 0.9, 31.2; odds ratio for male firearm decedents: 1.6, 95% confidence interval: 0.7, 3.4).

CONCLUSIONS

Nearly half of suicide decedents have a typology characterized by low probability of diagnosis of behavioral health conditions. Suicide screening could likely be enhanced using improved indicators of lived experience and behavioral health.

Artificial Intelligence: Review of Current and Future Applications in Medicine

Background

The role of artificial intelligence (AI) in health care is expanding rapidly. Currently, there are at least 29 US Food and Drug Administration-approved AI health care devices that apply to numerous medical specialties and many more are in development.

Observations

With increasing expectations for all health care sectors to deliver timely, fiscally-responsible, high-quality health care, AI has potential utility in numerous areas, such as image analysis, improved workflow and efficiency, public health, and epidemiology, to aid in processing large volumes of patient and medical data. In this review, we describe basic terminology, principles, and general AI applications relating to health care. We then discuss current and future applications for a variety of medical specialties. Finally, we discuss the future potential of AI along with the potential risks and limitations of current AI technology.

Conclusions

AI can improve diagnostic accuracy, increase patient safety, assist with patient triage, monitor disease progression, and assist with treatment decisions.

Rates and Correlates of Suicidality in VA Intensive Case Management Programs

There has been extensive concern about suicide among veterans, but no study has examined rates and correlates of suicidality in the highly vulnerable group of veterans receiving Veterans Health Administration (VHA) intensive case management services. Veterans participating in a national program evaluation were surveyed at the time of program entry and 6 months later. Sociodemographic and clinical characteristics were documented along with elements of program service delivery. Chi square tests were used to compare rates of suicidality (defined as either having made or threatened an attempt) at baseline and at the 6-month follow-up. Analysis of variance was also used to compare suicidal and non-suicidal veterans at follow-up. Logistic regression analysis was then used to identify independent correlates of suicidality 6 months after program entry. Among the 9921 veterans who later completed follow-up assessments 989 (10.0%) had reported suicidal behavior at program entry as compared to only 250 (2.51%) at 6 months (p < 0.0001). Multivariable logistic regression analysis showed suicidality at 6 months to be associated with suicidality at admission, increased subjective distress on the Brief Symptom Inventory (especially on depression items), violent behavior and decreased quality of life since admission, along with a greater likelihood of receiving crisis intervention, but not other services. Among veterans receiving intensive case management services from VHA, suicidal behavior declined by 75% from admission to 6 months (10-2.5%) and was associated with suicidality prior to program entry, worsening subjective symptoms and greater receipt of crisis intervention services.

AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients

The COVID-19 pandemic has adverse consequences on human psychology and behavior long after initial recovery from the virus. These COVID-19 health sequelae, if undetected and left untreated, may lead to more enduring mental health problems, and put vulnerable individuals at risk of developing more serious psychopathologies. Therefore, an early distinction of such vulnerable individuals from those who are more resilient is important to undertake timely preventive interventions. The main aim of this article is to present a comprehensive multimodal conceptual approach for addressing these potential psychological and behavioral mental health changes using state-of-the-art tools and means of artificial intelligence (AI). Mental health COVID-19 recovery programs at post-COVID clinics based on AI prediction and prevention strategies may significantly improve the global mental health of ex-COVID-19 patients. Most COVID-19 recovery programs currently involve specialists such as pulmonologists, cardiologists, and neurologists, but there is a lack of psychiatrist care. The focus of this article is on new tools which can enhance the current limited psychiatrist resources and capabilities in coping with the upcoming challenges related to widespread mental health disorders. Patients affected by COVID-19 are more vulnerable to psychological and behavioral changes than non-COVID populations and therefore they deserve careful clinical psychological screening in post-COVID clinics. However, despite significant advances in research, the pace of progress in prevention of psychiatric disorders in these patients is still insufficient. Current approaches for the diagnosis of psychiatric disorders largely rely on clinical rating scales, as well as self-rating questionnaires that are inadequate for comprehensive assessment of ex-COVID-19 patients' susceptibility to mental health deterioration. These limitations can presumably be overcome by applying state-of-the-art AI-based tools in diagnosis, prevention, and treatment of psychiatric disorders in acute phase of disease to prevent more chronic psychiatric consequences.

Comparing the predictive value of screening to the use of electronic health record data for detecting future suicidal thoughts and behavior in an urban pediatric emergency department: A preliminary analysis

AIM

Brief screening and predictive modeling have garnered attention for utility at identifying individuals at risk of suicide. Although previous research has investigated these methods, little is known about how these methods compare against each other or work in combination in the pediatric population.

METHODS

Patients were aged 8-18 years old who presented from January 1, 2017, to June 30, 2019, to a Pediatric Emergency Department (PED). All patients were screened with the Ask Suicide Questionnaire (ASQ) as part of a universal screening approach. For all models, we used 5-fold cross-validation. We compared four models: Model 1 only included the ASQ; Model 2 included the ASQ and EHR data gathered at the time of ED visit (EHR data); Model 3 only included EHR data; and Model 4 included EHR data and a single item from the ASQ that asked about a lifetime history of suicide attempt. The main outcome was subsequent PED visit with suicide-related presenting problem within a 3-month follow-up period.

RESULTS

Of the N = 13,420 individuals, n = 141 had a subsequent suicide-related PED visit. Approximately 63% identified as Black. Results showed that a model based only on EHR data (Model 3) had an area under the curve (AUC) of 0.775 compared to the ASQ alone (Model 1), which had an AUC of 0.754. Combining screening and EHR data (Model 4) resulted in a 17.4% (absolute difference = 3.6%) improvement in sensitivity and 13.4% increase in AUC (absolute difference = 6.6%) compared to screening alone (Model 1).

CONCLUSION

Our findings show that predictive modeling based on EHR data is helpful either in the absence or as an addition to brief suicide screening. This is the first study to compare brief suicide screening to EHR-based predictive modeling and adds to our understanding of how best to identify youth at risk of suicidal thoughts and behaviors in clinical care settings.