Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach

Machine learning and the prediction of suicide in psychiatric populations: a systematic review

Machine learning (ML) has emerged as a promising tool to enhance suicidal prediction. However, as many large-sample studies mixed psychiatric and non-psychiatric populations, a formal psychiatric diagnosis emerged as a strong predictor of suicidal risk, overshadowing more subtle risk factors specific to distinct populations. To overcome this limitation, we conducted a systematic review of ML studies evaluating suicidal behaviors exclusively in psychiatric clinical populations. A systematic literature search was performed from inception through November 17, 2022 on PubMed, EMBASE, and Scopus following the PRISMA guidelines. Original research using ML techniques to assess the risk of suicide or predict suicide attempts in the psychiatric population were included. An assessment for bias risk was performed using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines. About 1032 studies were retrieved, and 81 satisfied the inclusion criteria and were included for qualitative synthesis. Clinical and demographic features were the most frequently employed and random forest, support vector machine, and convolutional neural network performed better in terms of accuracy than other algorithms when directly compared. Despite heterogeneity in procedures, most studies reported an accuracy of 70% or greater based on features such as previous attempts, severity of the disorder, and pharmacological treatments. Although the evidence reported is promising, ML algorithms for suicidal prediction still present limitations, including the lack of neurobiological and imaging data and the lack of external validation samples. Overcoming these issues may lead to the development of models to adopt in clinical practice. Further research is warranted to boost a field that holds the potential to critically impact suicide mortality.

Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: A systematic review

OBJECTIVE

Natural language processing (NLP) combined with machine learning (ML) techniques are increasingly used to process unstructured/free-text patient-reported outcome (PRO) data available in electronic health records (EHRs). This systematic review summarizes the literature reporting NLP/ML systems/toolkits for analyzing PROs in clinical narratives of EHRs and discusses the future directions for the application of this modality in clinical care.

METHODS

We searched PubMed, Scopus, and Web of Science for studies written in English between 1/1/2000 and 12/31/2020. Seventy-nine studies meeting the eligibility criteria were included. We abstracted and summarized information related to the study purpose, patient population, type/source/amount of unstructured PRO data, linguistic features, and NLP systems/toolkits for processing unstructured PROs in EHRs.

RESULTS

Most of the studies used NLP/ML techniques to extract PROs from clinical narratives (n = 74) and mapped the extracted PROs into specific PRO domains for phenotyping or clustering purposes (n = 26). Some studies used NLP/ML to process PROs for predicting disease progression or onset of adverse events (n = 22) or developing/validating NLP/ML pipelines for analyzing unstructured PROs (n = 19). Studies used different linguistic features, including lexical, syntactic, semantic, and contextual features, to process unstructured PROs. Among the 25 NLP systems/toolkits we identified, 15 used rule-based NLP, 6 used hybrid NLP, and 4 used non-neural ML algorithms embedded in NLP.

CONCLUSIONS

This study supports the potential utility of different NLP/ML techniques in processing unstructured PROs available in EHRs for clinical care. Though using annotation rules for NLP/ML to analyze unstructured PROs is dominant, deploying novel neural ML-based methods is warranted.

Identifying suicide documentation in clinical notes through zero-shot learning

Background and Aims

In deep learning, a major difficulty in identifying suicidality and its risk factors in clinical notes is the lack of training samples given the small number of true positive instances among the number of patients screened. This paper describes a novel methodology that identifies suicidality in clinical notes by addressing this data sparsity issue through zero-shot learning. Our general aim was to develop a tool that leveraged zero-shot learning to effectively identify suicidality documentation in all types of clinical notes.

Methods

US Veterans Affairs clinical notes served as data. The training data set label was determined using diagnostic codes of suicide attempt and self-harm. We used a base string associated with the target label of suicidality to provide auxiliary information by narrowing the positive training cases to those containing the base string. We trained a deep neural network by mapping the training documents' contents to a semantic space. For comparison, we trained another deep neural network using the identical training data set labels, and bag-of-words features.

Results

The zero-shot learning model outperformed the baseline model in terms of area under the curve, sensitivity, specificity, and positive predictive value at multiple probability thresholds. In applying a 0.90 probability threshold, the methodology identified notes documenting suicidality but not associated with a relevant ICD-10-CM code, with 94% accuracy.

Conclusion

This method can effectively identify suicidality without manual annotation.

Assessment of machine learning algorithms in national data to classify the risk of self-harm among young adults in hospital: A retrospective study

BACKGROUND

Self-harm is one of the most common presentations at accident and emergency departments in the UK and is a strong predictor of suicide risk. The UK Government has prioritised identifying risk factors and developing preventative strategies for self-harm. Machine learning offers a potential method to identify complex patterns with predictive value for the risk of self-harm.

METHODS

National data in the UK Mental Health Services Data Set were isolated for patients aged 18-30 years who started a mental health hospital admission between Aug 1, 2020 and Aug 1, 2021, and had been discharged by Jan 1, 2022. Data were obtained on age group, gender, ethnicity, employment status, marital status, accommodation status and source of admission to hospital and used to construct seven machine learning models that were used individually and as an ensemble to predict hospital stays that would be associated with a risk of self-harm.

OUTCOMES

The training dataset included 23 808 items (including 1081 episodes of self-harm) and the testing dataset 5951 items (including 270 episodes of self-harm). The best performing algorithms were the random forest model (AUC-ROC 0.70, 95%CI:0.66-0.74) and the ensemble model (AUC-ROC 0.77 95%CI:0.75-0.79).

INTERPRETATION

Machine learning algorithms could predict hospital stays with a high risk of self-harm based on readily available data that are routinely collected by health providers and recorded in the Mental Health Services Data Set. The findings should be validated externally with other real-world, prospective data.

FUNDING

This study was supported by the Midlands and Lancashire Commissioning Support Unit.

Applying Deep Learning Model to Predict Diagnosis Code of Medical Records

The International Classification of Diseases (ICD) code is a diagnostic classification standard that is frequently used as a referencing system in healthcare and insurance. However, it takes time and effort to find and use the right diagnosis code based on a patient's medical records. In response, deep learning (DL) methods have been developed to assist physicians in the ICD coding process. Our findings propose a deep learning model that utilized clinical notes from medical records to predict ICD-10 codes. Our research used text-based medical data from the outpatient department (OPD) of a university hospital from January to December 2016. The dataset used clinical notes from five departments, and a total of 21,953 medical records were collected. Clinical notes consisted of a subjective component, objective component, assessment, plan (SOAP) notes, diagnosis code, and a drug list. The dataset was divided into two groups: 90% for training and 10% for test cases. We applied natural language processing (NLP) technique (word embedding, Word2Vector) to process the data. A deep learning-based convolutional neural network (CNN) model was created based on the information presented above. Three metrics (precision, recall, and F-score) were used to calculate the achievement of the deep learning CNN model. Clinically acceptable results were achieved through the deep learning model for five departments (precision: 0.53-0.96; recall: 0.85-0.99; and F-score: 0.65-0.98). With a precision of 0.95, a recall of 0.99, and an F-score of 0.98, the deep learning model performed the best in the department of cardiology. Our proposed CNN model significantly improved the prediction performance for an automated ICD-10 code prediction system based on prior clinical information. This CNN model could reduce the laborious task of manual coding and could assist physicians in making a better diagnosis.

An efficient landmark model for prediction of suicide attempts in multiple clinical settings

Growing evidence has shown that applying machine learning models to large clinical data sources may exceed clinician performance in suicide risk stratification. However, many existing prediction models either suffer from "temporal bias" (a bias that stems from using case-control sampling) or require training on all available patient visit data. Here, we adopt a "landmark model" framework that aligns with clinical practice for prediction of suicide-related behaviors (SRBs) using a large electronic health record database. Using the landmark approach, we developed models for SRB prediction (regularized Cox regression and random survival forest) that establish a time-point (e.g., clinical visit) from which predictions are made over user-specified prediction windows using historical information up to that point. We applied this approach to cohorts from three clinical settings: general outpatient, psychiatric emergency department, and psychiatric inpatients, for varying prediction windows and lengths of historical data. Models achieved high discriminative performance (area under the Receiver Operating Characteristic curve 0.74-0.93 for the Cox model) across different prediction windows and settings, even with relatively short periods of historical data. In short, we developed accurate, dynamic SRB risk prediction models with the landmark approach that reduce bias and enhance the reliability and portability of suicide risk prediction models.

Improving Suicide Prevention Through Evidence-Based Strategies: A Systematic Review

Objective

The authors sought to identify scalable evidence-based suicide prevention strategies.

Methods

A search of PubMed and Google Scholar identi- fied 20,234 articles published between September 2005 and December 2019, of which 97 were randomized controlled trials with suicidal behavior or ideation as primary outcomes or epidemiological studies of limiting access to lethal means, using educational approaches, and the impact of antidepressant treatment.

Results

Training primary care physicians in depression rec- ognition and treatment prevents suicide. Educating youths on depression and suicidal behavior, as well as active out- reach to psychiatric patients after discharge or a suicidal crisis, prevents suicidal behavior. Meta-analyses find that antidepressants prevent suicide attempts, but individual randomized controlled trials appear to be underpowered. Ketamine reduces suicidal ideation in hours but is untested for suicidal behavior prevention. Cognitive-behavioral therapy and dialectical behavior therapy prevent suicidal behavior. Active screening for suicidal ideation or behavior is not proven to be better than just screening for depression. Education of gatekeepers about youth suicidal behavior lacks effectiveness. No randomized trials have been reported for gatekeeper training for prevention of adult suicidal behavior. Algorithm-driven electronic health record screening, Internet-based screening, and smartphone passive monitoring to identify high-risk patients are under-studied. Means restriction, including of firearms, prevents suicide but is sporadically employed in the United States, even though firearms are used in half of all U.S. suicides.

Conclusions

Training general practitioners warrants wider implementation and testing in other nonpsychiatrist physi- cian settings. Active follow-up of patients after discharge or a suicide-related crisis should be routine, and restricting firearm access by at-risk individuals warrants wider use. Combination approaches in health care systems show promise in reducing suicide in several countries, but evaluating the benefit attributable to each component is essential. Further suicide rate reduction requires evaluating newer approaches, such as electronic health record-derived algorithms, Internet-based screening methods, ketamine's potential benefit for preventing attempts, and passive monitoring of acute suicide risk change.Reprinted from Am J Psychiatry 2021; 178:611-624, with permission from American Psychiatric Association Publishing. Copyright © 2021.

Predictive Modeling for Suicide-Related Outcomes and Risk Factors among Patients with Pain Conditions: A Systematic Review

Suicide is a leading cause of death in the US. Patients with pain conditions have higher suicidal risks. In a systematic review searching observational studies from multiple sources (e.g., MEDLINE) from 1 January 2000–12 September 2020, we evaluated existing suicide prediction models’ (SPMs) performance and identified risk factors and their derived data sources among patients with pain conditions. The suicide-related outcomes included suicidal ideation, suicide attempts, suicide deaths, and suicide behaviors. Among the 87 studies included (with 8 SPM studies), 107 suicide risk factors (grouped into 27 categories) were identified. The most frequently occurring risk factor category was depression and their severity (33%). Approximately 20% of the risk factor categories would require identification from data sources beyond structured data (e.g., clinical notes). For 8 SPM studies (only 2 performing validation), the reported prediction metrics/performance varied: C-statistics (n = 3 studies) ranged 0.67–0.84, overall accuracy(n = 5): 0.78–0.96, sensitivity(n = 2): 0.65–0.91, and positive predictive values(n = 3): 0.01–0.43. Using the modified Quality in Prognosis Studies tool to assess the risk of biases, four SPM studies had moderate-to-high risk of biases. This systematic review identified a comprehensive list of risk factors that may improve predicting suicidal risks for patients with pain conditions. Future studies need to examine reasons for performance variations and SPM’s clinical utility.

A Critical Review of Text Mining Applications for Suicide Research

Purpose of Review

Applying text mining to suicide research holds a great deal of promise. In this manuscript, literature from 2019 to 2021 is critically reviewed for text mining projects that use electronic health records, social media data, and death records.

Recent Findings

Text mining has helped identify risk factors for suicide in general and specific populations (e.g., older adults), has been combined with structured variables in EHRs to predict suicide risk, and has been used to track trends in social media suicidal discourse following population level events (e.g., COVID-19, celebrity suicides).

Summary

Future research should utilize text mining along with data linkage methods to capture more complete information on risk factors and outcomes across data sources (e.g., combining death records and EHRs), evaluate effectiveness of NLP-based intervention programs that use suicide risk prediction, establish standards for reporting accuracy of text mining programs to enable comparison across studies, and incorporate implementation science to understand feasibility, acceptability, and technical considerations.

A Mental Health Chatbot with Cognitive Skills for Personalised Behavioural Activation and Remote Health Monitoring

Mental health issues are at the forefront of healthcare challenges facing contemporary human society. These issues are most prevalent among working-age people, impacting negatively on the individual, his/her family, workplace, community, and the economy. Conventional mental healthcare services, although highly effective, cannot be scaled up to address the increasing demand from affected individuals, as evidenced in the first two years of the COVID-19 pandemic. Conversational agents, or chatbots, are a recent technological innovation that has been successfully adapted for mental healthcare as a scalable platform of cross-platform smartphone applications that provides first-level support for such individuals. Despite this disposition, mental health chatbots in the extant literature and practice are limited in terms of the therapy provided and the level of personalisation. For instance, most chatbots extend Cognitive Behavioural Therapy (CBT) into predefined conversational pathways that are generic and ineffective in recurrent use. In this paper, we postulate that Behavioural Activation (BA) therapy and Artificial Intelligence (AI) are more effectively materialised in a chatbot setting to provide recurrent emotional support, personalised assistance, and remote mental health monitoring. We present the design and development of our BA-based AI chatbot, followed by its participatory evaluation in a pilot study setting that confirmed its effectiveness in providing support for individuals with mental health issues.

Detection of self-harm and suicidal ideation in emergency department triage notes

OBJECTIVE

Accurate identification of self-harm presentations to Emergency Departments (ED) can lead to more timely mental health support, aid in understanding the burden of suicidal intent in a population, and support impact evaluation of public health initiatives related to suicide prevention. Given lack of manual self-harm reporting in ED, we aim to develop an automated system for the detection of self-harm presentations directly from ED triage notes.

MATERIALS AND METHODS

We frame this as supervised classification using natural language processing (NLP), utilizing a large data set of 477 627 free-text triage notes from ED presentations in 2012-2018 to The Royal Melbourne Hospital, Australia. The data were highly imbalanced, with only 1.4% of triage notes relating to self-harm. We explored various preprocessing techniques, including spelling correction, negation detection, bigram replacement, and clinical concept recognition, and several machine learning methods.

RESULTS

Our results show that machine learning methods dramatically outperform keyword-based methods. We achieved the best results with a calibrated Gradient Boosting model, showing 90% Precision and 90% Recall (PR-AUC 0.87) on blind test data. Prospective validation of the model achieves similar results (88% Precision; 89% Recall).

DISCUSSION

ED notes are noisy texts, and simple token-based models work best. Negation detection and concept recognition did not change the results while bigram replacement significantly impaired model performance.

CONCLUSION

This first NLP-based classifier for self-harm in ED notes has practical value for identifying patients who would benefit from mental health follow-up in ED, and for supporting surveillance of self-harm and suicide prevention efforts in the population.

Improving Suicide Prevention Through Evidence-Based Strategies: A Systematic Review

OBJECTIVE

The authors sought to identify scalable evidence-based suicide prevention strategies.

METHODS

A search of PubMed and Google Scholar identified 20,234 articles published between September 2005 and December 2019, of which 97 were randomized controlled trials with suicidal behavior or ideation as primary outcomes or epidemiological studies of limiting access to lethal means, using educational approaches, and the impact of antidepressant treatment.

RESULTS

Training primary care physicians in depression recognition and treatment prevents suicide. Educating youths on depression and suicidal behavior, as well as active outreach to psychiatric patients after discharge or a suicidal crisis, prevents suicidal behavior. Meta-analyses find that antidepressants prevent suicide attempts, but individual randomized controlled trials appear to be underpowered. Ketamine reduces suicidal ideation in hours but is untested for suicidal behavior prevention. Cognitive-behavioral therapy and dialectical behavior therapy prevent suicidal behavior. Active screening for suicidal ideation or behavior is not proven to be better than just screening for depression. Education of gatekeepers about youth suicidal behavior lacks effectiveness. No randomized trials have been reported for gatekeeper training for prevention of adult suicidal behavior. Algorithm-driven electronic health record screening, Internet-based screening, and smartphone passive monitoring to identify high-risk patients are understudied. Means restriction, including of firearms, prevents suicide but is sporadically employed in the United States, even though firearms are used in half of all U.S. suicides.

CONCLUSIONS

Training general practitioners warrants wider implementation and testing in other nonpsychiatrist physician settings. Active follow-up of patients after discharge or a suicide-related crisis should be routine, and restricting firearm access by at-risk individuals warrants wider use. Combination approaches in health care systems show promise in reducing suicide in several countries, but evaluating the benefit attributable to each component is essential. Further suicide rate reduction requires evaluating newer approaches, such as electronic health record-derived algorithms, Internet-based screening methods, ketamine's potential benefit for preventing attempts, and passive monitoring of acute suicide risk change.

Diagnostic Classification and Prognostic Prediction Using Common Genetic Variants in Autism Spectrum Disorder: Genotype-Based Deep Learning

Background

In the United States, about 3 million people have autism spectrum disorder (ASD), and around 1 out of 59 children are diagnosed with ASD. People with ASD have characteristic social communication deficits and repetitive behaviors. The causes of this disorder remain unknown; however, in up to 25% of cases, a genetic cause can be identified. Detecting ASD as early as possible is desirable because early detection of ASD enables timely interventions in children with ASD. Identification of ASD based on objective pathogenic mutation screening is the major first step toward early intervention and effective treatment of affected children.

Objective

Recent investigation interrogated genomics data for detecting and treating autism disorders, in addition to the conventional clinical interview as a diagnostic test. Since deep neural networks perform better than shallow machine learning models on complex and high-dimensional data, in this study, we sought to apply deep learning to genetic data obtained across thousands of simplex families at risk for ASD to identify contributory mutations and to create an advanced diagnostic classifier for autism screening.

Methods

After preprocessing the genomics data from the Simons Simplex Collection, we extracted top ranking common variants that may be protective or pathogenic for autism based on a chi-square test. A convolutional neural network–based diagnostic classifier was then designed using the identified significant common variants to predict autism. The performance was then compared with shallow machine learning–based classifiers and randomly selected common variants.

Results

The selected contributory common variants were significantly enriched in chromosome X while chromosome Y was also discriminatory in determining the identification of autistic individuals from nonautistic individuals. The ARSD, MAGEB16, and MXRA5 genes had the largest effect in the contributory variants. Thus, screening algorithms were adapted to include these common variants. The deep learning model yielded an area under the receiver operating characteristic curve of 0.955 and an accuracy of 88% for identifying autistic individuals from nonautistic individuals. Our classifier demonstrated a considerable improvement of ~13% in terms of classification accuracy compared to standard autism screening tools.

Conclusions

Common variants are informative for autism identification. Our findings also suggest that the deep learning process is a reliable method for distinguishing the diseased group from the control group based on the common variants of autism.

Applying Convolutional Neural Networks to Predict the ICD-9 Codes of Medical Records

The International Statistical Classification of Disease and Related Health Problems (ICD) is an international standard system for categorizing and reporting diseases, injuries, disorders, and health conditions. Most previously-proposed disease predicting systems need clinical information collected by the medical staff from the patients in hospitals. In this paper, we propose a deep learning algorithm to classify disease types and identify diagnostic codes by using only the subjective component of progress notes in medical records. In this study, we have a dataset, consisting of about one hundred and sixty-eight thousand medical records, from a medical center, collected during 2003 and 2017. First, we apply standard text processing procedures to parse the sentences and word embedding techniques for vector representations. Next, we build a convolution neural network model on the medical records to predict the ICD-9 code by using a subjective component of the progress note. The prediction performance is evaluated by ten-fold cross-validation and yields an accuracy of 0.409, recall of 0.409 and precision of 0.436. If we only consider the “chapter match” of ICD-9 code, our model achieves an accuracy of 0.580, recall of 0.580, and precision of 0.582. Since our diagnostic code prediction model is solely based on subjective components (mainly, patients’ self-report descriptions), the proposed approach could serve as a remote and self-diagnosis assistance tool, prior to seeking medical advice or going to the hospital. In addition, our work may be used as a primary evaluation tool for discomfort in the rural area where medical resources are restricted.