Novel Use of Natural Language Processing (NLP) to Predict Suicidal Ideation and Psychiatric Symptoms in a Text-Based Mental Health Intervention in Madrid

Deep learning for identifying personal and family history of suicidal thoughts and behaviors from EHRs

Personal and family history of suicidal thoughts and behaviors (PSH and FSH, respectively) are significant risk factors associated with suicides. Research is limited in automatic identification of such data from clinical notes in Electronic Health Records. This study developed deep learning (DL) tools utilizing transformer models (Bio_ClinicalBERT and GatorTron) to detect PSH and FSH in clinical notes derived from three academic medical centers, and compared their performance with a rule-based natural language processing tool. For detecting PSH, the rule-based approach obtained an F1-score of 0.75 ± 0.07, while the Bio_ClinicalBERT and GatorTron DL tools scored 0.83 ± 0.09 and 0.84 ± 0.07, respectively. For detecting FSH, the rule-based approach achieved an F1-score of 0.69 ± 0.11, compared to 0.89 ± 0.10 for Bio_ClinicalBERT and 0.92 ± 0.07 for GatorTron. Across sites, the DL tools identified more than 80% of patients at elevated risk for suicide who remain undiagnosed and untreated.

MONDEP: A unified SpatioTemporal MONitoring Framework for National DEPression Forecasting

Depression has become a prevalent mental disorder that significantly affects a person's emotions, behaviors, physical health, ability to perform daily tasks, and ability to maintain healthy relationships. Untreated depression can escalate the risk of suicide, making the situation even worse. Despite an abundance of models previously proposed for forecasting depression, the issue of foretelling the overall number of patients at each administrative level remains under-investigated. Therefore, in this paper, we propose a simple but effective SpatioTemporal Monitoring Framework for National Depression Forecasting (MONDEP). In particular, we analyze national depression statistics data in Thailand as a case study and create prediction models for a real-time depression forecasting system using machine learning and deep learning approaches. In order to forecast the prevalence of depression at various administrative levels, we use the hierarchical structure of depression aggregation. The proposed framework consists of three modules: Data Pre-processing to extract and pre-process the raw data, Exploratory Data Analysis (EDA) to visualize and analyze the data to get insight, and Model Training and Testing to predict future depression cases. The objective of our research is to construct a comprehensive MONDEP framework that utilizes machine learning and deep learning to predict depression profiles at the district and national levels using multivariate time series across various administrative levels. Our study illustrates the considerable association between a spatial-temporal component and demonstrates how depression profiles may be represented by employing lower administrative-level data to estimate the general level of mental health across the nation. Additionally, the best performance across all criteria is obtained when a deep learning model is used to exploit multivariate time series, showing a 13% improvement in MAE measure compared to the SARIMAX baseline. We believe the proposed framework could be used as a point of reference for decision-making regarding the management of depression and has the potential to be incredibly helpful for policymakers in successfully managing mental health services on time.

Diagnosis of mental disorders using machine learning: Literature review and bibliometric mapping from 2012 to 2023

Background

Mental disorders (MDs) are becoming a leading burden in non-communicable diseases (NCDs). As per the World Health Organization's 2022 assessment report, there was a steep increase of 25 % in MDs during the COVID-19 pandemic. Early diagnosis of MDs can significantly improve treatment outcome and save disability-adjusted life years (DALYs). In recent times, the application of machine learning (ML) and deep learning (DL)) has shown promising results in the diagnosis of MDs, and the field has witnessed a huge research output in the form of research publications. Therefore, a bibliometric mapping along with a review of recent advancements is required.

Methods

This study presents a bibliometric analysis and review of the research, published over the last 10 years. Literature searches were conducted in the Scopus database for the period from January 1, 2012, to June 9, 2023. The data was filtered and screened to include only relevant and reliable publications. A total of 2811 journal articles were found. The data was exported to a comma-separated value (CSV) format for further analysis. Furthermore, a review of 40 selected studies was performed.

Results

The popularity of ML techniques in diagnosing MDs has been growing, with an annual research growth rate of 17.05 %. The Journal of Affective Disorders published the most documents (n = 97), while Wang Y. (n = 64) has published the most articles. Lotka's law is observed, with a minority of authors contributing the majority of publications. The top affiliating institutes are the West China Hospital of Sichuan University followed by the University of California, with China and the US dominating the top 10 institutes. While China has more publications, papers affiliated with the US receive more citations. Depression and schizophrenia are the primary focuses of ML and deep learning (DL) in mental disease detection. Co-occurrence network analysis reveals that ML is associated with depression, schizophrenia, autism, anxiety, ADHD, obsessive-compulsive disorder, and PTSD. Popular algorithms include support vector machine (SVM) classifier, decision tree classifier, and random forest classifier. Furthermore, DL is linked to neuroimaging techniques such as MRI, fMRI, and EEG, as well as bipolar disorder. Current research trends encompass DL, LSTM, generalized anxiety disorder, feature fusion, and convolutional neural networks.

Predicting Suicidal Ideation, Planning, and Attempts among the Adolescent Population of the United States

Suicide is the second leading cause of death among individuals aged 5 to 24 in the United States (US). However, the precursors to suicide often do not surface, making suicide prevention challenging. This study aims to develop a machine learning model for predicting suicide ideation (SI), suicide planning (SP), and suicide attempts (SA) among adolescents in the US during the coronavirus pandemic. We used the 2021 Adolescent Behaviors and Experiences Survey Data. Class imbalance was addressed using the proposed data augmentation method tailored for binary variables, Modified Synthetic Minority Over-Sampling Technique. Five different ML models were trained and compared. SHapley Additive exPlanations analysis was conducted for explainability. The Logistic Regression model, identified as the most effective, showed superior performance across all targets, achieving high scores in recall: 0.82, accuracy: 0.80, and area under the Receiver Operating Characteristic curve: 0.88. Variables such as sad feelings, hopelessness, sexual behavior, and being overweight were noted as the most important predictors. Our model holds promise in helping health policymakers design effective public health interventions. By identifying vulnerable sub-groups within regions, our model can guide the implementation of tailored interventions that facilitate early identification and referral to medical treatment.

Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing

Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.

Predicting adolescent suicidal behavior following inpatient discharge using structured and unstructured data

BACKGROUND

The objective was to develop and assess performance of an algorithm predicting suicide-related ICD codes within three months of psychiatric discharge.

METHODS

This prognostic study used a retrospective cohort of EHR data from 2789 youth (12 to 20 years old) hospitalized in a safety net institution in the Northeastern United States. The dataset combined structured data with unstructured data obtained through natural language processing of clinical notes. Machine learning approaches compared gradient boosting to random forest analyses.

RESULTS

Area under the ROC and precision-recall curve were 0.88 and 0.17, respectively, for the final Gradient Boosting model. The cutoff point of the model-generated predicted probabilities of suicide that optimally classified the individual as high risk or not was 0.009. When applying the chosen cutoff (0.009) to the hold-out testing set, the model correctly identified 8 positive cases out of 10, and 418 negative cases out 548. The corresponding performance metrics showed 80 % sensitivity, 76 % specificity, 6 % PPV, 99 % NPV, F-1 score of 0.11, and an accuracy of 76 %.

LIMITATIONS

The data in this study comes from a single health system, possibly introducing bias in the model's algorithm. Thus, the model may have underestimated the incidence of suicidal behavior in the study population. Further research should include multiple system EHRs.

CONCLUSIONS

These performance metrics suggest a benefit to including both unstructured and structured data in design of predictive algorithms for suicidal behavior, which can be integrated into psychiatric services to help assess risk.

Analysis and evaluation of explainable artificial intelligence on suicide risk assessment

This study explores the effectiveness of Explainable Artificial Intelligence (XAI) for predicting suicide risk from medical tabular data. Given the common challenge of limited datasets in health-related Machine Learning (ML) applications, we use data augmentation in tandem with ML to enhance the identification of individuals at high risk of suicide. We use SHapley Additive exPlanations (SHAP) for XAI and traditional correlation analysis to rank feature importance, pinpointing primary factors influencing suicide risk and preventive measures. Experimental results show the Random Forest (RF) model is excelling in accuracy, F1 score, and AUC (>97% across metrics). According to SHAP, anger issues, depression, and social isolation emerge as top predictors of suicide risk, while individuals with high incomes, esteemed professions, and higher education present the lowest risk. Our findings underscore the effectiveness of ML and XAI in suicide risk assessment, offering valuable insights for psychiatrists and facilitating informed clinical decisions.

Exploring the Association Between Textual Parameters and Psychological and Cognitive Factors

Background

Textual data analysis has become a popular method for examining complex human behavior in various fields, including psychology, psychiatry, sociology, computer science, data mining, forensic sciences, and communication studies. However, identifying the most relevant textual parameters for analyzing complex behavior is still a challenge.

Goal of Study

This paper aims to explore potential textual parameters that could be useful in analyzing behavior through complex textual data. Furthermore, we have examined the randomly generated text based on different textual parameters.

Methods

To achieve this goal, we conducted a comprehensive review of the literature on textual data analysis and identified several potential topics that could be relevant, such as sentiment analysis, discourse analysis, lexical analysis, and syntactic analysis. We discuss the theoretical background and practical implications of each parameter and provide examples of how they have been used in previous research. Furthermore, we highlight the importance of considering the context in which these parameters are applied and the need for interdisciplinary collaboration to gain a deeper understanding of complex behavior through textual data analysis. Furthermore, we have provided Python code in the Supplementary Materials to facilitate a comprehensive analysis of such behaviors. In addition, to generate the text for analysis, we utilized ChatGPT 3.5 Turbo by requesting it to generate a random text of 1000 words divided into five paragraphs. Afterwards, we applied the provided Python code to analyze the randomly generated text.

Conclusion

Overall, this paper provides a foundation for researchers to identify relevant textual parameters to analyze complex human behavior in their respective fields such as linguistics, sociology, psychiatry, and psychology.

Detection of Personal and Family History of Suicidal Thoughts and Behaviors using Deep Learning and Natural Language Processing: A Multi-Site Study

Objective

Personal and family history of suicidal thoughts and behaviors (PSH and FSH, respectively) are significant risk factors associated with future suicide events. These are often captured in narrative clinical notes in electronic health records (EHRs). Collaboratively, Weill Cornell Medicine (WCM), Northwestern Medicine (NM), and the University of Florida (UF) developed and validated deep learning (DL)-based natural language processing (NLP) tools to detect PSH and FSH from such notes. The tool's performance was further benchmarked against a method relying exclusively on ICD-9/10 diagnosis codes.

Materials and Methods

We developed DL-based NLP tools utilizing pre-trained transformer models Bio_ClinicalBERT and GatorTron, and compared them with expert-informed, rule-based methods. The tools were initially developed and validated using manually annotated clinical notes at WCM. Their portability and performance were further evaluated using clinical notes at NM and UF.

Results

The DL tools outperformed the rule-based NLP tool in identifying PSH and FHS. For detecting PSH, the rule-based system obtained an F1-score of 0.75 ± 0.07, while the Bio_ClinicalBERT and GatorTron DL tools scored 0.83 ± 0.09 and 0.84 ± 0.07, respectively. For detecting FSH, the rule-based NLP tool's F1-score was 0.69 ± 0.11, compared to 0.89 ± 0.10 for Bio_ClinicalBERT and 0.92 ± 0.07 for GatorTron. For the gold standard corpora across the three sites, only 2.2% (WCM), 9.3% (NM), and 7.8% (UF) of patients reported to have an ICD-9/10 diagnosis code for suicidal thoughts and behaviors prior to the clinical notes report date. The best performing GatorTron DL tool identified 93.0% (WCM), 80.4% (NM), and 89.0% (UF) of patients with documented PSH, and 85.0%(WCM), 89.5%(NM), and 100%(UF) of patients with documented FSH in their notes.

Discussion

While PSH and FSH are significant risk factors for future suicide events, little effort has been made previously to identify individuals with these history. To address this, we developed a transformer based DL method and compared with conventional rule-based NLP approach. The varying effectiveness of the rule-based tools across sites suggests a need for improvement in its dictionary-based approach. In contrast, the performances of the DL tools were higher and comparable across sites. Furthermore, DL tools were fine-tuned using only small number of annotated notes at each site, underscores its greater adaptability to local documentation practices and lexical variations.

Conclusion

Variations in local documentation practices across health care systems pose challenges to rule-based NLP tools. In contrast, the developed DL tools can effectively extract PSH and FSH information from unstructured clinical notes. These tools will provide clinicians with crucial information for assessing and treating patients at elevated risk for suicide who are rarely been diagnosed.

Machine learning and the prediction of suicide in psychiatric populations: a systematic review

Machine learning (ML) has emerged as a promising tool to enhance suicidal prediction. However, as many large-sample studies mixed psychiatric and non-psychiatric populations, a formal psychiatric diagnosis emerged as a strong predictor of suicidal risk, overshadowing more subtle risk factors specific to distinct populations. To overcome this limitation, we conducted a systematic review of ML studies evaluating suicidal behaviors exclusively in psychiatric clinical populations. A systematic literature search was performed from inception through November 17, 2022 on PubMed, EMBASE, and Scopus following the PRISMA guidelines. Original research using ML techniques to assess the risk of suicide or predict suicide attempts in the psychiatric population were included. An assessment for bias risk was performed using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines. About 1032 studies were retrieved, and 81 satisfied the inclusion criteria and were included for qualitative synthesis. Clinical and demographic features were the most frequently employed and random forest, support vector machine, and convolutional neural network performed better in terms of accuracy than other algorithms when directly compared. Despite heterogeneity in procedures, most studies reported an accuracy of 70% or greater based on features such as previous attempts, severity of the disorder, and pharmacological treatments. Although the evidence reported is promising, ML algorithms for suicidal prediction still present limitations, including the lack of neurobiological and imaging data and the lack of external validation samples. Overcoming these issues may lead to the development of models to adopt in clinical practice. Further research is warranted to boost a field that holds the potential to critically impact suicide mortality.

Identifying features of risk periods for suicide attempts using document frequency and language use in electronic health records

Background

Individualising mental healthcare at times when a patient is most at risk of suicide involves shifting research emphasis from static risk factors to those that may be modifiable with interventions. Currently, risk assessment is based on a range of extensively reported stable risk factors, but critical to dynamic suicide risk assessment is an understanding of each individual patient's health trajectory over time. The use of electronic health records (EHRs) and analysis using machine learning has the potential to accelerate progress in developing early warning indicators.

Setting

EHR data from the South London and Maudsley NHS Foundation Trust (SLaM) which provides secondary mental healthcare for 1.8 million people living in four South London boroughs.

Objectives

To determine whether the time window proximal to a hospitalised suicide attempt can be discriminated from a distal period of lower risk by analysing the documentation and mental health clinical free text data from EHRs and (i) investigate whether the rate at which EHR documents are recorded per patient is associated with a suicide attempt; (ii) compare document-level word usage between documents proximal and distal to a suicide attempt; and (iii) compare n-gram frequency related to third-person pronoun use proximal and distal to a suicide attempt using machine learning.

Methods

The Clinical Record Interactive Search (CRIS) system allowed access to de-identified information from the EHRs. CRIS has been linked with Hospital Episode Statistics (HES) data for Admitted Patient Care. We analysed document and event data for patients who had at some point between 1 April 2006 and 31 March 2013 been hospitalised with a HES ICD-10 code related to attempted suicide (X60-X84; Y10-Y34; Y87.0/Y87.2).

Findings

n = 8,247 patients were identified to have made a hospitalised suicide attempt. Of these, n = 3,167 (39.8%) of patients had at least one document available in their EHR prior to their first suicide attempt. N = 1,424 (45.0%) of these patients had been "monitored" by mental healthcare services in the past 30 days. From 60 days prior to a first suicide attempt, there was a rapid increase in the monitoring level (document recording of the past 30 days) increasing from 35.1 to 45.0%. Documents containing words related to prescribed medications/drugs/overdose/poisoning/addiction had the highest odds of being a risk indicator used proximal to a suicide attempt (OR 1.88; precision 0.91 and recall 0.93), and documents with words citing a care plan were associated with the lowest risk for a suicide attempt (OR 0.22; precision 1.00 and recall 1.00). Function words, word sequence, and pronouns were most common in all three representations (uni-, bi-, and tri-gram).

Conclusion

EHR documentation frequency and language use can be used to distinguish periods distal from and proximal to a suicide attempt. However, in our study 55.0% of patients with documentation, prior to their first suicide attempt, did not have a record in the preceding 30 days, meaning that there are a high number who are not seen by services at their most vulnerable point.

Finding the Best Match - a Case Study on the (Text-)Feature and Model Choice in Digital Mental Health Interventions

With the need for psychological help long exceeding the supply, finding ways of scaling, and better allocating mental health support is a necessity. This paper contributes by investigating how to best predict intervention dropout and failure to allow for a need-based adaptation of treatment. We systematically compare the predictive power of different text representation methods (metadata, TF-IDF, sentiment and topic analysis, and word embeddings) in combination with supplementary numerical inputs (socio-demographic, evaluation, and closed-question data). Additionally, we address the research gap of which ML model types - ranging from linear to sophisticated deep learning models - are best suited for different features and outcome variables. To this end, we analyze nearly 16.000 open-text answers from 849 German-speaking users in a Digital Mental Health Intervention (DMHI) for stress. Our research proves that - contrary to previous findings - there is great promise in using neural network approaches on DMHI text data. We propose a task-specific LSTM-based model architecture to tackle the challenge of long input sequences and thereby demonstrate the potential of word embeddings (AUC scores of up to 0.7) for predictions in DMHIs. Despite the relatively small data set, sequential deep learning models, on average, outperform simpler features such as metadata and bag-of-words approaches when predicting dropout. The conclusion is that user-generated text of the first two sessions carries predictive power regarding patients' dropout and intervention failure risk. Furthermore, the match between the sophistication of features and models needs to be closely considered to optimize results, and additional non-text features increase prediction results.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00148-z.

Natural language processing system for rapid detection and intervention of mental health crisis chat messages

Patients experiencing mental health crises often seek help through messaging-based platforms, but may face long wait times due to limited message triage capacity. Here we build and deploy a machine-learning-enabled system to improve response times to crisis messages in a large, national telehealth provider network. We train a two-stage natural language processing (NLP) system with key word filtering followed by logistic regression on 721 electronic medical record chat messages, of which 32% are potential crises (suicidal/homicidal ideation, domestic violence, or non-suicidal self-injury). Model performance is evaluated on a retrospective test set (4/1/21-4/1/22, N = 481) and a prospective test set (10/1/22-10/31/22, N = 102,471). In the retrospective test set, the model has an AUC of 0.82 (95% CI: 0.78-0.86), sensitivity of 0.99 (95% CI: 0.96-1.00), and PPV of 0.35 (95% CI: 0.309-0.4). In the prospective test set, the model has an AUC of 0.98 (95% CI: 0.966-0.984), sensitivity of 0.98 (95% CI: 0.96-0.99), and PPV of 0.66 (95% CI: 0.626-0.692). The daily median time from message receipt to crisis specialist triage ranges from 8 to 13 min, compared to 9 h before the deployment of the system. We demonstrate that a NLP-based machine learning model can reliably identify potential crisis chat messages in a telehealth setting. Our system integrates into existing clinical workflows, suggesting that with appropriate training, humans can successfully leverage ML systems to facilitate triage of crisis messages.

Combining clinical notes with structured electronic health records enhances the prediction of mental health crises

An automatic prediction of mental health crises can improve caseload prioritization and enable preventative interventions, improving patient outcomes and reducing costs. We combine structured electronic health records (EHRs) with clinical notes from 59,750 de-identified patients to predict the risk of mental health crisis relapse within the next 28 days. The results suggest that an ensemble machine learning model that relies on structured EHRs and clinical notes when available, and relying solely on structured data when the notes are unavailable, offers superior performance over models trained with either of the two data streams alone. Furthermore, the study provides key takeaways related to the required amount of clinical notes to add value in predictive analytics. This study sheds light on the untapped potential of clinical notes in the prediction of mental health crises and highlights the importance of choosing an appropriate machine learning method to combine structured and unstructured EHRs.

Identifying suicide documentation in clinical notes through zero-shot learning

Background and Aims

In deep learning, a major difficulty in identifying suicidality and its risk factors in clinical notes is the lack of training samples given the small number of true positive instances among the number of patients screened. This paper describes a novel methodology that identifies suicidality in clinical notes by addressing this data sparsity issue through zero-shot learning. Our general aim was to develop a tool that leveraged zero-shot learning to effectively identify suicidality documentation in all types of clinical notes.

Methods

US Veterans Affairs clinical notes served as data. The training data set label was determined using diagnostic codes of suicide attempt and self-harm. We used a base string associated with the target label of suicidality to provide auxiliary information by narrowing the positive training cases to those containing the base string. We trained a deep neural network by mapping the training documents' contents to a semantic space. For comparison, we trained another deep neural network using the identical training data set labels, and bag-of-words features.

Results

The zero-shot learning model outperformed the baseline model in terms of area under the curve, sensitivity, specificity, and positive predictive value at multiple probability thresholds. In applying a 0.90 probability threshold, the methodology identified notes documenting suicidality but not associated with a relevant ICD-10-CM code, with 94% accuracy.

Conclusion

This method can effectively identify suicidality without manual annotation.

Health Equity in Clinical Research Informatics

OBJECTIVES

Through a scoping review, we examine in this survey what ways health equity has been promoted in clinical research informatics with patient implications and especially published in the year of 2021 (and some in 2022).

METHOD

A scoping review was conducted guided by using methods described in the Joanna Briggs Institute Manual. The review process consisted of five stages: 1) development of aim and research question, 2) literature search, 3) literature screening and selection, 4) data extraction, and 5) accumulate and report results.

RESULTS

From the 478 identified papers in 2021 on the topic of clinical research informatics with focus on health equity as a patient implication, 8 papers met our inclusion criteria. All included papers focused on artificial intelligence (AI) technology. The papers addressed health equity in clinical research informatics either through the exposure of inequity in AI-based solutions or using AI as a tool for promoting health equity in the delivery of healthcare services. While algorithmic bias poses a risk to health equity within AI-based solutions, AI has also uncovered inequity in traditional treatment and demonstrated effective complements and alternatives that promotes health equity.

CONCLUSIONS

Clinical research informatics with implications for patients still face challenges of ethical nature and clinical value. However, used prudently-for the right purpose in the right context-clinical research informatics could bring powerful tools in advancing health equity in patient care.

Text-based predictions of COVID-19 diagnosis from self-reported chemosensory descriptions

BACKGROUND

There is a prevailing view that humans' capacity to use language to characterize sensations like odors or tastes is poor, providing an unreliable source of information.

METHODS

Here, we developed a machine learning method based on Natural Language Processing (NLP) using Large Language Models (LLM) to predict COVID-19 diagnosis solely based on text descriptions of acute changes in chemosensation, i.e., smell, taste and chemesthesis, caused by the disease. The dataset of more than 1500 subjects was obtained from survey responses early in the COVID-19 pandemic, in Spring 2020.

RESULTS

When predicting COVID-19 diagnosis, our NLP model performs comparably (AUC ROC ~ 0.65) to models based on self-reported changes in function collected via quantitative rating scales. Further, our NLP model could attribute importance of words when performing the prediction; sentiment and descriptive words such as "smell", "taste", "sense", had strong contributions to the predictions. In addition, adjectives describing specific tastes or smells such as "salty", "sweet", "spicy", and "sour" also contributed considerably to predictions.

CONCLUSIONS

Our results show that the description of perceptual symptoms caused by a viral infection can be used to fine-tune an LLM model to correctly predict and interpret the diagnostic status of a subject. In the future, similar models may have utility for patient verbatims from online health portals or electronic health records.

Dynamic suicide topic modelling: Deriving population-specific, psychosocial and time-sensitive suicide risk variables from Electronic Health Record psychotherapy notes

In the machine learning subfield of natural language processing, a topic model is a type of unsupervised method that is used to uncover abstract topics within a corpus of text. Dynamic topic modelling (DTM) is used for capturing change in these topics over time. The study deploys DTM on corpus of electronic health record psychotherapy notes. This retrospective study examines whether DTM helps distinguish closely matched patients that did and did not die by suicide. Cohort consists of United States Department of Veterans Affairs (VA) patients diagnosed with Posttraumatic Stress Disorder (PTSD) between 2004 and 2013. Each case (those who died by suicide during the year following diagnosis) was matched with five controls (those who remained alive) that shared psychotherapists and had similar suicide risk based on VA's suicide prediction algorithm. Cohort was restricted to patients who received psychotherapy for 9+ months after initial PTSD diagnoses (cases = 77; controls = 362). For cases, psychotherapy notes from diagnosis until death were examined. For controls, psychotherapy notes from diagnosis until matched case's death date were examined. A Python-based DTM algorithm was utilized. Derived topics identified population-specific themes, including PTSD, psychotherapy, medication, communication and relationships. Control topics changed significantly more over time than case topics. Topic differences highlighted engagement, expressivity and therapeutic alliance. This study strengthens groundwork for deriving population-specific, psychosocial and time-sensitive suicide risk variables.

Leveraging Natural Language Processing to Improve Electronic Health Record Suicide Risk Prediction for Veterans Health Administration Users

Background: Suicide risk prediction models frequently rely on structured electronic health record (EHR) data, including patient demographics and health care usage variables. Unstructured EHR data, such as clinical notes, may improve predictive accuracy by allowing access to detailed information that does not exist in structured data fields. To assess comparative benefits of including unstructured data, we developed a large case-control dataset matched on a state-of-the-art structured EHR suicide risk algorithm, utilized natural language processing (NLP) to derive a clinical note predictive model, and evaluated to what extent this model provided predictive accuracy over and above existing predictive thresholds.

Methods: We developed a matched case-control sample of Veterans Health Administration (VHA) patients in 2017 and 2018. Each case (all patients that died by suicide in that interval, n = 4,584) was matched with 5 controls (patients who remained alive during treatment year) who shared the same suicide risk percentile. All sample EHR notes were selected and abstracted using NLP methods. We applied machine-learning classification algorithms to NLP output to develop predictive models. We calculated area under the curve (AUC) and suicide risk concentration to evaluate predictive accuracy overall and for high-risk patients.

Results: The best performing NLP-derived models provided 19% overall additional predictive accuracy (AUC = 0.69; 95% CI, 0.67, 0.72) and 6-fold additional risk concentration for patients at the highest risk tier (top 0.1%), relative to the structured EHR model.

Conclusions: The NLP-supplemented predictive models provided considerable benefit when compared to conventional structured EHR models. Results support future structured and unstructured EHR risk model integrations.

Indirect Identification of Perinatal Psychosocial Risks from Natural Language

During the perinatal period, psychosocial health risks, including depression and intimate partner violence, are associated with serious adverse health outcomes for birth parents and children. To appropriately intervene, healthcare professionals must first identify those at risk, yet stigma often prevents people from directly disclosing the information needed to prompt an assessment. In this research we use short diary entries to indirectly elicit information that could indicate psychosocial risks, then examine patterns that emerge in the language of those at risk. We find that diary entries exhibit consistent themes, extracted using topic modeling, and emotional perspective, drawn from dictionary-informed sentiment features. Using these features, we use regularized regression to predict screening measures for depression and psychological aggression by an intimate partner. Journal text entries quantified through topic models and sentiment features show promise for depression prediction, corresponding with self-reported screening measures almost as well as closed-form questions. Text-based features are less useful in predicting intimate partner violence, but topic models generate themes that align with known risk correlates. The indirect features uncovered in this research could aid in the detection and analysis of stigmatized risks.

Text mining methods for the characterisation of suicidal thoughts and behaviour

Traditional research methods have shown low predictive value for suicidal risk assessments and limitations to be applied in clinical practice. The authors sought to evaluate natural language processing as a new tool for assessing self-injurious thoughts and behaviors and emotions related. We used MEmind project to assess 2838 psychiatric outpatients. Anonymous unstructured responses to the open-ended question "how are you feeling today?" were collected according to their emotional state. Natural language processing was used to process the patients' writings. The texts were automatically represented (corpus) and analyzed to determine their emotional content and degree of suicidal risk. Authors compared the patients' texts with a question used to assess lack of desire to live, as a suicidal risk assessment tool. Corpus consists of 5,489 short free-text documents containing 12,256 tokenized or unique words. The natural language processing showed an ROC-AUC score of 0.9638 when compared with the responses to lack of a desire to live question. Natural language processing shows encouraging results for classifying subjects according to their desire not to live as a measure of suicidal risk using patients' free texts. It is also easily applicable to clinical practice and facilitates real-time communication with patients, allowing better intervention strategies to be designed.

Measuring the impact of simulation debriefing on the practices of interprofessional trauma teams using natural language processing

BACKGROUND

Natural language processing (NLP) may be a tool for automating trauma teamwork assessment in simulated scenarios.

METHODS

Using the Trauma Nontechnical Skills Assessment (T-NOTECHS), raters assessed video recordings of trauma teams in simulated pre-debrief (Sim1) and post-debrief (Sim2) trauma resuscitations. We developed codes through directed content analysis and created algorithms capturing teamwork-related discourse through NLP. Using a within subjects pre-post design (n = 150), we compared changes in teams' Sim1 versus Sim2 T-NOTECHS scores and automatically coded discourse to identify which NLP algorithms could identify skills assessed by the T-NOTECHS.

RESULTS

Automatically coded behaviors revealed significant post-debrief increases in teams' simulation discourse: Verbalizing Findings, Acknowledging Communication, Directed Communication, Directing Assessment and Role Assignment, and Leader as Hub for Information.

CONCLUSIONS

Our results suggest NLP can capture changes in trauma team discourse. These findings have implications for the expedition of team assessment and innovations in real-time feedback when paired with speech-to-text technology.

Application of Natural Language Processing (NLP) in Detecting and Preventing Suicide Ideation: A Systematic Review

(1) Introduction: Around a million people are reported to die by suicide every year, and due to the stigma associated with the nature of the death, this figure is usually assumed to be an underestimate. Machine learning and artificial intelligence such as natural language processing has the potential to become a major technique for the detection, diagnosis, and treatment of people. (2) Methods: PubMed, EMBASE, MEDLINE, PsycInfo, and Global Health databases were searched for studies that reported use of NLP for suicide ideation or self-harm. (3) Result: The preliminary search of 5 databases generated 387 results. Removal of duplicates resulted in 158 potentially suitable studies. Twenty papers were finally included in this review. (4) Discussion: Studies show that combining structured and unstructured data in NLP data modelling yielded more accurate results than utilizing either alone. Additionally, to reduce suicides, people with mental problems must be continuously and passively monitored. (5) Conclusions: The use of AI&ML opens new avenues for considerably guiding risk prediction and advancing suicide prevention frameworks. The review's analysis of the included research revealed that the use of NLP may result in low-cost and effective alternatives to existing resource-intensive methods of suicide prevention.

A Hybrid Deep Learning Model Using Grid Search and Cross-Validation for Effective Classification and Prediction of Suicidal Ideation from Social Network Data

Suicide deaths due to depression and mental stress are growing rapidly at an alarming rate. People freely express their feelings and emotions on social network sites while they feel hesitant to express such feelings during face-to-face interactions with their dear ones. In this study, a dataset comprising 20,000 posts was taken from Reddit and preprocessed into tokens using a variety of effective word2vec techniques. A new hybrid approach is proposed by combining the attention model in a convolutional neural network and long-short-term- memory. The objective of this research is to develop an effective learning model to evaluate the data on social media for the efficient and accurate identification of people with suicidal ideation. The proposed attention convolution long short-term memory (ACL) model uses hyperparameter tuning using a grid search to select optimized hyperparameters. From the experimental evaluation, it is shown that the proposed model, that is, ACL with Glove embedding after hyperparameter tuning gives the highest Accuracy of 88.48%, Precision of 87.36%, F1 score of 90.82% and specificity of 79.23% and ACL with Random embedding gives the highest Recall of 94.94% when compared to the state-of-the-art algorithms.

Identification of maternal depression risk from natural language collected in a mobile health app

Depression is one of the most common pregnancy complications, affecting approximately 15% of pregnant people. While valid psychometric measures of depression risk exist, they are not consistently administered at routine prenatal care, exacerbating the problem of adequate detection. The language we use in daily life offers a window into our psychological wellbeing. In this longitudinal observational cohort study of prenatal patients using a prenatal care mobile health app, we examine how features of app-entered natural language and other app-entered patient-reported data may be used as indicators for validated depression risk measures. Patient participants (n=1091) were prescribed a prenatal care app as part of a quality improvement initiative in the UPMC healthcare system from September 2019 - May 2022. Natural language from open-ended writing prompts in the app and self-reported daily mood, were entered by patients using the tool. Participants also completed a validated measure of depression risk - the Edinburgh Postnatal Depression Scale (EPDS) - at least once in their pregnancy. A variety of natural language processing tools were used to score sentiment, categorize topics, and capture other semantic and syntactic information from text entries. LASSO was used to model the relationship between the natural language features and depression risk. Open-ended text within a 30-day and 60-day timeframe of completing an EPDS was found to be moderately predictive of moderate to severe depression risk (AUROC=0.66 and 0.67, for each respective timeframe). When combined with average daily reported mood, open-ended text showed good predictive power (AUROC=0.87). Consistently predictive language features across all models included themes of "money" and "sadness." The combination of natural language and other user-reported data collected through a mobile health app offers an opportunity for identifying depression risk among a pregnant population.

Leveraging unstructured electronic medical record notes to derive population-specific suicide risk models

Electronic medical record (EMR)-based suicide risk prediction methods typically rely on analysis of structured variables such as demographics, visit history, and prescription data. Leveraging unstructured EMR notes may improve predictive accuracy by allowing access to nuanced clinical information. We utilized natural language processing (NLP) to analyze a large EMR note corpus to develop a data-driven suicide risk prediction model. We developed a matched case-control sample of U.S. Department of Veterans Affairs (VA) patients in 2015 and 2016. We randomly matched each case (all patients that died by suicide in that interval, n = 5029) with five controls (patients that remained alive). We processed note corpus using NLP methods and applied machine-learning classification algorithms to output. We calculated area under the curve (AUC) and risk tiers to determine predictive accuracy. NLP-derived models demonstrated strong predictive accuracy. Patients that scored within top 10% of risk model accounted for up to 29% of suicide decedents. NLP-derived model compares positively to other leading prediction methods. Our approach is highly implementable, only requiring access to text data and open-source software. Additional studies should evaluate ensemble models incorporating NLP-derived information alongside more typical structured variables.

Identify Caregiver Availability Using Medical Notes: Rule-Based Natural Language Processing

Background

Identifying caregiver availability, particularly for patients with dementia or those with a disability, is critical to informing the appropriate care planning by the health systems, hospitals, and providers. This information is not readily available, and there is a paucity of pragmatic approaches to automatically identifying caregiver availability and type.

Objective

Our main objective was to use medical notes to assess caregiver availability and type for hospitalized patients with dementia. Our second objective was to identify whether the patient lived at home or resided at an institution.

Methods

In this retrospective cohort study, we used 2016-2019 telephone-encounter medical notes from a single institution to develop a rule-based natural language processing (NLP) algorithm to identify the patient’s caregiver availability and place of residence. Using note-level data, we compared the results of the NLP algorithm with human-conducted chart abstraction for both training (749/976, 77%) and test sets (227/976, 23%) for a total of 223 adults aged 65 years and older diagnosed with dementia. Our outcomes included determining whether the patients (1) reside at home or in an institution, (2) have a formal caregiver, and (3) have an informal caregiver.

Results

Test set results indicated that our NLP algorithm had high level of accuracy and reliability for identifying whether patients had an informal caregiver (F₁=0.94, accuracy=0.95, sensitivity=0.97, and specificity=0.93), but was relatively less able to identify whether the patient lived at an institution (F₁=0.64, accuracy=0.90, sensitivity=0.51, and specificity=0.98). The most common explanations for NLP misclassifications across all categories were (1) incomplete or misspelled facility names; (2) past, uncertain, or undecided status; (3) uncommon abbreviations; and (4) irregular use of templates.

Conclusions

This innovative work was the first to use medical notes to pragmatically determine caregiver availability. Our NLP algorithm identified whether hospitalized patients with dementia have a formal or informal caregiver and, to a lesser extent, whether they lived at home or in an institutional setting. There is merit in using NLP to identify caregivers. This study serves as a proof of concept. Future work can use other approaches and further identify caregivers and the extent of their availability.

Natural language processing to identify case factors in child protection court proceedings

Social work case files hold rich detail about the lives and needs of vulnerable groups. Traditional case-reading studies to gain generalisable knowledge are resource-intensive, however, and sample sizes thereby limited. The advent of ‘big data’ technology, and vast repositories of centrally stored electronic records offer social work researchers novel alternatives, including data linkage and predictive risk modelling using administrative data. Free-text documents, however – including assessments, reports, and case chronologies – remain a largely untapped resource. This paper describes how 5000 social work court statements held by the Child and Family Court Advisory Support Service in England (Cafcass) were analysed using natural language processing (NLP) based on simple rules and mathematical principles. Thirteen factors relating to harm and risk to children involved in care proceedings in England were identified by automated computer techniques, and almost 90% agreement with professional readers achieved when the factors were clear-cut. The study represents an innovative approach for social work research on complex social problems. In conclusion, the paper discusses learning points; practical implications; future research avenues; and the technical and ethical challenges of NLP.

A Critical Review of Text Mining Applications for Suicide Research

Purpose of Review

Applying text mining to suicide research holds a great deal of promise. In this manuscript, literature from 2019 to 2021 is critically reviewed for text mining projects that use electronic health records, social media data, and death records.

Recent Findings

Text mining has helped identify risk factors for suicide in general and specific populations (e.g., older adults), has been combined with structured variables in EHRs to predict suicide risk, and has been used to track trends in social media suicidal discourse following population level events (e.g., COVID-19, celebrity suicides).

Summary

Future research should utilize text mining along with data linkage methods to capture more complete information on risk factors and outcomes across data sources (e.g., combining death records and EHRs), evaluate effectiveness of NLP-based intervention programs that use suicide risk prediction, establish standards for reporting accuracy of text mining programs to enable comparison across studies, and incorporate implementation science to understand feasibility, acceptability, and technical considerations.

Use of Artificial Intelligence-Based Strategies for Assessing Suicidal Behavior and Mental Illness: A Literature Review

Mental illness leading to suicide attempts is prevalent in a large portion of the population especially in low and middle-income nations. There remains a significant social stigma associated with mental illness that can lead to stigmatization of patients. Hence, patients are reluctant to communicate their problems to health care providers. Physicians have difficulty in timely identification of patients at risk for suicide. Novel and rigorously designed strategies are needed to determine the population at risk for suicide. This would be the first step in overcoming the multitude of barriers in the management of mental illness. Clinical tools and the use of electronic medical records (EMR) are time intensive. Recently, several artificial intelligence (AI)-based predictive technologies have gained momentum. The aim of this review is to summarize the recent advances in this landscape.

Analysis of a national response to a White House directive for ending veteran suicide

OBJECTIVE

Analyze responses to a national request for information (RFI) to uncover gaps in policy, practice, and understanding of veteran suicide to inform federal research strategy.

DATA SOURCE

An RFI with 21 open-ended questions generated from Presidential Executive Order #1386, administered nationally from July 3 to August 5, 2019.

STUDY DESIGN

Semi-structured, open-ended responses analyzed using a collaborative qualitative and text-mining data process.

DATA EXTRACTION METHODS

We aligned traditional qualitative methods with natural language processing (NLP) text-mining techniques to analyze 9040 open-ended question responses from 722 respondents to provide results within 3 months. Narrative inquiry and the medical explanatory model guided the data extraction and analytic process.

RESULTS

Five major themes were identified: risk factors, risk assessment, prevention and intervention, barriers to care, and data/research. Individuals and organizations mentioned different concepts within the same themes. In responses about risk factors, individuals frequently mentioned generic terms like "illness" while organizations mentioned specific terms like "traumatic brain injury." Organizations and individuals described unique barriers to care and emphasized ways to integrate data and research to improve points of care. Organizations often identified lack of funding as barriers while individuals often identified key moments for prevention such as military transitions and ensuring care providers have military cultural understanding.

CONCLUSIONS

This study provides an example of a rapid, adaptive analysis of a large body of qualitative, public response data about veteran suicide to support a federal strategy for an important public health topic. Combining qualitative and text-mining methods allowed a representation of voices and perspectives including the lived experiences of individuals who described stories of military transition, treatments that worked or did not, and the perspective of organizations treating veterans for suicide. The results supported the development of a national strategy to reduce suicide risks for veterans as well as civilians.

The usefulness of NLP techniques for predicting peaks in firefighter interventions due to rare events

In some countries such as France, the number of operations assisted by firefighters has shown an almost linear increase over the years, contrary to their resource capacity. For this reason, predicting the number of interventions has become a necessity. Initially, time series models were developed with several types of qualitative and quantitative features, including the alert level of the bulletins, to predict the operational load. We realized that interventions related to human activities are quite predictable. However, the recognition of interventions due to rare events such as storms or floods needs more than quantitative meteorological data to be identified, since there are almost always zero cases. Thus, this work proposes the application of natural language processing techniques, namely long short-term memory, convolutional neural networks, FlauBERT, and CamemBERT to extract features from the texts of weather bulletins in order to recognize periods with peak interventions, where the intense workload of firefighters is caused by rare events. Four categories identified as Emergency Person Rescue, Total Person Rescue, interventions related to Heating, and Storm/Flood were our targets for the multilabel classification models developed. The results showed a remarkable accuracy of 80%, 86%, 92%, and 86% for Emergency Rescue People, Total Rescue People, Heating, and Storm/Flood, respectively.

Listening to Mental Health Crisis Needs at Scale: Using Natural Language Processing to Understand and Evaluate a Mental Health Crisis Text Messaging Service

The current mental health crisis is a growing public health issue requiring a large-scale response that cannot be met with traditional services alone. Digital support tools are proliferating, yet most are not systematically evaluated, and we know little about their users and their needs. Shout is a free mental health text messaging service run by the charity Mental Health Innovations, which provides support for individuals in the UK experiencing mental or emotional distress and seeking help. Here we study a large data set of anonymised text message conversations and post-conversation surveys compiled through Shout. This data provides an opportunity to hear at scale from those experiencing distress; to better understand mental health needs for people not using traditional mental health services; and to evaluate the impact of a novel form of crisis support. We use natural language processing (NLP) to assess the adherence of volunteers to conversation techniques and formats, and to gain insight into demographic user groups and their behavioural expressions of distress. Our textual analyses achieve accurate classification of conversation stages (weighted accuracy = 88%), behaviours (1-hamming loss = 95%) and texter demographics (weighted accuracy = 96%), exemplifying how the application of NLP to frontline mental health data sets can aid with post-hoc analysis and evaluation of quality of service provision in digital mental health services.

Play the Pain: A Digital Strategy for Play-Oriented Research and Action

The value of understanding patients' illness experience and social contexts for advancing medicine and clinical care is widely acknowledged. However, methodologies for rigorous and inclusive data gathering and integrative analysis of biomedical, cultural, and social factors are limited. In this paper, we propose a digital strategy for large-scale qualitative health research, using play (as a state of being, a communication mode or context, and a set of imaginative, expressive, and game-like activities) as a research method for recursive learning and action planning. Our proposal builds on Gregory Bateson's cybernetic approach to knowledge production. Using chronic pain as an example, we show how pragmatic, structural and cultural constraints that define the relationship of patients to the healthcare system can give rise to conflicted messaging that impedes inclusive health research. We then review existing literature to illustrate how different types of play including games, chatbots, virtual worlds, and creative art making can contribute to research in chronic pain. Inspired by Frederick Steier's application of Bateson's theory to designing a science museum, we propose DiSPORA (Digital Strategy for Play-Oriented Research and Action), a virtual citizen science laboratory which provides a framework for delivering health information, tools for play-based experimentation, and data collection capacity, but is flexible in allowing participants to choose the mode and the extent of their interaction. Combined with other data management platforms used in epidemiological studies of neuropsychiatric illness, DiSPORA offers a tool for large-scale qualitative research, digital phenotyping, and advancing personalized medicine.

Automatic gender detection in Twitter profiles for health-related cohort studies

Objective

Biomedical research involving social media data is gradually moving from population-level to targeted, cohort-level data analysis. Though crucial for biomedical studies, social media user’s demographic information (eg, gender) is often not explicitly known from profiles. Here, we present an automatic gender classification system for social media and we illustrate how gender information can be incorporated into a social media-based health-related study.

Materials and Methods

We used a large Twitter dataset composed of public, gender-labeled users (Dataset-1) for training and evaluating the gender detection pipeline. We experimented with machine learning algorithms including support vector machines (SVMs) and deep-learning models, and public packages including M3. We considered users’ information including profile and tweets for classification. We also developed a meta-classifier ensemble that strategically uses the predicted scores from the classifiers. We then applied the best-performing pipeline to Twitter users who have self-reported nonmedical use of prescription medications (Dataset-2) to assess the system’s utility.

Results and Discussion

We collected 67 181 and 176 683 users for Dataset-1 and Dataset-2, respectively. A meta-classifier involving SVM and M3 performed the best (Dataset-1 accuracy: 94.4% [95% confidence interval: 94.0–94.8%]; Dataset-2: 94.4% [95% confidence interval: 92.0–96.6%]). Including automatically classified information in the analyses of Dataset-2 revealed gender-specific trends—proportions of females closely resemble data from the National Survey of Drug Use and Health 2018 (tranquilizers: 0.50 vs 0.50; stimulants: 0.50 vs 0.45), and the overdose Emergency Room Visit due to Opioids by Nationwide Emergency Department Sample (pain relievers: 0.38 vs 0.37).

Conclusion

Our publicly available, automated gender detection pipeline may aid cohort-specific social media data analyses (https://bitbucket.org/sarkerlab/gender-detection-for-public).

Addressing Biodisaster X Threats With Artificial Intelligence and 6G Technologies: Literature Review and Critical Insights

BACKGROUND

With advances in science and technology, biotechnology is becoming more accessible to people of all demographics. These advances inevitably hold the promise to improve personal and population well-being and welfare substantially. It is paradoxical that while greater access to biotechnology on a population level has many advantages, it may also increase the likelihood and frequency of biodisasters due to accidental or malicious use. Similar to "Disease X" (describing unknown naturally emerging pathogenic diseases with a pandemic potential), we term this unknown risk from biotechnologies "Biodisaster X." To date, no studies have examined the potential role of information technologies in preventing and mitigating Biodisaster X.

OBJECTIVE

This study aimed to explore (1) what Biodisaster X might entail and (2) solutions that use artificial intelligence (AI) and emerging 6G technologies to help monitor and manage Biodisaster X threats.

METHODS

A review of the literature on applying AI and 6G technologies for monitoring and managing biodisasters was conducted on PubMed, using articles published from database inception through to November 16, 2020.

RESULTS

Our findings show that Biodisaster X has the potential to upend lives and livelihoods and destroy economies, essentially posing a looming risk for civilizations worldwide. To shed light on Biodisaster X threats, we detailed effective AI and 6G-enabled strategies, ranging from natural language processing to deep learning-based image analysis to address issues ranging from early Biodisaster X detection (eg, identification of suspicious behaviors), remote design and development of pharmaceuticals (eg, treatment development), and public health interventions (eg, reactive shelter-at-home mandate enforcement), as well as disaster recovery (eg, sentiment analysis of social media posts to shed light on the public's feelings and readiness for recovery building).

CONCLUSIONS

Biodisaster X is a looming but avoidable catastrophe. Considering the potential human and economic consequences Biodisaster X could cause, actions that can effectively monitor and manage Biodisaster X threats must be taken promptly and proactively. Rather than solely depending on overstretched professional attention of health experts and government officials, it is perhaps more cost-effective and practical to deploy technology-based solutions to prevent and control Biodisaster X threats. This study discusses what Biodisaster X could entail and emphasizes the importance of monitoring and managing Biodisaster X threats by AI techniques and 6G technologies. Future studies could explore how the convergence of AI and 6G systems may further advance the preparedness for high-impact, less likely events beyond Biodisaster X.

Machine Learning and Natural Language Processing in Mental Health: Systematic Review

BACKGROUND

Machine learning systems are part of the field of artificial intelligence that automatically learn models from data to make better decisions. Natural language processing (NLP), by using corpora and learning approaches, provides good performance in statistical tasks, such as text classification or sentiment mining.

OBJECTIVE

The primary aim of this systematic review was to summarize and characterize, in methodological and technical terms, studies that used machine learning and NLP techniques for mental health. The secondary aim was to consider the potential use of these methods in mental health clinical practice.

METHODS

This systematic review follows the PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis) guidelines and is registered with PROSPERO (Prospective Register of Systematic Reviews; number CRD42019107376). The search was conducted using 4 medical databases (PubMed, Scopus, ScienceDirect, and PsycINFO) with the following keywords: machine learning, data mining, psychiatry, mental health, and mental disorder. The exclusion criteria were as follows: languages other than English, anonymization process, case studies, conference papers, and reviews. No limitations on publication dates were imposed.

RESULTS

A total of 327 articles were identified, of which 269 (82.3%) were excluded and 58 (17.7%) were included in the review. The results were organized through a qualitative perspective. Although studies had heterogeneous topics and methods, some themes emerged. Population studies could be grouped into 3 categories: patients included in medical databases, patients who came to the emergency room, and social media users. The main objectives were to extract symptoms, classify severity of illness, compare therapy effectiveness, provide psychopathological clues, and challenge the current nosography. Medical records and social media were the 2 major data sources. With regard to the methods used, preprocessing used the standard methods of NLP and unique identifier extraction dedicated to medical texts. Efficient classifiers were preferred rather than transparent functioning classifiers. Python was the most frequently used platform.

CONCLUSIONS

Machine learning and NLP models have been highly topical issues in medicine in recent years and may be considered a new paradigm in medical research. However, these processes tend to confirm clinical hypotheses rather than developing entirely new information, and only one major category of the population (ie, social media users) is an imprecise cohort. Moreover, some language-specific features can improve the performance of NLP methods, and their extension to other languages should be more closely investigated. However, machine learning and NLP techniques provide useful information from unexplored data (ie, patients' daily habits that are usually inaccessible to care providers). Before considering It as an additional tool of mental health care, ethical issues remain and should be discussed in a timely manner. Machine learning and NLP methods may offer multiple perspectives in mental health research but should also be considered as tools to support clinical practice.

Information and Communication Technology Use in Suicide Prevention: Scoping Review

BACKGROUND

The use of information and communication technology (ICT) in suicide prevention has progressed rapidly over the past decade. ICT plays a major role in suicide prevention, but research on best and promising practices has been slow.

OBJECTIVE

This paper aims to explore the existing literature on ICT use in suicide prevention to answer the following question: what are the best and most promising ICT practices for suicide prevention?

METHODS

A scoping search was conducted using the following databases: PubMed, PsycINFO, Sociological Abstracts, and IEEE Xplore. These databases were searched for articles published between January 1, 2013, and December 31, 2018. The five stages of the scoping review process were as follows: identifying research questions; targeting relevant studies; selecting studies; charting data; and collating, summarizing, and reporting the results. The World Health Organization suicide prevention model was used according to the continuum of universal, selective, and indicated prevention.

RESULTS

Of the 3848 studies identified, 115 (2.99%) were selected. Of these, 10 regarded the use of ICT in universal suicide prevention, 53 referred to the use of ICT in selective suicide prevention, and 52 dealt with the use of ICT in indicated suicide prevention.

CONCLUSIONS

The use of ICT plays a major role in suicide prevention, and many promising programs were identified through this scoping review. However, large-scale evaluation studies are needed to further examine the effectiveness of these programs and strategies. In addition, safety and ethics protocols for ICT-based interventions are recommended.

Naturally occurring language as a source of evidence in suicide prevention

We discuss computational language analysis as it pertains to suicide prevention research, with an emphasis on providing non-technologists with an understanding of key issues and, equally important, considering its relation to the broader enterprise of suicide prevention. Our emphasis here is on naturally occurring language in social media, motivated by its non-intrusive ability to yield high-value information that in the past has been largely unavailable to clinicians.

The use of text-based responses to improve our understanding and prediction of suicide risk

OBJECTIVE

Text-based responses may provide significant contributions to suicide risk prediction, yet research including text data is limited. This may be due to a lack of exposure and familiarity with statistical analyses for this data structure.

METHOD

The current study provides an overview of data processing and statistical algorithms for text data, guided by an empirical example of 947 online participants who completed both open-ended items and traditional self-report measures. We give an introduction to a number of text-based statistical approaches, including dictionary-based methods, topic modeling, word embeddings, and deep learning.

RESULTS

We analyze responses from the open-ended question "How do you feel today?", detailing characteristics of the responses, as well as predicting past-year suicidal ideation.

CONCLUSIONS

We see the analysis of text from social media, open-ended questions, and other text sources (i.e., medical records) as an important form of complementary assessment to traditional scales, shedding insight on what we are missing in our current set of questionnaires, which may ultimately serve to improve both our understanding and prediction of suicide.

Using natural language processing to classify social work interventions

OBJECTIVES

Health care organizations are increasingly employing social workers to address patients' social needs. However, social work (SW) activities in health care settings are largely captured as text data within electronic health records (EHRs), making measurement and analysis difficult. This study aims to extract and classify, from EHR notes, interventions intended to address patients' social needs using natural language processing (NLP) and machine learning (ML) algorithms.

STUDY DESIGN

Secondary data analysis of a longitudinal cohort.

METHODS

We extracted 815 SW encounter notes from the EHR system of a federally qualified health center. We reviewed the literature to derive a 10-category classification scheme for SW interventions. We applied NLP and ML algorithms to categorize the documented SW interventions in EHR notes according to the 10-category classification scheme.

RESULTS

Most of the SW notes (n = 598; 73.4%) contained at least 1 SW intervention. The most frequent interventions offered by social workers included care coordination (21.5%), education (21.0%), financial planning (18.5%), referral to community services and organizations (17.1%), and supportive counseling (15.3%). High-performing classification algorithms included the kernelized support vector machine (SVM) (accuracy, 0.97), logistic regression (accuracy, 0.96), linear SVM (accuracy, 0.95), and multinomial naive Bayes classifier (accuracy, 0.92).

CONCLUSIONS

NLP and ML can be utilized for automated identification and classification of SW interventions documented in EHRs. Health care administrators can leverage this automated approach to gain better insight into the most needed social interventions in the patient population served by their organizations. Such information can be applied in managerial decisions related to SW staffing, resource allocation, and patients' social needs.

Prediction of Loneliness in Older Adults Using Natural Language Processing: Exploring Sex Differences in Speech

OBJECTIVE

The growing pandemic of loneliness has great relevance to aging populations, though assessments are limited by self-report approaches. This paper explores the use of artificial intelligence (AI) technology to evaluate interviews on loneliness, notably, employing natural language processing (NLP) to quantify sentiment and features that indicate loneliness in transcribed speech text of older adults.

DESIGN

Participants completed semi-structured qualitative interviews regarding the experience of loneliness and a quantitative self-report scale (University of California Los Angeles or UCLA Loneliness scale) to assess loneliness. Lonely and non-lonely participants (based on qualitative and quantitative assessments) were compared.

SETTING

Independent living sector of a senior housing community in San Diego County.

PARTICIPANTS

Eighty English-speaking older adults with age range 66-94 (mean 83 years).

MEASUREMENTS

Interviews were audiotaped and manually transcribed. Transcripts were examined using NLP approaches to quantify sentiment and expressed emotions.

RESULTS

Lonely individuals (by qualitative assessments) had longer responses with greater expression of sadness to direct questions about loneliness. Women were more likely to endorse feeling lonely during the qualitative interview. Men used more fearful and joyful words in their responses. Using linguistic features, machine learning models could predict qualitative loneliness with 94% precision (sensitivity = 0.90, specificity = 1.00) and quantitative loneliness with 76% precision (sensitivity = 0.57, specificity = 0.89).

CONCLUSIONS

AI (e.g., NLP and machine learning approaches) can provide unique insights into how linguistic features of transcribed speech data may reflect loneliness. Eventually linguistic features could be used to assess loneliness of individuals, despite limitations of commercially developed natural language understanding programs.

Social Media and Suicide: A Validation of Terms to Help Identify Suicide-related Social Media Posts

BACKGROUND

Communication plays an important role in the prevention of suicide, a leading cause of death in the United States. Prior research suggests people who die by suicide often communicate their intent to more than one member of their social network. The ubiquity of social media in modern society means an individual's social network may be larger than ever before, which has contributed to a proliferation of colloquial terms and phrases to describe suicide.

AIMS

The present study collected and validated suicide-related terms from the U.S. English language in 2018-2019. By validating clinical and lay terms with people on the front lines of suicide prevention, the study provides a necessary foundation for lexical analyses of suicide communication on social media.

METHOD

98 terms related to suicide were collected from online, academic, and other sources. Mental health professionals and members of the electronic mailing list of the American Association of Suicidology were asked to validate terms.

RESULTS

The survey validated common terms used to communicate about suicide.

LIMITATIONS

The lexicon did not capture international phrases. It also did not document less direct language, such as expressions of emotion.

Artificial Intelligence and Suicide Prevention: A Systematic Review of Machine Learning Investigations

Suicide is a leading cause of death that defies prediction and challenges prevention efforts worldwide. Artificial intelligence (AI) and machine learning (ML) have emerged as a means of investigating large datasets to enhance risk detection. A systematic review of ML investigations evaluating suicidal behaviors was conducted using PubMed/MEDLINE, PsychInfo, Web-of-Science, and EMBASE, employing search strings and MeSH terms relevant to suicide and AI. Databases were supplemented by hand-search techniques and Google Scholar. Inclusion criteria: (1) journal article, available in English, (2) original investigation, (3) employment of AI/ML, (4) evaluation of a suicide risk outcome. N = 594 records were identified based on abstract search, and 25 hand-searched reports. N = 461 reports remained after duplicates were removed, n = 316 were excluded after abstract screening. Of n = 149 full-text articles assessed for eligibility, n = 87 were included for quantitative synthesis, grouped according to suicide behavior outcome. Reports varied widely in methodology and outcomes. Results suggest high levels of risk classification accuracy (>90%) and Area Under the Curve (AUC) in the prediction of suicidal behaviors. We report key findings and central limitations in the use of AI/ML frameworks to guide additional research, which hold the potential to impact suicide on broad scale.

A Framework for Applying Natural Language Processing in Digital Health Interventions

BACKGROUND

Digital health interventions (DHIs) are poised to reduce target symptoms in a scalable, affordable, and empirically supported way. DHIs that involve coaching or clinical support often collect text data from 2 sources: (1) open correspondence between users and the trained practitioners supporting them through a messaging system and (2) text data recorded during the intervention by users, such as diary entries. Natural language processing (NLP) offers methods for analyzing text, augmenting the understanding of intervention effects, and informing therapeutic decision making.

OBJECTIVE

This study aimed to present a technical framework that supports the automated analysis of both types of text data often present in DHIs. This framework generates text features and helps to build statistical models to predict target variables, including user engagement, symptom change, and therapeutic outcomes.

METHODS

We first discussed various NLP techniques and demonstrated how they are implemented in the presented framework. We then applied the framework in a case study of the Healthy Body Image Program, a Web-based intervention trial for eating disorders (EDs). A total of 372 participants who screened positive for an ED received a DHI aimed at reducing ED psychopathology (including binge eating and purging behaviors) and improving body image. These users generated 37,228 intervention text snippets and exchanged 4285 user-coach messages, which were analyzed using the proposed model.

RESULTS

We applied the framework to predict binge eating behavior, resulting in an area under the curve between 0.57 (when applied to new users) and 0.72 (when applied to new symptom reports of known users). In addition, initial evidence indicated that specific text features predicted the therapeutic outcome of reducing ED symptoms.

CONCLUSIONS

The case study demonstrates the usefulness of a structured approach to text data analytics. NLP techniques improve the prediction of symptom changes in DHIs. We present a technical framework that can be easily applied in other clinical trials and clinical presentations and encourage other groups to apply the framework in similar contexts.

Stigma, biomarkers, and algorithmic bias: recommendations for precision behavioral health with artificial intelligence

Effective implementation of artificial intelligence in behavioral healthcare delivery depends on overcoming challenges that are pronounced in this domain. Self and social stigma contribute to under-reported symptoms, and under-coding worsens ascertainment. Health disparities contribute to algorithmic bias. Lack of reliable biological and clinical markers hinders model development, and model explainability challenges impede trust among users. In this perspective, we describe these challenges and discuss design and implementation recommendations to overcome them in intelligent systems for behavioral and mental health.