Application of Natural Language Processing (NLP) in Detecting and Preventing Suicide Ideation: A Systematic Review

Large Language Models Versus Expert Clinicians in Crisis Prediction Among Telemental Health Patients: Comparative Study

BACKGROUND

Due to recent advances in artificial intelligence, large language models (LLMs) have emerged as a powerful tool for a variety of language-related tasks, including sentiment analysis, and summarization of provider-patient interactions. However, there is limited research on these models in the area of crisis prediction.

OBJECTIVE

This study aimed to evaluate the performance of LLMs, specifically OpenAI's generative pretrained transformer 4 (GPT-4), in predicting current and future mental health crisis episodes using patient-provided information at intake among users of a national telemental health platform.

METHODS

Deidentified patient-provided data were pulled from specific intake questions of the Brightside telehealth platform, including the chief complaint, for 140 patients who indicated suicidal ideation (SI), and another 120 patients who later indicated SI with a plan during the course of treatment. Similar data were pulled for 200 randomly selected patients, treated during the same time period, who never endorsed SI. In total, 6 senior Brightside clinicians (3 psychologists and 3 psychiatrists) were shown patients' self-reported chief complaint and self-reported suicide attempt history but were blinded to the future course of treatment and other reported symptoms, including SI. They were asked a simple yes or no question regarding their prediction of endorsement of SI with plan, along with their confidence level about the prediction. GPT-4 was provided with similar information and asked to answer the same questions, enabling us to directly compare the performance of artificial intelligence and clinicians.

RESULTS

Overall, the clinicians' average precision (0.7) was higher than that of GPT-4 (0.6) in identifying the SI with plan at intake (n=140) versus no SI (n=200) when using the chief complaint alone, while sensitivity was higher for the GPT-4 (0.62) than the clinicians' average (0.53). The addition of suicide attempt history increased the clinicians' average sensitivity (0.59) and precision (0.77) while increasing the GPT-4 sensitivity (0.59) but decreasing the GPT-4 precision (0.54). Performance decreased comparatively when predicting future SI with plan (n=120) versus no SI (n=200) with a chief complaint only for the clinicians (average sensitivity=0.4; average precision=0.59) and the GPT-4 (sensitivity=0.46; precision=0.48). The addition of suicide attempt history increased performance comparatively for the clinicians (average sensitivity=0.46; average precision=0.69) and the GPT-4 (sensitivity=0.74; precision=0.48).

CONCLUSIONS

GPT-4, with a simple prompt design, produced results on some metrics that approached those of a trained clinician. Additional work must be done before such a model can be piloted in a clinical setting. The model should undergo safety checks for bias, given evidence that LLMs can perpetuate the biases of the underlying data on which they are trained. We believe that LLMs hold promise for augmenting the identification of higher-risk patients at intake and potentially delivering more timely care to patients.

War, emotions, mental health, and artificial intelligence

During the war time dysregulation of negative emotions such as fear, anger, hatred, frustration, sadness, humiliation, and hopelessness can overrule normal societal values, culture, and endanger global peace and security, and mental health in affected societies. Therefore, it is understandable that the range and power of negative emotions may play important roles in consideration of human behavior in any armed conflict. The estimation and assessment of dominant negative emotions during war time are crucial but are challenged by the complexity of emotions' neuro-psycho-physiology. Currently available natural language processing (NLP) tools have comprehensive computational methods to analyze and understand the emotional content of related textual data in war-inflicted societies. Innovative AI-driven technologies incorporating machine learning, neuro-linguistic programming, cloud infrastructure, and novel digital therapeutic tools and applications present an immense potential to enhance mental health care worldwide. This advancement could make mental health services more cost-effective and readily accessible. Due to the inadequate number of psychiatrists and limited psychiatric resources in coping with mental health consequences of war and traumas, new digital therapeutic wearable devices supported by AI tools and means might be promising approach in psychiatry of future. Transformation of negative dominant emotional maps might be undertaken by the simultaneous combination of online cognitive behavioral therapy (CBT) on individual level, as well as usage of emotionally based strategic communications (EBSC) on a public level. The proposed positive emotional transformation by means of CBT and EBSC may provide important leverage in efforts to protect mental health of civil population in war-inflicted societies. AI-based tools that can be applied in design of EBSC stimuli, like Open AI Chat GPT or Google Gemini may have great potential to significantly enhance emotionally based strategic communications by more comprehensive understanding of semantic and linguistic analysis of available text datasets of war-traumatized society. Human in the loop enhanced by Chat GPT and Gemini can aid in design and development of emotionally annotated messages that resonate among targeted population, amplifying the impact of strategic communications in shaping human dominant emotional maps into a more positive by CBT and EBCS.

Using Large Language Models to Understand Suicidality in a Social Media-Based Taxonomy of Mental Health Disorders: Linguistic Analysis of Reddit Posts

Background

Rates of suicide have increased by over 35% since 1999. Despite concerted efforts, our ability to predict, explain, or treat suicide risk has not significantly improved over the past 50 years.

Objective

The aim of this study was to use large language models to understand natural language use during public web-based discussions (on Reddit) around topics related to suicidality.

Methods

We used large language model-based sentence embedding to extract the latent linguistic dimensions of user postings derived from several mental health-related subreddits, with a focus on suicidality. We then applied dimensionality reduction to these sentence embeddings, allowing them to be summarized and visualized in a lower-dimensional Euclidean space for further downstream analyses. We analyzed 2.9 million posts extracted from 30 subreddits, including r/SuicideWatch, between October 1 and December 31, 2022, and the same period in 2010.

Results

Our results showed that, in line with existing theories of suicide, posters in the suicidality community (r/SuicideWatch) predominantly wrote about feelings of disconnection, burdensomeness, hopeless, desperation, resignation, and trauma. Further, we identified distinct latent linguistic dimensions (well-being, seeking support, and severity of distress) among all mental health subreddits, and many of the resulting subreddit clusters were in line with a statistically driven diagnostic classification system-namely, the Hierarchical Taxonomy of Psychopathology (HiTOP)-by mapping onto the proposed superspectra.

Conclusions

Overall, our findings provide data-driven support for several language-based theories of suicide, as well as dimensional classification systems for mental health disorders. Ultimately, this novel combination of natural language processing techniques can assist researchers in gaining deeper insights about emotions and experiences shared on the web and may aid in the validation and refutation of different mental health theories.

Natural language sentiment as an indicator of depression and anxiety symptoms: a longitudinal mixed methods study1

The study tested how the use of positive- (e.g. beautiful) and negative-valenced (e.g. horrible) words in natural language and its change in time affects the severity of depression and anxiety symptoms among depressed and non-depressed individuals. This longitudinal mixed methods study (N = 40 participants, n = 1440 narratives) with three measurements within a year showed that at the between-person level the use of negative-valenced words was strongly associated with the increase in anxiety and depression symptoms over time while the use of positive-valenced words was slightly associated with the decrease in anxiety and depression symptom. These effects were not supported for within-person level (i.e. changes in word usage). No significant differences were observed in the effects between depressed and non-depressed groups. Summing up, the overall use of positive- and negative-valenced words (particularly negative-valenced words) had a stronger effect on the severity of psychopathological symptoms than their change over time. The results were discussed in the context of natural language processing and its application in diagnosing depression and anxiety symptoms.

Evidence for the biopsychosocial model of suicide: a review of whole person modeling studies using machine learning

Background

Traditional approaches to modeling suicide-related thoughts and behaviors focus on few data types from often-siloed disciplines. While psychosocial aspects of risk for these phenotypes are frequently studied, there is a lack of research assessing their impact in the context of biological factors, which are important in determining an individual's fulsome risk profile. To directly test this biopsychosocial model of suicide and identify the relative importance of predictive measures when considered together, a transdisciplinary, multivariate approach is needed. Here, we systematically review the emerging literature on large-scale studies using machine learning to integrate measures of psychological, social, and biological factors simultaneously in the study of suicide.

Methods

We conducted a systematic review of studies that used machine learning to model suicide-related outcomes in human populations including at least one predictor from each of biological, psychological, and sociological data domains. Electronic databases MEDLINE, EMBASE, PsychINFO, PubMed, and Web of Science were searched for reports published between August 2013 and August 30, 2023. We evaluated populations studied, features emerging most consistently as risk or resilience factors, methods used, and strength of evidence for or against the biopsychosocial model of suicide.

Results

Out of 518 full-text articles screened, we identified a total of 20 studies meeting our inclusion criteria, including eight studies conducted in general population samples and 12 in clinical populations. Common important features identified included depressive and anxious symptoms, comorbid psychiatric disorders, social behaviors, lifestyle factors such as exercise, alcohol intake, smoking exposure, and marital and vocational status, and biological factors such as hypothalamic-pituitary-thyroid axis activity markers, sleep-related measures, and selected genetic markers. A minority of studies conducted iterative modeling testing each data type for contribution to model performance, instead of reporting basic measures of relative feature importance.

Conclusion

Studies combining biopsychosocial measures to predict suicide-related phenotypes are beginning to proliferate. This literature provides some early empirical evidence for the biopsychosocial model of suicide, though it is marred by harmonization challenges. For future studies, more specific definitions of suicide-related outcomes, inclusion of a greater breadth of biological data, and more diversity in study populations will be needed.

The now and future of ChatGPT and GPT in psychiatry

ChatGPT has sparked extensive discussions within the healthcare community since its November 2022 release. However, potential applications in the field of psychiatry have received limited attention. Deep learning has proven beneficial to psychiatry, and GPT is a powerful deep learning-based language model with immense potential for this field. Despite the convenience of ChatGPT, this advanced chatbot currently has limited practical applications in psychiatry. It may be used to support psychiatrists in routine tasks such as completing medical records, facilitating communications between clinicians and with patients, polishing academic writings and presentations, and programming and performing analyses for research. The current training and application of ChatGPT require using appropriate prompts to maximize appropriate outputs and minimize deleterious inaccuracies and phantom errors. Moreover, future GPT advances that incorporate empathy, emotion recognition, personality assessment, and detection of mental health warning signs are essential for its effective integration into psychiatric care. In the near future, developing a fully-automated psychotherapy system trained for expert communication (such as psychotherapy verbatim) is conceivable by building on foundational GPT technology. This dream system should integrate practical 'real world' inputs and friendly AI user and patient interfaces via clinically validated algorithms, voice comprehension/generation modules, and emotion discrimination algorithms based on facial expressions and physiological inputs from wearable devices. In addition to the technology challenges, we believe it is critical to establish generally accepted ethical standards for applying ChatGPT-related tools in all mental healthcare environments, including telemedicine and academic/training settings.