1
|
Khosravi H, Ahmed I, Choudhury A. Predicting Suicidal Ideation, Planning, and Attempts among the Adolescent Population of the United States. Healthcare (Basel) 2024; 12:1262. [PMID: 38998797 PMCID: PMC11241284 DOI: 10.3390/healthcare12131262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Revised: 06/20/2024] [Accepted: 06/22/2024] [Indexed: 07/14/2024] Open
Abstract
Suicide is the second leading cause of death among individuals aged 5 to 24 in the United States (US). However, the precursors to suicide often do not surface, making suicide prevention challenging. This study aims to develop a machine learning model for predicting suicide ideation (SI), suicide planning (SP), and suicide attempts (SA) among adolescents in the US during the coronavirus pandemic. We used the 2021 Adolescent Behaviors and Experiences Survey Data. Class imbalance was addressed using the proposed data augmentation method tailored for binary variables, Modified Synthetic Minority Over-Sampling Technique. Five different ML models were trained and compared. SHapley Additive exPlanations analysis was conducted for explainability. The Logistic Regression model, identified as the most effective, showed superior performance across all targets, achieving high scores in recall: 0.82, accuracy: 0.80, and area under the Receiver Operating Characteristic curve: 0.88. Variables such as sad feelings, hopelessness, sexual behavior, and being overweight were noted as the most important predictors. Our model holds promise in helping health policymakers design effective public health interventions. By identifying vulnerable sub-groups within regions, our model can guide the implementation of tailored interventions that facilitate early identification and referral to medical treatment.
Collapse
Affiliation(s)
- Hamed Khosravi
- Industrial and Management Systems Engineering, West Virginia University, Morgantown, WV 26506, USA
| | - Imtiaz Ahmed
- Industrial and Management Systems Engineering, West Virginia University, Morgantown, WV 26506, USA
| | - Avishek Choudhury
- Industrial and Management Systems Engineering, West Virginia University, Morgantown, WV 26506, USA
| |
Collapse
|
2
|
Li CY, Xiao Y, Chen TT, Zhu SY. Epidemiological characteristics and behaviors of online broadcast suicidality in China: implications for targeted prevention strategies. Front Public Health 2024; 12:1396460. [PMID: 38774054 PMCID: PMC11106420 DOI: 10.3389/fpubh.2024.1396460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Accepted: 04/19/2024] [Indexed: 05/24/2024] Open
Abstract
Objectives Suicide is a global health concern, exacerbated by stigma around mental illnesses. Online platforms like Twitter and Sina Weibo have seen a rise in "online broadcast suicide," where individuals share suicidal thoughts and actions. However, there is limited understanding of the epidemiological characteristics, particularly in China. This study aims to analyze the demographics and behaviors of individuals engaging in online broadcast suicide in China to inform targeted prevention strategies. Methods A total of 525 incidents were identified through systematic retrieval of relevant news reports from online sources. Subsequently, a content analysis was performed on these reports to extract detailed information on the characteristics of each individual incident. Results Among the incidents analyzed, the male-to-female ratio was 1:1.6, with an average age of 23.1 ± 5.9 years. Approximately 71.9% took place in Southern China. Unemployment was reported in 15.0% of incidents. Relationship breakup (62.3%) was cited as the leading cause of suicide. Wrist cutting (58.2%) emerged as the predominant suicide method, and home (36.2%) was the most common location for these tragic events. Instant messaging apps were the primary platforms (54.7%) for conveying suicidal thoughts and actions. Additionally, among the 525 incidents examined, 12.0% disclosed having a mental disorder, and 7.6% had a history of prior suicide attempts. Significant variations were observed across age, gender, region, and occupation categories. Conclusion This study emphasizes the importance of developing suicide prevention programs for internet users. Besides, interventions should be customized to meet the specific needs of various populations.
Collapse
Affiliation(s)
- Chun-ya Li
- Psychosomatic Medical Center, The Fourth People's Hospital of Chengdu, Chengdu, China
| | - Yu Xiao
- Psychosomatic Medical Center, The Fourth People's Hospital of Chengdu, Chengdu, China
- Psychosomatic Medical Center, The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Chengdu, China
| | - Ting-ting Chen
- Mental Health Center, West China Hospital of Sichuan University, Chengdu, China
| | - Shao-yi Zhu
- Department of Psychiatry, Shantou University Mental Health Center, Shantou, China
| |
Collapse
|
3
|
Hill RM, Hussain Z, Vieyra B, Gallagher A. Reporting Ethical Procedures in Suicide Prevention Research: Current Status and Recommendations. Arch Suicide Res 2023; 27:1373-1390. [PMID: 36415164 DOI: 10.1080/13811118.2022.2131493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE Ethical concerns frequently arise in suicide prevention research regarding participant safety and confidentiality. Despite a substantial literature on managing and navigating ethical concerns in suicide research, little attention has been paid to the reporting of ethical procedures. Furthermore, standard procedures for reporting ethical risk management procedures have not been developed. METHOD A review of the current literature was performed to examine the current state of reporting of ethical procedures within suicide research. Articles published in 2020 (N = 263) from three suicide-focused publications were screened and then coded (n = 131) to identify reporting of procedures for the ethical conduct of research and suicide risk management steps taken by the research teams. RESULTS The majority of articles reported ethical review or approval (84.7%) and reported the use of an informed consent process (77.9%). Only 28.2% included risk mitigation procedures. Of those 29.7% of those articles reported conducting risk evaluation, 66.7% reported resource dissemination, and 51.4% reported an intervention. CONCLUSION As empirical support for brief interventions accrues, suicide prevention researchers should consider establishing standards for the reporting of procedures to ensure the safety of participants with suicidal risk.HighlightsReporting suicide safety protocols helps ensure high ethical standards in research.Fewer than 1/3 of articles reviewed reported risk mitigation procedures in 2020.Standard procedures for reporting safety protocols in suicide research are needed.
Collapse
|
4
|
Dobbs MF, McGowan A, Selloni A, Bilgrami Z, Sarac C, Cotter M, Herrera SN, Cecchi GA, Goodman M, Corcoran CM, Srivastava A. Linguistic correlates of suicidal ideation in youth at clinical high-risk for psychosis. Schizophr Res 2023; 259:20-27. [PMID: 36933977 PMCID: PMC10504409 DOI: 10.1016/j.schres.2023.03.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 03/04/2023] [Accepted: 03/05/2023] [Indexed: 03/20/2023]
Abstract
Suicidal ideation (SI) is prevalent among individuals at clinical high-risk for psychosis (CHR). Natural language processing (NLP) provides an efficient method to identify linguistic markers of suicidality. Prior work has demonstrated that an increased use of "I", as well as words with semantic similarity to "anger", "sadness", "stress" and "lonely", are correlated with SI in other cohorts. The current project analyzes data collected in an SI supplement to an NIH R01 study of thought disorder and social cognition in CHR. This study is the first to use NLP analyses of spoken language to identify linguistic correlates of recent suicidal ideation among CHR individuals. The sample included 43 CHR individuals, 10 with recent suicidal ideation and 33 without, as measured by the Columbia-Suicide Severity Rating Scale, as well as 14 healthy volunteers without SI. NLP methods include part-of-speech (POS) tagging, a GoEmotions-trained BERT Model, and Zero-Shot Learning. As hypothesized, individuals at CHR for psychosis who endorsed recent SI utilized more words with semantic similarity to "anger" compared to those who did not. Words with semantic similarity to "stress", "loneliness", and "sadness" were not significantly different between the two CHR groups. Contrary to our hypotheses, CHR individuals with recent SI did not use the word "I" more than those without recent SI. As anger is not characteristic of CHR, findings have implications for the consideration of subthreshold anger-related sentiment in suicidal risk assessment. As NLP is scalable, findings suggest that language markers may improve suicide screening and prediction in this population.
Collapse
Affiliation(s)
- Matthew F Dobbs
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA.
| | - Alessia McGowan
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA.
| | - Alexandria Selloni
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA.
| | - Zarina Bilgrami
- Department of Psychology, Emory University, 201 Dowman Dr, Atlanta, GA 3032, USA.
| | - Cansu Sarac
- Department of Psychology, Long Island University-Brooklyn, 1 University Plaza, Brooklyn, NY 11201, USA.
| | - Matthew Cotter
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA.
| | - Shaynna N Herrera
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA.
| | - Guillermo A Cecchi
- IBM T. J. Watson Research Center, 1101 Kitchawan Rd, Yorktown Heights, NY 10598, USA.
| | - Marianne Goodman
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA; VISN 2 Mental Illness Research, Education, and Clinical Center (MIRECC), James J. Peters Veterans Affairs Medical Center, 130 W Kingsbridge Rd, The Bronx, NY 10468, USA.
| | - Cheryl M Corcoran
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA; VISN 2 Mental Illness Research, Education, and Clinical Center (MIRECC), James J. Peters Veterans Affairs Medical Center, 130 W Kingsbridge Rd, The Bronx, NY 10468, USA.
| | - Agrima Srivastava
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Pl, New York, NY 10029, USA.
| |
Collapse
|
5
|
Warner EN, Ammerman RT, Glauser TA, Pestian JP, Agasthya G, Strawn JR. Developmental Epidemiology of Pediatric Anxiety Disorders. Child Adolesc Psychiatr Clin N Am 2023; 32:511-530. [PMID: 37201964 DOI: 10.1016/j.chc.2023.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
This review summarizes the developmental epidemiology of childhood and adolescent anxiety disorders. It discusses the coronavirus disease of 2019 (COVID-19) pandemic, sex differences, longitudinal course, and stability of anxiety disorders in addition to recurrence and remission. The trajectory of anxiety disorders-whether homotypic (ie, the same anxiety disorder persists over time) or heterotypic (ie, an anxiety disorder shifts to a different diagnosis over time) is discussed with regard to social, generalized, and separation anxiety disorders as well as specific phobia, and panic disorder. Finally, strategies for early recognition, prevention, and treatment of disorders are discussed.
Collapse
Affiliation(s)
- Emily N Warner
- University of Cincinnati College of Medicine, Cincinnati, OH, USA; Department of Environmental and Public Health Sciences, University of Cincinnati.
| | - Robert T Ammerman
- Division of Behavioral Medicine and Clinical Psychology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Tracy A Glauser
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - John P Pestian
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Greeshma Agasthya
- Oak Ridge National Laboratory, Computational Sciences and Engineering Division, Advanced Computing for Health Sciences Section
| | - Jeffrey R Strawn
- University of Cincinnati College of Medicine, Cincinnati, OH, USA; Division of Psychiatry, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA; Division of Clinical Pharmacology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| |
Collapse
|
6
|
Min S, Shin D, Rhee SJ, Park CHK, Yang JH, Song Y, Kim MJ, Kim K, Cho WI, Kwon OC, Ahn YM, Lee H. Acoustic Analysis of Speech for Screening for Suicide Risk: Machine Learning Classifiers for Between- and Within-Person Evaluation of Suicidality. J Med Internet Res 2023; 25:e45456. [PMID: 36951913 PMCID: PMC10131783 DOI: 10.2196/45456] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2023] [Revised: 02/21/2023] [Accepted: 02/26/2023] [Indexed: 03/01/2023] Open
Abstract
BACKGROUND Assessing a patient's suicide risk is challenging for health professionals because it depends on voluntary disclosure by the patient and often has limited resources. The application of novel machine learning approaches to determine suicide risk has clinical utility. OBJECTIVE This study aimed to investigate cross-sectional and longitudinal approaches to assess suicidality based on acoustic voice features of psychiatric patients using artificial intelligence. METHODS We collected 348 voice recordings during clinical interviews of 104 patients diagnosed with mood disorders at baseline and 2, 4, 8, and 12 months after recruitment. Suicidality was assessed using the Beck Scale for Suicidal Ideation and suicidal behavior using the Columbia Suicide Severity Rating Scale. The acoustic features of the voice, including temporal, formal, and spectral features, were extracted from the recordings. A between-person classification model that examines the vocal characteristics of individuals cross sectionally to detect individuals at high risk for suicide and a within-person classification model that detects considerable worsening of suicidality based on changes in acoustic features within an individual were developed and compared. Internal validation was performed using 10-fold cross validation of audio data from baseline to 2-month and external validation was performed using data from 2 to 4 months. RESULTS A combined set of 12 acoustic features and 3 demographic variables (age, sex, and past suicide attempts) were included in the single-layer artificial neural network for the between-person classification model. Furthermore, 13 acoustic features were included in the extreme gradient boosting machine learning algorithm for the within-person model. The between-person classifier was able to detect high suicidality with 69% accuracy (sensitivity 74%, specificity 62%, area under the receiver operating characteristic curve 0.62), whereas the within-person model was able to predict worsening suicidality over 2 months with 79% accuracy (sensitivity 68%, specificity 84%, area under receiver operating characteristic curve 0.67). The second model showed 62% accuracy in predicting increased suicidality in external sets. CONCLUSIONS Within-person analysis using changes in acoustic features within an individual is a promising approach to detect increased suicidality. Automated analysis of voice can be used to support the real-time assessment of suicide risk in primary care or telemedicine.
Collapse
Affiliation(s)
- Sooyeon Min
- Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
- Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Daun Shin
- Department of Psychiatry, Korea University College of Medicine, Seoul, Republic of Korea
| | - Sang Jin Rhee
- Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
| | - C Hyung Keun Park
- Department of Psychiatry, Asan Medical Center, Seoul, Republic of Korea
| | - Jeong Hun Yang
- Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
| | - Yoojin Song
- Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
| | - Min Ji Kim
- Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Kyungdo Kim
- Department of Biomedical Engineering, Duke University, Durham, NC, United States
| | - Won Ik Cho
- Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea
| | | | - Yong Min Ahn
- Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
- Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Hyunju Lee
- Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
| |
Collapse
|
7
|
Gauld C, Pignon B, Fourneret P, Dubertret C, Tebeka S. Comparison of relative areas of interest between major depression disorder and postpartum depression. Prog Neuropsychopharmacol Biol Psychiatry 2023; 121:110671. [PMID: 36341842 DOI: 10.1016/j.pnpbp.2022.110671] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 10/11/2022] [Accepted: 10/26/2022] [Indexed: 11/06/2022]
Abstract
INTRODUCTION Postpartum depression (PPD) is defined as a major depressive disorder (MDD) beginning after childbirth. Wide debates aim to better understand PPD's specificities compared with MDD. One of the keys in differentiating PPD from MDD is to systematically study scientific "Areas Of Interest" (AOIs) of these disorders. METHODS In November 2021, we performed an extraction and textual computational analysis of associated terms for PPD and MDD, using the biomedical database PubMed. We performed an undirected lexical network analysis to map the 150 first terms in space. Then, we used an unsupervised machine learning technique to detect word patterns and automatically cluster AOIs with a topic-modeling analysis. RESULTS We identified 30,000 articles of the 554,724 articles for MDD and 15,642 articles for PPD. Four AOIs were detected in the MDD network: mood disorders and their treatments, risk factors, consequences and quality of life, and mental health and comorbidities. Five AOIs were detected in the PPD network: mood disorders and treatments, risk factors, consequences and child health, patient's background, and the challenges of screening. DISCUSSION AND CONCLUSION Limitations are both methodological, in particular due to the qualitative interpretation of AOIs, and are also related to the difficult transferability of these research results to the clinical practice. The partial overlap between AOIs for MDD and for PPD suggest that the latter is a particular form of the former.
Collapse
Affiliation(s)
- Christophe Gauld
- Department of Psychopathology of Child and Adolescent Development, Hospices Civils de Lyon, Lyon 1, France; UMR CNRS 8590 IHPST, Sorbonne University, Paris 1, France.
| | - Baptiste Pignon
- Univ Paris-Est-Créteil (UPEC), AP-HP, Hôpitaux Universitaires « H. Mondor », France; DMU IMPACT, INSERM, IMRB, Translational Neuropsychiatry, Fondation FondaMental, F-94010 Creteil, France
| | - Pierre Fourneret
- Department of Psychopathology of Child and Adolescent Development, Hospices Civils de Lyon, Lyon 1, France; Marc Jeannerod Institute of Cognitive Sciences UMR 5229, CNRS & Claude Bernard University, Lyon 1, France
| | - Caroline Dubertret
- Université de Paris, INSERM UMR1266, Institute of Psychiatry and Neurosciences, Team 1, Paris, France; Department of Psychiatry, AP-HP, Louis Mourier Hospital, F-92700 Colombes, France
| | - Sarah Tebeka
- Université de Paris, INSERM UMR1266, Institute of Psychiatry and Neurosciences, Team 1, Paris, France; Department of Psychiatry, AP-HP, Louis Mourier Hospital, F-92700 Colombes, France
| |
Collapse
|
8
|
Psycholinguistic changes in the communication of adolescent users in a suicidal ideation online community during the COVID-19 pandemic. Eur Child Adolesc Psychiatry 2022; 32:975-985. [PMID: 36018514 PMCID: PMC9415261 DOI: 10.1007/s00787-022-02067-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Accepted: 08/05/2022] [Indexed: 11/03/2022]
Abstract
Since the outbreak of the COVID-19 pandemic, increases in suicidal ideation and suicide attempts in adolescents have been registered. Many adolescents experiencing suicidal ideation turn to online communities for social support. In this retrospective observational study, we investigated the communication-language style, contents and user activity-in 7975 unique posts and 51,119 comments by N = 2862 active adolescent users in a large suicidal ideation support community (SISC) on the social media website reddit.com in the onset period of the COVID-19 pandemic. We found significant relative changes in language style markers for hopelessness such as negative emotion words (+ 10.00%) and positive emotion words (- 3.45%) as well as for social disengagement such as social references (- 8.63%) and 2nd person pronouns (- 33.97%) since the outbreak of the pandemic. Using topic modeling with Latent Dirichlet Allocation (LDA), we identified significant changes in content for the topics Hopelessness (+ 23.98%), Suicide Methods (+ 17.11%), Social Support (- 14.91%), and Reaching Out to users (- 28.97%). Changes in user activity point to an increased expression of mental health issues and decreased engagement with other users. The results indicate a potential shift in communication patterns with more adolescent users expressing their suicidal ideation rather than relating with or supporting other users during the COVID-19 pandemic.
Collapse
|
9
|
Homan S, Gabi M, Klee N, Bachmann S, Moser AM, Duri' M, Michel S, Bertram AM, Maatz A, Seiler G, Stark E, Kleim B. Linguistic features of suicidal thoughts and behaviors: A systematic review. Clin Psychol Rev 2022; 95:102161. [DOI: 10.1016/j.cpr.2022.102161] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2021] [Revised: 03/28/2022] [Accepted: 04/27/2022] [Indexed: 12/13/2022]
|
10
|
Machine learning for suicidal ideation identification: A systematic literature review. COMPUTERS IN HUMAN BEHAVIOR 2022. [DOI: 10.1016/j.chb.2021.107095] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
11
|
Cohen J, Wright-Berryman J, Rohlfs L, Trocinski D, Daniel L, Klatt TW. Integration and Validation of a Natural Language Processing Machine Learning Suicide Risk Prediction Model Based on Open-Ended Interview Language in the Emergency Department. Front Digit Health 2022; 4:818705. [PMID: 35187527 PMCID: PMC8847784 DOI: 10.3389/fdgth.2022.818705] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 01/10/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Emergency departments (ED) are an important intercept point for identifying suicide risk and connecting patients to care, however, more innovative, person-centered screening tools are needed. Natural language processing (NLP) -based machine learning (ML) techniques have shown promise to assess suicide risk, although whether NLP models perform well in differing geographic regions, at different time periods, or after large-scale events such as the COVID-19 pandemic is unknown. OBJECTIVE To evaluate the performance of an NLP/ML suicide risk prediction model on newly collected language from the Southeastern United States using models previously tested on language collected in the Midwestern US. METHOD 37 Suicidal and 33 non-suicidal patients from two EDs were interviewed to test a previously developed suicide risk prediction NLP/ML model. Model performance was evaluated with the area under the receiver operating characteristic curve (AUC) and Brier scores. RESULTS NLP/ML models performed with an AUC of 0.81 (95% CI: 0.71-0.91) and Brier score of 0.23. CONCLUSION The language-based suicide risk model performed with good discrimination when identifying the language of suicidal patients from a different part of the US and at a later time period than when the model was originally developed and trained.
Collapse
Affiliation(s)
| | - Jennifer Wright-Berryman
- Department of Social Work, College of Allied Health Sciences, University of Cincinnati, Cincinnati, OH, United States
| | | | | | | | | |
Collapse
|
12
|
Cohen J, Wright-Berryman J, Rohlfs L, Wright D, Campbell M, Gingrich D, Santel D, Pestian J. A Feasibility Study Using a Machine Learning Suicide Risk Prediction Model Based on Open-Ended Interview Language in Adolescent Therapy Sessions. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17218187. [PMID: 33167554 PMCID: PMC7663991 DOI: 10.3390/ijerph17218187] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 10/30/2020] [Accepted: 11/02/2020] [Indexed: 12/21/2022]
Abstract
Background: As adolescent suicide rates continue to rise, innovation in risk identification is warranted. Machine learning can identify suicidal individuals based on their language samples. This feasibility pilot was conducted to explore this technology’s use in adolescent therapy sessions and assess machine learning model performance. Method: Natural language processing machine learning models to identify level of suicide risk using a smartphone app were tested in outpatient therapy sessions. Data collection included language samples, depression and suicidality standardized scale scores, and therapist impression of the client’s mental state. Previously developed models were used to predict suicidal risk. Results: 267 interviews were collected from 60 students in eight schools by ten therapists, with 29 students indicating suicide or self-harm risk. During external validation, models were trained on suicidal speech samples collected from two separate studies. We found that support vector machines (AUC: 0.75; 95% CI: 0.69–0.81) and logistic regression (AUC: 0.76; 95% CI: 0.70–0.82) lead to good discriminative ability, with an extreme gradient boosting model performing the best (AUC: 0.78; 95% CI: 0.72–0.84). Conclusion: Voice collection technology and associated procedures can be integrated into mental health therapists’ workflow. Collected language samples could be classified with good discrimination using machine learning methods.
Collapse
Affiliation(s)
- Joshua Cohen
- Clarigent Health, 5412 Courseview Drive, Suite 210, Mason, OH 45040, USA; (L.R.); (D.W.); (M.C.)
- Correspondence:
| | - Jennifer Wright-Berryman
- Department of Social Work, College of Allied Health Sciences, University of Cincinnati, Cincinnati, OH 45221, USA;
| | - Lesley Rohlfs
- Clarigent Health, 5412 Courseview Drive, Suite 210, Mason, OH 45040, USA; (L.R.); (D.W.); (M.C.)
| | - Donald Wright
- Clarigent Health, 5412 Courseview Drive, Suite 210, Mason, OH 45040, USA; (L.R.); (D.W.); (M.C.)
| | - Marci Campbell
- Clarigent Health, 5412 Courseview Drive, Suite 210, Mason, OH 45040, USA; (L.R.); (D.W.); (M.C.)
| | - Debbie Gingrich
- The Children’s Home, 5050 Madison Road, Cincinnati, OH 45227, USA;
| | - Daniel Santel
- Department of Pediatrics, Division of Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229, USA; (D.S.); (J.P.)
| | - John Pestian
- Department of Pediatrics, Division of Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229, USA; (D.S.); (J.P.)
| |
Collapse
|