1
|
Mitra A, Chen K, Liu W, Kessler RC, Yu H. Predicting Suicide Among US Veterans Using Natural Language Processing-enriched Social and Behavioral Determinants of Health. RESEARCH SQUARE 2024:rs.3.rs-4290732. [PMID: 38746180 PMCID: PMC11092830 DOI: 10.21203/rs.3.rs-4290732/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
Despite recognizing the critical association between social and behavioral determinants of health (SBDH) and suicide risk, SBDHs from unstructured electronic health record (EHR) notes for suicide predictive modeling remain underutilized. This study investigates the impact of SBDH, identified from both structured and unstructured data utilizing a natural language processing (NLP) system, on suicide prediction within 7, 30, 90, and 180 days of discharge. Using EHR data of 2,987,006 Veterans between October 1, 2009, and September 30, 2015, from the US Veterans Health Administration (VHA), we designed a case-control study that demonstrates that incorporating structured and NLP-extracted SBDH significantly enhances the performance of three architecturally distinct suicide predictive models - elastic-net logistic regression, random forest (RF), and multilayer perceptron. For example, RF achieved notable improvements in suicide prediction within 180 days of discharge, with an increase in the area under the receiver operating characteristic curve from 83.57-84.25% (95% CI = 0.63%-0.98%, p-val < 0.001) and the area under the precision recall curve from 57.38-59.87% (95% CI = 3.86%-4.82%, p-val < 0.001) after integrating NLP-extracted SBDH. These findings underscore the potential of NLP-extracted SBDH in enhancing suicide prediction across various prediction timeframes, offering valuable insights for healthcare practitioners and policymakers.
Collapse
Affiliation(s)
| | | | | | | | - Hong Yu
- University of Massachusetts Amherst
| |
Collapse
|
2
|
Dhaubhadel S, Ganguly K, Ribeiro RM, Cohn JD, Hyman JM, Hengartner NW, Kolade B, Singley A, Bhattacharya T, Finley P, Levin D, Thelen H, Cho K, Costa L, Ho YL, Justice AC, Pestian J, Santel D, Zamora-Resendiz R, Crivelli S, Tamang S, Martins S, Trafton J, Oslin DW, Beckham JC, Kimbrel NA, McMahon BH. High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning. Sci Rep 2024; 14:1793. [PMID: 38245528 PMCID: PMC10799879 DOI: 10.1038/s41598-024-51762-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Accepted: 01/09/2024] [Indexed: 01/22/2024] Open
Abstract
We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of [Formula: see text] 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.
Collapse
Affiliation(s)
| | - Kumkum Ganguly
- Los Alamos National Laboratory, Los Alamos, NM, 87545, USA
| | - Ruy M Ribeiro
- Los Alamos National Laboratory, Los Alamos, NM, 87545, USA
| | - Judith D Cohn
- Los Alamos National Laboratory, Los Alamos, NM, 87545, USA
| | - James M Hyman
- Los Alamos National Laboratory, Los Alamos, NM, 87545, USA
| | | | - Beauty Kolade
- Los Alamos National Laboratory, Los Alamos, NM, 87545, USA
| | - Anna Singley
- Los Alamos National Laboratory, Los Alamos, NM, 87545, USA
| | | | | | - Drew Levin
- Sandia National Laboratory, Albuquerque, NM, 87123, USA
| | - Haedi Thelen
- Sandia National Laboratory, Albuquerque, NM, 87123, USA
| | - Kelly Cho
- Massachusetts Veterans Epidemiology and Research Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
- Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, USA
| | - Lauren Costa
- Massachusetts Veterans Epidemiology and Research Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | - Yuk-Lam Ho
- Massachusetts Veterans Epidemiology and Research Information Center (MAVERIC), VA Boston Healthcare System, Boston, USA
| | - Amy C Justice
- VA Connecticut Healthcare System, Yale Schools of Medicine and Public Health, Yale University, West Haven, CT, USA
| | - John Pestian
- Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
| | - Daniel Santel
- Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
| | - Rafael Zamora-Resendiz
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, 1 Cyclotron Rd, Berkeley, CA, 94720, USA
| | - Silvia Crivelli
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, 1 Cyclotron Rd, Berkeley, CA, 94720, USA
| | - Suzanne Tamang
- Program Evaluation and Resource Center, Office of Mental Health and Suicide Prevention, Veterans Affairs Palo Alto Health Care System, Menlo Park, CA, USA
- Department of Medicine, Stanford University, Stanford, California, USA
| | - Susana Martins
- Program Evaluation and Resource Center, Office of Mental Health and Suicide Prevention, Veterans Affairs Palo Alto Health Care System, Menlo Park, CA, USA
| | - Jodie Trafton
- Program Evaluation and Resource Center, Office of Mental Health and Suicide Prevention, Veterans Affairs Palo Alto Health Care System, Menlo Park, CA, USA
| | - David W Oslin
- Cpl Michael J Crescenz VA Medical Center, VISN 4 Mental Illness Research, Education, and Clinical Center; Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 3535 Market Street, Philadelphia, PA, 19104, USA
| | - Jean C Beckham
- Durham Veterans Affairs (VA) Health Care System, Durham, NC, USA
- VA Mid-Atlantic Mental Illness Research, Education and Clinical Center, Durham, NC, USA
- Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, USA
| | - Nathan A Kimbrel
- Durham Veterans Affairs (VA) Health Care System, Durham, NC, USA
- VA Mid-Atlantic Mental Illness Research, Education and Clinical Center, Durham, NC, USA
- VA Health Services Research and Development Center of Innovation to Accelerate Discovery and Practice Transformation, Durham, NC, USA
- Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, USA
| | | |
Collapse
|