1
|
Ripperger MA, Kolli J, Wilimitis D, Robinson K, Reale C, Novak LL, Cunningham CA, Kasuske LM, Grover SG, Ribeiro JD, Walsh CG. External Validation and Updating of a Statistical Civilian-Based Suicide Risk Model in US Naval Primary Care. JAMA Netw Open 2023; 6:e2342750. [PMID: 37938841 PMCID: PMC10632956 DOI: 10.1001/jamanetworkopen.2023.42750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Accepted: 09/29/2023] [Indexed: 11/10/2023] Open
Abstract
Importance Suicide remains an ongoing concern in the US military. Statistical models have not been broadly disseminated for US Navy service members. Objective To externally validate and update a statistical suicide risk model initially developed in a civilian setting with an emphasis on primary care. Design, Setting, and Participants This retrospective cohort study used data collected from 2007 through 2017 among active-duty US Navy service members. The external civilian model was applied to every visit at Naval Medical Center Portsmouth (NMCP), its NMCP Naval Branch Health Clinics (NBHCs), and TRICARE Prime Clinics (TPCs) that fall within the NMCP area. The model was retrained and recalibrated using visits to NBHCs and TPCs and updated using Department of Defense (DoD)-specific billing codes and demographic characteristics, including expanded race and ethnicity categories. Domain and temporal analyses were performed with bootstrap validation. Data analysis was performed from September 2020 to December 2022. Exposure Visit to US NMCP. Main Outcomes and Measures Recorded suicidal behavior on the day of or within 30 days of a visit. Performance was assessed using area under the receiver operating curve (AUROC), area under the precision recall curve (AUPRC), Brier score, and Spiegelhalter z-test statistic. Results Of the 260 583 service members, 6529 (2.5%) had a recorded suicidal behavior, 206 412 (79.2%) were male; 104 835 (40.2%) were aged 20 to 24 years; and 9458 (3.6%) were Asian, 56 715 (21.8%) were Black or African American, and 158 277 (60.7%) were White. Applying the civilian-trained model resulted in an AUROC of 0.77 (95% CI, 0.74-0.79) and an AUPRC of 0.004 (95% CI, 0.003-0.005) at NBHCs with poor calibration (Spiegelhalter P < .001). Retraining the algorithm improved AUROC to 0.92 (95% CI, 0.91-0.93) and AUPRC to 0.66 (95% CI, 0.63-0.68). Number needed to screen in the top risk tiers was 366 for the external model and 200 for the retrained model; the lower number indicates better performance. Domain validation showed AUROC of 0.90 (95% CI, 0.90-0.91) and AUPRC of 0.01 (95% CI, 0.01-0.01), and temporal validation showed AUROC of 0.75 (95% CI, 0.72-0.78) and AUPRC of 0.003 (95% CI, 0.003-0.005). Conclusions and Relevance In this cohort study of active-duty Navy service members, a civilian suicide attempt risk model was externally validated. Retraining and updating with DoD-specific variables improved performance. Domain and temporal validation results were similar to external validation, suggesting that implementing an external model in US Navy primary care clinics may bypass the need for costly internal development and expedite the automation of suicide prevention in these clinics.
Collapse
Affiliation(s)
- Michael A. Ripperger
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Jhansi Kolli
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Drew Wilimitis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Katelyn Robinson
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Carrie Reale
- Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Laurie L. Novak
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | | | - Lalon M. Kasuske
- Daniel K. Inouye Graduate School of Nursing, Uniformed Services University of the Health Sciences, Bethesda, Maryland
| | | | | | - Colin G. Walsh
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Psychiatry, Vanderbilt University Medical Center, Nashville, Tennessee
| |
Collapse
|
2
|
Edwards ER, Coolidge B, Ruiz D, Epshteyn G, Krauss A, Gorman D, Connelly B, Redden C, El-Meouchy P, Geraci J. Situational stress and suicide attempt behavior in Army soldiers and veterans: Insights from the Army Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study. Suicide Life Threat Behav 2023; 53:642-654. [PMID: 37306332 DOI: 10.1111/sltb.12970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 04/07/2023] [Accepted: 05/07/2023] [Indexed: 06/13/2023]
Abstract
INTRODUCTION Prevention of suicide-related behaviors is considered a top clinical priority within the Departments of Veterans Affairs and Defense. Despite previous literature attesting to the likely importance of situational stress as a key correlate of acute changes in suicide risk, longitudinal research into associations between situational stress and suicide-related outcomes among military personnel has been relatively limited. METHODS The current study examined associations between situational stress, recent suicide attempt, and future suicide attempt using data from 14,508 Army soldiers and recently discharged veterans enrolled in the Army Study to Assess Risk and Resilience in Servicemembers-Longitudinal Studies (STARRS-LS). RESULTS Recent situational stress was more common among recently discharged veterans (vs. soldiers), those with a recent suicide attempt (vs. those without), and those with a subsequent suicide attempt (vs. those without). Job loss was more closely associated with suicide attempts among soldiers, whereas financial crisis, police contact, and death, illness, or injury of close others were more closely associated with suicide attempts among recently discharged veterans. CONCLUSION Findings further highlight situational stress as a salient risk factor for suicide-related outcomes among military personnel, particularly among recently discharged veterans. Implications for screening and treatment of at-risk military personnel are discussed.
Collapse
Affiliation(s)
- Emily R Edwards
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut, USA
| | - Brettland Coolidge
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
- Department of Psychology, University of Central Florida, Orlando, Florida, USA
| | - Danny Ruiz
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
| | - Gabriella Epshteyn
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
- Department of Psychology, University of Rhode Island, Kingston, Rhode Island, USA
| | - Alison Krauss
- VA VISN 17 Center of Excellence for Research on Returning War Veterans, Central Texas Veterans Health Care System, Waco, Texas, USA
| | - Daniel Gorman
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
| | - Brigid Connelly
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
- Department of Counseling & Clinical Psychology, Teachers College, Columbia University, New York, New York, USA
| | - Clare Redden
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
- Department of Counseling & Clinical Psychology, Teachers College, Columbia University, New York, New York, USA
| | - Paul El-Meouchy
- Department of Psychology, Fielding Graduate University, Santa Barbara, California, USA
| | - Joseph Geraci
- VISN 2 Mental Illness, Research, Education, and Clinical Center (MIRECC), Department of Veterans Affairs, Bronx, New York, USA
- Department of Counseling & Clinical Psychology, Teachers College, Columbia University, New York, New York, USA
| |
Collapse
|
3
|
Geraci JC, Dichiara A, Greene A, Gromatsky M, Finley E, Kilby D, Frankfurt S, Edwards ER, Kurz AS, Sokol Y, Sullivan SR, Mobbs M, Seim RW, Goodman M. Supporting servicemembers and veterans during their transition to civilian life using certified sponsors: A three-arm randomized controlled trial. Psychol Serv 2023; 20:248-259. [PMID: 37384439 PMCID: PMC10755060 DOI: 10.1037/ser0000764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/01/2023]
Abstract
Transitioning servicemembers and veterans (TSMVs) face difficulties throughout their reintegration to civilian life, including challenges with employment, poor social connection, and elevated risk for suicide. To meet the needs of this high-risk population, national initiatives have leveraged community-based interventions. Authors conducted a three-arm randomized controlled trial (n = 200) to evaluate two community-based interventions. The first, Team Red, White, and Blue (RWB), connects TSMVs to their community through physical/social activities. The second, Expiration Term of Service Sponsorship Program (ETS-SP) provides one-on-one certified sponsors to TSMVs who provide support during the reintegration process. TSMVs were assessed at baseline, 3, 6, and 12 months. The primary hypothesis was not supported as reintegration difficulties and social support were not significantly different for participants randomly assigned to the two community-based interventions (Arm-2/RWB and Arm-3/RWB + ETS-SP), when the data from the separate arms were collapsed and combined, compared to the waitlist. The results did support the secondary hypothesis as Arm-3/RWB + ETS-SP had less reintegration difficulties over 12 months and initially had more social support compared to Arm-2/RWB, which suggest that augmenting interventions with sponsors outperforms participation in community-based interventions alone. Overall, the results show some limitations of the studied community-based interventions, as implemented and researched within this study. The authors identified factors that may have contributed to the null findings for the primary hypothesis, which can be addressed in future studies, such as addressing the unique needs of TSMVs, enrolling TSMVs into interventions prior to military discharge, measuring and improving participation levels, and providing stepped-care interventions based on risk levels. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Collapse
Affiliation(s)
- Joseph C. Geraci
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX
- Resilience Center for Veterans & Families, Teachers College, Columbia University, NY, NY
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Ariana Dichiara
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
| | - Ashley Greene
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Molly Gromatsky
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Erin Finley
- Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX
- Center for the Study of Healthcare Innovation, Implementation, and Policy (CSHIIP), VA Greater Los Angeles Healthcare System
| | - Daniel Kilby
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Resilience Center for Veterans & Families, Teachers College, Columbia University, NY, NY
| | - Sheila Frankfurt
- Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX
- Central Texas Veterans Healthcare System, Temple, TX
| | - Emily R. Edwards
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Department of Psychiatry, Yale School of Medicine, New Haven, CT
| | - A. Solomon Kurz
- Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX
| | - Yosef Sokol
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Sarah R. Sullivan
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
| | - Meaghan Mobbs
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
| | - Richard W. Seim
- Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX
| | - Marianne Goodman
- Transitioning Servicemember/Veteran And Suicide Prevention Center (TASC), VISN 2 Mental Illness Research, Education and Clinical Center, James J. Peters VA Medical Center, Bronx, NY
- Center of Excellence for Research on Returning War Veterans, VISN 17, Doris Miller VA Medical Center, Waco, TX
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY
| |
Collapse
|
4
|
Dempsey CL, Benedek DM, Zuromski KL, Nock MK, Brent DA, Ao J, Georg MW, Haller K, Aliaga PA, Heeringa SG, Kessler RC, Stein MB, Ursano RJ. Recent Stressful Experiences and Suicide Risk: Implications for Suicide Prevention and Intervention in U.S. Army Soldiers. PSYCHIATRIC RESEARCH AND CLINICAL PRACTICE 2023; 5:24-36. [PMID: 36909141 PMCID: PMC9997076 DOI: 10.1176/appi.prcp.20220027] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 11/01/2022] [Indexed: 01/15/2023] Open
Abstract
Objectives To identify the extent to which the presence of recent stressful events are risk factors for suicide among active-duty soldiers as reported by informants. Methods Next-of-kin (NOK) and supervisors (SUP) of active duty soldiers (n = 135) who died by suicide and two groups of living controls: propensity-matched (n = 128) and soldiers who reported suicidal ideation in the past year, but did not die (SI) (n = 108) provided data via structured interviews from the Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Multivariate logistic regression analyses were used to create a risk score for suicide. Results The odds of suicide increased significantly for soldiers experiencing relationship problems, military punishment, and perceived failure or humiliation in the month prior to death. Suicide risk models with these risk factors predicted suicide death among those who reported SI in the past year (OR = 5.9, [95% CI = 1.5, 24.0] χ 2 = 6.24, p = 0.0125, AUC, 0.73 (0.7, 0.8) NOK) and (OR = 8.6, [95% CI = 1.4, 51.5] χ 2 = 5.49, p = 0.0191, AUC, 0.78 (0.7, 0.8); SUP) suggesting the combination of these recent stressors may contribute to the transition from ideation to action. Conclusions Our findings suggest for the first time recent stressors distinguished suicide ideating controls from suicide decedents in the month prior to death as reported by informants. Implications for preventive intervention efforts for clinicians, supervisors and family members in identifying the transition from ideation to action are discussed.
Collapse
Affiliation(s)
- Catherine L. Dempsey
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
- Henry M. Jackson Foundation for the Advancement of Military Medicine, IncBethesdaMD
| | - David M. Benedek
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
| | | | | | - David A. Brent
- Department of PsychiatryUniversity of Pittsburgh School of MedicinePittsburghPA
| | - Jingning Ao
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
- Henry M. Jackson Foundation for the Advancement of Military Medicine, IncBethesdaMD
| | - Matthew W. Georg
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
- Henry M. Jackson Foundation for the Advancement of Military Medicine, IncBethesdaMD
| | - Katy Haller
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
- Henry M. Jackson Foundation for the Advancement of Military Medicine, IncBethesdaMD
| | - Pablo A. Aliaga
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
- Henry M. Jackson Foundation for the Advancement of Military Medicine, IncBethesdaMD
| | | | | | - Murray B. Stein
- Department of Psychiatry and Department of Family Medicine & Public HealthUniversity of California San DiegoLa JollaCA
- VA San Diego Healthcare SystemSan DiegoCA
| | - Robert J. Ursano
- Department of PsychiatryCenter for the Study of Traumatic StressUniformed Services University of the Health SciencesBethesdaMD
| |
Collapse
|