Predictive modeling of initiation and delayed mental health contact for depression

BACKGROUND

Depression is prevalent among Operation Enduring Freedom and Operation Iraqi Freedom (OEF/OIF) Veterans, yet rates of Veteran mental health care utilization remain modest. The current study examined: factors in electronic health records (EHR) associated with lack of treatment initiation and treatment delay; the accuracy of regression and machine learning models to predict initiation of treatment.

METHODS

We obtained data from the VA Corporate Data Warehouse (CDW). EHR data were extracted for 127,423 Veterans who deployed to Iraq/Afghanistan after 9/11 with a positive depression screen and a first depression diagnosis between 2001 and 2021. We also obtained 12-month pre-diagnosis and post-diagnosis patient data. Retrospective cohort analysis was employed to test if predictors can reliably differentiate patients who initiated, delayed, or received no mental health treatment associated with their depression diagnosis.

RESULTS

108,457 Veterans with depression, initiated depression-related care (55,492 Veterans delayed treatment beyond one month). Those who were male, without VA disability benefits, with a mild depression diagnosis, and had a history of psychotherapy were less likely to initiate treatment. Among those who initiated care, those with single and mild depression episodes at baseline, with either PTSD or who lacked comorbidities were more likely to delay treatment for depression. A history of mental health treatment, of an anxiety disorder, and a positive depression screen were each related to faster treatment initiation. Classification of patients was modest (ROC AUC = 0.59 95%CI = 0.586-0.602; machine learning F-measure = 0.46).

CONCLUSIONS

Having VA disability benefits was the strongest predictor of treatment initiation after a depression diagnosis and a history of mental health treatment was the strongest predictor of delayed initiation of treatment. The complexity of the relationship between VA benefits and history of mental health care with treatment initiation after a depression diagnosis is further discussed. Modest classification accuracy with currently known predictors suggests the need to identify additional predictors of successful depression management.

Mental health treatment utilization patterns among 108,457 Afghanistan and Iraq veterans with depression

People with depression often underutilize mental health care. This study was conceived as a first step toward a clinical decision support tool that helps identify patients who are at higher risk of underutilizing care. The primary goals were to (a) describe treatment utilization patterns, early termination, and return to care; (b) identify factors associated with early termination of treatment; and (c) evaluate the accuracy of regression models to predict early termination. These goals were evaluated in a retrospective cohort analysis of 108,457 U.S. veterans who received care from the Veterans Health Administration between 2001 and 2021. Our final sample was 16.5% female with an average age of 34.5. Veterans were included if they had a depression diagnosis, a positive depression screen, and received general health care services at least a year before and after their depression diagnosis. Using treatment quality guidelines, the threshold for treatment underutilization was defined as receiving fewer than four psychotherapy sessions or less than 84 days of antidepressants. Over one fifth of veterans (21.6%) received less than the minimally recommended care for depression. The odds of underutilizing treatment increased with lack of Veterans Administration benefits, male gender, racial/ethnic minority status, and having received mental health treatment in the past (adjusted OR > 1.1). Posttraumatic stress disorder comorbidity correlated with increased depression treatment utilization (adjusted OR < .9). Models with demographic and clinical information from medical records performed modestly in classifying patients who underutilized depression treatment (area under the curve = 0.595, 95% CI [0.588, 0.603]). Most veterans in this cohort received at least the minimum recommended treatment for depression. To improve the prediction of underutilization, patient factors associated with treatment underutilization likely need to be supplemented by additional clinical information. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Machine learning to develop a predictive model of pressure injury in persons with spinal cord injury

STUDY DESIGN

A 5-year longitudinal, retrospective, cohort study.

OBJECTIVES

Develop a prediction model based on electronic health record (EHR) data to identify veterans with spinal cord injury/diseases (SCI/D) at highest risk for new pressure injuries (PIs).

SETTING

Structured (coded) and text EHR data, for veterans with SCI/D treated in a VHA SCI/D Center between October 1, 2008, and September 30, 2013.

METHODS

A total of 4709 veterans were available for analysis after randomly selecting 175 to act as a validation (gold standard) sample. Machine learning models were created using ten-fold cross validation and three techniques: (1) two-step logistic regression; (2) regression model employing adaptive LASSO; (3) and gradient boosting. Models based on each method were compared using area under the receiver-operating curve (AUC) analysis.

RESULTS

The AUC value for the gradient boosting model was 0.62 (95% CI = 0.54-0.70), for the logistic regression model it was 0.67 (95% CI = 0.59-0.75), and for the adaptive LASSO model it was 0.72 (95% CI = 0.65-80). Based on these results, the adaptive LASSO model was chosen for interpretation. The strongest predictors of new PI cases were having fewer total days in the hospital in the year before the annual exam, higher vs. lower weight and most severe vs. less severe grade of injury based on the American Spinal Cord Injury Association (ASIA) Impairment Scale.

CONCLUSIONS

While the analyses resulted in a potentially useful predictive model, clinical implications were limited because modifiable risk factors were absent in the models.

Complementary and Integrative Health Approaches and Pain Care Quality in the Veterans Health Administration Primary Care Setting: A Quasi-Experimental Analysis

Background: Complementary and integrative health (CIH) approaches have been recommended in national and international clinical guidelines for chronic pain management. We set out to determine whether exposure to CIH approaches is associated with pain care quality (PCQ) in the Veterans Health Administration (VHA) primary care setting. Methods: We followed a cohort of 62,721 Veterans with newly diagnosed musculoskeletal disorders between October 2016 and September 2017 over 1-year. PCQ scores were derived from primary care progress notes using natural language processing. CIH exposure was defined as documentation of acupuncture, chiropractic or massage therapies by providers. Propensity scores (PSs) were used to match one control for each Veteran with CIH exposure. Generalized estimating equations were used to examine associations between CIH exposure and PCQ scores, accounting for potential selection and confounding bias. Results: CIH was documented for 14,114 (22.5%) Veterans over 16,015 primary care clinic visits during the follow-up period. The CIH exposure group and the 1:1 PS-matched control group achieved superior balance on all measured baseline covariates, with standardized differences ranging from 0.000 to 0.045. CIH exposure was associated with an adjusted rate ratio (aRR) of 1.147 (95% confidence interval [CI]: 1.142, 1.151) on PCQ total score (mean: 8.36). Sensitivity analyses using an alternative PCQ scoring algorithm (aRR: 1.155; 95% CI: 1.150-1.160) and redefining CIH exposure by chiropractic alone (aRR: 1.118; 95% CI: 1.110-1.126) derived consistent results. Discussion: Our data suggest that incorporating CIH approaches may reflect higher overall quality of care for patients with musculoskeletal pain seen in primary care settings, supporting VHA initiatives and the Declaration of Astana to build comprehensive, sustainable primary care capacity for pain management. Future investigation is warranted to better understand whether and to what degree the observed association may reflect the therapeutic benefits patients actually received or other factors such as empowering provider-patient education and communication about these approaches.

Mental Health Diagnoses are Not Associated With Indicators of Lower Quality Pain Care in Electronic Health Records of a National Sample of Veterans Treated in Veterans Health Administration Primary Care Settings

Prior research has demonstrated disparities in general medical care for patients with mental health conditions, but little is known about disparities in pain care. The objective of this retrospective cohort study was to determine whether mental health conditions are associated with indicators of pain care quality (PCQ) as documented by primary care clinicians in the Veterans Health Administration (VHA). We used natural language processing to analyze electronic health record data from a national sample of Veterans with moderate to severe musculoskeletal pain during primary care visits in the Fiscal Year 2017. Twelve PCQ indicators were annotated from clinician progress notes as present or absent; PCQ score was defined as the sum of these indicators. Generalized estimating equation Poisson models examined associations among mental health diagnosis categories and PCQ scores. The overall mean PCQ score across 135,408 person-visits was 8.4 (SD = 2.3). In the final adjusted model, post-traumatic stress disorder was associated with higher PCQ scores (RR = 1.006, 95%CI 1.002-1.010, P = .007). Depression, alcohol use disorder, other substance use disorder, schizophrenia, and bipolar disorder diagnoses were not associated with PCQ scores. Overall, results suggest that in this patient population, presence of a mental health condition is not associated with lower quality pain care. PERSPECTIVE: This study used a natural language processing approach to analyze medical records to determine whether mental health conditions are associated with indicators of pain care quality as documented by primary care clinicians. Findings suggest that presence of a diagnosed mental health condition is not associated with lower quality pain care.

Measuring pain care quality in the Veterans Health Administration primary care setting

ABSTRACT

The lack of a reliable approach to assess quality of pain care hinders quality improvement initiatives. Rule-based natural language processing algorithms were used to extract pain care quality (PCQ) indicators from documents of Veterans Health Administration primary care providers for veterans diagnosed within the past year with musculoskeletal disorders with moderate-to-severe pain intensity across 2 time periods 2013 to 2014 (fiscal year [FY] 2013) and 2017 to 2018 (FY 2017). Patterns of documentation of PCQ indicators for 64,444 veterans and 124,408 unique visits (FY 2013) and 63,427 veterans and 146,507 visits (FY 2017) are described. The most commonly documented PCQ indicators in each cohort were presence of pain, etiology or source, and site of pain (greater than 90% of progress notes), while least commonly documented were sensation, what makes pain better or worse, and pain's impact on function (documented in fewer than 50%). A PCQ indicator score (maximum = 12) was calculated for each visit in FY 2013 (mean = 7.8, SD = 1.9) and FY 2017 (mean = 8.3, SD = 2.3) by adding one point for every indicator documented. Standardized Cronbach alpha for total PCQ scores was 0.74 in the most recent data (FY 2017). The mean PCQ indicator scores across patient characteristics and types of healthcare facilities were highly stable. Estimates of the frequency of documentation of PCQ indicators have face validity and encourage further evaluation of the reliability, validity, and utility of the measure. A reliable measure of PCQ fills an important scientific knowledge and practice gap.

Leveraging Electronic Health Care Record Information to Measure Pressure Ulcer Risk in Veterans With Spinal Cord Injury: A Longitudinal Study Protocol

Background

Pressure ulcers (PrUs) are a frequent, serious, and costly complication for veterans with spinal cord injury (SCI). The health care team should periodically identify PrU risk, although there is no tool in the literature that has been found to be reliable, valid, and sensitive enough to assess risk in this vulnerable population.

Objective

The immediate goal is to develop a risk assessment model that validly estimates the probability of developing a PrU. The long-term goal is to assist veterans with SCI and their providers in preventing PrUs through an automated system of risk assessment integrated into the veteran’s electronic health record (EHR).

Methods

This 5-year longitudinal, retrospective, cohort study targets 12,344 veterans with SCI who were cared for in the Veterans Health Administration (VHA) in fiscal year (FY) 2009 and had no record of a PrU in the prior 12 months. Potential risk factors identified in the literature were reviewed by an expert panel that prioritized factors and determined if these were found in structured data or unstructured form in narrative clinical notes for FY 2009-2013. These data are from the VHA enterprise Corporate Data Warehouse that is derived from the EHR structured (ie, coded in database/table) or narrative (ie, text in clinical notes) data for FY 2009-2013.

Results

This study is ongoing and final results are expected in 2017. Thus far, the expert panel reviewed the initial list of risk factors extracted from the literature; the panel recommended additions and omissions and provided insights about the format in which the documentation of the risk factors might exist in the EHR. This list was then iteratively refined through review and discussed with individual experts in the field. The cohort for the study was then identified, and all structured, unstructured, and semistructured data were extracted. Annotation schemas were developed, samples of documents were extracted, and annotations are ongoing. Operational definitions of structured data elements have been created and steps to create an analytic dataset are underway.

Conclusions

To our knowledge, this is the largest cohort employed to identify PrU risk factors in the United States. It also represents the first time natural language processing and statistical text mining will be used to expand the number of variables available for analysis. A major strength of this quantitative study is that all VHA SCI centers were included in the analysis, reducing potential for selection bias and providing increased power for complex statistical analyses. This longitudinal study will eventually result in a risk prediction tool to assess PrU risk that is reliable and valid, and that is sensitive to this vulnerable population.

A Case Study of Data Quality in Text Mining Clinical Progress Notes

Text analytic methods are often aimed at extracting useful information from the vast array of unstructured, free format text documents that are created by almost all organizational processes. The success of any text mining application rests on the quality of the underlying data being analyzed, including both predictive features and outcome labels. In this case study, some focused experiments regarding data quality are used to assess the robustness of Statistical Text Mining (STM) algorithms when applied to clinical progress notes. In particular, the experiments consider the impacts of task complexity (by removing signals), training set size, and target outcome quality. While this research is conducted using a dataset drawn from the medical domain, the data quality issues explored are of more general interest.

TagLine: Information Extraction for Semi-Structured Text in Medical Progress Notes

Statistical text mining and natural language processing have been shown to be effective for extracting useful information from medical documents. However, neither technique is effective at extracting the information stored in semi-structure text elements. A prototype system (TagLine) was developed to extract information from the semi-structured text using machine learning and a rule based annotator. Features for the learning machine were suggested by prior work, and by examining text, and selecting attributes that help distinguish classes of text lines. Classes were derived empirically from text and guided by an ontology developed by the VHA's Consortium for Health Informatics Research (CHIR). Decision trees were evaluated for class predictions on 15,103 lines of text achieved an overall accuracy of 98.5 percent. The class labels applied to the lines were then used for annotating semi-structured text elements. TagLine achieved F-measure over 0.9 for each of the structures, which included tables, slots and fillers.

Finding falls in ambulatory care clinical documents using statistical text mining

OBJECTIVE

To determine how well statistical text mining (STM) models can identify falls within clinical text associated with an ambulatory encounter.

MATERIALS AND METHODS

2241 patients were selected with a fall-related ICD-9-CM E-code or matched injury diagnosis code while being treated as an outpatient at one of four sites within the Veterans Health Administration. All clinical documents within a 48-h window of the recorded E-code or injury diagnosis code for each patient were obtained (n=26 010; 611 distinct document titles) and annotated for falls. Logistic regression, support vector machine, and cost-sensitive support vector machine (SVM-cost) models were trained on a stratified sample of 70% of documents from one location (dataset Atrain) and then applied to the remaining unseen documents (datasets Atest-D).

RESULTS

All three STM models obtained area under the receiver operating characteristic curve (AUC) scores above 0.950 on the four test datasets (Atest-D). The SVM-cost model obtained the highest AUC scores, ranging from 0.953 to 0.978. The SVM-cost model also achieved F-measure values ranging from 0.745 to 0.853, sensitivity from 0.890 to 0.931, and specificity from 0.877 to 0.944.

DISCUSSION

The STM models performed well across a large heterogeneous collection of document titles. In addition, the models also generalized across other sites, including a traditionally bilingual site that had distinctly different grammatical patterns.

CONCLUSIONS

The results of this study suggest STM-based models have the potential to improve surveillance of falls. Furthermore, the encouraging evidence shown here that STM is a robust technique for mining clinical documents bodes well for other surveillance-related topics.

Using ensemble models to classify the sentiment expressed in suicide notes

In 2007, suicide was the tenth leading cause of death in the U.S. Given the significance of this problem, suicide was the focus of the 2011 Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing (NLP) shared task competition (track two). Specifically, the challenge concentrated on sentiment analysis, predicting the presence or absence of 15 emotions (labels) simultaneously in a collection of suicide notes spanning over 70 years. Our team explored multiple approaches combining regular expression-based rules, statistical text mining (STM), and an approach that applies weights to text while accounting for multiple labels. Our best submission used an ensemble of both rules and STM models to achieve a micro-averaged F₁ score of 0.5023, slightly above the mean from the 26 teams that competed (0.4875).

Identification of a potent anti-IL-15 antibody with opposing mechanisms of action in vitro and in vivo

BACKGROUND AND PURPOSE

Interleukin-15 (IL-15) is important in the activation and proliferation of lymphocytic cell populations and is implicated in inflammatory disease. We report the characterization of a novel monoclonal antibody DISC0280 which is specific for human IL-15.

EXPERIMENTAL APPROACH

DISC0280 was characterized in a direct binding assay of IL-15 with IL-15 receptor α (IL-15Rα) and by its ability to alter IL-15 mediated proliferation of a range of cell lines (cytotoxic T lymphocyte line-2, M-07e, KIT225). A pharmacodynamic model injecting male C57/BL6 mice with IL-15 or IL-15/IL-15Rα, with or without DISC0280, and assessing changes in lymphocytic cell populations and serum cytokines was utilized.

KEY RESULTS

DISC0280 inhibited the binding of IL-15 to IL-15Rα and also potently inhibits IL-15 dependent proliferation of cells expressing IL-15Rα, shared interleukin 2/ interleukin 15 receptor β chain (IL-15Rβ) and common gamma chain (γ(c) ). DISC0280 also inhibited the IL-15 dependent proliferation of M-07e cells that only express IL-15Rβ/γ(c) subunits. Human IL-15 injected into mice caused an increase in NK1.1(+) and CD3(+) cells in the spleen and peripheral blood and these effects were unexpectedly potentiated by giving DISC0280 with human IL-15. This increase in cells caused by DISC0280/IL-15 co-administration was greater than that observed when IL-15 was administered complexed with soluble IL-15Rα.

CONCLUSIONS AND IMPLICATIONS

The ability of DISC0280 to bind to the IL-15Rα-binding site on IL-15 allows trans-presentation of IL-15 by DISC0280 in vivo, similar to the trans-presentation by soluble IL-15Rα. DISC0280 may be therefore suitable as a clinical substitute for IL-15.