Contemporary Analysis of Electronic Frailty Measurement in Older Adults with Multiple Myeloma Treated in the National US Veterans Affairs Healthcare System

Natural Language Processing Algorithm to Extract Multiple Myeloma Stage From Oncology Notes in the Veterans Affairs Healthcare System

PURPOSE

Stage in multiple myeloma (MM) is an essential measure of disease risk, but its measurement in large databases is often lacking. We aimed to develop and validate a natural language processing (NLP) algorithm to extract oncologists' documentation of stage in the national Veterans Affairs (VA) Healthcare System.

METHODS

Using nationwide electronic health record (EHR) and cancer registry data from the VA Corporate Data Warehouse, we developed and validated a rule-based NLP algorithm to extract oncologist-determined MM stage. To that end, a clinician annotated MM stage within over 5,000 short snippets of clinical notes, and annotated MM stage at MM treatment initiation for 200 patients. These were allocated into snippet- and patient-level development and validation sets. We developed MM stage extraction and roll-up algorithms within the development sets. After the algorithms were finalized, we validated them using standard measures in held-out validation sets.

RESULTS

We developed algorithms for three different MM staging systems that have been in widespread use (Revised International Staging System [R-ISS], International Staging System [ISS], and Durie-Salmon [DS]) and for stage reported without a clearly defined system. Precision and recall were uniformly high for MM stage at the snippet level, ranging from 0.92 to 0.99 for the different MM staging systems. Performance in identifying for MM stage at treatment initiation at the patient level was also excellent, with precision of 0.92, 0.96, 0.90, and 0.86 and recall of 0.99, 0.98, 0.94, and 0.92 for R-ISS, ISS, DS, and unclear stage, respectively.

CONCLUSION

Our MM stage extraction algorithm uses rule-based NLP and data aggregation to accurately measure MM stage documented in oncology notes and pathology reports in VA's national EHR system. It may be adapted to other systems where MM stage is recorded in clinical notes.

Current Health Status of Gulf War Deployed and Gulf War Era Veterans Who Use Veterans Affairs Health Care

Background: Although some recent studies have examined the health of female Gulf War (GW) deployed and non-deployed GW era veterans, these all relied on self-report, which can be inaccurate and subject to recall bias. This study investigated the current health of GW deployed and non-deployed GW era female and male veterans using Veterans Health Administration (VHA) electronic health records (EHR). Methods: We performed a cohort study of deployed GW and non-deployed GW era veterans, identified from a list from the Defense Manpower Data Center (DMDC). We used the VA-Frailty Index (VA-FI), calculated with VHA administrative claims and EHR, as a proxy measure of current health. Results: We identified 402,869 veterans (351,496 GW deployed; 51,3373 non-deployed GW era; 38,555 female) in VHA databases. Deployed female veterans had the highest VA-FI (i.e., were frailest) despite being younger than deployed and non-deployed male veterans and non-deployed female veterans. Compared with deployed male veterans, deployed females were more likely to be pre-frail, mildly, and moderately frail. Health differences between deployed and non-deployed female veterans were more prominent among older (60+ years) than younger (<60 years) veterans. Conclusions: Mirroring reports from recent, smaller survey studies of users and non-users of VA health care, findings from this cohort study indicate that deployed female GW veterans who use VA health care are frailer and have more health deficits than non-deployed female GW era and deployed male GW veterans. Because deployed female GW veterans appear to have additional health care needs, this may warrant increased outreach from women's clinics at VA hospitals.

How I treat multiple myeloma in geriatric patients

ABSTRACT

Multiple myeloma (MM) is primarily a disease of older patients. Until recently, geriatric aspects in the context of MM have been poorly investigated. Treatment outcomes for geriatric patients with MM are often compromised by comorbidities and an enhanced susceptibility to adverse events from therapy. Assessment of patient frailty has become more frequent and will be useful in the context of significant and continuous advances in therapy. The recent emergence of immunotherapy with CD38 monoclonal antibodies and upcoming immunooncology drugs, such as bispecific antibodies, will lead to additional therapeutic progress. The applicability of these new molecules to older and frail patients is a key clinical question. Here, we present 2 patient cases derived from clinical practice. We review current frailty scores and standards of care for older, newly diagnosed patients with MM, including frail subgroups, and discuss ways to tailor treatment, as well as treatment perspectives in this population.

Construct validity of the electronic Veterans Affairs Frailty Index against clinician frailty assessment

BACKGROUND

Electronic frailty indices (eFIs) can expand measurement of frailty in research and practice and have demonstrated predictive validity in associations with clinical outcomes. However, their construct validity is less well studied. We aimed to assess the construct validity of the VA-FI, an eFI developed for use in the U.S. Veterans Affairs Healthcare System.

METHODS

Veterans who underwent comprehensive geriatric assessments between January 31, 2019 and June 6, 2022 at VA Boston and had sufficient data documented for a comprehensive geriatric assessment-frailty index (CGA-FI) were included. The VA-FI, based on diagnostic and procedural codes, and the CGA-FI, based on geriatrician-measured deficits, were calculated for each patient. Geriatricians also assessed the Clinical Frailty Scale (CFS), functional status (ADLs and IADLs), and 4-meter gait speed (4MGS).

RESULTS

A total of 132 veterans were included, with median age 81.4 years (IQR 75.8-88.7). Across increasing levels of VA-FI (<0.2; 0.2-0.4; >0.4), mean CGA-FI increased (0.24; 0.30; 0.40). The VA-FI was moderately correlated with the CGA-FI (r 0.45, p < 0.001). Every 0.1-unit increase in the VA-FI was associated with an increase in the CGA-FI (linear regression beta 0.05; 95% confidence interval [CI] 0.03-0.06), higher CFS category (ordinal regression OR 1.69; 95% CI 1.24-2.30), higher odds of ADL dependency (logistic regression OR 1.59; 95% CI 1.20-2.11), IADL dependency (logistic regression OR 1.68; 95% CI 1.23-2.30), and a decrease in 4MGS (linear regression beta -0.07, 95% CI -0.12 to -0.02). All models were adjusted for age and race, and associations held after further adjustment for the Charlson Comorbidity Index.

CONCLUSION

Our results demonstrate the construct validity of the VA-FI through its associations with clinical measures of frailty, including summary frailty measures, functional status, and objective physical performance. Our findings complement others' in showing that eFIs can capture functional and mobility domains of frailty beyond just comorbidity and may be useful to measure frailty among populations and individuals.

Baseline correlates of frailty and its association with survival in United States veterans with acute myeloid leukemia

Frailty is an important construct to measure in acute myeloid leukemia (AML). We used the Veterans Affairs Frailty Index (VA-FI) - calculated using readily available data within the VA's electronic health records - to measure frailty in U.S. veterans with AML. Of the 1166 newly diagnosed and treated veterans with AML between 2012 and 2022, 722 (62%) veterans with AML were classified as frail (VA-FI > 0.2). At a median follow-up of 252.5 days, moderate-severely frail veterans had significantly worse survival than mildly frail, and non-frail veterans (median survival 179 vs. 306 vs. 417 days, p < .001). Increasing VA-FI severity was associated with higher mortality. A model with VA-FI in addition to the European LeukemiaNet (ELN) risk classification and other covariates statistically outperformed a model containing the ELN risk and other covariates alone (p < .001). These findings support the VA-FI as a tool to expand frailty measurement in research and clinical practice for informing prognosis in veterans with AML.

More intensive therapy as more effective treatment for frail patients with multiple myeloma [corrected]

Although randomized controlled trial data suggest that the more intensive triplet bortezomib-lenalidomide-dexamethasone (VRd) is superior to the less intensive doublet lenalidomide-dexamethasone (Rd) in patients newly diagnosed with multiple myeloma (MM), guidelines have historically recommended Rd over VRd for patients who are frail and may not tolerate a triplet. We identified 2573 patients (median age, 69.7 years) newly diagnosed with MM who were initiated on VRd (990) or Rd (1583) in the national US Veterans Affairs health care System from 2004 to 2020. We measured frailty using the Veterans Affairs Frailty Index. To reduce imbalance in confounding, we matched patients for MM stage and 1:1 based on a propensity score. Patients who were moderate-severely frail had a higher prevalence of stage III MM and myeloma-related frailty deficits than patients who were not frail. VRd vs Rd was associated with lower mortality (hazard ratio [HR], 0.81; 95% confidence interval [CI], 0.70-0.94) in the overall matched population. Patients who were moderate-severely frail demonstrated the strongest association (HR 0.74; 95% CI, 0.56-0.97), whereas the association weakened in those who were mildly frail (HR, 0.80; 95% CI, 0.61-1.05) and nonfrail (HR, 0.86; 95% CI, 0.67-1.10). VRd vs Rd was associated with a modestly higher incidence of hospitalizations in the overall population, but this association weakened in patients who were moderate-severely frail. Our findings confirm the benefit of VRd over Rd in US veterans and further suggest that this benefit is strongest in patients with the highest levels of frailty, arguing that more intensive treatment of myeloma may be more effective treatment of frailty itself.

Natural Language Processing-Assisted Classification Models to Confirm Monoclonal Gammopathy of Undetermined Significance and Progression in Veterans' Electronic Health Records

PURPOSE

To develop and validate natural language processing (NLP)-assisted machine learning (ML)-based classification models to confirm diagnoses of monoclonal gammopathy of undetermined significance (MGUS) and multiple myeloma (MM) from electronic health records (EHRs) in the Veterans Health Administration (VHA).

MATERIALS AND METHODS

We developed precompiled lexicons and classification rules as features for the following ML classifiers: logistic regression, random forest, and support vector machines (SVMs). These features were trained on 36,044 EHR documents from a random sample of 400 patients with at least one International Classification of Disease code for MGUS diagnosis from 1999 to 2021. The best-performing feature combination was calibrated in the validation set (17,826 documents/200 patients) and evaluated in the testing set (9,250 documents/100 patients). Model performance in diagnosis confirmation was compared with manual chart review results (gold standard) using recall, precision, accuracy, and F1 score. For patients correctly labeled as disease-positive, the difference between model-identified diagnosis dates and the gold standard was also computed.

RESULTS

In the testing set, the NLP-assisted classification model using SVMs achieved best performance in both MGUS and MM confirmation with recall/precision/accuracy/F1 of 98.8%/93.3%/93.0%/96.0% for MGUS and 100.0%/92.3%/99.0%/96.0% for MM. Dates of diagnoses matched (±45 days) with those of gold standard in 73.0% of model-confirmed MGUS and 84.6% of model-confirmed MM.

CONCLUSION

An NLP-assisted classification model can reliably confirm MGUS and MM diagnoses and dates and extract laboratory results using automated interpretation of EHR data. This algorithm has the potential to be adapted to other disease areas in VHA EHR system.

Assessment of an embedded primary care-derived electronic health record (EHR) frailty index (eFI) in older adults with acute myeloid leukemia

INTRODUCTION

Assessing frailty is integral to treatment decision-making for older adults with acute myeloid leukemia (AML). Prior electronic frailty indices (eFI) derive from an accumulated-deficit model and are associated with mortality in older primary care populations. We evaluated use of an embedded eFI in AML by describing baseline eFI categories by treatment type and exploring associations between eFI categories, survival, and treatment received.

MATERIALS AND METHODS

This was a retrospective study of subjects ≥60 years old with new AML treated at an academic medical center from 1/2018-10/2020. The eFI requires ≥2 ambulatory visits over two years and uses demographics, vitals, ICD-10 codes, outpatient labs, and available functional information from Medicare Annual Wellness Visits. Frailty was defined as fit (eFI ≤ 0.10), pre-frail (0.10 < eFI ≤ 0.21), and frail (eFI > 0.21). Chemotherapy was intensive (anthracycline-based) or less-intensive (hypomethylating agent, low dose cytarabine +/- venetoclax). Therapy type, pre-treatment characteristics, and chemotherapy cycles were compared by eFI category using chi-square and Fisher's exact tests and ANOVA. Median survival was compared by eFI category using log-rank tests stratified by therapy type.

RESULTS

Among 166 older adults treated for AML (mean age 74 years, 61% male, 85% Caucasian), only 79 (48%) had a calculable eFI score before treatment. Of these, baseline eFI category was associated with treatment received (fit (n = 31): 68% intensive, 32% less intensive; pre-frail (n = 38): 37% intensive, 63% less intensive; frail (n = 10): 0% intensive, 100% less intensive; not calculable (n = 87): 48% intensive, 52% less-intensive; p < 0.01). The prevalence of congestive heart failure and secondary AML differed by frailty status (p < 0.01). Median survival did not differ between eFI categories for intensively (p = 0.48) or less-intensively (p = 0.09) treated patients. For those with less-intensive therapy who lived ≥6 months, eFI category was not associated with the number of chemotherapy cycles received (p = 0.97). The main reason for an incalculable eFI was a lack of outpatient visits in our health system prior to AML diagnosis.

DISCUSSION

A primary care-derived eFI was incalculable for half of older adults with AML at an academic medical center. Frailty was associated with chemotherapy intensity but not survival or treatment duration. Next steps include testing adaptations of the eFI to the AML setting.

Frailty and survival among veterans treated with abiraterone or enzalutamide for metastatic castration-resistant prostate cancer

INTRODUCTION

Abiraterone and enzalutamide are treatments for metastatic castration-resistant prostate cancer (mCRPC). Due to a lack of head-to-head trials, they are prescribed interchangeably. However, the drugs have different pharmacokinetics and thus may have differing efficacy and adverse effects influenced by patient functional status and comorbid diseases. Additionally, mCRPC mainly affects older adults and since the prevalence of frailty increases with age, frailty is an important patient factor to consider in personalizing drug selection.

MATERIALS AND METHODS

We conducted a retrospective observational study of US veterans treated with abiraterone or enzalutamide for mCRPC from September 2014 to June 2017. Frailty was assessed using the Veterans Affairs Frailty Index (VA-FI), which utilizes administrative codes to assign a standardized frailty score. Patients were categorized as frail if VA-FI scores were > 0.2. The primary outcome was difference in overall survival (OS) between the two treatment groups. Cox regression modeling and propensity score matching was used to compare between abiraterone and enzalutamide treatments.

RESULTS

We identified 5,822 veterans, 57% of whom were initially treated with abiraterone and 43% with enzalutamide. Frail patients (n = 2,314; 39.7%) were older, with a mean age of 76.1 versus 74.9 years in the non-frail group (n = 3,508; 60.3%, p < 0.001) and had shorter OS compared to non-frail patients regardless of treatment group (18.5 vs. 26.6 months, p < 0.001). Among non-frail patients there was no significant difference in OS between abiraterone and enzalutamide treatment (27.7 vs 26.1 months, p = 0.07). However, frail patients treated with enzalutamide versus abiraterone had improved OS (20.7 vs 17.2 months, p < 0.001). In a propensity score matched analysis of frail patients (n = 2,070), enzalutamide was associated with greater median OS (24.1 vs 20.9 months, p < 0.001). In patients with dementia, enzalutamide was associated with longer OS (19.4 vs. 16.6 months, p = 0.003).

DISCUSSION

In this study of 5822 US veterans with mCRPC, treatment with enzalutamide was associated with improved OS compared to abiraterone among frail veterans and veterans with dementia, but not among non-frail veterans. Future studies should evaluate interactions between frailty and cancer treatments to optimize selection of therapy among frail adults.

Dynamic frailty risk assessment among older adults with multiple myeloma: A population-based cohort study

Multiple myeloma (MM) is a cancer of older adults and those who are more frail are at high risk of poor outcomes. Current tools for identifying and categorizing frail patients are often static and measured only at the time of diagnosis. The concept of dynamic frailty (i.e. frailty changing over time) is largely unexplored in MM. In our study, adults with newly-diagnosed MM who received novel drugs between the years 2007-2014 were identified in the Surveillance, Epidemiology, and End Results (SEER)-Medicare linked databases. Using a previously published cumulative deficit approach, a frailty index score was calculated at diagnosis and each landmark interval (1-yr, 2-yr, 3-yr post diagnosis). The association of frailty with overall survival (OS) both at baseline and at each landmark interval as well as factors associated with worsening frailty status over time were evaluated. Overall, 4617 patients were included. At baseline, 39% of the patients were categorized as moderately frail or severely frail. Among those who had 3 years of follow-up, frailty categorization changed post diagnosis in 93% of the cohort (78% improved and 72% deteriorated at least at one time point during the follow up period). In a landmark analysis, the predictive ability of frailty at the time of diagnosis decreased over time for OS (Harrell's C Statistic 0.65 at diagnosis, 0.63 at 1-yr, 0.62 at 2-yr, and 0.60 at 3-yr) and was inferior compared to current frailty status at each landmark interval. Our study is one of the first to demonstrate the dynamic nature of frailty among older adults with MM. Frailty may improve or deteriorate over time. Current frailty status is a better predictor of outcomes than frailty status at time of diagnosis, indicating the need for re-measurement in this high-risk patient population.

Validation of algorithms to select patients with multiple myeloma and patients initiating myeloma treatment in the national Veterans Affairs Healthcare System

BACKGROUND

We aimed to evaluate and compare the performance of multiple myeloma (MM) selection algorithms for use in Veterans Affairs (VA) research.

METHODS

Using the VA Corporate Data Warehouse (CDW), the VA Cancer Registry (VACR), and VA pharmacy data, we randomly selected 500 patients from 01/01/1999 to 06/01/2021 who had (1) either one MM diagnostic code OR were listed in the VACR as having MM AND (2) at least one MM treatment code. A team reviewed oncology notes for each veteran to annotate details regarding MM diagnosis and initial treatment within VA. We evaluated inter-annotator agreement and compared the performance of four published algorithms (two developed and validated external to VA data and two used in VA data).

RESULTS

A total of 859 patients were reviewed to obtain 500 patients who were annotated as having MM and initiating MM treatment in VA. Agreement was high among annotators for all variables: MM diagnosis (98.3% agreement, Kappa = 0.93); initial treatment in VA (91.8% agreement; Kappa = 0.77); and initial treatment classification (87.6% agreement; Kappa = 0.86). VA Algorithms were more specific and had higher PPVs than non-VA algorithms for both MM diagnosis and initial treatment in VA. We developed the "VA Recommended Algorithm," which had the highest PPV among all algorithms in identifying patients diagnosed with MM (PPV = 0.98, 95% CI = 0.95-0.99) and in identifying patients who initiated their MM treatment in VA (PPV = 0.93, 95% CI = 0.90-0.96).

CONCLUSION

Our VA Recommended Algorithm optimizes sensitivity and PPV for cohort selection and treatment classification.

Frail Multiple Myeloma Patients Deserve More Than Just a Score

Frailty is a hot topic in the field of multiple myeloma (MM). Clinicians have realised that frail myeloma patients can struggle with treatment, resulting in dose reductions and treatment discontinuation, which risk shorter progression-free and overall survival. Efforts have focused on the validity of existing frailty scores and on the development of new indices to identify frail patients more accurately. This review article explores the challenges of the existing frailty scores, including the International Myeloma Working Group (IMWG) frailty score, the revised Myeloma Co-morbidity Index (R-MCI), and the Myeloma Risk Profile (MRP). We conclude that the missing link is for frailty scoring to translate into a tool useful in real-world clinical practice. The future of frailty scores lies in their ability to be woven into clinical trials, to create a robust clinical evidence base for treatment selection and dose modification, and also to identify a cohort of patients who merit additional support from the wider MM multidisciplinary team.

Sentiment analysis of medical record notes for lung cancer patients at the Department of Veterans Affairs

Natural language processing of medical records offers tremendous potential to improve the patient experience. Sentiment analysis of clinical notes has been performed with mixed results, often highlighting the issue that dictionary ratings are not domain specific. Here, for the first time, we re-calibrate the labMT sentiment dictionary on 3.5M clinical notes describing 10,000 patients diagnosed with lung cancer at the Department of Veterans Affairs. The sentiment score of notes was calculated for two years after date of diagnosis and evaluated against a lab test (platelet count) and a combination of data points (treatments). We found that the oncology specific labMT dictionary, after re-calibration for the clinical oncology domain, produces a promising signal in notes that can be detected based on a comparative analysis to the aforementioned parameters.

Identifying Frail Patients by Using Electronic Health Records in Primary Care: Current Status and Future Directions

With the rapidly aging population, frailty, characterized by an increased risk of adverse outcomes, has become a major public health problem globally. Several frailty guidelines or consensuses recommend screening for frailty, especially in primary care settings. However, most of the frailty assessment tools are based on questionnaires or physical examinations, adding to the clinical workload, which is the major obstacle to converting frailty research into clinical practice. Medical data naturally generated by routine clinical work containing frailty indicators are stored in electronic health records (EHRs) (also called electronic health record (EHR) data), which provide resources and possibilities for frailty assessment. We reviewed several frailty assessment tools based on primary care EHRs and summarized the features and novel usage of these tools, as well as challenges and trends. Further research is needed to develop and validate frailty assessment tools based on EHRs in primary care in other parts of the world.

Frailty assessment using routine clinical data: An integrative review

BACKGROUND

Frailty is a common but complex problem in older adults. Frailty assessment using routine clinical data has been suggested as a pragmatic approach based on electronic health records from primary care center or hospital settings.

PURPOSE

We aimed to explore the tools and outcome variables used in the published studies on frailty assessment using routine clinical data.

METHODS

An integrative literature review was conducted using the method of Whittemore and Knafl. A literature search was conducted in PubMed, EMBASE, and CINAHL from January 2010 to October 2021.

RESULTS

A total of 45 studies and thirteen frailty assessment tools were analyzed. The assessment items were generally biased toward frailty's risk factors rather than the mechanisms or phenotypes of frailty. Similar to using conventional tools, routine clinical data-based frailty was associated with adverse health outcomes.

CONCLUSIONS

Frailty assessment based on routine clinical data could efficiently evaluate frailty using electronic health records from primary care centers or hospitals. However, they need refinement to consider the risk factors, mechanisms, and frailty phenotypes.

Geriatric Oncology: From Research to Clinical Practice

The incidence of cancer in older adults is strongly increasing due to the ageing of the population [...].