Predictors of Documented Goals-of-Care Discussion for Hospitalized Patients With Chronic Illness

Artificial intelligence for better goals of care documentation

OBJECTIVES

Lower rates of goals of care (GOC) conversations have been observed in non-white hospitalised patients, which may contribute to racial disparities in end-of-life care. We aimed to assess how a targeted initiative to increase GOC documentation rates is associated with GOC documentation by race.

METHODS

We retrospectively assessed GOC documentation during a targeted GOC initiative for adult patients with an artificial intelligence predicted elevated risk of mortality. Patients were admitted to an urban academic medical centre in Pittsburgh, Pennsylvania between July 2021 and 31 December 2022.

RESULTS

The 3643 studied patients had a median age of 72 (SD 13.0) and were predominantly white (87%) with 42% admitted to an intensive care unit and 15% dying during admission. GOC documentation was completed for 28% (n=1019/3643). By race, GOC was documented for 30% black (n=105/351), 28% white (n=883/3161) and 24% other (n=31/131) patients (p=0.3933). There was no statistical difference in the rate of documented GOC among races over time (p=0.5142).

CONCLUSIONS

A targeted initiative to increase documented GOC conversations for hospitalised patients with an elevated risk of mortality is associated with similar documentation rates across racial groups. Further research is needed to assess whether this initiative may promote racial equity in GOC documentation in other settings.

Equity in Using Artificial Intelligence Mortality Predictions to Target Goals of Care Documentation

BACKGROUND

Artificial intelligence (AI) algorithms are increasingly used to target patients with elevated mortality risk scores for goals-of-care (GOC) conversations.

OBJECTIVE

To evaluate the association between the presence or absence of AI-generated mortality risk scores with GOC documentation.

DESIGN

Retrospective cross-sectional study at one large academic medical center between July 2021 and December 2022.

PARTICIPANTS

Hospitalized adult patients with AI-defined Serious Illness Risk Indicator (SIRI) scores indicating > 30% 90-day mortality risk (defined as "elevated" SIRI) or no SIRI scores due to insufficient data.

INTERVENTION

A targeted intervention to increase GOC documentation for patients with AI-generated scores predicting elevated risk of mortality.

MAIN MEASURES

Odds ratios comparing GOC documentation for patients with elevated or no SIRI scores with similar severity of illness using propensity score matching and risk-adjusted mixed-effects logistic regression.

KEY RESULTS

Among 13,710 patients with elevated (n = 3643, 27%) or no (n = 10,067, 73%) SIRI scores, the median age was 64 years (SD 18). Twenty-five percent were non-White, 18% had Medicaid, 43% were admitted to an intensive care unit, and 11% died during admission. Patients lacking SIRI scores were more likely to be younger (median 60 vs. 72 years, p < 0.0001), be non-White (29% vs. 13%, p < 0.0001), and have Medicaid (22% vs. 9%, p < 0.0001). Patients with elevated versus no SIRI scores were more likely to have GOC documentation in the unmatched (aOR 2.5, p < 0.0001) and propensity-matched cohorts (aOR 2.1, p < 0.0001).

CONCLUSIONS

Using AI predictions of mortality to target GOC documentation may create differences in documentation prevalence between patients with and without AI mortality prediction scores with similar severity of illness. These finding suggest using AI to target GOC documentation may have the unintended consequence of disadvantaging severely ill patients lacking AI-generated scores from receiving targeted GOC documentation, including patients who are more likely to be non-White and have Medicaid insurance.

Classification of Documented Goals of Care Among Hospitalized Patients with High Mortality Risk: a Mixed-Methods Feasibility Study

BACKGROUND

The ability to classify patients' goals of care (GOC) from clinical documentation would facilitate serious illness communication quality improvement efforts and pragmatic measurement of goal-concordant care. Feasibility of this approach remains unknown.

OBJECTIVE

To evaluate the feasibility of classifying patients' GOC from clinical documentation in the electronic health record (EHR), describe the frequency and patterns of changes in patients' goals over time, and identify barriers to reliable goal classification.

DESIGN

Retrospective, mixed-methods chart review study.

PARTICIPANTS

Adults with high (50-74%) and very high (≥ 75%) 6-month mortality risk admitted to three urban hospitals.

MAIN MEASURES

Two physician coders independently reviewed EHR notes from 6 months before through 6 months after admission to identify documented GOC discussions and classify GOC. GOC were classified into one of four prespecified categories: (1) comfort-focused, (2) maintain or improve function, (3) life extension, or (4) unclear. Coder interrater reliability was assessed using kappa statistics. Barriers to classifying GOC were assessed using qualitative content analysis.

KEY RESULTS

Among 85 of 109 (78%) patients, 338 GOC discussions were documented. Inter-rater reliability was substantial (75% interrater agreement; Cohen's kappa = 0.67; 95% CI, 0.60-0.73). Patients' initial documented goal was most frequently "life extension" (N = 37, 44%), followed by "maintain or improve function" (N = 28, 33%), "unclear" (N = 17, 20%), and "comfort-focused" (N = 3, 4%). Among the 66 patients whose goals' classification changed over time, most changed to "comfort-focused" goals (N = 49, 74%). Primary reasons for unclear goals were the observation of concurrently held or conditional goals, patient and family uncertainty, and limited documentation.

CONCLUSIONS

Clinical notes in the EHR can be used to reliably classify patients' GOC into discrete, clinically germane categories. This work motivates future research to use natural language models to promote scalability of the approach in clinical care and serious illness research.

Selecting outcomes for pragmatic clinical trials in dementia care: The IMPACT Collaboratory iLibrary

BACKGROUND

Many interventions improve care and outcomes for people with Alzheimer's Disease and related dementias (ADRD), yet are never disseminated. Pragmatic trials facilitate the adoption and dissemination of best practices, but gaps in pragmatic outcome measurement are a critical obstacle. Our objectives are (1) to describe the development and structure of the IMbedded Pragmatic ADRD Clinical Trials Collaboratory (IMPACT) iLibrary of potential outcome measures for ADRD pragmatic trials, and (2) to assess their pragmatic characteristics.

METHODS

We identified potential outcome measures from several sources: a database of administrative and clinical outcome measures from ADRD clinical trials registered in ClinicalTrials.gov, published reviews, and IMPACT pilot pragmatic trial outcome measures. The iLibrary reports (a) number of items, (b) completion time, (c) readability for diverse populations, (d) cost or copyright barriers to use, (e) method of administration, (f) assessor training burden, and (g) feasibility of data capture and interpretation in routine care; a summary of pragmatic characteristics of each outcome measure (high, moderate, low); items or descriptions of items; and links to primary citations regarding development or psychometric properties.

RESULTS

We included 140 outcome measures in the iLibrary: 66 administrative (100% were pragmatic) and 74 clinical (52% were pragmatic). The most commonly addressed outcome domains from administrative assessments included physical function, quality of care or communication concerns, and psychological symptoms or distress behaviors. The most commonly addressed outcome domains from clinical assessments were psychological symptoms or distress behaviors, physical function, cognitive function, and health-related quality of life.

CONCLUSIONS

Pragmatic outcome measures are brief, meaningful to diverse populations, easily scored and interpreted by clinicians, and available in electronic format for analysis. The iLibrary can facilitate the selection of measures for a wide range of outcomes relevant to people with ADRD and their care partners.

Disparities in Preoperative Goals of Care Documentation in Veterans

Importance

Preoperative goals of care discussion and documentation are important for patients undergoing surgery, a major health care stressor that incurs risk.

Objective

To assess the association of race, ethnicity, and other factors, including history of mental health disability, with disparities in preoperative goals of care documentation among veterans.

Design, Setting, and Participants

This retrospective cross-sectional study assessed data from the Veterans Healthcare Administration (VHA) of 229 737 veterans who underwent surgical procedures between January 1, 2017, and October 18, 2022.

Exposures

Patient-level (ie, race, ethnicity, medical comorbidities, history of mental health comorbidity) and system-level (ie, facility complexity level) factors.

Main Outcomes and Measures

Preoperative life-sustaining treatment (LST) note documentation or no LST note documentation within 30 days prior to or on day of surgery. The standardized mean differences were calculated to assess the magnitude of differences between groups. Odds ratios (ORs) and 95% CIs were estimated with logistic regression.

Results

In this study, 13 408 patients (5.8%) completed preoperative LST from 229 737 VHA patients (209 123 [91.0%] male; 20 614 [9.0%] female; mean [SD] age, 65.5 [11.9] years) who received surgery. Compared with patients who did complete preoperative LST, patients tended to complete preoperative documentation less often if they were female (19 914 [9.2%] vs 700 [5.2%]), Black individuals (42 571 [19.7%] vs 2416 [18.0%]), Hispanic individuals (11 793 [5.5%] vs 631 [4.7%]), or from rural areas (75 637 [35.0%] vs 4273 [31.9%]); had a history of mental health disability (65 974 [30.5%] vs 4053 [30.2%]); or were seen at lowest-complexity (ie, level 3) facilities (7849 [3.6%] vs 78 [0.6%]). Over time, despite the COVID-19 pandemic, patients undergoing surgical procedures completed preoperative LST increasingly more often. Covariate-adjusted estimates of preoperative LST completion demonstrated that patients of racial or ethnic minority background (Black patients: OR, 0.79; 95% CI, 0.77-0.80; P <.001; patients selecting other race: OR, 0.78; 95% CI, 0.74-0.81; P <.001; Hispanic patients: OR, 0.78; 95% CI, 0.76-0.81; P <.001) and patients from rural regions (OR, 0.91; 95% CI, 0.90-0.93; P <.001) had lower likelihoods of completing LST compared with patients who were White or non-Hispanic and patients from urban areas. Patients with any mental health disability history also had lower likelihood of completing preoperative LST than those without a history (OR, 0.93; 95% CI, 0.92-0.94; P = .001).

Conclusions and Relevance

In this cross-sectional study, disparities in documentation rates within a VHA cohort persisted based on race, ethnicity, rurality of patient residence, history of mental health disability, and access to high-volume, high-complexity facilities.

Intervention to Promote Communication About Goals of Care for Hospitalized Patients With Serious Illness: A Randomized Clinical Trial

Importance

Discussions about goals of care are important for high-quality palliative care yet are often lacking for hospitalized older patients with serious illness.

Objective

To evaluate a communication-priming intervention to promote goals-of-care discussions between clinicians and hospitalized older patients with serious illness.

Design, Setting, and Participants

A pragmatic, randomized clinical trial of a clinician-facing communication-priming intervention vs usual care was conducted at 3 US hospitals within 1 health care system, including a university, county, and community hospital. Eligible hospitalized patients were aged 55 years or older with any of the chronic illnesses used by the Dartmouth Atlas project to study end-of-life care or were aged 80 years or older. Patients with documented goals-of-care discussions or a palliative care consultation between hospital admission and eligibility screening were excluded. Randomization occurred between April 2020 and March 2021 and was stratified by study site and history of dementia.

Intervention

Physicians and advance practice clinicians who were treating the patients randomized to the intervention received a 1-page, patient-specific intervention (Jumpstart Guide) to prompt and guide goals-of-care discussions.

Main Outcomes and Measures

The primary outcome was the proportion of patients with electronic health record-documented goals-of-care discussions within 30 days. There was also an evaluation of whether the effect of the intervention varied by age, sex, history of dementia, minoritized race or ethnicity, or study site.

Results

Of 3918 patients screened, 2512 were enrolled (mean age, 71.7 [SD, 10.8] years and 42% were women) and randomized (1255 to the intervention group and 1257 to the usual care group). The patients were American Indian or Alaska Native (1.8%), Asian (12%), Black (13%), Hispanic (6%), Native Hawaiian or Pacific Islander (0.5%), non-Hispanic (93%), and White (70%). The proportion of patients with electronic health record-documented goals-of-care discussions within 30 days was 34.5% (433 of 1255 patients) in the intervention group vs 30.4% (382 of 1257 patients) in the usual care group (hospital- and dementia-adjusted difference, 4.1% [95% CI, 0.4% to 7.8%]). The analyses of the treatment effect modifiers suggested that the intervention had a larger effect size among patients with minoritized race or ethnicity. Among 803 patients with minoritized race or ethnicity, the hospital- and dementia-adjusted proportion with goals-of-care discussions was 10.2% (95% CI, 4.0% to 16.5%) higher in the intervention group than in the usual care group. Among 1641 non-Hispanic White patients, the adjusted proportion with goals-of-care discussions was 1.6% (95% CI, -3.0% to 6.2%) higher in the intervention group than in the usual care group. There was no evidence of differential treatment effects of the intervention on the primary outcome by age, sex, history of dementia, or study site.

Conclusions and Relevance

Among hospitalized older adults with serious illness, a pragmatic clinician-facing communication-priming intervention significantly improved documentation of goals-of-care discussions in the electronic health record, with a greater effect size in racially or ethnically minoritized patients.

Trial Registration

ClinicalTrials.gov Identifier: NCT04281784.

Assessment of Natural Language Processing of Electronic Health Records to Measure Goals-of-Care Discussions as a Clinical Trial Outcome

IMPORTANCE

Many clinical trial outcomes are documented in free-text electronic health records (EHRs), making manual data collection costly and infeasible at scale. Natural language processing (NLP) is a promising approach for measuring such outcomes efficiently, but ignoring NLP-related misclassification may lead to underpowered studies.

OBJECTIVE

To evaluate the performance, feasibility, and power implications of using NLP to measure the primary outcome of EHR-documented goals-of-care discussions in a pragmatic randomized clinical trial of a communication intervention.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study compared the performance, feasibility, and power implications of measuring EHR-documented goals-of-care discussions using 3 approaches: (1) deep-learning NLP, (2) NLP-screened human abstraction (manual verification of NLP-positive records), and (3) conventional manual abstraction. The study included hospitalized patients aged 55 years or older with serious illness enrolled between April 23, 2020, and March 26, 2021, in a pragmatic randomized clinical trial of a communication intervention in a multihospital US academic health system.

MAIN OUTCOMES AND MEASURES

Main outcomes were natural language processing performance characteristics, human abstractor-hours, and misclassification-adjusted statistical power of methods of measuring clinician-documented goals-of-care discussions. Performance of NLP was evaluated with receiver operating characteristic (ROC) curves and precision-recall (PR) analyses and examined the effects of misclassification on power using mathematical substitution and Monte Carlo simulation.

RESULTS

A total of 2512 trial participants (mean [SD] age, 71.7 [10.8] years; 1456 [58%] female) amassed 44 324 clinical notes during 30-day follow-up. In a validation sample of 159 participants, deep-learning NLP trained on a separate training data set from identified patients with documented goals-of-care discussions with moderate accuracy (maximal F1 score, 0.82; area under the ROC curve, 0.924; area under the PR curve, 0.879). Manual abstraction of the outcome from the trial data set would require an estimated 2000 abstractor-hours and would power the trial to detect a risk difference of 5.4% (assuming 33.5% control-arm prevalence, 80% power, and 2-sided α = .05). Measuring the outcome by NLP alone would power the trial to detect a risk difference of 7.6%. Measuring the outcome by NLP-screened human abstraction would require 34.3 abstractor-hours to achieve estimated sensitivity of 92.6% and would power the trial to detect a risk difference of 5.7%. Monte Carlo simulations corroborated misclassification-adjusted power calculations.

CONCLUSIONS AND RELEVANCE

In this diagnostic study, deep-learning NLP and NLP-screened human abstraction had favorable characteristics for measuring an EHR outcome at scale. Adjusted power calculations accurately quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of studies using NLP would be beneficial.