Effects of Neighborhood-level Data on Performance and Algorithmic Equity of a Model That Predicts 30-day Heart Failure Readmissions at an Urban Academic Medical Center

Mitigating Racial And Ethnic Bias And Advancing Health Equity In Clinical Algorithms: A Scoping Review

In August 2022 the Department of Health and Human Services (HHS) issued a notice of proposed rulemaking prohibiting covered entities, which include health care providers and health plans, from discriminating against individuals when using clinical algorithms in decision making. However, HHS did not provide specific guidelines on how covered entities should prevent discrimination. We conducted a scoping review of literature published during the period 2011-22 to identify health care applications, frameworks, reviews and perspectives, and assessment tools that identify and mitigate bias in clinical algorithms, with a specific focus on racial and ethnic bias. Our scoping review encompassed 109 articles comprising 45 empirical health care applications that included tools tested in health care settings, 16 frameworks, and 48 reviews and perspectives. We identified a wide range of technical, operational, and systemwide bias mitigation strategies for clinical algorithms, but there was no consensus in the literature on a single best practice that covered entities could employ to meet the HHS requirements. Future research should identify optimal bias mitigation methods for various scenarios, depending on factors such as patient population, clinical setting, algorithm design, and types of bias to be addressed.

Can a 5-to-90-day Mortality Predictor Perform Consistently Across Time and Equitably Across Populations?

Advance care planning (ACP) facilitates end-of-life care, yet many die without it. Timely and accurate mortality prediction may encourage ACP. However, performance of predictors typically differs among sub-populations (e.g., rural vs. urban) and worsens over time ("concept drift"). Therefore, we assessed performance equity and consistency for a novel 5-to-90-day mortality predictor across various demographies, geographies, and timeframes (n = 76,812 total encounters). Predictions were made for the first day of included adult inpatient admissions on a retrospective dataset. AUC-PR remained at 29% both pre-COVID (throughout 2018) and during COVID (8 months in 2021). Pre-COVID-19 recall and precision were 58% and 25% respectively at the 12.5% certainty cutoff, and 12% and 44% at the 37.5% cutoff. During COVID-19, recall and precision were 59% and 26% at the 12.5% cutoff, and 11% and 43% at the 37.5% cutoff. Pre-COVID, compared to the overall population, recall was lower at the 12.5% cutoff in the White, non-Hispanic subgroup and at both cutoffs in the rural subgroup. During COVID-19, precision at the 12.5% cutoff was lower than that of the overall population for the non-White and non-White female subgroups. No other significant differences were seen between subgroups and the corresponding overall population. Overall performance during COVID was unchanged from pre-pandemic performance. Although some comparisons (especially precision at the 37.5% cutoff) were underpowered, precision at the 12.5% cutoff was equitable across most demographies, regardless of the pandemic. Mortality prediction to prioritize ACP conversations can be provided consistently and equitably across many studied timeframes and sub-populations.

Social Determinants of Health and Their Impact on the Black Race Coefficient in Serum Creatinine-Based Estimation of GFR: Secondary Analysis of MDRD and CRIC Studies

BACKGROUND

The cause for differences in serum creatinine between Black and non-Black individuals incorporated into prior GFR-estimating equations is not understood. We explored whether social determinants of health can account for this difference.

METHODS

We conducted a secondary analysis of baseline data of the Modification of Diet in Renal Disease and Chronic Renal Insufficiency Cohort studies ( N =1628 and 1423, respectively). Data in both study cohorts were stratified by race (Black versus non-Black). We first evaluated the extent to which the coefficient of Black race in estimating GFR from creatinine is explained by correlations of race with social determinants of health and non-GFR determinants of creatinine. Second, we evaluated whether the difference between race groups in adjusted mean creatinine can be explained by social determinants of health and non-GFR determinants of creatinine.

RESULTS

In models regressing measured GFR on creatinine, age, sex, and race, the coefficient for Black race was 21% (95% confidence interval, 0.176 to 0.245) in Modification of Diet in Renal Disease and 13% (95% confidence interval, 0.097 to 0.155) in the Chronic Renal Insufficiency Cohort and was not attenuated by the addition of social determinants of health, alone or in combination. In both studies, the coefficient for Black race was larger at lower versus higher income levels. In models, regressing creatinine on measured GFR, age, and sex, mean creatinine was higher in Black versus non-Black participants in both studies, with no effect of social determinants of health.

CONCLUSIONS

Adjustment for selected social determinants of health did not influence the relationship between Black race and creatinine-based estimated GFR.

Development and validation of a prediction model for actionable aspects of frailty in the text of clinicians' encounter notes

OBJECTIVE

Frailty is a prevalent risk factor for adverse outcomes among patients with chronic lung disease. However, identifying frail patients who may benefit from interventions is challenging using standard data sources. We therefore sought to identify phrases in clinical notes in the electronic health record (EHR) that describe actionable frailty syndromes.

MATERIALS AND METHODS

We used an active learning strategy to select notes from the EHR and annotated each sentence for 4 actionable aspects of frailty: respiratory impairment, musculoskeletal problems, fall risk, and nutritional deficiencies. We compared the performance of regression, tree-based, and neural network models to predict the labels for each sentence. We evaluated performance with the scaled Brier score (SBS), where 1 is perfect and 0 is uninformative, and the positive predictive value (PPV).

RESULTS

We manually annotated 155 952 sentences from 326 patients. Elastic net regression had the best performance across all 4 frailty aspects (SBS 0.52, 95% confidence interval [CI] 0.49-0.54) followed by random forests (SBS 0.49, 95% CI 0.47-0.51), and multi-task neural networks (SBS 0.39, 95% CI 0.37-0.42). For the elastic net model, the PPV for identifying the presence of respiratory impairment was 54.8% (95% CI 53.3%-56.6%) at a sensitivity of 80%.

DISCUSSION

Classification models using EHR notes can effectively identify actionable aspects of frailty among patients living with chronic lung disease. Regression performed better than random forest and neural network models.

CONCLUSIONS

NLP-based models offer promising support to population health management programs that seek to identify and refer community-dwelling patients with frailty for evidence-based interventions.