Using measures of race to make clinical predictions: Decision making, patient health, and fairness

Guidance for unbiased predictive information for healthcare decision-making and equity (GUIDE): considerations when race may be a prognostic factor

Clinical prediction models (CPMs) are tools that compute the risk of an outcome given a set of patient characteristics and are routinely used to inform patients, guide treatment decision-making, and resource allocation. Although much hope has been placed on CPMs to mitigate human biases, CPMs may potentially contribute to racial disparities in decision-making and resource allocation. While some policymakers, professional organizations, and scholars have called for eliminating race as a variable from CPMs, others raise concerns that excluding race may exacerbate healthcare disparities and this controversy remains unresolved. The Guidance for Unbiased predictive Information for healthcare Decision-making and Equity (GUIDE) provides expert guidelines for model developers and health system administrators on the transparent use of race in CPMs and mitigation of algorithmic bias across contexts developed through a 5-round, modified Delphi process from a diverse 14-person technical expert panel (TEP). Deliberations affirmed that race is a social construct and that the goals of prediction are distinct from those of causal inference, and emphasized: the importance of decisional context (e.g., shared decision-making versus healthcare rationing); the conflicting nature of different anti-discrimination principles (e.g., anticlassification versus antisubordination principles); and the importance of identifying and balancing trade-offs in achieving equity-related goals with race-aware versus race-unaware CPMs for conditions where racial identity is prognostically informative. The GUIDE, comprising 31 key items in the development and use of CPMs in healthcare, outlines foundational principles, distinguishes between bias and fairness, and offers guidance for examining subgroup invalidity and using race as a variable in CPMs. This GUIDE presents a living document that supports appraisal and reporting of bias in CPMs to support best practice in CPM development and use.

Predictive Factors of Apparent Treatment Resistant Hypertension Among Patients With Hypertension Identified Using Electronic Health Records

BACKGROUND

Early identification of a patient with resistant hypertension (RH) enables quickly intensified treatment, short-interval follow-up, or perhaps case management to bring his or her blood pressure under control and reduce the risk of complications.

OBJECTIVE

To identify predictors of RH among individuals with newly diagnosed hypertension (HTN), while comparing different prediction models and techniques for managing missing covariates using electronic health records data.

DESIGN

Risk prediction study in a retrospective cohort.

PARTICIPANTS

Adult patients with incident HTN treated in any of the primary care clinics of one health system between April 2013 and December 2016.

MAIN MEASURES

Predicted risk of RH at the time of HTN identification and candidate predictors for variable selection in future model development.

KEY RESULTS

Among 26,953 individuals with incident HTN, 613 (2.3%) met criteria for RH after 4.7 months (interquartile range, 1.2-11.3). Variables selected by the least absolute shrinkage and selection operator (LASSO), included baseline systolic blood pressure (SBP) and its missing indicator (a dummy variable created if baseline SBP is absent), use of antihypertensive medication at the time of cohort entry, body mass index, and atherosclerosis risk. The random forest technique achieved the highest area under the curve (AUC) of 0.893 (95% CI, 0.881-0.904) and the best calibration with a calibration slope of 1.01. Complete case analysis is not a valuable option (AUC = 0.625).

CONCLUSIONS

Machine learning techniques and traditional logistic regression exhibited comparable levels of predictive performance after handling the missingness. We suggest that the variables identified by this study may be good candidates for clinical prediction models to alert clinicians to the need for short-interval follow up and more intensive early therapy for HTN.

Race as a Risk Marker, Not a Risk Factor: Revising Race-Based Algorithms to Protect Racially Oppressed Patients

Emerging consensus in the medical and public health spheres encourages removing race and ethnicity from algorithms used in clinical decision-making. Although clinical algorithms remain appealing given their promise to lighten the cognitive load of medical practice and save time for providers, they risk exacerbating existing health disparities. Race is a strong risk marker of health outcomes, yet it is not a risk factor. The use of race as a factor in medical algorithms suggests that the effect of race is intrinsic to the patient or that its effects can be distinct or separated from other social and environmental variables. By contrast, incisive public health analysis coupled with a race-conscious perspective recognizes that race serves as a marker of countless other dynamic variables and that structural racism, rather than race, compromises the health of racially oppressed individuals. This perspective offers a historical and theoretical context for the current debates regarding the use of race in clinical algorithms, clinical and epidemiologic perspectives on "risk," and future directions for research and policy interventions that combat color-evasive racism and follow the principles of race-conscious medicine.

The limits of fair medical imaging AI in real-world generalization

As artificial intelligence (AI) rapidly approaches human-level performance in medical imaging, it is crucial that it does not exacerbate or propagate healthcare disparities. Previous research established AI's capacity to infer demographic data from chest X-rays, leading to a key concern: do models using demographic shortcuts have unfair predictions across subpopulations? In this study, we conducted a thorough investigation into the extent to which medical AI uses demographic encodings, focusing on potential fairness discrepancies within both in-distribution training sets and external test sets. Our analysis covers three key medical imaging disciplines-radiology, dermatology and ophthalmology-and incorporates data from six global chest X-ray datasets. We confirm that medical imaging AI leverages demographic shortcuts in disease classification. Although correcting shortcuts algorithmically effectively addresses fairness gaps to create 'locally optimal' models within the original data distribution, this optimality is not true in new test settings. Surprisingly, we found that models with less encoding of demographic attributes are often most 'globally optimal', exhibiting better fairness during model evaluation in new test environments. Our work establishes best practices for medical imaging models that maintain their performance and fairness in deployments beyond their initial training contexts, underscoring critical considerations for AI clinical deployments across populations and sites.

A Clinician's Guide to Understanding Bias in Critical Clinical Prediction Models

This narrative review focuses on the role of clinical prediction models in supporting informed decision-making in critical care, emphasizing their 2 forms: traditional scores and artificial intelligence (AI)-based models. Acknowledging the potential for both types to embed biases, the authors underscore the importance of critical appraisal to increase our trust in models. The authors outline recommendations and critical care examples to manage risk of bias in AI models. The authors advocate for enhanced interdisciplinary training for clinicians, who are encouraged to explore various resources (books, journals, news Web sites, and social media) and events (Datathons) to deepen their understanding of risk of bias.

A Systematic Review of Artificial Intelligence Models for Time-to-Event Outcome Applied in Cardiovascular Disease Risk Prediction

Artificial intelligence (AI) based predictive models for early detection of cardiovascular disease (CVD) risk are increasingly being utilised. However, AI based risk prediction models that account for right-censored data have been overlooked. This systematic review (PROSPERO protocol CRD42023492655) includes 33 studies that utilised machine learning (ML) and deep learning (DL) models for survival outcome in CVD prediction. We provided details on the employed ML and DL models, eXplainable AI (XAI) techniques, and type of included variables, with a focus on social determinants of health (SDoH) and gender-stratification. Approximately half of the studies were published in 2023 with the majority from the United States. Random Survival Forest (RSF), Survival Gradient Boosting models, and Penalised Cox models were the most frequently employed ML models. DeepSurv was the most frequently employed DL model. DL models were better at predicting CVD outcomes than ML models. Permutation-based feature importance and Shapley values were the most utilised XAI methods for explaining AI models. Moreover, only one in five studies performed gender-stratification analysis and very few incorporate the wide range of SDoH factors in their prediction model. In conclusion, the evidence indicates that RSF and DeepSurv models are currently the optimal models for predicting CVD outcomes. This study also highlights the better predictive ability of DL survival models, compared to ML models. Future research should ensure the appropriate interpretation of AI models, accounting for SDoH, and gender stratification, as gender plays a significant role in CVD occurrence.

Uses and limitations of artificial intelligence for oncology

Modern artificial intelligence (AI) tools built on high-dimensional patient data are reshaping oncology care, helping to improve goal-concordant care, decrease cancer mortality rates, and increase workflow efficiency and scope of care. However, data-related concerns and human biases that seep into algorithms during development and post-deployment phases affect performance in real-world settings, limiting the utility and safety of AI technology in oncology clinics. To this end, the authors review the current potential and limitations of predictive AI for cancer diagnosis and prognostication as well as of generative AI, specifically modern chatbots, which interfaces with patients and clinicians. They conclude the review with a discussion on ongoing challenges and regulatory opportunities in the field.

Development of a Computable Phenotype for Prehospital Pediatric Asthma Encounters

INTRODUCTION

Asthma exacerbations are a common cause of pediatric Emergency Medical Services (EMS) encounters. Accordingly, prehospital management of pediatric asthma exacerbations has been designated an EMS research priority. However, accurate identification of pediatric asthma exacerbations from the prehospital record is nuanced and difficult due to the heterogeneity of asthma symptoms, especially in children. Therefore, this study's objective was to develop a prehospital-specific pediatric asthma computable phenotype (CP) that could accurately identify prehospital encounters for pediatric asthma exacerbations.

METHODS

This is a retrospective observational study of patient encounters for ages 2-18 years from the ESO Data Collaborative between 2018 and 2021. We modified two existing rule-based pediatric asthma CPs and created three new CPs (one rule-based and two machine learning-based). Two pediatric emergency medicine physicians independently reviewed encounters to assign labels of asthma exacerbation or not. Taking that labeled encounter data, a 50/50 train/test split was used to create training and test sets from the labeled data. A 90/10 split was used to create a small validation set from the training set. We used specificity, sensitivity, positive predictive value (PPV), negative predictive value (NPV) and macro F1 to compare performance across all CP models.

RESULTS

After applying the inclusion and exclusion criteria, 24,283 patient encounters remained. The machine-learning models exhibited the best performance for the identification of pediatric asthma exacerbations. A multi-layer perceptron-based model had the best performance in all metrics, with an F1 score of 0.95, specificity of 1.00, sensitivity of 0.91, negative predictive value of 0.98, and positive predictive value of 1.00.

CONCLUSION

We modified existing and developed new pediatric asthma CPs to retrospectively identify prehospital pediatric asthma exacerbation encounters. We found that machine learning-based models greatly outperformed rule-based models. Given the high performance of the machine-learning models, the development and application of machine learning-based CPs for other conditions and diseases could help accelerate EMS research and ultimately enhance clinical care by accurately identifying patients with conditions of interest.

Risk scores, label bias, and everything but the kitchen sink

In designing risk assessment algorithms, many scholars promote a "kitchen sink" approach, reasoning that more information yields more accurate predictions. We show, however, that this rationale often fails when algorithms are trained to predict a proxy of the true outcome, for instance, predicting arrest as a proxy for criminal behavior. With this "label bias," one should exclude a feature if its correlation with the proxy and its correlation with the true outcome have opposite signs, conditional on the other model features. This criterion is often satisfied when a feature is weakly correlated with the true outcome, and, additionally, that feature and the true outcome are both direct causes of the proxy outcome. For example, criminal behavior and geography may be weakly correlated and, due to patterns of police deployment, direct causes of one's arrest record-suggesting that excluding geography in criminal risk assessment will weaken an algorithm's performance in predicting arrest but will improve its capacity to predict actual crime.

Prediction of non emergent acute care utilization and cost among patients receiving Medicaid

Patients receiving Medicaid often experience social risk factors for poor health and limited access to primary care, leading to high utilization of emergency departments and hospitals (acute care) for non-emergent conditions. As programs proactively outreach Medicaid patients to offer primary care, they rely on risk models historically limited by poor-quality data. Following initiatives to improve data quality and collect data on social risk, we tested alternative widely-debated strategies to improve Medicaid risk models. Among a sample of 10 million patients receiving Medicaid from 26 states and Washington DC, the best-performing model tripled the probability of prospectively identifying at-risk patients versus a standard model (sensitivity 11.3% [95% CI 10.5, 12.1%] vs 3.4% [95% CI 3.0, 4.0%]), without increasing "false positives" that reduce efficiency of outreach (specificity 99.8% [95% CI 99.6, 99.9%] vs 99.5% [95% CI 99.4, 99.7%]), and with a ~ tenfold improved coefficient of determination when predicting costs (R2: 0.195-0.412 among population subgroups vs 0.022-0.050). Our best-performing model also reversed the lower sensitivity of risk prediction for Black versus White patients, a bias present in the standard cost-based model. Our results demonstrate a modeling approach to substantially improve risk prediction performance and equity for patients receiving Medicaid.