Mining for equitable health: Assessing the impact of missing data in electronic health records

Race adjustments in clinical algorithms can help correct for racial disparities in data quality

Despite ethical and historical arguments for removing race from clinical algorithms, the consequences of removal remain unclear. Here, we highlight a largely undiscussed consideration in this debate: varying data quality of input features across race groups. For example, family history of cancer is an essential predictor in cancer risk prediction algorithms but is less reliably documented for Black participants and may therefore be less predictive of cancer outcomes. Using data from the Southern Community Cohort Study, we assessed whether race adjustments could allow risk prediction models to capture varying data quality by race, focusing on colorectal cancer risk prediction. We analyzed 77,836 adults with no history of colorectal cancer at baseline. The predictive value of self-reported family history was greater for White participants than for Black participants. We compared two cancer risk prediction algorithms-a race-blind algorithm which included standard colorectal cancer risk factors but not race, and a race-adjusted algorithm which additionally included race. Relative to the race-blind algorithm, the race-adjusted algorithm improved predictive performance, as measured by goodness of fit in a likelihood ratio test (P-value: <0.001) and area under the receiving operating characteristic curve among Black participants (P-value: 0.006). Because the race-blind algorithm underpredicted risk for Black participants, the race-adjusted algorithm increased the fraction of Black participants among the predicted high-risk group, potentially increasing access to screening. More broadly, this study shows that race adjustments may be beneficial when the data quality of key predictors in clinical algorithms differs by race group.

Centralized and Federated Models for the Analysis of Clinical Data

The progress of precision medicine research hinges on the gathering and analysis of extensive and diverse clinical datasets. With the continued expansion of modalities, scales, and sources of clinical datasets, it becomes imperative to devise methods for aggregating information from these varied sources to achieve a comprehensive understanding of diseases. In this review, we describe two important approaches for the analysis of diverse clinical datasets, namely the centralized model and federated model. We compare and contrast the strengths and weaknesses inherent in each model and present recent progress in methodologies and their associated challenges. Finally, we present an outlook on the opportunities that both models hold for the future analysis of clinical data.

Incorporating informatively collected laboratory data from EHR in clinical prediction models

BACKGROUND

Electronic Health Records (EHR) are widely used to develop clinical prediction models (CPMs). However, one of the challenges is that there is often a degree of informative missing data. For example, laboratory measures are typically taken when a clinician is concerned that there is a need. When data are the so-called Not Missing at Random (NMAR), analytic strategies based on other missingness mechanisms are inappropriate. In this work, we seek to compare the impact of different strategies for handling missing data on CPMs performance.

METHODS

We considered a predictive model for rapid inpatient deterioration as an exemplar implementation. This model incorporated twelve laboratory measures with varying levels of missingness. Five labs had missingness rate levels around 50%, and the other seven had missingness levels around 90%. We included them based on the belief that their missingness status can be highly informational for the prediction. In our study, we explicitly compared the various missing data strategies: mean imputation, normal-value imputation, conditional imputation, categorical encoding, and missingness embeddings. Some of these were also combined with the last observation carried forward (LOCF). We implemented logistic LASSO regression, multilayer perceptron (MLP), and long short-term memory (LSTM) models as the downstream classifiers. We compared the AUROC of testing data and used bootstrapping to construct 95% confidence intervals.

RESULTS

We had 105,198 inpatient encounters, with 4.7% having experienced the deterioration outcome of interest. LSTM models generally outperformed other cross-sectional models, where embedding approaches and categorical encoding yielded the best results. For the cross-sectional models, normal-value imputation with LOCF generated the best results.

CONCLUSION

Strategies that accounted for the possibility of NMAR missing data yielded better model performance than those did not. The embedding method had an advantage as it did not require prior clinical knowledge. Using LOCF could enhance the performance of cross-sectional models but have countereffects in LSTM models.

A reference architecture for personal health data spaces using decentralized content-addressable storage networks

Introduction

This paper addresses the dilemmas of accessibility, comprehensiveness, and ownership related to health data. To resolve these dilemmas, we propose and justify a novel, globally scalable reference architecture for a Personal Health Data Space (PHDS). This architecture leverages decentralized content-addressable storage (DCAS) networks, ensuring that the data subject retains complete control and ownership of their personal health data. In today's globalized world, where people are increasingly mobile for work and leisure, healthcare is transitioning from episodic symptom-based treatment toward continuity of care. The main aims of this are patient engagement, illness prevention, and active and healthy longevity. This shift, along with the secondary use of health data for societal benefit, has intensified the challenges associated with health data accessibility, comprehensiveness, and ownership.

Method

The study is structured around four health data use case scenarios from the Estonian National Health Information System (EHIS): primary medical use, medical emergency use, secondary use, and personal use. We analyze these use cases from the perspectives of accessibility, comprehensiveness, and ownership. Additionally, we examine the security, privacy, and interoperability aspects of health data.

Results

The proposed architectural solution allows individuals to consolidate all their health data into a unified Personal Health Record (PHR). This data can come from various healthcare institutions, mobile applications, medical devices for home use, and personal health notes.

Discussions

The comprehensive PHR can then be shared with healthcare providers in a semantically interoperable manner, regardless of their location or the information systems they use. Furthermore, individuals maintain the autonomy to share, sell, or donate their anonymous or pseudonymous health data for secondary use with different systems worldwide. The proposed reference architecture aligns with the principles of the European Health Data Space (EHDS) initiative, enhancing health data management by providing a secure, cost-effective, and sustainable solution.

To weight or not to weight? The effect of selection bias in 3 large electronic health record-linked biobanks and recommendations for practice

OBJECTIVES

To develop recommendations regarding the use of weights to reduce selection bias for commonly performed analyses using electronic health record (EHR)-linked biobank data.

MATERIALS AND METHODS

We mapped diagnosis (ICD code) data to standardized phecodes from 3 EHR-linked biobanks with varying recruitment strategies: All of Us (AOU; n = 244 071), Michigan Genomics Initiative (MGI; n = 81 243), and UK Biobank (UKB; n = 401 167). Using 2019 National Health Interview Survey data, we constructed selection weights for AOU and MGI to represent the US adult population more. We used weights previously developed for UKB to represent the UKB-eligible population. We conducted 4 common analyses comparing unweighted and weighted results.

RESULTS

For AOU and MGI, estimated phecode prevalences decreased after weighting (weighted-unweighted median phecode prevalence ratio [MPR]: 0.82 and 0.61), while UKB estimates increased (MPR: 1.06). Weighting minimally impacted latent phenome dimensionality estimation. Comparing weighted versus unweighted phenome-wide association study for colorectal cancer, the strongest associations remained unaltered, with considerable overlap in significant hits. Weighting affected the estimated log-odds ratio for sex and colorectal cancer to align more closely with national registry-based estimates.

DISCUSSION

Weighting had a limited impact on dimensionality estimation and large-scale hypothesis testing but impacted prevalence and association estimation. When interested in estimating effect size, specific signals from untargeted association analyses should be followed up by weighted analysis.

CONCLUSION

EHR-linked biobanks should report recruitment and selection mechanisms and provide selection weights with defined target populations. Researchers should consider their intended estimands, specify source and target populations, and weight EHR-linked biobank analyses accordingly.

Machine learning integrative approaches to advance computational immunology

The study of immunology, traditionally reliant on proteomics to evaluate individual immune cells, has been revolutionized by single-cell RNA sequencing. Computational immunologists play a crucial role in analysing these datasets, moving beyond traditional protein marker identification to encompass a more detailed view of cellular phenotypes and their functional roles. Recent technological advancements allow the simultaneous measurements of multiple cellular components-transcriptome, proteome, chromatin, epigenetic modifications and metabolites-within single cells, including in spatial contexts within tissues. This has led to the generation of complex multiscale datasets that can include multimodal measurements from the same cells or a mix of paired and unpaired modalities. Modern machine learning (ML) techniques allow for the integration of multiple "omics" data without the need for extensive independent modelling of each modality. This review focuses on recent advancements in ML integrative approaches applied to immunological studies. We highlight the importance of these methods in creating a unified representation of multiscale data collections, particularly for single-cell and spatial profiling technologies. Finally, we discuss the challenges of these holistic approaches and how they will be instrumental in the development of a common coordinate framework for multiscale studies, thereby accelerating research and enabling discoveries in the computational immunology field.

Filling the gaps: leveraging large language models for temporal harmonization of clinical text across multiple medical visits for clinical prediction

Electronic health records offer great promise for early disease detection, treatment evaluation, information discovery, and other important facets of precision health. Clinical notes, in particular, may contain nuanced information about a patient's condition, treatment plans, and history that structured data may not capture. As a result, and with advancements in natural language processing, clinical notes have been increasingly used in supervised prediction models. To predict long-term outcomes such as chronic disease and mortality, it is often advantageous to leverage data occurring at multiple time points in a patient's history. However, these data are often collected at irregular time intervals and varying frequencies, thus posing an analytical challenge. Here, we propose the use of large language models (LLMs) for robust temporal harmonization of clinical notes across multiple visits. We compare multiple state-of-the-art LLMs in their ability to generate useful information during time gaps, and evaluate performance in supervised deep learning models for clinical prediction.

Pragmatic strategies to address health disparities along the continuum of care in chronic liver disease

Racial, ethnic, and socioeconomic disparities exist in the prevalence and natural history of chronic liver disease, access to care, and clinical outcomes. Solutions to improve health equity range widely, from digital health tools to policy changes. The current review outlines the disparities along the chronic liver disease health care continuum from screening and diagnosis to the management of cirrhosis and considerations of pre-liver and post-liver transplantation. Using a health equity research and implementation science framework, we offer pragmatic strategies to address barriers to implementing high-quality equitable care for patients with chronic liver disease.

Participant flow diagrams for health equity in AI

Selection bias can arise through many aspects of a study, including recruitment, inclusion/exclusion criteria, input-level exclusion and outcome-level exclusion, and often reflects the underrepresentation of populations historically disadvantaged in medical research. The effects of selection bias can be further amplified when non-representative samples are used in artificial intelligence (AI) and machine learning (ML) applications to construct clinical algorithms. Building on the "Data Cards" initiative for transparency in AI research, we advocate for the addition of a participant flow diagram for AI studies detailing relevant sociodemographic and/or clinical characteristics of excluded participants across study phases, with the goal of identifying potential algorithmic biases before their clinical implementation. We include both a model for this flow diagram as well as a brief case study explaining how it could be implemented in practice. Through standardized reporting of participant flow diagrams, we aim to better identify potential inequities embedded in AI applications, facilitating more reliable and equitable clinical algorithms.

Algorithmic Individual Fairness and Healthcare: A Scoping Review

Objective

Statistical and artificial intelligence algorithms are increasingly being developed for use in healthcare. These algorithms may reflect biases that magnify disparities in clinical care, and there is a growing need for understanding how algorithmic biases can be mitigated in pursuit of algorithmic fairness. Individual fairness in algorithms constrains algorithms to the notion that "similar individuals should be treated similarly." We conducted a scoping review on algorithmic individual fairness to understand the current state of research in the metrics and methods developed to achieve individual fairness and its applications in healthcare.

Methods

We searched three databases, PubMed, ACM Digital Library, and IEEE Xplore, for algorithmic individual fairness metrics, algorithmic bias mitigation, and healthcare applications. Our search was restricted to articles published between January 2013 and September 2023. We identified 1,886 articles through database searches and manually identified one article from which we included 30 articles in the review. Data from the selected articles were extracted, and the findings were synthesized.

Results

Based on the 30 articles in the review, we identified several themes, including philosophical underpinnings of fairness, individual fairness metrics, mitigation methods for achieving individual fairness, implications of achieving individual fairness on group fairness and vice versa, fairness metrics that combined individual fairness and group fairness, software for measuring and optimizing individual fairness, and applications of individual fairness in healthcare.

Conclusion

While there has been significant work on algorithmic individual fairness in recent years, the definition, use, and study of individual fairness remain in their infancy, especially in healthcare. Future research is needed to apply and evaluate individual fairness in healthcare comprehensively.

Race, ethnicity, ancestry, and aspects that impact HLA data and matching for transplant

Race, ethnicity, and ancestry are terms that are often misinterpreted and/or used interchangeably. There is lack of consensus in the scientific literature on the definition of these terms and insufficient guidelines on the proper classification, collection, and application of this data in the scientific community. However, defining groups for human populations is crucial for multiple healthcare applications and clinical research. Some examples impacted by population classification include HLA matching for stem-cell or solid organ transplant, identifying disease associations and/or adverse drug reactions, defining social determinants of health, understanding diverse representation in research studies, and identifying potential biases. This article describes aspects of race, ethnicity and ancestry information that impact the stem-cell or solid organ transplantation field with particular focus on HLA data collected from donors and recipients by donor registries or transplant centers.

Leveraging informative missing data to learn about acute respiratory distress syndrome and mortality in long-term hospitalized COVID-19 patients throughout the years of the pandemic

Electronic health records (EHRs) contain a wealth of information that can be used to further precision health. One particular data element in EHRs that is not only under-utilized but oftentimes unaccounted for is missing data. However, missingness can provide valuable information about comorbidities and best practices for monitoring patients, which could save lives and reduce burden on the healthcare system. We characterize patterns of missing data in laboratory measurements collected at the University of Pennsylvania Hospital System from long-term COVID-19 patients and focus on the changes in these patterns between 2020 and 2021. We investigate how these patterns are associated with comorbidities such as acute respiratory distress syndrome (ARDS), and 90-day mortality in ARDS patients. This work displays how knowledge and experience can change the way clinicians and hospitals manage a novel disease. It can also provide insight into best practices when it comes to patient monitoring to improve outcomes.

Electronic health records and stratified psychiatry: bridge to precision treatment?

The use of a stratified psychiatry approach that combines electronic health records (EHR) data with machine learning (ML) is one potentially fruitful path toward rapidly improving precision treatment in clinical practice. This strategy, however, requires confronting pervasive methodological flaws as well as deficiencies in transparency and reporting in the current conduct of ML-based studies for treatment prediction. EHR data shares many of the same data quality issues as other types of data used in ML prediction, plus some unique challenges. To fully leverage EHR data's power for patient stratification, increased attention to data quality and collection of patient-reported outcome data is needed.

Dissecting Clinical Features of COVID-19 in a Cohort of 21,312 Acute Care Patients

COVID-19 has resulted in over 645 million hospitalization and 7 million deaths globally. However, many questions still remain about clinical complications in COVID-19 and if these complications changed with different circulating SARS-CoV-2 strains. We analyzed a 2.5-year retrospective cohort of 47,063 encounters for 21,312 acute care patients at five Central Texas hospitals and define distinct trajectory groups (TGs) with latent class mixed modeling, based on the World Health Organization COVID-19 Ordinal Scale. Using this TG framework, we evaluated the association of demographics, diagnoses, vitals, labs, imaging, consultations, and medications with COVID-19 severity and broad clinical outcomes. Patients within 6 distinct TGs differed in manifestations of multi-organ disease and multiple clinical factors. The proportion of mild patients increased over time, particularly during Omicron waves. Age separated mild and fatal patients, though did not distinguish patients with severe versus critical disease. Male and Hispanic/Latino demographics were associated with more severe/critical TGs. More severe patients had a higher rate of neuropsychiatric diagnoses, consultations, and brain imaging, which did not change significantly in severe patients across SARS-CoV-2 variant waves. More severely affected patients also demonstrated an immunological signature of high neutrophils and immature granulocytes, and low lymphocytes and monocytes. Interestingly, low albumin was one of the best lab predictors of COVID-19 severity in association with higher malnutrition in severe/critical patients, raising concern of nutritional insufficiency influencing COVID-19 outcomes. Despite this, only a small fraction of severe/critical patients had nutritional labs checked (pre-albumin, thiamine, Vitamin D, B vitamins) or received targeted interventions to address nutritional deficiencies such as vitamin replacement. Our findings underscore the significant link between COVID-19 severity, neuropsychiatric complications, and nutritional insufficiency as key risk factors of COVID-19 outcomes and raise the question of the need for more widespread early assessment of patients' neurological, psychiatric, and nutritional status in acute care settings to help identify those at risk of severe disease outcomes.

Data Missingness Reporting and Use of Methods to Address It in Critical Care Cohort Studies

IMPORTANCE

Failure to recognize and address data missingness in cohort studies may lead to biased results. Although Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines advocate data missingness reporting, the degree to which missingness is reported and addressed in the critical care literature remains unclear.

OBJECTIVES

To review published ICU cohort studies to characterize data missingness reporting and the use of methods to address it.

DESIGN SETTING AND PARTICIPANTS

We searched the 2022 table of contents of 29 critical care/critical care subspecialty journals having a 2021 impact factor greater than or equal to 3 to identify published prospective clinical or retrospective database cohort studies enrolling greater than or equal to 100 patients.

MAIN OUTCOMES AND MEASURES

In duplicate, two trained researchers conducted a manuscript/supplemental material PDF word search for "missing*" and extracted study type, patient age, ICU type, sample size, missingness reporting, and the use of methods to address it.

RESULTS

A total of 656 studies were reviewed. Of the 334 of 656 (50.9%) studies mentioning missingness, missingness was reported for greater than or equal to 1 variable in 234 (70.1%) and it exceeded 5% for at least one variable in 160 (47.9%). Among the 334 studies mentioning missingness, 88 (26.3%) used exclusion criteria, 36 (10.8%) used complete-case analysis, and 164 (49.1%) used a formal method to avoid missingness. In these 164 studies, imputation only was used in 100 (61.0%), an analytic strategy only in 24 (14.6%), and both in 40 (24.4%). Only missingness greater than 5% (in ≥ 1 variable) was independently associated with greater use of a missingness method (adjusted odds ratio 2.91; 95% CI, 1.85-4.60). Among 140 studies using imputation, multiple imputation was used in 87 studies (62.1%) and simple imputation in 49 studies (35.0%). For the 64 studies using an analytic method, 12 studies (18.8%) assigned missingness as an unknown category, whereas sensitivity analysis was used in 47 studies (73.4%).

CONCLUSIONS AND RELEVANCE

Among published critical care cohort studies, only half mentioned result missingness, one-third reported actual missingness and only one-quarter used a method to manage missingness. Educational strategies to promote missingness reporting and resolution methods are required.

Sources of Disparities in Surveillance Mammography Performance and Risk-Guided Recommendations for Supplemental Breast Imaging: A Simulation Study

BACKGROUND

Surveillance mammography is recommended for all women with a history of breast cancer. Risk-guided surveillance incorporating advanced imaging modalities based on individual risk of a second cancer could improve cancer detection. However, personalized surveillance may also amplify disparities.

METHODS

In simulated populations using inputs from the Breast Cancer Surveillance Consortium (BCSC), we investigated race- and ethnicity-based disparities. Disparities were decomposed into those due to primary breast cancer and treatment characteristics, social determinants of health (SDOH) and differential error in second cancer ascertainment by modeling populations with or without variation across race and ethnicity in the distribution of these characteristics. We estimated effects of disparities on mammography performance and supplemental imaging recommendations stratified by race and ethnicity.

RESULTS

In simulated cohorts based on 65,446 BCSC surveillance mammograms, when only cancer characteristics varied by race and ethnicity, mammograms for Black women had lower sensitivity compared with the overall population (64.1% vs. 71.1%). Differences between Black women and the overall population were larger when both cancer characteristics and SDOH varied by race and ethnicity (53.8% vs. 71.1%). Basing supplemental imaging recommendations on high predicted second cancer risk resulted in less frequent recommendations for Hispanic (6.7%) and Asian/Pacific Islander women (6.4%) compared with the overall population (10.0%).

CONCLUSIONS

Variation in cancer characteristics and SDOH led to disparities in surveillance mammography performance and recommendations for supplemental imaging.

IMPACT

Risk-guided surveillance imaging may exacerbate disparities. Decision-makers should consider implications for equity in cancer outcomes resulting from implementing risk-guided screening programs. See related In the Spotlight, p. 1479.

Weighted Lottery to Equitably Allocate Scarce Supply of COVID-19 Monoclonal Antibody

Importance

Equitable allocation of scarce medications is an important health policy goal. There are few data about attempts to achieve equitable allocation in the community setting.

Objective

To describe the development and use of a weighted lottery to allocate a scarce supply of tixagevimab with cilgavimab as preexposure prophylaxis to COVID-19 for immunocompromised individuals and examine whether this promoted equitable allocation to disadvantaged populations.

Design, Setting, and Participants

This quality improvement study analyzed a weighted lottery process from December 8, 2021, to February 23, 2022, that assigned twice the odds of drug allocation of 450 tixagevimab with cilgavimab doses to individuals residing in highly disadvantaged neighborhoods according to the US Area Deprivation Index (ADI) in a 35-hospital system in Pennsylvania, New York, and Maryland. In all, 10 834 individuals were eligible for the lottery. Weighted lottery results were compared with 10 000 simulated unweighted lotteries in the same cohort performed after drug allocation occurred.

Main Outcomes

Proportion of individuals from disadvantaged neighborhoods and Black individuals who were allocated and received tixagevimab with cilgavimab.

Results

Of the 10 834 eligible individuals, 1800 (16.6%) were from disadvantaged neighborhoods and 767 (7.1%) were Black. Mean (SD) age was 62.9 (18.8) years, and 5471 (50.5%) were women. A higher proportion of individuals from disadvantaged neighborhoods was allocated the drug in the ADI-weighted lottery compared with the unweighted lottery (29.1% vs 16.6%; P < .001). The proportion of Black individuals allocated the drug was greater in the weighted lottery (9.1% vs 7.1%; P < .001). Among the 450 individuals allocated tixagevimab with cilgavimab in the ADI-weighted lottery, similar proportions of individuals from disadvantaged neighborhoods accepted the allocation and received the drug compared with those from other neighborhoods (27.5% vs 27.9%; P = .93). However, Black individuals allocated the drug were less likely to receive it compared with White individuals (3 of 41 [7.3%] vs 118 of 402 [29.4%]; P = .003).

Conclusions and Relevance

The findings of this quality improvement study suggest an ADI-weighted lottery process to allocate scarce resources is feasible in a large health system and resulted in more drug allocation to and receipt of drug by individuals who reside in disadvantaged neighborhoods. Although the ADI-weighted lottery also resulted in more drug allocation to Black individuals compared with an unweighted process, they were less likely to accept allocation and receive it compared with White individuals. Further strategies are needed to ensure that Black individuals receive scarce medications allocated.

Informative missingness: What can we learn from patterns in missing laboratory data in the electronic health record?

BACKGROUND

In electronic health records, patterns of missing laboratory test results could capture patients' course of disease as well as ​​reflect clinician's concerns or worries for possible conditions. These patterns are often understudied and overlooked. This study aims to identify informative patterns of missingness among laboratory data collected across 15 healthcare system sites in three countries for COVID-19 inpatients.

METHODS

We collected and analyzed demographic, diagnosis, and laboratory data for 69,939 patients with positive COVID-19 PCR tests across three countries from 1 January 2020 through 30 September 2021. We analyzed missing laboratory measurements across sites, missingness stratification by demographic variables, temporal trends of missingness, correlations between labs based on missingness indicators over time, and clustering of groups of labs based on their missingness/ordering pattern.

RESULTS

With these analyses, we identified mapping issues faced in seven out of 15 sites. We also identified nuances in data collection and variable definition for the various sites. Temporal trend analyses may support the use of laboratory test result missingness patterns in identifying severe COVID-19 patients. Lastly, using missingness patterns, we determined relationships between various labs that reflect clinical behaviors.

CONCLUSION

In this work, we use computational approaches to relate missingness patterns to hospital treatment capacity and highlight the heterogeneity of looking at COVID-19 over time and at multiple sites, where there might be different phases, policies, etc. Changes in missingness could suggest a change in a patient's condition, and patterns of missingness among laboratory measurements could potentially identify clinical outcomes. This allows sites to consider missing data as informative to analyses and help researchers identify which sites are better poised to study particular questions.