Hospital-acquired Clostridioides difficile infection among patients at an urban safety-net hospital in Philadelphia: Demographics, neighborhood deprivation, and the transferability of national statistics

Generalizability of predictive models for Clostridioides difficile infection, severity and recurrence at an urban safety-net hospital

INTRODUCTION

Predictive models for Clostridioides difficile infection can identify high-risk patients and aid clinicians in preventing infection. Issues of generalizability regarding current predictive models have been acknowledged but, to the authors' knowledge, have never been quantified.

METHODS

C. difficile infection, severity and recurrence predictive models were created using multi-variate logistic regression through case-control sampling from an urban safety-net hospital. Models were validated using five-fold cross-validation, and inverse probability weights (IPW) based on two different catchment area definitions were used to improve external validity. Akaike Information Criterion (AIC), area under the receiver operating characteristic curve (AUROC), and sensitivity and specificity with bootstrapped confidence intervals (CI) were used to assess and compare model fit and performance.

RESULTS

Changes in performance before and after weighting were small across all models, although differences were more apparent after weighting the recurrence model (AUROC values of 0.78, 0.76 and 0.71 for the unweighted and two weighted models, respectively). Overall, the infection model performed the best (AUROC 0.82, 95% CI 0.78-0.85), followed by the recurrence model (AUROC 0.78, 95% CI 0.69-0.86) and then the severity model (AUROC 0.70, 95% CI 0.63-0.78).

CONCLUSIONS

The performance of the models after weighting did not change drastically, suggesting that the models predicting C. difficile infection, severity and recurrence may not be impacted by patient selection factors. However, other researchers may wish to consider addressing these catchment forces using IPW.

Bias correction models for electronic health records data in the presence of non-random sampling

Electronic health records (EHRs) contain rich clinical information for millions of patients and are increasingly used for public health research. However, non-random inclusion of subjects in EHRs can result in selection bias, with factors such as demographics, socioeconomic status, healthcare referral patterns, and underlying health status playing a role. While this issue has been well documented, little work has been done to develop or apply bias-correction methods, often due to the fact that most of these factors are unavailable in EHRs. To address this gap, we propose a series of Heckman type bias correction methods by incorporating social determinants of health selection covariates to model the EHR non-random sampling probability. Through simulations under various settings, we demonstrate the effectiveness of our proposed method in correcting biases in both the association coefficient and the outcome mean. Our method augments the utility of EHRs for public health inferences, as we show by estimating the prevalence of cardiovascular disease and its correlation with risk factors in the New York City network of EHRs.

A narrative review on the validity of electronic health record-based research in epidemiology

Electronic health records (EHRs) are widely used in epidemiological research, but the validity of the results is dependent upon the assumptions made about the healthcare system, the patient, and the provider. In this review, we identify four overarching challenges in using EHR-based data for epidemiological analysis, with a particular emphasis on threats to validity. These challenges include representativeness of the EHR to a target population, the availability and interpretability of clinical and non-clinical data, and missing data at both the variable and observation levels. Each challenge reveals layers of assumptions that the epidemiologist is required to make, from the point of patient entry into the healthcare system, to the provider documenting the results of the clinical exam and follow-up of the patient longitudinally; all with the potential to bias the results of analysis of these data. Understanding the extent of as well as remediating potential biases requires a variety of methodological approaches, from traditional sensitivity analyses and validation studies, to newer techniques such as natural language processing. Beyond methods to address these challenges, it will remain crucial for epidemiologists to engage with clinicians and informaticians at their institutions to ensure data quality and accessibility by forming multidisciplinary teams around specific research projects.