Prediction of risk of acquiring urinary tract infection during hospital stay based on machine-learning: A retrospective cohort study

Machine learning evaluation of inequities and disparities associated with nurse sensitive indicator safety events

PURPOSE

To use machine learning to examine health equity and clinical outcomes in patients who experienced a nurse sensitive indicator (NSI) event, defined as a fall, a hospital-acquired pressure injury (HAPI) or a hospital-acquired infection (HAI).

DESIGN

This was a retrospective observational study from a single academic hospital over six calendar years (2016-2021). Machine learning was used to examine patients with an NSI compared to those without.

METHODS

Inclusion criteria: all adult inpatient admissions (2016-2021). Three approaches were used to analyze the NSI group compared to the No-NSI group. In the univariate analysis, descriptive statistics, and absolute standardized differences (ASDs) were employed to compare the demographics and clinical variables of patients who experienced a NSI and those who did not experience any NSIs. For the multivariate analysis, a light grading boosting machine (LightGBM) model was utilized to comprehensively examine the relationships associated with the development of an NSI. Lastly, a simulation study was conducted to quantify the strength of associations obtained from the machine learning model.

RESULTS

From 163,507 admissions, 4643 (2.8%) were associated with at least one NSI. The mean, standard deviation (SD) age was 59.5 (18.2) years, males comprised 82,397 (50.4%). Non-Hispanic White 84,760 (51.8%), non-Hispanic Black 8703 (5.3%), non-Hispanic Asian 23,368 (14.3%), non-Hispanic Other 14,284 (8.7%), and Hispanic 30,271 (18.5%). Race and ethnicity alone were not associated with occurrence of an NSI. The NSI group had a statistically significant longer length of stay (LOS), longer intensive care unit (ICU) LOS, and was more likely to have an emergency admission compared to the group without an NSI. The simulation study results demonstrated that likelihood of NSI was higher in patients admitted under the major diagnostic categories (MDC) associated with circulatory, digestive, kidney/urinary tract, nervous, and infectious and parasitic disease diagnoses.

CONCLUSION

In this study, race/ethnicity was not associated with the risk of an NSI event. The risk of an NSI event was associated with emergency admission, longer LOS, longer ICU-LOS and certain MDCs (circulatory, digestive, kidney/urinary, nervous, infectious, and parasitic diagnoses).

CLINICAL RELEVANCE

Machine learning methodologies provide a new mechanism to investigate NSI events through the lens of health equity/disparity. Understanding which patients are at higher risk for adverse outcomes can help hospitals improve nursing care and prevent NSI injury and harm.

Clinically explainable machine learning models for early identification of patients at risk of hospital-acquired urinary tract infection

BACKGROUND

Machine learning (ML) models for early identification of patients at risk of hospital-acquired urinary tract infection (HA-UTI) may enable timely and targeted preventive and therapeutic strategies. However, clinicians are often challenged in the interpretation of the predictive outcomes provided by the ML models, which often reach different performances.

AIM

To train ML models for predicting patients at risk of HA-UTI using available data from electronic health records at the time of hospital admission. This study focused on the performance of different ML models and clinical explainability.

METHODS

This retrospective study investigated patient data representing 138,560 hospital admissions in the North Denmark Region from 1st January 2017 to 31st December 2018. Fifty-one health sociodemographic and clinical features were extracted as the full dataset, and χ2 test and expert knowledge were used for feature selection, resulting in two reduced datasets. Seven different ML models were trained and compared between the three datasets. The SHapley Additive exPlanation (SHAP) method was used to support population- and patient-level explainability.

FINDINGS

The best-performing ML model was the neural network model based on the full dataset, with an area under the curve (AUC) of 0.758. The neural network model was also the best-performing ML model based on the reduced datasets, with an AUC of 0.746. Clinical explainability was demonstrated with a SHAP summary and forceplot.

CONCLUSION

Within 24 h of hospital admission, the ML models were able to identify patients at risk of developing HA-UTI, providing new opportunities to develop efficient strategies for the prevention of HA-UTI. SHAP was used to demonstrate how risk predictions can be explained at individual patient level and for the patient population in general.

A step forward in the diagnosis of urinary tract infections: from machine learning to clinical practice

Objectives

Urinary tract infections (UTIs) are common infections within the Emergency Department (ED), causing increased laboratory workloads and unnecessary antibiotics prescriptions. The aim of this study was to improve UTI diagnostics in clinical practice by application of machine learning (ML) models for real-time UTI prediction.

Methods

In a retrospective study, patient information and outcomes from Emergency Department patients, with positive and negative culture results, were used to design models - 'Random Forest' and 'Neural Network' - for the prediction of UTIs. The performance of these predictive models was validated in a cross-sectional study. In a quasi-experimental study, the impact of UTI risk assessment was investigated by evaluating changes in the behaviour of clinicians, measuring changes in antibiotic prescriptions and urine culture requests.

Results

First, we trained and tested two different predictive models with 8692 cases. Second, we investigated the performance of the predictive models in clinical practice with 962 cases (Area under the curve was between 0.81 to 0.88). The best performance was the combination of both models. Finally, the assessment of the risk for UTIs was implemented into clinical practice and allowed for the reduction of unnecessary urine cultures and antibiotic prescriptions for patients with a low risk of UTI, as well as targeted diagnostics and treatment for patients with a high risk of UTI.

Conclusion

The combination of modern urinalysis diagnostic technologies with digital health solutions can help to further improve UTI diagnostics with positive impact on laboratory workloads and antimicrobial stewardship.

Advancing Patient Safety: The Future of Artificial Intelligence in Mitigating Healthcare-Associated Infections: A Systematic Review

BACKGROUND

Healthcare-associated infections are infections that patients acquire during hospitalization or while receiving healthcare in other facilities. They represent the most frequent negative outcome in healthcare, can be entirely prevented, and pose a burden in terms of financial and human costs. With the development of new AI and ML algorithms, hospitals could develop new and automated surveillance and prevention models for HAIs, leading to improved patient safety. The aim of this review is to systematically retrieve, collect, and summarize all available information on the application and impact of AI in HAI surveillance and/or prevention.

METHODS

We conducted a systematic review of the literature using PubMed and Scopus to find articles related to the implementation of artificial intelligence in the surveillance and/or prevention of HAIs.

RESULTS

We identified a total of 218 articles, of which only 35 were included in the review. Most studies were conducted in the US (n = 10, 28.6%) and China (n = 5; 14.3%) and were published between 2021 and 2023 (26 articles, 74.3%) with an increasing trend over time. Most focused on the development of ML algorithms for the identification/prevention of surgical site infections (n = 18; 51%), followed by HAIs in general (n = 9; 26%), hospital-acquired urinary tract infections (n = 5; 9%), and healthcare-associated pneumonia (n = 3; 9%). Only one study focused on the proper use of personal protective equipment (PPE) and included healthcare workers as the study population. Overall, the trend indicates that several AI/ML models can effectively assist clinicians in everyday decisions, by identifying HAIs early or preventing them through personalized risk factors with good performance. However, only a few studies have reported an actual implementation of these models, which proved highly successful. In one case, manual workload was reduced by nearly 85%, while another study observed a decrease in the local hospital's HAI incidence from 1.31% to 0.58%.

CONCLUSIONS

AI has significant potential to improve the prevention, diagnosis, and management of healthcare-associated infections, offering benefits such as increased accuracy, reduced workloads, and cost savings. Although some AI applications have already been tested and validated, adoption in healthcare is hindered by barriers such as high implementation costs, technological limitations, and resistance from healthcare workers. Overcoming these challenges could allow AI to be more widely and cost-effectively integrated, ultimately improving patient care and infection management.

Artificial intelligence and prescription of antibiotic therapy: present and future

INTRODUCTION

In the past few years, the use of artificial intelligence in healthcare has grown exponentially. Prescription of antibiotics is not exempt from its rapid diffusion, and various machine learning (ML) techniques, from logistic regression to deep neural networks and large language models, have been explored in the literature to support decisions regarding antibiotic prescription.

AREAS COVERED

In this narrative review, we discuss promises and challenges of the application of ML-based clinical decision support systems (ML-CDSSs) for antibiotic prescription. A search was conducted in PubMed up to April 2024.

EXPERT OPINION

Prescribing antibiotics is a complex process involving various dynamic phases. In each of these phases, the support of ML-CDSSs has shown the potential, and also the actual ability in some studies, to favorably impacting relevant clinical outcomes. Nonetheless, before widely exploiting this massive potential, there are still crucial challenges ahead that are being intensively investigated, pertaining to the transparency of training data, the definition of the sufficient degree of prediction explanations when predictions are obtained through black box models, and the legal and ethical framework for decision responsibility whenever an antibiotic prescription is supported by ML-CDSSs.

Impact of Insurance on Readmission Rates, Healthcare Expenditures, and Length of Hospital Stay among Patients with Chronic Ambulatory Care Sensitive Conditions in China

Background: The disparities in healthcare access due to varying insurance coverage significantly impact hospital outcomes, yet what is unclear is the role of insurance in providing care once the patient is in the hospital for a preventable admission, particularly in a weak gatekeeping environment. This study aimed to investigate the association between insurance types and readmission rates, healthcare expenditures, and length of hospital stay among patients with chronic ambulatory care sensitive conditions (ACSCs) in China. Methods: This retrospective observational study utilized hospitalization data collected from the Nanhai District, Foshan City, between 2016 and 2020. Generalized linear models (GLMs) were employed to analyze the relationship between medical insurance types and readmission rates, lengths of hospital stay, total medical expenses, out-of-pocket expenses, and insurance-covered expenses. Results: A total of 185,384 records were included. Among these, the participants covered by urban employee basic medical insurance (UEBMI) with 44,415 records and urban and rural resident basic medical insurance (URRBMI) with 80,752 records generally experienced more favorable outcomes compared to self-pay patients. Specifically, they had lower readmission rates (OR = 0.57, 95% CI: 0.36 to 0.90; OR = 0.59, 95% CI: 0.42 to 0.84) and reduced out-of-pocket expenses (β = -0.54, 95% CI: -0.94 to -0.14; β = -0.41, 95% CI: -0.78 to -0.05). However, they also experienced slightly longer lengths of hospital stay (IRR = 1.08, 95% CI: 1.03 to 1.14; IRR = 1.11, 95% CI: 1.04 to 1.18) and higher total medical expenses (β = 0.26, 95% CI: 0.09 to 0.44; β = 0.25, 95% CI: 0.10 to 0.40). Conclusions: This study found that different types of health insurance were associated with varying clinical outcomes among patients with chronic ambulatory care sensitive conditions (ACSCs) in China. Since the hospitalization of these patients was initially avoidable, disparities in readmission rates, lengths of hospital stay, and medical expenses among avoidable inpatient cases exacerbated the health gap between different insurance types. Addressing the disparities among different types of insurance can help reduce unplanned hospitalizations and promote health equity.

Artificial intelligence and machine learning applications in urinary tract infections identification and prediction: a systematic review and meta-analysis

BACKGROUND

Urinary tract infections (UTIs) have been one of the most common bacterial infections in clinical practice worldwide. Artificial intelligence (AI) and machine learning (ML) based algorithms have been increasingly applied in UTI case identification and prediction. However, the overall performance of AI/ML algorithms in identifying and predicting UTI has not been evaluated. The purpose of this paper is to quantitatively evaluate the application value of AI/ML in identifying and predicting UTI cases.

METHODS

MEDLINE, EMBASE, Web of Science, and PubMed databases were systematically searched for articles published up to December 31, 2023. Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) and Prediction Model Risk of Bias Assessment Tool (PROBAST) were used to assess the risk of bias. Study characteristics and detailed algorithm information were extracted. Pooled sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were synthesized using a bivariate mix-effects model. Meta-regression and subgroup analysis were conducted to test the source of heterogeneity.

RESULTS

In total, 11 studies with 14 AI/ML models were included in the final meta-analysis. The overall pooled AUC was 0.89 (95%CI 0.86-0.92). Additionally, the pooled Sen, Spe, PLR, NLR, and DOR were 0.78 (95%CI 0.71-0.84), 0.89 (95%CI 0.83-0.93), 6.99 (95%CI 4.38-11.14), 0.25 (95%CI 0.18-0.34) and 28.07 (95%CI 14.27-55.20), respectively. The results of meta-regression suggested that reference standard definitions might be the source of heterogeneity.

CONCLUSION

AI/ML algorithms appear to be promising to help clinicians detect and identify patients at high risk of UTIs. However, further studies are demanded to evaluate the application value of AI/ML more thoroughly.

Predictive analytics for early detection of hospital-acquired complications: An artificial intelligence approach

BACKGROUND

Hospital-acquired complications (HACs) have an adverse impact on patient recovery by impeding their path to full recovery and increasing healthcare costs.

OBJECTIVE

The aim of this study was to create a HAC risk prediction machine learning (ML) framework using hospital administrative data collections within North Metropolitan Health Service (NMHS), Western Australia.

METHOD

A retrospective cohort study was performed among 64,315 patients between July 2020 to June 2022 to develop an automated ML framework by inputting HAC and the healthcare site to obtain site-specific predictive algorithms for patients admitted to the hospital in NMHS. Univariate analysis was used for initial feature screening for 270 variables. Of these, 77 variables had significant relationship with any HAC. After excluding non-contemporaneous data, 37 variables were included in developing the ML framework based on logistic regression (LR), decision tree (DT) and random forest (RF) models to predict occurrence of four specific HACs: delirium, aspiration pneumonia, pneumonia and urinary tract infection.

RESULTS

All models exhibited similar performance with area under the curve scores around 0.90 for both training and testing datasets. For sensitivity, DT and RF exceeded LR performance while on average, false positives were lowest for LR-based models. Patient's length of stay, Charlson Index, operation length and intensive care unit stay were common predictors.

CONCLUSION

Integrating ML-based risk detection systems into clinical workflows can potentially enhance patient safety and optimise resource allocation. LR-based models exhibited best performance.

IMPLICATIONS

We have successfully developed a "real-time" risk prediction model, where patient risk scores are calculated and reviewed daily.

Risk score models for urinary tract infection hospitalization

Annually, urinary tract infections (UTIs) affect over a hundred million people worldwide. Early detection of high-risk individuals can help prevent hospitalization for UTIs, which imposes significant economic and social burden on patients and caregivers. We present two methods to generate risk score models for UTI hospitalization. We utilize a sample of patients from the insurance claims data provided by the Centers for Medicare and Medicaid Services to develop and validate the proposed methods. Our dataset encompasses a wide range of features, such as demographics, medical history, and healthcare utilization of the patients along with provider quality metrics and community-based metrics. The proposed methods scale and round the coefficients of an underlying logistic regression model to create scoring tables. We present computational experiments to evaluate the prediction performance of both models. We also discuss different features of these models with respect to their impact on interpretability. Our findings emphasize the effectiveness of risk score models as practical tools for identifying high-risk patients and provide a quantitative assessment of the significance of various risk factors in UTI hospitalizations such as admission to ICU in the last 3 months, cognitive disorders and low inpatient, outpatient and carrier costs in the last 6 months.

Automating detection of diagnostic error of infectious diseases using machine learning

Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.

A study on the risk stratification for patients within 24 hours of admission for risk of hospital-acquired urinary tract infection using Bayesian network models

Early identification of patients at risk of hospital-acquired urinary tract infections (HA-UTI) enables the initiation of timely targeted preventive and therapeutic strategies. Machine learning (ML) models have shown great potential for this purpose. However, existing ML models in infection control have demonstrated poor ability to support explainability, which challenges the interpretation of the result in clinical practice, limiting the adaption of the ML models into a daily clinical routine. In this study, we developed Bayesian Network (BN) models to enable explainable assessment within 24 h of admission for risk of HA-UTI. Our dataset contained 138,250 unique hospital admissions. We included data on admission details, demographics, lifestyle factors, comorbidities, vital parameters, laboratory results, and urinary catheter. Models developed from a reduced set of five features were characterized by transparency compared to models developed from a full set of 50 features. The expert-based clinical BN model over the reduced feature space showed the highest performance (area under the curve = 0.746) compared to the naïve- and tree-augmented-naïve BN models. Moreover, models developed from expert-based knowledge were characterized by enhanced explainability.

Smart Diagnosis of Urinary Tract Infections: is Artificial Intelligence the Fast-Lane Solution?

PURPOSE OF REVIEW

Artificial intelligence (AI) can significantly improve physicians' workflow when examining patients with UTI. However, most contemporary reviews are focused on examining the usage of AI with a restricted quantity of data, analyzing only a subset of AI algorithms, or performing narrative work without analyzing all dedicated studies. Given the preceding, the goal of this work was to conduct a mini-review to determine the current state of AI-based systems as a support in UTI diagnosis.

RECENT FINDINGS

There are sufficient publications to comprehend the potential applications of artificial intelligence in the diagnosis of UTIs. Existing research in this field, in general, publishes performance metrics that are exemplary. However, upon closer inspection, many of the available publications are burdened with flaws associated with the improper use of artificial intelligence, such as the use of a small number of samples, their lack of heterogeneity, and the absence of external validation. AI-based models cannot be classified as full-fledged physician assistants in diagnosing UTIs due to the fact that these limitations and flaws represent only a portion of all potential obstacles. Instead, such studies should be evaluated as exploratory, with a focus on the importance of future work that complies with all rules governing the use of AI. AI algorithms have demonstrated their potential for UTI diagnosis. However, further studies utilizing large, heterogeneous, prospectively collected datasets, as well as external validations, are required to define the actual clinical workflow value of artificial intelligence.

Development and validation of a parsimonious prediction model for positive urine cultures in outpatient visits

Urine culture is often considered the gold standard for detecting the presence of bacteria in the urine. Since culture is expensive and often requires 24-48 hours, clinicians often rely on urine dipstick test, which is considerably cheaper than culture and provides instant results. Despite its ease of use, urine dipstick test may lack sensitivity and specificity. In this paper, we use a real-world dataset consisting of 17,572 outpatient encounters who underwent urine cultures, collected between 2015 and 2021 at a large multi-specialty hospital in Abu Dhabi, United Arab Emirates. We develop and evaluate a simple parsimonious prediction model for positive urine cultures based on a minimal input set of ten features selected from the patient's presenting vital signs, history, and dipstick results. In a test set of 5,339 encounters, the parsimonious model achieves an area under the receiver operating characteristic curve (AUROC) of 0.828 (95% CI: 0.810-0.844) for predicting a bacterial count ≥ 105 CFU/ml, outperforming a model that uses dipstick features only that achieves an AUROC of 0.786 (95% CI: 0.769-0.806). Our proposed model can be easily deployed at point-of-care, highlighting its value in improving the efficiency of clinical workflows, especially in low-resource settings.

Data Science Implementation Trends in Nursing Practice: A Review of the 2021 Literature

OBJECTIVES

The goal of this work was to provide a review of the implementation of data science-driven applications focused on structural or outcome-related nurse-sensitive indicators in the literature in 2021. By conducting this review, we aim to inform readers of trends in the nursing indicators being addressed, the patient populations and settings of focus, and lessons and challenges identified during the implementation of these tools.

METHODS

We conducted a rigorous descriptive review of the literature to identify relevant research published in 2021. We extracted data on model development, implementation-related strategies and measures, lessons learned, and challenges and stakeholder involvement. We also assessed whether reports of data science application implementations currently follow the guidelines of the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by AI (DECIDE-AI) framework.

RESULTS

Of 4,943 articles found in PubMed (NLM) and CINAHL (EBSCOhost), 11 were included in the final review and data extraction. Systems leveraging data science were developed for adult patient populations and were primarily deployed in hospital settings. The clinical domains targeted included mortality/deterioration, utilization/resource allocation, and hospital-acquired infections/COVID-19. The composition of development teams and types of stakeholders involved varied. Research teams more frequently reported on implementation methods than implementation results. Most studies provided lessons learned that could help inform future implementations of data science systems in health care.

CONCLUSION

In 2021, very few studies report on the implementation of data science-driven applications focused on structural- or outcome-related nurse-sensitive indicators. This gap in the sharing of implementation strategies needs to be addressed in order for these systems to be successfully adopted in health care settings.