"A catalyst for action": Factors for implementing clinical risk prediction models of infection in home care settings

Predictive analytics for early detection of hospital-acquired complications: An artificial intelligence approach

BACKGROUND

Hospital-acquired complications (HACs) have an adverse impact on patient recovery by impeding their path to full recovery and increasing healthcare costs.

OBJECTIVE

The aim of this study was to create a HAC risk prediction machine learning (ML) framework using hospital administrative data collections within North Metropolitan Health Service (NMHS), Western Australia.

METHOD

A retrospective cohort study was performed among 64,315 patients between July 2020 to June 2022 to develop an automated ML framework by inputting HAC and the healthcare site to obtain site-specific predictive algorithms for patients admitted to the hospital in NMHS. Univariate analysis was used for initial feature screening for 270 variables. Of these, 77 variables had significant relationship with any HAC. After excluding non-contemporaneous data, 37 variables were included in developing the ML framework based on logistic regression (LR), decision tree (DT) and random forest (RF) models to predict occurrence of four specific HACs: delirium, aspiration pneumonia, pneumonia and urinary tract infection.

RESULTS

All models exhibited similar performance with area under the curve scores around 0.90 for both training and testing datasets. For sensitivity, DT and RF exceeded LR performance while on average, false positives were lowest for LR-based models. Patient's length of stay, Charlson Index, operation length and intensive care unit stay were common predictors.

CONCLUSION

Integrating ML-based risk detection systems into clinical workflows can potentially enhance patient safety and optimise resource allocation. LR-based models exhibited best performance.

IMPLICATIONS

We have successfully developed a "real-time" risk prediction model, where patient risk scores are calculated and reviewed daily.

Implementation and clinical utility of a Computer-Aided Risk Score for Mortality (CARM): a qualitative study

OBJECTIVES

The Computer-Aided Risk Score for Mortality (CARM) estimates the risk of in-hospital mortality following acute admission to the hospital by automatically amalgamating physiological measures, blood tests, gender, age and COVID-19 status. Our aims were to implement the score with a small group of practitioners and understand their first-hand experience of interacting with the score in situ.

DESIGN

Pilot implementation evaluation study involving qualitative interviews.

SETTING

This study was conducted in one of the two National Health Service hospital trusts in the North of England in which the score was developed.

PARTICIPANTS

Medical, older person and ICU/anaesthetic consultants and specialist grade registrars (n=116) and critical outreach nurses (n=7) were given access to CARM. Nine interviews were conducted in total, with eight doctors and one critical care outreach nurse.

INTERVENTIONS

Participants were given access to the CARM score, visible after login to the patients' electronic record, along with information about the development and intended use of the score.

RESULTS

Four themes and 14 subthemes emerged from reflexive thematic analysis: (1) current use (including support or challenge clinical judgement and decision making, communicating risk of mortality and professional curiosity); (2) barriers and facilitators to use (including litigation, resource needs, perception of the evidence base, strengths and limitations), (3) implementation support needs (including roll-out and integration, access, training and education); and (4) recommendations for development (including presentation and functionality and potential additional data). Barriers and facilitators to use, and recommendations for development featured highly across most interviews.

CONCLUSION

Our in situ evaluation of the pilot implementation of CARM demonstrated its scope in supporting clinical decision making and communicating risk of mortality between clinical colleagues and with service users. It suggested to us barriers to implementation of the score. Our findings may support those seeking to develop, implement or improve the adoption of risk scores.

Prediction of Poststroke Urinary Tract Infection Risk in Immobile Patients Using Machine Learning: a observational cohort study

BACKGROUND

Urinary tract infection (UTI) is one of major nosocomial infections significantly affecting the outcomes of immobile stroke patients. Previous studies have identified several risk factors, but it's still challenging to accurately estimate personal UTI risk.

OBJECTIVES

We aimed to develop predictive models for UTI risk identification for immobile stroke patients.

METHODS

Research data were collected from our previous multi-centre study. Derivation cohort included 3982 immobile stroke patients collected from November 1, 2015 to June 30, 2016; external validation cohort included 3837 patients collected from November 1, 2016 to July 30, 2017. 6 machine learning models and an ensemble learning model were derived based on 80% of derivation cohort and effectiveness was evaluated with the remaining 20%. We used Shapley additive explanation values to determine feature importance and examine the clinical significance of prediction models.

RESULTS

2.59% (103/3982) patients were diagnosed with UTI in derivation cohort, 1.38% (53/3837) in external cohort. The ensemble learning model performed the best in area under the receiver operating characteristic (ROC) curve in internal validation (82.2%); second best in external validation (80.8%). In addition, the ensemble learning model performed the best sensitivity in both internal and external validation sets (80.9% and 81.1%, respectively). We also identified seven UTI risk factors (pneumonia, glucocorticoid use, female sex, mixed cerebrovascular disease, increased age, prolonged length of stay, and duration of catheterization).

CONCLUSIONS

Our ensemble learning model demonstrated promising performance. Future work should continue to develop a more concise scoring tool based on machine learning models and prospectively examining the model in practical use, thus improving clinical outcomes.