Avoidable Serum Potassium Testing in the Cardiac ICU: Development and Testing of a Machine-Learning Model

The Pediatric Data Science and Analytics Subgroup of the Pediatric Acute Lung Injury and Sepsis Investigators Network: Use of Supervised Machine Learning Applications in Pediatric Critical Care Medicine Research

OBJECTIVE

Perform a scoping review of supervised machine learning in pediatric critical care to identify published applications, methodologies, and implementation frequency to inform best practices for the development, validation, and reporting of predictive models in pediatric critical care.

DESIGN

Scoping review and expert opinion.

SETTING

We queried CINAHL Plus with Full Text (EBSCO), Cochrane Library (Wiley), Embase (Elsevier), Ovid Medline, and PubMed for articles published between 2000 and 2022 related to machine learning concepts and pediatric critical illness. Articles were excluded if the majority of patients were adults or neonates, if unsupervised machine learning was the primary methodology, or if information related to the development, validation, and/or implementation of the model was not reported. Article selection and data extraction were performed using dual review in the Covidence tool, with discrepancies resolved by consensus.

SUBJECTS

Articles reporting on the development, validation, or implementation of supervised machine learning models in the field of pediatric critical care medicine.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Of 5075 identified studies, 141 articles were included. Studies were primarily (57%) performed at a single site. The majority took place in the United States (70%). Most were retrospective observational cohort studies. More than three-quarters of the articles were published between 2018 and 2022. The most common algorithms included logistic regression and random forest. Predicted events were most commonly death, transfer to ICU, and sepsis. Only 14% of articles reported external validation, and only a single model was implemented at publication. Reporting of validation methods, performance assessments, and implementation varied widely. Follow-up with authors suggests that implementation remains uncommon after model publication.

CONCLUSIONS

Publication of supervised machine learning models to address clinical challenges in pediatric critical care medicine has increased dramatically in the last 5 years. While these approaches have the potential to benefit children with critical illness, the literature demonstrates incomplete reporting, absence of external validation, and infrequent clinical implementation.

Prediction of Impending Central Line Associated Bloodstream Infections in Hospitalized Cardiac Patients: Development and Testing of a Machine-Learning Model

BACKGROUND

While modeling of central line-associated blood stream infection (CLABSI) risk factors is common, models that predict an impending CLABSI in real time are lacking.

AIM

To build a prediction model which identifies patients who will develop a CLABSI in the ensuing 24 hours.

METHODS

We collected variables potentially related to infection identification in all patients admitted to the cardiac ICU or cardiac ward at Boston Children's Hospital in whom a central venous catheter (CVC) was in place between January 2010 and August 2020, excluding those with a diagnosis of bacterial endocarditis. We created models predicting whether a patient would develop CLABSI in the ensuing 24 hours. We assessed model performance based on area under the curve (AUC), sensitivity, and false positive rate (FPR) of models run on an independent testing set (40%).

FINDINGS

104,035 patient-days and 139,662 line-days corresponding to 7,468 unique patients were included in the analysis. There were 399 positive blood cultures (0.38%), most commonly with Staphylococcus aureus (23% of infections). Major predictors included a prior history of infection, elevated maximum heart rate, elevated maximum temperature, elevated C-reactive protein, exposure to parenteral nutrition, and use of alteplase for CVC clearance. The model identified 25% of positive cultures with an FPR of 0.11% (AUC = 0.82).

CONCLUSIONS

A machine learning model can be used to predict 25% of patients with impending CLABSI with only 1.1/1,000 of these predictions being incorrect. Once prospectively validated, this tool may allow for early treatment or prevention.