1
|
Heneghan JA, Walker SB, Fawcett A, Bennett TD, Dziorny AC, Sanchez-Pinto LN, Farris RW, Winter MC, Badke C, Martin B, Brown SR, McCrory MC, Ness-Cochinwala M, Rogerson C, Baloglu O, Harwayne-Gidansky I, Hudkins MR, Kamaleswaran R, Gangadharan S, Tripathi S, Mendonca EA, Markovitz BP, Mayampurath A, Spaeder MC. The Pediatric Data Science and Analytics Subgroup of the Pediatric Acute Lung Injury and Sepsis Investigators Network: Use of Supervised Machine Learning Applications in Pediatric Critical Care Medicine Research. Pediatr Crit Care Med 2024; 25:364-374. [PMID: 38059732 PMCID: PMC10994770 DOI: 10.1097/pcc.0000000000003425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/08/2023]
Abstract
OBJECTIVE Perform a scoping review of supervised machine learning in pediatric critical care to identify published applications, methodologies, and implementation frequency to inform best practices for the development, validation, and reporting of predictive models in pediatric critical care. DESIGN Scoping review and expert opinion. SETTING We queried CINAHL Plus with Full Text (EBSCO), Cochrane Library (Wiley), Embase (Elsevier), Ovid Medline, and PubMed for articles published between 2000 and 2022 related to machine learning concepts and pediatric critical illness. Articles were excluded if the majority of patients were adults or neonates, if unsupervised machine learning was the primary methodology, or if information related to the development, validation, and/or implementation of the model was not reported. Article selection and data extraction were performed using dual review in the Covidence tool, with discrepancies resolved by consensus. SUBJECTS Articles reporting on the development, validation, or implementation of supervised machine learning models in the field of pediatric critical care medicine. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS Of 5075 identified studies, 141 articles were included. Studies were primarily (57%) performed at a single site. The majority took place in the United States (70%). Most were retrospective observational cohort studies. More than three-quarters of the articles were published between 2018 and 2022. The most common algorithms included logistic regression and random forest. Predicted events were most commonly death, transfer to ICU, and sepsis. Only 14% of articles reported external validation, and only a single model was implemented at publication. Reporting of validation methods, performance assessments, and implementation varied widely. Follow-up with authors suggests that implementation remains uncommon after model publication. CONCLUSIONS Publication of supervised machine learning models to address clinical challenges in pediatric critical care medicine has increased dramatically in the last 5 years. While these approaches have the potential to benefit children with critical illness, the literature demonstrates incomplete reporting, absence of external validation, and infrequent clinical implementation.
Collapse
Affiliation(s)
- Julia A. Heneghan
- Division of Pediatric Critical Care, University of Minnesota Masonic Children’s Hospital; Minneapolis, MN
| | - Sarah B. Walker
- Department of Pediatrics (Critical Care), Northwestern University Feinberg School of Medicine and Ann & Robert H. Lurie Children’s Hospital of Chicago; Chicago, IL
| | - Andrea Fawcett
- Department of Clinical and Organizational Development; Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL
| | - Tellen D. Bennett
- Departments of Biomedical Informatics and Pediatrics (Critical Care Medicine), University of Colorado School of Medicine; Aurora, CO
| | - Adam C. Dziorny
- Department of Pediatrics, University of Rochester; Rochester, NY
| | - L. Nelson Sanchez-Pinto
- Department of Pediatrics (Critical Care) and Preventive Medicine (Health & Biomedical Informatics), Northwestern University Feinberg School of Medicine and Ann & Robert H. Lurie Children’s Hospital of Chicago; Chicago, IL
| | - Reid W.D. Farris
- Department of Pediatrics, University of Washington and Seattle Children’s Hospital; Seattle, WA
| | - Meredith C. Winter
- Department of Anesthesiology Critical Care Medicine, Children’s Hospital Los Angeles and Keck School of Medicine, University of Southern California; Los Angeles, CA
| | - Colleen Badke
- Department of Pediatrics (Critical Care), Northwestern University Feinberg School of Medicine and Ann & Robert H. Lurie Children’s Hospital of Chicago; Chicago, IL
| | - Blake Martin
- Departments of Biomedical Informatics and Pediatrics (Critical Care Medicine), University of Colorado School of Medicine; Aurora, CO
| | - Stephanie R. Brown
- Section of Pediatric Critical Care, Oklahoma Children’s Hospital and Department of Pediatrics, University of Oklahoma Health Sciences Center, Oklahoma City, OK
| | - Michael C. McCrory
- Department of Anesthesiology, Wake Forest University School of Medicine; Winston Salem, NC
| | | | - Colin Rogerson
- Division of Critical Care, Department of Pediatrics, Indiana University; Indianapolis, IN
| | - Orkun Baloglu
- Pediatric Critical Care Medicine and Pediatric Cardiology, Cleveland Clinic Children’s Center for Artificial Intelligence (C4AI), Cleveland Clinic; Cleveland, OH
| | | | - Matthew R. Hudkins
- Division of Pediatric Critical Care, Department of Pediatrics, Oregon Health & Science University; Portland, OR
| | - Rishikesan Kamaleswaran
- Departments of Biomedical Informatics and Pediatrics, Emory University School of Medicine; Department of Biomedical Engineering, Georgia Institute of Technology; Atlanta, GA
| | - Sandeep Gangadharan
- Department of Pediatrics, Mount Sinai Icahn School of Medicine; New York, NY
| | - Sandeep Tripathi
- Department of Pediatrics. University of Illinois College of Medicine at Peoria/OSF HealthCare, Children’s Hospital of Illinois; Peoria, IL
| | - Eneida A. Mendonca
- Division of Biomedical Informatics, Department of Pediatrics, Cincinnati Children’s Hospital Medical Center and University of Cincinnati; Cincinnati, OH
| | - Barry P. Markovitz
- Division of Pediatric Critical Care, Department of Pediatrics, University of Utah Spencer F Eccles School of Medicine, Intermountain Primary Children’s Hospital; Salt Lake City, UT
| | - Anoop Mayampurath
- Department of Biostatistics & Medical Informatics, University of Wisconsin-Madison; Madison, WI
| | - Michael C. Spaeder
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA
| |
Collapse
|
2
|
Bonello K, Emani S, Sorensen A, Shaw L, Godsay M, Delgado M, Sperotto F, Santillana M, Kheir JN. Prediction of Impending Central Line Associated Bloodstream Infections in Hospitalized Cardiac Patients: Development and Testing of a Machine-Learning Model. J Hosp Infect 2022; 127:44-50. [PMID: 35738317 DOI: 10.1016/j.jhin.2022.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2022] [Revised: 06/06/2022] [Accepted: 06/06/2022] [Indexed: 10/18/2022]
Abstract
BACKGROUND While modeling of central line-associated blood stream infection (CLABSI) risk factors is common, models that predict an impending CLABSI in real time are lacking. AIM To build a prediction model which identifies patients who will develop a CLABSI in the ensuing 24 hours. METHODS We collected variables potentially related to infection identification in all patients admitted to the cardiac ICU or cardiac ward at Boston Children's Hospital in whom a central venous catheter (CVC) was in place between January 2010 and August 2020, excluding those with a diagnosis of bacterial endocarditis. We created models predicting whether a patient would develop CLABSI in the ensuing 24 hours. We assessed model performance based on area under the curve (AUC), sensitivity, and false positive rate (FPR) of models run on an independent testing set (40%). FINDINGS 104,035 patient-days and 139,662 line-days corresponding to 7,468 unique patients were included in the analysis. There were 399 positive blood cultures (0.38%), most commonly with Staphylococcus aureus (23% of infections). Major predictors included a prior history of infection, elevated maximum heart rate, elevated maximum temperature, elevated C-reactive protein, exposure to parenteral nutrition, and use of alteplase for CVC clearance. The model identified 25% of positive cultures with an FPR of 0.11% (AUC = 0.82). CONCLUSIONS A machine learning model can be used to predict 25% of patients with impending CLABSI with only 1.1/1,000 of these predictions being incorrect. Once prospectively validated, this tool may allow for early treatment or prevention.
Collapse
Affiliation(s)
- Kristin Bonello
- Department of Cardiology, Boston Children's Hospital; Department of Paediatrics, Harvard Medical School, Boston, Massachusetts
| | - Sivaram Emani
- Department of Cardiology, Boston Children's Hospital; Department of Paediatrics, Harvard Medical School, Boston, Massachusetts
| | | | - Lauren Shaw
- Department of Cardiology, Boston Children's Hospital
| | | | | | - Francesca Sperotto
- Department of Cardiology, Boston Children's Hospital; Department of Paediatrics, Harvard Medical School, Boston, Massachusetts; Paediatric Cardiac Intensive Care Unit, Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Mauricio Santillana
- Harvard Institute for Applied Computational Science, Harvard University, Cambridge, Massachusetts; Computational Health Informatics Program, Boston Children's Hospital
| | - John N Kheir
- Department of Cardiology, Boston Children's Hospital; Department of Paediatrics, Harvard Medical School, Boston, Massachusetts.
| |
Collapse
|