Machine learning models for early sepsis recognition in the neonatal intensive care unit using readily available electronic health record data

A Risk Prediction Model to Identify Newborns at Risk for Missing Early Childhood Vaccinations

BACKGROUND

Approximately 30% of US children aged 24 months have not received all recommended vaccines. This study aimed to develop a prediction model to identify newborns at high risk for missing early childhood vaccines.

METHODS

A retrospective cohort included 9080 infants born weighing ≥2000 g at an academic medical center between 2008 and 2013. Electronic medical record data were linked to vaccine data from the Washington State Immunization Information System. Risk models were constructed using derivation and validation samples. K-fold cross-validation identified risk factors for model inclusion based on alpha = 0.01. For each patient in the derivation set, the total number of weighted adverse risk factors was calculated and used to establish groups at low, medium, or high risk for undervaccination. Logistic regression evaluated the likelihood of not completing the 7-vaccine series by age 19 months. The final model was tested using the validation sample.

RESULTS

Overall, 53.6% failed to complete the 7-vaccine series by 19 months. Six risk factors were identified: race/ethnicity, maternal language, insurance status, birth hospitalization length of stay, medical service, and hepatitis B vaccine receipt. Likelihood of non-completion was greater in the high (77.1%; adjusted odds ratio [AOR] 5.6; 99% confidence interval [CI]: 4.2, 7.4) and medium (52.7%; AOR 1.9; 99% CI: 1.6, 2.2) vs low (38.7%) risk groups in the derivation sample. Similar results were observed in the validation sample.

CONCLUSIONS

Our prediction model using information readily available in birth hospitalization records consistently identified newborns at high risk for undervaccination. Early identification of high-risk families could be useful for initiating timely, tailored vaccine interventions.

Machine Learning Models for Classification of Cushing's Syndrome Using Retrospective Data

Accurate classification of Cushing's Syndrome (CS) plays a critical role in providing the early and correct diagnosis of CS that may facilitate treatment and improve patient outcomes. Diagnosis of CS is a complex process, which requires careful and concurrent interpretation of signs and symptoms, multiple biochemical test results, and findings of medical imaging by physicians with a high degree of specialty and knowledge to make correct judgments. In this article, we explore the state of the art machine learning algorithms to demonstrate their potential as a clinical decision support system to analyze and classify CS to facilitate the diagnosis, prognosis, and treatment of CS. Prominent algorithms are compared using nested cross-validation and various class comparison strategies including multiclass, one vs. all, and one vs. one binary classification. Our findings show that Random Forest (RF) algorithm is most suitable for the classification of CS. We demonstrate that the proposed approach can classify CS with an average accuracy of 92% and an average F1 score of 91.5%, depending on the class comparison strategy and selected features. RF-based one vs. all binary classification model achieves sensitivity of 97.6%, precision of 91.1%, and specificity of 87.1% to discriminate CS from non-CS on the test dataset. RF-based multiclass classification model achieves average per class sensitivity of 91.8%, average per class specificity of 97.1%, and average per class precision of 92.1% to classify different subtypes of CS on the test dataset. Clinical performance evaluation suggests that the developed models can help improve physicians' judgment in diagnosing CS.

Aiding clinical assessment of neonatal sepsis using hematological analyzer data with machine learning techniques

INTRODUCTION

Early diagnosis and antibiotic administration are essential for reducing sepsis morbidity and mortality; however, diagnosis remains difficult due to complex pathogenesis and presentation. We created a machine learning model for bacterial sepsis identification in the neonatal intensive care unit (NICU) using hematological analyzer data.

METHODS

Hematological analyzer data were gathered from NICU patients up to 48 hours prior to clinical evaluation for bacterial sepsis. Five models, Support Vector Machine, K-nearest-neighbors, Logistic Regression, Random Forest (RF), and Extreme Gradient boosting (XGBoost), were trained on 60 hematological and nine clinical variables for 2357 cases (1692 control, 665 septic). Clinical feature only models (nine variables) were additionally trained and compared with models including hematological variables. Feature importance was used to assess relative contributions of parameters to performance.

RESULTS

The three best performing models were RF, Logistic Regression, and XGBoost. RF achieved an average accuracy of 0.74, AUC-ROC of 0.73, Sensitivity of 0.38, and Specificity of 0.88. Logistic Regression achieved an average accuracy of 0.70, AUC-ROC of 0.74, Sensitivity of 0.62, and Specificity of 0.73. XGBoost achieved an average accuracy of 0.72, AUC-ROC of 0.71, Sensitivity of 0.40, and Specificity of 0.85. All models with hematological variables had significantly stronger performance than models trained on only clinical features. Neutrophil parameters had the highest average feature importance.

CONCLUSIONS

Machine learning models using hematological analyzer data can classify NICU patients as sepsis positive or negative with stronger performance compared to clinical feature only models. Hematological analyzer variables could augment current sepsis classification machine learning algorithms.

Development and validation of an early warning tool for sepsis and decompensation in children during emergency department triage

This study was designed to develop and validate an early warning system for sepsis based on a predictive model of critical decompensation. Data from the electronic medical records for 537,837 visits to a pediatric Emergency Department (ED) from March 2013 to December 2019 were collected. A multiclass stochastic gradient boosting model was built to identify early warning signs associated with death, severe sepsis, non-severe sepsis, and bacteremia. Model features included triage vital signs, previous diagnoses, medications, and healthcare utilizations within 6 months of the index ED visit. There were 483 patients who had severe sepsis and/or died, 1102 had non-severe sepsis, 1103 had positive bacteremia tests, and the remaining had none of the events. The most important predictors were age, heart rate, length of stay of previous hospitalizations, temperature, systolic blood pressure, and prior sepsis. The one-versus-all area under the receiver operator characteristic curve (AUROC) were 0.979 (0.967, 0.991), 0.990 (0.985, 0.995), 0.976 (0.972, 0.981), and 0.968 (0.962, 0.974) for death, severe sepsis, non-severe sepsis, and bacteremia without sepsis respectively. The multi-class macro average AUROC and area under the precision recall curve were 0.977 and 0.316 respectively. The study findings were used to develop an automated early warning decision tool for sepsis. Implementation of this model in pediatric EDs will allow sepsis-related critical decompensation to be predicted accurately after a few seconds of triage.

Early Detection of Late Onset Sepsis in Premature Infants Using Visibility Graph Analysis of Heart Rate Variability

OBJECTIVE

This study was designed to test the diagnostic value of visibility graph features derived from the heart rate time series to predict late onset sepsis (LOS) in preterm infants using machine learning.

METHODS

The heart rate variability (HRV) data was acquired from 49 premature newborns hospitalized in neonatal intensive care units (NICU). The LOS group consisted of patients who received more than five days of antibiotics, at least 72 hours after birth. The control group consisted of infants who did not receive antibiotics. HRV features in the days prior to the start of antibiotics (LOS group) or in a randomly selected period (control group) were compared against a baseline value calculated during a calibration period. After automatic feature selection, four machine learning algorithms were trained. All the tests were done using two variants of the feature set: one only included traditional HRV features, and the other additionally included visibility graph features. Performance was studied using area under the receiver operating characteristics curve (AUROC).

RESULTS

The best performance for detecting LOS was obtained with logistic regression, using the feature set including visibility graph features, with AUROC of 87.7% during the six hours preceding the start of antibiotics, and with predictive potential (AUROC above 70%) as early as 42 h before start of antibiotics.

CONCLUSION

These results demonstrate the usefulness of introducing visibility graph indexes in HRV analysis for sepsis prediction in newborns.

SIGNIFICANCE

The method proposed the possibility of non-invasive, real-time monitoring of risk of LOS in a NICU setting.

Clinician involvement in research on machine learning-based predictive clinical decision support for the hospital setting: A scoping review

OBJECTIVE

The study sought to describe the prevalence and nature of clinical expert involvement in the development, evaluation, and implementation of clinical decision support systems (CDSSs) that utilize machine learning to analyze electronic health record data to assist nurses and physicians in prognostic and treatment decision making (ie, predictive CDSSs) in the hospital.

MATERIALS AND METHODS

A systematic search of PubMed, CINAHL, and IEEE Xplore and hand-searching of relevant conference proceedings were conducted to identify eligible articles. Empirical studies of predictive CDSSs using electronic health record data for nurses or physicians in the hospital setting published in the last 5 years in peer-reviewed journals or conference proceedings were eligible for synthesis. Data from eligible studies regarding clinician involvement, stage in system design, predictive CDSS intention, and target clinician were charted and summarized.

RESULTS

Eighty studies met eligibility criteria. Clinical expert involvement was most prevalent at the beginning and late stages of system design. Most articles (95%) described developing and evaluating machine learning models, 28% of which described involving clinical experts, with nearly half functioning to verify the clinical correctness or relevance of the model (47%).

DISCUSSION

Involvement of clinical experts in predictive CDSS design should be explicitly reported in publications and evaluated for the potential to overcome predictive CDSS adoption challenges.

CONCLUSIONS

If present, clinical expert involvement is most prevalent when predictive CDSS specifications are made or when system implementations are evaluated. However, clinical experts are less prevalent in developmental stages to verify clinical correctness, select model features, preprocess data, or serve as a gold standard.

Temporal bias in case-control design: preventing reliable predictions of the future

One of the primary tools that researchers use to predict risk is the case-control study. We identify a flaw, temporal bias, that is specific to and uniquely associated with these studies that occurs when the study period is not representative of the data that clinicians have during the diagnostic process. Temporal bias acts to undermine the validity of predictions by over-emphasizing features close to the outcome of interest. We examine the impact of temporal bias across the medical literature, and highlight examples of exaggerated effect sizes, false-negative predictions, and replication failure. Given the ubiquity and practical advantages of case-control studies, we discuss strategies for estimating the influence of and preventing temporal bias where it exists.

Early Detection of Sepsis With Machine Learning Techniques: A Brief Clinical Perspective

Sepsis is a major cause of death worldwide. Over the past years, prediction of clinically relevant events through machine learning models has gained particular attention. In the present perspective, we provide a brief, clinician-oriented vision on the following relevant aspects concerning the use of machine learning predictive models for the early detection of sepsis in the daily practice: (i) the controversy of sepsis definition and its influence on the development of prediction models; (ii) the choice and availability of input features; (iii) the measure of the model performance, the output, and their usefulness in the clinical practice. The increasing involvement of artificial intelligence and machine learning in health care cannot be disregarded, despite important pitfalls that should be always carefully taken into consideration. In the long run, a rigorous multidisciplinary approach to enrich our understanding in the application of machine learning techniques for the early recognition of sepsis may show potential to augment medical decision-making when facing this heterogeneous and complex syndrome.

Data analytics and clinical feature ranking of medical records of patients with sepsis

Background

Sepsis is a life-threatening clinical condition that happens when the patient’s body has an excessive reaction to an infection, and should be treated in one hour. Due to the urgency of sepsis, doctors and physicians often do not have enough time to perform laboratory tests and analyses to help them forecast the consequences of the sepsis episode. In this context, machine learning can provide a fast computational prediction of sepsis severity, patient survival, and sequential organ failure by just analyzing the electronic health records of the patients. Also, machine learning can be employed to understand which features in the medical records are more predictive of sepsis severity, of patient survival, and of sequential organ failure in a fast and non-invasive way.

Dataset and methods

In this study, we analyzed a dataset of electronic health records of 364 patients collected between 2014 and 2016. The medical record of each patient has 29 clinical features, and includes a binary value for survival, a binary value for septic shock, and a numerical value for the sequential organ failure assessment (SOFA) score. We disjointly utilized each of these three factors as an independent target, and employed several machine learning methods to predict it (binary classifiers for survival and septic shock, and regression analysis for the SOFA score). Afterwards, we used a data mining approach to identify the most important dataset features in relation to each of the three targets separately, and compared these results with the results achieved through a standard biostatistics approach.

Results and conclusions

Our results showed that machine learning can be employed efficiently to predict septic shock, SOFA score, and survival of patients diagnoses with sepsis, from their electronic health records data. And regarding clinical feature ranking, our results showed that Random Forests feature selection identified several unexpected symptoms and clinical components as relevant for septic shock, SOFA score, and survival. These discoveries can help doctors and physicians in understanding and predicting septic shock. We made the analyzed dataset and our developed software code publicly available online.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13040-021-00235-0).

Clinical and vital sign changes associated with late-onset sepsis in very low birth weight infants at 3 NICUs

BACKGROUND

In premature infants, clinical changes frequently occur due to sepsis or non-infectious conditions, and distinguishing between these is challenging. Baseline risk factors, vital signs, and clinical signs guide decisions to culture and start antibiotics. We sought to compare heart rate (HR) and oxygenation (SpO2) patterns as well as baseline variables and clinical signs prompting sepsis work-ups ultimately determined to be late-onset sepsis (LOS) and sepsis ruled out (SRO).

METHODS

At three NICUs, we reviewed records of very low birth weight (VLBW) infants around their first sepsis work-up diagnosed as LOS or SRO. Clinical signs prompting the evaluation were determined from clinician documentation. HR-SpO2 data, when available, were analyzed for mean, standard deviation, skewness, kurtosis, and cross-correlation. We used LASSO and logistic regression to assess variable importance and associations with LOS compared to SRO.

RESULTS

We analyzed sepsis work-ups in 408 infants (173 LOS, 235 SRO). Compared to infants with SRO, those with LOS were of lower GA and BW, and more likely to have a central catheter and mechanical ventilation. Clinical signs cited more often in LOS included hypotension, acidosis, abdominal distension, lethargy, oliguria, and abnormal CBC or CRP(p <  0.05). HR-SpO2 data were available in 266 events. Cross-correlation HR-SpO2 before the event was associated with LOS after adjusting for GA, BW, and postnatal age. A model combining baseline, clinical and HR-SpO2 variables had AUC 0.821.

CONCLUSION

In VLBW infants at 3-NICUs, we describe the baseline, clinical, and HR-SpO2 variables associated with LOS versus SRO.

Application of AI and IoT in Clinical Medicine: Summary and Challenges

The application of artificial intelligence (AI) technology in the medical field has experienced a long history of development. In turn, some long-standing points and challenges in the medical field have also prompted diverse research teams to continue to explore AI in depth. With the development of advanced technologies such as the Internet of Things (IoT), cloud computing, big data, and 5G mobile networks, AI technology has been more widely adopted in the medical field. In addition, the in-depth integration of AI and IoT technology enables the gradual improvement of medical diagnosis and treatment capabilities so as to provide services to the public in a more effective way. In this work, we examine the technical basis of IoT, cloud computing, big data analysis and machine learning involved in clinical medicine, combined with concepts of specific algorithms such as activity recognition, behavior recognition, anomaly detection, assistant decision-making system, to describe the scenario-based applications of remote diagnosis and treatment collaboration, neonatal intensive care unit, cardiology intensive care unit, emergency first aid, venous thromboembolism, monitoring nursing, image-assisted diagnosis, etc. We also systematically summarize the application of AI and IoT in clinical medicine, analyze the main challenges thereof, and comment on the trends and future developments in this field.

Stratified Management for Bacterial Infections in Late Preterm and Term Neonates: Current Strategies and Future Opportunities Toward Precision Medicine

Bacterial infections remain a major cause of morbidity and mortality in the neonatal period. Therefore, many neonates, including late preterm and term neonates, are exposed to antibiotics in the first weeks of life. Data on the importance of inter-individual differences and disease signatures are accumulating. Differences that may potentially influence treatment requirement and success rate. However, currently, many neonates are treated following a "one size fits all" approach, based on general protocols and standard antibiotic treatment regimens. Precision medicine has emerged in the last years and is perceived as a new, holistic, way of stratifying patients based on large-scale data including patient characteristics and disease specific features. Specific to sepsis, differences in disease susceptibility, disease severity, immune response and pharmacokinetics and -dynamics can be used for the development of treatment algorithms helping clinicians decide when and how to treat a specific patient or a specific subpopulation. In this review, we highlight the current and future developments that could allow transition to a more precise manner of antibiotic treatment in late preterm and term neonates, and propose a research agenda toward precision medicine for neonatal bacterial infections.

Supervised Machine Learning Applied to Automate Flash and Prolonged Capillary Refill Detection by Pulse Oximetry

Objective

Develop an automated approach to detect flash (<1.0 s) or prolonged (>2.0 s) capillary refill time (CRT) that correlates with clinician judgment by applying several supervised machine learning (ML) techniques to pulse oximeter plethysmography data.

Materials and Methods

Data was collected in the Pediatric Intensive Care Unit (ICU), Cardiac ICU, Progressive Care Unit, and Operating Suites in a large academic children’s hospital. Ninety-nine children and 30 adults were enrolled in testing and validation cohorts, respectively. Patients had 5 paired CRT measurements by a modified pulse oximeter device and a clinician, generating 485 waveform pairs for model training. Supervised ML models using gradient boosting (XGBoost), logistic regression (LR), and support vector machines (SVMs) were developed to detect flash (<1 s) or prolonged CRT (≥2 s) using clinician CRT assessment as the reference standard. Models were compared using Area Under the Receiver Operating Curve (AUC) and precision-recall curve (positive predictive value vs. sensitivity) analysis. The best performing model was externally validated with 90 measurement pairs from adult patients. Feature importance analysis was performed to identify key waveform characteristics.

Results

For flash CRT, XGBoost had a greater mean AUC (0.79, 95% CI 0.75–0.83) than logistic regression (0.77, 0.71–0.82) and SVM (0.72, 0.67–0.76) models. For prolonged CRT, XGBoost had a greater mean AUC (0.77, 0.72–0.82) than logistic regression (0.73, 0.68–0.78) and SVM (0.75, 0.70–0.79) models. Pairwise testing showed statistically significant improved performance comparing XGBoost and SVM; all other pairwise model comparisons did not reach statistical significance. XGBoost showed good external validation with AUC of 0.88. Feature importance analysis of XGBoost identified distinct key waveform characteristics for flash and prolonged CRT, respectively.

Conclusion

Novel application of supervised ML to pulse oximeter waveforms yielded multiple effective models to identify flash and prolonged CRT, using clinician judgment as the reference standard.

Tweet

Supervised machine learning applied to pulse oximeter waveform features predicts flash or prolonged capillary refill.

Supervised classification techniques for prediction of mortality in adult patients with sepsis

BACKGROUND

Sepsis mortality is still unacceptably high and an appropriate prognostic tool may increase the accuracy for clinical decisions.

OBJECTIVE

To evaluate several supervised techniques of Artificial Intelligence (AI) for classification and prediction of mortality, in adult patients hospitalized by emergency services with sepsis diagnosis.

METHODS

Secondary data analysis of a prospective cohort in three university hospitals in Medellín, Colombia. We included patients >18 years hospitalized for suspected or confirmed infection and any organ dysfunction according to the Sepsis-related Organ Failure Assessment. The outcome variable was hospital mortality and the prediction variables were grouped into those related to the initial clinical treatment and care or to the direct measurement of physiological disturbances. Four supervised classification techniques were analyzed: the C4.5 Decision Tree, Random Forest, artificial neural networks (ANN) and support vector machine (SVM) models. Their performance was evaluated by the concordance between the observed and predicted outcomes and by the discrimination according to AUC-ROC.

RESULTS

A total of 2510 patients with a median age of 62 years (IQR = 46-74) and an overall hospital mortality rate of 11.5% (n = 289). The best discrimination was provided by the SVM and ANN using physiological variables, with an AUC-ROC of 0.69 (95%CI: 0.62; 0.76) and AUC-ROC of 0.69 (95%CI: 0.61; 0.76) respectively.

CONCLUSION

Deep learning and AI are increasingly used as support tools in clinical medicine. Their performance in a syndrome as complex and heterogeneous as sepsis may be a new horizon in clinical research. SVM and ANN seem promising for improving sepsis classification and prognosis.

Joining Datasets Without Identifiers: Probabilistic Linkage of Virtual Pediatric Systems and PEDSnet

OBJECTIVES

To 1) probabilistically link two important pediatric data sources, Virtual Pediatric Systems and PEDSnet, 2) evaluate linkage accuracy overall and in patients with severe sepsis or septic shock, and 3) identify variables important to linkage accuracy.

DESIGN

Retrospective linkage of prospectively collected datasets from Virtual Pediatrics Systems, Inc (Los Angeles, CA) and the PEDSnet consortium.

SETTING

Single-center academic PICU.

PATIENTS

All PICU encounters between January 1, 2012, and December 31, 2017, that were deterministically matched between the two datasets.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We abstracted records from Virtual Pediatric Systems and PEDSnet corresponding to PICU encounters and probabilistically linked using 44 features shared by the two datasets. We generated a gold standard deterministic linkage using protected health information elements, which were then removed from datasets. We then calculated candidate pair log-likelihood ratios for all pairs of subjects and selected optimal pairs in a two-stage algorithm. A total of 22,051 gold standard PICU encounter pairs were identified over the study period. The optimal linkage model demonstrated excellent discrimination (area under the receiver operating characteristic curve > 0.99); 19,801 cases (89.9%) were matched with 13 false positives. The addition of two protected health information dates (admission month, birth day-of-year) increased to 20,189 (91.6%) the cases matched, with three false positives. Restricting to patients with Virtual Pediatric Systems diagnosis of severe sepsis or septic shock (n = 1,340 [6.1%]) matched 1,250 cases (93.2%) with zero false positives. Increased number of laboratory values present in the first 12 hours of admission significantly increased log-likelihood ratios, suggesting stronger candidate pair matching.

CONCLUSIONS

We demonstrated the use of probabilistic linkage to accurately join two complementary pediatric critical care datasets at a single academic PICU in the absence of protected health information. Combining datasets with curated diagnoses and granular measurements can validate patient acuity metrics and facilitate multicenter machine learning algorithms. We anticipate these methods will generalize to other common PICU diagnoses.

Machine Learning to Support Hemodynamic Intervention in the Neonatal Intensive Care Unit

Hemodynamic support in neonatal intensive care is directed at maintaining cardiovascular wellbeing. At present, monitoring of vital signs plays an essential role in augmenting care in a reactive manner. By applying machine learning techniques, a model can be trained to learn patterns in time series data, allowing the detection of adverse outcomes before they become clinically apparent. In this review we provide an overview of the different machine learning techniques that have been used to develop models in hemodynamic care for newborn infants. We focus on their potential benefits, research pitfalls, and challenges related to their implementation in clinical care.

Generalizability of SuperAlarm via Cross-Institutional Performance Evaluation

Bedside patient monitors are ubiquitous tools in modern critical care units to provide timely patient status. However, current systems suffer from high volume of false alarms leading to alarm fatigue, one of top technical hazards in clinical settings. Many studies are racing to develop improved algorithms towards precision patient monitoring, while little has been done to investigate the aspect of algorithm generalizability across different health institutions. Our group has been developing an evolving framework termed SuperAlarm that extracts multivariate patterns in data streams (monitor alarms, electronic health records and physiologic waveforms) of modern health enterprise to predict patient deterioration and has demonstrated great potential in mitigating alarm fatigue. In this study, we further investigate the generalizability of SuperAlarm by designing a comprehensive approach to achieve performance comparison in predicting in-hospital code blue (CB) events across two health institutions. SuperAlarm model trained with alarm data in one institution is tested on both internal and external test sets. Results show comparable performance with sensitivity up to 80% within one-hour window of events and over 90% in reduction of false alarms in both institutions. Cross-institutional performance agreement can be further improved by predicting a more stringent CB subtype (cardiopulmonary arrest), with internal sensitivity lying within 95% confident interval of external one up to 8-hour before event onset. The cross-institutional performance comparison offers first-hand knowledge on both advantages and challenges in generalizing a prediction algorithm across different institutions, which hold key information to guide the design of model training and deployment strategy.

Machine learning for early detection of sepsis: an internal and temporal validation study

OBJECTIVE

Determine if deep learning detects sepsis earlier and more accurately than other models. To evaluate model performance using implementation-oriented metrics that simulate clinical practice.

MATERIALS AND METHODS

We trained internally and temporally validated a deep learning model (multi-output Gaussian process and recurrent neural network [MGP-RNN]) to detect sepsis using encounters from adult hospitalized patients at a large tertiary academic center. Sepsis was defined as the presence of 2 or more systemic inflammatory response syndrome (SIRS) criteria, a blood culture order, and at least one element of end-organ failure. The training dataset included demographics, comorbidities, vital signs, medication administrations, and labs from October 1, 2014 to December 1, 2015, while the temporal validation dataset was from March 1, 2018 to August 31, 2018. Comparisons were made to 3 machine learning methods, random forest (RF), Cox regression (CR), and penalized logistic regression (PLR), and 3 clinical scores used to detect sepsis, SIRS, quick Sequential Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS). Traditional discrimination statistics such as the C-statistic as well as metrics aligned with operational implementation were assessed.

RESULTS

The training set and internal validation included 42 979 encounters, while the temporal validation set included 39 786 encounters. The C-statistic for predicting sepsis within 4 h of onset was 0.88 for the MGP-RNN compared to 0.836 for RF, 0.849 for CR, 0.822 for PLR, 0.756 for SIRS, 0.619 for NEWS, and 0.481 for qSOFA. MGP-RNN detected sepsis a median of 5 h in advance. Temporal validation assessment continued to show the MGP-RNN outperform all 7 clinical risk score and machine learning comparisons.

CONCLUSIONS

We developed and validated a novel deep learning model to detect sepsis. Using our data elements and feature set, our modeling approach outperformed other machine learning methods and clinical scores.

Augmented intelligence in pediatric anesthesia and pediatric critical care

PURPOSE OF REVIEW

Acute care technologies, including novel monitoring devices, big data, increased computing capabilities, machine-learning algorithms and automation, are converging. This enables the application of augmented intelligence for improved outcome predictions, clinical decision-making, and offers unprecedented opportunities to improve patient outcomes, reduce costs, and improve clinician workflow. This article briefly explores recent work in the areas of automation, artificial intelligence and outcome prediction models in pediatric anesthesia and pediatric critical care.

RECENT FINDINGS

Recent years have yielded little published research into pediatric physiological closed loop control (a type of automation) beyond studies focused on glycemic control for type 1 diabetes. However, there has been a greater range of research in augmented decision-making, leveraging artificial intelligence and machine-learning techniques, in particular, for pediatric ICU outcome prediction.

SUMMARY

Most studies focusing on artificial intelligence demonstrate good performance on prediction or classification, whether they use traditional statistical tools or novel machine-learning approaches. Yet the challenges of implementation, user acceptance, ethics and regulation cannot be underestimated. Areas in which there is easy access to routinely labeled data and robust outcomes, such as those collected through national networks and quality improvement programs, are likely to be at the forefront of the adoption of these advances.

Machine learning-based risk prediction of intrahospital clinical outcomes in patients undergoing TAVI

BACKGROUND

Currently, patient selection in TAVI is based upon a multidisciplinary heart team assessment of patient comorbidities and surgical risk stratification. In an era of increasing need for precision medicine and quickly expanding TAVI indications, machine learning has shown promise in making accurate predictions of clinical outcomes. This study aims to predict different intrahospital clinical outcomes in patients undergoing TAVI using a machine learning-based approach. The main clinical outcomes include all-cause mortality, stroke, major vascular complications, paravalvular leakage, and new pacemaker implantations.

METHODS AND RESULTS

The dataset consists of 451 consecutive patients undergoing elective TAVI between February 2014 and June 2016. The applied machine learning methods were neural networks, support vector machines, and random forests. Their performance was evaluated using five-fold nested cross-validation. Considering all 83 features, the performance of all machine learning models in predicting all-cause intrahospital mortality (AUC 0.94-0.97) was significantly higher than both the STS risk score (AUC 0.64), the STS/ACC TAVR score (AUC 0.65), and all machine learning models using baseline characteristics only (AUC 0.72-0.82). Using an extreme boosting gradient, baseline troponin T was found to be the most important feature among all input variables. Overall, after feature selection, there was a slightly inferior performance. Stroke, major vascular complications, paravalvular leakage, and new pacemaker implantations could not be accurately predicted.

CONCLUSIONS

Machine learning has the potential to improve patient selection and risk management of interventional cardiovascular procedures, as it is capable of making superior predictions compared to current logistic risk scores.

Deep Learning for Improved Risk Prediction in Surgical Outcomes

The Norwood surgical procedure restores functional systemic circulation in neonatal patients with single ventricle congenital heart defects, but this complex procedure carries a high mortality rate. In this study we address the need to provide an accurate patient specific risk prediction for one-year postoperative mortality or cardiac transplantation and prolonged length of hospital stay with the purpose of assisting clinicians and patients' families in the preoperative decision making process. Currently available risk prediction models either do not provide patient specific risk factors or only predict in-hospital mortality rates. We apply machine learning models to predict and calculate individual patient risk for mortality and prolonged length of stay using the Pediatric Heart Network Single Ventricle Reconstruction trial dataset. We applied a Markov Chain Monte-Carlo simulation method to impute missing data and then fed the selected variables to multiple machine learning models. The individual risk of mortality or cardiac transplantation calculation produced by our deep neural network model demonstrated 89 ± 4% accuracy and 0.95 ± 0.02 area under the receiver operating characteristic curve (AUROC). The C-statistics results for prediction of prolonged length of stay were 85 ± 3% accuracy and AUROC 0.94 ± 0.04. These predictive models and calculator may help to inform clinical and organizational decision making.

Performance of Pediatric Systemic Inflammatory Response Syndrome and Organ Dysfunction Criteria in Late-Onset Sepsis in a Quaternary Neonatal Intensive Care Unit: A Case-Control Study

OBJECTIVES

To evaluate accuracy of systemic inflammatory response syndrome (SIRS) criteria in identifying culture-proven late-onset neonatal sepsis and to assess prevalence of organ dysfunction and its relationship with SIRS criteria.

STUDY DESIGN

This was a retrospective case-control study of patients in the Children's Hospital of Philadelphia level IV neonatal intensive care unit undergoing sepsis evaluations (concurrent blood culture and antibiotics). During calendar years 2016-2017, 77 case and 77 control sepsis evaluations were identified. Cases included infants who had sepsis evaluations with positive blood cultures and antibiotic duration ≥7 days. Controls were matched by gestational and postmenstrual age, and had sepsis evaluations with negative blood cultures and antibiotic duration ≤48 hours. SIRS criteria were determined at time of sepsis evaluation, and organ dysfunction evaluated in the 72 hours following sepsis evaluation. Statistical analysis included descriptive statistics, Mann-Whitney tests, and χ² (Fisher exact) tests.

RESULTS

At time of sepsis evaluation, 42% of cases and 26% of controls met SIRS criteria. Among infants of ≤37 weeks postmenstrual age, SIRS criteria were met in only 17% of sepsis evaluations (4 of 23 in both cases and controls). Test characteristics for SIRS at diagnosis of culture-proven sepsis included sensitivity 42% and specificity 74%. Cases had higher rates of new organ dysfunction within 72 hours (40% vs 21%); however, 58% of cases developing organ dysfunction did not meet SIRS criteria at time of sepsis evaluation. Of 6 deaths (all cases with organ dysfunction), 2 did not meet SIRS criteria at sepsis evaluation.

CONCLUSIONS

SIRS criteria did not accurately identify culture-proven late-onset sepsis, with poorest accuracy in preterm infants. SIRS criteria did not predict later organ dysfunction or mortality.

Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone

BACKGROUND

Cardiovascular diseases kill approximately 17 million people globally every year, and they mainly exhibit as myocardial infarctions and heart failures. Heart failure (HF) occurs when the heart cannot pump enough blood to meet the needs of the body.Available electronic medical records of patients quantify symptoms, body features, and clinical laboratory test values, which can be used to perform biostatistics analysis aimed at highlighting patterns and correlations otherwise undetectable by medical doctors. Machine learning, in particular, can predict patients' survival from their data and can individuate the most important features among those included in their medical records.

METHODS

In this paper, we analyze a dataset of 299 patients with heart failure collected in 2015. We apply several machine learning classifiers to both predict the patients survival, and rank the features corresponding to the most important risk factors. We also perform an alternative feature ranking analysis by employing traditional biostatistics tests, and compare these results with those provided by the machine learning algorithms. Since both feature ranking approaches clearly identify serum creatinine and ejection fraction as the two most relevant features, we then build the machine learning survival prediction models on these two factors alone.

RESULTS

Our results of these two-feature models show not only that serum creatinine and ejection fraction are sufficient to predict survival of heart failure patients from medical records, but also that using these two features alone can lead to more accurate predictions than using the original dataset features in its entirety. We also carry out an analysis including the follow-up month of each patient: even in this case, serum creatinine and ejection fraction are the most predictive clinical features of the dataset, and are sufficient to predict patients' survival.

CONCLUSIONS

This discovery has the potential to impact on clinical practice, becoming a new supporting tool for physicians when predicting if a heart failure patient will survive or not. Indeed, medical doctors aiming at understanding if a patient will survive after heart failure may focus mainly on serum creatinine and ejection fraction.

Development and Validation of a Predictive Model of the Risk of Pediatric Septic Shock Using Data Known at the Time of Hospital Arrival

OBJECTIVE

To derive and validate a model of risk of septic shock among children with suspected sepsis, using data known in the electronic health record at hospital arrival.

STUDY DESIGN

This observational cohort study at 6 pediatric emergency department and urgent care sites used a training dataset (5 sites, April 1, 2013, to December 31, 2016), a temporal test set (5 sites, January 1, 2017 to June 30, 2018), and a geographic test set (a sixth site, April 1, 2013, to December 31, 2018). Patients 60 days to 18 years of age in whom clinicians suspected sepsis were included; patients with septic shock on arrival were excluded. The outcome, septic shock, was systolic hypotension with vasoactive medication or ≥30 mL/kg of isotonic crystalloid within 24 hours of arrival. Elastic net regularization, a penalized regression technique, was used to develop a model in the training set.

RESULTS

Of 2464 included visits, septic shock occurred in 282 (11.4%). The model had an area under the curve of 0.79 (0.76-0.83) in the training set, 0.75 (0.69-0.81) in the temporal test set, and 0.87 (0.73-1.00) in the geographic test set. With a threshold set to 90% sensitivity in the training set, the model yielded 82% (72%-90%) sensitivity and 48% (44%-52%) specificity in the temporal test set, and 90% (55%-100%) sensitivity and 32% (21%-46%) specificity in the geographic test set.

CONCLUSIONS

This model estimated the risk of septic shock in children at hospital arrival earlier than existing models. It leveraged the predictive value of routine electronic health record data through a modern predictive algorithm and has the potential to enhance clinical risk stratification in the critical moments before deterioration.

Comparison of Machine Learning Models for Prediction of Initial Intravenous Immunoglobulin Resistance in Children With Kawasaki Disease

We constructed an optimal machine learning (ML) method for predicting intravenous immunoglobulin (IVIG) resistance in children with Kawasaki disease (KD) using commonly available clinical and laboratory variables. We retrospectively collected 98 clinical records of hospitalized children with KD (2-109 months of age). We found that 20 (20%) children were resistant to initial IVIG therapy. We trained three ML techniques, including logistic regression, linear support vector machine, and eXtreme gradient boosting with 10 variables against IVIG resistance. Moreover, we estimated the predictive performance based on nested 5-fold cross-validation (CV). We also selected variables using the recursive feature elimination method and performed the nested 5-fold CV with selected variables in a similar manner. We compared ML models with the existing system regardless of their predictive performance. Results of the area under the receiver operator characteristic curve were in the range of 0.58-0.60 in the all-variable model and 0.60-0.75 in the select model. The specificities were more than 0.90 and higher than those in existing scoring systems, but the sensitivities were lower. Three ML models based on demographics and routine laboratory variables did not provide reliable performance. This is possibly the first study that has attempted to establish a better predictive model. Additional biomarkers are probably needed to generate an effective prediction model.

Neonatal Sepsis Diagnosis Decision-Making Based on Artificial Neural Networks

Neonatal sepsis remains difficult to diagnose due to its non-specific signs and symptoms. Traditional scoring systems help to discriminate between septic or not patients, but they do not consider every single patient particularity. Thus, the purpose of this study was to develop an early- and late-onset neonatal sepsis diagnosis model, based on clinical maternal and neonatal data from electronic records, at the time of clinical suspicion. A predictive model was obtained by training and validating an artificial Neural Networks (ANN) algorithm with a balanced dataset consisting of preterm and term non-septic or septic neonates (early- and late-onset), with negative and positive culture results, respectively, using 25 maternal and neonatal features. The outcome of the model was sepsis or not. The performance measures of the model, evaluated with an independent dataset, outperformed physician's diagnosis using the same features based on traditional scoring systems, with a 93.3% sensitivity, an 80.0% specificity, a 94.4% AUROC, and a regression coefficient of 0.974 between actual and simulated results. The model also performed well-relative to the state-of-the-art methods using similar maternal/neonatal variables. The top 10 factors estimating sepsis were maternal age, cervicovaginitis and neonatal: fever, apneas, platelet counts, gender, bradypnea, band cells, catheter use, and birth weight.

Computational Health Engineering Applied to Model Infectious Diseases and Antimicrobial Resistance Spread

Infectious diseases are the primary cause of mortality worldwide. The dangers of infectious disease are compounded with antimicrobial resistance, which remains the greatest concern for human health. Although novel approaches are under investigation, the World Health Organization predicts that by 2050, septicaemia caused by antimicrobial resistant bacteria could result in 10 million deaths per year. One of the main challenges in medical microbiology is to develop novel experimental approaches, which enable a better understanding of bacterial infections and antimicrobial resistance. After the introduction of whole genome sequencing, there was a great improvement in bacterial detection and identification, which also enabled the characterization of virulence factors and antimicrobial resistance genes. Today, the use of in silico experiments jointly with computational and machine learning offer an in depth understanding of systems biology, allowing us to use this knowledge for the prevention, prediction, and control of infectious disease. Herein, the aim of this review is to discuss the latest advances in human health engineering and their applicability in the control of infectious diseases. An in-depth knowledge of host–pathogen–protein interactions, combined with a better understanding of a host’s immune response and bacterial fitness, are key determinants for halting infectious diseases and antimicrobial resistance dissemination.