Predicting presumed serious infection among hospitalized children on central venous lines with machine learning

A common data model for the standardization of intensive care unit medication features

Objective

Common data models provide a standard means of describing data for artificial intelligence (AI) applications, but this process has never been undertaken for medications used in the intensive care unit (ICU). We sought to develop a common data model (CDM) for ICU medications to standardize the medication features needed to support future ICU AI efforts.

Materials and Methods

A 9-member, multi-professional team of ICU clinicians and AI experts conducted a 5-round modified Delphi process employing conference calls, web-based communication, and electronic surveys to define the most important medication features for AI efforts. Candidate ICU medication features were generated through group discussion and then independently scored by each team member based on relevance to ICU clinical decision-making and feasibility for collection and coding. A key consideration was to ensure the final ontology both distinguished unique medications and met Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles.

Results

Using a list of 889 ICU medications, the team initially generated 106 different medication features, and 71 were ranked as being core features for the CDM. Through this process, 106 medication features were assigned to 2 key feature domains: drug product-related (n = 43) and clinical practice-related (n = 63). Each feature included a standardized definition and suggested response values housed in the electronic data library. This CDM for ICU medications is available online.

Conclusion

The CDM for ICU medications represents an important first step for the research community focused on exploring how AI can improve patient outcomes and will require ongoing engagement and refinement.

Machine learning-based prediction of delirium 24 h after pediatric intensive care unit admission in critically ill children: A prospective cohort study

BACKGROUND

Accurately identifying patients at high risk of delirium is vital for timely preventive intervention measures. Approaches for identifying the risk of developing delirium among critically ill children are not well researched.

OBJECTIVE

To develop and validate machine learning-based models for predicting delirium among critically ill children 24 h after pediatric intensive care unit (PICU) admission.

DESIGN

A prospective cohort study.

SETTING

A large academic medical center with a 57-bed PICU in southwestern China from November 2019 to February 2022.

PARTICIPANTS

One thousand five hundred and seventy-six critically ill children requiring PICU stay over 24 h.

METHODS

Five machine learning algorithms were employed. Delirium was screened by bedside nurses twice a day using the Cornell Assessment of Pediatric Delirium. Twenty-four clinical features from medical and nursing records during hospitalization were used to inform the models. Model performance was assessed according to numerous learning metrics, including the area under the receiver operating characteristic curve (AUC).

RESULTS

Of the 1576 enrolled patients, 929 (58.9 %) were boys, and the age ranged from 28 days to 15 years with a median age of 12 months (IQR 3 to 60 months). Among them, 1126 patients were assigned to the training cohort, and 450 were assigned to the validation cohort. The AUCs ranged from 0.763 to 0.805 for the five models, among which the eXtreme Gradient Boosting (XGB) model performed best, achieving an AUC of 0.805 (95 % CI, 0.759-0.851), with 0.798 (95 % CI, 0.758-0.834) accuracy, 0.902 sensitivity, 0.839 positive predictive value, 0.640 F1-score and a Brier score of 0.144. Almost all models showed lower predictive performance in children younger than 24 months than in older children. The logistic regression model also performed well, with an AUC of 0.789 (95 % CI, 0.739, 0.838), just under that of the XGB model, and was subsequently transformed into a nomogram.

CONCLUSIONS

Machine learning-based models can be established and potentially help identify critically ill children who are at high risk of delirium 24 h after PICU admission. The nomogram may be a beneficial management tool for delirium for PICU practitioners at present.

Developing machine learning systems worthy of trust for infection science: a requirement for future implementation into clinical practice

Infection science is a discipline of healthcare which includes clinical microbiology, public health microbiology, mechanisms of microbial disease, and antimicrobial countermeasures. The importance of infection science has become more apparent in recent years during the SARS-CoV-2 (COVID-19) pandemic and subsequent highlighting of critical operational domains within infection science including the hospital, clinical laboratory, and public health environments to prevent, manage, and treat infectious diseases. However, as the global community transitions beyond the pandemic, the importance of infection science remains, with emerging infectious diseases, bloodstream infections, sepsis, and antimicrobial resistance becoming increasingly significant contributions to the burden of global disease. Machine learning (ML) is frequently applied in healthcare and medical domains, with growing interest in the application of ML techniques to problems in infection science. This has the potential to address several key aspects including improving patient outcomes, optimising workflows in the clinical laboratory, and supporting the management of public health. However, despite promising results, the implementation of ML into clinical practice and workflows is limited. Enabling the migration of ML models from the research to real world environment requires the development of trustworthy ML systems that support the requirements of users, stakeholders, and regulatory agencies. This paper will provide readers with a brief introduction to infection science, outline the principles of trustworthy ML systems, provide examples of the application of these principles in infection science, and propose future directions for moving towards the development of trustworthy ML systems in infection science.

Machine learning applications on neonatal sepsis treatment: a scoping review

INTRODUCTION

Neonatal sepsis is a major cause of health loss and mortality worldwide. Without proper treatment, neonatal sepsis can quickly develop into multisystem organ failure. However, the signs of neonatal sepsis are non-specific, and treatment is labour-intensive and expensive. Moreover, antimicrobial resistance is a significant threat globally, and it has been reported that over 70% of neonatal bloodstream infections are resistant to first-line antibiotic treatment. Machine learning is a potential tool to aid clinicians in diagnosing infections and in determining the most appropriate empiric antibiotic treatment, as has been demonstrated for adult populations. This review aimed to present the application of machine learning on neonatal sepsis treatment.

METHODS

PubMed, Embase, and Scopus were searched for studies published in English focusing on neonatal sepsis, antibiotics, and machine learning.

RESULTS

There were 18 studies included in this scoping review. Three studies focused on using machine learning in antibiotic treatment for bloodstream infections, one focused on predicting in-hospital mortality associated with neonatal sepsis, and the remaining studies focused on developing machine learning prediction models to diagnose possible sepsis cases. Gestational age, C-reactive protein levels, and white blood cell count were important predictors to diagnose neonatal sepsis. Age, weight, and days from hospital admission to blood sample taken were important to predict antibiotic-resistant infections. The best-performing machine learning models were random forest and neural networks.

CONCLUSION

Despite the threat antimicrobial resistance poses, there was a lack of studies focusing on the use of machine learning for aiding empirical antibiotic treatment for neonatal sepsis.

Recognition of outer membrane proteins using multiple feature fusion

Introduction: Outer membrane proteins are crucial in maintaining the structural stability and permeability of the outer membrane. Outer membrane proteins exhibit several functions such as antigenicity and strong immunogenicity, which have potential applications in clinical diagnosis and disease prevention. However, wet experiments for studying OMPs are time and capital-intensive, thereby necessitating the use of computational methods for their identification. Methods: In this study, we developed a computational model to predict outer membrane proteins. The non-redundant dataset consists of a positive set of 208 outer membrane proteins and a negative set of 876 non-outer membrane proteins. In this study, we employed the pseudo amino acid composition method to extract feature vectors and subsequently utilized the support vector machine for prediction. Results and Discussion: In the Jackknife cross-validation, the overall accuracy and the area under receiver operating characteristic curve were observed to be 93.19% and 0.966, respectively. These results demonstrate that our model can produce accurate predictions, and could serve as a valuable guide for experimental research on outer membrane proteins.

Age-based centiles for diastolic blood pressure among children in the out-of-hospital emergency setting

Objective

To compare Pediatric Advanced Life Support (PALS) diastolic blood pressure (DBP) criteria to empirically derived DBP criteria for the prediction of out-of-hospital interventions in children.

Methods

We performed a retrospective study of pediatric (<18 years) encounters from the ESO Data Collaborative, which includes approximately 2000 Emergency Medical Services agencies in the United States. We developed age-based centile curves for DBP using generalized additive models for location, scale, and shape. We compared the proportion of encounters with a low DBP when using empirically derived and PALS criteria and calculated their associations with the delivery of out-of-hospital interventions (advanced airway management, cardiopulmonary resuscitation, cardiac epinephrine, any systemic epinephrine, defibrillation, and bolus intravenous fluids).

Results

We included 343,129 encounters. When using PALS criteria, 155,564 (45.3%) were classified as having  abnormal DBP, including 120,624 (35.2%) with high DBP and 34,940 (10.2%) with low DBP. When using empirically-derived criteria, 18.6% had an abnormal DBP (ie, a DBP <10th or >90th centile). The accuracy of low DBP for out-of-hospital interventions between the two criteria was similar.

Conclusion

PALS criteria for DBP classified a high proportion of children as having abnormal vital signs, particularly with diastolic hypertension. Empirically derived DBP thresholds more accurately predict the delivery of key out-of-hospital interventions. If externally validated, correlated to in-hospital outcomes, and combined with thresholds for other vital signs, these may better predict the need for out-of-hospital interventions.

Evaluation of Structured, Semi-Structured, and Free-Text Electronic Health Record Data to Classify Hepatitis C Virus (HCV) Infection

Evaluation of the United States Centers for Disease Control and Prevention (CDC)-defined HCV-related risk factors are not consistently performed as part of routine care, rendering risk-based testing susceptible to clinician bias and missed diagnoses. This work uses natural language processing (NLP) and machine learning to identify patients who are at high risk for HCV infection. Models were developed and validated to predict patients with newly identified HCV infection (detectable RNA or reported HCV diagnosis). We evaluated models with three types of variables: structured (structured-based model), semi-structured and free-text notes (text-based model), and all variables (full-set model). We applied each model to three stratifications of data: patients with no history of HCV prior to 2020, patients with a history of HCV prior to 2020, and all patients. We used XGBoost and ten-fold C-statistic cross-validation to evaluate the generalizability of the models. There were 3564 unique patients, 487 with HCV infection. The average C-statistics on the structured-based, text-based, and full-set models for all the patients were 0.777 (95% CI: 0.744–0.810), 0.677 (95% CI: 0.631–0.723), and 0.774 (95% CI: 0.735–0.813), respectively. The full-set model performed slightly better than the structured-based model and similar to text-based models for patients with no history of HCV prior to 2020; average C-statistics of 0.780, 0.774, and 0.759, respectively. NLP was able to identify six more risk factors inconsistently coded in structured elements: incarceration, needlestick, substance use or abuse, sexually transmitted infections, piercings, and tattoos. The availability of model options (structured-based or text-based models) with a similar performance can provide deployment flexibility in situations where data is limited.

Integrating structured and unstructured data for timely prediction of bloodstream infection among children

BACKGROUND

Hospitalized children with central venous lines (CVLs) are at higher risk of hospital-acquired infections. Information in electronic health records (EHRs) can be employed in training deep learning models to predict the onset of these infections. We incorporated clinical notes in addition to structured EHR data to predict serious bloodstream infections, defined as positive blood culture followed by at least 4 days of new antimicrobial agent administration, among hospitalized children with CVLs.

METHODS

Structured EHR information and clinical notes were extracted for a retrospective cohort including all hospitalized patients with CVLs at a single tertiary care pediatric health system from 2013 to 2018. Deep learning models were trained to determine the added benefit of incorporating the information embedded in clinical notes in predicting serious bloodstream infection.

RESULTS

A total of 24,351 patient encounters met inclusion criteria. The best-performing model restricted to structured EHR data had a specificity of 0.951 and positive predictive value (PPV) of 0.056 when the sensitivity was set to 0.85. The addition of contextualized word embeddings improved the specificity to 0.981 and PPV to 0.113.

CONCLUSIONS

Integrating clinical notes with structured EHR data improved the prediction of serious bloodstream infections among pediatric patients with CVLs.

IMPACT

Developed an advanced infection prediction model in pediatrics that integrates the structured and unstructured EHRs. Extracted information from clinical notes to do timely prediction in a clinical setting. Developed a deep learning model framework that can be employed in predicting rare events in a complex and dynamic environment.

Prediction of apoptosis protein subcellular location based on amphiphilic pseudo amino acid composition

Introduction: Apoptosis proteins play an important role in the process of cell apoptosis, which makes the rate of cell proliferation and death reach a relative balance. The function of apoptosis protein is closely related to its subcellular location, it is of great significance to study the subcellular locations of apoptosis proteins. Many efforts in bioinformatics research have been aimed at predicting their subcellular location. However, the subcellular localization of apoptotic proteins needs to be carefully studied. Methods: In this paper, based on amphiphilic pseudo amino acid composition and support vector machine algorithm, a new method was proposed for the prediction of apoptosis proteins\x{2019} subcellular location. Results and Discussion: The method achieved good performance on three data sets. The Jackknife test accuracy of the three data sets reached 90.5%, 93.9% and 84.0%, respectively. Compared with previous methods, the prediction accuracies of APACC_SVM were improved.

Prediction of Lumbar Drainage-Related Meningitis Based on Supervised Machine Learning Algorithms

Background

Lumbar drainage is widely used in the clinic; however, forecasting lumbar drainage-related meningitis (LDRM) is limited. We aimed to establish prediction models using supervised machine learning (ML) algorithms.

Methods

We utilized a cohort of 273 eligible lumbar drainage cases. Data were preprocessed and split into training and testing sets. Optimal hyper-parameters were archived by 10-fold cross-validation and grid search. The support vector machine (SVM), random forest (RF), and artificial neural network (ANN) were adopted for model training. The area under the operating characteristic curve (AUROC) and precision-recall curve (AUPRC), true positive ratio (TPR), true negative ratio (TNR), specificity, sensitivity, accuracy, and kappa coefficient were used for model evaluation. All trained models were internally validated. The importance of features was also analyzed.

Results

In the training set, all the models had AUROC exceeding 0.8. SVM and the RF models had an AUPRC of more than 0.6, but the ANN model had an unexpectedly low AUPRC (0.380). The RF and ANN models revealed similar TPR, whereas the ANN model had a higher TNR and demonstrated better specificity, sensitivity, accuracy, and kappa efficiency. In the testing set, most performance indicators of established models decreased. However, the RF and AVM models maintained adequate AUROC (0.828 vs. 0.719) and AUPRC (0.413 vs. 0.520), and the RF model also had better TPR, specificity, sensitivity, accuracy, and kappa efficiency. Site leakage showed the most considerable mean decrease in accuracy.

Conclusions

The RF and SVM models could predict LDRM, in which the RF model owned the best performance, and site leakage was the most meaningful predictor.

DeepMC-iNABP: Deep learning for multiclass identification and classification of nucleic acid-binding proteins

Nucleic acid-binding proteins (NABPs), including DNA-binding proteins (DBPs) and RNA-binding proteins (RBPs), play vital roles in gene expression. Accurate identification of these proteins is crucial. However, there are two existing challenges: one is the problem of ignoring DNA- and RNA-binding proteins (DRBPs), and the other is a cross-predicting problem referring to DBP predictors predicting DBPs as RBPs, and vice versa. In this study, we proposed a computational predictor, called DeepMC-iNABP, with the goal of solving these difficulties by utilizing a multiclass classification strategy and deep learning approaches. DBPs, RBPs, DRBPs and non-NABPs as separate classes of data were used for training the DeepMC-iNABP model. The results on test data collected in this study and two independent test datasets showed that DeepMC-iNABP has a strong advantage in identifying the DRBPs and has the ability to alleviate the cross-prediction problem to a certain extent. The web-server of DeepMC-iNABP is freely available at http://www.deepmc-inabp.net/. The datasets used in this research can also be downloaded from the website.

Predicting bloodstream infection outcome using machine learning

Bloodstream infections (BSI) are a main cause of infectious disease morbidity and mortality worldwide. Early prediction of BSI patients at high risk of poor outcomes is important for earlier decision making and effective patient stratification. We developed electronic medical record-based machine learning models that predict patient outcomes of BSI. The area under the receiver-operating characteristics curve was 0.82 for a full featured inclusive model, and 0.81 for a compact model using only 25 features. Our models were trained using electronic medical records that include demographics, blood tests, and the medical and diagnosis history of 7889 hospitalized patients diagnosed with BSI. Among the implications of this work is implementation of the models as a basis for selective rapid microbiological identification, toward earlier administration of appropriate antibiotic therapy. Additionally, our models may help reduce the development of BSI and its associated adverse health outcomes and complications.

Deep Learning Model to Predict Serious Infection Among Children With Central Venous Lines

Objective: Predict the onset of presumed serious infection, defined as a positive blood culture drawn and new antibiotic course of at least 4 days (PSI*), among pediatric patients with Central Venous Lines (CVLs). Design: Retrospective cohort study. Setting: Single academic children's hospital. Patients: All hospital encounters from January 2013 to December 2018, excluding the ones without a CVL or with a length-of-stay shorter than 24 h. Measurements and Main Results: Clinical features including demographics, laboratory results, vital signs, characteristics of the CVLs and medications used were extracted retrospectively from electronic medical records. Data were aggregated across all hospitals within a single pediatric health system and used to train a deep learning model to predict the occurrence of PSI* during the next 48 h of hospitalization. The proposed model prediction was compared to prediction of PSI* by a marker of illness severity (PELOD-2). The baseline prevalence of line infections was 0.34% over all segmented 48-h time windows. Events were identified among cases using onset time. All data from admission till the onset was used for cases and among controls we used all data from admission till discharge. The benchmarks were aggregated over all 48 h time windows [N=748,380 associated with 27,137 patient encounters]. The model achieved an area under the receiver operating characteristic curve of 0.993 (95% CI = [0.990, 0.996]), the enriched positive predictive value (PPV) was 23 times greater than the base prevalence. Conversely, prediction by PELOD-2 achieved a lower PPV of 1.5% [0.9%, 2.1%] which was 5 times the baseline prevalence. Conclusion: A deep learning model that employs common clinical features in the electronic health record can help predict the onset of CLABSI in hospitalized children with central venous line 48 hours prior to the time of specimen collection.

Improvement of APACHE II score system for disease severity based on XGBoost algorithm

BACKGROUND

Prognostication is an essential tool for risk adjustment and decision making in the intensive care units (ICUs). In order to improve patient outcomes, we have been trying to develop a more effective model than Acute Physiology and Chronic Health Evaluation (APACHE) II to measure the severity of the patients in ICUs. The aim of the present study was to provide a mortality prediction model for ICUs patients, and to assess its performance relative to prediction based on the APACHE II scoring system.

METHODS

We used the Medical Information Mart for Intensive Care version III (MIMIC-III) database to build our model. After comparing the APACHE II with 6 typical machine learning (ML) methods, the best performing model was screened for external validation on anther independent dataset. Performance measures were calculated using cross-validation to avoid making biased assessments. The primary outcome was hospital mortality. Finally, we used TreeSHAP algorithm to explain the variable relationships in the extreme gradient boosting algorithm (XGBoost) model.

RESULTS

We picked out 14 variables with 24,777 cases to form our basic data set. When the variables were the same as those contained in the APACHE II, the accuracy of XGBoost (accuracy: 0.858) was higher than that of APACHE II (accuracy: 0.742) and other algorithms. In addition, it exhibited better calibration properties than other methods, the result in the area under the ROC curve (AUC: 0.76). we then expand the variable set by adding five new variables to improve the performance of our model. The accuracy, precision, recall, F1, and AUC of the XGBoost model increased, and were still higher than other models (0.866, 0.853, 0.870, 0.845, and 0.81, respectively). On the external validation dataset, the AUC was 0.79 and calibration properties were good.

CONCLUSIONS

As compared to conventional severity scores APACHE II, our XGBoost proposal offers improved performance for predicting hospital mortality in ICUs patients. Furthermore, the TreeSHAP can help to enhance the understanding of our model by providing detailed insights into the impact of different features on the disease risk. In sum, our model could help clinicians determine prognosis and improve patient outcomes.