Learning From Past Respiratory Infections to Predict COVID-19 Outcomes: Retrospective Study

Unbiased identification of risk factors for invasive Escherichia coli disease using machine learning

BACKGROUND

Invasive Escherichia coli disease (IED), also known as invasive extraintestinal pathogenic E. coli disease, is a leading cause of sepsis and bacteremia in older adults that can result in hospitalization and sometimes death and is frequently associated with antimicrobial resistance. Moreover, certain patient characteristics may increase the risk of developing IED. This study aimed to validate a machine learning approach for the unbiased identification of potential risk factors that correlate with an increased risk for IED.

METHODS

Using electronic health records from 6.5 million people, an XGBoost model was trained to predict IED from 663 distinct patient features, and the most predictive features were identified as potential risk factors. Using Shapley Additive predictive values, the specific relationships between features and the outcome of developing IED were characterized.

RESULTS

The model independently predicted that older age, a known risk factor for IED, increased the chance of developing IED. The model also predicted that a history of ≥ 1 urinary tract infection, as well as more frequent and/or more recent urinary tract infections, and ≥ 1 emergency department or inpatient visit increased the risk for IED. Outcomes were used to calculate risk ratios in selected subpopulations, demonstrating the impact of individual or combinations of features on the incidence of IED.

CONCLUSION

This study illustrates the viability and validity of using large electronic health records datasets and machine learning to identify correlating features and potential risk factors for infectious diseases, including IED. The next step is the independent validation of potential risk factors using conventional methods.

Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS)

The number of papers presenting machine learning (ML) models that are being submitted to and published in the Journal of Medical Internet Research and other JMIR Publications journals has steadily increased. Editors and peer reviewers involved in the review process for such manuscripts often go through multiple review cycles to enhance the quality and completeness of reporting. The use of reporting guidelines or checklists can help ensure consistency in the quality of submitted (and published) scientific manuscripts and, for example, avoid instances of missing information. In this Editorial, the editors of JMIR Publications journals discuss the general JMIR Publications policy regarding authors' application of reporting guidelines and specifically focus on the reporting of ML studies in JMIR Publications journals, using the Consolidated Reporting of Machine Learning Studies (CREMLS) guidelines, with an example of how authors and other journals could use the CREMLS checklist to ensure transparency and rigor in reporting.

Smoking on the risk of acute respiratory distress syndrome: a systematic review and meta-analysis

BACKGROUND

The relationship between smoking and the risk of acute respiratory distress syndrome (ARDS) has been recognized, but the conclusions have been inconsistent. This systematic review and meta-analysis investigated the association between smoking and ARDS risk in adults.

METHODS

The PubMed, EMBASE, Cochrane Library, and Web of Science databases were searched for eligible studies published from January 1, 2000, to December 31, 2023. We enrolled adult patients exhibiting clinical risk factors for ARDS and smoking condition. Outcomes were quantified using odds ratios (ORs) for binary variables and mean differences (MDs) for continuous variables, with a standard 95% confidence interval (CI).

RESULTS

A total of 26 observational studies involving 36,995 patients were included. The meta-analysis revealed a significant association between smoking and an increased risk of ARDS (OR 1.67; 95% CI 1.33-2.08; P < 0.001). Further analysis revealed that the associations between patient-reported smoking history and ARDS occurrence were generally similar to the results of all the studies (OR 1.78; 95% CI 1.38-2.28; P < 0.001). In contrast, patients identified through the detection of tobacco metabolites (cotinine, a metabolite of nicotine, and 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanol (NNAL), a metabolite of tobacco products) showed no significant difference in ARDS risk (OR 1.19; 95% CI 0.69-2.05; P = 0.53). The smoking group was younger than the control group (MD - 7.15; 95% CI - 11.58 to - 2.72; P = 0.002). Subgroup analysis revealed that smoking notably elevated the incidence of ARDS with extrapulmonary etiologies (OR 1.85; 95% CI 1.43-2.38; P < 0.001). Publication bias did not affect the integrity of our conclusions. Sensitivity analysis further reinforced the reliability of our aggregated outcomes.

CONCLUSIONS

There is a strong association between smoking and elevated ARDS risk. This emphasizes the need for thorough assessment of patients' smoking status, urging healthcare providers to vigilantly monitor individuals with a history of smoking, especially those with additional extrapulmonary risk factors for ARDS.

A deep learning model for predicting COVID-19 ARDS in critically ill patients

Background

The coronavirus disease 2019 (COVID-19) is an acute infectious pneumonia caused by a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection previously unknown to humans. However, predictive studies of acute respiratory distress syndrome (ARDS) in patients with COVID-19 are limited. In this study, we attempted to establish predictive models to predict ARDS caused by COVID-19 via a thorough analysis of patients' clinical data and CT images.

Method

The data of included patients were retrospectively collected from the intensive care unit in our hospital from April 2022 to June 2022. The primary outcome was the development of ARDS after ICU admission. We first established two individual predictive models based on extreme gradient boosting (XGBoost) and convolutional neural network (CNN), respectively; then, an integrated model was developed by combining the two individual models. The performance of all the predictive models was evaluated using the area under receiver operating characteristic curve (AUC), confusion matrix, and calibration plot.

Results

A total of 103 critically ill COVID-19 patients were included in this research, of which 23 patients (22.3%) developed ARDS after admission; five predictive variables were selected and further used to establish the machine learning models, and the XGBoost model yielded the most accurate predictions with the highest AUC (0.94, 95% CI: 0.91-0.96). The AUC of the CT-based convolutional neural network predictive model and the integrated model was 0.96 (95% CI: 0.93-0.98) and 0.97 (95% CI: 0.95-0.99), respectively.

Conclusion

An integrated deep learning model could be used to predict COVID-19 ARDS in critically ill patients.

Predicting the Next-Day Perceived and Physiological Stress of Pregnant Women by Using Machine Learning and Explainability: Algorithm Development and Validation

Background

Cognitive behavioral therapy–based interventions are effective in reducing prenatal stress, which can have severe adverse health effects on mothers and newborns if unaddressed. Predicting next-day physiological or perceived stress can help to inform and enable pre-emptive interventions for a likely physiologically and perceptibly stressful day. Machine learning models are useful tools that can be developed to predict next-day physiological and perceived stress by using data collected from the previous day. Such models can improve our understanding of the specific factors that predict physiological and perceived stress and allow researchers to develop systems that collect selected features for assessment in clinical trials to minimize the burden of data collection.

Objective

The aim of this study was to build and evaluate a machine-learned model that predicts next-day physiological and perceived stress by using sensor-based, ecological momentary assessment (EMA)–based, and intervention-based features and to explain the prediction results.

Methods

We enrolled pregnant women into a prospective proof-of-concept study and collected electrocardiography, EMA, and cognitive behavioral therapy intervention data over 12 weeks. We used the data to train and evaluate 6 machine learning models to predict next-day physiological and perceived stress. After selecting the best performing model, Shapley Additive Explanations were used to identify the feature importance and explainability of each feature.

Results

A total of 16 pregnant women enrolled in the study. Overall, 4157.18 hours of data were collected, and participants answered 2838 EMAs. After applying feature selection, 8 and 10 features were found to positively predict next-day physiological and perceived stress, respectively. A random forest classifier performed the best in predicting next-day physiological stress (F1 score of 0.84) and next-day perceived stress (F1 score of 0.74) by using all features. Although any subset of sensor-based, EMA-based, or intervention-based features could reliably predict next-day physiological stress, EMA-based features were necessary to predict next-day perceived stress. The analysis of explainability metrics showed that the prolonged duration of physiological stress was highly predictive of next-day physiological stress and that physiological stress and perceived stress were temporally divergent.

Conclusions

In this study, we were able to build interpretable machine learning models to predict next-day physiological and perceived stress, and we identified unique features that were highly predictive of next-day stress that can help to reduce the burden of data collection.

Clinicodemographic profile, intensive care unit utilization and mortality rate among COVID-19 patients admitted during the second wave in Bangladesh

INTRODUCTION

The second wave of COVID-19 arrived in Bangladesh in March 2021. This pilot research from a tertiary care COVID-dedicated hospital observed the clinicodemographic profile, intensive care unit (ICU) utilization, and mortality rate among COVID-19 patients admitted during the second wave.

METHODS

Reverse transcription-polymerase chain reaction or chest high-resolution computed tomography confirmed 972 COVID-19 cases included in this cross-sectional study from 24 March to 23 June 2021, recruited using convenience sampling. Data regarding clinicodemographic profile, ICU utilization and mortality rate were analyzed.

RESULTS

The mean study cohort age was 54.47±12.73 years, with most patients (48.3%) aged 41-60; 64.1% were men. Fever (77.9%) and cough (75.9%) were the most common symptoms, and hypertension (43.6%) and diabetes (42.15%) the most common comorbidities. Nearly half of patients had total lung involvement of 26%-50%, and 23.8% required ICU. Overall mortality was 16.5%, whereas the mortality rate among ICU admitted patients was 56.1%. The most important predictors of mortality were older age, chronic renal illness, the proportion of lung involvement and ICU requirement.

CONCLUSIONS

We found higher mortality and ICU utilization rate and greater total lung involvement during the second wave. The mortality rate among the elderly and ICU patients was also higher than earlier.

Learning from past respiratory failure patients to triage COVID-19 patient ventilator needs: A multi-institutional study

BACKGROUND

Unlike well-established diseases that base clinical care on randomized trials, past experiences, and training, prognosis in COVID19 relies on a weaker foundation. Knowledge from other respiratory failure diseases may inform clinical decisions in this novel disease. The objective was to predict 48-hour invasive mechanical ventilation (IMV) within 48 h in patients hospitalized with COVID-19 using COVID-like diseases (CLD).

METHODS

This retrospective multicenter study trained machine learning (ML) models on patients hospitalized with CLD to predict IMV within 48 h in COVID-19 patients. CLD patients were identified using diagnosis codes for bacterial pneumonia, viral pneumonia, influenza, unspecified pneumonia and acute respiratory distress syndrome (ARDS), 2008-2019. A total of 16 cohorts were constructed, including any combinations of the four diseases plus an exploratory ARDS cohort, to determine the most appropriate cohort to use. Candidate predictors included demographic and clinical parameters that were previously associated with poor COVID-19 outcomes. Model development included the implementation of logistic regression and three ensemble tree-based algorithms: decision tree, AdaBoost, and XGBoost. Models were validated in hospitalized COVID-19 patients at two healthcare systems, March 2020-July 2020. ML models were trained on CLD patients at Stanford Hospital Alliance (SHA). Models were validated on hospitalized COVID-19 patients at both SHA and Intermountain Healthcare.

RESULTS

CLD training data were obtained from SHA (n = 14,030), and validation data included 444 adult COVID-19 hospitalized patients from SHA (n = 185) and Intermountain (n = 259). XGBoost was the top-performing ML model, and among the 16 CLD training cohorts, the best model achieved an area under curve (AUC) of 0.883 in the validation set. In COVID-19 patients, the prediction models exhibited moderate discrimination performance, with the best models achieving an AUC of 0.77 at SHA and 0.65 at Intermountain. The model trained on all pneumonia and influenza cohorts had the best overall performance (SHA: positive predictive value (PPV) 0.29, negative predictive value (NPV) 0.97, positive likelihood ratio (PLR) 10.7; Intermountain: PPV, 0.23, NPV 0.97, PLR 10.3). We identified important factors associated with IMV that are not traditionally considered for respiratory diseases.

CONCLUSIONS

The performance of prediction models derived from CLD for 48-hour IMV in patients hospitalized with COVID-19 demonstrate high specificity and can be used as a triage tool at point of care. Novel predictors of IMV identified in COVID-19 are often overlooked in clinical practice. Lessons learned from our approach may assist other research institutes seeking to build artificial intelligence technologies for novel or rare diseases with limited data for training and validation.

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

OBJECTIVE

To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease.

DESIGN

Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group.

DATA SOURCES

PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020.

STUDY SELECTION

Studies that developed or validated a multivariable covid-19 related prediction model.

DATA EXTRACTION

At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).

RESULTS

37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models.

CONCLUSION

Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.

SYSTEMATIC REVIEW REGISTRATION

Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

READERS' NOTE

This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.