Development and validation of early warning score systems for COVID-19 patients

[Experience of nurses using the COVID-19 Early Warning Score in the care of COVID-19 patients: A qualitative study]

Experience of nurses using the COVID-19 Early Warning Score in the care of COVID-19 patients: A qualitative study Abstract: Background: In all phases of the COVID-19 disease, patients are at risk of acute deterioration. In order to identify patients at risk at an early stage, the COVID-19-EWS Salzburg was implemented in April 2020 at the University Hospital Salzburg. So far, the applicability, practicability and relevance of the EWS for acute inpatient COVID-19 care are unknown. Aim: The aim of this qualitative study was to describe the relevance and practicability of the COVID-19-EWS Salzburg as a risk assessment tool for acute inpatient COVID-19 care, based on the experiences of the nursing staff. Methods: Nine semi-structured expert interviews were conducted with the nursing staff of the COVID-19 acute care unit. The data were analysed by qualitative content analysis. Results: Nurses described the EWS as relevant to practice because the score facilitates decision-making, increases patient safety, and enhances interprofessional communication. Both the Early Warning Score (EWS) and experience in caring for COVID-19 patients were found to be relevant for decision-making in the context of managing clinical deterioration. The score provided a sense of security in the care of COVID-19 patients, particularly to new and inexperienced nurses. Conclusion: The participating nurses describe the COVID-19-EWS Salzburg as a useful and practical risk assessment instrument, complementing clinical judgment. A need for adaptation concerning the parameters oxygen saturation and oxygen requirement was identified.

Innovative applications of artificial intelligence during the COVID-19 pandemic

The COVID-19 pandemic has created unprecedented challenges worldwide. Artificial intelligence (AI) technologies hold tremendous potential for tackling key aspects of pandemic management and response. In the present review, we discuss the tremendous possibilities of AI technology in addressing the global challenges posed by the COVID-19 pandemic. First, we outline the multiple impacts of the current pandemic on public health, the economy, and society. Next, we focus on the innovative applications of advanced AI technologies in key areas such as COVID-19 prediction, detection, control, and drug discovery for treatment. Specifically, AI-based predictive analytics models can use clinical, epidemiological, and omics data to forecast disease spread and patient outcomes. Additionally, deep neural networks enable rapid diagnosis through medical imaging. Intelligent systems can support risk assessment, decision-making, and social sensing, thereby improving epidemic control and public health policies. Furthermore, high-throughput virtual screening enables AI to accelerate the identification of therapeutic drug candidates and opportunities for drug repurposing. Finally, we discuss future research directions for AI technology in combating COVID-19, emphasizing the importance of interdisciplinary collaboration. Though promising, barriers related to model generalization, data quality, infrastructure readiness, and ethical risks must be addressed to fully translate these innovations into real-world impacts. Multidisciplinary collaboration engaging diverse expertise and stakeholders is imperative for developing robust, responsible, and human-centered AI solutions against COVID-19 and future public health emergencies.

Machine Learning Models Versus the National Early Warning Score System for Predicting Deterioration: Retrospective Cohort Study in the United Arab Emirates

BACKGROUND

Early warning score systems are widely used for identifying patients who are at the highest risk of deterioration to assist clinical decision-making. This could facilitate early intervention and consequently improve patient outcomes; for example, the National Early Warning Score (NEWS) system, which is recommended by the Royal College of Physicians in the United Kingdom, uses predefined alerting thresholds to assign scores to patients based on their vital signs. However, there is limited evidence of the reliability of such scores across patient cohorts in the United Arab Emirates.

OBJECTIVE

Our aim in this study was to propose a data-driven model that accurately predicts in-hospital deterioration in an inpatient cohort in the United Arab Emirates.

METHODS

We conducted a retrospective cohort study using a real-world data set that consisted of 16,901 unique patients associated with 26,073 inpatient emergency encounters and 951,591 observation sets collected between April 2015 and August 2021 at a large multispecialty hospital in Abu Dhabi, United Arab Emirates. The observation sets included routine measurements of heart rate, respiratory rate, systolic blood pressure, level of consciousness, temperature, and oxygen saturation, as well as whether the patient was receiving supplementary oxygen. We divided the data set of 16,901 unique patients into training, validation, and test sets consisting of 11,830 (70%; 18,319/26,073, 70.26% emergency encounters), 3397 (20.1%; 5206/26,073, 19.97% emergency encounters), and 1674 (9.9%; 2548/26,073, 9.77% emergency encounters) patients, respectively. We defined an adverse event as the occurrence of admission to the intensive care unit, mortality, or both if the patient was admitted to the intensive care unit first. On the basis of 7 routine vital signs measurements, we assessed the performance of the NEWS system in detecting deterioration within 24 hours using the area under the receiver operating characteristic curve (AUROC). We also developed and evaluated several machine learning models, including logistic regression, a gradient-boosting model, and a feed-forward neural network.

RESULTS

In a holdout test set of 2548 encounters with 95,755 observation sets, the NEWS system achieved an overall AUROC value of 0.682 (95% CI 0.673-0.690). In comparison, the best-performing machine learning models, which were the gradient-boosting model and the neural network, achieved AUROC values of 0.778 (95% CI 0.770-0.785) and 0.756 (95% CI 0.749-0.764), respectively. Our interpretability results highlight the importance of temperature and respiratory rate in predicting patient deterioration.

CONCLUSIONS

Although traditional early warning score systems are the dominant form of deterioration prediction models in clinical practice today, we strongly recommend the development and use of cohort-specific machine learning models as an alternative. This is especially important in external patient cohorts that were unseen during model development.

Prognostic models in COVID-19 infection that predict severity: a systematic review

Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.

Self-Aware SGD: Reliable Incremental Adaptation Framework For Clinical AI Models

Healthcare is dynamic as demographics, diseases, and therapeutics constantly evolve. This dynamic nature induces inevitable distribution shifts in populations targeted by clinical AI models, often rendering them ineffective. Incremental learning provides an effective method of adapting deployed clinical models to accommodate these contemporary distribution shifts. However, since incremental learning involves modifying a deployed or in-use model, it can be considered unreliable as any adverse modification due to maliciously compromised or incorrectly labelled data can make the model unsuitable for the targeted application. This paper introduces self-aware stochastic gradient descent (SGD), an incremental deep learning algorithm that utilises a contextual bandit-like sanity check to only allow reliable modifications to a model. The contextual bandit analyses incremental gradient updates to isolate and filter unreliable gradients. This behaviour allows self-aware SGD to balance incremental training and integrity of a deployed model. Experimental evaluations on the Oxford University Hospital datasets highlight that self-aware SGD can provide reliable incremental updates for overcoming distribution shifts in challenging conditions induced by label noise.

Machine learning and predictive models: 2 years of Sars-CoV-2 pandemic in a single-center retrospective analysis

BACKGROUND

Since January 2020, coronavirus disease 19 (COVID-19) has rapidly spread all over the world. An early assessment of illness severity is crucial for the stratification of patients in order to address them to the right intensity path of care. We performed an analysis on a large cohort of COVID-19 patients (n=581) hospitalized between March 2020 and May 2021 in our intensive care unit (ICU) at Policlinico Riuniti di Foggia hospital. Through an integration of the scores, demographic data, clinical history, laboratory findings, respiratory parameters, a correlation analysis, and the use of machine learning our study aimed to develop a model to predict the main outcome.

METHODS

We deemed eligible for analysis all adult patients (age >18 years old) admitted to our department. We excluded all the patients with an ICU length of stay inferior to 24 h and the ones that declined to participate in our data collection. We collected demographic data, medical history, D-dimers, NEWS2, and MEWS scores on ICU admission and on ED admission, PaO2/FiO2 ratio on ICU admission, and the respiratory support modalities before the orotracheal intubation and the intubation timing (early vs late with a 48-h hospital length of stay cutoff). We further collected the ICU and hospital lengths of stay expressed in days of hospitalization, hospital location (high dependency unit, HDU, ED), and length of stay before and after ICU admission; the in-hospital mortality; and the in-ICU mortality. We performed univariate, bivariate, and multivariate statistical analyses.

RESULTS

SARS-CoV-2 mortality was positively correlated to age, length of stay in HDU, MEWS, and NEWS2 on ICU admission, D-dimer value on ICU admission, early orotracheal intubation, and late orotracheal intubation. We found a negative correlation between the PaO2/FiO2 ratio on ICU admission and NIV. No significant correlations with sex, obesity, arterial hypertension, chronic obstructive pulmonary disease, chronic kidney disease, cardiovascular disease, diabetes mellitus, dyslipidemia, and neither MEWS nor NEWS on ED admission were observed. Considering all the pre-ICU variables, none of the machine learning algorithms performed well in developing a prediction model accurate enough to predict the outcome although a secondary multivariate analysis focused on the ventilation modalities and the main outcome confirmed how the choice of the right ventilatory support with the right timing is crucial.

CONCLUSION

In our cohort of COVID patients, the choice of the right ventilatory support at the right time has been crucial, severity scores, and clinical judgment gave support in identifying patients at risk of developing a severe disease, comorbidities showed a lower weight than expected considering the main outcome, and machine learning method integration could be a fundamental statistical tool in the comprehensive evaluation of such complex diseases.

Development and validation of a dynamic 48-hour in-hospital mortality risk stratification for COVID-19 in a UK teaching hospital: a retrospective cohort study

OBJECTIVES

To develop a disease stratification model for COVID-19 that updates according to changes in a patient's condition while in hospital to facilitate patient management and resource allocation.

DESIGN

In this retrospective cohort study, we adopted a landmarking approach to dynamic prediction of all-cause in-hospital mortality over the next 48 hours. We accounted for informative predictor missingness and selected predictors using penalised regression.

SETTING

All data used in this study were obtained from a single UK teaching hospital.

PARTICIPANTS

We developed the model using 473 consecutive patients with COVID-19 presenting to a UK hospital between 1 March 2020 and 12 September 2020; and temporally validated using data on 1119 patients presenting between 13 September 2020 and 17 March 2021.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome is all-cause in-hospital mortality within 48 hours of the prediction time. We accounted for the competing risks of discharge from hospital alive and transfer to a tertiary intensive care unit for extracorporeal membrane oxygenation.

RESULTS

Our final model includes age, Clinical Frailty Scale score, heart rate, respiratory rate, oxygen saturation/fractional inspired oxygen ratio, white cell count, presence of acidosis (pH <7.35) and interleukin-6. Internal validation achieved an area under the receiver operating characteristic (AUROC) of 0.90 (95% CI 0.87 to 0.93) and temporal validation gave an AUROC of 0.86 (95% CI 0.83 to 0.88).

CONCLUSIONS

Our model incorporates both static risk factors (eg, age) and evolving clinical and laboratory data, to provide a dynamic risk prediction model that adapts to both sudden and gradual changes in an individual patient's clinical condition. On successful external validation, the model has the potential to be a powerful clinical risk assessment tool.

TRIAL REGISTRATION

The study is registered as 'researchregistry5464' on the Research Registry (www.researchregistry.com).

Threshold of increase in oxygen demand to predict mechanical ventilation use in novel coronavirus disease 2019: A retrospective cohort study incorporating restricted cubic spline regression

Background

Rapid deterioration of oxygenation occurs in novel coronavirus disease 2019 (COVID-19), and prediction of mechanical ventilation (MV) is needed for allocation of patients to intensive care unit. Since intubation is usually decided based on varying clinical conditions, such as required oxygen changes, we aimed to elucidate thresholds of increase in oxygen demand to predict MV use within 12 h.

Methods

A single-center retrospective cohort study using data between January 2020 and January 2021was conducted. Data were retrieved from the hospital data warehouse. Adult patients diagnosed with COVID-19 with a positive polymerase chain reaction (PCR) who needed oxygen during admission were included. Hourly increments in oxygen demand were calculated using two consecutive oxygen values. Covariates were selected from measurements at the closest time points of oxygen data. Prediction of MV use within 12 h by required oxygen changes was evaluated with the area under the receiver operating curves (AUCs). A threshold for increased MV use risk was obtained from restricted cubic spline curves.

Results

Among 66 eligible patients, 1835 oxygen data were analyzed. The AUC was 0.756 for predicting MV by oxygen demand changes, 0.888 by both amounts and changes in oxygen, and 0.933 by the model adjusted with respiratory rate, PCR quantification cycle (Ct), and days from PCR. The threshold of increments of required oxygen was identified as 0.44 L/min/h and the probability of MV use linearly increased afterward. In subgroup analyses, the threshold was lower (0.25 L/min/h) when tachypnea or frequent respiratory distress existed, whereas it was higher (1.00 L/min/h) when viral load is low (Ct ≥20 or days from PCR >7 days).

Conclusions

Hourly changes in oxygen demand predicted MV use within 12 h, with a threshold of 0.44 L/min/h. This threshold was lower with an unstable respiratory condition and higher with a low viral load.

An Ensemble Learning Model for COVID-19 Detection from Blood Test Samples

Current research endeavors in the application of artificial intelligence (AI) methods in the diagnosis of the COVID-19 disease has proven indispensable with very promising results. Despite these promising results, there are still limitations in real-time detection of COVID-19 using reverse transcription polymerase chain reaction (RT-PCR) test data, such as limited datasets, imbalance classes, a high misclassification rate of models, and the need for specialized research in identifying the best features and thus improving prediction rates. This study aims to investigate and apply the ensemble learning approach to develop prediction models for effective detection of COVID-19 using routine laboratory blood test results. Hence, an ensemble machine learning-based COVID-19 detection system is presented, aiming to aid clinicians to diagnose this virus effectively. The experiment was conducted using custom convolutional neural network (CNN) models as a first-stage classifier and 15 supervised machine learning algorithms as a second-stage classifier: K-Nearest Neighbors, Support Vector Machine (Linear and RBF), Naive Bayes, Decision Tree, Random Forest, MultiLayer Perceptron, AdaBoost, ExtraTrees, Logistic Regression, Linear and Quadratic Discriminant Analysis (LDA/QDA), Passive, Ridge, and Stochastic Gradient Descent Classifier. Our findings show that an ensemble learning model based on DNN and ExtraTrees achieved a mean accuracy of 99.28% and area under curve (AUC) of 99.4%, while AdaBoost gave a mean accuracy of 99.28% and AUC of 98.8% on the San Raffaele Hospital dataset, respectively. The comparison of the proposed COVID-19 detection approach with other state-of-the-art approaches using the same dataset shows that the proposed method outperforms several other COVID-19 diagnostics methods.