Managing COVID-19 With a Clinical Decision Support Tool in a Community Health Network: Algorithm Development and Validation

Expanding access to veterinary clinical decision support in resource-limited settings: a scoping review of clinical decision support tools in medicine and antimicrobial stewardship

Introduction

Digital clinical decision support (CDS) tools are of growing importance in supporting healthcare professionals in understanding complex clinical problems and arriving at decisions that improve patient outcomes. CDS tools are also increasingly used to improve antimicrobial stewardship (AMS) practices in healthcare settings. However, far fewer CDS tools are available in lowerand middle-income countries (LMICs) and in animal health settings, where their use in improving diagnostic and treatment decision-making is likely to have the greatest impact. The aim of this study was to evaluate digital CDS tools designed as a direct aid to support diagnosis and/or treatment decisionmaking, by reviewing their scope, functions, methodologies, and quality. Recommendations for the development of veterinary CDS tools in LMICs are then provided.

Methods

The review considered studies and reports published between January 2017 and October 2023 in the English language in peer-reviewed and gray literature.

Results

A total of 41 studies and reports detailing CDS tools were included in the final review, with 35 CDS tools designed for human healthcare settings and six tools for animal healthcare settings. Of the tools reviewed, the majority were deployed in high-income countries (80.5%). Support for AMS programs was a feature in 12 (29.3%) of the tools, with 10 tools in human healthcare settings. The capabilities of the CDS tools varied when reviewed against the GUIDES checklist.

Discussion

We recommend a methodological approach for the development of veterinary CDS tools in LMICs predicated on securing sufficient and sustainable funding. Employing a multidisciplinary development team is an important first step. Developing standalone CDS tools using Bayesian algorithms based on local expert knowledge will provide users with rapid and reliable access to quality guidance on diagnoses and treatments. Such tools are likely to contribute to improved disease management on farms and reduce inappropriate antimicrobial use, thus supporting AMS practices in areas of high need.

Machine and deep learning methods for clinical outcome prediction based on physiological data of COVID-19 patients: a scoping review

INTRODUCTION

Since the beginning of the COVID-19 pandemic, numerous machine and deep learning (MDL) methods have been proposed in the literature to analyze patient physiological data. The objective of this review is to summarize various aspects of these methods and assess their practical utility for predicting various clinical outcomes.

METHODS

We searched PubMed, Scopus, and Cochrane Library, screened and selected the studies matching the inclusion criteria. The clinical analysis focused on the characteristics of the patient cohorts in the studies included in this review, the specific tasks in the context of the COVID-19 pandemic that machine and deep learning methods were used for, and their practical limitations. The technical analysis focused on the details of specific MDL methods and their performance.

RESULTS

Analysis of the 48 selected studies revealed that the majority (∼54 %) of them examined the application of MDL methods for the prediction of survival/mortality-related patient outcomes, while a smaller fraction (∼13 %) of studies also examined applications to the prediction of patients' physiological outcomes and hospital resource utilization. 21 % of the studies examined the application of MDL methods to multiple clinical tasks. Machine and deep learning methods have been shown to be effective at predicting several outcomes of COVID-19 patients, such as disease severity, complications, intensive care unit (ICU) transfer, and mortality. MDL methods also achieved high accuracy in predicting the required number of ICU beds and ventilators.

CONCLUSION

Machine and deep learning methods have been shown to be valuable tools for predicting disease severity, organ dysfunction and failure, patient outcomes, and hospital resource utilization during the COVID-19 pandemic. The discovered knowledge and our conclusions and recommendations can also be useful to healthcare professionals and artificial intelligence researchers in managing future pandemics.

Simultaneous Quantitative SARS-CoV-2 Antigen and Host Antibody Detection and Pre-Screening Strategy at the Point of Care

As COVID-19 pandemic public health measures are easing globally, the emergence of new SARS-CoV-2 strains continue to present high risk for vulnerable populations. The antibody-mediated protection acquired from vaccination and/or infection is seen to wane over time and the immunocompromised populations can no longer expect benefit from monoclonal antibody prophylaxis. Hence, there is a need to monitor new variants and its effect on vaccine performance. In this context, surveillance of new SARS-CoV-2 infections and serology testing are gaining consensus for use as screening methods, especially for at-risk groups. Here, we described an improved COVID-19 screening strategy, comprising predictive algorithms and concurrent, rapid, accurate, and quantitative SARS-CoV-2 antigen and host antibody testing strategy, at point of care (POC). We conducted a retrospective analysis of 2553 pre- and asymptomatic patients who were tested for SARS-CoV-2 by RT-PCR. The pre-screening model had an AUC (CI) of 0.76 (0.73-0.78). Despite being the default method for screening, body temperature had lower AUC (0.52 [0.49-0.55]) compared to case incidence rate (0.65 [0.62-0.68]). POC assays for SARS-CoV-2 nucleocapsid protein (NP) and spike (S) receptor binding domain (RBD) IgG antibody showed promising preliminary results, demonstrating a convenient, rapid (<20 min), quantitative, and sensitive (ng/mL) antigen/antibody assay. This integrated pre-screening model and simultaneous antigen/antibody approach may significantly improve accuracy of COVID-19 infection and host immunity screening, helping address unmet needs for monitoring vaccine effectiveness and severe disease surveillance.

www.coronabambini.ch: Development and usage of an online decision support tool for paediatric COVID-19-testing in Switzerland: a cross-sectional analysis

OBJECTIVES

To describe the development and usage of www.coronabambini.ch as an example of a paediatric electronic public health application and to explore its potential and limitations in providing information on disease epidemiology and public health policy implementation.

DESIGN

We developed and maintained a non-commercial online decision support tool, www.coronabambini.ch, to translate the Swiss Federal Office of Public Health (FOPH) paediatric (age 0-18 years) COVID-19 guidelines around testing and school/daycare attendance for caregivers, teachers and healthcare personnel. We analysed the online decision tool as well as a voluntary follow-up survey from October 2020 to September 2021 to explore its potential as a surveillance tool for public health policy and epidemiology.

PARTICIPANTS

68 269 users accessed and 52 726 filled out the complete online decision tool. 3% (1399/52 726) filled out a voluntary follow-up. 92% (18 797/20 330) of users were parents.

RESULTS

Certain dynamics of the pandemic and changes in testing strategies were reflected in the data captured by www.coronabambini.ch, for example, in terms of disease epidemiology, gastrointestinal symptoms were reported more frequently in younger age groups (13% (3308/26 180) in children 0-5 years vs 9% (3934/42 089) in children ≥6 years, χ²=184, p≤0.001). As a reflection of public health policy, the proportion of users consulting the tool for a positive contact without symptoms in children 6-12 years increased from 4% (1415/32 215) to 6% (636/9872) after the FOPH loosened testing criteria in this age group, χ²=69, p≤0.001. Adherence to the recommendation was generally high (84% (1131/1352)) but differed by the type of recommendation: 89% (344/385) for 'stay at home and observe', 75% (232/310) for 'school attendance'.

CONCLUSIONS

Usage of www.coronabambini.ch was generally high in areas where it was developed and promoted. Certain patterns in epidemiology and adherence to public health policy could be depicted but selection bias was difficult to measure showing the potential and challenges of digital decision support as public health tools.

Chaos of COVID-19 Superspreading Events: An Analysis Via a Data-driven Approach

Superspreading has become a key mechanism of COVID-19 transmission which creates chaos. The classical approach of compartmental models may not sufficiently reflect the epidemiological situation amid superspreading events (SSEs). We perform a data-driven approach and recognise the deterministic chaos of confirmed cases. The first derivative ( ≈difference of total confirmed cases) and the second derivative ( ≈difference of the first derivative) are used upon SSEs to showcase the chaos. Varying solution trajectories, sensitivity and numerical unpredictability are the chaotic characteristics discussed here.

A Novel COVID-19 Severity Score is Associated With Survival in Patients Undergoing Percutaneous Dilational Tracheostomy

INTRODUCTION

Tracheostomy in patients with COVID-19 is a controversial and difficult clinical decision. We hypothesized that a recently validated COVID-19 Severity Score (CSS) would be associated with survival in patients considered for tracheostomy.

METHODS

We reviewed 77 mechanically ventilated COVID-19 patients evaluated for decision for percutaneous dilational tracheostomy (PDT) from March to June 2020 at a public tertiary care center. Decision for PDT was based on clinical judgment of the screening surgeons. The CSS was retrospectively calculated using mean biomarker values from admission to time of PDT consult. Our primary outcome was survival to discharge, and all patient charts were reviewed through August 31, 2021. ROC curve and Youden index were used to estimate an optimal cut-point for survival.

RESULTS

The mean CSS for 42 survivors significantly differed from that of 35 nonsurvivors (CSS 52 versus 66, P = 0.003). The Youden index returned an optimal CSS of 55 (95% confidence interval 43-72), which was associated with a sensitivity of 0.8 and a specificity of 0.6. The median CSS was 40 (interquartile range 27, 49) in the lower CSS (<55) group and 72 (interquartile range 66, 93) in the high CSS (≥55 group). Eighty-seven percent of lower CSS patients underwent PDT, with 74% survival, whereas 61% of high CSS patients underwent PDT, with only 41% surviving. Patients with high CSS had 77% lower odds of survival (odds ratio = 0.2, 95% confidence interval 0.1-0.7).

CONCLUSIONS

Higher CSS was associated with decreased survival in patients evaluated for PDT, with a score ≥55 predictive of mortality. The novel CSS may be a useful adjunct in determining which COVID-19 patients will benefit from tracheostomy. Further prospective validation of this tool is warranted.

Smart Diagnostics: Combining Artificial Intelligence and In Vitro Diagnostics

We are beginning a new era of Smart Diagnostics-integrated biosensors powered by recent innovations in embedded electronics, cloud computing, and artificial intelligence (AI). Universal and AI-based in vitro diagnostics (IVDs) have the potential to exponentially improve healthcare decision making in the coming years. This perspective covers current trends and challenges in translating Smart Diagnostics. We identify essential elements of Smart Diagnostics platforms through the lens of a clinically validated platform for digitizing biology and its ability to learn disease signatures. This platform for biochemical analyses uses a compact instrument to perform multiclass and multiplex measurements using fully integrated microfluidic cartridges compatible with the point of care. Image analysis digitizes biology by transforming fluorescence signals into inputs for learning disease/health signatures. The result is an intuitive Score reported to the patients and/or providers. This AI-linked universal diagnostic system has been validated through a series of large clinical studies and used to identify signatures for early disease detection and disease severity in several applications, including cardiovascular diseases, COVID-19, and oral cancer. The utility of this Smart Diagnostics platform may extend to multiple cell-based oncology tests via cross-reactive biomarkers spanning oral, colorectal, lung, bladder, esophageal, and cervical cancers, and is well-positioned to improve patient care, management, and outcomes through deployment of this resilient and scalable technology. Lastly, we provide a future perspective on the direction and trajectory of Smart Diagnostics and the transformative effects they will have on health care.

A Rapid and Sensitive Microfluidics-Based Tool for Seroprevalence Immunity Assessment of COVID-19 and Vaccination-Induced Humoral Antibody Response at the Point of Care

As of 8 August 2022, SARS-CoV-2, the causative agent of COVID-19, has infected over 585 million people and resulted in more than 6.42 million deaths worldwide. While approved SARS-CoV-2 spike (S) protein-based vaccines induce robust seroconversion in most individuals, dramatically reducing disease severity and the risk of hospitalization, poorer responses are observed in aged, immunocompromised individuals and patients with certain pre-existing health conditions. Further, it is difficult to predict the protection conferred through vaccination or previous infection against new viral variants of concern (VoC) as they emerge. In this context, a rapid quantitative point-of-care (POC) serological assay able to quantify circulating anti-SARS-CoV-2 antibodies would allow clinicians to make informed decisions on the timing of booster shots, permit researchers to measure the level of cross-reactive antibody against new VoC in a previously immunized and/or infected individual, and help assess appropriate convalescent plasma donors, among other applications. Utilizing a lab-on-a-chip ecosystem, we present proof of concept, optimization, and validation of a POC strategy to quantitate COVID-19 humoral protection. This platform covers the entire diagnostic timeline of the disease, seroconversion, and vaccination response spanning multiple doses of immunization in a single POC test. Our results demonstrate that this platform is rapid (~15 min) and quantitative for SARS-CoV-2-specific IgG detection.

A rapid review of machine learning approaches for telemedicine in the scope of COVID-19

The COVID-19 pandemic has rapidly spread around the world. The rapid transmission of the virus is a threat that hinders the ability to contain the disease propagation. The pandemic forced widespread conversion of in-person to virtual care delivery through telemedicine. Given this gap, this article aims at providing a literature review of machine learning-based telemedicine applications to mitigate COVID-19. A rapid review of the literature was conducted in six electronic databases published from 2015 through 2020. The process of data extraction was documented using a PRISMA flowchart for inclusion and exclusion of studies. As a result, the literature search identified 1.733 articles, from which 16 articles were included in the review. We developed an updated taxonomy and identified challenges, open questions, and current data types. Our taxonomy and discussion contribute with a significant degree of coverage from subjects related to the use of machine learning to improve telemedicine in response to the COVID-19 pandemic. The evidence identified by this rapid review suggests that machine learning, in combination with telemedicine, can provide a strategy to control outbreaks by providing smart triage of patients and remote monitoring. Also, the use of telemedicine during future outbreaks could be further explored and refined.

Clinical informatics solutions in COVID-19 pandemic: Scoping literature review

Background

The global outbreak of COVID-19 (coronavirus disease 2019) disease has highlighted the importance of disease monitoring, diagnosing, treating, and screening. Technology-based instruments could efficiently assist healthcare systems during pandemics by allowing rapid and widespread transfer of information, real-time tracking of data transfer, and virtualization of meetings and patient visits. Therefore, this study was conducted to investigate the applications of clinical informatics (CI) during the COVID-19 outbreak.

Methods

A comprehensive search was performed on Medline and Scopus databases in September 2020. Eligible studies were selected based on the inclusion and exclusion criteria. The extracted data from the studies reviewed were about study sample, study type, objectives, clinical informatics domain, applied method, sample size, outcomes, findings, and conclusion. The risk of bias was evaluated in the studies using appropriate instruments based on the type of each study. The selected studies were then subjected to thematic synthesis.

Results

In this review study, 72 out of 2716 retrieved articles met the inclusion criteria for full-text analysis. Most of the articles reviewed were done in China and the United States of America. The majority of the studies were conducted in the following CI domains: prediction models (60%), telehealth (36%), and mobile health (4%). Most of the studies in telehealth domain used synchronous methods, such as online and phone- or video-call consultations. Mobile applications were developed as self-triage, self-scheduling, and information delivery tools during the COVID-19 pandemic. The most common types of prediction models among the reviewed studies were neural network (49%), classification (42%), and linear models (4.5%).

Conclusion

The present study showed clinical informatics applications during COVID-19 and identified current gaps in this field. Health information technology and clinical informatics seem to be useful in assisting clinicians and managers to combat COVID-19. The most common domains in clinical informatics for research on the COVID-19 crisis were prediction models and telehealth. It is suggested that future researchers conduct scoping reviews to describe and analyze other levels of medical informatics, including bioinformatics, imaging informatics, and public health informatics.

Biodiagnostics in an era of global pandemics-From biosensing materials to data management

The novel corona virus SARS-CoV-2 (COVID-19) has exposed the world to challenges never before seen in fast diagnostics, monitoring, and prevention of the outbreak. As a result, different approaches for fast diagnostic and screening are made and yet to find the ideal way. The current mini-review provides and examines evidence-based innovative and rapid chemical sensing and related biodiagnostic solutions to deal with infectious disease and related pandemic emergencies, which could offer the best possible care for the general population and improve the approachability of the pandemic information, insights, and surrounding contexts. The review discusses how integration of sensing devices with big data analysis, artificial Intelligence or machine learning, and clinical decision support system, could improve the accuracy of the recorded patterns of the disease conditions within an ocean of information. At the end, the mini-review provides a prospective on the requirements to improve our coping of the pandemic-related biodiagnostics as well as future opportunities.

A scoping review of knowledge authoring tools used for developing computerized clinical decision support systems

Objective

Clinical Knowledge Authoring Tools (CKATs) are integral to the computerized Clinical Decision Support (CDS) development life cycle. CKATs enable authors to generate accurate, complete, and reliable digital knowledge artifacts in a relatively efficient and affordable manner. This scoping review aims to compare knowledge authoring tools and derive the common features of CKATs.

Materials and Methods

We performed a keyword-based literature search, followed by a snowball search, to identify peer-reviewed publications describing the development or use of CKATs. We used PubMed and Embase search engines to perform the initial search (n = 1579). After removing duplicate articles, nonrelevant manuscripts, and not peer-reviewed publication, we identified 47 eligible studies describing 33 unique CKATs. The reviewed CKATs were further assessed, and salient characteristics were extracted and grouped as common CKAT features.

Results

Among the identified CKATs, 55% use an open source platform, 70% provide an application programming interface for CDS system integration, and 79% provide features to validate/test the knowledge. The majority of the reviewed CKATs describe the flow of information, offer a graphical user interface for knowledge authors, and provide intellisense coding features (94%, 97%, and 97%, respectively). The composed list of criteria for CKAT included topics such as simulating the clinical setting, validating the knowledge, standardized clinical models and vocabulary, and domain independence. None of the reviewed CKATs met all common criteria.

Conclusion

Our scoping review highlights the key specifications for a CKAT. The CKAT specification proposed in this review can guide CDS authors in developing more targeted CKATs.

Robotics and artificial intelligence in healthcare during COVID-19 pandemic: A systematic review

The outbreak of the COVID-19 pandemic is unarguably the biggest catastrophe of the 21st century, probably the most significant global crisis after the second world war. The rapid spreading capability of the virus has compelled the world population to maintain strict preventive measures. The outrage of the virus has rampaged through the healthcare sector tremendously. This pandemic created a huge demand for necessary healthcare equipment, medicines along with the requirement for advanced robotics and artificial intelligence-based applications. The intelligent robot systems have great potential to render service in diagnosis, risk assessment, monitoring, telehealthcare, disinfection, and several other operations during this pandemic which has helped reduce the workload of the frontline workers remarkably. The long-awaited vaccine discovery of this deadly virus has also been greatly accelerated with AI-empowered tools. In addition to that, many robotics and Robotics Process Automation platforms have substantially facilitated the distribution of the vaccine in many arrangements pertaining to it. These forefront technologies have also aided in giving comfort to the people dealing with less addressed mental health complicacies. This paper investigates the use of robotics and artificial intelligence-based technologies and their applications in healthcare to fight against the COVID-19 pandemic. A systematic search following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method is conducted to accumulate such literature, and an extensive review on 147 selected records is performed.

Identification of high-risk COVID-19 patients using machine learning

The current COVID-19 public health crisis, caused by SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2), has produced a devastating toll both in terms of human life loss and economic disruption. In this paper we present a machine-learning algorithm capable of identifying whether a given patient (actually infected or suspected to be infected) is more likely to survive than to die, or vice-versa. We train this algorithm with historical data, including medical history, demographic data, as well as COVID-19-related information. This is extracted from a database of confirmed and suspected COVID-19 infections in Mexico, constituting the official COVID-19 data compiled and made publicly available by the Mexican Federal Government. We demonstrate that the proposed method can detect high-risk patients with high accuracy, in each of four identified clinical stages, thus improving hospital capacity planning and timely treatment. Furthermore, we show that our method can be extended to provide optimal estimators for hypothesis-testing techniques commonly-used in biological and medical statistics. We believe that our work could be of use in the context of the current pandemic in assisting medical professionals with real-time assessments so as to determine health care priorities.

Managing Pandemics with Health Informatics

OBJECTIVE

To summarize significant research contributions on managing pandemics with health informatics published in 2020.

METHODS

An extensive search using PubMed and Scopus was conducted to identify peer-reviewed articles published in 2020 that examined health informatics systems used during the global COVID-19 pandemic. The selection process comprised three steps: 1) 15 candidate best papers were first selected by the two section editors; 2) external reviewers from internationally renowned research teams reviewed each candidate best paper; and 3) the final selection of three best papers was conducted by the editorial committee of the International Medical Informatics Association (IMIA) Yearbook.

RESULTS

Selected best papers represent the important and diverse ways that health informatics supported clinical and public health responses to the global COVID-19 pandemic. Selected papers represent four groups of papers: 1) Use of analytics to screen, triage, and manage patients; 2) Use of telehealth and remote monitoring to manage patients and populations; 3) Use of EHR systems and administrative systems to manage internal operations of a hospital or health system; and 4) Use of informatics methods and systems by public health authorities to capture, store, manage, and visualize population-level data and information.

CONCLUSION

Health informatics played a critical role in managing patients and populations during the COVID-19 pandemic. Health care and public health organizations both leveraged available information systems and standards to rapidly identify cases, triage infected individuals, and monitor population trends. The selected best papers represent a fraction of the body of knowledge stemming from COVID-19, most of which is focused on pandemic response. Future work will be needed to help the world recover from the pandemic and strengthen the health information infrastructure in preparation for the next pandemic.

Pandemic information support lifecycle: Evidence from the evolution of mobile apps during COVID-19

Information sharing and consumption play an important role during a pandemic in managing constrained resources and devising effective plans to minimize a pandemic's impact. The type of support extended by information also changes as a pandemic evolves. In this paper, we present a novel framework to understand the different types of information support needed during a pandemic crisis. Adapting phases from the pandemic crisis management lifecycle, we propose five different overlapping phases of our proposed Pandemic Information Support Lifecycle (PISL): awareness information support, preventive care information support, active information support, confidence-building information support and evaluation information support. To validate the proposed PISL, we examine the evolution of new mobile apps during the current COVID-19 pandemic by developing a taxonomy for mobile app-based information support. The proposed lifecycle presents future phases of information support for the ongoing COVID-19 pandemic, while identifying specific areas that need additional research and mobile-based information support development.

Artificial Intelligence in Surveillance, Diagnosis, Drug Discovery and Vaccine Development against COVID-19

As of August 6th, 2021, the World Health Organization has notified 200.8 million laboratory-confirmed infections and 4.26 million deaths from COVID-19, making it the worst pandemic since the 1918 flu. The main challenges in mitigating COVID-19 are effective vaccination, treatment, and agile containment strategies. In this review, we focus on the potential of Artificial Intelligence (AI) in COVID-19 surveillance, diagnosis, outcome prediction, drug discovery and vaccine development. With the help of big data, AI tries to mimic the cognitive capabilities of a human brain, such as problem-solving and learning abilities. Machine Learning (ML), a subset of AI, holds special promise for solving problems based on experiences gained from the curated data. Advances in AI methods have created an unprecedented opportunity for building agile surveillance systems using the deluge of real-time data generated within a short span of time. During the COVID-19 pandemic, many reports have discussed the utility of AI approaches in prioritization, delivery, surveillance, and supply chain of drugs, vaccines, and non-pharmaceutical interventions. This review will discuss the clinical utility of AI-based models and will also discuss limitations and challenges faced by AI systems, such as model generalizability, explainability, and trust as pillars for real-life deployment in healthcare.

Artificial Intelligence in Surveillance, Diagnosis, Drug Discovery and Vaccine Development against COVID-19

As of August 6th, 2021, the World Health Organization has notified 200.8 million laboratory-confirmed infections and 4.26 million deaths from COVID-19, making it the worst pandemic since the 1918 flu. The main challenges in mitigating COVID-19 are effective vaccination, treatment, and agile containment strategies. In this review, we focus on the potential of Artificial Intelligence (AI) in COVID-19 surveillance, diagnosis, outcome prediction, drug discovery and vaccine development. With the help of big data, AI tries to mimic the cognitive capabilities of a human brain, such as problem-solving and learning abilities. Machine Learning (ML), a subset of AI, holds special promise for solving problems based on experiences gained from the curated data. Advances in AI methods have created an unprecedented opportunity for building agile surveillance systems using the deluge of real-time data generated within a short span of time. During the COVID-19 pandemic, many reports have discussed the utility of AI approaches in prioritization, delivery, surveillance, and supply chain of drugs, vaccines, and non-pharmaceutical interventions. This review will discuss the clinical utility of AI-based models and will also discuss limitations and challenges faced by AI systems, such as model generalizability, explainability, and trust as pillars for real-life deployment in healthcare.

Predictive Modeling of Morbidity and Mortality in Patients Hospitalized With COVID-19 and its Clinical Implications: Algorithm Development and Interpretation

Background

The COVID-19 pandemic began in early 2021 and placed significant strains on health care systems worldwide. There remains a compelling need to analyze factors that are predictive for patients at elevated risk of morbidity and mortality.

Objective

The goal of this retrospective study of patients who tested positive with COVID-19 and were treated at NYU (New York University) Langone Health was to identify clinical markers predictive of disease severity in order to assist in clinical decision triage and to provide additional biological insights into disease progression.

Methods

The clinical activity of 3740 patients at NYU Langone Hospital was obtained between January and August 2020; patient data were deidentified. Models were trained on clinical data during different parts of their hospital stay to predict three clinical outcomes: deceased, ventilated, or admitted to the intensive care unit (ICU).

Results

The XGBoost (eXtreme Gradient Boosting) model that was trained on clinical data from the final 24 hours excelled at predicting mortality (area under the curve [AUC]=0.92; specificity=86%; and sensitivity=85%). Respiration rate was the most important feature, followed by SpO₂ (peripheral oxygen saturation) and being aged 75 years and over. Performance of this model to predict the deceased outcome extended 5 days prior, with AUC=0.81, specificity=70%, and sensitivity=75%. When only using clinical data from the first 24 hours, AUCs of 0.79, 0.80, and 0.77 were obtained for deceased, ventilated, or ICU-admitted outcomes, respectively. Although respiration rate and SpO₂ levels offered the highest feature importance, other canonical markers, including diabetic history, age, and temperature, offered minimal gain. When lab values were incorporated, prediction of mortality benefited the most from blood urea nitrogen and lactate dehydrogenase (LDH). Features that were predictive of morbidity included LDH, calcium, glucose, and C-reactive protein.

Conclusions

Together, this work summarizes efforts to systematically examine the importance of a wide range of features across different endpoint outcomes and at different hospitalization time points.

Predictive modeling of morbidity and mortality in COVID-19 hospitalized patients and its clinical implications

Clinical activity of 3740 de-identified COVID-19 positive patients treated at NYU Langone Health (NYULH) were collected between January and August 2020. XGBoost model trained on clinical data from the final 24 hours excelled at predicting mortality (AUC=0.92, specificity=86% and sensitivity=85%). Respiration rate was the most important feature, followed by SpO2 and age 75+. Performance of this model to predict the deceased outcome extended 5 days prior with AUC=0.81, specificity=70%, sensitivity=75%. When only using clinical data from the first 24 hours, AUCs of 0.79, 0.80, and 0.77 were obtained for deceased, ventilated, or ICU admitted, respectively. Although respiration rate and SpO2 levels offered the highest feature importance, other canonical markers including diabetic history, age and temperature offered minimal gain. When lab values were incorporated, prediction of mortality benefited the most from blood urea nitrogen (BUN) and lactate dehydrogenase (LDH). Features predictive of morbidity included LDH, calcium, glucose, and C-reactive protein (CRP). Together this work summarizes efforts to systematically examine the importance of a wide range of features across different endpoint outcomes and at different hospitalization time points.

Clinical decision support tool for diagnosis of COVID-19 in hospitals

Background

The coronavirus infectious disease 19 (COVID-19) pandemic has resulted in significant morbidities, severe acute respiratory failures and subsequently emergency departments’ (EDs) overcrowding in a context of insufficient laboratory testing capacities. The development of decision support tools for real-time clinical diagnosis of COVID-19 is of prime importance to assist patients’ triage and allocate resources for patients at risk.

Methods and principal findings

From March 2 to June 15, 2020, clinical patterns of COVID-19 suspected patients at admission to the EDs of Liège University Hospital, consisting in the recording of eleven symptoms (i.e. dyspnoea, chest pain, rhinorrhoea, sore throat, dry cough, wet cough, diarrhoea, headache, myalgia, fever and anosmia) plus age and gender, were investigated during the first COVID-19 pandemic wave. Indeed, 573 SARS-CoV-2 cases confirmed by qRT-PCR before mid-June 2020, and 1579 suspected cases that were subsequently determined to be qRT-PCR negative for the detection of SARS-CoV-2 were enrolled in this study. Using multivariate binary logistic regression, two most relevant symptoms of COVID-19 were identified in addition of the age of the patient, i.e. fever (odds ratio [OR] = 3.66; 95% CI: 2.97–4.50), dry cough (OR = 1.71; 95% CI: 1.39–2.12), and patients older than 56.5 y (OR = 2.07; 95% CI: 1.67–2.58). Two additional symptoms (chest pain and sore throat) appeared significantly less associated to the confirmed COVID-19 cases with the same OR = 0.73 (95% CI: 0.56–0.94). An overall pondered (by OR) score (OPS) was calculated using all significant predictors. A receiver operating characteristic (ROC) curve was generated and the area under the ROC curve was 0.71 (95% CI: 0.68–0.73) rendering the use of the OPS to discriminate COVID-19 confirmed and unconfirmed patients. The main predictors were confirmed using both sensitivity analysis and classification tree analysis. Interestingly, a significant negative correlation was observed between the OPS and the cycle threshold (Ct values) of the qRT-PCR.

Conclusion and main significance

The proposed approach allows for the use of an interactive and adaptive clinical decision support tool. Using the clinical algorithm developed, a web-based user-interface was created to help nurses and clinicians from EDs with the triage of patients during the second COVID-19 wave.

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

OBJECTIVE

To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease.

DESIGN

Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group.

DATA SOURCES

PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020.

STUDY SELECTION

Studies that developed or validated a multivariable covid-19 related prediction model.

DATA EXTRACTION

At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).

RESULTS

37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models.

CONCLUSION

Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.

SYSTEMATIC REVIEW REGISTRATION

Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

READERS' NOTE

This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.