Evaluating a Widely Implemented Proprietary Deterioration Index Model Among Hospitalized COVID-19 Patients

Clinical prediction rules for adverse evolution in patients with COVID-19 by the Omicron variant

OBJECTIVE

We identify factors related to SARS-CoV-2 infection linked to hospitalization, ICU admission, and mortality and develop clinical prediction rules.

METHODS

Retrospective cohort study of 380,081 patients with SARS-CoV-2 infection from March 1, 2020 to January 9, 2022, including a subsample of 46,402 patients who attended Emergency Departments (EDs) having data on vital signs. For derivation and external validation of the prediction rule, two different periods were considered: before and after emergence of the Omicron variant, respectively. Data collected included sociodemographic data, COVID-19 vaccination status, baseline comorbidities and treatments, other background data and vital signs at triage at EDs. The predictive models for the EDs and the whole samples were developed using multivariate logistic regression models using Lasso penalization.

RESULTS

In the multivariable models, common predictive factors of death among EDs patients were greater age; being male; having no vaccination, dementia; heart failure; liver and kidney disease; hemiplegia or paraplegia; coagulopathy; interstitial pulmonary disease; malignant tumors; use chronic systemic use of steroids, higher temperature, low O2 saturation and altered blood pressure-heart rate. The predictors of an adverse evolution were the same, with the exception of liver disease and the inclusion of cystic fibrosis. Similar predictors were found to be related to hospital admission, including liver disease, arterial hypertension, and basal prescription of immunosuppressants. Similarly, models for the whole sample, without vital signs, are presented.

CONCLUSIONS

We propose risk scales, based on basic information, easily-calculable, high-predictive that also function with the current Omicron variant and may help manage such patients in primary, emergency, and hospital care.

Modelling the risk of hospital admission of lab confirmed SARS-CoV-2-infected patients in primary care: a population-based study

The objectives of this study are to develop a predictive model of hospital admission for COVID-19 to help in the activation of emergency services, early referrals from primary care, and the improvement of clinical decision-making in emergency room services. The method is the retrospective cohort study of 49,750 patients with microbiological confirmation of SARS-CoV-2 infection. The sample was randomly divided into two subsamples, for the purposes of derivation and validation of the prediction rule (60% and 40%, respectively). Data collected for this study included sociodemographic data, baseline comorbidities, baseline treatments, and other background data. Multilevel analyses with generalized estimated equations were used to develop the predictive model. Male sex and the gradual effect of age were the main risk factors for hospital admission. Regarding baseline comorbidities, coagulopathies, cancer, cardiovascular diseases, diabetes with organ damage, and liver disease were among the five most notable. Flu vaccination was a risk factor for hospital admission. Drugs that increased risk were chronic systemic steroids, immunosuppressants, angiotensin-converting enzyme inhibitors, and NSAIDs. The AUC of the risk score was 0.821 and 0.828 in the derivation and validation samples, respectively. Based on the risk score, five risk groups were derived with hospital admission ranging from 2.94 to 51.87%. In conclusion, we propose a classification system for people with COVID-19 with a higher risk of hospitalization, and indirectly with it a greater severity of the disease, easy to be completed both in primary care, as well as in emergency services and in hospital emergency room to help in clinical decision-making.Registration: ClinicalTrials.gov Identifier: NCT04463706.

Prognosis of COVID-19 pneumonia can be early predicted combining Age-adjusted Charlson Comorbidity Index, CRB score and baseline oxygen saturation

In potentially severe diseases in general and COVID-19 in particular, it is vital to early identify those patients who are going to progress to severe disease. A recent living systematic review dedicated to predictive models in COVID-19, critically appraises 145 models, 8 of them focused on prediction of severe disease and 23 on mortality. Unfortunately, in all 145 models, they found a risk of bias significant enough to finally "not recommend any for clinical use". Authors suggest concentrating on avoiding biases in sampling and prioritising the study of already identified predictive factors, rather than the identification of new ones that are often dependent on the database. Our objective is to develop a model to predict which patients with COVID-19 pneumonia are at high risk of developing severe illness or dying, using basic and validated clinical tools. We studied a prospective cohort of consecutive patients admitted in a teaching hospital during the "first wave" of the COVID-19 pandemic. Follow-up to discharge from hospital. Multiple logistic regression selecting variables according to clinical and statistical criteria. 404 consecutive patients were evaluated, 392 (97%) completed follow-up. Mean age was 61 years; 59% were men. The median burden of comorbidity was 2 points in the Age-adjusted Charlson Comorbidity Index, CRB was abnormal in 18% of patients and basal oxygen saturation on admission lower than 90% in 18%. A model composed of Age-adjusted Charlson Comorbidity Index, CRB score and basal oxygen saturation can predict unfavorable evolution or death with an area under the ROC curve of 0.85 (95% CI 0.80-0.89), and 0.90 (95% CI 0.86 to 0.94), respectively. Prognosis of COVID-19 pneumonia can be predicted without laboratory tests using two classic clinical tools and a pocket pulse oximeter.

Effects of hydroxychloroquine and its metabolites in patients with connective tissue diseases

Hydroxychloroquine has attracted attention in the treatment of COVID-19. Many conflicting findings have been reported regarding the efficacy and safety of this drug, which has been used safely in the rheumatological diseases for years. However, these studies lacked measurement methods that allow accurate assessment of hydroxychloroquine and its metabolite levels. The aim of this study was to measure hydroxychloroquine and its metabolite levels in whole blood samples of patients with rheumatoid arthritis (RA), systemic lupus erythematosus (SLE), Sjogren’s syndrome (SS) and scleroderma (Scl) by a robust, simple and accurate validated tandem mass spectrometric method, and to investigate the relationship between these levels with drug-related adverse effects and disease activity scores. The validated LC–MS/MS method was applied to measure blood hydroxychloroquine and its metabolite levels of patients with RA, SLE, SS, Scl. Various haematological and biochemical parameters were measured with Beckman-Coulter AU 5800 and Beckman Coulter LH 780 analyzers, respectively. QTc intervals were calculated with Bazett’s formula, and the patients were followed up by clinicians in terms of clinical findings and adverse effects. Hydroxychloroquine levels of patients were similar to previous studies. There was a negative correlation between disease activity scores and hydroxychloroquine levels, while the highest correlation was between QTc interval, creatinine and GFR levels with desethylchloroquine. Bidetylchloroquine had the highest correlation with RBC count and liver function tests. Our findings showed that hydroxychloroquine and its metabolite levels were associated with disease activity scores, renal, hepatic function, QTc prolongation, and hematological parameters.

Validated tool for early prediction of intensive care unit admission in COVID-19 patients

BACKGROUND

The novel coronavirus disease 2019 (COVID-19) pandemic is a global threat caused by the severe acute respiratory syndrome coronavirus-2.

AIM

To develop and validate a risk stratification tool for the early prediction of intensive care unit (ICU) admission among COVID-19 patients at hospital admission.

METHODS

The training cohort included COVID-19 patients admitted to the Wuhan Third Hospital. We selected 13 of 65 baseline laboratory results to assess ICU admission risk, which were used to develop a risk prediction model with the random forest (RF) algorithm. A nomogram for the logistic regression model was built based on six selected variables. The predicted models were carefully calibrated, and the predictive performance was evaluated and compared with two previously published models.

RESULTS

There were 681 and 296 patients in the training and validation cohorts, respectively. The patients in the training cohort were older than those in the validation cohort (median age: 63.0 vs 49.0 years, P < 0.001), and the percentages of male gender were similar (49.6% vs 49.3%, P = 0.958). The top predictors selected in the RF model were neutrophil-to-lymphocyte ratio, age, lactate dehydrogenase, C-reactive protein, creatinine, D-dimer, albumin, procalcitonin, glucose, platelet, total bilirubin, lactate and creatine kinase. The accuracy, sensitivity and specificity for the RF model were 91%, 88% and 93%, respectively, higher than those for the logistic regression model. The area under the receiver operating characteristic curve of our model was much better than those of two other published methods (0.90 vs 0.82 and 0.75). Model A underestimated risk of ICU admission in patients with a predicted risk less than 30%, whereas the RF risk score demonstrated excellent ability to categorize patients into different risk strata. Our predictive model provided a larger standardized net benefit across the major high-risk range compared with model A.

CONCLUSION

Our model can identify ICU admission risk in COVID-19 patients at admission, who can then receive prompt care, thus improving medical resource allocation.

How US law will evaluate artificial intelligence for covid-19

Daniel E Ho and colleagues explore the legal implications of using artificial intelligence in the response to covid-19 and call for more robust evaluation frameworks

Development of a prognostic model for mortality in COVID-19 infection using machine learning

Coronavirus disease 2019 (COVID-19) is a novel disease resulting from infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which has quickly risen since the beginning of 2020 to become a global pandemic. As a result of the rapid growth of COVID-19, hospitals are tasked with managing an increasing volume of these cases with neither a known effective therapy, an existing vaccine, nor well-established guidelines for clinical management. The need for actionable knowledge amidst the COVID-19 pandemic is dire and yet, given the urgency of this illness and the speed with which the healthcare workforce must devise useful policies for its management, there is insufficient time to await the conclusions of detailed, controlled, prospective clinical research. Thus, we present a retrospective study evaluating laboratory data and mortality from patients with positive RT-PCR assay results for SARS-CoV-2. The objective of this study is to identify prognostic serum biomarkers in patients at greatest risk of mortality. To this end, we develop a machine learning model using five serum chemistry laboratory parameters (c-reactive protein, blood urea nitrogen, serum calcium, serum albumin, and lactic acid) from 398 patients (43 expired and 355 non-expired) for the prediction of death up to 48 h prior to patient expiration. The resulting support vector machine model achieved 91% sensitivity and 91% specificity (AUC 0.93) for predicting patient expiration status on held-out testing data. Finally, we examine the impact of each feature and feature combination in light of different model predictions, highlighting important patterns of laboratory values that impact outcomes in SARS-CoV-2 infection.

Learning From Past Respiratory Infections to Predict COVID-19 Outcomes: Retrospective Study

BACKGROUND

For the clinical care of patients with well-established diseases, randomized trials, literature, and research are supplemented with clinical judgment to understand disease prognosis and inform treatment choices. In the void created by a lack of clinical experience with COVID-19, artificial intelligence (AI) may be an important tool to bolster clinical judgment and decision making. However, a lack of clinical data restricts the design and development of such AI tools, particularly in preparation for an impending crisis or pandemic.

OBJECTIVE

This study aimed to develop and test the feasibility of a "patients-like-me" framework to predict the deterioration of patients with COVID-19 using a retrospective cohort of patients with similar respiratory diseases.

METHODS

Our framework used COVID-19-like cohorts to design and train AI models that were then validated on the COVID-19 population. The COVID-19-like cohorts included patients diagnosed with bacterial pneumonia, viral pneumonia, unspecified pneumonia, influenza, and acute respiratory distress syndrome (ARDS) at an academic medical center from 2008 to 2019. In total, 15 training cohorts were created using different combinations of the COVID-19-like cohorts with the ARDS cohort for exploratory purposes. In this study, two machine learning models were developed: one to predict invasive mechanical ventilation (IMV) within 48 hours for each hospitalized day, and one to predict all-cause mortality at the time of admission. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, positive predictive value, and negative predictive value. We established model interpretability by calculating SHapley Additive exPlanations (SHAP) scores to identify important features.

RESULTS

Compared to the COVID-19-like cohorts (n=16,509), the patients hospitalized with COVID-19 (n=159) were significantly younger, with a higher proportion of patients of Hispanic ethnicity, a lower proportion of patients with smoking history, and fewer patients with comorbidities (P<.001). Patients with COVID-19 had a lower IMV rate (15.1 versus 23.2, P=.02) and shorter time to IMV (2.9 versus 4.1 days, P<.001) compared to the COVID-19-like patients. In the COVID-19-like training data, the top models achieved excellent performance (AUROC>0.90). Validating in the COVID-19 cohort, the top-performing model for predicting IMV was the XGBoost model (AUROC=0.826) trained on the viral pneumonia cohort. Similarly, the XGBoost model trained on all 4 COVID-19-like cohorts without ARDS achieved the best performance (AUROC=0.928) in predicting mortality. Important predictors included demographic information (age), vital signs (oxygen saturation), and laboratory values (white blood cell count, cardiac troponin, albumin, etc). Our models had class imbalance, which resulted in high negative predictive values and low positive predictive values.

CONCLUSIONS

We provided a feasible framework for modeling patient deterioration using existing data and AI technology to address data limitations during the onset of a novel, rapidly changing pandemic.

Managing COVID-19 With a Clinical Decision Support Tool in a Community Health Network: Algorithm Development and Validation

BACKGROUND

The coronavirus disease (COVID-19) pandemic has resulted in significant morbidity and mortality; large numbers of patients require intensive care, which is placing strain on health care systems worldwide. There is an urgent need for a COVID-19 disease severity assessment that can assist in patient triage and resource allocation for patients at risk for severe disease.

OBJECTIVE

The goal of this study was to develop, validate, and scale a clinical decision support system and mobile app to assist in COVID-19 severity assessment, management, and care.

METHODS

Model training data from 701 patients with COVID-19 were collected across practices within the Family Health Centers network at New York University Langone Health. A two-tiered model was developed. Tier 1 uses easily available, nonlaboratory data to help determine whether biomarker-based testing and/or hospitalization is necessary. Tier 2 predicts the probability of mortality using biomarker measurements (C-reactive protein, procalcitonin, D-dimer) and age. Both the Tier 1 and Tier 2 models were validated using two external datasets from hospitals in Wuhan, China, comprising 160 and 375 patients, respectively.

RESULTS

All biomarkers were measured at significantly higher levels in patients who died vs those who were not hospitalized or discharged (P<.001). The Tier 1 and Tier 2 internal validations had areas under the curve (AUCs) of 0.79 (95% CI 0.74-0.84) and 0.95 (95% CI 0.92-0.98), respectively. The Tier 1 and Tier 2 external validations had AUCs of 0.79 (95% CI 0.74-0.84) and 0.97 (95% CI 0.95-0.99), respectively.

CONCLUSIONS

Our results demonstrate the validity of the clinical decision support system and mobile app, which are now ready to assist health care providers in making evidence-based decisions when managing COVID-19 patient care. The deployment of these new capabilities has potential for immediate impact in community clinics and sites, where application of these tools could lead to improvements in patient outcomes and cost containment.

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

OBJECTIVE

To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease.

DESIGN

Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group.

DATA SOURCES

PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020.

STUDY SELECTION

Studies that developed or validated a multivariable covid-19 related prediction model.

DATA EXTRACTION

At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).

RESULTS

37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models.

CONCLUSION

Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.

SYSTEMATIC REVIEW REGISTRATION

Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

READERS' NOTE

This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.

An Updated Systematic Review of the Therapeutic Role of Hydroxychloroquine in Coronavirus Disease-19 (COVID-19)

BACKGROUND AND OBJECTIVE

The world is currently experiencing the Coronavirus Disease-19 (COVID-19) pandemic. There is no approved drug for the definitive treatment of the disease. Various drugs are being tried for the treatment of COVID-19, including hydroxychloroquine (HCQ). This study was performed to systematically review the therapeutic role of HCQ in COVID-19 from the available literature.

METHODS

PubMed, Embase, ClinicalTrials.gov, ICTRP (WHO), Cochrane Library databases, and two pre-print servers (medRxiv.org and Research Square) were searched for clinical studies that evaluated the therapeutic role of HCQ on COVID-19 until 10 May 2020. The available studies were critically analyzed and the data were extracted.

RESULTS

A total of 663 articles were screened and 12 clinical studies (seven peer-reviewed and published studies and five non-peer-reviewed studies from pre-print servers) with a total sample size of 3543 patients were included. Some of the clinical studies demonstrated good virological and clinical outcomes with HCQ alone or in combination with azithromycin in COVID-19 patients, although the studies had major methodological limitations. Some of the other studies showed negative results with HCQ therapy along with the risk of adverse reactions.

CONCLUSION

The results of efficacy and safety of HCQ in COVID-19, as obtained from the clinical studies, are not satisfactory, although many of these studies had major methodological limitations. Stronger evidence from well-designed robust randomized clinical trials is required before conclusively determining the role of HCQ in the treatment of COVID-19. Clinical prudence is required in advocating HCQ as a therapeutic armamentarium in COVID-19.