Rare Events in the ICU: An Emerging Challenge in Classification and Prediction

Validation and comparison of triage-based screening strategies for sepsis

OBJECTIVE

This study sought to externally validate and compare proposed methods for stratifying sepsis risk at emergency department (ED) triage.

METHODS

This nested case/control study enrolled ED patients from four hospitals in Utah and evaluated the performance of previously-published sepsis risk scores amenable to use at ED triage based on their area under the precision-recall curve (AUPRC, which balances positive predictive value and sensitivity) and area under the receiver operator characteristic curve (AUROC, which balances sensitivity and specificity). Score performance for predicting whether patients met Sepsis-3 criteria in the ED was compared to patients' assigned ED triage score (Canadian Triage Acuity Score [CTAS]) with adjustment for multiple comparisons.

RESULTS

Among 2000 case/control patients, 981 met Sepsis-3 criteria on final adjudication. The best performing sepsis risk scores were the Predict Sepsis version #3 (AUPRC 0.183, 95 % CI 0.148-0.256; AUROC 0.859, 95 % CI 0.843-0.875) and Borelli scores (AUPRC 0.127, 95 % CI 0.107-0.160, AUROC 0.845, 95 % CI 0.829-0.862), which significantly outperformed CTAS (AUPRC 0.038, 95 % CI 0.035-0.042, AUROC 0.650, 95 % CI 0.628-0.671, p < 0.001 for all AUPRC and AUROC comparisons). The Predict Sepsis and Borelli scores exhibited sensitivity of 0.670 and 0.678 and specificity of 0.902 and 0.834, respectively, at their recommended cutoff values and outperformed Systemic Inflammatory Response Syndrome (SIRS) criteria (AUPRC 0.083, 95 % CI 0.070-0.102, p = 0.052 and p = 0.078, respectively; AUROC 0.775, 95 % CI 0.756-0.795, p < 0.001 for both scores).

CONCLUSIONS

The Predict Sepsis and Borelli scores exhibited improved performance including increased specificity and positive predictive values for sepsis identification at ED triage compared to CTAS and SIRS criteria.

Deep Learning for Epidemiologists: An Introduction to Neural Networks

Deep learning methods are increasingly being applied to problems in medicine and health care. However, few epidemiologists have received formal training in these methods. To bridge this gap, this article introduces the fundamentals of deep learning from an epidemiologic perspective. Specifically, this article reviews core concepts in machine learning (e.g., overfitting, regularization, and hyperparameters); explains several fundamental deep learning architectures (convolutional neural networks, recurrent neural networks); and summarizes training, evaluation, and deployment of models. Conceptual understanding of supervised learning algorithms is the focus of the article; instructions on the training of deep learning models and applications of deep learning to causal learning are out of this article's scope. We aim to provide an accessible first step towards enabling the reader to read and assess research on the medical applications of deep learning and to familiarize readers with deep learning terminology and concepts to facilitate communication with computer scientists and machine learning engineers.

Prospective Evaluation of a Dynamic Acuity Score for Regularly Assessing a Critically Ill Patient's Risk of Mortality

OBJECTIVE

Predictive models developed for use in ICUs have been based on retrospectively collected data, which does not take into account the challenges associated with live, clinical data. This study sought to determine if a previously constructed predictive model of ICU mortality (ViSIG) is robust when using data collected prospectively in near real-time.

DESIGN

Prospectively collected data were aggregated and transformed to evaluate a previously developed rolling predictor of ICU mortality.

SETTING

Five adult ICUs at Robert Wood Johnson-Barnabas University Hospital and one adult ICU at Stamford Hospital.

PATIENTS

One thousand eight hundred and ten admissions from August to December 2020.

MEASUREMENTS AND MAIN RESULTS

The ViSIG Score, comprised of severity weights for heart rate, respiratory rate, oxygen saturation, mean arterial pressure, mechanical ventilation, and values for OBS Medical's Visensia Index. This information was collected prospectively, whereas data on discharge disposition was collected retrospectively to measure the ViSIG Score's accuracy. The distribution of patients' maximum ViSIG Score was compared with ICU mortality rate, and cut points determined where changes in mortality probability were greatest. The ViSIG Score was validated on new admissions. The ViSIG Score was able to stratify patients into three groups: 0-37 (low risk), 38-58 (moderate risk), and 59-100 (high risk), with mortality of 1.7%, 12.0%, and 39.8%, respectively ( p < 0.001). The sensitivity and specificity of the model to predict mortality for the high-risk group were 51% and 91%. Performance on the validation dataset remained high. There were similar increases across risk groups for length of stay, estimated costs, and readmission.

CONCLUSIONS

Using prospectively collected data, the ViSIG Score produced risk groups for mortality with good sensitivity and excellent specificity. A future study will evaluate making the ViSIG Score visible to clinicians to determine whether this metric can influence clinician behavior to reduce adverse outcomes.

Prediction of postoperative complications after oesophagectomy using machine-learning methods

BACKGROUND

Oesophagectomy is an operation with a high risk of postoperative complications. The aim of this single-centre retrospective study was to apply machine-learning methods to predict complications (Clavien-Dindo grade IIIa or higher) and specific adverse events.

METHODS

Patients with resectable adenocarcinoma or squamous cell carcinoma of the oesophagus and gastro-oesophageal junction who underwent Ivor Lewis oesophagectomy between 2016 and 2021 were included. The tested algorithms were logistic regression after recursive feature elimination, random forest, k-nearest neighbour, support vector machine, and neural network. The algorithms were also compared with a current risk score (the Cologne risk score).

RESULTS

457 patients had Clavien-Dindo grade IIIa or higher complications (52.9 per cent) versus 407 patients with Clavien-Dindo grade 0, I, or II complications (47.1 per cent). After 3-fold imputation and 3-fold cross-validation, the overall accuracies were: logistic regression after recursive feature elimination, 0.528; random forest, 0.535; k-nearest neighbour, 0.491; support vector machine, 0.511; neural network, 0.688; and Cologne risk score, 0.510. For medical complications, the results were: logistic regression after recursive feature elimination, 0.688; random forest, 0.664; k-nearest neighbour, 0.673; support vector machine, 0.681; neural network, 0.692; and Cologne risk score, 0.650. For surgical complications, the results were: logistic regression after recursive feature elimination, 0.621; random forest, 0.617; k-nearest neighbour, 0.620; support vector machine, 0.634; neural network, 0.667; and Cologne risk score, 0.624. The calculated area under the curve of the neural network was 0.672 for Clavien-Dindo grade IIIa or higher, 0.695 for medical complications, and 0.653 for surgical complications.

CONCLUSION

The neural network scored the highest accuracies compared with all of the other models for the prediction of postoperative complications after oesophagectomy.

The principles of whole-hospital predictive analytics monitoring for clinical medicine originated in the neonatal ICU

In 2011, a multicenter group spearheaded at the University of Virginia demonstrated reduced mortality from real-time continuous cardiorespiratory monitoring in the neonatal ICU using what we now call Artificial Intelligence, Big Data, and Machine Learning. The large, randomized heart rate characteristics trial made real, for the first time that we know of, the promise that early detection of illness would allow earlier and more effective intervention and improved patient outcomes. Currently, though, we hear as much of failures as we do of successes in the rapidly growing field of predictive analytics monitoring that has followed. This Perspective aims to describe the principles of how we developed heart rate characteristics monitoring for neonatal sepsis and then applied them throughout adult ICU and hospital medicine. It primarily reflects the work since the 1990s of the University of Virginia group: the theme is that sudden and catastrophic deteriorations can be preceded by subclinical but measurable physiological changes apparent in the continuous cardiorespiratory monitoring and electronic health record.

Automated multilabel diagnosis on electrocardiographic images and signals

The application of artificial intelligence (AI) for automated diagnosis of electrocardiograms (ECGs) can improve care in remote settings but is limited by the reliance on infrequently available signal-based data. We report the development of a multilabel automated diagnosis model for electrocardiographic images, more suitable for broader use. A total of 2,228,236 12-lead ECGs signals from 811 municipalities in Brazil are transformed to ECG images in varying lead conformations to train a convolutional neural network (CNN) identifying 6 physician-defined clinical labels spanning rhythm and conduction disorders, and a hidden label for gender. The image-based model performs well on a distinct test set validated by at least two cardiologists (average AUROC 0.99, AUPRC 0.86), an external validation set of 21,785 ECGs from Germany (average AUROC 0.97, AUPRC 0.73), and printed ECGs, with performance superior to signal-based models, and learning clinically relevant cues based on Grad-CAM. The model allows the application of AI to ECGs across broad settings.

On Missingness Features in Machine Learning Models for Critical Care: Observational Study

Background

Missing data in electronic health records is inevitable and considered to be nonrandom. Several studies have found that features indicating missing patterns (missingness) encode useful information about a patient’s health and advocate for their inclusion in clinical prediction models. But their effectiveness has not been comprehensively evaluated.

Objective

The goal of the research is to study the effect of including informative missingness features in machine learning models for various clinically relevant outcomes and explore robustness of these features across patient subgroups and task settings.

Methods

A total of 48,336 electronic health records from the 2012 and 2019 PhysioNet Challenges were used, and mortality, length of stay, and sepsis outcomes were chosen. The latter dataset was multicenter, allowing external validation. Gated recurrent units were used to learn sequential patterns in the data and classify or predict labels of interest. Models were evaluated on various criteria and across population subgroups evaluating discriminative ability and calibration.

Results

Generally improved model performance in retrospective tasks was observed on including missingness features. Extent of improvement depended on the outcome of interest (area under the curve of the receiver operating characteristic [AUROC] improved from 1.2% to 7.7%) and even patient subgroup. However, missingness features did not display utility in a simulated prospective setting, being outperformed (0.9% difference in AUROC) by the model relying only on pathological features. This was despite leading to earlier detection of disease (true positives), since including these features led to a concomitant rise in false positive detections.

Conclusions

This study comprehensively evaluated effectiveness of missingness features on machine learning models. A detailed understanding of how these features affect model performance may lead to their informed use in clinical settings especially for administrative tasks like length of stay prediction where they present the greatest benefit. While missingness features, representative of health care processes, vary greatly due to intra- and interhospital factors, they may still be used in prediction models for clinically relevant outcomes. However, their use in prospective models producing frequent predictions needs to be explored further.

Explainable Machine Learning on AmsterdamUMCdb for ICU Discharge Decision Support: Uniting Intensivists and Data Scientists

Unexpected ICU readmission is associated with longer length of stay and increased mortality. To prevent ICU readmission and death after ICU discharge, our team of intensivists and data scientists aimed to use AmsterdamUMCdb to develop an explainable machine learning–based real-time bedside decision support tool.

Use of Machine Learning Models to Predict Death After Acute Myocardial Infarction

Importance

Accurate prediction of adverse outcomes after acute myocardial infarction (AMI) can guide the triage of care services and shared decision-making, and novel methods hold promise for using existing data to generate additional insights.

Objective

To evaluate whether contemporary machine learning methods can facilitate risk prediction by including a larger number of variables and identifying complex relationships between predictors and outcomes.

Design, Setting, and Participants

This cohort study used the American College of Cardiology Chest Pain-MI Registry to identify all AMI hospitalizations between January 1, 2011, and December 31, 2016. Data analysis was performed from February 1, 2018, to October 22, 2020.

Main Outcomes and Measures

Three machine learning models were developed and validated to predict in-hospital mortality based on patient comorbidities, medical history, presentation characteristics, and initial laboratory values. Models were developed based on extreme gradient descent boosting (XGBoost, an interpretable model), a neural network, and a meta-classifier model. Their accuracy was compared against the current standard developed using a logistic regression model in a validation sample.

Results

A total of 755 402 patients (mean [SD] age, 65 [13] years; 495 202 [65.5%] male) were identified during the study period. In independent validation, 2 machine learning models, gradient descent boosting and meta-classifier (combination including inputs from gradient descent boosting and a neural network), marginally improved discrimination compared with logistic regression (C statistic, 0.90 for best performing machine learning model vs 0.89 for logistic regression). Nearly perfect calibration in independent validation data was found in the XGBoost (slope of predicted to observed events, 1.01; 95% CI, 0.99-1.04) and the meta-classifier model (slope of predicted-to-observed events, 1.01; 95% CI, 0.99-1.02), with more precise classification across the risk spectrum. The XGBoost model reclassified 32 393 of 121 839 individuals (27%) and the meta-classifier model reclassified 30 836 of 121 839 individuals (25%) deemed at moderate to high risk for death in logistic regression as low risk, which were more consistent with the observed event rates.

Conclusions and Relevance

In this cohort study using a large national registry, none of the tested machine learning models were associated with substantive improvement in the discrimination of in-hospital mortality after AMI, limiting their clinical utility. However, compared with logistic regression, XGBoost and meta-classifier models, but not the neural network, offered improved resolution of risk for high-risk individuals.

Observational Research for Therapies Titrated to Effect and Associated With Severity of Illness: Misleading Results From Commonly Used Statistical Methods

OBJECTIVES

In critically ill patients, treatment dose or intensity is often related to severity of illness and mortality risk, whereas overtreatment or undertreatment (relative to the individual need) may further increase the odds of death. We aimed to investigate how these relationships affect the results of common statistical methods used in observational studies.

DESIGN

Using Monte Carlo simulation, we generated data for 5,000 patients with a treatment dose related to the pretreatment mortality risk but with randomly distributed overtreatment or undertreatment. Significant overtreatment or undertreatment (relative to the optimal dose) further increased the mortality risk. A prognostic score that reflects the mortality risk and an outcome of death or survival was then generated. The study was analyzed: 1) using logistic regression to estimate the effect of treatment dose on outcome while controlling for prognostic score and 2) using propensity score matching and inverse probability weighting of the effect of high treatment dose on outcome. The data generation and analyses were repeated 1,500 times over sample sizes between 200 and 30,000 patients, with an increasing accuracy of the prognostic score and with different underlying assumptions.

SETTING

Computer-simulated studies.

MEASUREMENTS AND MAIN RESULTS

In the simulated 5,000-patient observational study, higher treatment dose was found to be associated with increased odds of death (p = 0.00001) while controlling for the prognostic score with logistic regression. Propensity-matched analysis led to similar results. Larger sample sizes led to equally biased estimates with narrower CIs. A perfect risk predictor negated the bias only under artificially perfect assumptions.

CONCLUSIONS

When a treatment dose is associated with severity of illness and should be dosed "enough," logistic regression, propensity score matching, and inverse probability weighting to adjust for confounding by severity of illness lead to biased results. Larger sample sizes lead to more precisely wrong estimates.

Development and Reporting of Prediction Models: Guidance for Authors From Editors of Respiratory, Sleep, and Critical Care Journals

Supplemental Digital Content is available in the text.

Prediction models aim to use available data to predict a health state or outcome that has not yet been observed. Prediction is primarily relevant to clinical practice, but is also used in research, and administration. While prediction modeling involves estimating the relationship between patient factors and outcomes, it is distinct from casual inference. Prediction modeling thus requires unique considerations for development, validation, and updating. This document represents an effort from editors at 31 respiratory, sleep, and critical care medicine journals to consolidate contemporary best practices and recommendations related to prediction study design, conduct, and reporting. Herein, we address issues commonly encountered in submissions to our various journals. Key topics include considerations for selecting predictor variables, operationalizing variables, dealing with missing data, the importance of appropriate validation, model performance measures and their interpretation, and good reporting practices. Supplemental discussion covers emerging topics such as model fairness, competing risks, pitfalls of “modifiable risk factors”, measurement error, and risk for bias. This guidance is not meant to be overly prescriptive; we acknowledge that every study is different, and no set of rules will fit all cases. Additional best practices can be found in the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines, to which we refer readers for further details.

A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation

Supplemental Digital Content is available in the text.

Objectives:

To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score.

Design:

Retrospective study.

Setting:

Quaternary care medical-surgical PICU.

Patients:

All patients admitted to the PICU from 2013 to 2019.

Interventions:

None.

Measurements and Main Results:

We investigated the performance of various machine learning algorithms using the same variables used to calculate the Pediatric Logistic Organ Dysfunction-2 score to predict PICU mortality. We used 10,194 patient records from 2013 to 2017 for training and 4,043 patient records from 2018 to 2019 as a holdout validation cohort. Mortality rate was 3.0% in the training cohort and 3.4% in the validation cohort. The best performing algorithm was a random forest model (area under the receiver operating characteristic curve, 0.867 [95% CI, 0.863–0.895]; area under the precision-recall curve, 0.327 [95% CI, 0.246–0.414]; F1, 0.396 [95% CI, 0.321–0.468]) and significantly outperformed the Pediatric Logistic Organ Dysfunction-2 score (area under the receiver operating characteristic curve, 0.761 [95% CI, 0.713–0.810]; area under the precision-recall curve (0.239 [95% CI, 0.165–0.316]; F1, 0.284 [95% CI, 0.209–0.360]), although this difference was reduced after retraining the Pediatric Logistic Organ Dysfunction-2 logistic regression model at the study institution. The random forest model also showed better calibration than the Pediatric Logistic Organ Dysfunction-2 score, and calibration of the random forest model remained superior to the retrained Pediatric Logistic Organ Dysfunction-2 model.

Conclusions:

A machine learning model achieved better performance than a logistic regression-based score for predicting ICU mortality. Better estimation of mortality risk can improve our ability to adjust for severity of illness in future studies, although external validation is required before this method can be widely deployed.

Prognostic models for predicting in-hospital paediatric mortality in resource-limited countries: a systematic review

OBJECTIVES

To identify and appraise the methodological rigour of multivariable prognostic models predicting in-hospital paediatric mortality in low-income and middle-income countries (LMICs).

DESIGN

Systematic review of peer-reviewed journals.

DATA SOURCES

MEDLINE, CINAHL, Google Scholar and Web of Science electronic databases since inception to August 2019.

ELIGIBILITY CRITERIA

We included model development studies predicting in-hospital paediatric mortality in LMIC.

DATA EXTRACTION AND SYNTHESIS

This systematic review followed the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies framework. The risk of bias assessment was conducted using Prediction model Risk of Bias Assessment Tool (PROBAST). No quantitative summary was conducted due to substantial heterogeneity that was observed after assessing the studies included.

RESULTS

Our search strategy identified a total of 4054 unique articles. Among these, 3545 articles were excluded after review of titles and abstracts as they covered non-relevant topics. Full texts of 509 articles were screened for eligibility, of which 15 studies reporting 21 models met the eligibility criteria. Based on the PROBAST tool, risk of bias was assessed in four domains; participant, predictors, outcome and analyses. The domain of statistical analyses was the main area of concern where none of the included models was judged to be of low risk of bias.

CONCLUSION

This review identified 21 models predicting in-hospital paediatric mortality in LMIC. However, most reports characterising these models are of poor quality when judged against recent reporting standards due to a high risk of bias. Future studies should adhere to standardised methodological criteria and progress from identifying new risk scores to validating or adapting existing scores.

PROSPERO REGISTRATION NUMBER

CRD42018088599.

Identification of newborns with birthweight ≥ 4,500g: Ultrasound within one- vs. two weeks of delivery

OBJECTIVE

Our objective was to compare the diagnostic characteristics of sonographic estimated fetal weight (SEFW) done within 7 versus 8-14 days before delivery for detection of fetal macrosomia (birthweight ≥ 4500 g).

STUDY DESIGN

We performed a multicenter, retrospective cohort study of all non-anomalous singletons with SEFW ≥ 4000 g by Registered Diagnostic Medical Sonographers conducted within 14 days of delivery. Cohorts were grouped by time interval between ultrasound and delivery: 0-7 days versus 8-14 days. The detection rate (DR) and false positive rate (FPR) for detection of birthweight (BW) ≥ 4500 g were compared between groups with subgroup analysis for diabetic women. Area under the receiver operator curve (AUC) was calculated to analyze all possible SEFW cutoffs within our cohort.

RESULTS

A total of 330 patients met inclusion criteria with 250 (75.8 %) having SEFW within 7 days and 80 (24.2 %) with SEFW 8-14 days prior to delivery. The rate of macrosomia was 15.1 % (N = 51). The DR for macrosomia was significantly higher when SEFW was performed within 7 days of delivery compared to 8-14 days among non-diabetic (73.0 % vs 7.1 %; p < 0.001) and diabetic women (76.5 % vs 16.7 %; p = 0.02). There was no significant change in FPR in either group. The AUC for detection of macrosomia was significantly higher when SEFW was performed within 7 days versus 8-14 days (0.89 vs 0.63; p < 0.01).

CONCLUSION

With SEFW ≥ 4000 g, the detection of BW ≥ 4500 g is significantly higher when the sonographic examination is within 7 days of birth irrespective of maternal diabetes.

Feasibility, reproducibility and diagnostic usefulness of right ventricular strain by 2-dimensional speckle-tracking echocardiography in ARDS patients: the ARD strain study

BACKGROUND

Right ventricular (RV) function evaluation by echocardiography is key in the management of ICU patients with acute respiratory distress syndrome (ARDS), however, it remains challenging. Quantification of RV deformation by speckle-tracking echocardiography (STE) is a recently available and reproducible technique that provides an integrated analysis of the RV. However, data are scarce regarding its use in critically ill patients. The aim of this study was to assess its feasibility and clinical usefulness in moderate-severe ARDS patients.

RESULTS

Forty-eight ARDS patients under invasive mechanical ventilation (MV) were consecutively enrolled in a prospective observational study. A full transthoracic echocardiography was performed within 36 h of MV initiation. STE-derived and conventional parameters were recorded. Strain imaging of the RV lateral, inferior and septal walls was highly feasible (47/48 (98%) patients). Interobserver reproducibility of RV strain values displayed good reliability (intraclass correlation coefficients (ICC) > 0.75 for all STE-derived parameters) in ARDS patients. ROC curve analysis showed that lateral, inferior, global (average of the 3 RV walls) longitudinal systolic strain (LSS) and global strain rate demonstrated significant diagnostic values when compared to several conventional indices (TAPSE, S', RV FAC). A RV global LSS value > - 13.7% differentiated patients with a TAPSE < vs > 12 mm with a sensitivity of 88% and a specificity of 83%. Regarding clinical outcomes, mortality and cumulative incidence of weaning from MV at day 28 were not different in patients with normal versus abnormal STE-derived parameters.

CONCLUSIONS

Global STE assessment of the RV was highly achievable and reproducible in moderate-severe ARDS patients under MV and additionally correlated with several conventional parameters of RV function. In our cohort, STE-derived parameters did not provide any incremental value in terms of survival or weaning from MV prediction. Further investigations are needed to evaluate their theranostic usefulness. Trial registration NCT02638844: NCT.

Heart Rate Variability as a Biomarker of Neurocardiogenic Injury After Subarachnoid Hemorrhage

BACKGROUND

The objective of this study was to examine whether heart rate variability (HRV) measures can be used to detect neurocardiogenic injury (NCI).

METHODS

Three hundred and twenty-six consecutive admissions with aneurysmal subarachnoid hemorrhage (SAH) met criteria for the study. Of 326 subjects, 56 (17.2%) developed NCI which we defined by wall motion abnormality with ventricular dysfunction on transthoracic echocardiogram or cardiac troponin-I > 0.3 ng/mL without electrocardiogram evidence of coronary artery insufficiency. HRV measures (in time and frequency domains, as well as nonlinear technique of detrended fluctuation analysis) were calculated over the first 48 h. We applied longitudinal multilevel linear regression to characterize the relationship of HRV measures with NCI and examine between-group differences at baseline and over time.

RESULTS

There was decreased vagal activity in NCI subjects with a between-group difference in low/high frequency ratio (β 3.42, SE 0.92, p = 0.0002), with sympathovagal balance in favor of sympathetic nervous activity. All time-domain measures were decreased in SAH subjects with NCI. An ensemble machine learning approach translated these measures into a classification tool that demonstrated good discrimination using the area under the receiver operating characteristic curve (AUROC 0.82), the area under precision recall curve (AUPRC 0.75), and a correct classification rate of 0.81.

CONCLUSIONS

HRV measures are significantly associated with our label of NCI and a machine learning approach using features derived from HRV measures can classify SAH patients that develop NCI.

An Evaluation of the Influence of Body Mass Index on Severity Scoring

OBJECTIVES

Although one third or more of critically ill patients in the United States are obese, obesity is not incorporated as a contributing factor in any of the commonly used severity of illness scores. We hypothesize that selected severity of illness scores would perform differently if body mass index categorization was incorporated and that the performance of these score models would improve after consideration of body mass index as an additional model feature.

DESIGN

Retrospective cohort analysis from a multicenter ICU database which contains deidentified data for more than 200,000 ICU admissions from 208 distinct ICUs across the United States between 2014 and 2015.

SETTING

First ICU admission of patients with documented height and weight.

PATIENTS

One-hundred eight-thousand four-hundred two patients from 189 different ICUs across United States were included in the analyses, of whom 4,661 (4%) were classified as underweight, 32,134 (30%) as normal weight, 32,278 (30%) as overweight, 30,259 (28%) as obese, and 9,070 (8%) as morbidly obese.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

To assess the effect of adding body mass index as a risk adjustment element to the Acute Physiology and Chronic Health Evaluation IV and Oxford Acute Severity of Illness scoring systems, we examined the impact of this addition on both discrimination and calibration. We performed three assessments based upon 1) the original scoring systems, 2) a recalibrated version of the systems, and 3) a recalibrated version incorporating body mass index as a covariate. We also performed a subgroup analysis in groups defined using World Health Organization guidelines for obesity. Incorporating body mass index into the models provided a minor improvement in both discrimination and calibration. In a subgroup analysis, model discrimination was higher in groups with higher body mass index, but calibration worsened.

CONCLUSIONS

The performance of ICU prognostic models utilizing body mass index category as a scoring element was inconsistent across body mass index categories. Overall, adding body mass index as a risk adjustment variable led only to a minor improvement in scoring system performance.

Predictive performances of STESS and EMSE in a Norwegian adult status epilepticus cohort

PURPOSE

"Status Epilepticus Severity Score" (STESS) and "Epidemiology-based Mortality Score in Status Epilepticus" (EMSE) are two clinical scoring systems aiming to predict mortality in status epilepticus (SE). The objective of this study was to compare their predictive performances in a cohort of 151 SE-patients from Oslo University Hospital in the period 2001-2017.

METHOD

Variables used to calculate STESS (age, previous seizures, worst SE-semiology, level of consciousness) and two different versions of EMSE, EMSE-EAC (etiology, age, comorbidities) and EMSE-EACE (etiology, age, comorbidities, EEG-pattern), as well as outcome were collected retrospectively. Receiver Operating Characteristic (ROC)-analyses, determination of best cut-off values, sensitivity (Se), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV) were performed. In addition, Precision-Recall curves (PRC) were produced, plotting PPV as a function of Se.

RESULTS

Thirteen patients (9%) died during their hospital stay. STESS did not accurately predict mortality, with a ROC-curve showing an area under the curve (AUC) of 0.625(95%CI = 0.472-0.783), p = 0.15. EMSE-EAC performed better with an AUC of 0.714(95%CI = 0.552-0.873), p = 0.01 and a best cut-off value of 37. Se was 69.2%, Sp 72.1%, PPV 19% and NPV 96.2%. EMSE-EACE performed best with an AUC of 0.855(95%CI = 0.736-0.976), p < 0.0005 and a best cut-off value of 79. Se was 77.8%, Sp 87.8%, PPV 36.8% and NPV 97.7%. The PRC showed areas under the PRC of 0.23 for EMSE-EAC and 0.46 for EMSE-EACE.

CONCLUSIONS

EMSE-EAC and EMSE-EACE performed better than STESS and may be useful in identifying the patients at risk of death in SE. PRC may give a more relevant visual representation of predictive utility than ROC-curves in situations of imbalanced datasets.

Critical Care, Critical Data

As big data, machine learning, and artificial intelligence continue to penetrate into and transform many facets of our lives, we are witnessing the emergence of these powerful technologies within health care. The use and growth of these technologies has been contingent on the availability of reliable and usable data, a particularly robust resource in critical care medicine where continuous monitoring forms a key component of the infrastructure of care. The response to this opportunity has included the development of open databases for research and other purposes; the development of a collaborative form of clinical data science intended to fully leverage these data resources, and the creation of data-driven applications for purposes such as clinical decision support. Most recently, data levels have reached the thresholds required for the development of robust artificial intelligence features for clinical purposes. The systematic capture and analysis of clinical data in both individuals and populations allows us to begin to move toward precision medicine in the intensive care unit (ICU). In this perspective review, we examine the fundamental role of data as we present the current progress that has been made toward an artificial intelligence (AI)-supported, data-driven precision critical care medicine.

Using the Shapes of Clinical Data Trajectories to Predict Mortality in ICUs

Supplemental Digital Content is available in the text.

1) To show how to exploit the information contained in the trajectories of time-varying patient clinical data for dynamic predictions of mortality in the ICU; and 2) to demonstrate the additional predictive value that can be achieved by incorporating this trajectory information.