Discovery and integration of univariate patterns from daily individual organ-failure scores for intensive care mortality prediction

Using the diagnostic odds ratio to select multivariate sequential patterns in order to build an interpretable pattern-based classifier in a clinical domain (Preprint)

Background

It is important to exploit all available data on patients in settings such as intensive care burn units (ICBUs), where several variables are recorded over time. It is possible to take advantage of the multivariate patterns that model the evolution of patients to predict their survival. However, pattern discovery algorithms generate a large number of patterns, of which only some are relevant for classification.

Objective

We propose to use the diagnostic odds ratio (DOR) to select multivariate sequential patterns used in the classification in a clinical domain, rather than employing frequency properties.

Methods

We used data obtained from the ICBU at the University Hospital of Getafe, where 6 temporal variables for 465 patients were registered every day during 5 days, and to model the evolution of these clinical variables, we used multivariate sequential patterns by applying 2 different discretization methods for the continuous attributes. We compared 4 ways in which to employ the DOR for pattern selection: (1) we used it as a threshold to select patterns with a minimum DOR; (2) we selected patterns whose differential DORs are higher than a threshold with regard to their extensions; (3) we selected patterns whose DOR CIs do not overlap; and (4) we proposed the combination of threshold and nonoverlapping CIs to select the most discriminative patterns. As a baseline, we compared our proposals with Jumping Emerging Patterns, one of the most frequently used techniques for pattern selection that utilizes frequency properties.

Results

We have compared the number and length of the patterns eventually selected, classification performance, and pattern and model interpretability. We show that discretization has a great impact on the accuracy of the classification model, but that a trade-off must be found between classification accuracy and the physicians’ capacity to interpret the patterns obtained. We have also identified that the experiments combining threshold and nonoverlapping CIs (Option 4) obtained the fewest number of patterns but also with the smallest size, thus implying the loss of an acceptable accuracy with regard to clinician interpretation. The best classification model according to the trade-off is a JRIP classifier with only 5 patterns (20 items) that was built using unsupervised correlation preserving discretization and differential DOR in a beam search for the best pattern. It achieves a specificity of 56.32% and an area under the receiver operating characteristic curve of 0.767.

Conclusions

A method for the classification of patients’ survival can benefit from the use of sequential patterns, as these patterns consider knowledge about the temporal evolution of the variables in the case of ICBU. We have proved that the DOR can be used in several ways, and that it is a suitable measure to select discriminative and interpretable quality patterns.

Early noncardiovascular organ failure and mortality in the cardiac intensive care unit

Background

Noncardiac organ failure has been associated with worse outcomes among a cardiac intensive care unit (CICU) population.

Hypothesis

We hypothesized that early organ failure based on the sequential organ failure assessment (SOFA) score would be associated with mortality in CICU patients.

Methods

Adult CICU patients from 2007 to 2015 were reviewed. Organ failure was defined as any SOFA organ subscore ≥3 on the first CICU day. Organ failure was evaluated as a predictor of hospital mortality and postdischarge survival after adjustment for illness severity and comorbidities.

Results

We included 10 004 patients with a mean age of 67 ± 15 years (37% female). Admission diagnoses included acute coronary syndrome in 43%, heart failure in 46%, cardiac arrest in 12%, and cardiogenic shock in 11%. Organ failure was present in 31%, including multiorgan failure in 12%. Hospital mortality was higher in patients with organ failure (22% vs 3%, adjusted OR 3.0, 95% CI 2.5‐3.7, P < .001). After adjustment, each failing organ system predicted twofold higher odds of hospital mortality (adjusted OR 1.9, 95% CI 1.1‐2.1, P < .001). Mortality risk was highest with cardiovascular, coagulation and liver failure. Among hospital survivors, organ failure was associated with higher adjusted postdischarge mortality risk (P < .001); multiorgan failure did not confer added long‐term mortality risk.

Conclusions

Early noncardiovascular organ failure, especially multiorgan failure, is associated with increased hospital mortality in CICU patients, and this risk continues after hospital discharge, emphasizing the need to promote early recognition of organ failure in CICU patients.

Predictive value of individual Sequential Organ Failure Assessment sub-scores for mortality in the cardiac intensive care unit

Purpose

To determine the impact of Sequential Organ Failure Assessment (SOFA) organ sub-scores for hospital mortality risk stratification in a contemporary cardiac intensive care unit (CICU) population.

Materials and methods

Adult CICU admissions between January 1, 2007 and December 31, 2015 were reviewed. The SOFA score and organ sub-scores were calculated on CICU day 1; patients with missing SOFA sub-score data were excluded. Discrimination for hospital mortality was assessed using area under the receiver-operator characteristic curve (AUROC) values, followed by multivariable logistic regression.

Results

We included 1214 patients with complete SOFA sub-score data. The mean age was 67 ± 16 years (38% female); all-cause hospital mortality was 26%. Day 1 SOFA score predicted hospital mortality with an AUROC of 0.72. Each SOFA organ sub-score predicted hospital mortality (all p <0.01), with AUROC values of 0.53 to 0.67. On multivariable analysis, only the cardiovascular, central nervous system, renal and respiratory SOFA sub-scores were associated with hospital mortality (all p <0.01). A simplified SOFA score containing the cardiovascular, central nervous system and renal sub-scores had an AUROC of 0.72.

Conclusions

In CICU patients with complete SOFA sub-score data, risk stratification for hospital mortality is determined primarily by the cardiovascular, central nervous system, renal and respiratory SOFA sub-scores.

Factors Influencing How Intensive Care Unit Nurses Allocate Their Time

Spending time with the patient is essential for intensive care unit (ICU) nurses to detect clinical change. This article reports on an examination of factors influencing nurses’ activity time allocation. Data were analyzed from a prospective time and motion study of medical ICU nurses. Nurse demographic data and observation, electronic locator technology, and electronic medical record log data were collected over 12 days from 11 registered nurses. Charlson Co-Morbidity Index and Sequential Organ Failure Assessment scores were calculated for patient assignments. Nurses averaged 78.04 ( SD = 47.85) min per patient on activities in the patient room. Years of ICU nursing experience and the patient’s Charlson Co-Morbidity Index was significantly associated with time spent in the patient’s room. Neither nursing education nor specialty certification was found to influence time spent in a patient’s room. Using technology can advance understanding of nurses’ time allocation leading to interventions optimizing time spent with the patient.

Prediction of Outcome From Adult Bacterial Meningitis in a High-HIV-Seroprevalence, Resource-Poor Setting Using the Malawi Adult Meningitis Score (MAMS)

Background.

Acute bacterial meningitis (ABM) in adults residing in resource-poor countries is associated with mortality rates >50%. To improve outcome, interventional trials and standardized clinical algorithms are urgently required. To optimize these processes, we developed and validated an outcome prediction tool to identify ABM patients at greatest risk of death.

Methods.

We derived a nomogram using mortality predictors derived from a logistic regression model of a discovery database of adult Malawian patients with ABM (n = 523 [65%] cerebrospinal fluid [CSF] culture positive). We validated the nomogram internally using a bootstrap procedure and subsequently used the nomogram scores to further interpret the effects of adjunctive dexamethasone and glycerol using clinical trial data from Malawi.

Results.

ABM mortality at 6-week follow-up was 54%. Five of 15 variables tested were strongly associated with poor outcome (CSF culture positivity, CSF white blood cell count, hemoglobin, Glasgow Coma Scale, and pulse rate), and were used in the derivation of the Malawi Adult Meningitis Score (MAMS) nomogram. The C-index (area under the curve) was 0.76 (95% confidence interval, .71–.80) and calibration was good (Hosmer-Lemeshow C-statistic = 5.48, df = 8, P = .705). Harmful effects of adjunctive glycerol were observed in groups with relatively low predicted risk of poor outcome (25%–50% risk): Case Fatality Rate of 21% in the placebo group and 52% in the glycerol group (P < .001). This effect was not seen with adjunctive dexamethasone.

Conclusions.

MAMS provides a novel tool for predicting prognosis and improving interpretation of ABM clinical trials by risk stratification in resource-poor settings. Whether MAMS can be applied to non-HIV-endemic countries requires further evaluation.

Cross-validation of a Sequential Organ Failure Assessment score-based model to predict mortality in patients with cancer admitted to the intensive care unit

PURPOSE

This study aims to validate the performance of the Sequential Organ Failure Assessment (SOFA) score to predict death of critically ill patients with cancer.

MATERIAL AND METHODS

We conducted a retrospective observational study including adults admitted to the intensive care unit (ICU) between January 1, 2006, and December 31, 2008. We randomly selected training and validation samples in medical and surgical admissions to predict ICU and in-hospital mortality. By using logistic regression, we calculated the probabilities of death in the training samples and applied them to the validation samples to test the goodness-of-fit of the models, construct receiver operator characteristics curves, and calculate the areas under the curve (AUCs).

RESULTS

In predicting mortality at discharge from the unit, the AUC from the validation group of medical admissions was 0.7851 (95% confidence interval [CI], 0.7437-0.8264), and the AUC from the surgical admissions was 0.7847 (95% CI, 0.6319-0.937). The AUCs of the SOFA score to predict mortality in the hospital after ICU admission were 0.7789 (95% CI, 0.74-0.8177) and 0.7572 (95% CI, 0.6719-0.8424) for the medical and surgical validations groups, respectively.

CONCLUSIONS

The SOFA score had good discrimination to predict ICU and hospital mortality. However, the observed underestimation of ICU deaths and unsatisfactory goodness-of-fit test of the model in surgical patients to indicate calibration of the score to predict ICU mortality is advised in this group.

Prediction of clinical conditions after coronary bypass surgery using dynamic data analysis

This work studies the impact of using dynamic information as features in a machine learning algorithm for the prediction task of classifying critically ill patients in two classes according to the time they need to reach a stable state after coronary bypass surgery: less or more than 9 h. On the basis of five physiological variables (heart rate, systolic arterial blood pressure, systolic pulmonary pressure, blood temperature and oxygen saturation), different dynamic features were extracted, namely the means and standard deviations at different moments in time, coefficients of multivariate autoregressive models and cepstral coefficients. These sets of features served subsequently as inputs for a Gaussian process and the prediction results were compared with the case where only admission data was used for the classification. The dynamic features, especially the cepstral coefficients (aROC: 0.749, Brier score: 0.206), resulted in higher performances when compared to static admission data (aROC: 0.547, Brier score: 0.247). The differences in performance are shown to be significant. In all cases, the Gaussian process classifier outperformed to logistic regression.

A novel time series analysis approach for prediction of dialysis in critically ill patients using echo-state networks

Background

Echo-state networks (ESN) are part of a group of reservoir computing methods and are basically a form of recurrent artificial neural networks (ANN). These methods can perform classification tasks on time series data. The recurrent ANN of an echo-state network has an 'echo-state' characteristic. This 'echo-state' functions as a fading memory: samples that have been introduced into the network in a further past, are faded away. The echo-state approach for the training of recurrent neural networks was first described by Jaeger H. et al. In clinical medicine, until this moment, no original research articles have been published to examine the use of echo-state networks.

Methods

This study examines the possibility of using an echo-state network for prediction of dialysis in the ICU. Therefore, diuresis values and creatinine levels of the first three days after ICU admission were collected from 830 patients admitted to the intensive care unit (ICU) between May 31th 2003 and November 17th 2007. The outcome parameter was the performance by the echo-state network in predicting the need for dialysis between day 5 and day 10 of ICU admission. Patients with an ICU length of stay <10 days or patients that received dialysis in the first five days of ICU admission were excluded. Performance by the echo-state network was then compared by means of the area under the receiver operating characteristic curve (AUC) with results obtained by two other time series analysis methods by means of a support vector machine (SVM) and a naive Bayes algorithm (NB).

Results

The AUC's in the three developed echo-state networks were 0.822, 0.818, and 0.817. These results were comparable to the results obtained by the SVM and the NB algorithm.

Conclusions

This proof of concept study is the first to evaluate the performance of echo-state networks in an ICU environment. This echo-state network predicted the need for dialysis in ICU patients. The AUC's of the echo-state networks were good and comparable to the performance of other classification algorithms. Moreover, the echo-state network was more easily configured than other time series modeling technologies.

Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic review

Introduction

To systematically review studies evaluating the performance of Sequential Organ Failure Assessment (SOFA)-based models for predicting mortality in patients in the intensive care unit (ICU).

Methods

Medline, EMBASE and other databases were searched for English-language articles with the major objective of evaluating the prognostic performance of SOFA-based models in predicting mortality in surgical and/or medical ICU admissions. The quality of each study was assessed based on a quality framework for prognostic models.

Results

Eighteen articles met all inclusion criteria. The studies differed widely in the SOFA derivatives used and in their methods of evaluation. Ten studies reported about developing a probabilistic prognostic model, only five of which used an independent validation data set. The other studies used the SOFA-based score directly to discriminate between survivors and non-survivors without fitting a probabilistic model. In five of the six studies, admission-based models (Acute Physiology and Chronic Health Evaluation (APACHE) II/III) were reported to have a slightly better discrimination ability than SOFA-based models at admission (the receiver operating characteristic curve (AUC) of SOFA-based models ranged between 0.61 and 0.88), and in one study a SOFA model had higher AUC than the Simplified Acute Physiology Score (SAPS) II model. Four of these studies used the Hosmer-Lemeshow tests for calibration, none of which reported a lack of fit for the SOFA models. Models based on sequential SOFA scores were described in 11 studies including maximum SOFA scores and maximum sum of individual components of the SOFA score (AUC range: 0.69 to 0.92) and delta SOFA (AUC range: 0.51 to 0.83). Studies comparing SOFA with other organ failure scores did not consistently show superiority of one scoring system to another. Four studies combined SOFA-based derivatives with admission severity of illness scores, and they all reported on improved predictions for the combination. Quality of studies ranged from 11.5 to 19.5 points on a 20-point scale.

Conclusions

Models based on SOFA scores at admission had only slightly worse performance than APACHE II/III and were competitive with SAPS II models in predicting mortality in patients in the general medical and/or surgical ICU. Models with sequential SOFA scores seem to have a comparable performance with other organ failure scores. The combination of sequential SOFA derivatives with APACHE II/III and SAPS II models clearly improved prognostic performance of either model alone. Due to the heterogeneity of the studies, it is impossible to draw general conclusions on the optimal mathematical model and optimal derivatives of SOFA scores. Future studies should use a standard evaluation methodology with a standard set of outcome measures covering discrimination, calibration and accuracy.