Discovery and inclusion of SOFA score episodes in mortality prediction

Observational study of missing SOFA score data frequency in RCTs relative to ICU length of stay

The Sequential Organ Failure Assessment, also known as SOFA score, was introduced to assess organ dysfunction of critical ill patients. However, understanding the impact of missing SOFA scores in randomized controlled trials and how this affect the validity and applicability of the SOFA score as a surrogate endpoint for predicting mortality has been a matter of interest. To address this, a secondary analysis of a systematic review was conducted to quantify the relationship between SOFA scores and the prediction of mortality in critically ill adults in randomized controlled trials (RCTs). The systematic review being referred to included 87 RCTs with a total of 12,064 critically ill patients. This analysis focused on missing SOFA score data in relation to the length of stay on the intensive care unit (ICU) and the methods used to handle missing data. SOFA score measurements from the included studies were categorized into three time frames: Early (t ≤ 4 days), Intermediate (t = 5-10 days) and Late (t > 10 days) measurement. Only one study reported a complete data set for calculating the SOFA score for an Early measurement. When considering all methods used to address missing data, 32% of studies still had missing data for Early measurements, and this percentage increased to 64% for Late measurements. These findings suggested that, over time, the number of studies with incomplete data sets has been increasing. The longer a patient is treated on the ICU, the higher the number of missing data which can impact the validity of SOFA score analyses. There was no clear trend towards a specific method for compensating missing data. An exemplary calculation demonstrated that ignoring missing data may lead to an underestimated variability of the treatment effect. This, in turn, could bias the interpretation of study results by policy- and clinical decision-makers. Overall, there are several limitations that need to be considered when using SOFA score as a surrogate endpoint for mortality. When employed as an outcome, the SOFA score is frequently missing and most studies do not adequately describe the amount or nature of missing data, or the methods used to handle missing data in the analysis.

Development and validation of prognostic machine learning models for short- and long-term mortality among acutely admitted patients based on blood tests

Several scores predicting mortality at the emergency department have been developed. However, all with shortcomings either simple and applicable in a clinical setting, with poor performance, or advanced, with high performance, but clinically difficult to implement. This study aimed to explore if machine learning algorithms could predict all-cause short- and long-term mortality based on the routine blood test collected at admission.

METHODS

We analyzed data from a retrospective cohort study, including patients > 18 years admitted to the Emergency Department (ED) of Copenhagen University Hospital Hvidovre, Denmark between November 2013 and March 2017. The primary outcomes were 3-, 10-, 30-, and 365-day mortality after admission. PyCaret, an automated machine learning library, was used to evaluate the predictive performance of fifteen machine learning algorithms using the area under the receiver operating characteristic curve (AUC).

RESULTS

Data from 48,841 admissions were analyzed, of these 34,190 (70%) were randomly divided into training data, and 14,651 (30%) were in test data. Eight machine learning algorithms achieved very good to excellent results of AUC on test data in a of range 0.85-0.93. In prediction of short-term mortality, lactate dehydrogenase (LDH), leukocyte counts and differentials, Blood urea nitrogen (BUN) and mean corpuscular hemoglobin concentration (MCHC) were the best predictors, whereas prediction of long-term mortality was favored by age, LDH, soluble urokinase plasminogen activator receptor (suPAR), albumin, and blood urea nitrogen (BUN).

CONCLUSION

The findings suggest that measures of biomarkers taken from one blood sample during admission to the ED can identify patients at high risk of short-and long-term mortality following emergency admissions.

Predictors of in-hospital mortality in elderly unvaccinated patients during SARS-CoV-2 Alpha variants epidemic

Background

COVID-19, caused by SARS-CoV-2, has caused a global pandemic. This study aimed to identify predictors of in-hospital mortality in unvaccinated elderly patients with COVID-19 by comparing various predictive factors between the survivors and non-survivors.

Methods

We retrospectively selected 132 unvaccinated patients aged over 65 years with COVID-19 at a hospital in Kanagawa, Japan, during SARS-CoV-2 Alpha variants epidemic. We compared the clinical characteristics, laboratory and radiological findings, treatment, and complications of the survivors and non-survivors. In logistic regression analysis, variables that were significant in the univariate analysis were subjected to multivariate analysis using the variable increase method.

Results

There were 119 and 13 patients in the survivor and non-survivor groups, respectively. Multivariate regression revealed increasing odds with the presence of ARDS and DIC (odd ratio (OR) = 16.35, 34.36; P=0.002, 0.001, respectively) and prolonged hospital stay (OR = 1.17; P=0.004).

Conclusions

We found the complications of ARDS and DIC and hospital length of stay to be independent predictors of in-hospital mortality in elderly unvaccinated patients with COVID-19. Establishing treatments and prevention methods for ARDS and DIC could result in lower mortality rates.

Advancing prognostic precision in pulmonary embolism: A clinical and laboratory-based artificial intelligence approach for enhanced early mortality risk stratification

BACKGROUND

Acute pulmonary embolism (PE) is a critical medical emergency that necessitates prompt identification and intervention. Accurate prognostication of early mortality is vital for recognizing patients at elevated risk for unfavourable outcomes and administering suitable therapy. Machine learning (ML) algorithms hold promise for enhancing the precision of early mortality prediction in PE patients.

OBJECTIVE

To devise an ML algorithm for early mortality prediction in PE patients by employing clinical and laboratory variables.

METHODS

This study utilized diverse oversampling techniques to improve the performance of various machine learning models including ANN, SVM, DT, RF, and AdaBoost for early mortality prediction. Appropriate oversampling methods were chosen for each model based on algorithm characteristics and dataset properties. Predictor variables included four lab tests, eight physiological time series indicators, and two general descriptors. Evaluation used metrics like accuracy, F1_score, precision, recall, Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) curves, providing a comprehensive view of models' predictive abilities.

RESULTS

The findings indicated that the RF model with random oversampling exhibited superior performance among the five models assessed, achieving elevated accuracy and precision alongside high recall for predicting the death class. The oversampling approaches effectively equalized the sample distribution among the classes and enhanced the models' performance.

CONCLUSIONS

The suggested ML technique can efficiently prognosticate mortality in patients afflicted with acute PE. The RF model with random oversampling can aid healthcare professionals in making well-informed decisions regarding the treatment of patients with acute PE. The study underscores the significance of oversampling methods in managing imbalanced data and emphasizes the potential of ML algorithms in refining early mortality prediction for PE patients.

Development and validation of a nomogram for predicting enteral feeding intolerance in critically ill patients (NOFI): Mixed retrospective and prospective cohort study

OBJECTIVE

Developing and validating a clinical prediction nomogram of enteral feeding intolerance (NOFI) in critically ill patients. So as to help clinicians implement pre-intervention for patients with high risk of enteral feeding intolerance (FI), formulate individualized feeding strategies, and reduce the probability of FI occurrence.

METHODS

From March 2018 to April 2023, patients who met the inclusion criteria but did not meet the exclusion criteria constituted the development cohort for retrospective analysis, and NOFI was developed. Patients recruited consecutively between May 2023 and July 2023 constituted the validation cohort for the prospective analysis for independent external validation of NOFI. Initially, a backward stepwise method was employed to conduct a multivariate logistic regression analysis in the development cohort, aiming to identify the optimal-fit model. Subsequently, a nomogram was derived from this model. The validation of the nomogram was carried out in an independent external validation cohort, where discrimination and calibration were evaluated. Additionally, a decision curve analysis was conducted to assess the net benefit of utilizing the nomogram for decision-making.

RESULTS

A total of 628 and 143 patients, 49.0 % and 51.7 % of patients occurred FI, were included in the development and validation cohort, respectively. We developed a NOFI in severely ill patients and the primary diagnosis, Acute gastrointestinal injury (AGI) grade, and APACHE II score were independent predictors of FI, with the OR of the primary diagnosis of circulatory disease being 2.281 (95 % CI, 1.364-3.816; P = 0.002); The OR of respiratory diseases was 0.424 (95 % CI, 0.259-0.594; P = 0.001); The OR of AGI grade was 4.920 (95 % CI, 3.773-6.416; P < 0.001), OR of APACHE II score was 1.100 (95 % CI, 1.059-1.143; P < 0.001). Independent external validation of the prediction model was performed. This model has good discrimination and calibration. The decision curve analysis of the nomogram provided better net benefit than the alternate options (full early enteral nutrition or delayed enteral nutrition).

CONCLUSIONS

The prediction of enteral feeding intolerance can be conveniently facilitated by the NOFI that integrates primary diagnosis, AGI grade, and APACHE II score in critically ill patients.

Predictive performance of SOFA (Sequential Organ Failure Assessment) and qSOFA (quick Sequential Organ Failure Assessment) for in-hospital mortality in ICU patients with COVID-19 of referral center in the north of Iran a retrospective study

Introduction

Patients diagnosed with Coronavirus disease 2019 exhibit varied clinical outcomes, with a reported mortality rate exceeding 30% in those requiring admission to the ICU. The objective of this study was to assess the predictive capacity of Sequential Organ Failure Assessment (SOFA) and quick Sequential Organ Failure Assessment (qSOFA) scores in determining mortality risk among severe COVID-19 patients.

Method and materials

This retrospective study was performed by analyzing the data of patients with COVID-19 who were hospitalized in the ICUs. Data collection of the parameters required to calculate the SOFA and qSOFA Scores were extracted from patient's medical records. All data analysis was performed using SPSS V.25. Significance level considered as P less than 0.05.

Findings

In this study, 258 patients were included. The results showed that the subjects ranged in age from 21 to 98 years with a mean and SD of 62.7±15.6. Of all patients, 127 (49.2%) were female and the rest were male. The mortality rate was 102 (39.5%). The underlying disease of diabetes mellitus with an odds ratio of 1.81 (CI=1.02-3.22) had a significant effect on mortality. In addition, a significant correlation was obtained between admission duration and SOFA score (r=0.147, P=0.018). The SOFA had a very high accuracy of 0.941 and at the cut-off point less than 5 had a sensitivity and specificity of 91.2% and 82.7%. In addition, qSOFA had high accuracy (0.914) and a sensitivity and specificity of 87.3% and 91.7% at the optimal cutting point of greater than 1.

Conclusion

The findings of present study illustrated that deceased COVID-19 patients admitted to the ICU had higher scores on both SOFA and qSOFA scales than surviving patients. Also, both scales have high sensitivity and specificity for anticipating of mortality in these patients. The underlying diabetes mellitus was associated with an increase in patient mortality.

Machine learning based early mortality prediction in the emergency department

BACKGROUND

It is a great challenge for emergency physicians to early detect the patient's deterioration and prevent unexpected death through a large amount of clinical data, which requires sufficient experience and keen insight.

OBJECTIVE

To evaluate the performance of machine learning models in quantifying the severity of emergency department (ED) patients and identifying high-risk patients.

METHODS

Using routinely-available demographics, vital signs and laboratory tests extracted from electronic health records (EHRs), a framework based on machine learning and feature engineering was proposed for mortality prediction. Patients who had one complete record of vital signs and laboratory tests in ED were included. The following patients were excluded: pediatric patients aged < 18 years, pregnant woman, and patients died or were discharged or hospitalized within 12 h after admission. Based on 76 original features extracted, 9 machine learning models were adopted to validate our proposed framework. Their optimal hyper-parameters were fine-tuned using the grid search method. The prediction results were evaluated on performance metrics (i.e., accuracy, area under the curve (AUC), recall and precision) with repeated 5-fold cross-validation (CV). The time window from patient admission to the prediction was analyzed at 12 h, 24 h, 48 h, and entire stay.

RESULTS

We studied a total of 1114 ED patients with 71.54% (797/1114) survival and 28.46% (317/1114) death in the hospital. The results revealed a more complete time window leads to better prediction performance. Using the entire stay records, the LightGBM model with refined feature engineering demonstrated high discrimination and achieved 93.6% (±0.008) accuracy, 97.6% (±0.003) AUC, 97.1% (±0.008) recall, and 94.2% (±0.006) precision, even if no diagnostic information was utilized.

CONCLUSIONS

This study quantifies the criticality of ED patients and appears to have significant potential as a clinical decision support tool in assisting physicians in their clinical routine. While the model requires validation before use elsewhere, the same methodology could be used to create a strong model for the new hospital.

Rapid Emergency Medicine Score as a Predictive Value for 30-day Outcome of Nonsurgical Patients Referred to the Emergency Department

Background and aims

Multiple scoring systems are designed and prepared nowadays that can be used to determine and predict the severity, morbidity, and mortality rate of patients. Among them, the rapid emergency medicine score (REMS) system has been designed to predict the motility of nonsurgical patients admitted to the emergency department (ED). This study was performed with the aim of evaluating the predictive value of REMS in the mortality rate of nonsurgical patients.

Materials and methods

This study was carried out in 2017 among 300 nonsurgical patients referred to the ED. Data were collected using a checklist containing two parts of demographic information and REMS scale.

Results

Based on the results, we found a significant correlation between the duration of hospitalization and other parameters of the study. The results of this study indicated that the REMS of patients increased by 11%, 3%, and 5%, per each unit rise in patient’s age, heart rate, and respiratory rate, respectively. On the contrary, 12% and 22% decrements for every unit increase in SPO₂ and GCS levels were observed, respectively. All the reported findings were statistically significant.

Conclusion

In sum, the outcomes of the present study corroborate the REMS system as a successful scale in predicting mortality and the duration of hospitalization in nonsurgical ED patients.

How to cite this article

Ala A, Vahdati SS, Jalali M, Parsay S. Rapid Emergency Medicine Score as a Predictive Value for 30-day Outcome of Nonsurgical Patients Referred to the Emergency Department. Indian J Crit Care Med 2020;24(6):418–422.

Improved ICU mortality prediction based on SOFA scores and gastrointestinal parameters

BACKGROUND

The Sequential Organ Failure Assessment (SOFA) score is commonly used in ICUs around the world, designed to assess the severity of the patient's clinical state based on function/dysfunction of six major organ systems. The goal of this work is to build a computational model to predict mortality based on a series of SOFA scores. In addition, we examined the possibility of improving the prediction by incorporating a new component designed to measure the performance of the gastrointestinal system, added to the other six components.

METHODS

In this retrospective study, we used patients' three latest SOFA scores recorded during an individual ICU stay as input to different machine learning models and ensemble learning models. We added three validated parameters representing gastrointestinal failure. Among others, we used classification models such as Support Vector Machines (SVMs), Neural Networks, Logistic Regression and a penalty function used to increase model robustness in regard to certain extreme cases, which may be found in ICU population. We used the Area under Curve (AUC) performance metric to examine performance.

RESULTS

We found an ensemble model of linear and logistic regression achieves a higher AUC compared related works in past years. After incorporating the gastrointestinal failure score along with the penalty function, our best performing ensemble model resulted in an additional improvement in terms of AUC metrics. We implemented and compared 36 different models that were built using both the information from the SOFA score as well as that of the gastrointestinal system. All compared models have approximately similar and relatively large AUC (between 0.8645 and 0.9146) with the best results are achieved by incorporating the gastrointestinal parameters into the prediction models.

CONCLUSIONS

Our findings indicate that gastrointestinal parameters carry significant information as a mortality predictor in addition to the conventional SOFA score. This information improves the predictive power of machine learning models by extending the SOFA to include information related to gastrointestinal organ system. The described method improves mortality prediction by considering the dynamics of the extended SOFA score. Although tested on a limited data set, the results' stability across different models suggests robustness in real-time use.

Development and validation of a nomogram for predicting self-propelled postpyloric placement of spiral nasoenteric tube in the critically ill: Mixed retrospective and prospective cohort study

BACKGROUND & AIMS

Equipment-aided or experience-dependent methods for postpyloric nasoenteric tube placement are not so readily accessible in the critically ill setting. Self-propelled postpyloric placement of a spiral nasoenteric tube can serve as an alternative approach. However, the success rate of this method is relatively low despite using prokinetics. This study aims to develop a user-friendly nomogram incorporating clinical markers to individually predict the probability of successful postpyloric nasoenteric tube placement and facilitate intensivists with improved decision-making before tube insertion.

METHODS

Patients consecutively recruited in the stage between May 2012 through December 2016 constituted the development cohort for retrospective analysis to internally test the nomogram, and patients in the stage between January 2017 through March 2018 constituted the validation cohort for prospective analysis to external validate the nomogram. A multivariate logistic regression analysis was firstly performed in the development cohort by a backward stepwise method to identify the best-fit model, from which a nomogram was obtained. The nomogram was validated in the independent external validation cohort concerning discrimination, calibration. A decision curve analysis was also performed to evaluate the net benefit of insertion decision with the nomogram.

RESULTS

A total of 364 and 119 patients, 52.7% and 55.5% with successful postpyloric placement, were included in the development and validation cohort, respectively. Predictors contained in the prediction nomogram included primary diagnosis, APACHE II score, AGI grade. The derived model showed good discrimination, with an area under the receiver operating characteristic curve (AUROC) of 0.809 (95%CI, 0.765-0.853) and good calibration. Application of the nomogram in the validation cohort also gave good discrimination with an AUROC of 0.776 (95%CI, 0.694-0.859) and good calibration. The decision curve analysis of the nomogram provided better net benefit than the alternate options (insert-all or insert-none).

CONCLUSIONS

A prediction nomogram that incorporates primary diagnosis, together with APACHE II score and AGI grade can be conveniently used to facilitate the pre-insertion individualized prediction of postpyloric nasoenteric tube placement in critically ill patients.

Presenting an efficient approach based on novel mapping for mortality prediction in intensive care unit cardiovascular patients

Intensive care unit (ICU) experienced and skillful people in this field should be employed because the equipment, facilities, and admitted patients have more special conditions than other departments. Our goal provides the best quality according to the condition each patient and prevent many unnecessary costs for preventive treatment. In this paper, the proposed system will first receive the patient's vital signs, which are recorded by the ICU monitoring. After the necessary processing, in case of observing changes in the normal state, risk alarms are transmitted to the nursing station so that nurses become aware of this condition and take all equipment to return the patient to normal condition and prevent his death. The applied graph in this study examines patients at any moment and displays the patient's future condition in a schematic manner after precise analyses. In this algorithm, after calculating the R-R intervals in the electrocardiogram signal, RRIs are thrown into a risk plot (RP) by a projectile. Given the amount of projectile RRI, one of the stairs can host that amount. After a few moments by springs embedded under the stairs, the drain of RRIs is done by the kinetic energy stored in the springs towards the valley of life. If the accumulation of quantities in a stair is too much, the spring will not be able to project those RRIs. By examining this situation, we will introduce an index to determine the risk of death for all patients. The results of this paper show that when a person is in normal condition, there is no density in a certain stair and the ball or the projected RRIs are not limited to a stair. In general, the results of this paper show that the lower amount of RRI dispersion in the RP leads to greater risk of entry into the death range and as this amount decrease, an immediate consideration is required. In conclusion, if the precise prediction of the future condition of ICU patients is available to nurses and doctors, more facilities and equipment could be provided to save their lives. •We focused on nonlinear methods with new aspects to extract mentioned dynamics.•This method can reduce the number of ICU nurses and give the special facilities for high-risk patients.•Our results confirm that it is possible to predict mortality based on the dynamical characteristics of HRV.

Bayesian averaging over decision tree models: An application for estimating uncertainty in trauma severity scoring

INTRODUCTION

For making reliable decisions, practitioners need to estimate uncertainties that exist in data and decision models. In this paper we analyse uncertainties of predicting survival probability for patients in trauma care. The existing prediction methodology employs logistic regression modelling of Trauma and Injury Severity Score (TRISS), which is based on theoretical assumptions. These assumptions limit the capability of TRISS methodology to provide accurate and reliable predictions.

METHODS

We adopt the methodology of Bayesian model averaging and show how this methodology can be applied to decision trees in order to provide practitioners with new insights into the uncertainty. The proposed method has been validated on a large set of 447,176 cases registered in the US National Trauma Data Bank in terms of discrimination ability evaluated with receiver operating characteristic (ROC) and precision-recall (PRC) curves.

RESULTS

Areas under curves were improved for ROC from 0.951 to 0.956 (p = 3.89 × 10^-18) and for PRC from 0.564 to 0.605 (p = 3.89 × 10^-18). The new model has significantly better calibration in terms of the Hosmer-Lemeshow Hˆ statistic, showing an improvement from 223.14 (the standard method) to 11.59 (p = 2.31 × 10^-18).

CONCLUSION

The proposed Bayesian method is capable of improving the accuracy and reliability of survival prediction. The new method has been made available for evaluation purposes as a web application.

Value of SOFA, APACHE IV and SAPS II scoring systems in predicting short-term mortality in patients with acute myocarditis

Acute myocarditis is an uncommon and potentially life-threatening disease. Scoring systems are essential for predicting outcome and evaluating the therapy effect of adult patients with acute myocarditis. The aim of this study was to determine the value of the Sequential Organ Failure Assessment (SOFA), Acute Physiology and Chronic Health Evaluation IV (APACHE IV) and second Simplified Acute Physiology Score (SAPS II) scoring systems in predicting short-term mortality of these patients. We retrospectively analyzed data from 305 adult patients suffering from acute myocarditis between April 2005 and August 2016. The association between the value of admission SOFA, APACHE IV and SAPS II scores and risk of short-term mortality was determined. Multivariate Cox analysis showed that SOFA, APACHE IV and SAPS II scores were independent risk factors of death in patients with acute myocarditis. For each scoring system, Kaplan–Meier analysis showed that the cumulative short-term mortality was significantly higher in patients with higher admission scores compared with those with lower admission scores. For the prediction of short-term mortality in a patient with acute myocarditis, SAPS II had the highest accuracy followed by the APACHE IV and SOFA scores.

Septic shock prediction for ICU patients via coupled HMM walking on sequential contrast patterns

BACKGROUND AND OBJECTIVE

Critical care patient events like sepsis or septic shock in intensive care units (ICUs) are dangerous complications which can cause multiple organ failures and eventual death. Preventive prediction of such events will allow clinicians to stage effective interventions for averting these critical complications.

METHODS

It is widely understood that physiological conditions of patients on variables such as blood pressure and heart rate are suggestive to gradual changes over a certain period of time, prior to the occurrence of a septic shock. This work investigates the performance of a novel machine learning approach for the early prediction of septic shock. The approach combines highly informative sequential patterns extracted from multiple physiological variables and captures the interactions among these patterns via coupled hidden Markov models (CHMM). In particular, the patterns are extracted from three non-invasive waveform measurements: the mean arterial pressure levels, the heart rates and respiratory rates of septic shock patients from a large clinical ICU dataset called MIMIC-II.

EVALUATION AND RESULTS

For baseline estimations, SVM and HMM models on the continuous time series data for the given patients, using MAP (mean arterial pressure), HR (heart rate), and RR (respiratory rate) are employed. Single channel patterns based HMM (SCP-HMM) and multi-channel patterns based coupled HMM (MCP-HMM) are compared against baseline models using 5-fold cross validation accuracies over multiple rounds. Particularly, the results of MCP-HMM are statistically significant having a p-value of 0.0014, in comparison to baseline models. Our experiments demonstrate a strong competitive accuracy in the prediction of septic shock, especially when the interactions between the multiple variables are coupled by the learning model.

CONCLUSIONS

It can be concluded that the novelty of the approach, stems from the integration of sequence-based physiological pattern markers with the sequential CHMM model to learn dynamic physiological behavior, as well as from the coupling of such patterns to build powerful risk stratification models for septic shock patients.

Designing optimal mortality risk prediction scores that preserve clinical knowledge

Many in-hospital mortality risk prediction scores dichotomize predictive variables to simplify the score calculation. However, hard thresholding in these additive stepwise scores of the form "add x points if variable v is above/below threshold t" may lead to critical failures. In this paper, we seek to develop risk prediction scores that preserve clinical knowledge embedded in features and structure of the existing additive stepwise scores while addressing limitations caused by variable dichotomization. To this end, we propose a novel score structure that relies on a transformation of predictive variables by means of nonlinear logistic functions facilitating smooth differentiation between critical and normal values of the variables. We develop an optimization framework for inferring parameters of the logistic functions for a given patient population via cyclic block coordinate descent. The parameters may readily be updated as the patient population and standards of care evolve. We tested the proposed methodology on two populations: (1) brain trauma patients admitted to the intensive care unit of the Dell Children's Medical Center of Central Texas between 2007 and 2012, and (2) adult ICU patient data from the MIMIC II database. The results are compared with those obtained by the widely used PRISM III and SOFA scores. The prediction power of a score is evaluated using area under ROC curve, Youden's index, and precision-recall balance in a cross-validation study. The results demonstrate that the new framework enables significant performance improvements over PRISM III and SOFA in terms of all three criteria.

Incorporating temporal EHR data in predictive models for risk stratification of renal function deterioration

Predictive models built using temporal data in electronic health records (EHRs) can potentially play a major role in improving management of chronic diseases. However, these data present a multitude of technical challenges, including irregular sampling of data and varying length of available patient history. In this paper, we describe and evaluate three different approaches that use machine learning to build predictive models using temporal EHR data of a patient. The first approach is a commonly used non-temporal approach that aggregates values of the predictors in the patient's medical history. The other two approaches exploit the temporal dynamics of the data. The two temporal approaches vary in how they model temporal information and handle missing data. Using data from the EHR of Mount Sinai Medical Center, we learned and evaluated the models in the context of predicting loss of estimated glomerular filtration rate (eGFR), the most common assessment of kidney function. Our results show that incorporating temporal information in patient's medical history can lead to better prediction of loss of kidney function. They also demonstrate that exactly how this information is incorporated is important. In particular, our results demonstrate that the relative importance of different predictors varies over time, and that using multi-task learning to account for this is an appropriate way to robustly capture the temporal dynamics in EHR data. Using a case study, we also demonstrate how the multi-task learning based model can yield predictive models with better performance for identifying patients at high risk of short-term loss of kidney function.

Glasgow Coma Scale score dominates the association between admission Sequential Organ Failure Assessment score and 30-day mortality in a mixed intensive care unit population

OBJECTIVE

The Sequential Organ Failure Assessment (SOFA) score, a measure of multiple-organ dysfunction syndrome, is used to predict mortality in critically ill patients by assigning equally weighted scores across 6 different organ systems. We hypothesized that specific organ systems would have a greater association with mortality than others.

DESIGN

We retrospectively studied patients admitted over a period of 4.2 years to a mixed-profile intensive care unit (ICU). We recorded age and comorbidities, and calculated SOFA organ scores. The primary outcome was 30-day all-cause mortality. We determined which organ subscores of the SOFA score were most associated with mortality using multiple analytic methods: random forests, conditional inference trees, distanced-based clustering techniques, and logistic regression.

SETTING

A 24-bed mixed-profile adult ICU that cares for medical, surgical, and trauma (level 1) patients at an academic referral center.

PATIENTS

All patients' first admission to the study ICU during the study period.

MEASUREMENTS AND MAIN RESULTS

We identified 9120 first admissions during the study period. Overall 30-day mortality was 12%. Multiple analytical methods all demonstrated that the best initial prediction variables were age and the central nervous system SOFA subscore, which is determined solely by Glasgow Coma Scale score.

CONCLUSIONS

In a mixed population of critically ill patients, the Glasgow Coma Scale score dominates the association between admission SOFA score and 30-day mortality. Future research into outcomes from multiple-organ dysfunction may benefit from new models for measuring organ dysfunction with special attention to neurologic dysfunction.

A methodology for interactive mining and visual analysis of clinical event patterns using electronic health record data

Patients' medical conditions often evolve in complex and seemingly unpredictable ways. Even within a relatively narrow and well-defined episode of care, variations between patients in both their progression and eventual outcome can be dramatic. Understanding the patterns of events observed within a population that most correlate with differences in outcome is therefore an important task in many types of studies using retrospective electronic health data. In this paper, we present a method for interactive pattern mining and analysis that supports ad hoc visual exploration of patterns mined from retrospective clinical patient data. Our approach combines (1) visual query capabilities to interactively specify episode definitions, (2) pattern mining techniques to help discover important intermediate events within an episode, and (3) interactive visualization techniques that help uncover event patterns that most impact outcome and how those associations change over time. In addition to presenting our methodology, we describe a prototype implementation and present use cases highlighting the types of insights or hypotheses that our approach can help uncover.

Prevalence of acute kidney injury and prognostic significance in patients with acute myocarditis

Objective

Myocarditis is an inflammation of the myocardium. The condition is commonly associated with rapid disease progression and often results in profound shock. Impaired renal function is the result of impairment in end-organ perfusion and is highly prevalent among critically ill patients. The aim of this study was to evaluate the incidence of acute kidney injury (AKI) and identify the relationship between AKI and the prognosis of patients with acute myocarditis.

Design, Measurements and Main Results

This retrospective study reviewed the medical records of 101 patients suffering from acute myocarditis between 1996 and 2011. Sixty of these patients (59%) developed AKI within 48 hours of being hospitalized. AKI defined as AKIN stage 3 (p = 0.007) and SOFA score (p = 0.03) were identified as predictors of in-hospital mortality in multivariate analysis. The conditional effect plot of the estimated risk against SOFA score upon admission categorized according to the AKIN stages showed that the risk of in-hospital mortality was highest among patients in AKIN stage 3 with a high SOFA score.

Conclusions

Among patients with acute myocarditis, AKI defined as AKIN stage 3 and elevated SOFA score were associated with unfavorable outcomes. AKIN classification is a simple, reproducible, and easily applied evaluation tool capable of providing objective information related to the clinical prognosis of patients with acute myocarditis.

Cross-validation of a Sequential Organ Failure Assessment score-based model to predict mortality in patients with cancer admitted to the intensive care unit

PURPOSE

This study aims to validate the performance of the Sequential Organ Failure Assessment (SOFA) score to predict death of critically ill patients with cancer.

MATERIAL AND METHODS

We conducted a retrospective observational study including adults admitted to the intensive care unit (ICU) between January 1, 2006, and December 31, 2008. We randomly selected training and validation samples in medical and surgical admissions to predict ICU and in-hospital mortality. By using logistic regression, we calculated the probabilities of death in the training samples and applied them to the validation samples to test the goodness-of-fit of the models, construct receiver operator characteristics curves, and calculate the areas under the curve (AUCs).

RESULTS

In predicting mortality at discharge from the unit, the AUC from the validation group of medical admissions was 0.7851 (95% confidence interval [CI], 0.7437-0.8264), and the AUC from the surgical admissions was 0.7847 (95% CI, 0.6319-0.937). The AUCs of the SOFA score to predict mortality in the hospital after ICU admission were 0.7789 (95% CI, 0.74-0.8177) and 0.7572 (95% CI, 0.6719-0.8424) for the medical and surgical validations groups, respectively.

CONCLUSIONS

The SOFA score had good discrimination to predict ICU and hospital mortality. However, the observed underestimation of ICU deaths and unsatisfactory goodness-of-fit test of the model in surgical patients to indicate calibration of the score to predict ICU mortality is advised in this group.

Computerized prediction of intensive care unit discharge after cardiac surgery: development and validation of a Gaussian processes model

Background

The intensive care unit (ICU) length of stay (LOS) of patients undergoing cardiac surgery may vary considerably, and is often difficult to predict within the first hours after admission. The early clinical evolution of a cardiac surgery patient might be predictive for his LOS. The purpose of the present study was to develop a predictive model for ICU discharge after non-emergency cardiac surgery, by analyzing the first 4 hours of data in the computerized medical record of these patients with Gaussian processes (GP), a machine learning technique.

Methods

Non-interventional study. Predictive modeling, separate development (n = 461) and validation (n = 499) cohort. GP models were developed to predict the probability of ICU discharge the day after surgery (classification task), and to predict the day of ICU discharge as a discrete variable (regression task). GP predictions were compared with predictions by EuroSCORE, nurses and physicians. The classification task was evaluated using aROC for discrimination, and Brier Score, Brier Score Scaled, and Hosmer-Lemeshow test for calibration. The regression task was evaluated by comparing median actual and predicted discharge, loss penalty function (LPF) ((actual-predicted)/actual) and calculating root mean squared relative errors (RMSRE).

Results

Median (P25-P75) ICU length of stay was 3 (2-5) days. For classification, the GP model showed an aROC of 0.758 which was significantly higher than the predictions by nurses, but not better than EuroSCORE and physicians. The GP had the best calibration, with a Brier Score of 0.179 and Hosmer-Lemeshow p-value of 0.382. For regression, GP had the highest proportion of patients with a correctly predicted day of discharge (40%), which was significantly better than the EuroSCORE (p < 0.001) and nurses (p = 0.044) but equivalent to physicians. GP had the lowest RMSRE (0.408) of all predictive models.

Conclusions

A GP model that uses PDMS data of the first 4 hours after admission in the ICU of scheduled adult cardiac surgery patients was able to predict discharge from the ICU as a classification as well as a regression task. The GP model demonstrated a significantly better discriminative power than the EuroSCORE and the ICU nurses, and at least as good as predictions done by ICU physicians. The GP model was the only well calibrated model.

Prediction of clinical conditions after coronary bypass surgery using dynamic data analysis

This work studies the impact of using dynamic information as features in a machine learning algorithm for the prediction task of classifying critically ill patients in two classes according to the time they need to reach a stable state after coronary bypass surgery: less or more than 9 h. On the basis of five physiological variables (heart rate, systolic arterial blood pressure, systolic pulmonary pressure, blood temperature and oxygen saturation), different dynamic features were extracted, namely the means and standard deviations at different moments in time, coefficients of multivariate autoregressive models and cepstral coefficients. These sets of features served subsequently as inputs for a Gaussian process and the prediction results were compared with the case where only admission data was used for the classification. The dynamic features, especially the cepstral coefficients (aROC: 0.749, Brier score: 0.206), resulted in higher performances when compared to static admission data (aROC: 0.547, Brier score: 0.247). The differences in performance are shown to be significant. In all cases, the Gaussian process classifier outperformed to logistic regression.

A novel time series analysis approach for prediction of dialysis in critically ill patients using echo-state networks

Background

Echo-state networks (ESN) are part of a group of reservoir computing methods and are basically a form of recurrent artificial neural networks (ANN). These methods can perform classification tasks on time series data. The recurrent ANN of an echo-state network has an 'echo-state' characteristic. This 'echo-state' functions as a fading memory: samples that have been introduced into the network in a further past, are faded away. The echo-state approach for the training of recurrent neural networks was first described by Jaeger H. et al. In clinical medicine, until this moment, no original research articles have been published to examine the use of echo-state networks.

Methods

This study examines the possibility of using an echo-state network for prediction of dialysis in the ICU. Therefore, diuresis values and creatinine levels of the first three days after ICU admission were collected from 830 patients admitted to the intensive care unit (ICU) between May 31th 2003 and November 17th 2007. The outcome parameter was the performance by the echo-state network in predicting the need for dialysis between day 5 and day 10 of ICU admission. Patients with an ICU length of stay <10 days or patients that received dialysis in the first five days of ICU admission were excluded. Performance by the echo-state network was then compared by means of the area under the receiver operating characteristic curve (AUC) with results obtained by two other time series analysis methods by means of a support vector machine (SVM) and a naive Bayes algorithm (NB).

Results

The AUC's in the three developed echo-state networks were 0.822, 0.818, and 0.817. These results were comparable to the results obtained by the SVM and the NB algorithm.

Conclusions

This proof of concept study is the first to evaluate the performance of echo-state networks in an ICU environment. This echo-state network predicted the need for dialysis in ICU patients. The AUC's of the echo-state networks were good and comparable to the performance of other classification algorithms. Moreover, the echo-state network was more easily configured than other time series modeling technologies.

Using hierarchical dynamic Bayesian networks to investigate dynamics of organ failure in patients in the Intensive Care Unit

In intensive care medicine close monitoring of organ failure status is important for the prognosis of patients and for choices regarding ICU management. Major challenges in analyzing the multitude of data pertaining to the functioning of the organ systems over time are to extract meaningful clinical patterns and to provide predictions for the future course of diseases. With their explicit states and probabilistic state transitions, Markov models seem to fit this purpose well. In complex domains such as intensive care a choice is often made between a simple model that is estimated from the data, or a more complex model in which the parameters are provided by domain experts. Our primary aim is to combine these approaches and develop a set of complex Markov models based on clinical data. In this paper we describe the design choices underlying the models, which enable them to identify temporal patterns, predict outcomes, and test clinical hypotheses. Our models are characterized by the choice of the dynamic hierarchical Bayesian network structure and the use of logistic regression equations in estimating the transition probabilities. We demonstrate the induction, inference, evaluation, and use of these models in practice in a case-study of patients with severe sepsis admitted to four Dutch ICUs.

Preliminary results of a prospective randomized trial of restrictive versus standard fluid regime in elective open abdominal aortic aneurysm repair

BACKGROUND

Open abdominal aortic aneurysm (AAA) repair is associated with a significant morbidity (primarily respiratory and cardiac complications) and an overall mortality rate of 4% to 10%. We tested the hypothesis that perioperative fluid restriction would reduce complications and improve outcome after elective open AAA repair.

METHODS

In a prospective randomized control trial, patients undergoing elective open infra-renal AAA repair were randomized to a "standard" or "restricted" perioperative fluid administration group. Primary outcome measure was rate of major complications (MC) after AAA repair and secondary outcome measures included: Sequential Organ Failure Assessment Score; FiO2/PO2 ratio; Urinary Albumin/Creatinine Ratio; Length-of-stay in, intensive care unit, high dependency unit, in-hospital. This prospective Randomized Controlled Trial was registered in a publicly accessible database and has the following ID number ISRCTN27753612.

RESULTS

Overall 22 patients were randomized, 1 was excluded on a priori criteria, leaving standard group (11) and restricted group (10) for analysis. No significant difference was noted between groups in respect to age, gender, American Society Anesthesiology class, Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity scores, operation time, and operation blood loss. There were no in-hospital deaths and no 30-day mortality. The cumulative fluid balance on day 5 postoperative was for standard group, 8242 +/- 714 mL, compared with restricted group, 2570 +/- 977 mL, P < 0.01. MC were significantly reduced in the restricted group (n = 10), 1 MC, compared with standard group (n = 11), 14 MC, P < 0.024. Total and postoperative length-of-stay in-hospital was significantly reduced in the restricted group, 9 +/- 1 and 8 +/- 1 days, compared with standard group, 18 +/- 5 and 16 +/- 5 days, P < 0.01 and P < 0.025, respectively.

CONCLUSIONS

Serious complications are common after elective open AAA repair, and we have shown for the first time that a restricted perioperative fluid regimen can prevent MC and significantly reduce overall hospital stay.

Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic review

Introduction

To systematically review studies evaluating the performance of Sequential Organ Failure Assessment (SOFA)-based models for predicting mortality in patients in the intensive care unit (ICU).

Methods

Medline, EMBASE and other databases were searched for English-language articles with the major objective of evaluating the prognostic performance of SOFA-based models in predicting mortality in surgical and/or medical ICU admissions. The quality of each study was assessed based on a quality framework for prognostic models.

Results

Eighteen articles met all inclusion criteria. The studies differed widely in the SOFA derivatives used and in their methods of evaluation. Ten studies reported about developing a probabilistic prognostic model, only five of which used an independent validation data set. The other studies used the SOFA-based score directly to discriminate between survivors and non-survivors without fitting a probabilistic model. In five of the six studies, admission-based models (Acute Physiology and Chronic Health Evaluation (APACHE) II/III) were reported to have a slightly better discrimination ability than SOFA-based models at admission (the receiver operating characteristic curve (AUC) of SOFA-based models ranged between 0.61 and 0.88), and in one study a SOFA model had higher AUC than the Simplified Acute Physiology Score (SAPS) II model. Four of these studies used the Hosmer-Lemeshow tests for calibration, none of which reported a lack of fit for the SOFA models. Models based on sequential SOFA scores were described in 11 studies including maximum SOFA scores and maximum sum of individual components of the SOFA score (AUC range: 0.69 to 0.92) and delta SOFA (AUC range: 0.51 to 0.83). Studies comparing SOFA with other organ failure scores did not consistently show superiority of one scoring system to another. Four studies combined SOFA-based derivatives with admission severity of illness scores, and they all reported on improved predictions for the combination. Quality of studies ranged from 11.5 to 19.5 points on a 20-point scale.

Conclusions

Models based on SOFA scores at admission had only slightly worse performance than APACHE II/III and were competitive with SAPS II models in predicting mortality in patients in the general medical and/or surgical ICU. Models with sequential SOFA scores seem to have a comparable performance with other organ failure scores. The combination of sequential SOFA derivatives with APACHE II/III and SAPS II models clearly improved prognostic performance of either model alone. Due to the heterogeneity of the studies, it is impossible to draw general conclusions on the optimal mathematical model and optimal derivatives of SOFA scores. Future studies should use a standard evaluation methodology with a standard set of outcome measures covering discrimination, calibration and accuracy.

Rating organ failure via adverse events using data mining in the intensive care unit

OBJECTIVE

The main intensive care unit (ICU) goal is to avoid or reverse the organ failure process by adopting a timely intervention. Within this context, early identification of organ impairment is a key issue. The sequential organ failure assessment (SOFA) is an expert-driven score that is widely used in European ICUs to quantify organ disorder. This work proposes a complementary data-driven approach based on adverse events, defined from commonly monitored biometrics. The aim is to study the impact of these events when predicting the risk of ICU organ failure.

MATERIALS AND METHODS

A large database was considered, with a total of 25,215 daily records taken from 4425 patients and 42 European ICUs. The input variables include the case mix (i.e. age, diagnosis, admission type and admission from) and adverse events defined from four bedside physiologic variables (i.e. systolic blood pressure, heart rate, pulse oximeter oxygen saturation and urine output). The output target is the organ status (i.e. normal, dysfunction or failure) of six organ systems (respiratory, coagulation, hepatic, cardiovascular, neurological and renal), as measured by the SOFA score. Two data mining (DM) methods were compared: multinomial logistic regression (MLR) and artificial neural networks (ANNs). These methods were tested in the R statistical environment, using 20 runs of a 5-fold cross-validation scheme. The area under the receiver operator characteristic (ROC) curve and Brier score were used as the discrimination and calibration measures.

RESULTS

The best performance was obtained by the ANNs, outperforming the MLR in both discrimination and calibration criteria. The ANNs obtained an average (over all organs) area under the ROC curve of 64, 69 and 74% and Brier scores of 0.18, 0.16 and 0.09 for the dysfunction, normal and failure organ conditions, respectively. In particular, very good results were achieved when predicting renal failure (ROC curve area of 76% and Brier score of 0.06).

CONCLUSION

Adverse events, taken from bedside monitored data, are important intermediate outcomes, contributing to a timely recognition of organ dysfunction and failure during ICU length of stay. The obtained results show that it is possible to use DM methods to get knowledge from easy obtainable data, thus making room for the development of intelligent clinical alarm monitoring.

Discovery and integration of univariate patterns from daily individual organ-failure scores for intensive care mortality prediction

OBJECTIVES

The current established mortality predictive models in the intensive care rely only on patient information gathered within the first 24 hours of admission. Recent research demonstrated the added prognostic value residing in the sequential organ-failure assessment (SOFA) score which quantifies on each day the cumulative patient organ derangement. The objective of this paper is to develop and study predictive models that also incorporate univariate patterns of the six individual organ systems underlining the SOFA score. A model for a given day d predicts the probability of in-hospital mortality.

MATERIALS AND METHODS

We use the logistic framework to combine a summary statistic of the historic SOFA information for a patient together with selected dummy variables indicating the occurrence of univariate frequent temporal patterns of individual organ system functioning. We demonstrate the application of our method to a large real-life data set from an intensive care unit (ICU) in a teaching hospital. Model performance is tested in terms of the AUC and the Brier score.

RESULTS

An algorithm for categorization, discovery, and selection of univariate patterns of individual organ scores and the induction of predictive models. The case-study resulted in six daily models corresponding to days 2-7. Their AUC ranged between 0.715 and 0.794 and the Brier scores between 0.161 and 0.216. Models using only admission data but recalibrated for days 2-7 generated AUC ranging between 0.643 and 0.761 and Brier scores ranged between 0.175 and 0.230.

CONCLUSIONS

The results show that temporal organ-failure episodes improve predictions' quality in terms of both discrimination and calibration. In addition, they enhance the interpretability of models. Our approach should be applicable to many other medical domains where severity scores and sub-scores are collected.