Prognosis in critical care

Association of bioimpedance analysis parameters trajectories with clinical outcomes in neurocritical patients

Background and objective

Neurocritical patients often experience uncontrolled high catabolic metabolism state during the acuta phase of the disease. The complex interactions of neuroendocrine, inflammation, and immune system lead to massive protein breakdown and changes in body composition. Bioelectrical impedance analysis (BIA) evaluates the content and proportions of body components based on the principles of bioelectricity. Its parameters reflect the overall health status of the body and the integrity of cellular structure and function, playing an important role in assessing the disease status and predicting prognosis of such patients. This study explored the association of BIA parameters trajectories with clinical outcomes in neurocritical patients.

Methods

This study prospectively collected BIA parameters of 127 neurocritical patients in the Department of Neurology admitted to the NICU for the first 1-7 days. All these patients were adults (≥18 years old) experiencing their first onset of illness and were in the acute phase of the disease. The group-based trajectory modeling (GBTM), which aims to identify individuals following similar developmental trajectories, was used to identify potential subgroups of individuals based on BIA parameters. The short-term prognosis of patients in each trajectory group with variations in phase angle (PA) and extracellular water/total body water (ECW/TBW) over time was differentially analyzed, and the logistic regression model was used to analyze the relationship between potential trajectory groups of PA and ECW/TBW and the short-term prognosis of neurocritical patients. The outcome was Glasgow Outcome Scale (GOS) score at discharge.

Results

Four PA trajectories and four ECW/TBW trajectories were detected respectively in neurocritical patients. Among them, compared with the other latent subgroups, the "Low PA rapidly decreasing subgroup" and the "High ECW/TBW slowly rising subgroup" had higher incidences of adverse outcomes at discharge (GOS:1-3), in-hospital mortality, and length of neurology intensive care unit stay (all P < 0.05). After correcting for potential confounders, compared with the "Low PA rapidly decreasing subgroup", the risk of adverse outcome (GOS:1-3) was lower in the other three PA trajectories, with OR values of 0.0003, 0.0004, and 0.003 respectively (all P < 0.05). Compared with the "High ECW/TBW slowly rising subgroup", the risk of adverse outcome (GOS:1-3) was lower in the other three ECW/TBW trajectories, with OR values of 0.013, 0.035 and 0.038 respectively (all P < 0.05).

Conclusion

Latent PA trajectories and latent ECW/TBW trajectories during 1-7 days after admission were associated with the clinical outcomes of neurocritical patients. The risk of adverse outcomes was highest in the "Low PA rapidly decreasing subgroup" and the "High ECW/TBW slowly rising subgroup". These results reflected the overall health status and nutritional condition of neurocritical patients at the onset of the disease, and demonstrated the dynamic change process in body composition caused by the inflammatory response during the acute phase of the disease. This provided a reference basis for the observation and prognostic evaluation of such patients.

Bias in artificial intelligence algorithms and recommendations for mitigation

The adoption of artificial intelligence (AI) algorithms is rapidly increasing in healthcare. Such algorithms may be shaped by various factors such as social determinants of health that can influence health outcomes. While AI algorithms have been proposed as a tool to expand the reach of quality healthcare to underserved communities and improve health equity, recent literature has raised concerns about the propagation of biases and healthcare disparities through implementation of these algorithms. Thus, it is critical to understand the sources of bias inherent in AI-based algorithms. This review aims to highlight the potential sources of bias within each step of developing AI algorithms in healthcare, starting from framing the problem, data collection, preprocessing, development, and validation, as well as their full implementation. For each of these steps, we also discuss strategies to mitigate the bias and disparities. A checklist was developed with recommendations for reducing bias during the development and implementation stages. It is important for developers and users of AI-based algorithms to keep these important considerations in mind to advance health equity for all populations.

Explainable ensemble machine learning model for prediction of 28-day mortality risk in patients with sepsis-associated acute kidney injury

Background

Sepsis-associated acute kidney injury (S-AKI) is a major contributor to mortality in intensive care units (ICU). Early prediction of mortality risk is crucial to enhance prognosis and optimize clinical decisions. This study aims to develop a 28-day mortality risk prediction model for S-AKI utilizing an explainable ensemble machine learning (ML) algorithm.

Methods

This study utilized data from the Medical Information Mart for Intensive Care IV (MIMIC-IV 2.0) database to gather information on patients with S-AKI. Univariate regression, correlation analysis and Boruta were combined for feature selection. To construct the four ML models, hyperparameters were tuned via random search and five-fold cross-validation. To evaluate the performance of all models, ROC, K-S, and LIFT curves were used. The discrimination of ML models and traditional scoring systems was compared using area under the receiver operating characteristic curve (AUC). Additionally, the SHapley Additive exPlanation (SHAP) was utilized to interpret the ML model and identify essential variables. To investigate the relationship between the top nine continuous variables and the risk of 28-day mortality. COX regression-restricted cubic splines were utilized while controlling for age and comorbidities.

Results

The study analyzed data from 9,158 patients with S-AKI, dividing them into a 28-day mortality group of 1,940 and a survival group of 7,578. The results showed that XGBoost was the best performing model of the four ML models with AUC of 0.873. All models outperformed APS-III 0.713 and SAPS-II 0.681. The K-S and LIFT curves indicated XGBoost as the most effective predictor for 28-day mortality risk. The model's performance was evaluated using ROCpr curves, calibration curves, accuracy, precision, and F1 scores. SHAP force plots were utilized to interpret and visualize the personalized predictive power of the 28-day mortality risk model. Additionally, COX regression restricted cubic splines revealed an interesting non-linear relationship between the top nine variables and 28-day mortality.

Conclusion

The use of ensemble ML models has shown to be more effective than the LR model and conventional scoring systems in predicting 28-day mortality risk in S-AKI patients. By visualizing the XGBoost model with the best predictive performance, clinicians are able to identify high-risk patients early on and improve prognosis.

Association of the time in targeted blood glucose range of 3.9-10 mmol/L with the mortality of critically ill patients with or without diabetes

Purpose

The relationship between the TIR and mortality may be influenced by the presence of diabetes and other glycemic indicators. The purpose of this study was to investigate the relationship between TIR and in-hospital mortality in diabetic and non-diabetic patients in ICU.

Methods

A total of 998 patients with severe diseases in the ICU were selected for this retrospective analysis. The TIR is defined as the percentage of time spent in the target blood glucose range of 3.9-10.0 mmol/L within 24 h. The relationship between TIR and in-hospital mortality in diabetic and non-diabetic patients was analyzed. The effect of glycemic variability was also analyzed.

Results

The binary logistic regression model showed that there was a significant association between the TIR and the in-hospital death of severely ill non-diabetic patients. Furthermore, TIR≥70% was significantly associated with in-hospital death (OR = 0.581, P = 0.003). The study found that the coefficient of variation (CV) was significantly associated with the mortality of severely ill diabetic patients (OR = 1.042, P = 0.027).

Conclusions

Both diabetic and non-diabetic critically ill patients should control blood glucose fluctuations and maintain blood glucose levels within the target range, it may be beneficial in reducing mortality.

The illness severity of patients reviewed by the medical emergency team: A scoping review

BACKGROUND

Medical emergency teams (METs) are internationally used to manage hospitalised deteriorating patients. Although triggers for MET review and hospital outcomes have previously been widely reported, the illness severity at the point of MET review has not been reported. As such, levels of clinical acuity and patient dependency representing the risk of exposure to short-term adverse clinical outcomes remain largely unknown.

OBJECTIVE

This scoping review sought to understand the illness severity of MET review recipients in terms of acuity and dependency.

METHODS

This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. The published and grey literature since 2009 was searched to identify relevant articles reporting illness severity scores associated with hospitalised adult inpatients reviewed by a MET. After applying the inclusion and exclusion criteria, 17 articles (16 quantitative studies, one mixed-methods study) were reviewed, summarised, collated, and reported.

RESULTS

A total of 17 studies reported clinical acuity metrics for patients reviewed by a MET. No studies described an integrated risk score encompassing acuity, patient dependency, or wider parameters that might be associated with increased patient risk or the need for intervention. Multi-MET review, the use of specialist interventions, and delayed/transfer to the intensive care unit were associated with a greater risk of clinical deterioration, higher clinical acuity score, and predicted mortality risk. A single dependency metric was not reported although organisational levels of care, the duration of MET review, MET interventions, chronic illness, and frailty were inferred proxy measures.

CONCLUSION

Of the 17 studies reviewed, no single study provided an integrated assessment of illness severity from which to stratify risk or support patient management processes. Patients reviewed by a MET have variable and rapidly changing health needs that make them particularly vulnerable. The lack of high-quality data reporting acuity and dependency limits our understanding of true clinical risk and subsequent opportunities for pathway development.

Predicting 30-Day Hospital Readmission Risk in a National Cohort of Patients with Cirrhosis

BACKGROUND

Early hospital readmission for patients with cirrhosis continues to challenge the healthcare system. Risk stratification may help tailor resources, but existing models were designed using small, single-institution cohorts or had modest performance.

AIMS

We leveraged a large clinical database from the Department of Veterans Affairs (VA) to design a readmission risk model for patients hospitalized with cirrhosis. Additionally, we analyzed potentially modifiable or unexplored readmission risk factors.

METHODS

A national VA retrospective cohort of patients with a history of cirrhosis hospitalized for any reason from January 1, 2006, to November 30, 2013, was developed from 123 centers. Using 174 candidate variables within demographics, laboratory results, vital signs, medications, diagnoses and procedures, and healthcare utilization, we built a 47-variable penalized logistic regression model with the outcome of all-cause 30-day readmission. We excluded patients who left against medical advice, transferred to a non-VA facility, or if the hospital length of stay was greater than 30 days. We evaluated calibration and discrimination across variable volume and compared the performance to recalibrated preexisting risk models for readmission.

RESULTS

We analyzed 67,749 patients and 179,298 index hospitalizations. The 30-day readmission rate was 23%. Ascites was the most common cirrhosis-related cause of index hospitalization and readmission. The AUC of the model was 0.670 compared to existing models (0.649, 0.566, 0.577). The Brier score of 0.165 showed good calibration.

CONCLUSION

Our model achieved better discrimination and calibration compared to existing models, even after local recalibration. Assessment of calibration by variable parsimony revealed performance improvements for increasing variable inclusion well beyond those detectable for discrimination.

A nonparametric updating method to correct clinical prediction model drift

OBJECTIVE

Clinical prediction models require updating as performance deteriorates over time. We developed a testing procedure to select updating methods that minimizes overfitting, incorporates uncertainty associated with updating sample sizes, and is applicable to both parametric and nonparametric models.

MATERIALS AND METHODS

We describe a procedure to select an updating method for dichotomous outcome models by balancing simplicity against accuracy. We illustrate the test's properties on simulated scenarios of population shift and 2 models based on Department of Veterans Affairs inpatient admissions.

RESULTS

In simulations, the test generally recommended no update under no population shift, no update or modest recalibration under case mix shifts, intercept correction under changing outcome rates, and refitting under shifted predictor-outcome associations. The recommended updates provided superior or similar calibration to that achieved with more complex updating. In the case study, however, small update sets lead the test to recommend simpler updates than may have been ideal based on subsequent performance.

DISCUSSION

Our test's recommendations highlighted the benefits of simple updating as opposed to systematic refitting in response to performance drift. The complexity of recommended updating methods reflected sample size and magnitude of performance drift, as anticipated. The case study highlights the conservative nature of our test.

CONCLUSIONS

This new test supports data-driven updating of models developed with both biostatistical and machine learning approaches, promoting the transportability and maintenance of a wide array of clinical prediction models and, in turn, a variety of applications relying on modern prediction tools.

Are Fetuin-A levels beneficial for estimating timing of sepsis occurrence?

Objectives:

To evaluated Fetuin-A levels of patients admitted in the intensive care unit with a diagnosis of sepsis.

Methods:

This study was conducted at the Faculty of Medicine, Çanakkale Onsekiz Mart University Hospital, Çanakkal, Turkey, between February 2015 and October 2015. Forty septic patients were included in the study. Subsequent to clinical suspicion of sepsis, serum levels of C-reactive protein (CRP) and procalcitonin; and white blood cell (WBC) counts were evaluated at 3 time-points: 0 (basal), 24, and 72 hours.

Results:

The mean Fetuin-A levels at the 3 time-points were 58.5 ± 29.2 ng/mL, 40.9 ± 23.6 ng/mL, and 47.8 ± 25.7 ng/mL, respectively. Fetuin-A levels at 24 hours were significantly lower than the basal level (p<0.05), where as no significant difference was observed between the basal levels and those at 72 hours (p>0.05). Correlation between the temporal changes in Fetuin-A levels and the changes in other inflammatory markers (CRP, procalcitonin and WBC) was examined. Fetuin A was found to have only a negative correlation with serum procalcitonin level (p<0.05).

Conclusion:

In this study, serum Fetuin-A levels in septic patients decreased significantly in the first 24 hours, followed by an insignificant increase at 72 hours. These findings suggest that monitoring of Fetuin-A levels may help predict the time of occurrence of sepsis and prognosis of sepsis.

Low vitamin D at ICU admission is associated with cancer, infections, acute respiratory insufficiency, and liver failure

OBJECTIVES

Vitamin D deficiency may be associated with comorbidities and poor prognosis. However, this association in patients in the intensive care unit (ICU) has not been fully elucidated. The aim of this study was to investigate whether the serum concentrations of 25-hydroxyvitamin D (25[OH]D) within the first 48 h after ICU admission are associated with prognostic indicators (Acute Physiology and Chronic Health Evaluation [APACHE] II, Sequential Organ Failure Assessment [SOFA] score, Charlson comorbidity index [CCI]), clinical complications, serum C-reactive protein (CRP) concentrations, mechanical ventilation duration, and mortality.

METHODS

Seventy-one patients were admitted to the ICU, and their concentrations of 25(OH)D in the first 48 h were analyzed. To evaluate the prognostic factors in the ICU, APACHE II scores, SOFA scores, CCI questionnaires, mechanical ventilation time, CRP, and mortality were used.

RESULTS

The mean concentration of 25(OH)D was 17.7 ± 8.27 ng/mL (range 3.5-37.5 ng/mL), with 91.6% presenting with deficiency at admission. Although no associations were found between serum 25(OH)D concentrations with mechanical ventilation time, CRP, mortality, and APACHE II and SOFA severity scores, we found associations with the CCI when adjusted by age (model 1: odds ratio [OR], 1.64; 95% confidence interval [CI], 1.14-2.34) and by age, sex and body mass index (model 2: OR, 1.59; 95% CI, 1.10-2.34). In addition, among the comorbidities present, 25(OH)D concentrations were inversely associated with cancer (crude model OR, 3.42; 95% CI, 1.21-9.64) and liver disease (crude model OR, 9.64; 95% CI, 2.28-40.60).

CONCLUSION

We found a strong association between 25(OH)D concentrations and the prognostic indicator CCI and clinical complications (acute respiratory insufficiency, acute liver failure, and infections), but no associations with the prognostic indicators APACHE II and SOFA score, CRP, mechanical ventilation duration, or mortality. The main comorbidities associated with low 25(OH)D were cancer and liver disease, suggesting that the determination of 25(OH)vitamin D is relevant during the ICU stay.

Calibration drift in regression and machine learning models for acute kidney injury

Objective

Predictive analytics create opportunities to incorporate personalized risk estimates into clinical decision support. Models must be well calibrated to support decision-making, yet calibration deteriorates over time. This study explored the influence of modeling methods on performance drift and connected observed drift with data shifts in the patient population.

Materials and Methods

Using 2003 admissions to Department of Veterans Affairs hospitals nationwide, we developed 7 parallel models for hospital-acquired acute kidney injury using common regression and machine learning methods, validating each over 9 subsequent years.

Results

Discrimination was maintained for all models. Calibration declined as all models increasingly overpredicted risk. However, the random forest and neural network models maintained calibration across ranges of probability, capturing more admissions than did the regression models. The magnitude of overprediction increased over time for the regression models while remaining stable and small for the machine learning models. Changes in the rate of acute kidney injury were strongly linked to increasing overprediction, while changes in predictor-outcome associations corresponded with diverging patterns of calibration drift across methods.

Conclusions

Efficient and effective updating protocols will be essential for maintaining accuracy of, user confidence in, and safety of personalized risk predictions to support decision-making. Model updating protocols should be tailored to account for variations in calibration drift across methods and respond to periods of rapid performance drift rather than be limited to regularly scheduled annual or biannual intervals.

Improving Personalized Clinical Risk Prediction Based on Causality-Based Association Rules

Developing clinical risk prediction models is one of the main tasks of healthcare data mining. Advanced data collection techniques in current Big Data era have created an emerging and urgent need for scalable, computer-based data mining methods. These methods can turn data into useful, personalized decision support knowledge in a flexible, cost-effective, and productive way. In our previous study, we developed a tool, called icuARM- II, that can generate personalized clinical risk prediction evidence using a temporal rule mining framework. However, the generation of final risk prediction possibility with icuARM-II still relied on human interpretation, which was subjective and, most of time, biased. In this study, we propose a new mechanism to improve icuARM-II's rule selection by including the concept of causal analysis. The generated risk prediction is quantitatively assessed using calibration statistics. To evaluate the performance of the new rule selection mechanism, we conducted a case study to predict short-term intensive care unit mortality based on personalized lab testing abnormalities. Our results demonstrated a better-calibrated ICU risk prediction using the new causality-base rule selection solution by comparing with conventional confidence-only rule selection methods.

Predictive modelling of survival and length of stay in critically ill patients using sequential organ failure scores

INTRODUCTION

The length of stay of critically ill patients in the intensive care unit (ICU) is an indication of patient ICU resource usage and varies considerably. Planning of postoperative ICU admissions is important as ICUs often have no nonoccupied beds available.

PROBLEM STATEMENT

Estimation of the ICU bed availability for the next coming days is entirely based on clinical judgement by intensivists and therefore too inaccurate. For this reason, predictive models have much potential for improving planning for ICU patient admission.

OBJECTIVE

Our goal is to develop and optimize models for patient survival and ICU length of stay (LOS) based on monitored ICU patient data. Furthermore, these models are compared on their use of sequential organ failure (SOFA) scores as well as underlying raw data as input features.

METHODOLOGY

Different machine learning techniques are trained, using a 14,480 patient dataset, both on SOFA scores as well as their underlying raw data values from the first five days after admission, in order to predict (i) the patient LOS, and (ii) the patient mortality. Furthermore, to help physicians in assessing the prediction credibility, a probabilistic model is tailored to the output of our best-performing model, assigning a belief to each patient status prediction. A two-by-two grid is built, using the classification outputs of the mortality and prolonged stay predictors to improve the patient LOS regression models.

RESULTS

For predicting patient mortality and a prolonged stay, the best performing model is a support vector machine (SVM) with GA,D=65.9% (area under the curve (AUC) of 0.77) and GS,L=73.2% (AUC of 0.82). In terms of LOS regression, the best performing model is support vector regression, achieving a mean absolute error of 1.79 days and a median absolute error of 1.22 days for those patients surviving a nonprolonged stay.

CONCLUSION

Using a classification grid based on the predicted patient mortality and prolonged stay, allows more accurate modeling of the patient LOS. The detailed models allow to support the decisions made by physicians in an ICU setting.

Grid Binary LOgistic REgression (GLORE): building shared models without sharing data

Objective

The classification of complex or rare patterns in clinical and genomic data requires the availability of a large, labeled patient set. While methods that operate on large, centralized data sources have been extensively used, little attention has been paid to understanding whether models such as binary logistic regression (LR) can be developed in a distributed manner, allowing researchers to share models without necessarily sharing patient data.

Material and methods

Instead of bringing data to a central repository for computation, we bring computation to the data. The Grid Binary LOgistic REgression (GLORE) model integrates decomposable partial elements or non-privacy sensitive prediction values to obtain model coefficients, the variance-covariance matrix, the goodness-of-fit test statistic, and the area under the receiver operating characteristic (ROC) curve.

Results

We conducted experiments on both simulated and clinically relevant data, and compared the computational costs of GLORE with those of a traditional LR model estimated using the combined data. We showed that our results are the same as those of LR to a 10⁻¹⁵ precision. In addition, GLORE is computationally efficient.

Limitation

In GLORE, the calculation of coefficient gradients must be synchronized at different sites, which involves some effort to ensure the integrity of communication. Ensuring that the predictors have the same format and meaning across the data sets is necessary.

Conclusion

The results suggest that GLORE performs as well as LR and allows data to remain protected at their original sites.

A patient-driven adaptive prediction technique to improve personalized risk estimation for clinical decision support

Objective

Competing tools are available online to assess the risk of developing certain conditions of interest, such as cardiovascular disease. While predictive models have been developed and validated on data from cohort studies, little attention has been paid to ensure the reliability of such predictions for individuals, which is critical for care decisions. The goal was to develop a patient-driven adaptive prediction technique to improve personalized risk estimation for clinical decision support.

Material and methods

A data-driven approach was proposed that utilizes individualized confidence intervals (CIs) to select the most ‘appropriate’ model from a pool of candidates to assess the individual patient's clinical condition. The method does not require access to the training dataset. This approach was compared with other strategies: the BEST model (the ideal model, which can only be achieved by access to data or knowledge of which population is most similar to the individual), CROSS model, and RANDOM model selection.

Results

When evaluated on clinical datasets, the approach significantly outperformed the CROSS model selection strategy in terms of discrimination (p<1e–14) and calibration (p<0.006). The method outperformed the RANDOM model selection strategy in terms of discrimination (p<1e–12), but the improvement did not achieve significance for calibration (p=0.1375).

Limitations

The CI may not always offer enough information to rank the reliability of predictions, and this evaluation was done using aggregation. If a particular individual is very different from those represented in a training set of existing models, the CI may be somewhat misleading.

Conclusion

This approach has the potential to offer more reliable predictions than those offered by other heuristics for disease risk estimation of individual patients.

Calibrating predictive model estimates to support personalized medicine

OBJECTIVE

Predictive models that generate individualized estimates for medically relevant outcomes are playing increasing roles in clinical care and translational research. However, current methods for calibrating these estimates lose valuable information. Our goal is to develop a new calibration method to conserve as much information as possible, and would compare favorably to existing methods in terms of important performance measures: discrimination and calibration.

MATERIAL AND METHODS

We propose an adaptive technique that utilizes individualized confidence intervals (CIs) to calibrate predictions. We evaluate this new method, adaptive calibration of predictions (ACP), in artificial and real-world medical classification problems, in terms of areas under the ROC curves, the Hosmer-Lemeshow goodness-of-fit test, mean squared error, and computational complexity.

RESULTS

ACP compared favorably to other calibration methods such as binning, Platt scaling, and isotonic regression. In several experiments, binning, isotonic regression, and Platt scaling failed to improve the calibration of a logistic regression model, whereas ACP consistently improved the calibration while maintaining the same discrimination or even improving it in some experiments. In addition, the ACP algorithm is not computationally expensive.

LIMITATIONS

The calculation of CIs for individual predictions may be cumbersome for certain predictive models. ACP is not completely parameter-free: the length of the CI employed may affect its results.

CONCLUSIONS

ACP can generate estimates that may be more suitable for individualized predictions than estimates that are calibrated using existing methods. Further studies are necessary to explore the limitations of ACP.

Survival of patients with hematological malignancy admitted to the intensive care unit: prognostic factors and outcome compared to unselected medical intensive care unit admissions, a parallel group study

Improved survival in patients with hematological malignancy (HM) admitted to the intensive care unit (ICU) has largely been reported in uncontrolled cohorts from single academic institutions. We compared hospital mortality between 147 patients with HM and 147 general medical admissions to five non-specialist ICUs. The proportion of patients surviving to hospital discharge was significantly worse in patients with HM (27% vs. 56%; p < 0.001). Six-month and 1-year survival in patients with HM was 21% and 18%, respectively. HM, greater age, mechanical ventilation (MV) and acute physiology and chronic health evaluation (APACHE) II score were independent predictors of poor outcome. For patients with HM, culture proven infection, age, MV and inotropes were negative predictors. Disease-specific factors including hematological diagnosis, neutropenia, remission status, prior stem cell transplant, time from diagnosis to admission and degree of prior treatment were not predictive. Overall survival of patients with HM was worse than that recently reported from specialist units.

Severity of illness scoring systems in the intensive care unit

OBJECTIVE

Adult intensive care unit prognostic models have been used for predicting patient outcome for three decades. The goal of this review is to describe the different versions of the main adult intensive care unit prognostic models and discuss their potential roles.

DATA SOURCE

PubMed search and review of the relevant medical literature.

SUMMARY

The main prognostic models for assessing the overall severity of illness in critically ill adults are Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score, and Mortality Probability Model. Simplified Acute Physiology Score and Mortality Probability Model have been updated to their third versions and Acute Physiology and Chronic Health Evaluation to its fourth version. The development of prognostic models is usually followed by internal and external validation and performance assessment. Performance is assessed by area under the receiver operating characteristic curve for discrimination and Hosmer-Lemeshow statistic for calibration. The areas under the receiver operating characteristic curve of Simplified Acute Physiology Score 3, Acute Physiology and Chronic Health Evaluation IV, and Mortality Probability Model0 III were 0.85, 0.88, and 0.82, respectively, and all these three fourth-generation models had good calibration. The models have been extensively used for case-mix adjustment in clinical research and epidemiology, but their role in benchmarking, performance improvement, resource use, and clinical decision support has been less well studied.

CONCLUSIONS

The fourth-generation Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score 3, Acute Physiology and Chronic Health Evaluation IV, and Mortality Probability Model0 III adult prognostic models, perform well in predicting mortality. Future studies are needed to determine their roles for benchmarking, performance improvement, resource use, and clinical decision support.

Prediction of clinical conditions after coronary bypass surgery using dynamic data analysis

This work studies the impact of using dynamic information as features in a machine learning algorithm for the prediction task of classifying critically ill patients in two classes according to the time they need to reach a stable state after coronary bypass surgery: less or more than 9 h. On the basis of five physiological variables (heart rate, systolic arterial blood pressure, systolic pulmonary pressure, blood temperature and oxygen saturation), different dynamic features were extracted, namely the means and standard deviations at different moments in time, coefficients of multivariate autoregressive models and cepstral coefficients. These sets of features served subsequently as inputs for a Gaussian process and the prediction results were compared with the case where only admission data was used for the classification. The dynamic features, especially the cepstral coefficients (aROC: 0.749, Brier score: 0.206), resulted in higher performances when compared to static admission data (aROC: 0.547, Brier score: 0.247). The differences in performance are shown to be significant. In all cases, the Gaussian process classifier outperformed to logistic regression.

Persisting high levels of plasma pentraxin 3 over the first days after severe sepsis and septic shock onset are associated with mortality

PURPOSE

Pentraxin 3 (PTX3) is an inflammatory mediator produced by neutrophils, macrophages, myeloid dendritic and endothelial cells. During sepsis a massive inflammatory activation and coagulation/fibrinolysis dysfunction occur. PTX3, as a mediator of inflammation, may represent an early marker of severity and outcome in sepsis.

METHODS

This study is based on a prospective trial regarding the impact of glycemic control on coagulation in sepsis. Ninety patients admitted to three general intensive care units were enrolled when severe sepsis or septic shock was diagnosed. At enrollment, we recorded sepsis signs, disease severity, coagulation activation [prothrombin fragments 1 + 2 (F(1+2))] and fibrinolysis inhibition [plasminogen activator inhibitor-1 (PAI-1)]. We measured plasma PTX3 levels at enrollment, everyday until day 7, then at days 9, 11, 13, 18, 23 and 28. Mortality was recorded at day 90.

RESULTS

Although not different on day 1, PTX3 remained significantly higher in non-survivors than in survivors over the first 5 days (p = 0.002 by general linear model). On day 1, PTX3 levels were higher in septic shock than in severely septic patients (p = 0.029). Day 1 PTX3 was significantly correlated with platelet count (p < 0.001), SAPS II score (p = 0.006) and SOFA score (p < 0.001). Day 1 PTX3 was correlated with F(1+2) concentration and with PAI-1 activity and concentration (p < 0.05 for all).

CONCLUSIONS

Persisting high levels of circulating PTX3 over the first days from sepsis onset may be associated with mortality. PTX3 correlates with severity of sepsis and with sepsis-associated coagulation/fibrinolysis dysfunction.

Reducing diagnostic error with computer-based clinical decision support

Information technology approaches to delivering diagnostic clinical decision support (CDS) are the subject of the papers to follow in the proceedings. These will address the history of CDS and present day approaches (Miller), evaluation of diagnostic CDS methods (Friedman), and the role of clinical documentation in supporting diagnostic decision making (Schiff). In addition, several other considerations relating to this topic are interesting to ponder. We are moving toward increased understanding of gene regulation and gene expression, identification of biomarkers, and the ability to predict patient response to disease and to tailor treatments to these individual variations-referred to as "personalized" or, more recently, "predictive" medicine. Consequently, diagnostic decision making is more and more linked to management decision making, and generic diagnostic labels like "diabetes" or "colon cancer" will no longer be sufficient, because they don't tell us what to do. Ultimately, if we have more complete data including more structured capture of phenomic data as well as the characterization of the patient's genome, direct prediction from responses of highly refined subsets of similar patients in a database can be used to select appropriate management, the effectiveness of which was demonstrated in projects in selected limited domains as early as the 1970s. In general, there are six classes of methodologies, including the above, which can be applied to delivering CDS. In addition, patients are becoming more knowledgeable and should be regarded as active participants, not only in helping to obtain data but also in their own status assessment and as recipients of decision support. With the above advances, this is a very promising time to be engaged in pursuit of methods of CDS.

Is there an advantage in scoring early embryos on more than one day?

BACKGROUND

This study was undertaken to determine what characteristics should be recorded on which days to build a predictive model for selection of Day 3 embryos.

METHODS

Embryos failing to form a clinical sac or that formed a viable fetus (to > or =12 weeks), and transferred singly (n = 269) or in pairs (n = 1326) were scored for early cleavage and pronuclear status on Day 1, and cell number, fragmentation, and symmetry on Days 2 and 3, with number of nuclei per blastomere also recorded on Day 2. Seven candidate models were identified using a priori clinical knowledge and univariate analyses. Each model was fit on a training-set and evaluated on a test-set with resampling, with discrimination assessed using the area under the ROC curve (AUC) and calibration assessed using the Hosmer-Lemeshow statistics.

RESULTS

Models built using Day 1, 2 or 3 scores independently on the 30 resampled data sets showed that Day 1 evaluations provided the poorest predictive value (median AUC = 0.683 versus 0.729 and 0.725, for Day 2 and 3). Combining information from Day 1, 2 and 3 marginally improved discrimination (median AUC = 0.737). Using the final Day 3 model fitted on the whole dataset, the median AUC was 0.732 (95% CI, 0.700-0.764), and 68.6% of embryos would be correctly classified with a cutoff probability equal to 0.3.

CONCLUSIONS

Day 2 or Day 3 evaluations alone are sufficient for morphological selection of cleavage stage embryos. The derived regression coefficients can be used prospectively in an algorithm to rank embryos for selection.

Predicting the outcome of patients with subarachnoid hemorrhage using machine learning techniques

BACKGROUND

Outcome prediction for subarachnoid hemorrhage (SAH) helps guide care and compare global management strategies. Logistic regression models for outcome prediction may be cumbersome to apply in clinical practice.

OBJECTIVE

To use machine learning techniques to build a model of outcome prediction that makes the knowledge discovered from the data explicit and communicable to domain experts.

MATERIAL AND METHODS

A derivation cohort (n = 441) of nonselected SAH cases was analyzed using different classification algorithms to generate decision trees and decision rules. Algorithms used were C4.5, fast decision tree learner, partial decision trees, repeated incremental pruning to produce error reduction, nearest neighbor with generalization, and ripple down rule learner. Outcome was dichotomized in favorable [Glasgow outcome scale (GOS) = I-II] and poor (GOS = III-V). An independent cohort (n = 193) was used for validation. An exploratory questionnaire was given to potential users (specialist doctors) to gather their opinion on the classifier and its usability in clinical routine.

RESULTS

The best classifier was obtained with the C4.5 algorithm. It uses only two attributes [World Federation of Neurological Surgeons (WFNS) and Fisher's scale] and leads to a simple decision tree. The accuracy of the classifier [area under the ROC curve (AUC) = 0.84; confidence interval (CI) = 0.80-0.88] is similar to that obtained by a logistic regression model (AUC = 0.86; CI = 0.83-0.89) derived from the same data and is considered better fit for clinical use.

Support vector machine versus logistic regression modeling for prediction of hospital mortality in critically ill patients with haematological malignancies

Background

Several models for mortality prediction have been constructed for critically ill patients with haematological malignancies in recent years. These models have proven to be equally or more accurate in predicting hospital mortality in patients with haematological malignancies than ICU severity of illness scores such as the APACHE II or SAPS II [1]. The objective of this study is to compare the accuracy of predicting hospital mortality in patients with haematological malignancies admitted to the ICU between models based on multiple logistic regression (MLR) and support vector machine (SVM) based models.

Methods

352 patients with haematological malignancies admitted to the ICU between 1997 and 2006 for a life-threatening complication were included. 252 patient records were used for training of the models and 100 were used for validation. In a first model 12 input variables were included for comparison between MLR and SVM. In a second more complex model 17 input variables were used. MLR and SVM analysis were performed independently from each other. Discrimination was evaluated using the area under the receiver operating characteristic (ROC) curves (± SE).

Results

The area under ROC curve for the MLR and SVM in the validation data set were 0.768 (± 0.04) vs. 0.802 (± 0.04) in the first model (p = 0.19) and 0.781 (± 0.05) vs. 0.808 (± 0.04) in the second more complex model (p = 0.44). SVM needed only 4 variables to make its prediction in both models, whereas MLR needed 7 and 8 variables in the first and second model respectively.

Conclusion

The discriminative power of both the MLR and SVM models was good. No statistically significant differences were found in discriminative power between MLR and SVM for prediction of hospital mortality in critically ill patients with haematological malignancies.

Rating organ failure via adverse events using data mining in the intensive care unit

OBJECTIVE

The main intensive care unit (ICU) goal is to avoid or reverse the organ failure process by adopting a timely intervention. Within this context, early identification of organ impairment is a key issue. The sequential organ failure assessment (SOFA) is an expert-driven score that is widely used in European ICUs to quantify organ disorder. This work proposes a complementary data-driven approach based on adverse events, defined from commonly monitored biometrics. The aim is to study the impact of these events when predicting the risk of ICU organ failure.

MATERIALS AND METHODS

A large database was considered, with a total of 25,215 daily records taken from 4425 patients and 42 European ICUs. The input variables include the case mix (i.e. age, diagnosis, admission type and admission from) and adverse events defined from four bedside physiologic variables (i.e. systolic blood pressure, heart rate, pulse oximeter oxygen saturation and urine output). The output target is the organ status (i.e. normal, dysfunction or failure) of six organ systems (respiratory, coagulation, hepatic, cardiovascular, neurological and renal), as measured by the SOFA score. Two data mining (DM) methods were compared: multinomial logistic regression (MLR) and artificial neural networks (ANNs). These methods were tested in the R statistical environment, using 20 runs of a 5-fold cross-validation scheme. The area under the receiver operator characteristic (ROC) curve and Brier score were used as the discrimination and calibration measures.

RESULTS

The best performance was obtained by the ANNs, outperforming the MLR in both discrimination and calibration criteria. The ANNs obtained an average (over all organs) area under the ROC curve of 64, 69 and 74% and Brier scores of 0.18, 0.16 and 0.09 for the dysfunction, normal and failure organ conditions, respectively. In particular, very good results were achieved when predicting renal failure (ROC curve area of 76% and Brier score of 0.06).

CONCLUSION

Adverse events, taken from bedside monitored data, are important intermediate outcomes, contributing to a timely recognition of organ dysfunction and failure during ICU length of stay. The obtained results show that it is possible to use DM methods to get knowledge from easy obtainable data, thus making room for the development of intelligent clinical alarm monitoring.

Simple coagulation tests improve survival prediction in patients with septic shock

BACKGROUND

Classic mortality prediction models in intensive care units (ICUs) are based on clinical scores, which do not contain any coagulation test (SAPS-II or SOFA scores).

OBJECTIVES

To determine whether coagulation tests can improve mortality prediction in patients with septic shock.

PATIENTS AND METHODS

One hundred fifty-eight consecutive patients with septic shock entering our institution's ICU were investigated on the first day of admission, and deaths were registered during the first month.

RESULTS

Among all the coagulation tests performed, only the fibrinogen (Fg) plasma level, together with the SAPS-II score and the age, were included in our simplified mortality score [area under the receiver operating curve (AUC) 0.927, standard deviation (SD) 0.030], which was more efficient than SAPS-II and SOFA scores themselves in predicting first-week mortality, its optimized cut-off having a very high negative predictive value (NPV) [0.989; 95% confidence interval (CI) 0.967-1.000)]. A simplified score predicting first-month mortality, containing the prothrombin ratio and the antithrombin activity values in addition to the age, the hemoglobin concentration, and the SAPS-II and SOFA scores (AUC 0.889, SD 0.026), was found to be superior to the SAPS-II and SOFA scores, the optimized cut-off value having a high NPV (0.952; 95% CI 0.888-1.000).

CONCLUSIONS

In patients admitted to an ICU with septic shock, some initial coagulation test values can help identify those who will survive in the first week and then in the first month.

Severity of Illness and Organ Failure Assessment in Adult Intensive Care Units

The critical care community has been using severity and organ failure assessment tools for over 2 decades. The major adult severity assessment models are Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score, and Mortality Probability Model. All three recent versions of these models perform well in predicting hospital mortality. Sequential Organ Failure Assessment score is the most used tool for assessment of multiple organ failure. These tools have been used extensively in clinical research involving critically ill patients and for benchmarking and the measurement of performance improvement. Their roles as clinical decision support tools at the bedside await future studies because of their unknown or poor performance at the individual patient level.