Predicting the outcome for individual patients with traumatic brain injury: a case-based review

Development and Validation of a Machine Learning COVID-19 Veteran (COVet) Deterioration Risk Score

BACKGROUND AND OBJECTIVE

To develop the COVid Veteran (COVet) score for clinical deterioration in Veterans hospitalized with COVID-19 and further validate this model in both Veteran and non-Veteran samples. No such score has been derived and validated while incorporating a Veteran sample.

DERIVATION COHORT

Adults (age ≥ 18 yr) hospitalized outside the ICU with a diagnosis of COVID-19 for model development to the Veterans Health Administration (VHA) (n = 80 hospitals).

VALIDATION COHORT

External validation occurred in a VHA cohort of 34 hospitals, as well as six non-Veteran health systems for further external validation (n = 21 hospitals) between 2020 and 2023.

PREDICTION MODEL

eXtreme Gradient Boosting machine learning methods were used, and performance was assessed using the area under the receiver operating characteristic curve and compared with the National Early Warning Score (NEWS). The primary outcome was transfer to the ICU or death within 24 hours of each new variable observation. Model predictor variables included demographics, vital signs, structured flowsheet data, and laboratory values.

RESULTS

A total of 96,908 admissions occurred during the study period, of which 59,897 were in the Veteran sample and 37,011 were in the non-Veteran sample. During external validation in the Veteran sample, the model demonstrated excellent discrimination, with an area under the receiver operating characteristic curve of 0.88. This was significantly higher than NEWS (0.79; p < 0.01). In the non-Veteran sample, the model also demonstrated excellent discrimination (0.86 vs. 0.79 for NEWS; p < 0.01). The top three variables of importance were eosinophil percentage, mean oxygen saturation in the prior 24-hour period, and worst mental status in the prior 24-hour period.

CONCLUSIONS

We used machine learning methods to develop and validate a highly accurate early warning score in both Veterans and non-Veterans hospitalized with COVID-19. The model could lead to earlier identification and therapy, which may improve outcomes.

Comparative analysis of CRASH and IMPACT in predicting the outcome of 340 patients with traumatic brain injury

Background

Both the International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) and the Corticosteroid randomization after significant head injury (CRASH) models are globally acknowledged prognostic algorithms for assessing traumatic brain injury (TBI) outcomes. The aim of this study is to externalize the validation process and juxtapose the prognostic accuracy of the CRASH and IMPACT models in moderate-to-severe TBI patients in the Chinese population.

Methods

We conducted a retrospective study encompassing a cohort of 340 adult TBI patients (aged > 18 years), presenting with Glasgow Coma Scale (GCS) scores ranging from 3 to 12. The data were accrued over 2 years (2020-2022). The primary endpoints were 14-day mortality rates and 6-month Glasgow Outcome Scale (GOS) scores. Analytical metrics, including the area under the receiver operating characteristic curve for discrimination and the Brier score for predictive precision were employed to quantitatively evaluate the model performance.

Results

Mortality rates at the 14-day and 6-month intervals, as well as the 6-month unfavorable GOS outcomes, were established to be 22.06, 40.29, and 65.59%, respectively. The IMPACT models had area under the curves (AUCs) of 0.873, 0.912, and 0.927 for the 6-month unfavorable GOS outcomes, with respective Brier scores of 0.14, 0.12, and 0.11. On the other hand, the AUCs associated with the six-month mortality were 0.883, 0.909, and 0.912, and the corresponding Brier scores were 0.15, 0.14, and 0.13, respectively. The CRASH models exhibited AUCs of 0.862 and 0.878 for the 6-month adverse outcomes, with uniform Brier scores of 0.18. The 14-day mortality rates had AUCs of 0.867 and 0.87, and corresponding Brier scores of 0.21 and 0.22, respectively.

Conclusion

Both the CRASH and IMPACT algorithms offer reliable prognostic estimations for patients suffering from craniocerebral injuries. However, compared to the CRASH model, the IMPACT model has superior predictive accuracy, albeit at the cost of increased computational intricacy.

Machine Learning Algorithm Predicts Mortality Risk in Intensive Care Unit for Patients with Traumatic Brain Injury

BACKGROUND

Numerous mortality prediction tools are currently available to assist patients with moderate to severe traumatic brain injury (TBI). However, an algorithm that utilizes various machine learning methods and employs diverse combinations of features to identify the most suitable predicting outcomes of brain injury patients in the intensive care unit (ICU) has not yet been well-established.

METHOD

Between January 2016 and December 2021, we retrospectively collected data from the electronic medical records of Chi Mei Medical Center, comprising 2260 TBI patients admitted to the ICU. A total of 42 features were incorporated into the analysis using four different machine learning models, which were then segmented into various feature combinations. The predictive performance was assessed using the area under the curve (AUC) of the receiver operating characteristic (ROC) curve and validated using the Delong test.

RESULT

The AUC for each model under different feature combinations ranged from 0.877 (logistic regression with 14 features) to 0.921 (random forest with 22 features). The Delong test indicated that the predictive performance of the machine learning models is better than that of traditional tools such as APACHE II and SOFA scores.

CONCLUSION

Our machine learning training demonstrated that the predictive accuracy of the LightGBM is better than that of APACHE II and SOFA scores. These features are readily available on the first day of patient admission to the ICU. By integrating this model into the clinical platform, we can offer clinicians an immediate prognosis for the patient, thereby establishing a bridge for educating and communicating with family members.

Machine learning-based prediction of in-ICU mortality in pneumonia patients

Conventional severity-of-illness scoring systems have shown suboptimal performance for predicting in-intensive care unit (ICU) mortality in patients with severe pneumonia. This study aimed to develop and validate machine learning (ML) models for mortality prediction in patients with severe pneumonia. This retrospective study evaluated patients admitted to the ICU for severe pneumonia between January 2016 and December 2021. The predictive performance was analyzed by comparing the area under the receiver operating characteristic curve (AU-ROC) of ML models to that of conventional severity-of-illness scoring systems. Three ML models were evaluated: (1) logistic regression with L2 regularization, (2) gradient-boosted decision tree (LightGBM), and (3) multilayer perceptron (MLP). Among the 816 pneumonia patients included, 223 (27.3%) patients died. All ML models significantly outperformed the Simplified Acute Physiology Score II (AU-ROC: 0.650 [0.584-0.716] vs 0.820 [0.771-0.869] for logistic regression vs 0.827 [0.777-0.876] for LightGBM 0.838 [0.791-0.884] for MLP; P < 0.001). In the analysis for NRI, the LightGBM and MLP models showed superior reclassification compared with the logistic regression model in predicting in-ICU mortality in all length of stay in the ICU subgroups; all age subgroups; all subgroups with any APACHE II score, PaO₂/FiO₂ ratio < 200; all subgroups with or without history of respiratory disease; with or without history of CVA or dementia; treatment with mechanical ventilation, and use of inotropic agents. In conclusion, the ML models have excellent performance in predicting in-ICU mortality in patients with severe pneumonia. Moreover, this study highlights the potential advantages of selecting individual ML models for predicting in-ICU mortality in different subgroups.

Predicting Clinical Outcomes 7-10 Years after Severe Traumatic Brain Injury: Exploring the Prognostic Utility of the IMPACT Lab Model and Cerebrospinal Fluid UCH-L1 and MAP-2

BACKGROUND

Severe traumatic brain injury (TBI) is a major contributor to disability and mortality in the industrialized world. Outcomes of severe TBI are profoundly heterogeneous, complicating outcome prognostication. Several prognostic models have been validated for acute prediction of 6-month global outcomes following TBI (e.g., morbidity/mortality). In this preliminary observational prognostic study, we assess the utility of the International Mission on Prognosis and Analysis of Clinical Trials in TBI (IMPACT) Lab model in predicting longer term global and cognitive outcomes (7-10 years post injury) and the extent to which cerebrospinal fluid (CSF) biomarkers enhance outcome prediction.

METHODS

Very long-term global outcome was assessed in a total of 59 participants (41 of whom did not survive their injuries) using the Glasgow Outcome Scale-Extended and Disability Rating Scale. More detailed outcome information regarding cognitive functioning in daily life was collected from 18 participants surviving to 7-10 years post injury using the Cognitive Subscale of the Functional Independence Measure. A subset (n = 10) of these participants also completed performance-based cognitive testing (Digit Span Test) by telephone. The IMPACT lab model was applied to determine its prognostic value in relation to very long-term outcomes as well as the additive effects of acute CSF ubiquitin C-terminal hydrolase-L1 (UCH-L1) and microtubule associated protein 2 (MAP-2) concentrations.

RESULTS

The IMPACT lab model discriminated favorable versus unfavorable 7- to 10-year outcome with an area under the receiver operating characteristic curve of 0.80. Higher IMPACT lab model risk scores predicted greater extent of very long-term morbidity (β = 0.488 p = 0.000) as well as reduced cognitive independence (β =  - 0.515, p = 0.034). Acute elevations in UCH-L1 levels were also predictive of lesser independence in cognitive activities in daily life at very long-term follow-up (β = 0.286, p = 0.048). Addition of two CSF biomarkers significantly improved prediction of very long-term neuropsychological performance among survivors, with the overall model (including IMPACT lab score, UCH-L1, and MAP-2) explaining 89.6% of variance in cognitive performance 7-10 years post injury (p = 0.008). Higher acute UCH-L1 concentrations were predictive of poorer cognitive performance (β =  - 0.496, p = 0.029), whereas higher acute MAP-2 concentrations demonstrated a strong cognitive protective effect (β = 0.679, p = 0.010).

CONCLUSIONS

Although preliminary, results suggest that existing prognostic models, including models with incorporation of CSF markers, may be applied to predict outcome of severe TBI years after injury. Continued research is needed examining early predictors of longer-term outcomes following TBI to identify potential targets for clinical trials that could impact long-ranging functional and cognitive outcomes.

A proposed novel traumatic brain injury classification system - an overview and inter-rater reliability validation on behalf of the Society of British Neurological Surgeons

INTRODUCTION

The measurement of traumatic brain injury (TBI) 'severity' has traditionally been based on the earliest Glasgow Coma Score (GCS) recorded, however, the underlying parenchymal pathology is highly heterogonous. This heterogeneity renders prediction of outcome on an individual patient level inaccurate and makes comparison between patients both in clinical practice and research difficult. The complexity of this heterogeneity has resulted in generic all encompassing 'traumatic brain injury protocols'. Early management and studies of neuro-protectants are often done irrespective of TBI type, yet it may well be that a specific treatment may be beneficial in a subset of TBI pathologies.

METHODS

A simple CT-based classification system rating the recognised types of blunt TBI (extradural, subdural, subarachnoid haemorrhage, contusions/intracerebral haematoma and diffuse axonal injury) as mild (1), moderate (2) or severe (3) is proposed. Hypoxic brain injury, a common secondary injury following TBI, is also included. Scores can be combined to reflect concomitant types of TBI and predominant location of injury is also recorded. To assess interrater reliability, 50 patient CT images were assessed by 5 independent clinicians of varying experience. Interrater reliability was calculated using overall agreement through Cronbach's alpha including confidence intervals for intra-class coefficients.

RESULTS

Interrater reliability scores showed strong agreement for same score and same injury for TBIs with blood on CT and Cronbach's alpha co-efficient (range 0.87-0.93) demonstrated excellent correlation between raters. Cronbach's alpha was not affected when individual raters were removed.

CONCLUSIONS

The proposed simple CT classification system has good inter-rater reliability and hence potentially could enable better individual prognostication and targeted treatments to be compared while also accounting for multiple intracranial injury types. Further studies are proposed and underway.

Comparison of Machine Learning Methods for Predicting Outcomes After In-Hospital Cardiac Arrest

OBJECTIVES

Prognostication of neurologic status among survivors of in-hospital cardiac arrests remains a challenging task for physicians. Although models such as the Cardiac Arrest Survival Post-Resuscitation In-hospital score are useful for predicting neurologic outcomes, they were developed using traditional statistical techniques. In this study, we derive and compare the performance of several machine learning models with each other and with the Cardiac Arrest Survival Post-Resuscitation In-hospital score for predicting the likelihood of favorable neurologic outcomes among survivors of resuscitation.

DESIGN

Analysis of the Get With The Guidelines-Resuscitation registry.

SETTING

Seven-hundred fifty-five hospitals participating in Get With The Guidelines-Resuscitation from January 1, 2001, to January 28, 2017.

PATIENTS

Adult in-hospital cardiac arrest survivors.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Of 117,674 patients in our cohort, 28,409 (24%) had a favorable neurologic outcome, as defined as survival with a Cerebral Performance Category score of less than or equal to 2 at discharge. Using patient characteristics, pre-existing conditions, prearrest interventions, and periarrest variables, we constructed logistic regression, support vector machines, random forests, gradient boosted machines, and neural network machine learning models to predict favorable neurologic outcome. Events prior to October 20, 2009, were used for model derivation, and all subsequent events were used for validation. The gradient boosted machine predicted favorable neurologic status at discharge significantly better than the Cardiac Arrest Survival Post-Resuscitation In-hospital score (C-statistic: 0.81 vs 0.73; p < 0.001) and outperformed all other machine learning models in terms of discrimination, calibration, and accuracy measures. Variables that were consistently most important for prediction across all models were duration of arrest, initial cardiac arrest rhythm, admission Cerebral Performance Category score, and age.

CONCLUSIONS

The gradient boosted machine algorithm was the most accurate for predicting favorable neurologic outcomes in in-hospital cardiac arrest survivors. Our results highlight the utility of machine learning for predicting neurologic outcomes in resuscitated patients.

Outcome Prediction after Moderate and Severe Traumatic Brain Injury: External Validation of Two Established Prognostic Models in 1742 European Patients

The International Mission on Prognosis and Analysis of Clinical Trials in Traumatic Brain Injury (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models predict functional outcome after moderate and severe traumatic brain injury (TBI). We aimed to assess their performance in a contemporary cohort of patients across Europe. The Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) core study is a prospective, observational cohort study in patients presenting with TBI and an indication for brain computed tomography. The CENTER-TBI core cohort consists of 4509 TBI patients available for analyses from 59 centers in 18 countries across Europe and Israel. The IMPACT validation cohort included 1173 patients with GCS ≤12, age ≥14, and 6-month Glasgow Outcome Scale-Extended (GOSE) available. The CRASH validation cohort contained 1742 patients with GCS ≤14, age ≥16, and 14-day mortality or 6-month GOSE available. Performance of the three IMPACT and two CRASH model variants was assessed with discrimination (area under the receiver operating characteristic curve; AUC) and calibration (comparison of observed vs. predicted outcome rates). For IMPACT, model discrimination was good, with AUCs ranging between 0.77 and 0.85 in 1173 patients and between 0.80 and 0.88 in the broader CRASH selection (n = 1742). For CRASH, AUCs ranged between 0.82 and 0.88 in 1742 patients and between 0.66 and 0.80 in the stricter IMPACT selection (n = 1173). Calibration of the IMPACT and CRASH models was generally moderate, with calibration-in-the-large and calibration slopes ranging between -2.02 and 0.61 and between 0.48 and 1.39, respectively. The IMPACT and CRASH models adequately identify patients at high risk for mortality or unfavorable outcome, which supports their use in research settings and for benchmarking in the context of quality-of-care assessment.

Prediction of in-hospital mortality in patients with post traumatic brain injury using National Trauma Registry and Machine Learning Approach

BACKGROUND

The use of machine learning techniques to predict diseases outcomes has grown significantly in the last decade. Several studies prove that the machine learning predictive techniques outperform the classical multivariate techniques. We aimed to build a machine learning predictive model to predict the in-hospital mortality for patients who sustained Traumatic Brain Injury (TBI).

METHODS

Adult patients with TBI who were hospitalized in the level 1 trauma center in the period from January 2014 to February 2019 were included in this study. Patients' demographics, injury characteristics and CT findings were used as predictors. The predictive performance of Artificial Neural Networks (ANN) and Support Vector Machines (SVM) was evaluated in terms of accuracy, Area Under the Curve (AUC), sensitivity, precision, Negative Predictive Value (NPV), specificity and F-score.

RESULTS

A total of 1620 eligible patients were included in the study (1417 survival and 203 non-survivals). Both models achieved accuracy over 91% and AUC over 93%. SVM achieved the optimal performance with accuracy 95.6% and AUC 96%.

CONCLUSIONS

for prediction of mortality in patients with TBI, SVM outperformed the well-known classical models that utilized the conventional multivariate analytical techniques.

Machine learning-based dynamic mortality prediction after traumatic brain injury

Our aim was to create simple and largely scalable machine learning-based algorithms that could predict mortality in a real-time fashion during intensive care after traumatic brain injury. We performed an observational multicenter study including adult TBI patients that were monitored for intracranial pressure (ICP) for at least 24 h in three ICUs. We used machine learning-based logistic regression modeling to create two algorithms (based on ICP, mean arterial pressure [MAP], cerebral perfusion pressure [CPP] and Glasgow Coma Scale [GCS]) to predict 30-day mortality. We used a stratified cross-validation technique for internal validation. Of 472 included patients, 92 patients (19%) died within 30 days. Following cross-validation, the ICP-MAP-CPP algorithm's area under the receiver operating characteristic curve (AUC) increased from 0.67 (95% confidence interval [CI] 0.60-0.74) on day 1 to 0.81 (95% CI 0.75-0.87) on day 5. The ICP-MAP-CPP-GCS algorithm's AUC increased from 0.72 (95% CI 0.64-0.78) on day 1 to 0.84 (95% CI 0.78-0.90) on day 5. Algorithm misclassification was seen among patients undergoing decompressive craniectomy. In conclusion, we present a new concept of dynamic prognostication for patients with TBI treated in the ICU. Our simple algorithms, based on only three and four main variables, discriminated between survivors and non-survivors with accuracies up to 81% and 84%. These open-sourced simple algorithms can likely be further developed, also in low and middle-income countries.

Blood Biomarkers for Traumatic Brain Injury: A Quantitative Assessment of Diagnostic and Prognostic Accuracy

Blood biomarkers have been explored for their potential to provide objective measures in the assessment of traumatic brain injury (TBI). However, it is not clear which biomarkers are best for diagnosis and prognosis in different severities of TBI. Here, we compare existing studies on the discriminative abilities of serum biomarkers for four commonly studied clinical situations: detecting concussion, predicting intracranial damage after mild TBI (mTBI), predicting delayed recovery after mTBI, and predicting adverse outcome after severe TBI (sTBI). We conducted a literature search of publications on biomarkers in TBI published up until July 2018. Operating characteristics were pooled for each biomarker for comparison. For detecting concussion, 4 biomarker panels and creatine kinase B type had excellent discriminative ability. For detecting intracranial injury and the need for a head CT scan after mTBI, 2 biomarker panels, and hyperphosphorylated tau had excellent operating characteristics. For predicting delayed recovery after mTBI, top candidates included calpain-derived αII-spectrin N-terminal fragment, tau A, neurofilament light, and ghrelin. For predicting adverse outcome following sTBI, no biomarker had excellent performance, but several had good performance, including markers of coagulation and inflammation, structural proteins in the brain, and proteins involved in homeostasis. The highest-performing biomarkers in each of these categories may provide insight into the pathophysiologies underlying mild and severe TBI. With further study, these biomarkers have the potential to be used alongside clinical and radiological data to improve TBI diagnostics, prognostics, and evidence-based medical management.

Mortality prediction in patients with isolated moderate and severe traumatic brain injury using machine learning models

BACKGROUND

The purpose of this study was to build a model of machine learning (ML) for the prediction of mortality in patients with isolated moderate and severe traumatic brain injury (TBI).

METHODS

Hospitalized adult patients registered in the Trauma Registry System between January 2009 and December 2015 were enrolled in this study. Only patients with an Abbreviated Injury Scale (AIS) score ≥ 3 points related to head injuries were included in this study. A total of 1734 (1564 survival and 170 non-survival) and 325 (293 survival and 32 non-survival) patients were included in the training and test sets, respectively.

RESULTS

Using demographics and injury characteristics, as well as patient laboratory data, predictive tools (e.g., logistic regression [LR], support vector machine [SVM], decision tree [DT], naive Bayes [NB], and artificial neural networks [ANN]) were used to determine the mortality of individual patients. The predictive performance was evaluated by accuracy, sensitivity, and specificity, as well as by area under the curve (AUC) measures of receiver operator characteristic curves. In the training set, all five ML models had a specificity of more than 90% and all ML models (except the NB) achieved an accuracy of more than 90%. Among them, the ANN had the highest sensitivity (80.59%) in mortality prediction. Regarding performance, the ANN had the highest AUC (0.968), followed by the LR (0.942), SVM (0.935), NB (0.908), and DT (0.872). In the test set, the ANN had the highest sensitivity (84.38%) in mortality prediction, followed by the SVM (65.63%), LR (59.38%), NB (59.38%), and DT (43.75%).

CONCLUSIONS

The ANN model provided the best prediction of mortality for patients with isolated moderate and severe TBI.

Prediction of Mortality in Patients with Isolated Traumatic Subarachnoid Hemorrhage Using a Decision Tree Classifier: A Retrospective Analysis Based on a Trauma Registry System

Background: In contrast to patients with traumatic subarachnoid hemorrhage (tSAH) in the presence of other types of intracranial hemorrhage, the prognosis of patients with isolated tSAH is good. The incidence of mortality in these patients ranges from 0–2.5%. However, few data or predictive models are available for the identification of patients with a high mortality risk. In this study, we aimed to construct a model for mortality prediction using a decision tree (DT) algorithm, along with data obtained from a population-based trauma registry, in a Level 1 trauma center. Methods: Five hundred and forty-five patients with isolated tSAH, including 533 patients who survived and 12 who died, between January 2009 and December 2016, were allocated to training (n = 377) or test (n = 168) sets. Using the data on demographics and injury characteristics, as well as laboratory data of the patients, classification and regression tree (CART) analysis was performed based on the Gini impurity index, using the rpart function in the rpart package in R. Results: In this established DT model, three nodes (head Abbreviated Injury Scale (AIS) score ≤4, creatinine (Cr) <1.4 mg/dL, and age <76 years) were identified as important determinative variables in the prediction of mortality. Of the patients with isolated tSAH, 60% of those with a head AIS >4 died, as did the 57% of those with an AIS score ≤4, but Cr ≥1.4 and age ≥76 years. All patients who did not meet the above-mentioned criteria survived. With all the variables in the model, the DT achieved an accuracy of 97.9% (sensitivity of 90.9% and specificity of 98.1%) and 97.7% (sensitivity of 100% and specificity of 97.7%), for the training set and test set, respectively. Conclusions: The study established a DT model with three nodes (head AIS score ≤4, Cr <1.4, and age <76 years) to predict fatal outcomes in patients with isolated tSAH. The proposed decision-making algorithm may help identify patients with a high risk of mortality.

The aggressiveness of neurotrauma practitioners and the influence of the IMPACT prognostic calculator

Published guidelines have helped to standardize the care of patients with traumatic brain injury; however, there remains substantial variation in the decision to pursue or withhold aggressive care. The International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) prognostic calculator offers the opportunity to study and decrease variability in physician aggressiveness. The authors wish to understand how IMPACT’s prognostic calculations currently influence patient care and to better understand physician aggressiveness. The authors conducted an anonymous international, multidisciplinary survey of practitioners who provide care to patients with traumatic brain injury. Questions were designed to determine current use rates of the IMPACT prognostic calculator and thresholds of age and risk for death or poor outcome that might cause practitioners to consider withholding aggressive care. Correlations between physician aggressiveness, putative predictors of aggressiveness, and demographics were examined. One hundred fifty-four responses were received, half of which were from physicians who were familiar with the IMPACT calculator. The most frequent use of the calculator was to improve communication with patients and their families. On average, respondents indicated that in patients older than 76 years or those with a >85% chance of death or poor outcome it might be reasonable to pursue non-aggressive care. These thresholds were robust and were not influenced by provider or institutional characteristics. This study demonstrates the need to educate physicians about the IMPACT prognostic calculator. The consensus values for age and prognosis identified in our study may be explored in future studies aimed at reducing variability in physician aggressiveness and should not serve as a basis for withdrawing care.