The aggressiveness of neurotrauma practitioners and the influence of the IMPACT prognostic calculator

Comparative analysis of CRASH and IMPACT in predicting the outcome of 340 patients with traumatic brain injury

Background

Both the International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) and the Corticosteroid randomization after significant head injury (CRASH) models are globally acknowledged prognostic algorithms for assessing traumatic brain injury (TBI) outcomes. The aim of this study is to externalize the validation process and juxtapose the prognostic accuracy of the CRASH and IMPACT models in moderate-to-severe TBI patients in the Chinese population.

Methods

We conducted a retrospective study encompassing a cohort of 340 adult TBI patients (aged > 18 years), presenting with Glasgow Coma Scale (GCS) scores ranging from 3 to 12. The data were accrued over 2 years (2020-2022). The primary endpoints were 14-day mortality rates and 6-month Glasgow Outcome Scale (GOS) scores. Analytical metrics, including the area under the receiver operating characteristic curve for discrimination and the Brier score for predictive precision were employed to quantitatively evaluate the model performance.

Results

Mortality rates at the 14-day and 6-month intervals, as well as the 6-month unfavorable GOS outcomes, were established to be 22.06, 40.29, and 65.59%, respectively. The IMPACT models had area under the curves (AUCs) of 0.873, 0.912, and 0.927 for the 6-month unfavorable GOS outcomes, with respective Brier scores of 0.14, 0.12, and 0.11. On the other hand, the AUCs associated with the six-month mortality were 0.883, 0.909, and 0.912, and the corresponding Brier scores were 0.15, 0.14, and 0.13, respectively. The CRASH models exhibited AUCs of 0.862 and 0.878 for the 6-month adverse outcomes, with uniform Brier scores of 0.18. The 14-day mortality rates had AUCs of 0.867 and 0.87, and corresponding Brier scores of 0.21 and 0.22, respectively.

Conclusion

Both the CRASH and IMPACT algorithms offer reliable prognostic estimations for patients suffering from craniocerebral injuries. However, compared to the CRASH model, the IMPACT model has superior predictive accuracy, albeit at the cost of increased computational intricacy.

Prognostication and Goals of Care Decisions in Severe Traumatic Brain Injury: A Survey of The Seattle International Severe Traumatic Brain Injury Consensus Conference Working Group

Abstract Best practice guidelines have advanced severe traumatic brain injury (TBI) care; however, there is little that currently informs goals of care decisions and processes despite their importance and frequency. Panelists from the Seattle International severe traumatic Brain Injury Consensus Conference (SIBICC) participated in a survey consisting of 24 questions. Questions queried use of prognostic calculators, variability in and responsibility for goals of care decisions, and acceptability of neurological outcomes, as well as putative means of improving decisions that might limit care. A total of 97.6% of the 42 SIBICC panelists completed the survey. Responses to most questions were highly variable. Overall, panelists reported infrequent use of prognostic calculators, and observed variability in patient prognostication and goals of care decisions. They felt that it would be beneficial for physicians to improve consensus on what constitutes an acceptable neurological outcome as well as what chance of achieving that outcome is acceptable. Panelists felt that the public should help to define what constitutes a good outcome and expressed some support for a "nihilism guard." More than 50% of panelists felt that if it was certain to be permanent, a vegetative state or lower severe disability would justify a withdrawal of care decision, whereas 15% felt that upper severe disability justified such a decision. Whether conceptualizing an ideal or existing prognostic calculator to predict death or an unacceptable outcome, on average a 64-69% chance of a poor outcome was felt to justify treatment withdrawal. These results demonstrate important variability in goals of care decision making and a desire to reduce this variability. Our panel of recognized TBI experts opined on the neurological outcomes and chances of those outcomes that might prompt consideration of care withdrawal; however, imprecision of prognostication and existing prognostication tools is a significant impediment to standardizing the approach to care-limiting decisions.

Prognostic Models for Traumatic Brain Injury Have Good Discrimination but Poor Overall Model Performance for Predicting Mortality and Unfavorable Outcomes

BACKGROUND

The most extensively validated prognostic models for traumatic brain injury (TBI) are the Corticoid Randomization after Significant Head Injury (CRASH) and International Mission on Prognosis and Analysis of Clinical Trials (IMPACT). Model characteristics outside of area under the curve (AUC) are rarely reported.

OBJECTIVE

To report the discriminative validity and overall model performance of the CRASH and IMPACT models for prognosticating death at 14 days (CRASH) and 6 months (IMPACT) and unfavorable outcomes at 6 months after TBI.

METHODS

This retrospective cohort study included prospectively collected patients with severe TBI treated at a single level I trauma center (n = 467). CRASH and IMPACT percent risk values for the given outcome were computed. Unfavorable outcome was defined as a Glasgow Outcome Scale-Extended score of 1 to 4 at 6 months. Binary logistic regressions and receiver operating characteristic analyses were used to differentiate patients from the CRASH and IMPACT prognostic models.

RESULTS

All models had low R 2 values (0.17-0.23) with AUC values from 0.77 to 0.81 and overall accuracies ranging from 72.4% to 78.3%. Sensitivity (35.3-50.0) and positive predictive values (66.7-69.2) were poor in the CRASH models, while specificity (52.3-53.1) and negative predictive values (58.1-63.6) were poor in IMPACT models. All models had unacceptable false positive rates (20.8%-33.3%).

CONCLUSION

Our results were consistent with previous literature regarding discriminative validity (AUC = 0.77-0.81). However, accuracy and false positive rates of both the CRASH and IMPACT models were poor.

Performance of CRASH and IMPACT Prognostic Models for Traumatic Brain Injury at 12 and 24 Months Post-Injury

The Corticoid Randomization after Significant Head Injury (CRASH) and International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) prognostic models are the most reported prognostic models for traumatic brain injury (TBI) in the scientific literature. However, these models were developed and validated to predict 6-month unfavorable outcome and mortality, and growing evidence supports continuous improvements in functional outcome after severe TBI up to 2 years post-injury. The purpose of this study was to evaluate CRASH and IMPACT model performance beyond 6 months post-injury to include 12 and 24 months post-injury. Discriminative validity remained consistent over time and comparable to earlier recovery time points (area under the curve = 0.77-0.83). Both models had poor fit for unfavorable outcomes, explaining less than one quarter of the variation in outcomes for severe TBI patients. The CRASH model had significant values for the Hosmer-Lemeshow test at 12 and 24 months, indicating poor model fit past the previous validation point. There is concern in the scientific literature that TBI prognostic models are being used by neurotrauma clinicians to support clinical decision making despite the goal of the models' development being to support research study design. The results of this study indicate that the CRASH and IMPACT models should not be used in routine clinical practice because of poor model fit that worsens over time and the large, unexplained variance in outcomes.

Prognostic Models in Severe Traumatic Brain Injury: A Systematic Review and Meta-analysis

This review aimed to analyze the results of investigations that performed external validation or that compared prognostic models to identify the models and their variations that showed the best performance in predicting mortality, survival, and unfavorable outcome after severe traumatic brain injury. Pubmed, Embase, Scopus, Web of Science, Cumulative Index to Nursing and Allied Health Literature, Google Scholar, TROVE, and Open Grey databases were searched. A total of 1616 studies were identified and screened, and 15 studies were subsequently included for analysis after applying the selection criteria. The Corticosteroid Randomization After Significant Head Injury (CRASH) and International Mission for Prognosis and Analysis of Clinical Trials in Traumatic Brain Injury (IMPACT) models were the most externally validated among studies of severe traumatic brain injury. The results of the review showed that most publications encountered an area under the curve ≥ 0.70. The area under the curve meta-analysis showed similarity between the CRASH and IMPACT models and their variations for predicting mortality and unfavorable outcomes. Calibration results showed that the variations of CRASH and IMPACT models demonstrated adequate calibration in most studies for both outcomes, but without a clear indication of uncertainties in the evaluations of these models. Based on the results of this meta-analysis, the choice of prognostic models for clinical application may depend on the availability of predictors, characteristics of the population, and trauma care services.

Outcome Prediction in Patients with Severe Traumatic Brain Injury Using Deep Learning from Head CT Scans

Background After severe traumatic brain injury (sTBI), physicians use long-term prognostication to guide acute clinical care yet struggle to predict outcomes in comatose patients. Purpose To develop and evaluate a prognostic model combining deep learning of head CT scans and clinical information to predict long-term outcomes after sTBI. Materials and Methods This was a retrospective analysis of two prospectively collected databases. The model-building set included 537 patients (mean age, 40 years ± 17 [SD]; 422 men) from one institution from November 2002 to December 2018. Transfer learning and curriculum learning were applied to a convolutional neural network using admission head CT to predict mortality and unfavorable outcomes (Glasgow Outcomes Scale scores 1-3) at 6 months. This was combined with clinical input for a holistic fusion model. The models were evaluated using an independent internal test set and an external cohort of 220 patients with sTBI (mean age, 39 years ± 17; 166 men) from 18 institutions in the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) study from February 2014 to April 2018. The models were compared with the International Mission on Prognosis and Analysis of Clinical Trials in TBI (IMPACT) model and the predictions of three neurosurgeons. Area under the receiver operating characteristic curve (AUC) was used as the main model performance metric. Results The fusion model had higher AUCs than did the IMPACT model in the prediction of mortality (AUC, 0.92 [95% CI: 0.86, 0.97] vs 0.80 [95% CI: 0.71, 0.88]; P < .001) and unfavorable outcomes (AUC, 0.88 [95% CI: 0.82, 0.94] vs 0.82 [95% CI: 0.75, 0.90]; P = .04) on the internal data set. For external TRACK-TBI testing, there was no evidence of a significant difference in the performance of any models compared with the IMPACT model (AUC, 0.83; 95% CI: 0.77, 0.90) in the prediction of mortality. The Imaging model (AUC, 0.73; 95% CI: 0.66-0.81; P = .02) and the fusion model (AUC, 0.68; 95% CI: 0.60, 0.76; P = .02) underperformed as compared with the IMPACT model (AUC, 0.83; 95% CI: 0.77, 0.89) in the prediction of unfavorable outcomes. The fusion model outperformed the predictions of the neurosurgeons. Conclusion A deep learning model of head CT and clinical information can be used to predict 6-month outcomes after severe traumatic brain injury. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Haller in this issue.

Outcome Prediction after Moderate and Severe Traumatic Brain Injury: External Validation of Two Established Prognostic Models in 1742 European Patients

The International Mission on Prognosis and Analysis of Clinical Trials in Traumatic Brain Injury (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models predict functional outcome after moderate and severe traumatic brain injury (TBI). We aimed to assess their performance in a contemporary cohort of patients across Europe. The Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) core study is a prospective, observational cohort study in patients presenting with TBI and an indication for brain computed tomography. The CENTER-TBI core cohort consists of 4509 TBI patients available for analyses from 59 centers in 18 countries across Europe and Israel. The IMPACT validation cohort included 1173 patients with GCS ≤12, age ≥14, and 6-month Glasgow Outcome Scale-Extended (GOSE) available. The CRASH validation cohort contained 1742 patients with GCS ≤14, age ≥16, and 14-day mortality or 6-month GOSE available. Performance of the three IMPACT and two CRASH model variants was assessed with discrimination (area under the receiver operating characteristic curve; AUC) and calibration (comparison of observed vs. predicted outcome rates). For IMPACT, model discrimination was good, with AUCs ranging between 0.77 and 0.85 in 1173 patients and between 0.80 and 0.88 in the broader CRASH selection (n = 1742). For CRASH, AUCs ranged between 0.82 and 0.88 in 1742 patients and between 0.66 and 0.80 in the stricter IMPACT selection (n = 1173). Calibration of the IMPACT and CRASH models was generally moderate, with calibration-in-the-large and calibration slopes ranging between -2.02 and 0.61 and between 0.48 and 1.39, respectively. The IMPACT and CRASH models adequately identify patients at high risk for mortality or unfavorable outcome, which supports their use in research settings and for benchmarking in the context of quality-of-care assessment.

The patient with severe traumatic brain injury: clinical decision-making: the first 60 min and beyond

PURPOSE OF REVIEW

There is an urgent need to discuss the uncertainties and paradoxes in clinical decision-making after severe traumatic brain injury (s-TBI). This could improve transparency, reduce variability of practice and enhance shared decision-making with proxies.

RECENT FINDINGS

Clinical decision-making on initiation, continuation and discontinuation of medical treatment may encompass substantial consequences as well as lead to presumed patient benefits. Such decisions, unfortunately, often lack transparency and may be controversial in nature. The very process of decision-making is frequently characterized by both a lack of objective criteria and the absence of validated prognostic models that could predict relevant outcome measures, such as long-term quality and satisfaction with life. In practice, while treatment-limiting decisions are often made in patients during the acute phase immediately after s-TBI, other such severely injured TBI patients have been managed with continued aggressive medical care, and surgical or other procedural interventions have been undertaken in the context of pursuing a more favorable patient outcome. Given this spectrum of care offered to identical patient cohorts, there is clearly a need to identify and decrease existing selectivity, and better ascertain the objective criteria helpful towards more consistent decision-making and thereby reduce the impact of subjective valuations of predicted patient outcome.

SUMMARY

Recent efforts by multiple medical groups have contributed to reduce uncertainty and to improve care and outcome along the entire chain of care. Although an unlimited endeavor for sustaining life seems unrealistic, treatment-limiting decisions should not deprive patients of a chance on achieving an outcome they would have considered acceptable.

Surgical decision making in the setting of severe traumatic brain injury: A survey of neurosurgeons

BACKGROUND

Surgical decision-making in severe traumatic brain injury (TBI) is complex. Neurosurgeons weigh risks and benefits of interventions that have the potential to both maximize the chance of recovery and prolong suffering. Inaccurate prognostication can lead to over- or under-estimation of outcomes and influence treatment recommendations.

OBJECTIVE

To evaluate the impact of evidence-based risk estimates on neurosurgeon treatment recommendations and prognostic beliefs in severe TBI.

METHODS

In a survey-based randomized experiment, a total of 139 neurosurgeons were presented with two hypothetical patient with severe TBI and subdural hematoma; the intervention group received additional evidence-based risk estimates for each patient. The main outcome was neurosurgeon treatment recommendation of non-surgical management. Secondary outcomes included prediction of functional recovery at six months.

RESULTS

In the first patient scenario, 22% of neurosurgeons recommended non-surgical management and provision of evidence-based risk estimates increased the propensity to recommend non-surgical treatment (odds ratio [OR]: 2.81, 95% CI: 1.21-6.98; p = 0.02). Neurosurgeon prognostic beliefs of 6-month functional recovery were variable in both control (median 20%, IQR: 10%-40%) and intervention (30% IQR: 10%-50%) groups and neurosurgeons were less likely to recommend non-surgical management when they believed prognosis was favorable (odds ratio [OR] per percentage point increase in 6-month functional recovery: 0.97, 95% confidence interval [CI]: 0.95-0.99). The results for the second patient scenario were qualitatively similar.

CONCLUSIONS

Our findings show that the provision of evidence-based risk predictions can influence neurosurgeon treatment recommendations and prognostication, but the effect is modest and there remains large variability in neurosurgeon prognostication.

Prognosis in Moderate and Severe Traumatic Brain Injury: A Systematic Review of Contemporary Models and Validation Studies

Outcome prognostication in traumatic brain injury (TBI) is important but challenging due to heterogeneity of the disease. The aim of this systematic review is to present the current state-of-the-art on prognostic models for outcome after moderate and severe TBI and evidence on their validity. We searched for studies reporting on the development, validation or extension of prognostic models for functional outcome after TBI with Glasgow Coma Scale (GCS) ≤12 published between 2006-2018. Studies with patients age ≥14 years and evaluating a multi-variable prognostic model based on admission characteristics were included. Model discrimination was expressed with the area under the receiver operating characteristic curve (AUC), and model calibration with calibration slope and intercept. We included 58 studies describing 67 different prognostic models, comprising the development of 42 models, 149 external validations of 31 models, and 12 model extensions. The most common predictors were GCS (motor) score (n = 55), age (n = 54), and pupillary reactivity (n = 48). Model discrimination varied substantially between studies. The International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) models were developed on the largest cohorts (8509 and 10,008 patients, respectively) and were most often externally validated (n = 91), yielding AUCs ranging between 0.65-0.90 and 0.66-1.00, respectively. Model calibration was reported with a calibration intercept and slope for seven models in 53 validations, and was highly variable. In conclusion, the discriminatory validity of the IMPACT and CRASH prognostic models is supported across a range of settings. The variation in calibration, reflecting heterogeneity in reliability of predictions, motivates continuous validation and updating if clinical implementation is pursued.