Development and validation of a predictive model for the prognosis in aneurysmal subarachnoid hemorrhage

Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review update

BACKGROUND

A systematic review of clinical prediction models for aneurysmal subarachnoid hemorrhage (aSAH) reported in 2011 noted that clinical prediction models for aSAH were developed using poor methods and were not externally validated. This study aimed to update the above review to guide the future development of predictive models in aSAH.

METHODS

We systematically searched Embase and MEDLINE databases (January 2010 to February 2022) for articles that reported the development of a clinical prediction model to predict functional outcomes in aSAH. Our reviews are based on the items included in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (PRISMA) checklist, and on data abstracted from each study in accord with the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) 2014 checklist. Bias and applicability were assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST).

RESULTS

We reviewed data on 30 466 patients contributing to 29 prediction models abstracted from 22 studies identified from an initial search of 7858 studies. Most models were developed using logistic regression (n=20) or machine learning (n=9) with prognostic variables selected through a range of methods. Age (n=13), World Federation of Neurological Surgeons (WFNS) grade (n=11), hypertension (n=6), aneurysm size (n=5), Fisher grade (n=12), Hunt and Hess score (n=5), and Glasgow Coma Scale (n=8) were the variables most frequently included in the reported models. External validation was performed in only four studies. All but one model had a high or unclear risk of bias due to poor performance or lack of validation.

CONCLUSION

Externally validated models for the prediction of functional outcome in aSAH patients have now become available. However, most of them still have a high risk of bias.

A 90-Day Prognostic Model Based on the Early Brain Injury Indicators after Aneurysmal Subarachnoid Hemorrhage: the TAPS Score

This study aimed to establish a new scoring model based on the early brain injury (EBI) indicators to predict the 90-day functional outcomes in patients with aneurysmal subarachnoid hemorrhage (aSAH). We retrospectively enrolled 825 patients and prospectively enrolled 108 patients with aSAH who underwent surgical clipping or endovascular coiling (derivation cohort = 640; validation cohort = 185; prospective cohort = 108) in our institute. We established a logistic regression model based on independent risk factors associated with 90-day unfavorable outcomes. The discrimination of the prognostic model was assessed by the area under the curve in a receiver operating characteristic curve analysis. The Hosmer-Lemeshow goodness-of-fit test and a calibration plot were used to evaluate the calibration of the prediction model. The developed scoring model named "TAPS" (total score, 0-7 points) included the following admission variables: age > 55 years old, WFNS grade of 4-5, mFS grade of 3-4, Graeb score of 5-12, white blood cell count > 11.28 × 109/L, and surgical clipping. The model showed good discrimination with the area under the curve in the derivation, validation, and prospective cohorts which were 0.816 (p < 0.001, 95%CI = 0.77-0.86), 0.810 (p < 0.001, 95%CI = 0.73-0.90), and 0.803 (p < 0.001, 95%CI = 0.70-0.91), respectively. The model also demonstrated good calibration (Hosmer-Lemeshow goodness-of-fit test: X2 = 1.75, df = 8, p = 0.988). Compared with other predictive models, TAPS is an easy handle tool for predicting the 90-day unfavorable outcomes of aSAH patients, which can help clinicians better understand the concept of EBI and quickly identify those patients at risk of poor prognosis, providing more positive treatment strategies. Trial registration: NCT04785976. Registered 5 March 2021-retrospectively registered, http://www.clinicaltrials.gov .

Utility of serum amyloid A as a potential prognostic biomarker of aneurysmal subarachnoid hemorrhage

Objectives

Inflammation plays a vital role in the aneurysmal subarachnoid hemorrhage (aSAH), while serum amyloid A (SAA) has been identified as an inflammatory biomarker. The present study aimed to elucidate the relationship between SAA concentrations and prognosis in aSAH.

Methods

From prospective analyses of patients admitted to our department between March 2016 and August 2022, aSAH patients with complete medical records were evaluated. Meanwhile, the healthy control group consisted of the age and sex matched individuals who came to our hospital for healthy examination between March 2018 and August 2022. SAA level was measured by enzyme-linked immunosorbent assay kit (Invitrogen Corp). The Glasgow Outcome Scale (GOS) was used to classify patients into good (GOS score of 4 or 5) and poor (GOS score of 1, 2, or 3) outcome.

Results

456 patients were enrolled in the study, thereinto, 200 (43.86%) patients had a poor prognosis at the 3-months follow-up. Indeed, the SAA of poor outcome group were significantly increased compared to good outcome group and healthy control group [36.44 (32.23-41.00) vs. 28.99 (14.67-34.12) and 5.64 (3.43-7.45), P < 0.001]. In multivariate analyses, SAA served for independently predicting the poor outcome after aICH at 3 months [OR:1.129 (95% CI, 1.081-1.177), P < 0.001]. After adjusting the underlying confounding factors, the odds ratio (OR) of depression after aSAH was 2.247 (95% CI: 1.095-4.604, P = 0.021) for the highest tertile of SAA relative to the lowest tertile. With an AUC of 0.807 (95% CI, 0.623-0.747), SAA demonstrated an obviously better discriminatory ability relative to CRP, WBC, and IL-6. SAA as an indicator for predicting poor outcome after aSAH had an optimal cut-off value of 30.28, and the sensitivity and specificity were 61.9 and 78.7%, respectively.

Conclusions

Elevated level of SAA was associated with poor outcome at 3 months, suggesting that SAA might be a useful inflammatory markers to predict prognosis after aSAH.

Improving the models for prognosis of aneurysmal subarachnoid hemorrhage with the neutrophil-to-albumin ratio

Objective

Many peripheral inflammatory markers were reported to be associated with the prognosis of aneurysmal subarachnoid hemorrhage (aSAH). We aimed to identify the most promising inflammatory factor that can improve existing predictive models.

Methods

The study was based on data from a 10 year retrospective cohort study at Sichuan University West China Hospital. We selected the well-known SAFIRE and Subarachnoid Hemorrhage International Trialists' (SAHIT) models as the basic models. We compared the performance of the models after including the inflammatory markers and that of the original models. The developed models were internally and temporally validated.

Results

A total of 3,173 patients were included in this study, divided into the derivation cohort (n = 2,525) and the validation cohort (n = 648). Most inflammatory markers could improve the SAH model for mortality prediction in patients with aSAH, and the neutrophil-to-albumin ratio (NAR) performed best among all the included inflammatory markers. By incorporating NAR, the modified SAFIRE and SAHIT models improved the area under the receiver operator characteristics curve (SAFIRE+NAR vs. SAFIRE: 0.794 vs. 0.778, p = 0.012; SAHIT+NAR vs. SAHIT: 0.831 vs. 0.819, p = 0.016) and categorical net reclassification improvement (SAFIRE+NAR: 0.0727, p = 0.002; SAHIT+NAR: 0.0810, p < 0.001).

Conclusion

This study illustrated that among the inflammatory markers associated with aSAH prognosis, NAR could improve the SAFIRE and SAHIT models for 3 month mortality of aSAH.

Potential role of serum hypoxia-inducible factor 1alpha as a biomarker of delayed cerebral ischemia and poor clinical outcome after human aneurysmal subarachnoid hemorrhage: A prospective, longitudinal, multicenter, and observational study

Objective

Hypoxia-inducible factor 1alpha (HIF-1α) functions as a crucial transcriptional mediator in hypoxic and ischemic brain response. We endeavored to assess the prognostic significance of serum HIF-1α in human aneurysmal subarachnoid hemorrhage (aSAH).

Methods

In this prospective, longitudinal, multicenter, and observational study of 257 patients with aSAH and 100 healthy controls, serum HIF-1α levels were quantified. Univariate analyses, followed by multivariate analyses, were performed to discern the relationship between serum HIF-1α levels and severity and delayed cerebral ischemia (DCI) plus poststroke 6-month poor outcome [extended Glasgow outcome scale (GOSE) scores of 1-4]. Predictive efficiency was determined under the receiver operating characteristic (ROC) curve.

Results

There were significantly increased serum HIF-lα levels after aSAH, in comparison to controls (median, 288.0 vs. 102.6 pg/ml; P < 0.001). Serum HIF-lα levels were independently correlated with Hunt-Hess scores [β, 78.376; 95% confidence interval (CI): 56.446-100.305; P = 0.001] and modified Fisher scores (β, 52.037; 95% CI: 23.461-80.614; P = 0.002). Serum HIF-lα levels displayed significant efficiency for discriminating DCI risk [area under ROC curve (AUC), 0.751; 95% CI: 0.687-0.815; P < 0.001] and poor outcome (AUC, 0.791; 95% CI: 0.736-0.846; P < 0.001). Using the Youden method, serum HIF-1α levels >229.3 pg/ml predicted the development of DCI with 92.3% sensitivity and 48.4% specificity and serum HIF-1α levels >384.0 pg/ml differentiated the risk of a poor prognosis with 71.4% sensitivity and 81.1% specificity. Serum HIF-1α levels >229.3 pg/ml were independently predictive of DCI [odds ratio (OR), 3.061; 95% CI: 1.045-8.965; P = 0.041] and serum HIF-1α levels >384.0 pg/ml were independently associated with a poor outcome (OR, 2.907; 95% CI: 1.403-6.024; P = 0.004). The DCI predictive ability of their combination was significantly superior to those of Hunt-Hess scores (AUC, 0.800; 95% CI: 0.745-0.855; P = 0.039) and modified Fisher scores (AUC, 0.784; 95% CI: 0.726-0.843; P = 0.004). The prognostic predictive ability of their combination substantially exceeded those of Hunt-Hess scores (AUC, 0.839; 95% CI: 0.791-0.886; P < 0.001) and modified Fisher scores (AUC, 0.844; 95% CI: 0.799-0.890; P < 0.001).

Conclusion

Elevated serum HIF-lα levels after aSAH, in independent correlation with stroke severity, were independently associated with DCI and 6-month poor outcome, substantializing serum HIF-lα as a potential prognostic biomarker of aSAH.

Neutrophil to lymphocyte ratio predicting poor outcome after aneurysmal subarachnoid hemorrhage: A retrospective study and updated meta-analysis

Background

The relationship between neutrophil to lymphocyte ratio (NLR) and poor outcome of aneurysmal subarachnoid hemorrhage (aSAH) is controversial. We aim to evaluate the relationship between NLR on admission and the poor outcome after aSAH.

Method

Part I: Retrospective analysis of aSAH patients in our center. Baseline characteristics of patients were collected and compared. Multivariate analysis was used to evaluate parameters independently related to poor outcome. Receiver operating characteristic (ROC) curve analysis was used to determine the best cut-off value of NLR. Part II: Systematic review and meta-analysis of relevant literature. Related literature was selected through the database. The pooled odds ratio (OR) and corresponding 95% confidence interval (CI) were calculated to evaluate the correlation between NLR and outcome measures.

Results

Part I: A total of 240 patients with aSAH were enrolled, and 52 patients had a poor outcome. Patients with poor outcome at 3 months had a higher admission NLR, Hunt & Hess score, Barrow Neurological Institute (BNI) scale score, Subarachnoid Hemorrhage Early Brain Edema Score (SEBES), and proportion of hypertension history. After adjustment, NLR at admission remained an independent predictor of poor outcome in aSAH patients (OR 0.76, 95% CI 0.69-0.83; P < 0.001). The best cut-off value of NLR in ROC analysis is 12.03 (area under the curve 0.805, 95% CI 0.735 - 0.875; P < 0.001). Part II: A total of 16 literature were included. Pooled results showed that elevated NLR was significantly associated with poor outcome (OR 1.31, 95% CI 1.14-1.49; P < 0.0001) and delayed cerebral ischemia (DCI) occurrence (OR 1.32, 95% CI 1.11-1.56; P = 0.002). The results are more reliable in large sample sizes, low NLR cut-off value, multicenter, or prospective studies.

Conclusion

Elevated NLR is an independent predictor of poor outcome and DCI occurrence in aSAH.

The Prognostic Value of Neutrophil-to-Lymphocyte Ratio in Patients With Aneurysmal Subarachnoid Hemorrhage: A Systematic Review and Meta-Analysis of Observational Studies

The neutrophil–to-lymphocyte ratio (NLR), as an essential systemic inflammation factor, has been widely used as a prognostic indicator in various diseases, such as malignant tumors, cardiovascular disease, and intracranial hemorrhage. An increasing number of studies have believed that NLR is a valuable predictor of prognosis for patients with aneurysmal subarachnoid hemorrhage (aSAH). However, these results remain controversial. In the current study, we planned to carry out a systematic review and meta-analysis to investigate the association between NLR and poor outcome, and the occurrence of delayed cerebral ischemia (DCI). We carried out a comprehensive search for published literatures on PubMed, EMBASE, Cochrane Library, and Web of Science databases from inception to April 1, 2021. We conducted an assessment of all included studies based on the principles proposed in the Newcastle-Ottawa Quality Assessment Scale (NOS). Poor outcome and the occurrence of DCI were considered as the main outcome measure. We calculated the pooled odds ratio (OR) and corresponding 95% confidence interval (CI) to examine the strength of the association of NLR with poor outcome or the occurrence of DCI. We strictly selected a total of 10 studies comprising 4,989 patients. Nine studies reported the association between NLR and poor outcome, and five studies reported the association between NLR and the occurrence of DCI. The pooled results indicated higher NLR was significantly associated with both poorer outcomes (OR = 1.32, 95%CI 1.11–1.57; P = 0.002, I² = 87%), and the occurrence of DCI (OR = 1.72, 95%CI 1.22–2.41; P = 0.002, I² = 82%) in aSAH patients. The NLR is a valuable indicator of inflammation to independently predict poor outcome and occurrence of DCI after aSAH, where a higher NLR is significantly associated with poor outcomes and occurrence of DCI. These findings suggest that the NLR can help clinicians evaluate the prognosis and identify potentially severe patients early, which may contribute to better management and improve poor prognosis of aSAH patients.

An Externally Validated Dynamic Nomogram for Predicting Unfavorable Prognosis in Patients With Aneurysmal Subarachnoid Hemorrhage

Background: Aneurysmal subarachnoid hemorrhage (aSAH) leads to severe disability and functional dependence. However, no reliable method exists to predict the clinical prognosis after aSAH. Thus, this study aimed to develop a web-based dynamic nomogram to precisely evaluate the risk of poor outcomes in patients with aSAH.

Methods: Clinical patient data were retrospectively analyzed at two medical centers. One center with 126 patients was used to develop the model. Least absolute shrinkage and selection operator (LASSO) analysis was used to select the optimal variables. Multivariable logistic regression was applied to identify independent prognostic factors and construct a nomogram based on the selected variables. The C-index and Hosmer–Lemeshow p-value and Brier score was used to reflect the discrimination and calibration capacities of the model. Receiver operating characteristic curve and calibration curve (1,000 bootstrap resamples) were generated for internal validation, while another center with 84 patients was used to validate the model externally. Decision curve analysis (DCA) and clinical impact curves (CICs) were used to evaluate the clinical usefulness of the nomogram.

Results: Unfavorable prognosis was observed in 46 (37%) patients in the training cohort and 24 (29%) patients in the external validation cohort. The independent prognostic factors of the nomogram, including neutrophil-to-lymphocyte ratio (NLR) (p = 0.005), World Federation of Neurosurgical Societies (WFNS) grade (p = 0.002), and delayed cerebral ischemia (DCI) (p = 0.0003), were identified using LASSO and multivariable logistic regression. A dynamic nomogram (https://hu-ping.shinyapps.io/DynNomapp/) was developed. The nomogram model demonstrated excellent discrimination, with a bias-corrected C-index of 0.85, and calibration capacities (Hosmer–Lemeshow p-value, 0.412; Brier score, 0.12) in the training cohort. Application of the model to the external validation cohort yielded a C-index of 0.84 and a Brier score of 0.13. Both DCA and CIC showed a superior overall net benefit over the entire range of threshold probabilities.

Conclusion: This study identified that NLR on admission, WFNS grade, and DCI independently predicted unfavorable prognosis in patients with aSAH. These factors were used to develop a web-based dynamic nomogram application to calculate the precise probability of a poor patient outcome. This tool will benefit personalized treatment and patient management and help neurosurgeons make better clinical decisions.

Dynamic Detection of Delayed Cerebral Ischemia: A Study in 3 Centers

BACKGROUND AND PURPOSE

Delayed cerebral ischemia (DCI) after aneurysmal subarachnoid hemorrhage negatively impacts long-term recovery but is often detected too late to prevent damage. We aim to develop hourly risk scores using routinely collected clinical data to detect DCI.

METHODS

A DCI classification model was trained using vital sign measurements (heart rate, blood pressure, respiratory rate, and oxygen saturation) and demographics routinely collected for clinical care. Twenty-two time-varying physiological measures were computed including mean, SD, and cross-correlation of heart rate time series with each of the other vitals. Classification was achieved using an ensemble approach with L2-regularized logistic regression, random forest, and support vector machines models. Classifier performance was determined by area under the receiver operating characteristic curves and confusion matrices. Hourly DCI risk scores were generated as the posterior probability at time t using the Ensemble classifier on cohorts recruited at 2 external institutions (n=38 and 40).

RESULTS

Three hundred ten patients were included in the training model (median, 54 years old [interquartile range, 45-65]; 80.2% women, 28.4% Hunt and Hess scale 4-5, 38.7% Modified Fisher Scale 3-4); 101 (33%) developed DCI with a median onset day 6 (interquartile range, 5-8). Classification accuracy before DCI onset was 0.83 (interquartile range, 0.76-0.83) area under the receiver operating characteristic curve. Risk scores applied to external institution datasets correctly predicted 64% and 91% of DCI events as early as 12 hours before clinical detection, with 2.7 and 1.6 true alerts for every false alert.

CONCLUSIONS

An hourly risk score for DCI derived from routine vital signs may have the potential to alert clinicians to DCI, which could reduce neurological injury.