Predicting Outcomes for Cerebral Aneurysms Treated with Flow Diversion: A Comparison Between 4 Grading Scales

Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study

Background: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons.

Objectives: The recently emerged automated machine learning (AutoML) has shown promise in making ML more accessible to non-computer experts. We aimed to evaluate the feasibility of applying AutoML to develop the ML models for treatment outcome prediction.

Methods: The patients with aneurysms treated by endovascular treatment were prospectively recruited from 2016 to 2020. Treatment was considered successful if angiographic complete occlusion was achieved at follow-up. A statistical prediction model was developed using multivariate logistic regression. In addition, two ML models were developed. One was developed manually and the other was developed by AutoML. Three models were compared based on their area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC).

Results: The aneurysm size, stent-assisted coiling (SAC), and posterior circulation were the three significant and independent variables associated with treatment outcome. The statistical model showed an AUPRC of 0.432 and AUROC of 0.745. The conventional manually trained ML model showed an improved AUPRC of 0.545 and AUROC of 0.781. The AutoML derived ML model showed the best performance with AUPRC of 0.632 and AUROC of 0.832, significantly better than the other two models.

Conclusions: This study demonstrated the feasibility of using AutoML to develop a high-quality ML model, which may outperform the statistical model and manually derived ML models. AutoML could be a useful tool that makes ML more accessible to the clinical researchers.

A novel score for evaluating cerebral aneurysms treated with flow diversion: 4F-flow diversion predictive score

Background and purpose

Although grading scales for angiography outcomes following cerebral aneurysm treatment with flow diversion have been published, physicians have not widely adopted these scales in practice. The aim of this study is to propose and validate a novel Flow diversion Predictive Score (4F-FPS) grading scale based on previously established scales that is simple and reliable.

Methods

We retrospectively analyzed consecutive patients who underwent endovascular treatment for cerebral aneurysms with flow diversion between January 2014 and September 2019. The included patients were randomly divided into the derivation and validation group in a 70/30 ratio, respectively. Aneurysms were classified as incomplete or complete occlusion based on final angiography outcomes. 4F-FPS was derived to predict aneurysm occlusion from multivariate logistic regression analyses in the derivation group and validated with previously published grading scales in the validation group.

Results

Overall, 662 patients [mean age, 53.8 years; 72.5% (480/662) female] with 662 aneurysms treated with the Pipeline^TM flow diverter were included [69.9% (463/662) derivation group, 30.1% (199/662) validation group]. The incidence of aneurysm occlusion was 82.7%. 4F-FPS demonstrated significant discrimination in 10-fold cross validation [mean receiver operating characteristic (ROC) area, 0.862 ± 0.055] and calibration (Cox & Snell R ², 0.251; Nagelkerke R ², 0.413) in the derivation group. The ROC area of 4F-FPS score in both the derivation and validation groups is the largest compared with previously published grading scales/scores (p < 0.05), which shows better sensitivity and specificity. The 4F-FPS score showed excellent prediction, discrimination, and calibration properties.

Conclusion

The 4F-FPS score is a simple and reliable tool to predict angiography outcome after flow diversion treatment. If widely adopted, it may provide a common language to be used in future reporting of flow diversion results for clinical trials and daily practice.

Clinical trial registration

http://www.clinicaltrials.gov. Unique identifier: NCT03831672.