Quantitative Evaluation of Performance in Interventional Neuroradiology: An Integrated Curriculum Featuring Theoretical and Practical Challenges

Evaluation of intracranial stenting in a simulated training and assessment environment for neuroendovascular procedures

Purpose

Given the inherent complexity of neurointerventional procedures and the associated risks of ionizing radiation exposure, it is crucial to prioritize ongoing training and improve safety protocols. The aim of this study is to assess a training and evaluation in-vitro environment using a vascular model of M1 stenosis, within a clinical angiography suite, without relying on animal models or X-ray radiation.

Materials and methods

Using a transparent model replicating M1 stenosis, we conducted intracranial stenting procedures with four different setups (Gateway & Wingspan, Gateway & Enterprise, Neurospeed & Acclino, and Pharos Vitesse). A video camera was integrated with the angiography system's monitor for real-time visualization, while a foot switch was employed to simulate live fluoroscopy. Three neuroradiologists with varying levels of expertise performed each procedure for three times. The total duration of fluoroscopy as well as the time from passing the stenosis with the wire to completion of the procedure were recorded using a dedicated software designed for this experimental setup.

Results

Compared to the Gateway & Wingspan procedure, the total fluoroscopy time reduced significantly with the Gateway & Enterprise, Neurospeed & Acclino, and Pharos Vitesse procedures by 51.56 s, 111.33 s, and 144.89 s, respectively (p < 0.001). Additionally, physicians with under 2 years and over 5 years of experience reduced FT by 62.83 s and 106.42 s, respectively, (p < 0.001), compared to a novice physician. Similar trends were noted for the time of wire distal to stenosis, with significant reductions for Neurospeed & Acclino and Pharos Vitesse compared to both Gateway & Wingspan as well as Gateway & Enterprise (all p < 0.001).

Conclusion

Procedures requiring wire exchange maneuvers exhibited nearly twice the fluoroscopy time in comparison to balloon-mounted stenting or stent-placement via PTA balloon catheters. The more experienced neuroradiologist demonstrated significantly quicker performance in line with expectations in a real-life clinical setting, when compared to the less experienced interventionalist. This in-vitro setup allowed the evaluation of alternative technical approaches and differences in experience of operators without the use of animal models or X-ray. The setup combines advantages of simulators and silicone vessel models in a realistic working environment.

Can training on ex-vivo models increase neurointerventionalists’ subjective self-confidence in the operating room?

In a changing learning environment where young neurointerventionalists spend less time in the operating room, computer simulators have been established as a new training model. Our aim was the comparison of silicone models and computer simulators, and the evaluation of their influence on subjective self-confidence of operators. Pre- and postquestionnaires of 27 participants and 9 tutors were evaluated after the participation in a three-days interventional stroke course using silicone models and computer simulators. Training on computer simulators was considered as more realistic and important before patient contact than training on silicone models. Participants rated their own abilities as significantly better after participation in the course and felt significantly better prepared for patient care. Training on computer simulators can increase the subjective self-confidence of trainees. We suggest a stepwise training program, comprising both ex-vivo and the porcine in-vivo model, finished by conventional operating room teaching, to prepare neuroradiologists for optimal patient care when performing interventions.

Use of Endovascular Simulator in Training of Neurosurgery Residents - A Review and Single Institution Experience

Simulators for surgical procedures and interventions have undergone significant technological advancement in the past decade and are becoming more commonplace in medical training. Neurosurgery residents across multiple training levels underwent performance evaluation using a neuro-interventional simulator, employing a variety of metrics for assessment. We identified seven core metrics used in the evaluation of neurosurgery residents performing simulated mechanical thrombectomies. Additionally, a systematic PubMed search for studies related to Neurointerventional Radiology training via simulation was performed. The purpose of this study is to examine the validity and benefits of training with these simulation devices and compare our institution's experience. Additionally, an exploration of their applicability to neurosurgery resident training is discussed.

Confidence of treatment decision and perceived risk of procedure-related neurological complications in the management of unruptured intracranial aneurysms

BACKGROUND AND PURPOSE

To evaluate factors influencing the confidence of management recommendation for unruptured intracranial aneurysms (UIAs) and to assess the ability of neurointerventionalists to predict procedure-related neurological complications compared with a 3-point risk score.

MATERIALS AND METHODS

Twenty-eight neurointerventionalists were asked to evaluate digital subtraction angiographies examinations of patients with UIAs by determining the best management approach, their level of confidence in their management recommendation, and estimating the risk of procedure-related neurological complications. Knowledge and experience in interventional neuroradiology (INR) of each participant were assessed.

RESULTS

Reliability was moderate regarding any treatment recommendation (ICC=0.49) and low regarding the estimation of risk of complications (ICC=0.38). The recommendation of clipping was less likely with more experience in INR (OR=0.6) and more likely with increasing knowledge (OR=1.7). Odds of recommending WEB device were lower with more experience in INR (OR=0.6), higher in patients with multiple aneurysms (OR=3.6) and increasing neck width (OR=2.7). The recommendation of stent-assisted coiling was more likely with increasing neck width (OR=2.4) and when cerebral ischemic comorbidities were present (OR=2.9). The participants were significantly worse than the risk score (mean area under the curve of 0.53) and not better than random guess in predicting complications. Neither knowledge nor experience in INR was significantly associated with the participants' ability to predict neurological complications.

CONCLUSIONS

Our study shows a moderate interrater reliability of treatment recommendations of UIAs. Confidence in treatment recommendation varied significantly according to recommended treatments. Overall performance in predicting neurological complications was worse than the risk score and not better than random guess.

Factors Influencing Confidence in Diagnostic Ratings and Retreatment Recommendations in Coiled Aneurysms

BACKGROUND AND PURPOSE

Angiographic occlusion and retreatment of coiled aneurysms are commonly used as surrogate end points in clinical trials. We aimed to evaluate the influence of aneurysm, patient, and rater characteristics on the confidence of visual evaluation of aneurysm coiling and retreatment decisions.

MATERIALS AND METHODS

Twenty-six participants of the Advanced Course in Endovascular Interventional Neuroradiology of the European Society of Neuroradiology were asked to evaluate digital subtraction angiography examinations of patients who had undergone endovascular coiling, by determining the grade of aneurysm occlusion, the change between immediate postprocedural and follow-up angiograms, their level of confidence, the technical difficulty of retreatment, and the best therapeutic approach. The experience, knowledge, and skills of each participant were assessed. The influence of rater and case characteristics on indicated confidence in diagnostic ratings and retreatment recommendations was analyzed.

RESULTS

Interrater reliability was moderate regarding the assessment of aneurysm occlusion grade (intraclass correlation coefficient = 0.581) and substantial regarding change (intraclass correlation coefficient = 0.776). Overall confidence in the diagnostic rating was high (median, "very certain"). Confidence was statistically significantly higher in cases that were generally rated as "worse." The odds of recommending retreatment were significantly higher in cases that were generally rated with higher mean confidence.

CONCLUSIONS

Although overall confidence in the diagnostic rating was high, our study confirms the suboptimal interrater reliability of visual assessment of aneurysm occlusion as well as retreatment recommendations, rendering both questionable as primary outcome measures. Besides recurrence status, recommendation of retreatment is significantly influenced by patient age, aneurysm neck width, and characteristics of the therapist.