Interobserver variability in the characterization of giant intracranial aneurysms with special emphasis on aneurysm diameter and shape

Assessing accuracy and consistency in intracranial aneurysm sizing: human expertise vs. artificial intelligence

Intracranial aneurysms (IAs) are a common vascular pathology and are associated with a risk of rupture, which is often fatal. Aneurysm growth of more than 1 mm is considered a surrogate of rupture risk, therefore, this study presents a comprehensive analysis of intracranial aneurysm measurements utilizing a dataset comprising 358 IA from 248 computed tomography angiography (CTA) scans measured by four junior raters and one senior rater. The study explores the variability in sizing assessments by employing both human raters and an Artificial Intelligence (AI) system. Our findings reveal substantial inter- and intra-rater variability among junior raters, contrasting with the lower intra-rater variability observed in the senior rater. Standard deviations of all raters were above the threshold for IA growth (1 mm). Additionally, the study identifies a systemic bias, indicating a tendency for human experts to measure aneurysms smaller than the AI system. Our findings emphasize the challenges in human assessment while also showcasing the capacity of AI technology to improve the precision and reliability of intracranial aneurysm assessments, especially beneficial for junior raters. The potential of AI was particularly evident in the task of monitoring IA at various intervals, where the AI-based approach surpassed junior raters and achieved performance comparable to senior raters.

Reliability and accuracy assessment of morphometric measurements obtained with software for three-dimensional reconstruction of brain aneurysms relative to cerebral angiography measures

OBJECTIVE

To analyze the reliability and accuracy of morphological measurements of software employed to three-dimensionally reconstruct aneurysms and vessels (VMTKlab, version 1.6.1,) with computed tomography angiography (CTA) as the source of images. Agreement with measurements from three-dimensional digital subtraction angiography (3 D-DSA) was evaluated.

METHODS

We evaluated 40 patients presenting with aneurysmal subarachnoid hemorrhage (aSAH). We analyzed four main variables of the aneurysm morphology: absolute height (size), neck (maximum neck width), perpendicular height, and maximum width. The CTA images were uploaded to the software and then segmented to reconstruct the aneurysm. This new method was compared to the current gold standard-3D reconstruction of pretreatment cerebral angiography. We used intraclass correlation coefficient (ICC) and Bland-Altman plot analyses to evaluate the agreement between these methods.

RESULTS

The ICCs obtained for absolute height, neck, perpendicular height, and maximum width were 0.85, 0.57, 0.85, and 0.89, respectively. This implied good agreement except for the neck of the aneurysm (moderate agreement). Bland-Altman plots are presented for the four indexes. The average of the differences was not significant in terms of absolute height, perpendicular height, and maximum width indicating good agreement. However, it was significant for the neck of the aneurysm.

CONCLUSIONS

We report good agreement between the values generated using VMTKlab and cerebral angiography for three of the four main variables. Discrepancies in neck diameter are not surprising and its underestimation with a traditional delineation from cerebral angiography has been reported before.

Giant intracranial aneurysms of the posterior circulation and their relation to the brainstem: analysis of risk factors for neurological deficits

OBJECTIVE

Giant posterior circulation aneurysms (GPCirAs) usually cause substantial mass effect on the brainstem, which may lead to neurological deficits. So far, there has been no systematic investigation of factors associated with such deficits in GPCirA. The authors aim to examine the risk factors for cranial nerve deficit (CND), motor deficit, and disability in patients with GPCirA.

METHODS

Using MR images obtained in 30 patients with unruptured GPCirA, the authors examined GPCirA volume, presence of hydrocephalus or partial thrombosis (PT) of the aneurysm, and the degree of brainstem displacement measured by the distance between the McRae line and the tip of the GPCirA (∆MT). They evaluated associations between these factors and neurological deficits.

RESULTS

Thirty GPCirAs in 30 patients were included. The prevalence of CNDs was 50%. Patients with CNDs significantly differed from those without CNDs in terms of age (mean 51.0 years [SD 15.0 years] vs 69.0 years [SD 21.0 years], p = 0.01) and in ∆MT (median 50.7 mm [IQR 39.2-53.9 mm] vs 39.0 mm [IQR 32.3-45.9 mm], p = 0.02). The prevalence of motor deficits was 33.3%. Patients with motor deficits showed a larger ∆MT (median 50.5 mm [IQR 40.8-54.6 mm]) compared with those without (∆MT: median 39.1 mm [IQR 32.8-50.5 mm], p = 0.04). GPCirA volume was larger in patients with poor modified Rankin Scale (mRS) scores (median 14.9 cm3 [IQR 8.6-18.7 cm3]) than in those with mRS scores of 0-2 (median 6.8 cm3 [IQR 4.4-11.7 cm3], p = 0.03). After adjusting for patient age and the occurrence of hydrocephalus or PT, the authors found that higher degrees of disability were significantly associated with aneurysm volume (OR 1.13, 95% CI 1.0-1.3; p = 0.04), but not with ∆MT. The occurrence of CND or motor deficit was not associated with any of the examined variables. There was no correlation between GPCirA volume and ∆MT (rs = 0.01, p = 0.96). The prevalence of neurological deficits did not differ between GPCirA at the basilar apex, the basilar trunk, the vertebrobasilar junction, or the vertebral artery.

CONCLUSIONS

In this study, the neurological condition of the patients was associated only with GPCirA volume and not with the degree of brainstem displacement, the occurrence of PT or hydrocephalus, or the exact location of the GPCirA. These findings highlight the clinical relevance of GPCirA volume and suggest that factors such as brainstem displacement or PT should play less of a role when finding arguments for or against treatment of GPCirA.Clinical trial registration no.: NCT02066493 (clinicaltrials.gov).

Clinical implications and radiographic characteristics of the relation between giant intracranial aneurysms of the posterior circulation and the brainstem

BACKGROUND

Giant intracranial aneurysms of the posterior circulation (GPCirA) are rare entities compressing the brainstem and adjacent structures. Previous evidence has shown that the amount of brainstem shift away from the cranial base is not associated with neurological deficits. This raises the question whether other factors may be associated with neurological deficits.

METHODS

All data were extracted from the Giant Intracranial Aneurysm Registry, an international multicenter prospective study on giant intracranial aneurysms. We grouped GPCirA according to the mass effect on the brainstem (lateral versus medial). Brainstem compression was evaluated with two indices: (a) brainstem compression ratio (BCR) or diameter of the compressed brainstem to the assumed normal diameter of the brainstem and (b) aneurysm to brainstem ratio (ABR) or diameter of the aneurysm to the diameter of the compressed brainstem. We examined associations between neurological deficits and GPCirA characteristics using binary regression analysis.

RESULTS

Twenty-eight GPCirA were included. Twenty GPCirA showed medial (71.4%) and 8 lateral compression of the brainstem (28.6%). Baseline characteristics did not differ between the groups for patient age, aneurysm diameter, aneurysm volume, modified Rankin Scale (mRS), motor deficit (MD), or cranial nerve deficits (CND). Mean BCR was 53.0 in the medial and 54.0 in the lateral group (p = 0.92). The mean ABR was 2.9 in the medial and 2.3 in the lateral group (p = 0.96). In the entire cohort, neither BCR nor ABR nor GPCirA volumes were associated with the occurrence of CND or MD. In contrast, disability (mRS) was significantly associated with ABR (OR 1.94 (95% CI 1.01-3.70; p = 0.045) and GPCirA volumes (OR 1.21 (95% CI 1.01-1.44); p = 0.035), but not with BCR.

CONCLUSION

In this cohort of patients with GPCirA, neither the degree of lateral projection nor the amount of brainstem compression predicted neurological deficits. Disability was associated only with aneurysm volume. When designing treatment strategies for GPCirA, aneurysm laterality or the amount of brainstem compression should be viewed as less relevant while the high risk of rupture of such giant lesions should be emphasized.

TRIAL REGISTRATION

The registry is listed at clinicaltrials.gov under the registration no. NCT02066493.

Temporary Worsening of Perianeurysmal Edema Following Clipping of a Partially Thrombosed Giant Pericallosal Artery Aneurysm

We present a case of a partially thrombosed giant aneurysm of the pericallosal artery that experienced a temporary worsening of perianeurysmal edema after clipping. A 66-year-old man presented with progressive weakness of the right leg. Imaging studies revealed a partially thrombosed giant aneurysm at the bifurcation of the left pericallosal artery with concomitant perianeurysmal edema. The aneurysm was thought responsible for the symptoms of the patient, and he agreed to undergo clipping surgery. The aneurysmal neck was clipped using two long aneurysm clips without dissecting the aneurysmal dome. The patient woke up from anesthesia with the right-sided hemiparesis, which progressed to hemiplegia by 12 h after surgery. Brain computed tomography revealed worsening of the perianeurysmal edema. Fortunately, his symptoms resolved completely by 2 weeks after surgery with conservative management. Perianeurysmal edema is frequently observed in patients with partially thrombosed giant aneurysms. Although worsening of perianeurysmal edema is a relatively common complication of endovascular procedures, its occurrence after clipping has rarely been reported. The worsening may have been induced by release of inflammatory cytokines from aneurysm wall, which was rendered ischemic due to obliteration of the vasa vasorum by the clips. Perianeurysmal edema may develop or worsen after clipping of a partially thrombosed giant aneurysm if the interface between the aneurysm and surrounding brain is not fully dissected.

Computer-Assisted Three-Dimensional Morphology Evaluation of Intracranial Aneurysms

OBJECTIVE

Precise morphologic evaluation is important for intracranial aneurysm (IA) management. At present, clinicians manually measure the IA size and neck diameter on 2-dimensional (2D) digital subtraction angiographic (DSA) images and categorize the IA shape as regular or irregular on 3-dimensional (3D)-DSA images, which could result in inconsistency and bias. We investigated whether a computer-assisted 3D analytical approach could improve IA morphology assessment.

METHODS

Five neurointerventionists evaluated the size, neck diameter, and shape of 39 IAs using current and computer-assisted 3D approaches. In the computer-assisted 3D approach, the size, neck diameter, and undulation index (UI, a shape irregularity metric) were extracted using semiautomated reconstruction of aneurysm geometry using 3D-DSA, followed by IA neck identification and computerized geometry assessment.

RESULTS

The size and neck diameter measured using the manual 2D approach were smaller than computer-assisted 3D measurements by 2.01 mm (P < 0.001) and 1.85 mm (P < 0.001), respectively. Applying the definitions of small IAs (<7 mm) and narrow-necked IAs (<4 mm) from the reported data, interrater variation in manual 2D measurements resulted in inconsistent classification of the size of 14 IAs and the necks of 19 IAs. Visual inspection resulted in an inconsistent shape classification for 23 IAs among the raters. Greater consistency was achieved using the computer-assisted 3D approach for size (intraclass correlation coefficient [ICC], 1.00), neck measurements (ICC, 0.96), and shape quantification (UI; ICC, 0.94).

CONCLUSIONS

Computer-assisted 3D morphology analysis can improve accuracy and consistency in measurements compared with manual 2D measurements. It can also more reliably quantify shape irregularity using the UI. Future application of computer-assisted analysis tools could help clinicians standardize morphology evaluations, leading to more consistent IA evaluations.

Decision-making and neurosurgeons' agreement in the management of aneurysmal subarachnoid haemorrhage based on computed tomography angiography

BACKGROUND

To determine the neurosurgeon's agreement in aneurysmal subarachnoid haemorrhage (aSAH) management with special emphasis on the rater's level of experience. A secondary aim was to analyse potential aneurysm variables associated with the therapeutic recommendation.

METHOD

Basic clinical information and admission computed tomography angiography (CTA) images of 30 consecutive aSAH patients were provided. Twelve neurosurgeons independently evaluated aneurysm characteristics and gave recommendations regarding the emergency management and aneurysm occlusion therapy. Inter-rater variability and predictors of treatment recommendation were evaluated.

RESULTS

There was an overall moderate agreement in treatment decision [κ = 0.43; 95% confidence interval ((CI), 0.387-0.474] with moderate agreement for surgical (κ = 0.43; 95% CI, 0.386-0.479) and endovascular treatment recommendation (κ = 0.45; 95% CI, 0.398-0.49). Agreement on detailed treatment recommendations including clip, coil, bypass, stent, flow diverter and ventriculostomy was low to moderate. Inter-rater agreement did not significantly differ between residents and consultants. Middle cerebral artery (MCA) aneurysm location was a positive predictor of surgical treatment [odds ratio (OR), 49.57; 95% CI, 10.416-235.865; p < 0.001], while patients aged >65 years (OR, 0.12; 95% CI, 0.03-0.0434; p = 0.001), fusiform aneurysm type (OR, 0.18; 95% CI, 0.044-0.747; p = 0.018) and intracerebral haematoma (ICA) aneurysm location (OR, 0.24; 95% CI, 0.088-0.643; p = 0.005) were associated with a recommendation for endovascular treatment.

CONCLUSIONS

Agreement on aSAH management varies considerably across neurosurgeons, while therapeutic decision-making is challenging on an individual patient level. However, patients aged >65 years, fusiform aneurysm shape and ICA location were associated with endovascular treatment recommendation, while MCA aneurysm location remains a surgical domain in the opinion of neurosurgeons without formal endovascular training.

Interrater Agreement in the Radiologic Characterization of Ruptured Intracranial Aneurysms Based on Computed Tomography Angiography

OBJECTIVE

To determine interrater agreement in the initial radiologic characterization of ruptured intracranial aneurysms based on computed tomography angiography (CTA) with special emphasis on the rater's level of experience.

METHODS

One junior and one senior rater of 5 high-volume neurovascular tertiary centers evaluated anonymized CTA images of 30 consecutive patients with aneurysmal subarachnoid hemorrhage. Each rater described location, side, size, and morphology in a standardized manner. Interrater variability was analyzed using intraclass correlation and Fleiss' kappa analysis.

RESULTS

There was a high level of agreement for location (κ = 0.76, 95% confidence interval [CI] 0.74-0.79), side (κ = 0.95, CI 0.91-0.99), maximum diameter (intraclass correlation coefficient [ICC] 0.81, CI 0.70-0.90), and dome (ICC 0.78, CI 0.66-0.88) of intracranial aneurysms. In contrast, a lower level of agreement was observed for aneurysms' neck diameter (ICC 0.39, CI 0.28-0.58), the presence of multiple aneurysms (κ = 0.35, CI 0.30-0.40), and aneurysm morphology (blister κ = 0.11, CI -0.05 to 0.07; fusiform κ = 0.54, CI 0.48-0.60; multilobular, κ = 0.39 CI 0.33-0.45). The interrater agreement in the senior rater group was greater than in the junior rater group.

CONCLUSIONS

Interrater agreement confirms the benefit of CTA as initial diagnostic imaging in ruptured intracranial aneurysms but not for aneurysm morphology and presence of multiple aneurysms. A trend towards greater interrater agreement between more experienced raters was noticed.

Intraobserver and interobserver variability in CT angiography and MR angiography measurements of the size of cerebral aneurysms

PURPOSE

Accurate and reliable measurement of aneurysm size is important for treatment planning. The purpose of this study was to determine intraobserver and interobserver variability of CTA and MRA for measurement of the size of cerebral aneurysms.

METHODS

Thirty patients with 33 unruptured cerebral aneurysms (saccular, >3 mm in their maximal dimension, with no daughter sacs or lobulations) who underwent 256-row multislice CTA, 3-D TOF MRA at 3.0T, and 3D rotational angiography (3DRA) were retrospectively analyzed. Three independent observers measured the neck, height, and width of the aneurysms using the CTA and MRA images. Intraobserver and interobserver variability of CTA and MRA measurements was evaluated using the standardized difference and intraclass correlation coefficient, with 3DRA measurements as the reference standard. In addition, the mean values of the measurements using CTA and MRA were compared with those using 3DRA.

RESULTS

The overall intraobserver and interobserver standardized differences in CTA/MRA were 12.83-15.92%/13.48-17.45% and 14.08-17.00%/12.08-17.67%, respectively. The overall intraobserver and interobserver intraclass correlation coefficients of CTA/MRA were 0.88-0.98/0.84-0.96 and 0.86-0.98/0.85-0.95, respectively. Compared to the height and width measurements, measurements of the neck dimensions showed higher intraobserver and interobserver variability. The sizes of the cerebral aneurysms measured by CTA and MRA were 1.13-9.26 and 5.20-9.67% larger than those measured by 3DRA, respectively; however, these differences were not statistically significant.

CONCLUSION

There were no noticeable differences between intraobserver and interobserver variability for both CTA- and MRA-based measurements of the size of cerebral aneurysms.