Automated software supported versus manual aorto-iliac diameter measurements in CT angiography of patients with abdominal aortic aneurysms: Assessment of inter- and intraobserver variation

Comparison of Three Workstations For Abdominal Aortic Aneurysm Sizing: Impact in Decision Making and Graft Selection

PURPOSE

The aim of this study was to assess the agreement of multiplanar reconstruction (MPR) and semiautomated central lumen line (CLL) analysis of abdominal aortic aneurysms (AAA), with 3 different software workstations (WS1, WS2, WS3) and 2 experienced practitioners as well as to analyze its eventual impact in graft selection.

MATERIALS AND METHODS

Twenty computed tomography (CT) angiography data sets were randomly chosen from a series of 100 consecutive studies. Measurements were performed twice by each reader, in random order, and included 8 parameters (5 diameters and 3 lengths). Each observer performed a complete set of 60 studies. Intra-observer and interobserver variability for every WS was assessed. Measurements were evaluated using Bland-Altman analysis, correlation coefficients (r), and concordance correlation coefficients (CCC [95% confidence interval (CI)]).

RESULTS

A high overall agreement between repeated measurements for both observers was obtained (r=0.989; CCC=0.988 [0.982-0.992] and r=0.998; CCC=0.996 [0.994-0.997], for observers 1 and 2, respectively). However, reproducibility for individual parameters was excellent for observer 2 and only moderate for observer 1. A high overall agreement was obtained for interobserver concordance (r=0.987; CCC=0.986 [0.982-0.989]). When analyzing for individual parameters, greatest interobserver differences were found at CLL measurement of the diameter of aortic neck (WS2) and bifurcation (WS1 and WS2) as well as iliac diameter in all 3 WS for both CLL and MPR. Similar differences were observed in paired comparison between WS when involving these parameters. Careful inspection of Bland-Altman charts revealed some cases of disagreement between WS and observers that would affect decision making on graft selection, changing the neck diameter to a different size, in 2 cases when measuring with WS1, and iliac diameter in 4 cases (2 of them with WS1 and 2 with WS2). Greatest discordance was observed regarding ipsilateral iliac length affecting 7 measurements that would lead to change the length of the selected limb graft (2 with WS1, 3 with WS2, and 2 with WS3).

CONCLUSIONS

Although a high agreement between different observers using different WS for AAA measurements is to be expected, small differences may lead to the selection of a different graft size. The use of a single software by experienced users, and double check by a different one, may be advisable.

CLINICAL IMPACT

Influence of inter and intraobserver variability in CT measurements during planning of endovascular aneurysm repair (EVAR) has been extensively reviewed. However, its impact in graft selection (final choose of diameter and lengths) has been scarcely analyzed. The results of this study suggest that, although a high agreement between different observers using different workstations for AAA measurements is to be expected, small differences may lead to the selection of a different graft size. The use of a single software by experienced users, and double check by a different one, may be advisable.

Artificial Intelligence Application to Screen Abdominal Aortic Aneurysm Using Computed tomography Angiography

The aim of our study is to validate a totally automated deep learning (DL)-based segmentation pipeline to screen abdominal aortic aneurysms (AAA) in computed tomography angiography (CTA) scans. We retrospectively evaluated 73 thoraco-abdominal CTAs (48 AAA and 25 control CTA) by means of a DL-based segmentation pipeline built on a 2.5D convolutional neural network (CNN) architecture to segment lumen and thrombus of the aorta. The maximum aortic diameter of the abdominal tract was compared using a threshold value (30 mm). Blinded manual measurements from a radiologist were done in order to create a true comparison. The screening pipeline was tested on 48 patients with aneurysm and 25 without aneurysm. The average diameter manually measured was 51.1 ± 14.4 mm for patients with aneurysms and 21.7 ± 3.6 mm for patients without aneurysms. The pipeline correctly classified 47 AAA out of 48 and 24 control patients out of 25 with 97% accuracy, 98% sensitivity, and 96% specificity. The automated pipeline of aneurysm measurements in the abdominal tract reported a median error with regard to the maximum abdominal diameter measurement of 1.3 mm. Our approach allowed for the maximum diameter of 51.2 ± 14.3 mm in patients with aneurysm and 22.0 ± 4.0 mm in patients without an aneurysm. The DL-based screening for AAA is a feasible and accurate method, calling for further validation using a larger pool of diagnostic images towards its clinical use.

Automatic measurement of maximal diameter of abdominal aortic aneurysm on computed tomography angiography using artificial intelligence

INTRODUCTION

The treatment of abdominal aortic aneurysm (AAA) relies on surgical repair and the indication mainly depends on its size evaluated by the maximal diameter (Dmax). The aim of this study was to evaluate a new automatic method based on artificial intelligence (AI) to measure the Dmax on computed tomography angiography (CTA).

METHODS

A fully automatic segmentation of the vascular system was performed using a hybrid method combining expert system with supervised deep learning (DL). The aorta centreline was extracted from the segmented aorta and the aortic diameters were automatically calculated. Results were compared to manual segmentation performed by two human operators.

RESULTS

The median absolute error between the two human operators was 1.2 mm (IQR 0.5- 1.9). The automatic method using the DL algorithm demonstrated correlation with the human segmentation, with a median absolute error of 0.8 (0.5- 4.2) mm and a coefficient correlation of 0.91 (p<0.001).

CONCLUSION

Although validation in larger cohorts is required, this method brings perspectives to develop new tools to standardize and automate the measurement of AAA Dmax in order to help clinicians in the decision-making process.

Clinical Validation of a Semi-Automated Software for Maximal Diameter Measurements for Endovascular Repair Follow-up

PURPOSE

To compare automated measurements of maximal diameter (D_max) of abdominal aortic aneurysm (AAA) orthogonal to luminal or outer wall envelope centerline for endovascular repair (EVAR) follow-up.

MATERIAL AND METHODS

Eighty-three consecutive patients with AAA treated by EVAR who had at least 1 computed tomography (CT) scan before and 2 CT scans after EVAR with at least 5 months' interval were included. Three-dimensional reconstruction of the AAA was achieved with dedicated segmentation software. Performances of automated calculation algorithms of D_max perpendicular to lumen or outer wall envelope centerlines were then compared to manual measurement of D_max on double-oblique multiplanar reconstruction (gold standard). Accuracy of automated D_max measurements at baseline, follow-up, and progression over time was evaluated by calculation of mean error, Bland-Altman plot, and regression models.

RESULTS

Disagreement in D_max measurements between outer wall envelope algorithm and manual method was insignificant (mean error: baseline, -0.07 ± 1.66 mm, P = .7; first follow-up, 0.24 ± 1.69 mm, P = .2; last follow-up, -0.41 ± 2.74 mm, P = .17); whereas significant discrepancies were found between the luminal algorithm and the manual method (mean error: baseline, -1.24 ± 2.01 mm, P < .01; first follow-up, -1.49 ± 3.30 mm, P < .01; last follow-up, -1.78 ± 3.60 mm, P < .01). D_max progression results were more accurate with AAA outer wall envelope algorithm compared to luminal method (P = .2).

CONCLUSIONS

AAA outer wall envelope segmentation is recommended to enable automated calculation of D_max perpendicular to its centerline during EVAR follow-up.

How Precise Are Preinterventional Measurements Using Centerline Analysis Applications? Objective Ground Truth Evaluation Reveals Software-Specific Centerline Characteristics

PURPOSE

To evaluate different centerline analysis applications using objective ground truth from realistic aortic aneurysm phantoms with precisely defined geometry and centerlines to overcome the lack of unknown true dimensions in previously published in vivo validation studies.

METHODS

Three aortic phantoms were created using computer-aided design (CAD) software and a 3-dimensional (3D) printer. Computed tomography angiograms (CTAs) of phantoms and 3 patients were analyzed with 3 clinically approved and 1 research software application. The 3D centerline coordinates, intraluminal diameters, and lengths were validated against CAD ground truth using a dedicated evaluation software platform.

RESULTS

The 3D centerline position mean error ranged from 0.7±0.8 to 2.9±2.5 mm between tested applications. All applications calculated centerlines significantly different from ground truth. Diameter mean errors varied from 0.5±1.2 to 1.1±1.0 mm among 3 applications, but exceeded 8.0±11.0 mm with one application due to an unsteady distortion of luminal dimensions along the centerline. All tested commercially available software tools systematically underestimated centerline total lengths by -4.6±0.9 mm to -10.4±4.3 mm (maximum error -14.6 mm). Applications with the highest 3D centerline accuracy yielded the most precise diameter and length measurements.

CONCLUSION

One clinically approved application did not provide reproducible centerline-based analysis results, while another approved application showed length errors that might influence stent-graft choice and procedure success. The variety and specific characteristics of endovascular aneurysm repair planning software tools require scientific evaluation and user awareness.

Semiautomatic sizing software in emergency endovascular aneurysm repair for ruptured abdominal aortic aneurysms

PURPOSE

In emergency endovascular repair (EVAR) of ruptured aneurysms of the aorta (rAAA), anatomical suitability must be determined. Semiautomatic three-dimensional assessment of the aortoiliac arteries has the potential to standardise measurements. This study assesses the fitness for purpose of such a semiautomatic approach for rAAA and determined interobserver agreement on suitability.

MATERIALS AND METHODS

Interobserver study with six trained observers (4 vascular surgeons, 2 radiologists) blindly assessing preoperative computed tomography angiography scans of 50 consecutive patients with rAAA. A central lumen line (CLL) was generated, and perpendicular diameters, length along the CLL, and EVAR suitability were determined using dedicated sizing software (3mensio; 3mensio Vascular; Bilthoven, The Netherlands). Success of generating a CLL, time of assessment, and interobserver agreement was determined.

RESULTS

In the majority of the patients (median 76 %, range 64-78 %), a CLL was semiautomatically generated. The median duration of CLL generation and performance measurements was 7.5 min (interquartile range 5.5-10.6). Agreement on suitability was moderate for the entire group (Fleiss' κ = 0.55, confidence interval 0.48-0.62) and ranged from moderate to good (Cohen's κ = 0.40-0.72) between observer pairs.

CONCLUSION

Assessing EVAR suitability of rAAA patients using dedicated sizing software is possible in the majority of patients. The measurements can be performed in a reasonable amount of time, and the agreement of suitability for EVAR in patients with rAAA is moderate. Improvements and additional research are necessary to replace the current axial measurement.

A systematic review of protocols for the three-dimensional morphologic assessment of abdominal aortic aneurysms using computed tomographic angiography

The morphology of infrarenal abdominal aortic aneurysms (AAAs) directly influences the perioperative outcome and long-term durability of endovascular aneurysm repair. A variety of methods have been proposed for the characterization of AAA morphology using reconstructed three-dimensional (3D) computed tomography (CT) images. At present, there is lack of consensus as to which of these methods is most applicable to clinical practice or research. The purpose of this review was to evaluate existing protocols that used 3D CT images in the assessment of various aspects of AAA morphology. An electronic search was performed, from January 1996 to the end of October 2010, using the Embase and Medline databases. The literature review conformed to PRISMA statement standards. The literature search identified 604 articles, of which 31 studies met inclusion criteria. Only 15 of 31 studies objectively assessed reproducibility. Existing published protocols were insufficient to define a single evidence-based methodology for preoperative assessment of AAA morphology. Further development and expert consensus are required to establish a standardized and validated protocol to determine precisely how morphology relates to outcomes after endovascular aneurysm repair.

Accuracy and variability of semiautomatic centerline analysis versus manual aortic measurement techniques for TEVAR

OBJECTIVES

This study aims to test whether inter-observer variability and time of diameter measurements for thoracic endovascular aortic repair (TEVAR) are improved by semiautomatic centerline analysis compared to manual assessment.

METHODS

Preoperative computed tomography (CT) angiographies of 30 patients with thoracic aortic disease (mean age 66.8 ± 11.6 years, 23 males) were retrospectively analysed by two blinded experts in vascular radiology. Maximum aortic diameters at three positions relevant to TEVAR were assessed (P1, distal to left common carotid artery; P2, distal to left subclavian artery; and P3, proximal to coeliac trunk) using three measurement techniques: manual axial slices (axial), manual double-oblique multiplanar reformations (MPRs) and semiautomatic centerline analysis.

RESULTS

Diameter measurements by both centerline analysis and the axial technique did not significantly differ from MPR (p = 0.17 and p = 0.37). Total deviation index for 0.9 was for P1 2.7 mm (axial), 3.7 mm (MPR), 1.8 mm (centerline); for P2 2.0 mm (axial), 3.6 mm (MPR), 1.8 mm (centerline); and for P3 3.0 mm (axial), 3.5 mm (MPR), 2.5 mm (centerline). Measurement time using centerline analysis was significantly shorter than for assessment by MPR.

CONCLUSIONS

Centerline analysis provides the least variable and fast diameter measurements in TEVAR patients with the same accuracy as the current reference standard MPR.

Evaluation of computer-assisted quantification of carotid artery stenosis

The purpose of this study was to evaluate the influence of advanced software assistance on the assessment of carotid artery stenosis; particularly, the inter-observer variability of readers with different level of experience is to be investigated. Forty patients with suspected carotid artery stenosis received head and neck dual-energy CT angiography as part of their pre-interventional workup. Four blinded readers with different levels of experience performed standard imaging interpretation. At least 1 day later, they performed quantification using an advanced vessel analysis software including automatic dual-energy bone and hard plaque removal, automatic and semiautomatic vessel segmentation, as well as creation of curved planar reformation. Results were evaluated for the reproducibility of stenosis quantification of different readers by calculating the kappa and correlation values. Consensus reading of the two most experienced readers was used as the standard of reference. For standard imaging interpretation, experienced readers reached very good (k = 0.85) and good (k = 0.78) inter-observer variability. Inexperienced readers achieved moderate (k = 0.6) and fair (k = 0.24) results. Sensitivity values 80%, 91%, 83%, 77% and specificity values 100%, 84%, 82%, 53% were achieved for significant area stenosis >70%. For grading using advanced vessel analysis software, all readers achieved good inter-observer variability (k = 0.77, 0.72, 0.71, and 0.77). Specificity values of 97%, 95%, 95%, 93% and sensitivity values of 84%, 78%, 86%, 92% were achieved. In conclusion, when supported by advanced vessel analysis software, experienced readers are able to achieve good reproducibility. Even inexperienced readers are able to achieve good results in the assessment of carotid artery stenosis when using advanced vessel analysis software.

Two-dimensional versus three-dimensional CT angiography in analysis of anatomical suitability for stentgraft repair of abdominal aortic aneurysms

BACKGROUND

The morphological analysis prior to endovascular abdominal aneurysm repair (EVAR) plays an important role in long-term outcomes. Post-imaging analysis of computed tomographic angiography (CTA) by three-dimensional reconstruction with central lumen line detection (CLL 3D-CTA) enables measurements to be made in orthogonal slices. This might be more precise than equal post-imaging analysis in axial slices by two-dimensional computed tomographic angiography (2D-CTA).

PURPOSE

To evaluate the intra- and interobserver variability of CLL 3D-CTA and 2D-CTA post-imaging analysis methods and the agreement between them in pre-EVAR suitability analysis of patients with abdominal aortic aneurysm (AAA).

MATERIAL AND METHODS

Anonymized CTA data-sets from 70 patients with AAA were analyzed retrospectively. Length measurements included proximal and distal aortic neck lengths and total distance from the lower renal artery to the higher iliac bifurcation. Width measurements included proximal and distal neck diameters, maximum AAA diameter and common iliac diameters just above the iliac bifurcations. The measurements were performed in random order by two vascular surgeons, twice per method with 1-month interval between readings. In the CLL 3D-CTA method we used semi-automated CLL detection by software and manual measurements on CTA slices perpendicular to CLL. The equal measurements in 2D-CTA were performed manually on axial CTA slices using a DICOM viewer workstation. The intra- and interobserver variability, as well as the agreement between the two methods were assessed by Bland-Altman test and bivariate correlation analysis.

RESULTS

The intraobserver variability was significantly higher in 2D-CTA measurements for both readers. The interobserver variability was significant in 2D-CTA measurements of proximal neck dimensions while the agreement in CLL 3D-CTA analysis between the two readers was excellent in all studied parameters. The agreement between the two suitability analysis techniques was poor for both readers, especially in measurements of proximal neck's dimensions and in total aortoiliac length (p = 0.001).

CONCLUSION

In pre-EVAR morphological evaluation of AAAs the CLL-3D CTA post-imaging analysis has better intra- and interobserver correlation than 2D-CTA and might represent a useful tool for the proper selection of endograft's type and size.

Sizing for endovascular aneurysm repair: clinical evaluation of a new automated three-dimensional software

BACKGROUND

To assess the reproducibility and accuracy of the sizing procedure before aortic endograft implantation using new sizing automated software as compared with standard radiological procedures.

METHODS

On the basis of original spiral-computed tomography images, the sizing of 32 patients with abdominal aortic aneurysm treated by endovascular aneurysm repair (EVAR) was retrospectively compared. The first sizing was performed by a radiologist using a standard workstation (General electrics) and software (Advanced vessel analysis). The second was performed twice by two surgeons using a personal computer with automatic three-dimensional sizing software (Endosize; Therenva, Rennes, France). All diameters and lengths required before EVAR were measured (17 items). Moreover, 13 qualitative criteria regarding EVAR feasibility, including neck length, were compared. Intra- and interobserver variability with Endosize, as well as the variability between the two measurement methods were analyzed using the intraclass correlation coefficient (ICC) and Bland and Altman's method. Qualitative variables were analyzed using Fischer's exact test and kappa coefficient.

RESULTS

Intraobserver variability with Endosize proved to be efficient. None of the ICCs were lower than 0.9, and more than 90% of the absolute differences between two measurements were less than 2 mm. Interobserver variability with Endosize was assessed in a similar manner. Measurement variability of vessel diameters was less marked than that of vessel lengths. This trend was observed for all datasets. Comparison of the two measurement methods demonstrated a good correlation (minimum ICC = 0.697; maximum ICC = 0.974), although less so than that observed using Endosize. Mean time consumption using Endosize was 13.1 ± 4.53 minutes (range: 7.2-32.7). Analysis of the alarm sets demonstrated a high agreement between observers (kappa coefficient = 0.81).

CONCLUSIONS

Sizing using the Endosize software is as reliable as conventional radiological procedures. Sizing by surgeons using an automated, user-friendly, and mobile tool appears to be reproducible.

Imaging in endovascular therapy: our future

The endovascular therapist now has many modern imaging techniques available to plan and execute treatment, whereas in the past vascular surgeons relied mostly on clinical examination and arteriography. Advances in computer technology have enabled fast acquisition and processing of the large amounts of digital data essential to capture the dynamic information from fast-flowing blood at high resolution. Functional imaging has begun to play a role in predicting stability of progressive vascular disease and the need for and risks of intervention. Computing power now affords the interventionist the ability to handle imaging data in powerful 3-dimensional programs and electronically "in-lay" a variety of devices to plan complex endovascular procedures from the familiar platform of a laptop. In four major clinical areas, carotid intervention, peripheral intervention, endoluminal grafting, and cardiac imaging, we review the latest advances and changes with an eye toward how we should best be using imaging in our patients undergoing endovascular treatment...now and into the future.

Intraobserver and interobserver variability of 64-row computed tomography abdominal aortic aneurysm neck measurements

BACKGROUND

Integrity of the abdominal aortic aneurysm (AAA) neck is crucial for the long-term success of endovascular AAA repair (EVAR). However, suitable tools for reliable assessment of changes in small aortic volumes are lacking. The purpose of this study was to assess the intraobserver and interobserver variability of software-enhanced 64-row computed tomographic angiography (CTA) AAA neck volume measurements in patients after EVAR.

METHODS

A total of 25 consecutive patients successfully treated by EVAR underwent 64-row follow-up CTA in 1.5-mm collimation. Manual CTA measurements were performed twice by three blinded and independent readers in random order with at least a 4-week interval between readings. Maximum and minimum transverse aortic neck diameters were measured twice on two different levels within the proximal neck. Volumetry of the proximal aortic neck was performed by using dedicated software. Variability was calculated as 1.96 SD of the mean arithmetic difference according to Bland and Altman. Two-sided and paired t tests were used to compare measurements. P values <.05 were considered to indicate statistical significance.

RESULTS

Intraobserver agreement was excellent for dedicated aneurysmal neck volumetry, with mean differences of less than 1 mL (P > .05), whereas it was poor for transverse aortic neck diameter measurements (P < .05). However, interobserver variability was statistically significant for both neck volumetry (P < .005) and neck diameter measurements (P < .015).

CONCLUSIONS

The reliability of dedicated AAA neck volumetry by using 64-row CTA is excellent for serial measurements by individual readers, but not between different readers. Therefore, studies should be performed with aortic neck volumetry by a single experienced reader.