The use of computer-aided detection for the assessment of pulmonary arterial filling defects at computed tomographic angiography

Pixel-level annotated dataset of computed tomography angiography images of acute pulmonary embolism

Pulmonary embolism has a high incidence and mortality, especially if undiagnosed. The examination of choice for diagnosing the disease is computed tomography pulmonary angiography. As many factors can lead to misinterpretations and diagnostic errors, different groups are utilizing deep learning methods to help improve this process. The diagnostic accuracy of these methods tends to increase by augmenting the training dataset. Deep learning methods can potentially benefit from the use of images acquired with devices from different vendors. To the best of our knowledge, we have developed the first public dataset annotated at the pixel and image levels and the first pixel-level annotated dataset to contain examinations performed with equipment from Toshiba and GE. This dataset includes 40 examinations, half performed with each piece of equipment, representing samples from two medical services. We also included measurements related to the cardiac and circulatory consequences of pulmonary embolism. We encourage the use of this dataset to develop, evaluate and compare the performance of new AI algorithms designed to diagnose PE.

Computer-assisted detection of acute pulmonary embolism at CT pulmonary angiography in children and young adults: a diagnostic performance analysis

Background

To diagnose pulmonary embolism (PE) in children and adults since evaluating tiny pulmonary vasculature beyond segmental level is a challenging and demanding task with thousands of images.

Purpose

To evaluate the effect of computer-assisted detection (CAD) on acute PE on CTPA in children and young adults by readers with varying experience levels.

Material and Methods

Six radiologists were retrospectively divided into three groups according to experience levels and assessed the CTPA studies on a per-emboli basis. All readers identified independently the PE presence, and ranked diagnostic confidence on a 5-point scale with and without CAD. Reading time, sensitivities, specificities, accuracies, positive predictive values (PPVs), and negative predictive values (NPVs) were calculated for each reading.

Results

The sensitivities and NPVs differed significantly in most readers ( P = 0.004, 0.001, 0.010, 0.010, and 0.012 for sensitivities and P = 0.011, 0.003, 0.016, 0.017, and 0.019 for NPVs) except for reader 6 ( P = 0.148 and 0.165, respectively), and the accuracies of all readers differed significantly (all P < 0.05) in peripheral PE (beyond segmental level) detection readings with CAD versus without CAD between two reading methods. The overall time using CAD was longer than those without CAD (76.6 ± 54.4 s vs. 49.4 ± 17.7 s, P = 0.000) for all readers. Significant differences were found for confidence scores in inter-group measurements with CAD ( P = 0.045) and without CAD ( P < 0.001).

Conclusion

At the expense of longer reading time, the use of the CAD algorithms improves sensitivities, NPVs, and the accuracies of readers in peripheral PE detection, especially for readers with a poor level of interpretation experience.

Stand-alone performance of a computer-assisted detection prototype for detection of acute pulmonary embolism: a multi-institutional comparison

OBJECTIVE

To assess whether the performance of a computer-assisted detection (CAD) algorithm for acute pulmonary embolism (PE) differs in pulmonary CT angiographies acquired at various institutions.

METHODS

In this retrospective study, we included 40 consecutive scans with and 40 without PE from 3 institutions (n = 240) using 64-slice scanners made by different manufacturers (General Electric; Philips; Siemens). CAD markers were classified as true or false positive (FP) using independent evaluation by two readers and consultation of a third chest radiologist in discordant cases. Image quality parameters were subjectively scored using 4/5-point scales. Image noise and vascular enhancement were measured. Statistical analysis was done to correlate image quality of the three institutions with CAD stand-alone performance.

RESULTS

Patient groups were comparable with respect to age (p = 0.22), accompanying lung disease (p = 0.12) and inpatient/outpatient ratio (p = 0.67). The sensitivity was 100% (34/34), 97% (37/38) and 92% (33/36), and the specificity was 18% (8/44), 15% (6/41) and 13% (5/39). Neither significantly differed between the institutions (p = 0.21 and p = 0.820, respectively). The mean number of FP findings (4.5, 6.2 and 3.7) significantly varied (p = 0.02 and p = 0.03), but median numbers (2, 3 and 3) were comparable. Image quality parameters were significantly associated with the number of FP findings (p<0.05) but not with sensitivity. After correcting for noise and vascular enhancement, the number of FPs did not significantly differ between the three institutions (p = 0.43).

CONCLUSIONS

CAD stand-alone performance is independent of scanner type but strongly related to image quality and thus scanning protocols.

Carotid artery stenosis: performance of advanced vessel analysis software in evaluating CTA

OBJECTIVES

The aim of this study was to evaluate time efficiency and diagnostic reproducibility of an advanced vessel analysis software for diagnosis of carotid artery stenosis.

MATERIAL AND METHODS

40 patients with suspected carotid artery stenosis received head and neck DE-CTA as part of their pre-interventional workup. Acquired data were evaluated by 2 independent radiologists. Stenosis grading was performed by MPR eyeballing with freely adjustable MPRs and with a preliminary prototype of the meanwhile available client-server and advanced visualization software syngo.via CT Vascular (Siemens Healthcare, Erlangen, Germany). Stenoses were graded according to the following 5 categories: I: 0%, II: 1-50%, III: 51-69%, IV: 70-99% and V: total occlusion. Furthermore, time to diagnosis for each carotid artery was recorded.

RESULTS

Both readers achieved very good specificity values and good respectively very good sensitivity values without significant differences between both reading methods. Furthermore, there was a very good correlation between both readers for both reading methods without significant differences (kappa value: standard image interpretation k=0.809; advanced vessel analysis software k=0.863). Using advanced vessel analysis software resulted in a significant time saving (p<0.0001) for both readers. Time to diagnosis could be decreased by approximately 55%.

CONCLUSIONS

Advanced vessel analysis application CT Vascular of the new imaging software syngo.via (Siemens Healthcare, Forchheim, Germany) provides a high rate of reproducibility in assessment of carotid artery stenosis. Furthermore a significant time saving in comparison to standard image interpretation is achievable.

Automation bias: a systematic review of frequency, effect mediators, and mitigators

Automation bias (AB)--the tendency to over-rely on automation--has been studied in various academic fields. Clinical decision support systems (CDSS) aim to benefit the clinical decision-making process. Although most research shows overall improved performance with use, there is often a failure to recognize the new errors that CDSS can introduce. With a focus on healthcare, a systematic review of the literature from a variety of research fields has been carried out, assessing the frequency and severity of AB, the effect mediators, and interventions potentially mitigating this effect. This is discussed alongside automation-induced complacency, or insufficient monitoring of automation output. A mix of subject specific and freetext terms around the themes of automation, human-automation interaction, and task performance and error were used to search article databases. Of 13 821 retrieved papers, 74 met the inclusion criteria. User factors such as cognitive style, decision support systems (DSS), and task specific experience mediated AB, as did attitudinal driving factors such as trust and confidence. Environmental mediators included workload, task complexity, and time constraint, which pressurized cognitive resources. Mitigators of AB included implementation factors such as training and emphasizing user accountability, and DSS design factors such as the position of advice on the screen, updated confidence levels attached to DSS output, and the provision of information versus recommendation. By uncovering the mechanisms by which AB operates, this review aims to help optimize the clinical decision-making process for CDSS developers and healthcare practitioners.

Computer-assisted detection of pulmonary embolism: evaluation of pulmonary CT angiograms performed in an on-call setting

Purpose

The purpose of the study was to assess the stand-alone performance of computer-assisted detection (CAD) for evaluation of pulmonary CT angiograms (CTPA) performed in an on-call setting.

Methods

In this institutional review board-approved study, we retrospectively included 292 consecutive CTPA performed during night shifts and weekends over a period of 16 months. Original reports were compared with a dedicated CAD system for pulmonary emboli (PE). A reference standard for the presence of PE was established using independent evaluation by two readers and consultation of a third experienced radiologist in discordant cases.

Results

Original reports had described 225 negative studies and 67 positive studies for PE. CAD found PE in seven patients originally reported as negative but identified by independent evaluation: emboli were located in segmental (n = 2) and subsegmental arteries (n = 5). The negative predictive value (NPV) of the CAD algorithm was 92% (44/48). On average there were 4.7 false positives (FP) per examination (median 2, range 0–42). In 72% of studies ≤5 FP were found, 13% of studies had ≥10 FP.

Conclusion

CAD identified small emboli originally missed under clinical conditions and found 93% of the isolated subsegmental emboli. On average there were 4.7 FP per examination.