Lung nodule CAD software as a second reader: a multicenter study

Deep-learning for automated detection of MSU deposits on DECT: evaluating impact on efficiency and reader confidence

Introduction

Dual-energy CT (DECT) is a non-invasive way to determine the presence of monosodium urate (MSU) crystals in the workup of gout. Color-coding distinguishes MSU from calcium following material decomposition and post-processing. Manually identifying these foci (most commonly labeled green) is tedious, and an automated detection system could streamline the process. This study aims to evaluate the impact of a deep-learning (DL) algorithm developed for detecting green pixelations on DECT on reader time, accuracy, and confidence.

Methods

We collected a sample of positive and negative DECTs, reviewed twice-once with and once without the DL tool-with a 2-week washout period. An attending musculoskeletal radiologist and a fellow separately reviewed the cases, simulating clinical workflow. Metrics such as time taken, confidence in diagnosis, and the tool's helpfulness were recorded and statistically analyzed.

Results

We included thirty DECTs from different patients. The DL tool significantly reduced the reading time for the trainee radiologist (p = 0.02), but not for the attending radiologist (p = 0.15). Diagnostic confidence remained unchanged for both (p = 0.45). However, the DL model identified tiny MSU deposits that led to a change in diagnosis in two cases for the in-training radiologist and one case for the attending radiologist. In 3/3 of these cases, the diagnosis was correct when using DL.

Conclusions

The implementation of the developed DL model slightly reduced reading time for our less experienced reader and led to improved diagnostic accuracy. There was no statistically significant difference in diagnostic confidence when studies were interpreted without and with the DL model.

Incorporation of CAD (computer-aided detection) with thin-slice lung CT in routine 18F-FDG PET/CT imaging read-out protocol for detection of lung nodules

OBJECTIVE

To evaluate the detection rate and performance of 18F-FDG PET alone (PET), the combination of PET and low-dose thick-slice CT (PET/lCT), PET and diagnostic thin-slice CT (PET/dCT), and additional computer-aided detection (PET/dCT/CAD) for lung nodules (LN)/metastases in tumor patients. Along with this, assessment of inter-reader agreement and time requirement for different techniques were evaluated as well.

METHODS

In 100 tumor patients (56 male, 44 female; age range: 22-93 years, mean age: 60 years) 18F-FDG PET images, low-dose CT with shallow breathing (5 mm slice thickness), and diagnostic thin-slice CT (1 mm slice thickness) in full inspiration were retrospectively evaluated by three readers with variable experience (junior, mid-level, and senior) for the presence of lung nodules/metastases and additionally analyzed with CAD. Time taken for each analysis and number of the nodules detected were assessed. Sensitivity, specificity, positive and negative predictive value, accuracy, and Receiver operating characteristic (ROC) analysis of each technique was calculated. Histopathology and/or imaging follow-up served as reference standard for the diagnosis of metastases.

RESULTS

Three readers, on an average, detected 40 LN in 17 patients with PET only, 121 LN in 37 patients using ICT, 283 LN in 60 patients with dCT, and 282 LN in 53 patients with CAD. On average, CAD detected 49 extra LN, missed by the three readers without CAD, whereas CAD overall missed 53 LN. There was very good inter-reader agreement regarding the diagnosis of metastases for all four techniques (kappa: 0.84-0.93). The average time required for the evaluation of LN in PET, lCT, dCT, and CAD was 25, 31, 60, and 40 s, respectively; the assistance of CAD lead to average 33% reduction in time requirement for evaluation of lung nodules compared to dCT. The time-saving effect was highest in the less experienced reader. Regarding the diagnosis of metastases, sensitivity and specificity combined of all readers were 47.8%/96.2% for PET, 80.0%/81.9% for PET/lCT, 100%/56.7% for PET/dCT, and 95.6%/64.3% for PET/CAD. No significant difference was observed regarding the ROC AUC (area under the curve) between the imaging methods.

CONCLUSION

Implementation of CAD for the detection of lung nodules/metastases in routine 18F-FDG PET/CT read-out is feasible. The combination of diagnostic thin-slice CT and CAD significantly increases the detection rate of lung nodules in tumor patients compared to the standard PET/CT read-out. PET combined with low-dose CT showed the best balance between sensitivity and specificity regarding the diagnosis of metastases per patient. CAD reduces the time required for lung nodule/metastasis detection, especially for less experienced readers.

The current status and future of FDA-approved artificial intelligence tools in chest radiology in the United States

Artificial intelligence (AI) is becoming more widespread within radiology. Capabilities that AI algorithms currently provide include detection, segmentation, classification, and quantification of pathological findings. Artificial intelligence software have created challenges for the traditional United States Food and Drug Administration (FDA) approval process for medical devices given their abilities to evolve over time with incremental data input. Currently, there are 190 FDA-approved radiology AI-based software devices, 42 of which pertain specifically to thoracic radiology. The majority of these algorithms are approved for the detection and/or analysis of pulmonary nodules, for monitoring placement of endotracheal tubes and indwelling catheters, for detection of emergent findings, and for assessment of pulmonary parenchyma; however, as technology evolves, there are many other potential applications that can be explored. For example, evaluation of non-idiopathic pulmonary fibrosis interstitial lung diseases, synthesis of imaging, clinical and/or laboratory data to yield comprehensive diagnoses, and survival or prognosis prediction of certain pathologies. With increasing physician and developer engagement, transparency and frequent communication between developers and regulatory agencies, such as the FDA, AI medical devices will be able to provide a critical supplement to patient management and ultimately enhance physicians' ability to improve patient care.

Artificial intelligence software in pulmonary nodule assessment

Background:

This study tests the impact of the addition of autonomous computed tomography (CT) interpreting software to radiologist assessment of pulmonary nodules.

Methods:

Computed tomography scans for nodule assessment were identified retrospectively. Lung cancer risk factors, initial radiologist (RAD) report, Philips Lung Nodule software report (computer-aided nodule (CAD)) and radiologist report following the review of CT images and CAD (RAD + CAD) were collected. Follow-up recommendations based on current guidelines were derived from each report.

Results:

In all, 100 patients were studied. Median maximal diameter of the largest nodule reported by RAD and RAD + CAD were similar at 10.0 and 9.0 mm, respectively ( p = 0.06) but were reported as larger by CAD at 11.8 mm ( p < 0.001). Follow-up recommendations derived from RAD + CAD were less intensive in 23 (23%) and more intensive in 34 (34%) than that of RAD.

Discussion:

This study suggests that autonomous software use can alter radiologist assessment of pulmonary nodules such that suggested follow-up is altered.

Radiologists with and without deep learning-based computer-aided diagnosis: comparison of performance and interobserver agreement for characterizing and diagnosing pulmonary nodules/masses

OBJECTIVES

To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)-based computer-aided diagnosis (CAD).

METHODS

We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3-5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs).

RESULTS

The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD.

CONCLUSIONS

DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists.

KEY POINTS

• Deep learning-based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.

Solitary Pulmonary Nodule Evaluation: Pearls and Pitfalls

Lung nodules are frequently encountered while interpreting chest CTs and are challenging to detect, characterize, and manage given they can represent both benign or malignant etiologies. An understanding of features associated with malignancy and causes of interpretive pitfalls is helpful to avoid misdiagnoses. This review addresses pertinent topics related to the etiologies for missed lung nodules on radiography and CT. Additionally, CT imaging technical pitfalls and challenges in addition to issues in the evaluation of nodule morphology, attenuation, and size will be discussed. Nodule management guidelines will be addressed as well as recent investigations that further our understanding of lung nodules.

Clinical impact of a deep learning system for automated detection of missed pulmonary nodules on routine body computed tomography including the chest region

OBJECTIVES

To evaluate the clinical impact of a deep learning system (DLS) for automated detection of pulmonary nodules on computed tomography (CT) images as a second reader.

METHODS

This single-centre retrospective study screened 21,150 consecutive body CT studies from September 2018 to February 2019. Pulmonary nodules detected by the DLS on axial CT images but not mentioned in initial radiology reports were flagged. Flagged images were scored by four board-certificated radiologists each with at least 5 years of experience. Nodules with scores of 2 (understandable miss) or 3 (should not be missed) were then categorised as unlikely to be clinically significant (2a or 3a) or likely to be clinically significant (2b or 3b) according to the 2017 Fleischner guidelines for pulmonary nodules. The miss rate was defined as the total number of studies receiving scores of 2 or 3 divided by total screened studies.

RESULTS

Among 172 nodules flagged by the DLS, 60 (35%) missed nodules were confirmed by the radiologists. The nodules were further categorised as 2a, 2b, 3a, and 3b in 24, 14, 10, and 12 studies, respectively, with an overall positive predictive value of 35%. Missed pulmonary nodules were identified in 0.3% of all CT images, and one-third of these lesions were considered clinically significant.

CONCLUSIONS

Use of DLS-assisted automated detection as a second reader can identify missed pulmonary nodules, some of which may be clinically significant.

CLINICAL RELEVANCE/APPLICATION

Use of DLS to help radiologists detect pulmonary lesions may improve patient care.

KEY POINTS

• DLS-assisted automated detection as a second reader is feasible in a large consecutive cohort. • Performance of combined radiologists and DLS was better than DLS or radiologists alone. • Pulmonary nodules were missed more frequently in abdomino-pelvis CT than the thoracic CT.

Assisted versus Manual Interpretation of Low-Dose CT Scans for Lung Cancer Screening: Impact on Lung-RADS Agreement

Purpose To compare the inter- and intraobserver agreement and reading times achieved when assigning Lung Imaging Reporting and Data System (Lung-RADS) categories to baseline and follow-up lung cancer screening studies by using a dedicated CT lung screening viewer with integrated nodule detection and volumetric support with those achieved by using a standard picture archiving and communication system (PACS)-like viewer. Materials and Methods Data were obtained from the National Lung Screening Trial (NLST). By using data recorded by NLST radiologists, scans were assigned to Lung-RADS categories. For each Lung-RADS category (1 or 2, 3, 4A, and 4B), 40 CT scans (20 baseline scans and 20 follow-up scans) were randomly selected for 160 participants (median age, 61 years; interquartile range, 58-66 years; 61 women) in total. Seven blinded observers independently read all CT scans twice in a randomized order with a 2-week washout period: once by using the standard PACS-like viewer and once by using the dedicated viewer. Observers were asked to assign a Lung-RADS category to each scan and indicate the risk-dominant nodule. Inter- and intraobserver agreement was analyzed by using Fleiss κ values and Cohen weighted κ values, respectively. Reading times were compared by using a Wilcoxon signed rank test. Results The interobserver agreement was moderate for the standard viewer and substantial for the dedicated viewer, with Fleiss κ values of 0.58 (95% CI: 0.55, 0.60) and 0.66 (95% CI: 0.64, 0.68), respectively. The intraobserver agreement was substantial, with a mean Cohen weighted κ value of 0.67. The median reading time was significantly reduced from 160 seconds with the standard viewer to 86 seconds with the dedicated viewer (P < .001). Conclusion Lung-RADS interobserver agreement increased from moderate to substantial when using the dedicated CT lung screening viewer. The median reading time was substantially reduced when scans were read by using the dedicated CT lung screening viewer. Keywords: CT, Thorax, Lung, Computer Applications-Detection/Diagnosis, Observer Performance, Technology Assessment Supplemental material is available for this article. © RSNA, 2021.

Performance and reading time of lung nodule identification on multidetector CT with or without an artificial intelligence-powered computer-aided detection system

AIM

To compare the performance and reading time of different readers using automatic artificial intelligence (AI)-powered computer-aided detection (CAD) to detect lung nodules in different reading modes.

MATERIALS AND METHODS

One hundred and fifty multidetector computed tomography (CT) datasets containing 340 nodules ≤10 mm in diameter were collected retrospectively. A CAD with vessel-suppressed function was used to interpret the images. Three junior and three senior readers were assigned to read (1) CT images without CAD, (2) second-read using CAD in which CAD was applied only after initial unassisted assessment, and (3) a concurrent read with CAD in which CAD was applied at the start of assessment. Diagnostic performances and reading times were compared using analysis of variance.

RESULTS

For all readers, the mean sensitivity improved from 64% (95% confidence interval [CI]: 62%, 66%) for the without-CAD mode to 82% (95% CI: 80%, 84%) for the second-reading mode and to 80% (95% CI: 79%, 82%) for the concurrent-reading mode (p<0.001). There was no significant difference between the two modes in terms of the mean sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for both junior and senior readers and all readers (p>0.05). The reading time of all readers was significantly shorter for the concurrent-reading mode (124 ± 25 seconds) compared to without CAD (156 ± 34 seconds; p<0.001) and the second-reading mode (197 ± 46 seconds; p<0.001).

CONCLUSION

In CAD for lung nodules at CT, the second-reading mode and concurrent-reading mode may improve detection performance for all readers in both screening and clinical routine practice. Concurrent use of CAD is more efficient for both junior and senior readers.

Artificial intelligence for detection and characterization of pulmonary nodules in lung cancer CT screening: ready for practice?

Lung cancer computed tomography (CT) screening trials using low-dose CT have repeatedly demonstrated a reduction in the number of lung cancer deaths in the screening group compared to a control group. With various countries currently considering the implementation of lung cancer screening, recurring discussion points are, among others, the potentially high false positive rates, cost-effectiveness, and the availability of radiologists for scan interpretation. Artificial intelligence (AI) has the potential to increase the efficiency of lung cancer screening. We discuss the performance levels of AI algorithms for various tasks related to the interpretation of lung screening CT scans, how they compare to human experts, and how AI and humans may complement each other. We discuss how AI may be used in the lung cancer CT screening workflow according to the current evidence and describe the additional research that will be required before AI can take a more prominent role in the analysis of lung screening CT scans.

How to Read Articles That Use Machine Learning: Users' Guides to the Medical Literature

In recent years, many new clinical diagnostic tools have been developed using complicated machine learning methods. Irrespective of how a diagnostic tool is derived, it must be evaluated using a 3-step process of deriving, validating, and establishing the clinical effectiveness of the tool. Machine learning-based tools should also be assessed for the type of machine learning model used and its appropriateness for the input data type and data set size. Machine learning models also generally have additional prespecified settings called hyperparameters, which must be tuned on a data set independent of the validation set. On the validation set, the outcome against which the model is evaluated is termed the reference standard. The rigor of the reference standard must be assessed, such as against a universally accepted gold standard or expert grading.

Integration of fully automated computer-aided pulmonary nodule detection into CT pulmonary angiography studies in the emergency department: effect on workflow and diagnostic accuracy

PURPOSE

To assess the feasibility of implementing fully automated computer-aided diagnosis (CAD) for detection of pulmonary nodules on CT pulmonary angiography (CTPA) studies in emergency setting.

MATERIALS AND METHODS

CTPA of 48 emergency patients was retrospectively reviewed. Fully automated CAD nodule detection was performed at the scanner and results were automatically submitted to PACS. A third-year radiology resident (RAD1) and a cardiothoracic radiologist with 6 years' experience (RAD2) reviewed the scans independently to detect pulmonary nodules in two different sessions 8 weeks apart: session 1, CAD was reviewed first and then all images were reviewed; session 2, CAD was reviewed last after all images were reviewed. Time spent by RAD to evaluate image sets was measured for each case. Fisher's exact test and t test were used.

RESULTS

There were 17 male and 31 female patients with mean ± SD age of 48.7 ± 16.4 years. Using CAD at the beginning was associated with lower average reading time for both readers. However, difference in reading time did not reach statistical significance for RAD1 (RAD1 94.6 s vs. 102.7 s, P > 0.05; RAD2 61.1 s vs. 76.5 s, P < 0.05). Using CAD at the end significantly increased rate of RAD1 and RAD2 nodule detection by 34% (2.52 vs. 2.12 nodule/scan, P < 0.05) and 27% (2.23 vs. 1.81 nodule/scan, P < 0.05), respectively.

CONCLUSION

Routine utilization of CAD in emergency setting is feasible and can improve detection rate of pulmonary nodules significantly. Different methods of incorporating CAD in detecting pulmonary nodules can improve both the rate of detection and interpretation speed.

Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists

Background

The study was conducted to evaluate the performance of a state‐of‐the‐art commercial deep learning‐based computer‐aided diagnosis (DL‐CAD) system for detecting and characterizing pulmonary nodules.

Methods

Pulmonary nodules in 346 healthy subjects (male: female = 221:125, mean age 51 years) from a lung cancer screening program conducted from March to November 2017 were screened using a DL‐CAD system and double reading independently, and their performance in nodule detection and characterization were evaluated. An expert panel combined the results of the DL‐CAD system and double reading as the reference standard.

Results

The DL‐CAD system showed a higher detection rate than double reading, regardless of nodule size (86.2% vs. 79.2%; P < 0.001): nodules ≥ 5 mm (96.5% vs. 88.0%; P = 0.008); nodules < 5 mm (84.3% vs. 77.5%; P < 0.001). However, the false positive rate (per computed tomography scan) of the DL‐CAD system (1.53, 529/346) was considerably higher than that of double reading (0.13, 44/346; P < 0.001). Regarding nodule characterization, the sensitivity and specificity of the DL‐CAD system for distinguishing solid nodules > 5 mm (90.3% and 100.0%, respectively) and ground‐glass nodules (100.0% and 96.1%, respectively) were close to that of double reading, but dropped to 55.5% and 93%, respectively, when discriminating part solid nodules.

Conclusion

Our DL‐CAD system detected significantly more nodules than double reading. In the future, false positive findings should be further reduced and characterization accuracy improved.

Radiologist performance in the detection of lung cancer using CT

AIM

To measure the level of radiologists' performance in lung cancer detection, and to explore radiologists' performance in cancer specialised and non-specialised centres.

MATERIALS AND METHODS

Thirty radiologists read 60 chest computed tomography (CT) examinations. Thirty cases had surgically or biopsy-proven lung cancer and 30 were cancer-free cases. The cancer cases were validated by four expert radiologists who located the malignant lung nodules. Reader performance was evaluated by calculating sensitivity, location sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC). In addition, sensitivity at fixed specificity (0.794) was computed from each reader's estimated ROC curve.

RESULTS

The radiologists had a mean sensitivity of 0.749, sensitivity at fixed specificity of 0.744, location sensitivity of 0.666, specificity of 0.81 and AUC of 0.846. Radiologists in the specialised and non-specialised cancer centres had the following (specialised, non-specialised) pairs of values: sensitivity=(0.80, 0.719); sensitivity for fixed 0.794 specificity=(0.752, 0.740); location sensitivity=(0.712, 0.637); specificity=(0.794, 0.82) and AUC=(0.846, 0.846).

CONCLUSION

The efficacy of radiologists was comparable to other studies. Furthermore, AUC outcomes were similar for specialised and non-specialised cancer centre radiologists, suggesting they have similar discriminatory ability and that the higher sensitivity and lower specificity for specialised-centre radiologists can be attributed to them being less conservative in interpreting case images.

Technical Note: Image filtering to make computer-aided detection robust to image reconstruction kernel choice in lung cancer CT screening

PURPOSE

In lung cancer computed tomography (CT) screening, the performance of a computer-aided detection (CAD) system depends on the selection of the image reconstruction kernel. To reduce this dependence on reconstruction kernels, the authors propose a novel application of an image filtering method previously proposed by their group.

METHODS

The proposed filtering process uses the ratio of modulation transfer functions (MTFs) of two reconstruction kernels as a filtering function in the spatial-frequency domain. This method is referred to as MTFratio filtering. Test image data were obtained from CT screening scans of 67 subjects who each had one nodule. Images were reconstructed using two kernels: fSTD (for standard lung imaging) and fSHARP (for sharp edge-enhancement lung imaging). The MTFratio filtering was implemented using the MTFs measured for those kernels and was applied to the reconstructed fSHARP images to obtain images that were similar to the fSTD images. A mean filter and a median filter were applied (separately) for comparison. All reconstructed and filtered images were processed using their prototype CAD system.

RESULTS

The MTFratio filtered images showed excellent agreement with the fSTD images. The standard deviation for the difference between these images was very small, ∼6.0 Hounsfield units (HU). However, the mean and median filtered images showed larger differences of ∼48.1 and ∼57.9 HU from the fSTD images, respectively. The free-response receiver operating characteristic (FROC) curve for the fSHARP images indicated poorer performance compared with the FROC curve for the fSTD images. The FROC curve for the MTFratio filtered images was equivalent to the curve for the fSTD images. However, this similarity was not achieved by using the mean filter or median filter.

CONCLUSIONS

The accuracy of MTFratio image filtering was verified and the method was demonstrated to be effective for reducing the kernel dependence of CAD performance.

A method for evaluating the performance of computer-aided detection of pulmonary nodules in lung cancer CT screening: detection limit for nodule size and density

OBJECTIVE

We propose the application of virtual nodules to evaluate the performance of computer-aided detection (CAD) of lung nodules in cancer screening using low-dose CT.

METHODS

The virtual nodules were generated based on the spatial resolution measured for a CT system used in an institution providing cancer screening and were fused into clinical lung images obtained at that institution, allowing site specificity. First, we validated virtual nodules as an alternative to artificial nodules inserted into a phantom. In addition, we compared the results of CAD analysis between the real nodules (n = 6) and the corresponding virtual nodules. Subsequently, virtual nodules of various sizes and contrasts between nodule density and background density (ΔCT) were inserted into clinical images (n = 10) and submitted for CAD analysis.

RESULTS

In the validation study, 46 of 48 virtual nodules had the same CAD results as artificial nodules (kappa coefficient = 0.913). Real nodules and the corresponding virtual nodules showed the same CAD results. The detection limits of the tested CAD system were determined in terms of size and density of peripheral lung nodules; we demonstrated that a nodule with a 5-mm diameter was detected when the nodule had a ΔCT > 220 HU.

CONCLUSION

Virtual nodules are effective in evaluating CAD performance using site-specific scan/reconstruction conditions. Advances in knowledge: Virtual nodules can be an effective means of evaluating site-specific CAD performance. The methodology for guiding the detection limit for nodule size/density might be a useful evaluation strategy.

Computer-aided detection (CAD) of solid pulmonary nodules in chest x-ray equivalent ultralow dose chest CT - first in-vivo results at dose levels of 0.13mSv

OBJECTIVES

To determine the value of computer-aided detection (CAD) for solid pulmonary nodules in ultralow radiation dose single-energy computed tomography (CT) of the chest using third-generation dual-source CT at 100kV and fixed tube current at 70 mAs with tin filtration.

METHODS

202 consecutive patients undergoing clinically indicated standard dose chest CT (1.8±0.7 mSv) were prospectively included and scanned with an additional ultralow dose CT (0.13±0.01 mSv) in the same session. Standard of reference (SOR) was established by consensus reading of standard dose CT by two radiologists. CAD was performed in standard dose and ultralow dose CT with two different reconstruction kernels. CAD detection rate of nodules was evaluated including subgroups of different nodule sizes (<5, 5-7, >7mm). Sensitivity was further analysed in multivariable mixed effects logistic regression.

RESULTS

The SOR included 279 solid nodules (mean diameter 4.3±3.4mm, range 1-24mm). There was no significant difference in per-nodule sensitivity of CAD in standard dose with 70% compared to 68% in ultralow dose CT both overall and in different size subgroups (all p>0.05). CAD led to a significant increase of sensitivity for both radiologists reading the ultralow dose CT scans (all p<0.001). In multivariable analysis, the use of CAD (p<0.001), and nodule size (p<0.0001) were independent predictors for nodule detection, but not BMI (p=0.933) and the use of contrast agents (p=0.176).

CONCLUSIONS

Computer-aided detection of solid pulmonary nodules using ultralow dose CT with chest X-ray equivalent radiation dose has similar sensitivities to those from standard dose CT. Adding CAD in ultralow dose CT significantly improves the sensitivity of radiologists.

Lung nodule and cancer detection in computed tomography screening

Fundamental to the diagnosis of lung cancer in computed tomography (CT) scans is the detection and interpretation of lung nodules. As the capabilities of CT scanners have advanced, higher levels of spatial resolution reveal tinier lung abnormalities. Not all detected lung nodules should be reported; however, radiologists strive to detect all nodules that might have relevance to cancer diagnosis. Although medium to large lung nodules are detected consistently, interreader agreement and reader sensitivity for lung nodule detection diminish substantially as the nodule size falls below 8 to 10 mm. The difficulty in establishing an absolute reference standard presents a challenge to the reliability of studies performed to evaluate lung nodule detection. In the interest of improving detection performance, investigators are using eye tracking to analyze the effectiveness with which radiologists search CT scans relative to their ability to recognize nodules within their search path in order to determine whether strategies might exist to improve performance across readers. Beyond the viewing of transverse CT reconstructions, image processing techniques such as thin-slab maximum-intensity projections are used to substantially improve reader performance. Finally, the development of computer-aided detection has continued to evolve with the expectation that one day it will serve routinely as a tireless partner to the radiologist to enhance detection performance without significant prolongation of the interpretive process. This review provides an introduction to the current understanding of these varied issues as we enter the era of widespread lung cancer screening.

A comparison of axial versus coronal image viewing in computer-aided detection of lung nodules on CT

PURPOSE

To compare primarily viewing axial images (Axial mode) versus coronal reconstruction images (Coronal mode) in computer-aided detection (CAD) of lung nodules on multidetector computed tomography (CT) in terms of detection performance and reading time.

MATERIALS AND METHODS

Sixty CT data sets from two institutions were collected prospectively. Ten observers (6 radiologists, 4 pulmonologists) with varying degrees of experience interpreted the data sets using CAD as a second reader (performing nodule detection first without then with aid). The data sets were interpreted twice, once each for Axial and Coronal modes, in two sessions held 4 weeks apart. Jackknife free-response receiver-operating characteristic analysis was used to compare detection performances in the two modes.

RESULTS

Mean figure-of-merit values with and without aid were 0.717 and 0.684 in Axial mode and 0.702 and 0.671 in Coronal mode; use of CAD significantly increased the performance of observers in both modes (P < 0.01). Mean reading times for radiologists did not significantly differ between Axial (156 ± 74 s) and Coronal mode (164 ± 69 s; P = 0.08). Mean reading times for pulmonologists were significantly lower in Coronal (112 ± 53 s) than in Axial mode (130 ± 80 s; P < 0.01).

CONCLUSION

There was no statistically significant difference between Axial and Coronal modes for lung nodule detection with CAD.

Improving the radiologist-CAD interaction: designing for appropriate trust

Computer-aided diagnosis (CAD) has great potential to improve radiologists' diagnostic performance. However, the reported performance of the radiologist-CAD team is lower than what might be expected based on the performance of the radiologist and the CAD system in isolation. This indicates that the interaction between radiologists and the CAD system is not optimal. An important factor in the interaction between humans and automated aids (such as CAD) is trust. Suboptimal performance of the human-automation team is often caused by an inappropriate level of trust in the automation. In this review, we examine the role of trust in the radiologist-CAD interaction and suggest ways to improve the output of the CAD system so that it allows radiologists to calibrate their trust in the CAD system more effectively. Observer studies of the CAD systems show that radiologists often have an inappropriate level of trust in the CAD system. They sometimes under-trust CAD, thereby reducing its potential benefits, and sometimes over-trust it, leading to diagnostic errors they would not have made without CAD. Based on the literature on trust in human-automation interaction and the results of CAD observer studies, we have identified four ways to improve the output of CAD so that it allows radiologists to form a more appropriate level of trust in CAD. Designing CAD systems for appropriate trust is important and can improve the performance of the radiologist-CAD team. Future CAD research and development should acknowledge the importance of the radiologist-CAD interaction, and specifically the role of trust therein, in order to create the perfect artificial partner for the radiologist. This review focuses on the role of trust in the radiologist-CAD interaction. The aim of the review is to encourage CAD developers to design for appropriate trust and thereby improve the performance of the radiologist-CAD team.

Role of the texture features of images in the diagnosis of solitary pulmonary nodules in different sizes

OBJECTIVE

To explore the role of the texture features of images in the diagnosis of solitary pulmonary nodules (SPNs) in different sizes.

MATERIALS AND METHODS

A total of 379 patients with pathologically confirmed SPNs were enrolled in this study. They were divided into three groups based on the SPN sizes: ≤10, 11-20, and >20 mm. Their texture features were segmented and extracted. The differences in the image features between benign and malignant SPNs were compared. The SPNs in these three groups were determined and analyzed with the texture features of images.

RESULTS

These 379 SPNs were successfully segmented using the 2D Otsu threshold method and the self-adaptive threshold segmentation method. The texture features of these SPNs were obtained using the method of grey level co-occurrence matrix (GLCM). Of these 379 patients, 120 had benign SPNs and 259 had malignant SPNs. The entropy, contrast, energy, homogeneity, and correlation were 3.5597±0.6470, 0.5384±0.2561, 0.1921±0.1256, 0.8281±0.0604, and 0.8748±0.0740 in the benign SPNs and 3.8007±0.6235, 0.6088±0.2961, 0.1673±0.1070, 0.7980±0.0555, and 0.8550±0.0869 in the malignant SPNs (all P<0.05). The sensitivity, specificity, and accuracy of the texture features of images were 83.3%, 90.0%, and 86.8%, respectively, for SPNs sized ≤10 mm, and were 86.6%, 88.2%, and 87.1%, respectively, for SPNs sized 
11-20 mm and 94.7%, 91.8%, and 93.9%, respectively, for SPNs sized >20 mm.

CONCLUSIONS

The entropy and contrast of malignant pulmonary nodules have been demonstrated to be higher in comparison to those of benign pulmonary nodules, while the energy, homogeneity correlation of malignant pulmonary nodules are lower than those of benign pulmonary nodules. The texture features of images can reflect the tissue features and have high sensitivity, specificity, and accuracy in differentiating SPNs. The sensitivity and accuracy increase for larger SPNs.

Optimal dose levels in screening chest CT for unimpaired detection and volumetry of lung nodules, with and without computer assisted detection at minimal patient radiation

OBJECTIVES

The aim of this phantom study was to minimize the radiation dose by finding the best combination of low tube current and low voltage that would result in accurate volume measurements when compared to standard CT imaging without significantly decreasing the sensitivity of detecting lung nodules both with and without the assistance of CAD.

METHODS

An anthropomorphic chest phantom containing artificial solid and ground glass nodules (GGNs, 5-12 mm) was examined with a 64-row multi-detector CT scanner with three tube currents of 100, 50 and 25 mAs in combination with three tube voltages of 120, 100 and 80 kVp. This resulted in eight different protocols that were then compared to standard CT sensitivity (100 mAs/120 kVp). For each protocol, at least 127 different nodules were scanned in 21-25 phantoms. The nodules were analyzed in two separate sessions by three independent, blinded radiologists and computer-aided detection (CAD) software.

RESULTS

The mean sensitivity of the radiologists for identifying solid lung nodules on a standard CT was 89.7% ± 4.9%. The sensitivity was not significantly impaired when the tube and current voltage were lowered at the same time, except at the lowest exposure level of 25 mAs/80 kVp [80.6% ± 4.3% (p = 0.031)]. Compared to the standard CT, the sensitivity for detecting GGNs was significantly lower at all dose levels when the voltage was 80 kVp; this result was independent of the tube current. The CAD significantly increased the radiologists' sensitivity for detecting solid nodules at all dose levels (5-11%). No significant volume measurement errors (VMEs) were documented for the radiologists or the CAD software at any dose level.

CONCLUSIONS

Our results suggest a CT protocol with 25 mAs and 100 kVp is optimal for detecting solid and ground glass nodules in lung cancer screening. The use of CAD software is highly recommended at all dose levels.

Impact of a computer-aided detection (CAD) system integrated into a picture archiving and communication system (PACS) on reader sensitivity and efficiency for the detection of lung nodules in thoracic CT exams

The objective of this study is to assess the impact on nodule detection and efficiency using a computer-aided detection (CAD) device seamlessly integrated into a commercially available picture archiving and communication system (PACS). Forty-eight consecutive low-dose thoracic computed tomography studies were retrospectively included from an ongoing multi-institutional screening study. CAD results were sent to PACS as a separate image series for each study. Five fellowship-trained thoracic radiologists interpreted each case first on contiguous 5 mm sections, then evaluated the CAD output series (with CAD marks on corresponding axial sections). The standard of reference was based on three-reader agreement with expert adjudication. The time to interpret CAD marking was automatically recorded. A total of 134 true-positive nodules, measuring 3 mm and larger were included in our study; with 85 ≥ 4 and 50 ≥ 5 mm in size. Readers detection improved significantly in each size category when using CAD, respectively, from 44 to 57 % for ≥3 mm, 48 to 61 % for ≥4 mm, and 44 to 60 % for ≥5 mm. CAD stand-alone sensitivity was 65, 68, and 66 % for nodules ≥3, ≥4, and ≥5 mm, respectively, with CAD significantly increasing the false positives for two readers only. The average time to interpret and annotate a CAD mark was 15.1 s, after localizing it in the original image series. The integration of CAD into PACS increases reader sensitivity with minimal impact on interpretation time and supports such implementation into daily clinical practice.

Development and evaluation of a software tool for the generation of virtual liver lesions in multidetector-row CT datasets

RATIONALE AND OBJECTIVES

Development and evaluation of a software tool for the insertion of simulated hypodense liver lesions in multidetector-row computed tomography (CT) datasets.

MATERIALS AND METHODS

Forty software-generated hypodense liver lesions were inserted at random locations in 20 CT datasets by using the "alpha blending" technique and compared with 40 real metastatic lesions. The location, diameter (5-20 mm) and density of the simulated lesions were individually adjusted to closely resemble real lesions in each patient. Three blinded readers evaluated all 80 lesions twice in a 2-week interval using a five-point Likert confidence scale under standardized conditions. Nonparametric tests were used to statistically evaluate possible differences in scoring between real and simulated lesions. The correctness of the observer rating for real and simulated lesions was compared to chance distribution using the chi-squared statistics. The inter- and intraobserver variability was determined using Kendall's coefficient of concordance.

RESULTS

The observer study did not reveal significant differences between the scoring for real versus simulated lesions for any of the readers (P > .05). The distribution of correct and false scoring of the lesions was not significantly different from chance distribution (P > .05). Inter- and intraobserver agreement was poor (Kendall W coefficient = 0.12/0.13).

CONCLUSION

The proposed algorithm is suitable for creating realistic virtual liver lesions in CT datasets.

Quantitative analysis of multiparametric prostate MR images: differentiation between prostate cancer and normal tissue and correlation with Gleason score--a computer-aided diagnosis development study

PURPOSE

To evaluate the potential utility of a number of parameters obtained at T2-weighted, diffusion-weighted, and dynamic contrast material-enhanced multiparametric magnetic resonance (MR) imaging for computer-aided diagnosis (CAD) of prostate cancer and assessment of cancer aggressiveness.

MATERIALS AND METHODS

In this institutional review board-approved HIPAA-compliant study, multiparametric MR images were acquired with an endorectal coil in 48 patients with prostate cancer (median age, 62.5 years; age range, 44-73 years) who subsequently underwent prostatectomy. A radiologist and a pathologist identified 104 regions of interest (ROIs) (61 cancer ROIs, 43 normal ROIs) based on correlation of histologic and MR findings. The 10th percentile and average apparent diffusion coefficient (ADC) values, T2-weighted signal intensity histogram skewness, and Tofts K(trans) were analyzed, both individually and combined, via linear discriminant analysis, with receiver operating characteristic curve analysis with area under the curve (AUC) as figure of merit, to distinguish cancer foci from normal foci. Spearman rank-order correlation (ρ) was calculated between cancer foci Gleason score (GS) and image features.

RESULTS

AUC (maximum likelihood estimate ± standard error) values in the differentiation of prostate cancer from normal foci of 10th percentile ADC, average ADC, T2-weighted skewness, and K(trans) were 0.92 ± 0.03, 0.89 ± 0.03, 0.86 ± 0.04, and 0.69 ± 0.04, respectively. The combination of 10th percentile ADC, average ADC, and T2-weighted skewness yielded an AUC value for the same task of 0.95 ± 0.02. GS correlated moderately with 10th percentile ADC (ρ = -0.34, P = .008), average ADC (ρ = -0.30, P = .02), and K(trans) (ρ = 0.38, P = .004).

CONCLUSION

The combination of 10th percentile ADC, average ADC, and T2-weighted skewness with CAD is promising in the differentiation of prostate cancer from normal tissue. ADC image features and K(trans) moderately correlate with GS.

Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume

Objective

To evaluate performance of computer-aided detection (CAD) beyond double reading for pulmonary nodules on low-dose computed tomography (CT) by nodule volume.

Methods

A total of 400 low-dose chest CT examinations were randomly selected from the NELSON lung cancer screening trial. CTs were evaluated by two independent readers and processed by CAD. A total of 1,667 findings marked by readers and/or CAD were evaluated by a consensus panel of expert chest radiologists. Performance was evaluated by calculating sensitivity of pulmonary nodule detection and number of false positives, by nodule characteristics and volume.

Results

According to the screening protocol, 90.9 % of the findings could be excluded from further evaluation, 49.2 % being small nodules (less than 50 mm³). Excluding small nodules reduced false-positive detections by CAD from 3.7 to 1.9 per examination. Of 151 findings that needed further evaluation, 33 (21.9 %) were detected by CAD only, one of them being diagnosed as lung cancer the following year. The sensitivity of nodule detection was 78.1 % for double reading and 96.7 % for CAD. A total of 69.7 % of nodules undetected by readers were attached nodules of which 78.3 % were vessel-attached.

Conclusions

CAD is valuable in lung cancer screening to improve sensitivity of pulmonary nodule detection beyond double reading, at a low false-positive rate when excluding small nodules.

Key Points

• Computer-aided detection (CAD) has known advantages for computed tomography (CT).

• Combined CAD/nodule size cut-off parameters assist CT lung cancer screening.

• This combination improves the sensitivity of pulmonary nodule detection by CT.

• It increases the positive predictive value for cancer detection.

Influence of nodule detection software on radiologists' confidence in identifying pulmonary nodules with computed tomography

PURPOSE

With advances in technology, detection of small pulmonary nodules is increasing. Nodule detection software (NDS) has been developed to assist radiologists with pulmonary nodule diagnosis. Although it may increase sensitivity for small nodules, often there is an accompanying increase in false-positive findings. We designed a study to examine the extent to which computed tomography (CT) NDS influences the confidence of radiologists in identifying small pulmonary nodules.

MATERIALS AND METHODS

Eight radiologists (readers) with different levels of experience examined thoracic CT scans of 131 cases and identified all the clinically relevant pulmonary nodules. The reference standard was established by an expert, dedicated thoracic radiologist. For each nodule, the readers recorded nodule size, density, location, and confidence level. Two weeks (or more) later, the readers reinterpreted the same scans; however, this time they were provided marks, when present, as indicated by NDS and asked to reassess their level of confidence. The effect of NDS on changes in reader confidence was assessed using multivariable generalized linear regression models.

RESULTS

A total of 327 unique nodules were identified. Declines in confidence were significantly (P<0.05) associated with the absence of an NDS mark and smaller nodules (odds ratio=71.0, 95% confidence interval =14.8-339.7). Among nodules with pre-NDS confidence less than 100%, increases in confidence were significantly (P<0.05) associated with the presence of an NDS mark (odds ratio=6.0, 95% confidence interval =2.7-13.6) and larger nodules. Secondary findings showed that NDS did not improve reader diagnostic accuracy.

CONCLUSION

Although in this study NDS does not seem to enhance reader accuracy, the confidence of the radiologists in identifying small pulmonary nodules with CT is greatly influenced by NDS.

Evaluation of a method of computer-aided detection (CAD) of pulmonary nodules with computed tomography

PURPOSE

The authors sought to compare the sensitivity and reading time obtained using computer-aided detection (CAD) software as second reader (SR) or concurrent reader (CR) in the identification of pulmonary nodules.

MATERIALS AND METHODS

Unenhanced CT scans of 100 consecutive cancer patients were retrospectively reviewed by four readers to identify all solid, noncalcified pulmonary nodules ranging from 3 to 30 mm in diameter. The sensitivity and reading time of each reader and of CAD alone were calculated at 3-mm and 5-mm thresholds with respect to the reference standard, consisting of a consensus reading by the four radiologists involved in the study. The McNemar test was used to compare the sensitivities obtained by reading without CAD (readers 1 and 2), with CAD as SR (readers 1 and 2 with a 2-month delay), and with CAD as CR (readers 3 and 4). The paired Student's t test was used to compare reading times. A value of p<0.05 was considered statistically significant.

RESULTS

A total of 258 and 224 nodules were identified at 3-mm and 5-mm thresholds, respectively. The sensitivity of CAD alone was 62.79% and 67.41% at the 3-mm and 5-mm threshold values respectively, with 4.15 and 2.96 false-positive findings per examination. CAD as SR produced a significant increase in sensitivity (p<0.001) in nodule detection with respect to reading without CAD both at 3 mm (12.01%) and 5 mm (10.04%); the average increase in sensitivity obtained when comparing CAD as SR to CAD as CR was statistically significant (p<0.025) both at the 3-mm (5.35%) and 5-mm (4.68%) thresholds. CAD as CR produced a nonsignificant increase in sensitivity compared with reading without CAD (p>0.05). Mean reading time using CAD as SR (330 s) was significantly longer than reading without CAD (135 s, p<0.001) and reading with CAD as CR (195 s, p<0.025).

CONCLUSIONS

The use of CAD as CR, without any significant increase in reading time, produces no significant increase in sensitivity in pulmonary nodule detection when compared with reading without CAD (p>0.05); CAD as SR, at the cost of longer reading times, increases sensitivity when compared with reading without CAD (p<0.001) or with CAD as CR (p<0.025).

Assessing operating characteristics of CAD algorithms in the absence of a gold standard

PURPOSE

The authors examine potential bias when using a reference reader panel as "gold standard" for estimating operating characteristics of CAD algorithms for detecting lesions. As an alternative, the authors propose latent class analysis (LCA), which does not require an external gold standard to evaluate diagnostic accuracy.

METHODS

A binomial model for multiple reader detections using different diagnostic protocols was constructed, assuming conditional independence of readings given true lesion status. Operating characteristics of all protocols were estimated by maximum likelihood LCA. Reader panel and LCA based estimates were compared using data simulated from the binomial model for a range of operating characteristics. LCA was applied to 36 thin section thoracic computed tomography data sets from the Lung Image Database Consortium (LIDC): Free search markings of four radiologists were compared to markings from four different CAD assisted radiologists. For real data, bootstrap-based resampling methods, which accommodate dependence in reader detections, are proposed to test of hypotheses of differences between detection protocols.

RESULTS

In simulation studies, reader panel based sensitivity estimates had an average relative bias (ARB) of -23% to -27%, significantly higher (p-value < 0.0001) than LCA (ARB--2% to -6%). Specificity was well estimated by both reader panel (ARB -0.6% to -0.5%) and LCA (ARB 1.4%-0.5%). Among 1145 lesion candidates LIDC considered, LCA estimated sensitivity of reference readers (55%) was significantly lower (p-value 0.006) than CAD assisted readers' (68%). Average false positives per patient for reference readers (0.95) was not significantly lower (p-value 0.28) than CAD assisted readers' (1.27).

CONCLUSIONS

Whereas a gold standard based on a consensus of readers may substantially bias sensitivity estimates, LCA may be a significantly more accurate and consistent means for evaluating diagnostic accuracy.

Lung nodule computer-aided detection as a second reader: influence on radiology residents

OBJECTIVE

The purpose of this study was to evaluate the use of a computed tomographic lung nodule computer-aided detection (CAD) software as a second reader for radiology residents.

METHODS

The study involved 110 cases from 4 sites. Three expert radiologists identified nodules that were 4 to 30 mm in maximum diameter to form the ground truth. These cases were then interpreted by 6 board-certified radiologists and 6 radiology residents. The residents read each case without and then with a CAD software (Lung Nodule Assesment, Extended Brilliance Workspace; Philips Healthcare, Highlands Heights, OH) to identify nodules that were 4 to 30 mm in maximum diameter.

RESULTS

The experts identified 91 nodules as the ground truth for the study. The mean sensitivity of the 6 board-certified radiologists was 89%. The mean sensitivity of the residents was 85% without the CAD and 90% (P < 0.05) with the CAD as a second reader.

CONCLUSIONS

The CAD software can help improve the sensitivity of residents in the detection of pulmonary nodules on computed tomography, making them comparable with board-certified radiologists.

CAD (computed-aided detection) and CADx (computer aided diagnosis) systems in identifying and characterising lung nodules on chest CT: overview of research, developments and new prospects

Computer-aided detection (CAD) systems allow the automatic identification of lung nodules on chest computed tomography (CT), providing a second opinion to the radiologist's judgement and a volumetric evaluation of lesions - a very important aspect in oncological patients. The natural evolution of these systems has led to the introduction of computer-aided diagnosis (CADx) systems, which are able not only to identify nodules but also to characterise them by determining a likelihood of malignancy or benignity. The aim of this article is to describe the main technical principles of CAD and CADx systems, their applicability and influence in clinical practice and new prospects for their future development.

Chest tomosynthesis: technical principles and clinical update

Digital tomosynthesis is a radiographic technique that can produce an arbitrary number of section images of a patient from a single pass of the X-ray tube. It utilizes a conventional X-ray tube, a flat-panel detector, a computer-controlled tube mover, and special reconstruction algorithms to produce section images. While it does not have the depth resolution of computed tomography (CT), tomosynthesis provides some of the tomographic benefits of CT but at lower cost and radiation dose than CT. Compared to conventional chest radiography, chest tomosynthesis results in improved visibility of normal structures such as vessels, airway and spine. By reducing visual clutter from overlying normal anatomy, it also enhances detection of small lung nodules. This review article outlines the components of a tomosynthesis system, discusses results regarding improved lung nodule detection from the recent literature, and presents examples of nodule detection from a clinical trial in human subjects. Possible implementation strategies for use in clinical chest imaging are discussed.

A novel multithreshold method for nodule detection in lung CT

Multislice computed tomography (MSCT) is a valuable tool for lung cancer detection, thanks to its ability to identify noncalcified nodules of small size (from about 3 mm). Due to the large number of images generated by MSCT, there is much interest in developing computer-aided detection (CAD) systems that could assist radiologists in the lung nodule detection task. A complete multistage CAD system, including lung boundary segmentation, regions of interest (ROIs) selection, feature extraction, and false positive reduction is presented. The selection of ROIs is based on a multithreshold surface-triangulation approach. Surface triangulation is performed at different threshold values, varying from a minimum to a maximum value in a wide range. At a given threshold value, a ROI is defined as the volume inside a connected component of the triangulated isosurface. The evolution of a ROI as a function of the threshold can be represented by a treelike structure. A multithreshold ROI is defined as a path on this tree, which starts from a terminal ROI and ends on the root ROI. For each ROI, the volume, surface area, roundness, density, and moments of inertia are computed as functions of the threshold and used as input to a classification system based on artificial neural networks. The method is suitable to detect different types of nodules, including juxta-pleural nodules and nodules connected to blood vessels. A training set of 109 low-dose MSCT scans made available by the Pisa center of the Italung-CT trial and annotated by expert radiologists was used for the algorithm design and optimization. The system performance was tested on an independent set of 23 low-dose MSCT scans coming from the Pisa Italung-CT center and on 83 scans made available by the Lung Image Database Consortium (LIDC) annotated by four expert radiologists. On the Italung-CT test set, for nodules having a diameter greater than or equal to 3 mm, the system achieved 84% and 71% sensitivity at false positive/scan rates of 10 and 4, respectively. For nodules having a diameter greater than or equal to 4 mm, the sensitivities were 97% and 80% at false positive/scan rates of 10 and 4, respectively. On the LIDC data set, the system achieved a 79% sensitivity at a false positive/scan rate of 4 in the detection of nodules with a diameter greater than or equal to 3 mm that have been annotated by all four radiologists.

Computer-aided detection (CAD) of lung nodules in CT scans: radiologist performance and reading time with incremental CAD assistance

OBJECTIVE

The diagnostic performance of radiologists using incremental CAD assistance for lung nodule detection on CT and their temporal variation in performance during CAD evaluation was assessed.

METHODS

CAD was applied to 20 chest multidetector-row computed tomography (MDCT) scans containing 190 non-calcified > or =3-mm nodules. After free search, three radiologists independently evaluated a maximum of up to 50 CAD detections/patient. Multiple free-response ROC curves were generated for free search and successive CAD evaluation, by incrementally adding CAD detections one at a time to the radiologists' performance.

RESULTS

The sensitivity for free search was 53% (range, 44%-59%) at 1.15 false positives (FP)/patient and increased with CAD to 69% (range, 59-82%) at 1.45 FP/patient. CAD evaluation initially resulted in a sharp rise in sensitivity of 14% with a minimal increase in FP over a time period of 100 s, followed by flattening of the sensitivity increase to only 2%. This transition resulted from a greater prevalence of true positive (TP) versus FP detections at early CAD evaluation and not by a temporal change in readers' performance. The time spent for TP (9.5 s +/- 4.5 s) and false negative (FN) (8.4 s +/- 6.7 s) detections was similar; FP decisions took two- to three-times longer (14.4 s +/- 8.7 s) than true negative (TN) decisions (4.7 s +/- 1.3 s).

CONCLUSIONS

When CAD output is ordered by CAD score, an initial period of rapid performance improvement slows significantly over time because of non-uniformity in the distribution of TP CAD output and not to a changing reader performance over time.

Commercially available computer-aided detection system for pulmonary nodules on thin-section images using 64 detectors-row CT: preliminary study of 48 cases

RATIONALE AND OBJECTIVES

Most studies of computer-aided detection (CAD) for pulmonary nodules have focused on solid nodule detection. The aim of this study was to evaluate the performance of a commercially available CAD system in the detection of pulmonary nodules with or without ground-glass opacity (GGO) using 64-detector-row computed tomography compared to visual interpretation.

MATERIALS AND METHODS

Computed tomographic examinations were performed on 48 patients with existing or suspicious pulmonary nodules on chest radiography. Three radiologists independently reported the location and pattern (GGO, solid, or part solid) of each nodule candidate on computed tomographic scans, assigned each a confidence score, and then analyzed all scans using the CAD system. A reference standard was established by a consensus panel of different radiologists, who found 229 noncalcified nodules with diameters > or = 4 mm. True-positive and false-positive results and confidence levels were used to generate jackknife alternative free-response receiver-operating characteristic plots.

RESULTS

The sensitivity of GGO for 3 radiologists (60%-80%) was significantly higher than that for the CAD system (21%) (McNemar's test, P < .0001). For overall and solid nodules, the figure-of-merit values without and with the CAD system were significantly different (P = .005-.04) on jackknife alternative free-response receiver-operating characteristic analysis. For GGO and part-solid nodules, the figure-of-merit values with the CAD system were greater than those without the CAD system, indicating no significant differences.

CONCLUSION

Radiologists are significantly superior to this CAD system in the detection of GGO, but the CAD system can still play a complementary role in detecting nodules with or without GGO.