Comparison of dual-energy subtraction and electronic bone suppression combined with computer-aided detection on chest radiographs: effect on human observers' performance in nodule detection

Imaging of Solid Pulmonary Nodules

Early detection with accurate classification of solid pulmonary nodules is critical in reducing lung cancer morbidity and mortality. Computed tomography (CT) remains the most widely used imaging examination for pulmonary nodule evaluation; however, other imaging modalities, such as PET/CT and MRI, are increasingly used for nodule characterization. Current advances in solid nodule imaging are largely due to developments in machine learning, including automated nodule segmentation and computer-aided detection. This review explores current multi-modality solid pulmonary nodule detection and characterization with discussion of radiomics and risk prediction models.

Review on chest pathogies detection systems using deep learning techniques

Chest radiography is the standard and most affordable way to diagnose, analyze, and examine different thoracic and chest diseases. Typically, the radiograph is examined by an expert radiologist or physician to decide about a particular anomaly, if exists. Moreover, computer-aided methods are used to assist radiologists and make the analysis process accurate, fast, and more automated. A tremendous improvement in automatic chest pathologies detection and analysis can be observed with the emergence of deep learning. The survey aims to review, technically evaluate, and synthesize the different computer-aided chest pathologies detection systems. The state-of-the-art of single and multi-pathologies detection systems, which are published in the last five years, are thoroughly discussed. The taxonomy of image acquisition, dataset preprocessing, feature extraction, and deep learning models are presented. The mathematical concepts related to feature extraction model architectures are discussed. Moreover, the different articles are compared based on their contributions, datasets, methods used, and the results achieved. The article ends with the main findings, current trends, challenges, and future recommendations.

Artificial Intelligence (AI) for Lung Nodules, From the AJR Special Series on AI Applications

Interest in artificial intelligence (AI) applications for lung nodules continues to grow among radiologists, particularly with the expanding eligibility criteria and clinical utilization of lung cancer screening CT. AI has been heavily investigated for detecting and characterizing lung nodules and for guiding prognostic assessment. AI tools have also been used for image postprocessing (e.g., rib suppression on radiography or vessel suppression on CT) and for noninterpretive aspects of reporting and workflow, including management of nodule follow-up. Despite growing interest in and rapid development of AI tools and FDA approval of AI tools for pulmonary nodule evaluation, integration into clinical practice has been limited. Challenges to clinical adoption have included concerns about generalizability, regulatory issues, technical hurdles in implementation, and human skepticism. Further validation of AI tools for clinical use and demonstration of benefit in terms of patient-oriented outcomes also are needed. This article provides an overview of potential applications of AI tools in the imaging evaluation of lung nodules and discusses the challenges faced by practices interested in clinical implementation of such tools.

Bone Suppression on Chest Radiographs for Pulmonary Nodule Detection: Comparison between a Generative Adversarial Network and Dual-Energy Subtraction

OBJECTIVE

To compare the effects of bone suppression imaging using deep learning (BSp-DL) based on a generative adversarial network (GAN) and bone subtraction imaging using a dual energy technique (BSt-DE) on radiologists' performance for pulmonary nodule detection on chest radiographs (CXRs).

MATERIALS AND METHODS

A total of 111 adults, including 49 patients with 83 pulmonary nodules, who underwent both CXR using the dual energy technique and chest CT, were enrolled. Using CT as a reference, two independent radiologists evaluated CXR images for the presence or absence of pulmonary nodules in three reading sessions (standard CXR, BSt-DE CXR, and BSp-DL CXR). Person-wise and nodule-wise performances were assessed using receiver-operating characteristic (ROC) and alternative free-response ROC (AFROC) curve analyses, respectively. Subgroup analyses based on nodule size, location, and the presence of overlapping bones were performed.

RESULTS

BSt-DE with an area under the AFROC curve (AUAFROC) of 0.996 and 0.976 for readers 1 and 2, respectively, and BSp-DL with AUAFROC of 0.981 and 0.958, respectively, showed better nodule-wise performance than standard CXR (AUAFROC of 0.907 and 0.808, respectively; p ≤ 0.005). In the person-wise analysis, BSp-DL with an area under the ROC curve (AUROC) of 0.984 and 0.931 for readers 1 and 2, respectively, showed better performance than standard CXR (AUROC of 0.915 and 0.798, respectively; p ≤ 0.011) and comparable performance to BSt-DE (AUROC of 0.988 and 0.974; p ≥ 0.064). BSt-DE and BSp-DL were superior to standard CXR for detecting nodules overlapping with bones (p < 0.017) or in the upper/middle lung zone (p < 0.017). BSt-DE was superior (p < 0.017) to BSp-DL in detecting peripheral and sub-centimeter nodules.

CONCLUSION

BSp-DL (GAN-based bone suppression) showed comparable performance to BSt-DE and can improve radiologists' performance in detecting pulmonary nodules on CXRs. Nevertheless, for better delineation of small and peripheral nodules, further technical improvements are required.

Improved detection of solitary pulmonary nodules on radiographs compared with deep bone suppression imaging

Background

The present study aimed to investigate whether deep bone suppression imaging (BSI) could increase the diagnostic performance for solitary pulmonary nodule detection compared with digital tomosynthesis (DTS), dual-energy subtraction (DES) radiography, and conventional chest radiography (CCR).

Methods

A total of 256 patients (123 with a solitary pulmonary nodule, 133 with normal findings) were included in the study. The confidence score of 6 observers determined the presence or absence of pulmonary nodules in each patient. These were first analyzed using a CCR image, then with CCR plus deep BSI, then with CCR plus DES radiography, and finally with DTS images. Receiver-operating characteristic curves were used to evaluate the performance of the 6 observers in the detection of pulmonary nodules.

Results

For the 6 observers, the average area under the curve improved significantly from 0.717 with CCR to 0.848 with CCR plus deep BSI (P<0.01), 0.834 with CCR plus DES radiography (P<0.01), and 0.939 with DTS (P<0.01). Comparisons between CCR and CCR plus deep BSI found that the sensitivities of the assessments by the 3 residents increased from 53.2% to 69.5% (P=0.014) for nodules located in the upper lung field, from 30.6% to 44.6% (P=0.015) for nodules that were partially/completely obscured by the bone, and from 33.2% to 45.8% (P=0.006) for nodules <10 mm.

Conclusions

The deep BSI technique can significantly increase the sensitivity of radiology residents for solitary pulmonary nodules compared with CCR. Increased detection was seen mainly for smaller nodules, nodules with partial/complete obscuration, and nodules located in the upper lung field.

Chest radiographs and machine learning - Past, present and future

Despite its simple acquisition technique, the chest X-ray remains the most common first-line imaging tool for chest assessment globally. Recent evidence for image analysis using modern machine learning points to possible improvements in both the efficiency and the accuracy of chest X-ray interpretation. While promising, these machine learning algorithms have not provided comprehensive assessment of findings in an image and do not account for clinical history or other relevant clinical information. However, the rapid evolution in technology and evidence base for its use suggests that the next generation of comprehensive, well-tested machine learning algorithms will be a revolution akin to early advances in X-ray technology. Current use cases, strengths, limitations and applications of chest X-ray machine learning systems are discussed.

Lung cancer screening by single-shot dual-energy subtraction using flat-panel detector

Purpose

The purpose of this study was to evaluate the usefulness of single-shot dual-energy subtraction (DES) method using a flat-panel detector for lung cancer screening

Materials and methods

The subjects were 13,315 residents (5801 males and 7514 females) aged 50 years or older (50–97 years, with an intermediate value of 68 years) who underwent lung cancer screening for a period of 1 year and 6 months from January 2019 to June 2020. We investigated whether the number of lung cancers detected, the detection rate, and the rate of required scrutiny changed, when DES images were added to the judgment based on conventional chest radiography.

Results

When DES images were added, the number and percentage of cancer detection increased from 16 (0.12%) to 23 (0.17%) (P < 0.05). Five of the newly detected 7 lung cancers were in the early stages of resectable cancer. The rate of participants requiring scrutiny increased slightly from 1.1 to 1.3%.

Conclusion

DES method improved the detection of lung cancer in screening. The increase in the percentage of participants requiring scrutiny was negligible.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.

Dual energy imaging in cardiothoracic pathologies: A primer for radiologists and clinicians

Recent advances in dual-energy imaging techniques, dual-energy subtraction radiography (DESR) and dual-energy CT (DECT), offer new and useful additional information to conventional imaging, thus improving assessment of cardiothoracic abnormalities. DESR facilitates detection and characterization of pulmonary nodules. Other advantages of DESR include better depiction of pleural, lung parenchymal, airway and chest wall abnormalities, detection of foreign bodies and indwelling devices, improved visualization of cardiac and coronary artery calcifications helping in risk stratification of coronary artery disease, and diagnosing conditions like constrictive pericarditis and valvular stenosis. Commercially available DECT approaches are classified into emission based (dual rotation/spin, dual source, rapid kilovoltage switching and split beam) and detector-based (dual layer) systems. DECT provide several specialized image reconstructions. Virtual non-contrast images (VNC) allow for radiation dose reduction by obviating need for true non contrast images, low energy virtual mono-energetic images (VMI) boost contrast enhancement and help in salvaging otherwise non-diagnostic vascular studies, high energy VMI reduce beam hardening artifacts from metallic hardware or dense contrast material, and iodine density images allow quantitative and qualitative assessment of enhancement/iodine distribution. The large amount of data generated by DECT can affect interpreting physician efficiency but also limit clinical adoption of the technology. Optimization of the existing workflow and streamlining the integration between post-processing software and picture archiving and communication system (PACS) is therefore warranted.

Value of bone suppression software in chest radiographs for improving image quality and reducing radiation dose

OBJECTIVES

To compare image quality and radiation dose between dual-energy subtraction (DES)-based bone suppression images (D-BSIs) and software-based bone suppression images (S-BSIs).

METHODS

Chest radiographs (CXRs) of forty adult patients were obtained with the two X-ray devices, one with DES and one with bone suppression software. Three image quality metrics (relative mean absolute error (RMAE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM)) between original CXR and BSI for each of D-BSI and S-SBI groups were calculated for each bone and soft tissue areas. Two readers rated the visual image quality for original CXR and BSI for each of D-BSI and S-SBI groups. The dose area product (DAP) values were recorded. Paired t test was used to compare the image quality and DAP values between D-BSI and S-BSI groups.

RESULTS

In bone areas, S-BSIs had better SSIM values than D-BSI (94.57 vs. 87.77) but worse RMAE and PSNR values (0.50 vs. 0.20; 20.93 vs. 34.37) (all p < 0.001). In soft tissue areas, S-BSIs had better SSIM values than D-BSI (97.56 vs. 91.42) but similar RMAE and PSNR values (0.29 vs. 0.27; 31.35 vs. 29.87) (all p < 0.001). Both readers gave S-BSIs significantly higher image quality scores than D-BSI (p < 0.001). The mean DAP in software-related images (0.98 dGy·cm²) was significantly lower than that in the DES-related images (1.48 dGy·cm²) (p < 0.001).

CONCLUSION

Bone suppression software significantly improved the image quality of bone suppression images with a relatively lower radiation dose, compared with dual-energy subtraction technique.

KEY POINTS

• Bone suppression software preserves structure similarity of soft tissues better than dual-energy subtraction technique in bone suppression images. • Bone suppression software achieves superior image quality for lung lesions than dual-energy subtraction technique in bone suppression images. • Bone suppression software can decrease the radiation dose over the hardware-based image processing technique.

Putting artificial intelligence (AI) on the spot: machine learning evaluation of pulmonary nodules

Lung cancer remains the leading cause of cancer related death world-wide despite advances in treatment. This largely relates to the fact that many of these patients already have advanced diseases at the time of initial diagnosis. As most lung cancers present as nodules initially, an accurate classification of pulmonary nodules as early lung cancers is critical to reducing lung cancer morbidity and mortality. There have been significant recent advances in artificial intelligence (AI) for lung nodule evaluation. Deep learning (DL) and convolutional neural networks (CNNs) have shown promising results in pulmonary nodule detection and have also excelled in segmentation and classification of pulmonary nodules. This review aims to provide an overview of progress that has been made in AI recently for pulmonary nodule detection and characterization with the ultimate goal of lung cancer prediction and classification while outlining some of the pitfalls and challenges that remain to bring such advancements to routine clinical use.

Extravalidation and reproducibility results of a commercial deep learning-based automatic detection algorithm for pulmonary nodules on chest radiographs at tertiary hospital

INTRODUCTION

To extra validate and evaluate the reproducibility of a commercial deep convolutional neural network (DCNN) algorithm for pulmonary nodules on chest radiographs (CRs) and to compare its performance with radiologists.

METHODS

This retrospective study enrolled 434 CRs (normal to abnormal ratio, 246:188) from 378 patients that visited a tertiary hospital. DCNN performance was compared with two radiology residents and two thoracic radiologists. Abnormality assessment (using the area under the receiver operating ch3cteristics (AUROC)) and nodule detection (using jackknife alternative free-response ROC (JAFROC)) were compared among three groups (DCNN only, radiologist without DCNN and radiologist with DCNN). A subset of 56 paired cases, having two CRs taken within a 7-day period, were assessed for intraobserver reproducibility using the intraclass correlation coefficient. Independent characteristics of pulmonary nodules detected by DCNN were assessed by multiple logistic regression analysis.

RESULTS

The AUROC for abnormality detection for the three groups were 0.87, 0.93 and 0.96, respectively (P < 0.05), whereas the JAFROC analysis of nodule detection was 0.926, 0.929 and 0.964. Reproducibility for the three groups was 0.80, 0.67 and 0.80, which shows an increase in radiologists using DCNN (P < 0.05). Nodules detected by DCNN were more solid, round-shaped and well marginated, not masked and laterally located (P < 0.05).

CONCLUSIONS

Extra validation results of DCNN showed high ROC results and there was a significant improvement in the performance when radiologists used DCNN. Reproducibility by DCNN alone showed good agreement, and there was an improvement from moderate to good agreement for radiologists using DCNN.

Learning to detect chest radiographs containing pulmonary lesions using visual attention networks

Machine learning approaches hold great potential for the automated detection of lung nodules on chest radiographs, but training algorithms requires very large amounts of manually annotated radiographs, which are difficult to obtain. The increasing availability of PACS (Picture Archiving and Communication System), is laying the technological foundations needed to make available large volumes of clinical data and images from hospital archives. Binary labels indicating whether a radiograph contains a pulmonary lesion can be extracted at scale, using natural language processing algorithms. In this study, we propose two novel neural networks for the detection of chest radiographs containing pulmonary lesions. Both architectures make use of a large number of weakly-labelled images combined with a smaller number of manually annotated x-rays. The annotated lesions are used during training to deliver a type of visual attention feedback informing the networks about their lesion localisation performance. The first architecture extracts saliency maps from high-level convolutional layers and compares the inferred position of a lesion against the true position when this information is available; a localisation error is then back-propagated along with the softmax classification error. The second approach consists of a recurrent attention model that learns to observe a short sequence of smaller image portions through reinforcement learning; the reward function penalises the exploration of areas, within an image, that are unlikely to contain nodules. Using a repository of over 430,000 historical chest radiographs, we present and discuss the proposed methods over related architectures that use either weakly-labelled or annotated images only.

Memory bias in observer-performance literature

The objective of our study was to determine how authors of published observer–performance experiments dealt with memory bias in study design. We searched American Journal of Roentgenology online and Radiology using “observer study” and “observer performance.” We included articles from 1970 or later that reported an observer performance experiment using human observers. We recorded the methods used by the authors to order presentation of the conditions being tested and images within sets for viewing. We recorded use and length of any time gap between viewings. We included 110 experiments. Forty-five used methods not subject to memory bias. Of 68 remaining experiments, 30 (44.1%) ordered the viewing of tested conditions to decrease memory bias. Fifteen (22.1%) ordered the tested conditions in ways that may create memory bias. Eleven (16.2%) intermixed the tested conditions. Forty-three (63.2%) used random or pseudorandom ordering of images within sets. Forty-six (67.6%) used a time gap (median 14 days) between viewings. Six (8.8%) did not use a time gap. Thirty-six (52.9%) did not indicate what methods they used in at least one studied parameter. Therefore, we conclude that 22.1% of the experiments could improve their methods of ordering tested conditions. Completeness of reporting could be improved by including more details regarding methods of ameliorating memory bias.

Bone subtraction radiography in adult patients with cystic fibrosis

Background Bone subtraction radiography allows reading pulmonary changes of chest radiographs more accurately without superimposition of bones. Purpose To evaluate the value of bone subtraction chest radiography using dual energy (DE) bone subtracted lung images compared to conventional radiographs (CR) in adult patients with cystic fibrosis (CF). Material and Methods Forty-nine DE radiographs of 24 patients (16 men) with CF (mean age, 32 years; age range, 18-71 years) were included. Lung function tests were performed within 10 days of the radiographs. Two radiologists evaluated all CR, DE, and CR + DE radiographs using the modified Chrispin-Norman score (CNS) and a five-point score for the confidence. Findings were statistically evaluated by Friedman ANOVA and Wilcoxon matched-pairs test. Results There was significant difference of CNS between CR and DE ( P = 0.044) as well as CR and CR + DE ( P < 0.001). CNS of CR images showed moderate correlation with FEV1% (R = 0.287, P = 0.046) while DE and CR + DE correlated poorly with FEV1% (R = 0.023, P = 0.874 and R = 0.04, P = 0.785). A higher confidence was achieved with bone-subtracted radiographs compared to radiographs alone (median, CR 3.3, DE 3.9, CR + DE 4.1, for both P < 0.001). Conclusion DE radiographs are reliable for the evaluation of adult patients with CF in acute exacerbation. For yearly surveillance, CR and DE radiographs may play a limited role. However, in clinical routine, DE radiographs are useful for adult CF patients and may depict more accurately inflammatory changes than CR.

A novel bone suppression method that improves lung nodule detection : Suppressing dedicated bone shadows in radiographs while preserving the remaining signal

PURPOSE

Suppressing thoracic bone shadows in chest radiographs has been previously reported to improve the detection rates for solid lung nodules, however at the cost of increased false detection rates. These bone suppression methods are based on an artificial neural network that was trained using dual-energy subtraction images in order to mimic their appearance.

METHOD

Here, a novel approach is followed where all bone shadows crossing the lung field are suppressed sequentially leaving the intercostal space unaffected. Given a contour delineating a bone, its image region is spatially transferred to separate normal image gradient components from tangential component. Smoothing the normal partial gradient along the contour results in a reconstruction of the image representing the bone shadow only, because all other overlaid signals tend to cancel out each other in this representation.

RESULTS

The method works even with highly contrasted overlaid objects such as a pacemaker. The approach was validated in a reader study with two experienced chest radiologists, and these images helped improving both the sensitivity and the specificity of the readers for the detection and localization of solid lung nodules. The AUC improved significantly from 0.596 to 0.655 on a basis of 146 images from patients and normals with a total of 123 confirmed lung nodules.

CONCLUSION

Subtracting all reconstructed bone shadows from the original image results in a soft image where lung nodules are no longer obscured by bone shadows. Both the sensitivity and the specificity of experienced radiologists increased.

Imaging the solitary pulmonary nodule

The development of widespread lung cancer screening programs has the potential to dramatically increase the number of thoracic computed tomography (CT) examinations performed annually in the United States, resulting in a greater number of newly detected, indeterminate solitary pulmonary nodules (SPNs). Additional imaging studies, such as fluorodeoxyglucose F 18 (FDG)-positron emission tomography (PET), have been shown to provide valuable information in the assessment of indeterminate SPNs. Newer technologies, such as contrast-enhanced dual-energy chest CT and FDG-PET/CT, also have the potential to facilitate diagnosis of potentially malignant SPNs.

Computer-aided nodule detection system: results in an unselected series of consecutive chest radiographs

RATIONALE AND OBJECTIVES

To evaluate the performance of a computer-aided detection (CAD) system with bone suppression imaging when applied to unselected consecutive chest radiographs (CXRs) with computed tomography (CT) correlation.

MATERIALS AND METHODS

This study included 586 consecutive patients with standard or portable CXRs who had a chest CT scan on the same day. Among the 586 CXRs, 438 had various abnormalities, including 46 CXRs with 66 lung nodules, and 148 CXRs had no significant abnormalities. A commercially available CAD system was applied to all 586 CXRs. True nodules and false positives (FPs) marked on CXRs by the CAD system were evaluated based on the corresponding chest CT findings.

RESULTS

The CAD system marked 47 of 66 (71%) lung nodules in this consecutive series of CXRs. The mean FP rate per image was 1.3 across all 586 CXRs, with 1.5 FPs per image on the 438 abnormal CXRs and 0.8 FPs per image on the 148 normal CXRs. A total of 41% of the 752 FP marks were related to non-nodule pathologic findings.

CONCLUSIONS

A currently available CAD system marked 71% of radiologist-identified lung nodules in a large consecutive series of CXRs, and 41% of "false" marks were caused by pathologic findings.

Feasibility study of dual energy radiographic imaging for target localization in radiotherapy for lung tumors

Purpose

Dual-energy (DE) radiographic imaging improves tissue discrimination by separating soft from hard tissues in the acquired images. This study was to establish a mathematic model of DE imaging based on intrinsic properties of tissues and quantitatively evaluate the feasibility of applying the DE imaging technique to tumor localization in radiotherapy.

Methods

We investigated the dependence of DE image quality on the radiological equivalent path length (EPL) of tissues with two phantoms using a stereoscopic x-ray imaging unit. 10 lung cancer patients who underwent radiotherapy each with gold markers implanted in the tumor were enrolled in the study approved by the hospital's Ethics Committee. The displacements of the centroids of the delineated gross tumor volumes (GTVs) in the digitally reconstructed radiograph (DRR) and in the bone-canceled DE image were compared with the averaged displacements of the centroids of gold markers to evaluate the feasibility of using DE imaging for tumor localization.

Results

The results of the phantom study indicated that the contrast-to-noise ratio (CNR) was linearly dependent on the difference of EPL and a mathematical model was established. The objects and backgrounds corresponding to ΔEPL less than 0.08 are visually indistinguishable in the bone-canceled DE image. The analysis of patient data showed that the tumor contrast in the bone-canceled images was improved significantly as compared with that in the original radiographic images and the accuracy of tumor localization using the DE imaging technique was comparable with that of using fiducial makers.

Conclusion

It is feasible to apply the technique for tumor localization in radiotherapy.

Lung cancer imaging

Lung cancer remains the leading cause of cancer-related deaths in the US. Imaging plays an important role in the diagnosis, staging, and follow-up evaluation of patients with lung cancer. With recent advances in technology, it is important to update and standardize the radiological practices in lung cancer evaluation. In this article, the authors review the main clinical applications of different imaging modalities and the most common radiological presentations of lung cancer.

Computer-aided detection improves detection of pulmonary nodules in chest radiographs beyond the support by bone-suppressed images

PURPOSE

To evaluate the added value of computer-aided detection (CAD) for lung nodules on chest radiographs when radiologists have bone-suppressed images (BSIs) available.

MATERIALS AND METHODS

Written informed consent was waived by the institutional review board. Selection of study images and study setup was reviewed and approved by the institutional review boards. Three hundred posteroanterior (PA) and lateral chest radiographs (189 radiographs with negative findings and 111 radiographs with a solitary nodule) in 300 subjects were selected from image archives at four institutions. PA images were processed by using a commercially available CAD, and PA BSIs were generated. Five radiologists and three residents evaluated the radiographs with BSIs available, first, without CAD and, second, after inspection of the CAD marks. Readers marked locations suspicious for a nodule and provided a confidence score for that location to be a nodule. Location-based receiver operating characteristic analysis was performed by using jackknife alternative free-response receiver operating characteristic analysis. Area under the curve (AUC) functioned as figure of merit, and P values were computed with the Dorfman-Berbaum-Metz method.

RESULTS

Average nodule size was 16.2 mm. Stand-alone CAD reached a sensitivity of 74% at 1.0 false-positive mark per image. Without CAD, average AUC for observers was 0.812. With CAD, performance significantly improved to an AUC of 0.841 (P = .0001). CAD detected 127 of 239 nodules that were missed after evaluation of the radiographs together with BSIs pooled over all observers. Only 57 of these detections were eventually marked by the observers after review of CAD candidates.

CONCLUSION

CAD improved radiologists' performance for the detection of lung nodules on chest radiographs, even when baseline performance was optimized by providing lateral radiographs and BSIs. Still, most of the true-positive CAD candidates are dismissed by observers.