Improved detection of pulmonary nodules on energy-subtracted chest radiographs with a commercial computer-aided diagnosis software: comparison with human observers

Computer-Aided Detection of Seven Chest Pathologies on Standard Posteroanterior Chest X-Rays Compared to Radiologists Reading Dual-Energy Subtracted Radiographs

RATIONALE AND OBJECTIVES

Retrospective performance evaluation of a computer-aided detection (CAD) system on standard posteroanterior (PA) chest radiographs (PA-CXR) in detection of pulmonary nodules, infectious consolidation, pneumothorax, pleural effusion, aortic calcification, cardiomegaly and rib fractures compared to radiologists analyzing PA-CXR including dual-energy subtraction radiography (further termed as DESR).

MATERIALS AND METHODS

PA-CXR/DESR images of 197 patients were included. All patients underwent chest CT (gold standard) within a short interval (mean 28 hours). All images were evaluated by three blinded readers for the presence of pulmonary nodules, infectious consolidation, pneumothorax, pleural effusion, aortic calcification, cardiomegaly, and rib fractures. Meanwhile PA-CXR were analyzed by a CAD software. CAD results were compared to the majority result of the three readers. Sensitivity and specificity were calculated. McNemar's test was applied to test for significant differences. Interobserver agreement was defined using Cohen's kappa (κ).

RESULTS

Sensitivity of the CAD software was significantly higher (p < 0.05) for detection of infectious consolidation and pulmonary nodules (67.9% vs 26.8% and 54% vs 35.6%, respectively; p < 0.001) compared to radiologists analyzing DESR images. For the residual evaluated pathologies no statistical significant differences could be found. Overall, mean inter observer agreement between the three radiologists was moderate (k = 0.534). The best interobserver agreement could be reached for pneumothorax (k = 0.708) and pleural effusion (k = 0.699), while the worst was obtained for rib fractures (k = 0.412).

CONCLUSION

The CAD system has the potential to improve the detection of infectious consolidation and pulmonary nodules on CXR images.

Computer-Aided Diagnosis Improves the Detection of Clinically Significant Prostate Cancer on Multiparametric-MRI: A Multi-Observer Performance Study Involving Inexperienced Readers

Recently, Computer Aided Diagnosis (CAD) systems have been proposed to help radiologists in detecting and characterizing Prostate Cancer (PCa). However, few studies evaluated the performances of these systems in a clinical setting, especially when used by non-experienced readers. The main aim of this study is to assess the diagnostic performance of non-experienced readers when reporting assisted by the likelihood map generated by a CAD system, and to compare the results with the unassisted interpretation. Three resident radiologists were asked to review multiparametric-MRI of patients with and without PCa, both unassisted and assisted by a CAD system. In both reading sessions, residents recorded all positive cases, and sensitivity, specificity, negative and positive predictive values were computed and compared. The dataset comprised 90 patients (45 with at least one clinically significant biopsy-confirmed PCa). Sensitivity significantly increased in the CAD assisted mode for patients with at least one clinically significant lesion (GS > 6) (68.7% vs. 78.1%, p = 0.018). Overall specificity was not statistically different between unassisted and assisted sessions (94.8% vs. 89.6, p = 0.072). The use of the CAD system significantly increases the per-patient sensitivity of inexperienced readers in the detection of clinically significant PCa, without negatively affecting specificity, while significantly reducing overall reporting time.

Dual energy imaging in cardiothoracic pathologies: A primer for radiologists and clinicians

Recent advances in dual-energy imaging techniques, dual-energy subtraction radiography (DESR) and dual-energy CT (DECT), offer new and useful additional information to conventional imaging, thus improving assessment of cardiothoracic abnormalities. DESR facilitates detection and characterization of pulmonary nodules. Other advantages of DESR include better depiction of pleural, lung parenchymal, airway and chest wall abnormalities, detection of foreign bodies and indwelling devices, improved visualization of cardiac and coronary artery calcifications helping in risk stratification of coronary artery disease, and diagnosing conditions like constrictive pericarditis and valvular stenosis. Commercially available DECT approaches are classified into emission based (dual rotation/spin, dual source, rapid kilovoltage switching and split beam) and detector-based (dual layer) systems. DECT provide several specialized image reconstructions. Virtual non-contrast images (VNC) allow for radiation dose reduction by obviating need for true non contrast images, low energy virtual mono-energetic images (VMI) boost contrast enhancement and help in salvaging otherwise non-diagnostic vascular studies, high energy VMI reduce beam hardening artifacts from metallic hardware or dense contrast material, and iodine density images allow quantitative and qualitative assessment of enhancement/iodine distribution. The large amount of data generated by DECT can affect interpreting physician efficiency but also limit clinical adoption of the technology. Optimization of the existing workflow and streamlining the integration between post-processing software and picture archiving and communication system (PACS) is therefore warranted.

Bone subtraction radiography in adult patients with cystic fibrosis

Background Bone subtraction radiography allows reading pulmonary changes of chest radiographs more accurately without superimposition of bones. Purpose To evaluate the value of bone subtraction chest radiography using dual energy (DE) bone subtracted lung images compared to conventional radiographs (CR) in adult patients with cystic fibrosis (CF). Material and Methods Forty-nine DE radiographs of 24 patients (16 men) with CF (mean age, 32 years; age range, 18-71 years) were included. Lung function tests were performed within 10 days of the radiographs. Two radiologists evaluated all CR, DE, and CR + DE radiographs using the modified Chrispin-Norman score (CNS) and a five-point score for the confidence. Findings were statistically evaluated by Friedman ANOVA and Wilcoxon matched-pairs test. Results There was significant difference of CNS between CR and DE ( P = 0.044) as well as CR and CR + DE ( P < 0.001). CNS of CR images showed moderate correlation with FEV1% (R = 0.287, P = 0.046) while DE and CR + DE correlated poorly with FEV1% (R = 0.023, P = 0.874 and R = 0.04, P = 0.785). A higher confidence was achieved with bone-subtracted radiographs compared to radiographs alone (median, CR 3.3, DE 3.9, CR + DE 4.1, for both P < 0.001). Conclusion DE radiographs are reliable for the evaluation of adult patients with CF in acute exacerbation. For yearly surveillance, CR and DE radiographs may play a limited role. However, in clinical routine, DE radiographs are useful for adult CF patients and may depict more accurately inflammatory changes than CR.

An Official American Thoracic Society Research Statement: A Research Framework for Pulmonary Nodule Evaluation and Management

BACKGROUND

Pulmonary nodules are frequently detected during diagnostic chest imaging and as a result of lung cancer screening. Current guidelines for their evaluation are largely based on low-quality evidence, and patients and clinicians could benefit from more research in this area.

METHODS

In this research statement from the American Thoracic Society, a multidisciplinary group of clinicians, researchers, and patient advocates reviewed available evidence for pulmonary nodule evaluation, characterized six focus areas to direct future research efforts, and identified fundamental gaps in knowledge and strategies to address them. We did not use formal mechanisms to prioritize one research area over another or to achieve consensus.

RESULTS

There was widespread agreement that novel tests (including novel imaging tests and biopsy techniques, biomarkers, and prognostic models) may improve diagnostic accuracy for identifying cancerous nodules. Before they are used in clinical practice, however, better evidence is needed to show that they improve more distal outcomes of importance to patients. In addition, the pace of research and the quality of clinical care would be improved by the development of registries that link demographic and nodule characteristics with patient-level outcomes. Methods to share data from registries are also necessary.

CONCLUSIONS

This statement may help researchers to develop impactful and innovative research projects and enable funders to better judge research proposals. We hope that it will accelerate the pace and increase the efficiency of discovery to improve the quality of care for patients with pulmonary nodules.

Role of the texture features of images in the diagnosis of solitary pulmonary nodules in different sizes

OBJECTIVE

To explore the role of the texture features of images in the diagnosis of solitary pulmonary nodules (SPNs) in different sizes.

MATERIALS AND METHODS

A total of 379 patients with pathologically confirmed SPNs were enrolled in this study. They were divided into three groups based on the SPN sizes: ≤10, 11-20, and >20 mm. Their texture features were segmented and extracted. The differences in the image features between benign and malignant SPNs were compared. The SPNs in these three groups were determined and analyzed with the texture features of images.

RESULTS

These 379 SPNs were successfully segmented using the 2D Otsu threshold method and the self-adaptive threshold segmentation method. The texture features of these SPNs were obtained using the method of grey level co-occurrence matrix (GLCM). Of these 379 patients, 120 had benign SPNs and 259 had malignant SPNs. The entropy, contrast, energy, homogeneity, and correlation were 3.5597±0.6470, 0.5384±0.2561, 0.1921±0.1256, 0.8281±0.0604, and 0.8748±0.0740 in the benign SPNs and 3.8007±0.6235, 0.6088±0.2961, 0.1673±0.1070, 0.7980±0.0555, and 0.8550±0.0869 in the malignant SPNs (all P<0.05). The sensitivity, specificity, and accuracy of the texture features of images were 83.3%, 90.0%, and 86.8%, respectively, for SPNs sized ≤10 mm, and were 86.6%, 88.2%, and 87.1%, respectively, for SPNs sized 
11-20 mm and 94.7%, 91.8%, and 93.9%, respectively, for SPNs sized >20 mm.

CONCLUSIONS

The entropy and contrast of malignant pulmonary nodules have been demonstrated to be higher in comparison to those of benign pulmonary nodules, while the energy, homogeneity correlation of malignant pulmonary nodules are lower than those of benign pulmonary nodules. The texture features of images can reflect the tissue features and have high sensitivity, specificity, and accuracy in differentiating SPNs. The sensitivity and accuracy increase for larger SPNs.

A comparison of computer-aided detection (CAD) effectiveness in pulmonary nodule identification using different methods of bone suppression in chest radiographs

This study aimed to compare the diagnostic effectiveness of computer-aided detection (CAD) software (OnGuard™ 5.2) in combination with hardware-based bone suppression (dual-energy subtraction radiography (DESR)), software-based bone suppression (SoftView™, version 2.4), and standard posteroanterior images with no bone suppression. A retrospective pilot study compared the diagnostic performance of two commercially available methods of bone suppression when used with commercially available CAD software. Chest images from 27 patients with computed tomography (CT) and pathology-proven malignant pulmonary nodules (8-34 mm) and 25 CT-negative patient controls were used for analysis. The Friedman, McNemar, and chi-square tests were used to compare diagnostic performance and the kappa statistic was used to evaluate method agreement. The average number of regions of interest and false-positives per image identified by CAD were not found to be significantly different regardless of the bone suppression methods evaluated. Similarly, the sensitivity, specificity, and test efficiency were not found to be significantly different. Agreement between the methods was between poor and excellent. The accuracy of CAD (OnGuard™, version 5.2) is not statistically different with either DESR or SoftView™ (version 2.4) bone suppression technology in digital chest images for pulmonary nodule identification. Low values for sensitivity (<80 %) and specificity (<50 %) may limit their utility for clinical radiology.

Bone suppressed images improve radiologists' detection performance for pulmonary nodules in chest radiographs

OBJECTIVES

To assess the effect of bone suppression imaging on observer performance in detecting lung nodules in chest radiographs.

MATERIALS AND METHODS

Posteroanterior (PA) and lateral digital chest radiographs of 111 (average age 65) patients with a CT proven solitary nodule (median diameter 15 mm), and 189 (average age 63) controls were read by 5 radiologists and 3 residents. Conspicuity of nodules on the radiographs was classified in obvious (n = 32), moderate (n = 32), subtle (n = 29) and very subtle (n = 18). Observers read the PA and lateral chest radiographs without and with an additional PA bone suppressed image (BSI) (ClearRead Bone Suppression 2.4, Riverain Technologies, Ohio) within one reading session. Multi reader multi case (MRMC) receiver operating characteristics (ROC) were used for statistical analysis.

RESULTS

ROC analysis showed improved detection with use of BSI compared to chest radiographs alone (AUC = 0.883 versus 0.855; p = 0.004). Performance also increased at high specificities exceeding 80% (pAUC = 0.136 versus 0.124; p = 0.0007). Operating at a specificity of 90%, sensitivity increased with BSI from 66% to 71% (p = 0.0004). Increase of detection performance was highest for nodules with moderate and subtle conspicuity (p = 0.02; p = 0.03).

CONCLUSION

Bone suppressed images improve radiologists' detection performance for pulmonary nodules, especially for those of moderate and subtle conspicuity.

Comparison of dual-energy subtraction and electronic bone suppression combined with computer-aided detection on chest radiographs: effect on human observers' performance in nodule detection

OBJECTIVE

The objective of our study was to compare the effect of dual-energy subtraction and bone suppression software alone and in combination with computer-aided detection (CAD) on the performance of human observers in lung nodule detection.

MATERIALS AND METHODS

One hundred one patients with from one to five lung nodules measuring 5-29 mm and 42 subjects with no nodules were retrospectively selected and randomized. Three independent radiologists marked suspicious-appearing lesions on the original chest radiographs, dual-energy subtraction images, and bone-suppressed images before and after postprocessing with CAD. Marks of the observers and CAD marks were compared with CT as the reference standard. Data were analyzed using nonparametric tests and the jackknife alternative free-response receiver operating characteristic (JAFROC) method.

RESULTS

Using dual-energy subtraction alone (p = 0.0198) or CAD alone (p = 0.0095) improved the detection rate compared with using the original conventional chest radiograph. The combination of bone suppression and CAD provided the highest sensitivity (51.6%) and the original nonenhanced conventional chest radiograph alone provided the lowest (46.9%; p = 0.0049). Dual-energy subtraction and bone suppression provided the same false-positive (p = 0.2702) and true-positive (p = 0.8451) rates. Up to 22.9% of lesions were found only by the CAD program and were missed by the readers. JAFROC showed no difference in the performance between modalities (p = 0.2742-0.5442).

CONCLUSION

Dual-energy subtraction and the electronic bone suppression program used in this study provided similar detection rates for pulmonary nodules. Additionally, CAD alone or combined with bone suppression can significantly improve the sensitivity of human observers for pulmonary nodule detection.

Prostate cancer: computer-aided diagnosis with multiparametric 3-T MR imaging--effect on observer performance

PURPOSE

To determine the effect of computer-aided diagnosis (CAD) on less-experienced and experienced observer performance in differentiation of benign from malignant prostate lesions at 3-T multiparametric magnetic resonance (MR) imaging.

MATERIALS AND METHODS

The institutional review board waived the need for informed consent. Retrospectively, 34 patients were included who had prostate cancer and had undergone multiparametric MR imaging, including T2-weighted, diffusion-weighted, and dynamic contrast material-enhanced MR imaging prior to radical prostatectomy. Six radiologists less experienced in prostate imaging and four radiologists experienced in prostate imaging were asked to characterize different regions suspicious for cancer as benign or malignant on multiparametric MR images first without and subsequently with CAD software. The effect of CAD was analyzed by using a multiple-reader, multicase, receiver operating characteristic analysis and a linear mixed-model analysis.

RESULTS

In 34 patients, 206 preannotated regions, including 67 malignant and 64 benign regions in the peripheral zone (PZ) and 19 malignant and 56 benign regions in the transition zone (TZ), were evaluated. Stand-alone CAD had an overall area under the receiver operating characteristic curve (AUC) of 0.90. For PZ and TZ lesions, the AUCs were 0.92 and 0.87, respectively. Without CAD, less-experienced observers had an overall AUC of 0.81, which significantly increased to 0.91 (P = .001) with CAD. For experienced observers, the AUC without CAD was 0.88, which increased to 0.91 (P = .17) with CAD. For PZ lesions, less-experienced observers increased their AUC from 0.86 to 0.95 (P < .001) with CAD. Experienced observers showed an increase from 0.91 to 0.93 (P = .13). For TZ lesions, less-experienced observers significantly increased their performance from 0.72 to 0.79 (P = .01) with CAD and experienced observers increased their performance from 0.81 to 0.82 (P = .42).

CONCLUSION

Addition of CAD significantly improved the performance of less-experienced observers in distinguishing benign from malignant lesions; when less-experienced observers used CAD, they reached similar performance as experienced observers. The stand-alone performance of CAD was similar to performance of experienced observers.

Computer-aided detection of small pulmonary nodules in chest radiographs: an observer study

RATIONALE AND OBJECTIVES

To evaluate the impact of computer-aided detection (CAD, IQQA-Chest; EDDA Technology, Princeton Junction, NJ) used as second reader on the detection of small pulmonary nodules in chest radiography (CXR).

MATERIALS AND METHODS

A total of 113 patients (mean age 62 years) with CT and CXR within 6 weeks were selected. Fifty-nine patients showed 101 pulmonary nodules (diameter 5-15mm); the remaining 54 patients served as negative controls. Six readers of varying experience individually evaluated the CXR without and with CAD as second reader in two separate reading sessions. The sensitivity per lesion, figure of merit (FOM), and mean false positive per image (mFP) were calculated. Institutional review board approval was waived.

RESULTS

With CAD, the sensitivity increased for inexperienced readers (39% vs. 45%, P < .05) and remained unchanged for experienced readers (50% vs. 51%). The mFP nonsignificantly increased for both inexperienced and experienced readers (0.27 vs. 0.34 and 0.16 vs. 0.21). The mean FOM did not significantly differ for readings without and with CAD irrespective of reader experience (0.71 vs. 0.71 and 0.84 vs. 0.87). All readers together dismissed 33% of true-positive CAD candidates. False-positive candidates by CAD provoked 40% of all false-positive marks made by the readers.

CONCLUSION

CAD improves the sensitivity of inexperienced readers for the detection of small nodules at the expense of loss of specificity. Overall performance by means of FOM was therefore not affected. To use CAD more beneficial, readers need to improve their ability to differentiate true from false-positive CAD candidates.

Predicting readers' diagnostic accuracy with a new CAD algorithm

RATIONALE AND OBJECTIVES

Before computer-aided detection (CAD) algorithms can be used in clinical practice, they must be shown to improve readers' diagnostic accuracy over their unaided performance. This is usually accomplished through a large multireader, multicase (MRMC) clinical trial. It is burdensome, however, for an MRMC study to be performed with each new release of a CAD algorithm. The aim of this report is to present an approach for building models to predict readers' accuracy with a new CAD algorithm.

MATERIALS AND METHODS

A modeling approach for predicting readers' results with a new CAD algorithm is described. Multiple-variable logistic regression was used to build models for readers' sensitivity and false-positive rate, given the results of an MRMC study with an older CAD algorithm and the stand-alone performance results of a new CAD algorithm. Data from a large lung MRMC CAD trial are used to illustrate the modeling approach and test the ability of the models to predict readers' accuracy with the new CAD algorithm.

RESULTS

The model overestimated the readers' actual sensitivity with the new CAD algorithm, but this did not reach statistical significance (0.621 vs 0.603, P = .147). The observed and predicted false-positive rates also did not differ significantly (0.275 vs 0.285, P = .250).

CONCLUSIONS

Using one clinical study as a test case, it is shown that the modeling approach is feasible. More testing of the approach is needed to determine if and under what circumstances it can be used as an alternative to a full-scale MRMC study. Meanwhile, the approach can be used to determine if a new CAD algorithm is likely to improve readers' accuracy before embarking on a full-scale MRMC study.