Missed lung cancer on chest radiography and computed tomography

Clinical and chest radiographic features of missed lung cancer and their association with patient outcomes

PURPOSE

To examine clinical and chest radiographic features of missed lung cancer (MLC) and explore their association with patient outcomes.

METHODS

We retrospectively reviewed chest radiographs obtained at least six months before lung cancer (LC) diagnosis in 95 patients to identify the first positive chest radiograph showing MLC. We assessed chest radiographic features of MLC and their association with patient outcomes.

RESULTS

Seventy-five (78.9%) patients (39 men, 36 women; mean age, 64.5 ± 10.5 years) had MLC. The median diagnostic delay was 31.3 months (6.6-128.0 months). The median MLC size was 16 mm (5-57 mm), and 54.7%, 68.0%, and 74.7% of MLC were in the left lung, the middle/lower zones, and the outer two-thirds of the lung, respectively. MLC exhibited a round/oval shape, partly/poorly defined margin, irregular/spiculated border, a density less than the aortic knob, and anatomical superimposition in 57.3%, 77.3%, 61.3%, 85.3%, and 88.0% of cases, respectively. Thirty-five (46.7%) patients had stage III + IV LC at diagnosis. Thirty-one (41.3%) patients died. MLC in the inner one-third of the lung, exhibiting a density equal to/greater than the aortic knob, or superimposed by midline structures was significantly associated with stage III + IV LC at diagnosis. The 3-year all-cause mortality significantly increased when MLC was in the upper zone, superimposed by pulmonary vessels, superimposed by pulmonary vessels plus ribs, or superimposed by pulmonary vessels plus in the inner one-third of the lung.

CONCLUSION

MLC with some radiographic features pertaining to their location, density, and superimposed structures was found to portend a worse outcome.

Validation of deep learning-based computer-aided detection software use for interpretation of pulmonary abnormalities on chest radiographs and examination of factors that influence readers' performance and final diagnosis

PURPOSE

To evaluate the performance of a deep learning-based computer-aided detection (CAD) software for detecting pulmonary nodules, masses, and consolidation on chest radiographs (CRs) and to examine the effect of readers' experience and data characteristics on the sensitivity and final diagnosis.

MATERIALS AND METHODS

The CRs of 453 patients were retrospectively selected from two institutions. Among these CRs, 60 images with abnormal findings (pulmonary nodules, masses, and consolidation) and 140 without abnormal findings were randomly selected for sequential observer-performance testing. In the test, 12 readers (three radiologists, three pulmonologists, three non-pulmonology physicians, and three junior residents) interpreted 200 images with and without CAD, and the findings were compared. Weighted alternative free-response receiver operating characteristic (wAFROC) figure of merit (FOM) was used to analyze observer performance. The lesions that readers initially missed but CAD detected were stratified by anatomic location and degree of subtlety, and the adoption rate was calculated. Fisher's exact test was used for comparison.

RESULTS

The mean wAFROC FOM score of the 12 readers significantly improved from 0.746 to 0.810 with software assistance (P = 0.007). In the reader group with < 6 years of experience, the mean FOM score significantly improved from 0.680 to 0.779 (P = 0.011), while that in the reader group with ≥ 6 years of experience increased from 0.811 to 0.841 (P = 0.12). The sensitivity of the CAD software and the adoption rate for the lesions with subtlety level 2 or 3 (obscure) lesions were significantly lower than for level 4 or 5 (distinct) lesions (50% vs. 93%, P < 0.001; and 55% vs. 74%, P = 0.04, respectively).

CONCLUSION

CAD software use improved doctors' performance in detecting nodules/masses and consolidation on CRs, particularly for non-expert doctors, by preventing doctors from missing distinct lesions rather than helping them to detect obscure lesions.

Artificial neural network in the discrimination of lung cancer based on infrared spectroscopy

Given the increasing prevalence of lung cancer worldwide, an auxiliary diagnostic method is needed alongside the microscopic examination of biopsy samples, which is dependent on the skills and experience of pathologists. Thus, this study aimed to advance lung cancer diagnosis by developing five (5) artificial neural network (NN) models that can discriminate malignant from benign samples based on infrared spectral data of lung tumors (n = 122; 56 malignant, 66 benign). NNs were benchmarked with classical machine learning (CML) models. Stratified 10-fold cross-validation was performed to evaluate the NN models, and the performance metrics—area under the curve (AUC), accuracy (ACC) positive predictive value (PPV), negative predictive value (NPV), specificity rate (SR), and recall rate (RR)—were averaged for comparison. All NNs were able to outperform the CML models, however, support vector machine is relatively comparable to NNs. Among the NNs, CNN performed best with an AUC of 92.28% ± 7.36%, ACC of 98.45% ± 1.72%, PPV of 96.62% ± 2.30%, NPV of 90.50% ± 11.92%, SR of 96.01% ± 3.09%, and RR of 89.21% ± 12.93%. In conclusion, NNs can be potentially used as a computational tool in lung cancer diagnosis based on infrared spectroscopy of lung tissues.

Clinical impact of a deep learning system for automated detection of missed pulmonary nodules on routine body computed tomography including the chest region

OBJECTIVES

To evaluate the clinical impact of a deep learning system (DLS) for automated detection of pulmonary nodules on computed tomography (CT) images as a second reader.

METHODS

This single-centre retrospective study screened 21,150 consecutive body CT studies from September 2018 to February 2019. Pulmonary nodules detected by the DLS on axial CT images but not mentioned in initial radiology reports were flagged. Flagged images were scored by four board-certificated radiologists each with at least 5 years of experience. Nodules with scores of 2 (understandable miss) or 3 (should not be missed) were then categorised as unlikely to be clinically significant (2a or 3a) or likely to be clinically significant (2b or 3b) according to the 2017 Fleischner guidelines for pulmonary nodules. The miss rate was defined as the total number of studies receiving scores of 2 or 3 divided by total screened studies.

RESULTS

Among 172 nodules flagged by the DLS, 60 (35%) missed nodules were confirmed by the radiologists. The nodules were further categorised as 2a, 2b, 3a, and 3b in 24, 14, 10, and 12 studies, respectively, with an overall positive predictive value of 35%. Missed pulmonary nodules were identified in 0.3% of all CT images, and one-third of these lesions were considered clinically significant.

CONCLUSIONS

Use of DLS-assisted automated detection as a second reader can identify missed pulmonary nodules, some of which may be clinically significant.

CLINICAL RELEVANCE/APPLICATION

Use of DLS to help radiologists detect pulmonary lesions may improve patient care.

KEY POINTS

• DLS-assisted automated detection as a second reader is feasible in a large consecutive cohort. • Performance of combined radiologists and DLS was better than DLS or radiologists alone. • Pulmonary nodules were missed more frequently in abdomino-pelvis CT than the thoracic CT.

Artificial intelligence-supported lung cancer detection by multi-institutional readers with multi-vendor chest radiographs: a retrospective clinical validation study

Background

We investigated the performance improvement of physicians with varying levels of chest radiology experience when using a commercially available artificial intelligence (AI)-based computer-assisted detection (CAD) software to detect lung cancer nodules on chest radiographs from multiple vendors.

Methods

Chest radiographs and their corresponding chest CT were retrospectively collected from one institution between July 2017 and June 2018. Two author radiologists annotated pathologically proven lung cancer nodules on the chest radiographs while referencing CT. Eighteen readers (nine general physicians and nine radiologists) from nine institutions interpreted the chest radiographs. The readers interpreted the radiographs alone and then reinterpreted them referencing the CAD output. Suspected nodules were enclosed with a bounding box. These bounding boxes were judged correct if there was significant overlap with the ground truth, specifically, if the intersection over union was 0.3 or higher. The sensitivity, specificity, accuracy, PPV, and NPV of the readers’ assessments were calculated.

Results

In total, 312 chest radiographs were collected as a test dataset, including 59 malignant images (59 nodules of lung cancer) and 253 normal images. The model provided a modest boost to the reader’s sensitivity, particularly helping general physicians. The performance of general physicians was improved from 0.47 to 0.60 for sensitivity, from 0.96 to 0.97 for specificity, from 0.87 to 0.90 for accuracy, from 0.75 to 0.82 for PPV, and from 0.89 to 0.91 for NPV while the performance of radiologists was improved from 0.51 to 0.60 for sensitivity, from 0.96 to 0.96 for specificity, from 0.87 to 0.90 for accuracy, from 0.76 to 0.80 for PPV, and from 0.89 to 0.91 for NPV. The overall increase in the ratios of sensitivity, specificity, accuracy, PPV, and NPV were 1.22 (1.14–1.30), 1.00 (1.00–1.01), 1.03 (1.02–1.04), 1.07 (1.03–1.11), and 1.02 (1.01–1.03) by using the CAD, respectively.

Conclusion

The AI-based CAD was able to improve the ability of physicians to detect nodules of lung cancer in chest radiographs. The use of a CAD model can indicate regions physicians may have overlooked during their initial assessment.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-021-08847-9.

Extravalidation and reproducibility results of a commercial deep learning-based automatic detection algorithm for pulmonary nodules on chest radiographs at tertiary hospital

INTRODUCTION

To extra validate and evaluate the reproducibility of a commercial deep convolutional neural network (DCNN) algorithm for pulmonary nodules on chest radiographs (CRs) and to compare its performance with radiologists.

METHODS

This retrospective study enrolled 434 CRs (normal to abnormal ratio, 246:188) from 378 patients that visited a tertiary hospital. DCNN performance was compared with two radiology residents and two thoracic radiologists. Abnormality assessment (using the area under the receiver operating ch3cteristics (AUROC)) and nodule detection (using jackknife alternative free-response ROC (JAFROC)) were compared among three groups (DCNN only, radiologist without DCNN and radiologist with DCNN). A subset of 56 paired cases, having two CRs taken within a 7-day period, were assessed for intraobserver reproducibility using the intraclass correlation coefficient. Independent characteristics of pulmonary nodules detected by DCNN were assessed by multiple logistic regression analysis.

RESULTS

The AUROC for abnormality detection for the three groups were 0.87, 0.93 and 0.96, respectively (P < 0.05), whereas the JAFROC analysis of nodule detection was 0.926, 0.929 and 0.964. Reproducibility for the three groups was 0.80, 0.67 and 0.80, which shows an increase in radiologists using DCNN (P < 0.05). Nodules detected by DCNN were more solid, round-shaped and well marginated, not masked and laterally located (P < 0.05).

CONCLUSIONS

Extra validation results of DCNN showed high ROC results and there was a significant improvement in the performance when radiologists used DCNN. Reproducibility by DCNN alone showed good agreement, and there was an improvement from moderate to good agreement for radiologists using DCNN.

The Use of Artificial Intelligence in the Differentiation of Malignant and Benign Lung Nodules on Computed Tomograms Proven by Surgical Pathology

The purpose of this work was to evaluate the performance of an existing commercially available artificial intelligence (AI) software system in differentiating malignant and benign lung nodules. The AI tool consisted of a vessel-suppression function and a deep-learning-based computer-aided-detection (VS-CAD) analyzer. Fifty patients (32 females, mean age 52 years) with 75 lung nodules (47 malignant and 28 benign) underwent low-dose computed tomography (LDCT) followed by surgical excision and the pathological analysis of their 75 nodules within a 3 month time frame. All 50 cases were then processed by the AI software to generate corresponding VS images and CAD outcomes. All 75 pathologically proven lung nodules were well delineated by vessel-suppressed images. Three (6.4%) of the 47 lung cancer cases, and 11 (39.3%) of the 28 benign nodules were ignored and not detected by the AI without showing a CAD analysis summary. The AI system/radiologists produced a sensitivity and specificity (shown in %) of 93.6/89.4 and 39.3/82.1 in distinguishing malignant from benign nodules, respectively. AI sensitivity was higher than that of radiologists, though not statistically significant (p = 0.712). Specificity obtained by the radiologists was significantly higher than that of the VS-CAD AI (p = 0.003). There was no significant difference between the malignant and benign lesions with respect to age, gender, pure ground-glass pattern, the diameter and location of the nodules, or nodules <6 vs. ≥6 mm. However, more part-solid nodules were proven to be malignant than benign (90.9% vs. 9.1%), and more solid nodules were proven to be benign than malignant (86.7% vs. 13.3%) with statistical significance (p = 0.001 and <0.001, respectively). A larger cohort and prospective study are required to validate the AI performance.

Is there an advantage to using computer aided detection for the early detection of pulmonary nodules within chest X-Ray imaging?

OBJECTIVE

Using published literature, this research examines whether Computer-aided Detection (CAD) identifies more Pulmonary Nodules (PN) within Chest X-ray (CXR) systems, compared to radiologist diagnosis without CAD.

KEY FINDINGS

Although the primary papers were pointing to CAD being a beneficial system in the diagnosis of PN detection, a regression analysis of the data available within these papers showed no correlation between the higher sensitivity of CAD against the detrimental high False Positives (FP) of CAD. Findings of the studies were deemed inconclusive.

CONCLUSION

Further research is recommended to review the potential of CAD on CXR PN detection.

IMPLICATIONS FOR PRACTICE

CAD acting as a second reader could potentially reduce interpreter error rate.

Deep Convolutional Neural Network-based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs

Background Multicenter studies are required to validate the added benefit of using deep convolutional neural network (DCNN) software for detecting malignant pulmonary nodules on chest radiographs. Purpose To compare the performance of radiologists in detecting malignant pulmonary nodules on chest radiographs when assisted by deep learning-based DCNN software with that of radiologists or DCNN software alone in a multicenter setting. Materials and Methods Investigators at four medical centers retrospectively identified 600 lung cancer-containing chest radiographs and 200 normal chest radiographs. Each radiograph with a lung cancer had at least one malignant nodule confirmed by CT and pathologic examination. Twelve radiologists from the four centers independently analyzed the chest radiographs and marked regions of interest. Commercially available deep learning-based computer-aided detection software separately trained, tested, and validated with 19 330 radiographs was used to find suspicious nodules. The radiologists then reviewed the images with the assistance of DCNN software. The sensitivity and number of false-positive findings per image of DCNN software, radiologists alone, and radiologists with the use of DCNN software were analyzed by using logistic regression and Poisson regression. Results The average sensitivity of radiologists improved (from 65.1% [1375 of 2112; 95% confidence interval {CI}: 62.0%, 68.1%] to 70.3% [1484 of 2112; 95% CI: 67.2%, 73.1%], P < .001) and the number of false-positive findings per radiograph declined (from 0.2 [488 of 2400; 95% CI: 0.18, 0.22] to 0.18 [422 of 2400; 95% CI: 0.16, 0.2], P < .001) when the radiologists re-reviewed radiographs with the DCNN software. For the 12 radiologists in this study, 104 of 2400 radiographs were positively changed (from false-negative to true-positive or from false-positive to true-negative) using the DCNN, while 56 of 2400 radiographs were changed negatively. Conclusion Radiologists had better performance with deep convolutional network software for the detection of malignant pulmonary nodules on chest radiographs than without. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Jacobson in this issue.

The Added Value of Computer-aided Detection of Small Pulmonary Nodules and Missed Lung Cancers

Lung cancer at its earliest stage is typically manifested on computed tomography as a pulmonary nodule, which could be detected by low-dose multidetector computed tomography technology and the use of thinner collimation. Within the last 2 decades, computer-aided detection (CAD) of pulmonary nodules has been developed to meet the increasing demand for lung cancer screening computed tomography with a larger set of images per scan. This review introduced the basic techniques and then summarized the up-to-date applications of CAD systems in clinical and research programs and in the low-dose lung cancer screening trials, especially in the detection of small pulmonary nodules and missed lung cancers. Many studies have already shown that the CAD systems could increase the sensitivity and reduce the false-positive rate in the diagnosis of pulmonary nodules, especially for the small and isolated nodules. Further improvements to the current CAD schemes are needed to detect nodules accurately, particularly for subsolid nodules.

Screening for early stage lung cancer and its correlation with lung nodule detection

Currently, the most effective way of reducing lung cancer mortality is early diagnosis of lung cancer. The National Lung Screening Trial has proved the efficacy of lung cancer screening using low-dose computed tomography to reduce lung cancer mortality. However, many questions remain surrounding lung cancer screening implementation, among which include how to select the optimal risk population, the personalized screening interval based different levels of risk, methods to improve diagnostic discrimination between malignant and benign disease in detected lung nodules, and the roles of biomolecular markers in stratifying risk and in guiding the management of indeterminate nodules. This review concentrates on the latest developments of lung cancer screening and provides an overview of the main unanswered questions on lung nodule detection.

Missed lung cancer: when, where, and why?

Missed lung cancer is a source of concern among radiologists and an important medicolegal challenge. In 90% of the cases, errors in diagnosis of lung cancer occur on chest radiographs. It may be challenging for radiologists to distinguish a lung lesion from bones, pulmonary vessels, mediastinal structures, and other complex anatomical structures on chest radiographs. Nevertheless, lung cancer can also be overlooked on computed tomography (CT) scans, regardless of the context, either if a clinical or radiologic suspect exists or for other reasons. Awareness of the possible causes of overlooking a pulmonary lesion can give radiologists a chance to reduce the occurrence of this eventuality. Various factors contribute to a misdiagnosis of lung cancer on chest radiographs and on CT, often very similar in nature to each other. Observer error is the most significant one and comprises scanning error, recognition error, decision-making error, and satisfaction of search. Tumor characteristics such as lesion size, conspicuity, and location are also crucial in this context. Even technical aspects can contribute to the probability of skipping lung cancer, including image quality and patient positioning and movement. Albeit it is hard to remove missed lung cancer completely, strategies to reduce observer error and methods to improve technique and automated detection may be valuable in reducing its likelihood.

X-ray Dark-field Radiography - In-Vivo Diagnosis of Lung Cancer in Mice

Accounting for about 1.5 million deaths annually, lung cancer is the prevailing cause of cancer deaths worldwide, mostly associated with long-term smoking effects. Numerous small-animal studies are performed currently in order to better understand the pathogenesis of the disease and to develop treatment strategies. Within this letter, we propose to exploit X-ray dark-field imaging as a novel diagnostic tool for the detection of lung cancer on projection radiographs. Here, we demonstrate in living mice bearing lung tumors, that X-ray dark-field radiography provides significantly improved lung tumor detection rates without increasing the number of false-positives, especially in the case of small and superimposed nodules, when compared to conventional absorption-based imaging. While this method still needs to be adapted to larger mammals and finally humans, the technique presented here can already serve as a valuable tool in evaluating novel lung cancer therapies, tested in mice and other small animal models.

The impact of greyscale inversion for nodule detection in an anthropomorphic chest phantom: a free-response observer study

Objective:

The aim of this work was to assess the impact of greyscale inversion on nodule detection on posteroanterior chest X-ray images. Previous work has attempted this, with no consensus opinion formed. We assessed the value of “fast-flicking” between standard and inverted display modes for nodule detection.

Methods:

Six consultant radiologists (with 5–32 years' reporting experience) completed an observer task under the free-response paradigm. An anthropomorphic chest phantom was loaded with 50 different configurations of simulated nodules (1–4 nodules per case) measuring 5, 8, 10 and 12 mm in spherical diameter; each configuration represented a single case. In addition, 25 cases contained no nodules. Images were displayed in three modes: (i) standard, (ii) inverted and (iii) fast-flicking between standard and inverted display modes. Each observer completed the study in a different order of display (i, ii, iii) using a calibrated 5-megapixel monitor. Nodules were localized with mouse clicks and ratings assigned using a 1–10 discrete slider-bar confidence scale. Rjafroc (Pittsburgh, PA) was used for data analysis; differences in nodule detection performance were considered significant at 0.05.

Results:

The observer-averaged weighted jackknife alternative free-response receiver-operating characteristic figures of merit were 0.715 (standard), 0.684 (inverted) and 0.717 (fast-flicking). Random-reader fixed-case analysis revealed no statistically significant difference between any treatment pair [F(2,8) = 1.22; p = 0.345].

Conclusion:

No statistically significant difference in nodule detection was found for the three display conditions.

Advances in knowledge:

We have investigated the impact of fast-flicking between standard and inverted display modes for the detection of nodules. We found no benefit.

Errors in imaging patients in the emergency setting

Emergency and trauma care produces a "perfect storm" for radiological errors: uncooperative patients, inadequate histories, time-critical decisions, concurrent tasks and often junior personnel working after hours in busy emergency departments. The main cause of diagnostic errors in the emergency department is the failure to correctly interpret radiographs, and the majority of diagnoses missed on radiographs are fractures. Missed diagnoses potentially have important consequences for patients, clinicians and radiologists. Radiologists play a pivotal role in the diagnostic assessment of polytrauma patients and of patients with non-traumatic craniothoracoabdominal emergencies, and key elements to reduce errors in the emergency setting are knowledge, experience and the correct application of imaging protocols. This article aims to highlight the definition and classification of errors in radiology, the causes of errors in emergency radiology and the spectrum of diagnostic errors in radiography, ultrasonography and CT in the emergency setting.

Errors in Medical Imaging and Radiography Practice: A Systematic Review

BACKGROUND

Errors in health care can harm patients and undermine public trust, yet many are preventable. In medical imaging and radiography, errors can cause increased radiation dose, misdiagnosis, and clinical mismanagement.

AIM

The purpose of this review was to identify the type and prevalence of errors directly associated with radiography practice and the imaging cycle, with a view to developing recommendations to reduce common errors.

METHOD

A systematic review was undertaken of current literature obtained through the Ovid Medline and PubMed databases. A total of 41 useable articles were analysed into a priori categories of the medical imaging cycle: preprocedural, procedural, and postprocedural.

FINDINGS

This review found that errors may occur during any phase of the cycle and that communication breakdown, especially during handover periods, was the main contributing factor to errors. Although the importance of incident reporting is well recognised, feedback to users is often limited.

CONCLUSIONS

A systematic approach to radiographic practice may assist in reducing communication-related errors. Future research is required to determine how extending radiographers' roles or using electronic ordering systems could also help to reduce errors.

Errors in multidetector row computed tomography

Multidetector row computed tomography (MDCT) represents the technique of choice for the majority of pathologies today and is responsible for the majority of diagnoses. However, despite the low number of studies dedicated to errors in MDCT, CT reporting seems especially prone to generating errors and errors are an inevitable part of MDCT practice. Most of these arise during image interpretation but, differently from other radiological techniques, the awareness of radiologists regarding technical CT aspects and pathologies substantially contribute in generating errors, in particular because CT technology expands rapidly and radiologists do not routinely receive specific and appropriate training for its use and because CT examinations are not the same for each patient and each pathology and the choice of the most appropriate CT examination (including the dose exposure to the patient) presumes a very large awareness from radiologists. This review is aimed at increasing awareness regarding the type of errors in MDCT and in particular to also highlight technical and procedural errors.

Lung tumors on multimodal radiographs derived from grating-based X-ray imaging--a feasibility study

PURPOSE

The purpose of this study was to assess whether grating-based X-ray imaging may have a role in imaging of pulmonary nodules on radiographs.

MATERIALS AND METHODS

A mouse lung containing multiple lung tumors was imaged using a small-animal scanner with a conventional X-ray source and a grating interferometer for phase-contrast imaging. We qualitatively compared the signal characteristics of lung nodules on transmission, dark-field and phase-contrast images. Furthermore, we quantitatively compared signal characteristics of lung tumors and the adjacent lung tissue and calculated the corresponding contrast-to-noise ratios.

RESULTS

Of the 5 tumors visualized on the transmission image, 3/5 tumors were clearly visualized and 1 tumor was faintly visualized in the dark-field image as areas of decreased small angle scattering. In the phase-contrast images, 3/5 tumors were clearly visualized, while the remaining 2 tumors were faintly visualized by the phase-shift occurring at their edges. No additional tumors were visualized in either the dark-field or phase-contrast images. Compared to the adjacent lung tissue, lung tumors were characterized by a significant decrease in transmission signal (median 0.86 vs. 0.91, p = 0.04) and increase in dark-field signal (median 0.71 vs. 0.65, p = 0.04). Median contrast-to-noise ratios for the visualization of lung nodules were 4.4 for transmission images and 1.7 for dark-field images (p = 0.04).

CONCLUSION

Lung nodules can be visualized on all three radiograph modalities derived from grating-based X-ray imaging. However, our initial data suggest that grating-based multimodal X-ray imaging does not increase the sensitivity of chest radiographs for the detection of lung nodules.