Prospective Study of Spatial Distribution of Missed Lung Nodules by Readers in CT Lung Screening Using Computer-assisted Detection

Effect of emphysema on AI software and human reader performance in lung nodule detection from low-dose chest CT

BACKGROUND

Emphysema influences the appearance of lung tissue in computed tomography (CT). We evaluated whether this affects lung nodule detection by artificial intelligence (AI) and human readers (HR).

METHODS

Individuals were selected from the "Lifelines" cohort who had undergone low-dose chest CT. Nodules in individuals without emphysema were matched to similar-sized nodules in individuals with at least moderate emphysema. AI results for nodular findings of 30-100 mm3 and 101-300 mm3 were compared to those of HR; two expert radiologists blindly reviewed discrepancies. Sensitivity and false positives (FPs)/scan were compared for emphysema and non-emphysema groups.

RESULTS

Thirty-nine participants with and 82 without emphysema were included (n = 121, aged 61 ± 8 years (mean ± standard deviation), 58/121 males (47.9%)). AI and HR detected 196 and 206 nodular findings, respectively, yielding 109 concordant nodules and 184 discrepancies, including 118 true nodules. For AI, sensitivity was 0.68 (95% confidence interval 0.57-0.77) in emphysema versus 0.71 (0.62-0.78) in non-emphysema, with FPs/scan 0.51 and 0.22, respectively (p = 0.028). For HR, sensitivity was 0.76 (0.65-0.84) and 0.80 (0.72-0.86), with FPs/scan of 0.15 and 0.27 (p = 0.230). Overall sensitivity was slightly higher for HR than for AI, but this difference disappeared after the exclusion of benign lymph nodes. FPs/scan were higher for AI in emphysema than in non-emphysema (p = 0.028), while FPs/scan for HR were higher than AI for 30-100 mm3 nodules in non-emphysema (p = 0.009).

CONCLUSIONS

AI resulted in more FPs/scan in emphysema compared to non-emphysema, a difference not observed for HR.

RELEVANCE STATEMENT

In the creation of a benchmark dataset to validate AI software for lung nodule detection, the inclusion of emphysema cases is important due to the additional number of FPs.

KEY POINTS

• The sensitivity of nodule detection by AI was similar in emphysema and non-emphysema. • AI had more FPs/scan in emphysema compared to non-emphysema. • Sensitivity and FPs/scan by the human reader were comparable for emphysema and non-emphysema. • Emphysema and non-emphysema representation in benchmark dataset is important for validating AI.

Performance changes due to differences among annotating radiologists for training data in computerized lesion detection

PURPOSE

The quality and bias of annotations by annotators (e.g., radiologists) affect the performance changes in computer-aided detection (CAD) software using machine learning. We hypothesized that the difference in the years of experience in image interpretation among radiologists contributes to annotation variability. In this study, we focused on how the performance of CAD software changes with retraining by incorporating cases annotated by radiologists with varying experience.

METHODS

We used two types of CAD software for lung nodule detection in chest computed tomography images and cerebral aneurysm detection in magnetic resonance angiography images. Twelve radiologists with different years of experience independently annotated the lesions, and the performance changes were investigated by repeating the retraining of the CAD software twice, with the addition of cases annotated by each radiologist. Additionally, we investigated the effects of retraining using integrated annotations from multiple radiologists.

RESULTS

The performance of the CAD software after retraining differed among annotating radiologists. In some cases, the performance was degraded compared to that of the initial software. Retraining using integrated annotations showed different performance trends depending on the target CAD software, notably in cerebral aneurysm detection, where the performance decreased compared to using annotations from a single radiologist.

CONCLUSIONS

Although the performance of the CAD software after retraining varied among the annotating radiologists, no direct correlation with their experience was found. The performance trends differed according to the type of CAD software used when integrated annotations from multiple radiologists were used.

Study rationale and design of the PEOPLHE trial

PURPOSE

Lung cancer screening (LCS) by low-dose computed tomography (LDCT) demonstrated a 20-40% reduction in lung cancer mortality. National stakeholders and international scientific societies are increasingly endorsing LCS programs, but translating their benefits into practice is rather challenging. The "Model for Optimized Implementation of Early Lung Cancer Detection: Prospective Evaluation Of Preventive Lung HEalth" (PEOPLHE) is an Italian multicentric LCS program aiming at testing LCS feasibility and implementation within the national healthcare system. PEOPLHE is intended to assess (i) strategies to optimize LCS workflow, (ii) radiological quality assurance, and (iii) the need for dedicated resources, including smoking cessation facilities.

METHODS

PEOPLHE aims to recruit 1.500 high-risk individuals across three tertiary general hospitals in three different Italian regions that provide comprehensive services to large populations to explore geographic, demographic, and socioeconomic diversities. Screening by LDCT will target current or former (quitting < 10 years) smokers (> 15 cigarettes/day for > 25 years, or > 10 cigarettes/day for > 30 years) aged 50-75 years. Lung nodules will be volumetric measured and classified by a modified PEOPLHE Lung-RADS 1.1 system. Current smokers will be offered smoking cessation support.

CONCLUSION

The PEOPLHE program will provide information on strategies for screening enrollment and smoking cessation interventions; administrative, organizational, and radiological needs for performing a state-of-the-art LCS; collateral and incidental findings (both pulmonary and extrapulmonary), contributing to the LCS implementation within national healthcare systems.

Deep learning for the detection of benign and malignant pulmonary nodules in non-screening chest CT scans

BACKGROUND

Outside a screening program, early-stage lung cancer is generally diagnosed after the detection of incidental nodules in clinically ordered chest CT scans. Despite the advances in artificial intelligence (AI) systems for lung cancer detection, clinical validation of these systems is lacking in a non-screening setting.

METHOD

We developed a deep learning-based AI system and assessed its performance for the detection of actionable benign nodules (requiring follow-up), small lung cancers, and pulmonary metastases in CT scans acquired in two Dutch hospitals (internal and external validation). A panel of five thoracic radiologists labeled all nodules, and two additional radiologists verified the nodule malignancy status and searched for any missed cancers using data from the national Netherlands Cancer Registry. The detection performance was evaluated by measuring the sensitivity at predefined false positive rates on a free receiver operating characteristic curve and was compared with the panel of radiologists.

RESULTS

On the external test set (100 scans from 100 patients), the sensitivity of the AI system for detecting benign nodules, primary lung cancers, and metastases is respectively 94.3% (82/87, 95% CI: 88.1-98.8%), 96.9% (31/32, 95% CI: 91.7-100%), and 92.0% (104/113, 95% CI: 88.5-95.5%) at a clinically acceptable operating point of 1 false positive per scan (FP/s). These sensitivities are comparable to or higher than the radiologists, albeit with a slightly higher FP/s (average difference of 0.6).

CONCLUSIONS

The AI system reliably detects benign and malignant pulmonary nodules in clinically indicated CT scans and can potentially assist radiologists in this setting.

Computer-aided detection of pulmonary nodules based on convolutional neural networks: a review

Computer-aided detection (CADe) technology has been proven to increase the detection rate of pulmonary nodules that has important clinical significance for the early diagnosis of lung cancer. In this study, we systematically review the latest techniques in pulmonary nodule CADe based on deep learning models with convolutional neural networks in computed tomography images. First, the brief descriptions and popular architecture of convolutional neural networks are introduced. Second, several common public databases and evaluation metrics are briefly described. Third, state-of-the-art approaches with excellent performances are selected. Subsequently, we combine the clinical diagnostic process and the traditional four steps of pulmonary nodule CADe into two stages, namely, data preprocessing and image analysis. Further, the major optimizations of deep learning models and algorithms are highlighted according to the progressive evaluation effect of each method, and some clinical evidence is added. Finally, various methods are summarized and compared. The innovative or valuable contributions of each method are expected to guide future research directions. The analyzed results show that deep learning-based methods significantly transformed the detection of pulmonary nodules, and the design of these methods can be inspired by clinical imaging diagnostic procedures. Moreover, focusing on the image analysis stage will result in improved returns. In particular, optimal results can be achieved by optimizing the steps of candidate nodule generation and false positive reduction. End-to-end methods, with greater operating speeds and lower computational consumptions, are superior to other methods in CADe of pulmonary nodules.

Paraneoplastic Cerebellar Degeneration and Lambert-Eaton Myasthenic Syndrome with SOX-1 Antibodies

A 69-year-old man was admitted to our hospital for progressive muscle weakness in both lower limbs and limb ataxia (day 0). Nerve conduction studies showed low compound muscle action potential amplitudes at rest and increased amplitudes after maximum voluntary contraction. Blood testing revealed SOX-1 antibodies. He was diagnosed with paraneoplastic cerebellar degeneration and Lambert-Eaton myasthenic syndrome (PCD-LEMS). He died from aspiration pneumonia on day 9. Small-cell lung carcinoma (SCLC), which had not been obvious on computed tomography, was found during the autopsy. Patients with PCD-LEMS who test positive for SOX-1 antibodies should be carefully evaluated for SCLC.