Does a Deep Learning-Based Computer-Assisted Diagnosis System Outperform Conventional Double Reading by Radiologists in Distinguishing Benign and Malignant Lung Nodules?

Standalone deep learning versus experts for diagnosis lung cancer on chest computed tomography: a systematic review

PURPOSE

To compare the diagnostic performance of standalone deep learning (DL) algorithms and human experts in lung cancer detection on chest computed tomography (CT) scans.

MATERIALS AND METHODS

This study searched for studies on PubMed, Embase, and Web of Science from their inception until November 2023. We focused on adult lung cancer patients and compared the efficacy of DL algorithms and expert radiologists in disease diagnosis on CT scans. Quality assessment was performed using QUADAS-2, QUADAS-C, and CLAIM. Bivariate random-effects and subgroup analyses were performed for tasks (malignancy classification vs invasiveness classification), imaging modalities (CT vs low-dose CT [LDCT] vs high-resolution CT), study region, software used, and publication year.

RESULTS

We included 20 studies on various aspects of lung cancer diagnosis on CT scans. Quantitatively, DL algorithms exhibited superior sensitivity (82%) and specificity (75%) compared to human experts (sensitivity 81%, specificity 69%). However, the difference in specificity was statistically significant, whereas the difference in sensitivity was not statistically significant. The DL algorithms' performance varied across different imaging modalities and tasks, demonstrating the need for tailored optimization of DL algorithms. Notably, DL algorithms matched experts in sensitivity on standard CT, surpassing them in specificity, but showed higher sensitivity with lower specificity on LDCT scans.

CONCLUSION

DL algorithms demonstrated improved accuracy over human readers in malignancy and invasiveness classification on CT scans. However, their performance varies by imaging modality, underlining the importance of continued research to fully assess DL algorithms' diagnostic effectiveness in lung cancer.

CLINICAL RELEVANCE STATEMENT

DL algorithms have the potential to refine lung cancer diagnosis on CT, matching human sensitivity and surpassing in specificity. These findings call for further DL optimization across imaging modalities, aiming to advance clinical diagnostics and patient outcomes.

KEY POINTS

Lung cancer diagnosis by CT is challenging and can be improved with AI integration. DL shows higher accuracy in lung cancer detection on CT than human experts. Enhanced DL accuracy could lead to improved lung cancer diagnosis and outcomes.

Validation of a commercially available CAD-system for lung nodule detection and characterization using CT-scans

OBJECTIVES

This study aims to externally validate a commercially available Computer-Aided Detection (CAD)-system for the automatic detection and characterization of solid, part-solid, and ground-glass lung nodules (LN) on CT scans.

METHODS

This retrospective study encompasses 263 chest CT scans performed between January 2020 and December 2021 at a Dutch university hospital. All scans were read by a radiologist (R1) and compared with the initial radiology report. Conflicting scans were assessed by an adjudicating radiologist (R2). All scans were also processed by CAD. The standalone performance of CAD in terms of sensitivity and false-positive (FP)-rate for detection was calculated together with the sensitivity for characterization, including texture, calcification, speculation, and location. The R1's detection sensitivity was also assessed.

RESULTS

A total of 183 true nodules were identified in 121 nodule-containing scans (142 non-nodule-containing scans), of which R1 identified 165/183 (90.2%). CAD detected 149 nodules, of which 12 were not identified by R1, achieving a sensitivity of 149/183 (81.4%) with an FP-rate of 49/121 (0.405). CAD's detection sensitivity for solid, part-solid, and ground-glass LNs was 82/94 (87.2%), 42/47 (89.4%), and 25/42 (59.5%), respectively. The classification accuracy for solid, part-solid, and ground-glass LNs was 81/82 (98.8%), 16/42 (38.1%), and 18/25 (72.0%), respectively. Additionally, CAD demonstrated overall classification accuracies of 137/149 (91.9%), 123/149 (82.6%), and 141/149 (94.6%) for calcification, spiculation, and location, respectively.

CONCLUSIONS

Although the overall detection rate of this system slightly lags behind that of a radiologist, CAD is capable of detecting different LNs and thereby has the potential to enhance a reader's detection rate. While promising characterization performances are obtained, the tool's performance in terms of texture classification remains a subject of concern.

CLINICAL RELEVANCE STATEMENT

Numerous lung nodule computer-aided detection-systems are commercially available, with some of them solely being externally validated based on their detection performance on solid nodules. We encourage researchers to assess performances by incorporating all relevant characteristics, including part-solid and ground-glass nodules.

KEY POINTS

Few computer-aided detection (CAD) systems are externally validated for automatic detection and characterization of lung nodules. A detection sensitivity of 81.4% and an overall texture classification sensitivity of 77.2% were measured utilizing CAD. CAD has the potential to increase single reader detection rate, however, improvement in texture classification is required.

The Development of an Intelligent Agent to Detect and Non-Invasively Characterize Lung Lesions on CT Scans: Ready for the "Real World"?

(1) Background: Once lung lesions are identified on CT scans, they must be characterized by assessing the risk of malignancy. Despite the promising performance of computer-aided systems, some limitations related to the study design and technical issues undermine these tools' efficiency; an "intelligent agent" to detect and non-invasively characterize lung lesions on CT scans is proposed. (2) Methods: Two main modules tackled the detection of lung nodules on CT scans and the diagnosis of each nodule into benign and malignant categories. Computer-aided detection (CADe) and computer aided-diagnosis (CADx) modules relied on deep learning techniques such as Retina U-Net and the convolutional neural network; (3) Results: Tests were conducted on one publicly available dataset and two local datasets featuring CT scans acquired with different devices to reveal deep learning performances in "real-world" clinical scenarios. The CADe module reached an accuracy rate of 78%, while the CADx's accuracy, specificity, and sensitivity stand at 80%, 73%, and 85.7%, respectively; (4) Conclusions: Two different deep learning techniques have been adapted for CADe and CADx purposes in both publicly available and private CT scan datasets. Experiments have shown adequate performance in both detection and diagnosis tasks. Nevertheless, some drawbacks still characterize the supervised learning paradigm employed in networks such as CNN and Retina U-Net in real-world clinical scenarios, with CT scans from different devices with different sensors' fingerprints and spatial resolution. Continuous reassessment of CADe and CADx's performance is needed during their implementation in clinical practice.

A Propensity-Matched Analysis of Survival of Clinically Diagnosed Early-Stage Lung Cancer and Biopsy-Proven Early-Stage Non-Small Cell Lung Cancer Following Stereotactic Ablative Radiotherapy

Purpose

Stereotactic body radiotherapy (SBRT) has been increasingly regarded as a reasonable option for early-stage lung cancer patients without pretreatment pathologic results, but the efficacy and safety in a Chinese population remains unclear. The aim of this study was to compare survival outcomes and toxicities between patients with clinically diagnosed early-stage lung cancer or biopsy-proven early-stage non-small cell lung cancer and to demonstrate the rationality of this treatment.

Material and Methods

From May 2012 to December 2018, 56 patients with clinically diagnosed early-stage lung cancer and 60 patients with early-stage biopsy-proven were selected into non-pathological group and pathological group, respectively. Propensity score matching (PSM) was performed to reduce patient selection bias. Survival analysis with log-rank test was used to assess the differences of treatment outcomes, which included local control (LC), progression-free survival (PFS), and overall survival (OS).

Results

The median age was 76 (range 47–93) years, and the median follow-up time was 58.3 (range 4.3–95.1) months in the cohort without pathologic results. The median age was 74 (range 57–88) years, and the median follow-up time was 56.3 (range 2.6–94) months in the cohort with pathologic results. 45 matched-pair were analyzed. The 5-year LC, PFS, and OS rates in matched-pair patients with or without pathologic biopsy were 85.5% and 89.8%, 40.6% and 70.9%, and 63.2% and 76.1%, respectively. On Kaplan-Meier survival analysis after PSM analysis, there was no significant difference between patients with pathologic results versus patients with no pathologic results in terms of LC (P= 0.498) and OS (P=0.141). Of the matched-pair patients treated with SBRT, only 1 patient experienced grade 3 or above radiation pneumonitis.

Conclusion

For early-stage lung cancer patients with medically inoperable or not suitable for invasive diagnosis, SBRT may be a good local treatment.

Machine Learning and Syncope Management in the ED: The Future Is Coming

In recent years, machine learning (ML) has been promisingly applied in many fields of clinical medicine, both for diagnosis and prognosis prediction. Aims of this narrative review were to summarize the basic concepts of ML applied to clinical medicine and explore its main applications in the emergency department (ED) setting, with a particular focus on syncope management. Through an extensive literature search in PubMed and Embase, we found increasing evidence suggesting that the use of ML algorithms can improve ED triage, diagnosis, and risk stratification of many diseases. However, the lacks of external validation and reliable diagnostic standards currently limit their implementation in clinical practice. Syncope represents a challenging problem for the emergency physician both because its diagnosis is not supported by specific tests and the available prognostic tools proved to be inefficient. ML algorithms have the potential to overcome these limitations and, in the future, they could support the clinician in managing syncope patients more efficiently. However, at present only few studies have addressed this issue, albeit with encouraging results.