Determining the invasiveness of ground-glass nodules using a 3D multi-task network

Res-TransNet: A Hybrid deep Learning Network for Predicting Pathological Subtypes of lung Adenocarcinoma in CT Images

This study aims to develop a CT-based hybrid deep learning network to predict pathological subtypes of early-stage lung adenocarcinoma by integrating residual network (ResNet) with Vision Transformer (ViT). A total of 1411 pathologically confirmed ground-glass nodules (GGNs) retrospectively collected from two centers were used as internal and external validation sets for model development. 3D ResNet and ViT were applied to investigate two deep learning frameworks to classify three subtypes of lung adenocarcinoma namely invasive adenocarcinoma (IAC), minimally invasive adenocarcinoma and adenocarcinoma in situ, respectively. To further improve the model performance, four Res-TransNet based models were proposed by integrating ResNet and ViT with different ensemble learning strategies. Two classification tasks involving predicting IAC from Non-IAC (Task1) and classifying three subtypes (Task2) were designed and conducted in this study. For Task 1, the optimal Res-TransNet model yielded area under the receiver operating characteristic curve (AUC) values of 0.986 and 0.933 on internal and external validation sets, which were significantly higher than that of ResNet and ViT models (p < 0.05). For Task 2, the optimal fusion model generated the accuracy and weighted F1 score of 68.3% and 66.1% on the external validation set. The experimental results demonstrate that Res-TransNet can significantly increase the classification performance compared with the two basic models and have the potential to assist radiologists in precision diagnosis.

Predicting Invasiveness of Lung Adenocarcinoma at Chest CT with Deep Learning Ternary Classification Models

Background Preoperative discrimination of preinvasive, minimally invasive, and invasive adenocarcinoma at CT informs clinical management decisions but may be challenging for classifying pure ground-glass nodules (pGGNs). Deep learning (DL) may improve ternary classification. Purpose To determine whether a strategy that includes an adjudication approach can enhance the performance of DL ternary classification models in predicting the invasiveness of adenocarcinoma at chest CT and maintain performance in classifying pGGNs. Materials and Methods In this retrospective study, six ternary models for classifying preinvasive, minimally invasive, and invasive adenocarcinoma were developed using a multicenter data set of lung nodules. The DL-based models were progressively modified through framework optimization, joint learning, and an adjudication strategy (simulating a multireader approach to resolving discordant nodule classifications), integrating two binary classification models with a ternary classification model to resolve discordant classifications sequentially. The six ternary models were then tested on an external data set of pGGNs imaged between December 2019 and January 2021. Diagnostic performance including accuracy, specificity, and sensitivity was assessed. The χ2 test was used to compare model performance in different subgroups stratified by clinical confounders. Results A total of 4929 nodules from 4483 patients (mean age, 50.1 years ± 9.5 [SD]; 2806 female) were divided into training (n = 3384), validation (n = 579), and internal (n = 966) test sets. A total of 361 pGGNs from 281 patients (mean age, 55.2 years ± 11.1 [SD]; 186 female) formed the external test set. The proposed strategy improved DL model performance in external testing (P < .001). For classifying minimally invasive adenocarcinoma, the accuracy was 85% and 79%, sensitivity was 75% and 63%, and specificity was 89% and 85% for the model with adjudication (model 6) and the model without (model 3), respectively. Model 6 showed a relatively narrow range (maximum minus minimum) across diagnostic indexes (accuracy, 1.7%; sensitivity, 7.3%; specificity, 0.9%) compared with the other models (accuracy, 0.6%-10.8%; sensitivity, 14%-39.1%; specificity, 5.5%-17.9%). Conclusion Combining framework optimization, joint learning, and an adjudication approach improved DL classification of adenocarcinoma invasiveness at chest CT. Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Sohn and Fields in this issue.

An ensemble deep learning model for risk stratification of invasive lung adenocarcinoma using thin-slice CT

Lung cancer screening using computed tomography (CT) has increased the detection rate of small pulmonary nodules and early-stage lung adenocarcinoma. It would be clinically meaningful to accurate assessment of the nodule histology by CT scans with advanced deep learning algorithms. However, recent studies mainly focus on predicting benign and malignant nodules, lacking of model for the risk stratification of invasive adenocarcinoma. We propose an ensemble multi-view 3D convolutional neural network (EMV-3D-CNN) model to study the risk stratification of lung adenocarcinoma. We include 1075 lung nodules (≤30 mm and ≥4 mm) with preoperative thin-section CT scans and definite pathology confirmed by surgery. Our model achieves a state-of-art performance of 91.3% and 92.9% AUC for diagnosis of benign/malignant and pre-invasive/invasive nodules, respectively. Importantly, our model outperforms senior doctors in risk stratification of invasive adenocarcinoma with 77.6% accuracy [i.e., Grades 1, 2, 3]). It provides detailed predictive histological information for the surgical management of pulmonary nodules. Finally, for user-friendly access, the proposed model is implemented as a web-based system ( https://seeyourlung.com.cn ).

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

A deep learning approach for anterior cruciate ligament rupture localization on knee MR images

Purpose: To develop and evaluate a deep learning-based method to localize and classify anterior cruciate ligament (ACL) ruptures on knee MR images by using arthroscopy as the reference standard.

Methods: We proposed a fully automated ACL rupture localization system to localize and classify ACL ruptures. The classification of ACL ruptures was based on the projection coordinates of the ACL rupture point on the line connecting the center coordinates of the femoral and tibial footprints. The line was divided into three equal parts and the position of the projection coordinates indicated the classification of the ACL ruptures (femoral side, middle and tibial side). In total, 85 patients (mean age: 27; male: 56) who underwent ACL reconstruction surgery under arthroscopy were included. Three clinical readers evaluated the datasets separately and their diagnostic performances were compared with those of the model. The performance metrics included the accuracy, error rate, sensitivity, specificity, precision, and F1-score. A one-way ANOVA was used to evaluate the performance of the convolutional neural networks (CNNs) and clinical readers. Intraclass correlation coefficients (ICC) were used to assess interobserver agreement between the clinical readers.

Results: The accuracy of ACL localization was 3.77 ± 2.74 and 4.68 ± 3.92 (mm) for three-dimensional (3D) and two-dimensional (2D) CNNs, respectively. There was no significant difference in the ACL rupture location performance between the 3D and 2D CNNs or among the clinical readers (Accuracy, p < 0.01). The 3D CNNs performed best among the five evaluators in classifying the femoral side (sensitivity of 0.86 and specificity of 0.79), middle side (sensitivity of 0.71 and specificity of 0.84) and tibial side ACL rupture (sensitivity of 0.71 and specificity of 0.99), and the overall accuracy for sides classifying of ACL rupture achieved 0.79.

Conclusion: The proposed deep learning-based model achieved high diagnostic performances in locating and classifying ACL fractures on knee MR images.

Clinical impact of a deep learning system for automated detection of missed pulmonary nodules on routine body computed tomography including the chest region

OBJECTIVES

To evaluate the clinical impact of a deep learning system (DLS) for automated detection of pulmonary nodules on computed tomography (CT) images as a second reader.

METHODS

This single-centre retrospective study screened 21,150 consecutive body CT studies from September 2018 to February 2019. Pulmonary nodules detected by the DLS on axial CT images but not mentioned in initial radiology reports were flagged. Flagged images were scored by four board-certificated radiologists each with at least 5 years of experience. Nodules with scores of 2 (understandable miss) or 3 (should not be missed) were then categorised as unlikely to be clinically significant (2a or 3a) or likely to be clinically significant (2b or 3b) according to the 2017 Fleischner guidelines for pulmonary nodules. The miss rate was defined as the total number of studies receiving scores of 2 or 3 divided by total screened studies.

RESULTS

Among 172 nodules flagged by the DLS, 60 (35%) missed nodules were confirmed by the radiologists. The nodules were further categorised as 2a, 2b, 3a, and 3b in 24, 14, 10, and 12 studies, respectively, with an overall positive predictive value of 35%. Missed pulmonary nodules were identified in 0.3% of all CT images, and one-third of these lesions were considered clinically significant.

CONCLUSIONS

Use of DLS-assisted automated detection as a second reader can identify missed pulmonary nodules, some of which may be clinically significant.

CLINICAL RELEVANCE/APPLICATION

Use of DLS to help radiologists detect pulmonary lesions may improve patient care.

KEY POINTS

• DLS-assisted automated detection as a second reader is feasible in a large consecutive cohort. • Performance of combined radiologists and DLS was better than DLS or radiologists alone. • Pulmonary nodules were missed more frequently in abdomino-pelvis CT than the thoracic CT.

[Clinical Study of Artificial Intelligence-assisted Diagnosis System in Predicting the Invasive Subtypes of Early-stage Lung Adenocarcinoma Appearing as Pulmonary Nodules]

BACKGROUND

Lung cancer is the cancer with the highest mortality at home and abroad at present. The detection of lung nodules is a key step to reducing the mortality of lung cancer. Artificial intelligence-assisted diagnosis system presents as the state of the art in the area of nodule detection, differentiation between benign and malignant and diagnosis of invasive subtypes, however, a validation with clinical data is necessary for further application. Therefore, the aim of this study is to evaluate the effectiveness of artificial intelligence-assisted diagnosis system in predicting the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules.

METHODS

Clinical data of 223 patients with early-stage lung adenocarcinoma appearing as pulmonary nodules admitted to the Lanzhou University Second Hospital from January 1st, 2016 to December 31th, 2021 were retrospectively analyzed, which were divided into invasive adenocarcinoma group (n=170) and non-invasive adenocarcinoma group (n=53), and the non-invasive adenocarcinoma group was subdivided into minimally invasive adenocarcinoma group (n=31) and preinvasive lesions group (n=22). The malignant probability and imaging characteristics of each group were compared to analyze their predictive ability for the invasive subtypes of early-stage lung adenocarcinoma. The concordance between qualitative diagnostic results of artificial intelligence-assisted diagnosis of the invasive subtypes of early-stage lung adenocarcinoma and postoperative pathology was then analyzed.

RESULTS

In different invasive subtypes of early-stage lung adenocarcinoma, the mean CT value of pulmonary nodules (P<0.001), diameter (P<0.001), volume (P<0.001), malignant probability (P<0.001), pleural retraction sign (P<0.001), lobulation (P<0.001), spiculation (P<0.001) were significantly different. At the same time, it was also found that with the increased invasiveness of different invasive subtypes of early-stage lung adenocarcinoma, the proportion of dominant signs of each group gradually increased. On the issue of binary classification, the sensitivity, specificity, and area under the curve (AUC) values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 81.76%, 92.45% and 0.871 respectively. On the issue of three classification, the accuracy, recall rate, F1 score, and AUC values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 83.86%, 85.03%, 76.46% and 0.879 respectively.

CONCLUSIONS

Artificial intelligence-assisted diagnosis system could predict the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules, and has a certain predictive value. With the optimization of algorithms and the improvement of data, it may provide guidance for individualized treatment of patients.

[Chinese Experts Consensus on Artificial Intelligence Assisted Management for Pulmonary Nodule (2022 Version)]

Low-dose computed tomography (CT) for lung cancer screening has been proven to reduce lung cancer deaths in the screening group compared with the control group. The increasing number of pulmonary nodules being detected by CT scans significantly increase the workload of the radiologists for scan interpretation. Artificial intelligence (AI) has the potential to increase the efficiency of pulmonary nodule discrimination and has been tested in preliminary studies for nodule management. As more and more artificial AI products are commercialized, the consensus statement has been organized in a collaborative effort by Thoracic Surgery Committee, Department of Simulated Medicine, Wu Jieping Medical Foundation to aid clinicians in the application of AI-assisted management for pulmonary nodules.
.

IMAL-Net: Interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images

PURPOSE

Feature maps created from deep convolutional neural networks (DCNNs) have been widely used for visual explanation of DCNN-based classification tasks. However, many clinical applications such as benign-malignant classification of lung nodules normally require quantitative and objective interpretability, rather than just visualization. In this paper, we propose a novel interpretable multi-task attention learning network named IMAL-Net for early invasive adenocarcinoma screening in chest computed tomography images, which takes advantage of segmentation prior to assist interpretable classification.

METHODS

Two sub-ResNets are firstly integrated together via a prior-attention mechanism for simultaneous nodule segmentation and invasiveness classification. Then, numerous radiomic features from the segmentation results are concatenated with high-level semantic features from the classification subnetwork by FC layers to achieve superior performance. Meanwhile, an end-to-end feature selection mechanism (named FSM) is designed to quantify crucial radiomic features greatly affecting the prediction of each sample, and thus it can provide clinically applicable interpretability to the prediction result.

RESULTS

Nodule samples from a total of 1626 patients were collected from two grade-A hospitals for large-scale verification. Five-fold cross validation demonstrated that the proposed IMAL-Net can achieve an AUC score of 93.8% ± 1.1% and a recall score of 93.8% ± 2.8% for identification of invasive lung adenocarcinoma.

CONCLUSIONS

It can be concluded that fusing semantic features and radiomic features can achieve obvious improvements in the invasiveness classification task. Moreover, by learning more fine-grained semantic features and highlighting the most important radiomics features, the proposed attention and FSM mechanisms not only can further improve the performance but also can be used for both visual explanations and objective analysis of the classification results.

Consecutive Serial Non-Contrast CT Scan-Based Deep Learning Model Facilitates the Prediction of Tumor Invasiveness of Ground-Glass Nodules

Introduction

Tumors are continuously evolving biological systems which can be monitored by medical imaging. Previous studies only focus on single timepoint images, whether the performance could be further improved by using serial noncontrast CT imaging obtained during nodule follow-up management remains unclear. In this study, we evaluated DL model for predicting tumor invasiveness of GGNs through analyzing time series CT images.

Methods

A total of 168 pathologically confirmed GGN cases (48 noninvasive lesions and 120 invasive lesions) were retrospectively collected and randomly assigned to the development dataset (n = 123) and independent testing dataset (n = 45). All patients underwent consecutive noncontrast CT examinations, and the baseline CT and 3-month follow-up CT images were collected. The gross region of interest (ROI) patches containing only tumor region and the full ROI patches including both tumor and peritumor regions were cropped from CT images. A baseline model was built on the image features and demographic features. Four DL models were proposed: two single-DL model using gross ROI (model 1) or full ROI patches (model 3) from baseline CT images, and two serial-DL models using gross ROI (model 2) or full ROI patches (model 4) from consecutive CT images (baseline scan and 3-month follow-up scan). In addition, a combined model integrating serial full ROI patches and clinical information was also constructed. The performance of these predictive models was assessed with respect to discrimination and clinical usefulness.

Results

The area under the curve (AUC) of the baseline model, models 1, 2, 3, and 4 were 0.562 [(95% confidence interval (C)], 0.406~0.710), 0.693 (95% CI, 0.538-0.822), 0.787 (95% CI, 0.639-0.895), 0.727 (95% CI, 0.573-0.849), and 0.811 (95% CI, 0.667-0.912) in the independent testing dataset, respectively. The results indicated that the peritumor region had potential to contribute to tumor invasiveness prediction, and the model performance was further improved by integrating imaging scans at multiple timepoints. Furthermore, the combined model showed best discrimination ability, with AUC, sensitivity, specificity, and accuracy achieving 0.831 (95% CI, 0.690-0.926), 86.7%, 73.3%, and 82.2%, respectively.

Conclusion

The DL model integrating full ROIs from serial CT images shows improved predictive performance in differentiating noninvasive from invasive GGNs than the model using only baseline CT images, which could benefit the clinical management of GGNs.