1
|
Su Y, Xia X, Sun R, Yuan J, Hua Q, Han B, Gong J, Nie S. Res-TransNet: A Hybrid deep Learning Network for Predicting Pathological Subtypes of lung Adenocarcinoma in CT Images. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01149-z. [PMID: 38861071 DOI: 10.1007/s10278-024-01149-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 04/30/2024] [Accepted: 05/22/2024] [Indexed: 06/12/2024]
Abstract
This study aims to develop a CT-based hybrid deep learning network to predict pathological subtypes of early-stage lung adenocarcinoma by integrating residual network (ResNet) with Vision Transformer (ViT). A total of 1411 pathologically confirmed ground-glass nodules (GGNs) retrospectively collected from two centers were used as internal and external validation sets for model development. 3D ResNet and ViT were applied to investigate two deep learning frameworks to classify three subtypes of lung adenocarcinoma namely invasive adenocarcinoma (IAC), minimally invasive adenocarcinoma and adenocarcinoma in situ, respectively. To further improve the model performance, four Res-TransNet based models were proposed by integrating ResNet and ViT with different ensemble learning strategies. Two classification tasks involving predicting IAC from Non-IAC (Task1) and classifying three subtypes (Task2) were designed and conducted in this study. For Task 1, the optimal Res-TransNet model yielded area under the receiver operating characteristic curve (AUC) values of 0.986 and 0.933 on internal and external validation sets, which were significantly higher than that of ResNet and ViT models (p < 0.05). For Task 2, the optimal fusion model generated the accuracy and weighted F1 score of 68.3% and 66.1% on the external validation set. The experimental results demonstrate that Res-TransNet can significantly increase the classification performance compared with the two basic models and have the potential to assist radiologists in precision diagnosis.
Collapse
Affiliation(s)
- Yue Su
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Xianwu Xia
- Department of Oncology Intervention, Municipal Hospital Affiliated of Taizhou University, Zhejiang, Taizhou, 318000, China
| | - Rong Sun
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Jianjun Yuan
- Department of Oncology Intervention, Municipal Hospital Affiliated of Taizhou University, Zhejiang, Taizhou, 318000, China
| | - Qianjin Hua
- Department of Oncology Intervention, Municipal Hospital Affiliated of Taizhou University, Zhejiang, Taizhou, 318000, China
| | - Baosan Han
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China.
- Department of Breast Surgery, Xinhua Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200092, China.
| | - Jing Gong
- Department of Radiology, Fudan University Shanghai Cancer Center, 270 Dongan Road, Shanghai, 200032, China.
- Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, 200032, China.
| | - Shengdong Nie
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China.
| |
Collapse
|
2
|
Pan Z, Hu G, Zhu Z, Tan W, Han W, Zhou Z, Song W, Yu Y, Song L, Jin Z. Predicting Invasiveness of Lung Adenocarcinoma at Chest CT with Deep Learning Ternary Classification Models. Radiology 2024; 311:e232057. [PMID: 38591974 DOI: 10.1148/radiol.232057] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/10/2024]
Abstract
Background Preoperative discrimination of preinvasive, minimally invasive, and invasive adenocarcinoma at CT informs clinical management decisions but may be challenging for classifying pure ground-glass nodules (pGGNs). Deep learning (DL) may improve ternary classification. Purpose To determine whether a strategy that includes an adjudication approach can enhance the performance of DL ternary classification models in predicting the invasiveness of adenocarcinoma at chest CT and maintain performance in classifying pGGNs. Materials and Methods In this retrospective study, six ternary models for classifying preinvasive, minimally invasive, and invasive adenocarcinoma were developed using a multicenter data set of lung nodules. The DL-based models were progressively modified through framework optimization, joint learning, and an adjudication strategy (simulating a multireader approach to resolving discordant nodule classifications), integrating two binary classification models with a ternary classification model to resolve discordant classifications sequentially. The six ternary models were then tested on an external data set of pGGNs imaged between December 2019 and January 2021. Diagnostic performance including accuracy, specificity, and sensitivity was assessed. The χ2 test was used to compare model performance in different subgroups stratified by clinical confounders. Results A total of 4929 nodules from 4483 patients (mean age, 50.1 years ± 9.5 [SD]; 2806 female) were divided into training (n = 3384), validation (n = 579), and internal (n = 966) test sets. A total of 361 pGGNs from 281 patients (mean age, 55.2 years ± 11.1 [SD]; 186 female) formed the external test set. The proposed strategy improved DL model performance in external testing (P < .001). For classifying minimally invasive adenocarcinoma, the accuracy was 85% and 79%, sensitivity was 75% and 63%, and specificity was 89% and 85% for the model with adjudication (model 6) and the model without (model 3), respectively. Model 6 showed a relatively narrow range (maximum minus minimum) across diagnostic indexes (accuracy, 1.7%; sensitivity, 7.3%; specificity, 0.9%) compared with the other models (accuracy, 0.6%-10.8%; sensitivity, 14%-39.1%; specificity, 5.5%-17.9%). Conclusion Combining framework optimization, joint learning, and an adjudication approach improved DL classification of adenocarcinoma invasiveness at chest CT. Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Sohn and Fields in this issue.
Collapse
Affiliation(s)
- Zhengsong Pan
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Ge Hu
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Zhenchen Zhu
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Weixiong Tan
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Wei Han
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Zhen Zhou
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Wei Song
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Yizhou Yu
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Lan Song
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| | - Zhengyu Jin
- From the Department of Radiology (Z.P., Z. Zhu, W.S., L.S., Z.J.), Medical Research Center (G.H.), State Key Laboratory of Complex Severe and Rare Disease (G.H.), Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 1 Shuaifuyuan, Dongcheng District, Beijing 100730, China; 4 + 4 Medical Doctor Program (Z.P., Z. Zhu), Department of Epidemiology and Health Statistics (W.H.), Institute of Basic Medicine Sciences (W.H.), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China; Deepwise AI Laboratory, Beijing Deepwise & League of PhD Technology, Beijing, China (W.T., Z. Zhou, Y.Y.); and Department of Computer Science, The University of Hong Kong, Hong Kong, China (Y.Y.)
| |
Collapse
|
3
|
Zhou J, Hu B, Feng W, Zhang Z, Fu X, Shao H, Wang H, Jin L, Ai S, Ji Y. An ensemble deep learning model for risk stratification of invasive lung adenocarcinoma using thin-slice CT. NPJ Digit Med 2023; 6:119. [PMID: 37407729 DOI: 10.1038/s41746-023-00866-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 06/26/2023] [Indexed: 07/07/2023] Open
Abstract
Lung cancer screening using computed tomography (CT) has increased the detection rate of small pulmonary nodules and early-stage lung adenocarcinoma. It would be clinically meaningful to accurate assessment of the nodule histology by CT scans with advanced deep learning algorithms. However, recent studies mainly focus on predicting benign and malignant nodules, lacking of model for the risk stratification of invasive adenocarcinoma. We propose an ensemble multi-view 3D convolutional neural network (EMV-3D-CNN) model to study the risk stratification of lung adenocarcinoma. We include 1075 lung nodules (≤30 mm and ≥4 mm) with preoperative thin-section CT scans and definite pathology confirmed by surgery. Our model achieves a state-of-art performance of 91.3% and 92.9% AUC for diagnosis of benign/malignant and pre-invasive/invasive nodules, respectively. Importantly, our model outperforms senior doctors in risk stratification of invasive adenocarcinoma with 77.6% accuracy [i.e., Grades 1, 2, 3]). It provides detailed predictive histological information for the surgical management of pulmonary nodules. Finally, for user-friendly access, the proposed model is implemented as a web-based system ( https://seeyourlung.com.cn ).
Collapse
Affiliation(s)
- Jing Zhou
- Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, China
| | - Bin Hu
- Department of Thoracic Surgery, Beijing Institute of Respiratory Medicine and Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Wei Feng
- Department of Cardiothoracic Surgery, The Third Xiangya Hospital of Central South University, Changsha, China
| | - Zhang Zhang
- Department of Thoracic Surgery, Changsha Central Hospital, Changsha, China
| | - Xiaotong Fu
- Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, China
| | - Handie Shao
- Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, China
| | - Hansheng Wang
- Guanghua School of Management, Peking University, Beijing, China
| | - Longyu Jin
- Department of Cardiothoracic Surgery, The Third Xiangya Hospital of Central South University, Changsha, China
| | - Siyuan Ai
- Department of Thoracic Surgery, Beijing LIANGXIANG Hospital, Beijing, China
| | - Ying Ji
- Department of Thoracic Surgery, Beijing Institute of Respiratory Medicine and Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China.
| |
Collapse
|
4
|
Zhao Y, Wang X, Che T, Bao G, Li S. Multi-task deep learning for medical image computing and analysis: A review. Comput Biol Med 2023; 153:106496. [PMID: 36634599 DOI: 10.1016/j.compbiomed.2022.106496] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 12/06/2022] [Accepted: 12/27/2022] [Indexed: 12/29/2022]
Abstract
The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.
Collapse
Affiliation(s)
- Yan Zhao
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China
| | - Xiuying Wang
- School of Computer Science, The University of Sydney, Sydney, NSW, 2008, Australia.
| | - Tongtong Che
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China
| | - Guoqing Bao
- School of Computer Science, The University of Sydney, Sydney, NSW, 2008, Australia
| | - Shuyu Li
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, 100875, China.
| |
Collapse
|
5
|
Qu C, Yang H, Wang C, Wang C, Ying M, Chen Z, Yang K, Zhang J, Li K, Dimitriou D, Tsai TY, Liu X. A deep learning approach for anterior cruciate ligament rupture localization on knee MR images. Front Bioeng Biotechnol 2022; 10:1024527. [PMID: 36246358 PMCID: PMC9561886 DOI: 10.3389/fbioe.2022.1024527] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2022] [Accepted: 09/14/2022] [Indexed: 11/13/2022] Open
Abstract
Purpose: To develop and evaluate a deep learning-based method to localize and classify anterior cruciate ligament (ACL) ruptures on knee MR images by using arthroscopy as the reference standard. Methods: We proposed a fully automated ACL rupture localization system to localize and classify ACL ruptures. The classification of ACL ruptures was based on the projection coordinates of the ACL rupture point on the line connecting the center coordinates of the femoral and tibial footprints. The line was divided into three equal parts and the position of the projection coordinates indicated the classification of the ACL ruptures (femoral side, middle and tibial side). In total, 85 patients (mean age: 27; male: 56) who underwent ACL reconstruction surgery under arthroscopy were included. Three clinical readers evaluated the datasets separately and their diagnostic performances were compared with those of the model. The performance metrics included the accuracy, error rate, sensitivity, specificity, precision, and F1-score. A one-way ANOVA was used to evaluate the performance of the convolutional neural networks (CNNs) and clinical readers. Intraclass correlation coefficients (ICC) were used to assess interobserver agreement between the clinical readers. Results: The accuracy of ACL localization was 3.77 ± 2.74 and 4.68 ± 3.92 (mm) for three-dimensional (3D) and two-dimensional (2D) CNNs, respectively. There was no significant difference in the ACL rupture location performance between the 3D and 2D CNNs or among the clinical readers (Accuracy, p < 0.01). The 3D CNNs performed best among the five evaluators in classifying the femoral side (sensitivity of 0.86 and specificity of 0.79), middle side (sensitivity of 0.71 and specificity of 0.84) and tibial side ACL rupture (sensitivity of 0.71 and specificity of 0.99), and the overall accuracy for sides classifying of ACL rupture achieved 0.79. Conclusion: The proposed deep learning-based model achieved high diagnostic performances in locating and classifying ACL fractures on knee MR images.
Collapse
Affiliation(s)
- Cheng Qu
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Heng Yang
- College of Electrical Engineering, Sichuan University, Chengdu, China
| | - Cong Wang
- School of Biomedical Engineering and Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Chongyang Wang
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Mengjie Ying
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zheyi Chen
- Department of Radiology, Shanghai Municipal Eighth People’s Hospital, Shanghai, China
| | - Kai Yang
- Department of Radiology, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Jing Zhang
- College of Electrical Engineering, Sichuan University, Chengdu, China
| | - Kang Li
- West China Hospital, Sichuan University, Chengdu, China
| | - Dimitris Dimitriou
- Department of Orthopedics, Balgrist University Hospital, University of Zürich, Zurich, Switzerland
| | - Tsung-Yuan Tsai
- School of Biomedical Engineering and Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China
- *Correspondence: Tsung-Yuan Tsai, ; Xudong Liu,
| | - Xudong Liu
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
- *Correspondence: Tsung-Yuan Tsai, ; Xudong Liu,
| |
Collapse
|
6
|
Chen K, Lai YC, Vanniarajan B, Wang PH, Wang SC, Lin YC, Ng SH, Tran P, Lin G. Clinical impact of a deep learning system for automated detection of missed pulmonary nodules on routine body computed tomography including the chest region. Eur Radiol 2022; 32:2891-2900. [PMID: 34999920 DOI: 10.1007/s00330-021-08412-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Revised: 09/29/2021] [Accepted: 10/13/2021] [Indexed: 11/27/2022]
Abstract
OBJECTIVES To evaluate the clinical impact of a deep learning system (DLS) for automated detection of pulmonary nodules on computed tomography (CT) images as a second reader. METHODS This single-centre retrospective study screened 21,150 consecutive body CT studies from September 2018 to February 2019. Pulmonary nodules detected by the DLS on axial CT images but not mentioned in initial radiology reports were flagged. Flagged images were scored by four board-certificated radiologists each with at least 5 years of experience. Nodules with scores of 2 (understandable miss) or 3 (should not be missed) were then categorised as unlikely to be clinically significant (2a or 3a) or likely to be clinically significant (2b or 3b) according to the 2017 Fleischner guidelines for pulmonary nodules. The miss rate was defined as the total number of studies receiving scores of 2 or 3 divided by total screened studies. RESULTS Among 172 nodules flagged by the DLS, 60 (35%) missed nodules were confirmed by the radiologists. The nodules were further categorised as 2a, 2b, 3a, and 3b in 24, 14, 10, and 12 studies, respectively, with an overall positive predictive value of 35%. Missed pulmonary nodules were identified in 0.3% of all CT images, and one-third of these lesions were considered clinically significant. CONCLUSIONS Use of DLS-assisted automated detection as a second reader can identify missed pulmonary nodules, some of which may be clinically significant. CLINICAL RELEVANCE/APPLICATION Use of DLS to help radiologists detect pulmonary lesions may improve patient care. KEY POINTS • DLS-assisted automated detection as a second reader is feasible in a large consecutive cohort. • Performance of combined radiologists and DLS was better than DLS or radiologists alone. • Pulmonary nodules were missed more frequently in abdomino-pelvis CT than the thoracic CT.
Collapse
Affiliation(s)
- Kueian Chen
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
- Imaging Core Laboratory, Institute for Radiological Research, Chang Gung Memorial Hospital at Linkou and Chang Gung University, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
| | - Ying-Chieh Lai
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
- Imaging Core Laboratory, Institute for Radiological Research, Chang Gung Memorial Hospital at Linkou and Chang Gung University, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
- Clinical Metabolomics Core Laboratory, Chang Gung Memorial Hospital at Linkou, Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
| | | | - Pieh-Hsu Wang
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
| | - Shao-Chung Wang
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
| | - Yu-Chun Lin
- Imaging Core Laboratory, Institute for Radiological Research, Chang Gung Memorial Hospital at Linkou and Chang Gung University, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
- Clinical Metabolomics Core Laboratory, Chang Gung Memorial Hospital at Linkou, Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
| | - Shu-Hang Ng
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan
| | - Pelu Tran
- FerrumFerrum Health, Santa Clara, CA, USA
| | - Gigin Lin
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan.
- Imaging Core Laboratory, Institute for Radiological Research, Chang Gung Memorial Hospital at Linkou and Chang Gung University, 5 Fuhsing St., Taoyuan, 33382, Guishan, Taiwan.
- Clinical Metabolomics Core Laboratory, Chang Gung Memorial Hospital at Linkou, Fuhsing St., Taoyuan, 33382, Guishan, Taiwan.
| |
Collapse
|
7
|
[Clinical Study of Artificial Intelligence-assisted Diagnosis System in Predicting the
Invasive Subtypes of Early-stage Lung Adenocarcinoma Appearing as Pulmonary Nodules]. ZHONGGUO FEI AI ZA ZHI = CHINESE JOURNAL OF LUNG CANCER 2022; 25:245-252. [PMID: 35477188 PMCID: PMC9051300 DOI: 10.3779/j.issn.1009-3419.2022.102.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
BACKGROUND Lung cancer is the cancer with the highest mortality at home and abroad at present. The detection of lung nodules is a key step to reducing the mortality of lung cancer. Artificial intelligence-assisted diagnosis system presents as the state of the art in the area of nodule detection, differentiation between benign and malignant and diagnosis of invasive subtypes, however, a validation with clinical data is necessary for further application. Therefore, the aim of this study is to evaluate the effectiveness of artificial intelligence-assisted diagnosis system in predicting the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules. METHODS Clinical data of 223 patients with early-stage lung adenocarcinoma appearing as pulmonary nodules admitted to the Lanzhou University Second Hospital from January 1st, 2016 to December 31th, 2021 were retrospectively analyzed, which were divided into invasive adenocarcinoma group (n=170) and non-invasive adenocarcinoma group (n=53), and the non-invasive adenocarcinoma group was subdivided into minimally invasive adenocarcinoma group (n=31) and preinvasive lesions group (n=22). The malignant probability and imaging characteristics of each group were compared to analyze their predictive ability for the invasive subtypes of early-stage lung adenocarcinoma. The concordance between qualitative diagnostic results of artificial intelligence-assisted diagnosis of the invasive subtypes of early-stage lung adenocarcinoma and postoperative pathology was then analyzed. RESULTS In different invasive subtypes of early-stage lung adenocarcinoma, the mean CT value of pulmonary nodules (P<0.001), diameter (P<0.001), volume (P<0.001), malignant probability (P<0.001), pleural retraction sign (P<0.001), lobulation (P<0.001), spiculation (P<0.001) were significantly different. At the same time, it was also found that with the increased invasiveness of different invasive subtypes of early-stage lung adenocarcinoma, the proportion of dominant signs of each group gradually increased. On the issue of binary classification, the sensitivity, specificity, and area under the curve (AUC) values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 81.76%, 92.45% and 0.871 respectively. On the issue of three classification, the accuracy, recall rate, F1 score, and AUC values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 83.86%, 85.03%, 76.46% and 0.879 respectively. CONCLUSIONS Artificial intelligence-assisted diagnosis system could predict the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules, and has a certain predictive value. With the optimization of algorithms and the improvement of data, it may provide guidance for individualized treatment of patients.
Collapse
|
8
|
[Chinese Experts Consensus on Artificial Intelligence Assisted Management for
Pulmonary Nodule (2022 Version)]. ZHONGGUO FEI AI ZA ZHI = CHINESE JOURNAL OF LUNG CANCER 2022; 25:219-225. [PMID: 35340198 PMCID: PMC9051301 DOI: 10.3779/j.issn.1009-3419.2022.102.08] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Low-dose computed tomography (CT) for lung cancer screening has been proven to reduce lung cancer deaths in the screening group compared with the control group. The increasing number of pulmonary nodules being detected by CT scans significantly increase the workload of the radiologists for scan interpretation. Artificial intelligence (AI) has the potential to increase the efficiency of pulmonary nodule discrimination and has been tested in preliminary studies for nodule management. As more and more artificial AI products are commercialized, the consensus statement has been organized in a collaborative effort by Thoracic Surgery Committee, Department of Simulated Medicine, Wu Jieping Medical Foundation to aid clinicians in the application of AI-assisted management for pulmonary nodules.
.
Collapse
|
9
|
Wang J, Yuan C, Han C, Wen Y, Lu H, Liu C, She Y, Deng J, Li B, Qian D, Chen C. IMAL-Net: Interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images. Med Phys 2021; 48:7913-7929. [PMID: 34674280 DOI: 10.1002/mp.15293] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Revised: 08/26/2021] [Accepted: 09/29/2021] [Indexed: 12/17/2022] Open
Abstract
PURPOSE Feature maps created from deep convolutional neural networks (DCNNs) have been widely used for visual explanation of DCNN-based classification tasks. However, many clinical applications such as benign-malignant classification of lung nodules normally require quantitative and objective interpretability, rather than just visualization. In this paper, we propose a novel interpretable multi-task attention learning network named IMAL-Net for early invasive adenocarcinoma screening in chest computed tomography images, which takes advantage of segmentation prior to assist interpretable classification. METHODS Two sub-ResNets are firstly integrated together via a prior-attention mechanism for simultaneous nodule segmentation and invasiveness classification. Then, numerous radiomic features from the segmentation results are concatenated with high-level semantic features from the classification subnetwork by FC layers to achieve superior performance. Meanwhile, an end-to-end feature selection mechanism (named FSM) is designed to quantify crucial radiomic features greatly affecting the prediction of each sample, and thus it can provide clinically applicable interpretability to the prediction result. RESULTS Nodule samples from a total of 1626 patients were collected from two grade-A hospitals for large-scale verification. Five-fold cross validation demonstrated that the proposed IMAL-Net can achieve an AUC score of 93.8% ± 1.1% and a recall score of 93.8% ± 2.8% for identification of invasive lung adenocarcinoma. CONCLUSIONS It can be concluded that fusing semantic features and radiomic features can achieve obvious improvements in the invasiveness classification task. Moreover, by learning more fine-grained semantic features and highlighting the most important radiomics features, the proposed attention and FSM mechanisms not only can further improve the performance but also can be used for both visual explanations and objective analysis of the classification results.
Collapse
Affiliation(s)
- Jun Wang
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Cheng Yuan
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Can Han
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Yaofeng Wen
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Hongbing Lu
- College of Computer Science and Technology, Zhejiang University, Hangzhou, China
| | - Chen Liu
- Department of Radiology, Southwest Hospital, Third Military University (Army Medical University), Chongqing, China
| | - Yunlang She
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Jiajun Deng
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| | - Biao Li
- Department of Nuclear Medicine, Ruijin Hospital, Shanghai, China
| | - Dahong Qian
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Chang Chen
- Department of Thoracic Surgery, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, China
| |
Collapse
|
10
|
Xu Y, Li Y, Yin H, Tang W, Fan G. Consecutive Serial Non-Contrast CT Scan-Based Deep Learning Model Facilitates the Prediction of Tumor Invasiveness of Ground-Glass Nodules. Front Oncol 2021; 11:725599. [PMID: 34568054 PMCID: PMC8461974 DOI: 10.3389/fonc.2021.725599] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Accepted: 08/19/2021] [Indexed: 01/31/2023] Open
Abstract
Introduction Tumors are continuously evolving biological systems which can be monitored by medical imaging. Previous studies only focus on single timepoint images, whether the performance could be further improved by using serial noncontrast CT imaging obtained during nodule follow-up management remains unclear. In this study, we evaluated DL model for predicting tumor invasiveness of GGNs through analyzing time series CT images. Methods A total of 168 pathologically confirmed GGN cases (48 noninvasive lesions and 120 invasive lesions) were retrospectively collected and randomly assigned to the development dataset (n = 123) and independent testing dataset (n = 45). All patients underwent consecutive noncontrast CT examinations, and the baseline CT and 3-month follow-up CT images were collected. The gross region of interest (ROI) patches containing only tumor region and the full ROI patches including both tumor and peritumor regions were cropped from CT images. A baseline model was built on the image features and demographic features. Four DL models were proposed: two single-DL model using gross ROI (model 1) or full ROI patches (model 3) from baseline CT images, and two serial-DL models using gross ROI (model 2) or full ROI patches (model 4) from consecutive CT images (baseline scan and 3-month follow-up scan). In addition, a combined model integrating serial full ROI patches and clinical information was also constructed. The performance of these predictive models was assessed with respect to discrimination and clinical usefulness. Results The area under the curve (AUC) of the baseline model, models 1, 2, 3, and 4 were 0.562 [(95% confidence interval (C)], 0.406~0.710), 0.693 (95% CI, 0.538-0.822), 0.787 (95% CI, 0.639-0.895), 0.727 (95% CI, 0.573-0.849), and 0.811 (95% CI, 0.667-0.912) in the independent testing dataset, respectively. The results indicated that the peritumor region had potential to contribute to tumor invasiveness prediction, and the model performance was further improved by integrating imaging scans at multiple timepoints. Furthermore, the combined model showed best discrimination ability, with AUC, sensitivity, specificity, and accuracy achieving 0.831 (95% CI, 0.690-0.926), 86.7%, 73.3%, and 82.2%, respectively. Conclusion The DL model integrating full ROIs from serial CT images shows improved predictive performance in differentiating noninvasive from invasive GGNs than the model using only baseline CT images, which could benefit the clinical management of GGNs.
Collapse
Affiliation(s)
- Yao Xu
- Department of Radiology, Second Affiliated Hospital of Soochow University, Suzhou, China
| | - Yu Li
- Department of Radiology, Dushuhu Public Hospital Affiliated of Soochow University, Suzhou, China
| | - Hongkun Yin
- Department of Advanced Research, Infervision Medical Technology Co. Ltd, Beijing, China
| | - Wen Tang
- Department of Advanced Research, Infervision Medical Technology Co. Ltd, Beijing, China
| | - Guohua Fan
- Department of Radiology, Second Affiliated Hospital of Soochow University, Suzhou, China
| |
Collapse
|