1
|
Li J, Zhang P, Yang X, Zhu L, Wang T, Zhang P, Liu R, Sheng B, Wang K. SSM-Net: Semi-supervised multi-task network for joint lesion segmentation and classification from pancreatic EUS images. Artif Intell Med 2024; 154:102919. [PMID: 38941908 DOI: 10.1016/j.artmed.2024.102919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 04/30/2024] [Accepted: 06/19/2024] [Indexed: 06/30/2024]
Abstract
Pancreatic cancer does not show specific symptoms, which makes the diagnosis of early stages difficult with established image-based screening methods and therefore has the worst prognosis among all cancers. Although endoscopic ultrasonography (EUS) has a key role in diagnostic algorithms for pancreatic diseases, B-mode imaging of the pancreas can be affected by confounders such as chronic pancreatitis, which can make both pancreatic lesion segmentation and classification laborious and highly specialized. To address these challenges, this work proposes a semi-supervised multi-task network (SSM-Net) to leverage unlabeled and labeled EUS images for joint pancreatic lesion classification and segmentation. Specifically, we first devise a saliency-aware representation learning module (SRLM) on a large number of unlabeled images to train a feature extraction encoder network for labeled images by computing a contrastive loss with a semantic saliency map, which is obtained by our spectral residual module (SRM). Moreover, for labeled EUS images, we devise channel attention blocks (CABs) to refine the features extracted from the pre-trained encoder on unlabeled images for segmenting lesions, and then devise a merged global attention module (MGAM) and a feature similarity loss (FSL) for obtaining a lesion classification result. We collect a large-scale EUS-based pancreas image dataset (LS-EUSPI) consisting of 9,555 pathologically proven labeled EUS images (499 patients from four categories) and 15,500 unlabeled EUS images. Experimental results on the LS-EUSPI dataset and a public thyroid gland lesion dataset show that our SSM-Net clearly outperforms state-of-the-art methods.
Collapse
Affiliation(s)
- Jiajia Li
- School of Chemistry and Chemical Engineering and National Center for Translational Medicine, Shanghai Jiao Tong University, Shanghai, China.
| | - Pingping Zhang
- Department of Gastroenterology, Changhai Hospital, Second Military Medical University/Naval Medical University, Shanghai 200433, China.
| | - Xia Yang
- Department of Gastroenterology, Shandong Provincial Hospital Affiliated to Shangdong First Medical University, Jinan, Shandong, 250021, China.
| | - Lei Zhu
- Hong Kong University of Science and Technology (Guangzhou), Nansha, Guangzhou, 511400, Guangdong, China; Hong Kong University of Science and Technology, Hong Kong Special Administrative Region, China.
| | - Teng Wang
- Department of Gastroenterology, Changhai Hospital, Second Military Medical University/Naval Medical University, Shanghai 200433, China.
| | - Ping Zhang
- Department of Computer Science and Engineering, Ohio State University, Columbus, OH 43210, USA; Department of Biomedical Informatics, Ohio State University, Columbus, OH 43210, USA.
| | - Ruhan Liu
- Furong Laboratory, Central South University, Changsha, Hunan, China; Department of Dermatology, Xiangya Hospital, Central South University, Changsha, Hunan, China.
| | - Bin Sheng
- Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China.
| | - Kaixuan Wang
- Department of Gastroenterology, Changhai Hospital, Second Military Medical University/Naval Medical University, Shanghai 200433, China.
| |
Collapse
|
2
|
Liu J, Qi L, Xu Q, Chen J, Cui S, Li F, Wang Y, Cheng S, Tan W, Zhou Z, Wang J. A Self-supervised Learning-Based Fine-Grained Classification Model for Distinguishing Malignant From Benign Subcentimeter Solid Pulmonary Nodules. Acad Radiol 2024:S1076-6332(24)00287-3. [PMID: 38777719 DOI: 10.1016/j.acra.2024.05.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Revised: 05/02/2024] [Accepted: 05/05/2024] [Indexed: 05/25/2024]
Abstract
RATIONALE AND OBJECTIVES Diagnosing subcentimeter solid pulmonary nodules (SSPNs) remains challenging in clinical practice. Deep learning may perform better than conventional methods in differentiating benign and malignant pulmonary nodules. This study aimed to develop and validate a model for differentiating malignant and benign SSPNs using CT images. MATERIALS AND METHODS This retrospective study included consecutive patients with SSPNs detected between January 2015 and October 2021 as an internal dataset. Malignancy was confirmed pathologically; benignity was confirmed pathologically or via follow-up evaluations. The SSPNs were segmented manually. A self-supervision pre-training-based fine-grained network was developed for predicting SSPN malignancy. The pre-trained model was established using data from the National Lung Screening Trial, Lung Nodule Analysis 2016, and a database of 5478 pulmonary nodules from the previous study, with subsequent fine-tuning using the internal dataset. The model's efficacy was investigated using an external cohort from another center, and its accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were determined. RESULTS Overall, 1276 patients (mean age, 56 ± 10 years; 497 males) with 1389 SSPNs (mean diameter, 7.5 ± 2.0 mm; 625 benign) were enrolled. The internal dataset was specifically enriched for malignancy. The model's performance in the internal testing set (316 SSPNs) was: AUC, 0.964 (95% confidence interval (95%CI): 0.942-0.986); accuracy, 0.934; sensitivity, 0.965; and specificity, 0.908. The model's performance in the external test set (202 SSPNs) was: AUC, 0.945 (95% CI: 0.910-0.979); accuracy, 0.911; sensitivity, 0.977; and specificity, 0.860. CONCLUSION This deep learning model was robust and exhibited good performance in predicting the malignancy of SSPNs, which could help optimize patient management.
Collapse
Affiliation(s)
- Jianing Liu
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Linlin Qi
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Qian Xu
- Department of Computed Tomography and Magnetic Resonance, The Fourth Hospital of Hebei Medical University, Shijiazhuang, He Bei, China
| | - Jiaqi Chen
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Shulei Cui
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Fenglan Li
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Yawen Wang
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Sainan Cheng
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China
| | - Weixiong Tan
- Beijing Deepwise & League of PhD Technology Co. Ltd, Beijing, China
| | - Zhen Zhou
- Beijing Deepwise & League of PhD Technology Co. Ltd, Beijing, China
| | - Jianwei Wang
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Panjiayuan Nanli, Chaoyang District, Beijing 100021, China.
| |
Collapse
|
3
|
Fang X, Shi F, Liu F, Wei Y, Li J, Wu J, Wang T, Lu J, Shao C, Bian Y. Tracheal computed tomography radiomics model for prediction of the Omicron variant of severe acute respiratory syndrome coronavirus 2. RADIOLOGIE (HEIDELBERG, GERMANY) 2024:10.1007/s00117-024-01275-3. [PMID: 38446170 DOI: 10.1007/s00117-024-01275-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 01/11/2024] [Indexed: 03/07/2024]
Abstract
OBJECTIVES The Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is highly contagious, fast-spreading, and insidious. Most patients present with normal findings on lung computed tomography (CT). The current study aimed to develop and validate a tracheal CT radiomics model to predict Omicron variant infection. MATERIALS AND METHODS In this retrospective study, a radiomics model was developed based on a training set consisting of 157 patients with an Omicron variant infection and 239 healthy controls between 1 January and 30 April 2022. A set of morphological expansions, with dilations of 1, 3, 5, 7, and 9 voxels, was applied to the trachea, and radiomic features were extracted from different dilation voxels of the trachea. Logistic regression (LR), support vector machines (SVM), and random forests (RF) were developed and evaluated; the models were validated on 67 patients with the Omicron variant and on 103 healthy controls between 1 May and 30 July 2022. RESULTS Logistic regression with 12 radiomic features extracted from the tracheal wall with dilation of 5 voxels achieved the highest classification performance compared with the other models. The LR model achieved an area under the curve of 0.993 (95% confidence interval [CI]: 0.987-0.998) in the training set and 0.989 (95% CI: 0.979-0.999) in the validation set. Sensitivity, specificity, and accuracy of the model for the training set were 0.994, 0.946, and 0.965, respectively, whereas those for the validation set were 0.970, 0.952, and 0.959, respectively. CONCLUSION The tracheal CT radiomics model reliably identified the Omicron variant of SARS-CoV‑2, and may help in clinical decision-making in future, especially in cases of normal lung CT findings.
Collapse
Affiliation(s)
- Xu Fang
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China
| | - Feng Shi
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Fang Liu
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China
| | - Ying Wei
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Jing Li
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China
| | - Jiaojiao Wu
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Tiegong Wang
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China
| | - Jianping Lu
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China
| | - Chengwei Shao
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China.
| | - Yun Bian
- Department of Radiology, Changhai Hospital, Naval Medical University, 168 Changhai Road, 200433, Shanghai, China.
| |
Collapse
|
4
|
Wu R, Liang C, Zhang J, Tan Q, Huang H. Multi-kernel driven 3D convolutional neural network for automated detection of lung nodules in chest CT scans. BIOMEDICAL OPTICS EXPRESS 2024; 15:1195-1218. [PMID: 38404310 PMCID: PMC10890889 DOI: 10.1364/boe.504875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 12/27/2023] [Accepted: 12/28/2023] [Indexed: 02/27/2024]
Abstract
The accurate position detection of lung nodules is crucial in early chest computed tomography (CT)-based lung cancer screening, which helps to improve the survival rate of patients. Deep learning methodologies have shown impressive feature extraction ability in the CT image analysis task, but it is still a challenge to develop a robust nodule detection model due to the salient morphological heterogeneity of nodules and complex surrounding environment. In this study, a multi-kernel driven 3D convolutional neural network (MK-3DCNN) is proposed for computerized nodule detection in CT scans. In the MK-3DCNN, a residual learning-based encoder-decoder architecture is introduced to employ the multi-layer features of the deep model. Considering the various nodule sizes and shapes, a multi-kernel joint learning block is developed to capture 3D multi-scale spatial information of nodule CT images, and this is conducive to improving nodule detection performance. Furthermore, a multi-mode mixed pooling strategy is designed to replace the conventional single-mode pooling manner, and it reasonably integrates the max pooling, average pooling, and center cropping pooling operations to obtain more comprehensive nodule descriptions from complicated CT images. Experimental results on the public dataset LUNA16 illustrate that the proposed MK-3DCNN method achieves more competitive nodule detection performance compared to some state-of-the-art algorithms. The results on our constructed clinical dataset CQUCH-LND indicate that the MK-3DCNN has a good prospect in clinical practice.
Collapse
Affiliation(s)
- Ruoyu Wu
- Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
| | - Changyu Liang
- Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 400030, China
| | - Jiuquan Zhang
- Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 400030, China
| | - QiJuan Tan
- Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 400030, China
| | - Hong Huang
- Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
| |
Collapse
|
5
|
Liu W, Shen N, Zhang L, Wang X, Chen B, Liu Z, Yang C. Research in the application of artificial intelligence to lung cancer diagnosis. Front Med (Lausanne) 2024; 11:1343485. [PMID: 38352145 PMCID: PMC10861801 DOI: 10.3389/fmed.2024.1343485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 01/02/2024] [Indexed: 02/16/2024] Open
Abstract
The morbidity and mortality rates in lung cancer are high worldwide. Early diagnosis and personalized treatment are important to manage this public health issue. In recent years, artificial intelligence (AI) has played increasingly important roles in early screening, auxiliary diagnosis, and prognostic assessment. AI uses algorithms to extract quantitative feature information from high-volume and high-latitude data and learn existing data to predict disease outcomes. In this review, we describe the current uses of AI in lung cancer-focused pathomics, imageomics, and genomics applications.
Collapse
Affiliation(s)
- Wenjuan Liu
- Department of Respiratory and Critical Care Medicine, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Nan Shen
- Department of Nephrology, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Limin Zhang
- Department of Respiratory and Critical Care Medicine, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Xiaoxi Wang
- Department of Clinical Laboratory, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Bainan Chen
- Department of Respiratory and Critical Care Medicine, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Zhuo Liu
- Department of Respiratory and Critical Care Medicine, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Chao Yang
- Department of Radiology, The First Affiliated Hospital of Dalian Medical University, Dalian, China
| |
Collapse
|
6
|
Zhang Y, Qu H, Tian Y, Na F, Yan J, Wu Y, Cui X, Li Z, Zhao M. PB-LNet: a model for predicting pathological subtypes of pulmonary nodules on CT images. BMC Cancer 2023; 23:936. [PMID: 37789252 PMCID: PMC10548640 DOI: 10.1186/s12885-023-11364-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 09/04/2023] [Indexed: 10/05/2023] Open
Abstract
OBJECTIVE To investigate the correlation between CT imaging features and pathological subtypes of pulmonary nodules and construct a prediction model using deep learning. METHODS We collected information of patients with pulmonary nodules treated by surgery and the reference standard for diagnosis was post-operative pathology. After using elastic distortion for data augmentation, the CT images were divided into a training set, a validation set and a test set in a ratio of 6:2:2. We used PB-LNet to analyze the nodules in pre-operative CT and predict their pathological subtypes. Accuracy was used as the model evaluation index and Class Activation Map was applied to interpreting the results. Comparative experiments with other models were carried out to achieve the best results. Finally, images from the test set without data augmentation were analyzed to judge the clinical utility. RESULTS Four hundred seventy-seven patients were included and the nodules were divided into six groups: benign lesions, precursor glandular lesions, minimally invasive adenocarcinoma, invasive adenocarcinoma Grade 1, Grade 2 and Grade 3. The accuracy of the test set was 0.84. Class Activation Map confirmed that PB-LNet classified the nodules mainly based on the lungs in CT images, which is in line with the actual situation in clinical practice. In comparative experiments, PB-LNet obtained the highest accuracy. Finally, 96 images from the test set without data augmentation were analyzed and the accuracy was 0.89. CONCLUSIONS In classifying CT images of lung nodules into six categories based on pathological subtypes, PB-LNet demonstrates satisfactory accuracy without the need of delineating nodules, while the results are interpretable. A high level of accuracy was also obtained when validating on real data, therefore demonstrates its usefulness in clinical practice.
Collapse
Affiliation(s)
- Yuchong Zhang
- Department of Medical Oncology, the First Hospital of China Medical University, NO.155, North Nanjing Street, Heping District, Shenyang, Liaoning Province, 110001, China
| | - Hui Qu
- College of Medicine and Biological Information Engineering, Northeastern University, NO. 3-11, Wenhua Road, Heping District, Shenyang, 110819, Liaoning Province, China
| | - Yumeng Tian
- Department of Medical Oncology, the First Hospital of China Medical University, NO.155, North Nanjing Street, Heping District, Shenyang, Liaoning Province, 110001, China
| | - Fangjian Na
- Network Information Center, China Medical University, NO.77 Puhe Road, Shenbei New District, Shenyang, Liaoning Province, 110122, China
| | - Jinshan Yan
- Department of Medical Oncology, the First Hospital of China Medical University, NO.155, North Nanjing Street, Heping District, Shenyang, Liaoning Province, 110001, China
| | - Ying Wu
- Phase I Clinical Trails Center, the First Hospital of China Medical University, 210 1st Baita Street, Hunnan Distriction, Shenyang, Liaoning Province, 110101, China
| | - Xiaoyu Cui
- College of Medicine and Biological Information Engineering, Northeastern University, NO. 3-11, Wenhua Road, Heping District, Shenyang, 110819, Liaoning Province, China.
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Shenyang, China.
| | - Zhi Li
- Department of Medical Oncology, the First Hospital of China Medical University, NO.155, North Nanjing Street, Heping District, Shenyang, Liaoning Province, 110001, China.
| | - Mingfang Zhao
- Department of Medical Oncology, the First Hospital of China Medical University, NO.155, North Nanjing Street, Heping District, Shenyang, Liaoning Province, 110001, China.
| |
Collapse
|
7
|
Chen X, Wang X, Lv J, Qin G, Zhou Z. An integrated network based on 2D/3D feature correlations for benign-malignant tumor classification and uncertainty estimation in digital breast tomosynthesis. Phys Med Biol 2023; 68:175046. [PMID: 37582379 DOI: 10.1088/1361-6560/acf092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 08/15/2023] [Indexed: 08/17/2023]
Abstract
Objective.Classification of benign and malignant tumors is important for the early diagnosis of breast cancer. Over the last decade, digital breast tomosynthesis (DBT) has gradually become an effective imaging modality for breast cancer diagnosis due to its ability to generate three-dimensional (3D) visualizations. However, computer-aided diagnosis (CAD) systems based on 3D images require high computational costs and time. Furthermore, there is considerable redundant information in 3D images. Most CAD systems are designed based on 2D images, which may lose the spatial depth information of tumors. In this study, we propose a 2D/3D integrated network for the diagnosis of benign and malignant breast tumors.Approach.We introduce a correlation strategy to describe feature correlations between slices in 3D volumes, corresponding to the tissue relationship and spatial depth features of tumors. The correlation strategy can be used to extract spatial features with little computational cost. In the prediction stage, 3D spatial correlation features and 2D features are both used for classification.Main results.Experimental results demonstrate that our proposed framework achieves higher accuracy and reliability than pure 2D or 3D models. Our framework has a high area under the curve of 0.88 and accuracy of 0.82. The parameter size of the feature extractor in our framework is only 35% of that of the 3D models. In reliability evaluations, our proposed model is more reliable than pure 2D or 3D models because of its effective and nonredundant features.Significance.This study successfully combines 3D spatial correlation features and 2D features for the diagnosis of benign and malignant breast tumors in DBT. In addition to high accuracy and low computational cost, our model is more reliable and can output uncertainty value. From this point of view, the proposed method has the potential to be applied in clinic.
Collapse
Affiliation(s)
- Xi Chen
- School of Information and Communications Engineering, Xi'an Jiaotong University, Xi'an, 710049, Shaanxi, People's Republic of China
| | - Xiaoyu Wang
- School of Information and Communications Engineering, Xi'an Jiaotong University, Xi'an, 710049, Shaanxi, People's Republic of China
| | - Jiahuan Lv
- School of Information and Communications Engineering, Xi'an Jiaotong University, Xi'an, 710049, Shaanxi, People's Republic of China
| | - Genggeng Qin
- Department of Radiology, Nanfang Hospital, Southern Medical University, Guangzhou, 510515, Guangdong, People's Republic of China
| | - Zhiguo Zhou
- Department of Biostatistics and Data Science, University of Kansas Medical Center, Kansas City, KS-66160, United States of America
| |
Collapse
|
8
|
Shao J, Feng J, Li J, Liang S, Li W, Wang C. Novel tools for early diagnosis and precision treatment based on artificial intelligence. CHINESE MEDICAL JOURNAL PULMONARY AND CRITICAL CARE MEDICINE 2023; 1:148-160. [PMID: 39171128 PMCID: PMC11332840 DOI: 10.1016/j.pccm.2023.05.001] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Indexed: 08/23/2024]
Abstract
Lung cancer has the highest mortality rate among all cancers in the world. Hence, early diagnosis and personalized treatment plans are crucial to improving its 5-year survival rate. Chest computed tomography (CT) serves as an essential tool for lung cancer screening, and pathology images are the gold standard for lung cancer diagnosis. However, medical image evaluation relies on manual labor and suffers from missed diagnosis or misdiagnosis, and physician heterogeneity. The rapid development of artificial intelligence (AI) has brought a whole novel opportunity for medical task processing, demonstrating the potential for clinical application in lung cancer diagnosis and treatment. AI technologies, including machine learning and deep learning, have been deployed extensively for lung nodule detection, benign and malignant classification, and subtype identification based on CT images. Furthermore, AI plays a role in the non-invasive prediction of genetic mutations and molecular status to provide the optimal treatment regimen, and applies to the assessment of therapeutic efficacy and prognosis of lung cancer patients, enabling precision medicine to become a reality. Meanwhile, histology-based AI models assist pathologists in typing, molecular characterization, and prognosis prediction to enhance the efficiency of diagnosis and treatment. However, the leap to extensive clinical application still faces various challenges, such as data sharing, standardized label acquisition, clinical application regulation, and multimodal integration. Nevertheless, AI holds promising potential in the field of lung cancer to improve cancer care.
Collapse
Affiliation(s)
- Jun Shao
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Jiaming Feng
- West China School of Medicine, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Jingwei Li
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Shufan Liang
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Weimin Li
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Chengdi Wang
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| |
Collapse
|
9
|
Sakshiwala, Singh MP. An ensemble of three-dimensional deep neural network models for multi-attribute scoring and classification of pulmonary nodules. Proc Inst Mech Eng H 2023; 237:946-957. [PMID: 37366554 DOI: 10.1177/09544119231182037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/28/2023]
Abstract
Lung cancer is the uncontrolled growth of cells that originates in the lung parenchyma or cells that line the air passages. These cells divide rapidly to form malicious tumors. This paper proposes a multi-task ensemble of three dimensional (3D) deep neural network (DNN) based model, namely: pre-trained EfficientNetB0, BiGRU-based SEResNext101, and the proposed LungNet. The ensemble model performs binary classification and regression tasks to accurately classify the benign and malignant pulmonary nodules. This study also explores the attribute importance and proposes a domain knowledge-based regularization technique. The proposed model is evaluated on the public benchmark LIDC-IDRI dataset. Through a comparative study, it was shown that when coefficients generated by the random forest (RF) are used in the loss function, the proposed ensemble model offers a better prediction capability of the accuracy of 96.4% compared to the state-of-the-art methods. In addition, the receiver operating characteristic curves show that the proposed ensemble model has better performance than the base learners. Thus, the proposed CAD-based model can efficiently detect malignant pulmonary nodules.
Collapse
Affiliation(s)
- Sakshiwala
- Department of Computer Science and Engineering, NIT Patna, Patna, Bihar, India
| | | |
Collapse
|
10
|
Mei M, Ye Z, Zha Y. An integrated convolutional neural network for classifying small pulmonary solid nodules. Front Neurosci 2023; 17:1152222. [PMID: 37332867 PMCID: PMC10272407 DOI: 10.3389/fnins.2023.1152222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 04/17/2023] [Indexed: 06/20/2023] Open
Abstract
Achieving accurate classification of benign and malignant pulmonary nodules is essential for treating some diseases. However, traditional typing methods have difficulty obtaining satisfactory results on small pulmonary solid nodules, mainly caused by two aspects: (1) noise interference from other tissue information; (2) missing features of small nodules caused by downsampling in traditional convolutional neural networks. To solve these problems, this paper proposes a new typing method to improve the diagnosis rate of small pulmonary solid nodules in CT images. Specifically, first, we introduce the Otsu thresholding algorithm to preprocess the data and filter the interference information. Then, to acquire more small nodule features, we add parallel radiomics to the 3D convolutional neural network. Radiomics can extract a large number of quantitative features from medical images. Finally, the classifier generated more accurate results by the visual and radiomic features. In the experiments, we tested the proposed method on multiple data sets, and the proposed method outperformed other methods in the small pulmonary solid nodule classification task. In addition, various groups of ablation experiments demonstrated that the Otsu thresholding algorithm and radiomics are helpful for the judgment of small nodules and proved that the Otsu thresholding algorithm is more flexible than the manual thresholding algorithm.
Collapse
Affiliation(s)
- Mengqing Mei
- School of Computer Science, Hubei University of Technology, Wuhan, China
| | - Zhiwei Ye
- School of Computer Science, Hubei University of Technology, Wuhan, China
| | - Yunfei Zha
- Department of Radiology, Renmin Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
11
|
Multiround Transfer Learning and Modified Generative Adversarial Network for Lung Cancer Detection. INT J INTELL SYST 2023. [DOI: 10.1155/2023/6376275] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2023]
Abstract
Lung cancer has been the leading cause of cancer death for many decades. With the advent of artificial intelligence, various machine learning models have been proposed for lung cancer detection (LCD). Typically, challenges in building an accurate LCD model are the small-scale datasets, the poor generalizability to detect unseen data, and the selection of useful source domains and prioritization of multiple source domains for transfer learning. In this paper, a multiround transfer learning and modified generative adversarial network (MTL-MGAN) algorithm is proposed for LCD. The MTL transfers the knowledge between the prioritized source domains and target domain to get rid of exhaust search of datasets prioritization among multiple datasets, maximizing the transferability with a multiround transfer learning process, and avoiding negative transfer via customization of loss functions in the aspects of domain, instance, and feature. In regard to the MGAN, it not only generates additional training data but also creates intermediate domains to bridge the gap between the source domains and target domains. 10 benchmark datasets are chosen for the performance evaluation and analysis of the MTL-MGAN. The proposed algorithm has significantly improved the accuracy compared with related works. To examine the contributions of the individual components of the MTL-MGAN, ablation studies are conducted to confirm the effectiveness of the prioritization algorithm, the MTL, the negative transfer avoidance via loss functions, and the MGAN. The research implications are to confirm the feasibility of multiround transfer learning to enhance the optimal solution of the target model and to provide a generic approach to bridge the gap between the source domain and target domain using MGAN.
Collapse
|
12
|
Keerthana R, Gladston A, Nehemiah HK. Transfer learning-based CNN diagnostic framework for diagnosis of COVID-19 from lung CT images. THE IMAGING SCIENCE JOURNAL 2023. [DOI: 10.1080/13682199.2023.2170768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/06/2023]
Affiliation(s)
- R. Keerthana
- Department of Computer Science and Engineering, Anna University Chennai, Chennai, India
| | - Angelin Gladston
- Department of Computer Science and Engineering, Anna University Chennai, Chennai, India
| | | |
Collapse
|
13
|
Zhang R, Wei Y, Shi F, Ren J, Zhou Q, Li W, Chen B. The diagnostic and prognostic value of radiomics and deep learning technologies for patients with solid pulmonary nodules in chest CT images. BMC Cancer 2022; 22:1118. [PMID: 36319968 PMCID: PMC9628173 DOI: 10.1186/s12885-022-10224-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2022] [Accepted: 10/17/2022] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND Solid pulmonary nodules are different from subsolid nodules and the diagnosis is much more challenging. We intended to evaluate the diagnostic and prognostic value of radiomics and deep learning technologies for solid pulmonary nodules. METHODS Retrospectively enroll patients with pathologically-confirmed solid pulmonary nodules and collect clinical data. Obtain pre-treatment high-resolution thoracic CT and manually delineate the nodule in 3D. Then, all patients were randomly divided into training and testing sets at a ratio of 7:3, and convolutional neural networks (CNN) models and random forest (RF) models were established. Survival analyses were performed for patients with solid adenocarcinomas. RESULTS Totally 720 solid pulmonary nodules were enrolled, 348 benign and 372 malignant. The CNN model with clinical features achieved the highest AUC [0.819, 95% confidence interval (CI): 0.760-0.877] with a sensitivity of 0.778, specificity of 0.788 and accuracy of 0.783. No significant differences were observed between the CNN and radiomics models. There were 295 solid adenocarcinomas in survival analysis. Different disease-free survival was observed between the low-risk and high-risk groups divided according to the radiomics Rad-score. However, the groups based on deep learning signatures showed similar survival. Cox regression analysis indicated that the radiomics Rad-score (hazard ratio: 5.08, 95% CI: 2.61-9.90) was an independent predictor of recurrence. CONCLUSIONS The radiomics and deep learning models can well predict the malignancy of solid pulmonary nodules. Radiomics signatures also demonstrate prognostic value in solid adenocarcinomas.
Collapse
Affiliation(s)
- Rui Zhang
- grid.13291.380000 0001 0807 1581Department of Pulmonary and Critical Care Medicine, West China Hospital, Sichuan University, 37 GuoXue Alley, Wuhou District, Chengdu, Sichuan Province 610041 People’s Republic of China
| | - Ying Wei
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Feng Shi
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Jing Ren
- grid.13291.380000 0001 0807 1581Department of Pulmonary and Critical Care Medicine, West China Hospital, Sichuan University, 37 GuoXue Alley, Wuhou District, Chengdu, Sichuan Province 610041 People’s Republic of China
| | - Qing Zhou
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Weimin Li
- grid.13291.380000 0001 0807 1581Department of Pulmonary and Critical Care Medicine, West China Hospital, Sichuan University, 37 GuoXue Alley, Wuhou District, Chengdu, Sichuan Province 610041 People’s Republic of China
| | - Bojiang Chen
- grid.13291.380000 0001 0807 1581Department of Pulmonary and Critical Care Medicine, West China Hospital, Sichuan University, 37 GuoXue Alley, Wuhou District, Chengdu, Sichuan Province 610041 People’s Republic of China
| |
Collapse
|
14
|
Lv Y, Wei Y, Xu K, Zhang X, Hua R, Huang J, Li M, Tang C, Yang L, Liu B, Yuan Y, Li S, Gao Y, Zhang X, Wu Y, Han Y, Shang Z, Yu H, Zhan Y, Shi F, Ye B. 3D deep learning versus the current methods for predicting tumor invasiveness of lung adenocarcinoma based on high-resolution computed tomography images. Front Oncol 2022; 12:995870. [PMID: 36338695 PMCID: PMC9634256 DOI: 10.3389/fonc.2022.995870] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2022] [Accepted: 09/30/2022] [Indexed: 11/22/2022] Open
Abstract
Background Different pathological subtypes of lung adenocarcinoma lead to different treatment decisions and prognoses, and it is clinically important to distinguish invasive lung adenocarcinoma from preinvasive adenocarcinoma (adenocarcinoma in situ and minimally invasive adenocarcinoma). This study aims to investigate the performance of the deep learning approach based on high-resolution computed tomography (HRCT) images in the classification of tumor invasiveness and compare it with the performances of currently available approaches. Methods In this study, we used a deep learning approach based on 3D conventional networks to automatically predict the invasiveness of pulmonary nodules. A total of 901 early-stage non-small cell lung cancer patients who underwent surgical treatment at Shanghai Chest Hospital between November 2015 and March 2017 were retrospectively included and randomly assigned to a training set (n=814) or testing set 1 (n=87). We subsequently included 116 patients who underwent surgical treatment and intraoperative frozen section between April 2019 and January 2020 to form testing set 2. We compared the performance of our deep learning approach in predicting tumor invasiveness with that of intraoperative frozen section analysis and human experts (radiologists and surgeons). Results The deep learning approach yielded an area under the receiver operating characteristic curve (AUC) of 0.946 for distinguishing preinvasive adenocarcinoma from invasive lung adenocarcinoma in the testing set 1, which is significantly higher than the AUCs of human experts (P<0.05). In testing set 2, the deep learning approach distinguished invasive adenocarcinoma from preinvasive adenocarcinoma with an AUC of 0.862, which is higher than that of frozen section analysis (0.755, P=0.043), senior thoracic surgeons (0.720, P=0.006), radiologists (0.766, P>0.05) and junior thoracic surgeons (0.768, P>0.05). Conclusions We developed a deep learning model that achieved comparable performance to intraoperative frozen section analysis in determining tumor invasiveness. The proposed method may contribute to clinical decisions related to the extent of surgical resection.
Collapse
Affiliation(s)
- Yilv Lv
- Department of Thoracic Surgery, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Ying Wei
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Kuan Xu
- Department of Thoracic Surgery, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Xiaobin Zhang
- Department of Thoracic Surgery, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Rong Hua
- Department of Thoracic Surgery, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Jia Huang
- Department of Oncologic Surgery, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Min Li
- Department of Radiology, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Cui Tang
- Department of Radiology, Yangpu Hospital, Tongji University, Shanghai, China
| | - Long Yang
- Department of Thoracic Surgery, Affiliated Hospital of Gansu Medical College, Pingliang, China
| | - Bingchun Liu
- Department of Thoracic Surgery, Weifang People’s Hospital, Weifang, China
| | - Yonggang Yuan
- Department of Thoracic Surgery, Qilu Hospital of Shandong University, Qingdao, China
| | - Siwen Li
- Department of Thoracic Surgery, Qingyuan People’s Hospital, Guangzhou Medical University, Guangzhou, China
| | - Yaozong Gao
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Xianjie Zhang
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Yifan Wu
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Yuchen Han
- Department of Pathology, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Zhanxian Shang
- Department of Pathology, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Hong Yu
- Department of Radiology, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
| | - Yiqiang Zhan
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Feng Shi
- Department of Research and Development, Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
- *Correspondence: Bo Ye, ; Feng Shi,
| | - Bo Ye
- Department of Thoracic Surgery, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
- *Correspondence: Bo Ye, ; Feng Shi,
| |
Collapse
|
15
|
Liu Y, He Q, Duan H, Shi H, Han A, He Y. Using Sparse Patch Annotation for Tumor Segmentation in Histopathological Images. SENSORS (BASEL, SWITZERLAND) 2022; 22:6053. [PMID: 36015814 PMCID: PMC9414209 DOI: 10.3390/s22166053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 08/05/2022] [Accepted: 08/10/2022] [Indexed: 06/15/2023]
Abstract
Tumor segmentation is a fundamental task in histopathological image analysis. Creating accurate pixel-wise annotations for such segmentation tasks in a fully-supervised training framework requires significant effort. To reduce the burden of manual annotation, we propose a novel weakly supervised segmentation framework based on sparse patch annotation, i.e., only small portions of patches in an image are labeled as 'tumor' or 'normal'. The framework consists of a patch-wise segmentation model called PSeger, and an innovative semi-supervised algorithm. PSeger has two branches for patch classification and image classification, respectively. This two-branch structure enables the model to learn more general features and thus reduce the risk of overfitting when learning sparsely annotated data. We incorporate the idea of consistency learning and self-training into the semi-supervised training strategy to take advantage of the unlabeled images. Trained on the BCSS dataset with only 25% of the images labeled (five patches for each labeled image), our proposed method achieved competitive performance compared to the fully supervised pixel-wise segmentation models. Experiments demonstrate that the proposed solution has the potential to reduce the burden of labeling histopathological images.
Collapse
Affiliation(s)
- Yiqing Liu
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen 518055, China
| | - Qiming He
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen 518055, China
| | - Hufei Duan
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen 518055, China
| | - Huijuan Shi
- Department of Pathology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou 510080, China
| | - Anjia Han
- Department of Pathology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou 510080, China
| | - Yonghong He
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen 518055, China
| |
Collapse
|
16
|
Eckardt JN, Bornhäuser M, Wendt K, Middeke JM. Semi-supervised learning in cancer diagnostics. Front Oncol 2022; 12:960984. [PMID: 35912249 PMCID: PMC9329803 DOI: 10.3389/fonc.2022.960984] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Accepted: 06/24/2022] [Indexed: 12/01/2022] Open
Abstract
In cancer diagnostics, a considerable amount of data is acquired during routine work-up. Recently, machine learning has been used to build classifiers that are tasked with cancer detection and aid in clinical decision-making. Most of these classifiers are based on supervised learning (SL) that needs time- and cost-intensive manual labeling of samples by medical experts for model training. Semi-supervised learning (SSL), however, works with only a fraction of labeled data by including unlabeled samples for information abstraction and thus can utilize the vast discrepancy between available labeled data and overall available data in cancer diagnostics. In this review, we provide a comprehensive overview of essential functionalities and assumptions of SSL and survey key studies with regard to cancer care differentiating between image-based and non-image-based applications. We highlight current state-of-the-art models in histopathology, radiology and radiotherapy, as well as genomics. Further, we discuss potential pitfalls in SSL study design such as discrepancies in data distributions and comparison to baseline SL models, and point out future directions for SSL in oncology. We believe well-designed SSL models to strongly contribute to computer-guided diagnostics in malignant disease by overcoming current hinderances in the form of sparse labeled and abundant unlabeled data.
Collapse
Affiliation(s)
- Jan-Niklas Eckardt
- Department of Internal Medicine I, University Hospital Carl Gustav Carus, Dresden, Germany
- Else Kröner Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- *Correspondence: Jan-Niklas Eckardt,
| | - Martin Bornhäuser
- Department of Internal Medicine I, University Hospital Carl Gustav Carus, Dresden, Germany
- German Consortium for Translational Cancer Research, Heidelberg, Germany
- National Center for Tumor Disease (NCT), Dresden, Germany
| | - Karsten Wendt
- Else Kröner Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- Institute of Software and Multimedia Technology, Technical University Dresden, Dresden, Germany
| | - Jan Moritz Middeke
- Department of Internal Medicine I, University Hospital Carl Gustav Carus, Dresden, Germany
- Else Kröner Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| |
Collapse
|