Semi-Supervised Deep Transfer Learning for Benign-Malignant Diagnosis of Pulmonary Nodules in Chest CT Images

SSM-Net: Semi-supervised multi-task network for joint lesion segmentation and classification from pancreatic EUS images

Pancreatic cancer does not show specific symptoms, which makes the diagnosis of early stages difficult with established image-based screening methods and therefore has the worst prognosis among all cancers. Although endoscopic ultrasonography (EUS) has a key role in diagnostic algorithms for pancreatic diseases, B-mode imaging of the pancreas can be affected by confounders such as chronic pancreatitis, which can make both pancreatic lesion segmentation and classification laborious and highly specialized. To address these challenges, this work proposes a semi-supervised multi-task network (SSM-Net) to leverage unlabeled and labeled EUS images for joint pancreatic lesion classification and segmentation. Specifically, we first devise a saliency-aware representation learning module (SRLM) on a large number of unlabeled images to train a feature extraction encoder network for labeled images by computing a contrastive loss with a semantic saliency map, which is obtained by our spectral residual module (SRM). Moreover, for labeled EUS images, we devise channel attention blocks (CABs) to refine the features extracted from the pre-trained encoder on unlabeled images for segmenting lesions, and then devise a merged global attention module (MGAM) and a feature similarity loss (FSL) for obtaining a lesion classification result. We collect a large-scale EUS-based pancreas image dataset (LS-EUSPI) consisting of 9,555 pathologically proven labeled EUS images (499 patients from four categories) and 15,500 unlabeled EUS images. Experimental results on the LS-EUSPI dataset and a public thyroid gland lesion dataset show that our SSM-Net clearly outperforms state-of-the-art methods.

A Self-supervised Learning-Based Fine-Grained Classification Model for Distinguishing Malignant From Benign Subcentimeter Solid Pulmonary Nodules

RATIONALE AND OBJECTIVES

Diagnosing subcentimeter solid pulmonary nodules (SSPNs) remains challenging in clinical practice. Deep learning may perform better than conventional methods in differentiating benign and malignant pulmonary nodules. This study aimed to develop and validate a model for differentiating malignant and benign SSPNs using CT images.

MATERIALS AND METHODS

This retrospective study included consecutive patients with SSPNs detected between January 2015 and October 2021 as an internal dataset. Malignancy was confirmed pathologically; benignity was confirmed pathologically or via follow-up evaluations. The SSPNs were segmented manually. A self-supervision pre-training-based fine-grained network was developed for predicting SSPN malignancy. The pre-trained model was established using data from the National Lung Screening Trial, Lung Nodule Analysis 2016, and a database of 5478 pulmonary nodules from the previous study, with subsequent fine-tuning using the internal dataset. The model's efficacy was investigated using an external cohort from another center, and its accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were determined.

RESULTS

Overall, 1276 patients (mean age, 56 ± 10 years; 497 males) with 1389 SSPNs (mean diameter, 7.5 ± 2.0 mm; 625 benign) were enrolled. The internal dataset was specifically enriched for malignancy. The model's performance in the internal testing set (316 SSPNs) was: AUC, 0.964 (95% confidence interval (95%CI): 0.942-0.986); accuracy, 0.934; sensitivity, 0.965; and specificity, 0.908. The model's performance in the external test set (202 SSPNs) was: AUC, 0.945 (95% CI: 0.910-0.979); accuracy, 0.911; sensitivity, 0.977; and specificity, 0.860.

CONCLUSION

This deep learning model was robust and exhibited good performance in predicting the malignancy of SSPNs, which could help optimize patient management.

Tracheal computed tomography radiomics model for prediction of the Omicron variant of severe acute respiratory syndrome coronavirus 2

OBJECTIVES

The Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is highly contagious, fast-spreading, and insidious. Most patients present with normal findings on lung computed tomography (CT). The current study aimed to develop and validate a tracheal CT radiomics model to predict Omicron variant infection.

MATERIALS AND METHODS

In this retrospective study, a radiomics model was developed based on a training set consisting of 157 patients with an Omicron variant infection and 239 healthy controls between 1 January and 30 April 2022. A set of morphological expansions, with dilations of 1, 3, 5, 7, and 9 voxels, was applied to the trachea, and radiomic features were extracted from different dilation voxels of the trachea. Logistic regression (LR), support vector machines (SVM), and random forests (RF) were developed and evaluated; the models were validated on 67 patients with the Omicron variant and on 103 healthy controls between 1 May and 30 July 2022.

RESULTS

Logistic regression with 12 radiomic features extracted from the tracheal wall with dilation of 5 voxels achieved the highest classification performance compared with the other models. The LR model achieved an area under the curve of 0.993 (95% confidence interval [CI]: 0.987-0.998) in the training set and 0.989 (95% CI: 0.979-0.999) in the validation set. Sensitivity, specificity, and accuracy of the model for the training set were 0.994, 0.946, and 0.965, respectively, whereas those for the validation set were 0.970, 0.952, and 0.959, respectively.

CONCLUSION

The tracheal CT radiomics model reliably identified the Omicron variant of SARS-CoV‑2, and may help in clinical decision-making in future, especially in cases of normal lung CT findings.

Multi-kernel driven 3D convolutional neural network for automated detection of lung nodules in chest CT scans

The accurate position detection of lung nodules is crucial in early chest computed tomography (CT)-based lung cancer screening, which helps to improve the survival rate of patients. Deep learning methodologies have shown impressive feature extraction ability in the CT image analysis task, but it is still a challenge to develop a robust nodule detection model due to the salient morphological heterogeneity of nodules and complex surrounding environment. In this study, a multi-kernel driven 3D convolutional neural network (MK-3DCNN) is proposed for computerized nodule detection in CT scans. In the MK-3DCNN, a residual learning-based encoder-decoder architecture is introduced to employ the multi-layer features of the deep model. Considering the various nodule sizes and shapes, a multi-kernel joint learning block is developed to capture 3D multi-scale spatial information of nodule CT images, and this is conducive to improving nodule detection performance. Furthermore, a multi-mode mixed pooling strategy is designed to replace the conventional single-mode pooling manner, and it reasonably integrates the max pooling, average pooling, and center cropping pooling operations to obtain more comprehensive nodule descriptions from complicated CT images. Experimental results on the public dataset LUNA16 illustrate that the proposed MK-3DCNN method achieves more competitive nodule detection performance compared to some state-of-the-art algorithms. The results on our constructed clinical dataset CQUCH-LND indicate that the MK-3DCNN has a good prospect in clinical practice.

Research in the application of artificial intelligence to lung cancer diagnosis

The morbidity and mortality rates in lung cancer are high worldwide. Early diagnosis and personalized treatment are important to manage this public health issue. In recent years, artificial intelligence (AI) has played increasingly important roles in early screening, auxiliary diagnosis, and prognostic assessment. AI uses algorithms to extract quantitative feature information from high-volume and high-latitude data and learn existing data to predict disease outcomes. In this review, we describe the current uses of AI in lung cancer-focused pathomics, imageomics, and genomics applications.

PB-LNet: a model for predicting pathological subtypes of pulmonary nodules on CT images

OBJECTIVE

To investigate the correlation between CT imaging features and pathological subtypes of pulmonary nodules and construct a prediction model using deep learning.

METHODS

We collected information of patients with pulmonary nodules treated by surgery and the reference standard for diagnosis was post-operative pathology. After using elastic distortion for data augmentation, the CT images were divided into a training set, a validation set and a test set in a ratio of 6:2:2. We used PB-LNet to analyze the nodules in pre-operative CT and predict their pathological subtypes. Accuracy was used as the model evaluation index and Class Activation Map was applied to interpreting the results. Comparative experiments with other models were carried out to achieve the best results. Finally, images from the test set without data augmentation were analyzed to judge the clinical utility.

RESULTS

Four hundred seventy-seven patients were included and the nodules were divided into six groups: benign lesions, precursor glandular lesions, minimally invasive adenocarcinoma, invasive adenocarcinoma Grade 1, Grade 2 and Grade 3. The accuracy of the test set was 0.84. Class Activation Map confirmed that PB-LNet classified the nodules mainly based on the lungs in CT images, which is in line with the actual situation in clinical practice. In comparative experiments, PB-LNet obtained the highest accuracy. Finally, 96 images from the test set without data augmentation were analyzed and the accuracy was 0.89.

CONCLUSIONS

In classifying CT images of lung nodules into six categories based on pathological subtypes, PB-LNet demonstrates satisfactory accuracy without the need of delineating nodules, while the results are interpretable. A high level of accuracy was also obtained when validating on real data, therefore demonstrates its usefulness in clinical practice.

An integrated network based on 2D/3D feature correlations for benign-malignant tumor classification and uncertainty estimation in digital breast tomosynthesis

Objective.Classification of benign and malignant tumors is important for the early diagnosis of breast cancer. Over the last decade, digital breast tomosynthesis (DBT) has gradually become an effective imaging modality for breast cancer diagnosis due to its ability to generate three-dimensional (3D) visualizations. However, computer-aided diagnosis (CAD) systems based on 3D images require high computational costs and time. Furthermore, there is considerable redundant information in 3D images. Most CAD systems are designed based on 2D images, which may lose the spatial depth information of tumors. In this study, we propose a 2D/3D integrated network for the diagnosis of benign and malignant breast tumors.Approach.We introduce a correlation strategy to describe feature correlations between slices in 3D volumes, corresponding to the tissue relationship and spatial depth features of tumors. The correlation strategy can be used to extract spatial features with little computational cost. In the prediction stage, 3D spatial correlation features and 2D features are both used for classification.Main results.Experimental results demonstrate that our proposed framework achieves higher accuracy and reliability than pure 2D or 3D models. Our framework has a high area under the curve of 0.88 and accuracy of 0.82. The parameter size of the feature extractor in our framework is only 35% of that of the 3D models. In reliability evaluations, our proposed model is more reliable than pure 2D or 3D models because of its effective and nonredundant features.Significance.This study successfully combines 3D spatial correlation features and 2D features for the diagnosis of benign and malignant breast tumors in DBT. In addition to high accuracy and low computational cost, our model is more reliable and can output uncertainty value. From this point of view, the proposed method has the potential to be applied in clinic.

Novel tools for early diagnosis and precision treatment based on artificial intelligence

Lung cancer has the highest mortality rate among all cancers in the world. Hence, early diagnosis and personalized treatment plans are crucial to improving its 5-year survival rate. Chest computed tomography (CT) serves as an essential tool for lung cancer screening, and pathology images are the gold standard for lung cancer diagnosis. However, medical image evaluation relies on manual labor and suffers from missed diagnosis or misdiagnosis, and physician heterogeneity. The rapid development of artificial intelligence (AI) has brought a whole novel opportunity for medical task processing, demonstrating the potential for clinical application in lung cancer diagnosis and treatment. AI technologies, including machine learning and deep learning, have been deployed extensively for lung nodule detection, benign and malignant classification, and subtype identification based on CT images. Furthermore, AI plays a role in the non-invasive prediction of genetic mutations and molecular status to provide the optimal treatment regimen, and applies to the assessment of therapeutic efficacy and prognosis of lung cancer patients, enabling precision medicine to become a reality. Meanwhile, histology-based AI models assist pathologists in typing, molecular characterization, and prognosis prediction to enhance the efficiency of diagnosis and treatment. However, the leap to extensive clinical application still faces various challenges, such as data sharing, standardized label acquisition, clinical application regulation, and multimodal integration. Nevertheless, AI holds promising potential in the field of lung cancer to improve cancer care.

An ensemble of three-dimensional deep neural network models for multi-attribute scoring and classification of pulmonary nodules

Lung cancer is the uncontrolled growth of cells that originates in the lung parenchyma or cells that line the air passages. These cells divide rapidly to form malicious tumors. This paper proposes a multi-task ensemble of three dimensional (3D) deep neural network (DNN) based model, namely: pre-trained EfficientNetB0, BiGRU-based SEResNext101, and the proposed LungNet. The ensemble model performs binary classification and regression tasks to accurately classify the benign and malignant pulmonary nodules. This study also explores the attribute importance and proposes a domain knowledge-based regularization technique. The proposed model is evaluated on the public benchmark LIDC-IDRI dataset. Through a comparative study, it was shown that when coefficients generated by the random forest (RF) are used in the loss function, the proposed ensemble model offers a better prediction capability of the accuracy of 96.4% compared to the state-of-the-art methods. In addition, the receiver operating characteristic curves show that the proposed ensemble model has better performance than the base learners. Thus, the proposed CAD-based model can efficiently detect malignant pulmonary nodules.

An integrated convolutional neural network for classifying small pulmonary solid nodules

Achieving accurate classification of benign and malignant pulmonary nodules is essential for treating some diseases. However, traditional typing methods have difficulty obtaining satisfactory results on small pulmonary solid nodules, mainly caused by two aspects: (1) noise interference from other tissue information; (2) missing features of small nodules caused by downsampling in traditional convolutional neural networks. To solve these problems, this paper proposes a new typing method to improve the diagnosis rate of small pulmonary solid nodules in CT images. Specifically, first, we introduce the Otsu thresholding algorithm to preprocess the data and filter the interference information. Then, to acquire more small nodule features, we add parallel radiomics to the 3D convolutional neural network. Radiomics can extract a large number of quantitative features from medical images. Finally, the classifier generated more accurate results by the visual and radiomic features. In the experiments, we tested the proposed method on multiple data sets, and the proposed method outperformed other methods in the small pulmonary solid nodule classification task. In addition, various groups of ablation experiments demonstrated that the Otsu thresholding algorithm and radiomics are helpful for the judgment of small nodules and proved that the Otsu thresholding algorithm is more flexible than the manual thresholding algorithm.

Multiround Transfer Learning and Modified Generative Adversarial Network for Lung Cancer Detection

Lung cancer has been the leading cause of cancer death for many decades. With the advent of artificial intelligence, various machine learning models have been proposed for lung cancer detection (LCD). Typically, challenges in building an accurate LCD model are the small-scale datasets, the poor generalizability to detect unseen data, and the selection of useful source domains and prioritization of multiple source domains for transfer learning. In this paper, a multiround transfer learning and modified generative adversarial network (MTL-MGAN) algorithm is proposed for LCD. The MTL transfers the knowledge between the prioritized source domains and target domain to get rid of exhaust search of datasets prioritization among multiple datasets, maximizing the transferability with a multiround transfer learning process, and avoiding negative transfer via customization of loss functions in the aspects of domain, instance, and feature. In regard to the MGAN, it not only generates additional training data but also creates intermediate domains to bridge the gap between the source domains and target domains. 10 benchmark datasets are chosen for the performance evaluation and analysis of the MTL-MGAN. The proposed algorithm has significantly improved the accuracy compared with related works. To examine the contributions of the individual components of the MTL-MGAN, ablation studies are conducted to confirm the effectiveness of the prioritization algorithm, the MTL, the negative transfer avoidance via loss functions, and the MGAN. The research implications are to confirm the feasibility of multiround transfer learning to enhance the optimal solution of the target model and to provide a generic approach to bridge the gap between the source domain and target domain using MGAN.

The diagnostic and prognostic value of radiomics and deep learning technologies for patients with solid pulmonary nodules in chest CT images

BACKGROUND

Solid pulmonary nodules are different from subsolid nodules and the diagnosis is much more challenging. We intended to evaluate the diagnostic and prognostic value of radiomics and deep learning technologies for solid pulmonary nodules.

METHODS

Retrospectively enroll patients with pathologically-confirmed solid pulmonary nodules and collect clinical data. Obtain pre-treatment high-resolution thoracic CT and manually delineate the nodule in 3D. Then, all patients were randomly divided into training and testing sets at a ratio of 7:3, and convolutional neural networks (CNN) models and random forest (RF) models were established. Survival analyses were performed for patients with solid adenocarcinomas.

RESULTS

Totally 720 solid pulmonary nodules were enrolled, 348 benign and 372 malignant. The CNN model with clinical features achieved the highest AUC [0.819, 95% confidence interval (CI): 0.760-0.877] with a sensitivity of 0.778, specificity of 0.788 and accuracy of 0.783. No significant differences were observed between the CNN and radiomics models. There were 295 solid adenocarcinomas in survival analysis. Different disease-free survival was observed between the low-risk and high-risk groups divided according to the radiomics Rad-score. However, the groups based on deep learning signatures showed similar survival. Cox regression analysis indicated that the radiomics Rad-score (hazard ratio: 5.08, 95% CI: 2.61-9.90) was an independent predictor of recurrence.

CONCLUSIONS

The radiomics and deep learning models can well predict the malignancy of solid pulmonary nodules. Radiomics signatures also demonstrate prognostic value in solid adenocarcinomas.

3D deep learning versus the current methods for predicting tumor invasiveness of lung adenocarcinoma based on high-resolution computed tomography images

Background

Different pathological subtypes of lung adenocarcinoma lead to different treatment decisions and prognoses, and it is clinically important to distinguish invasive lung adenocarcinoma from preinvasive adenocarcinoma (adenocarcinoma in situ and minimally invasive adenocarcinoma). This study aims to investigate the performance of the deep learning approach based on high-resolution computed tomography (HRCT) images in the classification of tumor invasiveness and compare it with the performances of currently available approaches.

Methods

In this study, we used a deep learning approach based on 3D conventional networks to automatically predict the invasiveness of pulmonary nodules. A total of 901 early-stage non-small cell lung cancer patients who underwent surgical treatment at Shanghai Chest Hospital between November 2015 and March 2017 were retrospectively included and randomly assigned to a training set (n=814) or testing set 1 (n=87). We subsequently included 116 patients who underwent surgical treatment and intraoperative frozen section between April 2019 and January 2020 to form testing set 2. We compared the performance of our deep learning approach in predicting tumor invasiveness with that of intraoperative frozen section analysis and human experts (radiologists and surgeons).

Results

The deep learning approach yielded an area under the receiver operating characteristic curve (AUC) of 0.946 for distinguishing preinvasive adenocarcinoma from invasive lung adenocarcinoma in the testing set 1, which is significantly higher than the AUCs of human experts (P<0.05). In testing set 2, the deep learning approach distinguished invasive adenocarcinoma from preinvasive adenocarcinoma with an AUC of 0.862, which is higher than that of frozen section analysis (0.755, P=0.043), senior thoracic surgeons (0.720, P=0.006), radiologists (0.766, P>0.05) and junior thoracic surgeons (0.768, P>0.05).

Conclusions

We developed a deep learning model that achieved comparable performance to intraoperative frozen section analysis in determining tumor invasiveness. The proposed method may contribute to clinical decisions related to the extent of surgical resection.

Using Sparse Patch Annotation for Tumor Segmentation in Histopathological Images

Tumor segmentation is a fundamental task in histopathological image analysis. Creating accurate pixel-wise annotations for such segmentation tasks in a fully-supervised training framework requires significant effort. To reduce the burden of manual annotation, we propose a novel weakly supervised segmentation framework based on sparse patch annotation, i.e., only small portions of patches in an image are labeled as 'tumor' or 'normal'. The framework consists of a patch-wise segmentation model called PSeger, and an innovative semi-supervised algorithm. PSeger has two branches for patch classification and image classification, respectively. This two-branch structure enables the model to learn more general features and thus reduce the risk of overfitting when learning sparsely annotated data. We incorporate the idea of consistency learning and self-training into the semi-supervised training strategy to take advantage of the unlabeled images. Trained on the BCSS dataset with only 25% of the images labeled (five patches for each labeled image), our proposed method achieved competitive performance compared to the fully supervised pixel-wise segmentation models. Experiments demonstrate that the proposed solution has the potential to reduce the burden of labeling histopathological images.

Semi-supervised learning in cancer diagnostics

In cancer diagnostics, a considerable amount of data is acquired during routine work-up. Recently, machine learning has been used to build classifiers that are tasked with cancer detection and aid in clinical decision-making. Most of these classifiers are based on supervised learning (SL) that needs time- and cost-intensive manual labeling of samples by medical experts for model training. Semi-supervised learning (SSL), however, works with only a fraction of labeled data by including unlabeled samples for information abstraction and thus can utilize the vast discrepancy between available labeled data and overall available data in cancer diagnostics. In this review, we provide a comprehensive overview of essential functionalities and assumptions of SSL and survey key studies with regard to cancer care differentiating between image-based and non-image-based applications. We highlight current state-of-the-art models in histopathology, radiology and radiotherapy, as well as genomics. Further, we discuss potential pitfalls in SSL study design such as discrepancies in data distributions and comparison to baseline SL models, and point out future directions for SSL in oncology. We believe well-designed SSL models to strongly contribute to computer-guided diagnostics in malignant disease by overcoming current hinderances in the form of sparse labeled and abundant unlabeled data.