Automated Lung Nodule Detection and Classification Using Deep Learning Combined with Multiple Strategies

Convolutional neural network for discriminating nasopharyngeal carcinoma and benign hyperplasia on MRI

OBJECTIVES

A convolutional neural network (CNN) was adapted to automatically detect early-stage nasopharyngeal carcinoma (NPC) and discriminate it from benign hyperplasia on a non-contrast-enhanced MRI sequence for potential use in NPC screening programs.

METHODS

We retrospectively analyzed 412 patients who underwent T2-weighted MRI, 203 of whom had biopsy-proven primary NPC confined to the nasopharynx (stage T1) and 209 had benign hyperplasia without NPC. Thirteen patients were sampled randomly to monitor the training process. We applied the Residual Attention Network architecture, adapted for three-dimensional MR images, and incorporated a slice-attention mechanism, to produce a CNN score of 0-1 for NPC probability. Threefold cross-validation was performed in 399 patients. CNN scores between the NPC and benign hyperplasia groups were compared using Student's t test. Receiver operating characteristic with the area under the curve (AUC) was performed to identify the optimal CNN score threshold.

RESULTS

In each fold, significant differences were observed in the CNN scores between the NPC and benign hyperplasia groups (p < .01). The AUCs ranged from 0.95 to 0.97 with no significant differences between the folds (p = .35 to .92). The combined AUC from all three folds (n = 399) was 0.96, with an optimal CNN score threshold of > 0.71, producing a sensitivity, specificity, and accuracy of 92.4%, 90.6%, and 91.5%, respectively, for NPC detection.

CONCLUSION

Our CNN method applied to T2-weighted MRI could discriminate between malignant and benign tissues in the nasopharynx, suggesting that it as a promising approach for the automated detection of early-stage NPC.

KEY POINTS

• The convolutional neural network (CNN)-based algorithm could automatically discriminate between malignant and benign diseases using T2-weighted fat-suppressed MR images. • The CNN-based algorithm had an accuracy of 91.5% with an area under the receiver operator characteristic curve of 0.96 for discriminating early-stage T1 nasopharyngeal carcinoma from benign hyperplasia. • The CNN-based algorithm had a sensitivity of 92.4% and specificity of 90.6% for detecting early-stage nasopharyngeal carcinoma.

Multiple Primary Lung Cancers: A New Challenge in the Era of Precision Medicine

With the widespread implementation of lung cancer screening, more and more patients are being diagnosed with multiple primary lung cancers (MPLCs). In the era of precision medicine, many controversies remain in differentiating MPLCs from intrapulmonary metastasis and the optimum treatment choice, especially in patients exhibiting similar histology. In this review, we summarize common diagnostic criteria and novel discrimination methods with a special emphasis on the emerging value of broad panel next-generation sequencing (NGS) for the diagnosis of MPLCs. We then discuss current advances regarding therapeutic approaches for MPLCs. Radical surgery is the main treatment modality, while stereotactic body radiotherapy (SBRT) is safe and feasible for early-stage MPLC patients with inoperable tumors. In addition, immunotherapy and targeted therapy, particularly epidermal growth factor receptor-tyrosine kinase inhibitors, are emerging therapeutic strategies that are still in their infancy. Characteristics of both genomic profiles and tumor microenvironment are currently being evaluated but warrant further exploration to facilitate the application of targeted systematic therapies in MPLC patients.

A multi-task pipeline with specialized streams for classification and segmentation of infection manifestations in COVID-19 scans

We are concerned with the challenge of coronavirus disease (COVID-19) detection in chest X-ray and Computed Tomography (CT) scans, and the classification and segmentation of related infection manifestations. Even though it is arguably not an established diagnostic tool, using machine learning-based analysis of COVID-19 medical scans has shown the potential to provide a preliminary digital second opinion. This can help in managing the current pandemic, and thus has been attracting significant research attention. In this research, we propose a multi-task pipeline that takes advantage of the growing advances in deep neural network models. In the first stage, we fine-tuned an Inception-v3 deep model for COVID-19 recognition using multi-modal learning, that is, using X-ray and CT scans. In addition to outperforming other deep models on the same task in the recent literature, with an attained accuracy of 99.4%, we also present comparative analysis for multi-modal learning against learning from X-ray scans alone. The second and the third stages of the proposed pipeline complement one another in dealing with different types of infection manifestations. The former features a convolutional neural network architecture for recognizing three types of manifestations, while the latter transfers learning from another knowledge domain, namely, pulmonary nodule segmentation in CT scans, to produce binary masks for segmenting the regions corresponding to these manifestations. Our proposed pipeline also features specialized streams in which multiple deep models are trained separately to segment specific types of infection manifestations, and we show the significant impact that this framework has on various performance metrics. We evaluate the proposed models on widely adopted datasets, and we demonstrate an increase of approximately 2.5% and 4.5% for dice coefficient and mean intersection-over-union (mIoU), respectively, while achieving 60% reduction in computational time, compared to the recent literature.

Artificial Intelligence in Lung Cancer: Bridging the Gap Between Computational Power and Clinical Decision-Making

Lung cancer remains the most common cause of cancer death worldwide. Recent advances in lung cancer screening, radiotherapy, surgical techniques, and systemic therapy have led to increasing complexity in diagnosis, treatment decision-making, and assessment of recurrence. Artificial intelligence (AI)-based prediction models are being developed to address these issues and may have a future role in screening, diagnosis, treatment selection, and decision-making around salvage therapy. Imaging plays an essential role in all components of lung cancer management and has the potential to play a key role in AI applications. Artificial intelligence has demonstrated value in prognostic biomarker discovery in lung cancer diagnosis, treatment, and response assessment, putting it at the forefront of the next phase of personalized medicine. However, although exploratory studies demonstrate potential utility, there is a need for rigorous validation and standardization before AI can be utilized in clinical decision-making. In this review, we will provide a summary of the current literature implementing AI for outcome prediction in lung cancer. We will describe the anticipated impact of AI on the management of patients with lung cancer and discuss the challenges of clinical implementation of these techniques.

CT-Based Deep Learning Model for Invasiveness Classification and Micropapillary Pattern Prediction Within Lung Adenocarcinoma

Objective: Identification of tumor invasiveness of pulmonary adenocarcinomas before surgery is one of the most important guides to surgical planning. Additionally, preoperative diagnosis of lung adenocarcinoma with micropapillary patterns is also critical for clinical decision making. We aimed to evaluate the accuracy of deep learning models on classifying invasiveness degree and attempted to predict the micropapillary pattern in lung adenocarcinoma.

Methods: The records of 291 histopathologically confirmed lung adenocarcinoma patients were retrospectively analyzed and consisted of 61 adenocarcinoma in situ, 80 minimally invasive adenocarcinoma, 117 invasive adenocarcinoma, and 33 invasive adenocarcinoma with micropapillary components (>5%). We constructed two diagnostic models, the Lung-DL model and the Dense model, based on the LeNet and the DenseNet architecture, respectively.

Results: For distinguishing the nodule invasiveness degree, the area under the curve (AUC) value of the diagnosis with the Lung-DL model is 0.88 and that with the Dense model is 0.86. In the prediction of the micropapillary pattern, overall accuracies of 92 and 72.91% were obtained for the Lung-DL model and the Dense model, respectively.

Conclusion: Deep learning was successfully used for the invasiveness classification of pulmonary adenocarcinomas. This is also the first time that deep learning techniques have been used to predict micropapillary patterns. Both tasks can increase efficiency and assist in the creation of precise individualized treatment plans.

[Technological Innovations in Pulmonology - Examples from Diagnostics and Therapy]

A significant proportion of the current technological developments in pneumology originate from the various areas of information technology. The spectrum ranges from smartphone apps to be used in daily life or in patient care to the use of artificial intelligence in screening and early detection of diseases. The diagnostic accuracy of apps for symptom analysis is currently very limited. Research projects are performed on the integration of symptoms and functional parameters into early detection, but also on mobility measurements as a prognostic marker in COPD. Lung cancer screening using computed tomography represents a major challenge. Here, artificial intelligence can help radiologists to cope with huge amounts of data. However, the quality of the software depends on the sufficient training of the system. Technological developments shape all fields of pneumology. For diagnostic and interventional endoscopy, they offer improved biopsy techniques and microstructural imaging. Advances in lung function measurements allow the differentiated analysis of respiratory mechanical disorders, and they could be transferred to ventilation technology. The translation of basic findings about the lung microbiome into patient care may perspectively help to better understand and treat COPD exacerbations.

Hybrid deep learning for detecting lung diseases from X-ray images

Lung disease is common throughout the world. These include chronic obstructive pulmonary disease, pneumonia, asthma, tuberculosis, fibrosis, etc. Timely diagnosis of lung disease is essential. Many image processing and machine learning models have been developed for this purpose. Different forms of existing deep learning techniques including convolutional neural network (CNN), vanilla neural network, visual geometry group based neural network (VGG), and capsule network are applied for lung disease prediction. The basic CNN has poor performance for rotated, tilted, or other abnormal image orientation. Therefore, we propose a new hybrid deep learning framework by combining VGG, data augmentation and spatial transformer network (STN) with CNN. This new hybrid method is termed here as VGG Data STN with CNN (VDSNet). As implementation tools, Jupyter Notebook, Tensorflow, and Keras are used. The new model is applied to NIH chest X-ray image dataset collected from Kaggle repository. Full and sample versions of the dataset are considered. For both full and sample datasets, VDSNet outperforms existing methods in terms of a number of metrics including precision, recall, F0.5 score and validation accuracy. For the case of full dataset, VDSNet exhibits a validation accuracy of 73%, while vanilla gray, vanilla RGB, hybrid CNN and VGG, and modified capsule network have accuracy values of 67.8%, 69%, 69.5% and 63.8%, respectively. When sample dataset rather than full dataset is used, VDSNet requires much lower training time at the expense of a slightly lower validation accuracy. Hence, the proposed VDSNet framework will simplify the detection of lung disease for experts as well as for doctors.

Efficient Pneumonia Detection in Chest Xray Images Using Deep Transfer Learning

Pneumonia causes the death of around 700,000 children every year and affects 7% of the global population. Chest X-rays are primarily used for the diagnosis of this disease. However, even for a trained radiologist, it is a challenging task to examine chest X-rays. There is a need to improve the diagnosis accuracy. In this work, an efficient model for the detection of pneumonia trained on digital chest X-ray images is proposed, which could aid the radiologists in their decision making process. A novel approach based on a weighted classifier is introduced, which combines the weighted predictions from the state-of-the-art deep learning models such as ResNet18, Xception, InceptionV3, DenseNet121, and MobileNetV3 in an optimal way. This approach is a supervised learning approach in which the network predicts the result based on the quality of the dataset used. Transfer learning is used to fine-tune the deep learning models to obtain higher training and validation accuracy. Partial data augmentation techniques are employed to increase the training dataset in a balanced way. The proposed weighted classifier is able to outperform all the individual models. Finally, the model is evaluated, not only in terms of test accuracy, but also in the AUC score. The final proposed weighted classifier model is able to achieve a test accuracy of 98.43% and an AUC score of 99.76 on the unseen data from the Guangzhou Women and Children's Medical Center pneumonia dataset. Hence, the proposed model can be used for a quick diagnosis of pneumonia and can aid the radiologists in the diagnosis process.

Noninvasive biomarkers for lung cancer diagnosis, where do we stand?

The 2010's saw demonstration of the power of lung cancer screening to reduce mortality. However, with implementation of lung cancer screening comes the challenge of diagnosing millions of lung nodules every year. When compared to other cancers with widespread screening strategies (breast, colorectal, cervical, prostate, and skin), obtaining a lung nodule tissue biopsy to confirm a positive screening test remains associated with higher morbidity and cost. Therefore, non-invasive diagnostic biomarkers may have a unique opportunity in lung cancer to greatly improve the management of patients at risk. This review covers recent advances in the field of liquid biomarkers and computed tomographic imaging features, with special attention to new methods for combination of biomarkers as well as the use of artificial intelligence for the discrimination of benign from malignant nodules.

Can human posture and range of motion be measured automatically by smart mobile applications?

Human posture and Range of Motion (ROM) are important components of a physical assessment and, from the collected data, it is possible to identify postural deviations such as scoliosis or joint and muscle limitations, hence identifying risks of more serious injuries. Posture assessment and ROM measures are also necessary metrics to monitor the effect of treatments used in the motor rehabilitation of patients, as well as to monitor their clinical progress. These evaluation processes are more frequently performed through visual inspection and manual palpation, which are simple and low cost methods. These methods, however, can be optimized with the use of tools such as photogrammetry and goniometry. Mobile solutions have also been developed to help health professionals to capture more objective data and with less risk of bias. Although there are already several systems proposed for assessing human posture and ROM in the literature, they have not been able to automatically identify and mark Anatomical and Segment Points (ASPs). The hypothesis presented here considers the development of a mobile application for automatic identification of ASPs by using machine learning algorithms and computer vision models associated with technologies embedded in smartphones. From ASPs identification, it will be possible to identify changes in postural alignment and ROM. In this context, our view is that an application derived from the hypothesis will serve as an additional tool to assist in the physical assessment process and, consequently, in the diagnosis of disorders related to postural and movement changes.

Quantifying the incremental value of deep learning: Application to lung nodule detection

We present a case study for implementing a machine learning algorithm with an incremental value framework in the domain of lung cancer research. Machine learning methods have often been shown to be competitive with prediction models in some domains; however, implementation of these methods is in early development. Often these methods are only directly compared to existing methods; here we present a framework for assessing the value of a machine learning model by assessing the incremental value. We developed a machine learning model to identify and classify lung nodules and assessed the incremental value added to existing risk prediction models. Multiple external datasets were used for validation. We found that our image model, trained on a dataset from The Cancer Imaging Archive (TCIA), improves upon existing models that are restricted to patient characteristics, but it was inconclusive about whether it improves on models that consider nodule features. Another interesting finding is the variable performance on different datasets, suggesting population generalization with machine learning models may be more challenging than is often considered.

A Two-Stage Framework for Automated Malignant Pulmonary Nodule Detection in CT Scans

This research is concerned with malignant pulmonary nodule detection (PND) in low-dose CT scans. Due to its crucial role in the early diagnosis of lung cancer, PND has considerable potential in improving the survival rate of patients. We propose a two-stage framework that exploits the ever-growing advances in deep neural network models, and that is comprised of a semantic segmentation stage followed by localization and classification. We employ the recently published DeepLab model for semantic segmentation, and we show that it significantly improves the accuracy of nodule detection compared to the classical U-Net model and its most recent variants. Using the widely adopted Lung Nodule Analysis dataset (LUNA16), we evaluate the performance of the semantic segmentation stage by adopting two network backbones, namely, MobileNet-V2 and Xception. We present the impact of various model training parameters and the computational time on the detection accuracy, featuring a 79.1% mean intersection-over-union (mIoU) and an 88.34% dice coefficient. This represents a mIoU increase of 60% and a dice coefficient increase of 30% compared to U-Net. The second stage involves feeding the output of the DeepLab-based semantic segmentation to a localization-then-classification stage. The second stage is realized using Faster RCNN and SSD, with an Inception-V2 as a backbone. On LUNA16, the two-stage framework attained a sensitivity of 96.4%, outperforming other recent models in the literature, including deep models. Finally, we show that adopting a transfer learning approach, particularly, the DeepLab model weights of the first stage of the framework, to infer binary (malignant-benign) labels on the Kaggle dataset for pulmonary nodules achieves a classification accuracy of 95.66%, which represents approximately 4% improvement over the recent literature.

Lung Nodule Detection in CT Images Using Statistical and Shape-Based Features

The lung tumor is among the most detrimental kinds of malignancy. It has a high occurrence rate and a high death rate, as it is frequently diagnosed at the later stages. Computed Tomography (CT) scans are broadly used to distinguish the disease; computer aided systems are being created to analyze the ailment at prior stages productively. In this paper, we present a fully automatic framework for nodule detection from CT images of lungs. A histogram of the grayscale CT image is computed to automatically isolate the lung locale from the foundation. The results are refined using morphological operators. The internal structures are then extracted from the parenchyma. A threshold-based technique is proposed to separate the candidate nodules from other structures, e.g., bronchioles and blood vessels. Different statistical and shape-based features are extracted for these nodule candidates to form nodule feature vectors which are classified using support vector machines. The proposed method is evaluated on a large lungs CT dataset collected from the Lung Image Database Consortium (LIDC). The proposed method achieved excellent results compared to similar existing methods; it achieves a sensitivity rate of 93.75%, which demonstrates its effectiveness.

A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images

Pneumonia is among the top diseases which cause most of the deaths all over the world. Virus, bacteria and fungi can all cause pneumonia. However, it is difficult to judge the pneumonia just by looking at chest X-rays. The aim of this study is to simplify the pneumonia detection process for experts as well as for novices. We suggest a novel deep learning framework for the detection of pneumonia using the concept of transfer learning. In this approach, features from images are extracted using different neural network models pretrained on ImageNet, which then are fed into a classifier for prediction. We prepared five different models and analyzed their performance. Thereafter, we proposed an ensemble model that combines outputs from all pretrained models, which outperformed individual models, reaching the state-of-the-art performance in pneumonia recognition. Our ensemble model reached an accuracy of 96.4% with a recall of 99.62% on unseen data from the Guangzhou Women and Children’s Medical Center dataset.

Automated Identification of Wood Veneer Surface Defects Using Faster Region-Based Convolutional Neural Network with Data Augmentation and Transfer Learning

In the lumber and wood processing industry, most visual quality inspections are still done by trained human operators. Visual inspection is a tedious and repetitive task that involves a high likelihood of human error. Currently, new automated solutions with high-resolution cameras and visual inspection algorithms are being tested, but they are not always fast and accurate enough for real-time industrial applications. This paper proposes an automatic visual inspection system for the location and classification of defects on the wood surface. We adopted a faster region-based convolutional neural network (faster R-CNN) for the identification of defects on wood veneer surfaces. Faster R-CNN has been successfully used in medical image processing and object tracking before, but it has not yet been applied for wood panel surface quality assurance. To improve the results, we used pre-trained AlexNet, VGG16, BNInception, and ResNet152 neural network models for transfer learning. The results of the experiments using a synthetically augmented dataset are presented. The best average accuracy of 80.6% was obtained using the pretrained ResNet152 neural network model. By combining all the defect classes, a 96.1% accuracy of finding wood panel surface defects was achieved.