Multi-level Ground Glass Nodule Detection and Segmentation in CT Lung Images. In: Yang G, Hawkes D, Rueckert D, Noble A, Taylor C, editors. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009. Berlin: Springer Berlin Heidelberg; 2009. pp. 715-23. [DOI: 10.1007/978-3-642-04271-3_87]

How AI Can Help in the Diagnostic Dilemma of Pulmonary Nodules

Simple Summary

Pulmonary nodules are considered a sign of bronchogenic carcinoma, detecting them early will reduce their progression and can save lives. Lung cancer is the second most common type of cancer in both men and women. This manuscript discusses the current applications of artificial intelligence (AI) in lung segmentation as well as pulmonary nodule segmentation and classification using computed tomography (CT) scans, published in the last two decades, in addition to the limitations and future prospects in the field of AI.

Abstract

Pulmonary nodules are the precursors of bronchogenic carcinoma, its early detection facilitates early treatment which save a lot of lives. Unfortunately, pulmonary nodule detection and classification are liable to subjective variations with high rate of missing small cancerous lesions which opens the way for implementation of artificial intelligence (AI) and computer aided diagnosis (CAD) systems. The field of deep learning and neural networks is expanding every day with new models designed to overcome diagnostic problems and provide more applicable and simply used models. We aim in this review to briefly discuss the current applications of AI in lung segmentation, pulmonary nodule detection and classification.

Feature-shared adaptive-boost deep learning for invasiveness classification of pulmonary subsolid nodules in CT images

PURPOSE

In clinical practice, invasiveness is an important reference indicator for differentiating the malignant degree of subsolid pulmonary nodules. These nodules can be classified as atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), or invasive adenocarcinoma (IAC). The automatic determination of a nodule's invasiveness based on chest CT scans can guide treatment planning. However, it is challenging, owing to the insufficiency of training data and their interclass similarity and intraclass variation. To address these challenges, we propose a two-stage deep learning strategy for this task: prior-feature learning followed by adaptive-boost deep learning.

METHODS

The adaptive-boost deep learning is proposed to train a strong classifier for invasiveness classification of subsolid nodules in chest CT images, using multiple 3D convolutional neural network (CNN)-based weak classifiers. Because ensembles of multiple deep 3D CNN models have a huge number of parameters and require large computing resources along with more training and testing time, the prior-feature learning is proposed to reduce the computations by sharing the CNN layers between all weak classifiers. Using this strategy, all weak classifiers can be integrated into a single network.

RESULTS

Tenfold cross validation of binary classification was conducted on a total of 1357 nodules, including 765 noninvasive (AAH and AIS) and 592 invasive nodules (MIA and IAC). Ablation experimental results indicated that the proposed binary classifier achieved an accuracy of 73.4 \% ± 1.4 with an AUC of 81.3 \% ± 2.2 . These results are superior compared to those achieved by three experienced chest imaging specialists who achieved an accuracy of 69.1 \% , 69.3 \% , and 67.9 \% , respectively. About 200 additional nodules were also collected. These nodules covered 50 cases for each category (AAH, AIS, MIA, and IAC, respectively). Both binary and multiple classifications were performed on these data and the results demonstrated that the proposed method definitely achieves better performance than the performance achieved by nonensemble deep learning methods.

CONCLUSIONS

It can be concluded that the proposed adaptive-boost deep learning can significantly improve the performance of invasiveness classification of pulmonary subsolid nodules in CT images, while the prior-feature learning can significantly reduce the total size of deep models. The promising results on clinical data show that the trained models can be used as an effective lung cancer screening tool in hospitals. Moreover, the proposed strategy can be easily extended to other similar classification tasks in 3D medical images.

Evolving the pulmonary nodules diagnosis from classical approaches to deep learning-aided decision support: three decades' development course and future prospect

PURPOSE

Lung cancer is the commonest cause of cancer deaths worldwide, and its mortality can be reduced significantly by performing early diagnosis and screening. Since the 1960s, driven by the pressing needs to accurately and effectively interpret the massive volume of chest images generated daily, computer-assisted diagnosis of pulmonary nodule has opened up new opportunities to relax the limitation from physicians' subjectivity, experiences and fatigue. And the fair access to the reliable and affordable computer-assisted diagnosis will fight the inequalities in incidence and mortality between populations. It has been witnessed that significant and remarkable advances have been achieved since the 1980s, and consistent endeavors have been exerted to deal with the grand challenges on how to accurately detect the pulmonary nodules with high sensitivity at low false-positive rate as well as on how to precisely differentiate between benign and malignant nodules. There is a lack of comprehensive examination of the techniques' development which is evolving the pulmonary nodules diagnosis from classical approaches to machine learning-assisted decision support. The main goal of this investigation is to provide a comprehensive state-of-the-art review of the computer-assisted nodules detection and benign-malignant classification techniques developed over three decades, which have evolved from the complicated ad hoc analysis pipeline of conventional approaches to the simplified seamlessly integrated deep learning techniques. This review also identifies challenges and highlights opportunities for future work in learning models, learning algorithms and enhancement schemes for bridging current state to future prospect and satisfying future demand.

CONCLUSION

It is the first literature review of the past 30 years' development in computer-assisted diagnosis of lung nodules. The challenges indentified and the research opportunities highlighted in this survey are significant for bridging current state to future prospect and satisfying future demand. The values of multifaceted driving forces and multidisciplinary researches are acknowledged that will make the computer-assisted diagnosis of pulmonary nodules enter into the main stream of clinical medicine and raise the state-of-the-art clinical applications as well as increase both welfares of physicians and patients. We firmly hold the vision that fair access to the reliable, faithful, and affordable computer-assisted diagnosis for early cancer diagnosis would fight the inequalities in incidence and mortality between populations, and save more lives.

Knowledge-based Collaborative Deep Learning for Benign-Malignant Lung Nodule Classification on Chest CT

The accurate identification of malignant lung nodules on chest CT is critical for the early detection of lung cancer, which also offers patients the best chance of cure. Deep learning methods have recently been successfully introduced to computer vision problems, although substantial challenges remain in the detection of malignant nodules due to the lack of large training data sets. In this paper, we propose a multi-view knowledge-based collaborative (MV-KBC) deep model to separate malignant from benign nodules using limited chest CT data. Our model learns 3-D lung nodule characteristics by decomposing a 3-D nodule into nine fixed views. For each view, we construct a knowledge-based collaborative (KBC) submodel, where three types of image patches are designed to fine-tune three pre-trained ResNet-50 networks that characterize the nodules' overall appearance, voxel, and shape heterogeneity, respectively. We jointly use the nine KBC submodels to classify lung nodules with an adaptive weighting scheme learned during the error back propagation, which enables the MV-KBC model to be trained in an end-to-end manner. The penalty loss function is used for better reduction of the false negative rate with a minimal effect on the overall performance of the MV-KBC model. We tested our method on the benchmark LIDC-IDRI data set and compared it to the five state-of-the-art classification approaches. Our results show that the MV-KBC model achieved an accuracy of 91.60% for lung nodule classification with an AUC of 95.70%. These results are markedly superior to the state-of-the-art approaches.

Expression of p53 in ground-glass nodule of lung cancer and non-lung cancer patients

The present study investigated the expression of p53 in ground-glass nodule (GGN) of lung cancer and non-lung cancer patients, and explored the correlation with prognosis. A total of 120 GGN patients admitted to the Department of Respiratory Medicine in the Second Affiliated Hospital of Zhejiang University School of Medicine during the period from March 2010 to March 2014 were selected. These patients included 60 lung cancer patients and 60 non-tumor patients. Biopsy or surgical specimens were collected. Fluorescence in situ hybridization (FISH) and immunohistochemistry (IHC) were used to detect p53 gene and protein expression in the two groups of GGN tissues. All patients were followed up for 3 years and the relationship between p53 protein expression and the overall survival (OS) of the two groups of patients was analyzed. In GGN cells of non-cancer patients, p53 absence was observed in 6 cases and the absence rate was 10.0%. In GGN cells of cancer patients, the absence rate was significantly higher than that of non-cancer GGN group (p<0.05). The positive rate of p53-positive cases in non-tumor patients GGN group was lower than that of in GGN tissues of lung cancer patients (p<0.05). There were no deaths in the GGN non-cancer group (n=60) within 3 years, while 43 deaths occurred in GGN lung cancer group. The median survival time and the 3-year survival rate of patients with p53 positive was lower than that of p53-negative patients (p<0.05). p53 was overexpressed in GGN of lung cancer patients, and p53 overexpression is significantly correlated with poor prognosis of lung cancer patients. p53 plays an important role in transformation from GGN to lung cancer. Detection of p53 expression in GGN tissue may provide guidance for the diagnosis and prognosis of lung cancer.

Application of the 3D slicer chest imaging platform segmentation algorithm for large lung nodule delineation

PURPOSE

Accurate segmentation of lung nodules is crucial in the development of imaging biomarkers for predicting malignancy of the nodules. Manual segmentation is time consuming and affected by inter-observer variability. We evaluated the robustness and accuracy of a publically available semiautomatic segmentation algorithm that is implemented in the 3D Slicer Chest Imaging Platform (CIP) and compared it with the performance of manual segmentation.

METHODS

CT images of 354 manually segmented nodules were downloaded from the LIDC database. Four radiologists performed the manual segmentation and assessed various nodule characteristics. The semiautomatic CIP segmentation was initialized using the centroid of the manual segmentations, thereby generating four contours for each nodule. The robustness of both segmentation methods was assessed using the region of uncertainty (δ) and Dice similarity index (DSI). The robustness of the segmentation methods was compared using the Wilcoxon-signed rank test (pWilcoxon<0.05). The Dice similarity index (DSIAgree) between the manual and CIP segmentations was computed to estimate the accuracy of the semiautomatic contours.

RESULTS

The median computational time of the CIP segmentation was 10 s. The median CIP and manually segmented volumes were 477 ml and 309 ml, respectively. CIP segmentations were significantly more robust than manual segmentations (median δCIP = 14ml, median dsiCIP = 99% vs. median δmanual = 222ml, median dsimanual = 82%) with pWilcoxon~10-16. The agreement between CIP and manual segmentations had a median DSIAgree of 60%. While 13% (47/354) of the nodules did not require any manual adjustment, minor to substantial manual adjustments were needed for 87% (305/354) of the nodules. CIP segmentations were observed to perform poorly (median DSIAgree≈50%) for non-/sub-solid nodules with subtle appearances and poorly defined boundaries.

CONCLUSION

Semi-automatic CIP segmentation can potentially reduce the physician workload for 13% of nodules owing to its computational efficiency and superior stability compared to manual segmentation. Although manual adjustment is needed for many cases, CIP segmentation provides a preliminary contour for physicians as a starting point.

A Segmentation Framework of Pulmonary Nodules in Lung CT Images

Accurate segmentation of pulmonary nodules is a prerequisite for acceptable performance of computer-aided detection (CAD) system designed for diagnosis of lung cancer from lung CT images. Accurate segmentation helps to improve the quality of machine level features which could improve the performance of the CAD system. The well-circumscribed solid nodules can be segmented using thresholding, but segmentation becomes difficult for part-solid, non-solid, and solid nodules attached with pleura or vessels. We proposed a segmentation framework for all types of pulmonary nodules based on internal texture (solid/part-solid and non-solid) and external attachment (juxta-pleural and juxta-vascular). In the proposed framework, first pulmonary nodules are categorized into solid/part-solid and non-solid category by analyzing intensity distribution in the core of the nodule. Two separate segmentation methods are developed for solid/part-solid and non-solid nodules, respectively. After determining the category of nodule, the particular algorithm is set to remove attached pleural surface and vessels from the nodule body. The result of segmentation is evaluated in terms of four contour-based metrics and six region-based metrics for 891 pulmonary nodules from Lung Image Database Consortium and Image Database Resource Initiative (LIDC/IDRI) public database. The experimental result shows that the proposed segmentation framework is reliable for segmentation of various types of pulmonary nodules with improved accuracy compared to existing segmentation methods.

Predicting adenocarcinoma recurrence using computational texture models of nodule components in lung CT

PURPOSE

To investigate the importance of presurgical computed tomography (CT) intensity and texture information from ground-glass opacities (GGO) and solid nodule components for the prediction of adenocarcinoma recurrence.

METHODS

For this study, 101 patients with surgically resected stage I adenocarcinoma were selected. During the follow-up period, 17 patients had disease recurrence with six associated cancer-related deaths. GGO and solid tumor components were delineated on presurgical CT scans by a radiologist. Computational texture models of GGO and solid regions were built using linear combinations of steerable Riesz wavelets learned with linear support vector machines (SVMs). Unlike other traditional texture attributes, the proposed texture models are designed to encode local image scales and directions that are specific to GGO and solid tissue. The responses of the locally steered models were used as texture attributes and compared to the responses of unaligned Riesz wavelets. The texture attributes were combined with CT intensities to predict tumor recurrence and patient hazard according to disease-free survival (DFS) time. Two families of predictive models were compared: LASSO and SVMs, and their survival counterparts: Cox-LASSO and survival SVMs.

RESULTS

The best-performing predictive model of patient hazard was associated with a concordance index (C-index) of 0.81 ± 0.02 and was based on the combination of the steered models and CT intensities with survival SVMs. The same feature group and the LASSO model yielded the highest area under the receiver operating characteristic curve (AUC) of 0.8 ± 0.01 for predicting tumor recurrence, although no statistically significant difference was found when compared to using intensity features solely. For all models, the performance was found to be significantly higher when image attributes were based on the solid components solely versus using the entire tumors (p < 3.08 × 10(-5)).

CONCLUSIONS

This study constitutes a novel perspective on how to interpret imaging information from CT examinations by suggesting that most of the information related to adenocarcinoma aggressiveness is related to the intensity and morphological properties of solid components of the tumor. The prediction of adenocarcinoma relapse was found to have low specificity but very high sensitivity. Our results could be useful in clinical practice to identify patients for which no recurrence is expected with a very high confidence using a presurgical CT scan only. It also provided an accurate estimation of the risk of recurrence after a given duration t from surgical resection (i.e., C-index = 0.81 ± 0.02).

Random Walk and Graph Cut for Co-Segmentation of Lung Tumor on PET-CT Images

Accurate lung tumor delineation plays an important role in radiotherapy treatment planning. Since the lung tumor has poor boundary in positron emission tomography (PET) images and low contrast in computed tomography (CT) images, segmentation of tumor in the PET and CT images is a challenging task. In this paper, we effectively integrate the two modalities by making fully use of the superior contrast of PET images and superior spatial resolution of CT images. Random walk and graph cut method is integrated to solve the segmentation problem, in which random walk is utilized as an initialization tool to provide object seeds for graph cut segmentation on the PET and CT images. The co-segmentation problem is formulated as an energy minimization problem which is solved by max-flow/min-cut method. A graph, including two sub-graphs and a special link, is constructed, in which one sub-graph is for the PET and another is for CT, and the special link encodes a context term which penalizes the difference of the tumor segmentation on the two modalities. To fully utilize the characteristics of PET and CT images, a novel energy representation is devised. For the PET, a downhill cost and a 3D derivative cost are proposed. For the CT, a shape penalty cost is integrated into the energy function which helps to constrain the tumor region during the segmentation. We validate our algorithm on a data set which consists of 18 PET-CT images. The experimental results indicate that the proposed method is superior to the graph cut method solely using the PET or CT is more accurate compared with the random walk method, random walk co-segmentation method, and non-improved graph cut method.

Deformable part models for object detection in medical images

BACKGROUND

Object detection in 3-D medical images is often necessary for constraining a segmentation or registration task. It may be a task in its own right as well, when instances of a structure, e.g. the lymph nodes, are searched. Problems from occlusion, illumination and projection do not arise, making the problem simpler than object detection in photographies. However, objects of interest are often not well contrasted against the background. Influence from noise and other artifacts is much stronger and shape and appearance may vary substantially within a class.

METHODS

Deformable models capture the characteristic shape of an anatomic object and use constrained deformation for hypothesing object boundaries in image regions of low or non-existing contrast. Learning these constraints requires a large sample data base. We show that training may be replaced by readily available user knowledge defining a prototypical deformable part model. If structures have a strong part-relationship, or if they may be found based on spatially related guiding structures, or if the deformation is rather restricted, the supporting data information suffices for solving the detection task. We use a finite element model to represent anatomic variation by elastic deformation. Complex shape variation may be represented by a hierarchical model with simpler part variation. The hierarchy may be represented explicitly as a hierarchy of sub-shapes, or implicitly by a single integrated model. Data support and model deformation of the complete model can be represented by an energy term, serving as quality-of-fit function for object detection.

RESULTS

The model was applied to detection and segmentation tasks in various medical applications in 2- and 3-D scenes. It has been shown that model fitting and object detection can be carried out efficiently by a combination of a local and global search strategy using models that are parameterized for the different tasks.

CONCLUSIONS

A part-based elastic model represents complex within-class object variation without training. The hierarchy of parts may specify relationship to neighboring anatomical objects in object detection or a part-decomposition of a complex anatomic structure. The intuitive way to incorporate domain knowledge has a high potential to serve as easily adaptable method to a wide range of different detection tasks in medical image analysis.

Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images

Subsolid pulmonary nodules occur less often than solid pulmonary nodules, but show a much higher malignancy rate. Therefore, accurate detection of this type of pulmonary nodules is crucial. In this work, a computer-aided detection (CAD) system for subsolid nodules in computed tomography images is presented and evaluated on a large data set from a multi-center lung cancer screening trial. The paper describes the different components of the CAD system and presents experiments to optimize the performance of the proposed CAD system. A rich set of 128 features is defined for subsolid nodule candidates. In addition to previously used intensity, shape and texture features, a novel set of context features is introduced. Experiments show that these features significantly improve the classification performance. Optimization and training of the CAD system is performed on a large training set from one site of a lung cancer screening trial. Performance analysis on an independent test from another site of the trial shows that the proposed system reaches a sensitivity of 80% at an average of only 1.0 false positive detections per scan. A retrospective analysis of the output of the CAD system by an experienced thoracic radiologist shows that the CAD system is able to find subsolid nodules which were not contained in the screening database.

Immunohistochemical analysis of breast tissue microarray images using contextual classifiers

BACKGROUND

Tissue microarrays (TMAs) are an important tool in translational research for examining multiple cancers for molecular and protein markers. Automatic immunohistochemical (IHC) scoring of breast TMA images remains a challenging problem.

METHODS

A two-stage approach that involves localization of regions of invasive and in-situ carcinoma followed by ordinal IHC scoring of nuclei in these regions is proposed. The localization stage classifies locations on a grid as tumor or non-tumor based on local image features. These classifications are then refined using an auto-context algorithm called spin-context. Spin-context uses a series of classifiers to integrate image feature information with spatial context information in the form of estimated class probabilities. This is achieved in a rotationally-invariant manner. The second stage estimates ordinal IHC scores in terms of the strength of staining and the proportion of nuclei stained. These estimates take the form of posterior probabilities, enabling images with uncertain scores to be referred for pathologist review.

RESULTS

The method was validated against manual pathologist scoring on two nuclear markers, progesterone receptor (PR) and estrogen receptor (ER). Errors for PR data were consistently lower than those achieved with ER data. Scoring was in terms of estimated proportion of cells that were positively stained (scored on an ordinal scale of 0-6) and perceived strength of staining (scored on an ordinal scale of 0-3). Average absolute differences between predicted scores and pathologist-assigned scores were 0.74 for proportion of cells and 0.35 for strength of staining (PR).

CONCLUSIONS

The use of context information via spin-context improved the precision and recall of tumor localization. The combination of the spin-context localization method with the automated scoring method resulted in reduced IHC scoring errors.

Computer-aided diagnosis systems for lung cancer: challenges and methodologies

This paper overviews one of the most important, interesting, and challenging problems in oncology, the problem of lung cancer diagnosis. Developing an effective computer-aided diagnosis (CAD) system for lung cancer is of great clinical importance and can increase the patient's chance of survival. For this reason, CAD systems for lung cancer have been investigated in a huge number of research studies. A typical CAD system for lung cancer diagnosis is composed of four main processing steps: segmentation of the lung fields, detection of nodules inside the lung fields, segmentation of the detected nodules, and diagnosis of the nodules as benign or malignant. This paper overviews the current state-of-the-art techniques that have been developed to implement each of these CAD processing steps. For each technique, various aspects of technical issues, implemented methodologies, training and testing databases, and validation methods, as well as achieved performances, are described. In addition, the paper addresses several challenges that researchers face in each implementation step and outlines the strengths and drawbacks of the existing approaches for lung cancer CAD systems.

Computer-aided detection of ground glass nodules in thoracic CT images using shape, intensity and context features

Ground glass nodules (GGNs) occur less frequent in computed tomography (CT) scans than solid nodules but have a much higher chance of being malignant. Accurate detection of these nodules is therefore highly important. A complete system for computer-aided detection of GGNs is presented consisting of initial segmentation steps, candidate detection, feature extraction and a two-stage classification process. A rich set of intensity, shape and context features is constructed to describe the appearance of GGN candidates. We apply a two-stage classification approach using a linear discriminant classifier and a GentleBoost classifier to efficiently classify candidate regions. The system is trained and independently tested on 140 scans that contained one or more GGNs from around 10,000 scans obtained in a lung cancer screening trial. The system shows a high sensitivity of 73% at only one false positive per scan.

Discriminative pathological context detection in thoracic images based on multi-level inference

Positron emission tomography - computed tomography (PET-CT) is now accepted as the best imaging technique to accurately stage lung cancer. The consistent and accurate interpretation of PET-CT images, however, is not a trivial task. We propose a discriminative, multi-level learning and inference method to automatically detect the pathological contexts in the thoracic PET-CT images, i.e. the primary tumor and its spatial relationships within the lung and mediastinum, and disease in regional lymph nodes. The detection results can also be used as features to retrieve similar images with previous diagnosis from an imaging database as a reference set to aid physicians in PET-CT scan interpretation. Our evaluation with clinical data from lung cancer patients suggests our approach is highly accurate.