Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis

Predicting Unnecessary Nodule Biopsies from a Small, Unbalanced, and Pathologically Proven Dataset by Transfer Learning

This study explores an automatic diagnosis method to predict unnecessary nodule biopsy from a small, unbalanced, and pathologically proven database. The automatic diagnosis method is based on a convolutional neural network (CNN) model. Because of the small and unbalanced samples, the presented method aims to improve the transfer learning capability via the VGG16 architecture and optimize the related transfer learning parameters. For comparison purpose, a traditional machine learning method is implemented, which extracts the texture features and classifies the features by support vector machine (SVM). The database includes 68 biopsied nodules, 16 are pathologically proven benign and the remaining 52 are malignant. To consider the volumetric data by the CNN model, each image slice from each nodule volume is selected randomly until all image slices of each nodule are utilized. The leave-one-out and 10-folder cross validations are applied to train and test the randomly selected 68 image slices (one image slice from one nodule) in each experiment, respectively. The averages over all the experimental outcomes are the final results. The experiments revealed that the features from both the medical and the natural images share the similarity of focusing on simpler and less-abstract objects, leading to the conclusion that not the more the transfer convolutional layers, the better the classification results. Transfer learning from other larger datasets can supply additional information to small and unbalanced datasets to improve the classification performance. The presented method has shown the potential to adapt CNN architecture to improve the prediction of unnecessary nodule biopsy from small, unbalanced, and pathologically proven volumetric dataset.

Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks

Fast diagnostic methods can control and prevent the spread of pandemic diseases like coronavirus disease 2019 (COVID-19) and assist physicians to better manage patients in high workload conditions. Although a laboratory test is the current routine diagnostic tool, it is time-consuming, imposing a high cost and requiring a well-equipped laboratory for analysis. Computed tomography (CT) has thus far become a fast method to diagnose patients with COVID-19. However, the performance of radiologists in diagnosis of COVID-19 was moderate. Accordingly, additional investigations are needed to improve the performance in diagnosing COVID-19. In this study is suggested a rapid and valid method for COVID-19 diagnosis using an artificial intelligence technique based. 1020 CT slices from 108 patients with laboratory proven COVID-19 (the COVID-19 group) and 86 patients with other atypical and viral pneumonia diseases (the non-COVID-19 group) were included. Ten well-known convolutional neural networks were used to distinguish infection of COVID-19 from non-COVID-19 groups: AlexNet, VGG-16, VGG-19, SqueezeNet, GoogleNet, MobileNet-V2, ResNet-18, ResNet-50, ResNet-101, and Xception. Among all networks, the best performance was achieved by ResNet-101 and Xception. ResNet-101 could distinguish COVID-19 from non-COVID-19 cases with an AUC of 0.994 (sensitivity, 100%; specificity, 99.02%; accuracy, 99.51%). Xception achieved an AUC of 0.994 (sensitivity, 98.04%; specificity, 100%; accuracy, 99.02%). However, the performance of the radiologist was moderate with an AUC of 0.873 (sensitivity, 89.21%; specificity, 83.33%; accuracy, 86.27%). ResNet-101 can be considered as a high sensitivity model to characterize and diagnose COVID-19 infections, and can be used as an adjuvant tool in radiology departments.

Homecare Robotic Systems for Healthcare 4.0: Visions and Enabling Technologies

Powered by the technologies that have originated from manufacturing, the fourth revolution of healthcare technologies is happening (Healthcare 4.0). As an example of such revolution, new generation homecare robotic systems (HRS) based on the cyber-physical systems (CPS) with higher speed and more intelligent execution are emerging. In this article, the new visions and features of the CPS-based HRS are proposed. The latest progress in related enabling technologies is reviewed, including artificial intelligence, sensing fundamentals, materials and machines, cloud computing and communication, as well as motion capture and mapping. Finally, the future perspectives of the CPS-based HRS and the technical challenges faced in each technical area are discussed.

Deep learning for automated cerebral aneurysm detection on computed tomography images

PURPOSE

Cerebrovascular aneurysms are being observed with rapidly increasing incidence. Therefore, tools are needed for accurate and efficient detection of aneurysms. We used deep learning techniques with CT angiography acquired from multiple medical centers and different machines to develop and evaluate an automatic detection model.

METHODS

In this study, we have introduced a deep learning model, the faster RCNN model, in order to develop a tool for automatic detection of aneurysms from medical images. The inputs of the model were 2D nearby projection (NP) images from 3D CTA, which were made by the NP method proposed in this study. This method made aneurysms clearly visible on images and improved the model's performance. The study included 311 patients with 352 aneurysms, selected from three hospitals, and 208 and 103 of these patients, respectively, were randomly selected to train and test the models.

RESULTS

The sensitivity of the trained model was 91.8%. For aneurysm sizes larger than 3 mm, the sensitivity of successful aneurysm detection was 96.7%. We achieved state-of-the-art sensitivity for > 3 mm aneurysms. The sensitivities also indicated that there was no significant difference among aneurysms at different locations in the body. Computing time for the detection process was less than 25 s per case.

CONCLUSIONS

We successfully developed a deep learning model that can automatically detect aneurysms. The model performed well for aneurysms of different sizes or in different locations. This finding indicates that the deep learning model has the potential to vastly improve clinician performance by providing automated aneurysm detection.

Evolving the pulmonary nodules diagnosis from classical approaches to deep learning-aided decision support: three decades' development course and future prospect

PURPOSE

Lung cancer is the commonest cause of cancer deaths worldwide, and its mortality can be reduced significantly by performing early diagnosis and screening. Since the 1960s, driven by the pressing needs to accurately and effectively interpret the massive volume of chest images generated daily, computer-assisted diagnosis of pulmonary nodule has opened up new opportunities to relax the limitation from physicians' subjectivity, experiences and fatigue. And the fair access to the reliable and affordable computer-assisted diagnosis will fight the inequalities in incidence and mortality between populations. It has been witnessed that significant and remarkable advances have been achieved since the 1980s, and consistent endeavors have been exerted to deal with the grand challenges on how to accurately detect the pulmonary nodules with high sensitivity at low false-positive rate as well as on how to precisely differentiate between benign and malignant nodules. There is a lack of comprehensive examination of the techniques' development which is evolving the pulmonary nodules diagnosis from classical approaches to machine learning-assisted decision support. The main goal of this investigation is to provide a comprehensive state-of-the-art review of the computer-assisted nodules detection and benign-malignant classification techniques developed over three decades, which have evolved from the complicated ad hoc analysis pipeline of conventional approaches to the simplified seamlessly integrated deep learning techniques. This review also identifies challenges and highlights opportunities for future work in learning models, learning algorithms and enhancement schemes for bridging current state to future prospect and satisfying future demand.

CONCLUSION

It is the first literature review of the past 30 years' development in computer-assisted diagnosis of lung nodules. The challenges indentified and the research opportunities highlighted in this survey are significant for bridging current state to future prospect and satisfying future demand. The values of multifaceted driving forces and multidisciplinary researches are acknowledged that will make the computer-assisted diagnosis of pulmonary nodules enter into the main stream of clinical medicine and raise the state-of-the-art clinical applications as well as increase both welfares of physicians and patients. We firmly hold the vision that fair access to the reliable, faithful, and affordable computer-assisted diagnosis for early cancer diagnosis would fight the inequalities in incidence and mortality between populations, and save more lives.

A cascaded dual-pathway residual network for lung nodule segmentation in CT images

It is difficult to obtain an accurate segmentation due to the variety of lung nodules in computed tomography (CT) images. In this study, we propose a data-driven model, called the Cascaded Dual-Pathway Residual Network (CDP-ResNet) to improve the segmentation of lung nodules in the CT images. Our approach incorporates the multi-view and multi-scale features of different nodules from CT images. The proposed residual block based dual-path network extracts local features and rich contextual information of lung nodules. In addition, we designed an improved weighted sampling strategy to select training samples based on the edge. The proposed method was extensively evaluated on an LIDC dataset, which contains 986 nodules. Experimental results show that the CDP-ResNet achieves superior segmentation performance with an average DICE score (standard deviation) of 81.58% (11.05) on the LIDC dataset. Moreover, we compared our results with those of four radiologists on the same dataset. The comparison shows that the CDP-ResNet is slightly better than human experts in terms of segmentation accuracy. Meanwhile, the proposed segmentation method outperforms existing methods.

Machine learning approach for distinguishing malignant and benign lung nodules utilizing standardized perinodular parenchymal features from CT

PURPOSE

Computed tomography (CT) is an effective method for detecting and characterizing lung nodules in vivo. With the growing use of chest CT, the detection frequency of lung nodules is increasing. Noninvasive methods to distinguish malignant from benign nodules have the potential to decrease the clinical burden, risk, and cost involved in follow-up procedures on the large number of false-positive lesions detected. This study examined the benefit of including perinodular parenchymal features in machine learning (ML) tools for pulmonary nodule assessment.

METHODS

Lung nodule cases with pathology confirmed diagnosis (74 malignant, 289 benign) were used to extract quantitative imaging characteristics from computed tomography scans of the nodule and perinodular parenchyma tissue. A ML tool development pipeline was employed using k-medoids clustering and information theory to determine efficient predictor sets for different amounts of parenchyma inclusion and build an artificial neural network classifier. The resulting ML tool was validated using an independent cohort (50 malignant, 50 benign).

RESULTS

The inclusion of parenchymal imaging features improved the performance of the ML tool over exclusively nodular features (P < 0.01). The best performing ML tool included features derived from nodule diameter-based surrounding parenchyma tissue quartile bands. We demonstrate similar high-performance values on the independent validation cohort (AUC-ROC = 0.965). A comparison using the independent validation cohort with the Fleischner pulmonary nodule follow-up guidelines demonstrated a theoretical reduction in recommended follow-up imaging and procedures.

CONCLUSIONS

Radiomic features extracted from the parenchyma surrounding lung nodules contain valid signals with spatial relevance for the task of lung cancer risk classification. Through standardization of feature extraction regions from the parenchyma, ML tool validation performance of 100% sensitivity and 96% specificity was achieved.

Application of deep learning (3-dimensional convolutional neural network) for the prediction of pathological invasiveness in lung adenocarcinoma: A preliminary study

To compare results for radiological prediction of pathological invasiveness in lung adenocarcinoma between radiologists and a deep learning (DL) system.Ninety patients (50 men, 40 women; mean age, 66 years; range, 40-88 years) who underwent pre-operative chest computed tomography (CT) with 0.625-mm slice thickness were included in this retrospective study. Twenty-four cases of adenocarcinoma in situ (AIS), 20 cases of minimally invasive adenocarcinoma (MIA), and 46 cases of invasive adenocarcinoma (IVA) were pathologically diagnosed. Three radiologists of different levels of experience diagnosed each nodule by using previously documented CT findings to predict pathological invasiveness. DL was structured using a 3-dimensional (3D) convolutional neural network (3D-CNN) constructed with 2 successive pairs of convolution and max-pooling layers, and 2 fully connected layers. The output layer comprises 3 nodes to recognize the 3 conditions of adenocarcinoma (AIS, MIA, and IVA) or 2 nodes for 2 conditions (AIS and MIA/IVA). Results from DL and the 3 radiologists were statistically compared.No significant differences in pathological diagnostic accuracy rates were seen between DL and the 3 radiologists (P >.11). Receiver operating characteristic analysis demonstrated that area under the curve for DL (0.712) was almost the same as that for the radiologist with extensive experience (0.714; P = .98). Compared with the consensus results from radiologists, DL offered significantly inferior sensitivity (P = .0005), but significantly superior specificity (P = .02).Despite the small training data set, diagnostic performance of DL was almost the same as the radiologist with extensive experience. In particular, DL provided higher specificity than radiologists.

An Appraisal of Nodule Diagnosis for Lung Cancer in CT Images

As "the second eyes" of radiologists, computer-aided diagnosis systems play a significant role in nodule detection and diagnosis for lung cancer. In this paper, we aim to provide a systematic survey of state-of-the-art techniques (both traditional techniques and deep learning techniques) for nodule diagnosis from computed tomography images. This review first introduces the current progress and the popular structure used for nodule diagnosis. In particular, we provide a detailed overview of the five major stages in the computer-aided diagnosis systems: data acquisition, nodule segmentation, feature extraction, feature selection and nodule classification. Second, we provide a detailed report of the selected works and make a comprehensive comparison between selected works. The selected papers are from the IEEE Xplore, Science Direct, PubMed, and Web of Science databases up to December 2018. Third, we discuss and summarize the better techniques used in nodule diagnosis and indicate the existing future challenges in this field, such as improving the area under the receiver operating characteristic curve and accuracy, developing new deep learning-based diagnosis techniques, building efficient feature sets (fusing traditional features and deep features), developing high-quality labeled databases with malignant and benign nodules and promoting the cooperation between medical organizations and academic institutions.

Post-imaging pulmonary nodule mathematical prediction models: are they clinically relevant?

OBJECTIVES

Post-imaging mathematical prediction models (MPMs) provide guidance for the management of solid pulmonary nodules by providing a lung cancer risk score from demographic and radiologists-indicated imaging characteristics. We hypothesized calibrating the MPM risk score threshold to a local study cohort would result in improved performance over the original recommended MPM thresholds. We compared the pre- and post-calibration performance of four MPM models and determined if improvement in MPM prediction occurs as nodules are imaged longitudinally.

MATERIALS AND METHODS

A common cohort of 317 individuals with computed tomography-detected, solid nodules (80 malignant, 237 benign) were used to evaluate the MPM performance. We created a web-based application for this study that allows others to easily calibrate thresholds and analyze the performance of MPMs on their local cohort. Thirty patients with repeated imaging were tested for improved performance longitudinally.

RESULTS

Using calibrated thresholds, Mayo Clinic and Brock University (BU) MPMs performed the best (AUC = 0.63, 0.61) compared to the Veteran's Affairs (0.51) and Peking University (0.55). Only BU had consensus with the original MPM threshold; the other calibrated thresholds improved MPM accuracy. No significant improvements in accuracy were found longitudinally between time points.

CONCLUSIONS

Calibration to a common cohort can select the best-performing MPM for your institution. Without calibration, BU has the most stable performance in solid nodules ≥ 8 mm but has only moderate potential to refine subjects into appropriate workup. Application of MPM is recommended only at initial evaluation as no increase in accuracy was achieved over time.

KEY POINTS

• Post-imaging lung cancer risk mathematical predication models (MPMs) perform poorly on local populations without calibration. • An application is provided to facilitate calibration to new study cohorts: the Mayo Clinic model, the U.S. Department of Veteran's Affairs model, the Brock University model, and the Peking University model. • No significant improvement in risk prediction occurred in nodules with repeated imaging sessions, indicating the potential value of risk prediction application is limited to the initial evaluation.

Automatic Pulmonary Nodule Detection Applying Deep Learning or Machine Learning Algorithms to the LIDC-IDRI Database: A Systematic Review

The aim of this study was to provide an overview of the literature available on machine learning (ML) algorithms applied to the Lung Image Database Consortium Image Collection (LIDC-IDRI) database as a tool for the optimization of detecting lung nodules in thoracic CT scans. This systematic review was compiled according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Only original research articles concerning algorithms applied to the LIDC-IDRI database were included. The initial search yielded 1972 publications after removing duplicates, and 41 of these articles were included in this study. The articles were divided into two subcategories describing their overall architecture. The majority of feature-based algorithms achieved an accuracy >90% compared to the deep learning (DL) algorithms that achieved an accuracy in the range of 82.2%⁻97.6%. In conclusion, ML and DL algorithms are able to detect lung nodules with a high level of accuracy, sensitivity, and specificity using ML, when applied to an annotated archive of CT scans of the lung. However, there is no consensus on the method applied to determine the efficiency of ML algorithms.

Deep stacked sparse auto-encoders for prediction of post-operative survival expectancy in thoracic lung cancer surgery

Lung cancer is the leading cause of cancer death in men and women. The prognostic value of survival after lung cancer surgery has an important role in decision-making for surgeons and patients. The combination of clinical features and CT scan information for diagnosis, treatment and survival of patients with lung cancer increases the accuracy of prediction using machine learning. Therefore, creating a computer intelligent method with low error and high accuracy to predict survival is an important challenge, and it is beneficial for decreasing mortality from lung cancer, and for planning treatment. In this work, we implemented a deep stacked sparse auto-encoder (DSSAE) approach on a thoracic surgery data set for 470 patients, and our results contributing to deep learning based on 16 features were more precise than other suggested techniques for predicting post-operative survival expectancy in thoracic lung cancer surgery. The proposed method achieved a sensitivity of 94%, specificity of 82.86% and g-mean of 88.25%.

Convolutional Neural Networks for Radiologic Images: A Radiologist's Guide

Deep learning has rapidly advanced in various fields within the past few years and has recently gained particular attention in the radiology community. This article provides an introduction to deep learning technology and presents the stages that are entailed in the design process of deep learning radiology research. In addition, the article details the results of a survey of the application of deep learning-specifically, the application of convolutional neural networks-to radiologic imaging that was focused on the following five major system organs: chest, breast, brain, musculoskeletal system, and abdomen and pelvis. The survey of the studies is followed by a discussion about current challenges and future trends and their potential implications for radiology. This article may be used as a guide for radiologists planning research in the field of radiologic image analysis using convolutional neural networks.

Thyroid Diagnosis from SPECT Images Using Convolutional Neural Network with Optimization

Thyroid disease has now become the second largest disease in the endocrine field; SPECT imaging is particularly important for the clinical diagnosis of thyroid diseases. However, there is little research on the application of SPECT images in the computer-aided diagnosis of thyroid diseases based on machine learning methods. A convolutional neural network with optimization-based computer-aided diagnosis of thyroid diseases using SPECT images is developed. Three categories of diseases are considered, and they are Graves' disease, Hashimoto disease, and subacute thyroiditis. A modified DenseNet architecture of convolutional neural network is employed, and the training method is improved. The architecture is modified by adding the trainable weight parameters to each skip connection in DenseNet. And the training method is improved by optimizing the learning rate with flower pollination algorithm for network training. Experimental results demonstrate that the proposed method of convolutional neural network is efficient for the diagnosis of thyroid diseases with SPECT images, and it has superior performance than other CNN methods.

Deep CNN models for pulmonary nodule classification: Model modification, model integration, and transfer learning

BACKGROUND

Deep learning has made spectacular achievements in analysing natural images, but it faces challenges for medical applications partly due to inadequate images.

OBJECTIVE

Aiming to classify malignant and benign pulmonary nodules using CT images, we explore different strategies to utilize the state-of-the-art deep convolutional neural networks (CNN).

METHODS

Experiments are conducted using the Lung Image Database Consortium image collection (LIDC-IDRI), which is a public database containing 1018 cases. Three strategies are implemented including to 1) modify some state-of-the-art CNN architectures, 2) integrate different CNNs and 3) adopt transfer learning. Totally, 11 deep CNN models are compared using the same dataset.

RESULTS

Study demonstrates that, for the model modification scheme, a concise CifarNet performs better than the other modified CNNs with more complex architectures, achieving an area under ROC curve of AUC = 0.90. Integrated CNN models do not significantly improve the classification performance, but the model complexity is reduced. Transfer learning outperforms the other two schemes and ResNet with fine-tuning leads to the best performance with an AUC = 0.94, as well as the sensitivity of 91% and an overall accuracy of 88%.

CONCLUSIONS

Model modification, model integration, and transfer learning can play important roles to identify and generate optimal deep CNN models in classifying pulmonary nodules based on CT images efficiently. Transfer learning is preferred when applying deep learning to medical imaging applications.

Detection of epithelial growth factor receptor (EGFR) mutations on CT images of patients with lung adenocarcinoma using radiomics and/or multi-level residual convolutionary neural networks

Background

We aim to analyze the ability to detect epithelial growth factor receptor (EGFR) mutations on chest CT images of patients with lung adenocarcinoma using radiomics and/or multi-level residual convolutionary neural networks (MCNNs).

Methods

We retrospectively collected 1,010 consecutive patients in Shanghai Chest Hospital from 2013 to 2017, among which 510 patients were EGFR-mutated and 500 patients were wild-type. The patients were randomly divided into a training set (810 patients) and a validation set (200 patients) according to a balanced distribution of clinical features. The CT images and the corresponding EGFR status measured by Amplification Refractory Mutation System (ARMS) method of the patients in the training set were utilized to construct both a radiomics-based model (M_Radiomics) and MCNNs-based model (M_MCNNs). The M_Radiomics and M_MCNNs were combined to build the Model_{Radiomics+MCNNs} (M_{Radiomics+MCNNs}). Clinical data of gender and smoking history constructed the clinical features-based model (M_Clinical). M_Clinical was then added into M_Radiomics, M_MCNNs, and M_{Radiomics+MCNNs} to establish the Model_{Radiomics+Clinical} (M_{Radiomics+Clinical}), the Model_{MCNNs+Clinical} (M_{MCNNs+Clinical}) and the Model_{Radiomics+MCNNs+Clinical} (M_{Radiomics+MCNNs+Clinical}). All the seven models were tested in the validation set to ascertain whether they were competent to detect EGFR mutations. The detection efficiency of each model was also compared in terms of area under the curve (AUC), sensitivity and specificity.

Results

The AUC of the M_Radiomics, M_MCNNs and M_{Radiomics+MCNNs} to predict EGFR mutations was 0.740, 0.810 and 0.811 respectively. The performance of M_MCNNs was better than that of M_Radiomics (P=0.0225). The addition of clinical features did not improve the AUC of the M_Radiomics (P=0.623), the M_MCNNs (P=0.114) and the M_{Radiomics+MCNNs} (P=0.058). The M_{Radiomics+MCNNs+Clinical} demonstrated the highest AUC value of 0.834. The M_MCNNs did not demonstrate any inferiority when compared with the M_{Radiomics+MCNNs} (P=0.742) and the M_{Radiomics+MCNNs+Clinical} (P=0.056).

Conclusions

Both of the M_Radiomics and the M_CNNs could predict EGFR mutations on CT images of patients with lung adenocarcinoma. The M_MCNNs outperformed the M_Radiomics in the detection of EGFR mutations. The combination of these two models, even added with clinical features, is not significantly more efficient than M_MCNNs alone.

Unsupervised Learning for Concept Detection in Medical Images: A Comparative Analysis

As digital medical imaging becomes more prevalent and archives increase in size, representation learning exposes an interesting opportunity for enhanced medical decision support systems. On the other hand, medical imaging data is often scarce and short on annotations. In this paper, we present an assessment of unsupervised feature learning approaches for images in biomedical literature which can be applied to automatic biomedical concept detection. Six unsupervised representation learning methods were built, including traditional bags of visual words, autoencoders, and generative adversarial networks. Each model was trained, and their respective feature spaces evaluated using images from the ImageCLEF 2017 concept detection task. The highest mean F1 score of 0.108 was obtained using representations from an adversarial autoencoder, which increased to 0.111 when combined with the representations from the sparse denoising autoencoder. We conclude that it is possible to obtain more powerful representations with modern deep learning approaches than with previously popular computer vision methods. The possibility of semi-supervised learning as well as its use in medical information retrieval problems are the next steps to be strongly considered.

CNN models discriminating between pulmonary micro-nodules and non-nodules from CT images

Background

Early and automatic detection of pulmonary nodules from CT lung screening is the prerequisite for precise management of lung cancer. However, a large number of false positives appear in order to increase the sensitivity, especially for detecting micro-nodules (diameter < 3 mm), which increases the radiologists’ workload and causes unnecessary anxiety for the patients. To decrease the false positive rate, we propose to use CNN models to discriminate between pulmonary micro-nodules and non-nodules from CT image patches.

Methods

A total of 13,179 micro-nodules and 21,315 non-nodules marked by radiologists are extracted with three different patch sizes (16 × 16, 32 × 32 and 64 × 64) from LIDC/IDRI database and used in the experiments. Three CNN models with different depths (1, 2 or 4 convolutional layers) are designed; their performances are evaluated by the fivefold cross-validation in term of the accuracy, area under the curve (AUC), F-score and sensitivity. The network parameters are also optimized.

Results

It is found that the performance of the CNN models is greatly dependent on the patches size and the number of convolutional layers. The CNN model with two convolutional layers presented the best performance in case of 32 × 32 patches size, achieving an accuracy of 88.28%, an AUC of 0.87, a F-score of 83.45% and a sensitivity of 83.82%.

Conclusions

The CNN models with appropriate depth and size of image patches can effectively discriminate between pulmonary micro-nodules and non-nodules, and reduce the false positives and help manage lung cancer precisely.

Survey on deep learning for radiotherapy

More than 50% of cancer patients are treated with radiotherapy, either exclusively or in combination with other methods. The planning and delivery of radiotherapy treatment is a complex process, but can now be greatly facilitated by artificial intelligence technology. Deep learning is the fastest-growing field in artificial intelligence and has been successfully used in recent years in many domains, including medicine. In this article, we first explain the concept of deep learning, addressing it in the broader context of machine learning. The most common network architectures are presented, with a more specific focus on convolutional neural networks. We then present a review of the published works on deep learning methods that can be applied to radiotherapy, which are classified into seven categories related to the patient workflow, and can provide some insights of potential future applications. We have attempted to make this paper accessible to both radiotherapy and deep learning communities, and hope that it will inspire new collaborations between these two communities to develop dedicated radiotherapy applications.

Deep learning aided decision support for pulmonary nodules diagnosing: a review

Deep learning techniques have recently emerged as promising decision supporting approaches to automatically analyze medical images for different clinical diagnosing purposes. Diagnosing of pulmonary nodules by using computer-assisted diagnosing has received considerable theoretical, computational, and empirical research work, and considerable methods have been developed for detection and classification of pulmonary nodules on different formats of images including chest radiographs, computed tomography (CT), and positron emission tomography in the past five decades. The recent remarkable and significant progress in deep learning for pulmonary nodules achieved in both academia and the industry has demonstrated that deep learning techniques seem to be promising alternative decision support schemes to effectively tackle the central issues in pulmonary nodules diagnosing, including feature extraction, nodule detection, false-positive reduction, and benign-malignant classification for the huge volume of chest scan data. The main goal of this investigation is to provide a comprehensive state-of-the-art review of the deep learning aided decision support for pulmonary nodules diagnosing. As far as the authors know, this is the first time that a review is devoted exclusively to deep learning techniques for pulmonary nodules diagnosing.

A new near-term breast cancer risk prediction scheme based on the quantitative analysis of ipsilateral view mammograms

PURPOSE

To help improve efficacy of screening mammography and eventually establish an optimal personalized screening paradigm, this study aimed to develop and test a new near-term breast cancer risk prediction scheme based on the quantitative analysis of ipsilateral view of the negative screening mammograms.

METHODS

The dataset includes digital mammograms acquired from 392 women with two sequential full-field digital mammography examinations. All the first ("prior") sets of mammograms were interpreted as negative during the original reading. In the sequential ("current") screening, 202 were proved positive and 190 remained negative/benign. For each pair of the "prior" ipsilateral mammograms, we adaptively fused the image features computed from two views. Using four different types of image features, we built four elastic net support vector machine (EnSVM) based classifiers. Then, the initial prediction scores form the 4 EnSVMs were combined to build a final artificial neural network (ANN) classifier that produces the final risk prediction score. The performance of the new scheme was evaluated by using a 10-fold cross-validation method and an assessment index of the area under the receiver operating characteristic curve (AUC).

RESULTS

A total number of 466 features were initially extracted from each pair of ipsilateral mammograms. Among them, 51 were selected to build the EnSVM based prediction scheme. The AUC = 0.737 ± 0.052 was yielded using the new scheme. Applying an optimal operating threshold, the prediction sensitivity was 60.4% (122 of 202) and the specificity was 79.0% (150 of 190).

CONCLUSION

The study results showed moderately high positive association between computed risk scores using the "prior" negative mammograms and the actual outcome of the image-detectable breast cancers in the next subsequent screening examinations. The study also demonstrated that quantitative analysis of the ipsilateral views of the mammograms enabled to provide useful information in predicting near-term breast cancer risk.

Association of Pathological Fibrosis With Renal Survival Using Deep Neural Networks

INTRODUCTION

Chronic kidney damage is routinely assessed semiquantitatively by scoring the amount of fibrosis and tubular atrophy in a renal biopsy sample. Although image digitization and morphometric techniques can better quantify the extent of histologic damage, we need more widely applicable ways to stratify kidney disease severity.

METHODS

We leveraged a deep learning architecture to better associate patient-specific histologic images with clinical phenotypes (training classes) including chronic kidney disease (CKD) stage, serum creatinine, and nephrotic-range proteinuria at the time of biopsy, and 1-, 3-, and 5-year renal survival. Trichrome-stained images processed from renal biopsy samples were collected on 171 patients treated at the Boston Medical Center from 2009 to 2012. Six convolutional neural network (CNN) models were trained using these images as inputs and the training classes as outputs, respectively. For comparison, we also trained separate classifiers using the pathologist-estimated fibrosis score (PEFS) as input and the training classes as outputs, respectively.

RESULTS

CNN models outperformed PEFS across the classification tasks. Specifically, the CNN model predicted the CKD stage more accurately than the PEFS model (κ = 0.519 vs. 0.051). For creatinine models, the area under curve (AUC) was 0.912 (CNN) versus 0.840 (PEFS). For proteinuria models, AUC was 0.867 (CNN) versus 0.702 (PEFS). AUC values for the CNN models for 1-, 3-, and 5-year renal survival were 0.878, 0.875, and 0.904, respectively, whereas the AUC values for PEFS model were 0.811, 0.800, and 0.786, respectively.

CONCLUSION

The study demonstrates a proof of principle that deep learning can be applied to routine renal biopsy images.

Can radiological technologists serve as primary screeners of low-dose computed tomography for the diagnosis of lung cancer?

BACKGROUND

The Accreditation Council for Lung Cancer CT Screening of Japan established guidelines for the certification of Radiological Technologists in 2009.

OBJECTIVE

To analyze the trends in examination pass rates of the Radiological Technologists and discuss the reasons.

METHODS

The cohort comprised 1593 Radiological Technologists (as examinees) based on 10-year of data (with a total of 17 examination runs). First, the examinees' written test results were analyzed. Second, an abnormal finding detection test was conducted using >100 client PCs connected to a dedicated server containing low-dose lung cancer CT screening images of 60 cases. The passing scores were correct answer rate >60% and sensitivity (TP) of >90%, respectively.

RESULTS

Overall, 1243 examinees passed with an overall rate of 78%. The average pass rate for the written test was 91%, whereas that for the abnormal findings detection test was 85%. There was a moderate correlation between the test pass rate and average years of clinical experience of the examinees for the abnormal findings detection test (R = 0.558), whereas no such correlation existed for the written test (R = 0.105).

CONCLUSIONS

In order for accredited Radiological Technologists to serve as primary screeners of low-dose computed tomography, it is important to revise the educational system according to current standard practices.

Projection decomposition algorithm for dual-energy computed tomography via deep neural network

BACKGROUND

Dual-energy computed tomography (DECT) has been widely used to improve identification of substances from different spectral information. Decomposition of the mixed test samples into two materials relies on a well-calibrated material decomposition function.

OBJECTIVE

This work aims to establish and validate a data-driven algorithm for estimation of the decomposition function.

METHODS

A deep neural network (DNN) consisting of two sub-nets is proposed to solve the projection decomposition problem. The compressing sub-net, substantially a stack auto-encoder (SAE), learns a compact representation of energy spectrum. The decomposing sub-net with a two-layer structure fits the nonlinear transform between energy projection and basic material thickness.

RESULTS

The proposed DNN not only delivers image with lower standard deviation and higher quality in both simulated and real data, and also yields the best performance in cases mixed with photon noise. Moreover, DNN costs only 0.4 s to generate a decomposition solution of 360 × 512 size scale, which is about 200 times faster than the competing algorithms.

CONCLUSIONS

The DNN model is applicable to the decomposition tasks with different dual energies. Experimental results demonstrated the strong function fitting ability of DNN. Thus, the Deep learning paradigm provides a promising approach to solve the nonlinear problem in DECT.

Vasculature surrounding a nodule: A novel lung cancer biomarker

PURPOSE

To investigate whether the vessels surrounding a nodule depicted on non-contrast, low-dose computed tomography (LDCT) can discriminate benign and malignant screen detected nodules.

MATERIALS AND METHODS

We collected a dataset consisting of LDCT scans acquired on 100 subjects from the Pittsburgh Lung Screening study (PLuSS). Fifty subjects were diagnosed with lung cancer and 50 subjects had suspicious nodules later proven benign. For the lung cancer cases, the location of the malignant nodule in the LDCT scans was known; while for the benign cases, the largest nodule in the LDCT scan was used in the analysis. A computer algorithm was developed to identify surrounding vessels and quantify the number and volume of vessels that were connected or near the nodule. A nonparametric receiver operating characteristic (ROC) analysis was performed based on a single nodule per subject to assess the discriminability of the surrounding vessels to provide a lung cancer diagnosis. Odds ratio (OR) were computed to determine the probability of a nodule being lung cancer based on the vessel features.

RESULTS

The areas under the ROC curves (AUCs) for vessel count and vessel volume were 0.722 (95% CI=0.616-0.811, p<0.01) and 0.676 (95% CI=0.565-0.772), respectively. The number of vessels attached to a nodule was significantly higher in the lung cancer group 9.7 (±9.6) compared to the non-lung cancer group 4.0 (±4.3) CONCLUSION: Our preliminary results showed that malignant nodules are often surrounded by more vessels compared to benign nodules, suggesting that the surrounding vessel characteristics could serve as lung cancer biomarker for indeterminate nodules detected during LDCT lung cancer screening using only the information collected during the initial visit.