Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?

Versatile Framework for Medical Image Processing and Analysis with Application to Automatic Bone Age Assessment

Deep learning technique has made a tremendous impact on medical image processing and analysis. Typically, the procedure of medical image processing and analysis via deep learning technique includes image segmentation, image enhancement, and classification or regression. A challenge for supervised deep learning frequently mentioned is the lack of annotated training data. In this paper, we aim to address the problems of training transferred deep neural networks with limited amount of annotated data. We proposed a versatile framework for medical image processing and analysis via deep active learning technique. The framework includes (1) applying deep active learning approach to segment specific regions of interest (RoIs) from raw medical image by using annotated data as few as possible; (2) generative adversarial Network is employed to enhance contrast, sharpness, and brightness of segmented RoIs; (3) Paced Transfer Learning (PTL) strategy which means fine-tuning layers in deep neural networks from top to bottom step by step to perform medical image classification or regression tasks. In addition, in order to understand the necessity of deep-learning-based medical image processing tasks and provide clues for clinical usage, class active map (CAM) is employed in our framework to visualize the feature maps. To illustrate the effectiveness of the proposed framework, we apply our framework to the bone age assessment (BAA) task using RSNA dataset and achieve the state-of-the-art performance. Experimental results indicate that the proposed framework can be effectively applied to medical image analysis task.

Radiogenomics for Precision Medicine With a Big Data Analytics Perspective

Precision medicine promises better healthcare delivery by improving clinical practice. Using evidence-based substratification of patients, the objective is to achieve better prognosis, diagnosis, and treatment that will transform existing clinical pathways toward optimizing care for the specific needs of each patient. The wealth of today's healthcare data, often characterized as big data, provides invaluable resources toward new knowledge discovery that has the potential to advance precision medicine. The latter requires interdisciplinary efforts that will capitalize the information, know-how, and medical data of newly formed groups fusing different backgrounds and expertise. The objective of this paper is to provide insights with respect to the state-of-the-art research in precision medicine. More specifically, our goal is to highlight the fundamental challenges in emerging fields of radiomics and radiogenomics by reviewing the case studies of Cancer and Alzheimer's disease, describe the computational challenges from a big data analytics perspective, and discuss standardization and open data initiatives that will facilitate the adoption of precision medicine methods and practices.

Sparse coding of pathology slides compared to transfer learning with deep neural networks

Background

Histopathology images of tumor biopsies present unique challenges for applying machine learning to the diagnosis and treatment of cancer. The pathology slides are high resolution, often exceeding 1GB, have non-uniform dimensions, and often contain multiple tissue slices of varying sizes surrounded by large empty regions. The locations of abnormal or cancerous cells, which may constitute a small portion of any given tissue sample, are not annotated. Cancer image datasets are also extremely imbalanced, with most slides being associated with relatively common cancers. Since deep representations trained on natural photographs are unlikely to be optimal for classifying pathology slide images, which have different spectral ranges and spatial structure, we here describe an approach for learning features and inferring representations of cancer pathology slides based on sparse coding.

Results

We show that conventional transfer learning using a state-of-the-art deep learning architecture pre-trained on ImageNet (RESNET) and fine tuned for a binary tumor/no-tumor classification task achieved between 85% and 86% accuracy. However, when all layers up to the last convolutional layer in RESNET are replaced with a single feature map inferred via a sparse coding using a dictionary optimized for sparse reconstruction of unlabeled pathology slides, classification performance improves to over 93%, corresponding to a 54% error reduction.

Conclusions

We conclude that a feature dictionary optimized for biomedical imagery may in general support better classification performance than does conventional transfer learning using a dictionary pre-trained on natural images.

Electronic supplementary material

The online version of this article (10.1186/s12859-018-2504-8) contains supplementary material, which is available to authorized users.

iBiopsy® for Precision Medicine

A high-throughput artificial intelligence-powered image-based phenotyping platform, iBiopsy® (Median Technologies, Valbonne, France), which aims to improve precision medicine, is discussed in the presented review. The article introduces novel concepts, including high-throughput, fully automated imaging biomarker extraction; unsupervised predictive learning; large-scale content- based image-based similarity search; the use of large-scale clinical data registries; and cloud-based big data analytics to the problems of disease subtyping and treatment planning. Unlike electronic health record-based approaches, which lack the detailed radiological, pathological, genomic, and molecular data necessary for accurate prediction, iBiopsy generates unique signatures as fingerprints of disease and tumour subtypes from target images. These signatures are then merged with any additional omics data and matched against a large-scale reference registry of deeply phenotyped patients. Initial applications targeted include hepatocellular carcinoma and other chronic liver diseases, such as nonalcoholic steatohepatitis. This new disruptive technology is expected to lead to the identification of appropriate therapies targeting specific molecular pathways involved in the detected phenotypes to bring personalised treatment to patients, taking into account individual biological variability, which is the principal aim of precision medicine.

Face Recognition Using the SR-CNN Model

In order to solve the problem of face recognition in complex environments being vulnerable to illumination change, object rotation, occlusion, and so on, which leads to the imprecision of target position, a face recognition algorithm with multi-feature fusion is proposed. This study presents a new robust face-matching method named SR-CNN, combining the rotation-invariant texture feature (RITF) vector, the scale-invariant feature transform (SIFT) vector, and the convolution neural network (CNN). Furthermore, a graphics processing unit (GPU) is used to parallelize the model for an optimal computational performance. The Labeled Faces in the Wild (LFW) database and self-collection face database were selected for experiments. It turns out that the true positive rate is improved by 10.97–13.24% and the acceleration ratio (the ratio between central processing unit (CPU) operation time and GPU time) is 5–6 times for the LFW face database. For the self-collection, the true positive rate increased by 12.65–15.31%, and the acceleration ratio improved by a factor of 6–7.

A pilot study using kernelled support tensor machine for distant failure prediction in lung SBRT

We developed a kernelled support tensor machine (KSTM)-based model with tumor tensors derived from pre-treatment PET and CT imaging as input to predict distant failure in early stage non-small cell lung cancer (NSCLC) treated with stereotactic body radiation therapy (SBRT). The patient cohort included 110 early stage NSCLC patients treated with SBRT, 25 of whom experienced failure at distant sites. Three-dimensional tumor tensors were constructed and used as input for the KSTM-based classifier. A KSTM iterative algorithm with a convergent proof was developed to train the weight vectors for every mode of the tensor for the classifier. In contrast to conventional radiomics approaches that rely on handcrafted imaging features, the KSTM-based classifier uses 3D imaging as input, taking full advantage of the imaging information. The KSTM-based classifier preserves the intrinsic 3D geometry structure of the medical images and the correlation in the original images and trains the classification hyper-plane in an adaptive feature tensor space. The KSTM-based predictive algorithm was compared with three conventional machine learning models and three radiomics approaches. For PET and CT, the KSTM-based predictive method achieved the highest prediction results among the seven methods investigated in this study based on 10-fold cross validation and independent testing.

H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes

Liver cancer is one of the leading causes of cancer death. To assist doctors in hepatocellular carcinoma diagnosis and treatment planning, an accurate and automatic liver and tumor segmentation method is highly demanded in the clinical practice. Recently, fully convolutional neural networks (FCNs), including 2-D and 3-D FCNs, serve as the backbone in many volumetric image segmentation. However, 2-D convolutions cannot fully leverage the spatial information along the third dimension while 3-D convolutions suffer from high computational cost and GPU memory consumption. To address these issues, we propose a novel hybrid densely connected UNet (H-DenseUNet), which consists of a 2-D DenseUNet for efficiently extracting intra-slice features and a 3-D counterpart for hierarchically aggregating volumetric contexts under the spirit of the auto-context algorithm for liver and tumor segmentation. We formulate the learning process of the H-DenseUNet in an end-to-end manner, where the intra-slice representations and inter-slice features can be jointly optimized through a hybrid feature fusion layer. We extensively evaluated our method on the data set of the MICCAI 2017 Liver Tumor Segmentation Challenge and 3DIRCADb data set. Our method outperformed other state-of-the-arts on the segmentation results of tumors and achieved very competitive performance for liver segmentation even with a single model.

Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training

To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.

Detection of herpesvirus capsids in transmission electron microscopy images using transfer learning

The detailed analysis of secondary envelopment of the Human betaherpesvirus 5/human cytomegalovirus (HCMV) from transmission electron microscopy (TEM) images is an important step towards understanding the mechanisms underlying the formation of infectious virions. As a step towards a software-based quantification of different stages of HCMV virion morphogenesis in TEM, we developed a transfer learning approach based on convolutional neural networks (CNNs) that automatically detects HCMV nucleocapsids in TEM images. In contrast to existing image analysis techniques that require time-consuming manual definition of structural features, our method automatically learns discriminative features from raw images without the need for extensive pre-processing. For this a constantly growing TEM image database of HCMV infected cells was available which is unique regarding image quality and size in the terms of virological EM. From the two investigated types of transfer learning approaches, namely feature extraction and fine-tuning, the latter enabled us to successfully detect HCMV nucleocapsids in TEM images. Our detection method has outperformed some of the existing image analysis methods based on discriminative textural indicators and radial density profiles for virus detection in TEM images. In summary, we could show that the method of transfer learning can be used for an automated detection of viral capsids in TEM images with high specificity using standard computers. This method is highly adaptable and in future could be easily extended to automatically detect and classify virions of other viruses and even distinguish different virion maturation stages.

Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning

Background: Accurate glioma grading before surgery is of the utmost importance in treatment planning and prognosis prediction. But previous studies on magnetic resonance imaging (MRI) images were not effective enough. According to the remarkable performance of convolutional neural network (CNN) in medical domain, we hypothesized that a deep learning algorithm can achieve high accuracy in distinguishing the World Health Organization (WHO) low grade and high grade gliomas.

Methods: One hundred and thirteen glioma patients were retrospectively included. Tumor images were segmented with a rectangular region of interest (ROI), which contained about 80% of the tumor. Then, 20% data were randomly selected and leaved out at patient-level as test dataset. AlexNet and GoogLeNet were both trained from scratch and fine-tuned from models that pre-trained on the large scale natural image database, ImageNet, to magnetic resonance images. The classification task was evaluated with five-fold cross-validation (CV) on patient-level split.

Results: The performance measures, including validation accuracy, test accuracy and test area under curve (AUC), averaged from five-fold CV of GoogLeNet which trained from scratch were 0.867, 0.909, and 0.939, respectively. With transfer learning and fine-tuning, better performances were obtained for both AlexNet and GoogLeNet, especially for AlexNet. Meanwhile, GoogLeNet performed better than AlexNet no matter trained from scratch or learned from pre-trained model.

Conclusion: In conclusion, we demonstrated that the application of CNN, especially trained with transfer learning and fine-tuning, to preoperative glioma grading improves the performance, compared with either the performance of traditional machine learning method based on hand-crafted features, or even the CNNs trained from scratch.

Forensic age estimation for pelvic X-ray images using deep learning

PURPOSE

To develop a deep learning bone age assessment model based on pelvic radiographs for forensic age estimation and compare its performance to that of the existing cubic regression model.

MATERIALS AND METHOD

A retrospective collection data of 1875 clinical pelvic radiographs between 10 and 25 years of age was obtained to develop the model. Model performance was assessed by comparing the testing results to estimated ages calculated directly using the existing cubic regression model based on ossification staging methods. The mean absolute error (MAE) and root-mean-squared error (RMSE) between the estimated ages and chronological age were calculated for both models.

RESULTS

For all test samples (between 10 and 25 years old), the mean MAE and RMSE between the automatic estimates using the proposed deep learning model and the reference standard were 0.94 and 1.30 years, respectively. For the test samples comparable to those of the existing cubic regression model (between 14 and 22 years old), the mean MAE and RMSE for the deep learning model were 0.89 and 1.21 years, respectively. For the existing cubic regression model, the mean MAE and RMSE were 1.05 and 1.61 years, respectively.

CONCLUSION

The deep learning convolutional neural network model achieves performance on par with the existing cubic regression model, demonstrating predictive ability capable of automated skeletal bone assessment based on pelvic radiographic images.

KEY POINTS

• The pelvis has considerable value in determining the bone age. • Deep learning can be used to create an automated bone age assessment model based on pelvic radiographs. • The deep learning convolutional neural network model achieves performance on par with the existing cubic regression model.

Preliminary Study of Chronic Liver Classification on Ultrasound Images Using an Ensemble Model

Chronic liver diseases are fifth leading cause of fatality in developing countries. Their early diagnosis is extremely important for timely treatment and salvage life. To examine abnormalities of liver, ultrasound imaging is the most frequently used modality. However, the visual differentiation between chronic liver and cirrhosis, and presence of heptocellular carcinomas (HCC) evolved over cirrhotic liver is difficult, as they appear almost similar in ultrasound images. In this paper, to deal with this difficult visualization problem, a method has been developed for classifying four liver stages, that is, normal, chronic, cirrhosis, and HCC evolved over cirrhosis. The method is formulated with selected set of "handcrafted" texture features obtained after hierarchal feature fusion. These multiresolution and higher order features, which are able to characterize echotexture and roughness of liver surface, are extracted by using ranklet, gray-level difference matrix and gray-level co-occurrence matrix methods. Thereafter, these features are applied on proposed ensemble classifier that is designed with voting algorithm in conjunction with three classifiers, namely, k-nearest neighbor (k-NN), support vector machine (SVM), and rotation forest. The experiments are conducted to evaluate the (a) effectiveness of "handcrafted" texture features, (b) performance of proposed ensemble model, (c) effectiveness of proposed ensemble strategy, (d) performance of different classifiers, and (e) performance of proposed ensemble model based on Convolutional Neural Networks (CNN) features to differentiate four liver stages. These experiments are carried out on database of 754 segmented regions of interest formed by clinically acquired ultrasound images. The results show that classification accuracy of 96.6% is obtained by use of proposed classifier model.

Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker

A computer-aided detection (CAD) tool for locating and detecting polyps can help reduce the chance of missing polyps during colonoscopy. Nevertheless, state-of-the-art algorithms were either computationally complex or suffered from low sensitivity and therefore unsuitable to be used in real clinical setting. In this paper, a novel regression-based Convolutional Neural Network (CNN) pipeline is presented for polyp detection during colonoscopy. The proposed pipeline was constructed in two parts: 1) to learn the spatial features of colorectal polyps, a fast object detection algorithm named ResYOLO was pre-trained with a large non-medical image database and further fine-tuned with colonoscopic images extracted from videos; and 2) temporal information was incorporated via a tracker named Efficient Convolution Operators (ECO) for refining the detection results given by ResYOLO. Evaluated on 17,574 frames extracted from 18 endoscopic videos of the AsuMayoDB, the proposed method was able to detect frames with polyps with a precision of 88.6%, recall of 71.6% and processing speed of 6.5 frames per second, i.e. the method can accurately locate polyps in more frames and at a faster speed compared to existing methods. In conclusion, the proposed method has great potential to be used to assist endoscopists in tracking polyps during colonoscopy.

Discrimination and conversion prediction of mild cognitive impairment using convolutional neural networks

Background

Recently, studies have demonstrated that machine learning techniques, particularly cutting-edge deep learning technology, have achieved significant progression on the classification of Alzheimer's disease (AD) and its prodromal phase, mild cognitive impairment (MCI). Moreover, accurate prediction of the progress and the conversion risk from MCI to probable AD has been of great importance in clinical application.

Methods

In this study, the baseline MR images and follow-up information during 3 years of 150 normal controls (NC), 150 patients with stable MCI (sMCI) and 157 converted MCI (cMCI) were collected from the Alzheimer's Disease Neuroimaging Initiative (ADNI). The deep convolutional neural networks (CNNs) were adopted to distinguish different stages of MCI from the NC group, and predict the conversion time from MCI to AD. Two CNN architectures including GoogleNet and CaffeNet were explored and evaluated in multiple classifications and estimations of conversion risk using transfer learning from pre-trained ImageNet (via fine-tuning) and five-fold cross-validation. A novel data augmentation approach using random views aggregation was applied to generate abundant image patches from the original MR scans.

Results

The GoogleNet acquired accuracies with 97.58%, 67.33% and 84.71% in three-way discrimination among the NC, sMCI and cMCI groups respectively, whereas the CaffeNet obtained promising accuracies of 98.71%, 72.04% and 92.35% in the NC, sMCI and cMCI classifications. Furthermore, the accuracy measures of conversion risk of patients with cMCI ranged from 71.25% to 83.25% in different time points using GoogleNet, whereas the CaffeNet achieved remarkable accuracy measures from 95.42% to 97.01% in conversion risk prediction.

Conclusions

The experimental results demonstrated that the proposed methods had prominent capability in classification among the 3 groups such as sMCI, cMCI and NC, and exhibited significant ability in conversion risk prediction of patients with MCI.

Automatic classification of dual-modalilty, smartphone-based oral dysplasia and malignancy images using deep learning

With the goal to screen high-risk populations for oral cancer in low- and middle-income countries (LMICs), we have developed a low-cost, portable, easy to use smartphone-based intraoral dual-modality imaging platform. In this paper we present an image classification approach based on autofluorescence and white light images using deep learning methods. The information from the autofluorescence and white light image pair is extracted, calculated, and fused to feed the deep learning neural networks. We have investigated and compared the performance of different convolutional neural networks, transfer learning, and several regularization techniques for oral cancer classification. Our experimental results demonstrate the effectiveness of deep learning methods in classifying dual-modal images for oral cancer detection.

Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging

PURPOSE

We propose an approach of 3D convolutional neural network to segment the prostate in MR images.

METHODS

A 3D deep dense multi-path convolutional neural network that follows the framework of the encoder-decoder design is proposed. The encoder is built based upon densely connected layers that learn the high-level feature representation of the prostate. The decoder interprets the features and predicts the whole prostate volume by utilizing a residual layout and grouped convolution. A set of sub-volumes of MR images, centered at the prostate, is generated and fed into the proposed network for training purpose. The performance of the proposed network is compared to previously reported approaches.

RESULTS

Two independent datasets were employed to assess the proposed network. In quantitative evaluations, the proposed network achieved 95.11 and 89.01 Dice coefficients for the two datasets. The segmentation results were robust to variations in MR images. In comparison experiments, the segmentation performance of the proposed network was comparable to the previously reported approaches. In qualitative evaluations, the segmentation results by the proposed network were well matched to the ground truth provided by human experts.

CONCLUSIONS

The proposed network is capable of segmenting the prostate in an accurate and robust manner. This approach can be applied to other types of medical images.

Classification of contrast-enhanced spectral mammography (CESM) images

PURPOSE

Contrast-enhanced spectral mammography (CESM) is a recently developed breast imaging technique. CESM relies on dual-energy acquisition following contrast agent injection to improve mammography sensitivity. CESM is comparable to contrast-enhanced MRI in terms of sensitivity, at a fraction of the cost. However, since lesion variability is large, even with the improved visibility provided by CESM, differentiation between benign and malignant enhancement is not accurate and a biopsy is usually performed for final assessment. Breast biopsies can be stressful to the patient and are expensive to healthcare systems. Moreover, as the biopsies results are most of the time benign, a specificity improvement in the radiologist diagnosis is required. This work presents a deep learning-based decision support system, which aims at improving the specificity of breast cancer diagnosis by CESM without affecting sensitivity.

METHODS

We compare two analysis approaches, fine-tuning a pretrained network and fully training a convolutional neural network, for classification of CESM breast mass as benign or malignant. Breast Imaging Reporting and Data Systems (BIRADS) is a radiological lexicon, used with breast images, to categorize lesions. We improve each classification network by incorporating BIRADS textual features as an additional input to the network. We evaluate two ways of BIRADS fusion as network input: feature fusion and decision fusion. This leads to multimodal network architectures. At classification, we also exploit information from apparently normal breast tissue in the CESM of the considered patient, leading to a patient-specific classification.

RESULTS

We evaluate performance using fivefold cross-validation, on 129 randomly selected breast lesions annotated by an experienced radiologist. Each annotation includes a contour of the mass in the image, biopsy-proven label of benign or malignant lesion and BIRADS descriptors. At 100% sensitivity, specificity of 66% was achieved using a multimodal network, which combines inputs at feature level and patient-specific classification.

CONCLUSIONS

The presented multimodal network may significantly reduce benign biopsies, without compromising sensitivity.

Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks

Remote sensing images have long been preferred to perform building damage assessments. The recently proposed methods to extract damaged regions from remote sensing imagery rely on convolutional neural networks (CNN). The common approach is to train a CNN independently considering each of the different resolution levels (satellite, aerial, and terrestrial) in a binary classification approach. In this regard, an ever-growing amount of multi-resolution imagery are being collected, but the current approaches use one single resolution as their input. The use of up/down-sampled images for training has been reported as beneficial for the image classification accuracy both in the computer vision and remote sensing domains. However, it is still unclear if such multi-resolution information can also be captured from images with different spatial resolutions such as imagery of the satellite and airborne (from both manned and unmanned platforms) resolutions. In this paper, three multi-resolution CNN feature fusion approaches are proposed and tested against two baseline (mono-resolution) methods to perform the image classification of building damages. Overall, the results show better accuracy and localization capabilities when fusing multi-resolution feature maps, specifically when these feature maps are merged and consider feature information from the intermediate layers of each of the resolution level networks. Nonetheless, these multi-resolution feature fusion approaches behaved differently considering each level of resolution. In the satellite and aerial (unmanned) cases, the improvements in the accuracy reached 2% while the accuracy improvements for the airborne (manned) case was marginal. The results were further confirmed by testing the approach for geographical transferability, in which the improvements between the baseline and multi-resolution experiments were overall maintained.

Retinal image quality assessment using deep learning

Poor-quality retinal images do not allow an accurate medical diagnosis, and it is inconvenient for a patient to return to a medical center to repeat the fundus photography exam. In this paper, a robust automatic system is proposed to assess the quality of retinal images at the moment of the acquisition, aiming at assisting health care professionals during a fundus photography exam. We propose a convolutional neural network (CNN) pretrained on non-medical images for extracting general image features. The weights of the CNN are further adjusted via a fine-tuning procedure, resulting in a performant classifier obtained only with a small quantity of labeled images. The CNN performance was evaluated on two publicly available databases (i.e., DRIMDB and ELSA-Brasil) using two different procedures: intra-database and inter-database cross-validation. The CNN achieved an area under the curve (AUC) of 99.98% on DRIMDB and an AUC of 98.56% on ELSA-Brasil in the inter-database experiment, where training and testing were not performed on the same database. These results show the robustness of the proposed model to various image acquisitions without requiring special adaptation, thus making it a good candidate for use in operational clinical scenarios.

Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification

This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.

Deep Learning in Microscopy Image Analysis: A Survey

Computerized microscopy image analysis plays an important role in computer aided diagnosis and prognosis. Machine learning techniques have powered many aspects of medical investigation and clinical practice. Recently, deep learning is emerging as a leading machine learning tool in computer vision and has attracted considerable attention in biomedical image analysis. In this paper, we provide a snapshot of this fast-growing field, specifically for microscopy image analysis. We briefly introduce the popular deep neural networks and summarize current deep learning achievements in various tasks, such as detection, segmentation, and classification in microscopy image analysis. In particular, we explain the architectures and the principles of convolutional neural networks, fully convolutional networks, recurrent neural networks, stacked autoencoders, and deep belief networks, and interpret their formulations or modelings for specific tasks on various microscopy images. In addition, we discuss the open challenges and the potential trends of future research in microscopy image analysis using deep learning.

Deep Learning Approach for Evaluating Knee MR Images: Achieving High Diagnostic Performance for Cartilage Lesion Detection

Purpose To determine the feasibility of using a deep learning approach to detect cartilage lesions (including cartilage softening, fibrillation, fissuring, focal defects, diffuse thinning due to cartilage degeneration, and acute cartilage injury) within the knee joint on MR images. Materials and Methods A fully automated deep learning-based cartilage lesion detection system was developed by using segmentation and classification convolutional neural networks (CNNs). Fat-suppressed T2-weighted fast spin-echo MRI data sets of the knee of 175 patients with knee pain were retrospectively analyzed by using the deep learning method. The reference standard for training the CNN classification was the interpretation provided by a fellowship-trained musculoskeletal radiologist of the presence or absence of a cartilage lesion within 17 395 small image patches placed on the articular surfaces of the femur and tibia. Receiver operating curve (ROC) analysis and the κ statistic were used to assess diagnostic performance and intraobserver agreement for detecting cartilage lesions for two individual evaluations performed by the cartilage lesion detection system. Results The sensitivity and specificity of the cartilage lesion detection system at the optimal threshold according to the Youden index were 84.1% and 85.2%, respectively, for evaluation 1 and 80.5% and 87.9%, respectively, for evaluation 2. Areas under the ROC curve were 0.917 and 0.914 for evaluations 1 and 2, respectively, indicating high overall diagnostic accuracy for detecting cartilage lesions. There was good intraobserver agreement between the two individual evaluations, with a κ of 0.76. Conclusion This study demonstrated the feasibility of using a fully automated deep learning-based cartilage lesion detection system to evaluate the articular cartilage of the knee joint with high diagnostic performance and good intraobserver agreement for detecting cartilage degeneration and acute cartilage injury. © RSNA, 2018 Online supplemental material is available for this article .

SD-CNN: A shallow-deep CNN for improved breast cancer diagnosis

Breast cancer is the second leading cause of cancer death among women worldwide. Nevertheless, it is also one of the most treatable malignances if detected early. Screening for breast cancer with full field digital mammography (FFDM) has been widely used. However, it demonstrates limited performance for women with dense breasts. An emerging technology in the field is contrast-enhanced digital mammography (CEDM), which includes a low energy (LE) image similar to FFDM, and a recombined image leveraging tumor neoangiogenesis similar to breast magnetic resonance imaging (MRI). CEDM has shown better diagnostic accuracy than FFDM. While promising, CEDM is not yet widely available across medical centers. In this research, we propose a Shallow-Deep Convolutional Neural Network (SD-CNN) where a shallow CNN is developed to derive "virtual" recombined images from LE images, and a deep CNN is employed to extract novel features from LE, recombined or "virtual" recombined images for ensemble models to classify the cases as benign vs. cancer. To evaluate the validity of our approach, we first develop a deep-CNN using 49 CEDM cases collected from Mayo Clinic to prove the contributions from recombined images for improved breast cancer diagnosis (0.85 in accuracy, 0.84 in AUC using LE imaging vs. 0.89 in accuracy, 0.91 in AUC using both LE and recombined imaging). We then develop a shallow-CNN using the same 49 CEDM cases to learn the nonlinear mapping from LE to recombined images. Next, we use 89 FFDM cases from INbreast, a public database to generate "virtual" recombined images. Using FFDM alone provides 0.84 in accuracy (AUC = 0.87), whereas SD-CNN improves the diagnostic accuracy to 0.90 (AUC = 0.92).

Artificial intelligence using neural network architecture for radiology (AINNAR): classification of MR imaging sequences

PURPOSE

The confusion of MRI sequence names could be solved if MR images were automatically identified after image data acquisition. We revealed the ability of deep learning to classify head MRI sequences.

MATERIALS AND METHODS

Seventy-eight patients with mild cognitive impairment (MCI) having apparently normal head MR images and 78 intracranial hemorrhage (ICH) patients with morphologically deformed head MR images were enrolled. Six imaging protocols were selected to be performed: T2-weighted imaging, fluid attenuated inversion recovery imaging, T2-star-weighted imaging, diffusion-weighted imaging, apparent diffusion coefficient mapping, and source images of time-of-flight magnetic resonance angiography. The proximal first image slices and middle image slices having ambiguous and distinctive contrast patterns, respectively, were classified by two deep learning imaging classifiers, AlexNet and GoogLeNet.

RESULTS

AlexNet had accuracies of 73.3%, 73.6%, 73.1%, and 60.7% in the middle slices of MCI group, middle slices of ICH group, first slices of MCI group, and first slices of ICH group, while GoogLeNet had accuracies of 100%, 98.1%, 93.1%, and 94.8%, respectively. AlexNet significantly had lower classification ability than GoogLeNet for all datasets.

CONCLUSIONS

GoogLeNet could judge the types of head MRI sequences with a small amount of training data, irrespective of morphological or contrast conditions.

Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images

Deep learning has emerged as a powerful artificial intelligence tool to interpret medical images for a growing variety of applications. However, the paucity of medical imaging data with high-quality annotations that is necessary for training such methods ultimately limits their performance. Medical data is challenging to acquire due to privacy issues, shortage of experts available for annotation, limited representation of rare conditions and cost. This problem has previously been addressed by using synthetically generated data. However, networks trained on synthetic data often fail to generalize to real data. Cinematic rendering simulates the propagation and interaction of light passing through tissue models reconstructed from CT data, enabling the generation of photorealistic images. In this paper, we present one of the first applications of cinematic rendering in deep learning, in which we propose to fine-tune synthetic data-driven networks using cinematically rendered CT data for the task of monocular depth estimation in endoscopy. Our experiments demonstrate that: (a) convolutional neural networks (CNNs) trained on synthetic data and fine-tuned on photorealistic cinematically rendered data adapt better to real medical images and demonstrate more robust performance when compared to networks with no fine-tuning, (b) these fine-tuned networks require less training data to converge to an optimal solution, and (c) fine-tuning with data from a variety of photorealistic rendering conditions of the same scene prevents the network from learning patient-specific information and aids in generalizability of the model. Our empirical evaluation demonstrates that networks fine-tuned with cinematically rendered data predict depth with 56.87% less error for rendered endoscopy images and 27.49% less error for real porcine colon endoscopy images.

Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype

RNA-based, multi-gene molecular assays are available and widely used for patients with ER-positive/HER2-negative breast cancers. However, RNA-based genomic tests can be costly and are not available in many countries. Methods for inferring molecular subtype from histologic images may identify patients most likely to benefit from further genomic testing. To identify patients who could benefit from molecular testing based on H&E stained histologic images, we developed an image analysis approach using deep learning. A training set of 571 breast tumors was used to create image-based classifiers for tumor grade, ER status, PAM50 intrinsic subtype, histologic subtype, and risk of recurrence score (ROR-PT). The resulting classifiers were applied to an independent test set (n = 288), and accuracy, sensitivity, and specificity of each was assessed on the test set. Histologic image analysis with deep learning distinguished low-intermediate vs. high tumor grade (82% accuracy), ER status (84% accuracy), Basal-like vs. non-Basal-like (77% accuracy), Ductal vs. Lobular (94% accuracy), and high vs. low-medium ROR-PT score (75% accuracy). Sampling considerations in the training set minimized bias in the test set. Incorrect classification of ER status was significantly more common for Luminal B tumors. These data provide proof of principle that molecular marker status, including a critical clinical biomarker (i.e., ER status), can be predicted with accuracy >75% based on H&E features. Image-based methods could be promising for identifying patients with a greater need for further genomic testing, or in place of classically scored variables typically accomplished using human-based scoring.

UNet++: A Nested U-Net Architecture for Medical Image Segmentation

In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network where the encoder and decoder sub-networks are connected through a series of nested, dense skip pathways. The re-designed skip pathways aim at reducing the semantic gap between the feature maps of the encoder and decoder sub-networks. We argue that the optimizer would deal with an easier learning task when the feature maps from the decoder and encoder networks are semantically similar. We have evaluated UNet++ in comparison with U-Net and wide U-Net architectures across multiple medical image segmentation tasks: nodule segmentation in the low-dose CT scans of chest, nuclei segmentation in the microscopy images, liver segmentation in abdominal CT scans, and polyp segmentation in colonoscopy videos. Our experiments demonstrate that UNet++ with deep supervision achieves an average IoU gain of 3.9 and 3.4 points over U-Net and wide U-Net, respectively.

Automatic polyp frame screening using patch based combined feature and dictionary learning

Polyps in the colon can potentially become malignant cancer tissues where early detection and removal lead to high survival rate. Certain types of polyps can be difficult to detect even for highly trained physicians. Inspired by aforementioned problem our study aims to improve the human detection performance by developing an automatic polyp screening framework as a decision support tool. We use a small image patch based combined feature method. Features include shape and color information and are extracted using histogram of oriented gradient and hue histogram methods. Dictionary learning based training is used to learn features and final feature vector is formed using sparse coding. For classification, we use patch image classification based on linear support vector machine and whole image thresholding. The proposed framework is evaluated using three public polyp databases. Our experimental results show that the proposed scheme successfully classified polyps and normal images with over 95% of classification accuracy, sensitivity, specificity and precision. In addition, we compare performance of the proposed scheme with conventional feature based methods and the convolutional neural network (CNN) based deep learning approach which is the state of the art technique in many image classification applications.

Optimize Transfer Learning for Lung Diseases in Bronchoscopy Using a New Concept: Sequential Fine-Tuning

Bronchoscopy inspection, as a follow-up procedure next to the radiological imaging, plays a key role in the diagnosis and treatment design for lung disease patients. When performing bronchoscopy, doctors have to make a decision immediately whether to perform a biopsy. Because biopsies may cause uncontrollable and life-threatening bleeding of the lung tissue, thus doctors need to be selective with biopsies. In this paper, to help doctors to be more selective on biopsies and provide a second opinion on diagnosis, we propose a computer-aided diagnosis (CAD) system for lung diseases, including cancers and tuberculosis (TB). Based on transfer learning (TL), we propose a novel TL method on the top of DenseNet: sequential fine-tuning (SFT). Compared with traditional fine-tuning (FT) methods, our method achieves the best performance. In a data set of recruited 81 normal cases, 76 TB cases and 277 lung cancer cases, SFT provided an overall accuracy of 82% while other traditional TL methods achieved an accuracy from 70% to 74%. The detection accuracy of SFT for cancers, TB, and normal cases are 87%, 54%, and 91%, respectively. This indicates that the CAD system has the potential to improve lung disease diagnosis accuracy in bronchoscopy and it may be used to be more selective with biopsies.

Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images

PURPOSE

The nonalcoholic fatty liver disease is the most common liver abnormality. Up to date, liver biopsy is the reference standard for direct liver steatosis quantification in hepatic tissue samples. In this paper we propose a neural network-based approach for nonalcoholic fatty liver disease assessment in ultrasound.

METHODS

We used the Inception-ResNet-v2 deep convolutional neural network pre-trained on the ImageNet dataset to extract high-level features in liver B-mode ultrasound image sequences. The steatosis level of each liver was graded by wedge biopsy. The proposed approach was compared with the hepatorenal index technique and the gray-level co-occurrence matrix algorithm. After the feature extraction, we applied the support vector machine algorithm to classify images containing fatty liver. Based on liver biopsy, the fatty liver was defined to have more than 5% of hepatocytes with steatosis. Next, we used the features and the Lasso regression method to assess the steatosis level.

RESULTS

The area under the receiver operating characteristics curve obtained using the proposed approach was equal to 0.977, being higher than the one obtained with the hepatorenal index method, 0.959, and much higher than in the case of the gray-level co-occurrence matrix algorithm, 0.893. For regression the Spearman correlation coefficients between the steatosis level and the proposed approach, the hepatorenal index and the gray-level co-occurrence matrix algorithm were equal to 0.78, 0.80 and 0.39, respectively.

CONCLUSIONS

The proposed approach may help the sonographers automatically diagnose the amount of fat in the liver. The presented approach is efficient and in comparison with other methods does not require the sonographers to select the region of interest.

Exploring transfer learning for gastrointestinal bleeding detection on small-size imbalanced endoscopy images

The success of Convolutional Neural Network (CNN) is attributed to their ability to learn rich midlevel image representations as opposed to hand-crafted low-level features used in many natural image classification methods. Learning CNN, however, amounts to estimating millions of parameters and requires a very large number of annotated image samples. In this paper, we explored transfer learning for gastrointestinal bleeding detection on small-size imbalanced endoscopy images, and showed how image representations learned with CNN on large-scale annotated datasets can be efficiently transferred to other tasks with limited amount of training data. We first transferred pre-trained Inception V3 model trained on the ImageNet dataset to compute mid-level image representation, and then fine-tuned the trained model with labeled endoscopy images, and resumed training from already learned weights. Additionally, we introduce both data augmentation and image resampling to increase the size of the training database and the positive sample rate to perform the Transfer Learning. Our results showed that our transfer learning method produces the best performance on AUC (the area under the receiver operating curve), Precision, Recall and Accuracy as compared to both the hand-crafted feature based method and training CNN model from-scratch method.

Predicting the progression of ophthalmic disease based on slit-lamp images using a deep temporal sequence network

Ocular images play an essential role in ophthalmology. Current research mainly focuses on computer-aided diagnosis using slit-lamp images, however few studies have been done to predict the progression of ophthalmic disease. Therefore exploring an effective approach of prediction can help to plan treatment strategies and to provide early warning for the patients. In this study, we present an end-to-end temporal sequence network (TempSeq-Net) to automatically predict the progression of ophthalmic disease, which includes employing convolutional neural network (CNN) to extract high-level features from consecutive slit-lamp images and applying long short term memory (LSTM) method to mine the temporal relationship of features. First, we comprehensively compare six potential combinations of CNNs and LSTM (or recurrent neural network) in terms of effectiveness and efficiency, to obtain the optimal TempSeq-Net model. Second, we analyze the impacts of sequence lengths on model's performance which help to evaluate their stability and validity and to determine the appropriate range of sequence lengths. The quantitative results demonstrated that our proposed model offers exceptional performance with mean accuracy (92.22), sensitivity (88.55), specificity (94.31) and AUC (97.18). Moreover, the model achieves real-time prediction with only 27.6ms for single sequence, and simultaneously predicts sequence data with lengths of 3-5. Our study provides a promising strategy for the progression of ophthalmic disease, and has the potential to be applied in other medical fields.

Multimodal 3D DenseNet for IDH Genotype Prediction in Gliomas

Non-invasive prediction of isocitrate dehydrogenase (IDH) genotype plays an important role in tumor glioma diagnosis and prognosis. Recently, research has shown that radiology images can be a potential tool for genotype prediction, and fusion of multi-modality data by deep learning methods can further provide complementary information to enhance prediction accuracy. However, it still does not have an effective deep learning architecture to predict IDH genotype with three-dimensional (3D) multimodal medical images. In this paper, we proposed a novel multimodal 3D DenseNet (M3D-DenseNet) model to predict IDH genotypes with multimodal magnetic resonance imaging (MRI) data. To evaluate its performance, we conducted experiments on the BRATS-2017 and The Cancer Genome Atlas breast invasive carcinoma (TCGA-BRCA) dataset to get image data as input and gene mutation information as the target, respectively. We achieved 84.6% accuracy (area under the curve (AUC) = 85.7%) on the validation dataset. To evaluate its generalizability, we applied transfer learning techniques to predict World Health Organization (WHO) grade status, which also achieved a high accuracy of 91.4% (AUC = 94.8%) on validation dataset. With the properties of automatic feature extraction, and effective and high generalizability, M3D-DenseNet can serve as a useful method for other multimodal radiogenomics problems and has the potential to be applied in clinical decision making.

Automatic Segmentation and Counting of Aphid Nymphs on Leaves Using Convolutional Neural Networks

The presence of pests is one of the main problems in crop production, and obtaining reliable statistics of pest infestation is essential for pest management. Detection of pests should be automated because human monitoring of pests is time-consuming and error-prone. Aphids are among the most destructive pests in greenhouses and they reproduce quickly. Automatic detection of aphid nymphs on leaves (especially on the lower surface) using image analysis is a challenging problem due to color similarity and complicated background. In this study, we propose a method for segmentation and counting of aphid nymphs on leaves using convolutional neural networks. Digital images of pakchoi leaves at different aphid infestation stages were obtained, and corresponding pixel-level binary mask annotated. In the test, segmentation results by the proposed method achieved high overlap with annotation by human experts (Dice coefficient of 0.8207). Automatic counting based on segmentation showed high precision (0.9563) and recall (0.9650). The correlation between aphid nymph count by the proposed method and manual counting was high (R2 = 0.99). The proposed method is generic and can be applied for other species of pests.

Meeting Report: Tissue-based Image Analysis

Quantitative image analysis (IA) is a rapidly evolving area of digital pathology. Although not a new concept, the quantification of histological features on photomicrographs used to be cumbersome, resource-intensive, and limited to specialists and specialized laboratories. Recent technological advances like highly efficient automated whole slide digitizer (scanner) systems, innovative IA platforms, and the emergence of pathologist-friendly image annotation and analysis systems mean that quantification of features on histological digital images will become increasingly prominent in pathologists' daily professional lives. The added value of quantitative IA in pathology includes confirmation of equivocal findings noted by a pathologist, increasing the sensitivity of feature detection, quantification of signal intensity, and improving efficiency. There is no denying that quantitative IA is part of the future of pathology; however, there are also several potential pitfalls when trying to estimate volumetric features from limited 2-dimensional sections. This continuing education session on quantitative IA offered a broad overview of the field; a hands-on toxicologic pathologist experience with IA principles, tools, and workflows; a discussion on how to apply basic stereology principles in order to minimize bias in IA; and finally, a reflection on the future of IA in the toxicologic pathology field.

Classification of Computed Tomography Images in Different Slice Positions Using Deep Learning

This study aimed at elucidating the relationship between the number of computed tomography (CT) images, including data concerning the accuracy of models and contrast enhancement for classifying the images. We enrolled 1539 patients who underwent contrast or noncontrast CT imaging, followed by dividing the CT imaging dataset for creating classification models into 10 classes for brain, neck, chest, abdomen, and pelvis with contrast-enhanced and plain imaging. The number of images prepared in each class were 100, 500, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, and 10,000. Accordingly, the names of datasets were defined as 0.1K, 0.5K, 1K, 2K, 3K, 4K, 5K, 6K, 7K, 8K, 9K, and 10K, respectively. We subsequently created and evaluated the models and compared the convolutional neural network (CNN) architecture between AlexNet and GoogLeNet. The time required for training models of AlexNet was lesser than that for GoogLeNet. The best overall accuracy for the classification of 10 classes was 0.721 with the 10K dataset of GoogLeNet. Furthermore, the best overall accuracy for the classification of the slice position without contrast media was 0.862 with the 2K dataset of AlexNet.

Prospects for Theranostics in Neurosurgical Imaging: Empowering Confocal Laser Endomicroscopy Diagnostics via Deep Learning

Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence imaging technology that has potential to increase intraoperative precision, extend resection, and tailor surgery for malignant invasive brain tumors because of its subcellular dimension resolution. Despite its promising diagnostic potential, interpreting the gray tone fluorescence images can be difficult for untrained users. CLE images can be distorted by motion artifacts, fluorescence signals out of detector dynamic range, or may be obscured by red blood cells, and thus interpreted as nondiagnostic (ND). However, just a single CLE image with a detectable pathognomonic histological tissue signature can suffice for intraoperative diagnosis. Dealing with the abundance of images from CLE is not unlike sifting through a myriad of genes, proteins, or other structural or metabolic markers to find something of commonality or uniqueness in cancer that might indicate a potential treatment scheme or target. In this review, we provide a detailed description of bioinformatical analysis methodology of CLE images that begins to assist the neurosurgeon and pathologist to rapidly connect on-the-fly intraoperative imaging, pathology, and surgical observation into a conclusionary system within the concept of theranostics. We present an overview and discuss deep learning models for automatic detection of the diagnostic CLE images and discuss various training regimes and ensemble modeling effect on power of deep learning predictive models. Two major approaches reviewed in this paper include the models that can automatically classify CLE images into diagnostic/ND, glioma/nonglioma, tumor/injury/normal categories, and models that can localize histological features on the CLE images using weakly supervised methods. We also briefly review advances in the deep learning approaches used for CLE image analysis in other organs. Significant advances in speed and precision of automated diagnostic frame selection would augment the diagnostic potential of CLE, improve operative workflow, and integration into brain tumor surgery. Such technology and bioinformatics analytics lend themselves to improved precision, personalization, and theranostics in brain tumor treatment.

Classification of Informative Frames in Colonoscopy Videos Using Convolutional Neural Networks with Binarized Weights

Colorectal cancer is one of the common cancers in the United States. Polyps are one of the major causes of colonic cancer, and early detection of polyps will increase the chance of cancer treatments. In this paper, we propose a novel classification of informative frames based on a convolutional neural network with binarized weights. The proposed CNN is trained with colonoscopy frames along with the labels of the frames as input data. We also used binarized weights and kernels to reduce the size of CNN and make it suitable for implementation in medical hardware. We evaluate our proposed method using Asu Mayo Test clinic database, which contains colonoscopy videos of different patients. Our proposed method reaches a dice score of 71.20% and accuracy of more than 90% using the mentioned dataset.

Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning

Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes (a.k.a. zero-shot learning). To address these problems, we propose a novel deep learning-based interactive segmentation framework by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine tuning. We applied this framework to two applications: 2-D segmentation of multiple organs from fetal magnetic resonance (MR) slices, where only two types of these organs were annotated for training and 3-D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only the tumor core in one MR sequence was annotated for training. Experimental results show that: 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.

DRUNET: a dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images

Given that the neural and connective tissues of the optic nerve head (ONH) exhibit complex morphological changes with the development and progression of glaucoma, their simultaneous isolation from optical coherence tomography (OCT) images may be of great interest for the clinical diagnosis and management of this pathology. A deep learning algorithm (custom U-NET) was designed and trained to segment 6 ONH tissue layers by capturing both the local (tissue texture) and contextual information (spatial arrangement of tissues). The overall Dice coefficient (mean of all tissues) was 0.91 ± 0.05 when assessed against manual segmentations performed by an expert observer. Further, we automatically extracted six clinically relevant neural and connective tissue structural parameters from the segmented tissues. We offer here a robust segmentation framework that could also be extended to the 3D segmentation of the ONH tissues.

Transfer Representation Learning using Inception-V3 for the Detection of Masses in Mammography

Breast cancer is the most prevalent cancer among women. The most common method to detect breast cancer is mammography. However, interpreting mammography is a challenging task that requires high skills and is timeconsuming. In this work, we propose a computer-aided diagnosis (CAD) scheme for mammography based on transfer representation learning using the Inception-V3 architecture. We evaluate the performance of the proposed scheme using the INBreast database, where the features are extracted from different layers of the architecture. In order to cope with the small dataset size limitation, we expand the training dataset by generating artificial mammograms and employing different augmentation techniques. The proposed scheme shows great potential with a maximal area under the receiver operating characteristics curve of 0.91.