851
|
Versatile Framework for Medical Image Processing and Analysis with Application to Automatic Bone Age Assessment. JOURNAL OF ELECTRICAL AND COMPUTER ENGINEERING 2018. [DOI: 10.1155/2018/2187247] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Deep learning technique has made a tremendous impact on medical image processing and analysis. Typically, the procedure of medical image processing and analysis via deep learning technique includes image segmentation, image enhancement, and classification or regression. A challenge for supervised deep learning frequently mentioned is the lack of annotated training data. In this paper, we aim to address the problems of training transferred deep neural networks with limited amount of annotated data. We proposed a versatile framework for medical image processing and analysis via deep active learning technique. The framework includes (1) applying deep active learning approach to segment specific regions of interest (RoIs) from raw medical image by using annotated data as few as possible; (2) generative adversarial Network is employed to enhance contrast, sharpness, and brightness of segmented RoIs; (3) Paced Transfer Learning (PTL) strategy which means fine-tuning layers in deep neural networks from top to bottom step by step to perform medical image classification or regression tasks. In addition, in order to understand the necessity of deep-learning-based medical image processing tasks and provide clues for clinical usage, class active map (CAM) is employed in our framework to visualize the feature maps. To illustrate the effectiveness of the proposed framework, we apply our framework to the bone age assessment (BAA) task using RSNA dataset and achieve the state-of-the-art performance. Experimental results indicate that the proposed framework can be effectively applied to medical image analysis task.
Collapse
|
852
|
Panayides AS, Pattichis MS, Leandrou S, Pitris C, Constantinidou A, Pattichis CS. Radiogenomics for Precision Medicine With a Big Data Analytics Perspective. IEEE J Biomed Health Inform 2018; 23:2063-2079. [PMID: 30596591 DOI: 10.1109/jbhi.2018.2879381] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Precision medicine promises better healthcare delivery by improving clinical practice. Using evidence-based substratification of patients, the objective is to achieve better prognosis, diagnosis, and treatment that will transform existing clinical pathways toward optimizing care for the specific needs of each patient. The wealth of today's healthcare data, often characterized as big data, provides invaluable resources toward new knowledge discovery that has the potential to advance precision medicine. The latter requires interdisciplinary efforts that will capitalize the information, know-how, and medical data of newly formed groups fusing different backgrounds and expertise. The objective of this paper is to provide insights with respect to the state-of-the-art research in precision medicine. More specifically, our goal is to highlight the fundamental challenges in emerging fields of radiomics and radiogenomics by reviewing the case studies of Cancer and Alzheimer's disease, describe the computational challenges from a big data analytics perspective, and discuss standardization and open data initiatives that will facilitate the adoption of precision medicine methods and practices.
Collapse
|
853
|
Fischer W, Moudgalya SS, Cohn JD, Nguyen NTT, Kenyon GT. Sparse coding of pathology slides compared to transfer learning with deep neural networks. BMC Bioinformatics 2018; 19:489. [PMID: 30577746 PMCID: PMC6302377 DOI: 10.1186/s12859-018-2504-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
Background Histopathology images of tumor biopsies present unique challenges for applying machine learning to the diagnosis and treatment of cancer. The pathology slides are high resolution, often exceeding 1GB, have non-uniform dimensions, and often contain multiple tissue slices of varying sizes surrounded by large empty regions. The locations of abnormal or cancerous cells, which may constitute a small portion of any given tissue sample, are not annotated. Cancer image datasets are also extremely imbalanced, with most slides being associated with relatively common cancers. Since deep representations trained on natural photographs are unlikely to be optimal for classifying pathology slide images, which have different spectral ranges and spatial structure, we here describe an approach for learning features and inferring representations of cancer pathology slides based on sparse coding. Results We show that conventional transfer learning using a state-of-the-art deep learning architecture pre-trained on ImageNet (RESNET) and fine tuned for a binary tumor/no-tumor classification task achieved between 85% and 86% accuracy. However, when all layers up to the last convolutional layer in RESNET are replaced with a single feature map inferred via a sparse coding using a dictionary optimized for sparse reconstruction of unlabeled pathology slides, classification performance improves to over 93%, corresponding to a 54% error reduction. Conclusions We conclude that a feature dictionary optimized for biomedical imagery may in general support better classification performance than does conventional transfer learning using a dictionary pre-trained on natural images. Electronic supplementary material The online version of this article (10.1186/s12859-018-2504-8) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Will Fischer
- Los Alamos National Laboratory, Los Alamos, NM, USA.
| | - Sanketh S Moudgalya
- Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology, Rochester, NY, USA
| | | | | | | |
Collapse
|
854
|
Brag J, Auffret M, Ramos C, Liu Y, Baudot P. iBiopsy® for Precision Medicine. EUROPEAN MEDICAL JOURNAL 2018. [DOI: 10.33590/emj/10310309] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
A high-throughput artificial intelligence-powered image-based phenotyping platform, iBiopsy® (Median Technologies, Valbonne, France), which aims to improve precision medicine, is discussed in the presented review. The article introduces novel concepts, including high-throughput, fully automated imaging biomarker extraction; unsupervised predictive learning; large-scale content- based image-based similarity search; the use of large-scale clinical data registries; and cloud-based big data analytics to the problems of disease subtyping and treatment planning. Unlike electronic health record-based approaches, which lack the detailed radiological, pathological, genomic, and molecular data necessary for accurate prediction, iBiopsy generates unique signatures as fingerprints of disease and tumour subtypes from target images. These signatures are then merged with any additional omics data and matched against a large-scale reference registry of deeply phenotyped patients. Initial applications targeted include hepatocellular carcinoma and other chronic liver diseases, such as nonalcoholic steatohepatitis. This new disruptive technology is expected to lead to the identification of appropriate therapies targeting specific molecular pathways involved in the detected phenotypes to bring personalised treatment to patients, taking into account individual biological variability, which is the principal aim of precision medicine.
Collapse
Affiliation(s)
- Johan Brag
- iBiopsy® Clinical Development, Median Technologies, Valbonne, France
| | - Michaël Auffret
- iBiopsy® Clinical Development, Median Technologies, Valbonne, France
| | - Corinne Ramos
- iBiopsy® Clinical Development, Median Technologies, Valbonne, France
| | - Yan Liu
- iBiopsy® Clinical Development, Median Technologies, Valbonne, France
| | - Pierre Baudot
- iBiopsy® Science & Image Processing, Median Technologies, Valbonne, France
| |
Collapse
|
855
|
Face Recognition Using the SR-CNN Model. SENSORS 2018; 18:s18124237. [PMID: 30513898 PMCID: PMC6308568 DOI: 10.3390/s18124237] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/27/2018] [Revised: 11/27/2018] [Accepted: 11/28/2018] [Indexed: 12/11/2022]
Abstract
In order to solve the problem of face recognition in complex environments being vulnerable to illumination change, object rotation, occlusion, and so on, which leads to the imprecision of target position, a face recognition algorithm with multi-feature fusion is proposed. This study presents a new robust face-matching method named SR-CNN, combining the rotation-invariant texture feature (RITF) vector, the scale-invariant feature transform (SIFT) vector, and the convolution neural network (CNN). Furthermore, a graphics processing unit (GPU) is used to parallelize the model for an optimal computational performance. The Labeled Faces in the Wild (LFW) database and self-collection face database were selected for experiments. It turns out that the true positive rate is improved by 10.97–13.24% and the acceleration ratio (the ratio between central processing unit (CPU) operation time and GPU time) is 5–6 times for the LFW face database. For the self-collection, the true positive rate increased by 12.65–15.31%, and the acceleration ratio improved by a factor of 6–7.
Collapse
|
856
|
Li S, Yang N, Li B, Zhou Z, Hao H, Folkert MR, Iyengar P, Westover K, Choy H, Timmerman R, Jiang S, Wang J. A pilot study using kernelled support tensor machine for distant failure prediction in lung SBRT. Med Image Anal 2018; 50:106-116. [PMID: 30266009 PMCID: PMC6237633 DOI: 10.1016/j.media.2018.09.004] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2017] [Revised: 07/20/2018] [Accepted: 09/07/2018] [Indexed: 12/27/2022]
Abstract
We developed a kernelled support tensor machine (KSTM)-based model with tumor tensors derived from pre-treatment PET and CT imaging as input to predict distant failure in early stage non-small cell lung cancer (NSCLC) treated with stereotactic body radiation therapy (SBRT). The patient cohort included 110 early stage NSCLC patients treated with SBRT, 25 of whom experienced failure at distant sites. Three-dimensional tumor tensors were constructed and used as input for the KSTM-based classifier. A KSTM iterative algorithm with a convergent proof was developed to train the weight vectors for every mode of the tensor for the classifier. In contrast to conventional radiomics approaches that rely on handcrafted imaging features, the KSTM-based classifier uses 3D imaging as input, taking full advantage of the imaging information. The KSTM-based classifier preserves the intrinsic 3D geometry structure of the medical images and the correlation in the original images and trains the classification hyper-plane in an adaptive feature tensor space. The KSTM-based predictive algorithm was compared with three conventional machine learning models and three radiomics approaches. For PET and CT, the KSTM-based predictive method achieved the highest prediction results among the seven methods investigated in this study based on 10-fold cross validation and independent testing.
Collapse
Affiliation(s)
- Shulong Li
- School of Biomedical Engineering, Guangdong Provincial Key Laboratory of Medical Image, Processing, Southern Medical University, Guangzhou 510515, China
| | - Ning Yang
- Department of Medical Imaging, Guangdong No.2 Provincial People's Hospital, Guangzhou 510317, China
| | - Bin Li
- School of Biomedical Engineering, Guangdong Provincial Key Laboratory of Medical Image, Processing, Southern Medical University, Guangzhou 510515, China
| | - Zhiguo Zhou
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Hongxia Hao
- School of Computer Science and Technology, Xidian University, Xi'an 710071, China
| | - Michael R Folkert
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Puneeth Iyengar
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Kenneth Westover
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Hak Choy
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Robert Timmerman
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Steve Jiang
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA
| | - Jing Wang
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas 75235, USA.
| |
Collapse
|
857
|
Li X, Chen H, Qi X, Dou Q, Fu CW, Heng PA. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2663-2674. [PMID: 29994201 DOI: 10.1109/tmi.2018.2845918] [Citation(s) in RCA: 755] [Impact Index Per Article: 125.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/19/2023]
Abstract
Liver cancer is one of the leading causes of cancer death. To assist doctors in hepatocellular carcinoma diagnosis and treatment planning, an accurate and automatic liver and tumor segmentation method is highly demanded in the clinical practice. Recently, fully convolutional neural networks (FCNs), including 2-D and 3-D FCNs, serve as the backbone in many volumetric image segmentation. However, 2-D convolutions cannot fully leverage the spatial information along the third dimension while 3-D convolutions suffer from high computational cost and GPU memory consumption. To address these issues, we propose a novel hybrid densely connected UNet (H-DenseUNet), which consists of a 2-D DenseUNet for efficiently extracting intra-slice features and a 3-D counterpart for hierarchically aggregating volumetric contexts under the spirit of the auto-context algorithm for liver and tumor segmentation. We formulate the learning process of the H-DenseUNet in an end-to-end manner, where the intra-slice representations and inter-slice features can be jointly optimized through a hybrid feature fusion layer. We extensively evaluated our method on the data set of the MICCAI 2017 Liver Tumor Segmentation Challenge and 3DIRCADb data set. Our method outperformed other state-of-the-arts on the segmentation results of tumors and achieved very competitive performance for liver segmentation even with a single model.
Collapse
|
858
|
Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.09.013] [Citation(s) in RCA: 309] [Impact Index Per Article: 51.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
|
859
|
Lei B, Huang S, Li R, Bian C, Li H, Chou YH, Cheng JZ. Segmentation of breast anatomy for automated whole breast ultrasound images with boundary regularized convolutional encoder–decoder network. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.09.043] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
860
|
Mahmood F, Chen R, Durr NJ. Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2572-2581. [PMID: 29993538 DOI: 10.1109/tmi.2018.2842767] [Citation(s) in RCA: 97] [Impact Index Per Article: 16.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.
Collapse
|
861
|
Devan KS, Walther P, von Einem J, Ropinski T, Kestler HA, Read C. Detection of herpesvirus capsids in transmission electron microscopy images using transfer learning. Histochem Cell Biol 2018; 151:101-114. [PMID: 30488339 DOI: 10.1007/s00418-018-1759-5] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/24/2018] [Indexed: 10/27/2022]
Abstract
The detailed analysis of secondary envelopment of the Human betaherpesvirus 5/human cytomegalovirus (HCMV) from transmission electron microscopy (TEM) images is an important step towards understanding the mechanisms underlying the formation of infectious virions. As a step towards a software-based quantification of different stages of HCMV virion morphogenesis in TEM, we developed a transfer learning approach based on convolutional neural networks (CNNs) that automatically detects HCMV nucleocapsids in TEM images. In contrast to existing image analysis techniques that require time-consuming manual definition of structural features, our method automatically learns discriminative features from raw images without the need for extensive pre-processing. For this a constantly growing TEM image database of HCMV infected cells was available which is unique regarding image quality and size in the terms of virological EM. From the two investigated types of transfer learning approaches, namely feature extraction and fine-tuning, the latter enabled us to successfully detect HCMV nucleocapsids in TEM images. Our detection method has outperformed some of the existing image analysis methods based on discriminative textural indicators and radial density profiles for virus detection in TEM images. In summary, we could show that the method of transfer learning can be used for an automated detection of viral capsids in TEM images with high specificity using standard computers. This method is highly adaptable and in future could be easily extended to automatically detect and classify virions of other viruses and even distinguish different virion maturation stages.
Collapse
Affiliation(s)
- K Shaga Devan
- Central Facility for Electron Microscopy, Ulm University, Ulm, Germany
| | - P Walther
- Central Facility for Electron Microscopy, Ulm University, Ulm, Germany.
| | - J von Einem
- Institute of Virology, Ulm University Medical Center, Ulm, Germany
| | - T Ropinski
- Institute of Media Informatics, Ulm University, Ulm, Germany
| | - H A Kestler
- Institute of Medical Systems Biology, Ulm University, Ulm, Germany
| | - C Read
- Central Facility for Electron Microscopy, Ulm University, Ulm, Germany.,Institute of Virology, Ulm University Medical Center, Ulm, Germany
| |
Collapse
|
862
|
Yang Y, Yan LF, Zhang X, Han Y, Nan HY, Hu YC, Hu B, Yan SL, Zhang J, Cheng DL, Ge XW, Cui GB, Zhao D, Wang W. Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning. Front Neurosci 2018; 12:804. [PMID: 30498429 PMCID: PMC6250094 DOI: 10.3389/fnins.2018.00804] [Citation(s) in RCA: 118] [Impact Index Per Article: 19.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2018] [Accepted: 10/16/2018] [Indexed: 12/27/2022] Open
Abstract
Background: Accurate glioma grading before surgery is of the utmost importance in treatment planning and prognosis prediction. But previous studies on magnetic resonance imaging (MRI) images were not effective enough. According to the remarkable performance of convolutional neural network (CNN) in medical domain, we hypothesized that a deep learning algorithm can achieve high accuracy in distinguishing the World Health Organization (WHO) low grade and high grade gliomas. Methods: One hundred and thirteen glioma patients were retrospectively included. Tumor images were segmented with a rectangular region of interest (ROI), which contained about 80% of the tumor. Then, 20% data were randomly selected and leaved out at patient-level as test dataset. AlexNet and GoogLeNet were both trained from scratch and fine-tuned from models that pre-trained on the large scale natural image database, ImageNet, to magnetic resonance images. The classification task was evaluated with five-fold cross-validation (CV) on patient-level split. Results: The performance measures, including validation accuracy, test accuracy and test area under curve (AUC), averaged from five-fold CV of GoogLeNet which trained from scratch were 0.867, 0.909, and 0.939, respectively. With transfer learning and fine-tuning, better performances were obtained for both AlexNet and GoogLeNet, especially for AlexNet. Meanwhile, GoogLeNet performed better than AlexNet no matter trained from scratch or learned from pre-trained model. Conclusion: In conclusion, we demonstrated that the application of CNN, especially trained with transfer learning and fine-tuning, to preoperative glioma grading improves the performance, compared with either the performance of traditional machine learning method based on hand-crafted features, or even the CNNs trained from scratch.
Collapse
Affiliation(s)
- Yang Yang
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Lin-Feng Yan
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Xin Zhang
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Yu Han
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Hai-Yan Nan
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Yu-Chuan Hu
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Bo Hu
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Song-Lin Yan
- Computer Network Information Center, Chinese Academy of Sciences, Beijing, China
| | - Jin Zhang
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Dong-Liang Cheng
- Student Brigade, Fourth Military Medical University, Xi'an, China
| | - Xiang-Wei Ge
- Student Brigade, Fourth Military Medical University, Xi'an, China
| | - Guang-Bin Cui
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Di Zhao
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Wen Wang
- Functional and Molecular Imaging Key Lab of Shaanxi Province, Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| |
Collapse
|
863
|
Sun W, Li M, Li Y, Wu Z, Sun Y, Lu S, Xiao Z, Zhao B, Sun K. The Use of Deep Learning to Fast Evaluate Organic Photovoltaic Materials. ADVANCED THEORY AND SIMULATIONS 2018. [DOI: 10.1002/adts.201800116] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Affiliation(s)
- Wenbo Sun
- MOE Key Laboratory of Low‐grade Energy Utilization Technologies and SystemsSchool of Energy and Power EngineeringChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
| | - Meng Li
- MOE Key Laboratory of Low‐grade Energy Utilization Technologies and SystemsSchool of Energy and Power EngineeringChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
| | - Yong Li
- MOE Key Laboratory of Low‐grade Energy Utilization Technologies and SystemsSchool of Energy and Power EngineeringChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
- School of Economics and Business AdministrationChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
| | - Zhou Wu
- MOE Key Laboratory of Dependable Service Computing in Cyber Physical SocietySchool of AutomationChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
| | - Yuyang Sun
- MOE Key Laboratory of Dependable Service Computing in Cyber Physical SocietySchool of AutomationChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
| | - Shirong Lu
- Chongqing Institute of Green and Intelligent TechnologyChinese Academy of Sciences 266 Fang Zheng Road Beibei, Chongqing 400714 China
| | - Zeyun Xiao
- Chongqing Institute of Green and Intelligent TechnologyChinese Academy of Sciences 266 Fang Zheng Road Beibei, Chongqing 400714 China
| | - Baomin Zhao
- Key Laboratory for Organic Electronics and Information DisplaysInstitute of Advanced MaterialsNanjing University of Posts and Telecommunications 66 New Model Road Nanjing 210023 China
| | - Kuan Sun
- MOE Key Laboratory of Low‐grade Energy Utilization Technologies and SystemsSchool of Energy and Power EngineeringChongqing University 174 Shazhengjie Shapingba, Chongqing 400044 China
| |
Collapse
|
864
|
Forensic age estimation for pelvic X-ray images using deep learning. Eur Radiol 2018; 29:2322-2329. [PMID: 30402703 DOI: 10.1007/s00330-018-5791-6] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2018] [Revised: 09/06/2018] [Accepted: 09/21/2018] [Indexed: 02/05/2023]
Abstract
PURPOSE To develop a deep learning bone age assessment model based on pelvic radiographs for forensic age estimation and compare its performance to that of the existing cubic regression model. MATERIALS AND METHOD A retrospective collection data of 1875 clinical pelvic radiographs between 10 and 25 years of age was obtained to develop the model. Model performance was assessed by comparing the testing results to estimated ages calculated directly using the existing cubic regression model based on ossification staging methods. The mean absolute error (MAE) and root-mean-squared error (RMSE) between the estimated ages and chronological age were calculated for both models. RESULTS For all test samples (between 10 and 25 years old), the mean MAE and RMSE between the automatic estimates using the proposed deep learning model and the reference standard were 0.94 and 1.30 years, respectively. For the test samples comparable to those of the existing cubic regression model (between 14 and 22 years old), the mean MAE and RMSE for the deep learning model were 0.89 and 1.21 years, respectively. For the existing cubic regression model, the mean MAE and RMSE were 1.05 and 1.61 years, respectively. CONCLUSION The deep learning convolutional neural network model achieves performance on par with the existing cubic regression model, demonstrating predictive ability capable of automated skeletal bone assessment based on pelvic radiographic images. KEY POINTS • The pelvis has considerable value in determining the bone age. • Deep learning can be used to create an automated bone age assessment model based on pelvic radiographs. • The deep learning convolutional neural network model achieves performance on par with the existing cubic regression model.
Collapse
|
865
|
Bharti P, Mittal D, Ananthasivan R. Preliminary Study of Chronic Liver Classification on Ultrasound Images Using an Ensemble Model. ULTRASONIC IMAGING 2018; 40:357-379. [PMID: 30015593 DOI: 10.1177/0161734618787447] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Chronic liver diseases are fifth leading cause of fatality in developing countries. Their early diagnosis is extremely important for timely treatment and salvage life. To examine abnormalities of liver, ultrasound imaging is the most frequently used modality. However, the visual differentiation between chronic liver and cirrhosis, and presence of heptocellular carcinomas (HCC) evolved over cirrhotic liver is difficult, as they appear almost similar in ultrasound images. In this paper, to deal with this difficult visualization problem, a method has been developed for classifying four liver stages, that is, normal, chronic, cirrhosis, and HCC evolved over cirrhosis. The method is formulated with selected set of "handcrafted" texture features obtained after hierarchal feature fusion. These multiresolution and higher order features, which are able to characterize echotexture and roughness of liver surface, are extracted by using ranklet, gray-level difference matrix and gray-level co-occurrence matrix methods. Thereafter, these features are applied on proposed ensemble classifier that is designed with voting algorithm in conjunction with three classifiers, namely, k-nearest neighbor (k-NN), support vector machine (SVM), and rotation forest. The experiments are conducted to evaluate the (a) effectiveness of "handcrafted" texture features, (b) performance of proposed ensemble model, (c) effectiveness of proposed ensemble strategy, (d) performance of different classifiers, and (e) performance of proposed ensemble model based on Convolutional Neural Networks (CNN) features to differentiate four liver stages. These experiments are carried out on database of 754 segmented regions of interest formed by clinically acquired ultrasound images. The results show that classification accuracy of 96.6% is obtained by use of proposed classifier model.
Collapse
Affiliation(s)
- Puja Bharti
- 1 Thapar Institute of Engineering & Technology, Patiala, India
| | - Deepti Mittal
- 1 Thapar Institute of Engineering & Technology, Patiala, India
| | | |
Collapse
|
866
|
Zhang R, Zheng Y, Poon CC, Shen D, Lau JY. Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker. PATTERN RECOGNITION 2018; 83:209-219. [PMID: 31105338 PMCID: PMC6519928 DOI: 10.1016/j.patcog.2018.05.026] [Citation(s) in RCA: 63] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
A computer-aided detection (CAD) tool for locating and detecting polyps can help reduce the chance of missing polyps during colonoscopy. Nevertheless, state-of-the-art algorithms were either computationally complex or suffered from low sensitivity and therefore unsuitable to be used in real clinical setting. In this paper, a novel regression-based Convolutional Neural Network (CNN) pipeline is presented for polyp detection during colonoscopy. The proposed pipeline was constructed in two parts: 1) to learn the spatial features of colorectal polyps, a fast object detection algorithm named ResYOLO was pre-trained with a large non-medical image database and further fine-tuned with colonoscopic images extracted from videos; and 2) temporal information was incorporated via a tracker named Efficient Convolution Operators (ECO) for refining the detection results given by ResYOLO. Evaluated on 17,574 frames extracted from 18 endoscopic videos of the AsuMayoDB, the proposed method was able to detect frames with polyps with a precision of 88.6%, recall of 71.6% and processing speed of 6.5 frames per second, i.e. the method can accurately locate polyps in more frames and at a faster speed compared to existing methods. In conclusion, the proposed method has great potential to be used to assist endoscopists in tracking polyps during colonoscopy.
Collapse
Affiliation(s)
- Ruikai Zhang
- Department of Surgery, The Chinese University of Hong Kong, Hong Kong
| | - Yali Zheng
- Department of Surgery, The Chinese University of Hong Kong, Hong Kong
| | - Carmen C.Y. Poon
- Department of Surgery, The Chinese University of Hong Kong, Hong Kong
- Corresponding author
| | - Dinggang Shen
- Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina, Chapel Hill, NC, USA
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
- Corresponding author at: Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina, Chapel Hill, NC, USA.
| | - James Y.W. Lau
- Department of Surgery, The Chinese University of Hong Kong, Hong Kong
| |
Collapse
|
867
|
Wu C, Guo S, Hong Y, Xiao B, Wu Y, Zhang Q. Discrimination and conversion prediction of mild cognitive impairment using convolutional neural networks. Quant Imaging Med Surg 2018; 8:992-1003. [PMID: 30598877 DOI: 10.21037/qims.2018.10.17] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
Background Recently, studies have demonstrated that machine learning techniques, particularly cutting-edge deep learning technology, have achieved significant progression on the classification of Alzheimer's disease (AD) and its prodromal phase, mild cognitive impairment (MCI). Moreover, accurate prediction of the progress and the conversion risk from MCI to probable AD has been of great importance in clinical application. Methods In this study, the baseline MR images and follow-up information during 3 years of 150 normal controls (NC), 150 patients with stable MCI (sMCI) and 157 converted MCI (cMCI) were collected from the Alzheimer's Disease Neuroimaging Initiative (ADNI). The deep convolutional neural networks (CNNs) were adopted to distinguish different stages of MCI from the NC group, and predict the conversion time from MCI to AD. Two CNN architectures including GoogleNet and CaffeNet were explored and evaluated in multiple classifications and estimations of conversion risk using transfer learning from pre-trained ImageNet (via fine-tuning) and five-fold cross-validation. A novel data augmentation approach using random views aggregation was applied to generate abundant image patches from the original MR scans. Results The GoogleNet acquired accuracies with 97.58%, 67.33% and 84.71% in three-way discrimination among the NC, sMCI and cMCI groups respectively, whereas the CaffeNet obtained promising accuracies of 98.71%, 72.04% and 92.35% in the NC, sMCI and cMCI classifications. Furthermore, the accuracy measures of conversion risk of patients with cMCI ranged from 71.25% to 83.25% in different time points using GoogleNet, whereas the CaffeNet achieved remarkable accuracy measures from 95.42% to 97.01% in conversion risk prediction. Conclusions The experimental results demonstrated that the proposed methods had prominent capability in classification among the 3 groups such as sMCI, cMCI and NC, and exhibited significant ability in conversion risk prediction of patients with MCI.
Collapse
Affiliation(s)
- Congling Wu
- Department of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China
| | - Shengwen Guo
- Department of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China
| | - Yanjia Hong
- Department of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China
| | - Benheng Xiao
- Department of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China
| | - Yupeng Wu
- Department of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China
| | - Qin Zhang
- Department of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China
| | | |
Collapse
|
868
|
Song B, Sunny S, Uthoff RD, Patrick S, Suresh A, Kolur T, Keerthi G, Anbarani A, Wilder-Smith P, Kuriakose MA, Birur P, Rodriguez JJ, Liang R. Automatic classification of dual-modalilty, smartphone-based oral dysplasia and malignancy images using deep learning. BIOMEDICAL OPTICS EXPRESS 2018; 9:5318-5329. [PMID: 30460130 PMCID: PMC6238918 DOI: 10.1364/boe.9.005318] [Citation(s) in RCA: 51] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/28/2018] [Revised: 08/10/2018] [Accepted: 09/12/2018] [Indexed: 05/04/2023]
Abstract
With the goal to screen high-risk populations for oral cancer in low- and middle-income countries (LMICs), we have developed a low-cost, portable, easy to use smartphone-based intraoral dual-modality imaging platform. In this paper we present an image classification approach based on autofluorescence and white light images using deep learning methods. The information from the autofluorescence and white light image pair is extracted, calculated, and fused to feed the deep learning neural networks. We have investigated and compared the performance of different convolutional neural networks, transfer learning, and several regularization techniques for oral cancer classification. Our experimental results demonstrate the effectiveness of deep learning methods in classifying dual-modal images for oral cancer detection.
Collapse
Affiliation(s)
- Bofan Song
- College of Optical Sciences, The University of Arizona, Tucson, AZ, USA
| | - Sumsum Sunny
- Mazumdar Shaw Medical Centre, Bangalore, India
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Ross D Uthoff
- College of Optical Sciences, The University of Arizona, Tucson, AZ, USA
| | | | - Amritha Suresh
- Mazumdar Shaw Medical Centre, Bangalore, India
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | | | - G Keerthi
- KLE Society Institute of Dental Sciences, Bangalore, India
| | - Afarin Anbarani
- Beckman Laser Institute, University of California, Irvine, Irvine, CA, USA
| | - Petra Wilder-Smith
- Beckman Laser Institute, University of California, Irvine, Irvine, CA, USA
| | - Moni Abraham Kuriakose
- Mazumdar Shaw Medical Centre, Bangalore, India
- Mazumdar Shaw Medical Foundation, Bangalore, India
| | - Praveen Birur
- Biocon Foundation, Bangalore, India
- KLE Society Institute of Dental Sciences, Bangalore, India
| | - Jeffrey J Rodriguez
- Dept. of Electrical and Computer Engineering, The University of Arizona, Tucson, AZ, USA
| | - Rongguang Liang
- College of Optical Sciences, The University of Arizona, Tucson, AZ, USA
| |
Collapse
|
869
|
To MNN, Vu DQ, Turkbey B, Choyke PL, Kwak JT. Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging. Int J Comput Assist Radiol Surg 2018; 13:1687-1696. [PMID: 30088208 PMCID: PMC6177294 DOI: 10.1007/s11548-018-1841-4] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2018] [Accepted: 07/27/2018] [Indexed: 01/22/2023]
Abstract
PURPOSE We propose an approach of 3D convolutional neural network to segment the prostate in MR images. METHODS A 3D deep dense multi-path convolutional neural network that follows the framework of the encoder-decoder design is proposed. The encoder is built based upon densely connected layers that learn the high-level feature representation of the prostate. The decoder interprets the features and predicts the whole prostate volume by utilizing a residual layout and grouped convolution. A set of sub-volumes of MR images, centered at the prostate, is generated and fed into the proposed network for training purpose. The performance of the proposed network is compared to previously reported approaches. RESULTS Two independent datasets were employed to assess the proposed network. In quantitative evaluations, the proposed network achieved 95.11 and 89.01 Dice coefficients for the two datasets. The segmentation results were robust to variations in MR images. In comparison experiments, the segmentation performance of the proposed network was comparable to the previously reported approaches. In qualitative evaluations, the segmentation results by the proposed network were well matched to the ground truth provided by human experts. CONCLUSIONS The proposed network is capable of segmenting the prostate in an accurate and robust manner. This approach can be applied to other types of medical images.
Collapse
Affiliation(s)
- Minh Nguyen Nhat To
- Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea
| | - Dang Quoc Vu
- Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea
| | - Baris Turkbey
- Molecular Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Peter L Choyke
- Molecular Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Jin Tae Kwak
- Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea.
| |
Collapse
|
870
|
Classification of contrast-enhanced spectral mammography (CESM) images. Int J Comput Assist Radiol Surg 2018; 14:249-257. [PMID: 30367322 DOI: 10.1007/s11548-018-1876-6] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2018] [Accepted: 10/13/2018] [Indexed: 10/28/2022]
Abstract
PURPOSE Contrast-enhanced spectral mammography (CESM) is a recently developed breast imaging technique. CESM relies on dual-energy acquisition following contrast agent injection to improve mammography sensitivity. CESM is comparable to contrast-enhanced MRI in terms of sensitivity, at a fraction of the cost. However, since lesion variability is large, even with the improved visibility provided by CESM, differentiation between benign and malignant enhancement is not accurate and a biopsy is usually performed for final assessment. Breast biopsies can be stressful to the patient and are expensive to healthcare systems. Moreover, as the biopsies results are most of the time benign, a specificity improvement in the radiologist diagnosis is required. This work presents a deep learning-based decision support system, which aims at improving the specificity of breast cancer diagnosis by CESM without affecting sensitivity. METHODS We compare two analysis approaches, fine-tuning a pretrained network and fully training a convolutional neural network, for classification of CESM breast mass as benign or malignant. Breast Imaging Reporting and Data Systems (BIRADS) is a radiological lexicon, used with breast images, to categorize lesions. We improve each classification network by incorporating BIRADS textual features as an additional input to the network. We evaluate two ways of BIRADS fusion as network input: feature fusion and decision fusion. This leads to multimodal network architectures. At classification, we also exploit information from apparently normal breast tissue in the CESM of the considered patient, leading to a patient-specific classification. RESULTS We evaluate performance using fivefold cross-validation, on 129 randomly selected breast lesions annotated by an experienced radiologist. Each annotation includes a contour of the mass in the image, biopsy-proven label of benign or malignant lesion and BIRADS descriptors. At 100% sensitivity, specificity of 66% was achieved using a multimodal network, which combines inputs at feature level and patient-specific classification. CONCLUSIONS The presented multimodal network may significantly reduce benign biopsies, without compromising sensitivity.
Collapse
|
871
|
Thompson RF, Valdes G, Fuller CD, Carpenter CM, Morin O, Aneja S, Lindsay WD, Aerts HJWL, Agrimson B, Deville C, Rosenthal SA, Yu JB, Thomas CR. Artificial Intelligence in Radiation Oncology Imaging. Int J Radiat Oncol Biol Phys 2018; 102:1159-1161. [PMID: 30353870 DOI: 10.1016/j.ijrobp.2018.05.070] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2018] [Revised: 05/17/2018] [Accepted: 05/26/2018] [Indexed: 12/19/2022]
Affiliation(s)
- Reid F Thompson
- Oregon Health & Science University, Portland, Oregon; VA Portland Health Care System, Portland, Oregon.
| | - Gilmer Valdes
- University of California San Francisco, San Francisco, California
| | | | | | - Olivier Morin
- University of California San Francisco, San Francisco, California
| | | | | | - Hugo J W L Aerts
- Dana Farber Cancer Institute, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | | | | | - Seth A Rosenthal
- Sutter Medical Group and Suttter Cancer Center, Sacramento, California
| | | | | |
Collapse
|
872
|
Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks. REMOTE SENSING 2018. [DOI: 10.3390/rs10101636] [Citation(s) in RCA: 53] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Remote sensing images have long been preferred to perform building damage assessments. The recently proposed methods to extract damaged regions from remote sensing imagery rely on convolutional neural networks (CNN). The common approach is to train a CNN independently considering each of the different resolution levels (satellite, aerial, and terrestrial) in a binary classification approach. In this regard, an ever-growing amount of multi-resolution imagery are being collected, but the current approaches use one single resolution as their input. The use of up/down-sampled images for training has been reported as beneficial for the image classification accuracy both in the computer vision and remote sensing domains. However, it is still unclear if such multi-resolution information can also be captured from images with different spatial resolutions such as imagery of the satellite and airborne (from both manned and unmanned platforms) resolutions. In this paper, three multi-resolution CNN feature fusion approaches are proposed and tested against two baseline (mono-resolution) methods to perform the image classification of building damages. Overall, the results show better accuracy and localization capabilities when fusing multi-resolution feature maps, specifically when these feature maps are merged and consider feature information from the intermediate layers of each of the resolution level networks. Nonetheless, these multi-resolution feature fusion approaches behaved differently considering each level of resolution. In the satellite and aerial (unmanned) cases, the improvements in the accuracy reached 2% while the accuracy improvements for the airborne (manned) case was marginal. The results were further confirmed by testing the approach for geographical transferability, in which the improvements between the baseline and multi-resolution experiments were overall maintained.
Collapse
|
873
|
Zago GT, Andreão RV, Dorizzi B, Teatini Salles EO. Retinal image quality assessment using deep learning. Comput Biol Med 2018; 103:64-70. [PMID: 30340214 DOI: 10.1016/j.compbiomed.2018.10.004] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2018] [Revised: 09/27/2018] [Accepted: 10/06/2018] [Indexed: 11/25/2022]
Abstract
Poor-quality retinal images do not allow an accurate medical diagnosis, and it is inconvenient for a patient to return to a medical center to repeat the fundus photography exam. In this paper, a robust automatic system is proposed to assess the quality of retinal images at the moment of the acquisition, aiming at assisting health care professionals during a fundus photography exam. We propose a convolutional neural network (CNN) pretrained on non-medical images for extracting general image features. The weights of the CNN are further adjusted via a fine-tuning procedure, resulting in a performant classifier obtained only with a small quantity of labeled images. The CNN performance was evaluated on two publicly available databases (i.e., DRIMDB and ELSA-Brasil) using two different procedures: intra-database and inter-database cross-validation. The CNN achieved an area under the curve (AUC) of 99.98% on DRIMDB and an AUC of 98.56% on ELSA-Brasil in the inter-database experiment, where training and testing were not performed on the same database. These results show the robustness of the proposed model to various image acquisitions without requiring special adaptation, thus making it a good candidate for use in operational clinical scenarios.
Collapse
Affiliation(s)
- Gabriel Tozatto Zago
- Department of Control and Automation Engineering, Instituto Federal do Espírito Santo, Brazil.
| | | | - Bernadette Dorizzi
- Télécom SudParis, Laboratoire SAMOVAR, 9 rue Charles Fourier, 91011, EVRY, France.
| | | |
Collapse
|
874
|
Iakovidis DK, Georgakopoulos SV, Vasilakakis M, Koulaouzidis A, Plagianakos VP. Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2196-2210. [PMID: 29994763 DOI: 10.1109/tmi.2018.2837002] [Citation(s) in RCA: 75] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.
Collapse
|
875
|
Building medical image classifiers with very limited data using segmentation networks. Med Image Anal 2018; 49:105-116. [DOI: 10.1016/j.media.2018.07.010] [Citation(s) in RCA: 44] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2017] [Revised: 07/19/2018] [Accepted: 07/30/2018] [Indexed: 11/22/2022]
|
876
|
Imrie F, Bradley AR, van der Schaar M, Deane CM. Protein Family-Specific Models Using Deep Neural Networks and Transfer Learning Improve Virtual Screening and Highlight the Need for More Data. J Chem Inf Model 2018; 58:2319-2330. [DOI: 10.1021/acs.jcim.8b00350] [Citation(s) in RCA: 76] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Affiliation(s)
- Fergus Imrie
- Oxford Protein Informatics Group, Department of Statistics, University of Oxford, Oxford OX1 3LB, U.K
| | - Anthony R. Bradley
- Structural Genomics Consortium, University of Oxford, Oxford OX3 7DQ, U.K
- Department of Chemistry, University of Oxford, Oxford OX1 3TA, U.K
- Diamond Light Source Ltd., Didcot OX11 0DE, U.K
| | - Mihaela van der Schaar
- Department of Engineering, University of Oxford, Oxford OX1 3PJ, U.K
- Alan Turing Institute, London NW1 2DB, U.K
| | - Charlotte M. Deane
- Oxford Protein Informatics Group, Department of Statistics, University of Oxford, Oxford OX1 3LB, U.K
| |
Collapse
|
877
|
Xing F, Xie Y, Su H, Liu F, Yang L. Deep Learning in Microscopy Image Analysis: A Survey. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2018; 29:4550-4568. [PMID: 29989994 DOI: 10.1109/tnnls.2017.2766168] [Citation(s) in RCA: 147] [Impact Index Per Article: 24.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Computerized microscopy image analysis plays an important role in computer aided diagnosis and prognosis. Machine learning techniques have powered many aspects of medical investigation and clinical practice. Recently, deep learning is emerging as a leading machine learning tool in computer vision and has attracted considerable attention in biomedical image analysis. In this paper, we provide a snapshot of this fast-growing field, specifically for microscopy image analysis. We briefly introduce the popular deep neural networks and summarize current deep learning achievements in various tasks, such as detection, segmentation, and classification in microscopy image analysis. In particular, we explain the architectures and the principles of convolutional neural networks, fully convolutional networks, recurrent neural networks, stacked autoencoders, and deep belief networks, and interpret their formulations or modelings for specific tasks on various microscopy images. In addition, we discuss the open challenges and the potential trends of future research in microscopy image analysis using deep learning.
Collapse
|
878
|
Liu F, Zhou Z, Samsonov A, Blankenbaker D, Larison W, Kanarek A, Lian K, Kambhampati S, Kijowski R. Deep Learning Approach for Evaluating Knee MR Images: Achieving High Diagnostic Performance for Cartilage Lesion Detection. Radiology 2018; 289:160-169. [PMID: 30063195 PMCID: PMC6166867 DOI: 10.1148/radiol.2018172986] [Citation(s) in RCA: 151] [Impact Index Per Article: 25.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2017] [Revised: 05/29/2018] [Accepted: 06/01/2018] [Indexed: 12/27/2022]
Abstract
Purpose To determine the feasibility of using a deep learning approach to detect cartilage lesions (including cartilage softening, fibrillation, fissuring, focal defects, diffuse thinning due to cartilage degeneration, and acute cartilage injury) within the knee joint on MR images. Materials and Methods A fully automated deep learning-based cartilage lesion detection system was developed by using segmentation and classification convolutional neural networks (CNNs). Fat-suppressed T2-weighted fast spin-echo MRI data sets of the knee of 175 patients with knee pain were retrospectively analyzed by using the deep learning method. The reference standard for training the CNN classification was the interpretation provided by a fellowship-trained musculoskeletal radiologist of the presence or absence of a cartilage lesion within 17 395 small image patches placed on the articular surfaces of the femur and tibia. Receiver operating curve (ROC) analysis and the κ statistic were used to assess diagnostic performance and intraobserver agreement for detecting cartilage lesions for two individual evaluations performed by the cartilage lesion detection system. Results The sensitivity and specificity of the cartilage lesion detection system at the optimal threshold according to the Youden index were 84.1% and 85.2%, respectively, for evaluation 1 and 80.5% and 87.9%, respectively, for evaluation 2. Areas under the ROC curve were 0.917 and 0.914 for evaluations 1 and 2, respectively, indicating high overall diagnostic accuracy for detecting cartilage lesions. There was good intraobserver agreement between the two individual evaluations, with a κ of 0.76. Conclusion This study demonstrated the feasibility of using a fully automated deep learning-based cartilage lesion detection system to evaluate the articular cartilage of the knee joint with high diagnostic performance and good intraobserver agreement for detecting cartilage degeneration and acute cartilage injury. © RSNA, 2018 Online supplemental material is available for this article .
Collapse
Affiliation(s)
- Fang Liu
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Zhaoye Zhou
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Alexey Samsonov
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Donna Blankenbaker
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Will Larison
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Andrew Kanarek
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Kevin Lian
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Shivkumar Kambhampati
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| | - Richard Kijowski
- From the Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Ave, Madison, WI 53705-2275 (F.L., A.S., D.B., W.L., A.K., K.L., S.K., R.K.); and Department of Biomedical Engineering, University of Minnesota, Minneapolis, Minn (Z.Z.)
| |
Collapse
|
879
|
Improving the performance of convolutional neural network for skin image classification using the response of image analysis filters. Neural Comput Appl 2018. [DOI: 10.1007/s00521-018-3711-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
880
|
Gao F, Wu T, Li J, Zheng B, Ruan L, Shang D, Patel B. SD-CNN: A shallow-deep CNN for improved breast cancer diagnosis. Comput Med Imaging Graph 2018; 70:53-62. [PMID: 30292910 DOI: 10.1016/j.compmedimag.2018.09.004] [Citation(s) in RCA: 73] [Impact Index Per Article: 12.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2018] [Revised: 08/28/2018] [Accepted: 09/13/2018] [Indexed: 10/28/2022]
Abstract
Breast cancer is the second leading cause of cancer death among women worldwide. Nevertheless, it is also one of the most treatable malignances if detected early. Screening for breast cancer with full field digital mammography (FFDM) has been widely used. However, it demonstrates limited performance for women with dense breasts. An emerging technology in the field is contrast-enhanced digital mammography (CEDM), which includes a low energy (LE) image similar to FFDM, and a recombined image leveraging tumor neoangiogenesis similar to breast magnetic resonance imaging (MRI). CEDM has shown better diagnostic accuracy than FFDM. While promising, CEDM is not yet widely available across medical centers. In this research, we propose a Shallow-Deep Convolutional Neural Network (SD-CNN) where a shallow CNN is developed to derive "virtual" recombined images from LE images, and a deep CNN is employed to extract novel features from LE, recombined or "virtual" recombined images for ensemble models to classify the cases as benign vs. cancer. To evaluate the validity of our approach, we first develop a deep-CNN using 49 CEDM cases collected from Mayo Clinic to prove the contributions from recombined images for improved breast cancer diagnosis (0.85 in accuracy, 0.84 in AUC using LE imaging vs. 0.89 in accuracy, 0.91 in AUC using both LE and recombined imaging). We then develop a shallow-CNN using the same 49 CEDM cases to learn the nonlinear mapping from LE to recombined images. Next, we use 89 FFDM cases from INbreast, a public database to generate "virtual" recombined images. Using FFDM alone provides 0.84 in accuracy (AUC = 0.87), whereas SD-CNN improves the diagnostic accuracy to 0.90 (AUC = 0.92).
Collapse
Affiliation(s)
- Fei Gao
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, 85281, USA
| | - Teresa Wu
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, 85281, USA.
| | - Jing Li
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, 85281, USA
| | - Bin Zheng
- School of Electrical and Computer Engineering, College of Engineering, University of Oklahoma, Norman, OK, 73019, USA
| | - Lingxiang Ruan
- The First Affiliated Hospital of Medical School of Zhejiang University, Hangzhou, China
| | - Desheng Shang
- The First Affiliated Hospital of Medical School of Zhejiang University, Hangzhou, China
| | - Bhavika Patel
- Department of Radiology, Mayo Clinic in Arizona, Scottsdale, AZ, 85259, USA
| |
Collapse
|
881
|
Noguchi T, Higa D, Asada T, Kawata Y, Machitori A, Shida Y, Okafuji T, Yokoyama K, Uchiyama F, Tajima T. Artificial intelligence using neural network architecture for radiology (AINNAR): classification of MR imaging sequences. Jpn J Radiol 2018; 36:691-697. [PMID: 30232585 DOI: 10.1007/s11604-018-0779-3] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2018] [Accepted: 09/14/2018] [Indexed: 12/17/2022]
Abstract
PURPOSE The confusion of MRI sequence names could be solved if MR images were automatically identified after image data acquisition. We revealed the ability of deep learning to classify head MRI sequences. MATERIALS AND METHODS Seventy-eight patients with mild cognitive impairment (MCI) having apparently normal head MR images and 78 intracranial hemorrhage (ICH) patients with morphologically deformed head MR images were enrolled. Six imaging protocols were selected to be performed: T2-weighted imaging, fluid attenuated inversion recovery imaging, T2-star-weighted imaging, diffusion-weighted imaging, apparent diffusion coefficient mapping, and source images of time-of-flight magnetic resonance angiography. The proximal first image slices and middle image slices having ambiguous and distinctive contrast patterns, respectively, were classified by two deep learning imaging classifiers, AlexNet and GoogLeNet. RESULTS AlexNet had accuracies of 73.3%, 73.6%, 73.1%, and 60.7% in the middle slices of MCI group, middle slices of ICH group, first slices of MCI group, and first slices of ICH group, while GoogLeNet had accuracies of 100%, 98.1%, 93.1%, and 94.8%, respectively. AlexNet significantly had lower classification ability than GoogLeNet for all datasets. CONCLUSIONS GoogLeNet could judge the types of head MRI sequences with a small amount of training data, irrespective of morphological or contrast conditions.
Collapse
Affiliation(s)
- Tomoyuki Noguchi
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan.
| | - Daichi Higa
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Takashi Asada
- Memory Clinics Ochanomizu, 4th floor, Ochanomizu Igaku Kaikan, 1-5-34, Yushima, Bunkyo-ku, Tokyo, 113-0034, Japan
| | - Yusuke Kawata
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Akihiro Machitori
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Yoshitaka Shida
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Takashi Okafuji
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Kota Yokoyama
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Fumiya Uchiyama
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| | - Tsuyoshi Tajima
- Department of Radiology, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku-ku, Tokyo, 162-8655, Japan
| |
Collapse
|
882
|
Mahmood F, Chen R, Sudarsky S, Yu D, Durr NJ. Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images. Phys Med Biol 2018; 63:185012. [PMID: 30113015 DOI: 10.1088/1361-6560/aada93] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Deep learning has emerged as a powerful artificial intelligence tool to interpret medical images for a growing variety of applications. However, the paucity of medical imaging data with high-quality annotations that is necessary for training such methods ultimately limits their performance. Medical data is challenging to acquire due to privacy issues, shortage of experts available for annotation, limited representation of rare conditions and cost. This problem has previously been addressed by using synthetically generated data. However, networks trained on synthetic data often fail to generalize to real data. Cinematic rendering simulates the propagation and interaction of light passing through tissue models reconstructed from CT data, enabling the generation of photorealistic images. In this paper, we present one of the first applications of cinematic rendering in deep learning, in which we propose to fine-tune synthetic data-driven networks using cinematically rendered CT data for the task of monocular depth estimation in endoscopy. Our experiments demonstrate that: (a) convolutional neural networks (CNNs) trained on synthetic data and fine-tuned on photorealistic cinematically rendered data adapt better to real medical images and demonstrate more robust performance when compared to networks with no fine-tuning, (b) these fine-tuned networks require less training data to converge to an optimal solution, and (c) fine-tuning with data from a variety of photorealistic rendering conditions of the same scene prevents the network from learning patient-specific information and aids in generalizability of the model. Our empirical evaluation demonstrates that networks fine-tuned with cinematically rendered data predict depth with 56.87% less error for rendered endoscopy images and 27.49% less error for real porcine colon endoscopy images.
Collapse
Affiliation(s)
- Faisal Mahmood
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, United States of America
| | | | | | | | | |
Collapse
|
883
|
Couture HD, Williams LA, Geradts J, Nyante SJ, Butler EN, Marron JS, Perou CM, Troester MA, Niethammer M. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 2018; 4:30. [PMID: 30182055 PMCID: PMC6120869 DOI: 10.1038/s41523-018-0079-1] [Citation(s) in RCA: 164] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2017] [Revised: 07/17/2018] [Accepted: 07/23/2018] [Indexed: 11/09/2022] Open
Abstract
RNA-based, multi-gene molecular assays are available and widely used for patients with ER-positive/HER2-negative breast cancers. However, RNA-based genomic tests can be costly and are not available in many countries. Methods for inferring molecular subtype from histologic images may identify patients most likely to benefit from further genomic testing. To identify patients who could benefit from molecular testing based on H&E stained histologic images, we developed an image analysis approach using deep learning. A training set of 571 breast tumors was used to create image-based classifiers for tumor grade, ER status, PAM50 intrinsic subtype, histologic subtype, and risk of recurrence score (ROR-PT). The resulting classifiers were applied to an independent test set (n = 288), and accuracy, sensitivity, and specificity of each was assessed on the test set. Histologic image analysis with deep learning distinguished low-intermediate vs. high tumor grade (82% accuracy), ER status (84% accuracy), Basal-like vs. non-Basal-like (77% accuracy), Ductal vs. Lobular (94% accuracy), and high vs. low-medium ROR-PT score (75% accuracy). Sampling considerations in the training set minimized bias in the test set. Incorrect classification of ER status was significantly more common for Luminal B tumors. These data provide proof of principle that molecular marker status, including a critical clinical biomarker (i.e., ER status), can be predicted with accuracy >75% based on H&E features. Image-based methods could be promising for identifying patients with a greater need for further genomic testing, or in place of classically scored variables typically accomplished using human-based scoring.
Collapse
Affiliation(s)
- Heather D Couture
- 1Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Lindsay A Williams
- 2Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Joseph Geradts
- 3Department of Pathology, Dana-Farber Cancer Institute, Boston, MA 02115 USA
| | - Sarah J Nyante
- 4Department of Radiology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Ebonee N Butler
- 2Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - J S Marron
- 5Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA.,6Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Charles M Perou
- 5Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA.,7Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Melissa A Troester
- 2Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA.,5Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| | - Marc Niethammer
- 1Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA.,8Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA
| |
Collapse
|
884
|
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. DEEP LEARNING IN MEDICAL IMAGE ANALYSIS AND MULTIMODAL LEARNING FOR CLINICAL DECISION SUPPORT : 4TH INTERNATIONAL WORKSHOP, DLMIA 2018, AND 8TH INTERNATIONAL WORKSHOP, ML-CDS 2018, HELD IN CONJUNCTION WITH MICCAI 2018, GRANADA, SPAIN, S... 2018; 11045:3-11. [PMID: 32613207 PMCID: PMC7329239 DOI: 10.1007/978-3-030-00889-5_1] [Citation(s) in RCA: 1109] [Impact Index Per Article: 184.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
Abstract
In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network where the encoder and decoder sub-networks are connected through a series of nested, dense skip pathways. The re-designed skip pathways aim at reducing the semantic gap between the feature maps of the encoder and decoder sub-networks. We argue that the optimizer would deal with an easier learning task when the feature maps from the decoder and encoder networks are semantically similar. We have evaluated UNet++ in comparison with U-Net and wide U-Net architectures across multiple medical image segmentation tasks: nodule segmentation in the low-dose CT scans of chest, nuclei segmentation in the microscopy images, liver segmentation in abdominal CT scans, and polyp segmentation in colonoscopy videos. Our experiments demonstrate that UNet++ with deep supervision achieves an average IoU gain of 3.9 and 3.4 points over U-Net and wide U-Net, respectively.
Collapse
|
885
|
Shin Y, Balasingham I. Automatic polyp frame screening using patch based combined feature and dictionary learning. Comput Med Imaging Graph 2018; 69:33-42. [PMID: 30172091 DOI: 10.1016/j.compmedimag.2018.08.001] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2018] [Revised: 07/29/2018] [Accepted: 08/13/2018] [Indexed: 12/15/2022]
Abstract
Polyps in the colon can potentially become malignant cancer tissues where early detection and removal lead to high survival rate. Certain types of polyps can be difficult to detect even for highly trained physicians. Inspired by aforementioned problem our study aims to improve the human detection performance by developing an automatic polyp screening framework as a decision support tool. We use a small image patch based combined feature method. Features include shape and color information and are extracted using histogram of oriented gradient and hue histogram methods. Dictionary learning based training is used to learn features and final feature vector is formed using sparse coding. For classification, we use patch image classification based on linear support vector machine and whole image thresholding. The proposed framework is evaluated using three public polyp databases. Our experimental results show that the proposed scheme successfully classified polyps and normal images with over 95% of classification accuracy, sensitivity, specificity and precision. In addition, we compare performance of the proposed scheme with conventional feature based methods and the convolutional neural network (CNN) based deep learning approach which is the state of the art technique in many image classification applications.
Collapse
Affiliation(s)
- Younghak Shin
- Department Electronic Systems at Norwegian University of Science and Technology (NTNU), Trondheim, Norway.
| | - Ilangko Balasingham
- Intervention Centre, Oslo University Hospital, Oslo NO-0027, Norway; Institute of Clinical Medicine, University of Oslo, and the Norwegian University of Science and Technology (NTNU), Norway.
| |
Collapse
|
886
|
Tan T, Li Z, Liu H, Zanjani FG, Ouyang Q, Tang Y, Hu Z, Li Q. Optimize Transfer Learning for Lung Diseases in Bronchoscopy Using a New Concept: Sequential Fine-Tuning. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE-JTEHM 2018; 6:1800808. [PMID: 30324036 PMCID: PMC6175035 DOI: 10.1109/jtehm.2018.2865787] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/27/2018] [Revised: 08/01/2018] [Accepted: 08/03/2018] [Indexed: 12/20/2022]
Abstract
Bronchoscopy inspection, as a follow-up procedure next to the radiological imaging, plays a key role in the diagnosis and treatment design for lung disease patients. When performing bronchoscopy, doctors have to make a decision immediately whether to perform a biopsy. Because biopsies may cause uncontrollable and life-threatening bleeding of the lung tissue, thus doctors need to be selective with biopsies. In this paper, to help doctors to be more selective on biopsies and provide a second opinion on diagnosis, we propose a computer-aided diagnosis (CAD) system for lung diseases, including cancers and tuberculosis (TB). Based on transfer learning (TL), we propose a novel TL method on the top of DenseNet: sequential fine-tuning (SFT). Compared with traditional fine-tuning (FT) methods, our method achieves the best performance. In a data set of recruited 81 normal cases, 76 TB cases and 277 lung cancer cases, SFT provided an overall accuracy of 82% while other traditional TL methods achieved an accuracy from 70% to 74%. The detection accuracy of SFT for cancers, TB, and normal cases are 87%, 54%, and 91%, respectively. This indicates that the CAD system has the potential to improve lung disease diagnosis accuracy in bronchoscopy and it may be used to be more selective with biopsies.
Collapse
Affiliation(s)
- Tao Tan
- Department of Biomedical EngineeringEindhoven University of Technology5600 MBEindhovenThe Netherlands.,ScreenPoint Medical6512 ABNijmegenThe Netherlands
| | - Zhang Li
- College of Aerospace Science and EngineeringNational University of Defense TechnologyChangsha410073China
| | - Haixia Liu
- School Of Computer ScienceUniversity of Nottingham Malaysia Campus43500SemenyihMalaysia
| | - Farhad G Zanjani
- Department of Electrical EngineeringEindhoven University of Technology5600 MBEindhovenThe Netherlands
| | - Quchang Ouyang
- Hunan Cancer Hospital, The Affiliated Cancer Hospital of Xiangya School of MedicineCentral South UniversityChangsha410000China
| | - Yuling Tang
- First Hospital of Changsha CityChangsha410000China
| | - Zheyu Hu
- Hunan Cancer Hospital, The Affiliated Cancer Hospital of Xiangya School of MedicineCentral South UniversityChangsha410000China
| | - Qiang Li
- Department of Respiratory MedicineShanghai East HospitalTongji University School of MedicineShanghai200120China
| |
Collapse
|
887
|
Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. Int J Comput Assist Radiol Surg 2018; 13:1895-1903. [PMID: 30094778 PMCID: PMC6223753 DOI: 10.1007/s11548-018-1843-2] [Citation(s) in RCA: 114] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2018] [Accepted: 07/31/2018] [Indexed: 02/06/2023]
Abstract
PURPOSE The nonalcoholic fatty liver disease is the most common liver abnormality. Up to date, liver biopsy is the reference standard for direct liver steatosis quantification in hepatic tissue samples. In this paper we propose a neural network-based approach for nonalcoholic fatty liver disease assessment in ultrasound. METHODS We used the Inception-ResNet-v2 deep convolutional neural network pre-trained on the ImageNet dataset to extract high-level features in liver B-mode ultrasound image sequences. The steatosis level of each liver was graded by wedge biopsy. The proposed approach was compared with the hepatorenal index technique and the gray-level co-occurrence matrix algorithm. After the feature extraction, we applied the support vector machine algorithm to classify images containing fatty liver. Based on liver biopsy, the fatty liver was defined to have more than 5% of hepatocytes with steatosis. Next, we used the features and the Lasso regression method to assess the steatosis level. RESULTS The area under the receiver operating characteristics curve obtained using the proposed approach was equal to 0.977, being higher than the one obtained with the hepatorenal index method, 0.959, and much higher than in the case of the gray-level co-occurrence matrix algorithm, 0.893. For regression the Spearman correlation coefficients between the steatosis level and the proposed approach, the hepatorenal index and the gray-level co-occurrence matrix algorithm were equal to 0.78, 0.80 and 0.39, respectively. CONCLUSIONS The proposed approach may help the sonographers automatically diagnose the amount of fat in the liver. The presented approach is efficient and in comparison with other methods does not require the sonographers to select the region of interest.
Collapse
|
888
|
Exploring transfer learning for gastrointestinal bleeding detection on small-size imbalanced endoscopy images. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2017:1994-1997. [PMID: 29060286 DOI: 10.1109/embc.2017.8037242] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
The success of Convolutional Neural Network (CNN) is attributed to their ability to learn rich midlevel image representations as opposed to hand-crafted low-level features used in many natural image classification methods. Learning CNN, however, amounts to estimating millions of parameters and requires a very large number of annotated image samples. In this paper, we explored transfer learning for gastrointestinal bleeding detection on small-size imbalanced endoscopy images, and showed how image representations learned with CNN on large-scale annotated datasets can be efficiently transferred to other tasks with limited amount of training data. We first transferred pre-trained Inception V3 model trained on the ImageNet dataset to compute mid-level image representation, and then fine-tuned the trained model with labeled endoscopy images, and resumed training from already learned weights. Additionally, we introduce both data augmentation and image resampling to increase the size of the training database and the positive sample rate to perform the Transfer Learning. Our results showed that our transfer learning method produces the best performance on AUC (the area under the receiver operating curve), Precision, Recall and Accuracy as compared to both the hand-crafted feature based method and training CNN model from-scratch method.
Collapse
|
889
|
Jiang J, Liu X, Liu L, Wang S, Long E, Yang H, Yuan F, Yu D, Zhang K, Wang L, Liu Z, Wang D, Xi C, Lin Z, Wu X, Cui J, Zhu M, Lin H. Predicting the progression of ophthalmic disease based on slit-lamp images using a deep temporal sequence network. PLoS One 2018; 13:e0201142. [PMID: 30063738 PMCID: PMC6067742 DOI: 10.1371/journal.pone.0201142] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2018] [Accepted: 06/12/2018] [Indexed: 11/21/2022] Open
Abstract
Ocular images play an essential role in ophthalmology. Current research mainly focuses on computer-aided diagnosis using slit-lamp images, however few studies have been done to predict the progression of ophthalmic disease. Therefore exploring an effective approach of prediction can help to plan treatment strategies and to provide early warning for the patients. In this study, we present an end-to-end temporal sequence network (TempSeq-Net) to automatically predict the progression of ophthalmic disease, which includes employing convolutional neural network (CNN) to extract high-level features from consecutive slit-lamp images and applying long short term memory (LSTM) method to mine the temporal relationship of features. First, we comprehensively compare six potential combinations of CNNs and LSTM (or recurrent neural network) in terms of effectiveness and efficiency, to obtain the optimal TempSeq-Net model. Second, we analyze the impacts of sequence lengths on model's performance which help to evaluate their stability and validity and to determine the appropriate range of sequence lengths. The quantitative results demonstrated that our proposed model offers exceptional performance with mean accuracy (92.22), sensitivity (88.55), specificity (94.31) and AUC (97.18). Moreover, the model achieves real-time prediction with only 27.6ms for single sequence, and simultaneously predicts sequence data with lengths of 3-5. Our study provides a promising strategy for the progression of ophthalmic disease, and has the potential to be applied in other medical fields.
Collapse
Affiliation(s)
- Jiewei Jiang
- School of Computer Science and Technology, Xidian University, Xi’an, China
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Xiyang Liu
- School of Computer Science and Technology, Xidian University, Xi’an, China
- School of Software, Xidian University, Xi’an, China
| | - Lin Liu
- School of Computer Science and Technology, Xidian University, Xi’an, China
| | - Shuai Wang
- School of Software, Xidian University, Xi’an, China
| | - Erping Long
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Haoqing Yang
- School of Computer Science and Technology, Xidian University, Xi’an, China
| | - Fuqiang Yuan
- School of Computer Science and Technology, Xidian University, Xi’an, China
| | - Deying Yu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
- Zhongshan School of Medicine, Sun Yat-sen University, Guangzhou, China
| | - Kai Zhang
- School of Computer Science and Technology, Xidian University, Xi’an, China
| | - Liming Wang
- School of Computer Science and Technology, Xidian University, Xi’an, China
- School of Software, Xidian University, Xi’an, China
| | - Zhenzhen Liu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Dongni Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Changzun Xi
- School of Computer Science and Technology, Xidian University, Xi’an, China
| | - Zhuoling Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Xiaohang Wu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Jiangtao Cui
- School of Computer Science and Technology, Xidian University, Xi’an, China
| | - Mingmin Zhu
- School of Mathematics and Statistics, Xidian University, Xi’an, China
| | - Haotian Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
890
|
Liang S, Zhang R, Liang D, Song T, Ai T, Xia C, Xia L, Wang Y. Multimodal 3D DenseNet for IDH Genotype Prediction in Gliomas. Genes (Basel) 2018; 9:E382. [PMID: 30061525 PMCID: PMC6115744 DOI: 10.3390/genes9080382] [Citation(s) in RCA: 65] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2018] [Revised: 07/06/2018] [Accepted: 07/16/2018] [Indexed: 01/08/2023] Open
Abstract
Non-invasive prediction of isocitrate dehydrogenase (IDH) genotype plays an important role in tumor glioma diagnosis and prognosis. Recently, research has shown that radiology images can be a potential tool for genotype prediction, and fusion of multi-modality data by deep learning methods can further provide complementary information to enhance prediction accuracy. However, it still does not have an effective deep learning architecture to predict IDH genotype with three-dimensional (3D) multimodal medical images. In this paper, we proposed a novel multimodal 3D DenseNet (M3D-DenseNet) model to predict IDH genotypes with multimodal magnetic resonance imaging (MRI) data. To evaluate its performance, we conducted experiments on the BRATS-2017 and The Cancer Genome Atlas breast invasive carcinoma (TCGA-BRCA) dataset to get image data as input and gene mutation information as the target, respectively. We achieved 84.6% accuracy (area under the curve (AUC) = 85.7%) on the validation dataset. To evaluate its generalizability, we applied transfer learning techniques to predict World Health Organization (WHO) grade status, which also achieved a high accuracy of 91.4% (AUC = 94.8%) on validation dataset. With the properties of automatic feature extraction, and effective and high generalizability, M3D-DenseNet can serve as a useful method for other multimodal radiogenomics problems and has the potential to be applied in clinical decision making.
Collapse
Affiliation(s)
- Sen Liang
- Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, and College of Computer Science and Technology, Jilin University, Changchun 130012, China.
| | - Rongguo Zhang
- Advanced Institute, Infervision, Beijing 100000, China.
| | - Dayang Liang
- School of Mechatronics Engineering, Nanchang University, Nanchang 330031, China.
| | - Tianci Song
- Advanced Institute, Infervision, Beijing 100000, China.
| | - Tao Ai
- Department of Radiology, Tongji Hospital, Wuhan 430030, China.
| | - Chen Xia
- Advanced Institute, Infervision, Beijing 100000, China.
| | - Liming Xia
- Department of Radiology, Tongji Hospital, Wuhan 430030, China.
| | - Yan Wang
- Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, and College of Computer Science and Technology, Jilin University, Changchun 130012, China.
- Cancer Systems Biology Center, China-Japan Union Hospital, Jilin University, Changchun 130033, China.
| |
Collapse
|
891
|
Automatic Segmentation and Counting of Aphid Nymphs on Leaves Using Convolutional Neural Networks. AGRONOMY-BASEL 2018. [DOI: 10.3390/agronomy8080129] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The presence of pests is one of the main problems in crop production, and obtaining reliable statistics of pest infestation is essential for pest management. Detection of pests should be automated because human monitoring of pests is time-consuming and error-prone. Aphids are among the most destructive pests in greenhouses and they reproduce quickly. Automatic detection of aphid nymphs on leaves (especially on the lower surface) using image analysis is a challenging problem due to color similarity and complicated background. In this study, we propose a method for segmentation and counting of aphid nymphs on leaves using convolutional neural networks. Digital images of pakchoi leaves at different aphid infestation stages were obtained, and corresponding pixel-level binary mask annotated. In the test, segmentation results by the proposed method achieved high overlap with annotation by human experts (Dice coefficient of 0.8207). Automatic counting based on segmentation showed high precision (0.9563) and recall (0.9650). The correlation between aphid nymph count by the proposed method and manual counting was high (R2 = 0.99). The proposed method is generic and can be applied for other species of pests.
Collapse
|
892
|
Saravanan C, Schumacher V, Brown D, Dunstan R, Galarneau JR, Odin M, Mishra S. Meeting Report: Tissue-based Image Analysis. Toxicol Pathol 2018; 45:983-1003. [PMID: 29162012 DOI: 10.1177/0192623317737468] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Quantitative image analysis (IA) is a rapidly evolving area of digital pathology. Although not a new concept, the quantification of histological features on photomicrographs used to be cumbersome, resource-intensive, and limited to specialists and specialized laboratories. Recent technological advances like highly efficient automated whole slide digitizer (scanner) systems, innovative IA platforms, and the emergence of pathologist-friendly image annotation and analysis systems mean that quantification of features on histological digital images will become increasingly prominent in pathologists' daily professional lives. The added value of quantitative IA in pathology includes confirmation of equivocal findings noted by a pathologist, increasing the sensitivity of feature detection, quantification of signal intensity, and improving efficiency. There is no denying that quantitative IA is part of the future of pathology; however, there are also several potential pitfalls when trying to estimate volumetric features from limited 2-dimensional sections. This continuing education session on quantitative IA offered a broad overview of the field; a hands-on toxicologic pathologist experience with IA principles, tools, and workflows; a discussion on how to apply basic stereology principles in order to minimize bias in IA; and finally, a reflection on the future of IA in the toxicologic pathology field.
Collapse
Affiliation(s)
- Chandra Saravanan
- 1 Translational Medicine: Preclinical Safety, Novartis Institutes for Biomedical Research, Cambridge, Massachusetts, USA
| | - Vanessa Schumacher
- 2 Roche Pharmaceutical Research and Early Development (pRED), Roche Innovation Center Basel, Basel, Switzerland
| | - Danielle Brown
- 3 Charles River Laboratories, Inc., Durham, North Carolina, USA
| | | | - Jean-Rene Galarneau
- 1 Translational Medicine: Preclinical Safety, Novartis Institutes for Biomedical Research, Cambridge, Massachusetts, USA
| | - Marielle Odin
- 2 Roche Pharmaceutical Research and Early Development (pRED), Roche Innovation Center Basel, Basel, Switzerland
| | | |
Collapse
|
893
|
Classification of Computed Tomography Images in Different Slice Positions Using Deep Learning. JOURNAL OF HEALTHCARE ENGINEERING 2018; 2018:1753480. [PMID: 30123439 PMCID: PMC6079460 DOI: 10.1155/2018/1753480] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/21/2018] [Accepted: 05/22/2018] [Indexed: 12/14/2022]
Abstract
This study aimed at elucidating the relationship between the number of computed tomography (CT) images, including data concerning the accuracy of models and contrast enhancement for classifying the images. We enrolled 1539 patients who underwent contrast or noncontrast CT imaging, followed by dividing the CT imaging dataset for creating classification models into 10 classes for brain, neck, chest, abdomen, and pelvis with contrast-enhanced and plain imaging. The number of images prepared in each class were 100, 500, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, and 10,000. Accordingly, the names of datasets were defined as 0.1K, 0.5K, 1K, 2K, 3K, 4K, 5K, 6K, 7K, 8K, 9K, and 10K, respectively. We subsequently created and evaluated the models and compared the convolutional neural network (CNN) architecture between AlexNet and GoogLeNet. The time required for training models of AlexNet was lesser than that for GoogLeNet. The best overall accuracy for the classification of 10 classes was 0.721 with the 10K dataset of GoogLeNet. Furthermore, the best overall accuracy for the classification of the slice position without contrast media was 0.862 with the 2K dataset of AlexNet.
Collapse
|
894
|
|
895
|
Izadyyazdanabadi M, Belykh E, Mooney MA, Eschbacher JM, Nakaji P, Yang Y, Preul MC. Prospects for Theranostics in Neurosurgical Imaging: Empowering Confocal Laser Endomicroscopy Diagnostics via Deep Learning. Front Oncol 2018; 8:240. [PMID: 30035099 PMCID: PMC6043805 DOI: 10.3389/fonc.2018.00240] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2018] [Accepted: 06/18/2018] [Indexed: 12/12/2022] Open
Abstract
Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence imaging technology that has potential to increase intraoperative precision, extend resection, and tailor surgery for malignant invasive brain tumors because of its subcellular dimension resolution. Despite its promising diagnostic potential, interpreting the gray tone fluorescence images can be difficult for untrained users. CLE images can be distorted by motion artifacts, fluorescence signals out of detector dynamic range, or may be obscured by red blood cells, and thus interpreted as nondiagnostic (ND). However, just a single CLE image with a detectable pathognomonic histological tissue signature can suffice for intraoperative diagnosis. Dealing with the abundance of images from CLE is not unlike sifting through a myriad of genes, proteins, or other structural or metabolic markers to find something of commonality or uniqueness in cancer that might indicate a potential treatment scheme or target. In this review, we provide a detailed description of bioinformatical analysis methodology of CLE images that begins to assist the neurosurgeon and pathologist to rapidly connect on-the-fly intraoperative imaging, pathology, and surgical observation into a conclusionary system within the concept of theranostics. We present an overview and discuss deep learning models for automatic detection of the diagnostic CLE images and discuss various training regimes and ensemble modeling effect on power of deep learning predictive models. Two major approaches reviewed in this paper include the models that can automatically classify CLE images into diagnostic/ND, glioma/nonglioma, tumor/injury/normal categories, and models that can localize histological features on the CLE images using weakly supervised methods. We also briefly review advances in the deep learning approaches used for CLE image analysis in other organs. Significant advances in speed and precision of automated diagnostic frame selection would augment the diagnostic potential of CLE, improve operative workflow, and integration into brain tumor surgery. Such technology and bioinformatics analytics lend themselves to improved precision, personalization, and theranostics in brain tumor treatment.
Collapse
Affiliation(s)
- Mohammadhassan Izadyyazdanabadi
- Active Perception Group, School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, United States.,Neurosurgery Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States
| | - Evgenii Belykh
- Neurosurgery Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States.,Department of Neurosurgery, Irkutsk State Medical University, Irkutsk, Russia
| | - Michael A Mooney
- Neurosurgery Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States
| | - Jennifer M Eschbacher
- Neurosurgery Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States
| | - Peter Nakaji
- Neurosurgery Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States
| | - Yezhou Yang
- Active Perception Group, School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, United States
| | - Mark C Preul
- Neurosurgery Research Laboratory, Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, United States
| |
Collapse
|
896
|
Zheng Y, Zhang R, Yu R, Jiang Y, Mak TWC, Wong SH, Lau JYW, Poon CCY. Localisation of Colorectal Polyps by Convolutional Neural Network Features Learnt from White Light and Narrow Band Endoscopic Images of Multiple Databases. 2018 40TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC) 2018; 2018:4142-4145. [PMID: 30441267 DOI: 10.1109/embc.2018.8513337] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
|
897
|
Akbari M, Mohrekesh M, Rafiei S, Reza Soroushmehr SM, Karimi N, Samavi S, Najarian K. Classification of Informative Frames in Colonoscopy Videos Using Convolutional Neural Networks with Binarized Weights. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2018:65-68. [PMID: 30440342 DOI: 10.1109/embc.2018.8512226] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Colorectal cancer is one of the common cancers in the United States. Polyps are one of the major causes of colonic cancer, and early detection of polyps will increase the chance of cancer treatments. In this paper, we propose a novel classification of informative frames based on a convolutional neural network with binarized weights. The proposed CNN is trained with colonoscopy frames along with the labels of the frames as input data. We also used binarized weights and kernels to reduce the size of CNN and make it suitable for implementation in medical hardware. We evaluate our proposed method using Asu Mayo Test clinic database, which contains colonoscopy videos of different patients. Our proposed method reaches a dice score of 71.20% and accuracy of more than 90% using the mentioned dataset.
Collapse
|
898
|
Wang G, Li W, Zuluaga MA, Pratt R, Patel PA, Aertsen M, Doel T, David AL, Deprest J, Ourselin S, Vercauteren T. Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:1562-1573. [PMID: 29969407 PMCID: PMC6051485 DOI: 10.1109/tmi.2018.2791721] [Citation(s) in RCA: 265] [Impact Index Per Article: 44.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/11/2017] [Revised: 01/04/2018] [Accepted: 01/05/2018] [Indexed: 05/22/2023]
Abstract
Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes (a.k.a. zero-shot learning). To address these problems, we propose a novel deep learning-based interactive segmentation framework by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine tuning. We applied this framework to two applications: 2-D segmentation of multiple organs from fetal magnetic resonance (MR) slices, where only two types of these organs were annotated for training and 3-D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only the tumor core in one MR sequence was annotated for training. Experimental results show that: 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.
Collapse
|
899
|
Devalla SK, Renukanand PK, Sreedhar BK, Subramanian G, Zhang L, Perera S, Mari JM, Chin KS, Tun TA, Strouthidis NG, Aung T, Thiéry AH, Girard MJA. DRUNET: a dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images. BIOMEDICAL OPTICS EXPRESS 2018; 9:3244-3265. [PMID: 29984096 PMCID: PMC6033560 DOI: 10.1364/boe.9.003244] [Citation(s) in RCA: 90] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2018] [Revised: 06/06/2018] [Accepted: 06/11/2018] [Indexed: 05/18/2023]
Abstract
Given that the neural and connective tissues of the optic nerve head (ONH) exhibit complex morphological changes with the development and progression of glaucoma, their simultaneous isolation from optical coherence tomography (OCT) images may be of great interest for the clinical diagnosis and management of this pathology. A deep learning algorithm (custom U-NET) was designed and trained to segment 6 ONH tissue layers by capturing both the local (tissue texture) and contextual information (spatial arrangement of tissues). The overall Dice coefficient (mean of all tissues) was 0.91 ± 0.05 when assessed against manual segmentations performed by an expert observer. Further, we automatically extracted six clinically relevant neural and connective tissue structural parameters from the segmented tissues. We offer here a robust segmentation framework that could also be extended to the 3D segmentation of the ONH tissues.
Collapse
Affiliation(s)
- Sripad Krishna Devalla
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
| | - Prajwal K Renukanand
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
| | - Bharathwaj K Sreedhar
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
| | - Giridhar Subramanian
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
| | - Liang Zhang
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
| | - Shamira Perera
- Duke-NUS, Graduate Medical School, Singapore
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
| | - Jean-Martial Mari
- GePaSud, Université de la Polynésie française, Tahiti, French Polynesia
| | - Khai Sing Chin
- Department of Statistics and Applied Probability, National University of Singapore, Singapore
| | - Tin A Tun
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
| | - Nicholas G Strouthidis
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
- NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
- Discipline of Clinical Ophthalmology and Eye Health, University of Sydney, Sydney, New South Wales, Australia
| | - Tin Aung
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
| | - Alexandre H Thiéry
- Department of Statistics and Applied Probability, National University of Singapore, Singapore
| | - Michaël J A Girard
- Ophthalmic Engineering & Innovation Laboratory, Department of Biomedical Engineering, Faculty of Engineering, National University of Singapore, Singapore
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
| |
Collapse
|
900
|
Mednikov Y, Nehemia S, Zheng B, Benzaquen O, Lederman D. Transfer Representation Learning using Inception-V3 for the Detection of Masses in Mammography. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2018:2587-2590. [PMID: 30440937 DOI: 10.1109/embc.2018.8512750] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Breast cancer is the most prevalent cancer among women. The most common method to detect breast cancer is mammography. However, interpreting mammography is a challenging task that requires high skills and is timeconsuming. In this work, we propose a computer-aided diagnosis (CAD) scheme for mammography based on transfer representation learning using the Inception-V3 architecture. We evaluate the performance of the proposed scheme using the INBreast database, where the features are extracted from different layers of the architecture. In order to cope with the small dataset size limitation, we expand the training dataset by generating artificial mammograms and employing different augmentation techniques. The proposed scheme shows great potential with a maximal area under the receiver operating characteristics curve of 0.91.
Collapse
|