1
|
Wan Y, Zheng Z, Liu R, Zhu Z, Zhou H, Zhang X, Boumaraf S. A Multi-Scale and Multi-Level Fusion Approach for Deep Learning-Based Liver Lesion Diagnosis in Magnetic Resonance Images with Visual Explanation. Life (Basel) 2021; 11:life11060582. [PMID: 34207262 PMCID: PMC8234101 DOI: 10.3390/life11060582] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2021] [Revised: 06/10/2021] [Accepted: 06/16/2021] [Indexed: 02/08/2023] Open
Abstract
Many computer-aided diagnosis methods, especially ones with deep learning strategies, of liver cancers based on medical images have been proposed. However, most of such methods analyze the images under only one scale, and the deep learning models are always unexplainable. In this paper, we propose a deep learning-based multi-scale and multi-level fusing approach of CNNs for liver lesion diagnosis on magnetic resonance images, termed as MMF-CNN. We introduce a multi-scale representation strategy to encode both the local and semi-local complementary information of the images. To take advantage of the complementary information of multi-scale representations, we propose a multi-level fusion method to combine the information of both the feature level and the decision level hierarchically and generate a robust diagnostic classifier based on deep learning. We further explore the explanation of the diagnosis decision of the deep neural network through visualizing the areas of interest of the network. A new scoring method is designed to evaluate whether the attention maps can highlight the relevant radiological features. The explanation and visualization make the decision-making process of the deep neural network transparent for the clinicians. We apply our proposed approach to various state-of-the-art deep learning architectures. The experimental results demonstrate the effectiveness of our approach.
Collapse
Affiliation(s)
- Yuchai Wan
- Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing 100048, China; (H.Z.); (X.Z.)
- Correspondence: (Y.W.); (Z.Z.)
| | - Zhongshu Zheng
- Beijing Lab of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing 100081, China;
| | - Ran Liu
- China South-to-North Water Diversion Corporation Limited, Beijing 100038, China;
| | - Zheng Zhu
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, 17, Panjiayuan NanLi, Chaoyang District, Beijing 100021, China
- Correspondence: (Y.W.); (Z.Z.)
| | - Hongen Zhou
- Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing 100048, China; (H.Z.); (X.Z.)
| | - Xun Zhang
- Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing 100048, China; (H.Z.); (X.Z.)
| | - Said Boumaraf
- Centre d’Exploitation des Systèmes de Télécommunications Spatiales (CESTS), Agence Spatiale Algérienne, Algiers, Algeria;
| |
Collapse
|
2
|
GPU-Accelerated GLRLM Algorithm for Feature Extraction of MRI. Sci Rep 2019; 9:10883. [PMID: 31350428 PMCID: PMC6659663 DOI: 10.1038/s41598-019-46622-w] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2019] [Accepted: 06/29/2019] [Indexed: 02/02/2023] Open
Abstract
The gray level run length matrix (GLRLM) whose entries are statistics recording distribution and relationship of images pixels is a widely used method for extracting statistical features for medical images, e.g., magnetic resonance (MR) images. Recently these features are usually employed in some artificial neural networks to identify and distinguish texture patterns. But GLRLM construction and features extraction are tedious and computationally intensive while the images are too big with high resolution, or there are too many small or intermediate Regions of Interest (ROI) to process in a single image, which makes the preprocess a time consuming stage. Hence, it is of great importance to accelerate the procedure which is nowadays possible with the rapid development of massively parallel Graphics Processing Unit, i.e. the GPU computing technology. In this article, we propose a new paradigm based on mature parallel primitives for generating GLRLMs and extracting multiple features for many ROIs simultaneously in a single image. Experiments show that such a paradigm is easy to implement and offers an acceleration over 5 fold increase in speed than an optimized serial counterpart.
Collapse
|
3
|
Perez-Moreno A, Dominguez M, Migliorelli F, Gratacos E, Palacio M, Bonet-Carne E. Clinical Feasibility of Quantitative Ultrasound Texture Analysis: A Robustness Study Using Fetal Lung Ultrasound Images. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2019; 38:1459-1476. [PMID: 30269384 DOI: 10.1002/jum.14824] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/14/2018] [Accepted: 08/20/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVES To compare the robustness of several methods based on quantitative ultrasound (US) texture analysis to evaluate its feasibility for extracting features from US images to use as a clinical diagnostic tool. METHODS We compared, ranked, and validated the robustness of 5 texture-based methods for extracting textural features from US images acquired under different conditions. For comparison and ranking purposes, we used 13,171 non-US images from widely known available databases (OUTEX [University of Oulu, Oulu, Finland] and PHOTEX [Texture Lab, Heriot-Watt University, Edinburgh, Scotland]), which were specifically acquired under different controlled parameters (illumination, resolution, and rotation) from 103 textures. The robustness of those methods with better results from the non-US images was validated by using 666 fetal lung US images acquired from singleton pregnancies. In this study, 2 similarity measurements (correlation and Chebyshev distances) were used to evaluate the repeatability of the features extracted from the same tissue images. RESULTS Three of the 5 methods (gray-level co-occurrence matrix, local binary patterns, and rotation-invariant local phase quantization) had favorably robust performance when using the non-US database. In fact, these methods showed similarity values close to 0 for the acquisition variations and delineations. Results from the US database confirmed robustness for all of the evaluated methods (gray-level co-occurrence matrix, local binary patterns, and rotation-invariant local phase quantization) when comparing the same texture obtained from different regions of the image (proximal/distal lungs and US machine brand stratification). CONCLUSIONS Our results confirmed that texture analysis can be robust (high similarity for different condition acquisitions) with potential to be included as a clinical tool.
Collapse
Affiliation(s)
- Alvaro Perez-Moreno
- Transmural Biotech SL, Barcelona, Spain
- Fetal I + D Fetal Medicine Research Center, Institut d'Investigacions Biomèdiques August Pi i Sunyer, University of Barcelona, Barcelona, Spain
| | | | - Federico Migliorelli
- Fetal I + D Fetal Medicine Research Center, Institut d'Investigacions Biomèdiques August Pi i Sunyer, University of Barcelona, Barcelona, Spain
- BCNatal, Barcelona Center for Maternal-Fetal and Neonatal Medicine, Hospital Clínic and Hospital Sant Joan de Deu, Barcelona, Spain
| | - Eduard Gratacos
- Fetal I + D Fetal Medicine Research Center, Institut d'Investigacions Biomèdiques August Pi i Sunyer, University of Barcelona, Barcelona, Spain
- BCNatal, Barcelona Center for Maternal-Fetal and Neonatal Medicine, Hospital Clínic and Hospital Sant Joan de Deu, Barcelona, Spain
- Center for Biomedical Research on Rare Diseases, Barcelona, Spain
| | - Montse Palacio
- Fetal I + D Fetal Medicine Research Center, Institut d'Investigacions Biomèdiques August Pi i Sunyer, University of Barcelona, Barcelona, Spain
- BCNatal, Barcelona Center for Maternal-Fetal and Neonatal Medicine, Hospital Clínic and Hospital Sant Joan de Deu, Barcelona, Spain
- Center for Biomedical Research on Rare Diseases, Barcelona, Spain
| | - Elisenda Bonet-Carne
- Transmural Biotech SL, Barcelona, Spain
- University College London, Microstructure Imaging Group, Center for Medical Image Computing, London, England
| |
Collapse
|
4
|
Kaur P, Singh G, Kaur P. An intelligent validation system for diagnostic and prognosis of ultrasound fetal growth analysis using Neuro-Fuzzy based on genetic algorithm. EGYPTIAN INFORMATICS JOURNAL 2019. [DOI: 10.1016/j.eij.2018.10.002] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
5
|
Classification of Liver Diseases Based on Ultrasound Image Texture Features. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9020342] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
This paper discusses using computer-aided diagnosis (CAD) to distinguish between hepatocellular carcinoma (HCC), i.e., the most common type of primary liver malignancy and a leading cause of death in people with cirrhosis worldwide, and liver abscess based on ultrasound image texture features and a support vector machine (SVM) classifier. Among 79 cases of liver diseases including 44 cases of liver cancer and 35 cases of liver abscess, this research extracts 96 features including 52 features of the gray-level co-occurrence matrix (GLCM) and 44 features of the gray-level run-length matrix (GLRLM) from the regions of interest (ROIs) in ultrasound images. Three feature selection models—(i) sequential forward selection (SFS), (ii) sequential backward selection (SBS), and (iii) F-score—are adopted to distinguish the two liver diseases. Finally, the developed system can classify liver cancer and liver abscess by SVM with an accuracy of 88.875%. The proposed methods for CAD can provide diagnostic assistance while distinguishing these two types of liver lesions.
Collapse
|
6
|
Faust O, Acharya UR, Meiburger KM, Molinari F, Koh JE, Yeong CH, Kongmebhol P, Ng KH. Comparative assessment of texture features for the identification of cancer in ultrasound images: a review. Biocybern Biomed Eng 2018. [DOI: 10.1016/j.bbe.2018.01.001] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
7
|
Parekh V, Jacobs MA. Radiomics: a new application from established techniques. EXPERT REVIEW OF PRECISION MEDICINE AND DRUG DEVELOPMENT 2016; 1:207-226. [PMID: 28042608 PMCID: PMC5193485 DOI: 10.1080/23808993.2016.1164013] [Citation(s) in RCA: 217] [Impact Index Per Article: 27.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
The increasing use of biomarkers in cancer have led to the concept of personalized medicine for patients. Personalized medicine provides better diagnosis and treatment options available to clinicians. Radiological imaging techniques provide an opportunity to deliver unique data on different types of tissue. However, obtaining useful information from all radiological data is challenging in the era of "big data". Recent advances in computational power and the use of genomics have generated a new area of research termed Radiomics. Radiomics is defined as the high throughput extraction of quantitative imaging features or texture (radiomics) from imaging to decode tissue pathology and creating a high dimensional data set for feature extraction. Radiomic features provide information about the gray-scale patterns, inter-pixel relationships. In addition, shape and spectral properties can be extracted within the same regions of interest on radiological images. Moreover, these features can be further used to develop computational models using advanced machine learning algorithms that may serve as a tool for personalized diagnosis and treatment guidance.
Collapse
Affiliation(s)
- Vishwa Parekh
- The Russell H. Morgan Department of Radiology and Radiological Science, Division of Cancer Imaging, The Johns Hopkins University School of Medicine, Baltimore, MD 21205
- Department of Computer Science, The Johns Hopkins University School of Medicine, Baltimore, MD 21205
| | - Michael A. Jacobs
- The Russell H. Morgan Department of Radiology and Radiological Science, Division of Cancer Imaging, The Johns Hopkins University School of Medicine, Baltimore, MD 21205
- Sidney Kimmel Comprehensive Cancer Center, The Johns Hopkins University School of Medicine, Baltimore, MD 21205
| |
Collapse
|
8
|
Wang L, Sun DW, Pu H, Zhu Z. Application of Hyperspectral Imaging to Discriminate the Variety of Maize Seeds. FOOD ANAL METHOD 2015. [DOI: 10.1007/s12161-015-0160-4] [Citation(s) in RCA: 48] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
9
|
Artificial neural network application in the diagnosis of disease conditions with liver ultrasound images. Adv Bioinformatics 2014; 2014:708279. [PMID: 25332717 PMCID: PMC4181903 DOI: 10.1155/2014/708279] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2014] [Revised: 08/06/2014] [Accepted: 08/07/2014] [Indexed: 11/17/2022] Open
Abstract
The preliminary study presented within this paper shows a comparative study of various texture features extracted from liver ultrasonic images by employing Multilayer Perceptron (MLP), a type of artificial neural network, to study the presence of disease conditions. An ultrasound (US) image shows echo-texture patterns, which defines the organ characteristics. Ultrasound images of liver disease conditions such as "fatty liver," "cirrhosis," and "hepatomegaly" produce distinctive echo patterns. However, various ultrasound imaging artifacts and speckle noise make these echo-texture patterns difficult to identify and often hard to distinguish visually. Here, based on the extracted features from the ultrasonic images, we employed an artificial neural network for the diagnosis of disease conditions in liver and finding of the best classifier that distinguishes between abnormal and normal conditions of the liver. Comparison of the overall performance of all the feature classifiers concluded that "mixed feature set" is the best feature set. It showed an excellent rate of accuracy for the training data set. The gray level run length matrix (GLRLM) feature shows better results when the network was tested against unknown data.
Collapse
|
10
|
SVM-based characterization of liver ultrasound images using wavelet packet texture descriptors. J Digit Imaging 2014; 26:530-43. [PMID: 23065124 DOI: 10.1007/s10278-012-9537-8] [Citation(s) in RCA: 72] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022] Open
Abstract
A system to characterize normal liver, cirrhotic liver and hepatocellular carcinoma (HCC) evolved on cirrhotic liver is proposed in this paper. The study is performed with 56 real ultrasound images (15 normal, 16 cirrhotic and 25 HCC liver images) taken from 56 subjects. A total of 180 nonoverlapping regions of interest (ROIs), i.e. 60 from each image class, are extracted by an experienced participating radiologist. The multiresolution wavelet packet texture descriptors, i.e. mean, standard deviation and energy features, are computed from all 180 ROIs by using various compact support wavelet filters including Haar, Daubechies (db4 and db6), biorthogonal (bior3.1,bior3.3 and bior4.4), symlets (sym3 and sym5) and coiflets (coif1 and coif2). It is observed that a combined texture descriptor feature vector of length 48 consisting of 16 mean, 16 standard deviation and 16 energy features estimated from all 16 subband feature images (wavelet packets) obtained by second-level decomposition with two-dimensional wavelet packet transform by using Haar wavelet filter gives the best characterization performance of 86.6 %. Feature selection by genetic algorithm-support vector machine method increased the classification accuracy to 88.8 % with sensitivity of 90 % for detecting normal and cirrhotic cases and sensitivity of 86.6 % for HCC cases. Considering limited sensitivity of B-mode ultrasound for detecting HCCs evolved on cirrhotic liver, the sensitivity of 86.6 % for HCC lesions obtained by the proposed computer-aided diagnostic system is quite promising and suggests that the proposed system can be used in a clinical environment to support radiologists in lesion interpretation.
Collapse
|
11
|
Andrade A, Silva JS, Santos J, Belo-Soares P. Classifier Approaches for Liver Steatosis using Ultrasound Images. ACTA ACUST UNITED AC 2012. [DOI: 10.1016/j.protcy.2012.09.084] [Citation(s) in RCA: 47] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
12
|
Mittal D, Kumar V, Saxena SC, Khandelwal N, Kalra N. Neural network based focal liver lesion diagnosis using ultrasound images. Comput Med Imaging Graph 2011; 35:315-23. [PMID: 21334176 DOI: 10.1016/j.compmedimag.2011.01.007] [Citation(s) in RCA: 54] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2010] [Revised: 12/21/2010] [Accepted: 01/24/2011] [Indexed: 12/18/2022]
|