1
|
Moreno Tarazona A, Bautista LX, Martínez F. Cardiac disease discrimination from 3D-convolutional kinematic patterns on cine-MRI sequences. BIOMEDICA : REVISTA DEL INSTITUTO NACIONAL DE SALUD 2024; 44:89-100. [PMID: 39079140 DOI: 10.7705/biomedica.7115] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Accepted: 02/13/2024] [Indexed: 08/04/2024]
Abstract
INTRODUCTION Cine-MRI (cine-magnetic resonance imaging) sequences are a key diagnostic tool to visualize anatomical information, allowing experts to localize and determine suspicious pathologies. Nonetheless, such analysis remains subjective and prone to diagnosis errors. OBJECTIVE To develop a binary and multi-class classification considering various cardiac conditions using a spatiotemporal model that highlights kinematic movements to characterize each disease. MATERIALS AND METHODS This research focuses on a 3D convolutional representation to characterize cardiac kinematic patterns during the cardiac cycle, which may be associated with pathologies. The kinematic maps are obtained from the apparent velocity maps computed from a dense optical flow strategy. Then, a 3D convolutional scheme learns to differentiate pathologies from kinematic maps. RESULTS The proposed strategy was validated with respect to the capability to discriminate among myocardial infarction, dilated cardiomyopathy, hypertrophic cardiomyopathy, abnormal right ventricle, and normal cardiac sequences. The proposed method achieves an average accuracy of 78.00% and a F1 score of 75.55%. Likewise, the approach achieved 92.31% accuracy for binary classification between pathologies and control cases. CONCLUSION The proposed method can support the identification of kinematically abnormal patterns associated with a pathological condition. The resultant descriptor, learned from the 3D convolutional net, preserves detailed spatiotemporal correlations and could emerge as possible digital biomarkers of cardiac diseases.
Collapse
Affiliation(s)
- Alejandra Moreno Tarazona
- Biomedical Imaging, Vision, and Learning Laboratory (BIVL2ab), Universidad Industrial de Santander, Bucaramanga, Colombia
| | - Lola Xiomara Bautista
- Biomedical Imaging, Vision, and Learning Laboratory (BIVL2ab), Universidad Industrial de Santander, Bucaramanga, Colombia
| | - Fabio Martínez
- Biomedical Imaging, Vision, and Learning Laboratory (BIVL2ab), Universidad Industrial de Santander, Bucaramanga, Colombia
| |
Collapse
|
2
|
Li D, Peng Y, Sun J, Guo Y. A task-unified network with transformer and spatial-temporal convolution for left ventricular quantification. Sci Rep 2023; 13:13529. [PMID: 37598235 PMCID: PMC10439898 DOI: 10.1038/s41598-023-40841-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 08/17/2023] [Indexed: 08/21/2023] Open
Abstract
Quantification of the cardiac function is vital for diagnosing and curing the cardiovascular diseases. Left ventricular function measurement is the most commonly used measure to evaluate the function of cardiac in clinical practice, how to improve the accuracy of left ventricular quantitative assessment results has always been the subject of research by medical researchers. Although considerable efforts have been put forward to measure the left ventricle (LV) automatically using deep learning methods, the accurate quantification is yet a challenge work as a result of the changeable anatomy structure of heart in the systolic diastolic cycle. Besides, most methods used direct regression method which lacks of visual based analysis. In this work, a deep learning segmentation and regression task-unified network with transformer and spatial-temporal convolution is proposed to segment and quantify the LV simultaneously. The segmentation module leverages a U-Net like 3D Transformer model to predict the contour of three anatomy structures, while the regression module learns spatial-temporal representations from the original images and the reconstruct feature map from segmentation path to estimate the finally desired quantification metrics. Furthermore, we employ a joint task loss function to train the two module networks. Our framework is evaluated on the MICCAI 2017 Left Ventricle Full Quantification Challenge dataset. The results of experiments demonstrate the effectiveness of our framework, which achieves competitive cardiac quantification metric results and at the same time produces visualized segmentation results that are conducive to later analysis.
Collapse
Affiliation(s)
- Dapeng Li
- Shandong University of Science and Technology, Qingdao, China
| | - Yanjun Peng
- Shandong University of Science and Technology, Qingdao, China.
- Shandong Province Key Laboratory of Wisdom Mining Information Technology, Qingdao, China.
| | - Jindong Sun
- Shandong University of Science and Technology, Qingdao, China
| | - Yanfei Guo
- Shandong University of Science and Technology, Qingdao, China
| |
Collapse
|
3
|
Wei H, Ma J, Zhou Y, Xue W, Ni D. Co-learning of appearance and shape for precise ejection fraction estimation from echocardiographic sequences. Med Image Anal 2023; 84:102686. [PMID: 36455332 DOI: 10.1016/j.media.2022.102686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 10/31/2022] [Accepted: 11/07/2022] [Indexed: 11/17/2022]
Abstract
Accurate estimation of ejection fraction (EF) from echocardiography is of great importance for evaluation of cardiac function. It is usually obtained by the Simpson's bi-plane method based on the segmentation of the left ventricle (LV) in two keyframes. However, obtaining accurate EF estimation from echocardiography is challenging due to (1) noisy appearance in ultrasound images, (2) temporal dynamic movement of myocardium, (3) sparse annotation of the full sequence, and (4) potential quality degradation during scanning. In this paper, we propose a multi-task semi-supervised framework, which is denoted as MCLAS, for precise EF estimation from echocardiographic sequences of two cardiac views. Specifically, we first propose a co-learning mechanism to explore the mutual benefits of cardiac segmentation and myocardium tracking iteratively on appearance level and shape level, therefore alleviating the noisy appearance and enforcing the temporal consistency of the segmentation results. This temporal consistency, as shown in our work, is critical for precise EF estimation. Then we propose two auxiliary tasks for the encoder, (1) view classification to help extract the discriminative features of each view, and automatize the whole pipeline of EF estimation in clinical practice, and (2) EF regression to help regularize the spatiotemporal embedding of the echocardiographic sequence. Both two auxiliary tasks can improve the segmentation-based EF prediction, especially for sequences of poor quality. Our method is capable of automating the whole pipeline of EF estimation, from view identification, cardiac structures segmentation to EF calculation. The effectiveness of our method is validated in aspects of segmentation, tracking, consistency analysis, and clinical parameters estimation. When compared with existing methods, our method shows obvious superiority for LV volumes on ED and ES phases, and EF estimation, with Pearson correlation of 0.975, 0.983 and 0.946, respectively. This is a significant improvement for echocardiography-based EF estimation and improves the potential of automated EF estimation in clinical practice. Besides, our method can obtain accurate and temporal-consistent segmentation for the in-between frames, which enables it for cardiac dynamic function evaluation.
Collapse
Affiliation(s)
- Hongrong Wei
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China
| | - Junqiang Ma
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China
| | - Yongjin Zhou
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China
| | - Wufeng Xue
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China.
| | - Dong Ni
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China.
| |
Collapse
|
4
|
Spatio-Temporal Knowledge Graph Based Forest Fire Prediction with Multi Source Heterogeneous Data. REMOTE SENSING 2022. [DOI: 10.3390/rs14143496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Forest fires have frequently occurred and caused great harm to people’s lives. Many researchers use machine learning techniques to predict forest fires by considering spatio-temporal data features. However, it is difficult to efficiently obtain the features from large-scale, multi-source, heterogeneous data. There is a lack of a method that can effectively extract features required by machine learning-based forest fire predictions from multi-source spatio-temporal data. This paper proposes a forest fire prediction method that integrates spatio-temporal knowledge graphs and machine learning models. This method can fuse multi-source heterogeneous spatio-temporal forest fire data by constructing a forest fire semantic ontology and a knowledge graph-based spatio-temporal framework. This paper defines the domain expertise of forest fire analysis as the semantic rules of the knowledge graph. This paper proposes a rule-based reasoning method to obtain the corresponding data for the specific machine learning-based forest fire prediction methods, which are dedicated to tackling the problem with real-time prediction scenarios. This paper performs experiments regarding forest fire predictions based on real-world data in the experimental areas Xichang and Yanyuan in Sichuan province. The results show that the proposed method is beneficial for the fusion of multi-source spatio-temporal data and highly improves the prediction performance in real forest fire prediction scenarios.
Collapse
|
5
|
Rong Y, Jiang Z, Wu W, Chen Q, Wei C, Fan Z, Chen H. Direct Estimation of Choroidal Thickness in Optical Coherence Tomography Images with Convolutional Neural Networks. J Clin Med 2022; 11:3203. [PMID: 35683590 PMCID: PMC9181751 DOI: 10.3390/jcm11113203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 05/24/2022] [Accepted: 06/02/2022] [Indexed: 11/17/2022] Open
Abstract
Automatic and accurate estimation of choroidal thickness plays a very important role in a computer-aided system for eye diseases. One of the most common methods for automatic estimation of choroidal thickness is segmentation-based methods, in which the boundaries of the choroid are first detected from optical coherence tomography (OCT) images. The choroidal thickness is then computed based on the detected boundaries. A shortcoming in the segmentation-based methods is that the estimating precision greatly depends on the segmentation results. To avoid the dependence on the segmentation step, in this paper, we propose a direct method based on convolutional neural networks (CNNs) for estimating choroidal thickness without segmentation. Concretely, a B-scan image is first cropped into several patches. A trained CNN model is then used to estimate the choroidal thickness for each patch. The mean thickness of the choroid in the B-scan is obtained by taking the average of the choroidal thickness on each patch. Then, 150 OCT volumes are collected to evaluate the proposed method. The experiments show that the results obtained by the proposed method are very competitive with those obtained by segmentation-based methods, which indicates that direct estimation of choroidal thickness is very promising.
Collapse
Affiliation(s)
- Yibiao Rong
- College of Engineering, Shantou University, Shantou 515063, China
- Key Laboratory of Digital Signal and Image Processing of Guangdong Provincial, Shantou University, Shantou 515063, China
| | - Zehua Jiang
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, Shantou 515051, China
- Medical College, Shantou University, Shantou 515063, China
| | - Weihang Wu
- College of Engineering, Shantou University, Shantou 515063, China
- Key Laboratory of Digital Signal and Image Processing of Guangdong Provincial, Shantou University, Shantou 515063, China
| | - Qifeng Chen
- College of Engineering, Shantou University, Shantou 515063, China
- Key Laboratory of Digital Signal and Image Processing of Guangdong Provincial, Shantou University, Shantou 515063, China
| | - Chuliang Wei
- College of Engineering, Shantou University, Shantou 515063, China
- Key Laboratory of Digital Signal and Image Processing of Guangdong Provincial, Shantou University, Shantou 515063, China
| | - Zhun Fan
- College of Engineering, Shantou University, Shantou 515063, China
- Key Laboratory of Digital Signal and Image Processing of Guangdong Provincial, Shantou University, Shantou 515063, China
| | - Haoyu Chen
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, Shantou 515051, China
- Medical College, Shantou University, Shantou 515063, China
| |
Collapse
|
6
|
Cui X, Cao Y, Liu Z, Sui X, Mi J, Zhang Y, Cui L, Li S. TRSA-Net: Task Relation Spatial co-Attention for Joint Segmentation, Quantification and Uncertainty Estimation on Paired 2D Echocardiography. IEEE J Biomed Health Inform 2022; 26:4067-4078. [PMID: 35503848 DOI: 10.1109/jbhi.2022.3171985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Clinical workflow of cardiac assessment on 2D echocardiography requires both accurate segmentation and quantification of the Left Ventricle (LV) from paired apical 4-chamber and 2-chamber. Moreover, uncertainty estimation is significant in clinically understanding the performance of a model. However, current research on 2D echocardiography ignores this vital task while joint segmentation with quantification, hence motivating the need for a unified optimization method. In this paper, we propose a multitask model with Task Relation Spatial co-Attention (referred as TRSA-Net) for joint segmentation, quantification, and uncertainty estimation on paired 2D echo. TRSA-Net achieves multitask joint learning by novelly exploring the spatial correlation between tasks. The task relation spatial co-attention learns the spatial mapping among task-specific features by non-local and co-excitation, which forcibly joints embedded spatial information in the segmentation and quantification. The Boundary-aware Structure Consistency (BSC) and Joint Indices Constraint (JIC) are integrated into the multitask learning optimization objective to guide the learning of segmentation and quantification paths. The BSC creatively promotes structural similarity of predictions, and JIC explores the internal relationship between three quantitative indices. We validate the efficacy of our TRSA-Net on the public CAMUS dataset. Extensive comparison and ablation experiments show that our approach can achieve competitive segmentation performance and highly accurate results on quantification.
Collapse
|
7
|
Martínez Carrillo F, Moreno Tarazona A, Guayacán Chaparro LC, Bautista Rozo LX, Pico JA. A fast right ventricle segmentation in cine-MRI from a dense hough representation. REVISTA POLITÉCNICA 2022. [DOI: 10.33571/rpolitec.v18n35a6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022] Open
Abstract
Segmentation of the right ventricle (RV) is essential for the diagnosis of multiple cardiac pathologies and conditions. However, its manual delineation is a tedious task and computational support is complex due to geometric and dynamic variability. This work introduces a dense Hough transform and representation (HT) that allows a nonparametric characterization of the shape, encoding each voxel by its curvature and orientation. This representation is integrated into a bayesian tracking approach, which efficiently segments the RV structure throughout the cardiac cycle. The proposed approach was evaluated on a public dataset, with 16 patients, achieving a Sørensen-Dice coefficient of 0.87 and 0.92, for complete volumes and basal structures, respectively. These results evidence an adequate fit of the proposed model with respect to RV shape throughout the entire cardiac cycle.
La segmentación del Ventrículo Derecho (VD) es esencial para el diagnóstico de múltiples patologías y condiciones cardiacas. Sin embargo, su delineación manual es una tarea tediosa y el soporte computacional resulta complejo debido a la variabilidad geométrica y dinámica. Este trabajo introduce una transformación y representación densa de Hough (TH) que permite una caracterización no paramétrica de la forma, codificando cada vóxel por su curvatura y orientación. Esta representación es integrada en un enfoque de seguimiento bayesiano, que logra de forma eficiente segmentar la estructura del VD, a lo largo del ciclo cardíaco. El enfoque propuesto fue evaluado en un conjunto de datos públicos, con 16 pacientes, logrando un coeficiente Sørensen-Dice de 0,87 y 0,92, para volúmenes completos y estructuras basales, respectivamente. Estos resultados evidencian una adecuada adaptación del modelo propuesto respecto a la forma del VD a lo largo de todo el ciclo cardíaco.
Collapse
|
8
|
Ensemble of 2D Residual Neural Networks Integrated with Atrous Spatial Pyramid Pooling Module for Myocardium Segmentation of Left Ventricle Cardiac MRI. MATHEMATICS 2022. [DOI: 10.3390/math10040627] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Cardiac disease diagnosis and identification is problematic mostly by inaccurate segmentation of the cardiac left ventricle (LV). Besides, LV segmentation is challenging since it involves complex and variable cardiac structures in terms of components and the intricacy of time-based crescendos. In addition, full segmentation and quantification of the LV myocardium border is even more challenging because of different shapes and sizes of the myocardium border zone. The foremost purpose of this research is to design a precise automatic segmentation technique employing deep learning models for the myocardium border using cardiac magnetic resonance imaging (MRI). The ASPP module (Atrous Spatial Pyramid Pooling) was integrated with a proposed 2D-residual neural network for segmentation of the myocardium border using a cardiac MRI dataset. Further, the ensemble technique based on a majority voting ensemble method was used to blend the results of recent deep learning models on different set of hyperparameters. The proposed model produced an 85.43% dice score on validation samples and 98.23% on training samples and provided excellent performance compared to recent deep learning models. The myocardium border was successfully segmented across diverse subject slices with different shapes, sizes and contrast using the proposed deep learning ensemble models. The proposed model can be employed for automatic detection and segmentation of the myocardium border for precise quantification of reflow, myocardial infarction, myocarditis, and h cardiomyopathy (HCM) for clinical applications.
Collapse
|
9
|
Abdou MA. Literature review: efficient deep neural networks techniques for medical image analysis. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-06960-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
10
|
Xiao X, Xu Y. Multi-target regression via self-parameterized Lasso and refactored target space. APPL INTELL 2021. [DOI: 10.1007/s10489-021-02238-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
11
|
Xue W, Li J, Hu Z, Kerfoot E, Clough J, Oksuz I, Xu H, Grau V, Guo F, Ng M, Li X, Li Q, Liu L, Ma J, Grinias E, Tziritas G, Yan W, Atehortúa A, Garreau M, Jang Y, Debus A, Ferrante E, Yang G, Hua T, Li S. Left Ventricle Quantification Challenge: A Comprehensive Comparison and Evaluation of Segmentation and Regression for Mid-Ventricular Short-Axis Cardiac MR Data. IEEE J Biomed Health Inform 2021; 25:3541-3553. [PMID: 33684050 PMCID: PMC7611810 DOI: 10.1109/jbhi.2021.3064353] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Automatic quantification of the left ventricle (LV) from cardiac magnetic resonance (CMR) images plays an important role in making the diagnosis procedure efficient, reliable, and alleviating the laborious reading work for physicians. Considerable efforts have been devoted to LV quantification using different strategies that include segmentation-based (SG) methods and the recent direct regression (DR) methods. Although both SG and DR methods have obtained great success for the task, a systematic platform to benchmark them remains absent because of differences in label information during model learning. In this paper, we conducted an unbiased evaluation and comparison of cardiac LV quantification methods that were submitted to the Left Ventricle Quantification (LVQuan) challenge, which was held in conjunction with the Statistical Atlases and Computational Modeling of the Heart (STACOM) workshop at the MICCAI 2018. The challenge was targeted at the quantification of 1) areas of LV cavity and myocardium, 2) dimensions of the LV cavity, 3) regional wall thicknesses (RWT), and 4) the cardiac phase, from mid-ventricle short-axis CMR images. First, we constructed a public quantification dataset Cardiac-DIG with ground truth labels for both the myocardium mask and these quantification targets across the entire cardiac cycle. Then, the key techniques employed by each submission were described. Next, quantitative validation of these submissions were conducted with the constructed dataset. The evaluation results revealed that both SG and DR methods can offer good LV quantification performance, even though DR methods do not require densely labeled masks for supervision. Among the 12 submissions, the DR method LDAMT offered the best performance, with a mean estimation error of 301 mm 2 for the two areas, 2.15 mm for the cavity dimensions, 2.03 mm for RWTs, and a 9.5% error rate for the cardiac phase classification. Three of the SG methods also delivered comparable performances. Finally, we discussed the advantages and disadvantages of SG and DR methods, as well as the unsolved problems in automatic cardiac quantification for clinical practice applications.
Collapse
Affiliation(s)
- Wufeng Xue
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, China; Department of Medical Imaging, Western University, London, ON N6A 3K7, Canada
| | - Jiahui Li
- Beijing University of Post and Telecommunication, Beijing, China
| | | | - Eric Kerfoot
- School of Biomedical Engineering & Imaging Sciences, King’s College London, UK
| | - James Clough
- School of Biomedical Engineering & Imaging Sciences, King’s College London, UK
| | - Ilkay Oksuz
- School of Biomedical Engineering & Imaging Sciences, King’s College London, UK
| | - Hao Xu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Vicente Grau
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Fumin Guo
- Sunnybrook Research Institute, Department of Medical Biophysics, University of Toronto, Canada
| | - Matthew Ng
- Sunnybrook Research Institute, Department of Medical Biophysics, University of Toronto, Canada
| | - Xiang Li
- Department of Radiology, Massachusetts General Hospital, Boston, USA
| | - Quanzheng Li
- Department of Radiology, Massachusetts General Hospital, Boston, USA
| | - Lihong Liu
- Pingan Technology (Shenzhen) Co.Ltd. Elias Grinias and Georgios Tziritas are with Department of Computer Science, University of Crete, Heraklion, Greece
| | - Jin Ma
- Pingan Technology (Shenzhen) Co.Ltd. Elias Grinias and Georgios Tziritas are with Department of Computer Science, University of Crete, Heraklion, Greece
| | - Elias Grinias
- Department of Computer Science, University of Crete, Heraklion, Greece
| | - Georgios Tziritas
- Department of Computer Science, University of Crete, Heraklion, Greece
| | - Wenjun Yan
- Department of Electrical Engineering, Fudan University, Shanghai, China
| | - Angélica Atehortúa
- LTSI UMR 1099, F-35000 Rennes, France; Universidad Nacional de Colombia, Bogotá, Colombia
| | | | - Yeonggul Jang
- Brain Korea 21 PLUS Project for Medical Science, Yonsei University
| | - Alejandro Debus
- Research Institute for Signals, Systems and Computational Intelligence, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
| | - Enzo Ferrante
- Research Institute for Signals, Systems and Computational Intelligence, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
| | - Guanyu Yang
- Centre de Recherche en Information Biomédicale Sino-Français (CRIBs), Southeast University, Nanjing, China; LIST, Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China
| | - Tiancong Hua
- Centre de Recherche en Information Biomedicale Sino-Francais (CRIBs), Southeast University, Nanjing, China
| | - Shuo Li
- Department of Medical Imaging, Western University, London, ON N6A 3K7, Canada
| |
Collapse
|
12
|
Huang X, Tian Y, Zhao S, Liu T, Wang W, Wang Q. Direct full quantification of the left ventricle via multitask regression and classification. APPL INTELL 2021. [DOI: 10.1007/s10489-020-02130-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
13
|
Learning local instance correlations for multi-target regression. APPL INTELL 2021. [DOI: 10.1007/s10489-020-02112-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
14
|
|
15
|
Vesal S, Gu M, Maier A, Ravikumar N. Spatio-Temporal Multi-Task Learning for Cardiac MRI Left Ventricle Quantification. IEEE J Biomed Health Inform 2021; 25:2698-2709. [PMID: 33351771 DOI: 10.1109/jbhi.2020.3046449] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Quantitative assessment of cardiac left ventricle (LV) morphology is essential to assess cardiac function and improve the diagnosis of different cardiovascular diseases. In current clinical practice, LV quantification depends on the measurement of myocardial shape indices, which is usually achieved by manual contouring of the endo- and epicardial. However, this process subjected to inter and intra-observer variability, and it is a time-consuming and tedious task. In this article, we propose a spatio-temporal multi-task learning approach to obtain a complete set of measurements quantifying cardiac LV morphology, regional-wall thickness (RWT), and additionally detecting the cardiac phase cycle (systole and diastole) for a given 3D Cine-magnetic resonance (MR) image sequence. We first segment cardiac LVs using an encoder-decoder network and then introduce a multitask framework to regress 11 LV indices and classify the cardiac phase, as parallel tasks during model optimization. The proposed deep learning model is based on the 3D spatio-temporal convolutions, which extract spatial and temporal features from MR images. We demonstrate the efficacy of the proposed method using cine-MR sequences of 145 subjects and comparing the performance with other state-of-the-art quantification methods. The proposed method obtained high prediction accuracy, with an average mean absolute error (MAE) of 129 mm 2, 1.23 mm, 1.76 mm, Pearson correlation coefficient (PCC) of 96.4%, 87.2%, and 97.5% for LV and myocardium (Myo) cavity regions, 6 RWTs, 3 LV dimensions, and an error rate of 9.0% for phase classification. The experimental results highlight the robustness of the proposed method, despite varying degrees of cardiac morphology, image appearance, and low contrast in the cardiac MR sequences.
Collapse
|
16
|
Penso M, Moccia S, Scafuri S, Muscogiuri G, Pontone G, Pepi M, Caiani EG. Automated left and right ventricular chamber segmentation in cardiac magnetic resonance images using dense fully convolutional neural network. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 204:106059. [PMID: 33812305 DOI: 10.1016/j.cmpb.2021.106059] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Accepted: 03/16/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND AND OBJECTIVE Segmentation of the left ventricular (LV) myocardium (Myo) and RV endocardium on cine cardiac magnetic resonance (CMR) images represents an essential step for cardiac-function evaluation and diagnosis. In order to have a common reference for comparing segmentation algorithms, several CMR image datasets were made available, but in general they do not include the most apical and basal slices, and/or gold standard tracing is limited to only one of the two ventricles, thus not fully corresponding to real clinical practice. Our aim was to develop a deep learning (DL) approach for automated segmentation of both RV and LV chambers from short-axis (SAX) CMR images, reporting separately the performance for basal slices, together with the applied criterion of choice. METHOD A retrospectively selected database (DB1) of 210 cine sequences (3 pathology groups) was considered: images (GE, 1.5 T) were acquired at Centro Cardiologico Monzino (Milan, Italy), and end-diastolic (ED) and end-systolic frames (ES) were manually segmented (gold standard, GS). Automatic ED and ES RV and LV segmentation were performed with a U-Net inspired architecture, where skip connections were redesigned introducing dense blocks to alleviate the semantic gap between the U-Net encoder and decoder. The proposed architecture was trained including: A) the basal slices where the Myo surrounded the LV for at least the 50% and all the other slice; B) all the slices where the Myo completely surrounded the LV. To evaluate the clinical relevance of the proposed architecture in a practical use case scenario, a graphical user interface was developed to allow clinicians to revise, and correct when needed, the automatic segmentation. Additionally, to assess generalizability, analysis of CMR images obtained in 12 healthy volunteers (DB2) with different equipment (Siemens, 3T) and settings was performed. RESULTS The proposed architecture outperformed the original U-Net. Comparing the performance on DB1 between the two criteria, no significant differences were measured when considering all slices together, but were present when only basal slices were examined. Automatic and manually-adjusted segmentation performed similarly compared to the GS (bias±95%LoA): LVEDV -1±12 ml, LVESV -1±14 ml, RVEDV 6±12 ml, RVESV 6±14 ml, ED LV mass 6±26 g, ES LV mass 5±26 g). Also, generalizability showed very similar performance, with Dice scores of 0.944 (LV), 0.908 (RV) and 0.852 (Myo) on DB1, and 0.940 (LV), 0.880 (RV), and 0.856 (Myo) on DB2. CONCLUSIONS Our results support the potential of DL methods for accurate LV and RV contours segmentation and the advantages of dense skip connections in alleviating the semantic gap generated when high level features are concatenated with lower level feature. The evaluation on our dataset, considering separately the performance on basal and apical slices, reveals the potential of DL approaches for fast, accurate and reliable automated cardiac segmentation in a real clinical setting.
Collapse
Affiliation(s)
- Marco Penso
- Department of Cardiovascular Imaging, Centro Cardiologico Monzino IRCCS, Milan, Italy.
| | - Sara Moccia
- Department of Information Engineering, Università Politecnica delle Marche, Ancona, Italy; The BioRobotics Institute, Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy.
| | - Stefano Scafuri
- Department of Cardiovascular Imaging, Centro Cardiologico Monzino IRCCS, Milan, Italy.
| | - Giuseppe Muscogiuri
- Department of Cardiovascular Imaging, Centro Cardiologico Monzino IRCCS, Milan, Italy.
| | - Gianluca Pontone
- Department of Cardiovascular Imaging, Centro Cardiologico Monzino IRCCS, Milan, Italy.
| | - Mauro Pepi
- Department of Cardiovascular Imaging, Centro Cardiologico Monzino IRCCS, Milan, Italy.
| | - Enrico Gianluca Caiani
- Department of Electronics, Information and Biomedical engineering, Politecnico di Milano, Milan, Italy; Consiglio Nazionale delle Ricerche, Istituto di Elettronica e di Ingegneria dell'Informazione e delle Telecomunicazioni, Milan, Italy.
| |
Collapse
|
17
|
Hussain MA, Hamarneh G, Garbi R. Cascaded Regression Neural Nets for Kidney Localization and Segmentation-free Volume Estimation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:1555-1567. [PMID: 33606626 DOI: 10.1109/tmi.2021.3060465] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Kidney volume is an essential biomarker for a number of kidney disease diagnoses, for example, chronic kidney disease. Existing total kidney volume estimation methods often rely on an intermediate kidney segmentation step. On the other hand, automatic kidney localization in volumetric medical images is a critical step that often precedes subsequent data processing and analysis. Most current approaches perform kidney localization via an intermediate classification or regression step. This paper proposes an integrated deep learning approach for (i) kidney localization in computed tomography scans and (ii) segmentation-free renal volume estimation. Our localization method uses a selection-convolutional neural network that approximates the kidney inferior-superior span along the axial direction. Cross-sectional (2D) slices from the estimated span are subsequently used in a combined sagittal-axial Mask-RCNN that detects the organ bounding boxes on the axial and sagittal slices, the combination of which produces a final 3D organ bounding box. Furthermore, we use a fully convolutional network to estimate the kidney volume that skips the segmentation procedure. We also present a mathematical expression to approximate the 'volume error' metric from the 'Sørensen-Dice coefficient.' We accessed 100 patients' CT scans from the Vancouver General Hospital records and obtained 210 patients' CT scans from the 2019 Kidney Tumor Segmentation Challenge database to validate our method. Our method produces a kidney boundary wall localization error of ~2.4mm and a mean volume estimation error of ~5%.
Collapse
|
18
|
Yu C, Gao Z, Zhang W, Yang G, Zhao S, Zhang H, Zhang Y, Li S. Multitask Learning for Estimating Multitype Cardiac Indices in MRI and CT Based on Adversarial Reverse Mapping. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021; 32:493-506. [PMID: 32310804 DOI: 10.1109/tnnls.2020.2984955] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The estimation of multitype cardiac indices from cardiac magnetic resonance imaging (MRI) and computed tomography (CT) images attracts great attention because of its clinical potential for comprehensive function assessment. However, the most exiting model can only work in one imaging modality (MRI or CT) without transferable capability. In this article, we propose the multitask learning method with the reverse inferring for estimating multitype cardiac indices in MRI and CT. Different from the existing forward inferring methods, our method builds a reverse mapping network that maps the multitype cardiac indices to cardiac images. The task dependencies are then learned and shared to multitask learning networks using an adversarial training approach. Finally, we transfer the parameters learned from MRI to CT. A series of experiments were conducted in which we first optimized the performance of our framework via ten-fold cross-validation of over 2900 cardiac MRI images. Then, the fine-tuned network was run on an independent data set with 2360 cardiac CT images. The results of all the experiments conducted on the proposed adversarial reverse mapping show excellent performance in estimating multitype cardiac indices.
Collapse
|
19
|
Sharma K, Alsadoon A, Prasad PWC, Al-Dala'in T, Nguyen TQV, Pham DTH. A novel solution of using deep learning for left ventricle detection: Enhanced feature extraction. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 197:105751. [PMID: 32957061 DOI: 10.1016/j.cmpb.2020.105751] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/06/2019] [Accepted: 09/05/2020] [Indexed: 05/20/2023]
Abstract
BACKGROUND AND AIM deep learning algorithms have not been successfully used for the left ventricle (LV) detection in echocardiographic images due to overfitting and vanishing gradient descent problem. This research aims to increase accuracy and improves the processing time of the left ventricle detection process by reducing the overfitting and vanishing gradient problem. METHODOLOGY the proposed system consists of an enhanced deep convolutional neural network with an extra convolutional layer, and dropout layer to solve the problem of overfitting and vanishing gradient. Data augmentation was used for increasing the accuracy of feature extraction for left ventricle detection. RESULTS four pathological groups of datasets were used for training and evaluation of the model: heart failure without infarction, heart failure with infarction, and hypertrophy, and healthy. The proposed model provided an accuracy of 94% in left ventricle detection for all the groups compared to the other current systems. The results showed that the processing time was reduced from 0.45 s to 0.34 s in an average. CONCLUSION the proposed system enhances accuracy and decreases processing time in the left ventricle detection. This paper solves the issues of overfitting of the data.
Collapse
Affiliation(s)
- Kiran Sharma
- School of Computing and Mathematics, Charles Sturt University, Sydney Campus, Australia
| | - Abeer Alsadoon
- School of Computing and Mathematics, Charles Sturt University, Sydney Campus, Australia.
| | - P W C Prasad
- School of Computing and Mathematics, Charles Sturt University, Sydney Campus, Australia
| | - Thair Al-Dala'in
- School of Computing and Mathematics, Charles Sturt University, Sydney Campus, Australia
| | - Tran Quoc Vinh Nguyen
- The University of Da Nang - University of Science and Education, Faculty of Information Technology, Vietnam
| | - Duong Thu Hang Pham
- The University of Da Nang - University of Science and Education, Faculty of Information Technology, Vietnam
| |
Collapse
|
20
|
Peña-Solórzano CA, Albrecht DW, Bassed RB, Burke MD, Dimmock MR. Findings from machine learning in clinical medical imaging applications - Lessons for translation to the forensic setting. Forensic Sci Int 2020; 316:110538. [PMID: 33120319 PMCID: PMC7568766 DOI: 10.1016/j.forsciint.2020.110538] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Revised: 04/28/2020] [Accepted: 10/04/2020] [Indexed: 12/18/2022]
Abstract
Machine learning (ML) techniques are increasingly being used in clinical medical imaging to automate distinct processing tasks. In post-mortem forensic radiology, the use of these algorithms presents significant challenges due to variability in organ position, structural changes from decomposition, inconsistent body placement in the scanner, and the presence of foreign bodies. Existing ML approaches in clinical imaging can likely be transferred to the forensic setting with careful consideration to account for the increased variability and temporal factors that affect the data used to train these algorithms. Additional steps are required to deal with these issues, by incorporating the possible variability into the training data through data augmentation, or by using atlases as a pre-processing step to account for death-related factors. A key application of ML would be then to highlight anatomical and gross pathological features of interest, or present information to help optimally determine the cause of death. In this review, we highlight results and limitations of applications in clinical medical imaging that use ML to determine key implications for their application in the forensic setting.
Collapse
Affiliation(s)
- Carlos A Peña-Solórzano
- Department of Medical Imaging and Radiation Sciences, Monash University, Wellington Rd, Clayton, Melbourne, VIC 3800, Australia.
| | - David W Albrecht
- Clayton School of Information Technology, Monash University, Wellington Rd, Clayton, Melbourne, VIC 3800, Australia.
| | - Richard B Bassed
- Victorian Institute of Forensic Medicine, 57-83 Kavanagh St., Southbank, Melbourne, VIC 3006, Australia; Department of Forensic Medicine, Monash University, Wellington Rd, Clayton, Melbourne, VIC 3800, Australia.
| | - Michael D Burke
- Victorian Institute of Forensic Medicine, 57-83 Kavanagh St., Southbank, Melbourne, VIC 3006, Australia; Department of Forensic Medicine, Monash University, Wellington Rd, Clayton, Melbourne, VIC 3800, Australia.
| | - Matthew R Dimmock
- Department of Medical Imaging and Radiation Sciences, Monash University, Wellington Rd, Clayton, Melbourne, VIC 3800, Australia.
| |
Collapse
|
21
|
Pereira RF, Rebelo MS, Moreno RA, Marco AG, Lima DM, Arruda MAF, Krieger JE, Gutierrez MA. Fully Automated Quantification of Cardiac Indices from Cine MRI Using a Combination of Convolution Neural Networks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:1221-1224. [PMID: 33018207 DOI: 10.1109/embc44109.2020.9176166] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Cardiovascular magnetic resonance imaging (CMRI) is one of the most accurate non-invasive modalities for evaluation of cardiac function, especially the left ventricle (LV). In this modality, the manual or semi-automatic delineation of LV by experts is currently the standard clinical practice for chambers segmentation. Despite these efforts, global quantification of LV remains a challenge. In this work, a combination of two convolutional neural network (CNN) architectures for quantitative evaluation of the LV is described, which estimates the cavity and the myocardium areas, endocardial cavity dimensions in three directions, and the myocardium regional wall thickness in six radial directions. The method was validated in CMRI exams of 56 patients (LVQuan19 dataset) and evaluated by metrics Dice Index, Mean Absolute Error, and Correlation with superior performance compared to the state-of-the-art methods. The combination of the CNN architectures provided a simpler yet fully automated approach, requiring no specialist interaction.Clinical Relevance- With the proposed method, it is possible to perform automatically the full quantification of regional clinically relevant parameters of the left ventricle in short-axis CMRI images with superior performance compared to state-of-the-art methods.
Collapse
|
22
|
Dynamically constructed network with error correction for accurate ventricle volume estimation. Med Image Anal 2020; 64:101723. [DOI: 10.1016/j.media.2020.101723] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2019] [Revised: 05/07/2020] [Accepted: 05/08/2020] [Indexed: 11/20/2022]
|
23
|
Ge R, Yang G, Chen Y, Luo L, Feng C, Ma H, Ren J, Li S. K-Net: Integrate Left Ventricle Segmentation and Direct Quantification of Paired Echo Sequence. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:1690-1702. [PMID: 31765307 DOI: 10.1109/tmi.2019.2955436] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The integration of segmentation and direct quantification on the left ventricle (LV) from the paired apical views(i.e., apical 4-chamber and 2-chamber together) echo sequence clinically achieves the comprehensive cardiac assessment: multiview segmentation for anatomical morphology, and multidimensional quantification for contractile function. Direct quantification of LV, i.e., to automatically quantify multiple LV indices directly from the image via task-aware feature representation and regression, avoids accumulative error from the inter-step target. This integration sequentially makes a stereoscopical reflection of cardiac activity jointly from the paired orthogonal cross views sequences, overcoming limited observation with a single plane. We propose a K-shaped Unified Network (K-Net), the first end-to-end framework to simultaneously segment LV from apical 4-chamber and 2-chamber views, and directly quantify LV from major- and minor-axis dimensions (1D), area (2D), and volume (3D), in sequence. It works via four components: 1) the K-Net architecture with the Attention Junction enables heterogeneous tasks learning of segmentation task of pixel-wise classification, and direct quantification task of image-wise regression, by interactively introducing the information from segmentation to jointly promote spatial attention map to guide quantification focusing on LV-related region, and transferring quantification feedback to make global constraint on segmentation; 2) the Bi-ResLSTMs distributed in K-Net layer-by-layer hierarchically extract spatial-temporal information in echo sequence, with bidirectional recurrent and short-cut connection to model spatial-temporal information among all frames; 3) the Information Valve tailing the Bi-ResLSTMs selectively exchanges information among multiple views, by stimulating complementary information and suppressing redundant information to make the efficient cross-flow for each view; 4) the Evolution Loss comprehensively guides sequential data learning, with static constraint for frame values, and dynamic constraint for inter-frame value changes. The experiments show that our K-Net gains high performance with a Dice coefficient up to 91.44% and a mean absolute error of the major-axis dimension down to 2.74mm, which reveal its clinical potential.
Collapse
|
24
|
Lin L, Tao X, Pang S, Su Z, Lu H, Li S, Feng Q, Chen B. Multiple Axial Spine Indices Estimation via Dense Enhancing Network With Cross-Space Distance-Preserving Regularization. IEEE J Biomed Health Inform 2020; 24:3248-3257. [PMID: 32142463 DOI: 10.1109/jbhi.2020.2977224] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Automatic estimation of axial spine indices is clinically desired for various spine computer aided procedures, such as disease diagnosis, therapeutic evaluation, pathophysiological understanding, risk assessment, and biomechanical modeling. Currently, the spine indices are manually measured by physicians, which is time-consuming and laborious. Even worse, the tedious manual procedure might result in inaccurate measurement. To deal with this problem, in this paper, we aim at developing an automatic method to estimate multiple indices from axial spine images. Inspired by the success of deep learning for regression problems and the densely connected network for image classification, we propose a dense enhancing network (DE-Net) which uses the dense enhancing blocks (DEBs) as its main body, where a feature enhancing layer is added to each of the bypass in a dense block. The DEB is designed to enhance discriminative feature embedding from the intervertebral disc and the dural sac areas. In addition, the cross-space distance-preserving regularization (CSDPR), which enforces consistent inter-sample distances between the output and the label spaces, is proposed to regularize the loss function of the DE-Net. To train and validate the proposed method, we collected 895 axial spine MRI images from 143 subjects and manually measured the indices as the ground truth. The results show that all deep learning models obtain very small prediction errors, and the proposed DE-Net with CSDPR acquires the smallest error among all methods, indicating that our method has great potential for spine computer aided procedures.
Collapse
|
25
|
Li T, Wei B, Cong J, Hong Y, Li S. Direct estimation of left ventricular ejection fraction via a cardiac cycle feature learning architecture. Comput Biol Med 2020; 118:103659. [DOI: 10.1016/j.compbiomed.2020.103659] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2019] [Revised: 02/11/2020] [Accepted: 02/11/2020] [Indexed: 12/28/2022]
|
26
|
Chen R, Xu C, Dong Z, Liu Y, Du X. DeepCQ: Deep multi-task conditional quantification network for estimation of left ventricle parameters. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 184:105288. [PMID: 31901611 DOI: 10.1016/j.cmpb.2019.105288] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 12/13/2019] [Accepted: 12/18/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic cardiac left ventricle (LV) quantification plays an important role in assessing cardiac function. Although many advanced methods have been put forward to quantify related LV parameters, automatic cardiac LV quantification is still a challenge task due to the anatomy construction complexity of heart. METHODS In this work, we propose a novel deep multi-task conditional quantification learning model (DeepCQ) which contains Segmentation module, Quantification encoder, and Dynamic analysis module. Besides, we also use task uncertainty loss function to update the parameters of the network in training. RESULTS The proposed framework is validated on the dataset from Left Ventricle Full Quantification Challenge MICCAI 2018 (https://lvquan18.github.io/). The experimental results show that DeepCQ outperforms the other advanced methods. CONCLUSIONS It illustrates that our method has a great potential in comprehensive cardiac function assessment and could play an auxiliary role in clinicians' diagnosis.
Collapse
Affiliation(s)
- Ruifeng Chen
- School of Computer Science and Technology, Anhui University, Anhui, China
| | - Chenchu Xu
- Department of Medical Imaging, Western University, London, Canada.
| | - Zhangfu Dong
- School of Computer Science and Technology, Anhui University, Anhui, China
| | - Yueguo Liu
- School of Computer Science and Technology, Anhui University, Anhui, China
| | - Xiuquan Du
- Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University, Anhui, China; School of Computer Science and Technology, Anhui University, Anhui, China.
| |
Collapse
|
27
|
Duchateau N, King AP, De Craene M. Machine Learning Approaches for Myocardial Motion and Deformation Analysis. Front Cardiovasc Med 2020; 6:190. [PMID: 31998756 PMCID: PMC6962100 DOI: 10.3389/fcvm.2019.00190] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2019] [Accepted: 12/12/2019] [Indexed: 12/21/2022] Open
Abstract
Information about myocardial motion and deformation is key to differentiate normal and abnormal conditions. With the advent of approaches relying on data rather than pre-conceived models, machine learning could either improve the robustness of motion quantification or reveal patterns of motion and deformation (rather than single parameters) that differentiate pathologies. We review machine learning strategies for extracting motion-related descriptors and analyzing such features among populations, keeping in mind constraints specific to the cardiac application.
Collapse
Affiliation(s)
| | - Andrew P. King
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
| | | |
Collapse
|
28
|
Luo G, Dong S, Wang W, Wang K, Cao S, Tam C, Zhang H, Howey J, Ohorodnyk P, Li S. Commensal correlation network between segmentation and direct area estimation for bi-ventricle quantification. Med Image Anal 2020; 59:101591. [DOI: 10.1016/j.media.2019.101591] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Revised: 08/25/2019] [Accepted: 10/21/2019] [Indexed: 10/25/2022]
|
29
|
Ge R, Yang G, Chen Y, Luo L, Feng C, Zhang H, Li S. PV-LVNet: Direct left ventricle multitype indices estimation from 2D echocardiograms of paired apical views with deep neural networks. Med Image Anal 2019; 58:101554. [DOI: 10.1016/j.media.2019.101554] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2018] [Revised: 05/15/2019] [Accepted: 09/04/2019] [Indexed: 11/16/2022]
|
30
|
Segmentation and quantification of infarction without contrast agents via spatiotemporal generative adversarial learning. Med Image Anal 2019; 59:101568. [PMID: 31622838 DOI: 10.1016/j.media.2019.101568] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2018] [Revised: 09/16/2019] [Accepted: 09/30/2019] [Indexed: 12/24/2022]
Abstract
Accurate and simultaneous segmentation and full quantification (all indices are required in a clinical assessment) of the myocardial infarction (MI) area are crucial for early diagnosis and surgical planning. Current clinical methods remain subject to potential high-risk, nonreproducibility and time-consumption issues. In this study, a deep spatiotemporal adversarial network (DSTGAN) is proposed as a contrast-free, stable and automatic clinical tool to simultaneously segment and quantify MIs directly from the cine MR image. The DSTGAN is implemented using a conditional generative model, which conditions the distributions of the objective cine MR image to directly optimize the generalized error of the mapping between the input and the output. The method consists of the following: (1) A multi-level and multi-scale spatiotemporal variation encoder learns a coarse to fine hierarchical feature to effectively encode the MI-specific morphological and kinematic abnormality structures, which vary for different spatial locations and time periods. (2) The top-down and cross-task generators learn the shared representations between segmentation and quantification to use the commonalities and differences between the two related tasks and enhance the generator preference. (3) Three inter-/intra-tasks to label the relatedness discriminators are iteratively imposed on the encoder and generator to detect and correct the inconsistencies in the label relatedness between and within tasks via adversarial learning. Our proposed method yields a pixel classification accuracy of 96.98%, and the mean absolute error of the MI centroid is 0.96 mm from 165 clinical subjects. These results indicate the potential of our proposed method in aiding standardized MI assessments.
Collapse
|
31
|
Automatic detection and localization of Focal Cortical Dysplasia lesions in MRI using fully convolutional neural network. Biomed Signal Process Control 2019. [DOI: 10.1016/j.bspc.2019.04.024] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
32
|
Pang S, Su Z, Leung S, Nachum IB, Chen B, Feng Q, Li S. Direct automated quantitative measurement of spine by cascade amplifier regression network with manifold regularization. Med Image Anal 2019; 55:103-115. [DOI: 10.1016/j.media.2019.04.012] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2018] [Revised: 02/25/2019] [Accepted: 04/17/2019] [Indexed: 11/30/2022]
|
33
|
Muthulakshmi M, Kavitha G. Deep CNN with LM learning based myocardial ischemia detection in cardiac magnetic resonance images. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2019:824-827. [PMID: 31946022 DOI: 10.1109/embc.2019.8856838] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Cardiovascular disease (CVD) is a chronic dysfunction caused by deterioration in cardiac physiology. It results in about 31% of mortality worldwide. Among CVDs, myocardial ischemia (MI) leads to restriction in blood supply to heart tissues. There is a need to develop an effective computer aided detection (CAD) system to reduce the fatality. In this work, an attempt is made to perform mass screening of myocardial ischemic subjects and left ventricle (LV) volume estimation from cardiac magnetic resonance (CMR) images using deep convolutional neural network (CNN) with Levenberg-Marquardt (LM) learning. LV volume measurement is an important predictor of myocardial ischemia. The CMR samples used in this analysis are obtained from Medical Image Computing and Computer Assisted Intervention (MICCAI) 2009 database. The results of the proposed model are compared with deep CNN based on gradient descent (GD) learning algorithm. The results show that deep CNN architecture with LM learning classifies ischemic subjects with high accuracy (86.39%) and sensitivity (90%). The LM learning based method gives an AUC of 0.93. The estimated LV volumes obtained from the trained network gives high correlation with the ground truth. Thus the results support that proposed framework of deep CNN architecture with LM learning can be used as an effective CAD system for diagnosis of cardiovascular disorders.
Collapse
|
34
|
Ivanov I, Lomaev Y, Barkovskaya A. Automatic calculation of left ventricular volume in magnetic resonance imaging using an image-based clustering approach. ACTA ACUST UNITED AC 2019. [DOI: 10.1088/1757-899x/537/4/042046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
35
|
|
36
|
The present and future of deep learning in radiology. Eur J Radiol 2019; 114:14-24. [PMID: 31005165 DOI: 10.1016/j.ejrad.2019.02.038] [Citation(s) in RCA: 167] [Impact Index Per Article: 33.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2018] [Revised: 02/17/2019] [Accepted: 02/26/2019] [Indexed: 12/18/2022]
Abstract
The advent of Deep Learning (DL) is poised to dramatically change the delivery of healthcare in the near future. Not only has DL profoundly affected the healthcare industry it has also influenced global businesses. Within a span of very few years, advances such as self-driving cars, robots performing jobs that are hazardous to human, and chat bots talking with human operators have proved that DL has already made large impact on our lives. The open source nature of DL and decreasing prices of computer hardware will further propel such changes. In healthcare, the potential is immense due to the need to automate the processes and evolve error free paradigms. The sheer quantum of DL publications in healthcare has surpassed other domains growing at a very fast pace, particular in radiology. It is therefore imperative for the radiologists to learn about DL and how it differs from other approaches of Artificial Intelligence (AI). The next generation of radiology will see a significant role of DL and will likely serve as the base for augmented radiology (AR). Better clinical judgement by AR will help in improving the quality of life and help in life saving decisions, while lowering healthcare costs. A comprehensive review of DL as well as its implications upon the healthcare is presented in this review. We had analysed 150 articles of DL in healthcare domain from PubMed, Google Scholar, and IEEE EXPLORE focused in medical imagery only. We have further examined the ethic, moral and legal issues surrounding the use of DL in medical imaging.
Collapse
|
37
|
Napel S, Mu W, Jardim‐Perassi BV, Aerts HJWL, Gillies RJ. Quantitative imaging of cancer in the postgenomic era: Radio(geno)mics, deep learning, and habitats. Cancer 2018; 124:4633-4649. [PMID: 30383900 PMCID: PMC6482447 DOI: 10.1002/cncr.31630] [Citation(s) in RCA: 110] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2018] [Revised: 07/11/2018] [Accepted: 07/17/2018] [Indexed: 11/07/2022]
Abstract
Although cancer often is referred to as "a disease of the genes," it is indisputable that the (epi)genetic properties of individual cancer cells are highly variable, even within the same tumor. Hence, preexisting resistant clones will emerge and proliferate after therapeutic selection that targets sensitive clones. Herein, the authors propose that quantitative image analytics, known as "radiomics," can be used to quantify and characterize this heterogeneity. Virtually every patient with cancer is imaged radiologically. Radiomics is predicated on the beliefs that these images reflect underlying pathophysiologies, and that they can be converted into mineable data for improved diagnosis, prognosis, prediction, and therapy monitoring. In the last decade, the radiomics of cancer has grown from a few laboratories to a worldwide enterprise. During this growth, radiomics has established a convention, wherein a large set of annotated image features (1-2000 features) are extracted from segmented regions of interest and used to build classifier models to separate individual patients into their appropriate class (eg, indolent vs aggressive disease). An extension of this conventional radiomics is the application of "deep learning," wherein convolutional neural networks can be used to detect the most informative regions and features without human intervention. A further extension of radiomics involves automatically segmenting informative subregions ("habitats") within tumors, which can be linked to underlying tumor pathophysiology. The goal of the radiomics enterprise is to provide informed decision support for the practice of precision oncology.
Collapse
Affiliation(s)
- Sandy Napel
- Department of RadiologyStanford UniversityStanfordCalifornia
| | - Wei Mu
- Department of Cancer PhysiologyH. Lee Moffitt Cancer CenterTampaFlorida
| | | | - Hugo J. W. L. Aerts
- Dana‐Farber Cancer Institute, Department of Radiology, Brigham and Women’s HospitalHarvard Medical SchoolBostonMassachusetts
| | - Robert J. Gillies
- Department of Cancer PhysiologyH. Lee Moffitt Cancer CenterTampaFlorida
| |
Collapse
|
38
|
Abstract
The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction, and intervention. Deep learning is a representation learning method that consists of layers that transform data nonlinearly, thus, revealing hierarchical relationships and structures. In this review, we survey deep learning application papers that use structured data, and signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.
Collapse
|
39
|
Montesinos-López OA, Montesinos-López A, Crossa J, Gianola D, Hernández-Suárez CM, Martín-Vallejo J. Multi-trait, Multi-environment Deep Learning Modeling for Genomic-Enabled Prediction of Plant Traits. G3 (BETHESDA, MD.) 2018; 8:3829-3840. [PMID: 30291108 PMCID: PMC6288830 DOI: 10.1534/g3.118.200728] [Citation(s) in RCA: 65] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/30/2018] [Accepted: 10/03/2018] [Indexed: 11/27/2022]
Abstract
Multi-trait and multi-environment data are common in animal and plant breeding programs. However, what is lacking are more powerful statistical models that can exploit the correlation between traits to improve prediction accuracy in the context of genomic selection (GS). Multi-trait models are more complex than univariate models and usually require more computational resources, but they are preferred because they can exploit the correlation between traits, which many times helps improve prediction accuracy. For this reason, in this paper we explore the power of multi-trait deep learning (MTDL) models in terms of prediction accuracy. The prediction performance of MTDL models was compared to the performance of the Bayesian multi-trait and multi-environment (BMTME) model proposed by Montesinos-López et al. (2016), which is a multi-trait version of the genomic best linear unbiased prediction (GBLUP) univariate model. Both models were evaluated with predictors with and without the genotype×environment interaction term. The prediction performance of both models was evaluated in terms of Pearson's correlation using cross-validation. We found that the best predictions in two of the three data sets were found under the BMTME model, but in general the predictions of both models, BTMTE and MTDL, were similar. Among models without the genotype×environment interaction, the MTDL model was the best, while among models with genotype×environment interaction, the BMTME model was superior. These results indicate that the MTDL model is very competitive for performing predictions in the context of GS, with the important practical advantage that it requires less computational resources than the BMTME model.
Collapse
Affiliation(s)
| | - Abelardo Montesinos-López
- Departamento de Matemáticas, Centro Universitario de Ciencias Exactas e Ingenierías (CUCEI), Universidad de Guadalajara, 44430, Guadalajara, Jalisco, México
| | - José Crossa
- Biometrics and Statistics Unit, Genetic Resources Program, International Maize and Wheat Improvement Center (CIMMYT), Apdo. Postal 6-641, 06600, Ciudad de México, México
| | - Daniel Gianola
- Departments of Animal Sciences, Dairy Science, and Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, Wisconsin 53706
| | | | - Javier Martín-Vallejo
- Departamento de Estadística, Universidad de Salamanca, c/Espejo 2, Salamanca, 37007, España
| |
Collapse
|
40
|
Zhang L, Gooya A, Pereanez M, Dong B, Piechnik S, Neubauer S, Petersen S, Frangi AF. Automatic Assessment of Full Left Ventricular Coverage in Cardiac Cine Magnetic Resonance Imaging with Fisher Discriminative 3D CNN. IEEE Trans Biomed Eng 2018; 66:1975-1986. [PMID: 30475705 DOI: 10.1109/tbme.2018.2881952] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/28/2024]
Abstract
Cardiac magnetic resonance (CMR) images play a growing role in the diagnostic imaging of cardiovascular diseases. Full coverage of the left ventricle (LV), from base to apex, is a basic criterion for CMR image quality and necessary for accurate measurement of cardiac volume and functional assessment. Incomplete coverage of the LV is identified through visual inspection, which is time-consuming and usually done retrospectively in the assessment of large imaging cohorts. This paper proposes a novel automatic method for determining LV coverage from CMR images by using Fisher-discriminative three-dimensional (FD3D) convolutional neural networks (CNNs). In contrast to our previous method employing 2D CNNs, this approach utilizes spatial contextual information in CMR volumes, extracts more representative high-level features and enhances the discriminative capacity of the baseline 2D CNN learning framework, thus achieving superior detection accuracy. A two-stage framework is proposed to identify missing basal and apical slices in measurements of CMR volume. First, the FD3D CNN extracts high-level features from the CMR stacks. These image representations are then used to detect the missing basal and apical slices. Compared to the traditional 3D CNN strategy, the proposed FD3D CNN minimizes within-class scatter and maximizes between-class scatter. We performed extensive experiments to validate the proposed method on more than 5,000 independent volumetric CMR scans from the UK Biobank study, achieving low error rates for missing basal/apical slice detection (4.9%/4.6%). The proposed method can also be adopted for assessing LV coverage for other types of CMR image data.
Collapse
|
41
|
Du X, Tang R, Yin S, Zhang Y, Li S. Direct Segmentation-Based Full Quantification for Left Ventricle via Deep Multi-Task Regression Learning Network. IEEE J Biomed Health Inform 2018; 23:942-948. [PMID: 30387757 DOI: 10.1109/jbhi.2018.2879188] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Quantitative analysis of the heart is extremely necessary and significant for detecting and diagnosing heart disease, yet there are still some challenges. In this study, we propose a new end-to-end segmentation-based deep multi-task regression learning model (Indices-JSQ) to make a holonomic quantitative analysis of the left ventricle (LV), which contains a segmentation network (Img2Contour) and multi-task regression network (Contour2Indices). First, Img2Contour, which contains a deep convolutional encoder-decoder module, is designed to obtain the LV contour. Then, the predicted contour is fed as input to Contour2Indices for full quantification. On the whole, we take into account the relationship between different tasks, which can serve as a complementary advantage. Meanwhile, instead of using images directly from the original dataset, we creatively use the segmented contour of the original image to estimate the cardiac indices to achieve better and more accurate results. We make experiments on MR sequences of 145 subjects and gain the experimental results of 157 mm 2, 2.43 mm, 1.29 mm, and 0.87 on areas, dimensions, regional wall thicknesses, and Dice Metric, respectively. It intuitively shows that the proposed method outperforms the other state-of-the-art methods and demonstrates that our method has a great potential in cardiac MR images segmentation, comprehensive clinical assessment, and diagnosis.
Collapse
|
42
|
Zhen X, Yu M, He X, Li S. Multi-Target Regression via Robust Low-Rank Learning. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2018; 40:497-504. [PMID: 28368816 DOI: 10.1109/tpami.2017.2688363] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Multi-target regression has recently regained great popularity due to its capability of simultaneously learning multiple relevant regression tasks and its wide applications in data mining, computer vision and medical image analysis, while great challenges arise from jointly handling inter-target correlations and input-output relationships. In this paper, we propose Multi-layer Multi-target Regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general framework via robust low-rank learning. Specifically, the MMR can explicitly encode inter-target correlations in a structure matrix by matrix elastic nets (MEN); the MMR can work in conjunction with the kernel trick to effectively disentangle highly complex nonlinear input-output relationships; the MMR can be efficiently solved by a new alternating optimization algorithm with guaranteed convergence. The MMR leverages the strength of kernel methods for nonlinear feature learning and the structural advantage of multi-layer learning architectures for inter-target correlation modeling. More importantly, it offers a new multi-layer learning paradigm for multi-target regression which is endowed with high generality, flexibility and expressive ability. Extensive experimental evaluation on 18 diverse real-world datasets demonstrates that our MMR can achieve consistently high performance and outperforms representative state-of-the-art algorithms, which shows its great effectiveness and generality for multivariate prediction.
Collapse
|
43
|
Full left ventricle quantification via deep multitask relationships learning. Med Image Anal 2018; 43:54-65. [DOI: 10.1016/j.media.2017.09.005] [Citation(s) in RCA: 98] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2017] [Revised: 08/25/2017] [Accepted: 09/18/2017] [Indexed: 12/22/2022]
|
44
|
Jang Y, Hong Y, Ha S, Kim S, Chang HJ. Automatic Segmentation of LV and RV in Cardiac MRI. LECTURE NOTES IN COMPUTER SCIENCE 2018. [DOI: 10.1007/978-3-319-75541-0_17] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
45
|
Pereira S, Meier R, McKinley R, Wiest R, Alves V, Silva CA, Reyes M. Enhancing interpretability of automatically extracted machine learning features: application to a RBM-Random Forest system on brain lesion segmentation. Med Image Anal 2017; 44:228-244. [PMID: 29289703 DOI: 10.1016/j.media.2017.12.009] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2017] [Revised: 10/15/2017] [Accepted: 12/12/2017] [Indexed: 12/19/2022]
Abstract
Machine learning systems are achieving better performances at the cost of becoming increasingly complex. However, because of that, they become less interpretable, which may cause some distrust by the end-user of the system. This is especially important as these systems are pervasively being introduced to critical domains, such as the medical field. Representation Learning techniques are general methods for automatic feature computation. Nevertheless, these techniques are regarded as uninterpretable "black boxes". In this paper, we propose a methodology to enhance the interpretability of automatically extracted machine learning features. The proposed system is composed of a Restricted Boltzmann Machine for unsupervised feature learning, and a Random Forest classifier, which are combined to jointly consider existing correlations between imaging data, features, and target variables. We define two levels of interpretation: global and local. The former is devoted to understanding if the system learned the relevant relations in the data correctly, while the later is focused on predictions performed on a voxel- and patient-level. In addition, we propose a novel feature importance strategy that considers both imaging data and target variables, and we demonstrate the ability of the approach to leverage the interpretability of the obtained representation for the task at hand. We evaluated the proposed methodology in brain tumor segmentation and penumbra estimation in ischemic stroke lesions. We show the ability of the proposed methodology to unveil information regarding relationships between imaging modalities and extracted features and their usefulness for the task at hand. In both clinical scenarios, we demonstrate that the proposed methodology enhances the interpretability of automatically learned features, highlighting specific learning patterns that resemble how an expert extracts relevant data from medical images.
Collapse
Affiliation(s)
- Sérgio Pereira
- CMEMS-UMinho Research Unit, University of Minho, Guimarães, Portugal; Centro Algoritmi, University of Minho, Braga, Portugal.
| | - Raphael Meier
- Institute for Surgical Technology and Biomechanics, University of Bern, Switzerland.
| | - Richard McKinley
- Support Center for Advanced Neuroimaging - Institute for Diagnostic and Interventional Neuroradiology, University Hospital and University of Bern, Switzerland.
| | - Roland Wiest
- Support Center for Advanced Neuroimaging - Institute for Diagnostic and Interventional Neuroradiology, University Hospital and University of Bern, Switzerland.
| | - Victor Alves
- Centro Algoritmi, University of Minho, Braga, Portugal.
| | - Carlos A Silva
- CMEMS-UMinho Research Unit, University of Minho, Guimarães, Portugal.
| | - Mauricio Reyes
- Institute for Surgical Technology and Biomechanics, University of Bern, Switzerland.
| |
Collapse
|
46
|
Xue W, Islam A, Bhaduri M, Li S. Direct Multitype Cardiac Indices Estimation via Joint Representation and Regression Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2017; 36:2057-2067. [PMID: 28574348 DOI: 10.1109/tmi.2017.2709251] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Cardiac indices estimation is of great importance during identification and diagnosis of cardiac disease in clinical routine. However, estimation of multitype cardiac indices with consistently reliable and high accuracy is still a great challenge due to the high variability of cardiac structures and the complexity of temporal dynamics in cardiac MR sequences. While efforts have been devoted into cardiac volumes estimation through feature engineering followed by a independent regression model, these methods suffer from the vulnerable feature representation and incompatible regression model. In this paper, we propose a semi-automated method for multitype cardiac indices estimation. After the manual labeling of two landmarks for ROI cropping, an integrated deep neural network Indices-Net is designed to jointly learn the representation and regression models. It comprises two tightly-coupled networks, such as a deep convolution autoencoder for cardiac image representation, and a multiple output convolution neural network for indices regression. Joint learning of the two networks effectively enhances the expressiveness of image representation with respect to cardiac indices, and the compatibility between image representation and indices regression, thus leading to accurate and reliable estimations for all the cardiac indices. When applied with five-fold cross validation on MR images of 145 subjects, Indices-Net achieves consistently low estimation error for LV wall thicknesses (1.44 ± 0.71 mm) and areas of cavity and myocardium (204 ± 133 mm2). It outperforms, with significant error reductions, segmentation method (55.1% and 17.4%), and two-phase direct volume-only methods (12.7% and 14.6%) for wall thicknesses and areas, respectively. These advantages endow the proposed method a great potential in clinical cardiac function assessment.
Collapse
|
47
|
Yang Y, Jiang H, Sun Q. A Multiorgan Segmentation Model for CT Volumes via Full Convolution-Deconvolution Network. BIOMED RESEARCH INTERNATIONAL 2017; 2017:6941306. [PMID: 29075646 PMCID: PMC5623798 DOI: 10.1155/2017/6941306] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/15/2017] [Revised: 08/03/2017] [Accepted: 08/10/2017] [Indexed: 01/11/2023]
Abstract
We propose a model with two-stage process for abdominal segmentation on CT volumes. First, in order to capture the details of organs, a full convolution-deconvolution network (FCN-DecNet) is constructed with multiple new unpooling, deconvolutional, and fusion layers. Then, we optimize the coarse segmentation results of FCN-DecNet by multiscale weights probabilistic atlas (MS-PA), which uses spatial and intensity characteristic of atlases. Our coarse-fine model takes advantage of intersubject variability, spatial location, and gray information of CT volumes to minimize the error of segmentation. Finally, using our model, we extract liver, spleen, and kidney with Dice index of 90.1 ± 1%, 89.0 ± 1.6%, and 89.0 ± 1.3%, respectively.
Collapse
Affiliation(s)
- Yangzi Yang
- Software College, Northeastern University, Shenyang 110819, China
| | - Huiyan Jiang
- Software College, Northeastern University, Shenyang 110819, China
| | - Qingjiao Sun
- Software College, Northeastern University, Shenyang 110819, China
| |
Collapse
|
48
|
Full Quantification of Left Ventricle via Deep Multitask Learning Network Respecting Intra- and Inter-Task Relatedness. ACTA ACUST UNITED AC 2017. [DOI: 10.1007/978-3-319-66179-7_32] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register]
|
49
|
Zhen X, Yu M, Islam A, Bhaduri M, Chan I, Li S. Descriptor Learning via Supervised Manifold Regularization for Multioutput Regression. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2017; 28:2035-2047. [PMID: 27295694 DOI: 10.1109/tnnls.2016.2573260] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Multioutput regression has recently shown great ability to solve challenging problems in both computer vision and medical image analysis. However, due to the huge image variability and ambiguity, it is fundamentally challenging to handle the highly complex input-target relationship of multioutput regression, especially with indiscriminate high-dimensional representations. In this paper, we propose a novel supervised descriptor learning (SDL) algorithm for multioutput regression, which can establish discriminative and compact feature representations to improve the multivariate estimation performance. The SDL is formulated as generalized low-rank approximations of matrices with a supervised manifold regularization. The SDL is able to simultaneously extract discriminative features closely related to multivariate targets and remove irrelevant and redundant information by transforming raw features into a new low-dimensional space aligned to targets. The achieved discriminative while compact descriptor largely reduces the variability and ambiguity for multioutput regression, which enables more accurate and efficient multivariate estimation. We conduct extensive evaluation of the proposed SDL on both synthetic data and real-world multioutput regression tasks for both computer vision and medical image analysis. Experimental results have shown that the proposed SDL can achieve high multivariate estimation accuracy on all tasks and largely outperforms the algorithms in the state of the arts. Our method establishes a novel SDL framework for multioutput regression, which can be widely used to boost the performance in different applications.
Collapse
|
50
|
Wu L, Cheng JZ, Li S, Lei B, Wang T, Ni D. FUIQA: Fetal Ultrasound Image Quality Assessment With Deep Convolutional Networks. IEEE TRANSACTIONS ON CYBERNETICS 2017; 47:1336-1349. [PMID: 28362600 DOI: 10.1109/tcyb.2017.2671898] [Citation(s) in RCA: 86] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
The quality of ultrasound (US) images for the obstetric examination is crucial for accurate biometric measurement. However, manual quality control is a labor intensive process and often impractical in a clinical setting. To improve the efficiency of examination and alleviate the measurement error caused by improper US scanning operation and slice selection, a computerized fetal US image quality assessment (FUIQA) scheme is proposed to assist the implementation of US image quality control in the clinical obstetric examination. The proposed FUIQA is realized with two deep convolutional neural network models, which are denoted as L-CNN and C-CNN, respectively. The L-CNN aims to find the region of interest (ROI) of the fetal abdominal region in the US image. Based on the ROI found by the L-CNN, the C-CNN evaluates the image quality by assessing the goodness of depiction for the key structures of stomach bubble and umbilical vein. To further boost the performance of the L-CNN, we augment the input sources of the neural network with the local phase features along with the original US data. It will be shown that the heterogeneous input sources will help to improve the performance of the L-CNN. The performance of the proposed FUIQA is compared with the subjective image quality evaluation results from three medical doctors. With comprehensive experiments, it will be illustrated that the computerized assessment with our FUIQA scheme can be comparable to the subjective ratings from medical doctors.
Collapse
|