1
|
Zhang Z, Yu C, Zhang H, Gao Z. Embedding Tasks Into the Latent Space: Cross-Space Consistency for Multi-Dimensional Analysis in Echocardiography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2215-2228. [PMID: 38329865 DOI: 10.1109/tmi.2024.3362964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/10/2024]
Abstract
Multi-dimensional analysis in echocardiography has attracted attention due to its potential for clinical indices quantification and computer-aided diagnosis. It can utilize various information to provide the estimation of multiple cardiac indices. However, it still has the challenge of inter-task conflict. This is owing to regional confusion, global abnormalities, and time-accumulated errors. Task mapping methods have the potential to address inter-task conflict. However, they may overlook the inherent differences between tasks, especially for multi-level tasks (e.g., pixel-level, image-level, and sequence-level tasks). This may lead to inappropriate local and spurious task constraints. We propose cross-space consistency (CSC) to overcome the challenge. The CSC embeds multi-level tasks to the same-level to reduce inherent task differences. This allows multi-level task features to be consistent in a unified latent space. The latent space extracts task-common features and constrains the distance in these features. This constrains the task weight region that satisfies multiple task conditions. Extensive experiments compare the CSC with fifteen state-of-the-art echocardiographic analysis methods on five datasets (10,908 patients). The result shows that the CSC can provide left ventricular (LV) segmentation, (DSC = 0.932), keypoint detection (MAE = 3.06mm), and keyframe identification (accuracy = 0.943). These results demonstrate that our method can provide a multi-dimensional analysis of cardiac function and is robust in large-scale datasets.
Collapse
|
2
|
Li D, Peng Y, Sun J, Guo Y. A task-unified network with transformer and spatial-temporal convolution for left ventricular quantification. Sci Rep 2023; 13:13529. [PMID: 37598235 PMCID: PMC10439898 DOI: 10.1038/s41598-023-40841-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 08/17/2023] [Indexed: 08/21/2023] Open
Abstract
Quantification of the cardiac function is vital for diagnosing and curing the cardiovascular diseases. Left ventricular function measurement is the most commonly used measure to evaluate the function of cardiac in clinical practice, how to improve the accuracy of left ventricular quantitative assessment results has always been the subject of research by medical researchers. Although considerable efforts have been put forward to measure the left ventricle (LV) automatically using deep learning methods, the accurate quantification is yet a challenge work as a result of the changeable anatomy structure of heart in the systolic diastolic cycle. Besides, most methods used direct regression method which lacks of visual based analysis. In this work, a deep learning segmentation and regression task-unified network with transformer and spatial-temporal convolution is proposed to segment and quantify the LV simultaneously. The segmentation module leverages a U-Net like 3D Transformer model to predict the contour of three anatomy structures, while the regression module learns spatial-temporal representations from the original images and the reconstruct feature map from segmentation path to estimate the finally desired quantification metrics. Furthermore, we employ a joint task loss function to train the two module networks. Our framework is evaluated on the MICCAI 2017 Left Ventricle Full Quantification Challenge dataset. The results of experiments demonstrate the effectiveness of our framework, which achieves competitive cardiac quantification metric results and at the same time produces visualized segmentation results that are conducive to later analysis.
Collapse
Affiliation(s)
- Dapeng Li
- Shandong University of Science and Technology, Qingdao, China
| | - Yanjun Peng
- Shandong University of Science and Technology, Qingdao, China.
- Shandong Province Key Laboratory of Wisdom Mining Information Technology, Qingdao, China.
| | - Jindong Sun
- Shandong University of Science and Technology, Qingdao, China
| | - Yanfei Guo
- Shandong University of Science and Technology, Qingdao, China
| |
Collapse
|
3
|
Jafari M, Shoeibi A, Khodatars M, Ghassemi N, Moridian P, Alizadehsani R, Khosravi A, Ling SH, Delfan N, Zhang YD, Wang SH, Gorriz JM, Alinejad-Rokny H, Acharya UR. Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review. Comput Biol Med 2023; 160:106998. [PMID: 37182422 DOI: 10.1016/j.compbiomed.2023.106998] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 03/01/2023] [Accepted: 04/28/2023] [Indexed: 05/16/2023]
Abstract
In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.
Collapse
Affiliation(s)
- Mahboobeh Jafari
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Afshin Shoeibi
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia; Data Science and Computational Intelligence Institute, University of Granada, Spain.
| | - Marjane Khodatars
- Data Science and Computational Intelligence Institute, University of Granada, Spain
| | - Navid Ghassemi
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Parisa Moridian
- Data Science and Computational Intelligence Institute, University of Granada, Spain
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Australia
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Australia
| | - Sai Ho Ling
- Faculty of Engineering and IT, University of Technology Sydney (UTS), Australia
| | - Niloufar Delfan
- Faculty of Computer Engineering, Dept. of Artificial Intelligence Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Yu-Dong Zhang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
| | - Shui-Hua Wang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
| | - Juan M Gorriz
- Data Science and Computational Intelligence Institute, University of Granada, Spain; Department of Psychiatry, University of Cambridge, UK
| | - Hamid Alinejad-Rokny
- BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia; UNSW Data Science Hub, The University of New South Wales, Sydney, NSW, 2052, Australia; Health Data Analytics Program, Centre for Applied Artificial Intelligence, Macquarie University, Sydney, 2109, Australia
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia; Dept. of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan
| |
Collapse
|
4
|
Yu C, Liu H, Zhang H. Distilling sub-space structure across views for cardiac indices estimation. Med Image Anal 2023; 85:102764. [PMID: 36791621 DOI: 10.1016/j.media.2023.102764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2021] [Revised: 01/20/2023] [Accepted: 02/02/2023] [Indexed: 02/09/2023]
Abstract
Cardiac indices estimation in multi-view images attracts great attention due to its capability for cardiac function assessment. However, the variation of the cardiac indices across views causes that most cardiac indices estimation methods can only be trained separately in each view, resulting in low data utilization. To solve this problem, we have proposed distilling the sub-space structure across views to explore the multi-view data fully for cardiac indices estimation. In particular, the sub-space structure is obtained via building a n×n covariance matrix to describe the correlation between the output dimensions of all views. Then, an alternate convex search algorithm is proposed to optimize the cross-view learning framework by which: (i) we train the model with regularization of sub-space structure in each view; (ii) we update the sub-space structure based on the learned parameters from all views. In the end, we have conducted a series of experiments to verify the effectiveness of our proposed framework. The model is trained on three views (short axis, 2-chamber view and 4-chamber view) with two modalities (magnetic resonance imaging and computed tomography). Compared to the state-of-the-art methods, our method has demonstrated superior performance on cardiac indices estimation tasks.
Collapse
Affiliation(s)
- Chengjin Yu
- College of Optical Science and Engineering, Zhejiang University, Hangzhou, China
| | - Huafeng Liu
- College of Optical Science and Engineering, Zhejiang University, Hangzhou, China
| | - Heye Zhang
- School of Biomedical Engineering, Sun Yat-sen University, Guangzhou, China.
| |
Collapse
|
5
|
Wei H, Ma J, Zhou Y, Xue W, Ni D. Co-learning of appearance and shape for precise ejection fraction estimation from echocardiographic sequences. Med Image Anal 2023; 84:102686. [PMID: 36455332 DOI: 10.1016/j.media.2022.102686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 10/31/2022] [Accepted: 11/07/2022] [Indexed: 11/17/2022]
Abstract
Accurate estimation of ejection fraction (EF) from echocardiography is of great importance for evaluation of cardiac function. It is usually obtained by the Simpson's bi-plane method based on the segmentation of the left ventricle (LV) in two keyframes. However, obtaining accurate EF estimation from echocardiography is challenging due to (1) noisy appearance in ultrasound images, (2) temporal dynamic movement of myocardium, (3) sparse annotation of the full sequence, and (4) potential quality degradation during scanning. In this paper, we propose a multi-task semi-supervised framework, which is denoted as MCLAS, for precise EF estimation from echocardiographic sequences of two cardiac views. Specifically, we first propose a co-learning mechanism to explore the mutual benefits of cardiac segmentation and myocardium tracking iteratively on appearance level and shape level, therefore alleviating the noisy appearance and enforcing the temporal consistency of the segmentation results. This temporal consistency, as shown in our work, is critical for precise EF estimation. Then we propose two auxiliary tasks for the encoder, (1) view classification to help extract the discriminative features of each view, and automatize the whole pipeline of EF estimation in clinical practice, and (2) EF regression to help regularize the spatiotemporal embedding of the echocardiographic sequence. Both two auxiliary tasks can improve the segmentation-based EF prediction, especially for sequences of poor quality. Our method is capable of automating the whole pipeline of EF estimation, from view identification, cardiac structures segmentation to EF calculation. The effectiveness of our method is validated in aspects of segmentation, tracking, consistency analysis, and clinical parameters estimation. When compared with existing methods, our method shows obvious superiority for LV volumes on ED and ES phases, and EF estimation, with Pearson correlation of 0.975, 0.983 and 0.946, respectively. This is a significant improvement for echocardiography-based EF estimation and improves the potential of automated EF estimation in clinical practice. Besides, our method can obtain accurate and temporal-consistent segmentation for the in-between frames, which enables it for cardiac dynamic function evaluation.
Collapse
Affiliation(s)
- Hongrong Wei
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China
| | - Junqiang Ma
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China
| | - Yongjin Zhou
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China
| | - Wufeng Xue
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China.
| | - Dong Ni
- School of Biomedical Engineering, Health Science Center, Shenzhen University, China; National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, China; Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, China.
| |
Collapse
|
6
|
Zhang J, Zhang Y, Zhang H, Zhang Q, Su W, Guo S, Wang Y. Segmentation of biventricle in cardiac cine MRI via nested capsule dense network. PeerJ Comput Sci 2022; 8:e1146. [PMID: 36532806 PMCID: PMC9748817 DOI: 10.7717/peerj-cs.1146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 10/13/2022] [Indexed: 06/17/2023]
Abstract
BACKGROUND Cardiac magnetic resonance image (MRI) has been widely used in diagnosis of cardiovascular diseases because of its noninvasive nature and high image quality. The evaluation standard of physiological indexes in cardiac diagnosis is essentially the accuracy of segmentation of left ventricle (LV) and right ventricle (RV) in cardiac MRI. The traditional symmetric single codec network structure such as U-Net tends to expand the number of channels to make up for lost information that results in the network looking cumbersome. METHODS Instead of a single codec, we propose a multiple codecs structure based on the FC-DenseNet (FCD) model and capsule convolution-capsule deconvolution, named Nested Capsule Dense Network (NCDN). NCDN uses multiple codecs to achieve multi-resolution, which makes it possible to save more spatial information and improve the robustness of the model. RESULTS The proposed model is tested on three datasets that include the York University Cardiac MRI dataset, Automated Cardiac Diagnosis Challenge (ACDC-2017), and the local dataset. The results show that the proposed NCDN outperforms most methods. In particular, we achieved nearly the most advanced accuracy performance in the ACDC-2017 segmentation challenge. This means that our method is a reliable segmentation method, which is conducive to the application of deep learning-based segmentation methods in the field of medical image segmentation.
Collapse
Affiliation(s)
- Jilong Zhang
- School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
| | - Yajuan Zhang
- School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
| | - Hongyang Zhang
- School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
| | - Quan Zhang
- School of Information and Communication Engineering, North University of China, Taiyuan, China
- Shanxi Provincial Key Laboratory for Biomedical Imaging and Big Data, North University of China, Taiyuan, China
| | - Weihua Su
- School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
| | - Shijie Guo
- Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, HeBUT, Tianjin, China
| | - Yuanquan Wang
- School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
- Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, HeBUT, Tianjin, China
| |
Collapse
|
7
|
Xiao X, Zhao J, Li S. Task relevance driven adversarial learning for simultaneous detection, size grading, and quantification of hepatocellular carcinoma via integrating multi-modality MRI. Med Image Anal 2022; 81:102554. [DOI: 10.1016/j.media.2022.102554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2021] [Revised: 07/12/2022] [Accepted: 07/18/2022] [Indexed: 11/26/2022]
|
8
|
RE-3DLVNet: Refined estimation of the left ventricle volume via interactive 3D segmentation and reinforced quantification. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
9
|
Tilborghs S, Bogaert J, Maes F. Shape constrained CNN for segmentation guided prediction of myocardial shape and pose parameters in cardiac MRI. Med Image Anal 2022; 81:102533. [DOI: 10.1016/j.media.2022.102533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 04/21/2022] [Accepted: 07/08/2022] [Indexed: 10/17/2022]
|
10
|
Delmondes PHM, Nunes FLS. A systematic review of multi-slice and multi-frame descriptors in cardiac MRI exams. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 221:106889. [PMID: 35649296 DOI: 10.1016/j.cmpb.2022.106889] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 04/13/2022] [Accepted: 05/11/2022] [Indexed: 06/15/2023]
Abstract
Computer-Aided Diagnosis systems have been developed to help medical professional in their decision making routines towards a more accurate diagnosis. These systems process medical exams such as Magnetic Resonance (MRI) in order to quantify meaningful features. These can be used with similarity-measuring techniques in a Content-Based Image Retrieval context, or inputted into a machine learning classifier in order to support early disease detection. For cardiac MRIs, single slice descriptors have been proposed in the two-dimensional domain, shape descriptors have been proposed in the three-dimensional domain, and previous reviews have mapped these two descriptor categories. Nonetheless, no systematic review on these descriptors have looked at full cardiac MRI images sets. We have reviewed the literature by searching for descriptors that consider the whole slice set (multi-slice) or frames (multi-frame) in cardiac MRI exams. We discuss descriptors and techniques, the datasets that were used, and the different evaluation metrics. Finally, we highlight literature gaps and research opportunities.
Collapse
|
11
|
Cui X, Cao Y, Liu Z, Sui X, Mi J, Zhang Y, Cui L, Li S. TRSA-Net: Task Relation Spatial co-Attention for Joint Segmentation, Quantification and Uncertainty Estimation on Paired 2D Echocardiography. IEEE J Biomed Health Inform 2022; 26:4067-4078. [PMID: 35503848 DOI: 10.1109/jbhi.2022.3171985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Clinical workflow of cardiac assessment on 2D echocardiography requires both accurate segmentation and quantification of the Left Ventricle (LV) from paired apical 4-chamber and 2-chamber. Moreover, uncertainty estimation is significant in clinically understanding the performance of a model. However, current research on 2D echocardiography ignores this vital task while joint segmentation with quantification, hence motivating the need for a unified optimization method. In this paper, we propose a multitask model with Task Relation Spatial co-Attention (referred as TRSA-Net) for joint segmentation, quantification, and uncertainty estimation on paired 2D echo. TRSA-Net achieves multitask joint learning by novelly exploring the spatial correlation between tasks. The task relation spatial co-attention learns the spatial mapping among task-specific features by non-local and co-excitation, which forcibly joints embedded spatial information in the segmentation and quantification. The Boundary-aware Structure Consistency (BSC) and Joint Indices Constraint (JIC) are integrated into the multitask learning optimization objective to guide the learning of segmentation and quantification paths. The BSC creatively promotes structural similarity of predictions, and JIC explores the internal relationship between three quantitative indices. We validate the efficacy of our TRSA-Net on the public CAMUS dataset. Extensive comparison and ablation experiments show that our approach can achieve competitive segmentation performance and highly accurate results on quantification.
Collapse
|
12
|
Ensemble of 2D Residual Neural Networks Integrated with Atrous Spatial Pyramid Pooling Module for Myocardium Segmentation of Left Ventricle Cardiac MRI. MATHEMATICS 2022. [DOI: 10.3390/math10040627] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Cardiac disease diagnosis and identification is problematic mostly by inaccurate segmentation of the cardiac left ventricle (LV). Besides, LV segmentation is challenging since it involves complex and variable cardiac structures in terms of components and the intricacy of time-based crescendos. In addition, full segmentation and quantification of the LV myocardium border is even more challenging because of different shapes and sizes of the myocardium border zone. The foremost purpose of this research is to design a precise automatic segmentation technique employing deep learning models for the myocardium border using cardiac magnetic resonance imaging (MRI). The ASPP module (Atrous Spatial Pyramid Pooling) was integrated with a proposed 2D-residual neural network for segmentation of the myocardium border using a cardiac MRI dataset. Further, the ensemble technique based on a majority voting ensemble method was used to blend the results of recent deep learning models on different set of hyperparameters. The proposed model produced an 85.43% dice score on validation samples and 98.23% on training samples and provided excellent performance compared to recent deep learning models. The myocardium border was successfully segmented across diverse subject slices with different shapes, sizes and contrast using the proposed deep learning ensemble models. The proposed model can be employed for automatic detection and segmentation of the myocardium border for precise quantification of reflow, myocardial infarction, myocarditis, and h cardiomyopathy (HCM) for clinical applications.
Collapse
|
13
|
Karpiel I, Ziębiński A, Kluszczyński M, Feige D. A Survey of Methods and Technologies Used for Diagnosis of Scoliosis. SENSORS (BASEL, SWITZERLAND) 2021; 21:8410. [PMID: 34960509 PMCID: PMC8707023 DOI: 10.3390/s21248410] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 12/04/2021] [Accepted: 12/09/2021] [Indexed: 02/07/2023]
Abstract
The purpose of this article is to present diagnostic methods used in the diagnosis of scoliosis in the form of a brief review. This article aims to point out the advantages of select methods. This article focuses on general issues without elaborating on problems strictly related to physiotherapy and treatment methods, which may be the subject of further discussions. By outlining and categorizing each method, we summarize relevant publications that may not only help introduce other researchers to the field but also be a valuable source for studying existing methods, developing new ones or choosing evaluation strategies.
Collapse
Affiliation(s)
- Ilona Karpiel
- Łukasiewicz Research Network—Institute of Medical Technology and Equipment, 118 Roosevelt, 41-800 Zabrze, Poland;
| | - Adam Ziębiński
- Department of Distributed Systems and Informatic Devices, Silesian University of Technology, 16 Akademicka, 44-100 Gliwice, Poland;
| | - Marek Kluszczyński
- Department of Health Sciences, Jan Dlugosz University, 4/8 Waszyngtona, 42-200 Częstochowa, Poland;
| | - Daniel Feige
- Łukasiewicz Research Network—Institute of Medical Technology and Equipment, 118 Roosevelt, 41-800 Zabrze, Poland;
- Department of Distributed Systems and Informatic Devices, Silesian University of Technology, 16 Akademicka, 44-100 Gliwice, Poland;
- PhD School, Silesian University of Technology, 2A Akademicka, 44-100 Gliwice, Poland
| |
Collapse
|
14
|
Langner T, Gustafsson FK, Avelin B, Strand R, Ahlström H, Kullberg J. Uncertainty-aware body composition analysis with deep regression ensembles on UK Biobank MRI. Comput Med Imaging Graph 2021; 93:101994. [PMID: 34624770 DOI: 10.1016/j.compmedimag.2021.101994] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2021] [Revised: 09/06/2021] [Accepted: 09/07/2021] [Indexed: 11/25/2022]
Abstract
Along with rich health-related metadata, medical images have been acquired for over 40,000 male and female UK Biobank participants, aged 44-82, since 2014. Phenotypes derived from these images, such as measurements of body composition from MRI, can reveal new links between genetics, cardiovascular disease, and metabolic conditions. In this work, six measurements of body composition and adipose tissues were automatically estimated by image-based, deep regression with ResNet50 neural networks from neck-to-knee body MRI. Despite the potential for high speed and accuracy, these networks produce no output segmentations that could indicate the reliability of individual measurements. The presented experiments therefore examine uncertainty quantification with mean-variance regression and ensembling to estimate individual measurement errors and thereby identify potential outliers, anomalies, and other failure cases automatically. In 10-fold cross-validation on data of about 8500 subjects, mean-variance regression and ensembling showed complementary benefits, reducing the mean absolute error across all predictions by 12%. Both improved the calibration of uncertainties and their ability to identify high prediction errors. With intra-class correlation coefficients (ICC) above 0.97, all targets except the liver fat content yielded relative measurement errors below 5%. Testing on another 1000 subjects showed consistent performance, and the method was finally deployed for inference to 30,000 subjects with missing reference values. The results indicate that deep regression ensembles could ultimately provide automated, uncertainty-aware measurements of body composition for more than 120,000 UK Biobank neck-to-knee body MRI that are to be acquired within the coming years.
Collapse
Affiliation(s)
- Taro Langner
- Uppsala University, Department of Surgical Sciences, Uppsala, Sweden.
| | | | - Benny Avelin
- Uppsala University, Department of Mathematics, Uppsala, Sweden
| | - Robin Strand
- Uppsala University, Department of Information Technology, Uppsala, Sweden
| | - Håkan Ahlström
- Uppsala University, Department of Surgical Sciences, Uppsala, Sweden; Antaros Medical AB, BioVenture Hub, Mölndal, Sweden
| | - Joel Kullberg
- Uppsala University, Department of Surgical Sciences, Uppsala, Sweden; Antaros Medical AB, BioVenture Hub, Mölndal, Sweden
| |
Collapse
|
15
|
Xue W, Li J, Hu Z, Kerfoot E, Clough J, Oksuz I, Xu H, Grau V, Guo F, Ng M, Li X, Li Q, Liu L, Ma J, Grinias E, Tziritas G, Yan W, Atehortúa A, Garreau M, Jang Y, Debus A, Ferrante E, Yang G, Hua T, Li S. Left Ventricle Quantification Challenge: A Comprehensive Comparison and Evaluation of Segmentation and Regression for Mid-Ventricular Short-Axis Cardiac MR Data. IEEE J Biomed Health Inform 2021; 25:3541-3553. [PMID: 33684050 PMCID: PMC7611810 DOI: 10.1109/jbhi.2021.3064353] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Automatic quantification of the left ventricle (LV) from cardiac magnetic resonance (CMR) images plays an important role in making the diagnosis procedure efficient, reliable, and alleviating the laborious reading work for physicians. Considerable efforts have been devoted to LV quantification using different strategies that include segmentation-based (SG) methods and the recent direct regression (DR) methods. Although both SG and DR methods have obtained great success for the task, a systematic platform to benchmark them remains absent because of differences in label information during model learning. In this paper, we conducted an unbiased evaluation and comparison of cardiac LV quantification methods that were submitted to the Left Ventricle Quantification (LVQuan) challenge, which was held in conjunction with the Statistical Atlases and Computational Modeling of the Heart (STACOM) workshop at the MICCAI 2018. The challenge was targeted at the quantification of 1) areas of LV cavity and myocardium, 2) dimensions of the LV cavity, 3) regional wall thicknesses (RWT), and 4) the cardiac phase, from mid-ventricle short-axis CMR images. First, we constructed a public quantification dataset Cardiac-DIG with ground truth labels for both the myocardium mask and these quantification targets across the entire cardiac cycle. Then, the key techniques employed by each submission were described. Next, quantitative validation of these submissions were conducted with the constructed dataset. The evaluation results revealed that both SG and DR methods can offer good LV quantification performance, even though DR methods do not require densely labeled masks for supervision. Among the 12 submissions, the DR method LDAMT offered the best performance, with a mean estimation error of 301 mm 2 for the two areas, 2.15 mm for the cavity dimensions, 2.03 mm for RWTs, and a 9.5% error rate for the cardiac phase classification. Three of the SG methods also delivered comparable performances. Finally, we discussed the advantages and disadvantages of SG and DR methods, as well as the unsolved problems in automatic cardiac quantification for clinical practice applications.
Collapse
Affiliation(s)
- Wufeng Xue
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, China; Department of Medical Imaging, Western University, London, ON N6A 3K7, Canada
| | - Jiahui Li
- Beijing University of Post and Telecommunication, Beijing, China
| | | | - Eric Kerfoot
- School of Biomedical Engineering & Imaging Sciences, King’s College London, UK
| | - James Clough
- School of Biomedical Engineering & Imaging Sciences, King’s College London, UK
| | - Ilkay Oksuz
- School of Biomedical Engineering & Imaging Sciences, King’s College London, UK
| | - Hao Xu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Vicente Grau
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Fumin Guo
- Sunnybrook Research Institute, Department of Medical Biophysics, University of Toronto, Canada
| | - Matthew Ng
- Sunnybrook Research Institute, Department of Medical Biophysics, University of Toronto, Canada
| | - Xiang Li
- Department of Radiology, Massachusetts General Hospital, Boston, USA
| | - Quanzheng Li
- Department of Radiology, Massachusetts General Hospital, Boston, USA
| | - Lihong Liu
- Pingan Technology (Shenzhen) Co.Ltd. Elias Grinias and Georgios Tziritas are with Department of Computer Science, University of Crete, Heraklion, Greece
| | - Jin Ma
- Pingan Technology (Shenzhen) Co.Ltd. Elias Grinias and Georgios Tziritas are with Department of Computer Science, University of Crete, Heraklion, Greece
| | - Elias Grinias
- Department of Computer Science, University of Crete, Heraklion, Greece
| | - Georgios Tziritas
- Department of Computer Science, University of Crete, Heraklion, Greece
| | - Wenjun Yan
- Department of Electrical Engineering, Fudan University, Shanghai, China
| | - Angélica Atehortúa
- LTSI UMR 1099, F-35000 Rennes, France; Universidad Nacional de Colombia, Bogotá, Colombia
| | | | - Yeonggul Jang
- Brain Korea 21 PLUS Project for Medical Science, Yonsei University
| | - Alejandro Debus
- Research Institute for Signals, Systems and Computational Intelligence, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
| | - Enzo Ferrante
- Research Institute for Signals, Systems and Computational Intelligence, sinc(i), FICH-UNL/CONICET, Santa Fe, Argentina
| | - Guanyu Yang
- Centre de Recherche en Information Biomédicale Sino-Français (CRIBs), Southeast University, Nanjing, China; LIST, Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China
| | - Tiancong Hua
- Centre de Recherche en Information Biomedicale Sino-Francais (CRIBs), Southeast University, Nanjing, China
| | - Shuo Li
- Department of Medical Imaging, Western University, London, ON N6A 3K7, Canada
| |
Collapse
|
16
|
Huang X, Tian Y, Zhao S, Liu T, Wang W, Wang Q. Direct full quantification of the left ventricle via multitask regression and classification. APPL INTELL 2021. [DOI: 10.1007/s10489-020-02130-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
17
|
Vesal S, Gu M, Maier A, Ravikumar N. Spatio-Temporal Multi-Task Learning for Cardiac MRI Left Ventricle Quantification. IEEE J Biomed Health Inform 2021; 25:2698-2709. [PMID: 33351771 DOI: 10.1109/jbhi.2020.3046449] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Quantitative assessment of cardiac left ventricle (LV) morphology is essential to assess cardiac function and improve the diagnosis of different cardiovascular diseases. In current clinical practice, LV quantification depends on the measurement of myocardial shape indices, which is usually achieved by manual contouring of the endo- and epicardial. However, this process subjected to inter and intra-observer variability, and it is a time-consuming and tedious task. In this article, we propose a spatio-temporal multi-task learning approach to obtain a complete set of measurements quantifying cardiac LV morphology, regional-wall thickness (RWT), and additionally detecting the cardiac phase cycle (systole and diastole) for a given 3D Cine-magnetic resonance (MR) image sequence. We first segment cardiac LVs using an encoder-decoder network and then introduce a multitask framework to regress 11 LV indices and classify the cardiac phase, as parallel tasks during model optimization. The proposed deep learning model is based on the 3D spatio-temporal convolutions, which extract spatial and temporal features from MR images. We demonstrate the efficacy of the proposed method using cine-MR sequences of 145 subjects and comparing the performance with other state-of-the-art quantification methods. The proposed method obtained high prediction accuracy, with an average mean absolute error (MAE) of 129 mm 2, 1.23 mm, 1.76 mm, Pearson correlation coefficient (PCC) of 96.4%, 87.2%, and 97.5% for LV and myocardium (Myo) cavity regions, 6 RWTs, 3 LV dimensions, and an error rate of 9.0% for phase classification. The experimental results highlight the robustness of the proposed method, despite varying degrees of cardiac morphology, image appearance, and low contrast in the cardiac MR sequences.
Collapse
|
18
|
Automatic segmentation of the cardiac MR images based on nested fully convolutional dense network with dilated convolution. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102684] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
19
|
Leclerc S, Smistad E, Ostvik A, Cervenansky F, Espinosa F, Espeland T, Rye Berg EA, Belhamissi M, Israilov S, Grenier T, Lartizien C, Jodoin PM, Lovstakken L, Bernard O. LU-Net: A Multistage Attention Network to Improve the Robustness of Segmentation of Left Ventricular Structures in 2-D Echocardiography. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2020; 67:2519-2530. [PMID: 32746187 DOI: 10.1109/tuffc.2020.3003403] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semiautomatically in clinical routine and is, thus, prone to interobserver and intraobserver variabilities. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a lack of robustness in terms of accuracy and number of outliers. The goal of this work is to introduce a novel network designed to improve the overall segmentation accuracy of left ventricular structures (endocardial and epicardial borders) while enhancing the estimation of the corresponding clinical indices and reducing the number of outliers. This network is based on a multistage framework where both the localization and segmentation steps are optimized jointly through an end-to-end scheme. Results obtained on a large open access data set show that our method outperforms the current best-performing deep learning solution with a lighter architecture and achieved an overall segmentation accuracy lower than the intraobserver variability for the epicardial border (i.e., on average a mean absolute error of 1.5 mm and a Hausdorff distance of 5.1mm) with 11% of outliers. Moreover, we demonstrate that our method can closely reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.96 and a mean absolute error of 7.6 ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.83 and an absolute mean error of 5.0%, producing scores that are slightly below the intraobserver margin. Based on this observation, areas for improvement are suggested.
Collapse
|
20
|
Zhang D, Yang G, Zhao S, Zhang Y, Ghista D, Zhang H, Li S. Direct Quantification of Coronary Artery Stenosis Through Hierarchical Attentive Multi-View Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:4322-4334. [PMID: 32804646 DOI: 10.1109/tmi.2020.3017275] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Quantification of coronary artery stenosis on X-ray angiography (XRA) images is of great importance during the intraoperative treatment of coronary artery disease. It serves to quantify the coronary artery stenosis by estimating the clinical morphological indices, which are essential in clinical decision making. However, stenosis quantification is still a challenging task due to the overlapping, diversity and small-size region of the stenosis in the XRA images. While efforts have been devoted to stenosis quantification through low-level features, these methods have difficulty in learning the real mapping from these features to the stenosis indices. These methods are still cumbersome and unreliable for the intraoperative procedures due to their two-phase quantification, which depends on the results of segmentation or reconstruction of the coronary artery. In this work, we are proposing a hierarchical attentive multi-view learning model (HEAL) to achieve a direct quantification of coronary artery stenosis, without the intermediate segmentation or reconstruction. We have designed a multi-view learning model to learn more complementary information of the stenosis from different views. For this purpose, an intra-view hierarchical attentive block is proposed to learn the discriminative information of stenosis. Additionally, a stenosis representation learning module is developed to extract the multi-scale features from the keyframe perspective for considering the clinical workflow. Finally, the morphological indices are directly estimated based on the multi-view feature embedding. Extensive experiment studies on clinical multi-manufacturer dataset consisting of 228 subjects show the superiority of our HEAL against nine comparing methods, including direct quantification methods and multi-view learning methods. The experimental results demonstrate the better clinical agreement between the ground truth and the prediction, which endows our proposed method with a great potential for the efficient intraoperative treatment of coronary artery disease.
Collapse
|
21
|
Pyo J, Hong SM, Kwon YS, Kim MS, Cho KH. Estimation of heavy metals using deep neural network with visible and infrared spectroscopy of soil. THE SCIENCE OF THE TOTAL ENVIRONMENT 2020; 741:140162. [PMID: 32886995 DOI: 10.1016/j.scitotenv.2020.140162] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2020] [Revised: 06/09/2020] [Accepted: 06/10/2020] [Indexed: 06/11/2023]
Abstract
Heavy metal contamination in soil disturbs the chemical, biological, and physical soil conditions and adversely affects the health of living organisms. Visible and near-infrared spectroscopy (VNIRS) shows a potential feasibility for estimating heavy metal elements in soil. Moreover, deep learning models have been shown to successfully deal with complex multi-dimensional and multivariate nonlinear data. Thus, this study implemented a deep learning method on reflectance spectra of soil samples to estimate heavy metal concentrations. A convolutional neural network (CNN) was adopted to estimate arsenic (As), copper (Cu), and lead (Pb) concentrations using measured soil reflectance. In addition, a convolutional autoencoder was utilized as a joint method with the CNN for dimensionality reduction of the reflectance spectra. Furthermore, artificial neural network (ANN) and random forest regression (RFR) models were built for heavy metal estimation. Principal component analysis was utilized for dimensionality reduction of the ANN and RFR models. Among these models, the CNN model with convolutional autoencoder showed the highest accuracies for As, Cu, and Pb estimates, having R2 values of 0.86, 0.74, and 0.82, respectively. The convolutional autoencoder disentangled the relevant features of multiple heavy metal elements and delivered robust features to the CNN model, thereby generating relatively accurate estimates.
Collapse
Affiliation(s)
- JongCheol Pyo
- School of Urban and Environmental Engineering, Ulsan National Institute of Science and Technology, Ulsan, 689-798, Republic of Korea
| | - Seok Min Hong
- School of Urban and Environmental Engineering, Ulsan National Institute of Science and Technology, Ulsan, 689-798, Republic of Korea
| | - Yong Sung Kwon
- School of Urban and Environmental Engineering, Ulsan National Institute of Science and Technology, Ulsan, 689-798, Republic of Korea
| | - Moon Sung Kim
- Environmental Microbial and Food Safety Laboratory, USDA Agricultural Research Service, Beltsville, MD 20705, USA
| | - Kyung Hwa Cho
- School of Urban and Environmental Engineering, Ulsan National Institute of Science and Technology, Ulsan, 689-798, Republic of Korea.
| |
Collapse
|
22
|
Langner T, Strand R, Ahlström H, Kullberg J. Large-scale biometry with interpretable neural network regression on UK Biobank body MRI. Sci Rep 2020; 10:17752. [PMID: 33082454 PMCID: PMC7576214 DOI: 10.1038/s41598-020-74633-5] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2020] [Accepted: 10/05/2020] [Indexed: 11/14/2022] Open
Abstract
In a large-scale medical examination, the UK Biobank study has successfully imaged more than 32,000 volunteer participants with magnetic resonance imaging (MRI). Each scan is linked to extensive metadata, providing a comprehensive medical survey of imaged anatomy and related health states. Despite its potential for research, this vast amount of data presents a challenge to established methods of evaluation, which often rely on manual input. To date, the range of reference values for cardiovascular and metabolic risk factors is therefore incomplete. In this work, neural networks were trained for image-based regression to infer various biological metrics from the neck-to-knee body MRI automatically. The approach requires no manual intervention or direct access to reference segmentations for training. The examined fields span 64 variables derived from anthropometric measurements, dual-energy X-ray absorptiometry (DXA), atlas-based segmentations, and dedicated liver scans. With the ResNet50, the standardized framework achieves a close fit to the target values (median R\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$^2 > 0.97$$\end{document}2>0.97) in cross-validation. Interpretation of aggregated saliency maps suggests that the network correctly targets specific body regions and limbs, and learned to emulate different modalities. On several body composition metrics, the quality of the predictions is within the range of variability observed between established gold standard techniques.
Collapse
Affiliation(s)
- Taro Langner
- Department of Surgical Sciences, Uppsala University, 751 85, Uppsala, Sweden.
| | - Robin Strand
- Department of Surgical Sciences, Uppsala University, 751 85, Uppsala, Sweden.,Department of Information Technology, Uppsala University, 751 85, Uppsala, Sweden
| | - Håkan Ahlström
- Department of Surgical Sciences, Uppsala University, 751 85, Uppsala, Sweden.,Antaros Medical AB, BioVenture Hub, 431 53, Mölndal, Sweden
| | - Joel Kullberg
- Department of Surgical Sciences, Uppsala University, 751 85, Uppsala, Sweden.,Antaros Medical AB, BioVenture Hub, 431 53, Mölndal, Sweden
| |
Collapse
|
23
|
Ge R, Yang G, Chen Y, Luo L, Feng C, Ma H, Ren J, Li S. K-Net: Integrate Left Ventricle Segmentation and Direct Quantification of Paired Echo Sequence. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:1690-1702. [PMID: 31765307 DOI: 10.1109/tmi.2019.2955436] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The integration of segmentation and direct quantification on the left ventricle (LV) from the paired apical views(i.e., apical 4-chamber and 2-chamber together) echo sequence clinically achieves the comprehensive cardiac assessment: multiview segmentation for anatomical morphology, and multidimensional quantification for contractile function. Direct quantification of LV, i.e., to automatically quantify multiple LV indices directly from the image via task-aware feature representation and regression, avoids accumulative error from the inter-step target. This integration sequentially makes a stereoscopical reflection of cardiac activity jointly from the paired orthogonal cross views sequences, overcoming limited observation with a single plane. We propose a K-shaped Unified Network (K-Net), the first end-to-end framework to simultaneously segment LV from apical 4-chamber and 2-chamber views, and directly quantify LV from major- and minor-axis dimensions (1D), area (2D), and volume (3D), in sequence. It works via four components: 1) the K-Net architecture with the Attention Junction enables heterogeneous tasks learning of segmentation task of pixel-wise classification, and direct quantification task of image-wise regression, by interactively introducing the information from segmentation to jointly promote spatial attention map to guide quantification focusing on LV-related region, and transferring quantification feedback to make global constraint on segmentation; 2) the Bi-ResLSTMs distributed in K-Net layer-by-layer hierarchically extract spatial-temporal information in echo sequence, with bidirectional recurrent and short-cut connection to model spatial-temporal information among all frames; 3) the Information Valve tailing the Bi-ResLSTMs selectively exchanges information among multiple views, by stimulating complementary information and suppressing redundant information to make the efficient cross-flow for each view; 4) the Evolution Loss comprehensively guides sequential data learning, with static constraint for frame values, and dynamic constraint for inter-frame value changes. The experiments show that our K-Net gains high performance with a Dice coefficient up to 91.44% and a mean absolute error of the major-axis dimension down to 2.74mm, which reveal its clinical potential.
Collapse
|
24
|
Dong S, Luo G, Tam C, Wang W, Wang K, Cao S, Chen B, Zhang H, Li S. Deep Atlas Network for Efficient 3D Left Ventricle Segmentation on Echocardiography. Med Image Anal 2020; 61:101638. [DOI: 10.1016/j.media.2020.101638] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2019] [Revised: 01/06/2020] [Accepted: 01/09/2020] [Indexed: 10/25/2022]
|
25
|
Zhao R, Chen X, Liu X, Chen Z, Guo F, Li S. Direct Cup-to-Disc Ratio Estimation for Glaucoma Screening via Semi-Supervised Learning. IEEE J Biomed Health Inform 2020; 24:1104-1113. [DOI: 10.1109/jbhi.2019.2934477] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
26
|
Lin L, Tao X, Pang S, Su Z, Lu H, Li S, Feng Q, Chen B. Multiple Axial Spine Indices Estimation via Dense Enhancing Network With Cross-Space Distance-Preserving Regularization. IEEE J Biomed Health Inform 2020; 24:3248-3257. [PMID: 32142463 DOI: 10.1109/jbhi.2020.2977224] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Automatic estimation of axial spine indices is clinically desired for various spine computer aided procedures, such as disease diagnosis, therapeutic evaluation, pathophysiological understanding, risk assessment, and biomechanical modeling. Currently, the spine indices are manually measured by physicians, which is time-consuming and laborious. Even worse, the tedious manual procedure might result in inaccurate measurement. To deal with this problem, in this paper, we aim at developing an automatic method to estimate multiple indices from axial spine images. Inspired by the success of deep learning for regression problems and the densely connected network for image classification, we propose a dense enhancing network (DE-Net) which uses the dense enhancing blocks (DEBs) as its main body, where a feature enhancing layer is added to each of the bypass in a dense block. The DEB is designed to enhance discriminative feature embedding from the intervertebral disc and the dural sac areas. In addition, the cross-space distance-preserving regularization (CSDPR), which enforces consistent inter-sample distances between the output and the label spaces, is proposed to regularize the loss function of the DE-Net. To train and validate the proposed method, we collected 895 axial spine MRI images from 143 subjects and manually measured the indices as the ground truth. The results show that all deep learning models obtain very small prediction errors, and the proposed DE-Net with CSDPR acquires the smallest error among all methods, indicating that our method has great potential for spine computer aided procedures.
Collapse
|
27
|
Wu H, Gao R, Sheng YP, Chen B, Li S. SDAE-GAN: Enable high-dimensional pathological images in liver cancer survival prediction with a policy gradient based data augmentation method. Med Image Anal 2020; 62:101640. [PMID: 32120270 DOI: 10.1016/j.media.2020.101640] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2019] [Revised: 12/09/2019] [Accepted: 01/09/2020] [Indexed: 02/07/2023]
Abstract
High-dimensional pathological images produced by Immunohistochemistry (IHC) methods consist of many pathological indexes, which play critical roles in cancer treatment planning. However, these indexes currently cannot be utilized in survival prediction because joining them with patients' clinicopathological features (e.g., age and tumor size) is challenging due to their high dimension and sparse features. To address this problem, we propose a novel two-stage survival prediction model named ICSPM to join the IHC images and clinicopathological features. For the first stage, our proposed SDAE-GAN compresses high-dimensional IHC images to flat, compact and representative feature vectors by compressing and reconstructing them. For the first time, SDAE-GAN integrates dense blocks, the stacked auto-encoder and the GAN architecture to maximize the ability to detect patterns in IHC images. In addition, we propose a novel policy gradient based data augmentation method to involve the diversity in IHC images without breaking patterns inside them. For the second stage, ICSPM adopts a DenseNet to join feature vectors and clinicopathological features for survival prediction. Experimental results demonstrate that ICSPM reached a state-of-the-art prediction accuracy of 0.72 on the five-year survival. ICSPM is the first work to enable high-dimensional IHC images in cancer survival prediction. We prove that high-dimensional IHC images and clinicopathological features provide valuable and complementary information in survival prediction.
Collapse
Affiliation(s)
- Hejun Wu
- Department of Computer Science, Sun Yat-sen University, Guangzhou, China
| | - Rong Gao
- Department of Computer Science, Sun Yat-sen University, Guangzhou, China
| | | | - Bo Chen
- School of Health Science, Western University, Canada
| | - Shuo Li
- Department of Medical Imaging and Medical Biophysics, Western University, Canada; Digital Imaging Group of London, Canada.
| |
Collapse
|
28
|
Chen R, Xu C, Dong Z, Liu Y, Du X. DeepCQ: Deep multi-task conditional quantification network for estimation of left ventricle parameters. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 184:105288. [PMID: 31901611 DOI: 10.1016/j.cmpb.2019.105288] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 12/13/2019] [Accepted: 12/18/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic cardiac left ventricle (LV) quantification plays an important role in assessing cardiac function. Although many advanced methods have been put forward to quantify related LV parameters, automatic cardiac LV quantification is still a challenge task due to the anatomy construction complexity of heart. METHODS In this work, we propose a novel deep multi-task conditional quantification learning model (DeepCQ) which contains Segmentation module, Quantification encoder, and Dynamic analysis module. Besides, we also use task uncertainty loss function to update the parameters of the network in training. RESULTS The proposed framework is validated on the dataset from Left Ventricle Full Quantification Challenge MICCAI 2018 (https://lvquan18.github.io/). The experimental results show that DeepCQ outperforms the other advanced methods. CONCLUSIONS It illustrates that our method has a great potential in comprehensive cardiac function assessment and could play an auxiliary role in clinicians' diagnosis.
Collapse
Affiliation(s)
- Ruifeng Chen
- School of Computer Science and Technology, Anhui University, Anhui, China
| | - Chenchu Xu
- Department of Medical Imaging, Western University, London, Canada.
| | - Zhangfu Dong
- School of Computer Science and Technology, Anhui University, Anhui, China
| | - Yueguo Liu
- School of Computer Science and Technology, Anhui University, Anhui, China
| | - Xiuquan Du
- Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University, Anhui, China; School of Computer Science and Technology, Anhui University, Anhui, China.
| |
Collapse
|
29
|
Luo G, Dong S, Wang W, Wang K, Cao S, Tam C, Zhang H, Howey J, Ohorodnyk P, Li S. Commensal correlation network between segmentation and direct area estimation for bi-ventricle quantification. Med Image Anal 2020; 59:101591. [DOI: 10.1016/j.media.2019.101591] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Revised: 08/25/2019] [Accepted: 10/21/2019] [Indexed: 10/25/2022]
|
30
|
Ge R, Yang G, Chen Y, Luo L, Feng C, Zhang H, Li S. PV-LVNet: Direct left ventricle multitype indices estimation from 2D echocardiograms of paired apical views with deep neural networks. Med Image Anal 2019; 58:101554. [DOI: 10.1016/j.media.2019.101554] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2018] [Revised: 05/15/2019] [Accepted: 09/04/2019] [Indexed: 11/16/2022]
|
31
|
Wang L, Xu Q, Leung S, Chung J, Chen B, Li S. Accurate automated Cobb angles estimation using multi-view extrapolation net. Med Image Anal 2019; 58:101542. [DOI: 10.1016/j.media.2019.101542] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2018] [Revised: 06/02/2019] [Accepted: 08/01/2019] [Indexed: 10/26/2022]
|
32
|
Abstract
OBJECTIVE. The recent advancement of deep learning techniques has profoundly impacted research on quantitative cardiac MRI analysis. The purpose of this article is to introduce the concept of deep learning, review its current applications on quantitative cardiac MRI, and discuss its limitations and challenges. CONCLUSION. Deep learning has shown state-of-the-art performance on quantitative analysis of multiple cardiac MRI sequences and holds great promise for future use in clinical practice and scientific research.
Collapse
|
33
|
Tobore I, Li J, Yuhang L, Al-Handarish Y, Kandwal A, Nie Z, Wang L. Deep Learning Intervention for Health Care Challenges: Some Biomedical Domain Considerations. JMIR Mhealth Uhealth 2019; 7:e11966. [PMID: 31376272 PMCID: PMC6696854 DOI: 10.2196/11966] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2018] [Revised: 04/14/2019] [Accepted: 06/12/2019] [Indexed: 01/10/2023] Open
Abstract
The use of deep learning (DL) for the analysis and diagnosis of biomedical and health care problems has received unprecedented attention in the last decade. The technique has recorded a number of achievements for unearthing meaningful features and accomplishing tasks that were hitherto difficult to solve by other methods and human experts. Currently, biological and medical devices, treatment, and applications are capable of generating large volumes of data in the form of images, sounds, text, graphs, and signals creating the concept of big data. The innovation of DL is a developing trend in the wake of big data for data representation and analysis. DL is a type of machine learning algorithm that has deeper (or more) hidden layers of similar function cascaded into the network and has the capability to make meaning from medical big data. Current transformation drivers to achieve personalized health care delivery will be possible with the use of mobile health (mHealth). DL can provide the analysis for the deluge of data generated from mHealth apps. This paper reviews the fundamentals of DL methods and presents a general view of the trends in DL by capturing literature from PubMed and the Institute of Electrical and Electronics Engineers database publications that implement different variants of DL. We highlight the implementation of DL in health care, which we categorize into biological system, electronic health record, medical image, and physiological signals. In addition, we discuss some inherent challenges of DL affecting biomedical and health domain, as well as prospective research directions that focus on improving health management by promoting the application of physiological signals and modern internet technology.
Collapse
Affiliation(s)
- Igbe Tobore
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China.,Graduate University, Chinese Academy of Sciences, Beijing, China
| | - Jingzhen Li
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Liu Yuhang
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yousef Al-Handarish
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Abhishek Kandwal
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Zedong Nie
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Lei Wang
- Center for Medical Robotics and Minimally Invasive Surgical Devices, Shenzhen Institutes of Advance Technology, Chinese Academy of Sciences, Shenzhen, China
| |
Collapse
|
34
|
Du X, Yin S, Tang R, Zhang Y, Li S. Cardiac-DeepIED: Automatic Pixel-Level Deep Segmentation for Cardiac Bi-Ventricle Using Improved End-to-End Encoder-Decoder Network. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2019; 7:1900110. [PMID: 30949419 PMCID: PMC6442749 DOI: 10.1109/jtehm.2019.2900628] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/05/2018] [Revised: 01/20/2019] [Accepted: 02/14/2019] [Indexed: 12/04/2022]
Abstract
Accurate segmentation of cardiac bi-ventricle (CBV) from magnetic resonance (MR) images has a great significance to analyze and evaluate the function of the cardiovascular system. However, the complex structure of CBV image makes fully automatic segmentation as a well-known challenge. In this paper, we propose an improved end-to-end encoder-decoder network for CBV segmentation from the pixel level view (Cardiac-DeepIED). In our framework, we explicitly solve the high variability of complex cardiac structures through an improved encoder-decoder architecture which consists of Fire dilated modules and D-Fire dilated modules. This improved encoder-decoder architecture has the advantages of being capable of obtaining semantic task-aware representation and preserving fine-grained information. In addition, our method can dynamically capture potential spatiotemporal correlations between consecutive cardiac MR images through specially designed convolutional long-term and short-term memory structure; it can simulate spatiotemporal contexts between consecutive frame images. The combination of these modules enables the entire network to get an accurate, robust segmentation result. The proposed method is evaluated on the 145 clinical subjects with leave-one-out cross-validation. The average dice metric (DM) is up to 0.96 (left ventricle), 0.89 (myocardium), and 0.903 (right ventricle). The performance of our method outperforms state-of-the-art methods. These results demonstrate the effectiveness and advantages of our method for CBV regions segmentation at the pixel-level. It also reveals the proposed automated segmentation system can be embedded into the clinical environment to accelerate the quantification of CBV and expanded to volume analysis, regional wall thickness analysis, and three LV dimensions analysis.
Collapse
Affiliation(s)
- Xiuquan Du
- School of Computer Science and TechnologyAnhui UniversityHefei230039China
| | - Susu Yin
- School of Computer Science and TechnologyAnhui UniversityHefei230039China
| | - Renjun Tang
- School of Computer Science and TechnologyAnhui UniversityHefei230039China
| | - Yanping Zhang
- School of Computer Science and TechnologyAnhui UniversityHefei230039China
| | - Shuo Li
- Department of Medical ImagingWestern UniversityLondonONN6A 3K7Canada
- Digital Imaging Group of LondonLondonONN6A 3K7Canada
| |
Collapse
|
35
|
Unsupervised pathology detection in medical images using conditional variational autoencoders. Int J Comput Assist Radiol Surg 2018; 14:451-461. [PMID: 30542975 DOI: 10.1007/s11548-018-1898-0] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2018] [Accepted: 12/04/2018] [Indexed: 10/27/2022]
Abstract
PURPOSE Pathology detection in medical image data is an important but a rather complicated task. In particular, the big variability of the pathologies is a challenge to automatic detection methods and even to machine learning methods. Supervised algorithms would usually learn the appearance of a single pathological structure based on a large annotated dataset. As such data is not usually available, especially in large amounts, in this work we pursue a different unsupervised approach. METHODS Our method is based on learning the entire variability of healthy data and detect pathologies by their differences to the learned norm. For this purpose, we use conditional variational autoencoders which learn the reconstruction and encoding distribution of healthy images and also have the ability to integrate certain prior knowledge about the data (condition). RESULTS Our experiments on different 2D and 3D datasets show that the approach is suitable for the detection of pathologies and deliver reasonable Dice coefficients and AUCs. Also this method can estimate missing correspondences in pathological images and thus can be used as a pre-step to a registration method. Our experiments show improving registration results on pathological data when using this approach. CONCLUSIONS Overall the presented approach is suitable for a rough pathology detection in medical images and can be successfully used as a preprocessing step to other image processing methods.
Collapse
|
36
|
Du X, Tang R, Yin S, Zhang Y, Li S. Direct Segmentation-Based Full Quantification for Left Ventricle via Deep Multi-Task Regression Learning Network. IEEE J Biomed Health Inform 2018; 23:942-948. [PMID: 30387757 DOI: 10.1109/jbhi.2018.2879188] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Quantitative analysis of the heart is extremely necessary and significant for detecting and diagnosing heart disease, yet there are still some challenges. In this study, we propose a new end-to-end segmentation-based deep multi-task regression learning model (Indices-JSQ) to make a holonomic quantitative analysis of the left ventricle (LV), which contains a segmentation network (Img2Contour) and multi-task regression network (Contour2Indices). First, Img2Contour, which contains a deep convolutional encoder-decoder module, is designed to obtain the LV contour. Then, the predicted contour is fed as input to Contour2Indices for full quantification. On the whole, we take into account the relationship between different tasks, which can serve as a complementary advantage. Meanwhile, instead of using images directly from the original dataset, we creatively use the segmented contour of the original image to estimate the cardiac indices to achieve better and more accurate results. We make experiments on MR sequences of 145 subjects and gain the experimental results of 157 mm 2, 2.43 mm, 1.29 mm, and 0.87 on areas, dimensions, regional wall thicknesses, and Dice Metric, respectively. It intuitively shows that the proposed method outperforms the other state-of-the-art methods and demonstrates that our method has a great potential in cardiac MR images segmentation, comprehensive clinical assessment, and diagnosis.
Collapse
|
37
|
Zotti C, Luo Z, Lalande A, Jodoin PM. Convolutional Neural Network With Shape Prior Applied to Cardiac MRI Segmentation. IEEE J Biomed Health Inform 2018; 23:1119-1128. [PMID: 30113903 DOI: 10.1109/jbhi.2018.2865450] [Citation(s) in RCA: 79] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In this paper, we present a novel convolutional neural network architecture to segment images from a series of short-axis cardiac magnetic resonance slices (CMRI). The proposed model is an extension of the U-net that embeds a cardiac shape prior and involves a loss function tailored to the cardiac anatomy. Since the shape prior is computed offline only once, the execution of our model is not limited by its calculation. Our system takes as input raw magnetic resonance images, requires no manual preprocessing or image cropping and is trained to segment the endocardium and epicardium of the left ventricle, the endocardium of the right ventricle, as well as the center of the left ventricle. With its multiresolution grid architecture, the network learns both high and low-level features useful to register the shape prior as well as accurately localize the borders of the cardiac regions. Experimental results obtained on the Automatic Cardiac Diagnostic Challenge - Medical Image Computing and Computer Assisted Intervention (ACDC-MICCAI) 2017 dataset show that our model segments multislices CMRI (left and right ventricle contours) in 0.18 s with an average Dice coefficient of [Formula: see text] and an average 3-D Hausdorff distance of [Formula: see text] mm.
Collapse
|
38
|
Automated comprehensive Adolescent Idiopathic Scoliosis assessment using MVC-Net. Med Image Anal 2018; 48:1-11. [PMID: 29803920 DOI: 10.1016/j.media.2018.05.005] [Citation(s) in RCA: 61] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2017] [Revised: 04/24/2018] [Accepted: 05/11/2018] [Indexed: 10/16/2022]
Abstract
Automated quantitative estimation of spinal curvature is an important task for the ongoing evaluation and treatment planning of Adolescent Idiopathic Scoliosis (AIS). It solves the widely accepted disadvantage of manual Cobb angle measurement (time-consuming and unreliable) which is currently the gold standard for AIS assessment. Attempts have been made to improve the reliability of automated Cobb angle estimation. However, it is very challenging to achieve accurate and robust estimation of Cobb angles due to the need for correctly identifying all the required vertebrae in both Anterior-posterior (AP) and Lateral (LAT) view x-rays. The challenge is especially evident in LAT x-ray where occlusion of vertebrae by the ribcage occurs. We therefore propose a novel Multi-View Correlation Network (MVC-Net) architecture that can provide a fully automated end-to-end framework for spinal curvature estimation in multi-view (both AP and LAT) x-rays. The proposed MVC-Net uses our newly designed multi-view convolution layers to incorporate joint features of multi-view x-rays, which allows the network to mitigate the occlusion problem by utilizing the structural dependencies of the two views. The MVC-Net consists of three closely-linked components: (1) a series of X-modules for joint representation of spinal structure (2) a Spinal Landmark Estimator network for robust spinal landmark estimation, and (3) a Cobb Angle Estimator network for accurate Cobb Angles estimation. By utilizing an iterative multi-task training algorithm to train the Spinal Landmark Estimator and Cobb Angle Estimator in tandem, the MVC-Net leverages the multi-task relationship between landmark and angle estimation to reliably detect all the required vertebrae for accurate Cobb angles estimation. Experimental results on 526 x-ray images from 154 patients show an impressive 4.04° Circular Mean Absolute Error (CMAE) in AP Cobb angle and 4.07° CMAE in LAT Cobb angle estimation, which demonstrates the MVC-Net's capability of robust and accurate estimation of Cobb angles in multi-view x-rays. Our method therefore provides clinicians with a framework for efficient, accurate, and reliable estimation of spinal curvature for comprehensive AIS assessment.
Collapse
|
39
|
Full left ventricle quantification via deep multitask relationships learning. Med Image Anal 2018; 43:54-65. [DOI: 10.1016/j.media.2017.09.005] [Citation(s) in RCA: 98] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2017] [Revised: 08/25/2017] [Accepted: 09/18/2017] [Indexed: 12/22/2022]
|