Learning-based ventricle detection from cardiac MR and CT images

Towards an ML-based semantic IoT for pandemic management: A survey of enabling technologies for COVID-19

The connection between humans and digital technologies has been documented extensively in the past decades but needs to be evaluated through the current global pandemic. Artificial Intelligence(AI), with its two strands, Machine Learning (ML) and Semantic Reasoning, has proven to be a great solution to provide efficient ways to prevent, diagnose and limit the spread of COVID-19. IoT solutions have been widely proposed for COVID-19 disease monitoring, infection geolocation, and social applications. In this paper, we investigate the usage of the three technologies for handling the COVID-19 pandemic. For this purpose, we surveyed the existing ML applications and algorithms proposed during the pandemic to detect COVID-19 disease using symptom factors and image processing. The survey includes existing approaches including semantic technologies and IoT systems for COVID-19. Based on the survey result, we classified the main challenges and the solutions that could solve them. The study proposes a conceptual framework for pandemic management and discusses challenges and trends for future research.

Deep Learning on Sparse Manifolds for Faster Object Segmentation

We propose a new combination of deep belief networks and sparse manifold learning strategies for the 2D segmentation of non-rigid visual objects. With this novel combination, we aim to reduce the training and inference complexities while maintaining the accuracy of machine learning-based non-rigid segmentation methodologies. Typical non-rigid object segmentation methodologies divide the problem into a rigid detection followed by a non-rigid segmentation, where the low dimensionality of the rigid detection allows for a robust training (i.e., a training that does not require a vast amount of annotated images to estimate robust appearance and shape models) and a fast search process during inference. Therefore, it is desirable that the dimensionality of this rigid transformation space is as small as possible in order to enhance the advantages brought by the aforementioned division of the problem. In this paper, we propose the use of sparse manifolds to reduce the dimensionality of the rigid detection space. Furthermore, we propose the use of deep belief networks to allow for a training process that can produce robust appearance models without the need of large annotated training sets. We test our approach in the segmentation of the left ventricle of the heart from ultrasound images and lips from frontal face images. Our experiments show that the use of sparse manifolds and deep belief networks for the rigid detection stage leads to segmentation results that are as accurate as the current state of the art, but with lower search complexity and training processes that require a small amount of annotated training data.

Automatic segmentation of left ventricle cavity from short-axis cardiac magnetic resonance images

In this paper, a computational framework is proposed to perform a fully automatic segmentation of the left ventricle (LV) cavity from short-axis cardiac magnetic resonance (CMR) images. In the initial phase, the region of interest (ROI) is automatically identified on the first image frame of the CMR slices. This is done by partitioning the image into different regions using a standard fuzzy c-means (FCM) clustering algorithm where the LV region is identified according to its intensity, size and circularity in the image. Next, LV segmentation is performed within the identified ROI by using a novel clustering method that utilizes an objective functional with a dissimilarity measure that incorporates a circular shape function. This circular shape-constrained FCM algorithm is able to differentiate pixels with similar intensity but are located in different regions (e.g. LV cavity and non-LV cavity), thus improving the accuracy of the segmentation even in the presence of papillary muscles. In the final step, the segmented LV cavity is propagated to the adjacent image frame to act as the ROI. The segmentation and ROI propagation are then iteratively executed until the segmentation has been performed for the whole cardiac sequence. Experiment results using the LV Segmentation Challenge validation datasets show that our proposed framework can achieve an average perpendicular distance (APD) shift of 2.23 ± 0.50 mm and the Dice metric (DM) index of 0.89 ± 0.03, which is comparable to the existing cutting edge methods. The added advantage over state of the art is that our approach is fully automatic, does not need manual initialization and does not require a prior trained model.

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.

Cardiac image segmentation from cine cardiac MRI using graph cuts and shape priors

In this paper, we propose a novel method for segmentation of the left ventricle, right ventricle, and myocardium from cine cardiac magnetic resonance images of the STACOM database. Our method incorporates prior shape information in a graph cut framework to achieve segmentation. Poor edge information and large within-patient shape variation of the different parts necessitates the inclusion of prior shape information. But large interpatient shape variability makes it difficult to have a generalized shape model. Therefore, for every dataset the shape prior is chosen as a single image clearly showing the different parts. Prior shape information is obtained from a combination of distance functions and orientation angle histograms of each pixel relative to the prior shape. To account for shape changes, pixels near the boundary are allowed to change their labels by appropriate formulation of the penalty and smoothness costs. Our method consists of two stages. In the first stage, segmentation is performed using only intensity information which is the starting point for the second stage combining intensity and shape information to get the final segmentation. Experimental results on different subsets of 30 real patient datasets show higher segmentation accuracy in using shape information and our method's superior performance over other competing methods.

Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data

We present a new statistical pattern recognition approach for the problem of left ventricle endocardium tracking in ultrasound data. The problem is formulated as a sequential importance resampling algorithm such that the expected segmentation of the current time step is estimated based on the appearance, shape, and motion models that take into account all previous and current images and previous segmentation contours produced by the method. The new appearance and shape models decouple the affine and nonrigid segmentations of the left ventricle to reduce the running time complexity. The proposed motion model combines the systole and diastole motion patterns and an observation distribution built by a deep neural network. The functionality of our approach is evaluated using a dataset of diseased cases containing 16 sequences and another dataset of normal cases comprised of four sequences, where both sets present long axis views of the left ventricle. Using a training set comprised of diseased and healthy cases, we show that our approach produces more accurate results than current state-of-the-art endocardium tracking methods in two test sequences from healthy subjects. Using three test sequences containing different types of cardiopathies, we show that our method correlates well with interuser statistics produced by four cardiologists.

Automatic segmentation of cardiac MRI cines validated for long axis views

Segmentation of cardiac magnetic resonance imaging is considered an important application in clinical practice. An automatic algorithm is proposed for segmentation of both endocardial and epicardial boundaries, in long-axis views. The data consisted of 126 patients, yielding 1008 traces. Estimated clinical parameters were highly correlated to gold standard measurements. The error between the automatic tracing and the gold standard was not significantly different than the error between two manual observers. In conclusion, a tool for segmenting the myocardial boundaries in the long-axis views is proposed, which works well, as demonstrated by the validation performed using a clinical dataset.

The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods

We present a new supervised learning model designed for the automatic segmentation of the left ventricle (LV) of the heart in ultrasound images. We address the following problems inherent to supervised learning models: 1) the need of a large set of training images; 2) robustness to imaging conditions not present in the training data; and 3) complex search process. The innovations of our approach reside in a formulation that decouples the rigid and nonrigid detections, deep learning methods that model the appearance of the LV, and efficient derivative-based search algorithms. The functionality of our approach is evaluated using a data set of diseased cases containing 400 annotated images (from 12 sequences) and another data set of normal cases comprising 80 annotated images (from two sequences), where both sets present long axis views of the LV. Using several error measures to compute the degree of similarity between the manual and automatic segmentations, we show that our method not only has high sensitivity and specificity but also presents variations with respect to a gold standard (computed from the manual annotations of two experts) within interuser variability on a subset of the diseased cases. We also compare the segmentations produced by our approach and by two state-of-the-art LV segmentation models on the data set of normal cases, and the results show that our approach produces segmentations that are comparable to these two approaches using only 20 training images and increasing the training set to 400 images causes our approach to be generally more accurate. Finally, we show that efficient search methods reduce up to tenfold the complexity of the method while still producing competitive segmentations. In the future, we plan to include a dynamical model to improve the performance of the algorithm, to use semisupervised learning methods to reduce even more the dependence on rich and large training sets, and to design a shape model less dependent on the training set.

Automated ventricular systems segmentation in brain CT images by combining low-level segmentation and high-level template matching

Background

Accurate analysis of CT brain scans is vital for diagnosis and treatment of Traumatic Brain Injuries (TBI). Automatic processing of these CT brain scans could speed up the decision making process, lower the cost of healthcare, and reduce the chance of human error. In this paper, we focus on automatic processing of CT brain images to segment and identify the ventricular systems. The segmentation of ventricles provides quantitative measures on the changes of ventricles in the brain that form vital diagnosis information.

Methods

First all CT slices are aligned by detecting the ideal midlines in all images. The initial estimation of the ideal midline of the brain is found based on skull symmetry and then the initial estimate is further refined using detected anatomical features. Then a two-step method is used for ventricle segmentation. First a low-level segmentation on each pixel is applied on the CT images. For this step, both Iterated Conditional Mode (ICM) and Maximum A Posteriori Spatial Probability (MASP) are evaluated and compared. The second step applies template matching algorithm to identify objects in the initial low-level segmentation as ventricles. Experiments for ventricle segmentation are conducted using a relatively large CT dataset containing mild and severe TBI cases.

Results

Experiments show that the acceptable rate of the ideal midline detection is over 95%. Two measurements are defined to evaluate ventricle recognition results. The first measure is a sensitivity-like measure and the second is a false positive-like measure. For the first measurement, the rate is 100% indicating that all ventricles are identified in all slices. The false positives-like measurement is 8.59%. We also point out the similarities and differences between ICM and MASP algorithms through both mathematically relationships and segmentation results on CT images.

Conclusion

The experiments show the reliability of the proposed algorithms. The novelty of the proposed method lies in its incorporation of anatomical features for ideal midline detection and the two-step ventricle segmentation method. Our method offers the following improvements over existing approaches: accurate detection of the ideal midline and accurate recognition of ventricles using both anatomical features and spatial templates derived from Magnetic Resonance Images.

Segmenting and tracking the left ventricle by learning the dynamics in cardiac images

Having accurate left ventricle (LV) segmentations across a cardiac cycle provides useful quantitative (e.g. ejection fraction) and qualitative information for diagnosis of certain heart conditions. Existing LV segmentation techniques are founded mostly upon algorithms for segmenting static images. In order to exploit the dynamic structure of the heart in a principled manner, we approach the problem of LV segmentation as a recursive estimation problem. In our framework, LV boundaries constitute the dynamic system state to be estimated, and a sequence of observed cardiac images constitute the data. By formulating the problem as one of state estimation, the segmentation at each particular time is based not only on the data observed at that instant, but also on predictions based on past segmentations. This requires a dynamical system model of the LV, which we propose to learn from training data through an information-theoretic approach. To incorporate the learned dynamic model into our segmentation framework and obtain predictions, we use ideas from particle filtering. Our framework uses a curve evolution method to combine such predictions with the observed images to estimate the LV boundaries at each time. We demonstrate the effectiveness of the proposed approach on a large set of cardiac images. We observe that our approach provides more accurate segmentations than those from static image segmentation techniques, especially when the observed data are of limited quality.

Model-based processing scheme for quantitative 4-D cardiac MRI analysis

In this paper, we present an integrated model-based processing scheme for cardiac magnetic resonance imaging (MRI), embedded in an interactive computing environment suitable for quantitative cardiac analysis, which provides a set of functions for the extraction, modeling, and visualization of cardiac shape and deformation. The methods apply four-dimensional (4-D) processing (three spatial and one temporal) to multiphase multislice MRI acquisitions and produce a continuous 4-D model of the myocardial surface deformation. The model is used to measure diagnostically useful parameters, such as wall motion, myocardial thicking, and myocardial mass measurements. The proposed model-based shape extraction method has the advantage of integrating local information into an overall representation and produces a robust description of cardiac cavities. A learning segmentation process that incorporates a generating-shrinking neural network is combined with a spatiotemporal parametric modeling method through functional basis decomposition. A multiscale approach is adopted, which uses at each step a coarse-scale model defined at the previous step in order to constrain the boundary detection. The representation accuracy starts from a coarse but smooth estimation of the approximate cardiac shape and is gradually increased to the desired detail. The main advantages of the proposed methods are efficiency, lack of uncertainty about convergence, and robustness to image artifacts. Experimental results obtained from application to clinical multislice multiphase MRI examinations of normal volunteers and patients with medical record of myocardial infarction were satisfactory in terms of accuracy and robustness.

Multistage hybrid active appearance model matching: segmentation of left and right ventricles in cardiac MR images

A fully automated approach to segmentation of the left and right cardiac ventricles from magnetic resonance (MR) images is reported. A novel multistage hybrid appearance model methodology is presented in which a hybrid active shape model/active appearance model (AAM) stage helps avoid local minima of the matching function. This yields an overall more favorable matching result. An automated initialization method is introduced making the approach fully automated. Our method was trained in a set of 102 MR images and tested in a separate set of 60 images. In all testing cases, the matching resulted in a visually plausible and accurate mapping of the model to the image data. Average signed border positioning errors did not exceed 0.3 mm in any of the three determined contours-left-ventricular (LV) epicardium, LV and right-ventricular (RV) endocardium. The area measurements derived from the three contours correlated well with the independent standard (r = 0.96, 0.96, 0.90), with slopes and intercepts of the regression lines close to one and zero, respectively. Testing the reproducibility of the method demonstrated an unbiased performance with small range of error as assessed via Bland-Altman statistic. In direct border positioning error comparison, the multistage method significantly outperformed the conventional AAM (p < 0.001). The developed method promises to facilitate fully automated quantitative analysis of LV and RV morphology and function in clinical setting.

A multiresolution image segmentation technique based on pyramidal segmentation and fuzzy clustering

In this paper, an unsupervised image segmentation technique is presented, which combines pyramidal image segmentation with the fuzzy c-means clustering algorithm. Each layer of the pyramid is split into a number of regions by a root labeling technique, and then fuzzy c-means is used to merge the regions of the layer with the highest image resolution. A cluster validity functional is used to find the optimal number of objects automatically. Segmentation of a number of synthetic as well as clinical images is illustrated and two fully automatic segmentation approaches are evaluated, which determine the left ventricular volume (LV) in 140 cardiovascular magnetic resonance (MR) images. First fuzzy c-means is applied without pyramids. In the second approach the regions generated by pyramidal segmentation are merged by fuzzy c-means. The correlation coefficients of manually and automatically defined LV lumen of all 140 and 20 end-diastolic images were equal to 0.86 and 0.79, respectively, when images were segmented with fuzzy c-means alone. These coefficients increased to 0.90 and 0.93 when the pyramidal segmentation was combined with fuzzy c-means. This method can be applied to any dimensional representation and at any resolution level of an image series. The evaluation study shows good performance in detecting LV lumen in MR images.

Motion and deformation tracking for short-axis echo-planar myocardial perfusion imaging

The assessment of regional myocardial perfusion during the first-pass of a contrast agent bolus requires tracking of the signal time course for each myocardial segment so that a detailed perfusion map can be derived. To obtain such a map in practice, however, is not trivial because deformation of the shape of the myocardium and respiratory-induced motion render a major difficulty in this process. This study describes an automated approach for motion and deformation tracking of functional myocardial perfusion images. The effectiveness of the described method has been evaluated using a numerical phantom and results are compared with those from existing techniques which use deformable models. Preliminary results from applying our approach to 20 patients are discussed and compared with those from SPECT studies.