Segmentation and tracking in echocardiographic sequences: active contours guided by optical flow estimates

Coded Excitation for Ultrasonic Testing: A Review

Originating in the early 20th century, ultrasonic testing has found increasingly extensive applications in medicine, industry, and materials science. Achieving both a high signal-to-noise ratio and high efficiency is crucial in ultrasonic testing. The former means an increase in imaging clarity as well as the detection depth, while the latter facilitates a faster refresh of the image. It is difficult to balance these two indicators with a conventional short pulse to excite the probe, so in general handling methods, these two factors have a trade-off. To solve the above problems, coded excitation (CE) can increase the pulse duration and offers great potential to improve the signal-to-noise ratio with equivalent or even higher efficiency. In this paper, we first review the fundamentals of CE, including signal modulation, signal transmission, signal reception, pulse compression, and optimization methods. Then, we introduce the application of CE in different areas of ultrasonic testing, with a focus on industrial bulk wave single-probe detection, industrial guided wave detection, industrial bulk wave phased array detection, and medical phased array imaging. Finally, we point out the advantages as well as a few future directions of CE.

A Computer-Aided Markov Random Field Segmentation Algorithm for Assessing Fetal Ventricular Chambers

Congenital heart disease (CHD) is the most widely occurring congenital defect and accounts to about 28% of the overall congenital defects. Analysis of the development of the fetal heart thus plays an important role for detection of abnormality in early stages and to take corrective measures. Cardiac chamber analysis is one of the important diagnosing methods. Segmentation of the cardiac chambers must be done appropriately to avoid false interpretations. Effective segmentation of fetal ventricular chambers is a challenging task as the speckle noise inherent in ultrasound images cause blurring of the boundaries of anatomical structures. Several segmentation techniques have been proposed for extracting the fetal cardiac chambers. This article discusses the performance evaluation of automated, probability based segmentation approach, and Markov random field (MRF) for segmenting the fetal ventricular chambers of ultrasonic cineloop sequences. 837 ultrasonic biometery sequences of various gestations were collected from local diagnostic center after due ethical clearance and used for the study. In order to assess the efficiency of the segmentation technique, four metrics such as dice coefficient, true positive ratio (TPR), false positive ratio (FPR), similarity ratio (SIR), and precision (PR) were used. In order to perform ground truth validation, 56% of the data used in this study were annotated by clinical experts. The automated segmentation yielded comparable results with manual annotation. The technique results in average value of 0.68 for Dice coefficient, 0.723 for TPR, 0.604 for SIR, and 0.632 for PR.

Application of cascaded GAN based on CT scan in the diagnosis of aortic dissection

PURPOSE

Currently, Computed Tomography Angiography (CTA) is the most commonly used clinical method for the diagnosis of aortic dissection, which is much better than plain CT. However, CTA examination has some disadvantages such as time-consuming image processing, complicated procedure and injection of developer. CT plain scanning is widely used in the early diagnosis of arterial dissection because of its convenience, speed and popularity. In order not to delay the optimal diagnosis and treatment time of patients, we use deep learning technology and network model to synthesize plain CT images into CTA images. Patients can be timely professional related departments of clinical diagnosis and treatment, and reduce the rate of missed diagnosis. In this paper, we propose a CTA image synthesis technique for cardiac aortic dissection based on the cascaded generative adjunctive network model.

METHOD

Firstly, we registered CT images, and then used nnU-Net segmentation network model to obtain CT and CTA paired images containing only the aorta. Then we proposed a CTA image synthesis method for aortic dissection based on cascaded generative adversarial. The core idea is to build a cascade generator and double discriminator network based on DCT channel attention mechanism to further enhance the synthesis effect of CTA.

RESULTS

The model is trained and tested on CT plain scan and CTA image data set of aortic dissection. The results show that the proposed model achieves good results in CTA image synthesis. In the CT data set, the nnU-Net model improves 8.63% and reduces 10.87mm errors in the key index DSC and HD, respectively, compared with the benchmark model U-Net. In CTA data set, nnU-Net model improves 10.27% and reduces 6.56mm error in key index DSC and HD, respectively, compared with benchmark model U-Net. In the synthesis task, the cascaded generative adm network is superior to Pix2pix and Pix2pixHD network models in both PSNR and SSIM, which proves that our proposed model has significant advantages.

CONCLUSION

This study provides new possibilities for CTA image synthesis of aortic dissection, and improves the accuracy and efficiency of diagnosis, and hopes to provide substantial help for the diagnosis of aortic dissection.

Deep learning-based medical image segmentation of the aorta using XR-MSF-U-Net

PURPOSE

This paper proposes a CT images and MRI segmentation technology of cardiac aorta based on XR-MSF-U-Net model. The purpose of this method is to better analyze the patient's condition, reduce the misdiagnosis and mortality rate of cardiovascular disease in inhabitants, and effectively avoid the subjectivity and unrepeatability of manual segmentation of heart aorta, and reduce the workload of doctors.

METHOD

We implement the X ResNet (XR) convolution module to replace the different convolution kernels of each branch of two-layer convolution XR of common model U-Net, which can make the model extract more useful features more efficiently. Meanwhile, a plug and play attention module integrating multi-scale features Multi-scale features fusion module (MSF) is proposed, which integrates global local and spatial features of different receptive fields to enhance network details to achieve the goal of efficient segmentation of cardiac aorta through CT images and MRI.

RESULTS

The model is trained on common cardiac CT images and MRI data sets and tested on our collected data sets to verify the generalization ability of the model. The results show that the proposed XR-MSF-U-Net model achieves a good segmentation effect on CT images and MRI. In the CT data set, the XR-MSF-U-Net model improves 7.99% in key index DSC and reduces 11.01 mm in HD compared with the benchmark model U-Net, respectively. In the MRI data set, XR-MSF-U-Net model improves 10.19% and reduces 6.86 mm error in key index DSC and HD compared with benchmark model U-Net, respectively. And it is superior to similar models in segmentation effect, proving that this model has significant advantages.

CONCLUSION

This study provides new possibilities for the segmentation of aortic CT images and MRI, improves the accuracy and efficiency of diagnosis, and hopes to provide substantial help for the segmentation of aortic CT images and MRI.

Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization

Analyzing and understanding the movement of the mitral valve is of vital importance in cardiology, as the treatment and prevention of several serious heart diseases depend on it. Unfortunately, large amounts of noise as well as a highly varying image quality make the automatic tracking and segmentation of the mitral valve in two-dimensional echocardiographic videos challenging. In this paper, we present a fully automatic and unsupervised method for segmentation of the mitral valve in two-dimensional echocardiographic videos, independently of the echocardiographic view. We propose a bias-free variant of the robust non-negative matrix factorization (RNMF) along with a window-based localization approach, that is able to identify the mitral valve in several challenging situations. We improve the average f1-score on our dataset of 10 echocardiographic videos by 0.18 to a f1-score of 0.56.

Shape constraint function for artery tracking in ultrasound images

Ultrasound guided regional anesthesia (UGRA) has emerged as a powerful technique for pain management in the operating theatre. It uses ultrasound imaging to visualize anatomical structures, the needle insertion and the delivery of the anesthetic around the targeted nerve block. Detection of the nerves is a difficult task, however, due to the poor quality of the ultrasound images. Recent developments in pattern recognition and machine learning have heightened the need for computer aided systems in many applications. This type of system can improve UGRA practice. In many imaging situations nerves are not salient in images. Generally, practitioners rely on the arteries as key anatomical structures to confirm the positions of the nerves, making artery tracking an important aspect for UGRA procedure. However, artery tracking in a noisy environment is a challenging problem, due to the instability of the features. This paper proposes a new method for real-time artery tracking in ultrasound images. It is based on shape information to correct tracker location errors. A new objective function is proposed, which defines an artery as an elliptical shape, enabling its robust fitting in a noisy environment. This approach is incorporated in two well-known tracking algorithms, and shows a systematic improvement over the original trackers. Evaluations were performed on 71 videos of different axillary nerve blocks. The results obtained demonstrated the validity of the proposed method.

A Novel 2-D Speckle Tracking Method for High-Frame-Rate Echocardiography

Speckle tracking echocardiography (STE) is a clinical tool to noninvasively assess regional myocardial function through the quantification of regional motion and deformation. Even if the time resolution of STE can be improved by high-frame-rate (HFR) imaging, dedicated HFR STE algorithms have to be developed to detect very small interframe motions. Therefore, in this article, we propose a novel 2-D STE method, purposely developed for HFR echocardiography. The 2-D motion estimator consists of a two-step algorithm based on the 1-D cross correlations to separately estimate the axial and lateral displacements. The method was first optimized and validated on simulated data giving an accuracy of ~3.3% and ~10.5% for the axial and lateral estimates, respectively. Then, it was preliminarily tested in vivo on ten healthy volunteers showing its clinical applicability and feasibility. Moreover, the extracted clinical markers were in the same range as those reported in the literature. Also, the estimated peak global longitudinal strain was compared with that measured with a clinical scanner showing good correlation and negligible differences (-20.94% versus -20.31%, p -value = 0.44). In conclusion, a novel algorithm for STE was developed: the radio frequency (RF) signals were preferred for the axial motion estimation, while envelope data were preferred for the lateral motion. Furthermore, using 2-D kernels, even for 1-D cross correlation, makes the method less sensitive to noise.

Generalizable fully automated multi-label segmentation of four-chamber view echocardiograms based on deep convolutional adversarial networks

A major issue in translation of the artificial intelligence platforms for automatic segmentation of echocardiograms to clinics is their generalizability. The present study introduces and verifies a novel generalizable and efficient fully automatic multi-label segmentation method for four-chamber view echocardiograms based on deep fully convolutional networks (FCNs) and adversarial training. For the first time, we used generative adversarial networks for pixel classification training, a novel method in machine learning not currently used for cardiac imaging, to overcome the generalization problem. The method's performance was validated against manual segmentations as the ground-truth. Furthermore, to verify our method's generalizability in comparison with other existing techniques, we compared our method's performance with a state-of-the-art method on our dataset in addition to an independent dataset of 450 patients from the CAMUS (cardiac acquisitions for multi-structure ultrasound segmentation) challenge. On our test dataset, automatic segmentation of all four chambers achieved a dice metric of 92.1%, 86.3%, 89.6% and 91.4% for LV, RV, LA and RA, respectively. LV volumes' correlation between automatic and manual segmentation were 0.94 and 0.93 for end-diastolic volume and end-systolic volume, respectively. Excellent agreement with chambers' reference contours and significant improvement over previous FCN-based methods suggest that generative adversarial networks for pixel classification training can effectively design generalizable fully automatic FCN-based networks for four-chamber segmentation of echocardiograms even with limited number of training data.

Artificial intelligence and automation in valvular heart diseases

Artificial intelligence (AI) is gradually changing every aspect of social life, and healthcare is no exception. The clinical procedures that were supposed to, and could previously only be handled by human experts can now be carried out by machines in a more accurate and efficient way. The coming era of big data and the advent of supercomputers provides great opportunities to the development of AI technology for the enhancement of diagnosis and clinical decision-making. This review provides an introduction to AI and highlights its applications in the clinical flow of diagnosing and treating valvular heart diseases (VHDs). More specifically, this review first introduces some key concepts and subareas in AI. Secondly, it discusses the application of AI in heart sound auscultation and medical image analysis for assistance in diagnosing VHDs. Thirdly, it introduces using AI algorithms to identify risk factors and predict mortality of cardiac surgery. This review also describes the state-of-the-art autonomous surgical robots and their roles in cardiac surgery and intervention.

Ultrasound Imaging Characterization of Soft Tissue Dynamics of the Seated Human Body

To characterize the dynamics of internal soft organs and external anatomical structures, this paper presents a system that combines medical ultrasound imaging with an optical tracker and a vertical exciter that imparts whole-body vibrations on seated subjects. The spatial and temporal accuracy of the system was validated using a phantom with calibrated internal structures, resulting in 0.224 mm maximum root-mean-square (r.m.s.) position error and 13 ms maximum synchronization error between sensors. In addition to the dynamics of the head and sternum, stomach dynamics were characterized by extracting the centroid of the stomach from the ultrasound images. The system was used to characterize the subject-specific body dynamics as well as the intrasubject variabilities caused by excitation pattern (frequency up-sweep, down-sweep, and white noise, 1-10 Hz), excitation amplitude (1 and 2 m/s2 r.m.s.), seat compliance (rigid and soft), and stomach filling (empty and 500 mL water). Human subjects experiments (n = 3) yielded preliminary results for the frequency response of the head, sternum, and stomach. The method presented here provides the first detailed in vivo characterization of internal and external human body dynamics. Tissue dynamics characterized by the system can inform design of vehicle structures and adaptive control of seat and suspension systems, as well as validate finite element models for predicting passenger comfort in the early stages of vehicle design.

Carotid artery ultrasound image analysis: A review of the literature

Stroke is one of the prominent causes of death in the recent days. The existence of susceptible plaque in the carotid artery can be used in ascertaining the possibilities of cardiovascular diseases and long-term disabilities. The imaging modality used for early screening of the disease is B-mode ultrasound image of the person in the artery area. The objective of this article is to give a widespread review of the imaging modes and methods used for studying the carotid artery for identifying stroke, atherosclerosis and related cardiovascular diseases. We encompass the review in methods used for artery wall tracking, intima-media, and lumen segmentation which will help in finding the extent of the disease. Due to the characteristics of the imaging modality used, the images have speckle noise which worsens the image quality. Adaptive homomorphic filtering with wavelet and contourlet transforms, Levy Shrink, gamma distribution were used for image denoising. Learning-based neural network approaches for denoising give better edge preservation. Domain knowledge-based segmentation approaches have proved to provide more accurate intima-media thickness measurements. There is a requirement of useful fully automatic segmentation approaches, 3D, 4D systems, and plaque motion analysis. Taking into consideration the image priors like geometry, imaging physics, intensity and temporal data, image analysis has to be performed. Encouragingly more research has focused on content-specific segmentation and classification techniques. With the evaluation of machine learning algorithms, classifying the image as with or without a fat deposit has gained better accuracy and sensitivity. Machine learning-based approaches like self-organizing map, k-nearest neighborhood and support vector machine achieve promising accuracy and sensitivity in classification. The literature reveals that there is more scope in identifying a patient-specific model in a fully automatic manner.

Ship Segmentation in SAR Images by Improved Nonlocal Active Contour Model

Synthetic aperture radar (SAR) has been widely used in ocean surveillance. As an important part of shipping management and military applications, ship monitoring is a study hotspot in SAR image interpretation; hence, many researches focus on ship targets. Among these studies, ship segmentation is a basic work, but still remains challenging due to the speckle noise and the complicated backscattering phenomenology in SAR images. To solve the problems, this paper proposes a new method for ship segmentation by nonlocal processing. Firstly, the proposed nonlocal energy describes the nonlocal comparison of patches and optimizes regions with spatially-varying features. Secondly, we rewrite the energy functional by introducing a ratio distance defined with respect to the probability density functions of regions to overcome the influence of the multiplicative noise. Finally, the integral histogram is introduced into the pairwise interactions to fasten the speed of convergence. Several rounds of comparative experiments are implemented on real SAR data with different resolutions and bands. The results demonstrate that the proposed method is robust to the speckle noise and intensity variations and could achieve refined segmentation for ship targets.

A New Method for the Anterior Mitral Leaflet Segmentation in Echocardiography Videos using the Virtual M-mode Space

Rheumatic heart disease is responsible for the heart valve damage, caused by repeated episodes of rheumatic fever. The disease commonly inflames and scars the mitral valve of the heart, resulting in thicker, less mobile leaflets, with associated decrease in cardiac efficiency. It is important to measure and quantity the early manifestations of this disease, including variations of the thickness, shape and mobility of the leaflets. These manifestations are visible in an echocardiographic screening process. The first step towards the defined objective is to segment the anterior mitral leaflet throughout the cardiac cycle, enabling the future automatic quantification of mentioned clinical parameters. In this work, a new algorithm for the segmentation of the whole region of the anterior mitral leaflet in the virtual M-mode space is proposed. The algorithm requires a single initialization point on the posterior wall of the aorta, in the first frame of the video. A junction point is then computed, showing the location where the two leaflets connect. This junction point helps to automatically redefine the range of virtual M-mode images required to completely segment the region of the anterior mitral leaflet. The segmented anterior mitral leaflet region in the virtual M-mode space is transferred back to regular image space and its shape refined using localized active contours. Results suggest the suitability of the proposed algorithm for the segmentation of anterior mitral leaflet with a median Dice Similarity Coefficient of 0.63, and with median precision and recall of 58% and 73% respectively.

Non-invasive spinal vibration testing using ultrafast ultrasound imaging: A new way to measure spine function

Ultrafast ultrasound imaging is used to capture driven spinal vibrations as a new method for non-invasive spinal testing in living subjects. Previously, it has been shown that accelerometer-based vibration testing in cadaveric models can reveal the presence, location and magnitude of spinal pathology. However, this process remains an invasive procedure as current non-invasive sensors are inadequate. In this paper, the ability of non-invasive ultrafast ultrasound to quantify in vivo vertebral vibration response across a broad range of frequencies (10–100Hz) in anesthetized pig models is investigated. Close agreement with invasive accelerometer measurements is achieved using the non-invasive ultrasound method, opening up unique opportunities to investigate spinal pathologies.

Tracking large Anterior Mitral Leaflet displacements by incorporating optical flow in an active contours framework

Echocardiography is an important tool to detect early evidence of mitral valve degradation associated with rheumatic heart disease. The segmentation and tracking of the Anterior Mitral Leaflet helps to quantify the morphologic valve anomalies, such as the leaflet thickening, shape and the mobility changes. The tracking of this leaflet throughout the cardiac cycle is still an open challenge in the research community. The widely used active contours segmentation framework fails when faced with large leaflet displacement. In this work, we propose the integration of optical flow in an open-ended active contour framework to address this difficulty. This additional information promotes solutions with contours next to high leaflet displacements, resulting in superior performance. The algorithm was tested on 9 fully annotated real clinical videos, acquired from the parasternal long axis view. The algorithm is compared with our previous work. Results show a clear improvement in situations where the leaflet exhibits large displacement or irregular shapes, with an average error of 4.5 pixels and a standard deviation of 2 pixels.

Vessel tree tracking in angiographic sequences

We present a method to track vessels in angiography [contrast filled vessels in two-dimensional (2-D) x-ray fluoroscopy]. Finding correspondence of a vessel tree from consecutive angiogram frames provides significant value in computer-aided clinical applications such as fast vessel tree segmentation, three-dimensional (3-D) vessel topology reconstruction from corresponding centerlines, cardiac motion understanding, etc. However, establishing an accurate vessel tree correspondence (vessel tree tracking) is a nontrivial problem due to nonlinear periodic cardiac and breathing motion in 2-D views, foreshortening, false bifurcations due to 3-D to 2-D projection, occlusion from other anatomies, etc. The vessel tree is represented by BSpline curves. The control points of the BSpline curves are landmarks that are the tracking targets. Our method maximizes the appearance similarity while preserving the vessel structure. A directed acyclic graph (DAG) is employed to represent the appearance and shape structure of the vessel tree: nodes from the DAG encode the appearance of the vessel tree landmarks, and the edges encode the relative locations between landmarks. The vessel tree tracking problem turns into finding the most similar tree from the DAG in the next frame, and it is solved using an efficient dynamic programming algorithm. We performed evaluations on 62 x-ray angiography sequences (above 1000 frames). Experiment results show our algorithm is robust to these challenges and delivers better performance, compared to four existing methods.

Connecting Markov random fields and active contour models: application to gland segmentation and classification

We introduce a Markov random field (MRF)-driven region-based active contour model (MaRACel) for histological image segmentation. This Bayesian segmentation method combines a region-based active contour (RAC) with an MRF. State-of-the-art RAC models assume that every spatial location in the image is statistically independent, thereby ignoring valuable contextual information among spatial locations. To address this shortcoming, we incorporate an MRF prior into energy term of the RAC. This requires a formulation of the Markov prior consistent with the continuous variational framework characteristic of active contours; consequently, we introduce a continuous analog to the discrete Potts model. Based on the automated segmentation boundary of glands by MaRACel model, explicit shape descriptors are then employed to distinguish prostate glands belonging to Gleason patterns 3 (G3) and 4 (G4). To demonstrate the effectiveness of MaRACel, we compare its performance to the popular models proposed by Chan and Vese (CV) and Rousson and Deriche (RD) with respect to the following tasks: (1) the segmentation of prostatic acini (glands) and (2) the differentiation of G3 and G4 glands. On almost 600 prostate biopsy needle images, MaRACel was shown to have higher average dice coefficients, overlap ratios, sensitivities, specificities, and positive predictive values both in terms of segmentation accuracy and ability to discriminate between G3 and G4 glands compared to the CV and RD models.

Segmentation of Fetal Left Ventricle in Echocardiographic Sequences Based on Dynamic Convolutional Neural Networks

Segmentation of fetal left ventricle (LV) in echocardiographic sequences is important for further quantitative analysis of fetal cardiac function. However, image gross inhomogeneities and fetal random movements make the segmentation a challenging problem. In this paper, a dynamic convolutional neural networks (CNN) based on multiscale information and fine-tuning is proposed for fetal LV segmentation. The CNN is pretrained by amount of labeled training data. In the segmentation, the first frame of each echocardiographic sequence is delineated manually. The dynamic CNN is fine-tuned by deep tuning with the first frame and shallow tuning with the rest of frames, respectively, to adapt to the individual fetus. Additionally, to separate the connection region between LV and left atrium (LA), a matching approach, which consists of block matching and line matching, is used for mitral valve (MV) base points tracking. Advantages of our proposed method are compared with an active contour model (ACM), a dynamical appearance model (DAM), and a fixed multiscale CNN method. Experimental results in 51 echocardiographic sequences show that the segmentation results agree well with the ground truth, especially in the cases with leakage, blurry boundaries, and subject-to-subject variations. The CNN architecture can be simple, and the dynamic fine-tuning is efficient.

2D to 3D fusion of echocardiography and cardiac CT for TAVR and TAVI image guidance

This study proposed a registration framework to fuse 2D echocardiography images of the aortic valve with preoperative cardiac CT volume. The registration facilitates the fusion of CT and echocardiography to aid the diagnosis of aortic valve diseases and provide surgical guidance during transcatheter aortic valve replacement and implantation. The image registration framework consists of two major steps: temporal synchronization and spatial registration. Temporal synchronization allows time stamping of echocardiography time series data to identify frames that are at similar cardiac phase as the CT volume. Spatial registration is an intensity-based normalized mutual information method applied with pattern search optimization algorithm to produce an interpolated cardiac CT image that matches the echocardiography image. Our proposed registration method has been applied on the short-axis "Mercedes Benz" sign view of the aortic valve and long-axis parasternal view of echocardiography images from ten patients. The accuracy of our fully automated registration method was 0.81 ± 0.08 and 1.30 ± 0.13 mm in terms of Dice coefficient and Hausdorff distance for short-axis aortic valve view registration, whereas for long-axis parasternal view registration it was 0.79 ± 0.02 and 1.19 ± 0.11 mm, respectively. This accuracy is comparable to gold standard manual registration by expert. There was no significant difference in aortic annulus diameter measurement between the automatically and manually registered CT images. Without the use of optical tracking, we have shown the applicability of this technique for effective fusion of echocardiography with preoperative CT volume to potentially facilitate catheter-based surgery.

Estimating blood vessel areas in ultrasound images using a deformable template model

We consider the problem of obtaining interval estimates of vessel areas from ultrasound images of cross sections through the carotid artery. Robust and automatic estimates of the cross sectional area is of medical interest and of help in diagnosing atherosclerosis, which is caused by plaque deposits in the carotid artery. We approach this problem by using a deformable template to model the blood vessel outline, and use recent developments in ultrasound science to model the likelihood. We demonstrate that by using an explicit model for the outline, we can easily adjust for an important feature in the data: strong edge reflections called specular reflection. The posterior is challenging to explore, and naive standard MCMC algorithms simply converge too slowly. To obtain an efficient MCMC algorithm we make extensive use of computational efficient Gaussian Markov random fields, and use various block sampling constructions that jointly update large parts of the model.

Segmentation of the metacarpus and phalange in musculoskeletal ultrasound images using local active contours

This work presents a method for the automatic segmentation of metacarpus and phalange bones in ultrasound images of the second metacarpophalangeal joint (MCPJ) using Active Contours. The MCPJ is known to be the one of the first structures to be affected by rheumatic diseases like rheumatoid arthritis. The early detection and follow-up of this disease is important to prevent irreversible damage of the joints, which occurs continuously and faster if no treatment is used. To our knowledge, there is no automatic system to quantify the extension of the lesions resulting from rheumatic activity. The objective of this work is to identify the metacarpus and the phalange bones using local active contours. To our knowledge, there is no well established method for this problem and this technique has not been used yet in these structures. Results proved that the automatic segmentation is possible with an error of 3 pixels for a confidence of 80%.

Automatic Extraction of Central Tendon of Rectus Femoris (CT-RF) in Ultrasound Images Using a New Intensity-Compensated Free-Form Deformation-Based Tracking Algorithm With Local Shape Refinement

Ultrasonography is an important diagnostic imaging technique for visualization of tendons, which provides useful health diagnostic and fundamental information in neuromuscular studies of human motion systems. Conventional ultrasonic-based tendon studies, however, are highly dependent on subjective experience of operators due to various impairments of ultrasound images. Dynamic changes of muscle and tendon deformation in a sequence can hardly be manually processed. Consequently, there is an urgent need for automatic analysis of tendon behavior. This paper proposes an automatic ultrasonic tendon tracking algorithm to extract the shape deformation of central tendon of rectus femoris (CT-RF) from ultrasonic image sequences. The tracking problem is complicated by the highly deformable tendon, time-varying brightness, and the inconspicuousness of the target. To address this difficult tracking problem, we proposed a new intensity-compensated free-form deformation (IC-FFD)-based tracking algorithm with local shape refinement (LSR). Experimental results and comparison show that the proposed IC-FFD-LSR algorithm outperforms IC-FFD and conventional methods such as MI-FFD in CT-RF tracking.

Contour Tracking with a Spatio-Temporal Intensity Moment

Standard edge detection operators such as the Laplacian of Gaussian and the gradient of Gaussian can be used to track contours in image sequences. When using edge operators, a contour, which is determined on a frame of the sequence, is simply used as a starting contour to locate the nearest contour on the subsequent frame. However, the strategy used to look for the nearest edge points may not work when tracking contours of non isolated gray level discontinuities. In these cases, strategies derived from the optical flow equation, which look for similar gray level distributions, appear to be more appropriate since these can work with a lower frame rate than that needed for strategies based on pure edge detection operators. However, an optical flow strategy tends to propagate the localization errors through the sequence and an additional edge detection procedure is essential to compensate for such a drawback. In this paper a spatio-temporal intensity moment is proposed which integrates the two basic functions of edge detection and tracking.

Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey

Reviewed here are methods developed for following (i.e., tracking) structures in medical B-mode ultrasound time sequences during large-scale motion. The resulting motion estimation problem and its key components are defined. The main tracking approaches are described, and their strengths and weaknesses are discussed. Existing motion estimation methods, tested on multiple in vivo sequences, are categorized with respect to their clinical applications, namely, cardiac, respiratory and muscular motion. A large number of works in this field had to be discarded as thorough validation of the results was missing. The remaining relevant works identified indicate the possibility of reaching an average tracking accuracy up to 1-2 mm. Real-time performance can be achieved using several methods. Yet only very few of these have progressed to clinical practice. The latest trends include incorporation of complementary and prior information. Advances are expected from common evaluation databases and 4-D ultrasound scanning technologies.

Active Contours Driven by Multi-Feature Gaussian Distribution Fitting Energy with Application to Vessel Segmentation

Active contour models are of great importance for image segmentation and can extract smooth and closed boundary contours of the desired objects with promising results. However, they cannot work well in the presence of intensity inhomogeneity. Hence, a novel region-based active contour model is proposed by taking image intensities and ‘vesselness values’ from local phase-based vesselness enhancement into account simultaneously to define a novel multi-feature Gaussian distribution fitting energy in this paper. This energy is then incorporated into a level set formulation with a regularization term for accurate segmentations. Experimental results based on publicly available STructured Analysis of the Retina (STARE) demonstrate our model is more accurate than some existing typical methods and can successfully segment most small vessels with varying width.

Phase-based probabilistic active contour for nerve detection in ultrasound images for regional anesthesia

Ultrasound guided regional anesthesia (UGRA) is steadily growing in popularity, owing to advances in ultrasound imaging technology and the advantages that this technique presents for safety and efficiency. The aim of this work is to assist anaesthetists during the UGRA procedure by automatically detecting the nerve blocks in the ultrasound images. The main disadvantage of ultrasound images is the poor quality of the images, which are also affected by the speckle noise. Moreover, the nerve structure is not salient amid the other tissues, which makes its detection a challenging problem. In this paper we propose a new method to tackle the problem of nerve zone detection in ultrasound images. The method consists in a combination of three approaches: probabilistic, edge phase information and active contours. The gradient vector flow (GVF) is adopted as an edge-based active contour. The phase analysis of the monogenic signal is used to provide reliable edges for the GVF. Then, a learned probabilistic model reduces the false positives and increases the likelihood energy term of the target region. It yields a new external force field that attracts the active contour toward the desired region of interest. The proposed scheme has been applied to sciatic nerve regions. The qualitative and quantitative evaluations show a high accuracy and a significant improvement in performance.

Manifold learning for object tracking with multiple nonlinear models

This paper presents a novel manifold learning algorithm for high-dimensional data sets. The scope of the application focuses on the problem of motion tracking in video sequences. The framework presented is twofold. First, it is assumed that the samples are time ordered, providing valuable information that is not presented in the current methodologies. Second, the manifold topology comprises multiple charts, which contrasts to the most current methods that assume one single chart, being overly restrictive. The proposed algorithm, Gaussian process multiple local models (GP-MLM), can deal with arbitrary manifold topology by decomposing the manifold into multiple local models that are probabilistic combined using Gaussian process regression. In addition, the paper presents a multiple filter architecture where standard filtering techniques are integrated within the GP-MLM. The proposed approach exhibits comparable performance of state-of-the-art trackers, namely multiple model data association and deep belief networks, and compares favorably with Gaussian process latent variable models. Extensive experiments are presented using real video data, including a publicly available database of lip sequences and left ventricle ultrasound images, in which the GP-MLM achieves state of the art results.

Contour tracking in echocardiographic sequences via sparse representation and dictionary learning

This paper presents a dynamical appearance model based on sparse representation and dictionary learning for tracking both endocardial and epicardial contours of the left ventricle in echocardiographic sequences. Instead of learning offline spatiotemporal priors from databases, we exploit the inherent spatiotemporal coherence of individual data to constraint cardiac contour estimation. The contour tracker is initialized with a manual tracing of the first frame. It employs multiscale sparse representation of local image appearance and learns online multiscale appearance dictionaries in a boosting framework as the image sequence is segmented frame-by-frame sequentially. The weights of multiscale appearance dictionaries are optimized automatically. Our region-based level set segmentation integrates a spectrum of complementary multilevel information including intensity, multiscale local appearance, and dynamical shape prediction. The approach is validated on twenty-six 4D canine echocardiographic images acquired from both healthy and post-infarct canines. The segmentation results agree well with expert manual tracings. The ejection fraction estimates also show good agreement with manual results. Advantages of our approach are demonstrated by comparisons with a conventional pure intensity model, a registration-based contour tracker, and a state-of-the-art database-dependent offline dynamical shape model. We also demonstrate the feasibility of clinical application by applying the method to four 4D human data sets.

A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images

A collaborative framework was initiated to establish a community resource of ground truth segmentations from cardiac MRI. Multi-site, multi-vendor cardiac MRI datasets comprising 95 patients (73 men, 22 women; mean age 62.73±11.24years) with coronary artery disease and prior myocardial infarction, were randomly selected from data made available by the Cardiac Atlas Project (Fonseca et al., 2011). Three semi- and two fully-automated raters segmented the left ventricular myocardium from short-axis cardiac MR images as part of a challenge introduced at the STACOM 2011 MICCAI workshop (Suinesiaputra et al., 2012). Consensus myocardium images were generated based on the Expectation-Maximization principle implemented by the STAPLE algorithm (Warfield et al., 2004). The mean sensitivity, specificity, positive predictive and negative predictive values ranged between 0.63 and 0.85, 0.60 and 0.98, 0.56 and 0.94, and 0.83 and 0.92, respectively, against the STAPLE consensus. Spatial and temporal agreement varied in different amounts for each rater. STAPLE produced high quality consensus images if the region of interest was limited to the area of discrepancy between raters. To maintain the quality of the consensus, an objective measure based on the candidate automated rater performance distribution is proposed. The consensus segmentation based on a combination of manual and automated raters were more consistent than any particular rater, even those with manual input. The consensus is expected to improve with the addition of new automated contributions. This resource is open for future contributions, and is available as a test bed for the evaluation of new segmentation algorithms, through the Cardiac Atlas Project (www.cardiacatlas.org).

Detection of Cardiac Quiescence from B-Mode Echocardiography Using a Correlation-Based Frame-to-Frame Deviation Measure

Two novel methods for detecting cardiac quiescent phases from B-mode echocardiography using a correlation-based frame-to-frame deviation measure were developed. Accurate knowledge of cardiac quiescence is crucial to the performance of many imaging modalities, including computed tomography coronary angiography (CTCA). Synchronous electrocardiography (ECG) and echocardiography data were obtained from 10 healthy human subjects (four male, six female, 23–45 years) and the interventricular septum (IVS) was observed using the apical four-chamber echocardiographic view. The velocity of the IVS was derived from active contour tracking and verified using tissue Doppler imaging echocardiography methods. In turn, the frame-to-frame deviation methods for identifying quiescence of the IVS were verified using active contour tracking. The timing of the diastolic quiescent phase was found to exhibit both inter- and intra-subject variability, suggesting that the current method of CTCA gating based on the ECG is suboptimal and that gating based on signals derived from cardiac motion are likely more accurate in predicting quiescence for cardiac imaging. Two robust and efficient methods for identifying cardiac quiescent phases from B-mode echocardiographic data were developed and verified. The methods presented in this paper will be used to develop new CTCA gating techniques and quantify the resulting potential improvement in CTCA image quality.

Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data

We present a new statistical pattern recognition approach for the problem of left ventricle endocardium tracking in ultrasound data. The problem is formulated as a sequential importance resampling algorithm such that the expected segmentation of the current time step is estimated based on the appearance, shape, and motion models that take into account all previous and current images and previous segmentation contours produced by the method. The new appearance and shape models decouple the affine and nonrigid segmentations of the left ventricle to reduce the running time complexity. The proposed motion model combines the systole and diastole motion patterns and an observation distribution built by a deep neural network. The functionality of our approach is evaluated using a dataset of diseased cases containing 16 sequences and another dataset of normal cases comprised of four sequences, where both sets present long axis views of the left ventricle. Using a training set comprised of diseased and healthy cases, we show that our approach produces more accurate results than current state-of-the-art endocardium tracking methods in two test sequences from healthy subjects. Using three test sequences containing different types of cardiopathies, we show that our method correlates well with interuser statistics produced by four cardiologists.

Snake model-based lymphoma segmentation for sequential CT images

The measurement of the size of lesions in follow-up CT examinations of cancer patients is important to evaluate the success of treatment. This paper presents an automatic algorithm for identifying and segmenting lymph nodes in CT images across longitudinal time points. Firstly, a two-step image registration method is proposed to locate the lymph nodes including coarse registration based on body region detection and fine registration based on a double-template matching algorithm. Then, to make the initial segmentation approximate the boundaries of lymph nodes, the initial image registration result is refined with intensity and edge information. Finally, a snake model is used to evolve the refined initial curve and obtain segmentation results. Our algorithm was tested on 26 lymph nodes at multiple time points from 14 patients. The image at the earlier time point was used as the baseline image to be used in evaluating the follow-up image, resulting in 76 total test cases. Of the 76 test cases, we made a 76 (100%) successful detection and 38/40 (95%) correct clinical assessment according to Response Evaluation Criteria in Solid Tumors (RECIST). The quantitative evaluation based on several metrics, such as average Hausdorff distance, indicates that our algorithm is produces good results. In addition, the proposed algorithm is fast with an average computing time 2.58s. The proposed segmentation algorithm for lymph nodes is fast and can achieve high segmentation accuracy, which may be useful to automate the tracking and evaluation of cancer therapy.

Serial slice image segmentation of digital human based on adaptive geometric active contour tracking

Segmentation is one of the crucial problems for the digital human research, as currently digital human datasets are manually segmented by experts with anatomy knowledge. Due to the thin slice thickness of digital human data, the static slices can be regarded as a sequence of temporal deformation of the same slice. This gives light to the method of object contour tracking for the segmentation task for the digital human data. In this paper, we present an adaptive geometric active contour tracking method, based on a feature image of object contour, to segment tissues in digital human data. The feature image is constructed according to the matching degree of object contour points, image variance and gradient, and statistical models of the object and background colors. Utilizing the characteristics of the feature image, the traditional edge-based geometric active contour model is improved to adaptively evolve curve in any direction instead of the single direction. Experimental results demonstrate that the proposed method is robust to automatically handle the topological changes, and is effective for the segmentation of digital human data.

Extracting cardiac shapes and motion of the chick embryo heart outflow tract from four-dimensional optical coherence tomography images

Recent advances in optical coherence tomography (OCT), and the development of image reconstruction algorithms, enabled four-dimensional (4-D) (three-dimensional imaging over time) imaging of the embryonic heart. To further analyze and quantify the dynamics of cardiac beating, segmentation procedures that can extract the shape of the heart and its motion are needed. Most previous studies analyzed cardiac image sequences using manually extracted shapes and measurements. However, this is time consuming and subject to inter-operator variability. Automated or semi-automated analyses of 4-D cardiac OCT images, although very desirable, are also extremely challenging. This work proposes a robust algorithm to semi automatically detect and track cardiac tissue layers from 4-D OCT images of early (tubular) embryonic hearts. Our algorithm uses a two-dimensional (2-D) deformable double-line model (DLM) to detect target cardiac tissues. The detection algorithm uses a maximum-likelihood estimator and was successfully applied to 4-D in vivo OCT images of the heart outflow tract of day three chicken embryos. The extracted shapes captured the dynamics of the chick embryonic heart outflow tract wall, enabling further analysis of cardiac motion.

Computing myocardial motion in 4-dimensional echocardiography

We describe a novel method for computing dense 3D myocardial motion with high accuracy in four-dimensional (4D) echocardiography (3 dimensions spatial plus time). The method is based on a classic variational optical flow technique but exploits modern developments in optical flow research to utilize the full capabilities of 4D echocardiography. Using a variety of metrics, we present an in-depth performance evaluation of the method on synthetic, phantom, and intraoperative 4D transesophageal echocardiographic data. When compared with state-of-the-art optical flow and speckle tracking techniques currently found in 4D echocardiography, the method we present shows notable improvements in error rates. We believe the performance improvements shown can have a positive impact when the method is used as input for various applications, such as strain computation, biomechanical modeling, and automated diagnostics.

The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods

We present a new supervised learning model designed for the automatic segmentation of the left ventricle (LV) of the heart in ultrasound images. We address the following problems inherent to supervised learning models: 1) the need of a large set of training images; 2) robustness to imaging conditions not present in the training data; and 3) complex search process. The innovations of our approach reside in a formulation that decouples the rigid and nonrigid detections, deep learning methods that model the appearance of the LV, and efficient derivative-based search algorithms. The functionality of our approach is evaluated using a data set of diseased cases containing 400 annotated images (from 12 sequences) and another data set of normal cases comprising 80 annotated images (from two sequences), where both sets present long axis views of the LV. Using several error measures to compute the degree of similarity between the manual and automatic segmentations, we show that our method not only has high sensitivity and specificity but also presents variations with respect to a gold standard (computed from the manual annotations of two experts) within interuser variability on a subset of the diseased cases. We also compare the segmentations produced by our approach and by two state-of-the-art LV segmentation models on the data set of normal cases, and the results show that our approach produces segmentations that are comparable to these two approaches using only 20 training images and increasing the training set to 400 images causes our approach to be generally more accurate. Finally, we show that efficient search methods reduce up to tenfold the complexity of the method while still producing competitive segmentations. In the future, we plan to include a dynamical model to improve the performance of the algorithm, to use semisupervised learning methods to reduce even more the dependence on rich and large training sets, and to design a shape model less dependent on the training set.

An active contour based method for analyzing cardiac quiescence from echocardiography

A semi-automated method for analyzing cardiac quiescence of anatomical cardiac features from two-dimensional echocardiographic cine data is presented. The method utilizes both active contour and optical flow techniques for feature identification and tracking. A curvature-based potential surface was used in the active contour calculations to attract the contour to regions of inflection on the image surface rather than the standard gradient-based surface that attracts the contour to strong edges. After identifying the feature in each frame, the frame-to-frame correlation matrix of the feature was calculated with correlation values corresponding to how well the feature matched between frames. Therefore prolonged regions of high correlation correspond to periods of cardiac quiescence. The location and duration of these periods were automatically identified from the correlation matrix by finding the largest region around each time index with a mean correlation above a specified threshold. In parallel, the position of the feature was calculated for each frame by finding the centroid of the pixel locations inside the contour. From this trajectory, the magnitude of the two-dimensional velocity was calculated. These methods were used to analyze the quiescence of the interventricular septum from an apical four-chamber echocardiogram performed on a human subject. Correlation-derived quiescent phases were observed to coincide with periods of the cardiac cycle with minimal velocity magnitude.

Active ellipse model and automatic chamber detection in apical views of echocardiography images

In this article, an automatic method for detection of all chambers in apical two- and four-chamber views is proposed. The method is based on four evolving ellipses with their sizes and alignments (centre point) gradually changing through iterations until they reach to the point that approximates the chamber boundaries. The interaction between the internal, external and inter-elliptic forces controls the simultaneous evolution of ellipses. Since no prior assumption of the approximate location is required with our approach, the specialists are not required to locate the centre points of chambers in apical images, making the overall segmentation fully automated. Moreover, the resultant ellipse inside a chamber could be used as the initial contour in segmentation techniques such as active contour models, where the initial contour has a significant role for higher accuracy and faster convergence. The simplicity of equations developed in our approach make for a computationally faster algorithm, compared with former approaches that utilize morphologic operators. Our evolving ellipse does not go beyond the gaps, a problem that normally exists within boundaries in echo images, making our overall segmentation process more robust against the gaps. To evaluate the proposed method, a subset of 80 images is selected and three observers are requested to manually draw best ellipses inside the images and compare them with our results. The obtained dice coefficient results (87.62 ± 4.53% for observer-1, 83.18 ± 6.20% for observer-2, 86.02 ± 5.16% for observer-3) indicate that the proposed method has a useful performance.

MR-guided thermotherapy of abdominal organs using a robust PCA-based motion descriptor

Thermotherapies can now be guided in real-time using magnetic resonance imaging (MRI). This technique is rapidly gaining importance in interventional therapies for abdominal organs such as liver and kidney. An accurate online estimation and characterization of organ displacement is mandatory to prevent misregistration and correct for motion related thermometry artifacts. In addition, when the ablation is performed with an extracorporal heating device such as high intensity focused ultrasound (HIFU), the continuous estimation of the organ displacement is the basis for the dynamic adjustment of the focal point position to track the targeted pathological tissue. In this paper, we describe the use of an optimized principal component analysis (PCA)-based motion descriptor to characterize in real-time the complex organ deformation during the therapy. The PCA was used to detect, in a preparative learning step, spatio-temporal coherences in the motion of the targeted organ. During hyperthermia, incoherent motion patterns could be discarded, which enabled improvements in motion estimation robustness, the compensation of motion related errors in thermal maps, and the adjustment of the beam position. The suggested method was evaluated for a moving phantom, and tested in vivo in the kidney and the liver of 12 healthy volunteers under free breathing conditions. The ability to perform a MR-guided thermotherapy in vivo during HIFU intervention was finally demonstrated on a porcine kidney.

Automated regional analysis of B-mode ultrasound images of skeletal muscle movement

To understand the functional significance of skeletal muscle anatomy, a method of quantifying local shape changes in different tissue structures during dynamic tasks is required. Taking advantage of the good spatial and temporal resolution of B-mode ultrasound imaging, we describe a method of automatically segmenting images into fascicle and aponeurosis regions and tracking movement of features, independently, in localized portions of each tissue. Ultrasound images (25 Hz) of the medial gastrocnemius muscle were collected from eight participants during ankle joint rotation (2° and 20°), isometric contractions (1, 5, and 50 Nm), and deep knee bends. A Kanade-Lucas-Tomasi feature tracker was used to identify and track any distinctive and persistent features within the image sequences. A velocity field representation of local movement was then found and subdivided between fascicle and aponeurosis regions using segmentations from a multiresolution active shape model (ASM). Movement in each region was quantified by interpolating the effect of the fields on a set of probes. ASM segmentation results were compared with hand-labeled data, while aponeurosis and fascicle movement were compared with results from a previously documented cross-correlation approach. ASM provided good image segmentations (<1 mm average error), with fully automatic initialization possible in sequences from seven participants. Feature tracking provided similar length change results to the cross-correlation approach for small movements, while outperforming it in larger movements. The proposed method provides the potential to distinguish between active and passive changes in muscle shape and model strain distributions during different movements/conditions and quantify nonhomogeneous strain along aponeuroses.