Ultrasonic image analysis and image-guided interventions

Advancing multimodal medical image fusion: an adaptive image decomposition approach based on multilevel Guided filtering

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

Bend-insensitive fiber optic ultrasonic tracking probe for cardiovascular interventions

BACKGROUND

Transesophageal echocardiography (TEE) is widely used to guide medical device placement in minimally invasive cardiovascular procedures. However, visualization of the device tip with TEE can be challenging. Ultrasonic tracking, enabled by an integrated fiber optic ultrasound sensor (FOUS) that receives transmissions from the TEE probe, is very well suited to improving device localization in this context. The problem addressed in this study is that tight deflections of devices such as a steerable guide catheter can result in bending of the FOUS beyond its specifications and a corresponding loss of ultrasound sensitivity.

PURPOSE

A bend-insensitive FOUS was developed, and its utility with ultrasonic tracking of a steerable tip during TEE-based image guidance was demonstrated.

METHODS

Fiberoptic ultrasound sensors were fabricated using both standard and bend insensitive single mode fibers and subjected to static bending at the distal end. The interference transfer function and ultrasound sensitivities were compared for both types of FOUS. The bend-insensitive FOUS was integrated within a steerable guide catheter, which served as an exemplar device; the signal-to-noise ratio (SNR) of tracking signals from the catheter tip with a straight and a fully deflected distal end were measured in a cardiac ultrasound phantom for over 100 frames.

RESULTS

With tight bending at the distal end (bend radius < 10 mm), the standard FOUS experienced a complete loss of US sensitivity due to high attenuation in the fiber, whereas the bend-insensitive FOUS had largely unchanged performance, with a SNR of 47.7 for straight fiber and a SNR of 36.8 at a bend radius of 3.0 mm. When integrated into the steerable guide catheter, the mean SNRs of the ultrasonic tracking signals recorded with the catheter in a cardiac phantom were similar for straight and fully deflected distal ends: 195 and 163.

CONCLUSION

The FOUS fabricated from bend-insensitive fiber overcomes the bend restrictions associated with the FOUS fabricated from standard single mode fiber, thereby enabling its use in ultrasonic tracking in a wide range of cardiovascular devices.

Transforming obstetric ultrasound into data science using eye tracking, voice recording, transducer motion and ultrasound video

Ultrasound is the primary modality for obstetric imaging and is highly sonographer dependent. Long training period, insufficient recruitment and poor retention of sonographers are among the global challenges in the expansion of ultrasound use. For the past several decades, technical advancements in clinical obstetric ultrasound scanning have largely concerned improving image quality and processing speed. By contrast, sonographers have been acquiring ultrasound images in a similar fashion for several decades. The PULSE (Perception Ultrasound by Learning Sonographer Experience) project is an interdisciplinary multi-modal imaging study aiming to offer clinical sonography insights and transform the process of obstetric ultrasound acquisition and image analysis by applying deep learning to large-scale multi-modal clinical data. A key novelty of the study is that we record full-length ultrasound video with concurrent tracking of the sonographer's eyes, voice and the transducer while performing routine obstetric scans on pregnant women. We provide a detailed description of the novel acquisition system and illustrate how our data can be used to describe clinical ultrasound. Being able to measure different sonographer actions or model tasks will lead to a better understanding of several topics including how to effectively train new sonographers, monitor the learning progress, and enhance the scanning workflow of experts.

Simultaneous compression and speckle reduction of clinical breast and fetal ultrasound images using rate-fidelity optimized coding

Medical ultrasound images are inherently noised with speckle noise, which may interfere with Computer Aided Diagnostics (CAD) tasks, such as automatic segmentation. A compression and speckle de-noising method is proposed and tested on real clinical breast and fetal ultrasound images. The proposed algorithm is based on the optimization of quantization coefficients when applying Wavelet representation on the image, where the optimization is held such that a pre-defined mathematical fidelity criterion with respect to a desired de-speckled image is obtained. The proposed algorithm yields effective speckle reduction whilst preserving the edges in the images, with a reduced computational burden compared to other existing state-of-the-art methods, such as Optimal Bayesian Non-Local Means (OBNLM). In addition, the images are simultaneously compressed to a target bit-rate. The proposed algorithm is evaluated using both objective mathematical fidelity criteria (such as Structural Similarity and Edge Preserve) as well as subjective radiologists tests. The experimental results demonstrate the ability of the proposed method to achieve de-speckled images with compression ratios of approximately 30:1, whilst obtaining competitive subjective as well as objective fidelity measures with respect to the desired de-speckled images.

One Shot Segmentation: Unifying Rigid Detection and Non-Rigid Segmentation Using Elastic Regularization

This paper proposes a novel approach for the non-rigid segmentation of deformable objects in image sequences, which is based on one-shot segmentation that unifies rigid detection and non-rigid segmentation using elastic regularization. The domain of application is the segmentation of a visual object that temporally undergoes a rigid transformation (e.g., affine transformation) and a non-rigid transformation (i.e., contour deformation). The majority of segmentation approaches to solve this problem are generally based on two steps that run in sequence: a rigid detection, followed by a non-rigid segmentation. In this paper, we propose a new approach, where both the rigid and non-rigid segmentation are performed in a single shot using a sparse low-dimensional manifold that represents the visual object deformations. Given the multi-modality of these deformations, the manifold partitions the training data into several patches, where each patch provides a segmentation proposal during the inference process. These multiple segmentation proposals are merged using the classification results produced by deep belief networks (DBN) that compute the confidence on each segmentation proposal. Thus, an ensemble of DBN classifiers is used for estimating the final segmentation. Compared to current methods proposed in the field, our proposed approach is advantageous in four aspects: (i) it is a unified framework to produce rigid and non-rigid segmentations; (ii) it uses an ensemble classification process, which can help the segmentation robustness; (iii) it provides a significant reduction in terms of the number of dimensions of the rigid and non-rigid segmentations search spaces, compared to current approaches that divide these two problems; and (iv) this lower dimensionality of the search space can also reduce the need for large annotated training sets to be used for estimating the DBN models. Experiments on the problem of left ventricle endocardial segmentation from ultrasound images, and lip segmentation from frontal facial images using the extended Cohn-Kanade (CK+) database, demonstrate the potential of the methodology through qualitative and quantitative evaluations, and the ability to reduce the search and training complexities without a significant impact on the segmentation accuracy.

Deep learning approach for automatic out-of-plane needle localisation for semi-automatic ultrasound probe calibration

The authors present a deep learning algorithm for the automatic centroid localisation of out-of-plane US needle reflections to produce a semi-automatic ultrasound (US) probe calibration algorithm. A convolutional neural network was trained on a dataset of 3825 images at a 6 cm imaging depth to predict the position of the centroid of a needle reflection. Applying the automatic centroid localisation algorithm to a test set of 614 annotated images produced a root mean squared error of 0.62 and 0.74 mm (6.08 and 7.62 pixels) in the axial and lateral directions, respectively. The mean absolute errors associated with the test set were 0.50 ± 0.40 mm and 0.51 ± 0.54 mm (4.9 ± 3.96 pixels and 5.24 ± 5.52 pixels) for the axial and lateral directions, respectively. The trained model was able to produce visually validated US probe calibrations at imaging depths on the range of 4–8 cm, despite being solely trained at 6 cm. This work has automated the pixel localisation required for the guided-US calibration algorithm producing a semi-automatic implementation available open-source through 3D Slicer. The automatic needle centroid localisation improves the usability of the algorithm and has the potential to decrease the fiducial localisation and target registration errors associated with the guided-US calibration method.

A New Active Contours Approach for Finger Extensor Tendon Segmentation in Ultrasound Images Using Prior Knowledge and Phase Symmetry

This work proposes a new approach for the segmentation of the extensor tendon in ultrasound images of the second metacarpophalangeal joint (MCPJ). The MCPJ is known to be frequently involved in early stages of rheumatic diseases like rheumatoid arthritis. The early detection and follow up of these diseases is important to start and adapt the treatments properly and, in that way, preventing irreversible damage of the joints. This work relies on an active contours framework, preceded by a phase symmetry preprocessing and with prior knowledge energies, to automatically identify the extensor tendon. Active contours methods are widely used in ultrasound images because of their robustness to speckle noise and ability to join unconnected smaller regions into a coherent shape. The tendon is formulated as a line so open ended active contours were used. Phase symmetry highlights the tendon, by setting a proper scale range and angle span. The distance between structures and the tendon slope were also included to enforce the model based on anatomical characteristics. And finally, the concavity measures were used because, given the anatomy of the finger, we know that the tendon line should have less than two concavities. To solve the active contours energy minimization a genetic algorithm approach was used. Several energy metric configurations were compared using the modified Hausdorff distance and results showed that this segmentation is not only possible, but exhibits errors smaller than 0.5 mm with a confidence of 95% with the phase symmetry preprocessing and energies based on the line neighborhood, area ratio, slope, and concavity measurements.

Automatic pneumonia detection based on ultrasound video analysis

Pneumonia is a disease which causes high mortality in children under five years old, particularly in developing countries. This paper proposes a novel application of ultrasound video analysis for the detection of pneumonia. This application is based on the processing of small video chunks, in which an image processing algorithm analyzes each frame to get some overall video statistics. Then, based on these quantities, the likeness of presence of pneumonia in the video is determined. The algorithm exploits different geometrical properties of typical anatomical and pathological features that commonly appear in lung sonography and which are already clinically typified in the literature. Our technique has been tested on different transverse thoracic scanning protocols and probe's maneuvers, thus, under a variety of clinical and usage protocols. Then, it can be targeted towards screening applications. We present encouraging results (AUC measure between 0.7851 and 0.9177) based on the analysis of 346 videos with an average duration of eight seconds. The analyzed videos were taken from children who were between three and five years old. Finally, our algorithm can be used directly as a classifier, but we detail how its performance may be enhanced if used as a first stage of a larger pipeline of other complementary pneumonia detection processes.

Murine ultrasound-guided transabdominal para-aortic injections of self-assembling type I collagen oligomers

Abdominal aortic aneurysms (AAAs) represent a potentially life-threatening condition that predominantly affects the infrarenal aorta. Several preclinical murine models that mimic the human condition have been developed and are now widely used to investigate AAA pathogenesis. Cell- or pharmaceutical-based therapeutics designed to prevent AAA expansion are currently being evaluated with these animal models, but more minimally invasive strategies for delivery could improve their clinical translation. The purpose of this study was to investigate the use of self-assembling type I collagen oligomers as an injectable therapeutic delivery vehicle in mice. Here we show the success and reliability of a para-aortic, ultrasound-guided technique for injecting quickly-polymerizing collagen oligomer solutions into mice to form a collagen-fibril matrix at body temperature. A commonly used infrarenal mouse AAA model was used to determine the target location of these collagen injections. Ultrasound-guided, closed-abdominal injections supported consistent delivery of collagen to the area surrounding the infrarenal abdominal aorta halfway between the right renal artery and aortic trifurcation into the iliac and tail arteries. This minimally invasive approach yielded outcomes similar to open-abdominal injections into the same region. Histological analysis on tissue removed on day 14 post-operatively showed minimal in vivo degradation of the self-assembled fibrillar collagen and the majority of implants experienced minimal inflammation and cell invasion, further confirming this material's potential as a method for delivering therapeutics. Finally, we showed that the typical length and position of this infrarenal AAA model was statistically similar to the length and targeted location of the injected collagen, increasing its feasibility as a localized therapeutic delivery vehicle. Future preclinical and clinical studies are needed to determine if specific therapeutics incorporated into the self-assembling type I collagen matrix described here can be delivered near the aorta and locally limit AAA expansion.

Reflections on ultrasound image analysis

Ultrasound (US) image analysis has advanced considerably in twenty years. Progress in ultrasound image analysis has always been fundamental to the advancement of image-guided interventions research due to the real-time acquisition capability of ultrasound and this has remained true over the two decades. But in quantitative ultrasound image analysis - which takes US images and turns them into more meaningful clinical information - thinking has perhaps more fundamentally changed. From roots as a poor cousin to Computed Tomography (CT) and Magnetic Resonance (MR) image analysis, both of which have richer anatomical definition and thus were better suited to the earlier eras of medical image analysis which were dominated by model-based methods, ultrasound image analysis has now entered an exciting new era, assisted by advances in machine learning and the growing clinical and commercial interest in employing low-cost portable ultrasound devices outside traditional hospital-based clinical settings. This short article provides a perspective on this change, and highlights some challenges ahead and potential opportunities in ultrasound image analysis which may both have high impact on healthcare delivery worldwide in the future but may also, perhaps, take the subject further away from CT and MR image analysis research with time.

Evaluation of Medical Image Registration Techniques Based on Nature and Domain of the Transformation

A lot of research has been done during the past 20 years in the area of medical image registration for obtaining detailed, important, and complementary information from two or more images and aligning them into a single, more informative image. Nature of the transformation and domain of the transformation are two important medical image registration techniques that deal with characters of objects (motions) in images. This article presents a detailed survey of the registration techniques that belong to both categories with detailed elaboration on their features, issues, and challenges. An investigation estimating similarity and dissimilarity measures and performance evaluation is the main objective of this work. This article also provides reference knowledge in a compact form for researchers and clinicians looking for the proper registration technique for a particular application.

Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey

Reviewed here are methods developed for following (i.e., tracking) structures in medical B-mode ultrasound time sequences during large-scale motion. The resulting motion estimation problem and its key components are defined. The main tracking approaches are described, and their strengths and weaknesses are discussed. Existing motion estimation methods, tested on multiple in vivo sequences, are categorized with respect to their clinical applications, namely, cardiac, respiratory and muscular motion. A large number of works in this field had to be discarded as thorough validation of the results was missing. The remaining relevant works identified indicate the possibility of reaching an average tracking accuracy up to 1-2 mm. Real-time performance can be achieved using several methods. Yet only very few of these have progressed to clinical practice. The latest trends include incorporation of complementary and prior information. Advances are expected from common evaluation databases and 4-D ultrasound scanning technologies.

Feature extraction and classification for ultrasound images of lumbar spine with support vector machine

In this paper, we proposed a feature extraction and machine learning method for the classification of ultrasound images obtained from lumbar spine of pregnant patients in the transverse plane. A group of features, including matching values and positions, appearance of black pixels within predefined windows along the midline, are extracted from the ultrasound images using template matching and midline detection. Support vector machine (SVM) with Gaussian kernel is utilized to classify the bone images and interspinous images with optimal separation hyperplane. The SVM is trained with 800 images from 20 pregnant subjects and tested with 640 images from a separate set of 16 pregnant patients. A high success rate (97.25% on training set and 95.00% on test set) is achieved with the proposed method. The trained SVM model is further tested on 36 videos collected from 36 pregnant subjects and successfully identified the proper needle insertion site (interspinous region) on all of the cases. Therefore, the proposed method is able to identify the ultrasound images of lumbar spine in an automatic manner, so as to facilitate the anesthetists' work to identify the needle insertion point precisely and effectively.

Lumbar Ultrasound Image Feature Extraction and Classification with Support Vector Machine

Needle entry site localization remains a challenge for procedures that involve lumbar puncture, for example, epidural anesthesia. To solve the problem, we have developed an image classification algorithm that can automatically identify the bone/interspinous region for ultrasound images obtained from lumbar spine of pregnant patients in the transverse plane. The proposed algorithm consists of feature extraction, feature selection and machine learning procedures. A set of features, including matching values, positions and the appearance of black pixels within pre-defined windows along the midline, were extracted from the ultrasound images using template matching and midline detection methods. A support vector machine was then used to classify the bone images and interspinous images. The support vector machine model was trained with 1,040 images from 26 pregnant subjects and tested on 800 images from a separate set of 20 pregnant patients. A success rate of 95.0% on training set and 93.2% on test set was achieved with the proposed method. The trained support vector machine model was further tested on 46 off-line collected videos, and successfully identified the proper needle insertion site (interspinous region) in 45 of the cases. Therefore, the proposed method is able to process the ultrasound images of lumbar spine in an automatic manner, so as to facilitate the anesthetists' work of identifying the needle entry site.

Segmentation of tumor ultrasound image in HIFU therapy based on texture and boundary encoding

It is crucial in high intensity focused ultrasound (HIFU) therapy to detect the tumor precisely with less manual intervention for enhancing the therapy efficiency. Ultrasound image segmentation becomes a difficult task due to signal attenuation, speckle effect and shadows. This paper presents an unsupervised approach based on texture and boundary encoding customized for ultrasound image segmentation in HIFU therapy. The approach oversegments the ultrasound image into some small regions, which are merged by using the principle of minimum description length (MDL) afterwards. Small regions belonging to the same tumor are clustered as they preserve similar texture features. The mergence is completed by obtaining the shortest coding length from encoding textures and boundaries of these regions in the clustering process. The tumor region is finally selected from merged regions by a proposed algorithm without manual interaction. The performance of the method is tested on 50 uterine fibroid ultrasound images from HIFU guiding transducers. The segmentations are compared with manual delineations to verify its feasibility. The quantitative evaluation with HIFU images shows that the mean true positive of the approach is 93.53%, the mean false positive is 4.06%, the mean similarity is 89.92%, the mean norm Hausdorff distance is 3.62% and the mean norm maximum average distance is 0.57%. The experiments validate that the proposed method can achieve favorable segmentation without manual initialization and effectively handle the poor quality of the ultrasound guidance image in HIFU therapy, which indicates that the approach is applicable in HIFU therapy.

Automatic identification of needle insertion site in epidural anesthesia with a cascading classifier

Ultrasound imaging was used to detect the anatomic structure of lumbar spine from the transverse view, to facilitate needle insertion in epidural anesthesia. The interspinous images that represent proper needle insertion sites were identified automatically with image processing and pattern recognition techniques. On the basis of ultrasound video streams obtained in pregnant patients, the image processing and identification procedure in a previous work was tested and improved. The test results indicate that the pre-processing algorithm performs well on lumbar spine ultrasound images, whereas the classifier is not flexible enough for pregnant patients. To improve the accuracy of identification, we propose a cascading classifier that successfully located the proper needle insertion site on all of the 36 video streams collected from pregnant patients. The results indicate that the proposed image identification procedure is able to identify the ultrasound images of lumbar spine in an automatic manner, so as to facilitate the anesthetists' work to identify the needle insertion point precisely and effectively.

Despeckle filtering software toolbox for ultrasound imaging of the common carotid artery

Ultrasound imaging of the common carotid artery (CCA) is a non-invasive tool used in medicine to assess the severity of atherosclerosis and monitor its progression through time. It is also used in border detection and texture characterization of the atherosclerotic carotid plaque in the CCA, the identification and measurement of the intima-media thickness (IMT) and the lumen diameter that all are very important in the assessment of cardiovascular disease (CVD). Visual perception, however, is hindered by speckle, a multiplicative noise, that degrades the quality of ultrasound B-mode imaging. Noise reduction is therefore essential for improving the visual observation quality or as a pre-processing step for further automated analysis, such as image segmentation of the IMT and the atherosclerotic carotid plaque in ultrasound images. In order to facilitate this preprocessing step, we have developed in MATLAB(®) a unified toolbox that integrates image despeckle filtering (IDF), texture analysis and image quality evaluation techniques to automate the pre-processing and complement the disease evaluation in ultrasound CCA images. The proposed software, is based on a graphical user interface (GUI) and incorporates image normalization, 10 different despeckle filtering techniques (DsFlsmv, DsFwiener, DsFlsminsc, DsFkuwahara, DsFgf, DsFmedian, DsFhmedian, DsFad, DsFnldif, DsFsrad), image intensity normalization, 65 texture features, 15 quantitative image quality metrics and objective image quality evaluation. The software is publicly available in an executable form, which can be downloaded from http://www.cs.ucy.ac.cy/medinfo/. It was validated on 100 ultrasound images of the CCA, by comparing its results with quantitative visual analysis performed by a medical expert. It was observed that the despeckle filters DsFlsmv, and DsFhmedian improved image quality perception (based on the expert's assessment and the image texture and quality metrics). It is anticipated that the system could help the physician in the assessment of cardiovascular image analysis.