Patch-based segmentation using expert priors: application to hippocampus and ventricle segmentation

Quantifying and modelling tissue maturation in the living human fetal brain

Recent advances in medical imaging are beginning to allow us to quantify brain tissue maturation in the growing human brain prior to normal term age, and are beginning to shed new light on early human brain growth. These advances compliment the work already done in cellular level imaging in animal and post mortem studies of brain development. The opportunities for collaborative research that bridges the gap between macroscopic and microscopic windows on the developing brain are significant. The aim of this paper is to provide a review of the current research into MR imaging of the living fetal brain with the aim of motivating improved interfaces between the two fields. The review begins with a description of faster MRI techniques that are capable of freezing motion of the fetal head during the acquisition of a slice, and how these have been combined with advanced post-processing algorithms to build 3D images from motion scattered slices. Such rich data has motivated the development of techniques to automatically label developing tissue zones within MRI data allowing their quantification in 3D and 4D within the normally growing fetal brain. These methods have provided the basis for later work that has created the first maps of tissue growth rate and cortical folding in normally developing brains in-utero. These measurements provide valuable findings that compliment those derived from post-mortem anatomy, and additionally allow for the possibility of larger population studies of the influence of maternal environmental and genes on early brain development.

Optimal optode montage on electroencephalography/functional near-infrared spectroscopy caps dedicated to study epileptic discharges

Functional near-infrared spectroscopy (fNIRS), acquired simultaneously with electroencephalography (EEG), allows the investigation of hemodynamic brain responses to epileptic activity. Because the presumed epileptogenic focus is patient-specific, an appropriate source/detector (SD) montage has to be reconfigured for each patient. The combination of EEG and fNIRS, however, entails several constraints on montages, and finding an optimal arrangement of optodes on the cap is an important issue. We present a method for computing an optimal SD montage on an EEG/fNIRS cap that focuses on one or several specific brain regions; the montage maximizes the spatial sensitivity. We formulate this optimization problem as a linear integer programming problem. The method was evaluated on two EEG/fNIRS caps. We simulated absorbers at different locations on a head model and generated realistic optical density maps on the scalp. We found that the maps of optimal SD montages had spatial resolution properties comparable to those of regular SD arrangements for the whole head with significantly fewer sensors than regular SD arrangements. In addition, we observed that optimal montages yielded improved spatial density of fNIRS measurements over the targeted regions together with an increase in signal-to-noise ratio.

Accurate and Fully Automatic Hippocampus Segmentation Using Subject-Specific 3D Optimal Local Maps Into a Hybrid Active Contour Model

Assessing the structural integrity of the hippocampus (HC) is an essential step toward prevention, diagnosis, and follow-up of various brain disorders due to the implication of the structural changes of the HC in those disorders. In this respect, the development of automatic segmentation methods that can accurately, reliably, and reproducibly segment the HC has attracted considerable attention over the past decades. This paper presents an innovative 3-D fully automatic method to be used on top of the multiatlas concept for the HC segmentation. The method is based on a subject-specific set of 3-D optimal local maps (OLMs) that locally control the influence of each energy term of a hybrid active contour model (ACM). The complete set of the OLMs for a set of training images is defined simultaneously via an optimization scheme. At the same time, the optimal ACM parameters are also calculated. Therefore, heuristic parameter fine-tuning is not required. Training OLMs are subsequently combined, by applying an extended multiatlas concept, to produce the OLMs that are anatomically more suitable to the test image. The proposed algorithm was tested on three different and publicly available data sets. Its accuracy was compared with that of state-of-the-art methods demonstrating the efficacy and robustness of the proposed method.

Geodesic patch-based segmentation

Label propagation has been shown to be effective in many automatic segmentation applications. However, its reliance on accurate image alignment means that segmentation results can be affected by any registration errors which occur. Patch-based methods relax this dependence by avoiding explicit one-to-one correspondence assumptions between images but are still limited by the search window size. Too small, and it does not account for enough registration error; too big, and it becomes more likely to select incorrect patches of similar appearance for label fusion. This paper presents a novel patch-based label propagation approach which uses relative geodesic distances to define patient-specific coordinate systems as spatial context to overcome this problem. The approach is evaluated on multi-organ segmentation of 20 cardiac MR images and 100 abdominal CT images, demonstrating competitive results.

Optimized patchMatch for near real time and accurate label fusion

Automatic segmentation methods are important tools for quantitative analysis of magnetic resonance images. Recently, patch-based label fusion approaches demonstrated state-of-the-art segmentation accuracy. In this paper, we introduce a new patch-based method using the PatchMatch algorithm to perform segmentation of anatomical structures. Based on an Optimized PAtchMatch Label fusion (OPAL) strategy, the proposed method provides competitive segmentation accuracy in near real time. During our validation on hippocampus segmentation of 80 healthy subjects, OPAL was compared to several state-of-the-art methods. Results show that OPAL obtained the highest median Dice coefficient (89.3%) in less than 1 sec per subject. These results highlight the excellent performance of OPAL in terms of computation time and segmentation accuracy compared to recently published methods.

Label inference with registration and patch priors

In this paper, we present a novel label inference method that integrates registration and patch priors, and serves as a remedy for labelling errors around structural boundaries. With the initial label map provided by nonrigid registration methods, its corresponding signed distance function can be estimated and used to evaluate the segmentation confidence. The pixels with less confident labels are selected as candidate nodes to be refined and those with relatively confident results are settled as seeds. The affinity between seeds and candidate nodes, which consists of regular image lattice connections, registration prior based on signed distance and patch prior from the warped atlas, is encoded to guide the label inference procedure. For method evaluation, experiments have been carried out on two publicly available data sets and it only takes several seconds for our method to improve the segmentation quality significantly.

Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge

Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p<0.05) and had an efficient implementation with a run time of 8min and 3s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.

Magnetic Resonance Image Example-Based Contrast Synthesis

The performance of image analysis algorithms applied to magnetic resonance images is strongly influenced by the pulse sequences used to acquire the images. Algorithms are typically optimized for a targeted tissue contrast obtained from a particular implementation of a pulse sequence on a specific scanner. There are many practical situations, including multi-institution trials, rapid emergency scans, and scientific use of historical data, where the images are not acquired according to an optimal protocol or the desired tissue contrast is entirely missing. This paper introduces an image restoration technique that recovers images with both the desired tissue contrast and a normalized intensity profile. This is done using patches in the acquired images and an atlas containing patches of the acquired and desired tissue contrasts. The method is an example-based approach relying on sparse reconstruction from image patches. Its performance in demonstrated using several examples, including image intensity normalization, missing tissue contrast recovery, automatic segmentation, and multimodal registration. These examples demonstrate potential practical uses and also illustrate limitations of our approach.

Automatic hippocampus segmentation of 7.0 Tesla MR images by combining multiple atlases and auto-context models

In many neuroscience and clinical studies, accurate measurement of hippocampus is very important to reveal the inter-subject anatomical differences or the subtle intra-subject longitudinal changes due to aging or dementia. Although many automatic segmentation methods have been developed, their performances are still challenged by the poor image contrast of hippocampus in the MR images acquired especially from 1.5 or 3.0 Tesla (T) scanners. With the recent advance of imaging technology, 7.0 T scanner provides much higher image contrast and resolution for hippocampus study. However, the previous methods developed for segmentation of hippocampus from 1.5 T or 3.0 T images do not work for the 7.0 T images, due to different levels of imaging contrast and texture information. In this paper, we present a learning-based algorithm for automatic segmentation of hippocampi from 7.0 T images, by taking advantages of the state-of-the-art multi-atlas framework and also the auto-context model (ACM). Specifically, ACM is performed in each atlas domain to iteratively construct sequences of location-adaptive classifiers by integrating both image appearance and local context features. Due to the plenty texture information in 7.0 T images, more advanced texture features are also extracted and incorporated into the ACM during the training stage. Then, under the multi-atlas segmentation framework, multiple sequences of ACM-based classifiers are trained for all atlases to incorporate the anatomical variability. In the application stage, for a new image, its hippocampus segmentation can be achieved by fusing the labeling results from all atlases, each of which is obtained by applying the atlas-specific ACM-based classifiers. Experimental results on twenty 7.0 T images with the voxel size of 0.35×0.35×0.35 mm3 show very promising hippocampus segmentations (in terms of Dice overlap ratio 89.1±0.020), indicating high applicability for the future clinical and neuroscience studies.

Multi-atlas segmentation with joint label fusion and corrective learning-an open source implementation

Label fusion based multi-atlas segmentation has proven to be one of the most competitive techniques for medical image segmentation. This technique transfers segmentations from expert-labeled images, called atlases, to a novel image using deformable image registration. Errors produced by label transfer are further reduced by label fusion that combines the results produced by all atlases into a consensus solution. Among the proposed label fusion strategies, weighted voting with spatially varying weight distributions derived from atlas-target intensity similarity is a simple and highly effective label fusion technique. However, one limitation of most weighted voting methods is that the weights are computed independently for each atlas, without taking into account the fact that different atlases may produce similar label errors. To address this problem, we recently developed the joint label fusion technique and the corrective learning technique, which won the first place of the 2012 MICCAI Multi-Atlas Labeling Challenge and was one of the top performers in 2013 MICCAI Segmentation: Algorithms, Theory and Applications (SATA) challenge. To make our techniques more accessible to the scientific research community, we describe an Insight-Toolkit based open source implementation of our label fusion methods. Our implementation extends our methods to work with multi-modality imaging data and is more suitable for segmentation problems with multiple labels. We demonstrate the usage of our tools through applying them to the 2012 MICCAI Multi-Atlas Labeling Challenge brain image dataset and the 2013 SATA challenge canine leg image dataset. We report the best results on these two datasets so far.

Local label learning (LLL) for subcortical structure segmentation: application to hippocampus segmentation

Automatic and reliable segmentation of subcortical structures is an important but difficult task in quantitative brain image analysis. Multi-atlas based segmentation methods have attracted great interest due to their promising performance. Under the multi-atlas based segmentation framework, using deformation fields generated for registering atlas images onto a target image to be segmented, labels of the atlases are first propagated to the target image space and then fused to get the target image segmentation based on a label fusion strategy. While many label fusion strategies have been developed, most of these methods adopt predefined weighting models that are not necessarily optimal. In this study, we propose a novel local label learning strategy to estimate the target image's segmentation label using statistical machine learning techniques. In particular, we use a L1-regularized support vector machine (SVM) with a k nearest neighbor (kNN) based training sample selection strategy to learn a classifier for each of the target image voxel from its neighboring voxels in the atlases based on both image intensity and texture features. Our method has produced segmentation results consistently better than state-of-the-art label fusion methods in validation experiments on hippocampal segmentation of over 100 MR images obtained from publicly available and in-house datasets. Volumetric analysis has also demonstrated the capability of our method in detecting hippocampal volume changes due to Alzheimer's disease.

Out-of-atlas likelihood estimation using multi-atlas segmentation

PURPOSE

Multi-atlas segmentation has been shown to be highly robust and accurate across an extraordinary range of potential applications. However, it is limited to the segmentation of structures that are anatomically consistent across a large population of potential target subjects (i.e., multi-atlas segmentation is limited to "in-atlas" applications). Herein, the authors propose a technique to determine the likelihood that a multi-atlas segmentation estimate is representative of the problem at hand, and, therefore, identify anomalous regions that are not well represented within the atlases.

METHODS

The authors derive a technique to estimate the out-of-atlas (OOA) likelihood for every voxel in the target image. These estimated likelihoods can be used to determine and localize the probability of an abnormality being present on the target image.

RESULTS

Using a collection of manually labeled whole-brain datasets, the authors demonstrate the efficacy of the proposed framework on two distinct applications. First, the authors demonstrate the ability to accurately and robustly detect malignant gliomas in the human brain-an aggressive class of central nervous system neoplasms. Second, the authors demonstrate how this OOA likelihood estimation process can be used within a quality control context for diffusion tensor imaging datasets to detect large-scale imaging artifacts (e.g., aliasing and image shading).

CONCLUSIONS

The proposed OOA likelihood estimation framework shows great promise for robust and rapid identification of brain abnormalities and imaging artifacts using only weak dependencies on anomaly morphometry and appearance. The authors envision that this approach would allow for application-specific algorithms to focus directly on regions of high OOA likelihood, which would (1) reduce the need for human intervention, and (2) reduce the propensity for false positives. Using the dual perspective, this technique would allow for algorithms to focus on regions of normal anatomy to ascertain image quality and adapt to image appearance characteristics.

Automated abdominal multi-organ segmentation with subject-specific atlas generation

A robust automated segmentation of abdominal organs can be crucial for computer aided diagnosis and laparoscopic surgery assistance. Many existing methods are specialized to the segmentation of individual organs and struggle to deal with the variability of the shape and position of abdominal organs. We present a general, fully-automated method for multi-organ segmentation of abdominal computed tomography (CT) scans. The method is based on a hierarchical atlas registration and weighting scheme that generates target specific priors from an atlas database by combining aspects from multi-atlas registration and patch-based segmentation, two widely used methods in brain segmentation. The final segmentation is obtained by applying an automatically learned intensity model in a graph-cuts optimization step, incorporating high-level spatial knowledge. The proposed approach allows to deal with high inter-subject variation while being flexible enough to be applied to different organs. We have evaluated the segmentation on a database of 150 manually segmented CT images. The achieved results compare well to state-of-the-art methods, that are usually tailored to more specific questions, with Dice overlap values of 94%, 93%, 70%, and 92% for liver, kidneys, pancreas, and spleen, respectively.

Segmentation of neonatal brain MR images using patch-driven level sets

The segmentation of neonatal brain MR image into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF), is challenging due to the low spatial resolution, severe partial volume effect, high image noise, and dynamic myelination and maturation processes. Atlas-based methods have been widely used for guiding neonatal brain segmentation. Existing brain atlases were generally constructed by equally averaging all the aligned template images from a population. However, such population-based atlases might not be representative of a testing subject in the regions with high inter-subject variability and thus often lead to a low capability in guiding segmentation in those regions. Recently, patch-based sparse representation techniques have been proposed to effectively select the most relevant elements from a large group of candidates, which can be used to generate a subject-specific representation with rich local anatomical details for guiding the segmentation. Accordingly, in this paper, we propose a novel patch-driven level set method for the segmentation of neonatal brain MR images by taking advantage of sparse representation techniques. Specifically, we first build a subject-specific atlas from a library of aligned, manually segmented images by using sparse representation in a patch-based fashion. Then, the spatial consistency in the probability maps from the subject-specific atlas is further enforced by considering the similarities of a patch with its neighboring patches. Finally, the probability maps are integrated into a coupled level set framework for more accurate segmentation. The proposed method has been extensively evaluated on 20 training subjects using leave-one-out cross validation, and also on 132 additional testing subjects. Our method achieved a high accuracy of 0.919±0.008 for white matter and 0.901±0.005 for gray matter, respectively, measured by Dice ratio for the overlap between the automated and manual segmentations in the cortical region.

A probabilistic patch-based label fusion model for multi-atlas segmentation with registration refinement: application to cardiac MR images

The evaluation of ventricular function is important for the diagnosis of cardiovascular diseases. It typically involves measurement of the left ventricular (LV) mass and LV cavity volume. Manual delineation of the myocardial contours is time-consuming and dependent on the subjective experience of the expert observer. In this paper, a multi-atlas method is proposed for cardiac magnetic resonance (MR) image segmentation. The proposed method is novel in two aspects. First, it formulates a patch-based label fusion model in a Bayesian framework. Second, it improves image registration accuracy by utilizing label information, which leads to improvement of segmentation accuracy. The proposed method was evaluated on a cardiac MR image set of 28 subjects. The average Dice overlap metric of our segmentation is 0.92 for the LV cavity, 0.89 for the right ventricular cavity and 0.82 for the myocardium. The results show that the proposed method is able to provide accurate information for clinical diagnosis.

Bayesian Parameter Estimation and Segmentation in the Multi-Atlas Random Orbit Model

This paper examines the multiple atlas random diffeomorphic orbit model in Computational Anatomy (CA) for parameter estimation and segmentation of subcortical and ventricular neuroanatomy in magnetic resonance imagery. We assume that there exist multiple magnetic resonance image (MRI) atlases, each atlas containing a collection of locally-defined charts in the brain generated via manual delineation of the structures of interest. We focus on maximum a posteriori estimation of high dimensional segmentations of MR within the class of generative models representing the observed MRI as a conditionally Gaussian random field, conditioned on the atlas charts and the diffeomorphic change of coordinates of each chart that generates it. The charts and their diffeomorphic correspondences are unknown and viewed as latent or hidden variables. We demonstrate that the expectation-maximization (EM) algorithm arises naturally, yielding the likelihood-fusion equation which the a posteriori estimator of the segmentation labels maximizes. The likelihoods being fused are modeled as conditionally Gaussian random fields with mean fields a function of each atlas chart under its diffeomorphic change of coordinates onto the target. The conditional-mean in the EM algorithm specifies the convex weights with which the chart-specific likelihoods are fused. The multiple atlases with the associated convex weights imply that the posterior distribution is a multi-modal representation of the measured MRI. Segmentation results for subcortical and ventricular structures of subjects, within populations of demented subjects, are demonstrated, including the use of multiple atlases across multiple diseased groups.

Segmentation of MR images via discriminative dictionary learning and sparse coding: application to hippocampus labeling

We propose a novel method for the automatic segmentation of brain MRI images by using discriminative dictionary learning and sparse coding techniques. In the proposed method, dictionaries and classifiers are learned simultaneously from a set of brain atlases, which can then be used for the reconstruction and segmentation of an unseen target image. The proposed segmentation strategy is based on image reconstruction, which is in contrast to most existing atlas-based labeling approaches that rely on comparing image similarities between atlases and target images. In addition, we propose a Fixed Discriminative Dictionary Learning for Segmentation (F-DDLS) strategy, which can learn dictionaries offline and perform segmentations online, enabling a significant speed-up in the segmentation stage. The proposed method has been evaluated for the hippocampus segmentation of 80 healthy ICBM subjects and 202 ADNI images. The robustness of the proposed method, especially of our F-DDLS strategy, was validated by training and testing on different subject groups in the ADNI database. The influence of different parameters was studied and the performance of the proposed method was also compared with that of the nonlocal patch-based approach. The proposed method achieved a median Dice coefficient of 0.879 on 202 ADNI images and 0.890 on 80 ICBM subjects, which is competitive compared with state-of-the-art methods.

Volumetric analysis of medial temporal lobe structures in brain development from childhood to adolescence

Puberty is an important stage of development as a child's sexual and physical characteristics mature because of hormonal changes. To better understand puberty-related effects on brain development, we investigated the magnetic resonance imaging (MRI) data of 306 subjects from 4 to 18 years of age. Subjects were grouped into before and during puberty groups according to their sexual maturity levels measured by the puberty scores. An appearance model-based automatic segmentation method with patch-based local refinement was employed to segment the MRI data and extract the volumes of medial temporal lobe (MTL) structures including the amygdala (AG), the hippocampus (HC), the entorhinal/perirhinal cortex (EPC), and the parahippocampal cortex (PHC). Our analysis showed age-related volumetric changes for the AG, HC, right EPC, and left PHC but only before puberty. After onset of puberty, these volumetric changes then correlate more with sexual maturity level, as measured by the puberty score. When normalized for brain volume, the volumes of the right HC decrease for boys; the volumes of the left HC increase for girls; and the volumes of the left and right PHC decrease for boys. These findings suggest that the rising levels of testosterone in boys and estrogen in girls might have opposite effects, especially for the HC and the PHC. Our findings on sex-specific and sexual maturity-related volumes may be useful in better understanding the MTL developmental differences and related learning, memory, and emotion differences between boys and girls during puberty.

Sparse patch-based label propagation for accurate prostate localization in CT images

In this paper, we propose a new prostate computed tomography (CT) segmentation method for image guided radiation therapy. The main contributions of our method lie in the following aspects. 1) Instead of using voxel intensity information alone, patch-based representation in the discriminative feature space with logistic sparse LASSO is used as anatomical signature to deal with low contrast problem in prostate CT images. 2) Based on the proposed patch-based signature, a new multi-atlases label fusion method formulated under sparse representation framework is designed to segment prostate in the new treatment images, with guidance from the previous segmented images of the same patient. This method estimates the prostate likelihood of each voxel in the new treatment image from its nearby candidate voxels in the previous segmented images, based on the nonlocal mean principle and sparsity constraint. 3) A hierarchical labeling strategy is further designed to perform label fusion, where voxels with high confidence are first labeled for providing useful context information in the same image for aiding the labeling of the remaining voxels. 4) An online update mechanism is finally adopted to progressively collect more patient-specific information from newly segmented treatment images of the same patient, for adaptive and more accurate segmentation. The proposed method has been extensively evaluated on a prostate CT image database consisting of 24 patients where each patient has more than 10 treatment images, and further compared with several state-of-the-art prostate CT segmentation algorithms using various evaluation metrics. Experimental results demonstrate that the proposed method consistently achieves higher segmentation accuracy than any other methods under comparison.

Non-local means resolution enhancement of lung 4D-CT data

Image resolution in 4D-CT is a crucial bottleneck that needs to be overcome for improved dose planning in radiotherapy for lung cancer. In this paper, we propose a novel patch-based algorithm to enhance the image quality of 4D-CT data. Our premise is that anatomical information missing in one phase can be recovered from complementary information embedded in other phases. We employ a patch-based mechanism to propagate information across phases for reconstruction of intermediate slices in the axial direction, where resolution is normally the lowest. Specifically, structurally-matching and spatially-nearby patches are combined for reconstruction of each patch. For greater sensitivity to anatomical nuances, we further employ a quad-tree technique to adaptively partition each slice of the image in each phase for more fine-grained refinement. Our evaluation based on a public 4D-CT lung data indicates that our algorithm gives very promising results with significantly enhanced image structures.

Towards automated detection of depression from brain structural magnetic resonance images

INTRODUCTION

Depression is a major issue worldwide and is seen as a significant health problem. Stigma and patient denial, clinical experience, time limitations, and reliability of psychometrics are barriers to the clinical diagnoses of depression. Thus, the establishment of an automated system that could detect such abnormalities would assist medical experts in their decision-making process. This paper reviews existing methods for the automated detection of depression from brain structural magnetic resonance images (sMRI).

METHODS

Relevant sources were identified from various databases and online sites using a combination of keywords and terms including depression, major depressive disorder, detection, classification, and MRI databases. Reference lists of chosen articles were further reviewed for associated publications.

RESULTS

The paper introduces a generic structure for representing and describing the methods developed for the detection of depression from sMRI of the brain. It consists of a number of components including acquisition and preprocessing, feature extraction, feature selection, and classification.

CONCLUSION

Automated sMRI-based detection methods have the potential to provide an objective measure of depression, hence improving the confidence level in the diagnosis and prognosis of depression.

Representation learning: a unified deep learning framework for automatic prostate MR segmentation

Image representation plays an important role in medical image analysis. The key to the success of different medical image analysis algorithms is heavily dependent on how we represent the input data, namely features used to characterize the input image. In the literature, feature engineering remains as an active research topic, and many novel hand-crafted features are designed such as Haar wavelet, histogram of oriented gradient, and local binary patterns. However, such features are not designed with the guidance of the underlying dataset at hand. To this end, we argue that the most effective features should be designed in a learning based manner, namely representation learning, which can be adapted to different patient datasets at hand. In this paper, we introduce a deep learning framework to achieve this goal. Specifically, a stacked independent subspace analysis (ISA) network is adopted to learn the most effective features in a hierarchical and unsupervised manner. The learnt features are adapted to the dataset at hand and encode high level semantic anatomical information. The proposed method is evaluated on the application of automatic prostate MR segmentation. Experimental results show that significant segmentation accuracy improvement can be achieved by the proposed deep learning method compared to other state-of-the-art segmentation approaches.

Automatic prostate MR image segmentation with sparse label propagation and domain-specific manifold regularization

Automatic prostate segmentation in MR images plays an important role in prostate cancer diagnosis. However, there are two main challenges: (1) Large inter-subject prostate shape variations; (2) Inhomogeneous prostate appearance. To address these challenges, we propose a new hierarchical prostate MR segmentation method, with the main contributions lying in the following aspects: First, the most salient features are learnt from atlases based on a subclass discriminant analysis (SDA) method, which aims to find a discriminant feature subspace by simultaneously maximizing the inter-class distance and minimizing the intra-class variations. The projected features, instead of only voxel-wise intensity, will be served as anatomical signature of each voxel. Second, based on the projected features, a new multi-atlases sparse label fusion framework is proposed to estimate the prostate likelihood of each voxel in the target image from the coarse level. Third, a domain-specific semi-supervised manifold regularization method is proposed to incorporate the most reliable patient-specific information identified by the prostate likelihood map to refine the segmentation result from the fine level. Our method is evaluated on a T2 weighted prostate MR image dataset consisting of 66 patients and compared with two state-of-the-art segmentation methods. Experimental results show that our method consistently achieves the highest segmentation accuracies than other methods under comparison.

Automated segmentation of CBCT image using spiral CT atlases and convex optimization

Cone-beam computed tomography (CBCT) is an increasingly utilized imaging modality for the diagnosis and treatment planning of the patients with craniomaxillofacial (CMF) deformities. CBCT scans have relatively low cost and low radiation dose in comparison to conventional spiral CT scans. However, a major limitation of CBCT scans is the widespread image artifacts such as noise, beam hardening and inhomogeneity, causing great difficulties for accurate segmentation of bony structures from soft tissues, as well as separating mandible from maxilla. In this paper, we presented a novel fully automated method for CBCT image segmentation. In this method, we first estimated a patient-specific atlas using a sparse label fusion strategy from predefined spiral CT atlases. This patient-specific atlas was then integrated into a convex segmentation framework based on maximum a posteriori probability for accurate segmentation. Finally, the performance of our method was validated via comparisons with manual ground-truth segmentations.

Is synthesizing MRI contrast useful for inter-modality analysis?

Availability of multi-modal magnetic resonance imaging (MRI) databases opens up the opportunity to synthesize different MRI contrasts without actually acquiring the images. In theory such synthetic images have the potential to reduce the amount of acquisitions to perform certain analyses. However, to what extent they can substitute real acquisitions in the respective analyses is an open question. In this study, we used a synthesis method based on patch matching to test whether synthetic images can be useful in segmentation and inter-modality cross-subject registration of brain MRI. Thirty-nine T1 scans with 36 manually labeled structures of interest were used in the registration and segmentation of eight proton density (PD) scans, for which ground truth T1 data were also available. The results show that synthesized T1 contrast can considerably enhance the quality of non-linear registration compared with using the original PD data, and it is only marginally worse than using the original T1 scans. In segmentation, the relative improvement with respect to using the PD is smaller, but still statistically significant.

Contour-driven regression for label inference in atlas-based segmentation

We present a novel method for inferring tissue labels in atlas-based image segmentation using Gaussian process regression. Atlas-based segmentation results in probabilistic label maps that serve as input to our method. We introduce a contour-driven prior distribution over label maps to incorporate image features of the input scan into the label inference problem. The mean function of the Gaussian process posterior distribution yields the MAP estimate of the label map and is used in the subsequent voting. We demonstrate improved segmentation accuracy when our approach is combined with two different patch-based segmentation techniques. We focus on the segmentation of parotid glands in CT scans of patients with head and neck cancer, which is important for radiation therapy planning.

Minimizing joint risk of mislabeling for iterative Patch-based label fusion

Automated labeling of anatomical structures in medical images is very important in many neuroscience studies. Recently, patch-based labeling in the non-local manner has been widely investigated to alleviate the possible misalignment when registering atlases to the target image. However, the weights used for label fusion from the registered atlases in conventional methods are generally computed independently and thus lack the capability of preventing the ambiguous atlas patches from contributing to the label fusion. More critically, these weights are often calculated based only on the simple patch similarity, thus not necessarily providing optimal solution for label fusion. To address these issues, we present a novel patch-based label fusion method in multi-atlas scenario, for the goal of labeling each voxel in the target image by the best representative atlas patches that also have the lowest joint risk of mislabeling. Specifically, sparse coding is used to select a small number of atlas patches which best represent the underlying patch at each point of the target image, thus minimizing the chance of including the misleading atlas patches for labeling. Furthermore, we examine the joint risk of any pair of atlas patches in making similar labeling error, by analyzing the correlation of their morphological error patterns and also the labeling consensus among atlases. This joint risk will be further recursively updated based on the latest labeling results to correct the possible labeling errors. To demonstrate the performance of our proposed method, we have evaluated it on both whole brain parcellation and hippocampus segmentation, and achieved promising labeling results, compared with the state-of-the-art methods.

Robust GM/WM segmentation of the spinal cord with iterative non-local statistical fusion

New magnetic resonance imaging (MRI) sequences are enabling clinical study of the in vivo spinal cord's internal structure. Yet, low contrast-to-noise ratio, artifacts, and imaging distortions have limited the applicability of tissue segmentation techniques pioneered elsewhere in the central nervous system. Recently, methods have been presented for cord/non-cord segmentation on MRI and the feasibility of gray matter/white matter tissue segmentation has been evaluated. To date, no automated algorithms have been presented. Herein, we present a non-local multi-atlas framework that robustly identifies the spinal cord and segments its internal structure with submillimetric accuracy. The proposed algorithm couples non-local fusion with a large number of slice-based atlases (as opposed to typical volumetric ones). To improve performance, the fusion process is interwoven with registration so that segmentation information guides registration and vice versa. We demonstrate statistically significant improvement over state-of-the-art benchmarks in a study of 67 patients. The primary contributions of this work are (1) innovation in non-volumetric atlas information, (2) advancement of label fusion theory to include iterative registration/segmentation, and (3) the first fully automated segmentation algorithm for spinal cord internal structure on MRI.

Multi-atlas segmentation without registration: a supervoxel-based approach

Multi-atlas segmentation is a powerful segmentation technique. It has two components: label transfer that transfers segmentation labels from prelabeled atlases to a novel image and label fusion that combines the label transfer results. For reliable label transfer, most methods assume that the structure of interest to be segmented have localized spatial support across different subjects. Although the technique has been successful for many applications, the strong assumption also limits its applicability. For example, multi-atlas segmentation has not been applied for tumor segmentation because it is difficult to derive reliable label transfer for such applications due to the substantial variation in tumor locations. To address this limitation, we propose a label transfer technique for multi-atlas segmentation. Inspired by the Superparsing work [13], we approach this problem in two steps. Our method first oversegments images into homogeneous regions, called supervoxels. For a voxel in a novel image, to find its correspondence in atlases for label transfer, we first locate supervoxels in atlases that are most similar to the supervoxel the target voxel belongs to. Then, voxel-wise correspondence is found through searching for voxels that have most similar patches to the target voxel within the selected atlas supervoxels. We apply this technique for brain tumor segmentation and show promising results.

Multi-atlas segmentation with joint label fusion and corrective learning-an open source implementation

Label fusion based multi-atlas segmentation has proven to be one of the most competitive techniques for medical image segmentation. This technique transfers segmentations from expert-labeled images, called atlases, to a novel image using deformable image registration. Errors produced by label transfer are further reduced by label fusion that combines the results produced by all atlases into a consensus solution. Among the proposed label fusion strategies, weighted voting with spatially varying weight distributions derived from atlas-target intensity similarity is a simple and highly effective label fusion technique. However, one limitation of most weighted voting methods is that the weights are computed independently for each atlas, without taking into account the fact that different atlases may produce similar label errors. To address this problem, we recently developed the joint label fusion technique and the corrective learning technique, which won the first place of the 2012 MICCAI Multi-Atlas Labeling Challenge and was one of the top performers in 2013 MICCAI Segmentation: Algorithms, Theory and Applications (SATA) challenge. To make our techniques more accessible to the scientific research community, we describe an Insight-Toolkit based open source implementation of our label fusion methods. Our implementation extends our methods to work with multi-modality imaging data and is more suitable for segmentation problems with multiple labels. We demonstrate the usage of our tools through applying them to the 2012 MICCAI Multi-Atlas Labeling Challenge brain image dataset and the 2013 SATA challenge canine leg image dataset. We report the best results on these two datasets so far.

Multi-atlas segmentation with robust label transfer and label fusion

Multi-atlas segmentation has been widely applied in medical image analysis. This technique relies on image registration to transfer segmentation labels from pre-labeled atlases to a novel target image and applies label fusion to reduce errors produced by registration-based label transfer. To improve the performance of registration-based label transfer against registration errors, our first contribution is to propose a label transfer scheme that generates multiple warped versions of each atlas to one target image through registration paths obtained by composing inter-atlas registrations and atlas-target registrations. The problem of decreasing quality of warped atlases caused by accumulative errors in composing multiple registrations is properly addressed by an atlas selection method that is guided by atlas segmentations. To improve the performance of label fusion against registration errors, our second contribution is to integrate the probabilistic correspondence model employed by the non-local mean approach with the joint label fusion technique, both of which have shown excellent performance for label fusion. Experiments on mitral-valve segmentation in 3D transesophageal echocardiography (TEE) show the effectiveness of the proposed techniques.

Atlas encoding by randomized forests for efficient label propagation

We propose a method for multi-atlas label propagation based on encoding the individual atlases by randomized classification forests. Most current approaches perform a non-linear registration between all atlases and the target image, followed by a sophisticated fusion scheme. While these approaches can achieve high accuracy, in general they do so at high computational cost. This negatively affects the scalability to large databases and experimentation. To tackle this issue, we propose to use a small and deep classification forest to encode each atlas individually in reference to an aligned probabilistic atlas, resulting in an Atlas Forest (AF). At test time, each AF yields a probabilistic label estimate, and fusion is done by averaging. Our scheme performs only one registration per target image, achieves good results with a simple fusion scheme, and allows for efficient experimentation. In contrast to standard forest schemes, incorporation of new scans is possible without retraining, and target-specific selection of atlases remains possible. The evaluation on three different databases shows accuracy at the level of the state of the art, at a significantly lower runtime.

Non-local statistical label fusion for multi-atlas segmentation

Multi-atlas segmentation provides a general purpose, fully-automated approach for transferring spatial information from an existing dataset ("atlases") to a previously unseen context ("target") through image registration. The method to resolve voxelwise label conflicts between the registered atlases ("label fusion") has a substantial impact on segmentation quality. Ideally, statistical fusion algorithms (e.g., STAPLE) would result in accurate segmentations as they provide a framework to elegantly integrate models of rater performance. The accuracy of statistical fusion hinges upon accurately modeling the underlying process of how raters err. Despite success on human raters, current approaches inaccurately model multi-atlas behavior as they fail to seamlessly incorporate exogenous intensity information into the estimation process. As a result, locally weighted voting algorithms represent the de facto standard fusion approach in clinical applications. Moreover, regardless of the approach, fusion algorithms are generally dependent upon large atlas sets and highly accurate registration as they implicitly assume that the registered atlases form a collectively unbiased representation of the target. Herein, we propose a novel statistical fusion algorithm, Non-Local STAPLE (NLS). NLS reformulates the STAPLE framework from a non-local means perspective in order to learn what label an atlas would have observed, given perfect correspondence. Through this reformulation, NLS (1) seamlessly integrates intensity into the estimation process, (2) provides a theoretically consistent model of multi-atlas observation error, and (3) largely diminishes the need for large atlas sets and very high-quality registrations. We assess the sensitivity and optimality of the approach and demonstrate significant improvement in two empirical multi-atlas experiments.

Scoring by nonlocal image patch estimator for early detection of Alzheimer's disease

Detection of Alzheimer's disease (AD) at the first stages of the pathology is an important task to accelerate the development of new therapies and improve treatment. Compared to AD detection, the prediction of AD using structural MRI at the mild cognitive impairment (MCI) or pre-MCI stage is more complex because the associated anatomical changes are more subtle. In this study, we analyzed the capability of a recently proposed method, SNIPE (Scoring by Nonlocal Image Patch Estimator), to predict AD by analyzing entorhinal cortex (EC) and hippocampus (HC) scoring over the entire ADNI database (834 scans). Detection (AD vs. CN) and prediction (progressive — pMCI vs. stable — sMCI) efficiency of SNIPE were studied using volumetric and grading biomarkers. First, our results indicate that grading-based biomarkers are more relevant for prediction than volume-based biomarkers. Second, we show that HC-based biomarkers are more important than EC-based biomarkers for prediction. Third, we demonstrate that the results obtained by SNIPE are similar to or better than results obtained in an independent study using HC volume, cortical thickness, and tensor-based morphometry, individually and in combination. Fourth, a comparison of new patch-based methods shows that the nonlocal redundancy strategy involved in SNIPE obtained similar results to a new local sparse-based approach. Finally, we present the first results of patch-based morphometry to illustrate the progression of the pathology.

Nonlocal regularization for active appearance model: Application to medial temporal lobe segmentation

The human medial temporal lobe (MTL) is an important part of the limbic system, and its substructures play key roles in learning, memory, and neurodegeneration. The MTL includes the hippocampus (HC), amygdala (AG), parahippocampal cortex (PHC), entorhinal cortex, and perirhinal cortex--structures that are complex in shape and have low between-structure intensity contrast, making them difficult to segment manually in magnetic resonance images. This article presents a new segmentation method that combines active appearance modeling and patch-based local refinement to automatically segment specific substructures of the MTL including HC, AG, PHC, and entorhinal/perirhinal cortex from MRI data. Appearance modeling, relying on eigen-decomposition to analyze statistical variations in image intensity and shape information in study population, is used to capture global shape characteristics of each structure of interest with a generative model. Patch-based local refinement, using nonlocal means to compare the image local intensity properties, is applied to locally refine the segmentation results along the structure borders to improve structure delimitation. In this manner, nonlocal regularization and global shape constraints could allow more accurate segmentations of structures. Validation experiments against manually defined labels demonstrate that this new segmentation method is computationally efficient, robust, and accurate. In a leave-one-out validation on 54 normal young adults, the method yielded a mean Dice κ of 0.87 for the HC, 0.81 for the AG, 0.73 for the anterior parts of the parahippocampal gyrus (entorhinal and perirhinal cortex), and 0.73 for the posterior parahippocampal gyrus.

Delayed cortical development in fetuses with complex congenital heart disease

Neurologic impairment is a major complication of complex congenital heart disease (CHD). A growing body of evidence suggests that neurologic dysfunction may be present in a significant proportion of this high-risk population in the early newborn period prior to surgical interventions. We recently provided the first evidence that brain growth impairment in fetuses with complex CHD has its origins in utero. Here, we extend these observations by characterizing global and regional brain development in fetuses with hypoplastic left heart syndrome (HLHS), one of the most severe forms of CHD. Using advanced magnetic resonance imaging techniques, we compared in vivo brain growth in 18 fetuses with HLHS and 30 control fetuses from 25.4-37.0 weeks of gestation. Our findings demonstrate a progressive third trimester fall-off in cortical gray and white matter volumes (P < 0.001), and subcortical gray matter (P < 0.05) in fetuses with HLHS. Significant delays in cortical gyrification were also evident in HLHS fetuses (P < 0.001). In the HLHS fetus, local cortical folding delays were detected as early as 25 weeks in the frontal, parietal, calcarine, temporal, and collateral regions and appear to precede volumetric brain growth disturbances, which may be an early marker of elevated risk for third trimester brain growth failure.

A multi-modal approach to computer-assisted deep brain stimulation trajectory planning

PURPOSE

Both frame-based and frameless approaches to deep brain stimulation (DBS) require planning of insertion trajectories that mitigate hemorrhagic risk and loss of neurological function. Currently, this is done by manual inspection of multiple potential electrode trajectories on MR-imaging data. We propose and validate a method for computer-assisted DBS trajectory planning.

METHOD

Our framework integrates multi-modal MRI analysis (T1w, SWI, TOF-MRA) to compute suitable DBS trajectories that optimize the avoidance of specific critical brain structures. A cylinder model is used to process each trajectory and to evaluate complex surgical constraints described via a combination of binary and fuzzy segmented datasets. The framework automatically aggregates the multiple constraints into a unique ranking of recommended low-risk trajectories. Candidate trajectories are represented as a few well-defined cortical entry patches of best-ranked trajectories and presented to the neurosurgeon for final trajectory selection.

RESULTS

The proposed algorithm permits a search space containing over 8,000 possible trajectories to be processed in less than 20 s. A retrospective analysis on 14 DBS cases of patients with severe Parkinson's disease reveals that our framework can improve the simultaneous optimization of many pre-formulated surgical constraints. Furthermore, all automatically computed trajectories were evaluated by two neurosurgeons, were judged suitable for surgery and, in many cases, were judged preferable or equivalent to the manually planned trajectories used during the operation.

CONCLUSIONS

This work provides neurosurgeons with an intuitive and flexible decision-support system that allows objective and patient-specific optimization of DBS lead trajectories, which should improve insertion safety and reduce surgical time.

Automated segmentation of basal ganglia and deep brain structures in MRI of Parkinson's disease

PURPOSE

Template-based segmentation techniques have been developed to facilitate the accurate targeting of deep brain structures in patients with movement disorders. Three template-based brain MRI segmentation techniques were compared to determine the best strategy for segmenting the deep brain structures of patients with Parkinson's disease.

METHODS

T1-weighted and T2-weighted magnetic resonance (MR) image templates were created by averaging MR images of 57 patients with Parkinson's disease. Twenty-four deep brain structures were manually segmented on the templates. To validate the template-based segmentation, 14 of the 24 deep brain structures from the templates were manually segmented on 10 MR scans of Parkinson's patients as a gold standard. We compared the manual segmentations with three methods of automated segmentation: two registration-based approaches, automatic nonlinear image matching and anatomical labeling (ANIMAL) and symmetric image normalization (SyN), and one patch-label fusion technique. The automated labels were then compared with the manual labels using a Dice-kappa metric and center of gravity. A Friedman test was used to compare the Dice-kappa values and paired t tests for the center of gravity.

RESULTS

The Friedman test showed a significant difference between the three methods for both thalami (p < 0.05) and not for the subthalamic nuclei. Registration with ANIMAL was better than with SyN for the left thalamus and was better than the patch-based method for the right thalamus.

CONCLUSION

Although template-based approaches are the most used techniques to segment basal ganglia by warping onto MR images, we found that the patch-based method provided similar results and was less time-consuming. Patch-based method may be preferable for the subthalamic nucleus segmentation in patients with Parkinson's disease.