Optimal weights for local multi-atlas fusion using supervised learning and dynamic information (SuperDyn): Validation on hippocampus segmentation

Simultaneous image reconstruction and lesion segmentation in accelerated MRI using multitasking learning

PURPOSE

Magnetic resonance imaging (MRI) serves as an important medical imaging modality for a variety of clinical applications. However, the problem of long imaging time limited its wide usage. In addition, prolonged scan time will cause discomfort to the patient, leading to severe image artifacts. On the other hand, manually lesion segmentation is time consuming. Algorithm-based automatic lesion segmentation is still challenging, especially for accelerated imaging with low quality.

METHODS

In this paper, we proposed a multitask learning-based method to perform image reconstruction and lesion segmentation simultaneously, called "RecSeg". Our hypothesis is that both tasks can benefit from the usage of the proposed combined model. In the experiment, we validated the proposed multitask model on MR k-space data with different acceleration factors (2×, 4×, and 6×). Two connected U-nets were used for the tasks of liver and renal image reconstruction and segmentation. A total of 50 healthy subjects and 100 patients with hepatocellular carcinoma were included for training and testing. For the segmentation part, we use healthy subjects to verify organ segmentation, and hepatocellular carcinoma patients to verify lesion segmentation. The organs and lesions were manually contoured by an experienced radiologist.

RESULTS

Experimental results show that the proposed RecSeg yielded the highest PSNR (RecSeg: 32.39 ± 1.64 vs. KSVD: 29.53 ± 2.74 and single U-net: 31.18 ± 1.68, respectively, p < 0.05) and highest structural similarity index measure (SSIM) (RecSeg: 0.93 ± 0.01 vs. KSVD: 0.88 ± 0.02 and single U-net: 0.90 ± 0.01, respectively, p < 0.05) under 6× acceleration. Moreover, in the task of lesion segmentation, it is proposed that RecSeg produced the highest Dice score (RecSeg: 0.86 ± 0.01 vs. KSVD: 0.82 ± 0.01 and single U-net: 0.84 ± 0.01, respectively, p < 0.05).

CONCLUSIONS

This study focused on the simultaneous reconstruction of medical images and the segmentation of organs and lesions. It is observed that the multitask learning-based method can improve performances of both image reconstruction and lesion segmentation.

Pixel-Wise Classification in Hippocampus Histological Images

This paper presents a method for pixel-wise classification applied for the first time on hippocampus histological images. The goal is achieved by representing pixels in a 14-D vector, composed of grey-level information and moment invariants. Then, several popular machine learning models are used to categorize them, and multiple metrics are computed to evaluate the performance of the different models. The multilayer perceptron, random forest, support vector machine, and radial basis function networks were compared, achieving the multilayer perceptron model the highest result on accuracy metric, AUC, and F ₁ score with highly satisfactory results for substituting a manual classification task, due to an expert opinion in the hippocampus histological images.

Multi-Template Mesiotemporal Lobe Segmentation: Effects of Surface and Volume Feature Modeling

Numerous neurological disorders are associated with atrophy of mesiotemporal lobe structures, including the hippocampus (HP), amygdala (AM), and entorhinal cortex (EC). Accurate segmentation of these structures is, therefore, necessary for understanding the disease process and patient management. Recent multiple-template segmentation algorithms have shown excellent performance in HP segmentation. Purely surface-based methods precisely describe structural boundary but their performance likely depends on a large template library, as segmentation suffers when the boundaries of template and individual MRI are not well aligned while volume-based methods are less dependent. So far only few algorithms attempted segmentation of entire mesiotemporal structures including the parahippocampus. We compared performance of surface- and volume-based approaches in segmenting the three mesiotemporal structures and assess the effects of different environments (i.e., size of templates, under pathology). We also proposed an algorithm that combined surface- with volume-derived similarity measures for optimal template selection. To further improve the method, we introduced two new modules: (1) a non-linear registration that is driven by volume-based intensities and features sampled on deformable template surfaces; (2) a shape averaging based on regional weighting using multi-scale global-to-local icosahedron sampling. Compared to manual segmentations, our approach, namely HybridMulti showed high accuracy in 40 healthy controls (mean Dice index for HP/AM/EC = 89.7/89.3/82.9%) and 135 patients with temporal lobe epilepsy (88.7/89.0/82.6%). This accuracy was comparable across two different datasets of 1.5T and 3T MRI. It resulted in the best performance among tested multi-template methods that were either based on volume or surface data alone in terms of accuracy and sensitivity to detect atrophy related to epilepsy. Moreover, unlike purely surface-based multi-template segmentation, HybridMulti could maintain accurate performance even with a 50% template library size.

Multidimensional co-segmentation of longitudinal brain MRI ensembles in the presence of a neurodegenerative process

MRI Segmentation of a pathological brain poses a significant challenge, as the available anatomical priors that provide top-down information to aid segmentation are inadequate in the presence of abnormalities. This problem is further complicated for longitudinal data capturing impaired brain development or neurodegenerative conditions, since the dynamic of brain atrophies has to be considered as well. For these cases, the absence of compatible annotated training examples renders the commonly used multi-atlas or machine-learning approaches impractical. We present a novel segmentation approach that accounts for the lack of labeled data via multi-region multi-subject co-segmentation (MMCoSeg) of longitudinal MRI sequences. The underlying, unknown anatomy is learned throughout an iterative process, in which the segmentation of a region is supported both by the segmentation of the neighboring regions, which share common boundaries, and by the segmentation of corresponding regions, in the other jointly segmented images. A 4D multi-region atlas that models the spatio-temporal deformations and can be adapted to different subjects undergoing similar degeneration processes is reconstructed concurrently. An inducible mouse model of p25 accumulation (the CK-p25 mouse) that displays key pathological hallmarks of Alzheimer disease (AD) is used as a gold-standard to test the proposed algorithm by providing a conditional control of rapid neurodegeneration. Applying the MMCoSeg to a cohort of CK-p25 mice and littermate controls yields promising segmentation results that demonstrate high compatibility with expertise manual annotations. An extensive comparative analysis with respect to current well-established, atlas-based segmentation methods highlights the advantage of the proposed approach, which provides accurate segmentation of longitudinal brain MRIs in pathological conditions, where only very few annotated examples are available.

The bumps under the hippocampus

Shown in every neuroanatomy textbook, a key morphological feature is the bumpy ridges, which we refer to as hippocampal dentation, on the inferior aspect of the hippocampus. Like the folding of the cerebral cortex, hippocampal dentation allows for greater surface area in a confined space. However, examining numerous approaches to hippocampal segmentation and morphology analysis, virtually all published 3D renderings of the hippocampus show the inferior surface to be quite smooth or mildly irregular; we have rarely seen the characteristic bumpy structure on reconstructed 3D surfaces. The only exception is a 9.4T postmortem study (Yushkevich et al. [2009]: NeuroImage 44:385-398). An apparent question is, does this indicate that this specific morphological signature can only be captured using ultra high-resolution techniques? Or, is such information buried in the data we commonly acquire, awaiting a computation technique that can extract and render it clearly? In this study, we propose an automatic and robust super-resolution technique that captures the fine scale morphometric features of the hippocampus based on common 3T MR images. The method is validated on 9.4T ultra-high field images and then applied on 3T data sets. This method opens possibilities of future research on the hippocampus and other sub-cortical structural morphometry correlating the degree of dentation with a range of diseases including epilepsy, Alzheimer's disease, and schizophrenia. Hum Brain Mapp 39:472-490, 2018. © 2017 Wiley Periodicals, Inc.

Hippocampus Segmentation Based on Local Linear Mapping

We propose local linear mapping (LLM), a novel fusion framework for distance field (DF) to perform automatic hippocampus segmentation. A k-means cluster method is propose for constructing magnetic resonance (MR) and DF dictionaries. In LLM, we assume that the MR and DF samples are located on two nonlinear manifolds and the mapping from the MR manifold to the DF manifold is differentiable and locally linear. We combine the MR dictionary using local linear representation to present the test sample, and combine the DF dictionary using the corresponding coefficients derived from local linear representation procedure to predict the DF of the test sample. We then merge the overlapped predicted DF patch to obtain the DF value of each point in the test image via a confidence-based weighted average method. This approach enabled us to estimate the label of the test image according to the predicted DF. The proposed method was evaluated on brain images of 35 subjects obtained from SATA dataset. Results indicate the effectiveness of the proposed method, which yields mean Dice similarity coefficients of 0.8697, 0.8770 and 0.8734 for the left, right and bi-lateral hippocampus, respectively.

3D-SIFT-Flow for atlas-based CT liver image segmentation

PURPOSE

In this paper, the authors proposed a new 3D registration algorithm, 3D-scale invariant feature transform (SIFT)-Flow, for multiatlas-based liver segmentation in computed tomography (CT) images.

METHODS

In the registration work, the authors developed a new registration method that takes advantage of dense correspondence using the informative and robust SIFT feature. The authors computed the dense SIFT features for the source image and the target image and designed an objective function to obtain the correspondence between these two images. Labeling of the source image was then mapped to the target image according to the former correspondence, resulting in accurate segmentation. In the fusion work, the 2D-based nonparametric label transfer method was extended to 3D for fusing the registered 3D atlases.

RESULTS

Compared with existing registration algorithms, 3D-SIFT-Flow has its particular advantage in matching anatomical structures (such as the liver) that observe large variation/deformation. The authors observed consistent improvement over widely adopted state-of-the-art registration methods such as ELASTIX, ANTS, and multiatlas fusion methods such as joint label fusion. Experimental results of liver segmentation on the MICCAI 2007 Grand Challenge are encouraging, e.g., Dice overlap ratio 96.27% ± 0.96% by our method compared with the previous state-of-the-art result of 94.90% ± 2.86%.

CONCLUSIONS

Experimental results show that 3D-SIFT-Flow is robust for segmenting the liver from CT images, which has large tissue deformation and blurry boundary, and 3D label transfer is effective and efficient for improving the registration accuracy.

Atlas-based rib-bone detection in chest X-rays

This paper investigates using rib-bone atlases for automatic detection of rib-bones in chest X-rays (CXRs). We built a system that takes patient X-ray and model atlases as input and automatically computes the posterior rib borders with high accuracy and efficiency. In addition to conventional atlas, we propose two alternative atlases: (i) automatically computed rib bone models using Computed Tomography (CT) scans, and (ii) dual energy CXRs. We test the proposed approach with each model on 25 CXRs from the Japanese Society of Radiological Technology (JSRT) dataset and another 25 CXRs from the National Library of Medicine CXR dataset. We achieve an area under the ROC curve (AUC) of about 95% for Montgomery and 91% for JSRT datasets. Using the optimal operating point of the ROC curve, we achieve a segmentation accuracy of 88.91±1.8% for Montgomery and 85.48±3.3% for JSRT datasets. Our method produces comparable results with the state-of-the-art algorithms. The performance of our method is also excellent on challenging X-rays as it successfully addressed the rib-shape variance between patients and number of visible rib-bones due to patient respiration.

MUSE: MUlti-atlas region Segmentation utilizing Ensembles of registration algorithms and parameters, and locally optimal atlas selection

Atlas-based automated anatomical labeling is a fundamental tool in medical image segmentation, as it defines regions of interest for subsequent analysis of structural and functional image data. The extensive investigation of multi-atlas warping and fusion techniques over the past 5 or more years has clearly demonstrated the advantages of consensus-based segmentation. However, the common approach is to use multiple atlases with a single registration method and parameter set, which is not necessarily optimal for every individual scan, anatomical region, and problem/data-type. Different registration criteria and parameter sets yield different solutions, each providing complementary information. Herein, we present a consensus labeling framework that generates a broad ensemble of labeled atlases in target image space via the use of several warping algorithms, regularization parameters, and atlases. The label fusion integrates two complementary sources of information: a local similarity ranking to select locally optimal atlases and a boundary modulation term to refine the segmentation consistently with the target image's intensity profile. The ensemble approach consistently outperforms segmentations using individual warping methods alone, achieving high accuracy on several benchmark datasets. The MUSE methodology has been used for processing thousands of scans from various datasets, producing robust and consistent results. MUSE is publicly available both as a downloadable software package, and as an application that can be run on the CBICA Image Processing Portal (https://ipp.cbica.upenn.edu), a web based platform for remote processing of medical images.

Automatic segmentation of the hippocampus for preterm neonates from early-in-life to term-equivalent age

Introduction

The hippocampus, a medial temporal lobe structure central to learning and memory, is particularly vulnerable in preterm-born neonates. To date, segmentation of the hippocampus for preterm-born neonates has not yet been performed early-in-life (shortly after birth when clinically stable). The present study focuses on the development and validation of an automatic segmentation protocol that is based on the MAGeT-Brain (Multiple Automatically Generated Templates) algorithm to delineate the hippocampi of preterm neonates on their brain MRIs acquired at not only term-equivalent age but also early-in-life.

Methods

First, we present a three-step manual segmentation protocol to delineate the hippocampus for preterm neonates and apply this protocol on 22 early-in-life and 22 term images. These manual segmentations are considered the gold standard in assessing the automatic segmentations. MAGeT-Brain, automatic hippocampal segmentation pipeline, requires only a small number of input atlases and reduces the registration and resampling errors by employing an intermediate template library. We assess the segmentation accuracy of MAGeT-Brain in three validation studies, evaluate the hippocampal growth from early-in-life to term-equivalent age, and study the effect of preterm birth on the hippocampal volume. The first experiment thoroughly validates MAGeT-Brain segmentation in three sets of 10-fold Monte Carlo cross-validation (MCCV) analyses with 187 different groups of input atlases and templates. The second experiment segments the neonatal hippocampi on 168 early-in-life and 154 term images and evaluates the hippocampal growth rate of 125 infants from early-in-life to term-equivalent age. The third experiment analyzes the effect of gestational age (GA) at birth on the average hippocampal volume at early-in-life and term-equivalent age using linear regression.

Results

The final segmentations demonstrate that MAGeT-Brain consistently provides accurate segmentations in comparison to manually derived gold standards (mean Dice's Kappa > 0.79 and Euclidean distance <1.3 mm between centroids). Using this method, we demonstrate that the average volume of the hippocampus is significantly different (p < 0.0001) in early-in-life (621.8 mm³) and term-equivalent age (958.8 mm³). Using these differences, we generalize the hippocampal growth rate to 38.3 ± 11.7 mm³/week and 40.5 ± 12.9 mm³/week for the left and right hippocampi respectively. Not surprisingly, younger gestational age at birth is associated with smaller volumes of the hippocampi (p = 0.001).

Conclusions

MAGeT-Brain is capable of segmenting hippocampi accurately in preterm neonates, even at early-in-life. Hippocampal asymmetry with a larger right side is demonstrated on early-in-life images, suggesting that this phenomenon has its onset in the 3rd trimester of gestation. Hippocampal volume assessed at the time of early-in-life and term-equivalent age is linearly associated with GA at birth, whereby smaller volumes are associated with earlier birth.

•

We develop a MAGeT-Brain based automatic protocol to segment hippocampus in preterm neonates.

•

MAGeT-Brain can accurately segment hippocampus in preterm neonates, even at early-in-life.

•

Hippocampal asymmetry with a larger right side is demonstrated on early-in-life images.

•

Smaller hippocampal volumes are associated with earlier birth in preterm neonates.

An Optimized PatchMatch for multi-scale and multi-feature label fusion

Automatic segmentation methods are important tools for quantitative analysis of Magnetic Resonance Images (MRI). Recently, patch-based label fusion approaches have demonstrated state-of-the-art segmentation accuracy. In this paper, we introduce a new patch-based label fusion framework to perform segmentation of anatomical structures. The proposed approach uses an Optimized PAtchMatch Label fusion (OPAL) strategy that drastically reduces the computation time required for the search of similar patches. The reduced computation time of OPAL opens the way for new strategies and facilitates processing on large databases. In this paper, we investigate new perspectives offered by OPAL, by introducing a new multi-scale and multi-feature framework. During our validation on hippocampus segmentation we use two datasets: young adults in the ICBM cohort and elderly adults in the EADC-ADNI dataset. For both, OPAL is compared to state-of-the-art methods. Results show that OPAL obtained the highest median Dice coefficient (89.9% for ICBM and 90.1% for EADC-ADNI). Moreover, in both cases, OPAL produced a segmentation accuracy similar to inter-expert variability. On the EADC-ADNI dataset, we compare the hippocampal volumes obtained by manual and automatic segmentation. The volumes appear to be highly correlated that enables to perform more accurate separation of pathological populations.

Multi-atlas segmentation of biomedical images: A survey

Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing, et al. (2004), Klein, et al. (2005), and Heckemann, et al. (2006), is becoming one of the most widely-used and successful image segmentation techniques in biomedical applications. By manipulating and utilizing the entire dataset of "atlases" (training images that have been previously labeled, e.g., manually by an expert), rather than some model-based average representation, MAS has the flexibility to better capture anatomical variation, thus offering superior segmentation accuracy. This benefit, however, typically comes at a high computational cost. Recent advancements in computer hardware and image processing software have been instrumental in addressing this challenge and facilitated the wide adoption of MAS. Today, MAS has come a long way and the approach includes a wide array of sophisticated algorithms that employ ideas from machine learning, probabilistic modeling, optimization, and computer vision, among other fields. This paper presents a survey of published MAS algorithms and studies that have applied these methods to various biomedical problems. In writing this survey, we have three distinct aims. Our primary goal is to document how MAS was originally conceived, later evolved, and now relates to alternative methods. Second, this paper is intended to be a detailed reference of past research activity in MAS, which now spans over a decade (2003-2014) and entails novel methodological developments and application-specific solutions. Finally, our goal is to also present a perspective on the future of MAS, which, we believe, will be one of the dominant approaches in biomedical image segmentation.

Local manifold learning for multiatlas segmentation: application to hippocampal segmentation in healthy population and Alzheimer's disease

AIMS

Automated hippocampal segmentation is an important issue in many neuroscience studies.

METHODS

We presented and evaluated a novel segmentation method that utilized a manifold learning technique under the multiatlas-based segmentation scenario. A manifold representation of local patches for each voxel was achieved by applying an Isomap algorithm, which can then be used to obtain spatially local weights of atlases for label fusion. The obtained atlas weights potentially depended on all pairwise similarities of the population, which is in contrast to most existing label fusion methods that only rely on similarities between the target image and the atlases. The performance of the proposed method was evaluated for hippocampal segmentation and compared with two representative local weighted label fusion methods, that is, local majority voting and local weighted inverse distance voting, on an in-house dataset of 28 healthy adolescents (age range: 10-17 years) and two ADNI datasets of 100 participants (age range: 60-89 years). We also implemented hippocampal volumetric analysis and evaluated segmentation performance using atlases from a different dataset.

RESULTS

The median Dice similarities obtained by our proposed method were approximately 0.90 for healthy subjects and above 0.88 for two mixed diagnostic groups of ADNI subjects.

CONCLUSION

The experimental results demonstrated that the proposed method could obtain consistent and significant improvements over label fusion strategies that are implemented in the original space.

A logarithmic opinion pool based STAPLE algorithm for the fusion of segmentations with associated reliability weights

Pelvic floor dysfunction is common in women after childbirth and precise segmentation of magnetic resonance images (MRI) of the pelvic floor may facilitate diagnosis and treatment of patients. However, because of the complexity of its structures, manual segmentation of the pelvic floor is challenging and suffers from high inter and intra-rater variability of expert raters. Multiple template fusion algorithms are promising segmentation techniques for these types of applications, but they have been limited by imperfections in the alignment of templates to the target, and by template segmentation errors. A number of algorithms sought to improve segmentation performance by combining image intensities and template labels as two independent sources of information, carrying out fusion through local intensity weighted voting schemes. This class of approach is a form of linear opinion pooling, and achieves unsatisfactory performance for this application. We hypothesized that better decision fusion could be achieved by assessing the contribution of each template in comparison to a reference standard segmentation of the target image and developed a novel segmentation algorithm to enable automatic segmentation of MRI of the female pelvic floor. The algorithm achieves high performance by estimating and compensating for both imperfect registration of the templates to the target image and template segmentation inaccuracies. A local image similarity measure is used to infer a local reliability weight, which contributes to the fusion through a novel logarithmic opinion pooling. We evaluated our new algorithm in comparison to nine state-of-the-art segmentation methods and demonstrated our algorithm achieves the highest performance.

Gradient-based reliability maps for ACM-based segmentation of hippocampus

Automatic segmentation of deep brain structures, such as the hippocampus (HC), in MR images has attracted considerable scientific attention due to the widespread use of MRI and to the principal role of some structures in various mental disorders. In this literature, there exists a substantial amount of work relying on deformable models incorporating prior knowledge about structures' anatomy and shape information. However, shape priors capture global shape characteristics and thus fail to model boundaries of varying properties; HC boundaries present rich, poor, and missing gradient regions. On top of that, shape prior knowledge is blended with image information in the evolution process, through global weighting of the two terms, again neglecting the spatially varying boundary properties, causing segmentation faults. An innovative method is hereby presented that aims to achieve highly accurate HC segmentation in MR images, based on the modeling of boundary properties at each anatomical location and the inclusion of appropriate image information for each of those, within an active contour model framework. Hence, blending of image information and prior knowledge is based on a local weighting map, which mixes gradient information, regional and whole brain statistical information with a multi-atlas-based spatial distribution map of the structure's labels. Experimental results on three different datasets demonstrate the efficacy and accuracy of the proposed method.

High-Dimensional Medial Lobe Morphometry: An Automated MRI Biomarker for the New AD Diagnostic Criteria

Introduction. Medial temporal lobe atrophy assessment via magnetic resonance imaging (MRI) has been proposed in recent criteria as an in vivo diagnostic biomarker of Alzheimer's disease (AD). However, practical application of these criteria in a clinical setting will require automated MRI analysis techniques. To this end, we wished to validate our automated, high-dimensional morphometry technique to the hypothetical prediction of future clinical status from baseline data in a cohort of subjects in a large, multicentric setting, compared to currently known clinical status for these subjects. Materials and Methods. The study group consisted of 214 controls, 371 mild cognitive impairment (147 having progressed to probable AD and 224 stable), and 181 probable AD from the Alzheimer's Disease Neuroimaging Initiative, with data acquired on 58 different 1.5 T scanners. We measured the sensitivity and specificity of our technique in a hierarchical fashion, first testing the effect of intensity standardization, then between different volumes of interest, and finally its generalizability for a large, multicentric cohort. Results. We obtained 73.2% prediction accuracy with 79.5% sensitivity for the prediction of MCI progression to clinically probable AD. The positive predictive value was 81.6% for MCI progressing on average within 1.5 (0.3 s.d.) year. Conclusion. With high accuracy, the technique's ability to identify discriminant medial temporal lobe atrophy has been demonstrated in a large, multicentric environment. It is suitable as an aid for clinical diagnostic of AD.

Brain extraction based on locally linear representation-based classification

Brain extraction is an important procedure in brain image analysis. Although numerous brain extraction methods have been presented, enhancing brain extraction methods remains challenging because brain MRI images exhibit complex characteristics, such as anatomical variability and intensity differences across different sequences and scanners. To address this problem, we present a Locally Linear Representation-based Classification (LLRC) method for brain extraction. A novel classification framework is derived by introducing the locally linear representation to the classical classification model. Under this classification framework, a common label fusion approach can be considered as a special case and thoroughly interpreted. Locality is important to calculate fusion weights for LLRC; this factor is also considered to determine that Local Anchor Embedding is more applicable in solving locally linear coefficients compared with other linear representation approaches. Moreover, LLRC supplies a way to learn the optimal classification scores of the training samples in the dictionary to obtain accurate classification. The International Consortium for Brain Mapping and the Alzheimer's Disease Neuroimaging Initiative databases were used to build a training dataset containing 70 scans. To evaluate the proposed method, we used four publicly available datasets (IBSR1, IBSR2, LPBA40, and ADNI3T, with a total of 241 scans). Experimental results demonstrate that the proposed method outperforms the four common brain extraction methods (BET, BSE, GCUT, and ROBEX), and is comparable to the performance of BEaST, while being more accurate on some datasets compared with BEaST.

Multi-atlas segmentation of subcortical brain structures via the AutoSeg software pipeline

Automated segmenting and labeling of individual brain anatomical regions, in MRI are challenging, due to the issue of individual structural variability. Although atlas-based segmentation has shown its potential for both tissue and structure segmentation, due to the inherent natural variability as well as disease-related changes in MR appearance, a single atlas image is often inappropriate to represent the full population of datasets processed in a given neuroimaging study. As an alternative for the case of single atlas segmentation, the use of multiple atlases alongside label fusion techniques has been introduced using a set of individual “atlases” that encompasses the expected variability in the studied population. In our study, we proposed a multi-atlas segmentation scheme with a novel graph-based atlas selection technique. We first paired and co-registered all atlases and the subject MR scans. A directed graph with edge weights based on intensity and shape similarity between all MR scans is then computed. The set of neighboring templates is selected via clustering of the graph. Finally, weighted majority voting is employed to create the final segmentation over the selected atlases. This multi-atlas segmentation scheme is used to extend a single-atlas-based segmentation toolkit entitled AutoSeg, which is an open-source, extensible C++ based software pipeline employing BatchMake for its pipeline scripting, developed at the Neuro Image Research and Analysis Laboratories of the University of North Carolina at Chapel Hill. AutoSeg performs N4 intensity inhomogeneity correction, rigid registration to a common template space, automated brain tissue classification based skull-stripping, and the multi-atlas segmentation. The multi-atlas-based AutoSeg has been evaluated on subcortical structure segmentation with a testing dataset of 20 adult brain MRI scans and 15 atlas MRI scans. The AutoSeg achieved mean Dice coefficients of 81.73% for the subcortical structures.

Accurate and Fully Automatic Hippocampus Segmentation Using Subject-Specific 3D Optimal Local Maps Into a Hybrid Active Contour Model

Assessing the structural integrity of the hippocampus (HC) is an essential step toward prevention, diagnosis, and follow-up of various brain disorders due to the implication of the structural changes of the HC in those disorders. In this respect, the development of automatic segmentation methods that can accurately, reliably, and reproducibly segment the HC has attracted considerable attention over the past decades. This paper presents an innovative 3-D fully automatic method to be used on top of the multiatlas concept for the HC segmentation. The method is based on a subject-specific set of 3-D optimal local maps (OLMs) that locally control the influence of each energy term of a hybrid active contour model (ACM). The complete set of the OLMs for a set of training images is defined simultaneously via an optimization scheme. At the same time, the optimal ACM parameters are also calculated. Therefore, heuristic parameter fine-tuning is not required. Training OLMs are subsequently combined, by applying an extended multiatlas concept, to produce the OLMs that are anatomically more suitable to the test image. The proposed algorithm was tested on three different and publicly available data sets. Its accuracy was compared with that of state-of-the-art methods demonstrating the efficacy and robustness of the proposed method.

Automatic hippocampus segmentation of 7.0 Tesla MR images by combining multiple atlases and auto-context models

In many neuroscience and clinical studies, accurate measurement of hippocampus is very important to reveal the inter-subject anatomical differences or the subtle intra-subject longitudinal changes due to aging or dementia. Although many automatic segmentation methods have been developed, their performances are still challenged by the poor image contrast of hippocampus in the MR images acquired especially from 1.5 or 3.0 Tesla (T) scanners. With the recent advance of imaging technology, 7.0 T scanner provides much higher image contrast and resolution for hippocampus study. However, the previous methods developed for segmentation of hippocampus from 1.5 T or 3.0 T images do not work for the 7.0 T images, due to different levels of imaging contrast and texture information. In this paper, we present a learning-based algorithm for automatic segmentation of hippocampi from 7.0 T images, by taking advantages of the state-of-the-art multi-atlas framework and also the auto-context model (ACM). Specifically, ACM is performed in each atlas domain to iteratively construct sequences of location-adaptive classifiers by integrating both image appearance and local context features. Due to the plenty texture information in 7.0 T images, more advanced texture features are also extracted and incorporated into the ACM during the training stage. Then, under the multi-atlas segmentation framework, multiple sequences of ACM-based classifiers are trained for all atlases to incorporate the anatomical variability. In the application stage, for a new image, its hippocampus segmentation can be achieved by fusing the labeling results from all atlases, each of which is obtained by applying the atlas-specific ACM-based classifiers. Experimental results on twenty 7.0 T images with the voxel size of 0.35×0.35×0.35 mm3 show very promising hippocampus segmentations (in terms of Dice overlap ratio 89.1±0.020), indicating high applicability for the future clinical and neuroscience studies.

Local label learning (LLL) for subcortical structure segmentation: application to hippocampus segmentation

Automatic and reliable segmentation of subcortical structures is an important but difficult task in quantitative brain image analysis. Multi-atlas based segmentation methods have attracted great interest due to their promising performance. Under the multi-atlas based segmentation framework, using deformation fields generated for registering atlas images onto a target image to be segmented, labels of the atlases are first propagated to the target image space and then fused to get the target image segmentation based on a label fusion strategy. While many label fusion strategies have been developed, most of these methods adopt predefined weighting models that are not necessarily optimal. In this study, we propose a novel local label learning strategy to estimate the target image's segmentation label using statistical machine learning techniques. In particular, we use a L1-regularized support vector machine (SVM) with a k nearest neighbor (kNN) based training sample selection strategy to learn a classifier for each of the target image voxel from its neighboring voxels in the atlases based on both image intensity and texture features. Our method has produced segmentation results consistently better than state-of-the-art label fusion methods in validation experiments on hippocampal segmentation of over 100 MR images obtained from publicly available and in-house datasets. Volumetric analysis has also demonstrated the capability of our method in detecting hippocampal volume changes due to Alzheimer's disease.

Simultaneous truth and performance level estimation through fusion of probabilistic segmentations

Recent research has demonstrated that improved image segmentation can be achieved by multiple template fusion utilizing both label and intensity information. However, intensity weighted fusion approaches use local intensity similarity as a surrogate measure of local template quality for predicting target segmentation and do not seek to characterize template performance. This limits both the usefulness and accuracy of these techniques. Our work here was motivated by the observation that the local intensity similarity is a poor surrogate measure for direct comparison of the template image with the true image target segmentation. Although the true image target segmentation is not available, a high quality estimate can be inferred, and this in turn allows a principled estimate to be made of the local quality of each template at contributing to the target segmentation. We developed a fusion algorithm that uses probabilistic segmentations of the target image to simultaneously infer a reference standard segmentation of the target image and the local quality of each probabilistic segmentation. The concept of comparing templates to a hidden reference standard segmentation enables accurate assessments of the contribution of each template to inferring the target image segmentation to be made, and in practice leads to excellent target image segmentation. We have used the new algorithm for the multiple-template-based segmentation and parcellation of magnetic resonance images of the brain. Intensity and label map images of each one of the aligned templates are used to train a local Gaussian mixture model based classifier. Then, each classifier is used to compute the probabilistic segmentations of the target image. Finally, the generated probabilistic segmentations are fused together using the new fusion algorithm to obtain the segmentation of the target image. We evaluated our method in comparison to other state-of-the-art segmentation methods. We demonstrated that our new fusion algorithm has higher segmentation performance than these methods.

Segmentation of MR images via discriminative dictionary learning and sparse coding: application to hippocampus labeling

We propose a novel method for the automatic segmentation of brain MRI images by using discriminative dictionary learning and sparse coding techniques. In the proposed method, dictionaries and classifiers are learned simultaneously from a set of brain atlases, which can then be used for the reconstruction and segmentation of an unseen target image. The proposed segmentation strategy is based on image reconstruction, which is in contrast to most existing atlas-based labeling approaches that rely on comparing image similarities between atlases and target images. In addition, we propose a Fixed Discriminative Dictionary Learning for Segmentation (F-DDLS) strategy, which can learn dictionaries offline and perform segmentations online, enabling a significant speed-up in the segmentation stage. The proposed method has been evaluated for the hippocampus segmentation of 80 healthy ICBM subjects and 202 ADNI images. The robustness of the proposed method, especially of our F-DDLS strategy, was validated by training and testing on different subject groups in the ADNI database. The influence of different parameters was studied and the performance of the proposed method was also compared with that of the nonlocal patch-based approach. The proposed method achieved a median Dice coefficient of 0.879 on 202 ADNI images and 0.890 on 80 ICBM subjects, which is competitive compared with state-of-the-art methods.

Nonlocal regularization for active appearance model: Application to medial temporal lobe segmentation

The human medial temporal lobe (MTL) is an important part of the limbic system, and its substructures play key roles in learning, memory, and neurodegeneration. The MTL includes the hippocampus (HC), amygdala (AG), parahippocampal cortex (PHC), entorhinal cortex, and perirhinal cortex--structures that are complex in shape and have low between-structure intensity contrast, making them difficult to segment manually in magnetic resonance images. This article presents a new segmentation method that combines active appearance modeling and patch-based local refinement to automatically segment specific substructures of the MTL including HC, AG, PHC, and entorhinal/perirhinal cortex from MRI data. Appearance modeling, relying on eigen-decomposition to analyze statistical variations in image intensity and shape information in study population, is used to capture global shape characteristics of each structure of interest with a generative model. Patch-based local refinement, using nonlocal means to compare the image local intensity properties, is applied to locally refine the segmentation results along the structure borders to improve structure delimitation. In this manner, nonlocal regularization and global shape constraints could allow more accurate segmentations of structures. Validation experiments against manually defined labels demonstrate that this new segmentation method is computationally efficient, robust, and accurate. In a leave-one-out validation on 54 normal young adults, the method yielded a mean Dice κ of 0.87 for the HC, 0.81 for the AG, 0.73 for the anterior parts of the parahippocampal gyrus (entorhinal and perirhinal cortex), and 0.73 for the posterior parahippocampal gyrus.

Hippocampal atrophy in Alzheimer’s disease

SUMMARY New research criteria for Alzheimer’s disease (AD) and mild cognitive impairment (MCI) emphasize the use of imaging biomarkers in clinical diagnosis of these disorders. The volume loss of medial temporal lobe structures, especially hippocampal atrophy, is the best validated marker of AD. Manual tracing on MRI is the present gold standard for evaluating hippocampal volume; however, it is laborious and tracer-dependent. We categorized the most recent full- or semi-automated methods by the nature of the output of the method: size and shape of subcortical structures, cortical thickness, atrophy-rate and voxel- and region-based characteristics. The features of each method are introduced. The findings in structural MRI studies, especially in those studies utilizing the most recent methods, and the accuracies of those new methods in differentiating AD from healthy controls and stable MCI from progressive MCI are reviewed.

Simultaneous segmentation and grading of anatomical structures for patient's classification: application to Alzheimer's disease

In this paper, we propose an innovative approach to robustly and accurately detect Alzheimer's disease (AD) based on the distinction of specific atrophic patterns of anatomical structures such as hippocampus (HC) and entorhinal cortex (EC). The proposed method simultaneously performs segmentation and grading of structures to efficiently capture the anatomical alterations caused by AD. Known as SNIPE (Scoring by Non-local Image Patch Estimator), the novel proposed grading measure is based on a nonlocal patch-based frame-work and estimates the similarity of the patch surrounding the voxel under study with all the patches present in different training populations. In this study, the training library was composed of two populations: 50 cognitively normal subjects (CN) and 50 patients with AD, randomly selected from the ADNI database. During our experiments, the classification accuracy of patients (CN vs. AD) using several biomarkers was compared: HC and EC volumes, the grade of these structures and finally the combination of their volume and their grade. Tests were completed in a leave-one-out framework using discriminant analysis. First, we showed that biomarkers based on HC provide better classification accuracy than biomarkers based on EC. Second, we demonstrated that structure grading is a more powerful measure than structure volume to distinguish both populations with a classification accuracy of 90%. Finally, by adding the ages of subjects in order to better separate age-related structural changes from disease-related anatomical alterations, SNIPE obtained a classification accuracy of 93%.

A supervised patch-based approach for human brain labeling

We propose in this work a patch-based image labeling method relying on a label propagation framework. Based on image intensity similarities between the input image and an anatomy textbook, an original strategy which does not require any nonrigid registration is presented. Following recent developments in nonlocal image denoising, the similarity between images is represented by a weighted graph computed from an intensity-based distance between patches. Experiments on simulated and in vivo magnetic resonance images show that the proposed method is very successful in providing automated human brain labeling.

Iterative multi-atlas-based multi-image segmentation with tree-based registration

In this paper, we present a multi-atlas-based framework for accurate, consistent and simultaneous segmentation of a group of target images. Multi-atlas-based segmentation algorithms consider concurrently complementary information from multiple atlases to produce optimal segmentation outcomes. However, the accuracy of these algorithms relies heavily on the precise alignment of the atlases with the target image. In particular, the commonly used pairwise registration may result in inaccurate alignment especially between images with large shape differences. Additionally, when segmenting a group of target images, most current methods consider these images independently with disregard of their correlation, thus resulting in inconsistent segmentations of the same structures across different target images. We propose two novel strategies to address these limitations: 1) a novel tree-based groupwise registration method for concurrent alignment of both the atlases and the target images, and 2) an iterative groupwise segmentation method for simultaneous consideration of segmentation information propagated from all available images, including the atlases and other newly segmented target images. Evaluation based on various datasets indicates that the proposed multi-atlas-based multi-image segmentation (MABMIS) framework yields substantial improvements in terms of consistency and accuracy over methods that do not consider the group of target images holistically.

Mixture of segmenters with discriminative spatial regularization and sparse weight selection

This paper presents a novel segmentation algorithm which automatically learns the combination of weak segmenters and builds a strong one based on the assumption that the locally weighted combination varies w.r.t. both the weak segmenters and the training images. We learn the weighted combination during the training stage using a discriminative spatial regularization which depends on training set labels. A closed form solution to the cost function is derived for this approach. In the testing stage, a sparse regularization scheme is imposed to avoid overfitting. To the best of our knowledge, such a segmentation technique has never been reported in literature and we empirically show that it significantly improves on the performances of the weak segmenters. After showcasing the performance of the algorithm in the context of atlas-based segmentation, we present comparisons to the existing weak segmenter combination strategies on a hippocampal data set.