Nonlocal patch-based label fusion for hippocampus segmentation

Cross-Modality Multi-Atlas Segmentation Using Deep Neural Networks

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image; and the transformed atlas labels can be combined to generate target segmentation via label fusion schemes. Many conventional MAS methods employed the atlases from the same modality as the target image. However, the number of atlases with the same modality may be limited or even missing in many clinical applications. Besides, conventional MAS methods suffer from the computational burden of registration or label fusion procedures. In this work, we design a novel cross-modality MAS framework, which uses available atlases from a certain modality to segment a target image from another modality. To boost the computational efficiency of the framework, both the image registration and label fusion are achieved by well-designed deep neural networks. For the atlas-to-target image registration, we propose a bi-directional registration network (BiRegNet), which can efficiently align images from different modalities. For the label fusion, we design a similarity estimation network (SimNet), which estimates the fusion weight of each atlas by measuring its similarity to the target image. SimNet can learn multi-scale information for similarity estimation to improve the performance of label fusion. The proposed framework was evaluated by the left ventricle and liver segmentation tasks on the MM-WHS and CHAOS datasets, respectively. Results have shown that the framework is effective for cross-modality MAS in both registration and label fusion. The code will be released publicly on https://github.com/NanYoMy/cmmas once the manuscript is accepted.

Factorisation-Based Image Labelling

Segmentation of brain magnetic resonance images (MRI) into anatomical regions is a useful task in neuroimaging. Manual annotation is time consuming and expensive, so having a fully automated and general purpose brain segmentation algorithm is highly desirable. To this end, we propose a patched-based labell propagation approach based on a generative model with latent variables. Once trained, our Factorisation-based Image Labelling (FIL) model is able to label target images with a variety of image contrasts. We compare the effectiveness of our proposed model against the state-of-the-art using data from the MICCAI 2012 Grand Challenge and Workshop on Multi-Atlas Labelling. As our approach is intended to be general purpose, we also assess how well it can handle domain shift by labelling images of the same subjects acquired with different MR contrasts.

Pixel-Wise Classification in Hippocampus Histological Images

This paper presents a method for pixel-wise classification applied for the first time on hippocampus histological images. The goal is achieved by representing pixels in a 14-D vector, composed of grey-level information and moment invariants. Then, several popular machine learning models are used to categorize them, and multiple metrics are computed to evaluate the performance of the different models. The multilayer perceptron, random forest, support vector machine, and radial basis function networks were compared, achieving the multilayer perceptron model the highest result on accuracy metric, AUC, and F ₁ score with highly satisfactory results for substituting a manual classification task, due to an expert opinion in the hippocampus histological images.

Fast and Precise Hippocampus Segmentation Through Deep Convolutional Neural Network Ensembles and Transfer Learning

Automatic segmentation of the hippocampus from 3D magnetic resonance imaging mostly relied on multi-atlas registration methods. In this work, we exploit recent advances in deep learning to design and implement a fully automatic segmentation method, offering both superior accuracy and fast result. The proposed method is based on deep Convolutional Neural Networks (CNNs) and incorporates distinct segmentation and error correction steps. Segmentation masks are produced by an ensemble of three independent models, operating with orthogonal slices of the input volume, while erroneous labels are subsequently corrected by a combination of Replace and Refine networks. We explore different training approaches and demonstrate how, in CNN-based segmentation, multiple datasets can be effectively combined through transfer learning techniques, allowing for improved segmentation quality. The proposed method was evaluated using two different public datasets and compared favorably to existing methodologies. In the EADC-ADNI HarP dataset, the correspondence between the method's output and the available ground truth manual tracings yielded a mean Dice value of 0.9015, while the required segmentation time for an entire MRI volume was 14.8 seconds. In the MICCAI dataset, the mean Dice value increased to 0.8835 through transfer learning from the larger EADC-ADNI HarP dataset.

Automatic Labeling of MR Brain Images Through the Hashing Retrieval Based Atlas Forest

The multi-atlas method is one of the efficient and common automatic labeling method, which uses the prior information provided by expert-labeled images to guide the labeling of the target. However, most multi-atlas-based methods depend on the registration that may not give the correct information during the label propagation. To address the issue, we designed a new automatic labeling method through the hashing retrieval based atlas forest. The proposed method propagates labels without registration to reduce the errors, and constructs a target-oriented learning model to integrate information among the atlases. This method innovates a coarse classification strategy to preprocess the dataset, which retains the integrity of dataset and reduces computing time. Furthermore, the method considers each voxel in the atlas as a sample and encodes these samples with hashing for the fast sample retrieval. In the stage of labeling, the method selects suitable samples through hashing learning and trains atlas forests by integrating the information from the dataset. Then, the trained model is used to predict the labels of the target. Experimental results on two datasets illustrated that the proposed method is promising in the automatic labeling of MR brain images.

Fully automatic segmentation and objective assessment of atrial scars for long-standing persistent atrial fibrillation patients using late gadolinium-enhanced MRI

PURPOSE

Atrial fibrillation (AF) is the most common heart rhythm disorder and causes considerable morbidity and mortality, resulting in a large public health burden that is increasing as the population ages. It is associated with atrial fibrosis, the amount and distribution of which can be used to stratify patients and to guide subsequent electrophysiology ablation treatment. Atrial fibrosis may be assessed noninvasively using late gadolinium-enhanced (LGE) magnetic resonance imaging (MRI) where scar tissue is visualized as a region of signal enhancement. However, manual segmentation of the heart chambers and of the atrial scar tissue is time consuming and subject to interoperator variability, particularly as image quality in AF is often poor. In this study, we propose a novel fully automatic pipeline to achieve accurate and objective segmentation of the heart (from MRI Roadmap data) and of scar tissue within the heart (from LGE MRI data) acquired in patients with AF.

METHODS

Our fully automatic pipeline uniquely combines: (a) a multiatlas-based whole heart segmentation (MA-WHS) to determine the cardiac anatomy from an MRI Roadmap acquisition which is then mapped to LGE MRI, and (b) a super-pixel and supervised learning based approach to delineate the distribution and extent of atrial scarring in LGE MRI. We compared the accuracy of the automatic analysis to manual ground truth segmentations in 37 patients with persistent long-standing AF.

RESULTS

Both our MA-WHS and atrial scarring segmentations showed accurate delineations of cardiac anatomy (mean Dice = 89%) and atrial scarring (mean Dice = 79%), respectively, compared to the established ground truth from manual segmentation. In addition, compared to the ground truth, we obtained 88% segmentation accuracy, with 90% sensitivity and 79% specificity. Receiver operating characteristic analysis achieved an average area under the curve of 0.91.

CONCLUSION

Compared with previously studied methods with manual interventions, our innovative pipeline demonstrated comparable results, but was computed fully automatically. The proposed segmentation methods allow LGE MRI to be used as an objective assessment tool for localization, visualization, and quantitation of atrial scarring and to guide ablation treatment.

The bumps under the hippocampus

Shown in every neuroanatomy textbook, a key morphological feature is the bumpy ridges, which we refer to as hippocampal dentation, on the inferior aspect of the hippocampus. Like the folding of the cerebral cortex, hippocampal dentation allows for greater surface area in a confined space. However, examining numerous approaches to hippocampal segmentation and morphology analysis, virtually all published 3D renderings of the hippocampus show the inferior surface to be quite smooth or mildly irregular; we have rarely seen the characteristic bumpy structure on reconstructed 3D surfaces. The only exception is a 9.4T postmortem study (Yushkevich et al. [2009]: NeuroImage 44:385-398). An apparent question is, does this indicate that this specific morphological signature can only be captured using ultra high-resolution techniques? Or, is such information buried in the data we commonly acquire, awaiting a computation technique that can extract and render it clearly? In this study, we propose an automatic and robust super-resolution technique that captures the fine scale morphometric features of the hippocampus based on common 3T MR images. The method is validated on 9.4T ultra-high field images and then applied on 3T data sets. This method opens possibilities of future research on the hippocampus and other sub-cortical structural morphometry correlating the degree of dentation with a range of diseases including epilepsy, Alzheimer's disease, and schizophrenia. Hum Brain Mapp 39:472-490, 2018. © 2017 Wiley Periodicals, Inc.

Familiarity deficits in cognitively normal aging individuals with APOE ε4: A follow-up investigation of medial temporal lobe structural correlates

Introduction

The apolipoprotein E ε4 (APOE ε4) allele is a well-documented risk factor for Alzheimer's disease (AD). Accordingly, aging individuals carrying one or more ε4 alleles are at considerably greater risk of developing AD over time. In an effort to characterize early cognitive manifestations of AD, we previously outlined selective deficits in familiarity-based recognition in otherwise asymptomatic carriers of the APOE ε4 allele (Schoemaker et al., 2016). In this follow-up report, we aimed to explore the neural correlates of this selective cognitive impairment.

Methods

For this purpose, within the same population and using high-resolution structural neuroimaging, we explored relationships between volumes of the hippocampus, entorhinal, and perirhinal cortices and performance in recollection and familiarity.

Results

Overall, our results revealed significant positive relationships between familiarity performance and volumes of the perirhinal and entorhinal cortices in aging individuals with APOE ε4. In APOE ε4 carriers, a positive correlation between recollection performance and hippocampal volume was also found. In contrast, no correlation reached statistical significance in the group of noncarriers.

Conclusion

These findings suggest that familiarity performance might be a useful marker of the integrity of the rhinal cortex, especially in populations at risk of AD.

Evaluation of Multi-Atlas Label Fusion for In Vivo MRI Orbital Segmentation

Multi-atlas methods have been successful for brain segmentation, but their application to smaller anatomies remains relatively unexplored. We evaluate 7 statistical and voting-based label fusion algorithms (and 6 additional variants) to segment the optic nerves, eye globes and chiasm. For non-local STAPLE, we evaluate different intensity similarity measures (including mean square difference, locally normalized cross correlation, and a hybrid approach). Each algorithm is evaluated in terms of the Dice overlap and symmetric surface distance metrics. Finally, we evaluate refinement of label fusion results using a learning based correction method for consistent bias correction and Markov random field regularization. The multi-atlas labeling pipelines were evaluated on a cohort of 35 subjects including both healthy controls and patients. Across all three structures, NLSS with a mixed weighting type provided the most consistent results; for the optic nerve NLSS resulted in a median Dice similarity coefficient of 0.81, mean surface distance of 0.41 mm and Hausdorff distance 2.18 mm for the optic nerves. Joint label fusion resulted in slightly superior median performance for the optic nerves (0.82, 0.39 mm and 2.15 mm), but slightly worse on the globes. The fully automated multi-atlas labeling approach provides robust segmentations of orbital structures on MRI even in patients for whom significant atrophy (optic nerve head drusen) or inflammation (multiple sclerosis) is present.

Rapid automatic segmentation of the human cerebellum and its lobules (RASCAL)--implementation and application of the patch-based label-fusion technique with a template library to segment the human cerebellum

Reliable and fast segmentation of the human cerebellum with its complex architecture of lobes and lobules has been a challenge for the past decades. Emerging knowledge of the functional integration of the cerebellum in various sensori-motor and cognitive-behavioral circuits demands new automatic segmentation techniques, with accuracies similar to manual segmentations, but applicable to large subject numbers in a reasonable time frame. This article presents the development and application of a novel pipeline for rapid automatic segmentation of the human cerebellum and its lobules (RASCAL) combining patch-based label-fusion and a template library of manually labeled cerebella of 16 healthy controls from the International Consortium for Brain Mapping (ICBM) database. Leave-one-out experiments revealed a good agreement between manual and automatic segmentations (Dice kappa = 0.82). Intraclass correlation coefficients (ICC) were calculated to test reliability of segmented volumes and were highest (ICC > 0.9) for global measures (total and hemispherical grey and white matter) followed by larger lobules of the posterior lobe (ICC > 0.8). Further we applied the pipeline to all 152 young healthy controls of the ICBM database to look for hemispheric and gender differences. The results demonstrated larger native space volumes in men then women (mean (± SD) total cerebellar volume in women = 217 cm(3) (± 26), men = 259 cm(3) (± 29); P < 0.001). Significant gender-by-hemisphere interaction was only found in stereotaxic space volumes for white matter core (men > women) and anterior lobe volume (women > men). This new method shows great potential for the precise and efficient analysis of the cerebellum in large patient cohorts.

Accurate and Fully Automatic Hippocampus Segmentation Using Subject-Specific 3D Optimal Local Maps Into a Hybrid Active Contour Model

Assessing the structural integrity of the hippocampus (HC) is an essential step toward prevention, diagnosis, and follow-up of various brain disorders due to the implication of the structural changes of the HC in those disorders. In this respect, the development of automatic segmentation methods that can accurately, reliably, and reproducibly segment the HC has attracted considerable attention over the past decades. This paper presents an innovative 3-D fully automatic method to be used on top of the multiatlas concept for the HC segmentation. The method is based on a subject-specific set of 3-D optimal local maps (OLMs) that locally control the influence of each energy term of a hybrid active contour model (ACM). The complete set of the OLMs for a set of training images is defined simultaneously via an optimization scheme. At the same time, the optimal ACM parameters are also calculated. Therefore, heuristic parameter fine-tuning is not required. Training OLMs are subsequently combined, by applying an extended multiatlas concept, to produce the OLMs that are anatomically more suitable to the test image. The proposed algorithm was tested on three different and publicly available data sets. Its accuracy was compared with that of state-of-the-art methods demonstrating the efficacy and robustness of the proposed method.

A unified framework for cross-modality multi-atlas segmentation of brain MRI

Multi-atlas label fusion is a powerful image segmentation strategy that is becoming increasingly popular in medical imaging. A standard label fusion algorithm relies on independently computed pairwise registrations between individual atlases and the (target) image to be segmented. These registrations are then used to propagate the atlas labels to the target space and fuse them into a single final segmentation. Such label fusion schemes commonly rely on the similarity between intensity values of the atlases and target scan, which is often problematic in medical imaging - in particular, when the atlases and target images are obtained via different sensor types or imaging protocols. In this paper, we present a generative probabilistic model that yields an algorithm for solving the atlas-to-target registrations and label fusion steps simultaneously. The proposed model does not directly rely on the similarity of image intensities. Instead, it exploits the consistency of voxel intensities within the target scan to drive the registration and label fusion, hence the atlases and target image can be of different modalities. Furthermore, the framework models the joint warp of all the atlases, introducing interdependence between the registrations. We use variational expectation maximization and the Demons registration framework in order to efficiently identify the most probable segmentation and registrations. We use two sets of experiments to illustrate the approach, where proton density (PD) MRI atlases are used to segment T1-weighted brain scans and vice versa. Our results clearly demonstrate the accuracy gain due to exploiting within-target intensity consistency and integrating registration into label fusion.

Spatial Bias in Multi-Atlas Based Segmentation

Multi-atlas segmentation has been widely applied in medical image analysis. With deformable registration, this technique realizes label transfer from pre-labeled atlases to unknown images. When deformable registration produces error, label fusion that combines results produced by multiple atlases is an effective way for reducing segmentation errors. Among the existing label fusion strategies, similarity-weighted voting strategies with spatially varying weight distributions have been particularly successful. We show that, weighted voting based label fusion produces a spatial bias that under-segments structures with convex shapes. The bias can be approximated as applying spatial convolution to the ground truth spatial label probability maps, where the convolution kernel combines the distribution of residual registration errors and the function producing similarity-based voting weights. To reduce this bias, we apply a standard spatial deconvolution to the spatial probability maps obtained from weighted voting. In a brain image segmentation experiment, we demonstrate the spatial bias and show that our technique substantially reduces this spatial bias.

A Generative Model for Probabilistic Label Fusion of Multimodal Data

The maturity of registration methods, in combination with the increasing processing power of computers, has made multi-atlas segmentation methods practical. The problem of merging the deformed label maps from the atlases is known as label fusion. Even though label fusion has been well studied for intramodality scenarios, it remains relatively unexplored when the nature of the target data is multimodal or when its modality is different from that of the atlases. In this paper, we review the literature on label fusion methods and also present an extension of our previously published algorithm to the general case in which the target data are multimodal. The method is based on a generative model that exploits the consistency of voxel intensities within the target scan based on the current estimate of the segmentation. Using brain MRI scans acquired with a multiecho FLASH sequence, we compare the method with majority voting, statistical-atlas-based segmentation, the popular package FreeSurfer and an adaptive local multi-atlas segmentation method. The results show that our approach produces highly accurate segmentations (Dice 86.3% across 22 brain structures of interest), outperforming the competing methods.

Optimal weights for multi-atlas label fusion

Multi-atlas based segmentation has been applied widely in medical image analysis. For label fusion, previous studies show that image similarity-based local weighting techniques produce the most accurate results. However, these methods ignore the correlations between results produced by different atlases. Furthermore, they rely on pre-selected weighting models and ad hoc methods to choose model parameters. We propose a novel label fusion method to address these limitations. Our formulation directly aims at reducing the expectation of the combined error and can be efficiently solved in a closed form. In our hippocampus segmentation experiment, our method significantly outperforms similarity-based local weighting. Using 20 atlases, we produce results with 0.898 +/- 0.019 Dice overlap to manual labelings for controls.