MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans

Semantic segmentation of cerebrospinal fluid and brain volume with a convolutional neural network in pediatric hydrocephalus-transfer learning from existing algorithms

Background

For the segmentation of medical imaging data, a multitude of precise but very specific algorithms exist. In previous studies, we investigated the possibility of segmenting MRI data to determine cerebrospinal fluid and brain volume using a classical machine learning algorithm. It demonstrated good clinical usability and a very accurate correlation of the volumes to the single area determination in a reproducible axial layer. This study aims to investigate whether these established segmentation algorithms can be transferred to new, more generalizable deep learning algorithms employing an extended transfer learning procedure and whether medically meaningful segmentation is possible.

Methods

Ninety-five routinely performed true FISP MRI sequences were retrospectively analyzed in 43 patients with pediatric hydrocephalus. Using a freely available and clinically established segmentation algorithm based on a hidden Markov random field model, four classes of segmentation (brain, cerebrospinal fluid (CSF), background, and tissue) were generated. Fifty-nine randomly selected data sets (10,432 slices) were used as a training data set. Images were augmented for contrast, brightness, and random left/right and X/Y translation. A convolutional neural network (CNN) for semantic image segmentation composed of an encoder and corresponding decoder subnetwork was set up. The network was pre-initialized with layers and weights from a pre-trained VGG 16 model. Following the network was trained with the labeled image data set. A validation data set of 18 scans (3289 slices) was used to monitor the performance as the deep CNN trained. The classification results were tested on 18 randomly allocated labeled data sets (3319 slices) and on a T2-weighted BrainWeb data set with known ground truth.

Results

The segmentation of clinical test data provided reliable results (global accuracy 0.90, Dice coefficient 0.86), while the CNN segmentation of data from the BrainWeb data set showed comparable results (global accuracy 0.89, Dice coefficient 0.84). The segmentation of the BrainWeb data set with the classical FAST algorithm produced consistent findings (global accuracy 0.90, Dice coefficient 0.87). Likewise, the area development of brain and CSF in the long-term clinical course of three patients was presented.

Conclusion

Using the presented methods, we showed that conventional segmentation algorithms can be transferred to new advances in deep learning with comparable accuracy, generating a large number of training data sets with relatively little effort. A clinically meaningful segmentation possibility was demonstrated.

The data representativeness criterion: Predicting the performance of supervised classification based on data set similarity

In a broad range of fields it may be desirable to reuse a supervised classification algorithm and apply it to a new data set. However, generalization of such an algorithm and thus achieving a similar classification performance is only possible when the training data used to build the algorithm is similar to new unseen data one wishes to apply it to. It is often unknown in advance how an algorithm will perform on new unseen data, being a crucial reason for not deploying an algorithm at all. Therefore, tools are needed to measure the similarity of data sets. In this paper, we propose the Data Representativeness Criterion (DRC) to determine how representative a training data set is of a new unseen data set. We present a proof of principle, to see whether the DRC can quantify the similarity of data sets and whether the DRC relates to the performance of a supervised classification algorithm. We compared a number of magnetic resonance imaging (MRI) data sets, ranging from subtle to severe difference is acquisition parameters. Results indicate that, based on the similarity of data sets, the DRC is able to give an indication as to when the performance of a supervised classifier decreases. The strictness of the DRC can be set by the user, depending on what one considers to be an acceptable underperformance.

Analysis of MRI Data in Diagnostic Neuroradiology

Magnetic resonance imaging (MRI) is a noninvasive imaging tool for neuroradiological diagnosis. Numerous concepts of automated MRI analysis and the use of machine learning have been proposed to assist diagnosis and prognosis. While these academic innovations have proven effective in principle within controlled environments, their application to clinical practice has faced unmet requirements, such as the ability to perform reliably across a heterogeneous population, to work robustly in the presence of comorbidities, and to be invariant to scanner hardware and image quality. The lack of realistic confidence bounds and the inability to handle missing data have also reduced the application of most of these methods outside of academic studies. Mastering the complex challenges in the diagnostic process may help researchers discover novel biological constructs in multimodal data and improve stratification for clinical trials, paving the way for precision medicine. This review presents the state of the art of computerized brain MRI analysis for diagnostic purposes. We critically evaluate the current clinical usefulness of the methods and highlight challenges and future perspectives of the field.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

ExploreASL: An image processing pipeline for multi-center ASL perfusion MRI studies

Arterial spin labeling (ASL) has undergone significant development since its inception, with a focus on improving standardization and reproducibility of its acquisition and quantification. In a community-wide effort towards robust and reproducible clinical ASL image processing, we developed the software package ExploreASL, allowing standardized analyses across centers and scanners. The procedures used in ExploreASL capitalize on published image processing advancements and address the challenges of multi-center datasets with scanner-specific processing and artifact reduction to limit patient exclusion. ExploreASL is self-contained, written in MATLAB and based on Statistical Parameter Mapping (SPM) and runs on multiple operating systems. To facilitate collaboration and data-exchange, the toolbox follows several standards and recommendations for data structure, provenance, and best analysis practice. ExploreASL was iteratively refined and tested in the analysis of >10,000 ASL scans using different pulse-sequences in a variety of clinical populations, resulting in four processing modules: Import, Structural, ASL, and Population that perform tasks, respectively, for data curation, structural and ASL image processing and quality control, and finally preparing the results for statistical analyses on both single-subject and group level. We illustrate ExploreASL processing results from three cohorts: perinatally HIV-infected children, healthy adults, and elderly at risk for neurodegenerative disease. We show the reproducibility for each cohort when processed at different centers with different operating systems and MATLAB versions, and its effects on the quantification of gray matter cerebral blood flow. ExploreASL facilitates the standardization of image processing and quality control, allowing the pooling of cohorts which may increase statistical power and discover between-group perfusion differences. Ultimately, this workflow may advance ASL for wider adoption in clinical studies, trials, and practice.

Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis

The Sørensen-Dice index (SDI) is a widely used measure for evaluating medical image segmentation algorithms. It offers a standardized measure of segmentation accuracy which has proven useful. However, it offers diminishing insight when the number of objects is unknown, such as in white matter lesion segmentation of multiple sclerosis (MS) patients. We present a refinement for finer grained parsing of SDI results in situations where the number of objects is unknown. We explore these ideas with two case studies showing what can be learned from our two presented studies. Our first study explores an inter-rater comparison, showing that smaller lesions cannot be reliably identified. In our second case study, we demonstrate fusing multiple MS lesion segmentation algorithms based on the insights into the algorithms provided by our analysis to generate a segmentation that exhibits improved performance. This work demonstrates the wealth of information that can be learned from refined analysis of medical image segmentations.

Planar single plane area determination is a viable substitute for total volumetry of CSF and brain in childhood hydrocephalus

BACKGROUND

In the treatment of childhood hydrocephalus, 3D volumetry seems to have many advantages over classical planar index measurements for dedicated monitoring of changes in cerebrospinal fluid and brain volume. Nevertheless, this method requires extensive technical effort and access to the complete three-dimensional data set. Against this background, we evaluated the possibility of planar area determination in a single plane and the correlation to volumetry.

METHODS

138 routinely performed true FISP MRI sequences (1 mm isovoxel) were analyzed retrospectively in 68 patients with pediatric hydrocephalus. After preprocessing, the 3D-data sets were skull stripped to estimate the inner skull volume. A 2-class segmentation into different tissue types (brain matter and CSF) was performed, and the volumes of CSF (VCSF) and brain matter (VBrain) were calculated. A plane at the level of the foramina of Monro was manually identified in the ac-pc oriented data. In this plane, the areas of brain (ABrain) and CSF (ACSF) in cm² were calculated and used for further correlation analysis.

RESULTS

Mean VCSF was 340 ± 145 cm³ and VBrain 1173 ± 254 cm³. In the selected plane, ACSF was 26 ± 14 cm², and ABrain was 107 ± 25 cm². There was a very strong positive correlation between both ACSF and VCSF (r = 0.895) and between ABrain and VBrain (r = 0.846). The prediction equations for VBrain and VCSF were highly significant.

CONCLUSION

Planar area determination of brain and CSF correlates excellently with both VCSF and VBrain. Thus, areas can serve as a surrogate marker for total brain and CSF volumes for a quantitated objective tracking of changes during treatment of childhood hydrocephalus.

A 3D Spatially Weighted Network for Segmentation of Brain Tissue From MRI

The segmentation of brain tissue in MRI is valuable for extracting brain structure to aid diagnosis, treatment and tracking the progression of different neurologic diseases. Medical image data are volumetric and some neural network models for medical image segmentation have addressed this using a 3D convolutional architecture. However, this volumetric spatial information has not been fully exploited to enhance the representative ability of deep networks, and these networks have not fully addressed the practical issues facing the analysis of multimodal MRI data. In this paper, we propose a spatially-weighted 3D network (SW-3D-UNet) for brain tissue segmentation of single-modality MRI, and extend it using multimodality MRI data. We validate our model on the MRBrainS13 and MALC12 datasets. This unpublished model ranked first on the leaderboard of the MRBrainS13 Challenge.

Arterial CO2 pressure changes during hypercapnia are associated with changes in brain parenchymal volume

The Monro-Kellie hypothesis (MKH) states that volume changes in any intracranial component (blood, brain tissue, cerebrospinal fluid) should be counterbalanced by a co-occurring opposite change to maintain intracranial pressure within the fixed volume of the cranium. In this feasibility study, we investigate the MKH application to structural magnetic resonance imaging (MRI) in observing compensating intracranial volume changes during hypercapnia, which causes an increase in cerebral blood volume. Seven healthy subjects aged from 24 to 64 years (median 32), 4 males and 3 females, underwent a 3-T three-dimensional T1-weighted MRI under normocapnia and under hypercapnia. Intracranial tissue volumes were computed. According to the MKH, the significant increase in measured brain parenchymal volume (median 6.0 mL; interquartile range 4.5, 8.5; p = 0.016) during hypercapnia co-occurred with a decrease in intracranial cerebrospinal fluid (median -10.0 mL; interquartile range -13.5, -6.5; p = 0.034). These results convey several implications: (i) blood volume changes either caused by disorders, anaesthesia, or medication can affect outcome of brain volumetric studies; (ii) besides probing tissue displacement, this approach may assess the brain cerebrovascular reactivity. Future studies should explore the use of alternative sequences, such as three-dimensional T2-weighted imaging, for improved quantification of hypercapnia-induced volume changes.

Conventional and Deep Learning Methods for Skull Stripping in Brain MRI

Skull stripping in brain magnetic resonance volume has recently been attracting attention due to an increased demand to develop an efficient, accurate, and general algorithm for diverse datasets of the brain. Accurate skull stripping is a critical step for neuroimaging diagnostic systems because neither the inclusion of non-brain tissues nor removal of brain parts can be corrected in subsequent steps, which results in unfixed error through subsequent analysis. The objective of this review article is to give a comprehensive overview of skull stripping approaches, including recent deep learning-based approaches. In this paper, the current methods of skull stripping have been divided into two distinct groups—conventional or classical approaches, and convolutional neural networks or deep learning approaches. The potentials of several methods are emphasized because they can be applied to standard clinical imaging protocols. Finally, current trends and future developments are addressed giving special attention to recent deep learning algorithms.

MRI Cross-Modality Image-to-Image Translation

We present a cross-modality generation framework that learns to generate translated modalities from given modalities in MR images. Our proposed method performs Image Modality Translation (abbreviated as IMT) by means of a deep learning model that leverages conditional generative adversarial networks (cGANs). Our framework jointly exploits the low-level features (pixel-wise information) and high-level representations (e.g. brain tumors, brain structure like gray matter, etc.) between cross modalities which are important for resolving the challenging complexity in brain structures. Our framework can serve as an auxiliary method in medical use and has great application potential. Based on our proposed framework, we first propose a method for cross-modality registration by fusing the deformation fields to adopt the cross-modality information from translated modalities. Second, we propose an approach for MRI segmentation, translated multichannel segmentation (TMS), where given modalities, along with translated modalities, are segmented by fully convolutional networks (FCN) in a multichannel manner. Both of these two methods successfully adopt the cross-modality information to improve the performance without adding any extra data. Experiments demonstrate that our proposed framework advances the state-of-the-art on five brain MRI datasets. We also observe encouraging results in cross-modality registration and segmentation on some widely adopted brain datasets. Overall, our work can serve as an auxiliary method in medical use and be applied to various tasks in medical fields.

A deep error correction network for compressed sensing MRI

BACKGROUND

CS-MRI (compressed sensing for magnetic resonance imaging) exploits image sparsity properties to reconstruct MRI from very few Fourier k-space measurements. Due to imperfect modelings in the inverse imaging, state-of-the-art CS-MRI methods tend to leave structural reconstruction errors. Compensating such errors in the reconstruction could help further improve the reconstruction quality.

RESULTS

In this work, we propose a DECN (deep error correction network) for CS-MRI. The DECN model consists of three parts, which we refer to as modules: a guide, or template, module, an error correction module, and a data fidelity module. Existing CS-MRI algorithms can serve as the template module for guiding the reconstruction. Using this template as a guide, the error correction module learns a CNN (convolutional neural network) to map the k-space data in a way that adjusts for the reconstruction error of the template image. We propose a deep error correction network. Our experimental results show the proposed DECN CS-MRI reconstruction framework can considerably improve upon existing inversion algorithms by supplementing with an error-correcting CNN.

CONCLUSIONS

In the proposed a deep error correction framework, any off-the-shelf CS-MRI algorithm can be used as template generation. Then a deep neural network is used to compensate reconstruction errors. The promising experimental results validate the effectiveness and utility of the proposed framework.

Automatic volumetry of cerebrospinal fluid and brain volume in severe paediatric hydrocephalus, implementation and clinical course after intervention

BACKGROUND

In childhood hydrocephalus, both the amount of cerebrospinal fluid and the brain volume are relevant for the prognosis of the development and for therapy monitoring. Since classical planar measurements of ventricular size are subject to strong limitations, imprecise and neglect brain volume, 3D volumetry is most desirable. We used and evaluated the robust segmentation algorithms of the freely available FSL-toolbox in paediatric hydrocephalus patients before and after specific therapy.

METHODS

Retrospectively 76 pre- and postoperative high-resolution T2-weighted MRI sequences (true FISP, 1 mm isovoxel) were analyzed in 38 patients with paediatric hydrocephalus (mean 4.4 ± 5.1 years) who underwent surgical treatment (ventriculo-peritoneal (VP) shunt n = 22, endoscopic third ventriculostomy (ETV) n = 16). After preprocessing, the 3D-datasets were skull stripped to estimate the inner skull surface. Following, a 2 class segmentation into different tissue types (brain matter and CSF) was performed. The volumes of CSF and brain were calculated.

RESULTS

The method could be implemented in an automated fashion in all 76 MRIs. In the VP shunt cohort, the amount of CSF (p < 0.001) decreased. Consecutively brain volume increased significantly (p < 0.001). Following ETV, CSF volume (p = 0.019) decreased significantly (p = 0.012) although the reduction was less pronounced than after shunt implantation. Brain volume expanded (p = 0.02).

CONCLUSION

A reliable automated segmentation of CSF and brain could be performed with the implemented algorithm. The method was able to track changes after therapy and detected significant differences in CSF and brain volumes after shunting and after ETV.

Context-Aware Superpixel and Bilateral Entropy-Image Coherence Induces Less Entropy

Superpixel clustering is one of the most popular computer vision techniques that aggregates coherent pixels into perceptually meaningful groups, taking inspiration from Gestalt grouping rules. However, due to brain complexity, the underlying mechanisms of such perceptual rules are unclear. Thus, conventional superpixel methods do not completely follow them and merely generate a flat image partition rather than hierarchical ones like a human does. In addition, those methods need to initialize the total number of superpixels, which may not suit diverse images. In this paper, we first propose context-aware superpixel (CASP) that follows both Gestalt grouping rules and the top-down hierarchical principle. Thus, CASP enables to adapt the total number of superpixels to specific images automatically. Next, we propose bilateral entropy, with two aspects conditional intensity entropy and spatial occupation entropy, to evaluate the encoding efficiency of image coherence. Extensive experiments demonstrate CASP achieves better superpixel segmentation performance and less entropy than baseline methods. More than that, using Pearson’s correlation coefficient, a collection of data with a total of 120 samples demonstrates a strong correlation between local image coherence and superpixel segmentation performance. Our results inversely support the reliability of above-mentioned perceptual rules, and eventually, we suggest designing novel entropy criteria to test the encoding efficiency of more complex patterns.

A Deep Information Sharing Network for Multi-Contrast Compressed Sensing MRI Reconstruction

Compressed sensing (CS) theory can accelerate multi-contrast magnetic resonance imaging (MRI) by sampling fewer measurements within each contrast. However, conventional optimization-based reconstruction models suffer several limitations, including a strict assumption of shared sparse support, time-consuming optimization, and "shallow" models with difficulties in encoding the patterns contained in massive MRI data. In this paper, we propose the first deep learning model for multi-contrast CS-MRI reconstruction. We achieve information sharing through feature sharing units, which significantly reduces the number of model parameters. The feature sharing unit combines with a data fidelity unit to comprise an inference block, which are then cascaded with dense connections, allowing for efficient information transmission across different depths of the network. Experiments on various multi-contrast MRI datasets show that the proposed model outperforms both state-of-the-art single-contrast and multi-contrast MRI methods in accuracy and efficiency. We demonstrate that improved reconstruction quality can bring benefits to subsequent medical image analysis. Furthermore, the robustness of the proposed model to misregistration shows its potential in real MRI applications.

A novel hybrid atlas-free hierarchical graph-based segmentation of newborn brain MRI using wavelet filter banks

Objective: The newborn brain MRI (magnetic resonance imaging) tissue segmentation plays a vital part in assessment of primary brain growth. In the newborn stage (nearly less than 28 days old), in T1- as well as T2-weighted MR images similar levels of intensity are exhibited by WM and GM, makes segmentation of the tissue extremely challenging. In this newborn stage for tissue segmentation, very few methods are developed. Hence the development of accurate brain tissue segmentation of neonate is prime objective of this paper.Methods: In this research work, we propose a novel hybrid atlas-free hierarchical graph-based tissue segmentation method for newborn infants. Wavelet filter banks are a class of deep models wherein filters and local neighborhood processes are used alternately for efficient segmentation on the raw input images, and fuzzy-based SVM (support vector machine) is used for precise tissue classification.Results: Specifically, from T1, T2 images multimodality information are used as inputs and then as outputs the segmentation maps are generated. The proposed approach considerably outperforms preceding methods of tissue segmentation as reflected in results. With this approach, the newborn MRI images that are even suffered from noise, poor resolution or the low contrasted images are also segmented more effectively with precision of 90% and sensitivity 98%.Conclusion: In addition, our findings indicate that the incorporation of multi-modality image led to significant improvements in performance. Thus, the proposed work effectively tackles the unreliability as well as the other issues faced with the prior methodologies with an interactive accurate segmentation outline.

Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge

Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. The automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their methods on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge. Sixty T1 + FLAIR images from three MR scanners were released with the manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. The segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: 1) Dice similarity coefficient; 2) modified Hausdorff distance (95th percentile); 3) absolute log-transformed volume difference; 4) sensitivity for detecting individual lesions; and 5) F1-score for individual lesions. In addition, the methods were ranked on their inter-scanner robustness; 20 participants submitted their methods for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all the methods generalize to unseen scanners. The challenge remains open for future submissions and provides a public platform for method evaluation.

RETOUCH: The Retinal OCT Fluid Detection and Segmentation Benchmark and Challenge

Retinal swelling due to the accumulation of fluid is associated with the most vision-threatening retinal diseases. Optical coherence tomography (OCT) is the current standard of care in assessing the presence and quantity of retinal fluid and image-guided treatment management. Deep learning methods have made their impact across medical imaging, and many retinal OCT analysis methods have been proposed. However, it is currently not clear how successful they are in interpreting the retinal fluid on OCT, which is due to the lack of standardized benchmarks. To address this, we organized a challenge RETOUCH in conjunction with MICCAI 2017, with eight teams participating. The challenge consisted of two tasks: fluid detection and fluid segmentation. It featured for the first time: all three retinal fluid types, with annotated images provided by two clinical centers, which were acquired with the three most common OCT device vendors from patients with two different retinal diseases. The analysis revealed that in the detection task, the performance on the automated fluid detection was within the inter-grader variability. However, in the segmentation task, fusing the automated methods produced segmentations that were superior to all individual methods, indicating the need for further improvements in the segmentation performance.

Image Based Brain Segmentation: From Multi-Atlas Fusion to Deep Learning

Background:

This review aims to identify the development of the algorithms for brain tissue and structure segmentation in MRI images.

Discussion:

Starting from the results of the Grand Challenges on brain tissue and structure segmentation held in Medical Image Computing and Computer-Assisted Intervention (MICCAI), this review analyses the development of the algorithms and discusses the tendency from multi-atlas label fusion to deep learning. The intrinsic characteristics of the winners’ algorithms on the Grand Challenges from the year 2012 to 2018 are analyzed and the results are compared carefully.

Conclusion:

Although deep learning has got higher rankings in the challenge, it has not yet met the expectations in terms of accuracy. More effective and specialized work should be done in the future.

Volumetric assessment of extracranial carotid artery aneurysms

The extracranial carotid artery aneurysm (ECAA) is a rare pathology for which clinical treatment guidelines are lacking. In general, symptoms or growth of the aneurysm sac are thought to indicate intervention. ECAAs may present in a large variety of shapes and sizes, and conventional diameter measurements fail to indicate geometrical differences. Therefore, we propose a protocol to measure ECAA size by 3D volumetric assessment. The volumes of 40 ECAAs in computed tomography angiography (CTA) images were measured through manual segmentation, by two independent operators. Volumes of the entire internal carotid artery (ICA) and the ECAA were measured separately. Excellent inter- and intraoperator reliability was found for both ICA and ECAA volumes, with all intraclass correlation coefficients above 0.94. Bland-Altman analysis revealed normal differences for both inter- and intraoperator agreement. For all volumes, similarity of the segmentations was excellent. Outliers were explained by presence of intraluminal ECAA thrombus, which hampered identification of the aneurysm outer wall. These results implicate robustness of our protocol, which is designed as a step-up towards (semi)automatic volumetric measurements to monitor patients with ECAA. Future (semi)automatic volumetric assessments are recommended and such techniques can be developed and validated using the proposed protocol and manual reference segmentations.

Testing a deep convolutional neural network for automated hippocampus segmentation in a longitudinal sample of healthy participants

Subtle changes in hippocampal volumes may occur during both physiological and pathophysiological processes in the human brain. Assessing hippocampal volumes manually is a time-consuming procedure, however, creating a need for automated segmentation methods that are both fast and reliable over time. Segmentation algorithms that employ deep convolutional neural networks (CNN) have emerged as a promising solution for large longitudinal neuroimaging studies. However, for these novel algorithms to be useful in clinical studies, the accuracy and reproducibility should be established on independent datasets. Here, we evaluate the performance of a CNN-based hippocampal segmentation algorithm that was developed by Thyreau and colleagues - Hippodeep. We compared its segmentation outputs to manual segmentation and FreeSurfer 6.0 in a sample of 200 healthy participants scanned repeatedly at seven sites across Canada, as part of the Canadian Biomarker Integration Network in Depression consortium. The algorithm demonstrated high levels of stability and reproducibility of volumetric measures across all time points compared to the other two techniques. Although more rigorous testing in clinical populations is necessary, this approach holds promise as a viable option for tracking volumetric changes in longitudinal neuroimaging studies.

HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation

Recently, dense connections have attracted substantial attention in computer vision because they facilitate gradient flow and implicit deep supervision during training. Particularly, DenseNet that connects each layer to every other layer in a feed-forward fashion and has shown impressive performances in natural image classification tasks. We propose HyperDenseNet, a 3-D fully convolutional neural network that extends the definition of dense connectivity to multi-modal segmentation problems. Each imaging modality has a path, and dense connections occur not only between the pairs of layers within the same path but also between those across different paths. This contrasts with the existing multi-modal CNN approaches, in which modeling several modalities relies entirely on a single joint layer (or level of abstraction) for fusion, typically either at the input or at the output of the network. Therefore, the proposed network has total freedom to learn more complex combinations between the modalities, within and in-between all the levels of abstraction, which increases significantly the learning representation. We report extensive evaluations over two different and highly competitive multi-modal brain tissue segmentation challenges, iSEG 2017 and MRBrainS 2013, with the former focusing on six month infant data and the latter on adult images. HyperDenseNet yielded significant improvements over many state-of-the-art segmentation networks, ranking at the top on both benchmarks. We further provide a comprehensive experimental analysis of features re-use, which confirms the importance of hyper-dense connections in multi-modal representation learning. Our code is publicly available.

Benchmark on Automatic 6-month-old Infant Brain Segmentation Algorithms: The iSeg-2017 Challenge

Accurate segmentation of infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is an indispensable foundation for early studying of brain growth patterns and morphological changes in neurodevelopmental disorders. Nevertheless, in the isointense phase (approximately 6-9 months of age), due to inherent myelination and maturation process, WM and GM exhibit similar levels of intensity in both T1-weighted (T1w) and T2-weighted (T2w) MR images, making tissue segmentation very challenging. Despite many efforts were devoted to brain segmentation, only few studies have focused on the segmentation of 6-month infant brain images. With the idea of boosting methodological development in the community, iSeg-2017 challenge (http://iseg2017.web.unc.edu) provides a set of 6-month infant subjects with manual labels for training and testing the participating methods. Among the 21 automatic segmentation methods participating in iSeg-2017, we review the 8 top-ranked teams, in terms of Dice ratio, modified Hausdorff distance and average surface distance, and introduce their pipelines, implementations, as well as source codes. We further discuss limitations and possible future directions. We hope the dataset in iSeg-2017 and this review article could provide insights into methodological development for the community.

Transfer Learning for Image Segmentation by Combining Image Weighting and Kernel Learning

Many medical image segmentation methods are based on the supervised classification of voxels. Such methods generally perform well when provided with a training set that is representative of the test images to the segment. However, problems may arise when training and test data follow different distributions, for example, due to differences in scanners, scanning protocols, or patient groups. Under such conditions, weighting training images according to distribution similarity have been shown to greatly improve performance. However, this assumes that a part of the training data is representative of the test data; it does not make unrepresentative data more similar. We, therefore, investigate kernel learning as a way to reduce differences between training and test data and explore the added value of kernel learning for image weighting. We also propose a new image weighting method that minimizes maximum mean discrepancy (MMD) between training and test data, which enables the joint optimization of image weights and kernel. Experiments on brain tissue, white matter lesion, and hippocampus segmentation show that both kernel learning and image weighting, when used separately, greatly improve performance on heterogeneous data. Here, MMD weighting obtains similar performance to previously proposed image weighting methods. Combining image weighting and kernel learning, optimized either individually or jointly, can give a small additional improvement in performance.

Comparing fully automated state-of-the-art cerebellum parcellation from magnetic resonance images

The human cerebellum plays an essential role in motor control, is involved in cognitive function (i.e., attention, working memory, and language), and helps to regulate emotional responses. Quantitative in-vivo assessment of the cerebellum is important in the study of several neurological diseases including cerebellar ataxia, autism, and schizophrenia. Different structural subdivisions of the cerebellum have been shown to correlate with differing pathologies. To further understand these pathologies, it is helpful to automatically parcellate the cerebellum at the highest fidelity possible. In this paper, we coordinated with colleagues around the world to evaluate automated cerebellum parcellation algorithms on two clinical cohorts showing that the cerebellum can be parcellated to a high accuracy by newer methods. We characterize these various methods at four hierarchical levels: coarse (i.e., whole cerebellum and gross structures), lobe, subdivisions of the vermis, and the lobules. Due to the number of labels, the hierarchy of labels, the number of algorithms, and the two cohorts, we have restricted our analyses to the Dice measure of overlap. Under these conditions, machine learning based methods provide a collection of strategies that are efficient and deliver parcellations of a high standard across both cohorts, surpassing previous work in the area. In conjunction with the rank-sum computation, we identified an overall winning method.

Processing of structural neuroimaging data in young children: Bridging the gap between current practice and state-of-the-art methods

The structure of the brain is subject to very rapid developmental changes during early childhood. Pediatric studies based on Magnetic Resonance Imaging (MRI) over this age range have recently become more frequent, with the advantage of providing in vivo and non-invasive high-resolution images of the developing brain, toward understanding typical and atypical trajectories. However, it has also been demonstrated that application of currently standard MRI processing methods that have been developed with datasets from adults may not be appropriate for use with pediatric datasets. In this review, we examine the approaches currently used in MRI studies involving young children, including an overview of the rationale for new MRI processing methods that have been designed specifically for pediatric investigations. These methods are mainly related to the use of age-specific or 4D brain atlases, improved methods for quantifying and optimizing image quality, and provision for registration of developmental data obtained with longitudinal designs. The overall goal is to raise awareness of the existence of these methods and the possibilities for implementing them in developmental neuroimaging studies.

Repeatability and reproducibility of FreeSurfer, FSL-SIENAX and SPM brain volumetric measurements and the effect of lesion filling in multiple sclerosis

OBJECTIVES

To compare the cross-sectional robustness of commonly used volumetric software and effects of lesion filling in multiple sclerosis (MS).

METHODS

Nine MS patients (six females; age 38±13 years, disease duration 7.3±5.2 years) were scanned twice with repositioning on three MRI scanners (Siemens Aera 1.5T, Avanto 1.5T, Trio 3.0T) the same day. Volumetric T₁-weighted images were processed with FreeSurfer, FSL-SIENAX, SPM and SPM-CAT before and after 3D FLAIR lesion filling with LST. The whole-brain, grey matter (GM) and white matter (WM) volumes were calculated with and without normalisation to the intracranial volume or FSL-SIENAX scaling factor. Robustness was assessed using the coefficient of variation (CoV).

RESULTS

Variability in volumetrics was lower within than between scanners (CoV 0.17-0.96% vs. 0.65-5.0%, p<0.001). All software provided similarly robust segmentations of the brain volume on the same scanner (CoV 0.17-0.28%, p=0.076). Normalisation improved inter-scanner reproducibility in FreeSurfer and SPM-based methods, but the FSL-SIENAX scaling factor did not improve robustness. Generally, SPM-based methods produced the most consistent volumetrics, while FreeSurfer was more robust for WM volumes on different scanners. FreeSurfer had more robust normalised brain and GM volumes on different scanners than FSL-SIENAX (p=0.004). MS lesion filling changed the output of FSL-SIENAX, SPM and SPM-CAT but not FreeSurfer.

CONCLUSIONS

Consistent use of the same scanner is essential and normalisation to the intracranial volume is recommended for multiple scanners. Based on robustness, SPM-based methods are particularly suitable for cross-sectional volumetry. FreeSurfer poses a suitable alternative with WM segmentations less sensitive to MS lesions.

KEY POINTS

• The same scanner should be used for brain volumetry. If different scanners are used, the intracranial volume normalisation improves the FreeSurfer and SPM robustness (but not the FSL scaling factor). • FreeSurfer, FSL and SPM all provide robust measures of the whole brain volume on the same MRI scanner. SPM-based methods overall provide the most robust segmentations (except white matter segmentations on different scanners where FreeSurfer is more robust). • MS lesion filling with Lesion Segmentation Toolbox changes the output of FSL-SIENAX and SPM. FreeSurfer output is not affected by MS lesion filling since it already takes white matter hypointensities into account and is therefore particularly suitable for MS brain volumetry.

Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review

In recent years, deep convolutional neural networks (CNNs) have shown record-shattering performance in a variety of computer vision problems, such as visual object recognition, detection and segmentation. These methods have also been utilised in medical image analysis domain for lesion segmentation, anatomical segmentation and classification. We present an extensive literature review of CNN techniques applied in brain magnetic resonance imaging (MRI) analysis, focusing on the architectures, pre-processing, data-preparation and post-processing strategies available in these works. The aim of this study is three-fold. Our primary goal is to report how different CNN architectures have evolved, discuss state-of-the-art strategies, condense their results obtained using public datasets and examine their pros and cons. Second, this paper is intended to be a detailed reference of the research activity in deep CNN for brain MRI analysis. Finally, we present a perspective on the future of CNNs in which we hint some of the research directions in subsequent years.

Transfer learning by feature-space transformation: A method for Hippocampus segmentation across scanners

Many successful approaches in MR brain segmentation use supervised voxel classification, which requires manually labeled training images that are representative of the test images to segment. However, the performance of such methods often deteriorates if training and test images are acquired with different scanners or scanning parameters, since this leads to differences in feature representations between training and test data.

In this paper we propose a feature-space transformation (FST) to overcome such differences in feature representations. The proposed FST is derived from unlabeled images of a subject that was scanned with both the source and the target scan protocol. After an affine registration, these images give a mapping between source and target voxels in the feature space. This mapping is then used to map all training samples to the feature representation of the test samples.

We evaluated the benefit of the proposed FST on hippocampus segmentation. Experiments were performed on two datasets: one with relatively small differences between training and test images and one with large differences. In both cases, the FST significantly improved the performance compared to using only image normalization. Additionally, we showed that our FST can be used to improve the performance of a state-of-the-art patch-based-atlas-fusion technique in case of large differences between scanners.

•

We present a feature-space transformation for image segmentation across scanners.

•

This FST is trained on unlabeled images of subjects scanned with multiple scanners.

•

These are used to transform training samples to values observed in target samples.

•

The FST makes SVM hippocampus segmentation across scanners significantly better.

•

Our FST can also increase performance of patch-based fusion methods.

The association between brain volume, cortical brain infarcts, and physical frailty

Physical frailty is an age-associated syndrome of decreased reserve leading to vulnerability to physiological stressors and associated with negative outcomes. The underlying structural brain abnormalities of physical frailty are unclear. We investigated the association between brain volume, cortical brain infarcts, and physical frailty. In this multicenter study, 214 nondemented participants were classified as frail (n = 32), prefrail (n = 107), or nonfrail (n = 75) based on the Fried frailty phenotype. The associations between frailty and brain volumes and cortical brain infarcts were investigated by linear or logistic regression analyses. Participants in the frail group showed a lower total brain volume (−19.67 mL [95% confidence interval −37.84 to −1.50]) and lower gray matter volume (−12.19 mL [95% confidence interval −23.84 to −0.54]) compared to nonfrail participants. Frailty was associated with cortical brain infarcts [frail 16% [n = 5], prefrail 11% [n = 12], and nonfrail 3% [n = 2]). Reduced total brain volume and gray matter volume and increased cortical brain infarcts seem therefore to be part of the structural substrate of the physical frailty phenotype.

Measurement of Whole-Brain and Gray Matter Atrophy in Multiple Sclerosis: Assessment with MR Imaging

Purpose To compare available methods for whole-brain and gray matter (GM) atrophy estimation in multiple sclerosis (MS) in terms of repeatability (same magnetic resonance [MR] imaging unit) and reproducibility (different system/field strength) for their potential clinical applications. Materials and Methods The softwares ANTs-v1.9, CIVET-v2.1, FSL-SIENAX/SIENA-5.0.1, Icometrix-MSmetrix-1.7, and SPM-v12 were compared. This retrospective study, performed between March 2015 and March 2017, collected data from (a) eight simulated MR images and longitudinal data (2 weeks) from 10 healthy control subjects to assess the cross-sectional and longitudinal accuracy of atrophy measures, (b) test-retest MR images in 29 patients with MS acquired within the same day at different imaging unit field strengths/manufacturers to evaluate precision, and (c) longitudinal data (1 year) in 24 patients with MS for the agreement between methods. Tissue segmentation, image registration, and white matter (WM) lesion filling were also evaluated. Multiple paired t tests were used for comparisons. Results High values of accuracy (0.87-0.97) for whole-brain and GM volumes were found, with the lowest values for MSmetrix. ANTs showed the lowest mean error (0.02%) for whole-brain atrophy in healthy control subjects, with a coefficient of variation of 0.5%. SPM showed the smallest mean error (0.07%) and coefficient of variation (0.08%) for GM atrophy. Globally, good repeatability (P > .05) but poor reproducibility (P < .05) were found for all methods. WM lesion filling technique mainly affected ANTs, MSmetrix, and SPM results (P < .05). Conclusion From this comparison, it would be possible to select a software for atrophy measurement, depending on the requirements of the application (research center, clinical trial) and its goal (accuracy and repeatability or reproducibility). An improved reproducibility is required for clinical application. ^© RSNA, 2018 Online supplemental material is available for this article.

Fast CSF MRI for brain segmentation; Cross-validation by comparison with 3D T1-based brain segmentation methods

OBJECTIVE

In previous work we have developed a fast sequence that focusses on cerebrospinal fluid (CSF) based on the long T2 of CSF. By processing the data obtained with this CSF MRI sequence, brain parenchymal volume (BPV) and intracranial volume (ICV) can be automatically obtained. The aim of this study was to assess the precision of the BPV and ICV measurements of the CSF MRI sequence and to validate the CSF MRI sequence by comparison with 3D T1-based brain segmentation methods.

MATERIALS AND METHODS

Ten healthy volunteers (2 females; median age 28 years) were scanned (3T MRI) twice with repositioning in between. The scan protocol consisted of a low resolution (LR) CSF sequence (0:57min), a high resolution (HR) CSF sequence (3:21min) and a 3D T1-weighted sequence (6:47min). Data of the HR 3D-T1-weighted images were downsampled to obtain LR T1-weighted images (reconstructed imaging time: 1:59 min). Data of the CSF MRI sequences was automatically segmented using in-house software. The 3D T1-weighted images were segmented using FSL (5.0), SPM12 and FreeSurfer (5.3.0).

RESULTS

The mean absolute differences for BPV and ICV between the first and second scan for CSF LR (BPV/ICV: 12±9/7±4cc) and CSF HR (5±5/4±2cc) were comparable to FSL HR (9±11/19±23cc), FSL LR (7±4, 6±5cc), FreeSurfer HR (5±3/14±8cc), FreeSurfer LR (9±8, 12±10cc), and SPM HR (5±3/4±7cc), and SPM LR (5±4, 5±3cc). The correlation between the measured volumes of the CSF sequences and that measured by FSL, FreeSurfer and SPM HR and LR was very good (all Pearson's correlation coefficients >0.83, R2 .67-.97). The results from the downsampled data and the high-resolution data were similar.

CONCLUSION

Both CSF MRI sequences have a precision comparable to, and a very good correlation with established 3D T1-based automated segmentations methods for the segmentation of BPV and ICV. However, the short imaging time of the fast CSF MRI sequence is superior to the 3D T1 sequence on which segmentation with established methods is performed.

Automated versus manual segmentation of brain region volumes in former football players

Objectives

To determine whether or not automated FreeSurfer segmentation of brain regions considered important in repetitive head trauma can be analyzed accurately without manual correction.

Materials and methods

3 T MR neuroimaging was performed with automated FreeSurfer segmentation and manual correction of 11 brain regions in former National Football League (NFL) players with neurobehavioral symptoms and in control subjects. Automated segmentation and manually-corrected volumes were compared using an intraclass correlation coefficient (ICC). Linear mixed effects regression models were also used to estimate between-group mean volume comparisons and to correlate former NFL player brain volumes with neurobehavioral factors.

Results

Eighty-six former NFL players (55.2 ± 8.0 years) and 22 control subjects (57.0 ± 6.6 years) were evaluated. ICC was highly correlated between automated and manually-corrected corpus callosum volumes (0.911), lateral ventricular volumes (right 0.980, left 0.967), and amygdala-hippocampal complex volumes (right 0.713, left 0.731), but less correlated when amygdalae (right -0.170, left -0.090) and hippocampi (right 0.539, left 0.637) volumes were separately delineated and also less correlated for cingulate gyri volumes (right 0.639, left 0.351). Statistically significant differences between former NFL player and controls were identified in 8 of 11 regions with manual correction but in only 4 of 11 regions without such correction. Within NFL players, manually corrected brain volumes were significantly associated with 3 neurobehavioral factors, but a different set of 3 brain regions and neurobehavioral factor correlations was observed for brain region volumes segmented without manual correction.

Conclusions

Automated FreeSurfer segmentation of the corpus callosum, lateral ventricles, and amygdala-hippocampus complex may be appropriate for analysis without manual correction. However, FreeSurfer segmentation of the amygdala, hippocampus, and cingulate gyrus need further manual correction prior to performing group comparisons and correlations with neurobehavioral measures.

Comparison of image intensity, local, and multi-atlas priors in brain tissue classification

PURPOSE

Automated and accurate tissue classification in three-dimensional brain magnetic resonance images is essential in volumetric morphometry or as a preprocessing step for diagnosing brain diseases. However, noise, intensity in homogeneity, and partial volume effects limit the classification accuracy of existing methods. This paper provides a comparative study on the contributions of three commonly used image information priors for tissue classification in normal brains: image intensity, local, and multi-atlas priors.

METHODS

We compared the effectiveness of the three priors by comparing the four methods modeling them: K-Means (KM), KM combined with a Markov Random Field (KM-MRF), multi-atlas segmentation (MAS), and the combination of KM, MRF, and MAS (KM-MRF-MAS). The key parameters and factors in each of the four methods are analyzed, and the performance of all the models is compared quantitatively and qualitatively on both simulated and real data.

RESULTS

The KM-MRF-MAS model that combines the three image information priors performs best.

CONCLUSIONS

The image intensity prior is insufficient to generate reasonable results for a few images. Introducing local and multi-atlas priors results in improved brain tissue classification. This study provides a general guide on what image information priors can be used for effective brain tissue classification.

Evaluation of a deep learning approach for the segmentation of brain tissues and white matter hyperintensities of presumed vascular origin in MRI

Automatic segmentation of brain tissues and white matter hyperintensities of presumed vascular origin (WMH) in MRI of older patients is widely described in the literature. Although brain abnormalities and motion artefacts are common in this age group, most segmentation methods are not evaluated in a setting that includes these items. In the present study, our tissue segmentation method for brain MRI was extended and evaluated for additional WMH segmentation. Furthermore, our method was evaluated in two large cohorts with a realistic variation in brain abnormalities and motion artefacts. The method uses a multi-scale convolutional neural network with a T1-weighted image, a T2-weighted fluid attenuated inversion recovery (FLAIR) image and a T1-weighted inversion recovery (IR) image as input. The method automatically segments white matter (WM), cortical grey matter (cGM), basal ganglia and thalami (BGT), cerebellum (CB), brain stem (BS), lateral ventricular cerebrospinal fluid (lvCSF), peripheral cerebrospinal fluid (pCSF), and WMH. Our method was evaluated quantitatively with images publicly available from the MRBrainS13 challenge (n = 20), quantitatively and qualitatively in relatively healthy older subjects (n = 96), and qualitatively in patients from a memory clinic (n = 110). The method can accurately segment WMH (Overall Dice coefficient in the MRBrainS13 data of 0.67) without compromising performance for tissue segmentations (Overall Dice coefficients in the MRBrainS13 data of 0.87 for WM, 0.85 for cGM, 0.82 for BGT, 0.93 for CB, 0.92 for BS, 0.93 for lvCSF, 0.76 for pCSF). Furthermore, the automatic WMH volumes showed a high correlation with manual WMH volumes (Spearman's ρ = 0.83 for relatively healthy older subjects). In both cohorts, our method produced reliable segmentations (as determined by a human observer) in most images (relatively healthy/memory clinic: tissues 88%/77% reliable, WMH 85%/84% reliable) despite various degrees of brain abnormalities and motion artefacts. In conclusion, this study shows that a convolutional neural network-based segmentation method can accurately segment brain tissues and WMH in MR images of older patients with varying degrees of brain abnormalities and motion artefacts.

Diffusion MRI microstructure models with in vivo human brain Connectome data: results from a multi-group comparison

A large number of mathematical models have been proposed to describe the measured signal in diffusion-weighted (DW) magnetic resonance imaging (MRI). However, model comparison to date focuses only on specific subclasses, e.g. compartment models or signal models, and little or no information is available in the literature on how performance varies among the different types of models. To address this deficiency, we organized the 'White Matter Modeling Challenge' during the International Symposium on Biomedical Imaging (ISBI) 2015 conference. This competition aimed to compare a range of different kinds of models in their ability to explain a large range of measurable in vivo DW human brain data. Specifically, we assessed the ability of models to predict the DW signal accurately for new diffusion gradients and b values. We did not evaluate the accuracy of estimated model parameters, as a ground truth is hard to obtain. We used the Connectome scanner at the Massachusetts General Hospital, using gradient strengths of up to 300 mT/m and a broad set of diffusion times. We focused on assessing the DW signal prediction in two regions: the genu in the corpus callosum, where the fibres are relatively straight and parallel, and the fornix, where the configuration of fibres is more complex. The challenge participants had access to three-quarters of the dataset and their models were ranked on their ability to predict the remaining unseen quarter of the data. The challenge provided a unique opportunity for a quantitative comparison of diverse methods from multiple groups worldwide. The comparison of the challenge entries reveals interesting trends that could potentially influence the next generation of diffusion-based quantitative MRI techniques. The first is that signal models do not necessarily outperform tissue models; in fact, of those tested, tissue models rank highest on average. The second is that assuming a non-Gaussian (rather than purely Gaussian) noise model provides little improvement in prediction of unseen data, although it is possible that this may still have a beneficial effect on estimated parameter values. The third is that preprocessing the training data, here by omitting signal outliers, and using signal-predicting strategies, such as bootstrapping or cross-validation, could benefit the model fitting. The analysis in this study provides a benchmark for other models and the data remain available to build up a more complete comparison in the future.

Multimodal MEMPRAGE, FLAIR, and [Formula: see text] Segmentation to Resolve Dura and Vessels from Cortical Gray Matter

While widely in use in automated segmentation approaches for the detection of group differences or of changes associated with continuous predictors in gray matter volume, T1-weighted images are known to represent dura and cortical vessels with signal intensities similar to those of gray matter. By considering multiple signal sources at once, multimodal segmentation approaches may be able to resolve these different tissue classes and address this potential confound. We explored here the simultaneous use of FLAIR and apparent transverse relaxation rates (a signal related to T2* relaxation maps and having similar contrast) with T1-weighted images. Relative to T1-weighted images alone, multimodal segmentation had marked positive effects on 1. the separation of gray matter from dura, 2. the exclusion of vessels from the gray matter compartment, and 3. the contrast with extracerebral connective tissue. While obtainable together with the T1-weighted images without increasing scanning times, apparent transverse relaxation rates were less effective than added FLAIR images in providing the above mentioned advantages. FLAIR images also improved the detection of cortical matter in areas prone to susceptibility artifacts in standard MPRAGE T1-weighted images, while the addition of transverse relaxation maps exacerbated the effect of these artifacts on segmentation. Our results confirm that standard MPRAGE segmentation may overestimate gray matter volume by wrongly assigning vessels and dura to this compartment and show that multimodal approaches may greatly improve the specificity of cortical segmentation. Since multimodal segmentation is easily implemented, these benefits are immediately available to studies focusing on translational applications of structural imaging.