A review on brain structures segmentation in magnetic resonance imaging

Assessment of U-Net in the segmentation of short tracts: Transferring to clinical MRI routine

Accurately studying structural connectivity requires precise tract segmentation strategies. The U-Net network has been widely recognized for its exceptional capacity in image segmentation tasks and provides remarkable results in large tract segmentation when high-quality diffusion-weighted imaging (DWI) data are used. However, short tracts, which are associated with various neurological diseases, pose specific challenges, particularly when high-quality DWI data acquisition within clinical settings is concerned. Here, we aimed to evaluate the U-Net network ability to segment short tracts by using DWI data acquired in different experimental conditions. To this end, we conducted three types of training experiments involving 350 healthy subjects and 11 white matter tracts, including the anterior, posterior, and hippocampal commissure, fornix, and uncinated fasciculus. In the first experiment, the model was exclusively trained with high-quality data of the Human Connectome Project (HCP) dataset. The second experiment focused on images of healthy subjects acquired from a local hospital dataset, representing a typical clinical routine acquisition. In the third experiment, a hybrid training approach was employed, combining data of the HCP and local hospital datasets. Then, the best model was also tested in unseen DWIs of 10 epilepsy patients of the local hospital and 10 healthy subjects acquired on a scanner from another company. The outcomes of the third experiment demonstrated a notable enhancement in performance when contrasted with the preceding trials. Specifically, the short tracts within the local hospital dataset achieved Dice scores ranging between 0.60 and 0.65. Similar intervals were obtained with HCP data in the first experiment, and a substantial improvement compared to the scores between 0.37 and 0.50 obtained with the local hospital dataset at the same experiment. This improvement persisted when the method was applied to diverse scenarios, including different scanner acquisitions and epilepsy patients. These results indicate that combining datasets from different sources, coupled with resolution standardization strengthens the neural network ability to generalize predictions across a spectrum of datasets. Nevertheless, short tract segmentation performance is intricately linked to the training composition, to validation, and to testing data. Moreover, curved tracts have intricate structural nature, which adds complexities to their segmenting. Although the network training approach tested herein has provided promising results, caution must be taken when extrapolating its application to datasets acquired under distinct experimental conditions, even in the case of higher-quality data or analysis of long or short tracts.

Unsupervised model for structure segmentation applied to brain computed tomography

This article presents an unsupervised method for segmenting brain computed tomography scans. The proposed methodology involves image feature extraction and application of similarity and continuity constraints to generate segmentation maps of the anatomical head structures. Specifically designed for real-world datasets, this approach applies a spatial continuity scoring function tailored to the desired number of structures. The primary objective is to assist medical experts in diagnosis by identifying regions with specific abnormalities. Results indicate a simplified and accessible solution, reducing computational effort, training time, and financial costs. Moreover, the method presents potential for expediting the interpretation of abnormal scans, thereby impacting clinical practice. This proposed approach might serve as a practical tool for segmenting brain computed tomography scans, and make a significant contribution to the analysis of medical images in both research and clinical settings.

Preparation, evaluation and application of MRI detectable sunitinib-loaded calcium alginate/poly(acrylic acid) hydrogel microspheres

Transcatheter arterial chemoembolization (TACE) is an effective method for the treatment of unresectable hepatocellular carcinoma. Although many embolic agents have been developed in TACE, there are few ideal embolic agents that combine drug loading, imaging properties and vessel embolization. Here, we developed novel magnetic embolic microspheres that could simultaneously load sunitinib malate (SU), be detected by magnetic resonance imaging (MRI) and block blood vessels. Calcium alginate/poly (acrylic acid) hydrogel microspheres (CA/PAA-MDMs) with superparamagnetic iron oxide nanoparticles (SPIONs) modified by citric acid were prepared by a drip and photopolymerization method. The embolization and imaging properties of CA/PAA-MDMs were evaluated through a series of experiments such as morphology, X-ray diffraction and X-ray photoelectron spectroscopy, magnetic responsiveness analysis, elasticity, cytotoxicity, hemolysis test, in vitro MRI evaluation, rabbit ear embolization and histopathology. In addition, the ability of drug loading and drug release of CA/PAA-MDMs were investigated by using sunitinib (SU) as the model drug. In conclusion, CA/PAA-MDMs showed outstanding drug loading capability, excellent imaging property and embolization effect, which would be expected to be used as a potential biodegradable embolic agent in the clinical interventional therapy.

GPU-Based Parallel Processing Techniques for Enhanced Brain Magnetic Resonance Imaging Analysis: A Review of Recent Advances

The approach of using more than one processor to compute in order to overcome the complexity of different medical imaging methods that make up an overall job is known as GPU (graphic processing unit)-based parallel processing. It is extremely important for several medical imaging techniques such as image classification, object detection, image segmentation, registration, and content-based image retrieval, since the GPU-based parallel processing approach allows for time-efficient computation by a software, allowing multiple computations to be completed at once. On the other hand, a non-invasive imaging technology that may depict the shape of an anatomy and the biological advancements of the human body is known as magnetic resonance imaging (MRI). Implementing GPU-based parallel processing approaches in brain MRI analysis with medical imaging techniques might be helpful in achieving immediate and timely image capture. Therefore, this extended review (the extension of the IWBBIO2023 conference paper) offers a thorough overview of the literature with an emphasis on the expanding use of GPU-based parallel processing methods for the medical analysis of brain MRIs with the imaging techniques mentioned above, given the need for quicker computation to acquire early and real-time feedback in medicine. Between 2019 and 2023, we examined the articles in the literature matrix that include the tasks, techniques, MRI sequences, and processing results. As a result, the methods discussed in this review demonstrate the advancements achieved until now in minimizing computing runtime as well as the obstacles and problems still to be solved in the future.

Reliability of brain atrophy measurements in multiple sclerosis using MRI: an assessment of six freely available software packages for cross-sectional analyses

PURPOSE

Volume measurement using MRI is important to assess brain atrophy in multiple sclerosis (MS). However, differences between scanners, acquisition protocols, and analysis software introduce unwanted variability of volumes. To quantify theses effects, we compared within-scanner repeatability and between-scanner reproducibility of three different MR scanners for six brain segmentation methods.

METHODS

Twenty-one people with MS underwent scanning and rescanning on three 3 T MR scanners (GE MR750, Philips Ingenuity, Toshiba Vantage Titan) to obtain 3D T1-weighted images. FreeSurfer, FSL, SAMSEG, FastSurfer, CAT-12, and SynthSeg were used to quantify brain, white matter and (deep) gray matter volumes both from lesion-filled and non-lesion-filled 3D T1-weighted images. We used intra-class correlation coefficient (ICC) to quantify agreement; repeated-measures ANOVA to analyze systematic differences; and variance component analysis to quantify the standard error of measurement (SEM) and smallest detectable change (SDC).

RESULTS

For all six software, both between-scanner agreement (ICCs ranging 0.4-1) and within-scanner agreement (ICC range: 0.6-1) were typically good, and good to excellent (ICC > 0.7) for large structures. No clear differences were found between filled and non-filled images. However, gray and white matter volumes did differ systematically between scanners for all software (p < 0.05). Variance component analysis yielded within-scanner SDC ranging from 1.02% (SAMSEG, whole-brain) to 14.55% (FreeSurfer, CSF); and between-scanner SDC ranging from 4.83% (SynthSeg, thalamus) to 29.25% (CAT12, thalamus).

CONCLUSION

Volume measurements of brain, GM and WM showed high repeatability, and high reproducibility despite substantial differences between scanners. Smallest detectable change was high, especially between different scanners, which hampers the clinical implementation of atrophy measurements.

Applying Image-Based Food-Recognition Systems on Dietary Assessment: A Systematic Review

Dietary assessment can be crucial for the overall well-being of humans and, at least in some instances, for the prevention and management of chronic, life-threatening diseases. Recall and manual record-keeping methods for food-intake monitoring are available, but often inaccurate when applied for a long period of time. On the other hand, automatic record-keeping approaches that adopt mobile cameras and computer vision methods seem to simplify the process and can improve current human-centric diet-monitoring methods. Here we present an extended critical literature overview of image-based food-recognition systems (IBFRS) combining a camera of the user's mobile device with computer vision methods and publicly available food datasets (PAFDs). In brief, such systems consist of several phases, such as the segmentation of the food items on the plate, the classification of the food items in a specific food category, and the estimation phase of volume, calories, or nutrients of each food item. A total of 159 studies were screened in this systematic review of IBFRS. A detailed overview of the methods adopted in each of the 78 included studies of this systematic review of IBFRS is provided along with their performance on PAFDs. Studies that included IBFRS without presenting their performance in at least 1 of the above-mentioned phases were excluded. Among the included studies, 45 (58%) studies adopted deep learning methods and especially convolutional neural networks (CNNs) in at least 1 phase of the IBFRS with input PAFDs. Among the implemented techniques, CNNs outperform all other approaches on the PAFDs with a large volume of data, since the richness of these datasets provides adequate training resources for such algorithms. We also present evidence for the benefits of application of IBFRS in professional dietetic practice. Furthermore, challenges related to the IBFRS presented here are also thoroughly discussed along with future directions.

DBSegment: Fast and robust segmentation of deep brain structures considering domain generalization

Segmenting deep brain structures from magnetic resonance images is important for patient diagnosis, surgical planning, and research. Most current state-of-the-art solutions follow a segmentation-by-registration approach, where subject magnetic resonance imaging (MRIs) are mapped to a template with well-defined segmentations. However, registration-based pipelines are time-consuming, thus, limiting their clinical use. This paper uses deep learning to provide a one-step, robust, and efficient deep brain segmentation solution directly in the native space. The method consists of a preprocessing step to conform all MRI images to the same orientation, followed by a convolutional neural network using the nnU-Net framework. We use a total of 14 datasets from both research and clinical collections. Of these, seven were used for training and validation and seven were retained for testing. We trained the network to segment 30 deep brain structures, as well as a brain mask, using labels generated from a registration-based approach. We evaluated the generalizability of the network by performing a leave-one-dataset-out cross-validation, and independent testing on unseen datasets. Furthermore, we assessed cross-domain transportability by evaluating the results separately on different domains. We achieved an average dice score similarity of 0.89 ± 0.04 on the test datasets when compared to the registration-based gold standard. On our test system, the computation time decreased from 43 min for a reference registration-based pipeline to 1.3 min. Our proposed method is fast, robust, and generalizes with high reliability. It can be extended to the segmentation of other brain structures. It is publicly available on GitHub, and as a pip package for convenient usage.

Review on Hybrid Segmentation Methods for Identification of Brain Tumor in MRI

Modalities like MRI give information about organs and highlight diseases. Organ information is visualized in intensities. The segmentation method plays an important role in the identification of the region of interest (ROI). The ROI can be segmented from the image using clustering, features, and region extraction. Segmentation can be performed in steps; firstly, the region is extracted from the image. Secondly, feature extraction performed, and better features are selected. They can be shape, texture, or intensity. Thirdly, clustering segments the shape of tumor, tumor has specified shape, and shape is detected by feature. Clustering consists of FCM, K-means, FKM, and their hybrid. To support the segmentation, we conducted three studies (region extraction, feature, and clustering) which are discussed in the first line of this review paper. All these studies are targeting MRI as a modality. MRI visualization proved to be more accurate for the identification of diseases compared with other modalities. Information of the modality is compromised due to low pass image. In MRI Images, the tumor intensities are variable in tumor areas as well as in tumor boundaries.

Personalized auto-segmentation for magnetic resonance imaging guided adaptive radiotherapy of prostate cancer

PURPOSE

Fast and accurate delineation of organs on treatment-fraction images is critical in magnetic resonance imaging-guided adaptive radiotherapy (MRIgART). This study proposes a personalized auto-segmentation (AS) framework to assist online delineation of prostate cancer using MRIgART.

METHODS

Image data from 26 patients diagnosed with prostate cancer and treated using hypofractionated MRIgART (5 fractions per patient) were collected retrospectively. Daily pretreatment T2-weighted MRI was performed using a 1.5-T MRI system integrated into a Unity MR-linac. First-fraction image and contour data from 16 patients (80 image-sets) were used to train the population AS model, and the remaining 10 patients composed the test set. The proposed personalized AS framework contained two main steps. First, a convolutional neural network was employed to train the population model using the training set. Second, for each test patient, the population model was progressively fine-tuned with manually checked delineations of the patient's current and previous fractions to obtain a personalized model that was applied to the next fraction.

RESULTS

Compared with the population model, the personalized models substantially improved the mean Dice similarity coefficient from 0.79 to 0.93 for the prostate clinical target volume (CTV), 0.91 to 0.97 for the bladder, 0.82 to 0.92 for the rectum, 0.91 to 0.93 for the femoral heads, respectively.

CONCLUSIONS

The proposed method can achieve accurate segmentation and potentially shorten the overall online delineation time of MRIgART. This article is protected by copyright. All rights reserved.

Hyperconnected Openings Codified in a Max Tree Structure: An Application for Skull-Stripping in Brain MRI T1

This article presents two procedures involving a maximal hyperconnected function and a hyperconnected lower leveling to segment the brain in a magnetic resonance imaging T1 weighted using new openings on a max-tree structure. The openings are hyperconnected and are viscous transformations. The first procedure considers finding the higher hyperconnected maximum by using an increasing criterion that plays a central role during segmentation. The second procedure utilizes hyperconnected lower leveling, which acts as a marker, controlling the reconstruction process into the mask. As a result, the proposal allows an efficient segmentation of the brain to be obtained. In total, 38 magnetic resonance T1-weighted images obtained from the Internet Brain Segmentation Repository are segmented. The Jaccard and Dice indices are computed, compared, and validated with the efficiency of the Brain Extraction Tool software and other algorithms provided in the literature.

Effect of head motion-induced artefacts on the reliability of deep learning-based whole-brain segmentation

Due to their robustness and speed, recently developed deep learning-based methods have the potential to provide a faster and hence more scalable alternative to more conventional neuroimaging analysis pipelines in terms of whole-brain segmentation based on magnetic resonance (MR) images. These methods were also shown to have higher test-retest reliability, raising the possibility that they could also exhibit superior head motion tolerance. We investigated this by comparing the effect of head motion-induced artifacts in structural MR images on the consistency of segmentation performed by FreeSurfer and recently developed deep learning-based methods to a similar extent. We used state-of-the art neural network models (FastSurferCNN and Kwyk) and developed a new whole-brain segmentation pipeline (ReSeg) to examine whether reliability depends on choice of deep learning method. Structural MRI scans were collected from 110 participants under rest and active head motion and were evaluated for image quality by radiologists. Compared to FreeSurfer, deep learning-based methods provided more consistent segmentations across different levels of image quality, suggesting that they also have the advantage of providing more reliable whole-brain segmentations of MR images corrupted by motion-induced artifacts, and provide evidence for their practical applicability in the study of brain structural alterations in health and disease.

Towards an Architecture of a Multi-purpose, User-Extendable Reference Human Brain Atlas

Human brain atlas development is predominantly research-oriented and the use of atlases in clinical practice is limited. Here I introduce a new definition of a reference human brain atlas that serves education, research and clinical applications, and is extendable by its user. Subsequently, an architecture of a multi-purpose, user-extendable reference human brain atlas is proposed and its implementation discussed. The human brain atlas is defined as a vehicle to gather, present, use, share, and discover knowledge about the human brain with highly organized content, tools enabling a wide range of its applications, massive and heterogeneous knowledge database, and means for content and knowledge growing by its users. The proposed architecture determines major components of the atlas, their mutual relationships, and functional roles. It contains four functional units, core cerebral models, knowledge database, research and clinical data input and conversion, and toolkit (supporting processing, content extension, atlas individualization, navigation, exploration, and display), all united by a user interface. Each unit is described in terms of its function, component modules and sub-modules, data handling, and implementation aspects. This novel architecture supports brain knowledge gathering, presentation, use, sharing, and discovery and is broadly applicable and useful in student- and educator-oriented neuroeducation for knowledge presentation and communication, research for knowledge acquisition, aggregation and discovery, and clinical applications in decision making support for prevention, diagnosis, treatment, monitoring, and prediction. It establishes a backbone for designing and developing new, multi-purpose and user-extendable brain atlas platforms, serving as a potential standard across labs, hospitals, and medical schools.

Whole brain segmentation with full volume neural network

Whole brain segmentation is an important neuroimaging task that segments the whole brain volume into anatomically labeled regions-of-interest. Convolutional neural networks have demonstrated good performance in this task. Existing solutions, usually segment the brain image by classifying the voxels, or labeling the slices or the sub-volumes separately. Their representation learning is based on parts of the whole volume whereas their labeling result is produced by aggregation of partial segmentation. Learning and inference with incomplete information could lead to sub-optimal final segmentation result. To address these issues, we propose to adopt a full volume framework, which feeds the full volume brain image into the segmentation network and directly outputs the segmentation result for the whole brain volume. The framework makes use of complete information in each volume and can be implemented easily. An effective instance in this framework is given subsequently. We adopt the 3D high-resolution network (HRNet) for learning spatially fine-grained representations and the mixed precision training scheme for memory-efficient training. Extensive experiment results on a publicly available 3D MRI brain dataset show that our proposed model advances the state-of-the-art methods in terms of segmentation performance.

A comparison of automated atrophy measures across the frontotemporal dementia spectrum: Implications for trials

Background

Frontotemporal dementia (FTD) is a common cause of young onset dementia, and whilst there are currently no treatments, there are several promising candidates in development and early phase trials. Comprehensive investigations of neuroimaging markers of disease progression across the full spectrum of FTD disorders are lacking and urgently needed to facilitate these trials.

Objective

To investigate the comparative performance of multiple automated segmentation and registration pipelines used to quantify longitudinal whole-brain atrophy across the clinical, genetic and pathological subgroups of FTD, in order to inform upcoming trials about suitable neuroimaging-based endpoints.

Methods

Seventeen fully automated techniques for extracting whole-brain atrophy measures were applied and directly compared in a cohort of 226 participants who had undergone longitudinal structural 3D T1-weighted imaging. Clinical diagnoses were behavioural variant FTD (n = 56) and primary progressive aphasia (PPA, n = 104), comprising semantic variant PPA (n = 38), non-fluent variant PPA (n = 42), logopenic variant PPA (n = 18), and PPA-not otherwise specified (n = 6). 49 of these patients had either a known pathogenic mutation or postmortem confirmation of their underlying pathology. 66 healthy controls were included for comparison. Sample size estimates to detect a 30% reduction in atrophy (80% power; 0.05 significance) were computed to explore the relative feasibility of these brain measures as surrogate markers of disease progression and their ability to detect putative disease-modifying treatment effects.

Results

Multiple automated techniques showed great promise, detecting significantly increased rates of whole-brain atrophy (p<0.001) and requiring sample sizes of substantially less than 100 patients per treatment arm. Across the different FTD subgroups, direct measures of volume change consistently outperformed their indirect counterparts, irrespective of the initial segmentation quality. Significant differences in performance were found between both techniques and patient subgroups, highlighting the importance of informed biomarker choice based on the patient population of interest.

Conclusion

This work expands current knowledge and builds on the limited longitudinal investigations currently available in FTD, as well as providing valuable information about the potential of fully automated neuroimaging biomarkers for sporadic and genetic FTD trials.

Performance improvement of weakly supervised fully convolutional networks by skip connections for brain structure segmentation

PURPOSE

For the planning and navigation of neurosurgery, we have developed a fully convolutional network (FCN)-based method for brain structure segmentation on magnetic resonance (MR) images. The capability of an FCN depends on the quality of the training data (i.e., raw data and annotation data) and network architectures. The improvement of annotation quality is a significant concern because it requires much labor for labeling organ regions. To address this problem, we focus on skip connection architectures and reveal which skip connections are effective for training FCNs using sparsely annotated brain images.

METHODS

We tested 2D FCN architectures with four different types of skip connections. The first was a U-Net architecture with horizontal skip connections that transfer feature maps at the same scale from the encoder to the decoder. The second was a U-Net++ architecture with dense convolution layers and dense horizontal skip connections. The third was a full-resolution residual network (FRRN) architecture with vertical skip connections that pass feature maps between each downsampled scale path and the full-resolution scale path. The last one was a hybrid architecture with a combination of horizontal and vertical skip connections. We validated the effect of skip connections on medical image segmentation from sparse annotation based on these four FCN architectures, which were trained under the same conditions.

RESULTS

For multiclass segmentation of the cerebrum, cerebellum, brainstem, and blood vessels from sparsely annotated MR images, we performed a comparative evaluation of segmentation performance among the above four FCN approaches: U-Net, U-Net++, FRRN, and hybrid architectures. The experimental results show that the horizontal skip connections in the U-Net architectures were effective for the segmentation of larger sized objects, whereas the vertical skip connections in the FRRN architecture improved the segmentation of smaller sized objects. The hybrid architecture with both horizontal and vertical skip connections achieved the best results of the four FCN architectures. We then performed an ablation study to explore which skip connections in the FRRN architecture contributed to the improved segmentation of blood vessels. In the ablation study, we compared the segmentation performance between architectures with a horizontal path (HP), an HP and vertical up paths (HP+VUPs), an HP and vertical down paths (HP+VDPs), and an HP and vertical up and down paths (FRRN). We found that the vertical up paths were effective in improving the segmentation of smaller sized objects.

CONCLUSIONS

This paper investigated which skip connection architectures were effective for multiclass brain segmentation from sparse annotation. Consequently, using vertical skip connections with horizontal skip connections allowed FCNs to improve segmentation performance.

What is new in computer vision and artificial intelligence in medical image analysis applications

Computer vision and artificial intelligence applications in medicine are becoming increasingly important day by day, especially in the field of image technology. In this paper we cover different artificial intelligence advances that tackle some of the most important worldwide medical problems such as cardiology, cancer, dermatology, neurodegenerative disorders, respiratory problems, and gastroenterology. We show how both areas have resulted in a large variety of methods that range from enhancement, detection, segmentation and characterizations of anatomical structures and lesions to complete systems that automatically identify and classify several diseases in order to aid clinical diagnosis and treatment. Different imaging modalities such as computer tomography, magnetic resonance, radiography, ultrasound, dermoscopy and microscopy offer multiple opportunities to build automatic systems that help medical diagnosis, taking advantage of their own physical nature. However, these imaging modalities also impose important limitations to the design of automatic image analysis systems for diagnosis aid due to their inherent characteristics such as signal to noise ratio, contrast and resolutions in time, space and wavelength. Finally, we discuss future trends and challenges that computer vision and artificial intelligence must face in the coming years in order to build systems that are able to solve more complex problems that assist medical diagnosis.

Fully Automatic Adaptive Meshing Based Segmentation of the Ventricular System for Augmented Reality Visualization and Navigation

OBJECTIVE

Effective image segmentation of cerebral structures is fundamental to 3-dimensional techniques such as augmented reality. To be clinically viable, segmentation algorithms should be fully automatic and easily integrated in existing digital infrastructure. We created a fully automatic adaptive-meshing-based segmentation system for T1-weighted magnetic resonance images (MRI) to automatically segment the complete ventricular system, running in a cloud-based environment that can be accessed on an augmented reality device. This study aims to assess the accuracy and segmentation time of the system by comparing it to a manually segmented ground truth dataset.

METHODS

A ground truth (GT) dataset of 46 contrast-enhanced and non-contrast-enhanced T1-weighted MRI scans was manually segmented. These scans also were uploaded to our system to create a machine-segmented (MS) dataset. The GT data were compared with the MS data using the Sørensen-Dice similarity coefficient and 95% Hausdorff distance to determine segmentation accuracy. Furthermore, segmentation times for all GT and MS segmentations were measured.

RESULTS

Automatic segmentation was successful for 45 (98%) of 46 cases. Mean Sørensen-Dice similarity coefficient score was 0.83 (standard deviation [SD] = 0.08) and mean 95% Hausdorff distance was 19.06 mm (SD = 11.20). Segmentation time was significantly longer for the GT group (mean = 14405 seconds, SD = 7089) when compared with the MS group (mean = 1275 seconds, SD = 714) with a mean difference of 13,130 seconds (95% confidence interval 10,130-16,130).

CONCLUSIONS

The described adaptive meshing-based segmentation algorithm provides accurate and time-efficient automatic segmentation of the ventricular system from T1 MRI scans and direct visualization of the rendered surface models in augmented reality.

Loss Weightings for Improving Imbalanced Brain Structure Segmentation Using Fully Convolutional Networks

Brain structure segmentation on magnetic resonance (MR) images is important for various clinical applications. It has been automatically performed by using fully convolutional networks. However, it suffers from the class imbalance problem. To address this problem, we investigated how loss weighting strategies work for brain structure segmentation tasks with different class imbalance situations on MR images. In this study, we adopted segmentation tasks of the cerebrum, cerebellum, brainstem, and blood vessels from MR cisternography and angiography images as the target segmentation tasks. We used a U-net architecture with cross-entropy and Dice loss functions as a baseline and evaluated the effect of the following loss weighting strategies: inverse frequency weighting, median inverse frequency weighting, focal weighting, distance map-based weighting, and distance penalty term-based weighting. In the experiments, the Dice loss function with focal weighting showed the best performance and had a high average Dice score of 92.8% in the binary-class segmentation tasks, while the cross-entropy loss functions with distance map-based weighting achieved the Dice score of up to 93.1% in the multi-class segmentation tasks. The results suggested that the distance map-based and the focal weightings could boost the performance of cross-entropy and Dice loss functions in class imbalanced segmentation tasks, respectively.

Fuzzy C-Means Clustering Algorithm-Based Magnetic Resonance Imaging Image Segmentation for Analyzing the Effect of Edaravone on the Vascular Endothelial Function in Patients with Acute Cerebral Infarction

This paper aimed to discuss the denoising ability of magnetic resonance imaging (MRI) images based on fuzzy C-means clustering (FCM) algorithm and the influence of Butylphthalide combined with Edaravone treatment on nerve function and vascular endothelial function in patients with acute cerebral infarction (ACI). Based on FCM algorithm, Markov Random Field (MRF) model algorithm was introduced to obtain a novel algorithm (NFCM), which was compared with FCM and MRF algorithm in terms of misclassification rate (MCR) and difference of Kappa index (KI). 90 patients with ACI diagnosed in hospital from December 2018 to December 2019 were selected as subjects, who were divided into combined treatment group (conventional treatment + Edaravone + Butylphthalide) and Edaravone group (conventional treatment + Edaravone) randomly, each consisting of 45 cases. The National Institutes of Health Stroke Scale (NIHSS) score and endothelial function index level such as plasma nitric oxide (NO), human endothelin-1 (ET-1), and vascular endothelial cell growth factor (VEGF) were compared before and after treatment between the two groups. The results showed that the MCR of NFCM was evidently inferior to FCM and MRF, and the KI was notably higher relative to the other two algorithms. After treatment, the NIHSS score of the combined treatment group was (9.09 ± 1.86) points and that of Edaravone group was (14.97 ± 3.44) points, with evident difference between the two groups (P < 0.05). After treatment, the NO of the combined treatment was (54.63 ± 4.85), and that of Edaravone group was (41.54 ± 5.27), which was considerably different (P < 0.01), and the VEGF and ET-1 of combined treatment group were greatly inferior to Edaravone group (P < 0.01). It was revealed that the novel algorithm based on FCM can obtain more favorable quality and segmentation accuracy of MRI images. Moreover, Butylphthalide combined with Edaravone treatment can effectively improve nerve function, vascular endothelial function, and short-term prognosis in ACI, which was safe and worthy of clinical adoption.

Beware of white matter hyperintensities causing systematic errors in FreeSurfer gray matter segmentations!

Volumetric estimates of subcortical and cortical structures, extracted from T1-weighted MRIs, are widely used in many clinical and research applications. Here, we investigate the impact of the presence of white matter hyperintensities (WMHs) on FreeSurfer gray matter (GM) structure volumes and its possible bias on functional relationships. T1-weighted images from 1,077 participants (4,321 timepoints) from the Alzheimer's Disease Neuroimaging Initiative were processed with FreeSurfer version 6.0.0. WMHs were segmented using a previously validated algorithm on either T2-weighted or Fluid-attenuated inversion recovery images. Mixed-effects models were used to assess the relationships between overlapping WMHs and GM structure volumes and overall WMH burden, as well as to investigate whether such overlaps impact associations with age, diagnosis, and cognitive performance. Participants with higher WMH volumes had higher overlaps with GM volumes of bilateral caudate, cerebral cortex, putamen, thalamus, pallidum, and accumbens areas (p < .0001). When not corrected for WMHs, caudate volumes increased with age (p < .0001) and were not different between cognitively healthy individuals and age-matched probable Alzheimer's disease patients. After correcting for WMHs, caudate volumes decreased with age (p < .0001), and Alzheimer's disease patients had lower caudate volumes than cognitively healthy individuals (p < .01). Uncorrected caudate volume was not associated with ADAS13 scores, whereas corrected lower caudate volumes were significantly associated with poorer cognitive performance (p < .0001). Presence of WMHs leads to systematic inaccuracies in GM segmentations, particularly for the caudate, which can also change clinical associations. While specifically measured for the Freesurfer toolkit, this problem likely affects other algorithms.

A Robust and Accurate Deep-learning-based Method for the Segmentation of Subcortical Brain: Cross-dataset Evaluation of Generalization Performance

Purpose:

To analyze subcortical brain volume more reliably, we propose a deep learning segmentation method of subcortical brain based on magnetic resonance imaging (MRI) having high generalization performance, accuracy, and robustness.

Methods:

First, local images of three-dimensional (3D) bounding boxes were extracted for seven subcortical structures (thalamus, putamen, caudate, pallidum, hippocampus, amygdala, and accumbens) from a whole brain MR image as inputs to the neural network. Second, dilated convolution layers, which input information of variable scope, were introduced to the blocks that make up the neural network. These blocks were connected in parallel to simultaneously process global and local information obtained by the dilated convolution layers. To evaluate generalization performance, different datasets were used for training and testing sessions (cross-dataset evaluation) because subcortical brain segmentation in clinical analysis is assumed to be applied to unknown datasets.

Results:

The proposed method showed better generalization performance that can obtain stable accuracy for all structures, whereas the state-of-the-art deep learning method obtained extremely low accuracy for some structures. The proposed method performed segmentation for all samples without failing with significantly higher accuracy (P < 0.005) than conventional methods such as 3D U-Net, FreeSurfer, and Functional Magnetic Resonance Imaging of the Brain’s (FMRIB’s) Integrated Registration and Segmentation Tool in the FMRIB Software Library (FSL-FIRST). Moreover, when applying this proposed method to larger datasets, segmentation was robustly performed for all samples without producing segmentation results on the areas that were apparently different from anatomically relevant areas. On the other hand, FSL-FIRST produced segmentation results on the area that were apparently and largely different from the anatomically relevant area for about one-third to one-fourth of the datasets.

Conclusion:

The cross-dataset evaluation showed that the proposed method is superior to existing methods in terms of generalization performance, accuracy, and robustness.

Quantitative comparison of subcortical and ventricular volumetry derived from MPRAGE and MP2RAGE images using different brain morphometry software

OBJECTIVE

In brain volume assessment with MR imaging, it is of interest to know the effects of the pulse sequence and software used, to determine whether they provide equivalent data. The aim of this study was to compare cross-sectional volumes of subcortical and ventricular structures and their repeatability derived from MP2RAGE and MPRAGE images using MorphoBox, and FIRST or ALVIN.

MATERIALS AND METHODS

MPRAGE and MP2RAGE T1-weighted images were obtained from 24 healthy volunteers. Back-to-back scans were performed in 12 of them. Volumes, coefficients of variation, concordance, and correlations were determined.

RESULTS

Significant differences were found for volumes derived from MorphoBox and FIRST. Ventricular volumes determined by MorphoBox and ALVIN were similar. Differences between volumes obtained using MPRAGE and MP2RAGE were significant for a few regions. Coefficients of variation, ranged from 0.2 to 9.1%, showed a significant inverse correlation with the mean volume. There was a correlation between volume measures, but agreement was rated as poor for most regions.

CONCLUSION

MP2RAGE sequences and MorphoBox are valid options for assessing subcortical and ventricular volumes, in the same way as MPRAGE and FIRST or ALVIN, accepted tools for clinical research. However, caution is needed when comparing volumes obtained with different tools.

Artificial Intelligence in magnetic Resonance guided Radiotherapy: Medical and physical considerations on state of art and future perspectives

Over the last years, technological innovation in Radiotherapy (RT) led to the introduction of Magnetic Resonance-guided RT (MRgRT) systems. Due to the higher soft tissue contrast compared to on-board CT-based systems, MRgRT is expected to significantly improve the treatment in many situations. MRgRT systems may extend the management of inter- and intra-fraction anatomical changes, offering the possibility of online adaptation of the dose distribution according to daily patient anatomy and to directly monitor tumor motion during treatment delivery by means of a continuous cine MR acquisition. Online adaptive treatments require a multidisciplinary and well-trained team, able to perform a series of operations in a safe, precise and fast manner while the patient is waiting on the treatment couch. Artificial Intelligence (AI) is expected to rapidly contribute to MRgRT, primarily by safely and efficiently automatising the various manual operations characterizing online adaptive treatments. Furthermore, AI is finding relevant applications in MRgRT in the fields of image segmentation, synthetic CT reconstruction, automatic (on-line) planning and the development of predictive models based on daily MRI. This review provides a comprehensive overview of the current AI integration in MRgRT from a medical physicist's perspective. Medical physicists are expected to be major actors in solving new tasks and in taking new responsibilities: their traditional role of guardians of the new technology implementation will change with increasing emphasis on the managing of AI tools, processes and advanced systems for imaging and data analysis, gradually replacing many repetitive manual tasks.

Learning U-Net Based Multi-Scale Features in Encoding-Decoding for MR Image Brain Tissue Segmentation

Accurate brain tissue segmentation of MRI is vital to diagnosis aiding, treatment planning, and neurologic condition monitoring. As an excellent convolutional neural network (CNN), U-Net is widely used in MR image segmentation as it usually generates high-precision features. However, the performance of U-Net is considerably restricted due to the variable shapes of the segmented targets in MRI and the information loss of down-sampling and up-sampling operations. Therefore, we propose a novel network by introducing spatial and channel dimensions-based multi-scale feature information extractors into its encoding-decoding framework, which is helpful in extracting rich multi-scale features while highlighting the details of higher-level features in the encoding part, and recovering the corresponding localization to a higher resolution layer in the decoding part. Concretely, we propose two information extractors, multi-branch pooling, called MP, in the encoding part, and multi-branch dense prediction, called MDP, in the decoding part, to extract multi-scale features. Additionally, we designed a new multi-branch output structure with MDP in the decoding part to form more accurate edge-preserving predicting maps by integrating the dense adjacent prediction features at different scales. Finally, the proposed method is tested on datasets MRbrainS13, IBSR18, and ISeg2017. We find that the proposed network performs higher accuracy in segmenting MRI brain tissues and it is better than the leading method of 2018 at the segmentation of GM and CSF. Therefore, it can be a useful tool for diagnostic applications, such as brain MRI segmentation and diagnosing.

Transductive Transfer Learning for Domain Adaptation in Brain Magnetic Resonance Image Segmentation

Segmentation of brain images from Magnetic Resonance Images (MRI) is an indispensable step in clinical practice. Morphological changes of sub-cortical brain structures and quantification of brain lesions are considered biomarkers of neurological and neurodegenerative disorders and used for diagnosis, treatment planning, and monitoring disease progression. In recent years, deep learning methods showed an outstanding performance in medical image segmentation. However, these methods suffer from generalisability problem due to inter-centre and inter-scanner variabilities of the MRI images. The main objective of the study is to develop an automated deep learning segmentation approach that is accurate and robust to the variabilities in scanner and acquisition protocols. In this paper, we propose a transductive transfer learning approach for domain adaptation to reduce the domain-shift effect in brain MRI segmentation. The transductive scenario assumes that there are sets of images from two different domains: (1) source-images with manually annotated labels; and (2) target-images without expert annotations. Then, the network is jointly optimised integrating both source and target images into the transductive training process to segment the regions of interest and to minimise the domain-shift effect. We proposed to use a histogram loss in the feature level to carry out the latter optimisation problem. In order to demonstrate the benefit of the proposed approach, the method has been tested in two different brain MRI image segmentation problems using multi-centre and multi-scanner databases for: (1) sub-cortical brain structure segmentation; and (2) white matter hyperintensities segmentation. The experiments showed that the segmentation performance of a pre-trained model could be significantly improved by up to 10%. For the first segmentation problem it was possible to achieve a maximum improvement from 0.680 to 0.799 in average Dice Similarity Coefficient (DSC) metric and for the second problem the average DSC improved from 0.504 to 0.602. Moreover, the improvements after domain adaptation were on par or showed better performance compared to the commonly used traditional unsupervised segmentation methods (FIRST and LST), also achieving faster execution time. Taking this into account, this work presents one more step toward the practical implementation of deep learning algorithms into the clinical routine.

Magnetic resonance imaging manifestations of cerebral small vessel disease: automated quantification and clinical application

The common cerebral small vessel disease (CSVD) neuroimaging features visible on conventional structural magnetic resonance imaging include recent small subcortical infarcts, lacunes, white matter hyperintensities, perivascular spaces, microbleeds, and brain atrophy. The CSVD neuroimaging features have shared and distinct clinical consequences, and the automatic quantification methods for these features are increasingly used in research and clinical settings. This review article explores the recent progress in CSVD neuroimaging feature quantification and provides an overview of the clinical consequences of these CSVD features as well as the possibilities of using these features as endpoints in clinical trials. The added value of CSVD neuroimaging quantification is also discussed for researches focused on the mechanism of CSVD and the prognosis in subjects with CSVD.

Artificial intelligence for automatic cerebral ventricle segmentation and volume calculation: a clinical tool for the evaluation of pediatric hydrocephalus

OBJECTIVE

Imaging evaluation of the cerebral ventricles is important for clinical decision-making in pediatric hydrocephalus. Although quantitative measurements of ventricular size, over time, can facilitate objective comparison, automated tools for calculating ventricular volume are not structured for clinical use. The authors aimed to develop a fully automated deep learning (DL) model for pediatric cerebral ventricle segmentation and volume calculation for widespread clinical implementation across multiple hospitals.

METHODS

The study cohort consisted of 200 children with obstructive hydrocephalus from four pediatric hospitals, along with 199 controls. Manual ventricle segmentation and volume calculation values served as "ground truth" data. An encoder-decoder convolutional neural network architecture, in which T2-weighted MR images were used as input, automatically delineated the ventricles and output volumetric measurements. On a held-out test set, segmentation accuracy was assessed using the Dice similarity coefficient (0 to 1) and volume calculation was assessed using linear regression. Model generalizability was evaluated on an external MRI data set from a fifth hospital. The DL model performance was compared against FreeSurfer research segmentation software.

RESULTS

Model segmentation performed with an overall Dice score of 0.901 (0.946 in hydrocephalus, 0.856 in controls). The model generalized to external MR images from a fifth pediatric hospital with a Dice score of 0.926. The model was more accurate than FreeSurfer, with faster operating times (1.48 seconds per scan).

CONCLUSIONS

The authors present a DL model for automatic ventricle segmentation and volume calculation that is more accurate and rapid than currently available methods. With near-immediate volumetric output and reliable performance across institutional scanner types, this model can be adapted to the real-time clinical evaluation of hydrocephalus and improve clinician workflow.

Brain volumes quantification from MRI in healthy controls: Assessing correlation, agreement and robustness of a convolutional neural network-based software against FreeSurfer, CAT12 and FSL

BACKGROUND AND PURPOSE

There are instances in which an estimate of the brain volume should be obtained from MRI in clinical practice. Our objective is to calculate cross-sectional robustness of a convolutional neural network (CNN) based software (Entelai Pic) for brain volume estimation and compare it to traditional software such as FreeSurfer, CAT12 and FSL in healthy controls (HC).

MATERIALS AND METHODS

Sixteen HC were scanned four times, two different days on two different MRI scanners (1.5 T and 3 T). Volumetric T1-weighted images were acquired and post-processed with FreeSurfer v6.0.0, Entelai Pic v2, CAT12 v12.5 and FSL v5.0.9. Whole-brain, grey matter (GM), white matter (WM) and cerebrospinal fluid (CSF) volumes were calculated. Correlation and agreement between methods was assessed using intraclass correlation coefficient (ICC) and Bland Altman plots. Robustness was assessed using the coefficient of variation (CV).

RESULTS

Whole-brain volume estimation had better correlation between FreeSurfer and Entelai Pic (ICC (95% CI) 0.96 (0.94-0.97)) than FreeSurfer and CAT12 (0.92 (0.88-0.96)) and FSL (0.87 (0.79-0.91)). WM, GM and CSF showed a similar trend. Compared to FreeSurfer, Entelai Pic provided similarly robust segmentations of brain volumes both on same-scanner (mean CV 1.07, range 0.20-3.13% vs. mean CV 1.05, range 0.21-3.20%, p = 0.86) and on different-scanner variables (mean CV 3.84, range 2.49-5.91% vs. mean CV 3.84, range 2.62-5.13%, p = 0.96). Mean post-processing times were 480, 5, 40 and 5 min for FreeSurfer, Entelai Pic, CAT12 and FSL respectively.

CONCLUSION

Based on robustness and processing times, our CNN-based model is suitable for cross-sectional volumetry on clinical practice.

BISON: Brain tissue segmentation pipeline using T1 -weighted magnetic resonance images and a random forest classifier

PURPOSE

Tissue segmentation from T₁ -weighted (T1W) MRI is a critical requirement in many neuroscience and clinical applications. However, accurate tissue segmentation is challenging because of the variabilities in tissue intensity profiles caused by differences in scanner models, acquisition protocols, and age. In addition, many methods assume healthy anatomy and fail in the presence of pathology such as white matter hyperintensities (WMHs). We present BISON (Brain tISsue segmentatiON), a new pipeline for tissue segmentation using a random forest classifier and a set of intensity and location priors based on T1W MRI.

METHODS

BISON was developed and cross-validated using multiscanner manual labels of 72 subjects aged 5 to 96 years. We also assessed the test-retest reliability of BISON on two data sets: 20 subjects with scan/rescan MR images and manual segmentations and 90 scans from a single individual. The results were compared against Atropos, a state-of-the-art commonly used tissue classification method from advanced normalization tools (ANTs).

RESULTS

BISON cross-validation dice kappa values against manual segmentations of 72 MRI volumes yielded κ_GM = 0.88, κ_WM = 0.85, κ_CSF = 0.77, outperforming Atropos (κ_GM = 0.79, κ_WM = 0.84, κ_CSF = 0.64), test-retest values on 20 subjects of κ_GM = 0.94, κ_WM = 0.92, κ_CSF = 0.77 outperforming both manual (κ_GM = 0.92, κ_WM = 0.91, κ_CSF =0.74) and Atropos (κ_GM = 0.87, κ_WM = 0.92, κ_CSF = 0.79). Finally, BISON outperformed Atropos, FAST (fast automated segmentation tool) from the FMRIB (Functional Magnetic Resonance Imaging of the Brain) Software Library, and SPM12 (statistical parametric mapping 12) in the presence of WMHs.

CONCLUSION

BISON can provide accurate and robust segmentations in data from various age ranges and scanner models, making it ideal for performing tissue classification in large multicenter and multiscanner databases.

Inter-scanner reproducibility of brain volumetry: influence of automated brain segmentation software

Background

The inter-scanner reproducibility of brain volumetry is important in multi-site neuroimaging studies, where the reliability of automated brain segmentation (ABS) tools plays an important role. This study aimed to evaluate the influence of ABS tools on the consistency and reproducibility of the quantified brain volumetry from different scanners.

Methods

We included fifteen healthy volunteers who were scanned with 3D isotropic brain T1-weighted sequence on three different 3.0 Tesla MRI scanners (GE, Siemens and Philips). For each individual, the time span between image acquisitions on different scanners was limited to 1 h. All the T1-weighted images were processed with FreeSurfer v6.0, FSL v5.0 and AccuBrain^® with default settings to obtain volumetry of brain tissues (e.g. gray matter) and substructures (e.g. basal ganglia structures) if available. Coefficient of variation (CV) was calculated to test inter-scanner variability in brain volumetry of various structures as quantified by these ABS tools.

Results

The mean inter-scanner CV values per brain structure among three MRI scanners ranged from 6.946 to 12.29% (mean, 9.577%) for FreeSurfer, 7.245 to 20.98% (mean, 12.60%) for FSL and 1.348 to 8.800% (mean value, 3.546%) for AccuBrain^®. In addition, AccuBrain^® and FreeSurfer achieved the lowest mean values of region-specific CV between GE and Siemens scanners (from 0.818 to 5.958% for AccuBrain^®, and from 0.903 to 7.977% for FreeSurfer), while FSL-FIRST had the lowest mean values of region-specific CV between GE and Philips scanners (from 2.603 to 16.310%). AccuBrain^® also had the lowest mean values of region-specific CV between Siemens and Philips scanners (from 1.138 to 6.615%).

Conclusion

There is a large discrepancy in the inter-scanner reproducibility of brain volumetry when using different processing software. Image acquisition protocols and selection of ABS tool for brain volumetry quantification have impact on the robustness of results in multi-site studies.

Regional SUV quantification in hybrid PET/MR, a comparison of two atlas-based automatic brain segmentation methods

BACKGROUND

Quantitative analysis of brain positron-emission tomography (PET) depends on structural segmentation, which can be time-consuming and operator-dependent when performed manually. Previous automatic segmentation usually registered subjects' images onto an atlas template (defined as RSIAT here) for group analysis, which changed the individuals' images and probably affected regional PET segmentation. In contrast, we could register atlas template to subjects' images (RATSI), which created an individual atlas template and may be more accurate for PET segmentation. We segmented two representative brain areas in twenty Parkinson disease (PD) and eight multiple system atrophy (MSA) patients performed in hybrid positron-emission tomography/magnetic resonance imaging (PET/MR). The segmentation accuracy was evaluated using the Dice coefficient (DC) and Hausdorff distance (HD), and the standardized uptake value (SUV) measurements of these two automatic segmentation methods were compared, using manual segmentation as a reference.

RESULTS

The DC of RATSI increased, and the HD decreased significantly (P < 0.05) compared with the RSIAT in PD, while the results of one-way analysis of variance (ANOVA) found no significant differences in the SUVmean and SUVmax among the two automatic and the manual segmentation methods. Further, RATSI was used to compare regional differences in cerebral metabolism pattern between PD and MSA patients. The SUVmean in the segmented cerebellar gray matter for the MSA group was significantly lower compared with the PD group (P < 0.05), which is consistent with previous reports.

CONCLUSION

The RATSI was more accurate for the caudate nucleus and putamen automatic segmentation and can be used for regional PET analysis in hybrid PET/MR.

A fully automated pipeline for brain structure segmentation in multiple sclerosis

Accurate volume measurements of the brain structures are important for treatment evaluation and disease follow-up in multiple sclerosis (MS) patients. With the aim of obtaining reproducible measurements and avoiding the intra-/inter-rater variability that manual delineations introduce, several automated brain structure segmentation strategies have been proposed in recent years. However, most of these strategies tend to be affected by the abnormal MS lesion intensities, which corrupt the structure segmentation result. To address this problem, we recently reformulated two label fusion strategies of the state of the art, improving their segmentation performance on the lesion areas. Here, we integrate these reformulated strategies in a completely automated pipeline that includes pre-processing (inhomogeneity correction and intensity normalization), atlas selection, masked registration and label fusion, and combine them with an automated lesion segmentation method of the state of the art. We study the effect of automating the lesion mask acquisition on the structure segmentation result, analyzing the output of the proposed pipeline when used in combination with manually and automatically segmented lesion masks. We further analyze the effect of those masks on the segmentation result of the original label fusion strategies when combined with the well-established pre-processing step of lesion filling. The experiments performed show that, when the original methods are used to segment the lesion-filled images, significant structure volume differences are observed in a comparison between manually and automatically segmented lesion masks. The results indicate a mean volume decrease of 1.13%±1.93 in the cerebrospinal fluid, and a mean volume increase of 0.13%±0.14 and 0.05%±0.08 in the cerebral white matter and cerebellar gray matter, respectively. On the other hand, no significant volume differences were found when the proposed automated pipeline was used for segmentation, which demonstrates its robustness against variations in the lesion mask used.

Reliability assessment of tissue classification algorithms for multi-center and multi-scanner data

BACKGROUND

Gray and white matter volume difference and change are important imaging markers of pathology and disease progression in neurology and psychiatry. Such measures are usually estimated from tissue segmentation maps produced by publicly available image processing pipelines. However, the reliability of the produced segmentations when using multi-center and multi-scanner data remains understudied. Here, we assess the robustness of six publicly available tissue classification pipelines across images acquired from different MR scanners and sites.

METHODS

We used 90 T1-weighted images of a single individual, scanned in 73 sessions across 27 different sites to assess the robustness of the tissue classification tools. Variability in Dice similarity index values and tissue volumes was assessed for Atropos, BISON, Classify_Clean, FAST, FreeSurfer, and SPM12.

RESULTS

BISON had the highest overall Dice coefficient for GM, followed by SPM12 and Atropos; while Atropos had the highest overall Dice coefficient for WM, followed by BISON and SPM12. BISON had the lowest overall variability in its volumetric estimates, followed by FreeSurfer, and SPM12. All methods also had significant differences between some of their estimates across different scanner manufacturers (e.g. BISON had significantly higher GM estimates and correspondingly lower WM estimates for GE scans compared to Philips and Siemens), and different signal-to-noise ratio (SNR) levels (e.g. FAST and FreeSurfer had significantly higher WM volume estimates for high versus medium and low SNR tertiles as well as correspondingly lower GM volume estimates).

CONCLUSIONS

Our comparisons provide a benchmark on the reliability of the publicly used tissue classification techniques and the amount of variability that can be expected when using large multi-center and multi-scanner databases.

3D Visualization of Brain Tumors Using MR Images: A Survey

BACKGROUND

Three-Dimensional visualization of brain tumors is very useful in both diagnosis and treatment stages of brain cancer.

DISCUSSION

It helps the oncologist/neurosurgeon to take the best decision in Radiotherapy and/or surgical resection techniques. 3D visualization involves two main steps; tumor segmentation and 3D modeling.

CONCLUSION

In this article, we illustrate the most widely used segmentation and 3D modeling techniques for brain tumors visualization. We also survey the public databases available for evaluation of the mentioned techniques.

Using deep learning for a diffusion-based segmentation of the dentate nucleus and its benefits over atlas-based methods

The dentate nucleus (DN) is a gray matter structure deep in the cerebellum involved in motor coordination, sensory input integration, executive planning, language, and visuospatial function. The DN is an emerging biomarker of disease, informing studies that advance pathophysiologic understanding of neurodegenerative and related disorders. The main challenge in defining the DN radiologically is that, like many deep gray matter structures, it has poor contrast in T1-weighted magnetic resonance (MR) images and therefore requires specialized MR acquisitions for visualization. Manual tracing of the DN across multiple acquisitions is resource-intensive and does not scale well to large datasets. We describe a technique that automatically segments the DN using deep learning (DL) on common imaging sequences, such as T1-weighted, T2-weighted, and diffusion MR imaging. We trained a DL algorithm that can automatically delineate the DN and provide an estimate of its volume. The automatic segmentation achieved higher agreement to the manual labels compared to template registration, which is the current common practice in DN segmentation or multiatlas segmentation of manual labels. Across all sequences, the FA maps achieved the highest mean Dice similarity coefficient (DSC) of 0.83 compared to T1 imaging ( DSC = 0.76 ), T2 imaging ( DSC = 0.79 ), or a multisequence approach ( DSC = 0.80 ). A single atlas registration approach using the spatially unbiased atlas template of the cerebellum and brainstem template achieved a DSC of 0.23, and multi-atlas segmentation achieved a DSC of 0.33. Overall, we propose a method of delineating the DN on clinical imaging that can reproduce manual labels with higher accuracy than current atlas-based tools.

Automated Segmentation of Tissues Using CT and MRI: A Systematic Review

RATIONALE AND OBJECTIVES

The automated segmentation of organs and tissues throughout the body using computed tomography and magnetic resonance imaging has been rapidly increasing. Research into many medical conditions has benefited greatly from these approaches by allowing the development of more rapid and reproducible quantitative imaging markers. These markers have been used to help diagnose disease, determine prognosis, select patients for therapy, and follow responses to therapy. Because some of these tools are now transitioning from research environments to clinical practice, it is important for radiologists to become familiar with various methods used for automated segmentation.

MATERIALS AND METHODS

The Radiology Research Alliance of the Association of University Radiologists convened an Automated Segmentation Task Force to conduct a systematic review of the peer-reviewed literature on this topic.

RESULTS

The systematic review presented here includes 408 studies and discusses various approaches to automated segmentation using computed tomography and magnetic resonance imaging for neurologic, thoracic, abdominal, musculoskeletal, and breast imaging applications.

CONCLUSION

These insights should help prepare radiologists to better evaluate automated segmentation tools and apply them not only to research, but eventually to clinical practice.

Label fusion method combining pixel greyscale probability for brain MR segmentation

Multi-atlas-based segmentation (MAS) methods have demonstrated superior performance in the field of automatic image segmentation, and label fusion is an important part of MAS methods. In this paper, we propose a label fusion method that incorporates pixel greyscale probability information. The proposed method combines the advantages of label fusion methods based on sparse representation (SRLF) and weighted voting methods using patch similarity weights (PSWV) and introduces pixel greyscale probability information to improve the segmentation accuracy. We apply the proposed method to the segmentation of deep brain tissues in challenging 3D brain MR images from publicly available IBSR datasets, including images of the thalamus, hippocampus, caudate, putamen, pallidum and amygdala. The experimental results show that the proposed method has higher segmentation accuracy and robustness than the related methods. Compared with the state-of-the-art methods, the proposed method obtains the best putamen, pallidum and amygdala segmentation results and hippocampus and caudate segmentation results that are similar to those of the comparison methods.

Fast and accurate pseudo multispectral technique for whole-brain MRI tissue classification

Numerous strategies have been proposed to classify brain tissues into gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF). However, many of them fail when classifying specific regions with low contrast between tissues. In this work, we propose an alternative pseudo multispectral classification (PMC) technique using CIE LAB spaces instead of gray scale T1-weighted MPRAGE images, combined with a new preprocessing technique for contrast enhancement and an optimized iterative K-means clustering. To improve the accuracy of the classification process, gray scale images were converted to multispectral CIE LAB data by applying several transformation matrices. Thus, the amount of information associated with each image voxel was increased. The image contrast was then enhanced by applying a real time function that separates brain tissue distributions and improve image contrast in certain brain regions. The data were then classified using an optimized iterative and convergent K-means classifier. The performance of the proposed approach was assessed using simulation and in vivo human studies through comparison with three common software packages used for brain MR image segmentation, namely FSL, SPM8 and K-means clustering. In the presence of high SNR, the results showed that the four algorithms achieve a good classification. Conversely, in the presence of low SNR, PMC was shown to outperform the other methods by accurately recovering all tissue volumes. The quantitative assessment of brain tissue classification for simulated studies showed that the PMC algorithm resulted in a mean Jaccard index (JI) of 0.74 compared to 0.75 for FSL, 0.7 for SPM and 0.8 for K-means. The in vivo human studies showed that the PMC algorithm resulted in a mean JI of 0.92, which reflects a good spatial overlap between segmented and actual volumes, compared to 0.84 for FSL, 0.78 for SPM and 0.66 for K-means. The proposed algorithm presents a high potential for improving the accuracy of automatic brain tissues classification and was found to be accurate even in the presence of high noise level.

Non-linear registration improves statistical power to detect hippocampal atrophy in aging and dementia

OBJECTIVE

To compare the performance of different methods for determining hippocampal atrophy rates using longitudinal MRI scans in aging and Alzheimer's disease (AD).

BACKGROUND

Quantifying hippocampal atrophy caused by neurodegenerative diseases is important to follow the course of the disease. In dementia, the efficacy of new therapies can be partially assessed by measuring their effect on hippocampal atrophy. In radiotherapy, the quantification of radiation-induced hippocampal volume loss is of interest to quantify radiation damage. We evaluated plausibility, reproducibility and sensitivity of eight commonly used methods to determine hippocampal atrophy rates using test-retest scans.

MATERIALS AND METHODS

Manual, FSL-FIRST, FreeSurfer, multi-atlas segmentation (MALF) and non-linear registration methods (Elastix, NiftyReg, ANTs and MIRTK) were used to determine hippocampal atrophy rates on longitudinal T1-weighted MRI from the ADNI database. Appropriate parameters for the non-linear registration methods were determined using a small training dataset (N = 16) in which two-year hippocampal atrophy was measured using test-retest scans of 8 subjects with low and 8 subjects with high atrophy rates. On a larger dataset of 20 controls, 40 mild cognitive impairment (MCI) and 20  AD patients, one-year hippocampal atrophy rates were measured. A repeated measures ANOVA analysis was performed to determine differences between controls, MCI and AD patients. For each method we calculated effect sizes and the required sample sizes to detect one-year volume change between controls and MCI (NCTRL_MCI) and between controls and AD (NCTRL_AD). Finally, reproducibility of hippocampal atrophy rates was assessed using within-session rescans and expressed as an average distance measure DAve, which expresses the difference in atrophy rate, averaged over all subjects. The same DAve was used to determine the agreement between different methods.

RESULTS

Except for MALF, all methods detected a significant group difference between CTRL and AD, but none could find a significant difference between the CTRL and MCI. FreeSurfer and MIRTK required the lowest sample sizes (FreeSurfer: NCTRL_MCI = 115, NCTRL_AD = 17 with DAve = 3.26%; MIRTK: NCTRL_MCI = 97, NCTRL_AD = 11 with DAve = 3.76%), while ANTs was most reproducible (NCTRL_MCI = 162, NCTRL_AD = 37 with DAve = 1.06%), followed by Elastix (NCTRL_MCI = 226, NCTRL_AD = 15 with DAve = 1.78%) and NiftyReg (NCTRL_MCI = 193, NCTRL_AD = 14 with DAve = 2.11%). Manually measured hippocampal atrophy rates required largest sample sizes to detect volume change and were poorly reproduced (NCTRL_MCI = 452, NCTRL_AD = 87 with DAve = 12.39%). Atrophy rates of non-linear registration methods also agreed best with each other.

DISCUSSION AND CONCLUSION

Non-linear registration methods were most consistent in determining hippocampal atrophy and because of their better reproducibility, methods, such as ANTs, Elastix and NiftyReg, are preferred for determining hippocampal atrophy rates on longitudinal MRI. Since performances of non-linear registration methods are well comparable, the preferred method would mostly depend on computational efficiency.

Supervised Domain Adaptation for Automatic Sub-cortical Brain Structure Segmentation with Minimal User Interaction

In recent years, some convolutional neural networks (CNNs) have been proposed to segment sub-cortical brain structures from magnetic resonance images (MRIs). Although these methods provide accurate segmentation, there is a reproducibility issue regarding segmenting MRI volumes from different image domains - e.g., differences in protocol, scanner, and intensity profile. Thus, the network must be retrained from scratch to perform similarly in different imaging domains, limiting the applicability of such methods in clinical settings. In this paper, we employ the transfer learning strategy to solve the domain shift problem. We reduced the number of training images by leveraging the knowledge obtained by a pretrained network, and improved the training speed by reducing the number of trainable parameters of the CNN. We tested our method on two publicly available datasets - MICCAI 2012 and IBSR - and compared them with a commonly used approach: FIRST. Our method showed similar results to those obtained by a fully trained CNN, and our method used a remarkably smaller number of images from the target domain. Moreover, training the network with only one image from MICCAI 2012 and three images from IBSR datasets was sufficient to significantly outperform FIRST with (p < 0.001) and (p < 0.05), respectively.

HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation

Recently, dense connections have attracted substantial attention in computer vision because they facilitate gradient flow and implicit deep supervision during training. Particularly, DenseNet that connects each layer to every other layer in a feed-forward fashion and has shown impressive performances in natural image classification tasks. We propose HyperDenseNet, a 3-D fully convolutional neural network that extends the definition of dense connectivity to multi-modal segmentation problems. Each imaging modality has a path, and dense connections occur not only between the pairs of layers within the same path but also between those across different paths. This contrasts with the existing multi-modal CNN approaches, in which modeling several modalities relies entirely on a single joint layer (or level of abstraction) for fusion, typically either at the input or at the output of the network. Therefore, the proposed network has total freedom to learn more complex combinations between the modalities, within and in-between all the levels of abstraction, which increases significantly the learning representation. We report extensive evaluations over two different and highly competitive multi-modal brain tissue segmentation challenges, iSEG 2017 and MRBrainS 2013, with the former focusing on six month infant data and the latter on adult images. HyperDenseNet yielded significant improvements over many state-of-the-art segmentation networks, ranking at the top on both benchmarks. We further provide a comprehensive experimental analysis of features re-use, which confirms the importance of hyper-dense connections in multi-modal representation learning. Our code is publicly available.

Contrast Agents Delivery: An Up-to-Date Review of Nanodiagnostics in Neuroimaging

Neuroimaging is a highly important field of neuroscience, with direct implications for the early diagnosis and progression monitoring of brain-associated diseases. Neuroimaging techniques are categorized into structural, functional and molecular neuroimaging, each possessing advantages and disadvantages in terms of resolution, invasiveness, toxicity of contrast agents and costs. Nanotechnology-based approaches for neuroimaging mostly involve the development of nanocarriers for incorporating contrast agents or the use of nanomaterials as imaging agents. Inorganic and organic nanoparticles, liposomes, micelles, nanobodies and quantum dots are some of the most studied candidates for the delivery of contrast agents for neuroimaging. This paper focuses on describing the conventional modalities used for imaging and the applications of nanotechnology for developing novel strategies for neuroimaging. The aim is to highlight the roles of nanocarriers for enhancing and/or overcome the limitations associated with the most commonly utilized neuroimaging modalities. For future directions, several techniques that could benefit from the increased contrast induced by using imaging probes are presented.

An active texture-based digital atlas enables automated mapping of structures and markers across brains

Brain atlases enable the mapping of labeled cells and projections from different brains onto a standard coordinate system. We address two issues in the construction and use of atlases. First, expert neuroanatomists ascertain the fine-scale pattern of brain tissue, the 'texture' formed by cellular organization, to define cytoarchitectural borders. We automate the processes of localizing landmark structures and alignment of brains to a reference atlas using machine learning and training data derived from expert annotations. Second, we construct an atlas that is active; that is, augmented with each use. We show that the alignment of new brains to a reference atlas can continuously refine the coordinate system and associated variance. We apply this approach to the adult murine brainstem and achieve a precise alignment of projections in cytoarchitecturally ill-defined regions across brains from different animals.

Brain structure segmentation in the presence of multiple sclerosis lesions

Intensity-based multi-atlas segmentation strategies have shown to be particularly successful in segmenting brain images of healthy subjects. However, in the same way as most of the methods in the state of the art, their performance tends to be affected by the presence of MRI visible lesions, such as those found in multiple sclerosis (MS) patients. Here, we present an approach to minimize the effect of the abnormal lesion intensities on multi-atlas segmentation. We propose a new voxel/patch correspondence model for intensity-based multi-atlas label fusion strategies that leads to more accurate similarity measures, having a key role in the final brain segmentation. We present the theory of this model and integrate it into two well-known fusion strategies: Non-local Spatial STAPLE (NLSS) and Joint Label Fusion (JLF). The experiments performed show that our proposal improves the segmentation performance of the lesion areas. The results indicate a mean Dice Similarity Coefficient (DSC) improvement of 1.96% for NLSS (3.29% inside and 0.79% around the lesion masks) and, an improvement of 2.06% for JLF (2.31% inside and 1.42% around lesions). Furthermore, we show that, with the proposed strategy, the well-established preprocessing step of lesion filling can be disregarded, obtaining similar or even more accurate segmentation results.

•

We present an approach to improve multi-atlas brain parcellation of MS patients.

•

We integrate our model into 2 well-known segmentation strategies.

•

Our model improves the segmentation on the lesion areas.

•

The improvement on the lesion areas is also reflected in the global performance.

•

With our model, lesion filling can be omitted, obtaining at least similar results.

Patch spaces and fusion strategies in patch-based label fusion

In the field of multi-atlas segmentation, patch-based approaches have shown promising results in the segmentation of biomedical images. In the most common approach, registration is used to warp the atlases to the target space and then the warped atlas labelmaps are fused into a consensus segmentation based on local appearance information encoded in form of patches. The registration step establishes spatial correspondence, which is important to obtain anatomical priors. Patch-based label fusion in the target space has shown to produce very accurate segmentations although at the expense of registering all atlases to each target image. Moreover, appearance (i.e., patches) and label information used by label fusion is extracted from the warped atlases, which are subject to interpolation errors. In this work, we revisit and extend the patch-based label fusion framework, exploring the role of extracting this information from the native space of both atlases and target images, thus avoiding interpolation artifacts, but at the same time, we do it in a way that it does not sacrifice the anatomical priors derived by registration. We further propose a common formulation for two widely-used label fusion strategies, i.e., similarity-based and a particular type of learning-based label fusion. The proposed framework is evaluated on subcortical structure segmentation in adult brains and tissue segmentation in fetal brain MRI. Our results indicate that using atlas patches in their native space yields superior performance than warping the atlases to the target image. The learning-based approach tends to outperform the similarity-based approach, with the particularity that using patches in native space lessens the computational requirements of learning. As conclusion, the combination of learning-based label fusion and native atlas patches yields the best performance with reduced test times than conventional similarity-based approaches.

Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review

In recent years, deep convolutional neural networks (CNNs) have shown record-shattering performance in a variety of computer vision problems, such as visual object recognition, detection and segmentation. These methods have also been utilised in medical image analysis domain for lesion segmentation, anatomical segmentation and classification. We present an extensive literature review of CNN techniques applied in brain magnetic resonance imaging (MRI) analysis, focusing on the architectures, pre-processing, data-preparation and post-processing strategies available in these works. The aim of this study is three-fold. Our primary goal is to report how different CNN architectures have evolved, discuss state-of-the-art strategies, condense their results obtained using public datasets and examine their pros and cons. Second, this paper is intended to be a detailed reference of the research activity in deep CNN for brain MRI analysis. Finally, we present a perspective on the future of CNNs in which we hint some of the research directions in subsequent years.