Multi-atlas segmentation of biomedical images: A survey

Automated MicroCT-based bone and articular cartilage analysis using iterative shape averaging and atlas-based registration

Micro-computed tomography (μCT) and contrast-enhanced μCT are important tools for preclinical analysis of bone and articular cartilage (AC). Quantitative data from these modalities is highly dependent on the accuracy of tissue segmentations, which are often obtained via time-consuming manual contouring and are prone to inter- and intra-observer variability. Automated segmentation strategies could mitigate these issues, but few such approaches have been described in the context of μCT. Here, we validated a fully-automated strategy for bone and AC segmentation based on registration of an average tissue atlas. Femora from healthy and arthritic rats underwent μCT scanning, and epiphyseal trabecular bone and AC volumes were manually contoured by an expert. Average tissue atlases composed of 1, 3, 5, 10 and 20 pre-contoured training images (n = 10 atlases/group) were generated using iterative shape averaging and registered onto unknown images via affine and non-rigid registration. Atlas-based and expert-defined volumes for bone and AC were compared in terms of shape-based similarity metrics, as well as morphometric and densitometric parameters. Our results demonstrate that atlas-based registrations were capable of highly accurate and consistent segmentation. Atlases built from as few as 3 training images had no incidence of mal-registration and exhibited improved incidence of accurate registration, and higher sensitivity and specificity compared to atlases built from only one training image. Atlas-based segmentation of bone and AC from μCT images is a robust and accurate alternative to manual tissue segmentation, enabling faster, more consistent segmentation of pre-clinical datasets.

Multi-Atlas Segmentation of Anatomical Brain Structures Using Hierarchical Hypergraph Learning

Accurate segmentation of anatomical brain structures is crucial for many neuroimaging applications, e.g., early brain development studies and the study of imaging biomarkers of neurodegenerative diseases. Although multi-atlas segmentation (MAS) has achieved many successes in the medical imaging area, this approach encounters limitations in segmenting anatomical structures associated with poor image contrast. To address this issue, we propose a new MAS method that uses a hypergraph learning framework to model the complex subject-within and subject-to-atlas image voxel relationships and propagate the label on the atlas image to the target subject image. To alleviate the low-image contrast issue, we propose two strategies equipped with our hypergraph learning framework. First, we use a hierarchical strategy that exploits high-level context features for hypergraph construction. Because the context features are computed on the tentatively estimated probability maps, we can ultimately turn the hypergraph learning into a hierarchical model. Second, instead of only propagating the labels from the atlas images to the target subject image, we use a dynamic label propagation strategy that can gradually use increasing reliably identified labels from the subject image to aid in predicting the labels on the difficult-to-label subject image voxels. Compared with the state-of-the-art label fusion methods, our results show that the hierarchical hypergraph learning framework can substantially improve the robustness and accuracy in the segmentation of anatomical brain structures with low image contrast from magnetic resonance (MR) images.

Auto-segmentation of organs at risk for head and neck radiotherapy planning: From atlas-based to deep learning methods

Radiotherapy (RT) is one of the basic treatment modalities for cancer of the head and neck (H&N), which requires a precise spatial description of the target volumes and organs at risk (OARs) to deliver a highly conformal radiation dose to the tumor cells while sparing the healthy tissues. For this purpose, target volumes and OARs have to be delineated and segmented from medical images. As manual delineation is a tedious and time-consuming task subjected to intra/interobserver variability, computerized auto-segmentation has been developed as an alternative. The field of medical imaging and RT planning has experienced an increased interest in the past decade, with new emerging trends that shifted the field of H&N OAR auto-segmentation from atlas-based to deep learning-based approaches. In this review, we systematically analyzed 78 relevant publications on auto-segmentation of OARs in the H&N region from 2008 to date, and provided critical discussions and recommendations from various perspectives: image modality - both computed tomography and magnetic resonance image modalities are being exploited, but the potential of the latter should be explored more in the future; OAR - the spinal cord, brainstem, and major salivary glands are the most studied OARs, but additional experiments should be conducted for several less studied soft tissue structures; image database - several image databases with the corresponding ground truth are currently available for methodology evaluation, but should be augmented with data from multiple observers and multiple institutions; methodology - current methods have shifted from atlas-based to deep learning auto-segmentation, which is expected to become even more sophisticated; ground truth - delineation guidelines should be followed and participation of multiple experts from multiple institutions is recommended; performance metrics - the Dice coefficient as the standard volumetric overlap metrics should be accompanied with at least one distance metrics, and combined with clinical acceptability scores and risk assessments; segmentation performance - the best performing methods achieve clinically acceptable auto-segmentation for several OARs, however, the dosimetric impact should be also studied to provide clinically relevant endpoints for RT planning.

Migraine with aura in women is not associated with structural thalamic abnormalities

Migraine with aura is a highly prevalent disorder involving transient neurological disturbances associated with migraine headache. While the pathophysiology is incompletely understood, findings from clinical and basic science studies indicate a potential key role of the thalamus in the mechanisms underlying migraine with and without aura. Two recent, clinic-based MRI studies investigated the volumes of individual thalamic nuclei in migraine patients with and without aura using two different data analysis methods. Both studies found differences of thalamic nuclei volumes between patients and healthy controls, but the results of the studies were not consistent. Here, we investigated whether migraine with aura is associated with changes in thalamic volume by analysing MRI data obtained from a large, cross-sectional population-based study which specifically included women with migraine with aura (N = 156), unrelated migraine-free matched controls (N = 126), and migraine aura-free co-twins (N = 29) identified from the Danish Twin Registry. We used two advanced, validated analysis methods to assess the volume of the thalamus and its nuclei; the MAGeT Brain Algorithm and a recently developed FreeSurfer-based method based on a probabilistic atlas of the thalamic nuclei combining ex vivo MRI and histology. These approaches were very similar to the methods used in each of the two previous studies. Between-group comparisons were corrected for potential effects of age, educational level, BMI, smoking, alcohol, and hypertension using a linear mixed model. Further, we used linear mixed models and visual inspection of data to assess relations between migraine aura frequency and thalamic nuclei volumes in patients. In addition, we performed paired t-tests to compare volumes of twin pairs (N = 29) discordant for migraine with aura. None of our analyses showed any between-group differences in volume of the thalamus or of individual thalamic nuclei. Our results indicate that the pathophysiology of migraine with aura does not involve alteration of thalamic volume.

Sequential and Iterative Auto-Segmentation of High-Risk Clinical Target Volume for Radiotherapy of Nasopharyngeal Carcinoma in Planning CT Images

Background: Accurate segmentation of tumor targets is critical for maximizing tumor control and minimizing normal tissue toxicity. We proposed a sequential and iterative U-Net (SI-Net) deep learning method to auto-segment the high-risk primary tumor clinical target volume (CTVp1) for treatment planning of nasopharyngeal carcinoma (NPC) radiotherapy. Methods: The SI-Net is a variant of the U-Net architecture. The input of SI-Net includes one CT image, the CTVp1 contour on this image, and the next CT image. The output is the predicted CTVp1 contour on the next CT image. We designed the SI-Net, using the left side to learn the volumetric features and the right to localize the contour on the next image. Two prediction directions, one from inferior to superior (forward direction) and the other from superior to inferior (backward direction), were tested. The performance was compared between the SI-Net and the U-Net using Dice similarity coefficient (DSC), Jaccard index (JI), average surface distance (ASD), and Hausdorff distance (HD) metrics. Results: The DSC and JI values from the forward direction SI-Net model were 5 and 6% higher than those from the U-Net model (0.84 ± 0.04 vs. 0.80 ± 0.05 and 0.74 ± 0.05 vs. 0.69 ± 0.05, p < 0.001). The smaller ASD and HD values also indicated a better performance (2.8 ± 1.0 vs. 3.3 ± 1.0 mm and 8.7 ± 2.5 vs. 9.7 ± 2.7 mm, p < 0.01) for the SI-Net model. For the backward direction SI-Net model, the DSC and JI values were still better than those from the U-Net model (p < 0.01), although there were no significant differences in ASD and HD. Conclusions: The SI-Net model preserved the continuity between adjacent images and thus improved the segmentation accuracy compared with the conventional U-Net model. This model has potential of improving the efficiency and consistence of CTVp1 contouring for NPC patients.

Anthropometer3D: Automatic Multi-Slice Segmentation Software for the Measurement of Anthropometric Parameters from CT of PET/CT

Anthropometric parameters like muscle body mass (MBM), fat body mass (FBM), lean body mass (LBM), visceral adipose tissue (VAT), and subcutaneous adipose tissue (SAT) are used in oncology. Our aim was to develop and evaluate the software Anthropometer3D measuring these anthropometric parameters on the CT of PET/CT. This software performs a multi-atlas segmentation of CT of PET/CT with extrapolation coefficients for the body parts beyond the usual acquisition range (from the ischia to the eyes). The multi-atlas database is composed of 30 truncated CTs manually segmented to isolate three types of voxels (muscle, fat, and visceral fat). To evaluate Anthropomer3D, a leave-one-out cross-validation was performed to measure MBM, FBM, LBM, VAT, and SAT. The reference standard was based on the manual segmentation of the corresponding whole-body CT. A manual segmentation of one CT slice at level L3 was also used. Correlations were analyzed using Dice coefficient, intra-class coefficient correlation (ICC), and Bland-Altman plot. The population was heterogeneous (sex ratio 1:1; mean age 57 years old [min 23; max 74]; mean BMI 27 kg/m² [min 18; max 40]). Dice coefficients between reference standard and Anthropometer3D were excellent (mean+/-SD): muscle 0.95 ± 0.02, fat 1.00 ± 0.01, and visceral fat 0.97 ± 0.02. The ICC was almost perfect (minimal value of 95% CI of 0.97). All Bland-Altman plot values (mean difference, 95% CI and slopes) were better for Anthropometer3D compared to L3 level segmentation. Anthropometer3D allows multiple anthropometric measurements based on an automatic multi-slice segmentation. It is more precise than estimates using L3 level segmentation.

A Multi-atlas Approach for Active Bone Marrow Sparing Radiation Therapy: Implementation in the NRG-GY006 Trial

PURPOSE

Sparing active bone marrow (ABM) can reduce acute hematologic toxicity in patients undergoing chemoradiotherapy for cervical cancer, but ABM segmentation based on positron emission tomography/computed tomography (PET/CT) is costly. We sought to develop an atlas-based ABM segmentation method for implementation in a prospective clinical trial.

METHODS AND MATERIALS

A multiatlas was built on a training set of 144 patients and validated in 32 patients from the NRG-GY006 clinical trial. ABM for individual patients was defined as the subvolume of pelvic bone greater than the individual mean standardized uptake value on registered ¹⁸F-fluorodeoxyglucose PET/CT images. Atlas-based and custom ABM segmentations were compared using the Dice similarity coefficient and mean distance to agreement and used to generate ABM-sparing intensity modulated radiation therapy plans. Dose-volume metrics and normal tissue complication probabilities of the two approaches were compared using linear regression.

RESULTS

Atlas-based ABM volumes (mean [standard deviation], 548.4 [88.3] cm³) were slightly larger than custom ABM volumes (535.1 [93.2] cm³), with a Dice similarity coefficient of 0.73. Total pelvic bone marrow V₂₀ and D_mean were systematically higher and custom ABM V₁₀ was systematically lower with custom-based plans (slope: 1.021 [95% confidence interval (CI), 1.005-1.037], 1.014 [95% CI, 1.006-1.022], and 0.98 [95% CI, 0.97-0.99], respectively). We found no significant differences between atlas-based and custom-based plans in bowel, rectum, bladder, femoral heads, or target dose-volume metrics.

CONCLUSIONS

Atlas-based ABM segmentation can reduce pelvic bone marrow dose while achieving comparable target and other normal tissue dosimetry. This approach may allow ABM sparing in settings where PET/CT is unavailable.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

An Optimal, Generative Model for Estimating Multi-Label Probabilistic Maps

Multi-label probabilistic maps, a.k.a. probabilistic segmentations, parameterize a population of intimately co-existing anatomical shapes and are useful for various medical imaging applications, such as segmentation, anatomical atlases, shape analysis, and consensus generation. Existing methods to estimate probabilistic segmentations rely on ad hoc intermediate representations (e.g., average of Gaussian-smoothed label maps and smoothed signed distance maps) that do not necessarily conform to the underlying generative process. Generative modeling of such maps could help discover as well as aide in the statistical analysis of sub-groups in a population via clustering and mixture modeling techniques. In this paper, we propose an estimation of multi-label probabilistic maps and showcase their favorable performance for modeling anatomical shapes such as the left atrium of the human heart and brain structures. The proposed formulation relies on a constrained optimization in the natural parameter space of the exponential family form of categorical distributions. A smoothness prior provides generalizability in the model and helps achieve greater performance in modeling tasks for unseen samples. We demonstrate and compare the effectiveness of the proposed method for Bayesian image segmentation, multi-atlas segmentation, and shape-based clustering.

Multi-atlas image registration of clinical data with automated quality assessment using ventricle segmentation

Registration is a core component of many imaging pipelines. In case of clinical scans, with lower resolution and sometimes substantial motion artifacts, registration can produce poor results. Visual assessment of registration quality in large clinical datasets is inefficient. In this work, we propose to automatically assess the quality of registration to an atlas in clinical FLAIR MRI scans of the brain. The method consists of automatically segmenting the ventricles of a given scan using a neural network, and comparing the segmentation to the atlas ventricles propagated to image space. We used the proposed method to improve clinical image registration to a general atlas by computing multiple registrations - one directly to the general atlas and others via different age-specific atlases - and then selecting the registration that yielded the highest ventricle overlap. Finally, as an example application of the complete pipeline, a voxelwise map of white matter hyperintensity burden was computed using only the scans with registration quality above a predefined threshold. Methods were evaluated in a single-site dataset of more than 1000 scans, as well as a multi-center dataset comprising 142 clinical scans from 12 sites. The automated ventricle segmentation reached a Dice coefficient with manual annotations of 0.89 in the single-site dataset, and 0.83 in the multi-center dataset. Registration via age-specific atlases could improve ventricle overlap compared to a direct registration to the general atlas (Dice similarity coefficient increase up to 0.15). Experiments also showed that selecting scans with the registration quality assessment method could improve the quality of average maps of white matter hyperintensity burden, instead of using all scans for the computation of the white matter hyperintensity map. In this work, we demonstrated the utility of an automated tool for assessing image registration quality in clinical scans. This image quality assessment step could ultimately assist in the translation of automated neuroimaging pipelines to the clinic.

Three-dimensional choroidal vessel network quantification using swept source optical coherence tomography

Precise three-dimensional segmentation of choroidal vessels helps us understand the development and progression of multiple ocular diseases, such as agerelated macular degeneration and pathological myopia. Here we propose a novel automatic choroidal vessel segmentation framework for swept source optical coherence tomography (SS-OCT) to visualize and quantify three-dimensional choroidal vessel networks. Retinal pigment epithelium (RPE) was delineated from volumetric data and enface frames along the depth were extracted under the RPE. Choroidal vessels on the first enface frame were labeled by adaptive thresholding and each subsequent frame was segmented via segment propagation from the frame above and was in turn used as the reference for the next frame. Choroid boundary was determined by structural similarity index between adjacent frames. The framework was tested on 33 mm SS-OCT volumes acquired by a prototype SS-OCT system (PlexElite 9000, Zeiss Meditec, Dublin, CA, US), and vessel metrics including perfusion density, vessel density and mean vessel diameter were computed. Results from human subjects (N = 8) and non-human primates (N = 6) were summarized.Clinical Relevance- Accurate 3D choroid vessel segmentation can help clinicians better quantify blood perfusion which can lead to improved diagnosis and management of retinal eye diseases.

Groupwise registration with global-local graph shrinkage in atlas construction

Graph-based groupwise registration methods are widely used in atlas construction. Given a group of images, a graph is built whose nodes represent the images, and whose edges represent a geodesic path between two nodes. The distribution of images on an image manifold is explored through edge traversal in a graph. The final atlas is a mean image at the population center of the distribution on the manifold. The procedure of warping all images to the mean image turns to dynamic graph shrinkage in which nodes become closer to each other. Most conventional groupwise registration frameworks construct and shrink a graph without considering the local distribution of images on the dataset manifold and the local structure variations between image pairs. Neglecting the local information fundamentally decrease the accuracy and efficiency when population atlases are built for organs with large inter-subject anatomical variabilities. To overcome the problem, this paper proposes a global-local graph shrinkage approach that can generate accurate atlas. A connected graph is constructed automatically based on global similarities across the images to explore the global distribution. A local image distribution obtained by image clustering is used to simplify the edges of the constructed graph. Subsequently, local image similarities refine the deformation estimated through global image similarity for each image warping along the graph edges. Through the image warping, the overall simplified graph shrinks gradually to yield the atlas with respecting both global and local features. The proposed method is evaluated on 61 synthetic and 20 clinical liver datasets, and the results are compared with those of six state-of-the-art groupwise registration methods. The experimental results show that the proposed method outperforms non-global-local method approaches in terms of accuracy.

Serum adipokines/related inflammatory factors and ratios as predictors of infrapatellar fat pad volume in osteoarthritis: Applying comprehensive machine learning approaches

OBJECTIVE

The infrapatellar fat pad (IPFP) has been associated with knee osteoarthritis onset and progression. This study uses machine learning (ML) approaches to predict serum levels of some adipokines/related inflammatory factors and their ratios on knee IPFP volume of osteoarthritis patients.

METHODS

Serum and MRI were from the OAI at baseline. Variables comprised the 3 main osteoarthritis risk factors (age, gender, BMI), 6 adipokines, 3 inflammatory factors, and their 36 ratios. IPFP volume was assessed on MRI with a ML methodology. The best variables and models were identified in Total-cohort (n = 678), High-BMI (n = 341) and Low-BMI (n = 337), using a selection approach based on ML methods.

RESULTS

The best model for each group included three risk factors and adipsin/C-reactive protein combined for Total-cohort, adipsin/chemerin; High-BMI, chemerin/adiponectin HMW; and Low-BMI, interleukin-8. Gender separation improved the prediction (13-16%) compared to the BMI-based models. Reproducibility with osteoarthritis patients from a clinical trial was excellent (R: female 0.83, male 0.95). Pseudocodes based on gender were generated.

CONCLUSION

This study demonstrates for the first time that the combination of the serum levels of adipokines/inflammatory factors and the three main risk factors of osteoarthritis could predict IPFP volume with high reproducibility, with the superior performance of the model accounting for gender separation.

A fully automated pipeline for brain structure segmentation in multiple sclerosis

Accurate volume measurements of the brain structures are important for treatment evaluation and disease follow-up in multiple sclerosis (MS) patients. With the aim of obtaining reproducible measurements and avoiding the intra-/inter-rater variability that manual delineations introduce, several automated brain structure segmentation strategies have been proposed in recent years. However, most of these strategies tend to be affected by the abnormal MS lesion intensities, which corrupt the structure segmentation result. To address this problem, we recently reformulated two label fusion strategies of the state of the art, improving their segmentation performance on the lesion areas. Here, we integrate these reformulated strategies in a completely automated pipeline that includes pre-processing (inhomogeneity correction and intensity normalization), atlas selection, masked registration and label fusion, and combine them with an automated lesion segmentation method of the state of the art. We study the effect of automating the lesion mask acquisition on the structure segmentation result, analyzing the output of the proposed pipeline when used in combination with manually and automatically segmented lesion masks. We further analyze the effect of those masks on the segmentation result of the original label fusion strategies when combined with the well-established pre-processing step of lesion filling. The experiments performed show that, when the original methods are used to segment the lesion-filled images, significant structure volume differences are observed in a comparison between manually and automatically segmented lesion masks. The results indicate a mean volume decrease of 1.13%±1.93 in the cerebrospinal fluid, and a mean volume increase of 0.13%±0.14 and 0.05%±0.08 in the cerebral white matter and cerebellar gray matter, respectively. On the other hand, no significant volume differences were found when the proposed automated pipeline was used for segmentation, which demonstrates its robustness against variations in the lesion mask used.

The Tumor Target Segmentation of Nasopharyngeal Cancer in CT Images Based on Deep Learning Methods

Radiotherapy is the main treatment strategy for nasopharyngeal carcinoma. A major factor affecting radiotherapy outcome is the accuracy of target delineation. Target delineation is time-consuming, and the results can vary depending on the experience of the oncologist. Using deep learning methods to automate target delineation may increase its efficiency. We used a modified deep learning model called U-Net to automatically segment and delineate tumor targets in patients with nasopharyngeal carcinoma. Patients were randomly divided into a training set (302 patients), validation set (100 patients), and test set (100 patients). The U-Net model was trained using labeled computed tomography images from the training set. The U-Net was able to delineate nasopharyngeal carcinoma tumors with an overall dice similarity coefficient of 65.86% for lymph nodes and 74.00% for primary tumor, with respective Hausdorff distances of 32.10 and 12.85 mm. Delineation accuracy decreased with increasing cancer stage. Automatic delineation took approximately 2.6 hours, compared to 3 hours, using an entirely manual procedure. Deep learning models can therefore improve accuracy, consistency, and efficiency of target delineation in T stage, but additional physician input may be required for lymph nodes.

BPBSAM: Body part-specific burn severity assessment model

BACKGROUND AND OBJECTIVE

Burns are a serious health problem leading to several thousand deaths annually, and despite the growth of science and technology, automated burns diagnosis still remains a major challenge. Researchers have been exploring visual images-based automated approaches for burn diagnosis. Noting that the impact of a burn on a particular body part can be related to the skin thickness factor, we propose a deep convolutional neural network based body part-specific burns severity assessment model (BPBSAM).

METHOD

Considering skin anatomy, BPBSAM estimates burn severity using body part-specific support vector machines trained with CNN features extracted from burnt body part images. Thus BPBSAM first identifies the body part of the burn images using a convolutional neural network in training of which the challenge of limited availability of burnt body part images is successfully addressed by using available larger-size datasets of non-burn images of different body parts considered (face, hand, back, and inner forearm). We prepared a rich labelled burn images datasets: BI & UBI and trained several deep learning models with existing models as pipeline for body part classification and feature extraction for severity estimation.

RESULTS

The proposed novel BPBSAM method classified the severity of burn from color images of burn injury with an overall average F1 score of 77.8% and accuracy of 84.85% for the test BI dataset and 87.2% and 91.53% for the UBI dataset, respectively. For burn images body part classification, the average accuracy of around 93% is achieved, and for burn severity assessment, the proposed BPBSAM outperformed the generic method in terms of overall average accuracy by 10.61%, 4.55%, and 3.03% with pipelines ResNet50, VGG16, and VGG19, respectively.

CONCLUSIONS

The main contributions of this work along with burn images labelled datasets creation is that the proposed customized body part-specific burn severity assessment model can significantly improve the performance in spite of having small burn images dataset. This highly innovative customized body part-specific approach could also be used to deal with the burn region segmentation problem. Moreover, fine tuning on pre-trained non-burn body part images network has proven to be robust and reliable.

Tractostorm: The what, why, and how of tractography dissection reproducibility

Investigative studies of white matter (WM) brain structures using diffusion MRI (dMRI) tractography frequently require manual WM bundle segmentation, often called "virtual dissection." Human errors and personal decisions make these manual segmentations hard to reproduce, which have not yet been quantified by the dMRI community. It is our opinion that if the field of dMRI tractography wants to be taken seriously as a widespread clinical tool, it is imperative to harmonize WM bundle segmentations and develop protocols aimed to be used in clinical settings. The EADC-ADNI Harmonized Hippocampal Protocol achieved such standardization through a series of steps that must be reproduced for every WM bundle. This article is an observation of the problematic. A specific bundle segmentation protocol was used in order to provide a real-life example, but the contribution of this article is to discuss the need for reproducibility and standardized protocol, as for any measurement tool. This study required the participation of 11 experts and 13 nonexperts in neuroanatomy and "virtual dissection" across various laboratories and hospitals. Intra-rater agreement (Dice score) was approximately 0.77, while inter-rater was approximately 0.65. The protocol provided to participants was not necessarily optimal, but its design mimics, in essence, what will be required in future protocols. Reporting tractometry results such as average fractional anisotropy, volume or streamline count of a particular bundle without a sufficient reproducibility score could make the analysis and interpretations more difficult. Coordinated efforts by the diffusion MRI tractography community are needed to quantify and account for reproducibility of WM bundle extraction protocols in this era of open and collaborative science.

MonkeyCBP: A Toolbox for Connectivity-Based Parcellation of Monkey Brain

Non-human primate models are widely used in studying the brain mechanism underlying brain development, cognitive functions, and psychiatric disorders. Neuroimaging techniques, such as magnetic resonance imaging, play an important role in the examinations of brain structure and functions. As an indispensable tool for brain imaging data analysis, brain atlases have been extensively investigated, and a variety of versions constructed. These atlases diverge in the criteria based on which they are plotted. The criteria range from cytoarchitectonic features, neurotransmitter receptor distributions, myelination fingerprints, and transcriptomic patterns to structural and functional connectomic profiles. Among them, brainnetome atlas is tightly related to brain connectome information and built by parcellating the brain on the basis of the anatomical connectivity profiles derived from structural neuroimaging data. The pipeline for building the brainnetome atlas has been published as a toolbox named ATPP (A Pipeline for Automatic Tractography-Based Brain Parcellation). In this paper, we present a variation of ATPP, which is dedicated to monkey brain parcellation, to address the significant differences in the process between the two species. The new toolbox, MonkeyCBP, has major alterations in three aspects: brain extraction, image registration, and validity indices. By parcellating two different brain regions (posterior cingulate cortex) and (frontal pole) of the rhesus monkey, we demonstrate the efficacy of these alterations. The toolbox has been made public (https://github.com/bheAI/MonkeyCBP_CLI, https://github.com/bheAI/MonkeyCBP_GUI). It is expected that the toolbox can benefit the non-human primate neuroimaging community with high-throughput computation and low labor involvement.

Optimisation of 2D U-Net Model Components for Automatic Prostate Segmentation on MRI

In this paper, we develop an optimised state-of-the-art 2D U-Net model by studying the effects of the individual deep learning model components in performing prostate segmentation. We found that for upsampling, the combination of interpolation and convolution is better than the use of transposed convolution. For combining feature maps in each convolution block, it is only beneficial if a skip connection with concatenation is used. With respect to pooling, average pooling is better than strided-convolution, max, RMS or L2 pooling. Introducing a batch normalisation layer before the activation layer gives further performance improvement. The optimisation is based on a private dataset as it has a fixed 2D resolution and voxel size for every image which mitigates the need of a resizing operation in the data preparation process. Non-enhancing data preprocessing was applied and five-fold cross-validation was used to evaluate the fully automatic segmentation approach. We show it outperforms the traditional methods that were previously applied on the private dataset, as well as outperforming other comparable state-of-the-art 2D models on the public dataset PROMISE12.

VOTENET+ : AN IMPROVED DEEP LEARNING LABEL FUSION METHOD FOR MULTI-ATLAS SEGMENTATION

In this work, we improve the performance of multi-atlas segmentation (MAS) by integrating the recently proposed VoteNet model with the joint label fusion (JLF) approach. Specifically, we first illustrate that using a deep convolutional neural network to predict atlas probabilities can better distinguish correct atlas labels from incorrect ones than relying on image intensity difference as is typical in JLF. Motivated by this finding, we propose VoteNet+, an improved deep network to locally predict the probability of an atlas label to differ from the label of the target image. Furthermore, we show that JLF is more suitable for the VoteNet framework as a label fusion method than plurality voting. Lastly, we use Platt scaling to calibrate the probabilities of our new model. Results on LPBA40 3D MR brain images show that our proposed method can achieve better performance than VoteNet.

Automated Muscle Segmentation from Clinical CT Using Bayesian U-Net for Personalized Musculoskeletal Modeling

We propose a method for automatic segmentation of individual muscles from a clinical CT. The method uses Bayesian convolutional neural networks with the U-Net architecture, using Monte Carlo dropout that infers an uncertainty metric in addition to the segmentation label. We evaluated the performance of the proposed method using two data sets: 20 fully annotated CTs of the hip and thigh regions and 18 partially annotated CTs that are publicly available from The Cancer Imaging Archive (TCIA) database. The experiments showed a Dice coefficient (DC) of 0.891±0.016 (mean±std) and an average symmetric surface distance (ASD) of 0.994±0.230 mm over 19 muscles in the set of 20 CTs. These results were statistically significant improvements compared to the state-of-the-art hierarchical multi-atlas method which resulted in 0.845 ± 0.031 DC and 1.556 ± 0.444 mm ASD. We evaluated validity of the uncertainty metric in the multi-class organ segmentation problem and demonstrated a correlation between the pixels with high uncertainty and the segmentation failure. One application of the uncertainty metric in active-learning is demonstrated, and the proposed query pixel selection method considerably reduced the manual annotation cost for expanding the training data set. The proposed method allows an accurate patient-specific analysis of individual muscle shapes in a clinical routine. This would open up various applications including personalization of biomechanical simulation and quantitative evaluation of muscle atrophy.

Brain MRI analysis using a deep learning based evolutionary approach

Convolutional neural network (CNN) models have recently demonstrated impressive performance in medical image analysis. However, there is no clear understanding of why they perform so well, or what they have learned. In this paper, a three-dimensional convolutional neural network (3D-CNN) is employed to classify brain MRI scans into two predefined groups. In addition, a genetic algorithm based brain masking (GABM) method is proposed as a visualization technique that provides new insights into the function of the 3D-CNN. The proposed GABM method consists of two main steps. In the first step, a set of brain MRI scans is used to train the 3D-CNN. In the second step, a genetic algorithm (GA) is applied to discover knowledgeable brain regions in the MRI scans. The knowledgeable regions are those areas of the brain which the 3D-CNN has mostly used to extract important and discriminative features from them. For applying GA on the brain MRI scans, a new chromosome encoding approach is proposed. The proposed framework has been evaluated using ADNI (including 140 subjects for Alzheimer's disease classification) and ABIDE (including 1000 subjects for Autism classification) brain MRI datasets. Experimental results show a 5-fold classification accuracy of 0.85 for the ADNI dataset and 0.70 for the ABIDE dataset. The proposed GABM method has extracted 6 to 65 knowledgeable brain regions in ADNI dataset (and 15 to 75 knowledgeable brain regions in ABIDE dataset). These regions are interpreted as the segments of the brain which are mostly used by the 3D-CNN to extract features for brain disease classification. Experimental results show that besides the model interpretability, the proposed GABM method has increased final performance of the classification model in some cases with respect to model parameters.

Generalizing Deep Whole Brain Segmentation for Pediatric and Post- Contrast MRI with Augmented Transfer Learning

Generalizability is an important problem in deep neural networks, especially in the context of the variability of data acquisition in clinical magnetic resonance imaging (MRI). Recently, the Spatially Localized Atlas Network Tiles (SLANT) approach has been shown to effectively segment whole brain non-contrast T1w MRI with 132 volumetric labels. Enhancing generalizability of SLANT would enable broader application of volumetric assessment in multi-site studies. Transfer learning (TL) is commonly to update neural network weights for local factors; yet, it is commonly recognized to risk degradation of performance on the original validation/test cohorts. Here, we explore TL by data augmentation to address these concerns in the context of adapting SLANT to anatomical variation (e.g., adults versus children) and scanning protocol (e.g., non-contrast research T1w MRI versus contrast-enhanced clinical T1w MRI). We consider two datasets: First, 30 T1w MRI of young children with manually corrected volumetric labels, and accuracy of automated segmentation defined relative to the manually provided truth. Second, 36 paired datasets of pre- and post-contrast clinically acquired T1w MRI, and accuracy of the post-contrast segmentations assessed relative to the pre-contrast automated assessment. For both studies, we augment the original TL step of SLANT with either only the new data or with both original and new data. Over baseline SLANT, both approaches yielded significantly improved performance (pediatric: 0.89 vs. 0.82 DSC, p<0.001; contrast: 0.80 vs 0.76, p<0.001). The performance on the original test set decreased with the new-data only transfer learning approach, so data augmentation was superior to strict transfer learning.

Automated multi-atlas segmentation of gluteus maximus from Dixon and T1-weighted magnetic resonance images

OBJECTIVE

To design, develop and evaluate an automated multi-atlas method for segmentation and volume quantification of gluteus maximus from Dixon and T1-weighted images.

MATERIALS AND METHODS

The multi-atlas segmentation method uses an atlas library constructed from 15 Dixon MRI scans of healthy subjects. A non-rigid registration between each atlas and the target, followed by majority voting label fusion, is used in the segmentation. We propose a region of interest (ROI) to standardize the measurement of muscle bulk. The method was evaluated using the dice similarity coefficient (DSC) and the relative volume difference (RVD) as metrics, for Dixon and T1-weighted target images.

RESULTS

The mean(± SD) DSC was 0.94 ± 0.01 for Dixon images, while 0.93 ± 0.02 for T1-weighted. The RVD between the automated and manual segmentation had a mean(± SD) value of 1.5 ± 4.3% for Dixon and 1.5 ± 4.8% for T1-weighted images. In the muscle bulk ROI, the DSC was 0.95 ± 0.01 and the RVD was 0.6 ± 3.8%.

CONCLUSION

The method allows an accurate fully automated segmentation of gluteus maximus for Dixon and T1-weighted images and provides a relatively accurate volume measurement in shorter times (~ 20 min) than the current gold-standard manual segmentations (2 h). Visual inspection of the segmentation would be required when higher accuracy is needed.

Semantic segmentation of the multiform proximal femur and femoral head bones with the deep convolutional neural networks in low quality MRI sections acquired in different MRI protocols

Medical image segmentation is one of the most crucial issues in medical image processing and analysis. In general, segmentation of the various structures in medical images is performed for the further image analyzes such as quantification, assessment, diagnosis, prognosis and classification. In this paper, a research study for the 2D semantic segmentation of the multiform, both spheric and aspheric, femoral head and proximal femur bones in magnetic resonance imaging (MRI) sections of the patients with Legg-Calve-Perthes disease (LCPD) with the deep convolutional neural networks (CNNs) is presented. In the scope of the proposed study, bilateral hip MRI sections acquired in coronal plane were used. The main characteristic of the MRI sections that were used is to be low quality images which were obtained in different MRI protocols by using 3 different MRI scanners with 1.5 T imaging capability. In performance evaluations, promising segmentation results were achieved with deep CNNs in low quality MRI sections acquired in different MRI protocols. A success rate about 90% was observed in semantic segmentation of the multiform femoral head and proximal femur bones in a total of 194 MRI sections obtained from 33 MRI sequences of 13 patients with deep CNNs.

Optimizing HR-pQCT workflow: a comparison of bias and precision error for quantitative bone analysis

Manual correction of automatically generated contours for high-resolution peripheral quantitative computed tomography can be time consuming and introduces precision error. However, bias related to the automated protocol is unknown. This study provides insight into error bias that is present when using uncorrected contours and inter-operator precision error based on operator training.

INTRODUCTION

High-resolution peripheral quantitative computed tomography workflow includes manually correcting contours generated by the manufacturer's automated protocol. There is interest in minimizing corrections to save time and reduce precision error; however, bias related to the automated protocol is unknown. This study quantifies error bias when contours are uncorrected and identifies the impact of operator training on bias and precision error.

METHODS

Forty-five radii and tibiae scans across a representative range of bone density were analyzed using the automated and manually corrected contours of three operators, with training ranging from beginner to expert, and compared with a "ground truth" to estimate bias. Inter-operator precision was measured across operators.

RESULTS

The tibia had greater error bias than the radius when contours were uncorrected, with compartmental bone mineral densities and cortical microarchitecture having greatest biases, which could have significant implications for interpretation of studies using this skeletal site. Bias and precision error were greatest when contours were corrected by the beginner operator; however, when this operator was removed, bias was no longer present and inter-operator precision was between 0.01 and 3.74% for all parameters except cortical porosity.

CONCLUSION

These findings establish the need for manual correction and provide guidance on operator training needed to maximize workflow efficiency.

Minimally interactive placenta segmentation from three-dimensional ultrasound images

Purpose: Placental size in early pregnancy has been associated with important clinical outcomes, including fetal growth. However, extraction of placental size from three-dimensional ultrasound (3DUS) requires time-consuming interactive segmentation methods and is prone to user variability. We propose a semiautomated segmentation technique that requires minimal user input to robustly measure placental volume from 3DUS images. Approach: For semiautomated segmentation, a single, central 2D slice was manually annotated to initialize an automated multi-atlas label fusion (MALF) algorithm. The dataset consisted of 47 3DUS volumes obtained at 11 to 14 weeks in singleton pregnancies (28 anterior and 19 posterior). Twenty-six of these subjects were imaged twice within the same session. Dice overlap and surface distance were used to quantify the automated segmentation accuracy compared to expert manual segmentations. The mean placental volume measurements obtained by our method and VOCAL (virtual organ computer-aided analysis), a leading commercial semiautomated method, were compared to the manual reference set. The test-retest reliability was also assessed. Results: The overlap between our automated segmentation and manual (mean Dice: 0.824 ± 0.061 , median: 0.831) was within the range reported by other methods requiring extensive manual input. The average surface distance was 1.66 ± 0.96    mm . The correlation coefficient between test-retest volumes was r = 0.88 , and the intraclass correlation was ICC ( 1 ) = 0.86 . Conclusions: MALF is a promising method that can allow accurate and reliable segmentation of the placenta with minimal user interaction. Further refinement of this technique may allow for placental biometry to be incorporated into clinical pregnancy surveillance.

Self-fusion for OCT noise reduction

Reducing speckle noise is an important task for improving visual and automated assessment of retinal OCT images. Traditional image/signal processing methods only offer moderate speckle reduction; deep learning methods can be more effective but require substantial training data, which may not be readily available. We present a novel self-fusion method that offers effective speckle reduction comparable to deep learning methods, but without any external training data. We present qualitative and quantitative results in a variety of datasets from fovea and optic nerve head regions, with varying SNR values for input images.

Quantification in Musculoskeletal Imaging Using Computational Analysis and Machine Learning: Segmentation and Radiomics

Although still limited in clinical practice, quantitative analysis is expected to increase the value of musculoskeletal (MSK) imaging. Segmentation aims at isolating the tissues and/or regions of interest in the image and is crucial to the extraction of quantitative features such as size, signal intensity, or image texture. These features may serve to support the diagnosis and monitoring of disease. Radiomics refers to the process of extracting large amounts of features from radiologic images and combining them with clinical, biological, genetic, or any other type of complementary data to build diagnostic, prognostic, or predictive models. The advent of machine learning offers promising prospects for automatic segmentation and integration of large amounts of data. We present commonly used segmentation methods and describe the radiomics pipeline, highlighting the challenges to overcome for adoption in clinical practice. We provide some examples of applications from the MSK literature.

Evaluation of cortical segmentation pipelines on clinical neonatal MRI data

Magnetic Resonance Imaging (MRI) can provide 3D morphological information on brain structures. Such information is particularly relevant for carrying out morphometric brain analysis, especially in the newborn and in the case of prematurity. However, 3D neonatal MRI acquired in clinical environments are low-resolution, anisotropic images, making segmentation a challenging task. In this context, preprocessing techniques aim to increase the image resolution. Interpolation techniques were classically used; super-resolution (SR) techniques have recently appeared as an emerging alternative. In this paper, we evaluate the performance of different SR methods against the classical interpolation in the application of neonatal cortex segmentation. Additionally, we assess the robustness of different segmentation methods for each estimation of high resolution MRI input. Results are evaluated both qualitatively and quantitatively with neonatal clinical MRI.

Simultaneous Multi-Structure Segmentation of the Heart and Peripheral Tissues in Contrast Enhanced Cardiac Computed Tomography Angiography

Contrast enhanced cardiac computed tomography angiography (CTA) is a prominent imaging modality for diagnosing cardiovascular diseases non-invasively. It assists the evaluation of the coronary artery patency and provides a comprehensive assessment of structural features of the heart and great vessels. However, physicians are often required to evaluate different cardiac structures and measure their size manually. Such task is very time-consuming and tedious due to the large number of image slices in 3D data. We present a fully automatic method based on a combined multi-atlas and corrective segmentation approach to label the heart and its associated cardiovascular structures. This method also automatically separates other surrounding intrathoracic structures from CTA images. Quantitative assessment of the proposed method is performed on 36 studies with a reference standard obtained from expert manual segmentation of various cardiac structures. Qualitative evaluation is also performed by expert readers to score 120 studies of the automatic segmentation. The quantitative results showed an overall Dice of 0.93, Hausdorff distance of 7.94 mm, and mean surface distance of 1.03 mm between automatically and manually segmented cardiac structures. The visual assessment also attained an excellent score for the automatic segmentation. The average processing time was 2.79 minutes. Our results indicate the proposed automatic framework significantly improves accuracy and computational speed in conventional multi-atlas based approach, and it provides comprehensive and reliable multi-structural segmentation of CTA images that is valuable for clinical application.

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.

An empirical assessment of appearance descriptors applied to MRI for automated diagnosis of TLE and MCI

OBJECTIVE

Differential diagnosis of mild cognitive impairment MCI and temporal lobe epilepsy TLE is a debated issue, specifically because these conditions may coincide in the elderly population. We evaluate automated differential diagnosis based on characteristics derived from structural brain MRI of different brain regions.

METHODS

In 22 healthy controls, 19 patients with MCI, and 17 patients with TLE we used scale invariant feature transform (SIFT), local binary patterns (LBP), and wavelet-based features and investigate their predictive performance for MCI and TLE.

RESULTS

The classification based on SIFT features resulted in an accuracy of 81% of MCI vs. TLE and reasonable generalizability. Local binary patterns yielded satisfactory diagnostic performance with up to 94.74% sensitivity and 88.24% specificity in the right Thalamus for the distinction of MCI vs. TLE, but with limited generalizable. Wavelet features yielded similar results as LPB with 94.74% sensitivity and 82.35% specificity but generalize better.

SIGNIFICANCE

Features beyond volume analysis are a valid approach when applied to specific regions of the brain. Most significant information could be extracted from the thalamus, frontal gyri, and temporal regions, among others. These results suggest that analysis of changes of the central nervous system should not be limited to the most typical regions of interest such as the hippocampus and parahippocampal areas. Region-independent approaches can add considerable information for diagnosis. We emphasize the need to characterize generalizability in future studies, as our results demonstrate that not doing so can lead to overestimation of classification results.

LIMITATIONS

The data used within this study allows for separation of MCI and TLE subjects using a simple age threshold. While we present a strong indication that the presented method is age-invariant and therefore agnostic to this situation, new data would be needed for a rigorous empirical assessment of this findings.

Automated measurement of bone scan index from a whole-body bone scintigram

Purpose

We propose a deep learning-based image interpretation system for skeleton segmentation and extraction of hot spots of bone metastatic lesion from a whole-body bone scintigram followed by automated measurement of a bone scan index (BSI), which will be clinically useful.

Methods

The proposed system employs butterfly-type networks (BtrflyNets) for skeleton segmentation and extraction of hot spots of bone metastatic lesions, in which a pair of anterior and posterior images are processed simultaneously. BSI is then measured using the segmented bones and extracted hot spots. To further improve the networks, deep supervision (DSV) and residual learning technologies were introduced.

Results

We evaluated the performance of the proposed system using 246 bone scintigrams of prostate cancer in terms of accuracy of skeleton segmentation, hot spot extraction, and BSI measurement, as well as computational cost. In a threefold cross-validation experiment, the best performance was achieved by BtrflyNet with DSV for skeleton segmentation and BtrflyNet with residual blocks. The cross-correlation between the measured and true BSI was 0.9337, and the computational time for a case was 112.0 s.

Conclusion

We proposed a deep learning-based BSI measurement system for a whole-body bone scintigram and proved its effectiveness by threefold cross-validation study using 246 whole-body bone scintigrams. The automatically measured BSI and computational time for a case are deemed clinically acceptable and reliable.

Age-Dependent White Matter Characteristics of the Cerebellar Peduncles from Infancy Through Adolescence

Cerebellum-cerebrum connections are essential for many motor and cognitive functions and cerebellar disorders are prevalent in childhood. The middle (MCP), inferior (ICP), and superior cerebellar peduncles (SCP) are the major white matter pathways that permit communication between the cerebellum and the cerebrum. Knowledge about the microstructural properties of these cerebellar peduncles across childhood is limited. Here, we report on a diffusion magnetic resonance imaging tractography study to describe age-dependent characteristics of the cerebellar peduncles in a cross-sectional sample of infants, children, and adolescents from newborn to 17 years of age (N = 113). Scans were collected as part of clinical care; participants were restricted to those whose scans showed no abnormal findings and whose history and exam had no risk factors for cerebellar abnormalities. A novel automated tractography protocol was applied. Results showed that mean tract-FA increased, while mean tract-MD decreased from infancy to adolescence in all peduncles. Rapid changes were observed in both diffusion measures in the first 24 months of life, followed by gradual change at older ages. The shape of the tract profiles was similar across ages for all peduncles. These data are the first to characterize the variability of diffusion properties both across and within cerebellar white matter pathways that occur from birth through later adolescence. The data represent a rich normative data set against which white matter alterations seen in children with posterior fossa conditions can be compared. Ultimately, the data will facilitate the identification of sensitive biomarkers of cerebellar abnormalities.

Accuracy of computer-assisted image analysis in the diagnosis of maxillofacial radiolucent lesions: A systematic review and meta-analysis

OBJECTIVES

This study aimed to search for scientific evidence concerning the accuracy of computer-assisted analysis for diagnosing maxillofacial radiolucent lesions.

METHODS

A systematic review was conducted according to the statements of Preferred Reporting Items for Systematic Reviews and Meta-analyses Protocols and considering 10 databases, including the gray literature. Protocol was registered at the International Prospective Register of Systematic Reviews (CRD42018089945). The population, intervention, comparison and outcome strategy was used to define the eligibility criteria and only diagnostic test studies were included. Their risk of bias was assessed by the Joanna Briggs Institute Critical Appraisal tool. Random-effects model meta-analysis was performed and heterogeneity among the included studies was estimated using the I² statistic. The grade of recommendation, assessment, development, and evaluation (GRADE) tool assessed the quality of evidence and strength of recommendation across included studies.

RESULTS

Out of 715 identified citations, four papers, published between 2009 and 2017, fulfilled the criteria and were included in this systematic review. A total of 191 lesions, classified as periapical granuloma and cyst, dentigerous cyst or keratocystic odontogenic tumor, were analyzed. All selected articles scored low risk of bias. The pooled accuracy estimation, regardless of the classification method used, was 88.75% (95% CI = 85.19-92.30). Heterogeneity test reached moderate values (I² = 57.89%). According to the GRADE tool, the analyzed outcome was classified as having low level of certainty.

CONCLUSIONS

The overall evaluation showed all studies presented high accuracy rates of computer-aided diagnosis systems in classifying radiolucent maxillofacial lesions compared to histopathological biopsy. However, due to the moderate heterogeneity found among the studies included in this meta-analysis, a pragmatic recommendation about the use of computer-assisted analysis is not possible.

Multi-atlas label fusion with random local binary pattern features: Application to hippocampus segmentation

Automatic and reliable segmentation of the hippocampus from magnetic resonance (MR) brain images is extremely important in a variety of neuroimage studies. To improve the hippocampus segmentation performance, a local binary pattern based feature extraction method is developed for machine learning based multi-atlas hippocampus segmentation. Under the framework of multi-atlas image segmentation (MAIS), a set of selected atlases are registered to images to be segmented using a non-linear image registration algorithm. The registered atlases are then used as training data to build linear regression models for segmenting the images based on the image features, referred to as random local binary pattern (RLBP), extracted using a novel image feature extraction method. The RLBP based MAIS algorithm has been validated for segmenting hippocampus based on a data set of 135 T1 MR images which are from the Alzheimer’s Disease Neuroimaging Initiative database (adni.loni.usc.edu). By using manual segmentation labels produced by experienced tracers as the standard of truth, six segmentation evaluation metrics were used to evaluate the image segmentation results by comparing automatic segmentation results with the manual segmentation labels. We further computed Cohen’s d effect size to investigate the sensitivity of each segmenting method in detecting volumetric differences of the hippocampus between different groups of subjects. The evaluation results showed that our method was competitive to state-of-the-art label fusion methods in terms of accuracy. Hippocampal volumetric analysis showed that the proposed RLBP method performed well in detecting the volumetric differences of the hippocampus between groups of Alzheimer’s disease patients, mild cognitive impairment subjects, and normal controls. These results have demonstrated that the RLBP based multi-atlas image segmentation method could facilitate efficient and accurate extraction of the hippocampus and may help predict Alzheimer’s disease. The codes of the proposed method is available (https://www.nitrc.org/frs/?group_id=1242).

High-order Feature Learning for Multi-atlas based Label Fusion: Application to Brain Segmentation with MRI

Multi-atlas based segmentation methods have shown their effectiveness in brain regions-of-interesting (ROIs) segmentation, by propagating labels from multiple atlases to a target image based on the similarity between patches in the target image and multiple atlas images. Most of the existing multiatlas based methods use image intensity features to calculate the similarity between a pair of image patches for label fusion. In particular, using only low-level image intensity features cannot adequately characterize the complex appearance patterns (e.g., the high-order relationship between voxels within a patch) of brain magnetic resonance (MR) images. To address this issue, this paper develops a high-order feature learning framework for multi-atlas based label fusion, where high-order features of image patches are extracted and fused for segmenting ROIs of structural brain MR images. Specifically, an unsupervised feature learning method (i.e., means-covariances restricted Boltzmann machine, mcRBM) is employed to learn high-order features (i.e., mean and covariance features) of patches in brain MR images. Then, a group-fused sparsity dictionary learning method is proposed to jointly calculate the voting weights for label fusion, based on the learned high-order and the original image intensity features. The proposed method is compared with several state-of-the-art label fusion methods on ADNI, NIREP and LONI-LPBA40 datasets. The Dice ratio achieved by our method is 88:30%, 88:83%, 79:54% and 81:02% on left and right hippocampus on the ADNI, NIREP and LONI-LPBA40 datasets, respectively, while the best Dice ratio yielded by the other methods are 86:51%, 87:39%, 78:48% and 79:65% on three datasets, respectively.

TeTrIS: Template Transformer Networks for Image Segmentation With Shape Priors

In this paper, we introduce and compare different approaches for incorporating shape prior information into neural network-based image segmentation. Specifically, we introduce the concept of template transformer networks, where a shape template is deformed to match the underlying structure of interest through an end-to-end trained spatial transformer network. This has the advantage of explicitly enforcing shape priors, and this is free of discretization artifacts by providing a soft partial volume segmentation. We also introduce a simple yet effective way of incorporating priors in the state-of-the-art pixel-wise binary classification methods such as fully convolutional networks and U-net. Here, the template shape is given as an additional input channel, incorporating this information significantly reduces false positives. We report results on synthetic data and sub-voxel segmentation of coronary lumen structures in cardiac computed tomography showing the benefit of incorporating priors in neural network-based image segmentation.