Cortical Surface Parcellation using Spherical Convolutional Neural Networks

Automatic cortical surface parcellation in the fetal brain using attention-gated spherical U-net

Cortical surface parcellation for fetal brains is essential for the understanding of neurodevelopmental trajectories during gestations with regional analyses of brain structures and functions. This study proposes the attention-gated spherical U-net, a novel deep-learning model designed for automatic cortical surface parcellation of the fetal brain. We trained and validated the model using MRIs from 55 typically developing fetuses [gestational weeks: 32.9 ± 3.3 (mean ± SD), 27.4-38.7]. The proposed model was compared with the surface registration-based method, SPHARM-net, and the original spherical U-net. Our model demonstrated significantly higher accuracy in parcellation performance compared to previous methods, achieving an overall Dice coefficient of 0.899 ± 0.020. It also showed the lowest error in terms of the median boundary distance, 2.47 ± 1.322 (mm), and mean absolute percent error in surface area measurement, 10.40 ± 2.64 (%). In this study, we showed the efficacy of the attention gates in capturing the subtle but important information in fetal cortical surface parcellation. Our precise automatic parcellation model could increase sensitivity in detecting regional cortical anomalies and lead to the potential for early detection of neurodevelopmental disorders in fetuses.

A joint parcellation and boundary network with multi-rate-shared dilated graph attention for cortical surface parcellation

Cortical surface parcellation aims to segment the surface into anatomically and functionally significant regions, which are crucial for diagnosing and treating numerous neurological diseases. However, existing methods generally ignore the difficulty in learning labeling patterns of boundaries, hindering the performance of parcellation. To this end, this paper proposes a joint parcellation and boundary network (JPBNet) to promote the effectiveness of cortical surface parcellation. Its core is developing a multi-rate-shared dilated graph attention (MDGA) module and incorporating boundary learning into the parcellation process. The former, in particular, constructs a dilated graph attention strategy, extending the dilated convolution from regular data to irregular graph data. We fuse it with different dilated rates to extract context information in various scales by devising a shared graph attention layer. After that, a boundary enhancement module and a parcellation enhancement module based on graph attention mechanisms are built in each layer, forcing MDGA to capture informative and valuable features for boundary detection and parcellation tasks. Integrating MDGA, the boundary enhancement module, and the parcellation enhancement module at each layer to supervise boundary and parcellation information, an effective JPBNet is formed by stacking multiple layers. Experiments on the public dataset reveal that the proposed method outperforms comparison methods and performs well on boundaries for cortical surface parcellation.

Comprehensive shape analysis of the cortex in Huntington's disease

The striatum has traditionally been the focus of Huntington's disease research due to the primary insult to this region and its central role in motor symptoms. Beyond the striatum, evidence of cortical alterations caused by Huntington's disease has surfaced. However, findings are not coherent between studies which have used cortical thickness for Huntington's disease since it is the well-established cortical metric of interest in other diseases. In this study, we propose a more comprehensive approach to cortical morphology in Huntington's disease using cortical thickness, sulcal depth, and local gyrification index. Our results show consistency with prior findings in cortical thickness, including its limitations. Our comparison between cortical thickness and local gyrification index underscores the complementary nature of these two measures-cortical thickness detects changes in the sensorimotor and posterior areas while local gyrification index identifies insular differences. Since local gyrification index and cortical thickness measures detect changes in different regions, the two used in tandem could provide a clinically relevant measure of disease progression. Our findings suggest that differences in insular regions may correspond to earlier neurodegeneration and may provide a complementary cortical measure for detection of subtle early cortical changes due to Huntington's disease.

Integrating the BIDS Neuroimaging Data Format and Workflow Optimization for Large-Scale Medical Image Analysis

A robust medical image computing infrastructure must host massive multimodal archives, perform extensive analysis pipelines, and execute scalable job management. An emerging data format standard, the Brain Imaging Data Structure (BIDS), introduces complexities for interfacing with XNAT archives. Moreover, workflow integration is combinatorically problematic when matching large amount of processing to large datasets. Historically, workflow engines have been focused on refining workflows themselves instead of actual job generation. However, such an approach is incompatible with data centric architecture that hosts heterogeneous medical image computing. Distributed automation for XNAT toolkit (DAX) provides large-scale image storage and analysis pipelines with an optimized job management tool. Herein, we describe developments for DAX that allows for integration of XNAT and BIDS standards. We also improve DAX's efficiencies of diverse containerized workflows in a high-performance computing (HPC) environment. Briefly, we integrate YAML configuration processor scripts to abstract workflow data inputs, data outputs, commands, and job attributes. Finally, we propose an online database-driven mechanism for DAX to efficiently identify the most recent updated sessions, thereby improving job building efficiency on large projects. We refer the proposed overall DAX development in this work as DAX-1 (DAX version 1). To validate the effectiveness of the new features, we verified (1) the efficiency of converting XNAT data to BIDS format and the correctness of the conversion using a collection of BIDS standard containerized neuroimaging workflows, (2) how YAML-based processor simplified configuration setup via a sequence of application pipelines, and (3) the productivity of DAX-1 on generating actual HPC processing jobs compared with earlier DAX baseline method. The empirical results show that (1) DAX-1 converting XNAT data to BIDS has similar speed as accessing XNAT data only; (2) YAML can integrate to the DAX-1 with shallow learning curve for users, and (3) DAX-1 reduced the job/assessor generation latency by finding recent modified sessions. Herein, we present approaches for efficiently integrating XNAT and modern image formats with a scalable workflow engine for the large-scale dataset access and processing.

SPHARM-Net: Spherical Harmonics-Based Convolution for Cortical Parcellation

We present a spherical harmonics-based convolutional neural network (CNN) for cortical parcellation, which we call SPHARM-Net. Recent advances in CNNs offer cortical parcellation on a fine-grained triangle mesh of the cortex. Yet, most CNNs designed for cortical parcellation employ spatial convolution that depends on extensive data augmentation and allows only predefined neighborhoods of specific spherical tessellation. On the other hand, a rotation-equivariant convolutional filter avoids data augmentation, and rotational equivariance can be achieved in spectral convolution independent of a neighborhood definition. Nevertheless, the limited resources of a modern machine enable only a finite set of spectral components that might lose geometric details. In this paper, we propose (1) a constrained spherical convolutional filter that supports an infinite set of spectral components and (2) an end-to-end framework without data augmentation. The proposed filter encodes all the spectral components without the full expansion of spherical harmonics. We show that rotational equivariance drastically reduces the training time while achieving accurate cortical parcellation. Furthermore, the proposed convolution is fully composed of matrix transformations, which offers efficient and fast spectral processing. In the experiments, we validate SPHARM-Net on two public datasets with manual labels: Mindboggle-101 (N=101) and NAMIC (N=39). The experimental results show that the proposed method outperforms the state-of-the-art methods on both datasets even with fewer learnable parameters without rigid alignment and data augmentation. Our code is publicly available at https://github.com/Shape-Lab/SPHARM-Net.

Joint independent component analysis for hypothesizing spatiotemporal relationships between longitudinal gray and white matter changes in preclinical Alzheimer's disease

Characterizing relationships between gray matter (GM) and white matter (WM) in early Alzheimer's disease (AD) would improve understanding of how and when AD impacts the brain. However, modeling these relationships across brain regions and longitudinally remains a challenge. Thus, we propose extending joint independent component analysis (jICA) into spatiotemporal modeling of regional cortical thickness and WM bundle volumes leveraging multimodal MRI. We jointly characterize these GM and WM features in a normal aging (n=316) and an age- and sex-matched preclinical AD cohort (n=81) at each of two imaging sessions spaced three years apart, training on the normal aging population in cross-validation and interrogating the preclinical AD cohort. We find this joint model identifies reproducible, longitudinal changes in GM and WM between the two imaging sessions and that these changes are associated with preclinical AD and are plausible considering the literature. We compare this joint model to two focused models: (1) GM features at the first session and WM at the second and (2) vice versa. The joint model identifies components that correlate poorly with those from the focused models, suggesting the different models resolve different patterns. We find the strength of association with preclinical AD is improved in the GM to WM model, which supports the hypothesis that medial temporal and frontal thinning precedes volume loss in the uncinate fasciculus and inferior anterior-posterior association fibers. These results suggest that jICA effectively generates spatiotemporal hypotheses about GM and WM in preclinical AD, especially when specific intermodality relationships are considered a priori.

Multimodal neuroimaging in pediatric type 1 diabetes: a pilot multisite feasibility study of acquisition quality, motion, and variability

Type 1 diabetes (T1D) affects over 200,000 children and is associated with an increased risk of cognitive dysfunction. Prior imaging studies suggest the neurological changes underlying this risk are multifactorial, including macrostructural, microstructural, and inflammatory changes. However, these studies have yet to be integrated, limiting investigation into how these phenomena interact. To better understand these complex mechanisms of brain injury, a well-powered, prospective, multisite, and multimodal neuroimaging study is needed. We take the first step in accomplishing this with a preliminary characterization of multisite, multimodal MRI quality, motion, and variability in pediatric T1D. We acquire structural T1 weighted (T1w) MRI, diffusion tensor MRI (DTI), functional MRI (fMRI), and magnetic resonance spectroscopy (MRS) of 5-7 participants from each of two sites. First, we assess the contrast-to-noise ratio of the T1w MRI and find no differences between sites. Second, we characterize intervolume motion in DTI and fMRI and find it to be on the subvoxel level. Third, we investigate variability in regional gray matter volumes and local gyrification indices, bundle-wise DTI microstructural measures, and N-acetylaspartate to creatine ratios. We find the T1-based measures to be comparable between sites before harmonization and the DTI and MRS-based measures to be comparable after. We find a 5-15% coefficient of variation for most measures, suggesting ~150-200 participants per group on average are needed to detect a 5% difference across these modalities at 0.9 power. We conclude that multisite, multimodal neuroimaging of pediatric T1D is feasible with low motion artifact after harmonization of DTI and MRS.

S3Reg: Superfast Spherical Surface Registration Based on Deep Learning

Cortical surface registration is an essential step and prerequisite for surface-based neuroimaging analysis. It aligns cortical surfaces across individuals and time points to establish cross-sectional and longitudinal cortical correspondences to facilitate neuroimaging studies. Though achieving good performance, available methods are either time consuming or not flexible to extend to multiple or high dimensional features. Considering the explosive availability of large-scale and multimodal brain MRI data, fast surface registration methods that can flexibly handle multimodal features are desired. In this study, we develop a Superfast Spherical Surface Registration (S3Reg) framework for the cerebral cortex. Leveraging an end-to-end unsupervised learning strategy, S3Reg offers great flexibility in the choice of input feature sets and output similarity measures for registration, and meanwhile reduces the registration time significantly. Specifically, we exploit the powerful learning capability of spherical Convolutional Neural Network (CNN) to directly learn the deformation fields in spherical space and implement diffeomorphic design with "scaling and squaring" layers to guarantee topology-preserving deformations. To handle the polar-distortion issue, we construct a novel spherical CNN model using three orthogonal Spherical U-Nets. Experiments are performed on two different datasets to align both adult and infant multimodal cortical features. Results demonstrate that our S3Reg shows superior or comparable performance with state-of-the-art methods, while improving the registration time from 1 min to 10 sec.

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

With the advances of data-driven machine learning research, a wide variety of prediction problems have been tackled. It has become critical to explore how machine learning and specifically deep learning methods can be exploited to analyse healthcare data. A major limitation of existing methods has been the focus on grid-like data; however, the structure of physiological recordings are often irregular and unordered, which makes it difficult to conceptualise them as a matrix. As such, graph neural networks have attracted significant attention by exploiting implicit information that resides in a biological system, with interacting nodes connected by edges whose weights can be determined by either temporal associations or anatomical junctions. In this survey, we thoroughly review the different types of graph architectures and their applications in healthcare. We provide an overview of these methods in a systematic manner, organized by their domain of application including functional connectivity, anatomical structure, and electrical-based analysis. We also outline the limitations of existing techniques and discuss potential directions for future research.

Labeling lateral prefrontal sulci using spherical data augmentation and context-aware training

The inference of cortical sulcal labels often focuses on deep (primary and secondary) sulcal regions, whereas shallow (tertiary) sulcal regions are largely overlooked in the literature due to the scarcity of manual/well-defined annotations and their large neuroanatomical variability. In this paper, we present an automated framework for regional labeling of both primary/secondary and tertiary sulci of the dorsal portion of lateral prefrontal cortex (LPFC) using spherical convolutional neural networks. We propose two core components that enhance the inference of sulcal labels to overcome such large neuroanatomical variability: (1) surface data augmentation and (2) context-aware training. (1) To take into account neuroanatomical variability, we synthesize training data from the proposed feature space that embeds intermediate deformation trajectories of spherical data in a rigid to non-rigid fashion, which bridges an augmentation gap in conventional rotation data augmentation. (2) Moreover, we design a two-stage training process to improve labeling accuracy of tertiary sulci by informing the biological associations in neuroanatomy: inference of primary/secondary sulci and then their spatial likelihood to guide the definition of tertiary sulci. In the experiments, we evaluate our method on 13 deep and shallow sulci of human LPFC in two independent data sets with different age ranges: pediatric (N=60) and adult (N=36) cohorts. We compare the proposed method with a conventional multi-atlas approach and spherical convolutional neural networks without/with rotation data augmentation. In both cohorts, the proposed data augmentation improves labeling accuracy of deep and shallow sulci over the baselines, and the proposed context-aware training offers further improvement in the labeling of shallow sulci over the proposed data augmentation. We share our tools with the field and discuss applications of our results for understanding neuroanatomical-functional organization of LPFC and the rest of cortex (https://github.com/ilwoolyu/SphericalLabeling).

Spherical Deformable U-Net: Application to Cortical Surface Parcellation and Development Prediction

Convolutional Neural Networks (CNNs) have achieved overwhelming success in learning-related problems for 2D/3D images in the Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have an inherent spherical topology in a manifold space, e.g., the convoluted brain cortical surfaces represented by triangular meshes. There is no consistent neighborhood definition and thus no straightforward convolution/pooling operations for such cortical surface data. In this paper, leveraging the regular and hierarchical geometric structure of the resampled spherical cortical surfaces, we create the 1-ring filter on spherical cortical triangular meshes and accordingly develop convolution/pooling operations for constructing Spherical U-Net for cortical surface data. However, the regular nature of the 1-ring filter makes it inherently limited to model fixed geometric transformations. To further enhance the transformation modeling capability of Spherical U-Net, we introduce the deformable convolution and deformable pooling to cortical surface data and accordingly propose the Spherical Deformable U-Net (SDU-Net). Specifically, spherical offsets are learned to freely deform the 1-ring filter on the sphere to adaptively localize cortical structures with different sizes and shapes. We then apply the SDU-Net to two challenging and scientifically important tasks in neuroimaging: cortical surface parcellation and cortical attribute map prediction. Both applications validate the competitive performance of our approach in accuracy and computational efficiency in comparison with state-of-the-art methods.

Establishing Surface Correspondence for Post-surgical Cortical Thickness Changes in Temporal Lobe Epilepsy

In pre- and post-surgical surface shape analysis, establishing shape correspondence is necessary to investigate the postoperative surface changes. However, structural absence after the operation accompanies focal non-rigid changes, which leads to challenges in existing surface registration methods. In this paper, we present a fully automatic particle-based method to establish surface correspondence that can handle partial structural abnormality in the temporal lobe resection. Our method optimizes the coordinates of points which are modeled as particles on surfaces in a hierarchical way to reduce a chance of being trapped in a local minimum during the optimization. In the experiments, we evaluate the effectiveness of our method in comparison with conventional spherical registration (FreeSurfer) on two scenarios: cortical thickness changes in healthy controls within a short scan-rescan time window and patients with temporal lobe resection. The post-surgical scan is acquired at least 1 year after the presurgical scan. In region of interest-wise (ROI-wise) analysis, no changes on cortical thickness are found in both methods on the healthy control group. In patients, since there is no ground truth available, we instead investigated the disagreement between our method and FreeSurfer. We see poorly matched ROIs and large cortical thickness changes using FreeSurfer. On the contrary, our method shows well-matched ROIs and subtle cortical thickness changes. This suggests that the proposed method can establish a stable shape correspondence, which is not fully captured in a conventional spherical registration.

Joint analysis of structural connectivity and cortical surface features: correlates with mild traumatic brain injury

Mild traumatic brain injury (mTBI) is a complex syndrome that affects up to 600 per 100,000 individuals, with a particular concentration among military personnel. About half of all mTBI patients experience a diverse array of chronic symptoms which persist long after the acute injury. Hence, there is an urgent need for better understanding of the white matter and gray matter pathologies associated with mTBI to map which specific brain systems are impacted and identify courses of intervention. Previous works have linked mTBI to disruptions in white matter pathways and cortical surface abnormalities. Herein, we examine these hypothesized links in an exploratory study of joint structural connectivity and cortical surface changes associated with mTBI and its chronic symptoms. Briefly, we consider a cohort of 12 mTBI and 26 control subjects. A set of 588 cortical surface metrics and 4,753 structural connectivity metrics were extracted from cortical surface regions and diffusion weighted magnetic resonance imaging in each subject. Principal component analysis (PCA) was used to reduce the dimensionality of each metric set. We then applied independent component analysis (ICA) both to each PCA space individually and together in a joint ICA approach. We identified a stable independent component across the connectivity-only and joint ICAs which presented significant group differences in subject loadings (p<0.05, corrected). Additionally, we found that two mTBI symptoms, slowed thinking and forgetfulness, were significantly correlated (p<0.05, corrected) with mTBI subject loadings in a surface-only ICA. These surface-only loadings captured an increase in bilateral cortical thickness.

Joint cortical surface and structural connectivity analysis of Alzheimer's Disease

Prior neuroimaging studies have demonstrated isolated structural and connectivity changes in the brain due to Alzheimer's Disease (AD). However, how these changes relate to each other is not well understood. We present a preliminary study to begin to fill this gap by leveraging joint independent component analysis (jICA). We explore how jICA performs in an analysis of T1 and diffusion weighted MRI by characterizing the joint changes of complex cortical surface and structural connectivity metrics in AD in subjects from the Baltimore Longitudinal Study of Aging. We calculate 588 region-based cortical metrics and 4,753 fractional anisotropy-based connectivity metrics and project them into a low-dimensional manifold with principal component analysis. We perform jICA on the manifold and subsequently backproject the independent components to the original data space. We demonstrate component stability with 3-fold cross validation and find differential component loadings between 776 cognitively unimpaired control subjects and 23 with AD that generalizes across folds. In addition, we perform the same analysis on the surface and connectivity metrics separately and find that the joint approach identifies both novel and similar components to the separate approaches. To illustrate the joint approach's primary utility, we provide an example hypothesis for how surface and connectivity components may vary together with AD. These preliminary results suggest jointly varying independent cortical surface and structural connectivity components can be consistently extracted from MRI data and provide a data-driven way for generating novel hypotheses about AD that may not be captured by separate analyses.

AUTOMATIC LABELING OF CORTICAL SULCI USING SPHERICAL CONVOLUTIONAL NEURAL NETWORKS IN A DEVELOPMENTAL COHORT

In this paper, we present the automatic labeling framework for sulci in the human lateral prefrontal cortex (PFC). We adapt an existing spherical U-Net architecture with our recent surface data augmentation technique to improve the sulcal labeling accuracy in a developmental cohort. Specifically, our framework consists of the following key components: (1) augmented geometrical features being generated during cortical surface registration, (2) spherical U-Net architecture to efficiently fit the augmented features, and (3) postrefinement of sulcal labeling by optimizing spatial coherence via a graph cut technique. We validate our method on 30 healthy subjects with manual labeling of sulcal regions within PFC. In the experiments, we demonstrate significantly improved labeling performance (0.7749) in mean Dice overlap compared to that of multi-atlas (0.6410) and standard spherical U-Net (0.7011) approaches, respectively (p < 0.05). Additionally, the proposed method achieves a full set of sulcal labels in 20 seconds in this developmental cohort.