Geometric deep learning on brain shape predicts sex and age

A deep learning-based pipeline for developing multi-rib shape generative model with populational percentiles or anthropometrics as predictors

Rib cross-sectional shapes (characterized by the outer contour and cortical bone thickness) affect the rib mechanical response under impact loading, thereby influence the rib injury pattern and risk. A statistical description of the rib shapes or their correlations to anthropometrics is a prerequisite to the development of numerical human body models representing target demographics. Variational autoencoders (VAE) as anatomical shape generators remain to be explored in terms of utilizing the latent vectors to control or interpret the representativeness of the generated results. In this paper, we propose a pipeline for developing a multi-rib cross-sectional shape generative model from CT images, which consists of the achievement of rib cross-sectional shape data from CT images using an anatomical indexing system and regular grids, and a unified framework to fit shape distributions and associate shapes to anthropometrics for different rib categories. Specifically, we collected CT images including 3193 ribs, surface regular grid is generated for each rib based on anatomical coordinates, the rib cross-sectional shapes are characterized by nodal coordinates and cortical bone thickness. The tensor structure of shape data based on regular grids enable the implementation of CNNs in the conditional variational autoencoder (CVAE). The CVAE is trained against an auxiliary classifier to decouple the low-dimensional representations of the inter- and intra- variations and fit each intra-variation by a Gaussian distribution simultaneously. Random tree regressors are further leveraged to associate each continuous intra-class space with the corresponding anthropometrics of the subjects, i.e., age, height and weight. As a result, with the rib class labels and the latent vectors sampled from Gaussian distributions or predicted from anthropometrics as the inputs, the decoder can generate valid rib cross-sectional shapes of given class labels (male/female, 2nd to 11th ribs) for arbitrary populational percentiles or specific age, height and weight, which paves the road for future biomedical and biomechanical studies considering the diversity of rib shapes across the population.

Brain age prediction using combined deep convolutional neural network and multi-layer perceptron algorithms

The clinical applications of brain age prediction have expanded, particularly in anticipating the onset and prognosis of various neurodegenerative diseases. In the current study, we proposed a deep learning algorithm that leverages brain structural imaging data and enhances prediction accuracy by integrating biological sex information. Our model for brain age prediction, built on deep neural networks, employed a dataset of 3004 healthy subjects aged 18 and above. The T1-weighted images were minimally preprocessed and analyzed using the convolutional neural network (CNN) algorithm. The categorical sex information was then incorporated using the multi-layer perceptron (MLP) algorithm. We trained and validated both a CNN-only algorithm (utilizing only brain structural imaging data), and a combined CNN-MLP algorithm (using both structural brain imaging data and sex information) for age prediction. By integrating sex information with T1-weighted imaging data, our proposed CNN-MLP algorithm outperformed not only the CNN-only algorithm but also established algorithms, such as brainageR, in prediction accuracy. Notably, this hybrid CNN-MLP algorithm effectively distinguished between mild cognitive impairment and Alzheimer's disease groups by identifying variances in brain age gaps between them, highlighting the algorithm's potential for clinical application. Overall, these results underscore the enhanced precision of the CNN-MLP algorithm in brain age prediction, achieved through the integration of sex information.

Automatic landmark detection and registration of brain cortical surfaces via quasi-conformal geometry and convolutional neural networks

In medical imaging, surface registration is extensively used for performing systematic comparisons between anatomical structures, with a prime example being the highly convoluted brain cortical surfaces. To obtain a meaningful registration, a common approach is to identify prominent features on the surfaces and establish a low-distortion mapping between them with the feature correspondence encoded as landmark constraints. Prior registration works have primarily focused on using manually labeled landmarks and solving highly nonlinear optimization problems, which are time-consuming and hence hinder practical applications. In this work, we propose a novel framework for the automatic landmark detection and registration of brain cortical surfaces using quasi-conformal geometry and convolutional neural networks. We first develop a landmark detection network (LD-Net) that allows for the automatic extraction of landmark curves given two prescribed starting and ending points based on the surface geometry. We then utilize the detected landmarks and quasi-conformal theory for achieving the surface registration. Specifically, we develop a coefficient prediction network (CP-Net) for predicting the Beltrami coefficients associated with the desired landmark-based registration and a mapping network called the disk Beltrami solver network (DBS-Net) for generating quasi-conformal mappings from the predicted Beltrami coefficients, with the bijectivity guaranteed by quasi-conformal theory. Experimental results are presented to demonstrate the effectiveness of our proposed framework. Altogether, our work paves a new way for surface-based morphometry and medical shape analysis.

Whole-brain radiotherapy associated with structural changes resembling aging as determined by anatomic surface-based deep learning

BACKGROUND

Brain metastases are the most common intracranial tumors in adults and are associated with significant morbidity and mortality. Whole-brain radiotherapy (WBRT) is used frequently in patients for palliation, but can result in neurocognitive deficits. While dose-dependent injury to individual areas such as the hippocampus has been demonstrated, global structural shape changes after WBRT remain to be studied.

METHODS

We studied healthy controls and patients with brain metastases and examined MRI brain anatomic surface data before and after WBRT. We implemented a validated graph convolutional neural network model to estimate patient's "brain age". We further developed a mixed-effects linear model to compare the estimated age of the whole brain and substructures before and after WBRT.

RESULTS

4220 subjects were analyzed (4148 healthy controls and 72 patients). The median radiation dose was 30 Gy (range 25-37.5 Gy). The whole brain and substructures underwent structural change resembling rapid aging in radiated patients compared to healthy controls; the whole brain "aged" 9.32 times faster, the cortex 8.05 times faster, the subcortical structures 12.57 times faster, and the hippocampus 10.14 times faster. In a subset analysis, the hippocampus "aged" 8.88 times faster in patients after conventional WBRT versus after hippocampal avoidance (HA)-WBRT.

CONCLUSIONS

Our findings suggest that WBRT causes the brain and its substructures to undergo structural changes at a pace up to 13x of the normal aging pace, where hippocampal avoidance offers focal structural protection. Correlating these structural imaging changes with neurocognitive outcomes following WBRT or HA-WBRT would benefit from future analysis.

Stroke-GFCN: ischemic stroke lesion prediction with a fully convolutional graph network

Purpose

The interpretation of image data plays a critical role during acute brain stroke diagnosis, and promptly defining the requirement of a surgical intervention will drastically impact the patient's outcome. However, determining stroke lesions purely from images can be a daunting task. Many studies proposed automatic segmentation methods for brain stroke lesions from medical images in different modalities, though heretofore results do not satisfy the requirements to be clinically reliable. We investigate the segmentation of brain stroke lesions using a geometric deep learning model that takes advantage of the intrinsic interconnected diffusion features in a set of multi-modal inputs consisting of computer tomography (CT) perfusion parameters.

Approach

We propose a geometric deep learning model for the segmentation of ischemic stroke brain lesions that employs spline convolutions and unpooling/pooling operators on graphs to excerpt graph-structured features in a fully convolutional network architecture. In addition, we seek to understand the underlying principles governing the different components of our model. Accordingly, we structure the experiments in two parts: an evaluation of different architecture hyperparameters and a comparison with state-of-the-art methods.

Results

The ablation study shows that deeper layers obtain a higher Dice coefficient score (DCS) of up to 0.3654. Comparing different pooling and unpooling methods shows that the best performing unpooling method is the proportional approach, yet it often smooths the segmentation border. Unpooling achieves segmentation results more adapted to the lesion boundary corroborated with systematic lower values of Hausdorff distance. The model performs at the level of state-of-the-art models without optimized training methods, such as augmentation or patches, with a DCS of 0.4553 ± 0.0031 .

Conclusions

We proposed and evaluated an end-to-end trainable fully convolutional graph network architecture using spline convolutional layers for the ischemic stroke lesion prediction. We propose a model that employs graph-based operations to predict acute stroke brain lesions from CT perfusion parameters. Our results prove the feasibility of using geometric deep learning to solve segmentation problems, and our model shows a better performance than other models evaluated. The proposed model achieves improved metric values for the DCS metric, ranging from 8.61% to 69.05%, compared with other models trained under the same conditions. Next, we compare different pooling and unpooling operations in relation to their segmentation results, and we show that the model can produce segmentation outputs that adapt to irregular segmentation boundaries when using simple heuristic unpooling operations.

Automatic sex estimation using deep convolutional neural network based on orthopantomogram images

Sex estimation is very important in forensic applications as part of individual identification. Morphological sex estimation methods predominantly focus on anatomical measurements. Based on the close relationship between sex chromosome genes and facial characterization, craniofacial hard tissues morphology shows sex dimorphism. In order to establish a more labor-saving, rapid, and accurate reference for sex estimation, the study investigated a deep learning network-based artificial intelligence (AI) model using orthopantomograms (OPG) to estimate sex in northern Chinese subjects. In total, 10703 OPG images were divided into training (80%), validation (10%), and test sets (10%). At the same time, different age thresholds were selected to compare the accuracy differences between adults and minors. The accuracy of sex estimation using CNN (convolutional neural network) model was higher for adults (90.97%) compared with minors (82.64%). This work demonstrated that the proposed model trained with a large dataset could be used in automatic morphological sex-related identification with favorable performance and practical significance in forensic science for adults in northern China, while also providing a reference for minors to some extent.

From sMRI to task-fMRI: A unified geometric deep learning framework for cross-modal brain anatomo-functional mapping

Achieving predictions of brain functional activation patterns/task-fMRI maps from its underlying anatomy is an important yet challenging problem. Once successful, it will not only open up new ways to understand how brain anatomy influences functional organization of the brain, but also provide new technical support for the clinical use of anatomical information to guide the localization of cortical functional areas. However, due to the non-Euclidean complex architecture of brain anatomy and the inherent low signal-to-noise ratio (SNR) properties of fMRI signals, the key challenge in building such a cross-modal brain anatomo-functional mapping is how to effectively learn the context-aware information of brain anatomy and overcome the interference of noise-containing task-fMRI labels on the learning process. In this work, we propose a Unified Geometric Deep Learning framework (BrainUGDL) to perform the cross-modal brain anatomo-functional mapping task. Considering that both global and local structures of brain anatomy have an impact on brain functions from their respective perspectives, we innovatively propose the novel Global Graph Encoding (GGE) unit and Local Graph Attention (LGA) unit embedded into two parallel branches, focusing on learning the high-level global and local context information, respectively. Specifically, GGE learns the global context information of each mesh vertex by building and encoding global interactions, and LGA learns the local context information of each mesh vertex by selectively aggregating patch structure enhanced features from its spatial neighbors. The information learnt from the two branches is then fused to form a comprehensive representation of brain anatomical features for final brain function predictions. To address the inevitable measurement noise in task-fMRI labels, we further elaborate a novel uncertainty-filtered learning mechanism, which enables BrainUGDL to realize revised learning from the noise-containing labels through the estimated uncertainty. Experiments across seven open task-fMRI datasets from human connectome project (HCP) demonstrate the superiority of BrainUGDL. To our best knowledge, our proposed BrainUGDL is the first to achieve the prediction of individual task-fMRI maps solely based on brain sMRI data.

Geometric deep learning reveals a structuro-temporal understanding of healthy and pathologic brain aging

Background

Brain age has historically been investigated primarily at the whole brain level. The ability to deconstruct the brain into its composite parts and explore brain age at the sub-structure level offers unique advantages. These include the exploration of dynamic and interconnected relationships between different brain structures in healthy and pathologic aging. To achieve this, individual brain structures can be rendered as surface representations on which morphologic analysis is carried out. Combining the advantages of deep learning with the strengths of surface analysis, we investigate the aging process at the individual structure level with the hypothesis being that pathologic aging does not uniformly affect the aging process of individual structures.

Methods

MRI data, age at scan time and diagnosis of dementia were collected from seven publicly available data repositories. The data from 17,440 unique subjects were collected, representing a total of 26,276 T1-weighted MRI accounting for longitudinal acquisitions. Surfaces were extracted for the cortex and seven subcortical structures. Deep learning networks were trained to estimate a subject's age either using several structures together or a single structure. We conducted a cross-sectional analysis to assess the difference between the predicted and actual ages for all structures between healthy subjects, individuals with mild cognitive impairment (MCI) or Alzheimer's disease dementia (ADD). We then performed a longitudinal analysis to assess the difference in the aging pace for each structure between stable healthy controls and healthy controls converting to either MCI or ADD.

Findings

Using an independent cohort of healthy subjects, age was well estimated for all structures. Cross-sectional analysis identified significantly larger predicted age for all structures in patients with either MCI and ADD compared to healthy subjects. Longitudinal analysis revealed varying degrees of involvement of individual subcortical structures for both age difference across groups and aging pace across time. These findings were most notable in the whole brain, cortex, hippocampus and amygdala.

Conclusion

Although similar patterns of abnormal aging were found related to MCI and ADD, the involvement of individual subcortical structures varied greatly and was consistently more pronounced in ADD patients compared to MCI patients.

Behavioral Studies Using Large-Scale Brain Networks – Methods and Validations

Mapping human behaviors to brain activity has become a key focus in modern cognitive neuroscience. As methods such as functional MRI (fMRI) advance cognitive scientists show an increasing interest in investigating neural activity in terms of functional connectivity and brain networks, rather than activation in a single brain region. Due to the noisy nature of neural activity, determining how behaviors are associated with specific neural signals is not well-established. Previous research has suggested graph theory techniques as a solution. Graph theory provides an opportunity to interpret human behaviors in terms of the topological organization of brain network architecture. Graph theory-based approaches, however, only scratch the surface of what neural connections relate to human behavior. Recently, the development of data-driven methods, e.g., machine learning and deep learning approaches, provide a new perspective to study the relationship between brain networks and human behaviors across the whole brain, expanding upon past literatures. In this review, we sought to revisit these data-driven approaches to facilitate our understanding of neural mechanisms and build models of human behaviors. We start with the popular graph theory approach and then discuss other data-driven approaches such as connectome-based predictive modeling, multivariate pattern analysis, network dynamic modeling, and deep learning techniques that quantify meaningful networks and connectivity related to cognition and behaviors. Importantly, for each topic, we discuss the pros and cons of the methods in addition to providing examples using our own data for each technique to describe how these methods can be applied to real-world neuroimaging data.