Hierarchical Rough-to-Fine Model for Infant Age Prediction Based on Cortical Features

PDFF-CNN: An attention-guided dynamic multi-orientation feature fusion method for gestational age prediction on imbalanced fetal brain MRI dataset

BACKGROUND

Fetal brain magnetic resonance imaging (MRI)-based gestational age prediction has been widely used to characterize normal fetal brain development and diagnose congenital brain malformations.

PURPOSE

The uncertainty of fetal position and external interference leads to variable localization and direction of the fetal brain. In addition, pregnant women typically concentrate on receiving MRI scans during the fetal anomaly scanning week, leading to an imbalanced distribution of fetal brain MRI data. The above-mentioned problems pose great challenges for deep learning-based fetal brain MRI gestational age prediction.

METHODS

In this study, a pyramid squeeze attention (PSA)-guided dynamic feature fusion CNN (PDFF-CNN) is proposed to robustly predict gestational ages from fetal brain MRI images on an imbalanced dataset. PDFF-CNN contains four components: transformation module, feature extraction module, dynamic feature fusion module, and balanced mean square error (MSE) loss. The transformation and feature extraction modules are employed by using the PSA to learn multiscale and multi-orientation feature representations in a parallel weight-sharing Siamese network. The dynamic feature fusion module automatically learns the weights of feature vectors generated in the feature extraction module to dynamically fuse multiscale and multi-orientation brain sulci and gyri features. Considering the fact of the imbalanced dataset, the balanced MSE loss is used to mitigate the negative impact of imbalanced data distribution on gestational age prediction performance.

RESULTS

Evaluated on an imbalanced fetal brain MRI dataset of 1327 routine clinical T2-weighted MRI images from 157 subjects, PDFF-CNN achieved promising gestational age prediction performance with an overall mean absolute error of 0.848 weeks and anR 2 $R^2$ of 0.904. Furthermore, the attention activation maps of PDFF-CNN were derived, which revealed regional features that contributed to gestational age prediction at each gestational stage.

CONCLUSIONS

These results suggest that the proposed PDFF-CNN might have broad clinical applicability in guiding treatment interventions and delivery planning, which has the potential to be helpful with prenatal diagnosis.

JoCoRank: Joint correlation learning with ranking similarity regularization for imbalanced fetal brain age regression

Estimating fetal brain age based on sulci by magnetic resonance imaging (MRI) is clinically crucial in determining the normal development of fetal brains. Deep learning provides a possible way for fetal brain age estimation using MRI. Previous studies have mainly emphasized optimizing individual-wise correlation criteria, such as mean square error. However, they ignored the very important global and peer-wise criterion, which are essential for learning the structured relationships among regression samples. Moreover, the imbalanced label distribution introduces an adverse bias, which impairs the reliability and interpretation of correlation estimation and the model's fairness and generalizability. In this work, we propose a novel joint correlation learning with ranking similarity regularization (JoCoRank) algorithm for deep imbalanced regression of fetal brain age. Joint correlation learning concurrently captures individual, global, and peer-level valuable relationship information, and the customized optimization scheme for each criterion exhibits strong robustness against outliers and imbalanced regression. Ranking similarity regularization is designed to calibrate the biased feature representations by aligning the sorted list of neighbors in the label space with those in the feature space. A total of 1327 MRI images from 157 healthy fetuses between 22 and 34 weeks were collected at Wuhan Children's Hospital and utilized to evaluate the performance of JoCoRank in fetal brain age estimation. JoCoRank achieved promising results with an average mean absolute error of 0.693±0.064 weeks and R2 coefficient of 0.930±0.019. Our fetal brain age estimation algorithm would be useful for identifying abnormalities in fetal brain development and undertaking early intervention in clinical practice.

Human-to-monkey transfer learning identifies the frontal white matter as a key determinant for predicting monkey brain age

The application of artificial intelligence (AI) to summarize a whole-brain magnetic resonance image (MRI) into an effective "brain age" metric can provide a holistic, individualized, and objective view of how the brain interacts with various factors (e.g., genetics and lifestyle) during aging. Brain age predictions using deep learning (DL) have been widely used to quantify the developmental status of human brains, but their wider application to serve biomedical purposes is under criticism for requiring large samples and complicated interpretability. Animal models, i.e., rhesus monkeys, have offered a unique lens to understand the human brain - being a species in which aging patterns are similar, for which environmental and lifestyle factors are more readily controlled. However, applying DL methods in animal models suffers from data insufficiency as the availability of animal brain MRIs is limited compared to many thousands of human MRIs. We showed that transfer learning can mitigate the sample size problem, where transferring the pre-trained AI models from 8,859 human brain MRIs improved monkey brain age estimation accuracy and stability. The highest accuracy and stability occurred when transferring the 3D ResNet [mean absolute error (MAE) = 1.83 years] and the 2D global-local transformer (MAE = 1.92 years) models. Our models identified the frontal white matter as the most important feature for monkey brain age predictions, which is consistent with previous histological findings. This first DL-based, anatomically interpretable, and adaptive brain age estimator could broaden the application of AI techniques to various animal or disease samples and widen opportunities for research in non-human primate brains across the lifespan.

Multiscale functional connectivity patterns of the aging brain learned from harmonized rsfMRI data of the multi-cohort iSTAGING study

To learn multiscale functional connectivity patterns of the aging brain, we built a brain age prediction model of functional connectivity measures at seven scales on a large fMRI dataset, consisting of resting-state fMRI scans of 4186 individuals with a wide age range (22 to 97 years, with an average of 63) from five cohorts. We computed multiscale functional connectivity measures of individual subjects using a personalized functional network computational method, harmonized the functional connectivity measures of subjects from multiple datasets in order to build a functional brain age model, and finally evaluated how functional brain age gap correlated with cognitive measures of individual subjects. Our study has revealed that functional connectivity measures at multiple scales were more informative than those at any single scale for the brain age prediction, the data harmonization significantly improved the brain age prediction performance, and the data harmonization in the functional connectivity measures' tangent space worked better than in their original space. Moreover, brain age gap scores of individual subjects derived from the brain age prediction model were significantly correlated with clinical and cognitive measures. Overall, these results demonstrated that multiscale functional connectivity patterns learned from a large-scale multi-site rsfMRI dataset were informative for characterizing the aging brain and the derived brain age gap was associated with cognitive and clinical measures.

Brain Connectivity Based Graph Convolutional Networks and Its Application to Infant Age Prediction

Infancy is a critical period for the human brain development, and brain age is one of the indices for the brain development status associated with neuroimaging data. The difference between the predicted age based on neuroimaging and the chronological age can provide an important early indicator of deviation from the normal developmental trajectory. In this study, we utilize the Graph Convolutional Network (GCN) to predict the infant brain age based on resting-state fMRI data. The brain connectivity obtained from rs-fMRI can be represented as a graph with brain regions as nodes and functional connections as edges. However, since the brain connectivity is a fully connected graph with features on edges, current GCN cannot be directly used for it is a node-based method for sparse graphs. Hence, we propose an edge-based Graph Path Convolution (GPC) method, which aggregates the information from different paths and can be naturally applied on dense graphs. We refer the whole model as Brain Connectivity Graph Convolutional Networks (BC-GCN). Further, two upgraded network structures are proposed by including the residual and attention modules, referred as BC-GCN-Res and BC-GCN-SE to emphasize the information of the original data and enhance influential channels. Moreover, we design a two-stage coarse-to-fine framework, which determines the age group first and then predicts the age using group-specific BC-GCN-SE models. To avoid accumulated errors from the first stage, a cross-group training strategy is adopted for the second stage regression models. We conduct experiments on infant fMRI scans from 6 to 811 days of age. The coarse-to-fine framework shows significant improvements when being applied to several models (reducing error over 10 days). Comparing with state-of-the-art methods, our proposed model BC-GCN-SE with coarse-to-fine framework reduces the mean absolute error of the prediction from >70 days to 49.9 days. The code is now available at https://github.com/SCUT-Xinlab/BC-GCN.

Multi-channel attention-fusion neural network for brain age estimation: Accuracy, generality, and interpretation with 16,705 healthy MRIs across lifespan

Brain age estimated by machine learning from T1-weighted magnetic resonance images (T1w MRIs) can reveal how brain disorders alter brain aging and can help in the early detection of such disorders. A fundamental step is to build an accurate age estimator from healthy brain MRIs. We focus on this step, and propose a framework to improve the accuracy, generality, and interpretation of age estimation in healthy brain MRIs. For accuracy, we used one of the largest sample sizes (N = 16,705). For each subject, our proposed algorithm first explicitly splits the T1w image, which has been commonly treated as a single-channel 3D image in other studies, into two 3D image channels representing contrast and morphometry information. We further proposed a "fusion-with-attention" deep learning convolutional neural network (FiA-Net) to learn how to best fuse the contrast and morphometry image channels. FiA-Net recognizes varying contributions across image channels at different brain anatomy and different feature layers. In contrast, multi-channel fusion does not exist for brain age estimation, and is mostly attention-free in other medical image analysis tasks (e.g., image synthesis, or segmentation), where treating channels equally may not be optimal. For generality, we used lifespan data 0-97 years of age for real-world utility; and we thoroughly tested FiA-Net for multi-site and multi-scanner generality by two phases of cross-validations in discovery and replication data, compared to most other studies with only one phase of cross-validation. For interpretation, we directly measured each artificial neuron's correlation with the chronological age, compared to other studies looking at the saliency of features where salient features may or may not predict age. Overall, FiA-Net achieved a mean absolute error (MAE) of 3.00 years and Pearson correlation r=0.9840 with known chronological ages in healthy brain MRIs 0-97 years of age, comparing favorably with state-of-the-art algorithms and studies for accuracy and generality across sites and datasets. We also provided interpretations on how different artificial neurons and real neuroanatomy contribute to the age estimation.

Infant and Child MRI: A Review of Scanning Procedures

Magnetic resonance imaging (MRI) is a safe method to examine human brain. However, a typical MR scan is very sensitive to motion, and it requires the subject to lie still during the acquisition, which is a major challenge for pediatric scans. Consequently, in a clinical setting, sedation or general anesthesia is often used. In the research setting including healthy subjects anesthetics are not recommended for ethical reasons and potential longer-term harm. Here we review the methods used to prepare a child for an MRI scan, but also on the techniques and tools used during the scanning to enable a successful scan. Additionally, we critically evaluate how studies have reported the scanning procedure and success of scanning. We searched articles based on special subject headings from PubMed and identified 86 studies using brain MRI in healthy subjects between 0 and 6 years of age. Scan preparations expectedly depended on subject's age; infants and young children were scanned asleep after feeding and swaddling and older children were scanned awake. Comparing the efficiency of different procedures was difficult because of the heterogeneous reporting of the used methods and the success rates. Based on this review, we recommend more detailed reporting of scanning procedure to help find out which are the factors affecting the success of scanning. In the long term, this could help the research field to get high quality data, but also the clinical field to reduce the use of anesthetics. Finally, we introduce the protocol used in scanning 2 to 5-week-old infants in the FinnBrain Birth Cohort Study, and tips for calming neonates during the scans.

Age estimates from brain magnetic resonance images of children younger than two years of age using deep learning

The accuracy of brain age estimates from magnetic resonance (MR) images has improved with the advent of deep learning artificial intelligence (AI) models. However, most previous studies on predicting age emphasized aging from childhood to adulthood and old age, and few studies have focused on early brain development in children younger than 2 years of age. Here, we performed brain age estimates based on MR images in children younger than 2 years of age using deep learning. Our AI model, developed with one slice each of raw T1- and T2-weighted images from each subject, estimated brain age with a mean absolute error of 8.2 weeks (1.9 months). The estimates of our AI model were close to those of human specialists. The AI model also estimated the brain age of subjects with a myelination delay as significantly younger than the chronological age. These results indicate that the prediction accuracy of our AI model approached that of human specialists and that our simple method requiring less data and preprocessing facilitates a radiological assessment of brain development, such as monitoring maturational changes in myelination.

Disentangled-Multimodal Adversarial Autoencoder: Application to Infant Age Prediction With Incomplete Multimodal Neuroimages

Effective fusion of structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) data has the potential to boost the accuracy of infant age prediction thanks to the complementary information provided by different imaging modalities. However, functional connectivity measured by fMRI during infancy is largely immature and noisy compared to the morphological features from sMRI, thus making the sMRI and fMRI fusion for infant brain analysis extremely challenging. With the conventional multimodal fusion strategies, adding fMRI data for age prediction has a high risk of introducing more noises than useful features, which would lead to reduced accuracy than that merely using sMRI data. To address this issue, we develop a novel model termed as disentangled-multimodal adversarial autoencoder (DMM-AAE) for infant age prediction based on multimodal brain MRI. Specifically, we disentangle the latent variables of autoencoder into common and specific codes to represent the shared and complementary information among modalities, respectively. Then, cross-reconstruction requirement and common-specific distance ratio loss are designed as regularizations to ensure the effectiveness and thoroughness of the disentanglement. By arranging relatively independent autoencoders to separate the modalities and employing disentanglement under cross-reconstruction requirement to integrate them, our DMM-AAE method effectively restrains the possible interference cross modalities, while realizing effective information fusion. Taking advantage of the latent variable disentanglement, a new strategy is further proposed and embedded into DMM-AAE to address the issue of incompleteness of the multimodal neuroimages, which can also be used as an independent algorithm for missing modality imputation. By taking six types of cortical morphometric features from sMRI and brain functional connectivity from fMRI as predictors, the superiority of the proposed DMM-AAE is validated on infant age (35 to 848 days after birth) prediction using incomplete multimodal neuroimages. The mean absolute error of the prediction based on DMM-AAE reaches 37.6 days, outperforming state-of-the-art methods. Generally, our proposed DMM-AAE can serve as a promising model for prediction with multimodal data.

Brain Age Prediction of Children Using Routine Brain MR Images via Deep Learning

Predicting brain age of children accurately and quantitatively can give help in brain development analysis and brain disease diagnosis. Traditional methods to estimate brain age based on 3D magnetic resonance (MR), T1 weighted imaging (T1WI), and diffusion tensor imaging (DTI) need complex preprocessing and extra scanning time, decreasing clinical practice, especially in children. This research aims at proposing an end-to-end AI system based on deep learning to predict the brain age based on routine brain MR imaging. We spent over 5 years enrolling 220 stacked 2D routine clinical brain MR T1-weighted images of healthy children aged 0 to 5 years old and randomly divided those images into training data including 176 subjects and test data including 44 subjects. Data augmentation technology, which includes scaling, image rotation, translation, and gamma correction, was employed to extend the training data. A 10-layer 3D convolutional neural network (CNN) was designed for predicting the brain age of children and it achieved reliable and accurate results on test data with a mean absolute deviation (MAE) of 67.6 days, a root mean squared error (RMSE) of 96.1 days, a mean relative error (MRE) of 8.2%, a correlation coefficient (R) of 0.985, and a coefficient of determination (R 2) of 0.971. Specially, the performance on predicting the age of children under 2 years old with a MAE of 28.9 days, a RMSE of 37.0 days, a MRE of 7.8%, a R of 0.983, and a R 2 of 0.967 is much better than that over 2 with a MAE of 110.0 days, a RMSE of 133.5 days, a MRE of 8.2%, a R of 0.883, and a R 2 of 0.780.