Simultaneous and consistent labeling of longitudinal dynamic developing cortical surfaces in infants

Longitudinally consistent registration and parcellation of cortical surfaces using semi-supervised learning

Temporally consistent and accurate registration and parcellation of longitudinal cortical surfaces is of great importance in studying longitudinal morphological and functional changes of human brains. However, most existing methods are developed for registration or parcellation of a single cortical surface. When applying to longitudinal studies, these methods independently register/parcellate each surface from longitudinal scans, thus often generating longitudinally inconsistent and inaccurate results, especially in small or ambiguous cortical regions. Essentially, longitudinal cortical surface registration and parcellation are highly correlated tasks with inherently shared constraints on both spatial and temporal feature representations, which are unfortunately ignored in existing methods. To this end, we unprecedentedly propose a novel semi-supervised learning framework to exploit these inherent relationships from limited labeled data and extensive unlabeled data for more robust and consistent registration and parcellation of longitudinal cortical surfaces. Our method utilizes the spherical topology characteristic of cortical surfaces. It employs a spherical network to function as an encoder, which extracts high-level cortical features. Subsequently, we build two specialized decoders dedicated to the tasks of registration and parcellation, respectively. To extract more meaningful spatial features, we design a novel parcellation map similarity loss to utilize the relationship between registration and parcellation tasks, i.e., the parcellation map warped by the deformation field in registration should match the atlas parcellation map, thereby providing extra supervision for the registration task and augmented data for parcellation task by warping the atlas parcellation map to unlabeled surfaces. To enable temporally more consistent feature representation, we additionally enforce longitudinal consistency among longitudinal surfaces after registering them together using their concatenated features. Experiments on two longitudinal datasets of infants and adults have shown that our method achieves significant improvements on both registration/parcellation accuracy and longitudinal consistency compared to existing methods, especially in small and challenging cortical regions.

Modeling refined differences of cortical folding patterns via spatial, morphological, and temporal fusion representations

The gyrus, a pivotal cortical folding pattern, is essential for integrating brain structure-function. This study focuses on 2-Hinge and 3-Hinge folds, characterized by the gyral convergence from various directions. Existing voxel-level studies may not adequately capture the precise spatial relationships within cortical folding patterns, especially when relying solely on local cortical characteristics due to their variable shapes and homogeneous frequency-specific features. To overcome these challenges, we introduced a novel model that combines spatial distribution, morphological structure, and functional magnetic resonance imaging data. We utilized spatio-morphological residual representations to enhance and extract subtle variations in cortical spatial distribution and morphological structure during blood oxygenation, integrating these with functional magnetic resonance imaging embeddings using self-attention for spatio-morphological-temporal representations. Testing these representations for identifying cortical folding patterns, including sulci, gyri, 2-Hinge, and 2-Hinge folds, and evaluating the impact of phenotypic data (e.g. stimulus) on recognition, our experimental results demonstrate the model's superior performance, revealing significant differences in cortical folding patterns under various stimulus. These differences are also evident in the characteristics of sulci and gyri folds between genders, with 3-Hinge showing more variations. Our findings indicate that our representations of cortical folding patterns could serve as biomarkers for understanding brain structure-function correlations.

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

Studies have shown that there is a tight connection between cognition skills and brain morphology during infancy. Nonetheless, it is still a great challenge to predict individual cognitive scores using their brain morphological features, considering issues like the excessive feature dimension, small sample size and missing data. Due to the limited data, a compact but expressive feature set is desirable as it can reduce the dimension and avoid the potential overfitting issue. Therefore, we pioneer the path signature method to further explore the essential hidden dynamic patterns of longitudinal cortical features. To form a hierarchical and more informative temporal representation, in this work, a novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores. By introducing the existence embedding in path generation, we can improve the robustness against the missing data. Benefiting from the global temporal receptive field of CF-PSNet, characteristics consisted in the existing data can be fully leveraged. Further, as there is no need for the whole brain to work for a certain cognitive ability, a top K selection module is used to select the most influential brain regions, decreasing the model size and the risk of overfitting. Extensive experiments are conducted on an in-house longitudinal infant dataset within 9 time points. By comparing with several recent algorithms, we illustrate the state-of-the-art performance of our CF-PSNet (i.e., root mean square error of 0.027 with the time latency of 518 milliseconds for each sample).

A 4D infant brain volumetric atlas based on the UNC/UMN baby connectome project (BCP) cohort

Spatiotemporal (four-dimensional) infant-dedicated brain atlases are essential for neuroimaging analysis of early dynamic brain development. However, due to the substantial technical challenges in the acquisition and processing of infant brain MR images, 4D atlases densely covering the dynamic brain development during infancy are still scarce. Few existing ones generally have fuzzy tissue contrast and low spatiotemporal resolution, leading to degraded accuracy of atlas-based normalization and subsequent analyses. To address this issue, in this paper, we construct a 4D structural MRI atlas for infant brains based on the UNC/UMN Baby Connectome Project (BCP) dataset, which features a high spatial resolution, extensive age-range coverage, and densely sampled time points. Specifically, 542 longitudinal T1w and T2w scans from 240 typically developing infants up to 26-month of age were utilized for our atlas construction. To improve the co-registration accuracy of the infant brain images, which typically exhibit dynamic appearance with low tissue contrast, we employed the state-of-the-art registration method and leveraged our generated reliable brain tissue probability maps in addition to the intensity images to improve the alignment of individual images. To achieve consistent region labeling on both infant and adult brain images for facilitating region-based analysis across ages, we mapped the widely used Desikan cortical parcellation onto our atlas by following an age-decreasing mapping manner. Meanwhile, the typical subcortical structures were manually delineated to facilitate the studies related to the subcortex. Compared with the existing infant brain atlases, our 4D atlas has much higher spatiotemporal resolution and preserves more structural details, and thus can boost accuracy in neurodevelopmental analysis during infancy.

A Deep Network for Joint Registration and Parcellation of Cortical Surfaces

Cortical surface registration and parcellation are two essential steps in neuroimaging analysis. Conventionally, they are performed independently as two tasks, ignoring the inherent connections of these two closely-related tasks. Essentially, both tasks rely on meaningful cortical feature representations, so they can be jointly optimized by learning shared useful cortical features. To this end, we propose a deep learning framework for joint cortical surface registration and parcellation. Specifically, our approach leverages the spherical topology of cortical surfaces and uses a spherical network as the shared encoder to first learn shared features for both tasks. Then we train two task-specific decoders for registration and parcellation, respectively. We further exploit the more explicit connection between them by incorporating the novel parcellation map similarity loss to enforce the boundary consistency of regions, thereby providing extra supervision for the registration task. Conversely, parcellation network training also benefits from the registration, which provides a large amount of augmented data by warping one surface with manual parcellation map to another surface, especially when only few manually-labeled surfaces are available. Experiments on a dataset with more than 600 cortical surfaces show that our approach achieves large improvements on both parcellation and registration accuracy (over separately trained networks) and enables training high-quality parcellation and registration models using much fewer labeled data.

Reference-Relation Guided Autoencoder with Deep CCA Restriction for Awake-to-Sleep Brain Functional Connectome Prediction

The difficulty of acquiring resting-state fMRI of early developing children under the same condition leads to a dedicated protocol, i.e., scanning younger infants during sleep and older children during being awake, respectively. However, the obviously different brain activities of sleep and awake states arouse a new challenge of awake-to-sleep connectome prediction/translation, which remains unexplored despite its importance in the longitudinally-consistent delineation of brain functional development. Due to the data scarcity and huge differences between natural images and geometric data (e.g., brain connectome), existing methods tailored for image translation generally fail in predicting functional connectome from awake to sleep. To fill this critical gap, we unprecedentedly propose a novel reference-relation guided autoencoder with deep CCA restriction (R²AE-dCCA) for awake-to-sleep connectome prediction. Specifically, 1) A reference-autoencoder (RAE) is proposed to realize a guided generation from the source domain to the target domain. The limited paired data are thus greatly augmented by including the combinations of all the age-restricted neighboring subjects as the references, while the target-specific pattern is fully learned; 2) A relation network is then designed and embedded into RAE, which utilizes the similarity in the source domain to determine the belief-strength of the reference during prediction; 3) To ensure that the learned relation in the source domain can effectively guide the generation in the target domain, a deep CCA restriction is further employed to maintain the neighboring relation during translation; 4) New validation metrics dedicated for connectome prediction are also proposed. Experimental results showed that our proposed R²AE-dCCA produces better prediction accuracy and well maintains the modular structure of brain functional connectome in comparison with state-of-the-art methods.

Infant and Child MRI: A Review of Scanning Procedures

Magnetic resonance imaging (MRI) is a safe method to examine human brain. However, a typical MR scan is very sensitive to motion, and it requires the subject to lie still during the acquisition, which is a major challenge for pediatric scans. Consequently, in a clinical setting, sedation or general anesthesia is often used. In the research setting including healthy subjects anesthetics are not recommended for ethical reasons and potential longer-term harm. Here we review the methods used to prepare a child for an MRI scan, but also on the techniques and tools used during the scanning to enable a successful scan. Additionally, we critically evaluate how studies have reported the scanning procedure and success of scanning. We searched articles based on special subject headings from PubMed and identified 86 studies using brain MRI in healthy subjects between 0 and 6 years of age. Scan preparations expectedly depended on subject's age; infants and young children were scanned asleep after feeding and swaddling and older children were scanned awake. Comparing the efficiency of different procedures was difficult because of the heterogeneous reporting of the used methods and the success rates. Based on this review, we recommend more detailed reporting of scanning procedure to help find out which are the factors affecting the success of scanning. In the long term, this could help the research field to get high quality data, but also the clinical field to reduce the use of anesthetics. Finally, we introduce the protocol used in scanning 2 to 5-week-old infants in the FinnBrain Birth Cohort Study, and tips for calming neonates during the scans.

Multi-Regression based supervised sample selection for predicting baby connectome evolution trajectory from neonatal timepoint

The connectional map of the baby brain undergoes dramatic changes over the first year of postnatal development, which makes its mapping a challenging task, let alone learning how to predict its evolution. Currently, learning models for predicting brain connectomic developmental trajectories remain broadly absent despite their great potential in spotting atypical neurodevelopmental disorders early. This is most likely due to the scarcity and often incompleteness of longitudinal infant neuroimaging studies for training such models. In this paper, we propose the first approach for progressively predicting longitudinal development of brain networks during the postnatal period solely from a baseline connectome around birth. To this end, a supervised multi-regression sample selection strategy is designed to learn how to identify the best set of neighbors of a testing baseline connectome to eventually predict its evolution trajectory at follow-up timepoints. However, given that the training dataset may have missing samples (connectomes) at certain timepoints, this may affect the training of the predictive model. To overcome this problem, we perform a low-rank tensor completion based on a robust principal component analysis to impute the missing training connectomes by linearly approximating similar complete training networks. In the prediction step, our sample selection strategy aims to preserve spatiotemporal relationships between consecutive timepoints. Therefore, the proposed method learns how to identify the set of the local closest neighbors to a target network by training an ensemble of bidirectional regressors leveraging temporal dependency between consecutive timepoints with a recall to the baseline observations to progressively predict the evolution of a testing network over time. Our method achieves the best prediction results and better captures the dynamic changes of each brain connectome over time in comparison to its ablated versions using leave-one-out cross-validation strategy.

Infant Cognitive Scores Prediction with Multi-stream Attention-Based Temporal Path Signature Features

There is stunning rapid development of human brains in the first year of life. Some studies have revealed the tight connection between cognition skills and cortical morphology in this period. Nonetheless, it is still a great challenge to predict cognitive scores using brain morphological features, given issues like small sample size and missing data in longitudinal studies. In this work, for the first time, we introduce the path signature method to explore hidden analytical and geometric properties of longitudinal cortical morphology features. A novel BrainPSNet is proposed with a differentiable temporal path signature layer to produce informative representations of different time points and various temporal granules. Further, a two-stream neural network is included to combine groups of raw features and path signature features for predicting the cognitive score. More importantly, considering different influences of each brain region on the cognitive function, we design a learning-based attention mask generator to automatically weight regions correspondingly. Experiments are conducted on an in-house longitudinal dataset. By comparing with several recent algorithms, the proposed method achieves the state-of-the-art performance. The relationship between morphological features and cognitive abilities is also analyzed.

Disentangled Intensive Triplet Autoencoder for Infant Functional Connectome Fingerprinting

Functional connectome "fingerprint" is a highly characterized brain pattern that distinguishes one individual from others. Although its existence has been demonstrated in adults, an unanswered but fundamental question is whether such individualized pattern emerges since infancy. This problem is barely investigated despites its importance in identifying the origin of the intrinsic connectome patterns that mirror distinct behavioral phenotypes. However, addressing this knowledge gap is challenging because the conventional methods are only applicable to developed brains with subtle longitudinal changes and typically fail on the dramatically developing infant brains. To tackle this challenge, we invent a novel model, namely, disentangled intensive triplet autoencoder (DI-TAE). First, we introduce the triplet autoencoder to embed the original connectivity into a latent space with higher discriminative capability among infant individuals. Then, a disentanglement strategy is proposed to separate the latent variables into identity-code, age-code, and noise-code, which not only restrains the interference from age-related developmental variance, but also captures the identity-related invariance. Next, a cross-reconstruction loss and an intensive triplet loss are designed to guarantee the effectiveness of the disentanglement and enhance the inter-subject dissimilarity for better discrimination. Finally, a variance-guided bootstrap aggregating is developed for DI-TAE to further improve the performance of identification. DI-TAE is validated on three longitudinal resting-state fMRI datasets with 394 infant scans aged 16 to 874 days. Our proposed model outperforms other state-of-the-art methods by increasing the identification rate by more than 50%, and for the first time suggests the plausible existence of brain functional connectome "fingerprint" since early infancy.

Individual identification and individual variability analysis based on cortical folding features in developing infant singletons and twins

Studying the early dynamic development of cortical folding with remarkable individual variability is critical for understanding normal early brain development and related neurodevelopmental disorders. This study focuses on the fingerprinting capability and the individual variability of cortical folding during early brain development. Specifically, we aim to explore (a) whether the developing neonatal cortical folding is unique enough to be considered as a "fingerprint" that can reliably identify an individual within a cohort of infants; (b) which cortical regions manifest more individual variability and thus contribute more for infant identification; (c) whether the infant twins can be distinguished by cortical folding. Hence, for the first time, we conduct infant individual identification and individual variability analysis involving twins based on the developing cortical folding features (mean curvature, average convexity, and sulcal depth) in 472 neonates with 1,141 longitudinal MRI scans. Experimental results show that the infant individual identification achieves 100% accuracy when using the neonatal cortical folding features to predict the identities of 1- and 2-year-olds. Besides, we observe high identification capability in the high-order association cortices (i.e., prefrontal, lateral temporal, and inferior parietal regions) and two unimodal cortices (i.e., precentral gyrus and lateral occipital cortex), which largely overlap with the regions encoding remarkable individual variability in cortical folding during the first 2 years. For twins study, we show that even for monozygotic twins with identical genes and similar developmental environments, their cortical folding features are unique enough for accurate individual identification; and in some high-order association cortices, the differences between monozygotic twin pairs are significantly lower than those between dizygotic twins. This study thus provides important insights into individual identification and individual variability based on cortical folding during infancy.

SPHERICAL U-NET FOR INFANT CORTICAL SURFACE PARCELLATION

In human brain MRI studies, it is of great importance to accurately parcellate cortical surfaces into anatomically and functionally meaningful regions. In this paper, we propose a novel end-to-end deep learning method by formulating surface parcellation as a semantic segmentation task on the sphere. To extend the convolutional neural networks (CNNs) to the spherical space, corresponding operations of surface convolution, pooling and upsampling are first developed to deal with data representation on spherical surface meshes, and then spherical CNNs are constructed accordingly. Specifically, the U-Net and SegNet architectures are transformed to the spherical representation for neonatal cortical surface parcellation. Experimental results on 90 neonates indicate the effectiveness and efficiency of our proposed spherical U-Net, in comparison with the spherical SegNet and the previous patch-wise classification method.

Desikan-Killiany-Tourville Atlas Compatible Version of M-CRIB Neonatal Parcellated Whole Brain Atlas: The M-CRIB 2.0

Our recently published M-CRIB atlas comprises 100 neonatal brain regions including 68 compatible with the widely-used Desikan-Killiany adult cortical atlas. A successor to the Desikan-Killiany atlas is the Desikan-Killiany-Tourville atlas, in which some regions with unclear boundaries were removed, and many existing boundaries were revised to conform to clearer landmarks in sulcal fundi. Our first aim here was to modify cortical M-CRIB regions to comply with the Desikan-Killiany-Tourville protocol, in order to offer: (a) compatibility with this adult cortical atlas, (b) greater labeling accuracy due to clearer landmarks, and (c) optimisation of cortical regions for integration with surface-based infant parcellation pipelines. Secondly, we aimed to update subcortical regions in order to offer greater compatibility with subcortical segmentations produced in FreeSurfer. Data utilized were the T2-weighted MRI scans in our M-CRIB atlas, for 10 healthy neonates (post-menstrual age at MRI 40–43 weeks, four female), and corresponding parcellated images. Edits were performed on the parcellated images in volume space using ITK-SNAP. Cortical updates included deletion of frontal and temporal poles and ‘Banks STS,’ and modification of boundaries of many other regions. Changes to subcortical regions included the addition of ‘ventral diencephalon,’ and deletion of ‘subcortical matter’ labels. A detailed updated parcellation protocol was produced. The resulting whole-brain M-CRIB 2.0 atlas comprises 94 regions altogether. This atlas provides comparability with adult Desikan-Killiany-Tourville-labeled cortical data and FreeSurfer-labeed subcortical data, and is more readily adaptable for incorporation into surface-based neonatal parcellation pipelines. As such, it offers the ability to help facilitate a broad range of investigations into brain structure and function both at the neonatal time point and developmentally across the lifespan.

Exploring folding patterns of infant cerebral cortex based on multi-view curvature features: Methods and applications

The highly convoluted cortical folding of the human brain is intriguingly complex and variable across individuals. Exploring the underlying representative patterns of cortical folding is of great importance for many neuroimaging studies. At term birth, all major cortical folds are established and are minimally affected by the complicated postnatal environments; hence, neonates are the ideal candidates for exploring early postnatal cortical folding patterns, which yet remain largely unexplored. In this paper, we propose a novel method for exploring the representative regional folding patterns of infant brains. Specifically, first, multi-view curvature features are constructed to comprehensively characterize the complex characteristics of cortical folding. Second, for each view of curvature features, a similarity matrix is computed to measure the similarity of cortical folding in a specific region between any pair of subjects. Next, a similarity network fusion method is adopted to nonlinearly and adaptively fuse all the similarity matrices into a single one for retaining both shared and complementary similarity information of the multiple characteristics of cortical folding. Finally, based on the fused similarity matrix and a hierarchical affinity propagation clustering approach, all subjects are automatically grouped into several clusters to obtain the representative folding patterns. To show the applications, we have applied the proposed method to a large-scale dataset with 595 normal neonates and discovered representative folding patterns in several cortical regions, i.e., the superior temporal gyrus (STG), inferior frontal gyrus (IFG), precuneus, and cingulate cortex. Meanwhile, we have revealed sex difference in STG, IFG, and cingulate cortex, as well as hemispheric asymmetries in STG and cingulate cortex in terms of cortical folding patterns. Moreover, we have also validated the proposed method on a public adult dataset, i.e., the Human Connectome Project (HCP), and revealed that certain major cortical folding patterns of adults are largely established at term birth.

Processing of structural neuroimaging data in young children: Bridging the gap between current practice and state-of-the-art methods

The structure of the brain is subject to very rapid developmental changes during early childhood. Pediatric studies based on Magnetic Resonance Imaging (MRI) over this age range have recently become more frequent, with the advantage of providing in vivo and non-invasive high-resolution images of the developing brain, toward understanding typical and atypical trajectories. However, it has also been demonstrated that application of currently standard MRI processing methods that have been developed with datasets from adults may not be appropriate for use with pediatric datasets. In this review, we examine the approaches currently used in MRI studies involving young children, including an overview of the rationale for new MRI processing methods that have been designed specifically for pediatric investigations. These methods are mainly related to the use of age-specific or 4D brain atlases, improved methods for quantifying and optimizing image quality, and provision for registration of developmental data obtained with longitudinal designs. The overall goal is to raise awareness of the existence of these methods and the possibilities for implementing them in developmental neuroimaging studies.

Computational neuroanatomy of baby brains: A review

The first postnatal years are an exceptionally dynamic and critical period of structural, functional and connectivity development of the human brain. The increasing availability of non-invasive infant brain MR images provides unprecedented opportunities for accurate and reliable charting of dynamic early brain developmental trajectories in understanding normative and aberrant growth. However, infant brain MR images typically exhibit reduced tissue contrast (especially around 6 months of age), large within-tissue intensity variations, and regionally-heterogeneous, dynamic changes, in comparison with adult brain MR images. Consequently, the existing computational tools developed typically for adult brains are not suitable for infant brain MR image processing. To address these challenges, many infant-tailored computational methods have been proposed for computational neuroanatomy of infant brains. In this review paper, we provide a comprehensive review of the state-of-the-art computational methods for infant brain MRI processing and analysis, which have advanced our understanding of early postnatal brain development. We also summarize publically available infant-dedicated resources, including MRI datasets, computational tools, grand challenges, and brain atlases. Finally, we discuss the limitations in current research and suggest potential future research directions.

The developing human connectome project: A minimal processing pipeline for neonatal cortical surface reconstruction

The Developing Human Connectome Project (dHCP) seeks to create the first 4-dimensional connectome of early life. Understanding this connectome in detail may provide insights into normal as well as abnormal patterns of brain development. Following established best practices adopted by the WU-MINN Human Connectome Project (HCP), and pioneered by FreeSurfer, the project utilises cortical surface-based processing pipelines. In this paper, we propose a fully automated processing pipeline for the structural Magnetic Resonance Imaging (MRI) of the developing neonatal brain. This proposed pipeline consists of a refined framework for cortical and sub-cortical volume segmentation, cortical surface extraction, and cortical surface inflation, which has been specifically designed to address considerable differences between adult and neonatal brains, as imaged using MRI. Using the proposed pipeline our results demonstrate that images collected from 465 subjects ranging from 28 to 45 weeks post-menstrual age (PMA) can be processed fully automatically; generating cortical surface models that are topologically correct, and correspond well with manual evaluations of tissue boundaries in 85% of cases. Results improve on state-of-the-art neonatal tissue segmentation models and significant errors were found in only 2% of cases, where these corresponded to subjects with high motion. Downstream, these surfaces will enhance comparisons of functional and diffusion MRI datasets, supporting the modelling of emerging patterns of brain connectivity.

The developing human connectome project: A minimal processing pipeline for neonatal cortical surface reconstruction

The Developing Human Connectome Project (dHCP) seeks to create the first 4-dimensional connectome of early life. Understanding this connectome in detail may provide insights into normal as well as abnormal patterns of brain development. Following established best practices adopted by the WU-MINN Human Connectome Project (HCP), and pioneered by FreeSurfer, the project utilises cortical surface-based processing pipelines. In this paper, we propose a fully automated processing pipeline for the structural Magnetic Resonance Imaging (MRI) of the developing neonatal brain. This proposed pipeline consists of a refined framework for cortical and sub-cortical volume segmentation, cortical surface extraction, and cortical surface inflation, which has been specifically designed to address considerable differences between adult and neonatal brains, as imaged using MRI. Using the proposed pipeline our results demonstrate that images collected from 465 subjects ranging from 28 to 45 weeks post-menstrual age (PMA) can be processed fully automatically; generating cortical surface models that are topologically correct, and correspond well with manual evaluations of tissue boundaries in 85% of cases. Results improve on state-of-the-art neonatal tissue segmentation models and significant errors were found in only 2% of cases, where these corresponded to subjects with high motion. Downstream, these surfaces will enhance comparisons of functional and diffusion MRI datasets, supporting the modelling of emerging patterns of brain connectivity.

Learning-based subject-specific estimation of dynamic maps of cortical morphology at missing time points in longitudinal infant studies

Longitudinal neuroimaging analysis of the dynamic brain development in infants has received increasing attention recently. Many studies expect a complete longitudinal dataset in order to accurately chart the brain developmental trajectories. However, in practice, a large portion of subjects in longitudinal studies often have missing data at certain time points, due to various reasons such as the absence of scan or poor image quality. To make better use of these incomplete longitudinal data, in this paper, we propose a novel machine learning-based method to estimate the subject-specific, vertex-wise cortical morphological attributes at the missing time points in longitudinal infant studies. Specifically, we develop a customized regression forest, named dynamically assembled regression forest (DARF), as the core regression tool. DARF ensures the spatial smoothness of the estimated maps for vertex-wise cortical morphological attributes and also greatly reduces the computational cost. By employing a pairwise estimation followed by a joint refinement, our method is able to fully exploit the available information from both subjects with complete scans and subjects with missing scans for estimation of the missing cortical attribute maps. The proposed method has been applied to estimating the dynamic cortical thickness maps at missing time points in an incomplete longitudinal infant dataset, which includes 31 healthy infant subjects, each having up to five time points in the first postnatal year. The experimental results indicate that our proposed framework can accurately estimate the subject-specific vertex-wise cortical thickness maps at missing time points, with the average error less than 0.23 mm. Hum Brain Mapp 37:4129-4147, 2016. © 2016 Wiley Periodicals, Inc.

Developmental Patterns Based Individualized Parcellation of Infant Cortical Surface

The human cerebral cortex develops dynamically during the early postnatal stage, reflecting the underlying rapid changes of cortical microstructures and their connections, which jointly determine the functional principles of cortical regions. Hence, the dynamic cortical developmental patterns are ideal for defining the distinct cortical regions in microstructure and function for neurodevelopmental studies. Moreover, given the remarkable inter-subject variability in terms of cortical structure/function and their developmental patterns, the individualized cortical parcellation based on each infant's own developmental patterns is critical for precisely localizing personalized distinct cortical regions and also understanding inter-subject variability. To this end, we propose a novel method for individualized parcellation of the infant cortical surface into distinct and meaningful regions based on each individual's cortical developmental patterns. Specifically, to alleviate the effects of cortical measurement errors and also make the individualized cortical parcellation comparable across subjects, we first create a population-based cortical parcellation to capture the general developmental landscape of the cortex in an infant population. Then, this population-based parcellation is leveraged to guide the individualized parcellation based on each infant's own cortical developmental patterns in an iterative manner. At each iteration, the individualized parcellation is gradually updated based on 1) the prior information of the population-based parcellation, 2) the individualized parcellation at the previous iteration, and also 3) the developmental patterns of all vertices. Experiments on fifteen healthy infants, each with longitudinal MRI scans acquired at six time points (i.e., 1, 3, 6, 9, 12 and 18 months of age), show that our method generates a reliable and meaningful individualized cortical parcellation based on each infant's own developmental patterns.

Estimation of Clean and Centered Brain Network Atlases using Diffusive-Shrinking Graphs with Application to Developing Brains

Many methods have been developed to spatially normalize a population of brain images for estimating a mean image as a population-average atlas. However, methods for deriving a network atlas from a set of brain networks sitting on a complex manifold are still absent. Learning how to average brain networks across subjects constitutes a key step in creating a reliable mean representation of a population of brain networks, which can be used to spot abnormal deviations from the healthy network atlas. In this work, we propose a novel network atlas estimation framework, which guarantees that the produced network atlas is clean (for tuning down noisy measurements) and well-centered (for being optimally close to all subjects and representing the individual traits of each subject in the population). Specifically, for a population of brain networks, we first build a tensor, where each of its frontal-views (i.e., frontal matrices) represents a connectivity network matrix of a single subject in the population. Then, we use tensor robust principal component analysis for jointly denoising all subjects' networks through cleaving a sparse noisy network population tensor from a clean low-rank network tensor. Second, we build a graph where each node represents a frontal-view of the unfolded clean tensor (network), to leverage the local manifold structure of these networks when fusing them. Specifically, we progressively shrink the graph of networks towards the centered mean network atlas through non-linear diffusion along the local neighbors of each of its nodes. Our evaluation on the developing functional and morphological brain networks at 1, 3, 6, 9 and 12 months of age has showed a better centeredness of our network atlases, in comparison with the baseline network fusion method. Further cleaning of the population of networks produces even more centered atlases, especially for the noisy functional connectivity networks.

Can we predict subject-specific dynamic cortical thickness maps during infancy from birth?

Understanding the early dynamic development of the human cerebral cortex remains a challenging problem. Cortical thickness, as one of the most important morphological attributes of the cerebral cortex, is a sensitive indicator for both normal neurodevelopment and neuropsychiatric disorders, but its early postnatal development remains largely unexplored. In this study, we investigate a key question in neurodevelopmental science: can we predict the future dynamic development of cortical thickness map in an individual infant based on its available MRI data at birth? If this is possible, we might be able to better model and understand the early brain development and also early detect abnormal brain development during infancy. To this end, we develop a novel learning-based method, called Dynamically-Assembled Regression Forest (DARF), to predict the development of the cortical thickness map during the first postnatal year, based on neonatal MRI features. We applied our method to 15 healthy infants and predicted their cortical thickness maps at 3, 6, 9, and 12 months of age, with respectively mean absolute errors of 0.209 mm, 0.332 mm, 0.340 mm, and 0.321 mm. Moreover, we found that the prediction precision is region-specific, with high precision in the unimodal cortex and relatively low precision in the high-order association cortex, which may be associated with their differential developmental patterns. Additional experiments also suggest that using more early time points for prediction can further significantly improve the prediction accuracy. Hum Brain Mapp 38:2865-2874, 2017. © 2017 Wiley Periodicals, Inc.

Joint prediction of longitudinal development of cortical surfaces and white matter fibers from neonatal MRI

The human brain can be modeled as multiple interrelated shapes (or a multishape), each for characterizing one aspect of the brain, such as the cortex and white matter pathways. Predicting the developing multishape is a very challenging task due to the contrasting nature of the developmental trajectories of the constituent shapes: smooth for the cortical surface and non-smooth for white matter tracts due to changes such as bifurcation. We recently addressed this problem and proposed an approach for predicting the multishape developmental spatiotemporal trajectories of infant brains based only on neonatal MRI data using a set of geometric, dynamic, and fiber-to-surface connectivity features. In this paper, we propose two key innovations to further improve the prediction of multishape evolution. First, for a more accurate cortical surface prediction, instead of simply relying on one neonatal atlas to guide the prediction of the multishape, we propose to use multiple neonatal atlases to build a spatially heterogeneous atlas using the multidirectional varifold representation. This individualizes the atlas by locally maximizing its similarity to the testing baseline cortical shape for each cortical region, thereby better representing the baseline testing cortical surface, which founds the multishape prediction process. Second, for temporally consistent fiber prediction, we propose to reliably estimate spatiotemporal connectivity features using low-rank tensor completion, thereby capturing the variability and richness of the temporal development of fibers. Experimental results confirm that the proposed variants significantly improve the prediction performance of our original multishape prediction framework for both cortical surfaces and fiber tracts shape at 3, 6, and 9 months of age. Our pioneering model will pave the way for learning how to predict the evolution of anatomical shapes with abnormal changes. Ultimately, devising accurate shape evolution prediction models that can help quantify and predict the severity of a brain disorder as it progresses will be of great aid in individualized treatment planning.

Multidirectional and Topography-based Dynamic-scale Varifold Representations with Application to Matching Developing Cortical Surfaces

The human cerebral cortex is marked by great complexity as well as substantial dynamic changes during early postnatal development. To obtain a fairly comprehensive picture of its age-induced and/or disorder-related cortical changes, one needs to match cortical surfaces to one another, while maximizing their anatomical alignment. Methods that geodesically shoot surfaces into one another as currents (a distribution of oriented normals) and varifolds (a distribution of non-oriented normals) provide an elegant Riemannian framework for generic surface matching and reliable statistical analysis. However, both conventional current and varifold matching methods have two key limitations. First, they only use the normals of the surface to measure its geometry and guide the warping process, which overlooks the importance of the orientations of the inherently convoluted cortical sulcal and gyral folds. Second, the 'conversion' of a surface into a current or a varifold operates at a fixed scale under which geometric surface details will be neglected, which ignores the dynamic scales of cortical foldings. To overcome these limitations and improve varifold-based cortical surface registration, we propose two different strategies. The first strategy decomposes each cortical surface into its normal and tangent varifold representations, by integrating principal curvature direction field into the varifold matching framework, thus providing rich information of the orientation of cortical folding and better characterization of the complex cortical geometry. The second strategy explores the informative cortical geometric features to perform a dynamic-scale measurement of the cortical surface that depends on the local surface topography (e.g., principal curvature), thereby we introduce the concept of a topography-based dynamic-scale varifold. We tested the proposed varifold variants for registering 12 pairs of dynamically developing cortical surfaces from 0 to 6 months of age. Both variants improved the matching accuracy in terms of closeness to the target surface and the goodness of alignment with regional anatomical boundaries, when compared with three state-of-the-art methods: (1) diffeomorphic spectral matching, (2) conventional current-based surface matching, and (3) conventional varifold-based surface matching.

Predicting infant cortical surface development using a 4D varifold-based learning framework and local topography-based shape morphing

Longitudinal neuroimaging analysis methods have remarkably advanced our understanding of early postnatal brain development. However, learning predictive models to trace forth the evolution trajectories of both normal and abnormal cortical shapes remains broadly absent. To fill this critical gap, we pioneered the first prediction model for longitudinal developing cortical surfaces in infants using a spatiotemporal current-based learning framework solely from the baseline cortical surface. In this paper, we detail this prediction model and even further improve its performance by introducing two key variants. First, we use the varifold metric to overcome the limitations of the current metric for surface registration that was used in our preliminary study. We also extend the conventional varifold-based surface registration model for pairwise registration to a spatiotemporal surface regression model. Second, we propose a morphing process of the baseline surface using its topographic attributes such as normal direction and principal curvature sign. Specifically, our method learns from longitudinal data both the geometric (vertices positions) and dynamic (temporal evolution trajectories) features of the infant cortical surface, comprising a training stage and a prediction stage. In the training stage, we use the proposed varifold-based shape regression model to estimate geodesic cortical shape evolution trajectories for each training subject. We then build an empirical mean spatiotemporal surface atlas. In the prediction stage, given an infant, we select the best learnt features from training subjects to simultaneously predict the cortical surface shapes at all later timepoints, based on similarity metrics between this baseline surface and the learnt baseline population average surface atlas. We used a leave-one-out cross validation method to predict the inner cortical surface shape at 3, 6, 9 and 12 months of age from the baseline cortical surface shape at birth. Our method attained a higher prediction accuracy and better captured the spatiotemporal dynamic change of the highly folded cortical surface than the previous proposed prediction method.

Construction of 4D high-definition cortical surface atlases of infants: Methods and applications

In neuroimaging, cortical surface atlases play a fundamental role for spatial normalization, analysis, visualization, and comparison of results across individuals and different studies. However, existing cortical surface atlases created for adults are not suitable for infant brains during the first two postnatal years, which is the most dynamic period of postnatal structural and functional development of the highly-folded cerebral cortex. Therefore, spatiotemporal cortical surface atlases for infant brains are highly desired yet still lacking for accurate mapping of early dynamic brain development. To bridge this significant gap, leveraging our infant-dedicated computational pipeline for cortical surface-based analysis and the unique longitudinal infant MRI dataset acquired in our research center, in this paper, we construct the first spatiotemporal (4D) high-definition cortical surface atlases for the dynamic developing infant cortical structures at seven time points, including 1, 3, 6, 9, 12, 18, and 24 months of age, based on 202 serial MRI scans from 35 healthy infants. For this purpose, we develop a novel method to ensure the longitudinal consistency and unbiasedness to any specific subject and age in our 4D infant cortical surface atlases. Specifically, we first compute the within-subject mean cortical folding by unbiased groupwise registration of longitudinal cortical surfaces of each infant. Then we establish longitudinally-consistent and unbiased inter-subject cortical correspondences by groupwise registration of the geometric features of within-subject mean cortical folding across all infants. Our 4D surface atlases capture both longitudinally-consistent dynamic mean shape changes and the individual variability of cortical folding during early brain development. Experimental results on two independent infant MRI datasets show that using our 4D infant cortical surface atlases as templates leads to significantly improved accuracy for spatial normalization of cortical surfaces across infant individuals, in comparison to the infant surface atlases constructed without longitudinal consistency and also the FreeSurfer adult surface atlas. Moreover, based on our 4D infant surface atlases, for the first time, we reveal the spatially-detailed, region-specific correlation patterns of the dynamic cortical developmental trajectories between different cortical regions during early brain development.

Prediction of Longitudinal Development of Infant Cortical Surface Shape Using a 4D Current-Based Learning Framework

Understanding the early dynamics of the highly folded human cerebral cortex is still an actively evolving research field teeming with unanswered questions. Longitudinal neuroimaging analysis and modeling have become the new trend to advance research in this field. However, this is challenged by a limited number of acquisition timepoints and the absence of inter-subject matching between timepoints. In this paper, we propose a novel framework that unprecedentedly solves the problem of predicting the dynamic evolution of infant cortical surface shape solely from a single baseline shape based on a spatiotemporal (4D) current-based learning approach. Specifically, our method learns from longitudinal data both the geometric (vertices positions) and dynamic (temporal evolution trajectories) features of the infant cortical surface, comprising a training stage and a prediction stage. In the training stage, we first use the current-based shape regression model to set up the inter-subject cortical surface correspondences at baseline of all training subjects. We then estimate for each training subject the diffeomorphic temporal evolution trajectories of the cortical surface shape and build an empirical mean spatiotemporal surface atlas. In the prediction stage, given an infant, we first warp all training subjects onto its baseline cortical surface. Second, we select the most appropriate learnt features from training subjects to simultaneously predict the cortical surface shapes at all later timepoints from its baseline cortical surface, based on closeness metrics between this baseline surface and the learnt baseline population average surface atlas. We used the proposed framework to predict the inner cortical surface shape at 3, 6 and 9 months from the cortical shape at birth in 9 healthy infants. Our method predicted with good accuracy the spatiotemporal dynamic change of the highly folded cortex.

Spatial Patterns, Longitudinal Development, and Hemispheric Asymmetries of Cortical Thickness in Infants from Birth to 2 Years of Age

Cortical thickness (CT) is related to normal development and neurodevelopmental disorders. It remains largely unclear how the characteristic patterns of CT evolve in the first 2 years. In this paper, we systematically characterized for the first time the detailed vertex-wise patterns of spatial distribution, longitudinal development, and hemispheric asymmetries of CT at 0, 1, and 2 years of age, via surface-based analysis of 219 longitudinal magnetic resonance images from 73 infants. Despite the dynamic increase of CT in the first year and the little change of CT in the second year, we found that the overall spatial distribution of thin and thick cortices was largely present at birth, and evolved only modestly during the first 2 years. Specifically, the precentral gyrus, postcentral gyrus, occipital cortex, and superior parietal region had thin cortices, whereas the prefrontal, lateral temporal, insula, and inferior parietal regions had thick cortices. We revealed that in the first year thin cortices exhibited low growth rates of CT, whereas thick cortices exhibited high growth rates. We also found that gyri were thicker than sulci, and that the anterior bank of the central sulcus was thicker than the posterior bank. Moreover, we showed rightward hemispheric asymmetries of CT in the lateral temporal and posterior insula regions at birth, which shrank gradually in the first 2 years, and also leftward asymmetries in the medial prefrontal, paracentral, and anterior cingulate cortices, which expanded substantially during this period. This study provides the first comprehensive picture of early patterns and evolution of CT during infancy.

Multi-atlas segmentation of biomedical images: A survey

Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing, et al. (2004), Klein, et al. (2005), and Heckemann, et al. (2006), is becoming one of the most widely-used and successful image segmentation techniques in biomedical applications. By manipulating and utilizing the entire dataset of "atlases" (training images that have been previously labeled, e.g., manually by an expert), rather than some model-based average representation, MAS has the flexibility to better capture anatomical variation, thus offering superior segmentation accuracy. This benefit, however, typically comes at a high computational cost. Recent advancements in computer hardware and image processing software have been instrumental in addressing this challenge and facilitated the wide adoption of MAS. Today, MAS has come a long way and the approach includes a wide array of sophisticated algorithms that employ ideas from machine learning, probabilistic modeling, optimization, and computer vision, among other fields. This paper presents a survey of published MAS algorithms and studies that have applied these methods to various biomedical problems. In writing this survey, we have three distinct aims. Our primary goal is to document how MAS was originally conceived, later evolved, and now relates to alternative methods. Second, this paper is intended to be a detailed reference of past research activity in MAS, which now spans over a decade (2003-2014) and entails novel methodological developments and application-specific solutions. Finally, our goal is to also present a perspective on the future of MAS, which, we believe, will be one of the dominant approaches in biomedical image segmentation.

AUTOMATIC PARCELLATION OF CORTICAL SURFACES USING RANDOM FORESTS

Automatic and accurate parcellation of cortical surfaces into anatomically and functionally meaningful regions is of fundamental importance in brain mapping. In this paper, we propose a new method leveraging random forests and graph cuts methods to parcellate cortical surfaces into a set of gyral-based regions, using multiple surface atlases with manual labels by experts. Specifically, our method first takes advantage of random forests and auto-context methods to learn the optimal utilization of cortical features for rough parcellation and then the graph cuts method to further refine the parcellation for improved accuracy and spatial consistency. Particularly, to capitalize on random forests, we propose a novel definition of Haar-like features on cortical surfaces based on spherical mapping. The proposed method has been validated on cortical surfaces from 39 adult brain MR images, each with 35 regions manually labeled by a neuroanatomist, achieving the average Dice ratio of 0.902, higher than the-state-of-art methods.