Testing a convolutional neural network-based hippocampal segmentation method in a stroke population

Distinct brain morphometry patterns revealed by deep learning improve prediction of post-stroke aphasia severity

BACKGROUND

Emerging evidence suggests that post-stroke aphasia severity depends on the integrity of the brain beyond the lesion. While measures of lesion anatomy and brain integrity combine synergistically to explain aphasic symptoms, substantial interindividual variability remains unaccounted. One explanatory factor may be the spatial distribution of morphometry beyond the lesion (e.g., atrophy), including not just specific brain areas, but distinct three-dimensional patterns.

METHODS

Here, we test whether deep learning with Convolutional Neural Networks (CNNs) on whole brain morphometry (i.e., segmented tissue volumes) and lesion anatomy better predicts chronic stroke individuals with severe aphasia (N = 231) than classical machine learning (Support Vector Machines; SVMs), evaluating whether encoding spatial dependencies identifies uniquely predictive patterns.

RESULTS

CNNs achieve higher balanced accuracy and F1 scores, even when SVMs are nonlinear or integrate linear or nonlinear dimensionality reduction. Parity only occurs when SVMs access features learned by CNNs. Saliency maps demonstrate that CNNs leverage distributed morphometry patterns, whereas SVMs focus on the area around the lesion. Ensemble clustering of CNN saliencies reveals distinct morphometry patterns unrelated to lesion size, consistent across individuals, and which implicate unique networks associated with different cognitive processes as measured by the wider neuroimaging literature. Individualized predictions depend on both ipsilateral and contralateral features outside the lesion.

CONCLUSIONS

Three-dimensional network distributions of morphometry are directly associated with aphasia severity, underscoring the potential for CNNs to improve outcome prognostication from neuroimaging data, and highlighting the prospective benefits of interrogating spatial dependence at different scales in multivariate feature space.

Fine scale hippocampus morphology variation cross 552 healthy subjects from age 20 to 80

The cerebral cortex varies over the course of a person's life span: at birth, the surface is smooth, before becoming more bumpy (deeper sulci and thicker gyri) in middle age, and thinner in senior years. In this work, a similar phenomenon was observed on the hippocampus. It was previously believed the fine-scale morphology of the hippocampus could only be extracted only with high field scanners (7T, 9.4T); however, recent studies show that regular 3T MR scanners can be sufficient for this purpose. This finding opens the door for the study of fine hippocampal morphometry for a large amount of clinical data. In particular, a characteristic bumpy and subtle feature on the inferior aspect of the hippocampus, which we refer to as hippocampal dentation, presents a dramatic degree of variability between individuals from very smooth to highly dentated. In this report, we propose a combined method joining deep learning and sub-pixel level set evolution to efficiently obtain fine-scale hippocampal segmentation on 552 healthy subjects. Through non-linear dentation extraction and fitting, we reveal that the bumpiness of the inferior surface of the human hippocampus has a clear temporal trend. It is bumpiest between 40 and 50 years old. This observation should be aligned with neurodevelopmental and aging stages.

Automated hippocampal segmentation algorithms evaluated in stroke patients

Deep learning segmentation algorithms can produce reproducible results in a matter of seconds. However, their application to more complex datasets is uncertain and may fail in the presence of severe structural abnormalities-such as those commonly seen in stroke patients. In this investigation, six recent, deep learning-based hippocampal segmentation algorithms were tested on 641 stroke patients of a multicentric, open-source dataset ATLAS 2.0. The comparisons of the volumes showed that the methods are not interchangeable with concordance correlation coefficients from 0.266 to 0.816. While the segmentation algorithms demonstrated an overall good performance (volumetric similarity [VS] 0.816 to 0.972, DICE score 0.786 to 0.921, and Hausdorff distance [HD] 2.69 to 6.34), no single out-performing algorithm was identified: FastSurfer performed best in VS, QuickNat in DICE and average HD, and Hippodeep in HD. Segmentation performance was significantly lower for ipsilesional segmentation, with a decrease in performance as a function of lesion size due to the pathology-based domain shift. Only QuickNat showed a more robust performance in volumetric similarity. Even though there are many pre-trained segmentation methods, it is important to be aware of the possible decrease in performance for the segmentation results on the lesion side due to the pathology-based domain shift. The segmentation algorithm should be selected based on the research question and the evaluation parameter needed. More research is needed to improve current hippocampal segmentation methods.

Distinct brain morphometry patterns revealed by deep learning improve prediction of aphasia severity

Emerging evidence suggests that post-stroke aphasia severity depends on the integrity of the brain beyond the stroke lesion. While measures of lesion anatomy and brain integrity combine synergistically to explain aphasic symptoms, significant interindividual variability remains unaccounted for. A possible explanatory factor may be the spatial distribution of brain atrophy beyond the lesion. This includes not just the specific brain areas showing atrophy, but also distinct three-dimensional patterns of atrophy. Here, we tested whether deep learning with Convolutional Neural Networks (CNN) on whole brain morphometry (i.e., segmented tissue volumes) and lesion anatomy can better predict which individuals with chronic stroke (N=231) have severe aphasia, and whether encoding spatial dependencies in the data might be capable of improving predictions by identifying unique individualized spatial patterns. We observed that CNN achieves significantly higher accuracy and F1 scores than Support Vector Machine (SVM), even when the SVM is nonlinear or integrates linear and nonlinear dimensionality reduction techniques. Performance parity was only achieved when the SVM was directly trained on the latent features learned by the CNN. Saliency maps demonstrated that the CNN leveraged widely distributed patterns of brain atrophy predictive of aphasia severity, whereas the SVM focused almost exclusively on the area around the lesion. Ensemble clustering of CNN saliency maps revealed distinct morphometry patterns that were unrelated to lesion size, highly consistent across individuals, and implicated unique brain networks associated with different cognitive processes as measured by the wider neuroimaging literature. Individualized predictions of severity depended on both ipsilateral and contralateral features outside of the location of stroke. Our findings illustrate that three-dimensional network distributions of atrophy in individuals with aphasia are directly associated with aphasia severity, underscoring the potential for deep learning to improve prognostication of behavioral outcomes from neuroimaging data, and highlighting the prospective benefits of interrogating spatial dependence at different scales in multivariate feature space.

Multisite test-retest reliability and compatibility of brain metrics derived from FreeSurfer versions 7.1, 6.0, and 5.3

Automatic neuroimaging processing tools provide convenient and systematic methods for extracting features from brain magnetic resonance imaging scans. One tool, FreeSurfer, provides an easy-to-use pipeline to extract cortical and subcortical morphometric measures. There have been over 25 stable releases of FreeSurfer, with different versions used across published works. The reliability and compatibility of regional morphometric metrics derived from the most recent version releases have yet to be empirically assessed. Here, we used test-retest data from three public data sets to determine within-version reliability and between-version compatibility across 42 regional outputs from FreeSurfer versions 7.1, 6.0, and 5.3. Cortical thickness from v7.1 was less compatible with that of older versions, particularly along the cingulate gyrus, where the lowest version compatibility was observed (intraclass correlation coefficient 0.37-0.61). Surface area of the temporal pole, frontal pole, and medial orbitofrontal cortex, also showed low to moderate version compatibility. We confirm low compatibility between v6.0 and v5.3 of pallidum and putamen volumes, while those from v7.1 were compatible with v6.0. Replication in an independent sample showed largely similar results for measures of surface area and subcortical volumes, but had lower overall regional thickness reliability and compatibility. Batch effect correction may adjust for some inter-version effects when most sites are run with one version, but results vary when more sites are run with different versions. Age associations in a quality controlled independent sample (N = 106) revealed version differences in results of downstream statistical analysis. We provide a reference to highlight the regional metrics that may yield recent version-related inconsistencies in published findings. An interactive viewer is provided at http://data.brainescience.org/Freesurfer_Reliability/.

Chronic Stroke Sensorimotor Impairment Is Related to Smaller Hippocampal Volumes: An ENIGMA Analysis

Background Persistent sensorimotor impairments after stroke can negatively impact quality of life. The hippocampus is vulnerable to poststroke secondary degeneration and is involved in sensorimotor behavior but has not been widely studied within the context of poststroke upper-limb sensorimotor impairment. We investigated associations between non-lesioned hippocampal volume and upper limb sensorimotor impairment in people with chronic stroke, hypothesizing that smaller ipsilesional hippocampal volumes would be associated with greater sensorimotor impairment. Methods and Results Cross-sectional T1-weighted magnetic resonance images of the brain were pooled from 357 participants with chronic stroke from 18 research cohorts of the ENIGMA (Enhancing NeuoImaging Genetics through Meta-Analysis) Stroke Recovery Working Group. Sensorimotor impairment was estimated from the FMA-UE (Fugl-Meyer Assessment of Upper Extremity). Robust mixed-effects linear models were used to test associations between poststroke sensorimotor impairment and hippocampal volumes (ipsilesional and contralesional separately; Bonferroni-corrected, P<0.025), controlling for age, sex, lesion volume, and lesioned hemisphere. In exploratory analyses, we tested for a sensorimotor impairment and sex interaction and relationships between lesion volume, sensorimotor damage, and hippocampal volume. Greater sensorimotor impairment was significantly associated with ipsilesional (P=0.005; β=0.16) but not contralesional (P=0.96; β=0.003) hippocampal volume, independent of lesion volume and other covariates (P=0.001; β=0.26). Women showed progressively worsening sensorimotor impairment with smaller ipsilesional (P=0.008; β=-0.26) and contralesional (P=0.006; β=-0.27) hippocampal volumes compared with men. Hippocampal volume was associated with lesion size (P<0.001; β=-0.21) and extent of sensorimotor damage (P=0.003; β=-0.15). Conclusions The present study identifies novel associations between chronic poststroke sensorimotor impairment and ipsilesional hippocampal volume that are not caused by lesion size and may be stronger in women.

The impact of transfer learning on 3D deep learning convolutional neural network segmentation of the hippocampus in mild cognitive impairment and Alzheimer disease subjects

Research on segmentation of the hippocampus in magnetic resonance images through deep learning convolutional neural networks (CNNs) shows promising results, suggesting that these methods can identify small structural abnormalities of the hippocampus, which are among the earliest and most frequent brain changes associated with Alzheimer disease (AD). However, CNNs typically achieve the highest accuracy on datasets acquired from the same domain as the training dataset. Transfer learning allows domain adaptation through further training on a limited dataset. In this study, we applied transfer learning on a network called spatial warping network segmentation (SWANS), developed and trained in a previous study. We used MR images of patients with clinical diagnoses of mild cognitive impairment (MCI) and AD, segmented by two different raters. By using transfer learning techniques, we developed four new models, using different training methods. Testing was performed using 26% of the original dataset, which was excluded from training as a hold‐out test set. In addition, 10% of the overall training dataset was used as a hold‐out validation set. Results showed that all the new models achieved better hippocampal segmentation quality than the baseline SWANS model (p_s < .001), with high similarity to the manual segmentations (mean dice [best model] = 0.878 ± 0.003). The best model was chosen based on visual assessment and volume percentage error (VPE). The increased precision in estimating hippocampal volumes allows the detection of small hippocampal abnormalities already present in the MCI phase (SD = [3.9 ± 0.6]%), which may be crucial for early diagnosis.

The Enhancing NeuroImaging Genetics through Meta-Analysis Consortium: 10 Years of Global Collaborations in Human Brain Mapping

This Special Issue of Human Brain Mapping is dedicated to a 10-year anniversary of the Enhancing NeuroImaging Genetics through Meta-Analysis (ENIGMA) Consortium. It reports updates from a broad range of international neuroimaging projects that pool data from around the world to answer fundamental questions in neuroscience. Since ENIGMA was formed in December 2009, the initiative grew into a worldwide effort with over 2,000 participating scientists from 45 countries, and over 50 working groups leading large-scale studies of human brain disorders. Over the last decade, many lessons were learned on how best to pool brain data from diverse sources. Working groups were created to develop methods to analyze worldwide data from anatomical and diffusion magnetic resonance imaging (MRI), resting state and task-based functional MRI, electroencephalography (EEG), magnetoencephalography (MEG), and magnetic resonance spectroscopy (MRS). The quest to understand genetic effects on human brain development and disease also led to analyses of brain scans on an unprecedented scale. Genetic roadmaps of the human cortex were created by researchers worldwide who collaborated to perform statistically well-powered analyses of common and rare genetic variants on brain measures and rates of brain development and aging. Here, we summarize the 31 papers in this Special Issue, covering: (a) technical approaches to harmonize analysis of different types of brain imaging data, (b) reviews of the last decade of work by several of ENIGMA's clinical and technical working groups, and (c) new empirical papers reporting large-scale international brain mapping analyses in patients with substance use disorders, schizophrenia, bipolar disorders, major depression, posttraumatic stress disorder, obsessive compulsive disorder, epilepsy, and stroke.

Improved normalization of lesioned brains via cohort-specific templates

In MRI studies, spatial normalization is required to infer results at the group level. In the presence of a brain lesion, such as in stroke patients, the normalization process can be affected by tissue loss, spatial deformations, signal intensity changes, and other stroke sequelae that introduce confounds into the group analysis results. Previously, most neuroimaging studies with lesioned brains have used normalization methods optimized for intact brains, raising potential concerns about the accuracy of the resulting transformations and, in turn, their reported group level results. In this study, we demonstrate the benefits of creating an intermediate, cohort‐specific template in conjunction with diffeomorphism‐based methods to normalize structural MRI images in stroke patients. We show that including this cohort‐specific template improves accuracy compared to standard methods for normalizing lesioned brains. Critically, this method reduces overall differences in normalization accuracy between stroke patients and healthy controls, and may improve the localization and connectivity of BOLD signal in functional neuroimaging data.

Neuroimaging Advances in Diagnosis and Differentiation of HIV, Comorbidities, and Aging in the cART Era

In the "cART era" of more widely available and accessible treatment, aging and HIV-related comorbidities, including symptoms of brain dysfunction, remain common among HIV-infected individuals on suppressive treatment. A better understanding of the neurobiological consequences of HIV infection is essential for developing thorough treatment guidelines and for optimizing long-term neuropsychological outcomes and overall brain health. In this chapter, we first summarize magnetic resonance imaging (MRI) methods used in over two decades of neuroHIV research. These methods evaluate brain volumetric differences and circuitry disruptions in adults living with HIV, and help map clinical correlations with brain function and tissue microstructure. We then introduce and discuss aging and associated neurological complications in people living with HIV, and processes by which infection may contribute to the risk for late-onset dementias. We describe how new technologies and large-scale international collaborations are helping to disentangle the effect of genetic and environmental risk factors on brain aging and neurodegenerative diseases. We provide insights into how these advances, which are now at the forefront of Alzheimer's disease research, may advance the field of neuroHIV. We conclude with a summary of how we see the field of neuroHIV research advancing in the decades to come and highlight potential clinical implications.

Cortical Thickness Estimation in Individuals With Cerebral Small Vessel Disease, Focal Atrophy, and Chronic Stroke Lesions

BACKGROUND

Regional changes to cortical thickness in individuals with neurodegenerative and cerebrovascular diseases (CVD) can be estimated using specialized neuroimaging software. However, the presence of cerebral small vessel disease, focal atrophy, and cortico-subcortical stroke lesions, pose significant challenges that increase the likelihood of misclassification errors and segmentation failures.

PURPOSE

The main goal of this study was to examine a correction procedure developed for enhancing FreeSurfer's (FS's) cortical thickness estimation tool, particularly when applied to the most challenging MRI obtained from participants with chronic stroke and CVD, with varying degrees of neurovascular lesions and brain atrophy.

METHODS

In 155 CVD participants enrolled in the Ontario Neurodegenerative Disease Research Initiative (ONDRI), FS outputs were compared between a fully automated, unmodified procedure and a corrected procedure that accounted for potential sources of error due to atrophy and neurovascular lesions. Quality control (QC) measures were obtained from both procedures. Association between cortical thickness and global cognitive status as assessed by the Montreal Cognitive Assessment (MoCA) score was also investigated from both procedures.

RESULTS

Corrected procedures increased "Acceptable" QC ratings from 18 to 76% for the cortical ribbon and from 38 to 92% for tissue segmentation. Corrected procedures reduced "Fail" ratings from 11 to 0% for the cortical ribbon and 62 to 8% for tissue segmentation. FS-based segmentation of T1-weighted white matter hypointensities were significantly greater in the corrected procedure (5.8 mL vs. 15.9 mL, p < 0.001). The unmodified procedure yielded no significant associations with global cognitive status, whereas the corrected procedure yielded positive associations between MoCA total score and clusters of cortical thickness in the left superior parietal (p = 0.018) and left insula (p = 0.04) regions. Further analyses with the corrected cortical thickness results and MoCA subscores showed a positive association between left superior parietal cortical thickness and Attention (p < 0.001).

CONCLUSION

These findings suggest that correction procedures which account for brain atrophy and neurovascular lesions can significantly improve FS's segmentation results and reduce failure rates, thus maximizing power by preventing the loss of our important study participants. Future work will examine relationships between cortical thickness, cerebral small vessel disease, and cognitive dysfunction due to neurodegenerative disease in the ONDRI study.