Multi-Head Graph Convolutional Network for Structural Connectome Classification

We tackle classification based on brain connectivity derived from diffusion magnetic resonance images. We propose a machine-learning model inspired by graph convolutional networks (GCNs), which takes a brain-connectivity input graph and processes the data separately through a parallel GCN mechanism with multiple heads. The proposed network is a simple design that employs different heads involving graph convolutions focused on edges and nodes, thoroughly capturing representations from the input data. To test the ability of our model to extract complementary and representative features from brain connectivity data, we chose the task of sex classification. This quantifies the degree to which the connectome varies depending on the sex, which is important for improving our understanding of health and disease in both sexes. We show experiments on two publicly available datasets: PREVENT-AD (347 subjects) and OASIS3 (771 subjects). The proposed model demonstrates the highest performance compared to the existing machine-learning algorithms we tested, including classical methods and (graph and non-graph) deep learning. We provide a detailed analysis of each component of our model.

BOOSTING SKULL-STRIPPING PERFORMANCE FOR PEDIATRIC BRAIN IMAGES

Skull-stripping is the removal of background and non-brain anatomical features from brain images. While many skull-stripping tools exist, few target pediatric populations. With the emergence of multi-institutional pediatric data acquisition efforts to broaden the understanding of perinatal brain development, it is essential to develop robust and well-tested tools ready for the relevant data processing. However, the broad range of neuroanatomical variation in the developing brain, combined with additional challenges such as high motion levels, as well as shoulder and chest signal in the images, leaves many adult-specific tools ill-suited for pediatric skull-stripping. Building on an existing framework for robust and accurate skull-stripping, we propose developmental SynthStrip (d-SynthStrip), a skull-stripping model tailored to pediatric images. This framework exposes networks to highly variable images synthesized from label maps. Our model substantially outperforms pediatric baselines across scan types and age cohorts. In addition, the <1-minute runtime of our tool compares favorably to the fastest baselines. We distribute our model at https://w3id.org/synthstrip.

Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology

We present open-source tools for 3D analysis of photographs of dissected slices of human brains, which are routinely acquired in brain banks but seldom used for quantitative analysis. Our tools can: (i) 3D reconstruct a volume from the photographs and, optionally, a surface scan; and (ii) produce a high-resolution 3D segmentation into 11 brain regions per hemisphere (22 in total), independently of the slice thickness. Our tools can be used as a substitute for ex vivo magnetic resonance imaging (MRI), which requires access to an MRI scanner, ex vivo scanning expertise, and considerable financial resources. We tested our tools on synthetic and real data from two NIH Alzheimer's Disease Research Centers. The results show that our methodology yields accurate 3D reconstructions, segmentations, and volumetric measurements that are highly correlated to those from MRI. Our method also detects expected differences between post mortem confirmed Alzheimer's disease cases and controls. The tools are available in our widespread neuroimaging suite "FreeSurfer" ( https://surfer.nmr.mgh.harvard.edu/fswiki/PhotoTools ).

Scaling behaviours of deep learning and linear algorithms for the prediction of stroke severity

Deep learning has allowed for remarkable progress in many medical scenarios. Deep learning prediction models often require 105-107 examples. It is currently unknown whether deep learning can also enhance predictions of symptoms post-stroke in real-world samples of stroke patients that are often several magnitudes smaller. Such stroke outcome predictions however could be particularly instrumental in guiding acute clinical and rehabilitation care decisions. We here compared the capacities of classically used linear and novel deep learning algorithms in their prediction of stroke severity. Our analyses relied on a total of 1430 patients assembled from the MRI-Genetics Interface Exploration collaboration and a Massachusetts General Hospital-based study. The outcome of interest was National Institutes of Health Stroke Scale-based stroke severity in the acute phase after ischaemic stroke onset, which we predict by means of MRI-derived lesion location. We automatically derived lesion segmentations from diffusion-weighted clinical MRI scans, performed spatial normalization and included a principal component analysis step, retaining 95% of the variance of the original data. We then repeatedly separated a train, validation and test set to investigate the effects of sample size; we subsampled the train set to 100, 300 and 900 and trained the algorithms to predict the stroke severity score for each sample size with regularized linear regression and an eight-layered neural network. We selected hyperparameters on the validation set. We evaluated model performance based on the explained variance (R2) in the test set. While linear regression performed significantly better for a sample size of 100 patients, deep learning started to significantly outperform linear regression when trained on 900 patients. Average prediction performance improved by ∼20% when increasing the sample size 9× [maximum for 100 patients: 0.279 ± 0.005 (R2, 95% confidence interval), 900 patients: 0.337 ± 0.006]. In summary, for sample sizes of 900 patients, deep learning showed a higher prediction performance than typically employed linear methods. These findings suggest the existence of non-linear relationships between lesion location and stroke severity that can be utilized for an improved prediction performance for larger sample sizes.

A robust and interpretable deep learning framework for multi-modal registration via keypoints

We present KeyMorph, a deep learning-based image registration framework that relies on automatically detecting corresponding keypoints. State-of-the-art deep learning methods for registration often are not robust to large misalignments, are not interpretable, and do not incorporate the symmetries of the problem. In addition, most models produce only a single prediction at test-time. Our core insight which addresses these shortcomings is that corresponding keypoints between images can be used to obtain the optimal transformation via a differentiable closed-form expression. We use this observation to drive the end-to-end learning of keypoints tailored for the registration task, and without knowledge of ground-truth keypoints. This framework not only leads to substantially more robust registration but also yields better interpretability, since the keypoints reveal which parts of the image are driving the final alignment. Moreover, KeyMorph can be designed to be equivariant under image translations and/or symmetric with respect to the input image ordering. Finally, we show how multiple deformation fields can be computed efficiently and in closed-form at test time corresponding to different transformation variants. We demonstrate the proposed framework in solving 3D affine and spline-based registration of multi-modal brain MRI scans. In particular, we show registration accuracy that surpasses current state-of-the-art methods, especially in the context of large displacements. Our code is available at https://github.com/alanqrwang/keymorph.

Joint cortical registration of geometry and function using semi-supervised learning

Brain surface-based image registration, an important component of brain image analysis, establishes spatial correspondence between cortical surfaces. Existing iterative and learning-based approaches focus on accurate registration of folding patterns of the cerebral cortex, and assume that geometry predicts function and thus functional areas will also be well aligned. However, structure/functional variability of anatomically corresponding areas across subjects has been widely reported. In this work, we introduce a learning-based cortical registration framework, JOSA, which jointly aligns folding patterns and functional maps while simultaneously learning an optimal atlas. We demonstrate that JOSA can substantially improve registration performance in both anatomical and functional domains over existing methods. By employing a semi-supervised training strategy, the proposed framework obviates the need for functional data during inference, enabling its use in broad neuroscientific domains where functional data may not be observed. The source code of JOSA will be released to the public at https://voxelmorph.net.

A cellular resolution atlas of Broca's area

Brain cells are arranged in laminar, nuclear, or columnar structures, spanning a range of scales. Here, we construct a reliable cell census in the frontal lobe of human cerebral cortex at micrometer resolution in a magnetic resonance imaging (MRI)-referenced system using innovative imaging and analysis methodologies. MRI establishes a macroscopic reference coordinate system of laminar and cytoarchitectural boundaries. Cell counting is obtained with a digital stereological approach on the 3D reconstruction at cellular resolution from a custom-made inverted confocal light-sheet fluorescence microscope (LSFM). Mesoscale optical coherence tomography enables the registration of the distorted histological cell typing obtained with LSFM to the MRI-based atlas coordinate system. The outcome is an integrated high-resolution cellular census of Broca's area in a human postmortem specimen, within a whole-brain reference space atlas.

Multi-Head Graph Convolutional Network for Structural Connectome Classification

We tackle classification based on brain connectivity derived from diffusion magnetic resonance images. We propose a machine-learning model inspired by graph convolutional networks (GCNs), which takes a brain connectivity input graph and processes the data separately through a parallel GCN mechanism with multiple heads. The proposed network is a simple design that employs different heads involving graph convolutions focused on edges and nodes, capturing representations from the input data thoroughly. To test the ability of our model to extract complementary and representative features from brain connectivity data, we chose the task of sex classification. This quantifies the degree to which the connectome varies depending on the sex, which is important for improving our understanding of health and disease in both sexes. We show experiments on two publicly available datasets: PREVENT-AD (347 subjects) and OASIS3 (771 subjects). The proposed model demonstrates the highest performance compared to the existing machine-learning algorithms we tested, including classical methods and (graph and non-graph) deep learning. We provide a detailed analysis of each component of our model.

Supervision by Denoising

Learning-based image reconstruction models, such as those based on the U-Net, require a large set of labeled images if good generalization is to be guaranteed. In some imaging domains, however, labeled data with pixel- or voxel-level label accuracy are scarce due to the cost of acquiring them. This problem is exacerbated further in domains like medical imaging, where there is no single ground truth label, resulting in large amounts of repeat variability in the labels. Therefore, training reconstruction networks to generalize better by learning from both labeled and unlabeled examples (called semi-supervised learning) is problem of practical and theoretical interest. However, traditional semi-supervised learning methods for image reconstruction often necessitate handcrafting a differentiable regularizer specific to some given imaging problem, which can be extremely time-consuming. In this work, we propose "supervision by denoising" (SUD), a framework to supervise reconstruction models using their own denoised output as labels. SUD unifies stochastic averaging and spatial denoising techniques under a spatio-temporal denoising framework and alternates denoising and model weight update steps in an optimization framework for semi-supervision. As example applications, we apply SUD to two problems from biomedical imaging-anatomical brain reconstruction (3D) and cortical parcellation (2D)-to demonstrate a significant improvement in reconstruction over supervised-only and ensembling baselines. Our code available at https://github.com/seannz/sud.

Can we predict motion artifacts in clinical MRI before the scan completes?

Subject motion can cause artifacts in clinical MRI, frequently necessitating repeat scans. We propose to alleviate this inefficiency by predicting artifact scores from partial multi-shot multi-slice acquisitions, which may guide the operator in aborting corrupted scans early.

Hyper-convolutions via implicit kernels for medical image analysis

The convolutional neural network (CNN) is one of the most commonly used architectures for computer vision tasks. The key building block of a CNN is the convolutional kernel that aggregates information from the pixel neighborhood and shares weights across all pixels. A standard CNN's capacity, and thus its performance, is directly related to the number of learnable kernel weights, which is determined by the number of channels and the kernel size (support). In this paper, we present the hyper-convolution, a novel building block that implicitly encodes the convolutional kernel using spatial coordinates. Unlike a regular convolutional kernel, whose weights are independently learned, hyper-convolution kernel weights are correlated through an encoder that maps spatial coordinates to their corresponding values. Hyper-convolutions decouple kernel size from the total number of learnable parameters, enabling a more flexible architecture design. We demonstrate in our experiments that replacing regular convolutions with hyper-convolutions can improve performance with less parameters, and increase robustness against noise. We provide our code here: https://github.com/tym002/Hyper-Convolution.

Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

The relevance of rich club regions for functional outcome post-stroke is enhanced in women

This study aimed to investigate the influence of stroke lesions in predefined highly interconnected (rich-club) brain regions on functional outcome post-stroke, determine their spatial specificity and explore the effects of biological sex on their relevance. We analyzed MRI data recorded at index stroke and ~3-months modified Rankin Scale (mRS) data from patients with acute ischemic stroke enrolled in the multisite MRI-GENIE study. Spatially normalized structural stroke lesions were parcellated into 108 atlas-defined bilateral (sub)cortical brain regions. Unfavorable outcome (mRS > 2) was modeled in a Bayesian logistic regression framework. Effects of individual brain regions were captured as two compound effects for (i) six bilateral rich club and (ii) all further non-rich club regions. In spatial specificity analyses, we randomized the split into "rich club" and "non-rich club" regions and compared the effect of the actual rich club regions to the distribution of effects from 1000 combinations of six random regions. In sex-specific analyses, we introduced an additional hierarchical level in our model structure to compare male and female-specific rich club effects. A total of 822 patients (age: 64.7[15.0], 39% women) were analyzed. Rich club regions had substantial relevance in explaining unfavorable functional outcome (mean of posterior distribution: 0.08, area under the curve: 0.8). In particular, the rich club-combination had a higher relevance than 98.4% of random constellations. Rich club regions were substantially more important in explaining long-term outcome in women than in men. All in all, lesions in rich club regions were associated with increased odds of unfavorable outcome. These effects were spatially specific and more pronounced in women.

SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the contrast and resolution of the synthetic training data. Consequently, SynthSeg can segment real scans from a wide range of target domains without retraining or fine-tuning, which enables straightforward analysis of huge amounts of heterogeneous clinical data. Because SynthSeg only requires segmentations to be trained (no images), it can learn from labels obtained by automated methods on diverse populations (e.g., ageing and diseased), thus achieving robustness to a wide range of morphological variability. We demonstrate SynthSeg on 5,000 scans of six modalities (including CT) and ten resolutions, where it exhibits unparallelled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation. Finally, we demonstrate the generalisability of SynthSeg by applying it to cardiac MRI and CT scans.

SynthStrip: skull-stripping for any brain image

The removal of non-brain signal from magnetic resonance imaging (MRI) data, known as skull-stripping, is an integral component of many neuroimage analysis streams. Despite their abundance, popular classical skull-stripping methods are usually tailored to images with specific acquisition properties, namely near-isotropic resolution and T1-weighted (T1w) MRI contrast, which are prevalent in research settings. As a result, existing tools tend to adapt poorly to other image types, such as stacks of thick slices acquired with fast spin-echo (FSE) MRI that are common in the clinic. While learning-based approaches for brain extraction have gained traction in recent years, these methods face a similar burden, as they are only effective for image types seen during the training procedure. To achieve robust skull-stripping across a landscape of imaging protocols, we introduce SynthStrip, a rapid, learning-based brain-extraction tool. By leveraging anatomical segmentations to generate an entirely synthetic training dataset with anatomies, intensity distributions, and artifacts that far exceed the realistic range of medical images, SynthStrip learns to successfully generalize to a variety of real acquired brain images, removing the need for training data with target contrasts. We demonstrate the efficacy of SynthStrip for a diverse set of image acquisitions and resolutions across subject populations, ranging from newborn to adult. We show substantial improvements in accuracy over popular skull-stripping baselines - all with a single trained model. Our method and labeled evaluation data are available at https://w3id.org/synthstrip.

Deep profiling of multiple ischemic lesions in a large, multi-center cohort: Frequency, spatial distribution, and associations to clinical characteristics

Background purpose

A substantial number of patients with acute ischemic stroke (AIS) experience multiple acute lesions (MAL). We here aimed to scrutinize MAL in a large radiologically deep-phenotyped cohort.

Materials and methods

Analyses relied upon imaging and clinical data from the international MRI-GENIE study. Imaging data comprised both Fluid-attenuated inversion recovery (FLAIR) for white matter hyperintensity (WMH) burden estimation and diffusion-weighted imaging (DWI) sequences for the assessment of acute stroke lesions. The initial step featured the systematic evaluation of occurrences of MAL within one and several vascular supply territories. Associations between MAL and important imaging and clinical characteristics were subsequently determined. The interaction effect between single and multiple lesion status and lesion volume was estimated by means of Bayesian hierarchical regression modeling for both stroke severity and functional outcome.

Results

We analyzed 2,466 patients (age = 63.4 ± 14.8, 39% women), 49.7% of which presented with a single lesion. Another 37.4% experienced MAL in a single vascular territory, while 12.9% featured lesions in multiple vascular territories. Within most territories, MAL occurred as frequently as single lesions (ratio ∼1:1). Only the brainstem region comprised fewer patients with MAL (ratio 1:4). Patients with MAL presented with a significantly higher lesion volume and acute NIHSS (7.7 vs. 1.7 ml and 4 vs. 3, p FDR < 0.001). In contrast, patients with a single lesion were characterized by a significantly higher WMH burden (6.1 vs. 5.3 ml, p FDR = 0.048). Functional outcome did not differ significantly between patients with single versus multiple lesions. Bayesian analyses suggested that the association between lesion volume and stroke severity between single and multiple lesions was the same in case of anterior circulation stroke. In case of posterior circulation stroke, lesion volume was linked to a higher NIHSS only among those with MAL.

Conclusion

Multiple lesions, especially those within one vascular territory, occurred more frequently than previously reported. Overall, multiple lesions were distinctly linked to a higher acute stroke severity, a higher total DWI lesion volume and a lower WMH lesion volume. In posterior circulation stroke, lesion volume was linked to a higher stroke severity in multiple lesions only.

Learned iterative segmentation of highly variable anatomy from limited data: Applications to whole heart segmentation for congenital heart disease

Training deep learning models that segment an image in one step typically requires a large collection of manually annotated images that captures the anatomical variability in a cohort. This poses challenges when anatomical variability is extreme but training data is limited, as when segmenting cardiac structures in patients with congenital heart disease (CHD). In this paper, we propose an iterative segmentation model and show that it can be accurately learned from a small dataset. Implemented as a recurrent neural network, the model evolves a segmentation over multiple steps, from a single user click until reaching an automatically determined stopping point. We develop a novel loss function that evaluates the entire sequence of output segmentations, and use it to learn model parameters. Segmentations evolve predictably according to growth dynamics encapsulated by training data, which consists of images, partially completed segmentations, and the recommended next step. The user can easily refine the final segmentation by examining those that are earlier or later in the output sequence. Using a dataset of 3D cardiac MR scans from patients with a wide range of CHD types, we show that our iterative model offers better generalization to patients with the most severe heart malformations.

Migraine-Associated Common Genetic Variants Confer Greater Risk of Posterior vs. Anterior Circulation Ischemic Stroke☆

OBJECTIVE

To examine potential genetic relationships between migraine and the two distinct phenotypes posterior circulation ischemic stroke (PCiS) and anterior circulation ischemic stroke (ACiS), we generated migraine polygenic risk scores (PRSs) and compared these between PCiS and ACiS, and separately vs. non-stroke control subjects.

METHODS

Acute ischemic stroke cases were classified as PCiS or ACiS based on lesion location on diffusion-weighted MRI. Exclusion criteria were lesions in both vascular territories or uncertain territory; supratentorial PCiS with ipsilateral fetal posterior cerebral artery; and cases with atrial fibrillation. We generated migraine PRS for three migraine phenotypes (any migraine; migraine without aura; migraine with aura) using publicly available GWAS data and compared mean PRSs separately for PCiS and ACiS vs. non-stroke control subjects, and between each stroke phenotype.

RESULTS

Our primary analyses included 464 PCiS and 1079 ACiS patients with genetic European ancestry. Compared to non-stroke control subjects (n=15396), PRSs of any migraine were associated with increased risk of PCiS (p=0.01-0.03) and decreased risk of ACiS (p=0.010-0.039). Migraine without aura PRSs were significantly associated with PCiS (p=0.008-0.028), but not with ACiS. When comparing PCiS vs. ACiS directly, migraine PRSs were higher in PCiS vs. ACiS for any migraine (p=0.001-0.010) and migraine without aura (p=0.032-0.048). Migraine with aura PRS did not show a differential association in our analyses.

CONCLUSIONS

Our results suggest a stronger genetic overlap between unspecified migraine and migraine without aura with PCiS compared to ACiS. Possible shared mechanisms include dysregulation of cerebral vessel endothelial function.

SuperWarp: Supervised Learning and Warping on U-Net for Invariant Subvoxel-Precise Registration

In recent years, learning-based image registration methods have gradually moved away from direct supervision with target warps to instead use self-supervision, with excellent results in several registration benchmarks. These approaches utilize a loss function that penalizes the intensity differences between the fixed and moving images, along with a suitable regularizer on the deformation. However, since images typically have large untextured regions, merely maximizing similarity between the two images is not sufficient to recover the true deformation. This problem is exacerbated by texture in other regions, which introduces severe non-convexity into the landscape of the training objective and ultimately leads to overfitting. In this paper, we argue that the relative failure of supervised registration approaches can in part be blamed on the use of regular U-Nets, which are jointly tasked with feature extraction, feature matching and deformation estimation. Here, we introduce a simple but crucial modification to the U-Net that disentangles feature extraction and matching from deformation prediction, allowing the U-Net to warp the features, across levels, as the deformation field is evolved. With this modification, direct supervision using target warps begins to outperform self-supervision approaches that require segmentations, presenting new directions for registration when images do not have segmentations. We hope that our findings in this preliminary workshop paper will re-ignite research interest in supervised image registration techniques. Our code is publicly available from http://github.com/balbasty/superwarp.

TopoFit: Rapid Reconstruction of Topologically-Correct Cortical Surfaces

Mesh-based reconstruction of the cerebral cortex is a fundamental component in brain image analysis. Classical, iterative pipelines for cortical modeling are robust but often time-consuming, mostly due to expensive procedures that involve topology correction and spherical mapping. Recent attempts to address reconstruction with machine learning methods have accelerated some components in these pipelines, but these methods still require slow processing steps to enforce topological constraints that comply with known anatomical structure. In this work, we introduce a novel learning-based strategy, TopoFit, which rapidly fits a topologically-correct surface to the white-matter tissue boundary. We design a joint network, employing image and graph convolutions and an efficient symmetric distance loss, to learn to predict accurate deformations that map a template mesh to subject-specific anatomy. This technique encompasses the work of current mesh correction, fine-tuning, and inflation processes and, as a result, offers a 150× faster solution to cortical surface reconstruction compared to traditional approaches. We demonstrate that TopoFit is 1.8× more accurate than the current state-of-the-art deep-learning strategy, and it is robust to common failure modes, such as white-matter tissue hypointensities.

Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis

We propose neural network layers that explicitly combine frequency and image feature representations and show that they can be used as a versatile building block for reconstruction from frequency space data. Our work is motivated by the challenges arising in MRI acquisition where the signal is a corrupted Fourier transform of the desired image. The proposed joint learning schemes enable both correction of artifacts native to the frequency space and manipulation of image space representations to reconstruct coherent image structures at every layer of the network. This is in contrast to most current deep learning approaches for image reconstruction that treat frequency and image space features separately and often operate exclusively in one of the two spaces. We demonstrate the advantages of joint convolutional learning for a variety of tasks, including motion correction, denoising, reconstruction from undersampled acquisitions, and combined undersampling and motion correction on simulated and real world multicoil MRI data. The joint models produce consistently high quality output images across all tasks and datasets. When integrated into a state of the art unrolled optimization network with physics-inspired data consistency constraints for undersampled reconstruction, the proposed architectures significantly improve the optimization landscape, which yields an order of magnitude reduction of training time. This result suggests that joint representations are particularly well suited for MRI signals in deep learning networks. Our code and pretrained models are publicly available at https://github.com/nalinimsingh/interlacer.

SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to contrast introduced by magnetic resonance imaging (MRI). While classical registration methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning-based techniques are fast at test time but limited to registering images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency on training data by leveraging a generative strategy for diverse synthetic label maps and images that exposes networks to a wide range of variability, forcing them to learn more invariant features. This approach results in powerful networks that accurately generalize to a broad array of MRI contrasts. We present extensive experiments with a focus on 3D neuroimaging, showing that this strategy enables robust and accurate registration of arbitrary MRI contrasts even if the target contrast is not seen by the networks during training. We demonstrate registration accuracy surpassing the state of the art both within and across contrasts, using a single model. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images. Our code is available at doic https://w3id.org/synthmorph.

Learning the Effect of Registration Hyperparameters with HyperMorph

We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to compute a deformation field that aligns two images. Recent learning-based approaches leverage large image datasets to learn a function that rapidly estimates a deformation for a given image pair. In both strategies, the accuracy of the resulting spatial correspondences is strongly influenced by the choice of certain hyperparameter values. However, an effective hyperparameter search consumes substantial time and human effort as it often involves training multiple models for different fixed hyperparameter values and may lead to suboptimal registration. We propose an amortized hyperparameter learning strategy to alleviate this burden by learning the impact of hyperparameters on deformation fields. We design a meta network, or hypernetwork, that predicts the parameters of a registration network for input hyperparameters, thereby comprising a single model that generates the optimal deformation field corresponding to given hyperparameter values. This strategy enables fast, high-resolution hyperparameter search at test-time, reducing the inefficiency of traditional approaches while increasing flexibility. We also demonstrate additional benefits of HyperMorph, including enhanced robustness to model initialization and the ability to rapidly identify optimal hyperparameter values specific to a dataset, image contrast, task, or even anatomical region, all without the need to retrain models. We make our code publicly available at http://hypermorph.voxelmorph.net.

Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration

IMPORTANCE

When hospitals are at capacity, accurate deterioration indices could help identify low-risk patients as potential candidates for home care programs and alleviate hospital strain. To date, many existing deterioration indices are based entirely on structured data from the electronic health record (EHR) and ignore potentially useful information from other sources.

OBJECTIVE

To improve the accuracy of existing deterioration indices by incorporating unstructured imaging data from chest radiographs.

DESIGN, SETTING, AND PARTICIPANTS

Machine learning models were trained to predict deterioration of patients hospitalized with acute dyspnea using existing deterioration index scores and chest radiographs. Models were trained on hospitalized patients without coronavirus disease 2019 (COVID-19) and then subsequently tested on patients with COVID-19 between January 2020 and December 2020 at a single tertiary care center who had at least one radiograph taken within 48 hours of hospital admission.

MAIN OUTCOMES AND MEASURES

Patient deterioration was defined as the need for invasive or non-invasive mechanical ventilation, heated high flow nasal cannula, IV vasopressor administration or in-hospital mortality at any time following admission. The EPIC deterioration index was augmented with unstructured data from chest radiographs to predict risk of deterioration. We compared discriminative performance of the models with and without incorporating chest radiographs using area under the receiver operating curve (AUROC), focusing on comparing the fraction and total patients identified as low risk at different negative predictive values (NPV).

RESULTS

Data from 6278 hospitalizations were analyzed, including 5562 hospitalizations without COVID-19 (training cohort) and 716 with COVID-19 (216 in validation, 500 in held-out test cohort). At a NPV of 0.95, the best-performing image-augmented deterioration index identified 49 more (9.8%) individuals as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. At a NPV of 0.9, the EPIC image-augmented deterioration index identified 26 more individuals (5.2%) as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission.

CONCLUSION AND RELEVANCE

Augmenting existing deterioration indices with chest radiographs results in better identification of low-risk patients. The model augmentation strategy could be used in the future to incorporate other forms of unstructured data into existing disease models.

OUP accepted manuscript

Stroke represents a considerable burden of disease for both men and women. However, a growing body of literature suggests clinically relevant sex differences in the underlying causes, presentations and outcomes of acute ischaemic stroke. In a recent study, we reported sex divergences in lesion topographies: specific to women, acute stroke severity was linked to lesions in the left-hemispheric posterior circulation. We here determined whether these sex-specific brain manifestations also affect long-term outcomes. We relied on 822 acute ischaemic patients [age: 64.7 (15.0) years, 39% women] originating from the multi-centre MRI-GENIE study to model unfavourable outcomes (modified Rankin Scale >2) based on acute neuroimaging data in a Bayesian hierarchical framework. Lesions encompassing bilateral subcortical nuclei and left-lateralized regions in proximity to the insula explained outcomes across men and women (area under the curve = 0.81). A pattern of left-hemispheric posterior circulation brain regions, combining left hippocampus, precuneus, fusiform and lingual gyrus, occipital pole and latero-occipital cortex, showed a substantially higher relevance in explaining functional outcomes in women compared to men [mean difference of Bayesian posterior distributions (men – women) = −0.295 (90% highest posterior density interval = −0.556 to −0.068)]. Once validated in prospective studies, our findings may motivate a sex-specific approach to clinical stroke management and hold the promise of enhancing outcomes on a population level.

Mapping the subcortical connectivity of the human default mode network

The default mode network (DMN) mediates self-awareness and introspection, core components of human consciousness. Therapies to restore consciousness in patients with severe brain injuries have historically targeted subcortical sites in the brainstem, thalamus, hypothalamus, basal forebrain, and basal ganglia, with the goal of reactivating cortical DMN nodes. However, the subcortical connectivity of the DMN has not been fully mapped, and optimal subcortical targets for therapeutic neuromodulation of consciousness have not been identified. In this work, we created a comprehensive map of DMN subcortical connectivity by combining high-resolution functional and structural datasets with advanced signal processing methods. We analyzed 7 Tesla resting-state functional MRI (rs-fMRI) data from 168 healthy volunteers acquired in the Human Connectome Project. The rs-fMRI blood-oxygen-level-dependent (BOLD) data were temporally synchronized across subjects using the BrainSync algorithm. Cortical and subcortical DMN nodes were jointly analyzed and identified at the group level by applying a novel Nadam-Accelerated SCAlable and Robust (NASCAR) tensor decomposition method to the synchronized dataset. The subcortical connectivity map was then overlaid on a 7 Tesla 100 µm ex vivo MRI dataset for neuroanatomic analysis using automated segmentation of nuclei within the brainstem, thalamus, hypothalamus, basal forebrain, and basal ganglia. We further compared the NASCAR subcortical connectivity map with its counterpart generated from canonical seed-based correlation analyses. The NASCAR method revealed that BOLD signal in the central lateral nucleus of the thalamus and ventral tegmental area of the midbrain is strongly correlated with that of the DMN. In an exploratory analysis, additional subcortical sites in the median and dorsal raphe, lateral hypothalamus, and caudate nuclei were correlated with the cortical DMN. We also found that the putamen and globus pallidus are negatively correlated (i.e., anti-correlated) with the DMN, providing rs-fMRI evidence for the mesocircuit hypothesis of human consciousness, whereby a striatopallidal feedback system modulates anterior forebrain function via disinhibition of the central thalamus. Seed-based analyses yielded similar subcortical DMN connectivity, but the NASCAR result showed stronger contrast and better spatial alignment with dopamine immunostaining data. The DMN subcortical connectivity map identified here advances understanding of the subcortical regions that contribute to human consciousness and can be used to inform the selection of therapeutic targets in clinical trials for patients with disorders of consciousness.

A deep learning toolbox for automatic segmentation of subcortical limbic structures from MRI images

A tool was developed to automatically segment several subcortical limbic structures (nucleus accumbens, basal forebrain, septal nuclei, hypothalamus without mammillary bodies, the mammillary bodies, and fornix) using only a T1-weighted MRI as input. This tool fills an unmet need as there are few, if any, publicly available tools to segment these clinically relevant structures. A U-Net with spatial, intensity, contrast, and noise augmentation was trained using 39 manually labeled MRI data sets. In general, the Dice scores, true positive rates, false discovery rates, and manual-automatic volume correlation were very good relative to comparable tools for other structures. A diverse data set of 698 subjects were segmented using the tool; evaluation of the resulting labelings showed that the tool failed in less than 1% of cases. Test-retest reliability of the tool was excellent. The automatically segmented volume of all structures except mammillary bodies showed effectiveness at detecting either clinical AD effects, age effects, or both. This tool will be publicly released with FreeSurfer (surfer.nmr.mgh.harvard.edu/fswiki/ScLimbic). Together with the other cortical and subcortical limbic segmentations, this tool will allow FreeSurfer to provide a comprehensive view of the limbic system in an automated way.

Excessive White Matter Hyperintensity Increases Susceptibility to Poor Functional Outcomes After Acute Ischemic Stroke

Objective: To personalize the prognostication of post-stroke outcome using MRI-detected cerebrovascular pathology, we sought to investigate the association between the excessive white matter hyperintensity (WMH) burden unaccounted for by the traditional stroke risk profile of individual patients and their long-term functional outcomes after a stroke. Methods: We included 890 patients who survived after an acute ischemic stroke from the MRI-Genetics Interface Exploration (MRI-GENIE) study, for whom data on vascular risk factors (VRFs), including age, sex, atrial fibrillation, diabetes mellitus, hypertension, coronary artery disease, smoking, prior stroke history, as well as acute stroke severity, 3- to-6-month modified Rankin Scale score (mRS), WMH, and brain volumes, were available. We defined the unaccounted WMH (uWMH) burden via modeling of expected WMH burden based on the VRF profile of each individual patient. The association of uWMH and mRS score was analyzed by linear regression analysis. The odds ratios of patients who achieved full functional independence (mRS < 2) in between trichotomized uWMH burden groups were calculated by pair-wise comparisons. Results: The expected WMH volume was estimated with respect to known VRFs. The uWMH burden was associated with a long-term functional outcome (β = 0.104, p < 0.01). Excessive uWMH burden significantly reduced the odds of achieving full functional independence after a stroke compared to the low and average uWMH burden [OR = 0.4, 95% CI: (0.25, 0.63), p < 0.01 and OR = 0.61, 95% CI: (0.42, 0.87), p < 0.01, respectively]. Conclusion: The excessive amount of uWMH burden unaccounted for by the traditional VRF profile was associated with worse post-stroke functional outcomes. Further studies are needed to evaluate a lifetime brain injury reflected in WMH unrelated to the VRF profile of a patient as an important factor for stroke recovery and a plausible indicator of brain health.

MRI Radiomic Signature of White Matter Hyperintensities Is Associated With Clinical Phenotypes

OBJECTIVE

Neuroimaging measurements of brain structural integrity are thought to be surrogates for brain health, but precise assessments require dedicated advanced image acquisitions. By means of quantitatively describing conventional images, radiomic analyses hold potential for evaluating brain health. We sought to: (1) evaluate radiomics to assess brain structural integrity by predicting white matter hyperintensities burdens (WMH) and (2) uncover associations between predictive radiomic features and clinical phenotypes.

METHODS

We analyzed a multi-site cohort of 4,163 acute ischemic strokes (AIS) patients with T2-FLAIR MR images with total brain and WMH segmentations. Radiomic features were extracted from normal-appearing brain tissue (brain mask-WMH mask). Radiomics-based prediction of personalized WMH burden was done using ElasticNet linear regression. We built a radiomic signature of WMH with stable selected features predictive of WMH burden and then related this signature to clinical variables using canonical correlation analysis (CCA).

RESULTS

Radiomic features were predictive of WMH burden (R 2 = 0.855 ± 0.011). Seven pairs of canonical variates (CV) significantly correlated the radiomics signature of WMH and clinical traits with respective canonical correlations of 0.81, 0.65, 0.42, 0.24, 0.20, 0.15, and 0.15 (FDR-corrected p-values CV 1 - 6 < 0.001, p-value CV 7 = 0.012). The clinical CV1 was mainly influenced by age, CV2 by sex, CV3 by history of smoking and diabetes, CV4 by hypertension, CV5 by atrial fibrillation (AF) and diabetes, CV6 by coronary artery disease (CAD), and CV7 by CAD and diabetes.

CONCLUSION

Radiomics extracted from T2-FLAIR images of AIS patients capture microstructural damage of the cerebral parenchyma and correlate with clinical phenotypes, suggesting different radiographical textural abnormalities per cardiovascular risk profile. Further research could evaluate radiomics to predict the progression of WMH and for the follow-up of stroke patients' brain health.

Outcome after acute ischemic stroke is linked to sex-specific lesion patterns

Acute ischemic stroke affects men and women differently. In particular, women are often reported to experience higher acute stroke severity than men. We derived a low-dimensional representation of anatomical stroke lesions and designed a Bayesian hierarchical modeling framework tailored to estimate possible sex differences in lesion patterns linked to acute stroke severity (National Institute of Health Stroke Scale). This framework was developed in 555 patients (38% female). Findings were validated in an independent cohort (n = 503, 41% female). Here, we show brain lesions in regions subserving motor and language functions help explain stroke severity in both men and women, however more widespread lesion patterns are relevant in female patients. Higher stroke severity in women, but not men, is associated with left hemisphere lesions in the vicinity of the posterior circulation. Our results suggest there are sex-specific functional cerebral asymmetries that may be important for future investigations of sex-stratified approaches to management of acute ischemic stroke.

Reliability and sensitivity of two whole-brain segmentation approaches included in FreeSurfer - ASEG and SAMSEG

Accurate and reliable whole-brain segmentation is critical to longitudinal neuroimaging studies. We undertake a comparative analysis of two subcortical segmentation methods, Automatic Segmentation (ASEG) and Sequence Adaptive Multimodal Segmentation (SAMSEG), recently provided in the open-source neuroimaging package FreeSurfer 7.1, with regard to reliability, bias, sensitivity to detect longitudinal change, and diagnostic sensitivity to Alzheimer’s disease. First, we assess intra- and inter-scanner reliability for eight bilateral subcortical structures: amygdala, caudate, hippocampus, lateral ventricles, nucleus accumbens, pallidum, putamen and thalamus. For intra-scanner analysis we use a large sample of participants (n = 1629) distributed across the lifespan (age range = 4–93 years) and acquired on a 1.5T Siemens Avanto (n = 774) and a 3T Siemens Skyra (n = 855) scanners. For inter-scanner analysis we use a sample of 24 participants scanned on the day with three models of Siemens scanners: 1.5T Avanto, 3T Skyra and 3T Prisma. Second, we test how each method detects volumetric age change using longitudinal follow up scans (n = 491 for Avanto and n = 245 for Skyra; interscan interval = 1–10 years). Finally, we test sensitivity to clinically relevant change. We compare annual rate of hippocampal atrophy in cognitively normal older adults (n = 20), patients with mild cognitive impairment (n = 20) and Alzheimer’s disease (n = 20). We find that both ASEG and SAMSEG are reliable and lead to the detection of within-person longitudinal change, although with notable differences between age-trajectories for most structures, including hippocampus and amygdala. In summary, SAMSEG yields significantly lower differences between repeated measures for intra- and inter-scanner analysis without compromising sensitivity to changes and demonstrating ability to detect clinically relevant longitudinal changes.

LEARNING MRI CONTRAST-AGNOSTIC REGISTRATION

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to magnetic resonance imaging (MRI) contrast. While classical methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning methods are fast at test time but limited to images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency using a generative strategy that exposes networks to a wide range of images synthesized from segmentations during training, forcing them to generalize across contrasts. We show that networks trained within this framework generalize to a broad array of unseen MRI contrasts and surpass classical state-of-the-art brain registration accuracy by up to 12.4 Dice points for a variety of tested contrast combinations. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images during training.

JOINT SEGMENTATION OF MULTIPLE SCLEROSIS LESIONS AND BRAIN ANATOMY IN MRI SCANS OF ANY CONTRAST AND RESOLUTION WITH CNNs

We present the first deep learning method to segment Multiple Sclerosis lesions and brain structures from MRI scans of any (possibly multimodal) contrast and resolution. Our method only requires segmentations to be trained (no images), as it leverages the generative model of Bayesian segmentation to generate synthetic scans with simulated lesions, which are then used to train a CNN. Our method can be retrained to segment at any resolution by adjusting the amount of synthesised partial volume. By construction, the synthetic scans are perfectly aligned with their labels, which enables training with noisy labels obtained with automatic methods. The training data are generated on the fly, and aggressive augmentation (including artefacts) is applied for improved generalisation. We demonstrate our method on two public datasets, comparing it with a state-of-the-art Bayesian approach implemented in FreeSurfer, and dataset specific CNNs trained on real data. The code is available at https://github.com/BBillot/SynthSeg.

Abstract MP16: Excessive White Matter Hyperintensity Burden and Functional Outcomes After Acute Ischemic Stroke

Objective: Ability of the brain to recover after an acute ischemic stroke (AIS) is linked to the pre-stroke burden of white matter hyperintensity (WMH), a radiographic marker of brain health. We sought to determine the excessive WMH burden in an AIS population and investigate its association with 3-month stroke outcomes.

Data: We used 2,435 subjects from the MRI-GENIE study. Three-month functional outcomes of 872 subjects among those subjects were measured by 90-day modified Ranking Scale (mRS).

Methods: We automatically quantified WMH volume (WMHv) on FLAIR images and adjusted for a brain volume. We modeled a trend using the factor analysis (FA) log-linear regression using age, sex, atrial fibrillation, diabetes, hypertension, coronary artery disease and smoking as input variables. We categorized three WMH burden groups based on the conditional probability given by the model (LOW: lower 33%, MED: middle 34%, and HIGH: upper 33%). The subgroups were compared with respect to mRS (median and dichotomized odds ratio (OR) (good/poor: mRS 0-2/3-6)).

Results: Five FA components out of seven with significant relationship to WMHv (p<0.001) were used for the regression modeling (R 2 =0.359). The HIGH group showed higher median (median=2, IQR=2) mRS score than LOW (median=1, IQR=1) and MED (median=1, IQR=1). The odds (OR) of good AIS outcome for LOW and MED were 1.8 (p=0.0001) and 1.6 (p=0.006) times higher than HIGH, respectively.

Conclusion: Once accounted for clinical covariates, the excessive WMHv was associated with worse 3-month stroke outcomes. These data suggest that a life-time of injury to the white matter reflected in WMH is an important factor for stroke recovery and an indicator of the brain health.

Abstract 10: Radiomic Signature of the White Matter Hyperintensity Burden Correlates With Clinical Phenotypes

Introduction: Structural integrity of cerebral parenchyma is an essential radiographic equivalent of brain health; but its assessment usually requires dedicated advanced image acquisitions. Radiomics analyses bear the potential to describe radiophenotypes beyond what meets the naked eye. We sought to: 1) evaluate this novel approach to predict white matter hyperintensity (WMH) burden and 2) uncover latent clinico-radiological associations.

Methods: An international, multi-site cohort of 4,164 acute ischemic stroke (AIS) patients with FLAIR MRI (MRI-GENIE study) underwent total brain and WMH lesion segmentation using convolutional neural networks. Radiomic features (n=1905) were extracted from clinical FLAIR images outside of the WMH (brain mask - WMH mask). Prediction of the WMH burden using radiomics was done using LASSO regression. Radiomic signature of WMH was built with the most stable selected features, then compared to the clinical variables using canonical correlation analysis.

Results: In this cohort, (mean age=62.8±15.0, median WMH volume=4.2cc IQR 1.4-11.2), radiomic features were highly predictive of WMH burden (R2=0.8±0.012). Radiomic signature of WMH included 68 features. All 7 pairs of extracted canonical variates (CV) were statistically significant with respective canonical correlations of 0.79, 0.64, 0.44, 0.21, 0.16, 0.15 (Bonferroni corrected p-values CV1-6 <.001, p-value CV7 =.003). Upon examination, CV1 was mainly influenced by age, CV2 by sex, CV3 by history of smoking and diabetes mellitus (DM), CV4 by hypertension, CV5 by atrial fibrillation (AF) and DM, CV6 by coronary artery disease (CAD) and CV7 by CAD and DM.

Conclusion: Radiomics extracted from clinical grade FLAIR images of AIS patients seem able to capture structural integrity of the cerebral parenchyma and to correlate with clinical phenotypes. Further research could evaluate radiomics to predict the progression of cerebral small vessel disease on longitudinal data.

Cortical surface registration using unsupervised learning

Non-rigid cortical registration is an important and challenging task due to the geometric complexity of the human cortex and the high degree of inter-subject variability. A conventional solution is to use a spherical representation of surface properties and perform registration by aligning cortical folding patterns in that space. This strategy produces accurate spatial alignment, but often requires high computational cost. Recently, convolutional neural networks (CNNs) have demonstrated the potential to dramatically speed up volumetric registration. However, due to distortions introduced by projecting a sphere to a 2D plane, a direct application of recent learning-based methods to surfaces yields poor results. In this study, we present SphereMorph, a diffeomorphic registration framework for cortical surfaces using deep networks that addresses these issues. SphereMorph uses a UNet-style network associated with a spherical kernel to learn the displacement field and warps the sphere using a modified spatial transformer layer. We propose a resampling weight in computing the data fitting loss to account for distortions introduced by polar projection, and demonstrate the performance of our proposed method on two tasks, including cortical parcellation and group-wise functional area alignment. The experiments show that the proposed SphereMorph is capable of modeling the geometric registration problem in a CNN framework and demonstrate superior registration accuracy and computational efficiency. The source code of SphereMorph will be released to the public upon acceptance of this manuscript at https://github.com/voxelmorph/spheremorph.

Diffusion-Weighted Imaging, MR Angiography, and Baseline Data in a Systematic Multicenter Analysis of 3,301 MRI Scans of Ischemic Stroke Patients-Neuroradiological Review Within the MRI-GENIE Study

Background: Magnetic resonance imaging (MRI) serves as a cornerstone in defining stroke phenotype and etiological subtype through examination of ischemic stroke lesion appearance and is therefore an essential tool in linking genetic traits and stroke. Building on baseline MRI examinations from the centralized and structured radiological assessments of ischemic stroke patients in the Stroke Genetics Network, the results of the MRI-Genetics Interface Exploration (MRI-GENIE) study are described in this work.

Methods: The MRI-GENIE study included patients with symptoms caused by ischemic stroke (N = 3,301) from 12 international centers. We established and used a structured reporting protocol for all assessments. Two neuroradiologists, using a blinded evaluation protocol, independently reviewed the baseline diffusion-weighted images (DWIs) and magnetic resonance angiography images to determine acute lesion and vascular occlusion characteristics.

Results: In this systematic multicenter radiological analysis of clinical MRI from 3,301 acute ischemic stroke patients according to a structured prespecified protocol, we identified that anterior circulation infarcts were most prevalent (67.4%), that infarcts in the middle cerebral artery (MCA) territory were the most common, and that the majority of large artery occlusions 0 to 48 h from ictus were in the MCA territory. Multiple acute lesions in one or several vascular territories were common (11%). Of 2,238 patients with unilateral DWI lesions, 52.6% had left-sided infarct lateralization (P = 0.013 for χ² test).

Conclusions: This large-scale analysis of a multicenter MRI-based cohort of AIS patients presents a unique imaging framework facilitating the relationship between imaging and genetics for advancing the knowledge of genetic traits linked to ischemic stroke.

Multi-atlas image registration of clinical data with automated quality assessment using ventricle segmentation

Registration is a core component of many imaging pipelines. In case of clinical scans, with lower resolution and sometimes substantial motion artifacts, registration can produce poor results. Visual assessment of registration quality in large clinical datasets is inefficient. In this work, we propose to automatically assess the quality of registration to an atlas in clinical FLAIR MRI scans of the brain. The method consists of automatically segmenting the ventricles of a given scan using a neural network, and comparing the segmentation to the atlas ventricles propagated to image space. We used the proposed method to improve clinical image registration to a general atlas by computing multiple registrations - one directly to the general atlas and others via different age-specific atlases - and then selecting the registration that yielded the highest ventricle overlap. Finally, as an example application of the complete pipeline, a voxelwise map of white matter hyperintensity burden was computed using only the scans with registration quality above a predefined threshold. Methods were evaluated in a single-site dataset of more than 1000 scans, as well as a multi-center dataset comprising 142 clinical scans from 12 sites. The automated ventricle segmentation reached a Dice coefficient with manual annotations of 0.89 in the single-site dataset, and 0.83 in the multi-center dataset. Registration via age-specific atlases could improve ventricle overlap compared to a direct registration to the general atlas (Dice similarity coefficient increase up to 0.15). Experiments also showed that selecting scans with the registration quality assessment method could improve the quality of average maps of white matter hyperintensity burden, instead of using all scans for the computation of the white matter hyperintensity map. In this work, we demonstrated the utility of an automated tool for assessing image registration quality in clinical scans. This image quality assessment step could ultimately assist in the translation of automated neuroimaging pipelines to the clinic.

White matter hyperintensity burden in acute stroke patients differs by ischemic stroke subtype

OBJECTIVE

To examine etiologic stroke subtypes and vascular risk factor profiles and their association with white matter hyperintensity (WMH) burden in patients hospitalized for acute ischemic stroke (AIS).

METHODS

For the MRI Genetics Interface Exploration (MRI-GENIE) study, we systematically assembled brain imaging and phenotypic data for 3,301 patients with AIS. All cases underwent standardized web tool-based stroke subtyping with the Causative Classification of Ischemic Stroke (CCS). WMH volume (WMHv) was measured on T2 brain MRI scans of 2,529 patients with a fully automated deep-learning trained algorithm. Univariable and multivariable linear mixed-effects modeling was carried out to investigate the relationship of vascular risk factors with WMHv and CCS subtypes.

RESULTS

Patients with AIS with large artery atherosclerosis, major cardioembolic stroke, small artery occlusion (SAO), other, and undetermined causes of AIS differed significantly in their vascular risk factor profile (all p < 0.001). Median WMHv in all patients with AIS was 5.86 cm³ (interquartile range 2.18-14.61 cm³) and differed significantly across CCS subtypes (p < 0.0001). In multivariable analysis, age, hypertension, prior stroke, smoking (all p < 0.001), and diabetes mellitus (p = 0.041) were independent predictors of WMHv. When adjusted for confounders, patients with SAO had significantly higher WMHv compared to those with all other stroke subtypes (p < 0.001).

CONCLUSION

In this international multicenter, hospital-based cohort of patients with AIS, we demonstrate that vascular risk factor profiles and extent of WMH burden differ by CCS subtype, with the highest lesion burden detected in patients with SAO. These findings further support the small vessel hypothesis of WMH lesions detected on brain MRI of patients with ischemic stroke.

Brain Volume: An Important Determinant of Functional Outcome After Acute Ischemic Stroke

OBJECTIVE

To determine whether brain volume is associated with functional outcome after acute ischemic stroke (AIS).

PATIENTS AND METHODS

This study was conducted between July 1, 2014, and March 16, 2019. We analyzed cross-sectional data of the multisite, international hospital-based MRI-Genetics Interface Exploration study with clinical brain magnetic resonance imaging obtained on admission for index stroke and functional outcome assessment. Poststroke outcome was determined using the modified Rankin Scale score (0-6; 0 = asymptomatic; 6 = death) recorded between 60 and 190 days after stroke. Demographic characteristics and other clinical variables including acute stroke severity (measured as National Institutes of Health Stroke Scale score), vascular risk factors, and etiologic stroke subtypes (Causative Classification of Stroke system) were recorded during index admission.

RESULTS

Utilizing the data from 912 patients with AIS (mean ± SD age, 65.3±14.5 years; male, 532 [58.3%]; history of smoking, 519 [56.9%]; hypertension, 595 [65.2%]) in a generalized linear model, brain volume (per 155.1 cm³) was associated with age (β -0.3 [per 14.4 years]), male sex (β 1.0), and prior stroke (β -0.2). In the multivariable outcome model, brain volume was an independent predictor of modified Rankin Scale score (β -0.233), with reduced odds of worse long-term functional outcomes (odds ratio, 0.8; 95% CI, 0.7-0.9) in those with larger brain volumes.

CONCLUSION

Larger brain volume quantified on clinical magnetic resonance imaging of patients with AIS at the time of stroke purports a protective mechanism. The role of brain volume as a prognostic, protective biomarker has the potential to forge new areas of research and advance current knowledge of the mechanisms of poststroke recovery.

Detailed phenotyping of posterior vs. anterior circulation ischemic stroke: a multi-center MRI study

OBJECTIVE

Posterior circulation ischemic stroke (PCiS) constitutes 20-30% of ischemic stroke cases. Detailed information about differences between PCiS and anterior circulation ischemic stroke (ACiS) remains scarce. Such information might guide clinical decision making and prevention strategies. We studied risk factors and ischemic stroke subtypes in PCiS vs. ACiS and lesion location on magnetic resonance imaging (MRI) in PCiS.

METHODS

Out of 3,301 MRIs from 12 sites in the National Institute of Neurological Disorders and Stroke (NINDS) Stroke Genetics Network (SiGN), we included 2,381 cases with acute DWI lesions. The definition of ACiS or PCiS was based on lesion location. We compared the groups using Chi-squared and logistic regression.

RESULTS

PCiS occurred in 718 (30%) patients and ACiS in 1663 (70%). Diabetes and male sex were more common in PCiS vs. ACiS (diabetes 27% vs. 23%, p < 0.05; male sex 68% vs. 58%, p < 0.001). Both were independently associated with PCiS (diabetes, OR = 1.29; 95% CI 1.04-1.61; male sex, OR = 1.46; 95% CI 1.21-1.78). ACiS more commonly had large artery atherosclerosis (25% vs. 20%, p < 0.01) and cardioembolic mechanisms (17% vs. 11%, p < 0.001) compared to PCiS. Small artery occlusion was more common in PCiS vs. ACiS (20% vs. 14%, p < 0.001). Small artery occlusion accounted for 47% of solitary brainstem infarctions.

CONCLUSION

Ischemic stroke subtypes differ between the two phenotypes. Diabetes and male sex have a stronger association with PCiS than ACiS. Definitive MRI-based PCiS diagnosis aids etiological investigation and contributes additional insights into specific risk factors and mechanisms of injury in PCiS.

Trends and Focus of Machine Learning Applications for Health Research

IMPORTANCE

The use of machine learning applications related to health is rapidly increasing and may have the potential to profoundly affect the field of health care.

OBJECTIVE

To analyze submissions to a popular machine learning for health venue to assess the current state of research, including areas of methodologic and clinical focus, limitations, and underexplored areas.

DESIGN, SETTING, AND PARTICIPANTS

In this data-driven qualitative analysis, 166 accepted manuscript submissions to the Third Annual Machine Learning for Health workshop at the 32nd Conference on Neural Information Processing Systems on December 8, 2018, were analyzed to understand research focus, progress, and trends. Experts reviewed each submission against a rubric to identify key data points, statistical modeling and analysis of submitting authors was performed, and research topics were quantitatively modeled. Finally, an iterative discussion of topics common in submissions and invited speakers at the workshop was held to identify key trends.

MAIN OUTCOMES AND MEASURES

Frequency and statistical measures of methods, topics, goals, and author attributes were derived from an expert review of submissions guided by a rubric.

RESULTS

Of the 166 accepted submissions, 58 (34.9%) had clinician involvement and 83 submissions (50.0%) that focused on clinical practice included clinical collaborators. A total of 97 data sets (58.4%) used in submissions were publicly available or required a standard registration process. Clinical practice was the most common application area (70 manuscripts [42.2%]), with brain and mental health (25 [15.1%]), oncology (21 [12.7%]), and cardiovascular (19 [11.4%]) being the most common specialties.

CONCLUSIONS AND RELEVANCE

Trends in machine learning for health research indicate the importance of well-annotated, easily accessed data and the benefit from greater clinician involvement in the development of translational applications.

Unsupervised Deep Learning for Bayesian Brain MRI Segmentation

Probabilistic atlas priors have been commonly used to derive adaptive and robust brain MRI segmentation algorithms. Widely-used neuroimage analysis pipelines rely heavily on these techniques, which are often computationally expensive. In contrast, there has been a recent surge of approaches that leverage deep learning to implement segmentation tools that are computationally efficient at test time. However, most of these strategies rely on learning from manually annotated images. These supervised deep learning methods are therefore sensitive to the intensity profiles in the training dataset. To develop a deep learning-based segmentation model for a new image dataset (e.g., of different contrast), one usually needs to create a new labeled training dataset, which can be prohibitively expensive, or rely on suboptimal ad hoc adaptation or augmentation approaches. In this paper, we propose an alternative strategy that combines a conventional probabilistic atlas-based segmentation with deep learning, enabling one to train a segmentation model for new MRI scans without the need for any manually segmented images. Our experiments include thousands of brain MRI scans and demonstrate that the proposed method achieves good accuracy for a brain MRI segmentation task for different MRI contrasts, requiring only approximately 15 seconds at test time on a GPU.

White matter hyperintensity quantification in large-scale clinical acute ischemic stroke cohorts - The MRI-GENIE study

White matter hyperintensity (WMH) burden is a critically important cerebrovascular phenotype linked to prediction of diagnosis and prognosis of diseases, such as acute ischemic stroke (AIS). However, current approaches to its quantification on clinical MRI often rely on time intensive manual delineation of the disease on T2 fluid attenuated inverse recovery (FLAIR), which hinders high-throughput analyses such as genetic discovery.

In this work, we present a fully automated pipeline for quantification of WMH in clinical large-scale studies of AIS. The pipeline incorporates automated brain extraction, intensity normalization and WMH segmentation using spatial priors. We first propose a brain extraction algorithm based on a fully convolutional deep learning architecture, specifically designed for clinical FLAIR images. We demonstrate that our method for brain extraction outperforms two commonly used and publicly available methods on clinical quality images in a set of 144 subject scans across 12 acquisition centers, based on dice coefficient (median 0.95; inter-quartile range 0.94–0.95; p < 0.01) and Pearson correlation of total brain volume (r = 0.90). Subsequently, we apply it to the large-scale clinical multi-site MRI-GENIE study (N = 2783) and identify a decrease in total brain volume of −2.4 cc/year. Additionally, we show that the resulting total brain volumes can successfully be used for quality control of image preprocessing.

Finally, we obtain WMH volumes by building on an existing automatic WMH segmentation algorithm that delineates and distinguishes between different cerebrovascular pathologies. The learning method mimics expert knowledge of the spatial distribution of the WMH burden using a convolutional auto-encoder. This enables successful computation of WMH volumes of 2533 clinical AIS patients. We utilize these results to demonstrate the increase of WMH burden with age (0.950 cc/year) and show that single site estimates can be biased by the number of subjects recruited.

•

Fully automated high-throughput white matter hyperintensity segmentation pipeline.

•

Methodology designed for and applied to international clinical multi-site data.

•

Calculation of disease burden in 2533 acute ischemic stroke patients.

•

Total brain volume change with age (−2.4 cc/year) used in automated quality control.

•

Increase of white matter hyperintensity burden of 0.95 cc/year.

VoxelMorph: A Learning Framework for Deformable Medical Image Registration

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at https://github.com/voxelmorph/voxelmorph.

Iterative Segmentation from Limited Training Data: Applications to Congenital Heart Disease

We propose a new iterative segmentation model which can be accurately learned from a small dataset. A common approach is to train a model to directly segment an image, requiring a large collection of manually annotated images to capture the anatomical variability in a cohort. In contrast, we develop a segmentation model that recursively evolves a segmentation in several steps, and implement it as a recurrent neural network. We learn model parameters by optimizing the intermediate steps of the evolution in addition to the final segmentation. To this end, we train our segmentation propagation model by presenting incomplete and/or inaccurate input segmentations paired with a recommended next step. Our work aims to alleviate challenges in segmenting heart structures from cardiac MRI for patients with congenital heart disease (CHD), which encompasses a range of morphological deformations and topological changes. We demonstrate the advantages of this approach on a dataset of 20 images from CHD patients, learning a model that accurately segments individual heart chambers and great vessels. Compared to direct segmentation, the iterative method yields more accurate segmentation for patients with the most severe CHD malformations.

Medical Image Imputation from Image Collections

We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis impractical as many image analysis algorithms tend to fail when applied to such images. Highly specialized algorithms that explicitly handle sparse slice spacing do not generalize well across problem domains. In contrast, we aim to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. We introduce a generative model that captures fine-scale anatomical structure across subjects in clinical image collections and derive an algorithm for filling in the missing data in scans with large inter-slice spacing. Our experimental results demonstrate that the resulting method outperforms state-of-the-art upsampling super-resolution techniques, and promises to facilitate subsequent analysis not previously possible with scans of this quality. Our implementation is freely available at https://github.com/adalca/papago.