Filtering in tractography using autoencoders (FINTA)

Supervised tractogram filtering using Geometric Deep Learning

A tractogram is a virtual representation of the brain white matter. It is composed of millions of virtual fibers, encoded as 3D polylines, which approximate the white matter axonal pathways. To date, tractograms are the most accurate white matter representation and thus are used for tasks like presurgical planning and investigations of neuroplasticity, brain disorders, or brain networks. However, it is a well-known issue that a large portion of tractogram fibers is not anatomically plausible and can be considered artifacts of the tracking procedure. With Verifyber, we tackle the problem of filtering out such non-plausible fibers using a novel fully-supervised learning approach. Differently from other approaches based on signal reconstruction and/or brain topology regularization, we guide our method with the existing anatomical knowledge of the white matter. Using tractograms annotated according to anatomical principles, we train our model, Verifyber, to classify fibers as either anatomically plausible or non-plausible. The proposed Verifyber model is an original Geometric Deep Learning method that can deal with variable size fibers, while being invariant to fiber orientation. Our model considers each fiber as a graph of points, and by learning features of the edges between consecutive points via the proposed sequence Edge Convolution, it can capture the underlying anatomical properties. The output filtering results highly accurate and robust across an extensive set of experiments, and fast; with a 12GB GPU, filtering a tractogram of 1M fibers requires less than a minute.

Evaluation of tractogram filtering methods using human-like connectome phantoms

Tractography algorithms are prone to reconstructing spurious connections. The set of streamlines generated with tractography can be post-processed to retain the streamlines that are most biologically plausible. Several microstructure-informed filtering algorithms are available for this purpose, however, the comparative performance of these methods has not been extensively evaluated. In this study, we aim to evaluate streamline filtering and post-processing algorithms using simulated connectome phantoms. We first establish a framework for generating connectome phantoms featuring brain-like white matter fiber architectures. We then use our phantoms to systematically evaluate the performance of a range of streamline filtering algorithms, including SIFT, COMMIT, and LiFE. We find that all filtering methods successfully improve connectome accuracy, although filter performance depends on the complexity of the underlying white matter fiber architecture. Filtering algorithms can markedly improve tractography accuracy for simple tubular fiber bundles (F-measure deterministic- unfiltered: 0.49 and best filter: 0.72; F-measure probabilistic- unfiltered: 0.37 and best filter: 0.81), but for more complex brain-like fiber architectures, the improvement is modest (F-measure deterministic- unfiltered: 0.53 and best filter: 0.54; F-measure probabilistic- unfiltered: 0.46 and best filter: 0.50). Overall, filtering algorithms have the potential to improve the accuracy of connectome mapping pipelines, particularly for weighted connectomes and pipelines using probabilistic tractography methods. Our results highlight the need for further advances tractography and streamline filtering to improve the accuracy of connectome mapping.

FASSt : Filtering via Symmetric Autoencoder for Spherical Superficial White Matter Tractography

Superficial white matter (SWM) plays an important role in functioning of the human brain, and it contains a large amount of cortico-cortical connections. However, the difficulties of generating complete and reliable U-fibers make SWM-related analysis lag behind relatively matured Deep white matter (DWM) analysis. With the aid of some newly proposed surface-based SWM tractography algorithms, we have developed a specialized SWM filtering method based on a symmetric variational autoencoder (VAE). In this work, we first demonstrate the advantage of the spherical representation and generate these spherical tracts using the triangular mesh and the registered spherical surface. We then introduce the Filtering via symmetric Autoencoder for Spherical Superficial White Matter tractography (FASSt) framework with a novel symmetric weights module to perform the filtering task in a latent space. We evaluate and compare our method with the state-of-the-art clustering-based method on diffusion MRI data from Human Connectome Project (HCP). The results show that our proposed method outperform these clustering methods and achieves excellent performance in groupwise consistency and topographic regularity.

FIESTA: Autoencoders for accurate fiber segmentation in tractography

White matter bundle segmentation is a cornerstone of modern tractography to study the brain's structural connectivity in domains such as neurological disorders, neurosurgery, and aging. In this study, we present FIESTA (FIbEr Segmentation in Tractography using Autoencoders), a reliable and robust, fully automated, and easily semi-automatically calibrated pipeline based on deep autoencoders that can dissect and fully populate white matter bundles. This pipeline is built upon previous works that demonstrated how autoencoders can be used successfully for streamline filtering, bundle segmentation, and streamline generation in tractography. Our proposed method improves bundle segmentation coverage by recovering hard-to-track bundles with generative sampling through the latent space seeding of the subject bundle and the atlas bundle. A latent space of streamlines is learned using autoencoder-based modeling combined with contrastive learning. Using an atlas of bundles in standard space (MNI), our proposed method segments new tractograms using the autoencoder latent distance between each tractogram streamline and its closest neighbor bundle in the atlas of bundles. Intra-subject bundle reliability is improved by recovering hard-to-track streamlines, using the autoencoder to generate new streamlines that increase the spatial coverage of each bundle while remaining anatomically correct. Results show that our method is more reliable than state-of-the-art automated virtual dissection methods such as RecoBundles, RecoBundlesX, TractSeg, White Matter Analysis and XTRACT. Our framework allows for the transition from one anatomical bundle definition to another with marginal calibration efforts. Overall, these results show that our framework improves the practicality and usability of current state-of-the-art bundle segmentation framework.

BundleCleaner: Unsupervised Denoising and Subsampling of Diffusion MRI-Derived Tractography Data

We present BundleCleaner, an unsupervised multi-step framework that can filter, denoise and subsample bundles derived from diffusion MRI-based whole-brain tractography. Our approach considers both the global bundle structure and local streamline-wise features. We apply BundleCleaner to bundles generated from single-shell diffusion MRI data in an independent clinical sample of older adults from India using probabilistic tractography and the resulting 'cleaned' bundles can better align with the atlas bundles with reduced overreach. In a downstream tractometry analysis, we show that the cleaned bundles, represented with less than 20% of the original set of points, can robustly localize along-tract microstructural differences between 32 healthy controls and 34 participants with Alzheimer's disease ranging in age from 55 to 84 years old. Our approach can help reduce memory burden and improving computational efficiency when working with tractography data, and shows promise for large-scale multi-site tractometry.

Variational Autoencoders for Generating Synthetic Tractography-Based Bundle Templates in a Low-Data Setting

White matter tracts generated from whole brain tractography are often processed using automatic segmentation methods with standard atlases. Atlases are generated from hundreds of subjects, which becomes time-consuming to create and difficult to apply to all populations. In this study, we extended our prior work on using a deep generative model - a Convolutional Variational Autoencoder - to map complex and data-intensive streamlines to a low-dimensional latent space given a limited sample size of 50 subjects from the ADNI3 dataset, to generate synthetic population-specific bundle templates using Kernel Density Estimation (KDE) on streamline embeddings. We conducted a quantitative shape analysis by calculating bundle shape metrics, and found that our bundle templates better capture the shape distribution of the bundles than the atlas data used in the original segmentation derived from young healthy adults. We further demonstrated the use of our framework for direct bundle segmentation from whole-brain tractograms.

Deep Learning Methods for Identification of White Matter Fiber Tracts: Review of State-of-the-Art and Future Prospective

Quantitative analysis of white matter fiber tracts from diffusion Magnetic Resonance Imaging (dMRI) data is of great significance in health and disease. For example, analysis of fiber tracts related to anatomically meaningful fiber bundles is highly demanded in pre-surgical and treatment planning, and the surgery outcome depends on accurate segmentation of the desired tracts. Currently, this process is mainly done through time-consuming manual identification performed by neuro-anatomical experts. However, there is a broad interest in automating the pipeline such that it is fast, accurate, and easy to apply in clinical settings and also eliminates the intra-reader variabilities. Following the advancements in medical image analysis using deep learning techniques, there has been a growing interest in using these techniques for the task of tract identification as well. Recent reports on this application show that deep learning-based tract identification approaches outperform existing state-of-the-art methods. This paper presents a review of current tract identification approaches based on deep neural networks. First, we review the recent deep learning methods for tract identification. Next, we compare them with respect to their performance, training process, and network properties. Finally, we end with a critical discussion of open challenges and possible directions for future works.

Deep fiber clustering: Anatomically informed fiber clustering with self-supervised deep learning for fast and effective tractography parcellation

White matter fiber clustering is an important strategy for white matter parcellation, which enables quantitative analysis of brain connections in health and disease. In combination with expert neuroanatomical labeling, data-driven white matter fiber clustering is a powerful tool for creating atlases that can model white matter anatomy across individuals. While widely used fiber clustering approaches have shown good performance using classical unsupervised machine learning techniques, recent advances in deep learning reveal a promising direction toward fast and effective fiber clustering. In this work, we propose a novel deep learning framework for white matter fiber clustering, Deep Fiber Clustering (DFC), which solves the unsupervised clustering problem as a self-supervised learning task with a domain-specific pretext task to predict pairwise fiber distances. This process learns a high-dimensional embedding feature representation for each fiber, regardless of the order of fiber points reconstructed during tractography. We design a novel network architecture that represents input fibers as point clouds and allows the incorporation of additional sources of input information from gray matter parcellation. Thus, DFC makes use of combined information about white matter fiber geometry and gray matter anatomy to improve the anatomical coherence of fiber clusters. In addition, DFC conducts outlier removal naturally by rejecting fibers with low cluster assignment probability. We evaluate DFC on three independently acquired cohorts, including data from 220 individuals across genders, ages (young and elderly adults), and different health conditions (healthy control and multiple neuropsychiatric disorders). We compare DFC to several state-of-the-art white matter fiber clustering algorithms. Experimental results demonstrate superior performance of DFC in terms of cluster compactness, generalization ability, anatomical coherence, and computational efficiency.

Variational Autoencoders for Generating Synthetic Tractography-Based Bundle Templates in a Low-Data Setting

White matter tracts generated from whole brain tractography are often processed using automatic segmentation methods with standard atlases. Atlases are generated from hundreds of subjects, which becomes time-consuming to create and difficult to apply to all populations. In this study, we extended our prior work on using a deep generative model a Convolutional Variational Autoencoder - to map complex and data-intensive streamlines to a low-dimensional latent space given a limited sample size of 50 subjects from the ADNI3 dataset, to generate synthetic population-specific bundle templates using Kernel Density Estimation (KDE) on streamline embeddings. We conducted a quantitative shape analysis by calculating bundle shape metrics, and found that our bundle templates better capture the shape distribution of the bundles than the atlas data used in the original segmentation derived from young healthy adults. We further demonstrated the use of our framework for direct bundle segmentation from whole-brain tractograms.

Generative Sampling in Bundle Tractography using Autoencoders (GESTA)

Current tractography methods use the local orientation information to propagate streamlines from seed locations. Many such seeds provide streamlines that stop prematurely or fail to map the true white matter pathways because some bundles are "harder-to-track" than others. This results in tractography reconstructions with poor white and gray matter spatial coverage. In this work, we propose a generative, autoencoder-based method, named GESTA (Generative Sampling in Bundle Tractography using Autoencoders), that produces streamlines achieving better spatial coverage. Compared to other deep learning methods, our autoencoder-based framework uses a single model to generate streamlines in a bundle-wise fashion, and does not require to propagate local orientations. GESTA produces new and complete streamlines for any given white matter bundle, including hard-to-track bundles. Applied on top of a given tractogram, GESTA is shown to be effective in improving the white matter volume coverage in poorly populated bundles, both on synthetic and human brain in vivo data. Our streamline evaluation framework ensures that the streamlines produced by GESTA are anatomically plausible and fit well to the local diffusion signal. The streamline evaluation criteria assess anatomy (white matter coverage), local orientation alignment (direction), and geometry features of streamlines, and optionally, gray matter connectivity. The GESTA framework offers considerable gains in bundle overlap using a reduced set of seeding streamlines with a 1.5x improvement for the "Fiber Cup", and 6x for the ISMRM 2015 Tractography Challenge datasets. Similarly, it provides a 4x white matter volume increase on the BIL&GIN callosal homotopic dataset, and successfully populates bundles on the multi-subject, multi-site, whole-brain in vivo TractoInferno dataset. GESTA is thus a novel deep generative bundle tractography method that can be used to improve the tractography reconstruction of the white matter.

TractoInferno - A large-scale, open-source, multi-site database for machine learning dMRI tractography

TractoInferno is the world's largest open-source multi-site tractography database, including both research- and clinical-like human acquisitions, aimed specifically at machine learning tractography approaches and related ML algorithms. It provides 284 samples acquired from 3 T scanners across 6 different sites. Available data includes T1-weighted images, single-shell diffusion MRI (dMRI) acquisitions, spherical harmonics fitted to the dMRI signal, fiber ODFs, and reference streamlines for 30 delineated bundles generated using 4 tractography algorithms, as well as masks needed to run tractography algorithms. Manual quality control was additionally performed at multiple steps of the pipeline. We showcase TractoInferno by benchmarking the learn2track algorithm and 5 variations of the same recurrent neural network architecture. Creating the TractoInferno database required approximately 20,000 CPU-hours of processing power, 200 man-hours of manual QC, 3,000 GPU-hours of training baseline models, and 4 Tb of storage, to produce a final database of 350 Gb. By providing a standardized training dataset and evaluation protocol, TractoInferno is an excellent tool to address common issues in machine learning tractography.

FiberNeat: Unsupervised White Matter Tract Filtering

Whole-brain tractograms generated from diffusion MRI digitally represent the white matter structure of the brain and are composed of millions of streamlines. Such tractograms can have false positive and anatomically implausible streamlines. To obtain anatomically relevant streamlines and tracts, supervised and unsupervised methods can be used for tractogram clustering and tract extraction. Here we propose FiberNeat, an unsupervised white matter tract filtering method. FiberNeat takes an input set of streamlines that could either be unlabeled clusters or labeled tracts. Individual clusters/tracts are projected into a latent space using nonlinear dimensionality reduction techniques, t-SNE and UMAP, to find spurious and outlier streamlines. In addition, outlier streamline clusters are detected using DBSCAN and then removed from the data in streamline space. We performed quantitative comparisons with expertly delineated tracts. We ran FiberNeat on 131 participants' data from the ADNI3 dataset. We show that applying FiberNeat as a filtering step after bundle segmentation improves the quality of extracted tracts and helps improve tractometry.

Fast Streamline Search: An Exact Technique for Diffusion MRI Tractography

In this work, a hierarchical search algorithm is proposed to efficiently compute the distance between similar tractography streamlines. This hierarchical framework offers an upper bound and a lower bound for the point-wise distance between two streamlines, which guarantees the validity of a proximity search. The proposed streamline representation enables the use of space-partitioning search trees to increase the tractography clustering speed without reducing its accuracy. The resulting approach enables a fast reconstruction a sparse distance matrix between two sets of streamlines, for all similar streamlines within a given radius. Alongside a white matter atlas, this fast streamline search can be used for accurate and reproducible tractogram clustering.

Auto-encoded Latent Representations of White Matter Streamlines for Quantitative Distance Analysis

Parcellation of whole brain tractograms is a critical step to study brain white matter structures and connectivity patterns. The existing methods based on supervised classification of streamlines into predefined streamline bundle types are not designed to explore sub-bundle structures, and methods with manually designed features are expensive to compute streamline-wise similarities. To resolve these issues, we propose a novel atlas-free method that learns a latent space using a deep recurrent auto-encoder trained in an unsupervised manner. The method efficiently embeds any length of streamlines to fixed-size feature vectors, named streamline embedding, for tractogram parcellation using non-parametric clustering in the latent space. The method was evaluated on the ISMRM 2015 tractography challenge dataset with discrimination of major bundles using clustering algorithms and streamline querying based on similarity, as well as real tractograms of 102 subjects Human Connectome Project. The learnt latent streamline and bundle representations open the possibility of quantitative studies of arbitrary granularity of sub-bundle structures using generic data mining techniques.