Deep learning of brain magnetic resonance images: A brief review

A radiomics nomogram based on multiparametric MRI for diagnosing focal cortical dysplasia and initially identifying laterality

BACKGROUND

Focal cortical dysplasia (FCD) is the most common epileptogenic developmental malformation. The diagnosis of FCD is challenging. We generated a radiomics nomogram based on multiparametric magnetic resonance imaging (MRI) to diagnose FCD and identify laterality early.

METHODS

Forty-three patients treated between July 2017 and May 2022 with histopathologically confirmed FCD were retrospectively enrolled. The contralateral unaffected hemispheres were included as the control group. Therefore, 86 ROIs were finally included. Using January 2021 as the time cutoff, those admitted after January 2021 were included in the hold-out set (n = 20). The remaining patients were separated randomly (8:2 ratio) into training (n = 55) and validation (n = 11) sets. All preoperative and postoperative MR images, including T1-weighted (T1w), T2-weighted (T2w), fluid-attenuated inversion recovery (FLAIR), and combined (T1w + T2w + FLAIR) images, were included. The least absolute shrinkage and selection operator (LASSO) was used to select features. Multivariable logistic regression analysis was used to develop the diagnosis model. The performance of the radiomic nomogram was evaluated with an area under the curve (AUC), net reclassification improvement (NRI), integrated discrimination improvement (IDI), calibration and clinical utility.

RESULTS

The model-based radiomics features that were selected from combined sequences (T1w + T2w + FLAIR) had the highest performances in all models and showed better diagnostic performance than inexperienced radiologists in the training (AUCs: 0.847 VS. 0.664, p = 0.008), validation (AUC: 0.857 VS. 0.521, p = 0.155), and hold-out sets (AUCs: 0.828 VS. 0.571, p = 0.080). The positive values of NRI (0.402, 0.607, 0.424) and IDI (0.158, 0.264, 0.264) in the three sets indicated that the diagnostic performance of Model-Combined improved significantly. The radiomics nomogram fit well in calibration curves (p > 0.05), and decision curve analysis further confirmed the clinical usefulness of the nomogram. Additionally, the contrast (the radiomics feature) of the FCD lesions not only played a crucial role in the classifier but also had a significant correlation (r = -0.319, p < 0.05) with the duration of FCD.

CONCLUSION

The radiomics nomogram generated by logistic regression model-based multiparametric MRI represents an important advancement in FCD diagnosis and treatment.

fMRI-based spatio-temporal parcellations of the human brain

PURPOSE OF REVIEW

Human brain parcellation based on functional magnetic resonance imaging (fMRI) plays an essential role in neuroscience research. By segmenting vast and intricate fMRI data into functionally similar units, researchers can better decipher the brain's structure in both healthy and diseased states. This article reviews current methodologies and ideas in this field, while also outlining the obstacles and directions for future research.

RECENT FINDINGS

Traditional brain parcellation techniques, which often rely on cytoarchitectonic criteria, overlook the functional and temporal information accessible through fMRI. The adoption of machine learning techniques, notably deep learning, offers the potential to harness both spatial and temporal information for more nuanced brain segmentation. However, the search for a one-size-fits-all solution to brain segmentation is impractical, with the choice between group-level or individual-level models and the intended downstream analysis influencing the optimal parcellation strategy. Additionally, evaluating these models is complicated by our incomplete understanding of brain function and the absence of a definitive "ground truth".

SUMMARY

While recent methodological advancements have significantly enhanced our grasp of the brain's spatial and temporal dynamics, challenges persist in advancing fMRI-based spatio-temporal representations. Future efforts will likely focus on refining model evaluation and selection as well as developing methods that offer clear interpretability for clinical usage, thereby facilitating further breakthroughs in our comprehension of the brain.

Deep learning for accelerated and robust MRI reconstruction

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.

Brain age has limited utility as a biomarker for capturing fluid cognition in older individuals

One well-known biomarker candidate that supposedly helps capture fluid cognition is Brain Age, or a predicted value based on machine-learning models built to predict chronological age from brain MRI. To formally evaluate the utility of Brain Age for capturing fluid cognition, we built 26 age-prediction models for Brain Age based on different combinations of MRI modalities, using the Human Connectome Project in Aging (n=504, 36-100 years old). First, based on commonality analyses, we found a large overlap between Brain Age and chronological age: Brain Age could uniquely add only around 1.6% in explaining variation in fluid cognition over and above chronological age. Second, the age-prediction models that performed better at predicting chronological age did NOT necessarily create better Brain Age for capturing fluid cognition over and above chronological age. Instead, better-performing age-prediction models created Brain Age that overlapped larger with chronological age, up to around 29% out of 32%, in explaining fluid cognition. Third, Brain Age missed around 11% of the total variation in fluid cognition that could have been explained by the brain variation. That is, directly predicting fluid cognition from brain MRI data (instead of relying on Brain Age and chronological age) could lead to around a 1/3-time improvement of the total variation explained. Accordingly, we demonstrated the limited utility of Brain Age as a biomarker for fluid cognition and made some suggestions to ensure the utility of Brain Age in explaining fluid cognition and other phenotypes of interest.

Assistive tools for classifying neurological disorders using fMRI and deep learning: A guide and example

BACKGROUND

Deep-learning (DL) methods are rapidly changing the way researchers classify neurological disorders. For example, combining functional magnetic resonance imaging (fMRI) and DL has helped researchers identify functional biomarkers of neurological disorders (e.g., brain activation and connectivity) and pilot innovative diagnostic models. However, the knowledge required to perform DL analyses is often domain-specific and is not widely taught in the brain sciences (e.g., psychology, neuroscience, and cognitive science). Conversely, neurological diagnoses and neuroimaging training (e.g., fMRI) are largely restricted to the brain and medical sciences. In turn, these disciplinary knowledge barriers and distinct specializations can act as hurdles that prevent the combination of fMRI and DL pipelines. The complexity of fMRI and DL methods also hinders their clinical adoption and generalization to real-world diagnoses. For example, most current models are not designed for clinical settings or use by nonspecialized populations such as students, clinicians, and healthcare workers. Accordingly, there is a growing area of assistive tools (e.g., software and programming packages) that aim to streamline and increase the accessibility of fMRI and DL pipelines for the diagnoses of neurological disorders.

OBJECTIVES AND METHODS

In this study, we present an introductory guide to some popular DL and fMRI assistive tools. We also create an example autism spectrum disorder (ASD) classification model using assistive tools (e.g., Optuna, GIFT, and the ABIDE preprocessed repository), fMRI, and a convolutional neural network.

RESULTS

In turn, we provide researchers with a guide to assistive tools and give an example of a streamlined fMRI and DL pipeline.

CONCLUSIONS

We are confident that this study can help more researchers enter the field and create accessible fMRI and deep-learning diagnostic models for neurological disorders.

Brain tumor segmentation using neuro-technology enabled intelligence-cascaded U-Net model

According to experts in neurology, brain tumours pose a serious risk to human health. The clinical identification and treatment of brain tumours rely heavily on accurate segmentation. The varied sizes, forms, and locations of brain tumours make accurate automated segmentation a formidable obstacle in the field of neuroscience. U-Net, with its computational intelligence and concise design, has lately been the go-to model for fixing medical picture segmentation issues. Problems with restricted local receptive fields, lost spatial information, and inadequate contextual information are still plaguing artificial intelligence. A convolutional neural network (CNN) and a Mel-spectrogram are the basis of this cough recognition technique. First, we combine the voice in a variety of intricate settings and improve the audio data. After that, we preprocess the data to make sure its length is consistent and create a Mel-spectrogram out of it. A novel model for brain tumor segmentation (BTS), Intelligence Cascade U-Net (ICU-Net), is proposed to address these issues. It is built on dynamic convolution and uses a non-local attention mechanism. In order to reconstruct more detailed spatial information on brain tumours, the principal design is a two-stage cascade of 3DU-Net. The paper's objective is to identify the best learnable parameters that will maximize the likelihood of the data. After the network's ability to gather long-distance dependencies for AI, Expectation-Maximization is applied to the cascade network's lateral connections, enabling it to leverage contextual data more effectively. Lastly, to enhance the network's ability to capture local characteristics, dynamic convolutions with local adaptive capabilities are used in place of the cascade network's standard convolutions. We compared our results to those of other typical methods and ran extensive testing utilising the publicly available BraTS 2019/2020 datasets. The suggested method performs well on tasks involving BTS, according to the experimental data. The Dice scores for tumor core (TC), complete tumor, and enhanced tumor segmentation BraTS 2019/2020 validation sets are 0.897/0.903, 0.826/0.828, and 0.781/0.786, respectively, indicating high performance in BTS.

Joint learning framework of cross-modal synthesis and diagnosis for Alzheimer's disease by mining underlying shared modality information

Alzheimer's disease (AD) is one of the most common neurodegenerative disorders presenting irreversible progression of cognitive impairment. How to identify AD as early as possible is critical for intervention with potential preventive measures. Among various neuroimaging modalities used to diagnose AD, functional positron emission tomography (PET) has higher sensitivity than structural magnetic resonance imaging (MRI), but it is also costlier and often not available in many hospitals. How to leverage massive unpaired unlabeled PET to improve the diagnosis performance of AD from MRI becomes rather important. To address this challenge, this paper proposes a novel joint learning framework of unsupervised cross-modal synthesis and AD diagnosis by mining underlying shared modality information, improving the AD diagnosis from MRI while synthesizing more discriminative PET images. We mine underlying shared modality information in two aspects: diversifying modality information through the cross-modal synthesis network and locating critical diagnosis-related patterns through the AD diagnosis network. First, to diversify the modality information, we propose a novel unsupervised cross-modal synthesis network, which implements the inter-conversion between 3D PET and MRI in a single model modulated by the AdaIN module. Second, to locate shared critical diagnosis-related patterns, we propose an interpretable diagnosis network based on fully 2D convolutions, which takes either 3D synthesized PET or original MRI as input. Extensive experimental results on the ADNI dataset show that our framework can synthesize more realistic images, outperform the state-of-the-art AD diagnosis methods, and have better generalization on external AIBL and NACC datasets.

Automated High-Order Shimming for Neuroimaging Studies

B0 inhomogeneity presents a significant challenge in MRI and MR spectroscopy, particularly at high-field strengths, leading to image distortion, signal loss, and spectral broadening. Existing high-order shimming methods can alleviate these issues but often require time-consuming and subjective manual selection of regions of interest (ROIs). To address this, we proposed an automated high-order shimming (autoHOS) method, incorporating deep-learning-based brain extraction and image-based high-order shimming. This approach performs automated real-time brain extraction to define the ROI of the field map to be used in the shimming algorithm. The shimming performance of autoHOS was assessed through in vivo echo-planar imaging (EPI) and spectroscopic studies at both 3T and 7T field strengths. AutoHOS outperforms linear shimming and manual high-order shimming, enhancing both the image and spectral quality by reducing the EPI image distortion and narrowing the MRS spectral lineshapes. Therefore, autoHOS demonstrated a significant improvement in correcting B0 inhomogeneity while eliminating the need for additional user interaction.

Neuro-Vulnerability in Energy Metabolism Regulation: A Comprehensive Narrative Review

This comprehensive narrative review explores the concept of neuro-vulnerability in energy metabolism regulation and its implications for metabolic disorders. The review highlights the complex interactions among the neural, hormonal, and metabolic pathways involved in the regulation of energy metabolism. The key topics discussed include the role of organs, hormones, and neural circuits in maintaining metabolic balance. The review investigates the association between neuro-vulnerability and metabolic disorders, such as obesity, insulin resistance, and eating disorders, considering genetic, epigenetic, and environmental factors that influence neuro-vulnerability and subsequent metabolic dysregulation. Neuroendocrine interactions and the neural regulation of food intake and energy expenditure are examined, with a focus on the impact of neuro-vulnerability on appetite dysregulation and altered energy expenditure. The role of neuroinflammation in metabolic health and neuro-vulnerability is discussed, emphasizing the bidirectional relationship between metabolic dysregulation and neuroinflammatory processes. This review also evaluates the use of neuroimaging techniques in studying neuro-vulnerability and their potential applications in clinical settings. Furthermore, the association between neuro-vulnerability and eating disorders, as well as its contribution to obesity, is examined. Potential therapeutic interventions targeting neuro-vulnerability, including pharmacological treatments and lifestyle modifications, are reviewed. In conclusion, understanding the concept of neuro-vulnerability in energy metabolism regulation is crucial for addressing metabolic disorders. This review provides valuable insights into the underlying neurobiological mechanisms and their implications for metabolic health. Targeting neuro-vulnerability holds promise for developing innovative strategies in the prevention and treatment of metabolic disorders, ultimately improving metabolic health outcomes.

In-Domain Transfer Learning Strategy for Tumor Detection on Brain MRI

Transfer learning has gained importance in areas where there is a labeled data shortage. However, it is still controversial as to what extent natural image datasets as pre-training sources contribute scientifically to success in different fields, such as medical imaging. In this study, the effect of transfer learning for medical object detection was quantitatively compared using natural and medical image datasets. Within the scope of this study, transfer learning strategies based on five different weight initialization methods were discussed. A natural image dataset MS COCO and brain tumor dataset BraTS 2020 were used as the transfer learning source, and Gazi Brains 2020 was used for the target. Mask R-CNN was adopted as a deep learning architecture for its capability to effectively handle both object detection and segmentation tasks. The experimental results show that transfer learning from the medical image dataset was found to be 10% more successful and showed 24% better convergence performance than the MS COCO pre-trained model, although it contains fewer data. While the effect of data augmentation on the natural image pre-trained model was 5%, the same domain pre-trained model was measured as 2%. According to the most widely used object detection metric, transfer learning strategies using MS COCO weights and random weights showed the same object detection performance as data augmentation. The performance of the most effective strategies identified in the Mask R-CNN model was also tested with YOLOv8. Results showed that even if the amount of data is less than the natural dataset, in-domain transfer learning is more efficient than cross-domain transfer learning. Moreover, this study demonstrates the first use of the Gazi Brains 2020 dataset, which was generated to address the lack of labeled and qualified brain MRI data in the medical field for in-domain transfer learning. Thus, knowledge transfer was carried out from the deep neural network, which was trained with brain tumor data and tested on a different brain tumor dataset.

Deep-learning segmentation of fascicles from microCT of the human vagus nerve

Introduction

MicroCT of the three-dimensional fascicular organization of the human vagus nerve provides essential data to inform basic anatomy as well as the development and optimization of neuromodulation therapies. To process the images into usable formats for subsequent analysis and computational modeling, the fascicles must be segmented. Prior segmentations were completed manually due to the complex nature of the images, including variable contrast between tissue types and staining artifacts.

Methods

Here, we developed a U-Net convolutional neural network (CNN) to automate segmentation of fascicles in microCT of human vagus nerve.

Results

The U-Net segmentation of ~500 images spanning one cervical vagus nerve was completed in 24 s, versus ~40 h for manual segmentation, i.e., nearly four orders of magnitude faster. The automated segmentations had a Dice coefficient of 0.87, a measure of pixel-wise accuracy, thus suggesting a rapid and accurate segmentation. While Dice coefficients are a commonly used metric to assess segmentation performance, we also adapted a metric to assess fascicle-wise detection accuracy, which showed that our network accurately detects the majority of fascicles, but may under-detect smaller fascicles.

Discussion

This network and the associated performance metrics set a benchmark, using a standard U-Net CNN, for the application of deep-learning algorithms to segment fascicles from microCT images. The process may be further optimized by refining tissue staining methods, modifying network architecture, and expanding the ground-truth training data. The resulting three-dimensional segmentations of the human vagus nerve will provide unprecedented accuracy to define nerve morphology in computational models for the analysis and design of neuromodulation therapies.

A review of the application of three-dimensional convolutional neural networks for the diagnosis of Alzheimer's disease using neuroimaging

Alzheimer's disease (AD) is a degenerative disorder that leads to progressive, irreversible cognitive decline. To obtain an accurate and timely diagnosis and detect AD at an early stage, numerous approaches based on convolutional neural networks (CNNs) using neuroimaging data have been proposed. Because 3D CNNs can extract more spatial discrimination information than 2D CNNs, they have emerged as a promising research direction in the diagnosis of AD. The aim of this article is to present the current state of the art in the diagnosis of AD using 3D CNN models and neuroimaging modalities, focusing on the 3D CNN architectures and classification methods used, and to highlight potential future research topics. To give the reader a better overview of the content mentioned in this review, we briefly introduce the commonly used imaging datasets and the fundamentals of CNN architectures. Then we carefully analyzed the existing studies on AD diagnosis, which are divided into two levels according to their inputs: 3D subject-level CNNs and 3D patch-level CNNs, highlighting their contributions and significance in the field. In addition, this review discusses the key findings and challenges from the studies and highlights the lessons learned as a roadmap for future research. Finally, we summarize the paper by presenting some major findings, identifying open research challenges, and pointing out future research directions.

A Transformer-based Multi-features Fusion Model for Prediction of Conversion in Mild Cognitive Impairment

Mild cognitive impairment (MCI) is usually considered the early stage of Alzheimer's disease (AD). Therefore, the accurate identification of MCI individuals with high risk in converting to AD is essential for the potential prevention and treatment of AD. Recently, the great success of deep learning has sparked interest in applying deep learning to neuroimaging field. However, deep learning techniques are prone to overfitting since available neuroimaging datasets are not sufficiently large. Therefore, we proposed a deep learning model fusing cortical features to address the issue of fusion and classification blocks. To validate the effectiveness of the proposed model, we compared seven different models on the same dataset in the literature. The results show that our proposed model outperformed the competing models in the prediction of MCI conversion with an accuracy of 83.3% in the testing dataset. Subsequently, we used deep learning to characterize the contribution of brain regions and different cortical features to MCI progression. The results revealed that the caudal anterior cingulate and pars orbitalis contributed most to the classification task, and our model pays more attention to volume features and cortical thickness features.

Deep 3D Neural Network for Brain Structures Segmentation Using Self-Attention Modules in MRI Images

In recent years, the use of deep learning-based models for developing advanced healthcare systems has been growing due to the results they can achieve. However, the majority of the proposed deep learning-models largely use convolutional and pooling operations, causing a loss in valuable data and focusing on local information. In this paper, we propose a deep learning-based approach that uses global and local features which are of importance in the medical image segmentation process. In order to train the architecture, we used extracted three-dimensional (3D) blocks from the full magnetic resonance image resolution, which were sent through a set of successive convolutional neural network (CNN) layers free of pooling operations to extract local information. Later, we sent the resulting feature maps to successive layers of self-attention modules to obtain the global context, whose output was later dispatched to the decoder pipeline composed mostly of upsampling layers. The model was trained using the Mindboggle-101 dataset. The experimental results showed that the self-attention modules allow segmentation with a higher Mean Dice Score of 0.90 ± 0.036 compared with other UNet-based approaches. The average segmentation time was approximately 0.038 s per brain structure. The proposed model allows tackling the brain structure segmentation task properly. Exploiting the global context that the self-attention modules incorporate allows for more precise and faster segmentation. We segmented 37 brain structures and, to the best of our knowledge, it is the largest number of structures under a 3D approach using attention mechanisms.