MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans

A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation

Magnetic resonance imaging (MRI) is a ubiquitous medical imaging technology with applications in disease diagnostics, intervention, and treatment planning. Accurate MRI segmentation is critical for diagnosing abnormalities, monitoring diseases, and deciding on a course of treatment. With the advent of advanced deep learning frameworks, fully automated and accurate MRI segmentation is advancing. Traditional supervised deep learning techniques have advanced tremendously, reaching clinical-level accuracy in the field of segmentation. However, these algorithms still require a large amount of annotated data, which is oftentimes unavailable or impractical. One way to circumvent this issue is to utilize algorithms that exploit a limited amount of labeled data. This paper aims to review such state-of-the-art algorithms that use a limited number of annotated samples. We explain the fundamental principles of self-supervised learning, generative models, few-shot learning, and semi-supervised learning and summarize their applications in cardiac, abdomen, and brain MRI segmentation. Throughout this review, we highlight algorithms that can be employed based on the quantity of annotated data available. We also present a comprehensive list of notable publicly available MRI segmentation datasets. To conclude, we discuss possible future directions of the field-including emerging algorithms, such as contrastive language-image pretraining, and potential combinations across the methods discussed-that can further increase the efficacy of image segmentation with limited labels.

Fast MRI Reconstruction Using Deep Learning-based Compressed Sensing: A Systematic Review

Magnetic resonance imaging (MRI) has revolutionized medical imaging, providing a non-invasive and highly detailed look into the human body. However, the long acquisition times of MRI present challenges, causing patient discomfort, motion artifacts, and limiting real-time applications. To address these challenges, researchers are exploring various techniques to reduce acquisition time and improve the overall efficiency of MRI. One such technique is compressed sensing (CS), which reduces data acquisition by leveraging image sparsity in transformed spaces. In recent years, deep learning (DL) has been integrated with CS-MRI, leading to a new framework that has seen remarkable growth. DL-based CS-MRI approaches are proving to be highly effective in accelerating MR imaging without compromising image quality. This review comprehensively examines DL-based CS-MRI techniques, focusing on their role in increasing MR imaging speed. We provide a detailed analysis of each category of DL-based CS-MRI including end-to-end, unroll optimization, self-supervised, and federated learning. Our systematic review highlights significant contributions and underscores the exciting potential of DL in CS-MRI. Additionally, our systematic review efficiently summarizes key results and trends in DL-based CS-MRI including quantitative metrics, the dataset used, acceleration factors, and the progress of and research interest in DL techniques over time. Finally, we discuss potential future directions and the importance of DL-based CS-MRI in the advancement of medical imaging. To facilitate further research in this area, we provide a GitHub repository that includes up-to-date DL-based CS-MRI publications and publicly available datasets - https://github.com/mosaf/Awesome-DL-based-CS-MRI.

Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data

Many clinical and research studies of the human brain require accurate structural MRI segmentation. While traditional atlas-based methods can be applied to volumes from any acquisition site, recent deep learning algorithms ensure high accuracy only when tested on data from the same sites exploited in training (i.e., internal data). Performance degradation experienced on external data (i.e., unseen volumes from unseen sites) is due to the inter-site variability in intensity distributions, and to unique artefacts caused by different MR scanner models and acquisition parameters. To mitigate this site-dependency, often referred to as the scanner effect, we propose LOD-Brain, a 3D convolutional neural network with progressive levels-of-detail (LOD), able to segment brain data from any site. Coarser network levels are responsible for learning a robust anatomical prior helpful in identifying brain structures and their locations, while finer levels refine the model to handle site-specific intensity distributions and anatomical variations. We ensure robustness across sites by training the model on an unprecedentedly rich dataset aggregating data from open repositories: almost 27,000 T1w volumes from around 160 acquisition sites, at 1.5 - 3T, from a population spanning from 8 to 90 years old. Extensive tests demonstrate that LOD-Brain produces state-of-the-art results, with no significant difference in performance between internal and external sites, and robust to challenging anatomical variations. Its portability paves the way for large-scale applications across different healthcare institutions, patient populations, and imaging technology manufacturers. Code, model, and demo are available on the project website.

Deep learning based synthesis of MRI, CT and PET: Review and analysis

Medical image synthesis represents a critical area of research in clinical decision-making, aiming to overcome the challenges associated with acquiring multiple image modalities for an accurate clinical workflow. This approach proves beneficial in estimating an image of a desired modality from a given source modality among the most common medical imaging contrasts, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Positron Emission Tomography (PET). However, translating between two image modalities presents difficulties due to the complex and non-linear domain mappings. Deep learning-based generative modelling has exhibited superior performance in synthetic image contrast applications compared to conventional image synthesis methods. This survey comprehensively reviews deep learning-based medical imaging translation from 2018 to 2023 on pseudo-CT, synthetic MR, and synthetic PET. We provide an overview of synthetic contrasts in medical imaging and the most frequently employed deep learning networks for medical image synthesis. Additionally, we conduct a detailed analysis of each synthesis method, focusing on their diverse model designs based on input domains and network architectures. We also analyse novel network architectures, ranging from conventional CNNs to the recent Transformer and Diffusion models. This analysis includes comparing loss functions, available datasets and anatomical regions, and image quality assessments and performance in other downstream tasks. Finally, we discuss the challenges and identify solutions within the literature, suggesting possible future directions. We hope that the insights offered in this survey paper will serve as a valuable roadmap for researchers in the field of medical image synthesis.

Where is VALDO? VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra- and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3-Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.

Deep learning techniques for isointense infant brain tissue segmentation: a systematic literature review

Introduction

To improve comprehension of initial brain growth in wellness along with sickness, it is essential to precisely segment child brain magnetic resonance imaging (MRI) into white matter (WM) and gray matter (GM), along with cerebrospinal fluid (CSF). Nonetheless, in the isointense phase (6-8 months of age), the inborn myelination and development activities, WM along with GM display alike stages of intensity in both T1-weighted and T2-weighted MRI, making tissue segmentation extremely difficult.

Methods

The comprehensive review of studies related to isointense brain MRI segmentation approaches is highlighted in this publication. The main aim and contribution of this study is to aid researchers by providing a thorough review to make their search for isointense brain MRI segmentation easier. The systematic literature review is performed from four points of reference: (1) review of studies concerning isointense brain MRI segmentation; (2) research contribution and future works and limitations; (3) frequently applied evaluation metrics and datasets; (4) findings of this studies.

Results and discussion

The systemic review is performed on studies that were published in the period of 2012 to 2022. A total of 19 primary studies of isointense brain MRI segmentation were selected to report the research question stated in this review.

Fully automated measurement of intracranial CSF and brain parenchyma volumes in pediatric hydrocephalus by segmentation of clinical MRI studies

BACKGROUND

Brain parenchyma (BP) and intracranial cerebrospinal fluid (iCSF) volumes measured by fully automated segmentation of clinical brain MRI studies may be useful for the diagnosis and follow-up of pediatric hydrocephalus. However, previously published segmentation techniques either rely on dedicated sequences, not routinely used in clinical practice, or on spatial normalization, which has limited accuracy when severe brain distortions, such as in hydrocephalic patients, are present.

PURPOSE

We developed a fully automated method to measure BP and iCSF volumes from clinical brain MRI studies of pediatric hydrocephalus patients, exploiting the complementary information contained in T2- and T1-weighted images commonly used in clinical practice.

METHODS

The proposed procedure, following skull-stripping of the combined volumes, performed using a multiparametric method to obtain a reliable definition of the inner skull profile, maximizes the CSF-to-parenchyma contrast by dividing the T2w- by the T1w- volume after full-scale dynamic rescaling, thus allowing separation of iCSF and BP through a simple thresholding routine.

RESULTS

Validation against manual tracing on 23 studies (four controls and 19 hydrocephalic patients) showed excellent concordance (ICC > 0.98) and spatial overlap (Dice coefficients ranging from 77.2% for iCSF to 96.8% for intracranial volume). Accuracy was comparable to the intra-operator reproducibility of manual segmentation, as measured in 14 studies processed twice by the same experienced neuroradiologist. Results of the application of the algorithm to a dataset of 63 controls and 57 hydrocephalic patients (19 with parenchymal damage), measuring volumes' changes with normal development and in hydrocephalic patients, are also reported for demonstration purposes.

CONCLUSIONS

The proposed approach allows fully automated segmentation of BP and iCSF in clinical studies, also in severely distorted brains, enabling to assess age- and disease-related changes in intracranial tissue volume with an accuracy comparable to expert manual segmentation.

Multi-site, Multi-domain Airway Tree Modeling

Open international challenges are becoming the de facto standard for assessing computer vision and image analysis algorithms. In recent years, new methods have extended the reach of pulmonary airway segmentation that is closer to the limit of image resolution. Since EXACT'09 pulmonary airway segmentation, limited effort has been directed to the quantitative comparison of newly emerged algorithms driven by the maturity of deep learning based approaches and extensive clinical efforts for resolving finer details of distal airways for early intervention of pulmonary diseases. Thus far, public annotated datasets are extremely limited, hindering the development of data-driven methods and detailed performance evaluation of new algorithms. To provide a benchmark for the medical imaging community, we organized the Multi-site, Multi-domain Airway Tree Modeling (ATM'22), which was held as an official challenge event during the MICCAI 2022 conference. ATM'22 provides large-scale CT scans with detailed pulmonary airway annotation, including 500 CT scans (300 for training, 50 for validation, and 150 for testing). The dataset was collected from different sites and it further included a portion of noisy COVID-19 CTs with ground-glass opacity and consolidation. Twenty-three teams participated in the entire phase of the challenge and the algorithms for the top ten teams are reviewed in this paper. Both quantitative and qualitative results revealed that deep learning models embedded with the topological continuity enhancement achieved superior performance in general. ATM'22 challenge holds as an open-call design, the training data and the gold standard evaluation are available upon successful registration via its homepage (https://atm22.grand-challenge.org/).

Recent progress in transformer-based medical image analysis

The transformer is primarily used in the field of natural language processing. Recently, it has been adopted and shows promise in the computer vision (CV) field. Medical image analysis (MIA), as a critical branch of CV, also greatly benefits from this state-of-the-art technique. In this review, we first recap the core component of the transformer, the attention mechanism, and the detailed structures of the transformer. After that, we depict the recent progress of the transformer in the field of MIA. We organize the applications in a sequence of different tasks, including classification, segmentation, captioning, registration, detection, enhancement, localization, and synthesis. The mainstream classification and segmentation tasks are further divided into eleven medical image modalities. A large number of experiments studied in this review illustrate that the transformer-based method outperforms existing methods through comparisons with multiple evaluation metrics. Finally, we discuss the open challenges and future opportunities in this field. This task-modality review with the latest contents, detailed information, and comprehensive comparison may greatly benefit the broad MIA community.

Artificial Intelligence in Neuroradiology: A Review of Current Topics and Competition Challenges

There is an expanding body of literature that describes the application of deep learning and other machine learning and artificial intelligence methods with potential relevance to neuroradiology practice. In this article, we performed a literature review to identify recent developments on the topics of artificial intelligence in neuroradiology, with particular emphasis on large datasets and large-scale algorithm assessments, such as those used in imaging AI competition challenges. Numerous applications relevant to ischemic stroke, intracranial hemorrhage, brain tumors, demyelinating disease, and neurodegenerative/neurocognitive disorders were discussed. The potential applications of these methods to spinal fractures, scoliosis grading, head and neck oncology, and vascular imaging were also reviewed. The AI applications examined perform a variety of tasks, including localization, segmentation, longitudinal monitoring, diagnostic classification, and prognostication. While research on this topic is ongoing, several applications have been cleared for clinical use and have the potential to augment the accuracy or efficiency of neuroradiologists.

Intracerebral hemorrhage CT scan image segmentation with HarDNet based transformer

Although previous studies conducted on the segmentation of hemorrhage images were based on the U-Net model, which comprises an encoder-decoder architecture, these models exhibit low parameter passing efficiency between the encoder and decoder, large model size, and slow speed. Therefore, to overcome these drawbacks, this study proposes TransHarDNet, an image segmentation model for the diagnosis of intracerebral hemorrhage in CT scan images of the brain. In this model, the HarDNet block is applied to the U-Net architecture, and the encoder and decoder are connected using a transformer block. As a result, the network complexity was reduced and the inference speed improved while maintaining the high performance compared to conventional models. Furthermore, the superiority of the proposed model was verified by using 82,636 CT scan images showing five different types of hemorrhages to train and test the model. Experimental results showed that the proposed model exhibited a Dice coefficient and IoU of 0.712 and 0.597, respectively, in a test set comprising 1200 images of hemorrhage, indicating better performance compared to typical segmentation models such as U-Net, U-Net++, SegNet, PSPNet, and HarDNet. Moreover, the inference time was 30.78 frames per second (FPS), which was faster than all en-coder-decoder-based models except HarDNet.

Calibrating segmentation networks with margin-based label smoothing

Despite the undeniable progress in visual recognition tasks fueled by deep neural networks, there exists recent evidence showing that these models are poorly calibrated, resulting in over-confident predictions. The standard practices of minimizing the cross-entropy loss during training promote the predicted softmax probabilities to match the one-hot label assignments. Nevertheless, this yields a pre-softmax activation of the correct class that is significantly larger than the remaining activations, which exacerbates the miscalibration problem. Recent observations from the classification literature suggest that loss functions that embed implicit or explicit maximization of the entropy of predictions yield state-of-the-art calibration performances. Despite these findings, the impact of these losses in the relevant task of calibrating medical image segmentation networks remains unexplored. In this work, we provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses. Specifically, these losses could be viewed as approximations of a linear penalty (or a Lagrangian term) imposing equality constraints on logit distances. This points to an important limitation of such underlying equality constraints, whose ensuing gradients constantly push towards a non-informative solution, which might prevent from reaching the best compromise between the discriminative performance and calibration of the model during gradient-based optimization. Following our observations, we propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances. Comprehensive experiments on a variety of public medical image segmentation benchmarks demonstrate that our method sets novel state-of-the-art results on these tasks in terms of network calibration, whereas the discriminative performance is also improved. The code is available at https://github.com/Bala93/MarginLoss.

Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

Transformer, one of the latest technological advances of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that characterize the Transformers, we offer a comprehensive review of the state-of-the-art Transformer-based approaches for medical imaging and exhibit current research progresses made in the areas of medical image segmentation, recognition, detection, registration, reconstruction, enhancement, etc. In particular, what distinguishes our review lies in its organization based on the Transformer's key defining properties, which are mostly derived from comparing the Transformer and CNN, and its type of architecture, which specifies the manner in which the Transformer and CNN are combined, all helping the readers to best understand the rationale behind the reviewed approaches. We conclude with discussions of future perspectives.

ConvNets for automatic detection of polyglutamine SCAs from brain MRIs: state of the art applications

Polyglutamine spinocerebellar ataxias (polyQ SCAs) are a group of neurodegenerative diseases, clinically and genetically heterogeneous, characterized by loss of balance and motor coordination due to dysfunction of the cerebellum and its connections. The diagnosis of each type of polyQ SCA, alongside with genetic tests, includes medical images analysis, and its automation may help specialists to distinguish between each type. Convolutional neural networks (ConvNets or CNNs) have been recently used for medical image processing, with outstanding results. In this work, we present the main clinical and imaging features of polyglutamine SCAs, and the basics of CNNs. Finally, we review studies that have used this approach to automatically process brain medical images and may be applied to SCAs detection. We conclude by discussing the possible limitations and opportunities of using ConvNets for SCAs diagnose in the future.

Methods of Brain Extraction from Magnetic Resonance Images of Human Head: A Review

Medical images are providing vital information to aid physicians in diagnosing a disease afflicting the organ of a human body. Magnetic resonance imaging is an important imaging modality in capturing the soft tissues of the brain. Segmenting and extracting the brain is essential in studying the structure and pathological condition of brain. There are several methods that are developed for this purpose. Researchers in brain extraction or segmentation need to know the current status of the work that have been done. Such an information is also important for improving the existing method to get more accurate results or to reduce the complexity of the algorithm. In this paper we review the classical methods and convolutional neural network-based deep learning brain extraction methods.

Review of Semantic Segmentation of Medical Images Using Modified Architectures of UNET

In biomedical image analysis, information about the location and appearance of tumors and lesions is indispensable to aid doctors in treating and identifying the severity of diseases. Therefore, it is essential to segment the tumors and lesions. MRI, CT, PET, ultrasound, and X-ray are the different imaging systems to obtain this information. The well-known semantic segmentation technique is used in medical image analysis to identify and label regions of images. The semantic segmentation aims to divide the images into regions with comparable characteristics, including intensity, homogeneity, and texture. UNET is the deep learning network that segments the critical features. However, UNETs basic architecture cannot accurately segment complex MRI images. This review introduces the modified and improved models of UNET suitable for increasing segmentation accuracy.

A Fuzzy Consensus Clustering Algorithm for MRI Brain Tissue Segmentation

Brain tissue segmentation is an important component of the clinical diagnosis of brain diseases using multi-modal magnetic resonance imaging (MR). Brain tissue segmentation has been developed by many unsupervised methods in the literature. The most commonly used unsupervised methods are K-Means, Expectation-Maximization, and Fuzzy Clustering. Fuzzy clustering methods offer considerable benefits compared with the aforementioned methods as they are capable of handling brain images that are complex, largely uncertain, and imprecise. However, this approach suffers from the intrinsic noise and intensity inhomogeneity (IIH) in the data resulting from the acquisition process. To resolve these issues, we propose a fuzzy consensus clustering algorithm that defines a membership function resulting from a voting schema to cluster the pixels. In particular, we first pre-process the MRI data and employ several segmentation techniques based on traditional fuzzy sets and intuitionistic sets. Then, we adopted a voting schema to fuse the results of the applied clustering methods. Finally, to evaluate the proposed method, we used the well-known performance measures (boundary measure, overlap measure, and volume measure) on two publicly available datasets (OASIS and IBSR18). The experimental results show the superior performance of the proposed method in comparison with the recent state of the art. The performance of the proposed method is also presented using a real-world Autism Spectrum Disorder Detection problem with better accuracy compared to other existing methods.

Insights from auditory cortex for GABA+ magnetic resonance spectroscopy studies of aging

Changes in levels of the inhibitory neurotransmitter γ-aminobutyric acid (GABA) may underlie aging-related changes in brain function. GABA and co-edited macromolecules (GABA+) can be measured with MEGA-PRESS magnetic resonance spectroscopy (MRS). The current study investigated how changes in the aging brain impact the interpretation of GABA+ measures in bilateral auditory cortices of healthy young and older adults. Structural changes during aging appeared as decreasing proportion of grey matter in the MRS volume of interest and corresponding increase in cerebrospinal fluid. GABA+ referenced to H₂ O without tissue correction declined in aging. This decline persisted after correcting for tissue differences in MR-visible H₂ O and relaxation times but vanished after considering the different abundance of GABA+ in grey and white matter. However, GABA+ referenced to creatine and N-acetyl aspartate (NAA), which showed no dependence on tissue composition, decreased in aging. All GABA+ measures showed hemispheric asymmetry in young but not older adults. The study also considered aging-related effects on tissue segmentation and the impact of co-edited macromolecules. Tissue segmentation differed significantly between commonly used algorithms, but aging-related effects on tissue-corrected GABA+ were consistent across methods. Auditory cortex macromolecule concentration did not change with age, indicating that a decline in GABA caused the decrease in the compound GABA+ measure. Most likely, the macromolecule contribution to GABA+ leads to underestimating an aging-related decrease in GABA. Overall, considering multiple GABA+ measures using different reference signals strengthened the support for an aging-related decline in auditory cortex GABA levels.

MedicalSeg: A Medical GUI Application for Image Segmentation Management

In the field of medical imaging, the division of an image into meaningful structures using image segmentation is an essential step for pre-processing analysis. Many studies have been carried out to solve the general problem of the evaluation of image segmentation results. One of the main focuses in the computer vision field is based on artificial intelligence algorithms for segmentation and classification, including machine learning and deep learning approaches. The main drawback of supervised segmentation approaches is that a large dataset of ground truth validated by medical experts is required. In this sense, many research groups have developed their segmentation approaches according to their specific needs. However, a generalised application aimed at visualizing, assessing and comparing the results of different methods facilitating the generation of a ground-truth repository is not found in recent literature. In this paper, a new graphical user interface application (MedicalSeg) for the management of medical imaging based on pre-processing and segmentation is presented. The objective is twofold, first to create a test platform for comparing segmentation approaches, and secondly to generate segmented images to create ground truths that can then be used for future purposes as artificial intelligence tools. An experimental demonstration and performance analysis discussion are presented in this paper.

Validation of an automatic tool for the rapid measurement of brain atrophy and white matter hyperintensity: QyScore®

OBJECTIVES

QyScore® is an imaging analysis tool certified in Europe (CE marked) and the US (FDA cleared) for the automatic volumetry of grey and white matter (GM and WM respectively), hippocampus (HP), amygdala (AM), and white matter hyperintensity (WMH). Here we compare QyScore® performances with the consensus of expert neuroradiologists.

METHODS

Dice similarity coefficient (DSC) and the relative volume difference (RVD) for GM, WM volumes were calculated on 50 3DT1 images. DSC and the F1 metrics were calculated for WMH on 130 3DT1 and FLAIR images. For each index, we identified thresholds of reliability based on current literature review results. We hypothesized that DSC/F1 scores obtained using QyScore® markers would be higher than the threshold. In contrast, RVD scores would be lower. Regression analysis and Bland-Altman plots were obtained to evaluate QyScore® performance in comparison to the consensus of three expert neuroradiologists.

RESULTS

The lower bound of the DSC/F1 confidence intervals was higher than the threshold for the GM, WM, HP, AM, and WMH, and the higher bounds of the RVD confidence interval were below the threshold for the WM, GM, HP, and AM. QyScore®, compared with the consensus of three expert neuroradiologists, provides reliable performance for the automatic segmentation of the GM and WM volumes, and HP and AM volumes, as well as WMH volumes.

CONCLUSIONS

QyScore® represents a reliable medical device in comparison with the consensus of expert neuroradiologists. Therefore, QyScore® could be implemented in clinical trials and clinical routine to support the diagnosis and longitudinal monitoring of neurological diseases.

KEY POINTS

• QyScore® provides reliable automatic segmentation of brain structures in comparison with the consensus of three expert neuroradiologists. • QyScore® automatic segmentation could be performed on MRI images using different vendors and protocols of acquisition. In addition, the fast segmentation process saves time over manual and semi-automatic methods. • QyScore® could be implemented in clinical trials and clinical routine to support the diagnosis and longitudinal monitoring of neurological diseases.

Dense Dilated Deep Multiscale Supervised U-Network for biomedical image segmentation

Biomedical image segmentation is essential for computerized medical image analysis. Deep learning algorithms allow us to design state-of-the-art models for solving segmentation problems. The U-Net and its variants have provided positive results across various datasets. However, the existing networks have the same receptive field at each level and the models are supervised only at the shallow level. Considering these two ideas, we have proposed the D3MSU-Net where the field of view in each level is varied depending upon the depth of the resolution layer and the model is supervised at each resolution level. We have evaluated our network in eight benchmark datasets such as Electron Microscopy, Lung segmentation, Montgomery Chest X-ray, Covid-Radiopaedia, Wound, Medetec, Brain MRI, and Covid-19 lung CT dataset. Additionally, we have provided the performance for various ablations. The experimental results show the superiority of the proposed network. The proposed D3MSU-Net and ablation models are available at www.github.com/shirshabose/D3MSUNET.

Ten years of image analysis and machine learning competitions in dementia

Machine learning methods exploiting multi-parametric biomarkers, especially based on neuroimaging, have huge potential to improve early diagnosis of dementia and to predict which individuals are at-risk of developing dementia. To benchmark algorithms in the field of machine learning and neuroimaging in dementia and assess their potential for use in clinical practice and clinical trials, seven grand challenges have been organized in the last decade: MIRIAD (2012), Alzheimer's Disease Big Data DREAM (2014), CADDementia (2014), Machine Learning Challenge (2014), MCI Neuroimaging (2017), TADPOLE (2017), and the Predictive Analytics Competition (2019). Based on two challenge evaluation frameworks, we analyzed how these grand challenges are complementing each other regarding research questions, datasets, validation approaches, results and impact. The seven grand challenges addressed questions related to screening, clinical status estimation, prediction and monitoring in (pre-clinical) dementia. There was little overlap in clinical questions, tasks and performance metrics. Whereas this aids providing insight on a broad range of questions, it also limits the validation of results across challenges. The validation process itself was mostly comparable between challenges, using similar methods for ensuring objective comparison, uncertainty estimation and statistical testing. In general, winning algorithms performed rigorous data pre-processing and combined a wide range of input features. Despite high state-of-the-art performances, most of the methods evaluated by the challenges are not clinically used. To increase impact, future challenges could pay more attention to statistical analysis of which factors (i.e., features, models) relate to higher performance, to clinical questions beyond Alzheimer's disease, and to using testing data beyond the Alzheimer's Disease Neuroimaging Initiative. Grand challenges would be an ideal venue for assessing the generalizability of algorithm performance to unseen data of other cohorts. Key for increasing impact in this way are larger testing data sizes, which could be reached by sharing algorithms rather than data to exploit data that cannot be shared. Given the potential and lessons learned in the past ten years, we are excited by the prospects of grand challenges in machine learning and neuroimaging for the next ten years and beyond.

Cardiac MR: From Theory to Practice

Cardiovascular disease (CVD) is the leading single cause of morbidity and mortality, causing over 17. 9 million deaths worldwide per year with associated costs of over $800 billion. Improving prevention, diagnosis, and treatment of CVD is therefore a global priority. Cardiovascular magnetic resonance (CMR) has emerged as a clinically important technique for the assessment of cardiovascular anatomy, function, perfusion, and viability. However, diversity and complexity of imaging, reconstruction and analysis methods pose some limitations to the widespread use of CMR. Especially in view of recent developments in the field of machine learning that provide novel solutions to address existing problems, it is necessary to bridge the gap between the clinical and scientific communities. This review covers five essential aspects of CMR to provide a comprehensive overview ranging from CVDs to CMR pulse sequence design, acquisition protocols, motion handling, image reconstruction and quantitative analysis of the obtained data. (1) The basic MR physics of CMR is introduced. Basic pulse sequence building blocks that are commonly used in CMR imaging are presented. Sequences containing these building blocks are formed for parametric mapping and functional imaging techniques. Commonly perceived artifacts and potential countermeasures are discussed for these methods. (2) CMR methods for identifying CVDs are illustrated. Basic anatomy and functional processes are described to understand the cardiac pathologies and how they can be captured by CMR imaging. (3) The planning and conduct of a complete CMR exam which is targeted for the respective pathology is shown. Building blocks are illustrated to create an efficient and patient-centered workflow. Further strategies to cope with challenging patients are discussed. (4) Imaging acceleration and reconstruction techniques are presented that enable acquisition of spatial, temporal, and parametric dynamics of the cardiac cycle. The handling of respiratory and cardiac motion strategies as well as their integration into the reconstruction processes is showcased. (5) Recent advances on deep learning-based reconstructions for this purpose are summarized. Furthermore, an overview of novel deep learning image segmentation and analysis methods is provided with a focus on automatic, fast and reliable extraction of biomarkers and parameters of clinical relevance.

Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information

In clinical medicine, magnetic resonance imaging (MRI) is one of the most important tools for diagnosis, triage, prognosis, and treatment planning. However, MRI suffers from an inherent slow data acquisition process because data is collected sequentially in k-space. In recent years, most MRI reconstruction methods proposed in the literature focus on holistic image reconstruction rather than enhancing the edge information. This work steps aside this general trend by elaborating on the enhancement of edge information. Specifically, we introduce a novel parallel imaging coupled dual discriminator generative adversarial network (PIDD-GAN) for fast multi-channel MRI reconstruction by incorporating multi-view information. The dual discriminator design aims to improve the edge information in MRI reconstruction. One discriminator is used for holistic image reconstruction, whereas the other one is responsible for enhancing edge information. An improved U-Net with local and global residual learning is proposed for the generator. Frequency channel attention blocks (FCA Blocks) are embedded in the generator for incorporating attention mechanisms. Content loss is introduced to train the generator for better reconstruction quality. We performed comprehensive experiments on Calgary-Campinas public brain MR dataset and compared our method with state-of-the-art MRI reconstruction methods. Ablation studies of residual learning were conducted on the MICCAI13 dataset to validate the proposed modules. Results show that our PIDD-GAN provides high-quality reconstructed MR images, with well-preserved edge information. The time of single-image reconstruction is below 5ms, which meets the demand of faster processing.

Comparison of Two-Dimensional- and Three-Dimensional-Based U-Net Architectures for Brain Tissue Classification in One-Dimensional Brain CT

Brain tissue segmentation plays a crucial role in feature extraction, volumetric quantification, and morphometric analysis of brain scans. For the assessment of brain structure and integrity, CT is a non-invasive, cheaper, faster, and more widely available modality than MRI. However, the clinical application of CT is mostly limited to the visual assessment of brain integrity and exclusion of copathologies. We have previously developed two-dimensional (2D) deep learning-based segmentation networks that successfully classified brain tissue in head CT. Recently, deep learning-based MRI segmentation models successfully use patch-based three-dimensional (3D) segmentation networks. In this study, we aimed to develop patch-based 3D segmentation networks for CT brain tissue classification. Furthermore, we aimed to compare the performance of 2D- and 3D-based segmentation networks to perform brain tissue classification in anisotropic CT scans. For this purpose, we developed 2D and 3D U-Net-based deep learning models that were trained and validated on MR-derived segmentations from scans of 744 participants of the Gothenburg H70 Cohort with both CT and T1-weighted MRI scans acquired timely close to each other. Segmentation performance of both 2D and 3D models was evaluated on 234 unseen datasets using measures of distance, spatial similarity, and tissue volume. Single-task slice-wise processed 2D U-Nets performed better than multitask patch-based 3D U-Nets in CT brain tissue classification. These findings provide support to the use of 2D U-Nets to segment brain tissue in one-dimensional (1D) CT. This could increase the application of CT to detect brain abnormalities in clinical settings.

Association Between Cerebral Cortical Microinfarcts and Perilesional Cortical Atrophy on 3T MRI

BACKGROUND AND OBJECTIVES

Cerebral cortical microinfarcts (CMIs) are a novel MRI-marker of cerebrovascular disease (CeVD) that predicts accelerated cognitive decline. Presence of CMIs is known to be associated with global cortical atrophy, although the mechanism linking the two is unclear. Our primary objective was to examine the relation between CMIs and cortical atrophy and establish possible perilesional atrophy surrounding CMIs. Our secondary objective was to examine the role of cortical atrophy in CMI-associated cognitive impairment.

METHODS

Patients were recruited from two Singapore memory clinics between December 2010 and September 2013 and included if they received the diagnosis no objective cognitive impairment, cognitive impairment (with or without a history of stroke) or Alzheimer's or vascular dementia. Cortical thickness, chronic cortical microinfarcts and MRI-markers of CeVD were assessed on 3T MRI. Patients underwent cognitive testing. Cortical thickness was compared globally between patients with and without CMIs, regionally within individual patients with CMIs comparing brain regions with CMIs to the corresponding contralateral region without CMIs and locally within individuals patients in a 50 mm radius of CMIs. Global cortical thickness was analyzed as mediator in the relation between CMI and cognitive performance.

RESULTS

Of the 238 patients (mean age 72.5 SD 9.1 years) enrolled, 75 had ≥1 CMIs. Patient with CMIs had a 2.1% lower global cortical thickness (B=-.049 mm, 95% CI [.091; -.007] p=.022) compared to patients without CMIs, after correction for age, sex, education and intracranial volume. In patients with CMIs, cortical thickness in brain regions with CMIs was 2.2 % lower than in contralateral regions without CMIs (B=-.048 mm [-.071; -.026] p<.001). In a 20 mm radius area surrounding the CMI-core, cortical thickness was lower than in the area 20-50 mm from the CMI-core (Mean difference -.06 mm 95% CI [-.10; -.02] p=.002). Global cortical thickness was a significant mediator in the relationship between CMI presence and cognitive performance as measure with the Mini-Mental State Examination (B=-.12 [-.22; -.01] p=.025).

DISCUSSION

We found cortical atrophy surrounding CMIs, suggesting a perilesional effect in a cortical area many times larger than the CMI-core. Our findings support the notion that CMIs affect brain structure beyond the actual lesion site.

Deep learning from MRI-derived labels enables automatic brain tissue classification on human brain CT

Automatic methods for feature extraction, volumetry, and morphometric analysis in clinical neuroscience typically operate on images obtained with magnetic resonance (MR) imaging equipment. Although CT scans are less expensive to acquire and more widely available than MR scans, their application is currently limited to the visual assessment of brain integrity and the exclusion of co-pathologies. CT has rarely been used for tissue classification because the contrast between grey matter and white matter was considered insufficient. In this study, we propose an automatic method for segmenting grey matter (GM), white matter (WM), cerebrospinal fluid (CSF), and intracranial volume (ICV) from head CT images. A U-Net deep learning model was trained and validated on CT images with MRI-derived segmentation labels. We used data from 744 participants of the Gothenburg H70 Birth Cohort Studies for whom CT and T1-weighted MR images had been acquired on the same day. Our proposed model predicted brain tissue classes accurately from unseen CT images (Dice coefficients of 0.79, 0.82, 0.75, 0.93 and 0.98 for GM, WM, CSF, brain volume and ICV, respectively). To contextualize these results, we generated benchmarks based on established MR-based methods and intentional image degradation. Our findings demonstrate that CT-derived segmentations can be used to delineate and quantify brain tissues, opening new possibilities for the use of CT in clinical practice and research.

CEREBRUM-7T: Fast and Fully Volumetric Brain Segmentation of 7 Tesla MR Volumes

Ultra-high-field magnetic resonance imaging (MRI) enables sub-millimetre resolution imaging of the human brain, allowing the study of functional circuits of cortical layers at the meso-scale. An essential step in many functional and structural neuroimaging studies is segmentation, the operation of partitioning the MR images in anatomical structures. Despite recent efforts in brain imaging analysis, the literature lacks in accurate and fast methods for segmenting 7-tesla (7T) brain MRI. We here present CEREBRUM-7T, an optimised end-to-end convolutional neural network, which allows fully automatic segmentation of a whole 7T T1_w MRI brain volume at once, without partitioning the volume, pre-processing, nor aligning it to an atlas. The trained model is able to produce accurate multi-structure segmentation masks on six different classes plus background in only a few seconds. The experimental part, a combination of objective numerical evaluations and subjective analysis, confirms that the proposed solution outperforms the training labels it was trained on and is suitable for neuroimaging studies, such as layer functional MRI studies. Taking advantage of a fine-tuning operation on a reduced set of volumes, we also show how it is possible to effectively apply CEREBRUM-7T to different sites data. Furthermore, we release the code, 7T data, and other materials, including the training labels and the Turing test.

Neuropsychiatric systemic lupus erythematosus is associated with a distinct type and shape of cerebral white matter hyperintensities

Objectives

Advanced white matter hyperintensity (WMH) markers on brain MRI may help reveal underlying mechanisms and aid in the diagnosis of different phenotypes of SLE patients experiencing neuropsychiatric (NP) manifestations.

Methods

In this prospective cohort study, we included a clinically well-defined cohort of 155 patients consisting of 38 patients with NPSLE (26 inflammatory and 12 ischaemic phenotype) and 117 non-NPSLE patients. Differences in 3 T MRI WMH markers (volume, type and shape) were compared between patients with NPSLE and non-NPSLE and between patients with inflammatory and ischaemic NPSLE by linear and logistic regression analyses corrected for age, sex and intracranial volume.

Results

Compared with non-NPSLE [92% female; mean age 42 (13) years], patients with NPSLE [87% female; mean age 40 (14) years] showed a higher total WMH volume [B (95%-CI)]: 0.46 (0.0 7 ↔ 0.86); P = 0.021], a higher periventricular/confluent WMH volume [0.46 (0.0 6 ↔ 0.86); P = 0.024], a higher occurrence of periventricular with deep WMH type [0.32 (0.1 3 ↔ 0.77); P = 0.011], a higher number of deep WMH lesions [3.06 (1.2 1 ↔ 4.90); P = 0.001] and a more complex WMH shape [convexity: ‒0.07 (‒0.12 ↔ ‒0.02); P = 0.011, concavity index: 0.05 (0.0 1 ↔ 0.08); P = 0.007]. WMH shape was more complex in inflammatory NPSLE patients [89% female; mean age 39 (15) years] compared with patients with the ischaemic phenotype [83% female; mean age 41 (11) years] [concavity index: 0.08 (0.0 1 ↔ 0.15); P = 0.034].

Conclusion

We demonstrated that patients with NPSLE showed a higher periventricular/confluent WMH volume and more complex shape of WMH compared with non-NPSLE patients. This finding was particularly significant in inflammatory NPLSE patients, suggesting different or more severe underlying pathophysiological abnormalities.

HybridCTrm: Bridging CNN and Transformer for Multimodal Brain Image Segmentation

Multimodal medical image segmentation is always a critical problem in medical image segmentation. Traditional deep learning methods utilize fully CNNs for encoding given images, thus leading to deficiency of long-range dependencies and bad generalization performance. Recently, a sequence of Transformer-based methodologies emerges in the field of image processing, which brings great generalization and performance in various tasks. On the other hand, traditional CNNs have their own advantages, such as rapid convergence and local representations. Therefore, we analyze a hybrid multimodal segmentation method based on Transformers and CNNs and propose a novel architecture, HybridCTrm network. We conduct experiments using HybridCTrm on two benchmark datasets and compare with HyperDenseNet, a network based on fully CNNs. Results show that our HybridCTrm outperforms HyperDenseNet on most of the evaluation metrics. Furthermore, we analyze the influence of the depth of Transformer on the performance. Besides, we visualize the results and carefully explore how our hybrid methods improve on segmentations.

Realistic image normalization for multi-Domain segmentation

Image normalization is a building block in medical image analysis. Conventional approaches are customarily employed on a per-dataset basis. This strategy, however, prevents the current normalization algorithms from fully exploiting the complex joint information available across multiple datasets. Consequently, ignoring such joint information has a direct impact on the processing of segmentation algorithms. This paper proposes to revisit the conventional image normalization approach by, instead, learning a common normalizing function across multiple datasets. Jointly normalizing multiple datasets is shown to yield consistent normalized images as well as an improved image segmentation when intensity shifts are large. To do so, a fully automated adversarial and task-driven normalization approach is employed as it facilitates the training of realistic and interpretable images while keeping performance on par with the state-of-the-art. The adversarial training of our network aims at finding the optimal transfer function to improve both, jointly, the segmentation accuracy and the generation of realistic images. We have evaluated the performance of our normalizer on both infant and adult brain images from the iSEG, MRBrainS and ABIDE datasets. The results indicate that our contribution does provide an improved realism to the normalized images, while retaining a segmentation accuracy at par with the state-of-the-art learnable normalization approaches.

Deep learning methods for predicting brain abnormalities and compute human cognitive power using fMRI

In modern times, digital medical images play a significant progression in clinical diagnosis to treat the populace earlier to hoard their lives. Magnetic Resonance Imaging (MRI) is one of the most advanced medical imaging modalities that facilitate scanning various parts of the human body like the head, chest, abdomen, and pelvis and identify the diseases. Numerous studies on the same discipline have proposed different algorithms, techniques, and methods for analyzing medical digital images, especially MRI. Most of them have mainly focused on identifying and classifying the images as either normal or abnormal. Computing brainpower is essential to understand and handle various brain diseases efficiently in critical situations. This paper knuckles down to design and implement a computer-aided framework, enhancing the identification of humans’ cognitive power from their MRI. Images. The proposed framework converts the 3D DICOM images into 2D medical images, preprocessing, enhancement, learning, and extracting various image information to classify it as normal or abnormal and provide the brain’s cognitive power. This study widens the efficient use of machine learning methods, Voxel Residual Network (VRN), with multimodality fusion architecture to learn and analyze the image to classify and predict cognitive power. The experimental results denote that the proposed framework demonstrates better performance than the existing approaches.

Fully automatic brain tumor segmentation for 3D evaluation in augmented reality

OBJECTIVE

For currently available augmented reality workflows, 3D models need to be created with manual or semiautomatic segmentation, which is a time-consuming process. The authors created an automatic segmentation algorithm that generates 3D models of skin, brain, ventricles, and contrast-enhancing tumor from a single T1-weighted MR sequence and embedded this model into an automatic workflow for 3D evaluation of anatomical structures with augmented reality in a cloud environment. In this study, the authors validate the accuracy and efficiency of this automatic segmentation algorithm for brain tumors and compared it with a manually segmented ground truth set.

METHODS

Fifty contrast-enhanced T1-weighted sequences of patients with contrast-enhancing lesions measuring at least 5 cm3 were included. All slices of the ground truth set were manually segmented. The same scans were subsequently run in the cloud environment for automatic segmentation. Segmentation times were recorded. The accuracy of the algorithm was compared with that of manual segmentation and evaluated in terms of Sørensen-Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and 95th percentile of Hausdorff distance (HD95).

RESULTS

The mean ± SD computation time of the automatic segmentation algorithm was 753 ± 128 seconds. The mean ± SD DSC was 0.868 ± 0.07, ASSD was 1.31 ± 0.63 mm, and HD95 was 4.80 ± 3.18 mm. Meningioma (mean 0.89 and median 0.92) showed greater DSC than metastasis (mean 0.84 and median 0.85). Automatic segmentation had greater accuracy for measuring DSC (mean 0.86 and median 0.87) and HD95 (mean 3.62 mm and median 3.11 mm) of supratentorial metastasis than those of infratentorial metastasis (mean 0.82 and median 0.81 for DSC; mean 5.26 mm and median 4.72 mm for HD95).

CONCLUSIONS

The automatic cloud-based segmentation algorithm is reliable, accurate, and fast enough to aid neurosurgeons in everyday clinical practice by providing 3D augmented reality visualization of contrast-enhancing intracranial lesions measuring at least 5 cm3. The next steps involve incorporation of other sequences and improving accuracy with 3D fine-tuning in order to expand the scope of augmented reality workflow.

An atlas of classifiers-a machine learning paradigm for brain MRI segmentation

We present the Atlas of Classifiers (AoC)-a conceptually novel framework for brain MRI segmentation. The AoC is a spatial map of voxel-wise multinomial logistic regression (LR) functions learned from the labeled data. Upon convergence, the resulting fixed LR weights, a few for each voxel, represent the training dataset. It can, therefore, be considered as a light-weight learning machine, which despite its low capacity does not underfit the problem. The AoC construction is independent of the actual intensities of the test images, providing the flexibility to train it on the available labeled data and use it for the segmentation of images from different datasets and modalities. In this sense, it does not overfit the training data, as well. The proposed method has been applied to numerous publicly available datasets for the segmentation of brain MRI tissues and is shown to be robust to noise and outreach commonly used methods. Promising results were also obtained for multi-modal, cross-modality MRI segmentation. Finally, we show how AoC trained on brain MRIs of healthy subjects can be exploited for lesion segmentation of multiple sclerosis patients.

Automated multiclass tissue segmentation of clinical brain MRIs with lesions

•

A U-Net incorporating spatial prior information can successfully segment 6 brain tissue types.

•

The U-Net was able to segment gray and white matter in the presence of lesions.

•

The U-Net surpassed the performance of its source algorithm in an external dataset.

•

Segmentations were produced in a hundredth of the time of its predecessor algorithm.

Delineation and quantification of normal and abnormal brain tissues on Magnetic Resonance Images is fundamental to the diagnosis and longitudinal assessment of neurological diseases. Here we sought to develop a convolutional neural network for automated multiclass tissue segmentation of brain MRIs that was robust at typical clinical resolutions and in the presence of a variety of lesions. We trained a 3D U-Net for full brain multiclass tissue segmentation from a prior atlas-based segmentation method on an internal dataset that consisted of 558 clinical T1-weighted brain MRIs (453/52/53; training/validation/test) of patients with one of 50 different diagnostic entities (n = 362) or with a normal brain MRI (n = 196). We then used transfer learning to refine our model on an external dataset that consisted of 7 patients with hand-labeled tissue types. We evaluated the tissue-wise and intra-lesion performance with different loss functions and spatial prior information in the validation set and applied the best performing model to the internal and external test sets. The network achieved an average overall Dice score of 0.87 and volume similarity of 0.97 in the internal test set. Further, the network achieved a median intra-lesion tissue segmentation accuracy of 0.85 inside lesions within white matter and 0.61 inside lesions within gray matter. After transfer learning, the network achieved an average overall Dice score of 0.77 and volume similarity of 0.96 in the external dataset compared to human raters. The network had equivalent or better performance than the original atlas-based method on which it was trained across all metrics and produced segmentations in a hundredth of the time. We anticipate that this pipeline will be a useful tool for clinical decision support and quantitative analysis of clinical brain MRIs in the presence of lesions.

Privacy-Net: An Adversarial Approach for Identity-Obfuscated Segmentation of Medical Images

This paper presents a client/server privacy-preserving network in the context of multicentric medical image analysis. Our approach is based on adversarial learning which encodes images to obfuscate the patient identity while preserving enough information for a target task. Our novel architecture is composed of three components: 1) an encoder network which removes identity-specific features from input medical images, 2) a discriminator network that attempts to identify the subject from the encoded images, 3) a medical image analysis network which analyzes the content of the encoded images (segmentation in our case). By simultaneously fooling the discriminator and optimizing the medical analysis network, the encoder learns to remove privacy-specific features while keeping those essentials for the target task. Our approach is illustrated on the problem of segmenting brain MRI from the large-scale Parkinson Progression Marker Initiative (PPMI) dataset. Using longitudinal data from PPMI, we show that the discriminator learns to heavily distort input images while allowing for highly accurate segmentation results. Our results also demonstrate that an encoder trained on the PPMI dataset can be used for segmenting other datasets, without the need for retraining. The code is made available at: https://github.com/bachkimn/Privacy-Net-An-Adversarial-Approach-forIdentity-Obfuscated-Segmentation-of-MedicalImages.

APRNet: A 3D Anisotropic Pyramidal Reversible Network with Multi-modal Cross-Dimension Attention for Brain Tissue Segmentation in MR Images

Brain tissue segmentation in multi-modal magnetic resonance (MR) images is significant for the clinical diagnosis of brain diseases. Due to blurred boundaries, low contrast, and intricate anatomical relationships between brain tissue regions, automatic brain tissue segmentation without prior knowledge is still challenging. This paper presents a novel 3D fully convolutional network (FCN) for brain tissue segmentation, called APRNet. In this network, we first propose a 3D anisotropic pyramidal convolutional reversible residual sequence (3DAPC-RRS) module to integrate the intra-slice information with the inter-slice information without significant memory consumption; secondly, we design a multi-modal cross-dimension attention (MCDA) module to automatically capture the effective information in each dimension of multi-modal images; then, we apply 3DAPC-RRS modules and MCDA modules to a 3D FCN with multiple encoded streams and one decoded stream for constituting the overall architecture of APRNet. We evaluated APRNet on two benchmark challenges, namely MRBrainS13 and iSeg-2017. The experimental results show that APRNet yields state-of-the-art segmentation results on both benchmark challenge datasets and achieves the best segmentation performance on the cerebrospinal fluid region. Compared with other methods, our proposed approach exploits the complementary information of different modalities to segment brain tissue regions in both adult and infant MR images, and it achieves the average Dice coefficient of 87.22% and 93.03% on the MRBrainS13 and iSeg-2017 testing data, respectively. The proposed method is beneficial for quantitative brain analysis in the clinical study, and our code is made publicly available.

A deep cascade of ensemble of dual domain networks with gradient-based T1 assistance and perceptual refinement for fast MRI reconstruction

Deep learning networks have shown promising results in fast magnetic resonance imaging (MRI) reconstruction. In our work, we develop deep networks to further improve the quantitative and the perceptual quality of reconstruction. To begin with, we propose reconsynergynet (RSN), a network that combines the complementary benefits of independently operating on both the image and the Fourier domain. For a single-coil acquisition, we introduce deep cascade RSN (DC-RSN), a cascade of RSN blocks interleaved with data fidelity (DF) units. Secondly, we improve the structure recovery of DC-RSN for T2 weighted Imaging (T2WI) through assistance of T1 weighted imaging (T1WI), a sequence with short acquisition time. T1 assistance is provided to DC-RSN through a gradient of log feature (GOLF) fusion. Furthermore, we propose perceptual refinement network (PRN) to refine the reconstructions for better visual information fidelity (VIF), a metric highly correlated to radiologist's opinion on the image quality. Lastly, for multi-coil acquisition, we propose variable splitting RSN (VS-RSN), a deep cascade of blocks, each block containing RSN, multi-coil DF unit, and a weighted average module. We extensively validate our models DC-RSN and VS-RSN for single-coil and multi-coil acquisitions and report the state-of-the-art performance. We obtain a SSIM of 0.768, 0.923, and 0.878 for knee single-coil-4x, multi-coil-4x, and multi-coil-8x in fastMRI, respectively. We also conduct experiments to demonstrate the efficacy of GOLF based T1 assistance and PRN.

Learning U-Net Based Multi-Scale Features in Encoding-Decoding for MR Image Brain Tissue Segmentation

Accurate brain tissue segmentation of MRI is vital to diagnosis aiding, treatment planning, and neurologic condition monitoring. As an excellent convolutional neural network (CNN), U-Net is widely used in MR image segmentation as it usually generates high-precision features. However, the performance of U-Net is considerably restricted due to the variable shapes of the segmented targets in MRI and the information loss of down-sampling and up-sampling operations. Therefore, we propose a novel network by introducing spatial and channel dimensions-based multi-scale feature information extractors into its encoding-decoding framework, which is helpful in extracting rich multi-scale features while highlighting the details of higher-level features in the encoding part, and recovering the corresponding localization to a higher resolution layer in the decoding part. Concretely, we propose two information extractors, multi-branch pooling, called MP, in the encoding part, and multi-branch dense prediction, called MDP, in the decoding part, to extract multi-scale features. Additionally, we designed a new multi-branch output structure with MDP in the decoding part to form more accurate edge-preserving predicting maps by integrating the dense adjacent prediction features at different scales. Finally, the proposed method is tested on datasets MRbrainS13, IBSR18, and ISeg2017. We find that the proposed network performs higher accuracy in segmenting MRI brain tissues and it is better than the leading method of 2018 at the segmentation of GM and CSF. Therefore, it can be a useful tool for diagnostic applications, such as brain MRI segmentation and diagnosing.

Different phenotypes of neuropsychiatric systemic lupus erythematosus are related to a distinct pattern of structural changes on brain MRI

Objectives

The underlying structural brain correlates of neuropsychiatric involvement in systemic lupus erythematosus (NPSLE) remain unclear, thus hindering correct diagnosis.

We compared brain tissue volumes between a clinically well-defined cohort of patients with NPSLE and SLE patients with neuropsychiatric syndromes not attributed to SLE (non-NPSLE). Within the NPSLE patients, we also examined differences between patients with two distinct disease phenotypes: ischemic and inflammatory.

Methods

In this prospective (May 2007 to April 2015) cohort study, we included 38 NPSLE patients (26 inflammatory and 12 ischemic) and 117 non-NPSLE patients. All patients underwent a 3-T brain MRI scan that was used to automatically determine white matter, grey matter, white matter hyperintensities (WMH) and total brain volumes. Group differences in brain tissue volumes were studied with linear regression analyses corrected for age, gender, and total intracranial volume and expressed as B values and 95% confidence intervals.

Results

NPSLE patients showed higher WMH volume compared to non-NPSLE patients (p = 0.004). NPSLE inflammatory patients showed lower total brain (p = 0.014) and white matter volumes (p = 0.020), and higher WMH volume (p = 0.002) compared to non-NPSLE patients. Additionally, NPSLE inflammatory patients showed lower white matter (p = 0.020) and total brain volumes (p = 0.038) compared to NPSLE ischemic patients.

Conclusion

We showed that different phenotypes of NPSLE were related to distinct patterns of underlying structural brain MRI changes. Especially the inflammatory phenotype of NPSLE was associated with the most pronounced brain volume changes, which might facilitate the diagnostic process in SLE patients with neuropsychiatric symptoms.

Key Points

• Neuropsychiatric systemic lupus erythematosus (NPSLE) patients showed a higher WMH volume compared to SLE patients with neuropsychiatric syndromes not attributed to SLE (non-NPSLE).

• NPSLE patients with inflammatory phenotype showed a lower total brain and white matter volume, and a higher volume of white matter hyperintensities, compared to non-NPSLE patients.

• NPSLE patients with inflammatory phenotype showed lower white matter and total brain volumes compared to NPSLE patients with ischemic phenotype.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-07970-2.

A Deep Attentive Convolutional Neural Network for Automatic Cortical Plate Segmentation in Fetal MRI

Fetal cortical plate segmentation is essential in quantitative analysis of fetal brain maturation and cortical folding. Manual segmentation of the cortical plate, or manual refinement of automatic segmentations is tedious and time-consuming. Automatic segmentation of the cortical plate, on the other hand, is challenged by the relatively low resolution of the reconstructed fetal brain MRI scans compared to the thin structure of the cortical plate, partial voluming, and the wide range of variations in the morphology of the cortical plate as the brain matures during gestation. To reduce the burden of manual refinement of segmentations, we have developed a new and powerful deep learning segmentation method. Our method exploits new deep attentive modules with mixed kernel convolutions within a fully convolutional neural network architecture that utilizes deep supervision and residual connections. We evaluated our method quantitatively based on several performance measures and expert evaluations. Results show that our method outperforms several state-of-the-art deep models for segmentation, as well as a state-of-the-art multi-atlas segmentation technique. We achieved average Dice similarity coefficient of 0.87, average Hausdorff distance of 0.96 mm, and average symmetric surface difference of 0.28 mm on reconstructed fetal brain MRI scans of fetuses scanned in the gestational age range of 16 to 39 weeks (28.6± 5.3). With a computation time of less than 1 minute per fetal brain, our method can facilitate and accelerate large-scale studies on normal and altered fetal brain cortical maturation and folding.

Neuropsychiatric disease classification using functional connectomics - results of the connectomics in neuroimaging transfer learning challenge

Large, open-source datasets, such as the Human Connectome Project and the Autism Brain Imaging Data Exchange, have spurred the development of new and increasingly powerful machine learning approaches for brain connectomics. However, one key question remains: are we capturing biologically relevant and generalizable information about the brain, or are we simply overfitting to the data? To answer this, we organized a scientific challenge, the Connectomics in NeuroImaging Transfer Learning Challenge (CNI-TLC), held in conjunction with MICCAI 2019. CNI-TLC included two classification tasks: (1) diagnosis of Attention-Deficit/Hyperactivity Disorder (ADHD) within a pre-adolescent cohort; and (2) transference of the ADHD model to a related cohort of Autism Spectrum Disorder (ASD) patients with an ADHD comorbidity. In total, 240 resting-state fMRI (rsfMRI) time series averaged according to three standard parcellation atlases, along with clinical diagnosis, were released for training and validation (120 neurotypical controls and 120 ADHD). We also provided Challenge participants with demographic information of age, sex, IQ, and handedness. The second set of 100 subjects (50 neurotypical controls, 25 ADHD, and 25 ASD with ADHD comorbidity) was used for testing. Classification methodologies were submitted in a standardized format as containerized Docker images through ChRIS, an open-source image analysis platform. Utilizing an inclusive approach, we ranked the methods based on 16 metrics: accuracy, area under the curve, F1-score, false discovery rate, false negative rate, false omission rate, false positive rate, geometric mean, informedness, markedness, Matthew's correlation coefficient, negative predictive value, optimized precision, precision, sensitivity, and specificity. The final rank was calculated using the rank product for each participant across all measures. Furthermore, we assessed the calibration curves of each methodology. Five participants submitted their method for evaluation, with one outperforming all other methods in both ADHD and ASD classification. However, further improvements are still needed to reach the clinical translation of functional connectomics. We have kept the CNI-TLC open as a publicly available resource for developing and validating new classification methodologies in the field of connectomics.

A Review on Deep Learning Architecture and Methods for MRI Brain Tumour Segmentation

BACKGROUND

The automatic segmentation of brain tumour from MRI medical images is mainly covered in this review. Recently, state-of-the-art performance is provided by deep learning- based approaches in the field of image classification, segmentation, object detection, and tracking tasks.

INTRODUCTION

The core feature deep learning approach is the hierarchical representation of features from images, thus avoiding domain-specific handcrafted features.

METHODS

In this review paper, we have dealt with a review of Deep Learning Architecture and Methods for MRI Brain Tumour Segmentation. First, we have discussed the basic architecture and approaches for deep learning methods. Secondly, we have discussed the literature survey of MRI brain tumour segmentation using deep learning methods and its multimodality fusion. Then, the advantages and disadvantages of each method are analyzed and finally, it is concluded with a discussion on the merits and challenges of deep learning techniques.

RESULTS

The review of brain tumour identification using deep learning.

CONCLUSION

Techniques may help the researchers to have a better focus on it.

Semantic segmentation of cerebrospinal fluid and brain volume with a convolutional neural network in pediatric hydrocephalus-transfer learning from existing algorithms

Background

For the segmentation of medical imaging data, a multitude of precise but very specific algorithms exist. In previous studies, we investigated the possibility of segmenting MRI data to determine cerebrospinal fluid and brain volume using a classical machine learning algorithm. It demonstrated good clinical usability and a very accurate correlation of the volumes to the single area determination in a reproducible axial layer. This study aims to investigate whether these established segmentation algorithms can be transferred to new, more generalizable deep learning algorithms employing an extended transfer learning procedure and whether medically meaningful segmentation is possible.

Methods

Ninety-five routinely performed true FISP MRI sequences were retrospectively analyzed in 43 patients with pediatric hydrocephalus. Using a freely available and clinically established segmentation algorithm based on a hidden Markov random field model, four classes of segmentation (brain, cerebrospinal fluid (CSF), background, and tissue) were generated. Fifty-nine randomly selected data sets (10,432 slices) were used as a training data set. Images were augmented for contrast, brightness, and random left/right and X/Y translation. A convolutional neural network (CNN) for semantic image segmentation composed of an encoder and corresponding decoder subnetwork was set up. The network was pre-initialized with layers and weights from a pre-trained VGG 16 model. Following the network was trained with the labeled image data set. A validation data set of 18 scans (3289 slices) was used to monitor the performance as the deep CNN trained. The classification results were tested on 18 randomly allocated labeled data sets (3319 slices) and on a T2-weighted BrainWeb data set with known ground truth.

Results

The segmentation of clinical test data provided reliable results (global accuracy 0.90, Dice coefficient 0.86), while the CNN segmentation of data from the BrainWeb data set showed comparable results (global accuracy 0.89, Dice coefficient 0.84). The segmentation of the BrainWeb data set with the classical FAST algorithm produced consistent findings (global accuracy 0.90, Dice coefficient 0.87). Likewise, the area development of brain and CSF in the long-term clinical course of three patients was presented.

Conclusion

Using the presented methods, we showed that conventional segmentation algorithms can be transferred to new advances in deep learning with comparable accuracy, generating a large number of training data sets with relatively little effort. A clinically meaningful segmentation possibility was demonstrated.