Learning MRI artefact removal with unpaired data

The effect of head motion on brain age prediction using deep convolutional neural networks

Deep learning can be used effectively to predict participants' age from brain magnetic resonance imaging (MRI) data, and a growing body of evidence suggests that the difference between predicted and chronological age-referred to as brain-predicted age difference (brain-PAD)-is related to various neurological and neuropsychiatric disease states. A crucial aspect of the applicability of brain-PAD as a biomarker of individual brain health is whether and how brain-predicted age is affected by MR image artifacts commonly encountered in clinical settings. To investigate this issue, we trained and validated two different 3D convolutional neural network architectures (CNNs) from scratch and tested the models on a separate dataset consisting of motion-free and motion-corrupted T1-weighted MRI scans from the same participants, the quality of which were rated by neuroradiologists from a clinical diagnostic point of view. Our results revealed a systematic increase in brain-PAD with worsening image quality for both models. This effect was also observed for images that were deemed usable from a clinical perspective, with brains appearing older in medium than in good quality images. These findings were also supported by significant associations found between the brain-PAD and standard image quality metrics indicating larger brain-PAD for lower-quality images. Our results demonstrate a spurious effect of advanced brain aging as a result of head motion and underline the importance of controlling for image quality when using brain-predicted age based on structural neuroimaging data as a proxy measure for brain health.

Functional Hierarchy of the Human Neocortex from Cradle to Grave

Recent evidence indicates that the organization of the human neocortex is underpinned by smooth spatial gradients of functional connectivity (FC). These gradients provide crucial insight into the relationship between the brain's topographic organization and the texture of human cognition. However, no studies to date have charted how intrinsic FC gradient architecture develops across the entire human lifespan. In this work, we model developmental trajectories of the three primary gradients of FC using a large, high-quality, and temporally-dense functional MRI dataset spanning from birth to 100 years of age. The gradient axes, denoted as sensorimotor-association (SA), visual-somatosensory (VS), and modulation-representation (MR), encode crucial hierarchical organizing principles of the brain in development and aging. By tracking their evolution throughout the human lifespan, we provide the first ever comprehensive low-dimensional normative reference of global FC hierarchical architecture. We observe significant age-related changes in global network features, with global markers of hierarchical organization increasing from birth to early adulthood and decreasing thereafter. During infancy and early childhood, FC organization is shaped by primary sensory processing, dense short-range connectivity, and immature association and control hierarchies. Functional differentiation of transmodal systems supported by long-range coupling drives a convergence toward adult-like FC organization during late childhood, while adolescence and early adulthood are marked by the expansion and refinement of SA and MR hierarchies. While gradient topographies remain stable during late adulthood and aging, we observe decreases in global gradient measures of FC differentiation and complexity from 30 to 100 years. Examining cortical microstructure gradients alongside our functional gradients, we observed that structure-function gradient coupling undergoes differential lifespan trajectories across multiple gradient axes.

Unsupervised MRI motion artifact disentanglement: introducing MAUDGAN

Objective.This study developed an unsupervised motion artifact reduction method for magnetic resonance imaging (MRI) images of patients with brain tumors. The proposed novel design uses multi-parametric multicenter contrast-enhanced T1W (ceT1W) and T2-FLAIR MRI images.Approach.The proposed framework included two generators, two discriminators, and two feature extractor networks. A 3-fold cross-validation was used to train and fine-tune the hyperparameters of the proposed model using 230 brain MRI images with tumors, which were then tested on 148 patients'in-vivodatasets. An ablation was performed to evaluate the model's compartments. Our model was compared with Pix2pix and CycleGAN. Six evaluation metrics were reported, including normalized mean squared error (NMSE), structural similarity index (SSIM), multi-scale-SSIM (MS-SSIM), peak signal-to-noise ratio (PSNR), visual information fidelity (VIF), and multi-scale gradient magnitude similarity deviation (MS-GMSD). Artifact reduction and consistency of tumor regions, image contrast, and sharpness were evaluated by three evaluators using Likert scales and compared with ANOVA and Tukey's HSD tests.Main results.On average, our method outperforms comparative models to remove heavy motion artifacts with the lowest NMSE (18.34±5.07%) and MS-GMSD (0.07 ± 0.03) for heavy motion artifact level. Additionally, our method creates motion-free images with the highest SSIM (0.93 ± 0.04), PSNR (30.63 ± 4.96), and VIF (0.45 ± 0.05) values, along with comparable MS-SSIM (0.96 ± 0.31). Similarly, our method outperformed comparative models in removingin-vivomotion artifacts for different distortion levels except for MS- SSIM and VIF, which have comparable performance with CycleGAN. Moreover, our method had a consistent performance for different artifact levels. For the heavy level of motion artifacts, our method got the highest Likert scores of 2.82 ± 0.52, 1.88 ± 0.71, and 1.02 ± 0.14 (p-values≪0.0001) for our method, CycleGAN, and Pix2pix respectively. Similar trends were also found for other motion artifact levels.Significance.Our proposed unsupervised method was demonstrated to reduce motion artifacts from the ceT1W brain images under a multi-parametric framework.

Unsupervised deep learning model for correcting Nyquist ghosts of single-shot spatiotemporal encoding

PURPOSE

To design an unsupervised deep learning (DL) model for correcting Nyquist ghosts of single-shot spatiotemporal encoding (SPEN) and evaluate the model for real MRI applications.

METHODS

The proposed method consists of three main components: (1) an unsupervised network that combines Residual Encoder and Restricted Subspace Mapping (RERSM-net) and is trained to generate a phase-difference map based on the even and odd SPEN images; (2) a spin physical forward model to obtain the corrected image with the learned phase difference map; and (3) cycle-consistency loss that is explored for training the RERSM-net.

RESULTS

The proposed RERSM-net could effectively generate smooth phase difference maps and correct Nyquist ghosts of single-shot SPEN. Both simulation and real in vivo MRI experiments demonstrated that our method outperforms the state-of-the-art SPEN Nyquist ghost correction method. Furthermore, the ablation experiments of generating phase-difference maps show the advantages of the proposed unsupervised model.

CONCLUSION

The proposed method can effectively correct Nyquist ghosts for the single-shot SPEN sequence.

Stop moving: MR motion correction as an opportunity for artificial intelligence

Subject motion is a long-standing problem of magnetic resonance imaging (MRI), which can seriously deteriorate the image quality. Various prospective and retrospective methods have been proposed for MRI motion correction, among which deep learning approaches have achieved state-of-the-art motion correction performance. This survey paper aims to provide a comprehensive review of deep learning-based MRI motion correction methods. Neural networks used for motion artifacts reduction and motion estimation in the image domain or frequency domain are detailed. Furthermore, besides motion-corrected MRI reconstruction, how estimated motion is applied in other downstream tasks is briefly introduced, aiming to strengthen the interaction between different research areas. Finally, we identify current limitations and point out future directions of deep learning-based MRI motion correction.

Learning multi-site harmonization of magnetic resonance images without traveling human phantoms

Harmonization improves Magn. Reson. Imaging (MRI) data consistency and is central to effective integration of diverse imaging data acquired across multiple sites. Recent deep learning techniques for harmonization are predominantly supervised in nature and hence require imaging data of the same human subjects to be acquired at multiple sites. Data collection as such requires the human subjects to travel across sites and is hence challenging, costly, and impractical, more so when sufficient sample size is needed for reliable network training. Here we show how harmonization can be achieved with a deep neural network that does not rely on traveling human phantom data. Our method disentangles site-specific appearance information and site-invariant anatomical information from images acquired at multiple sites and then employs the disentangled information to generate the image of each subject for any target site. We demonstrate with more than 6,000 multi-site T1- and T2-weighted images that our method is remarkably effective in generating images with realistic site-specific appearances without altering anatomical details. Our method allows retrospective harmonization of data in a wide range of existing modern large-scale imaging studies, conducted via different scanners and protocols, without additional data collection.

Differential privacy preserved federated transfer learning for multi-institutional 68Ga-PET image artefact detection and disentanglement

PURPOSE

Image artefacts continue to pose challenges in clinical molecular imaging, resulting in misdiagnoses, additional radiation doses to patients and financial costs. Mismatch and halo artefacts occur frequently in gallium-68 (68Ga)-labelled compounds whole-body PET/CT imaging. Correcting for these artefacts is not straightforward and requires algorithmic developments, given that conventional techniques have failed to address them adequately. In the current study, we employed differential privacy-preserving federated transfer learning (FTL) to manage clinical data sharing and tackle privacy issues for building centre-specific models that detect and correct artefacts present in PET images.

METHODS

Altogether, 1413 patients with 68Ga prostate-specific membrane antigen (PSMA)/DOTA-TATE (TOC) PET/CT scans from 3 countries, including 8 different centres, were enrolled in this study. CT-based attenuation and scatter correction (CT-ASC) was used in all centres for quantitative PET reconstruction. Prior to model training, an experienced nuclear medicine physician reviewed all images to ensure the use of high-quality, artefact-free PET images (421 patients' images). A deep neural network (modified U2Net) was trained on 80% of the artefact-free PET images to utilize centre-based (CeBa), centralized (CeZe) and the proposed differential privacy FTL frameworks. Quantitative analysis was performed in 20% of the clean data (with no artefacts) in each centre. A panel of two nuclear medicine physicians conducted qualitative assessment of image quality, diagnostic confidence and image artefacts in 128 patients with artefacts (256 images for CT-ASC and FTL-ASC).

RESULTS

The three approaches investigated in this study for 68Ga-PET imaging (CeBa, CeZe and FTL) resulted in a mean absolute error (MAE) of 0.42 ± 0.21 (CI 95%: 0.38 to 0.47), 0.32 ± 0.23 (CI 95%: 0.27 to 0.37) and 0.28 ± 0.15 (CI 95%: 0.25 to 0.31), respectively. Statistical analysis using the Wilcoxon test revealed significant differences between the three approaches, with FTL outperforming CeBa and CeZe (p-value < 0.05) in the clean test set. The qualitative assessment demonstrated that FTL-ASC significantly improved image quality and diagnostic confidence and decreased image artefacts, compared to CT-ASC in 68Ga-PET imaging. In addition, mismatch and halo artefacts were successfully detected and disentangled in the chest, abdomen and pelvic regions in 68Ga-PET imaging.

CONCLUSION

The proposed approach benefits from using large datasets from multiple centres while preserving patient privacy. Qualitative assessment by nuclear medicine physicians showed that the proposed model correctly addressed two main challenging artefacts in 68Ga-PET imaging. This technique could be integrated in the clinic for 68Ga-PET imaging artefact detection and disentanglement using multicentric heterogeneous datasets.

Artificial Intelligence-Driven Single-Shot PET Image Artifact Detection and Disentanglement: Toward Routine Clinical Image Quality Assurance

PURPOSE

Medical imaging artifacts compromise image quality and quantitative analysis and might confound interpretation and misguide clinical decision-making. The present work envisions and demonstrates a new paradigm PET image Quality Assurance NETwork (PET-QA-NET) in which various image artifacts are detected and disentangled from images without prior knowledge of a standard of reference or ground truth for routine PET image quality assurance.

METHODS

The network was trained and evaluated using training/validation/testing data sets consisting of 669/100/100 artifact-free oncological 18 F-FDG PET/CT images and subsequently fine-tuned and evaluated on 384 (20% for fine-tuning) scans from 8 different PET centers. The developed DL model was quantitatively assessed using various image quality metrics calculated for 22 volumes of interest defined on each scan. In addition, 200 additional 18 F-FDG PET/CT scans (this time with artifacts), generated using both CT-based attenuation and scatter correction (routine PET) and PET-QA-NET, were blindly evaluated by 2 nuclear medicine physicians for the presence of artifacts, diagnostic confidence, image quality, and the number of lesions detected in different body regions.

RESULTS

Across the volumes of interest of 100 patients, SUV MAE values of 0.13 ± 0.04, 0.24 ± 0.1, and 0.21 ± 0.06 were reached for SUV mean , SUV max , and SUV peak , respectively (no statistically significant difference). Qualitative assessment showed a general trend of improved image quality and diagnostic confidence and reduced image artifacts for PET-QA-NET compared with routine CT-based attenuation and scatter correction.

CONCLUSION

We developed a highly effective and reliable quality assurance tool that can be embedded routinely to detect and correct for 18 F-FDG PET image artifacts in clinical setting with notably improved PET image quality and quantitative capabilities.

Correction of Arterial-Phase Motion Artifacts in Gadoxetic Acid-Enhanced Liver MRI Using an Innovative Unsupervised Network

This study aims to propose and evaluate DR-CycleGAN, a disentangled unsupervised network by introducing a novel content-consistency loss, for removing arterial-phase motion artifacts in gadoxetic acid-enhanced liver MRI examinations. From June 2020 to July 2021, gadoxetic acid-enhanced liver MRI data were retrospectively collected in this center to establish training and testing datasets. Motion artifacts were semi-quantitatively assessed using a five-point Likert scale (1 = no artifact, 2 = mild, 3 = moderate, 4 = severe, and 5 = non-diagnostic) and quantitatively evaluated using the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR). The datasets comprised a training dataset (308 examinations, including 58 examinations with artifact grade = 1 and 250 examinations with artifact grade ≥ 2), a paired test dataset (320 examinations, including 160 examinations with artifact grade = 1 and paired 160 examinations with simulated motion artifacts of grade ≥ 2), and an unpaired test dataset (474 examinations with artifact grade ranging from 1 to 5). The performance of DR-CycleGAN was evaluated and compared with a state-of-the-art network, Cycle-MedGAN V2.0. As a result, in the paired test dataset, DR-CycleGAN demonstrated significantly higher SSIM and PSNR values and lower motion artifact grades compared to Cycle-MedGAN V2.0 (0.89 ± 0.07 vs. 0.84 ± 0.09, 32.88 ± 2.11 vs. 30.81 ± 2.64, and 2.7 ± 0.7 vs. 3.0 ± 0.9, respectively; p < 0.001 each). In the unpaired test dataset, DR-CycleGAN also exhibited a superior motion artifact correction performance, resulting in a significant decrease in motion artifact grades from 2.9 ± 1.3 to 2.0 ± 0.6 compared to Cycle-MedGAN V2.0 (to 2.4 ± 0.9, p < 0.001). In conclusion, DR-CycleGAN effectively reduces motion artifacts in the arterial phase images of gadoxetic acid-enhanced liver MRI examinations, offering the potential to enhance image quality.

Unsupervised dual-domain disentangled network for removal of rigid motion artifacts in MRI

Motion artifacts in magnetic resonance imaging (MRI) have always been a serious issue because they can affect subsequent diagnosis and treatment. Supervised deep learning methods have been investigated for the removal of motion artifacts; however, they require paired data that are difficult to obtain in clinical settings. Although unsupervised methods are widely proposed to fully use clinical unpaired data, they generally focus on anatomical structures generated by the spatial domain while ignoring phase error (deviations or inaccuracies in phase information that are possibly caused by rigid motion artifacts during image acquisition) provided by the frequency domain. In this study, a 2D unsupervised deep learning method named unsupervised disentangled dual-domain network (UDDN) was proposed to effectively disentangle and remove unwanted rigid motion artifacts from images. In UDDN, a dual-domain encoding module was presented to capture different types of information from the spatial and frequency domains to enrich the information. Moreover, a cross-domain attention fusion module was proposed to effectively fuse information from different domains, reduce information redundancy, and improve the performance of motion artifact removal. UDDN was validated on a publicly available dataset and a clinical dataset. Qualitative and quantitative experimental results showed that our method could effectively remove motion artifacts and reconstruct image details. Moreover, the performance of UDDN surpasses that of several state-of-the-art unsupervised methods and is comparable with that of the supervised method. Therefore, our method has great potential for clinical application in MRI, such as real-time removal of rigid motion artifacts.

Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges

Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.

Deep learning-based motion quantification from k-space for fast model-based magnetic resonance imaging motion correction

BACKGROUND

Intra-scan rigid-body motion is a costly and ubiquitous problem in clinical magnetic resonance imaging (MRI) of the head.

PURPOSE

State-of-the-art methods for retrospective motion correction in MRI are often computationally expensive or in the case of image-to-image deep learning (DL) based methods can be prone to undesired alterations of the image (hallucinations'). In this work we introduce a novel rigid-body motion correction method which combines the advantages of classical model-driven and data-consistency (DC) preserving approaches with a novel DL algorithm, to provide fast and robust retrospective motion correction.

METHODS

The proposed Motion Parameter Estimating Densenet (MoPED) retrospectively estimates subject head motion during MRI acquisitions using a DL network with DenseBlocks and multitask learning. It quantifies the 2D rigid in-plane motion parameters slice-wise for each echo train (ET) of a Cartesian T2-weighted 2D Turbo-Spin-Echo sequence. The network receives a center patch of the motion corrupted k-space as well as an additional motion-free low-resolution reference scan to provide the ground truth orientation. The supervised training utilizes motion simulations based on 28 acquisitions with subject-wise training, validation, and test data splits of 70%, 23%, and 7%. During inference, MoPED is embedded in an iterative DC-driven motion correction algorithm which alternatingly updates estimates of the motion parameters and motion-corrected low-resolution k-space data. The estimated motion parameters are then used to reconstruct the final motion corrected image. The mean absolute/squared error and the Pearson correlation coefficient were used to analyze the motion parameter estimation quality on in-silico data in a quantitative evaluation. Structural similarity (SSIM), DC error and root mean squared error (RMSE) were used as metrics of image quality improvement. Furthermore, the generalization capability of the network was analyzed on two in-vivo motion volumes with 28 slices each and on one simulated T1-weighted volume.

RESULTS

The motion estimation achieves a Pearson correlation of 0.968 to the simulated ground-truth of the 2433 test data slices used. In-silico results indicate that MoPED decreases the time for the optimization by a factor of around 27 compared to a conventional method and is able to reduce the RMSE of the reconstructions and average DC error by more than a factor of two compared to uncorrected images. In-vivo experiments show a decrease in computation time by a factor of around 20, a RMSE decrease from 0.055 to 0.033 and an SSIM increase from 0.795 to 0.862. Furthermore, contrast independence is demonstrated as MoPED is also able to correct T1-weighted images in simulations without retraining. Due to the model-based correction, no hallucinations were observed.

CONCLUSIONS

Incorporating DL in a model-based motion correction algorithm shows great benefit on the optimization and computation time. The k-space-based estimation also allows a data consistent correction and therefore avoids the risk of hallucinations of image-to-image approaches.

Application and potential of artificial intelligence in neonatal medicine

Neonatal care is becoming increasingly complex with large amounts of rich, routinely recorded physiological, diagnostic and outcome data. Artificial intelligence (AI) has the potential to harness this vast quantity and range of information and become a powerful tool to support clinical decision making, personalised care, precise prognostics, and enhance patient safety. Current AI approaches in neonatal medicine include tools for disease prediction and risk stratification, neurological diagnostic support and novel image recognition technologies. Key to the integration of AI in neonatal medicine is the understanding of its limitations and a standardised critical appraisal of AI tools. Barriers and challenges to this include the quality of datasets used, performance assessment, and appropriate external validation and clinical impact studies. Improving digital literacy amongst healthcare professionals and cross-disciplinary collaborations are needed to harness the full potential of AI to help take the next significant steps in improving neonatal outcomes for high-risk infants.

A deep learning-based multisite neuroimage harmonization framework established with a traveling-subject dataset

The accumulation of multisite large-sample MRI datasets collected during large brain research projects in the last decade has provided critical resources for understanding the neurobiological mechanisms underlying cognitive functions and brain disorders. However, the significant site effects observed in imaging data and their derived structural and functional features have prevented the derivation of consistent findings across multiple studies. The development of harmonization methods that can effectively eliminate complex site effects while maintaining biological characteristics in neuroimaging data has become a vital and urgent requirement for multisite imaging studies. Here, we propose a deep learning-based framework to harmonize imaging data obtained from pairs of sites, in which site factors and brain features can be disentangled and encoded. We trained the proposed framework with a publicly available traveling subject dataset from the Strategic Research Program for Brain Sciences (SRPBS) and harmonized the gray matter volume maps derived from eight source sites to a target site. The proposed framework significantly eliminated intersite differences in gray matter volumes. The embedded encoders successfully captured both the abstract textures of site factors and the concrete brain features. Moreover, the proposed framework exhibited outstanding performance relative to conventional statistical harmonization methods in terms of site effect removal, data distribution homogenization, and intrasubject similarity improvement. Finally, the proposed harmonization network provided fixable expandability, through which new sites could be linked to the target site via indirect schema without retraining the whole model. Together, the proposed method offers a powerful and interpretable deep learning-based harmonization framework for multisite neuroimaging data that can enhance reliability and reproducibility in multisite studies regarding brain development and brain disorders.