3D MRI Reconstruction Based on 2D Generative Adversarial Network Super-Resolution

Towards MR contrast independent synthetic CT generation

The use of synthetic CT (sCT) in the radiotherapy workflow would reduce costs and scan time while removing the uncertainties around working with both MR and CT modalities. The performance of deep learning (DL) solutions for sCT generation is steadily increasing, however most proposed methods were trained and validated on private datasets of a single contrast from a single scanner. Such solutions might not perform equally well on other datasets, limiting their general usability and therefore value. Additionally, functional evaluations of sCTs such as dosimetric comparisons with CT-based dose calculations better show the impact of the methods, but the evaluations are more labor intensive than pixel-wise metrics. To improve the generalization of an sCT model, we propose to incorporate a pre-trained DL model to pre-process the input MR images by generating artificial proton density, T1 and T2 maps (i.e. contrast-independent quantitative maps), which are then used for sCT generation. Using a dataset of only T2w MR images, the robustness towards input MR contrasts of this approach is compared to a model that was trained using the MR images directly. We evaluate the generated sCTs using pixel-wise metrics and calculating mean radiological depths, as an approximation of the mean delivered dose. On T2w images acquired with the same settings as the training dataset, there was no significant difference between the performance of the models. However, when evaluated on T1w images, and a wide range of other contrasts and scanners from both public and private datasets, our approach outperforms the baseline model. Using a dataset of T2w MR images, our proposed model implements synthetic quantitative maps to generate sCT images, improving the generalization towards other contrasts. Our code and trained models are publicly available.

Generalizing the Enhanced-Deep-Super-Resolution Neural Network to Brain MR Images: A Retrospective Study on the Cam-CAN Dataset

The Enhanced-Deep-Super-Resolution (EDSR) model is a state-of-the-art convolutional neural network suitable for improving image spatial resolution. It was previously trained with general-purpose pictures and then, in this work, tested on biomedical magnetic resonance (MR) images, comparing the network outcomes with traditional up-sampling techniques. We explored possible changes in the model response when different MR sequences were analyzed. T1w and T2w MR brain images of 70 human healthy subjects (F:M, 40:30) from the Cambridge Centre for Ageing and Neuroscience (Cam-CAN) repository were down-sampled and then up-sampled using EDSR model and BiCubic (BC) interpolation. Several reference metrics were used to quantitatively assess the performance of up-sampling operations (RMSE, pSNR, SSIM, and HFEN). Two-dimensional and three-dimensional reconstructions were evaluated. Different brain tissues were analyzed individually. The EDSR model was superior to BC interpolation on the selected metrics, both for two- and three- dimensional reconstructions. The reference metrics showed higher quality of EDSR over BC reconstructions for all the analyzed images, with a significant difference of all the criteria in T1w images and of the perception-based SSIM and HFEN in T2w images. The analysis per tissue highlights differences in EDSR performance related to the gray-level values, showing a relative lack of outperformance in reconstructing hyperintense areas. The EDSR model, trained on general-purpose images, better reconstructs MR T1w and T2w images than BC, without any retraining or fine-tuning. These results highlight the excellent generalization ability of the network and lead to possible applications on other MR measurements.

A Systematic Review and Identification of the Challenges of Deep Learning Techniques for Undersampled Magnetic Resonance Image Reconstruction

Deep learning (DL) in magnetic resonance imaging (MRI) shows excellent performance in image reconstruction from undersampled k-space data. Artifact-free and high-quality MRI reconstruction is essential for ensuring accurate diagnosis, supporting clinical decision-making, enhancing patient safety, facilitating efficient workflows, and contributing to the validity of research studies and clinical trials. Recently, deep learning has demonstrated several advantages over conventional MRI reconstruction methods. Conventional methods rely on manual feature engineering to capture complex patterns and are usually computationally demanding due to their iterative nature. Conversely, DL methods use neural networks with hundreds of thousands of parameters and automatically learn relevant features and representations directly from the data. Nevertheless, there are some limitations to DL-based techniques concerning MRI reconstruction tasks, such as the need for large, labeled datasets, the possibility of overfitting, and the complexity of model training. Researchers are striving to develop DL models that are more efficient, adaptable, and capable of providing valuable information for medical practitioners. We provide a comprehensive overview of the current developments and clinical uses by focusing on state-of-the-art DL architectures and tools used in MRI reconstruction. This study has three objectives. Our main objective is to describe how various DL designs have changed over time and talk about cutting-edge tactics, including their advantages and disadvantages. Hence, data pre- and post-processing approaches are assessed using publicly available MRI datasets and source codes. Secondly, this work aims to provide an extensive overview of the ongoing research on transformers and deep convolutional neural networks for rapid MRI reconstruction. Thirdly, we discuss several network training strategies, like supervised, unsupervised, transfer learning, and federated learning for rapid and efficient MRI reconstruction. Consequently, this article provides significant resources for future improvement of MRI data pre-processing and fast image reconstruction.

Deep learning-based prediction of the retinal structural alterations after epiretinal membrane surgery

To generate and evaluate synthesized postoperative OCT images of epiretinal membrane (ERM) based on preoperative OCT images using deep learning methodology. This study included a total 500 pairs of preoperative and postoperative optical coherence tomography (OCT) images for training a neural network. 60 preoperative OCT images were used to test the neural networks performance, and the corresponding postoperative OCT images were used to evaluate the synthesized images in terms of structural similarity index measure (SSIM). The SSIM was used to quantify how similar the synthesized postoperative OCT image was to the actual postoperative OCT image. The Pix2Pix GAN model was used to generate synthesized postoperative OCT images. Total 60 synthesized OCT images were generated with training values at 800 epochs. The mean SSIM of synthesized postoperative OCT to the actual postoperative OCT was 0.913. Pix2Pix GAN model has a possibility to generate predictive postoperative OCT images following ERM removal surgery.

Improving the diagnostic performance of computed tomography angiography for intracranial large arterial stenosis by a novel super-resolution algorithm based on multi-scale residual denoising generative adversarial network

BACKGROUND

Computed tomography angiography (CTA) is very popular because it is characterized by rapidity and accessibility. However, CTA is inferior to digital subtraction angiography (DSA) in the diagnosis of intracranial artery stenosis or occlusion. DSA is an invasive examination, so we optimized the quality of cephalic CTA images.

METHODS

We used 5000 CTA images to train multi-scale residual denoising generative adversarial network (MRDGAN). And then 71 CTA images with intracranial large arterial stenosis were treated by Super-Resolution based on Generative Adversarial Network (SRGAN), Enhanced Super-Resolution based on Generative Adversarial Network (ESRGAN) and post-trained MRDGAN, respectively. Peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM) of the SRGAN, ESRGAN, MRDGAN and original CTA images were measured respectively. The qualities of MRDGAN and original images were visually assessed using a 4-point scale. The diagnostic coherence of digital subtraction angiography (DSA) with MRDGAN and original images was analyzed.

RESULTS

The PSNR was significantly higher in the MRDGAN CTA images (35.96 ± 1.51) than in the original (31.51 ± 1.43), SRGAN (25.75 ± 1.18) and ESRGAN (30.36 ± 1.05) CTA images (all P < 0.001). The SSIM was significantly higher in the MRDGAN CTA images (0.95 ± 0.02) than in the SRGAN (0.88 ± 0.03) and ESRGAN (0.90 ± 0.02) CTA images (all P < 0.01). The visual assessment was significantly higher in the MRDGAN CTA images (3.52 ± 0.58) than in the original CTA images (2.39 ± 0.69) (P < 0.05). The diagnostic coherence between MRDGAN and DSA (κ = 0.89) was superior to that between original images and DSA (κ = 0.62).

CONCLUSION

Our MRDGAN can effectively optimize original CTA images and improve its clinical diagnostic value for intracranial large artery stenosis.

Super-resolution application of generative adversarial network on brain time-of-flight MR angiography: image quality and diagnostic utility evaluation

OBJECTIVES

To develop a generative adversarial network (GAN) model to improve image resolution of brain time-of-flight MR angiography (TOF-MRA) and to evaluate the image quality and diagnostic utility of the reconstructed images.

METHODS

We included 180 patients who underwent 1-min low-resolution (LR) and 4-min high-resolution (routine) brain TOF-MRA scans. We used 50 patients' datasets for training, 12 for quantitative image quality evaluation, and the rest for diagnostic validation. We modified a pix2pix GAN to suit TOF-MRA datasets and fine-tuned GAN-related parameters, including loss functions. Maximum intensity projection images were generated and compared using multi-scale structural similarity (MS-SSIM) and information theoretic-based statistic similarity measure (ISSM) index. Two radiologists scored vessels' visibilities using a 5-point Likert scale. Finally, we evaluated sensitivities and specificities of GAN-MRA in depicting aneurysms, stenoses, and occlusions.

RESULTS

The optimal model was achieved with a lambda of 1e5 and L1 + MS-SSIM loss. Image quality metrics for GAN-MRA were higher than those for LR-MRA (MS-SSIM, 0.87 vs. 0.73; ISSM, 0.60 vs. 0.35; p.adjusted < 0.001). Vessels' visibility of GAN-MRA was superior to LR-MRA (rater A, 4.18 vs. 2.53; rater B, 4.61 vs. 2.65; p.adjusted < 0.001). In depicting vascular abnormalities, GAN-MRA showed comparable sensitivities and specificities, with greater sensitivity for aneurysm detection by one rater (93% vs. 84%, p < 0.05).

CONCLUSIONS

An optimized GAN could significantly improve the image quality and vessel visibility of low-resolution brain TOF-MRA with equivalent sensitivity and specificity in detecting aneurysms, stenoses, and occlusions.

KEY POINTS

• GAN could significantly improve the image quality and vessel visualization of low-resolution brain MR angiography (MRA). • With optimally adjusted training parameters, the GAN model did not degrade diagnostic performance by generating substantial false positives or false negatives. • GAN could be a promising approach for obtaining higher resolution TOF-MRA from images scanned in a fraction of time.

A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI

With the advances in brain imaging, magnetic resonance imaging (MRI) is evolving as a popular radiological tool in clinical diagnosis. Deep learning (DL) methods can detect abnormalities in brain images without an extensive manual feature extraction process. Generative adversarial network (GAN)-synthesized images have many applications in this field besides augmentation, such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. The existing literature was reviewed systematically to understand the role of GAN-synthesized dummy images in brain disease diagnosis. Web of Science and Scopus databases were extensively searched to find relevant studies from the last 6 years to write this systematic literature review (SLR). Predefined inclusion and exclusion criteria helped in filtering the search results. Data extraction is based on related research questions (RQ). This SLR identifies various loss functions used in the above applications and software to process brain MRIs. A comparative study of existing evaluation metrics for GAN-synthesized images helps choose the proper metric for an application. GAN-synthesized images will have a crucial role in the clinical sector in the coming years, and this paper gives a baseline for other researchers in the field.

The role of generative adversarial networks in brain MRI: a scoping review

The performance of artificial intelligence (AI) for brain MRI can improve if enough data are made available. Generative adversarial networks (GANs) showed a lot of potential to generate synthetic MRI data that can capture the distribution of real MRI. Besides, GANs are also popular for segmentation, noise removal, and super-resolution of brain MRI images. This scoping review aims to explore how GANs methods are being used on brain MRI data, as reported in the literature. The review describes the different applications of GANs for brain MRI, presents the most commonly used GANs architectures, and summarizes the publicly available brain MRI datasets for advancing the research and development of GANs-based approaches. This review followed the guidelines of PRISMA-ScR to perform the study search and selection. The search was conducted on five popular scientific databases. The screening and selection of studies were performed by two independent reviewers, followed by validation by a third reviewer. Finally, the data were synthesized using a narrative approach. This review included 139 studies out of 789 search results. The most common use case of GANs was the synthesis of brain MRI images for data augmentation. GANs were also used to segment brain tumors and translate healthy images to diseased images or CT to MRI and vice versa. The included studies showed that GANs could enhance the performance of AI methods used on brain MRI imaging data. However, more efforts are needed to transform the GANs-based methods in clinical applications.

Generative Adversarial Networks in Brain Imaging: A Narrative Review

Artificial intelligence (AI) is expected to have a major effect on radiology as it demonstrated remarkable progress in many clinical tasks, mostly regarding the detection, segmentation, classification, monitoring, and prediction of diseases. Generative Adversarial Networks have been proposed as one of the most exciting applications of deep learning in radiology. GANs are a new approach to deep learning that leverages adversarial learning to tackle a wide array of computer vision challenges. Brain radiology was one of the first fields where GANs found their application. In neuroradiology, indeed, GANs open unexplored scenarios, allowing new processes such as image-to-image and cross-modality synthesis, image reconstruction, image segmentation, image synthesis, data augmentation, disease progression models, and brain decoding. In this narrative review, we will provide an introduction to GANs in brain imaging, discussing the clinical potential of GANs, future clinical applications, as well as pitfalls that radiologists should be aware of.