Improving Structural MRI Preprocessing with Hybrid Transformer GANs

Multi-tailed vision transformer for efficient inference

Recently, Vision Transformer (ViT) has achieved promising performance in image recognition and gradually serves as a powerful backbone in various vision tasks. To satisfy the sequential input of Transformer, the tail of ViT first splits each image into a sequence of visual tokens with a fixed length. Then, the following self-attention layers construct the global relationship between tokens to produce useful representation for the downstream tasks. Empirically, representing the image with more tokens leads to better performance, yet the quadratic computational complexity of self-attention layer to the number of tokens could seriously influence the efficiency of ViT's inference. For computational reduction, a few pruning methods progressively prune uninformative tokens in the Transformer encoder, while leaving the number of tokens before the Transformer untouched. In fact, fewer tokens as the input for the Transformer encoder can directly reduce the following computational cost. In this spirit, we propose a Multi-Tailed Vision Transformer (MT-ViT) in the paper. MT-ViT adopts multiple tails to produce visual sequences of different lengths for the following Transformer encoder. A tail predictor is introduced to decide which tail is the most efficient for the image to produce accurate prediction. Both modules are optimized in an end-to-end fashion, with the Gumbel-Softmax trick. Experiments on ImageNet-1K demonstrate that MT-ViT can achieve a significant reduction on FLOPs with no degradation of the accuracy and outperform compared methods in both accuracy and FLOPs.

Speeding Up and Improving Image Quality in Glioblastoma MRI Protocol by Deep Learning Image Reconstruction

A fully diagnostic MRI glioma protocol is key to monitoring therapy assessment but is time-consuming and especially challenging in critically ill and uncooperative patients. Artificial intelligence demonstrated promise in reducing scan time and improving image quality simultaneously. The purpose of this study was to investigate the diagnostic performance, the impact on acquisition acceleration, and the image quality of a deep learning optimized glioma protocol of the brain. Thirty-three patients with histologically confirmed glioblastoma underwent standardized brain tumor imaging according to the glioma consensus recommendations on a 3-Tesla MRI scanner. Conventional and deep learning-reconstructed (DLR) fluid-attenuated inversion recovery, and T2- and T1-weighted contrast-enhanced Turbo spin echo images with an improved in-plane resolution, i.e., super-resolution, were acquired. Two experienced neuroradiologists independently evaluated the image datasets for subjective image quality, diagnostic confidence, tumor conspicuity, noise levels, artifacts, and sharpness. In addition, the tumor volume was measured in the image datasets according to Response Assessment in Neuro-Oncology (RANO) 2.0, as well as compared between both imaging techniques, and various clinical-pathological parameters were determined. The average time saving of DLR sequences was 30% per MRI sequence. Simultaneously, DLR sequences showed superior overall image quality (all p < 0.001), improved tumor conspicuity and image sharpness (all p < 0.001, respectively), and less image noise (all p < 0.001), while maintaining diagnostic confidence (all p > 0.05), compared to conventional images. Regarding RANO 2.0, the volume of non-enhancing non-target lesions (p = 0.963), enhancing target lesions (p = 0.993), and enhancing non-target lesions (p = 0.951) did not differ between reconstruction types. The feasibility of the deep learning-optimized glioma protocol was demonstrated with a 30% reduction in acquisition time on average and an increased in-plane resolution. The evaluated DLR sequences improved subjective image quality and maintained diagnostic accuracy in tumor detection and tumor classification according to RANO 2.0.

Positive Effect of Super-Resolved Structural Magnetic Resonance Imaging for Mild Cognitive Impairment Detection

This paper presents a novel approach to improving the detection of mild cognitive impairment (MCI) through the use of super-resolved structural magnetic resonance imaging (MRI) and optimized deep learning models. The study introduces enhancements to the perceptual quality of super-resolved 2D structural MRI images using advanced loss functions, modifications to the upscaler part of the generator, and experiments with various discriminators within a generative adversarial training setting. It empirically demonstrates the effectiveness of super-resolution in the MCI detection task, showcasing performance improvements across different state-of-the-art classification models. The paper also addresses the challenge of accurately capturing perceptual image quality, particularly when images contain checkerboard artifacts, and proposes a methodology that incorporates hyperparameter optimization through a Pareto optimal Markov blanket (POMB). This approach systematically explores the hyperparameter space, focusing on reducing overfitting and enhancing model generalizability. The research findings contribute to the field by demonstrating that super-resolution can significantly improve the quality of MRI images for MCI detection, highlighting the importance of choosing an adequate discriminator and the potential of super-resolution as a preprocessing step to boost classification model performance.

Arbitrary scale super-resolution diffusion model for brain MRI images

Given the constraints posed by hardware capacity, scan duration, and patient cooperation, the reconstruction of magnetic resonance imaging (MRI) images emerges as a pivotal aspect of medical imaging research. Currently, deep learning-based super-resolution (SR) methods have been widely discussed in medical image processing due to their ability to reconstruct high-quality, high resolution (HR) images from low resolution (LR) inputs. However, most existing MRI SR methods are designed for specific magnifications and cannot generate MRI images at arbitrary scales, which hinders the radiologists from fully visualizing the lesions. Moreover, current arbitrary scale SR methods often suffer from issues like excessive smoothing and artifacts. In this paper, we propose an Arbitrary Scale Super-Resolution Diffusion Model (ASSRDM), which combines implicit neural representation with the denoising diffusion probabilistic model to achieve arbitrary-scale, high-fidelity medical images SR. Moreover, we formulate a continuous resolution regulation mechanism, comprising a multi-scale LR guidance network and a scaling factor. The scaling factor finely adjusts the resolution and dynamically influences the weighting of LR details and synthesized features in the final output. This capability allows the model to seamlessly adapt to the requirements of continuous resolution adjustments. Additionally, the multi-scale LR guidance network provides the denoising block with multi-resolution LR features to enrich texture information and restore high-frequency details. Extensive experiments conducted on the IXI and fastMRI datasets demonstrate that our ASSRDM exhibits superior performance compared to existing techniques and has tremendous potential in clinical practice.