Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging: Combining physics and machine learning for improved medical imaging

Self-supervised learning for improved calibrationless radial MRI with NLINV-Net

PURPOSE

To develop a neural network architecture for improved calibrationless reconstruction of radial data when no ground truth is available for training.

METHODS

NLINV-Net is a model-based neural network architecture that directly estimates images and coil sensitivities from (radial) k-space data via nonlinear inversion (NLINV). Combined with a training strategy using self-supervision via data undersampling (SSDU), it can be used for imaging problems where no ground truth reconstructions are available. We validated the method for (1) real-time cardiac imaging and (2) single-shot subspace-based quantitative T1 mapping. Furthermore, region-optimized virtual (ROVir) coils were used to suppress artifacts stemming from outside the field of view and to focus the k-space-based SSDU loss on the region of interest. NLINV-Net-based reconstructions were compared with conventional NLINV and PI-CS (parallel imaging + compressed sensing) reconstruction and the effect of the region-optimized virtual coils and the type of training loss was evaluated qualitatively.

RESULTS

NLINV-Net-based reconstructions contain significantly less noise than the NLINV-based counterpart. ROVir coils effectively suppress streakings which are not suppressed by the neural networks while the ROVir-based focused loss leads to visually sharper time series for the movement of the myocardial wall in cardiac real-time imaging. For quantitative imaging, T1-maps reconstructed using NLINV-Net show similar quality as PI-CS reconstructions, but NLINV-Net does not require slice-specific tuning of the regularization parameter.

CONCLUSION

NLINV-Net is a versatile tool for calibrationless imaging which can be used in challenging imaging scenarios where a ground truth is not available.

Accelerated cardiac cine with spatio-coil regularized deep learning reconstruction

PURPOSE

To develop an iterative deep learning (DL) reconstruction with spatio-coil regularization and multichannel k-space data consistency for accelerated cine imaging.

METHODS

This study proposes a Spatio-Coil Regularized DL (SCR-DL) approach for iterative deep learning reconstruction incorporating multicoil information in data consistency and regularizer. SCR-DL uses shift-invariant convolutional kernels to interpolate missing k-space lines and reconstruct individual coil images, followed by a regularizer that operates simultaneously across spatial and coil dimensions using learned image priors. At 8-fold acceleration, SCR-DL was compared with Generalized Autocalibrating Partially Parallel Acquisition (GRAPPA), sensitivity encoding (SENSE)-based DL and spatio-temporal regularized (STR)-DL reconstruction. In the retrospective undersampled cine, images were quantitatively evaluated using normalized mean square error (NMSE) and structural similarity index measure (SSIM). Additionally, agreement for left-ventricular ejection fraction and left-ventricular mass were assessed using prospectively accelerated cine images at 2-fold and 8-fold accelerations.

RESULTS

The SCR-DL algorithm successfully reconstructed highly accelerated cine images. SCR-DL had significant improvements in NMSE (0.03 ± 0.02) and SSIM (91.4% ± 2.7%) compared with GRAPPA (NMSE: 0.09 ± 0.04, SSIM: 69.9% ± 11.1%; p < 0.001), SENSE-DL (NMSE: 0.07 ± 0.04, SSIM: 86.9% ± 3.2%; p < 0.001), and STR-DL (NMSE: 0.04 ± 0.03, SSIM: 90.0% ± 2.5%; p < 0.001) with retrospective undersampled cine. Despite the 3-fold reduction in scan time, there was no difference between left-ventricular ejection fraction (59.8 ± 4.5 vs. 60.8 ± 4.8, p = 0.46) or left-ventricular mass (73.6 ± 19.4 g vs. 73.2 ± 19.7 g, p = 0.95) between R = 2 and R = 8 prospectively accelerated cine images.

CONCLUSIONS

SCR-DL enabled highly accelerated cardiac cine imaging, significantly reducing breath-hold time. Compared with GRAPPA or SENSE-DL, images reconstructed with SCR-DL showed superior NMSE and SSIM.

Reducing energy consumption in musculoskeletal MRI using shorter scan protocols, optimized magnet cooling patterns, and deep learning sequences

OBJECTIVES

The unprecedented surge in energy costs in Europe, coupled with the significant energy consumption of MRI scanners in radiology departments, necessitates exploring strategies to optimize energy usage without compromising efficiency or image quality. This study investigates MR energy consumption and identifies strategies for improving energy efficiency, focusing on musculoskeletal MRI. We assess the potential savings achievable through (1) optimizing protocols, (2) incorporating deep learning (DL) accelerated acquisitions, and (3) optimizing the cooling system.

MATERIALS AND METHODS

Energy consumption measurements were performed on two MRI scanners (1.5-T Aera, 1.5-T Sola) in practices in Munich, Germany, between December 2022 and March 2023. Three levels of energy reduction measures were implemented and compared to the baseline. Wilcoxon signed-rank test with Bonferroni correction was conducted to evaluate the impact of sequence scan times and energy consumption.

RESULTS

Our findings showed significant energy savings by optimizing protocol settings and implementing DL technologies. Across all body regions, the average reduction in energy consumption was 72% with DL and 31% with economic protocols, accompanied by time reductions of 71% (DL) and 18% (economic protocols) compared to baseline. Optimizing the cooling system during the non-scanning time showed a 30% lower energy consumption.

CONCLUSION

Implementing energy-saving strategies, including economic protocols, DL accelerated sequences, and optimized magnet cooling, can significantly reduce energy consumption in MRI scanners. Radiology departments and practices should consider adopting these strategies to improve energy efficiency and reduce costs.

CLINICAL RELEVANCE STATEMENT

MRI scanner energy consumption can be substantially reduced by incorporating protocol optimization, DL accelerated acquisition, and optimized magnetic cooling into daily practice, thereby cutting costs and environmental impact.

KEY POINTS

Optimization of protocol settings reduced energy consumption by 31% and imaging time by 18%. DL technologies led to a 72% reduction in energy consumption of and a 71% reduction in time, compared to the standard MRI protocol. During non-scanning times, activating Eco power mode (EPM) resulted in a 30% reduction in energy consumption, saving 4881 € ($5287) per scanner annually.

Enhancing global sensitivity and uncertainty quantification in medical image reconstruction with Monte Carlo arbitrary-masked mamba

Deep learning has been extensively applied in medical image reconstruction, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) represent the predominant paradigms, each possessing distinct advantages and inherent limitations: CNNs exhibit linear complexity with local sensitivity, whereas ViTs demonstrate quadratic complexity with global sensitivity. The emerging Mamba has shown superiority in learning visual representation, which combines the advantages of linear scalability and global sensitivity. In this study, we introduce MambaMIR, an Arbitrary-Masked Mamba-based model with wavelet decomposition for joint medical image reconstruction and uncertainty estimation. A novel Arbitrary Scan Masking (ASM) mechanism "masks out" redundant information to introduce randomness for further uncertainty estimation. Compared to the commonly used Monte Carlo (MC) dropout, our proposed MC-ASM provides an uncertainty map without the need for hyperparameter tuning and mitigates the performance drop typically observed when applying dropout to low-level tasks. For further texture preservation and better perceptual quality, we employ the wavelet transformation into MambaMIR and explore its variant based on the Generative Adversarial Network, namely MambaMIR-GAN. Comprehensive experiments have been conducted for multiple representative medical image reconstruction tasks, demonstrating that the proposed MambaMIR and MambaMIR-GAN outperform other baseline and state-of-the-art methods in different reconstruction tasks, where MambaMIR achieves the best reconstruction fidelity and MambaMIR-GAN has the best perceptual quality. In addition, our MC-ASM provides uncertainty maps as an additional tool for clinicians, while mitigating the typical performance drop caused by the commonly used dropout.

Deep learning for accelerated and robust MRI reconstruction

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.

Large-scale 3D non-Cartesian coronary MRI reconstruction using distributed memory-efficient physics-guided deep learning with limited training data

OBJECT

To enable high-quality physics-guided deep learning (PG-DL) reconstruction of large-scale 3D non-Cartesian coronary MRI by overcoming challenges of hardware limitations and limited training data availability.

MATERIALS AND METHODS

While PG-DL has emerged as a powerful image reconstruction method, its application to large-scale 3D non-Cartesian MRI is hindered by hardware limitations and limited availability of training data. We combine several recent advances in deep learning and MRI reconstruction to tackle the former challenge, and we further propose a 2.5D reconstruction using 2D convolutional neural networks, which treat 3D volumes as batches of 2D images to train the network with a limited amount of training data. Both 3D and 2.5D variants of the PG-DL networks were compared to conventional methods for high-resolution 3D kooshball coronary MRI.

RESULTS

Proposed PG-DL reconstructions of 3D non-Cartesian coronary MRI with 3D and 2.5D processing outperformed all conventional methods both quantitatively and qualitatively in terms of image assessment by an experienced cardiologist. The 2.5D variant further improved vessel sharpness compared to 3D processing, and scored higher in terms of qualitative image quality.

DISCUSSION

PG-DL reconstruction of large-scale 3D non-Cartesian MRI without compromising image size or network complexity is achieved, and the proposed 2.5D processing enables high-quality reconstruction with limited training data.

Accelerated MRI reconstructions via variational network and feature domain learning

We introduce three architecture modifications to enhance the performance of the end-to-end (E2E) variational network (VarNet) for undersampled MRI reconstructions. We first implemented the Feature VarNet, which propagates information throughout the cascades of the network in an N-channel feature-space instead of a 2-channel feature-space. Then, we add an attention layer that utilizes the spatial locations of Cartesian undersampling artifacts to further improve performance. Lastly, we combined the Feature and E2E VarNets into the Feature-Image (FI) VarNet, to facilitate cross-domain learning and boost accuracy. Reconstructions were evaluated on the fastMRI dataset using standard metrics and clinical scoring by three neuroradiologists. Feature and FI VarNets outperformed the E2E VarNet for 4 × , 5 × and 8 × Cartesian undersampling in all studied metrics. FI VarNet secured second place in the public fastMRI leaderboard for 4 × Cartesian undersampling, outperforming all open-source models in the leaderboard. Radiologists rated FI VarNet brain reconstructions with higher quality and sharpness than the E2E VarNet reconstructions. FI VarNet excelled in preserving anatomical details, including blood vessels, whereas E2E VarNet discarded or blurred them in some cases. The proposed FI VarNet enhances the reconstruction quality of undersampled MRI and could enable clinically acceptable reconstructions at higher acceleration factors than currently possible.

McSTRA: A multi-branch cascaded swin transformer for point spread function-guided robust MRI reconstruction

Deep learning MRI reconstruction methods are often based on Convolutional neural network (CNN) models; however, they are limited in capturing global correlations among image features due to the intrinsic locality of the convolution operation. Conversely, the recent vision transformer models (ViT) are capable of capturing global correlations by applying self-attention operations on image patches. Nevertheless, the existing transformer models for MRI reconstruction rarely leverage the physics of MRI. In this paper, we propose a novel physics-based transformer model titled, the Multi-branch Cascaded Swin Transformers (McSTRA) for robust MRI reconstruction. McSTRA combines several interconnected MRI physics-related concepts with the Swin transformers: it exploits global MRI features via the shifted window self-attention mechanism; it extracts MRI features belonging to different spectral components via a multi-branch setup; it iterates between intermediate de-aliasing and data consistency via a cascaded network with intermediate loss computations; furthermore, we propose a point spread function-guided positional embedding generation mechanism for the Swin transformers which exploit the spread of the aliasing artifacts for effective reconstruction. With the combination of all these components, McSTRA outperforms the state-of-the-art methods while demonstrating robustness in adversarial conditions such as higher accelerations, noisy data, different undersampling protocols, out-of-distribution data, and abnormalities in anatomy.

A Theoretical Framework for Self-Supervised MR Image Reconstruction Using Sub-Sampling via Variable Density Noisier2Noise

In recent years, there has been attention on leveraging the statistical modeling capabilities of neural networks for reconstructing sub-sampled Magnetic Resonance Imaging (MRI) data. Most proposed methods assume the existence of a representative fully-sampled dataset and use fully-supervised training. However, for many applications, fully sampled training data is not available, and may be highly impractical to acquire. The development and understanding of self-supervised methods, which use only sub-sampled data for training, are therefore highly desirable. This work extends the Noisier2Noise framework, which was originally constructed for self-supervised denoising tasks, to variable density sub-sampled MRI data. We use the Noisier2Noise framework to analytically explain the performance of Self-Supervised Learning via Data Undersampling (SSDU), a recently proposed method that performs well in practice but until now lacked theoretical justification. Further, we propose two modifications of SSDU that arise as a consequence of the theoretical developments. Firstly, we propose partitioning the sampling set so that the subsets have the same type of distribution as the original sampling mask. Secondly, we propose a loss weighting that compensates for the sampling and partitioning densities. On the fastMRI dataset we show that these changes significantly improve SSDU's image restoration quality and robustness to the partitioning parameters.