Learning Data Consistency and its Application to Dynamic MR Imaging

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Integrating data distribution prior via Langevin dynamics for end-to-end MR reconstruction

PURPOSE

To develop a novel deep learning-based method inheriting the advantages of data distribution prior and end-to-end training for accelerating MRI.

METHODS

Langevin dynamics is used to formulate image reconstruction with data distribution before facilitate image reconstruction. The data distribution prior is learned implicitly through the end-to-end adversarial training to mitigate the hyper-parameter selection and shorten the testing time compared to traditional probabilistic reconstruction. By seamlessly integrating the deep equilibrium model, the iteration of Langevin dynamics culminates in convergence to a fix-point, ensuring the stability of the learned distribution.

RESULTS

The feasibility of the proposed method is evaluated on the brain and knee datasets. Retrospective results with uniform and random masks show that the proposed method demonstrates superior performance both quantitatively and qualitatively than the state-of-the-art.

CONCLUSION

The proposed method incorporating Langevin dynamics with end-to-end adversarial training facilitates efficient and robust reconstruction for MRI. Empirical evaluations conducted on brain and knee datasets compellingly demonstrate the superior performance of the proposed method in terms of artifact removing and detail preserving.

Accelerated MRI reconstructions via variational network and feature domain learning

We introduce three architecture modifications to enhance the performance of the end-to-end (E2E) variational network (VarNet) for undersampled MRI reconstructions. We first implemented the Feature VarNet, which propagates information throughout the cascades of the network in an N-channel feature-space instead of a 2-channel feature-space. Then, we add an attention layer that utilizes the spatial locations of Cartesian undersampling artifacts to further improve performance. Lastly, we combined the Feature and E2E VarNets into the Feature-Image (FI) VarNet, to facilitate cross-domain learning and boost accuracy. Reconstructions were evaluated on the fastMRI dataset using standard metrics and clinical scoring by three neuroradiologists. Feature and FI VarNets outperformed the E2E VarNet for 4 × , 5 × and 8 × Cartesian undersampling in all studied metrics. FI VarNet secured second place in the public fastMRI leaderboard for 4 × Cartesian undersampling, outperforming all open-source models in the leaderboard. Radiologists rated FI VarNet brain reconstructions with higher quality and sharpness than the E2E VarNet reconstructions. FI VarNet excelled in preserving anatomical details, including blood vessels, whereas E2E VarNet discarded or blurred them in some cases. The proposed FI VarNet enhances the reconstruction quality of undersampled MRI and could enable clinically acceptable reconstructions at higher acceleration factors than currently possible.

Accelerated submillimeter wave-encoded magnetic resonance imaging via deep untrained neural network

BACKGROUND

Wave gradient encoding can adequately utilize coil sensitivity profiles to facilitate higher accelerations in parallel magnetic resonance imaging (pMRI). However, there are limitations in mainstream pMRI and a few deep learning (DL) methods for recovering missing data under wave encoding framework: the former is prone to introduce errors from the auto-calibration signals (ACS) signal acquisition and is time-consuming, while the latter requires a large amount of training data.

PURPOSE

To tackle the above issues, an untrained neural network (UNN) model incorporating wave-encoded physical properties and deep generative model, named WDGM, was proposed with additional ACS- and training data-free.

METHODS

Generally, the proposed method can provide powerful missing data interpolation capability using the wave physical encoding framework and designed UNN to characterize the MR image (k-space data) priors. Specifically, the MRI reconstruction combining physical wave encoding and elaborate UNN is modeled as a generalized minimization problem. The designation of UNN is driven by the coil sensitivity maps (CSM) smoothness and k-space linear predictability. And then, the iterative paradigm to recover the full k-space signal is determined by the projected gradient descent, and the complex computation is unrolled to the network with optimized parameters by the optimizer. Simulated wave encoding and in vivo experiments are exploited to demonstrate the feasibility of the proposed method. The best quantitative metrics RMSE/SSIM/PSNR of 0.0413, 0.9514, and 37.4862 gave competitive results in all experiments with at least six-fold acceleration, respectively.

RESULTS

In vivo experiments of human brains and knees showed that the proposed method can achieve comparable reconstruction quality and even has superiority relative to the comparison, especially at a high resolution of 0.67 mm and fewer ACS. In addition, the proposed method has a higher computational efficiency achieving a computation time of 9.6 s/per slice.

CONCLUSIONS

The model proposed in this work addresses two limitations of MRI reconstruction in the wave encoding framework. The first is to eliminate the need for ACS signal acquisition to perform the time-consuming calibration process and to avoid errors such as motion during the acquisition procedure. Furthermore, the proposed method has clinical application friendly without the need to prepare large training datasets, which is difficult in the clinical. All results of the proposed method demonstrate more confidence in both quantitative and qualitative metrics. In addition, the proposed method can achieve higher computational efficiency.

Equilibrated Zeroth-Order Unrolled Deep Network for Parallel MR Imaging

In recent times, model-driven deep learning has evolved an iterative algorithm into a cascade network by replacing the regularizer's first-order information, such as the (sub)gradient or proximal operator, with a network module. This approach offers greater explainability and predictability compared to typical data-driven networks. However, in theory, there is no assurance that a functional regularizer exists whose first-order information matches the substituted network module. This implies that the unrolled network output may not align with the regularization models. Furthermore, there are few established theories that guarantee global convergence and robustness (regularity) of unrolled networks under practical assumptions. To address this gap, we propose a safeguarded methodology for network unrolling. Specifically, for parallel MR imaging, we unroll a zeroth-order algorithm, where the network module serves as a regularizer itself, allowing the network output to be covered by a regularization model. Additionally, inspired by deep equilibrium models, we conduct the unrolled network before backpropagation to converge to a fixed point and then demonstrate that it can tightly approximate the actual MR image. We also prove that the proposed network is robust against noisy interferences if the measurement data contain noise. Finally, numerical experiments indicate that the proposed network consistently outperforms state-of-the-art MRI reconstruction methods, including traditional regularization and unrolled deep learning techniques.

Complexities of deep learning-based undersampled MR image reconstruction

Artificial intelligence has opened a new path of innovation in magnetic resonance (MR) image reconstruction of undersampled k-space acquisitions. This review offers readers an analysis of the current deep learning-based MR image reconstruction methods. The literature in this field shows exponential growth, both in volume and complexity, as the capabilities of machine learning in solving inverse problems such as image reconstruction are explored. We review the latest developments, aiming to assist researchers and radiologists who are developing new methods or seeking to provide valuable feedback. We shed light on key concepts by exploring the technical intricacies of MR image reconstruction, highlighting the importance of raw datasets and the difficulty of evaluating diagnostic value using standard metrics.Relevance statement Increasingly complex algorithms output reconstructed images that are difficult to assess for robustness and diagnostic quality, necessitating high-quality datasets and collaboration with radiologists.Key points• Deep learning-based image reconstruction algorithms are increasing both in complexity and performance.• The evaluation of reconstructed images may mistake perceived image quality for diagnostic value.• Collaboration with radiologists is crucial for advancing deep learning technology.

Joint Image Reconstruction and Super-Resolution for Accelerated Magnetic Resonance Imaging

Magnetic resonance (MR) image reconstruction and super-resolution are two prominent techniques to restore high-quality images from undersampled or low-resolution k-space data to accelerate MR imaging. Combining undersampled and low-resolution acquisition can further improve the acceleration factor. Existing methods often treat the techniques of image reconstruction and super-resolution separately or combine them sequentially for image recovery, which can result in error propagation and suboptimal results. In this work, we propose a novel framework for joint image reconstruction and super-resolution, aiming to efficiently image recovery and enable fast imaging. Specifically, we designed a framework with a reconstruction module and a super-resolution module to formulate multi-task learning. The reconstruction module utilizes a model-based optimization approach, ensuring data fidelity with the acquired k-space data. Moreover, a deep spatial feature transform is employed to enhance the information transition between the two modules, facilitating better integration of image reconstruction and super-resolution. Experimental evaluations on two datasets demonstrate that our proposed method can provide superior performance both quantitatively and qualitatively.

Accelerated dynamic MR imaging with joint balanced low-rank tensor and sparsity constraints

BACKGROUND

Dynamic magnetic resonance imaging (DMRI) is an essential medical imaging technique, but the slow data acquisition process limits its further development.

PURPOSE

By exploiting the inherent spatio-temporal correlation of MR images, low-rank tensor based methods have been developed to accelerate imaging. However, the tensor rank used by these methods is defined by an unbalanced matricization scheme, which cannot capture the global correlation of DMR data efficiently during the reconstruction process.

METHODS

In this paper, an effective reconstruction model is proposed to achieve accurate reconstruction by using the tensor train (TT) rank defined by a well-balanced matricization scheme to exploit the hidden correlation of DMR data and combining sparsity. Meanwhile, the ket augmentation (KA) technology is introduced to preprocess the DMR data into a higher-order tensor through block structure addressing, which further improves the ability of TT rank to explore the local information of the image. In order to solve the proposed model, the alternating direction method of multipliers (ADMM) is used to decompose the optimization problem into several unconstrained subproblems.

RESULTS

The performance of the proposed method was validated on the 3D DMR image dataset by using different sampling trajectories and sampling rates. Extensive numerical experiments demonstrate that the reconstruction quality of the proposed method is superior to several state-of-the-art reconstruction methods.

CONCLUSIONS

The proposed method successfully utilizes the TT rank to explore the global correlation of DMR images, enabling more detailed information of the image to be captured. Besides, with the sparse priors, the proposed method can further improve the overall reconstruction quality for highly undersampled MR images.

Artificial intelligence and radiomics in magnetic resonance imaging of rectal cancer: a review

Rectal cancer (RC) is one of the most common tumours worldwide in both males and females, with significant morbidity and mortality rates, and it accounts for approximately one-third of colorectal cancers (CRCs). Magnetic resonance imaging (MRI) has been demonstrated to be accurate in evaluating the tumour location and stage, mucin content, invasion depth, lymph node (LN) metastasis, extramural vascular invasion (EMVI), and involvement of the mesorectal fascia (MRF). However, these features alone remain insufficient to precisely guide treatment decisions. Therefore, new imaging biomarkers are necessary to define tumour characteristics for staging and restaging patients with RC. During the last decades, RC evaluation via MRI-based radiomics and artificial intelligence (AI) tools has been a research hotspot. The aim of this review was to summarise the achievement of MRI-based radiomics and AI for the evaluation of staging, response to therapy, genotyping, prediction of high-risk factors, and prognosis in the field of RC. Moreover, future challenges and limitations of these tools that need to be solved to favour the transition from academic research to the clinical setting will be discussed.

K-UNN: k-space interpolation with untrained neural network

Recently, untrained neural networks (UNNs) have shown satisfactory performances for MR image reconstruction on random sampling trajectories without using additional full-sampled training data. However, the existing UNN-based approaches lack the modeling of physical priors, resulting in poor performance in some common scenarios (e.g., partial Fourier (PF), regular sampling, etc.) and the lack of theoretical guarantees for reconstruction accuracy. To bridge this gap, we propose a safeguarded k-space interpolation method for MRI using a specially designed UNN with a tripled architecture driven by three physical priors of the MR images (or k-space data), including transform sparsity, coil sensitivity smoothness, and phase smoothness. We also prove that the proposed method guarantees tight bounds for interpolated k-space data accuracy. Finally, ablation experiments show that the proposed method can characterize the physical priors of MR images well. Additionally, experiments show that the proposed method consistently outperforms traditional parallel imaging methods and existing UNNs, and is even competitive against supervised-trained deep learning methods in PF and regular undersampling reconstruction.

MRI-Based Artificial Intelligence in Rectal Cancer

Rectal cancer (RC) accounts for approximately one-third of colorectal cancer (CRC), with death rates increasing in patients younger than 50 years old. Magnetic resonance imaging (MRI) is routinely performed for tumor evaluation. However, the semantic features from images alone remain insufficient to guide treatment decisions. Functional MRIs are useful for revealing microstructural and functional abnormalities and nevertheless have low or modest repeatability and reproducibility. Therefore, during the preoperative evaluation and follow-up treatment of patients with RC, novel noninvasive imaging markers are needed to describe tumor characteristics to guide treatment strategies and achieve individualized diagnosis and treatment. In recent years, the development of artificial intelligence (AI) has created new tools for RC evaluation based on MRI. In this review, we summarize the research progress of AI in the evaluation of staging, prediction of high-risk factors, genotyping, response to therapy, recurrence, metastasis, prognosis, and segmentation with RC. We further discuss the challenges of clinical application, including improvement in imaging, model performance, and the biological meaning of features, which may also be major development directions in the future. EVIDENCE LEVEL: 5 TECHNICAL EFFICACY: Stage 2.

One-Dimensional Deep Low-Rank and Sparse Network for Accelerated MRI

Deep learning has shown astonishing performance in accelerated magnetic resonance imaging (MRI). Most state-of-the-art deep learning reconstructions adopt the powerful convolutional neural network and perform 2D convolution since many magnetic resonance images or their corresponding k-space are in 2D. In this work, we present a new approach that explores the 1D convolution, making the deep network much easier to be trained and generalized. We further integrate the 1D convolution into the proposed deep network, named as One-dimensional Deep Low-rank and Sparse network (ODLS), which unrolls the iteration procedure of a low-rank and sparse reconstruction model. Extensive results on in vivo knee and brain datasets demonstrate that, the proposed ODLS is very suitable for the case of limited training subjects and provides improved reconstruction performance than state-of-the-art methods both visually and quantitatively. Additionally, ODLS also shows nice robustness to different undersampling scenarios and some mismatches between the training and test data. In summary, our work demonstrates that the 1D deep learning scheme is memory-efficient and robust in fast MRI.

TranSMS: Transformers for Super-Resolution Calibration in Magnetic Particle Imaging

Magnetic particle imaging (MPI) offers exceptional contrast for magnetic nanoparticles (MNP) at high spatio-temporal resolution. A common procedure in MPI starts with a calibration scan to measure the system matrix (SM), which is then used to set up an inverse problem to reconstruct images of the MNP distribution during subsequent scans. This calibration enables the reconstruction to sensitively account for various system imperfections. Yet time-consuming SM measurements have to be repeated under notable changes in system properties. Here, we introduce a novel deep learning approach for accelerated MPI calibration based on Transformers for SM super-resolution (TranSMS). Low-resolution SM measurements are performed using large MNP samples for improved signal-to-noise ratio efficiency, and the high-resolution SM is super-resolved via model-based deep learning. TranSMS leverages a vision transformer module to capture contextual relationships in low-resolution input images, a dense convolutional module for localizing high-resolution image features, and a data-consistency module to ensure measurement fidelity. Demonstrations on simulated and experimental data indicate that TranSMS significantly improves SM recovery and MPI reconstruction for up to 64-fold acceleration in two-dimensional imaging.

A simultaneous multi-slice T2 mapping framework based on overlapping-echo detachment planar imaging and deep learning reconstruction

PURPOSE

Quantitative MRI (qMRI) is of great importance to clinical medicine and scientific research. However, most qMRI techniques are time-consuming and sensitive to motion, especially when a large 3D volume is imaged. To accelerate the acquisition, a framework is proposed to realize reliable simultaneous multi-slice T₂ mapping.

METHODS

The simultaneous multi-slice T₂ mapping framework is based on overlapping-echo detachment (OLED) planar imaging (dubbed SMS-OLED). Multi-slice overlapping-echo signals were generated by multiple excitation pulses together with echo-shifting gradients. The signals were excited and acquired with a single-channel coil. U-Net was used to reconstruct T₂ maps from the acquired overlapping-echo image.

RESULTS

Single-shot double-slice and two-shot triple-slice SMS-OLED scan schemes were designed according to the framework for evaluation. Simulations, water phantom, and in vivo rat brain experiments were carried out. Overlapping-echo signals were acquired, and T₂ maps were reconstructed and compared with references. The results demonstrate the superior performance of our method.

CONCLUSION

Two slices of T₂ maps can be obtained in a single shot within hundreds of milliseconds. Higher quality multi-slice T₂ maps can be obtained via multiple shots. SMS-OLED provides a lower specific absorption rate scheme compared with conventional SMS methods with a coil with only a single receiver channel. The new method is of potential in dynamic qMRI and functional qMRI where temporal resolution is vital.