J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction

2D CAIPI accelerated 3D multi-slab diffusion weighted EPI combined with qModeL reconstruction for fast high resolution microstructure imaging

PURPOSE

To develop acceleration strategies for 3D multi-slab diffusion weighted imaging (3D ms-DWI) for enabling applications that require simultaneously high spatial (1 mm isotropic) and angular (> 30 directions) resolutions.

METHODS

3D ms-DWI offers high SNR-efficiency, with the ability to achieve high isotropic spatial resolution, yet suffers from long scan-times for studies requiring high angular resolutions. We develop 6D k-q space acceleration strategies to reduce the scan-time. Specifically, we develop non-uniform 3D ky-kz under-sampling employing a shot-selective 2D CAIPI sampling approach. To achieve inter-shot phase-compensation, 2D navigators were employed that utilize the same CAIPI trajectory. An iterative model-based 3D multi-shot reconstruction was designed by incorporating phase into the forward encoding process. Additionally, the shot-selective non-uniform ky-kz CAIPI acceleration was randomized along the q-dimension. The 3D model-based multi-shot reconstruction is then extended to a joint reconstruction that simultaneously reconstructs all the q-space points, with the help of a spatial total variation and deep-learned q-space regularization.

RESULTS

The proposed reconstruction is shown to achieve adequate phase-compensation in both 2D CAIPI accelerated and additional ky-kz under-sampled cases. Using retrospective under-sampling experiments, we show that k-q accelerations close a factor of 12 can be achieved with a reconstruction error < 3% for both single and multi-shell data. This translates to a scan-time reduction by 3-fold for experiments with simultaneously high spatial and angular resolutions.

CONCLUSION

The proposed method facilitates the utilization of 3D ms-DWI for simultaneously high k-q resolution applications with close to 3× reduced scan-time.

Attention-Based MultiOffset Deep Learning Reconstruction of Chemical Exchange Saturation Transfer (AMO-CEST) MRI

One challenge of chemical exchange saturation transfer (CEST) magnetic resonance imaging (MRI) is the long scan time due to multiple acquisitions of images at different saturation frequency offsets. k-space under-sampling strategy is commonly used to accelerate MRI acquisition, while this could introduce artifacts and reduce signal-to-noise ratio (SNR). To accelerate CEST-MRI acquisition while maintaining suitable image quality, we proposed an attention-based multioffset deep learning reconstruction network (AMO-CEST) with a multiple radial k-space sampling strategy for CEST-MRI. The AMO-CEST also contains dilated convolution to enlarge the receptive field and data consistency module to preserve the sampled k-space data. We evaluated the proposed method on a mouse brain dataset containing 5760 CEST images acquired at a pre-clinical 3 T MRI scanner. Quantitative results demonstrated that AMO-CEST showed obvious improvement over zero-filling method with a PSNR enhancement of 11 dB, a SSIM enhancement of 0.15, and a NMSE decrease of [Formula: see text] in three acquisition orientations. Compared with other deep learning-based models, AMO-CEST showed visual and quantitative improvements in images from three different orientations. We also extracted molecular contrast maps, including the amide proton transfer (APT) and the relayed nuclear Overhauser enhancement (rNOE). The results demonstrated that the CEST contrast maps derived from the CEST images of AMO-CEST were comparable to those derived from the original high-resolution CEST images. The proposed AMO-CEST can efficiently reconstruct high-quality CEST images from under-sampled k-space data and thus has the potential to accelerate CEST-MRI acquisition.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

LUCMT: Learnable under-sampling and reconstructed network with cross multi-head attention transformer for accelerating MR image reconstruction

BACKGROUND AND OBJECTIVE

As a widely used technique for Magnetic Resonance Image (MRI) acceleration, compressed sensing MRI involves two main issues: designing an effective sampling strategy and reconstructing the image from significantly under-sampled K-space data. In this paper, an innovative approach is proposed to address these two challenges simultaneously.

METHODS

A novel MRI reconstruction method, termed as LUCMT, is implemented by integrating a learnable under-sampling strategy with a reconstruction network based on the Cross Multi-head Attention Transformer. In contrast to conventional static sampling methods, the proposed adaptive sampling scheme is processed optimally by learning the optimal sampling technique, which involves binarizing the sampling pattern by a sigmoid function and computing gradients by backpropagation. And the reconstruction network is designed by using CS-MRI depth unfolding network that incorporates a Cross Multi-head Attention (CMA) module with inertial and gradient descent terms.

RESULTS

T1 brain MR images from the FastMRI dataset are used to validate the performance of the proposed method. A series of experiments are conducted to validate the superior performance of our proposed network in terms of quantitative metrics and visual quality. Compared with other state-of-the-art reconstruction methods, LUCMT achieves better reconstruction performances with more accurate details. Specifically, LUCMT achieves PSNR and SSIM results of 41.87/0.9749, 46.64/0.9868, 50.41/0.9924, and 53.51/0.9955 at sampling rates of 10 %, 20 %, 30 %, and 40 %, respectively.

CONCLUSIONS

The proposed LUCMT method can provide a promising way for generating optimal under-sampling mask and accelerating MRI reconstruction accurately.

Deep learning for accelerated and robust MRI reconstruction

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.

HyperSLICE: HyperBand optimized spiral for low-latency interactive cardiac examination

PURPOSE

Interactive cardiac MRI is used for fast scan planning and MR-guided interventions. However, the requirement for real-time acquisition and near-real-time visualization constrains the achievable spatio-temporal resolution. This study aims to improve interactive imaging resolution through optimization of undersampled spiral sampling and leveraging of deep learning for low-latency reconstruction (deep artifact suppression).

METHODS

A variable density spiral trajectory was parametrized and optimized via HyperBand to provide the best candidate trajectory for rapid deep artifact suppression. Training data consisted of 692 breath-held CINEs. The developed interactive sequence was tested in simulations and prospectively in 13 subjects (10 for image evaluation, 2 during catheterization, 1 during exercise). In the prospective study, the optimized framework-HyperSLICE- was compared with conventional Cartesian real-time and breath-hold CINE imaging in terms quantitative and qualitative image metrics. Statistical differences were tested using Friedman chi-squared tests with post hoc Nemenyi test (p < 0.05).

RESULTS

In simulations the normalized RMS error, peak SNR, structural similarity, and Laplacian energy were all statistically significantly higher using optimized spiral compared to radial and uniform spiral sampling, particularly after scan plan changes (structural similarity: 0.71 vs. 0.45 and 0.43). Prospectively, HyperSLICE enabled a higher spatial and temporal resolution than conventional Cartesian real-time imaging. The pipeline was demonstrated in patients during catheter pull back, showing sufficiently fast reconstruction for interactive imaging.

CONCLUSION

HyperSLICE enables high spatial and temporal resolution interactive imaging. Optimizing the spiral sampling enabled better overall image quality and superior handling of image transitions compared with radial and uniform spiral trajectories.

SIMPLEX: Multiple phase-cycled bSSFP quantitative magnetization transfer imaging with physic-guided simulation learning of neural network

Most quantitative magnetization transfer (qMT) imaging methods require acquiring additional quantitative maps (such as T1) for data fitting. A method based on multiple phase-cycled bSSFP was recently proposed to enable high-resolution 3D qMT imaging based on least square fitting without any extra acquisition, and thus has high potential for simplifying the qMT procedure. However, the quantification of qMT parameters with this method was suboptimal, limiting its potential for clinical application despite its simpler protocol and higher spatial resolution. To improve the fitting of qMT data obtained with multiple phase-cycled bSSFP, we propose SIMulation-based Physics-guided Learning of neural network for qMT parameters EXtraction, or SIMPLEX. In contrast to previous deep learning supervised approaches for quantitative MR that require the acquisition of input data and corresponding ground truth for training, we leveraged the MR signal model to generate training samples without expensive data curation. The network was trained exclusively with simulation data by predicting the simulation parameters. The same network was applied directly to in-vivo data without additional training. The approach was verified with both simulation and in-vivo data. SIMPLEX showed a decrease in fitting mean squared error for all simulation data compared to the existing least-square fitting method. The in-vivo experiment revealed that the network performed well with the real in vivo data unseen during training. For all experiments, we observed that SIMPLEX consistently improved the quantification quality of the qMT parameters whilst being more robust to noise compared to the prior technique. The proposed SIMPLEX will expedite the routine clinical application of qMT by providing qMT parameters (exchange rate, pool fraction) as well as T1, T2, and ΔB0 maps simultaneously with high spatial resolution, better reliability, and reduced processing time.

Complexities of deep learning-based undersampled MR image reconstruction

Artificial intelligence has opened a new path of innovation in magnetic resonance (MR) image reconstruction of undersampled k-space acquisitions. This review offers readers an analysis of the current deep learning-based MR image reconstruction methods. The literature in this field shows exponential growth, both in volume and complexity, as the capabilities of machine learning in solving inverse problems such as image reconstruction are explored. We review the latest developments, aiming to assist researchers and radiologists who are developing new methods or seeking to provide valuable feedback. We shed light on key concepts by exploring the technical intricacies of MR image reconstruction, highlighting the importance of raw datasets and the difficulty of evaluating diagnostic value using standard metrics.Relevance statement Increasingly complex algorithms output reconstructed images that are difficult to assess for robustness and diagnostic quality, necessitating high-quality datasets and collaboration with radiologists.Key points• Deep learning-based image reconstruction algorithms are increasing both in complexity and performance.• The evaluation of reconstructed images may mistake perceived image quality for diagnostic value.• Collaboration with radiologists is crucial for advancing deep learning technology.

Optimizing variable flip angles in magnetization-prepared gradient-echo sequences for efficient 3D-T1ρ mapping

PURPOSE

To optimize the choice of the flip angles of magnetization-prepared gradient-echo sequences for improved accuracy, precision, and speed of 3D-T1ρ mapping.

METHODS

We propose a new optimization approach for finding variable flip-angle values that improve magnetization-prepared gradient-echo sequences used for 3D-T1ρ mapping. This new approach can improve the accuracy and SNR, while reducing filtering effects. We demonstrate the concept in the three different versions of the magnetization-prepared gradient-echo sequences that are typically used for 3D-T1ρ mapping and evaluate their performance in model agarose phantoms (n = 4) and healthy volunteers (n = 5) for knee joint imaging. We also tested the optimization with sequence parameters targeting faster acquisitions.

RESULTS

Our results show that optimized variable flip angle can improve the accuracy and the precision of the sequences, seen as a reduction of the mean of normalized absolute difference from about 5%-6% to 3%-4% in model phantoms and from 15%-16% to 11%-13% in the knee joint, and improving SNR from about 12-28 to 22-32 in agarose phantoms and about 7-14 to 13-17 in healthy volunteers. The optimization can also compensate for the loss in quality caused by making the sequence faster. This results in sequence configurations that acquire more data per unit of time with SNR and mean of normalized absolute difference measurements close to its slower versions.

CONCLUSION

The optimization of the variable flip angle can be used to increase accuracy and precision, and to improve the speed of the typical imaging sequences used for quantitative 3D-T1ρ mapping of the knee joint.

PARCEL: Physics-Based Unsupervised Contrastive Representation Learning for Multi-Coil MR Imaging

With the successful application of deep learning to magnetic resonance (MR) imaging, parallel imaging techniques based on neural networks have attracted wide attention. However, in the absence of high-quality, fully sampled datasets for training, the performance of these methods is limited. And the interpretability of models is not strong enough. To tackle this issue, this paper proposes a Physics-bAsed unsupeRvised Contrastive rEpresentation Learning (PARCEL) method to speed up parallel MR imaging. Specifically, PARCEL has a parallel framework to contrastively learn two branches of model-based unrolling networks from augmented undersampled multi-coil k-space data. A sophisticated co-training loss with three essential components has been designed to guide the two networks in capturing the inherent features and representations for MR images. And the final MR image is reconstructed with the trained contrastive networks. PARCEL was evaluated on two vivo datasets and compared to five state-of-the-art methods. The results show that PARCEL is able to learn essential representations for accurate MR reconstruction without relying on fully sampled datasets. The code will be made available at https://github.com/ternencewu123/PARCEL.

Emerging Trends in Fast MRI Using Deep-Learning Reconstruction on Undersampled k-Space Data: A Systematic Review

Magnetic Resonance Imaging (MRI) is an essential medical imaging modality that provides excellent soft-tissue contrast and high-resolution images of the human body, allowing us to understand detailed information on morphology, structural integrity, and physiologic processes. However, MRI exams usually require lengthy acquisition times. Methods such as parallel MRI and Compressive Sensing (CS) have significantly reduced the MRI acquisition time by acquiring less data through undersampling k-space. The state-of-the-art of fast MRI has recently been redefined by integrating Deep Learning (DL) models with these undersampled approaches. This Systematic Literature Review (SLR) comprehensively analyzes deep MRI reconstruction models, emphasizing the key elements of recently proposed methods and highlighting their strengths and weaknesses. This SLR involves searching and selecting relevant studies from various databases, including Web of Science and Scopus, followed by a rigorous screening and data extraction process using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. It focuses on various techniques, such as residual learning, image representation using encoders and decoders, data-consistency layers, unrolled networks, learned activations, attention modules, plug-and-play priors, diffusion models, and Bayesian methods. This SLR also discusses the use of loss functions and training with adversarial networks to enhance deep MRI reconstruction methods. Moreover, we explore various MRI reconstruction applications, including non-Cartesian reconstruction, super-resolution, dynamic MRI, joint learning of reconstruction with coil sensitivity and sampling, quantitative mapping, and MR fingerprinting. This paper also addresses research questions, provides insights for future directions, and emphasizes robust generalization and artifact handling. Therefore, this SLR serves as a valuable resource for advancing fast MRI, guiding research and development efforts of MRI reconstruction for better image quality and faster data acquisition.

Fast Multi-Contrast MRI Acquisition by Optimal Sampling of Information Complementary to Pre-Acquired MRI Contrast

Recent studies on multi-contrast MRI reconstruction have demonstrated the potential of further accelerating MRI acquisition by exploiting correlation between contrasts. Most of the state-of-the-art approaches have achieved improvement through the development of network architectures for fixed under-sampling patterns, without considering inter-contrast correlation in the under-sampling pattern design. On the other hand, sampling pattern learning methods have shown better reconstruction performance than those with fixed under-sampling patterns. However, most under-sampling pattern learning algorithms are designed for single contrast MRI without exploiting complementary information between contrasts. To this end, we propose a framework to optimize the under-sampling pattern of a target MRI contrast which complements the acquired fully-sampled reference contrast. Specifically, a novel image synthesis network is introduced to extract the redundant information contained in the reference contrast, which is exploited in the subsequent joint pattern optimization and reconstruction network. We have demonstrated superior performance of our learned under-sampling patterns on both public and in-house datasets, compared to the commonly used under-sampling patterns and state-of-the-art methods that jointly optimize the reconstruction network and the under-sampling patterns, up to 8-fold under-sampling factor.

AI in MRI: Computational Frameworks for a Faster, Optimized, and Automated Imaging Workflow

Over the last decade, artificial intelligence (AI) has made an enormous impact on a wide range of fields, including science, engineering, informatics, finance, and transportation [...].

Efficient Approximation of Jacobian Matrices Involving a Non-Uniform Fast Fourier Transform (NUFFT)

There is growing interest in learning Fourier domain sampling strategies (particularly for magnetic resonance imaging, MRI) using optimization approaches. For non-Cartesian sampling, the system models typically involve non-uniform fast Fourier transform (NUFFT) operations. Commonly used NUFFT algorithms contain frequency domain interpolation, which is not differentiable with respect to the sampling pattern, complicating the use of gradient methods. This paper describes an efficient and accurate approach for computing approximate gradients involving NUFFTs. Multiple numerical experiments validate the improved accuracy and efficiency of the proposed approximation. As an application to computational imaging, the NUFFT Jacobians were used to optimize non-Cartesian MRI sampling trajectories via data-driven stochastic optimization. Specifically, the sampling patterns were learned with respect to various model-based image reconstruction (MBIR) algorithms. The proposed approach enables sampling optimization for image sizes that are infeasible with standard auto-differentiation methods due to memory limits. The synergistic acquisition and reconstruction design leads to remarkably improved image quality. In fact, we show that model-based image reconstruction methods with suitably optimized imaging parameters can perform nearly as well as CNN-based methods.

Learning to Sense for Coded Diffraction Imaging

In this paper, we present a framework to learn illumination patterns to improve the quality of signal recovery for coded diffraction imaging. We use an alternating minimization-based phase retrieval method with a fixed number of iterations as the iterative method. We represent the iterative phase retrieval method as an unrolled network with a fixed number of layers where each layer of the network corresponds to a single step of iteration, and we minimize the recovery error by optimizing over the illumination patterns. Since the number of iterations/layers is fixed, the recovery has a fixed computational cost. Extensive experimental results on a variety of datasets demonstrate that our proposed method significantly improves the quality of image reconstruction at a fixed computational cost with illumination patterns learned only using a small number of training images.

Joint optimization of Cartesian sampling patterns and reconstruction for single-contrast and multi-contrast fast magnetic resonance imaging

BACKGROUND AND OBJECTIVE

Compressed sensing (CS) has gained increased attention in magnetic resonance imaging (MRI), leveraging its efficacy to accelerate image acquisition. Incoherence measurement and non-linear reconstruction are the most crucial guarantees of accurate restoration. However, the loose link between measurement and reconstruction hinders the further improvement of reconstruction quality, i.e., the default sampling pattern is not adaptively tailored to the downstream reconstruction method. When single-contrast reconstruction (SCR) has been upgraded to its multi-contrast reconstruction (MCR) variant, the identical morphologic information as a priori source could be integrated into the reconstruction procedure. How to measure less and reconstruct effectively by using the shareable morphologic information of various contrast images is an attractive topic.

METHODS

An adaptive sampling (AS) based end-to-end framework (ASSCR or ASMCR) is proposed to address this issue, which simultaneously optimizes sampling patterns and reconstruction from under-sampled data in SCR or MCR scenarios. Several deep probabilistic subsampling (DPS) modules are used in AS network to construct a sampling pattern generator. In SCR and MCR, a convolution block and a data consistency layer are iteratively applied in the reconstruction network. Specifically, the learned optimal sampling pattern output from the trained AS sub-net is used for under-sampling. Incoherence measurement for single-contrast images and the combination of sampling patterns for multi-contrast data are guided by the SCR/MCR sub-net.

RESULTS

Experiments were conducted on two single-contrast and one multi-contrast public MRI datasets. Compared with several state-of-the-art reconstruction methods, SCR results show that a learned sampling pattern brings the quality of the reconstructed image closer to the fully-sampled reference. With the addition of different contrast images, under-sampled images with higher acceleration factors could be well recovered.

CONCLUSION

The proposed method could improve the reconstruction quality of under-sampled images by using adaptive sampling patterns and learning-based reconstruction.

Joint Optimization of k-t Sampling Pattern and Reconstruction of DCE MRI for Pharmacokinetic Parameter Estimation

This work proposes to develop and evaluate a deep learning framework that jointly optimizes k-t sampling patterns and reconstruction for head and neck dynamic contrast-enhanced (DCE) MRI aiming to reduce bias and uncertainty of pharmacokinetic (PK) parameter estimation. 2D Cartesian phase encoding k-space subsampling patterns for a 3D spoiled gradient recalled echo (SPGR) sequence along a time course of DCE MRI were jointly optimized in a deep learning-based dynamic MRI reconstruction network by a loss function concerning both reconstruction image quality and PK parameter estimation accuracy. During training, temporal k-space data sharing scheme was optimized as well. The proposed method was trained and tested by multi-coil complex digital reference objects of DCE images (mcDROs). The PK parameters estimated by the proposed method were compared with two published iterative DCE MRI reconstruction schemes using normalized root mean squared errors (NRMSEs) and Bland-Altman analysis at temporal resolutions of [Formula: see text] = 2s, 3s, 4s, and 5s, which correspond to undersampling rates of R = 50, 34, 25, and 20. The proposed method achieved low PK parameter NRMSEs at all four temporal resolutions compared with the benchmark methods on testing mcDROs. The Bland-Altman plots demonstrated that the proposed method reduced PK parameter estimation bias and uncertainty in tumor regions at temporal resolution of 2s. The proposed method also showed robustness to contrast arrival timing variations across patients. This work provides a potential way to increase PK parameter estimation accuracy and precision, and thus facilitate the clinical translation of DCE MRI.

Simultaneously optimizing sampling pattern for joint acceleration of multi-contrast MRI using model-based deep learning

BACKGROUND

Acceleration of MR imaging (MRI) is a popular research area, and usage of deep learning for acceleration has become highly widespread in the MR community. Joint acceleration of multiple-acquisition MRI was proven to be effective over a single-acquisition approach. Also, optimization in the sampling pattern demonstrated its advantage over conventional undersampling pattern. However, optimizing the sampling patterns for joint acceleration of multiple-acquisition MRI has not been investigated well.

PURPOSE

To develop a model-based deep learning scheme to optimize sampling patterns for a joint acceleration of multi-contrast MRI.

METHODS

The proposed scheme combines sampling pattern optimization and multi-contrast MRI reconstruction. It was extended from the physics-guided method of the joint model-based deep learning (J-MoDL) scheme to optimize the separate sampling pattern for each of multiple contrasts simultaneously for their joint reconstruction. Tests were performed with three contrasts of T2-weighted, FLAIR, and T1-weighted images. The proposed multi-contrast method was compared to (i) single-contrast method with sampling optimization (baseline J-MoDL), (ii) multi-contrast method without sampling optimization, and (iii) multi-contrast method with single common sampling optimization for all contrasts. The optimized sampling patterns were analyzed for sampling location overlap across contrasts. The scheme was also tested in a data-driven scenario, where the inversion between input and label was learned from the under-sampled data directly and tested on knee datasets for generalization test.

RESULTS

The proposed scheme demonstrated a quantitative and qualitative advantage over the single-contrast scheme with sampling pattern optimization and the multi-contrast scheme without sampling pattern optimization. Optimizing the separate sampling pattern for each of the multi-contrasts was superior to optimizing only one common sampling pattern for all contrasts. The proposed scheme showed less overlap in sampling locations than the single-contrast scheme. The main hypothesis was also held in the data-driven situation as well. The brain-trained model worked well on the knee images, demonstrating its generalizability.

CONCLUSION

Our study introduced an effective scheme that combines the sampling optimization and the multi-contrast acceleration. The seamless combination resulted in superior performance over the other existing methods.

Accelerating multi-echo chemical shift encoded water-fat MRI using model-guided deep learning

PURPOSE

To accelerate chemical shift encoded (CSE) water-fat imaging by applying a model-guided deep learning water-fat separation (MGDL-WF) framework to the undersampled k-space data.

METHODS

A model-guided deep learning water-fat separation framework is proposed for the acceleration using Cartesian/radial undersampling data. The proposed MGDL-WF combines the power of CSE water-fat imaging model and data-driven deep learning by jointly using a multi-peak fat model and a modified residual U-net network. The model is used to guide the image reconstruction, and the network is used to capture the artifacts induced by the undersampling. A data consistency layer is used in MGDL-WF to ensure the output images to be consistent with the k-space measurements. A Gauss-Newton iteration algorithm is adapted for the gradient updating of the networks.

RESULTS

Compared with the compressed sensing water-fat separation (CS-WF) algorithm/2-step procedure algorithm, the MGDL-WF increased peak signal-to-noise ratio (PSNR) by 5.31/5.23, 6.11/4.54, and 4.75 dB/1.88 dB with Cartesian sampling, and by 4.13/6.53, 2.90/4.68, and 1.68 dB/3.48 dB with radial sampling, at acceleration rates (R) of 4, 6, and 8, respectively. By using MGDL-WF, radial sampling increased the PSNR by 2.07 dB at R = 8, compared with Cartesian sampling.

CONCLUSIONS

The proposed MGDL-WF enables exploiting features of the water images and fat images from the undersampled multi-echo data, leading to improved performance in the accelerated CSE water-fat imaging. By using MGDL-WF, radial sampling can further improve the image quality with comparable scan time in comparison with Cartesian sampling.

A review and experimental evaluation of deep learning methods for MRI reconstruction

Following the success of deep learning in a wide range of applications, neural network-based machine-learning techniques have received significant interest for accelerating magnetic resonance imaging (MRI) acquisition and reconstruction strategies. A number of ideas inspired by deep learning techniques for computer vision and image processing have been successfully applied to nonlinear image reconstruction in the spirit of compressed sensing for accelerated MRI. Given the rapidly growing nature of the field, it is imperative to consolidate and summarize the large number of deep learning methods that have been reported in the literature, to obtain a better understanding of the field in general. This article provides an overview of the recent developments in neural-network based approaches that have been proposed specifically for improving parallel imaging. A general background and introduction to parallel MRI is also given from a classical view of k-space based reconstruction methods. Image domain based techniques that introduce improved regularizers are covered along with k-space based methods which focus on better interpolation strategies using neural networks. While the field is rapidly evolving with plenty of papers published each year, in this review, we attempt to cover broad categories of methods that have shown good performance on publicly available data sets. Limitations and open problems are also discussed and recent efforts for producing open data sets and benchmarks for the community are examined.

2D probabilistic undersampling pattern optimization for MR image reconstruction

With 3D magnetic resonance imaging (MRI), a tradeoff exists between higher image quality and shorter scan time. One way to solve this problem is to reconstruct high-quality MRI images from undersampled k-space. There have been many recent studies exploring effective k-space undersampling patterns and designing MRI reconstruction methods from undersampled k-space, which are two necessary steps. Most studies separately considered these two steps, although in theory, their performance is dependent on each other. In this study, we propose a joint optimization model, trained end-to-end, to simultaneously optimize the undersampling pattern in the Fourier domain and the reconstruction model in the image domain. A 2D probabilistic undersampling layer was designed to optimize the undersampling pattern and probability distribution in a differentiable manner. A 2D inverse Fourier transform layer was implemented to connect the Fourier domain and the image domain during the forward and back propagation. Finally, we discovered an optimized relationship between the probability distribution of the undersampling pattern and its corresponding sampling rate. Further testing was performed using 3D T1-weighted MR images of the brain from the MICCAI 2013 Grand Challenge on Multi-Atlas Labeling dataset and locally acquired brain 3D T1-weighted MR images of healthy volunteers and contrast-enhanced 3D T1-weighted MR images of high-grade glioma patients. The results showed that the recovered MR images using our 2D probabilistic undersampling pattern (with or without the reconstruction network) significantly outperformed those using the existing start-of-the-art undersampling strategies for both qualitative and quantitative comparison, suggesting the advantages and some extent of the generalization of our proposed method.

Alternating Learning Approach for Variational Networks and Undersampling Pattern in Parallel MRI Applications

This work proposes an alternating learning approach to learn the sampling pattern (SP) and the parameters of variational networks (VN) in accelerated parallel magnetic resonance imaging (MRI). We investigate four variations of the learning approach, that alternates between improving the SP, using bias-accelerated subset selection, and improving parameters of the VN, using ADAM. The variations include the use of monotone or non-monotone alternating steps and systematic reduction of learning rates. The algorithms learn an effective pair to be used in future scans, including an SP that captures fewer k-space samples in which the generated undersampling artifacts are removed by the VN reconstruction. The quality of the VNs and SPs obtained by the proposed approaches is compared against different methods, including other kinds of joint learning methods and state-of-art reconstructions, on two different datasets at various acceleration factors (AF). We observed improvements visually and in three different figures of merit commonly used in deep learning (RMSE, SSIM, and HFEN) on AFs from 2 to 20 with brain and knee joint datasets when compared to the other approaches. The improvements ranged from 1% to 62% over the next best approach tested with VNs. The proposed approach has shown stable performance, obtaining similar learned SPs under different initial training conditions. We observe that the improvement is not only due to the learned sampling density, it is also due to the learned position of samples in k-space. The proposed approach was able to learn effective pairs of SPs and reconstruction VNs, improving 3D Cartesian accelerated parallel MRI applications.

Convex optimization algorithms in medical image reconstruction - in the age of AI

The past decade has seen the flourish of model based image reconstruction (MBIR) algorithms, which are often applications or adaptations of convex optimization algorithms from the optimization community. We review some state-of-the-art algorithms that have enjoyed wide popularity in medical image reconstruction, emphasize known connections between different algorithms, and discuss practical issues such as computation and memory cost. More recently, deep learning (DL) has forayed into medical imaging, where the latest development tries to exploit the synergy between DL and MBIR to elevate the MBIR's performance. We present existing approaches and emerging trends in DL-enhanced MBIR methods, with particular attention to the underlying role of convexity and convex algorithms on network architecture. We also discuss how convexity can be employed to improve the generalizability and representation power of DL networks in general.

Upstream Machine Learning in Radiology

Machine learning (ML) and Artificial intelligence (AI) has the potential to dramatically improve radiology practice at multiple stages of the imaging pipeline. Most of the attention has been garnered by applications focused on improving the end of the pipeline: image interpretation. However, this article reviews how AI/ML can be applied to improve upstream components of the imaging pipeline, including exam modality selection, hardware design, exam protocol selection, data acquisition, image reconstruction, and image processing. A breadth of applications and their potential for impact is shown across multiple imaging modalities, including ultrasound, computed tomography, and MRI.

Fast data-driven learning of parallel MRI sampling patterns for large scale problems

In this study, a fast data-driven optimization approach, named bias-accelerated subset selection (BASS), is proposed for learning efficacious sampling patterns (SPs) with the purpose of reducing scan time in large-dimensional parallel MRI. BASS is applicable when Cartesian fully-sampled k-space measurements of specific anatomy are available for training and the reconstruction method for undersampled measurements is specified; such information is used to define the efficacy of any SP for recovering the values at the non-sampled k-space points. BASS produces a sequence of SPs with the aim of finding one of a specified size with (near) optimal efficacy. BASS was tested with five reconstruction methods for parallel MRI based on low-rankness and sparsity that allow a free choice of the SP. Three datasets were used for testing, two of high-resolution brain images ([Formula: see text]-weighted images and, respectively, [Formula: see text]-weighted images) and another of knee images for quantitative mapping of the cartilage. The proposed approach has low computational cost and fast convergence; in the tested cases it obtained SPs up to 50 times faster than the currently best greedy approach. Reconstruction quality increased by up to 45% over that provided by variable density and Poisson disk SPs, for the same scan time. Optionally, the scan time can be nearly halved without loss of reconstruction quality. Quantitative MRI and prospective accelerated MRI results show improvements. Compared with greedy approaches, BASS rapidly learns effective SPs for various reconstruction methods, using larger SPs and larger datasets; enabling better selection of sampling-reconstruction pairs for specific MRI problems.

A Survey of Soft Computing Approaches in Biomedical Imaging

Medical imaging is an essential technique for the diagnosis and treatment of diseases in modern clinics. Soft computing plays a major role in the recent advances in medical imaging. It handles uncertainties and improves the qualities of an image. Until now, various soft computing approaches have been proposed for medical applications. This paper discusses various medical imaging modalities and presents a short review of soft computing approaches such as fuzzy logic, artificial neural network, genetic algorithm, machine learning, and deep learning. We also studied and compared each approach used for other imaging modalities based on the certain parameter used for the system evaluation. Finally, based on comparative analysis, the possible research strategies for further development are proposed. As far as we know, no previous work examined this issue.

qModeL: A plug-and-play model-based reconstruction for highly accelerated multi-shot diffusion MRI using learned priors

PURPOSE

To introduce a joint reconstruction method for highly undersampled multi-shot diffusion weighted (msDW) scans.

METHODS

Multi-shot EPI methods enable higher spatial resolution for diffusion MRI, but at the expense of long scan-time. Highly accelerated msDW scans are needed to enable their utilization in advanced microstructure studies, which require high q-space coverage. Previously, joint k-q undersampling methods coupled with compressed sensing were shown to enable very high acceleration factors. However, the reconstruction of this data using sparsity priors is challenging and is not suited for multi-shell data. We propose a new reconstruction that recovers images from the combined k-q data jointly. The proposed qModeL reconstruction brings together the advantages of model-based iterative reconstruction and machine learning, extending the idea of plug-and-play algorithms. Specifically, qModeL works by prelearning the signal manifold corresponding to the diffusion measurement space using deep learning. The prelearned manifold prior is incorporated into a model-based reconstruction to provide a voxel-wise regularization along the q-dimension during the joint recovery. Notably, the learning does not require in vivo training data and is derived exclusively from biophysical modeling. Additionally, a plug-and-play total variation denoising provides regularization along the spatial dimension. The proposed framework is tested on k-q undersampled single-shell and multi-shell msDW acquisition at various acceleration factors.

RESULTS

The qModeL joint reconstruction is shown to recover DWIs from 8-fold accelerated msDW acquisitions with error less than 5% for both single-shell and multi-shell data. Advanced microstructural analysis performed using the undersampled reconstruction also report reasonable accuracy.

CONCLUSION

qModeL enables the joint recovery of highly accelerated multi-shot dMRI utilizing learning-based priors. The bio-physically driven approach enables the use of accelerated multi-shot imaging for multi-shell sampling and advanced microstructure studies.

Deep learning in magnetic resonance image reconstruction

Magnetic resonance (MR) imaging visualises soft tissue contrast in exquisite detail without harmful ionising radiation. In this work, we provide a state-of-the-art review on the use of deep learning in MR image reconstruction from different image acquisition types involving compressed sensing techniques, parallel image acquisition and multi-contrast imaging. Publications with deep learning-based image reconstruction for MR imaging were identified from the literature (PubMed and Google Scholar), and a comprehensive description of each of the works was provided. A detailed comparison that highlights the differences, the data used and the performance of each of these works were also made. A discussion of the potential use cases for each of these methods is provided. The sparse image reconstruction methods were found to be most popular in using deep learning for improved performance, accelerating acquisitions by around 4-8 times. Multi-contrast image reconstruction methods rely on at least one pre-acquired image, but can achieve 16-fold, and even up to 32- to 50-fold acceleration depending on the set-up. Parallel imaging provides frameworks to be integrated in many of these methods for additional speed-up potential. The successful use of compressed sensing techniques and multi-contrast imaging with deep learning and parallel acquisition methods could yield significant MR acquisition speed-ups within clinical routines in the near future.

Learning the Sampling Pattern for MRI

The discovery of the theory of compressed sensing brought the realisation that many inverse problems can be solved even when measurements are "incomplete". This is particularly interesting in magnetic resonance imaging (MRI), where long acquisition times can limit its use. In this work, we consider the problem of learning a sparse sampling pattern that can be used to optimally balance acquisition time versus quality of the reconstructed image. We use a supervised learning approach, making the assumption that our training data is representative enough of new data acquisitions. We demonstrate that this is indeed the case, even if the training data consists of just 7 training pairs of measurements and ground-truth images; with a training set of brain images of size 192 by 192, for instance, one of the learned patterns samples only 35% of k-space, however results in reconstructions with mean SSIM 0.914 on a test set of similar images. The proposed framework is general enough to learn arbitrary sampling patterns, including common patterns such as Cartesian, spiral and radial sampling.

Deep Generalization of Structured Low-Rank Algorithms (Deep-SLR)

Structured low-rank (SLR) algorithms, which exploit annihilation relations between the Fourier samples of a signal resulting from different properties, is a powerful image reconstruction framework in several applications. This scheme relies on low-rank matrix completion to estimate the annihilation relations from the measurements. The main challenge with this strategy is the high computational complexity of matrix completion. We introduce a deep learning (DL) approach to significantly reduce the computational complexity. Specifically, we use a convolutional neural network (CNN)-based filterbank that is trained to estimate the annihilation relations from imperfect (under-sampled and noisy) k-space measurements of Magnetic Resonance Imaging (MRI). The main reason for the computational efficiency is the pre-learning of the parameters of the non-linear CNN from exemplar data, compared to SLR schemes that learn the linear filterbank parameters from the dataset itself. Experimental comparisons show that the proposed scheme can enable calibration-less parallel MRI; it can offer performance similar to SLR schemes while reducing the runtime by around three orders of magnitude. Unlike pre-calibrated and self-calibrated approaches, the proposed uncalibrated approach is insensitive to motion errors and affords higher acceleration. The proposed scheme also incorporates image domain priors that are complementary, thus significantly improving the performance over that of SLR schemes.