Manifold recovery using kernel low-rank regularization: application to dynamic imaging

Prospectively accelerated dynamic speech magnetic resonance imaging at 3 T using a self-navigated spiral-based manifold regularized scheme

This work develops and evaluates a self-navigated variable density spiral (VDS)-based manifold regularization scheme to prospectively improve dynamic speech magnetic resonance imaging (MRI) at 3 T. Short readout duration spirals (1.3-ms long) were used to minimize sensitivity to off-resonance. A custom 16-channel speech coil was used for improved parallel imaging of vocal tract structures. The manifold model leveraged similarities between frames sharing similar vocal tract postures without explicit motion binning. The self-navigating capability of VDS was leveraged to learn the Laplacian structure of the manifold. Reconstruction was posed as a sensitivity-encoding-based nonlocal soft-weighted temporal regularization scheme. Our approach was compared with view-sharing, low-rank, temporal finite difference, extra dimension-based sparsity reconstruction constraints. Undersampling experiments were conducted on five volunteers performing repetitive and arbitrary speaking tasks at different speaking rates. Quantitative evaluation in terms of mean square error over moving edges was performed in a retrospective undersampling experiment on one volunteer. For prospective undersampling, blinded image quality evaluation in the categories of alias artifacts, spatial blurring, and temporal blurring was performed by three experts in voice research. Region of interest analysis at articulator boundaries was performed in both experiments to assess articulatory motion. Improved performance with manifold reconstruction constraints was observed over existing constraints. With prospective undersampling, a spatial resolution of 2.4 × 2.4 mm2/pixel and a temporal resolution of 17.4 ms/frame for single-slice imaging, and 52.2 ms/frame for concurrent three-slice imaging, were achieved. We demonstrated implicit motion binning by analyzing the mechanics of the Laplacian matrix. Manifold regularization demonstrated superior image quality scores in reducing spatial and temporal blurring compared with all other reconstruction constraints. While it exhibited faint (nonsignificant) alias artifacts that were similar to temporal finite difference, it provided statistically significant improvements compared with the other constraints. In conclusion, the self-navigated manifold regularized scheme enabled robust high spatiotemporal resolution dynamic speech MRI at 3 T.

Deep learning for accelerated and robust MRI reconstruction

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.Affiliations [3 and 6] has been split into two different affiliations. Please check if action taken is appropriate and amend if necessary.looks good.

Technical Note: Swing golden angle - A navigator-interleaved golden angle trajectory with eddy current suppression - Application in free-running cardiac MRI

BACKGROUND

Golden angle (GA) radial trajectory is advantageous for dynamic magnetic resonance imaging (MRI). Recently, several advanced algorithms have been developed based on navigator-interleaved GA trajectory to realize free-running cardiac MRI. However, navigator-interleaved GA trajectory suffers from the eddy-current effect, which reduces the image quality.

PURPOSE

This work aims to integrate the navigator-interleaved GA trajectory with clinical cardiac MRI acquisition, with the minimum eddy-current artifacts. The ultimate goal is to realize a high-quality free-running cardiac imaging technique.

METHODS

In this paper, we propose a new "swing golden angle" (swingGA) radial profile order. SwingGA samples the k-space by rotating back and forth at the generalized golden ratio interval, with smoothly interleaved navigator readouts. The sampling efficiency and angle increment distributions were investigated by numerical simulations. Static phantom imaging experiments were conducted to evaluate the eddy current effect, compared with cartesian, golden angle radial (GA), and tiny golden angle (tGA) trajectories. Furthermore, 12 heart-healthy subjects (aged 21-25 years) were recruited for free-running cardiac imaging with different sampling trajectories. Dynamic images were reconstructed by a low-rank subspace-constrained algorithm. The image quality was evaluated by signal-to-noise-ratio and spectrum analysis in the heart region, and compared with traditional clinical cardiac MRI images.

RESULTS

SwingGA pattern achieves the highest sampling efficiency (mSE > 0.925) and the minimum azimuthal angle increment (mAD < 1.05). SwingGA can effectively suppress eddy currents in static phantom images, with the lowest normalized root mean square error (nRMSE) values among radial trajectories. For the in-vivo cardiac images, swingGA enjoys the highest SNR both in the blood pool and myocardium, and contains the minimum level of high-frequency artifacts. The free-running cardiac images have good consistency with traditional clinical cardiac MRI, and the swingGA sampling pattern achieves the best image quality among all sampling patterns.

CONCLUSIONS

The proposed swingGA sampling pattern can effectively improve the sampling efficiency and reduce the eddy currents for the navigator-interleaved GA sequence. SwingGA is a promising sampling pattern for free-running cardiac MRI.

Manifold Learning via Linear Tangent Space Alignment (LTSA) for Accelerated Dynamic MRI With Sparse Sampling

The spatial resolution and temporal frame-rate of dynamic magnetic resonance imaging (MRI) can be improved by reconstructing images from sparsely sampled k -space data with mathematical modeling of the underlying spatiotemporal signals. These models include sparsity models, linear subspace models, and non-linear manifold models. This work presents a novel linear tangent space alignment (LTSA) model-based framework that exploits the intrinsic low-dimensional manifold structure of dynamic images for accelerated dynamic MRI. The performance of the proposed method was evaluated and compared to state-of-the-art methods using numerical simulation studies as well as 2D and 3D in vivo cardiac imaging experiments. The proposed method achieved the best performance in image reconstruction among all the compared methods. The proposed method could prove useful for accelerating many MRI applications, including dynamic MRI, multi-parametric MRI, and MR spectroscopic imaging.

Variational Manifold Learning From Incomplete Data: Application to Multislice Dynamic MRI

Current deep learning-based manifold learning algorithms such as the variational autoencoder (VAE) require fully sampled data to learn the probability density of real-world datasets. However, fully sampled data is often unavailable in a variety of problems, including the recovery of dynamic and high-resolution magnetic resonance imaging (MRI). We introduce a novel variational approach to learn a manifold from undersampled data. The VAE uses a decoder fed by latent vectors, drawn from a conditional density estimated from the fully sampled images using an encoder. Since fully sampled images are not available in our setting, we approximate the conditional density of the latent vectors by a parametric model whose parameters are estimated from the undersampled measurements using back-propagation. We use the framework for the joint alignment and recovery of multi-slice free breathing and ungated cardiac MRI data from highly undersampled measurements. Experimental results demonstrate the utility of the proposed scheme in dynamic imaging alignment and reconstructions.

Dynamic Imaging Using Deep Bi-Linear Unsupervised Representation (DEBLUR)

Bilinear models such as low-rank and dictionary methods, which decompose dynamic data to spatial and temporal factor matrices are powerful and memory-efficient tools for the recovery of dynamic MRI data. Current bilinear methods rely on sparsity and energy compaction priors on the factor matrices to regularize the recovery. Motivated by deep image prior, we introduce a novel bilinear model, whose factor matrices are generated using convolutional neural networks (CNNs). The CNN parameters, and equivalently the factors, are learned from the undersampled data of the specific subject. Unlike current unrolled deep learning methods that require the storage of all the time frames in the dataset, the proposed approach only requires the storage of the factors or compressed representation; this approach allows the direct use of this scheme to large-scale dynamic applications, including free breathing cardiac MRI considered in this work. To reduce the run time and to improve performance, we initialize the CNN parameters using existing factor methods. We use sparsity regularization of the network parameters to minimize the overfitting of the network to measurement noise. Our experiments on free-breathing and ungated cardiac cine data acquired using a navigated golden-angle gradient-echo radial sequence show the ability of our method to provide reduced spatial blurring as compared to classical bilinear methods as well as a recent unsupervised deep-learning approach.

NC-PDNet: A Density-Compensated Unrolled Network for 2D and 3D Non-Cartesian MRI Reconstruction

Deep Learning has become a very promising avenue for magnetic resonance image (MRI) reconstruction. In this work, we explore the potential of unrolled networks for non-Cartesian acquisition settings. We design the NC-PDNet (Non-Cartesian Primal Dual Netwok), the first density-compensated (DCp) unrolled neural network, and validate the need for its key components via an ablation study. Moreover, we conduct some generalizability experiments to test this network in out-of-distribution settings, for example training on knee data and validating on brain data. The results show that NC-PDNet outperforms baseline (U-Net, Deep image prior) models both visually and quantitatively in all settings. In particular, in the 2D multi-coil acquisition scenario, the NC-PDNet provides up to a 1.2 dB improvement in peak signal-to-noise ratio (PSNR) over baseline networks, while also allowing a gain of at least 1dB in PSNR in generalization settings. We provide the open-source implementation of NC-PDNet, and in particular the Non-uniform Fourier Transform in TensorFlow, tested on 2D multi-coil and 3D single-coil k-space data.

Time-Dependent Deep Image Prior for Dynamic MRI

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. We introduce a generalized version of the deep-image-prior approach, which optimizes the weights of a reconstruction network to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in k -space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.

Learning Data Consistency and its Application to Dynamic MR Imaging

Magnetic resonance (MR) image reconstruction from undersampled k-space data can be formulated as a minimization problem involving data consistency and image prior. Existing deep learning (DL)-based methods for MR reconstruction employ deep networks to exploit the prior information and integrate the prior knowledge into the reconstruction under the explicit constraint of data consistency, without considering the real distribution of the noise. In this work, we propose a new DL-based approach termed Learned DC that implicitly learns the data consistency with deep networks, corresponding to the actual probability distribution of system noise. The data consistency term and the prior knowledge are both embedded in the weights of the networks, which provides an utterly implicit manner of learning reconstruction model. We evaluated the proposed approach with highly undersampled dynamic data, including the dynamic cardiac cine data with up to 24-fold acceleration and dynamic rectum data with the acceleration factor equal to the number of phases. Experimental results demonstrate the superior performance of the Learned DC both quantitatively and qualitatively than the state-of-the-art.

Dynamic Imaging Using a Deep Generative SToRM (Gen-SToRM) Model

We introduce a generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The model assumes that the images in the dataset are non-linear mappings of low-dimensional latent vectors. We use the deep convolutional neural network (CNN) to represent the non-linear transformation. The parameters of the generator as well as the low-dimensional latent vectors are jointly estimated only from the undersampled measurements. This approach is different from traditional CNN approaches that require extensive fully sampled training data. We penalize the norm of the gradients of the non-linear mapping to constrain the manifold to be smooth, while temporal gradients of the latent vectors are penalized to obtain a smoothly varying time-series. The proposed scheme brings in the spatial regularization provided by the convolutional network. The main benefit of the proposed scheme is the improvement in image quality and the orders-of-magnitude reduction in memory demand compared to traditional manifold models. To minimize the computational complexity of the algorithm, we introduce an efficient progressive training-in-time approach and an approximate cost function. These approaches speed up the image reconstructions and offers better reconstruction performance.

Recovery of surfaces and functions in high dimensions: sampling theory and links to neural networks

Several imaging algorithms including patch-based image denoising, image time series recovery, and convolutional neural networks can be thought of as methods that exploit the manifold structure of signals. While the empirical performance of these algorithms is impressive, the understanding of recovery of the signals and functions that live on manifold is less understood. In this paper, we focus on the recovery of signals that live on a union of surfaces. In particular, we consider signals living on a union of smooth band-limited surfaces in high dimensions. We show that an exponential mapping transforms the data to a union of low-dimensional subspaces. Using this relation, we introduce a sampling theoretical framework for the recovery of smooth surfaces from few samples and the learning of functions living on smooth surfaces. The low-rank property of the features is used to determine the number of measurements needed to recover the surface. Moreover, the low-rank property of the features also provides an efficient approach, which resembles a neural network, for the local representation of multidimensional functions on the surface. The direct representation of such a function in high dimensions often suffers from the curse of dimensionality; the large number of parameters would translate to the need for extensive training data. The low-rank property of the features can significantly reduce the number of parameters, which makes the computational structure attractive for learning and inference from limited labeled training data.

DEEP GENERATIVE STORM MODEL FOR DYNAMIC IMAGING

We introduce a novel generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The proposed generative framework represents the image time series as a smooth non-linear function of low-dimensional latent vectors that capture the cardiac and respiratory phases. The non-linear function is represented using a deep convolutional neural network (CNN). Unlike the popular CNN approaches that require extensive fully-sampled training data that is not available in this setting, the parameters of the CNN generator as well as the latent vectors are jointly estimated from the undersampled measurements using stochastic gradient descent. We penalize the norm of the gradient of the generator to encourage the learning of a smooth surface/manifold, while temporal gradients of the latent vectors are penalized to encourage the time series to be smooth. The main benefits of the proposed scheme are (a) the quite significant reduction in memory demand compared to the analysis based SToRM model, and (b) the spatial regularization brought in by the CNN model. We also introduce efficient progressive approaches to minimize the computational complexity of the algorithm.

Free-Breathing and Ungated Dynamic MRI Using Navigator-Less Spiral SToRM

We introduce a kernel low-rank algorithm to recover free-breathing and ungated dynamic MRI from spiral acquisitions without explicit k-space navigators. It is often challenging for low-rank methods to recover free-breathing and ungated images from undersampled measurements; extensive cardiac and respiratory motion often results in the Casorati matrix not being sufficiently low-rank. Therefore, we exploit the non-linear structure of the dynamic data, which gives the low-rank kernel matrix. Unlike prior work that rely on navigators to estimate the manifold structure, we propose a kernel low-rank matrix completion method to directly fill in the missing k-space data from variable density spiral acquisitions. We validate the proposed scheme using simulated data and in-vivo data. Our results show that the proposed scheme provides improved reconstructions compared to the classical methods such as low-rank and XD-GRASP. The comparison with breath-held cine data shows that the quantitative metrics agree, whereas the image quality is marginally lower.

High-Resolution Oscillating Steady-State fMRI Using Patch-Tensor Low-Rank Reconstruction

The goals of fMRI acquisition include high spatial and temporal resolutions with a high signal to noise ratio (SNR). Oscillating Steady-State Imaging (OSSI) is a new fMRI acquisition method that provides large oscillating signals with the potential for high SNR, but does so at the expense of spatial and temporal resolutions. The unique oscillation pattern of OSSI images makes it well suited for high-dimensional modeling. We propose a patch-tensor low-rank model to exploit the local spatial-temporal low-rankness of OSSI images. We also develop a practical sparse sampling scheme with improved sampling incoherence for OSSI. With an alternating direction method of multipliers (ADMM) based algorithm, we improve OSSI spatial and temporal resolutions with a factor of 12 acquisition acceleration and 1.3 mm isotropic spatial resolution in prospectively undersampled experiments. The proposed model yields high temporal SNR with more activation than other low-rank methods. Compared to the standard grad- ient echo (GRE) imaging with the same spatial-temporal resolution, 3D OSSI tensor model reconstruction demonstrates 2 times higher temporal SNR with 2 times more functional activation.

DYNAMIC MRI USING DEEP MANIFOLD SELF-LEARNING

We propose a deep self-learning algorithm to learn the manifold structure of free-breathing and ungated cardiac data and to recover the cardiac CINE MRI from highly undersampled measurements. Our method learns the manifold structure in the dynamic data from navigators using autoencoder network. The trained autoencoder is then used as a prior in the image reconstruction framework. We have tested the proposed method on free-breathing and ungated cardiac CINE data, which is acquired using a navigated golden-angle gradient-echo radial sequence. Results show the ability of our method to better capture the manifold structure, thus providing us reduced spatial and temporal blurring as compared to the SToRM reconstruction.

Reconstruction of Binary Shapes from Blurred Images via Hankel-structured Low-rank Matrix Recovery

With the dominance of digital imaging systems, we are often dealing with discrete-domain samples of an analog image. Due to physical limitations, all imaging devices apply a blurring kernel on the input image before taking samples to form the output pixels. In this paper, we focus on the reconstruction of binary shape images from few blurred samples. This problem has applications in medical imaging, shape processing, and image segmentation. Our method relies on representing the analog shape image in a discrete grid much finer than the sampling grid. We formulate the problem as the recovery of a rank r matrix that is formed by a Hankel structure on the pixels. We further propose efficient ADMM-based algorithms to recover the low-rank matrix in both noiseless and noisy settings. We also analytically investigate the number of required samples for successful recovery in the noiseless case. For this purpose, we study the problem in the random sampling framework, and show that with O(r log4(n1n2)) random samples (where the size of the image is assumed to be n1 x n2) we can guarantee the perfect reconstruction with high probability under mild conditions. We further prove the robustness of the proposed recovery in the noisy setting by showing that the reconstruction error in the noisy case is bounded when the input noise is bounded. Simulation results confirm that our proposed method outperform the conventional total variation minimization in the noiseless settings.