Image reconstruction of compressed sensing MRI using graph-based redundant wavelet transform

Deep plug-and-play MRI reconstruction based on multiple complementary priors

Magnetic resonance imaging (MRI) is widely used in clinical diagnosis as a safe, non-invasive, high-resolution medical imaging technology, but long scanning time has been a major challenge for this technology. The undersampling reconstruction method has become an important technical means to accelerate MRI by reducing the data sampling rate while maintaining high-quality imaging. However, traditional undersampling reconstruction techniques such as compressed sensing mainly rely on relatively single sparse or low-rank prior information to reconstruct the image, which has limitations in capturing the comprehensive features of images, resulting in the insufficient performance of the reconstructed image in terms of details and key information. In this paper, we propose a deep plug-and-play multiple complementary priors MRI reconstruction model, which combines traditional low-rank matrix recovery model methods and deep learning methods, and integrates global, local and nonlocal priors to improve reconstruction quality. Specifically, we capture the global features of the image through the matrix nuclear norm, and use the deep convolutional neural network denoiser Swin-Conv-UNet (SCUNet) and block-matching and 3-D filtering (BM3D) algorithm to preserve the local details and structural texture of the image, respectively. In addition, we utilize an efficient half-quadratic splitting (HQS) algorithm to solve the proposed model. The experimental results show that our proposed method has better reconstruction ability than the existing popular methods in terms of visual effects and numerical results.

Deep unfolding network with spatial alignment for multi-modal MRI reconstruction

Multi-modal Magnetic Resonance Imaging (MRI) offers complementary diagnostic information, but some modalities are limited by the long scanning time. To accelerate the whole acquisition process, MRI reconstruction of one modality from highly under-sampled k-space data with another fully-sampled reference modality is an efficient solution. However, the misalignment between modalities, which is common in clinic practice, can negatively affect reconstruction quality. Existing deep learning-based methods that account for inter-modality misalignment perform better, but still share two main common limitations: (1) The spatial alignment task is not adaptively integrated with the reconstruction process, resulting in insufficient complementarity between the two tasks; (2) the entire framework has weak interpretability. In this paper, we construct a novel Deep Unfolding Network with Spatial Alignment, termed DUN-SA, to appropriately embed the spatial alignment task into the reconstruction process. Concretely, we derive a novel joint alignment-reconstruction model with a specially designed aligned cross-modal prior term. By relaxing the model into cross-modal spatial alignment and multi-modal reconstruction tasks, we propose an effective algorithm to solve this model alternatively. Then, we unfold the iterative stages of the proposed algorithm and design corresponding network modules to build DUN-SA with interpretability. Through end-to-end training, we effectively compensate for spatial misalignment using only reconstruction loss, and utilize the progressively aligned reference modality to provide inter-modality prior to improve the reconstruction of the target modality. Comprehensive experiments on four real datasets demonstrate that our method exhibits superior reconstruction performance compared to state-of-the-art methods.

MRI reconstruction with enhanced self-similarity using graph convolutional network

BACKGROUND

Recent Convolutional Neural Networks (CNNs) perform low-error reconstruction in fast Magnetic Resonance Imaging (MRI). Most of them convolve the image with kernels and successfully explore the local information. Nonetheless, the non-local image information, which is embedded among image patches relatively far from each other, may be lost due to the limitation of the receptive field of the convolution kernel. We aim to incorporate a graph to represent non-local information and improve the reconstructed images by using the Graph Convolutional Enhanced Self-Similarity (GCESS) network.

METHODS

First, the image is reconstructed into the graph to extract the non-local self-similarity in the image. Second, GCESS uses spatial convolution and graph convolution to process the information in the image, so that local and non-local information can be effectively utilized. The network strengthens the non-local similarity between similar image patches while reconstructing images, making the reconstruction of structure more reliable.

RESULTS

Experimental results on in vivo knee and brain data demonstrate that the proposed method achieves better artifact suppression and detail preservation than state-of-the-art methods, both visually and quantitatively. Under 1D Cartesian sampling with 4 × acceleration (AF = 4), the PSNR of knee data reached 34.19 dB, 1.05 dB higher than that of the compared methods; the SSIM achieved 0.8994, 2% higher than the compared methods. Similar results were obtained for the reconstructed images under other sampling templates as demonstrated in our experiment.

CONCLUSIONS

The proposed method successfully constructs a hybrid graph convolution and spatial convolution network to reconstruct images. This method, through its training process, amplifies the non-local self-similarities, significantly benefiting the structural integrity of the reconstructed images. Experiments demonstrate that the proposed method outperforms the state-of-the-art reconstruction method in suppressing artifacts, as well as in preserving image details.

IWNeXt: an image-wavelet domain ConvNeXt-based network for self-supervised multi-contrast MRI reconstruction

Objective.Multi-contrast magnetic resonance imaging (MC MRI) can obtain more comprehensive anatomical information of the same scanning object but requires a longer acquisition time than single-contrast MRI. To accelerate MC MRI speed, recent studies only collect partial k-space data of one modality (target contrast) to reconstruct the remaining non-sampled measurements using a deep learning-based model with the assistance of another fully sampled modality (reference contrast). However, MC MRI reconstruction mainly performs the image domain reconstruction with conventional CNN-based structures by full supervision. It ignores the prior information from reference contrast images in other sparse domains and requires fully sampled target contrast data. In addition, because of the limited receptive field, conventional CNN-based networks are difficult to build a high-quality non-local dependency.Approach.In the paper, we propose an Image-Wavelet domain ConvNeXt-based network (IWNeXt) for self-supervised MC MRI reconstruction. Firstly, INeXt and WNeXt based on ConvNeXt reconstruct undersampled target contrast data in the image domain and refine the initial reconstructed result in the wavelet domain respectively. To generate more tissue details in the refinement stage, reference contrast wavelet sub-bands are used as additional supplementary information for wavelet domain reconstruction. Then we design a novel attention ConvNeXt block for feature extraction, which can capture the non-local information of the MC image. Finally, the cross-domain consistency loss is designed for self-supervised learning. Especially, the frequency domain consistency loss deduces the non-sampled data, while the image and wavelet domain consistency loss retain more high-frequency information in the final reconstruction.Main results.Numerous experiments are conducted on the HCP dataset and the M4Raw dataset with different sampling trajectories. Compared with DuDoRNet, our model improves by 1.651 dB in the peak signal-to-noise ratio.Significance.IWNeXt is a potential cross-domain method that can enhance the accuracy of MC MRI reconstruction and reduce reliance on fully sampled target contrast images.

A practical evaluation of measures derived from compressed sensing diffusion spectrum imaging

Diffusion Spectrum Imaging (DSI) using dense Cartesian sampling of q-space has been shown to provide important advantages for modeling complex white matter architecture. However, its adoption has been limited by the lengthy acquisition time required. Sparser sampling of q-space combined with compressed sensing (CS) reconstruction techniques has been proposed as a way to reduce the scan time of DSI acquisitions. However prior studies have mainly evaluated CS-DSI in post-mortem or non-human data. At present, the capacity for CS-DSI to provide accurate and reliable measures of white matter anatomy and microstructure in the living human brain remains unclear. We evaluated the accuracy and inter-scan reliability of 6 different CS-DSI schemes that provided up to 80% reductions in scan time compared to a full DSI scheme. We capitalized on a dataset of 26 participants who were scanned over eight independent sessions using a full DSI scheme. From this full DSI scheme, we subsampled images to create a range of CS-DSI images. This allowed us to compare the accuracy and inter-scan reliability of derived measures of white matter structure (bundle segmentation, voxel-wise scalar maps) produced by the CS-DSI and the full DSI schemes. We found that CS-DSI estimates of both bundle segmentations and voxel-wise scalars were nearly as accurate and reliable as those generated by the full DSI scheme. Moreover, we found that the accuracy and reliability of CS-DSI was higher in white matter bundles that were more reliably segmented by the full DSI scheme. As a final step, we replicated the accuracy of CS-DSI in a prospectively acquired dataset (n = 20, scanned once). Together, these results illustrate the utility of CS-DSI for reliably delineating in vivo white matter architecture in a fraction of the scan time, underscoring its promise for both clinical and research applications.

WRANet: wavelet integrated residual attention U-Net network for medical image segmentation

Medical image segmentation is crucial for the diagnosis and analysis of disease. Deep convolutional neural network methods have achieved great success in medical image segmentation. However, they are highly susceptible to noise interference during the propagation of the network, where weak noise can dramatically alter the network output. As the network deepens, it can face problems such as gradient explosion and vanishing. To improve the robustness and segmentation performance of the network, we propose a wavelet residual attention network (WRANet) for medical image segmentation. We replace the standard downsampling modules (e.g., maximum pooling and average pooling) in CNNs with discrete wavelet transform, decompose the features into low- and high-frequency components, and remove the high-frequency components to eliminate noise. At the same time, the problem of feature loss can be effectively addressed by introducing an attention mechanism. The combined experimental results show that our method can effectively perform aneurysm segmentation, achieving a Dice score of 78.99%, an IoU score of 68.96%, a precision of 85.21%, and a sensitivity score of 80.98%. In polyp segmentation, a Dice score of 88.89%, an IoU score of 81.74%, a precision rate of 91.32%, and a sensitivity score of 91.07% were achieved. Furthermore, our comparison with state-of-the-art techniques demonstrates the competitiveness of the WRANet network.

AutoBCS: Block-Based Image Compressive Sensing With Data-Driven Acquisition and Noniterative Reconstruction

Block compressive sensing (CS) is a well-known signal acquisition and reconstruction paradigm with widespread application prospects in science, engineering, and cybernetic systems. However, state-of-the-art block-based image CS (BCS) methods generally suffer from two issues. The sparsifying domain and the sensing matrices widely used for image acquisition are not data driven and, thus, both the features of the image and the relationships among subblock images are ignored. Moreover, it requires to address a high-dimensional optimization problem with extensive computational complexity for image reconstruction. In this article, we provide a deep learning (DL) strategy for BCS, called AutoBCS, which automatically takes the prior knowledge of images into account in the acquisition step and establishes a reconstruction model for performing fast image reconstruction. More precisely, we present a learning-based sensing matrix to accomplish image acquisition, thereby capturing and preserving more image characteristics than those captured by the existing methods. In addition, we build a noniterative reconstruction network, which provides an end-to-end BCS reconstruction framework to maximize image reconstruction efficiency. Furthermore, we investigate comprehensive comparison studies with both traditional BCS approaches and newly developed DL methods. Compared with these approaches, our proposed AutoBCS can not only provide superior performance in terms of image quality metrics (SSIM and PSNR) and visual perception but also automatically benefit reconstruction speed.

Undersampled MRI reconstruction based on spectral graph wavelet transform

Compressed sensing magnetic resonance imaging (CS-MRI) has exhibited great potential to accelerate magnetic resonance imaging if an image can be sparsely represented. How to sparsify the image significantly affects the reconstruction quality of images. In this paper, a spectral graph wavelet transform (SGWT) is introduced to sparsely represent magnetic resonance images in iterative image reconstructions. The SGWT is achieved by extending the traditional wavelets transform to the signal defined on the vertices of the weighted graph, i.e. the spectral graph domain. This SGWT uses only the connectivity information encoded in the edge weights, and does not rely on any other attributes of the vertices. Therefore, SGWT can be defined and calculated for any domain where the underlying relations between data locations can be represented by a weighted graph. Furthermore, we present a Chebyshev polynomial approximation algorithm for fast computing this SGWT transform. The l₁ norm regularized CS-MRI reconstruction model is introduced and solved by the projected iterative soft-thresholding algorithm to verify its feasibility. Numerical experiment results demonstrate that our proposed method outperforms several state-of-the-art sparsify transforms in terms of suppressing artifacts and achieving lower reconstruction errors on the tested datasets.

MR image reconstruction from undersampled data for image-guided radiation therapy using a patient-specific deep manifold image prior

Introduction

Recent advancements in radiotherapy (RT) have allowed for the integration of a Magnetic Resonance (MR) imaging scanner with a medical linear accelerator to use MR images for image guidance to position tumors against the treatment beam. Undersampling in MR acquisition is desired to accelerate the imaging process, but unavoidably deteriorates the reconstructed image quality. In RT, a high-quality MR image of a patient is available for treatment planning. In light of this unique clinical scenario, we proposed to exploit the patient-specific image prior to facilitate high-quality MR image reconstruction.

Methods

Utilizing the planning MR image, we established a deep auto-encoder to form a manifold of image patches of the patient. The trained manifold was then incorporated as a regularization to restore MR images of the same patient from undersampled data. We performed a simulation study using a patient case, a real patient study with three liver cancer patient cases, and a phantom experimental study using data acquired on an in-house small animal MR scanner. We compared the performance of the proposed method with those of the Fourier transform method, a tight-frame based Compressive Sensing method, and a deep learning method with a patient-generic manifold as the image prior.

Results

In the simulation study with 12.5% radial undersampling and 15% increase in noise, our method improved peak-signal-to-noise ratio by 4.46dB and structural similarity index measure by 28% compared to the patient-generic manifold method. In the experimental study, our method outperformed others by producing reconstructions of visually improved image quality.

Accelerated MRI Reconstruction With Separable and Enhanced Low-Rank Hankel Regularization

Magnetic resonance imaging serves as an essential tool for clinical diagnosis, however, suffers from a long acquisition time. Sparse sampling effectively saves this time but images need to be faithfully reconstructed from undersampled data. Among the existing reconstruction methods, the structured low-rank methods have advantages in robustness to the sampling patterns and lower error. However, the structured low-rank methods use the 2D or higher dimension k-space data to build a huge block Hankel matrix, leading to considerable time and memory consumption. To reduce the size of the Hankel matrix, we proposed to separably construct multiple small Hankel matrices from rows and columns of the k-space and then constrain the low-rankness on these small matrices. This separable model can significantly reduce the computational time but ignores the correlation existed in inter- and intra-row or column, resulting in increased reconstruction error. To improve the reconstructed image without obviously increasing the computation, we further introduced the self-consistency of k-space and virtual coil prior. Besides, the proposed separable model can be extended into other imaging scenarios which hold exponential characteristics in the parameter dimension. The in vivo experimental results demonstrated that the proposed method permits the lowest reconstruction error with a fast reconstruction. The proposed approach requires only 4% of the state-of-the-art STDLR-SPIRiT runtime for parallel imaging reconstruction, and achieves the fastest computational speed in parameter imaging reconstruction.

Study on Key Technologies of Virtual Interactive Surgical Simulation for 3D Reconstruction of Medical Images

Medical computed tomography (CT), nuclear magnetic resonance imaging (MRI), and other imaging facility produce a large amount of medical image data which has great diagnosis value. Traditional three-dimensional reconstruction of medical images has high requirements for graphics acceleration hardware and the processing speed. In this study, VxWorks embedded real-time process feature is used for CT or MRI DICOM data to real restoration and establishment of virtual three-dimensional model for realizing volume reconstruction, maximum density projection, multiplane reconstruction, dynamic interactive cutting of any surface, dynamic display of three-dimensional model and two-dimensional sectional image, surgical path planning and interactive surgical simulation, and determine the best surgical scheme. The practical application shows that the virtual simulation environment supports the seamless transplantation of code, function debugging, and interaction and solves the issue of high requirements for hardware. It can meet the needs of scientific research and teaching for clinicians and medical imaging workers and can be widely used in the training of virtual surgical anatomy for medical students.

Lossless Medical Image Compression by Using Difference Transform

This paper introduces a new method of compressing digital images by using the Difference Transform applied in medical imaging. The Difference Transform algorithm performs the decorrelation process of image data, and in this way improves the encoding process, achieving a file with a smaller size than the original. The proposed method proves to be competitive and in many cases better than the standards used for medical images such as TIFF or PNG. In addition, the Difference Transform can replace other transforms like Cosine or Wavelet.

MRI Reconstruction with Separate Magnitude and Phase Priors Based on Dual-Tree Complex Wavelet Transform

The methods of compressed sensing magnetic resonance imaging (CS-MRI) can be divided into two categories roughly based on the number of target variables. One group devotes to estimating the complex-valued MRI image. And the other calculates the magnitude and phase parts of the complex-valued MRI image, respectively, by enforcing separate penalties on them. We propose a new CS-based method based on dual-tree complex wavelet (DT CWT) sparsity, which is under the frame of the second class of CS-MRI. Owing to the separate regularization frame, this method reduces the impact of the phase jumps (that means the jumps or discontinuities of phase values) on magnitude reconstruction. Moreover, by virtue of the excellent features of DT CWT, such as nonoscillating envelope of coefficients and multidirectional selectivity, the proposed method is capable of capturing more details in the magnitude and phase images. The experimental results show that the proposed method recovers the image contour and edges information well and can eliminate the artifacts in magnitude results caused by phase jumps.

Discrete Shearlets as a Sparsifying Transform in Low-Rank Plus Sparse Decomposition for Undersampled (k, t)-Space MR Data

The discrete shearlet transformation accurately represents the discontinuities and edges occurring in magnetic resonance imaging, providing an excellent option of a sparsifying transform. In the present paper, we examine the use of discrete shearlets over other sparsifying transforms in a low-rank plus sparse decomposition problem, denoted by L+S. The proposed algorithm is evaluated on simulated dynamic contrast enhanced (DCE) and small bowel data. For the small bowel, eight subjects were scanned; the sequence was run first on breath-holding and subsequently on free-breathing, without changing the anatomical position of the subject. The reconstruction performance of the proposed algorithm was evaluated against k-t FOCUSS. L+S decomposition, using discrete shearlets as sparsifying transforms, successfully separated the low-rank (background and periodic motion) from the sparse component (enhancement or bowel motility) for both DCE and small bowel data. Motion estimated from low-rank of DCE data is closer to ground truth deformations than motion estimated from L and S. Motility metrics derived from the S component of free-breathing data were not significantly different from the ones from breath-holding data up to four-fold undersampling, indicating that bowel (rapid/random) motility is isolated in S. Our work strongly supports the use of discrete shearlets as a sparsifying transform in a L+S decomposition for undersampled MR data.

A review on deep learning MRI reconstruction without fully sampled k-space

BACKGROUND

Magnetic resonance imaging (MRI) is an effective auxiliary diagnostic method in clinical medicine, but it has always suffered from the problem of long acquisition time. Compressed sensing and parallel imaging are two common techniques to accelerate MRI reconstruction. Recently, deep learning provides a new direction for MRI, while most of them require a large number of data pairs for training. However, there are many scenarios where fully sampled k-space data cannot be obtained, which will seriously hinder the application of supervised learning. Therefore, deep learning without fully sampled data is indispensable.

MAIN TEXT

In this review, we first introduce the forward model of MRI as a classic inverse problem, and briefly discuss the connection of traditional iterative methods to deep learning. Next, we will explain how to train reconstruction network without fully sampled data from the perspective of obtaining prior information.

CONCLUSION

Although the reviewed methods are used for MRI reconstruction, they can also be extended to other areas where ground-truth is not available. Furthermore, we may anticipate that the combination of traditional methods and deep learning will produce better reconstruction results.

Homotopic Gradients of Generative Density Priors for MR Image Reconstruction

Deep learning, particularly the generative model, has demonstrated tremendous potential to significantly speed up image reconstruction with reduced measurements recently. Rather than the existing generative models that often optimize the density priors, in this work, by taking advantage of the denoising score matching, homotopic gradients of generative density priors (HGGDP) are exploited for magnetic resonance imaging (MRI) reconstruction. More precisely, to tackle the low-dimensional manifold and low data density region issues in generative density prior, we estimate the target gradients in higher-dimensional space. We train a more powerful noise conditional score network by forming high-dimensional tensor as the network input at the training phase. More artificial noise is also injected in the embedding space. At the reconstruction stage, a homotopy method is employed to pursue the density prior, such as to boost the reconstruction performance. Experiment results implied the remarkable performance of HGGDP in terms of high reconstruction accuracy. Only 10% of the k-space data can still generate image of high quality as effectively as standard MRI reconstructions with the fully sampled data.

Robust brain MR image compressive sensing via re-weighted total variation and sparse regression

Total variation (TV) and non-local self-similarity (NSS) are powerful tools for successfully enhancing compressive sensing performance. However, standard TV approaches often over-smooth detailed edges in the image, due to the uniform regularization of gradient magnitude. In this paper, a novel compressed sensing method for the reconstruction of medical images is proposed, the image edges are well preserved with the proposed reweighted TV. The redundancy of the NSS patch also is leveraged through the sparse regression model. The proposed model was solved with an efficient strategy of the Alternating Direction Method of Multipliers (ADMM) algorithm. Experimental results on thesimulated phantom, brain Magnetic resonance imaging (MRI) show that the proposed method outperforms the state-of-the-art compressed sensing approaches.

Accelerated image reconstruction with separable Hankel regularization in parallel MRI

Magnetic resonance imaging has been widely adopted in clinical diagnose, however, it suffers from relatively long data acquisition time. Sparse sampling with reconstruction can speed up the data acquisition duration. As the state-of-the-art magnetic resonance imaging methods, the structured low rank reconstruction approaches embrace the advantage of holding low reconstruction errors and permit flexible undersampling patterns. However, this type of method demands intensive computations and high memory consumptions, thereby resulting in a lengthy reconstruction time. In this work, we proposed a separable Hankel low rank reconstruction method to explore the low rankness of each row and each column. Furthermore, we utilized the self-consistence and conjugate symmetry property of k-space data. The experimental results demonstrated that the proposed method outperforms the state-of-the-art approaches in terms of lower reconstruction errors and better detail preservation. Besides, the proposed method requires much less computation and memory consumption.Clinical Relevance- Parallel imaging, image reconstruction, Hankel low-rank.

Transfer learning enhanced generative adversarial networks for multi-channel MRI reconstruction

Deep learning based generative adversarial networks (GAN) can effectively perform image reconstruction with under-sampled MR data. In general, a large number of training samples are required to improve the reconstruction performance of a certain model. However, in real clinical applications, it is difficult to obtain tens of thousands of raw patient data to train the model since saving k-space data is not in the routine clinical flow. Therefore, enhancing the generalizability of a network based on small samples is urgently needed. In this study, three novel applications were explored based on parallel imaging combined with the GAN model (PI-GAN) and transfer learning. The model was pre-trained with public Calgary brain images and then fine-tuned for use in (1) patients with tumors in our center; (2) different anatomies, including knee and liver; (3) different k-space sampling masks with acceleration factors (AFs) of 2 and 6. As for the brain tumor dataset, the transfer learning results could remove the artifacts found in PI-GAN and yield smoother brain edges. The transfer learning results for the knee and liver were superior to those of the PI-GAN model trained with its own dataset using a smaller number of training cases. However, the learning procedure converged more slowly in the knee datasets compared to the learning in the brain tumor datasets. The reconstruction performance was improved by transfer learning both in the models with AFs of 2 and 6. Of these two models, the one with AF = 2 showed better results. The results also showed that transfer learning with the pre-trained model could solve the problem of inconsistency between the training and test datasets and facilitate generalization to unseen data.

Synergistic multi-contrast cardiac magnetic resonance image reconstruction

Cardiac magnetic resonance imaging (CMR) is an important tool for the non-invasive diagnosis of a variety of cardiovascular diseases. Parametric mapping with multi-contrast CMR is able to quantify tissue alterations in myocardial disease and promises to improve patient care. However, magnetic resonance imaging is an inherently slow imaging modality, resulting in long acquisition times for parametric mapping which acquires a series of cardiac images with different contrasts for signal fitting or dictionary matching. Furthermore, extra efforts to deal with respiratory and cardiac motion by triggering and gating further increase the scan time. Several techniques have been developed to speed up CMR acquisitions, which usually acquire less data than that required by the Nyquist-Shannon sampling theorem, followed by regularized reconstruction to mitigate undersampling artefacts. Recent advances in CMR parametric mapping speed up CMR by synergistically exploiting spatial-temporal and contrast redundancies. In this article, we will review the recent developments in multi-contrast CMR image reconstruction for parametric mapping with special focus on low-rank and model-based reconstructions. Deep learning-based multi-contrast reconstruction has recently been proposed in other magnetic resonance applications. These developments will be covered to introduce the general methodology. Current technical limitations and potential future directions are discussed. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Spatiotemporal Flexible Sparse Reconstruction for Rapid Dynamic Contrast-enhanced MRI

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a tissue perfusion imaging technique. Some versatile free-breathing DCE-MRI techniques combining compressed sensing (CS) and parallel imaging with golden-angle radial sampling have been developed to improve motion robustness with high spatial and temporal resolution. These methods have demonstrated good diagnostic performance in clinical setting, but the reconstruction quality will degrade at high acceleration rates and overall reconstruction time remains long. In this paper, we proposed a new parallel CS reconstruction model for DCE-MRI that enforces flexible weighted sparse constraint along both spatial and temporal dimensions. Weights were introduced to flexibly adjust the importance of time and space sparsity, and we derived a fast-thresholding algorithm which was proven to be simple and efficient for solving the proposed reconstruction model. Results on both the brain tumor DCE and liver DCE show that, at relatively high acceleration factor of fast sampling, lowest reconstruction error and highest image structural similarity are obtained by the proposed method. Besides, the proposed method achieves faster reconstruction for liver datasets and better physiological measures are also obtained on tumor images.

Deep unregistered multi-contrast MRI reconstruction

Multiple magnetic resonance images of different contrasts are normally acquired for clinical diagnosis. Recently, research has shown that the previously acquired multi-contrast (MC) images of the same patient can be used as anatomical prior to accelerating magnetic resonance imaging (MRI). However, current MC-MRI networks are based on the assumption that the images are perfectly registered, which is rarely the case in real-world applications. In this paper, we propose an end-to-end deep neural network to reconstruct highly accelerated images by exploiting the shareable information from potentially misaligned reference images of an arbitrary contrast. Specifically, a spatial transformation (ST) module is designed and integrated into the reconstruction network to align the pre-acquired reference images with the images to be reconstructed. The misalignment is further alleviated by maximizing the normalized cross-correlation (NCC) between the MC images. The visualization of feature maps demonstrates that the proposed method effectively reduces the misalignment between the images for shareable information extraction when applied to the publicly available brain datasets. Additionally, the experimental results on these datasets show the proposed network allows the robust exploitation of shareable information across the misaligned MC images, leading to improved reconstruction results.

Radial Undersampling-Based Interpolation Scheme for Multislice CSMRI Reconstruction Techniques

Magnetic Resonance Imaging (MRI) is an important yet slow medical imaging modality. Compressed sensing (CS) theory has enabled to accelerate the MRI acquisition process using some nonlinear reconstruction techniques from even 10% of the Nyquist samples. In recent years, interpolated compressed sensing (iCS) has further reduced the scan time, as compared to CS, by exploiting the strong interslice correlation of multislice MRI. In this paper, an improved efficient interpolated compressed sensing (EiCS) technique is proposed using radial undersampling schemes. The proposed efficient interpolation technique uses three consecutive slices to estimate the missing samples of the central target slice from its two neighboring slices. Seven different evaluation metrics are used to analyze the performance of the proposed technique such as structural similarity index measurement (SSIM), feature similarity index measurement (FSIM), mean square error (MSE), peak signal to noise ratio (PSNR), correlation (CORR), sharpness index (SI), and perceptual image quality evaluator (PIQE) and compared with the latest interpolation techniques. The simulation results show that the proposed EiCS technique has improved image quality and performance using both golden angle and uniform angle radial sampling patterns, with an even lower sampling ratio and maximum information content and using a more practical sampling scheme.

A dual-task dual-domain model for blind MRI reconstruction

MRI reconstruction is the key technology to accelerate MR acquisition. Recent cascade models have gained satisfactory results, however, they deeply rely on the known sample mask, which we call it mask prior. To restore the MR image without mask prior, we designed an auxiliary network to estimate the mask from sampled k-space data. Experimentally, the sample mask can be completely estimated by the proposed network and be used to input to the cascade models. Moreover, we rethink the MRI reconstruction model as a k-space inpainting task. A dual-domain cascade network, which utilized partial convolutional layers to inpaint features in k-space, was presented to restore the MR image. Without the mask prior, our blind reconstruction model demonstrates the best reconstruction ability in both 4x acceleration and 8x acceleration.

Joint low-rank prior and difference of Gaussian filter for magnetic resonance image denoising

The low-rank matrix approximation (LRMA) is an efficient image denoising method to reduce additive Gaussian noise. However, the existing low-rank matrix approximation does not perform well in terms of Rician noise removal for magnetic resonance imaging (MRI). To this end, we propose a novel MR image denoising approach based on the extended difference of Gaussian (DoG) filter and nonlocal low-rank regularization. In the proposed method, a novel nonlocal self-similarity evaluation with the tight frame is exploited to improve the patch matching. To remove the Rician noise and preserve the edge details, the extended DoG filter is exploited to the nonlocal low-rank regularization model. The experimental results demonstrate that the proposed method can preserve more edge and fine structures while removing noise in MR image as compared with some of the existing methods.

A guaranteed convergence analysis for the projected fast iterative soft-thresholding algorithm in parallel MRI

Sparse sampling and parallel imaging techniques are two effective approaches to alleviate the lengthy magnetic resonance imaging (MRI) data acquisition problem. Promising data recoveries can be obtained from a few MRI samples with the help of sparse reconstruction models. To solve the optimization models, proper algorithms are indispensable. The pFISTA, a simple and efficient algorithm, has been successfully extended to parallel imaging. However, its convergence criterion is still an open question. Besides, the existing convergence criterion of single-coil pFISTA cannot be applied to the parallel imaging pFISTA, which, therefore, imposes confusions and difficulties on users about determining the only parameter - step size. In this work, we provide the guaranteed convergence analysis of the parallel imaging version pFISTA to solve the two well-known parallel imaging reconstruction models, SENSE and SPIRiT. Along with the convergence analysis, we provide recommended step size values for SENSE and SPIRiT reconstructions to obtain fast and promising reconstructions. Experiments on in vivo brain images demonstrate the validity of the convergence criterion.

High quality and fast compressed sensing MRI reconstruction via edge-enhanced dual discriminator generative adversarial network

Generative adversarial networks (GAN) are widely used for fast compressed sensing magnetic resonance imaging (CSMRI) reconstruction. However, most existing methods are difficult to make an effective trade-off between abstract global high-level features and edge features. It easily causes problems, such as significant remaining aliasing artifacts and clearly over-smoothed reconstruction details. To tackle these issues, we propose a novel edge-enhanced dual discriminator generative adversarial network architecture called EDDGAN for CSMRI reconstruction with high quality. In this model, we extract effective edge features by fusing edge information from different depths. Then, leveraging the relationship between abstract global high-level features and edge features, a three-player game is introduced to control the hallucination of details and stabilize the training process. The resulting EDDGAN can offer more focus on edge restoration and de-aliasing. Extensive experimental results demonstrate that our method consistently outperforms state-of-the-art methods and obtains reconstructed images with rich edge details. In addition, our method also shows remarkable generalization, and its time consumption for each 256 × 256 image reconstruction is approximately 8.39 ms.

On the regularization of feature fusion and mapping for fast MR multi-contrast imaging via iterative networks

Multi-contrast (MC) Magnetic Resonance Imaging (MRI) of the same patient usually requires long scanning times, despite the images sharing redundant information. In this work, we propose a new iterative network that utilizes the sharable information among MC images for MRI acceleration. The proposed network has reinforced data fidelity control and anatomy guidance through an iterative optimization procedure of Gradient Descent, leading to reduced uncertainties and improved reconstruction results. Through a convolutional network, the new method incorporates a learnable regularization unit that is capable of extracting, fusing, and mapping shareable information among different contrasts. Specifically, a dilated inception block is proposed to promote multi-scale feature extractions and increase the receptive field diversity for contextual information incorporation. Lastly, an optimal MC information feeding protocol is built through the design of a complementary feature extractor block. Comprehensive experiments demonstrated the superiority of the proposed network, both qualitatively and quantitatively.

Triple-D network for efficient undersampled magnetic resonance images reconstruction

Compressed sensing (CS) theory can help accelerate magnetic resonance imaging (MRI) by sampling partial k-space measurements. However, conventional optimization-based CS-MRI methods are often time-consuming and are based on fixed transform or shallow image dictionaries, which limits modeling capabilities. Recently, deep learning models have been used to solve the CS-MRI problem. However, recent researches have focused on modeling in image domain, and the potential of k-space modeling capability has not been utilized seriously. In this paper, we propose a deep model called Dual Domain Dense network (Triple-D network), which consisted of some k-space and image domain sub-network. These sub-networks are connected with dense connections, which can utilize feature maps at different levels to enhance performance. To further promote model capabilities, we use two strategies: multi-supervision strategies, which can avoid loss of supervision information; channel-wise attention layer (CA layer), which can adaptively adjust the weight of the feature map. Experimental results show that the proposed Triple-D network provides promising performance in CS-MRI, and it can effectively work on different sampling trajectories and noisy settings.

pFISTA-SENSE-ResNet for parallel MRI reconstruction

Magnetic resonance imaging has been widely applied in clinical diagnosis. However, it is limited by its long data acquisition time. Although the imaging can be accelerated by sparse sampling and parallel imaging, achieving promising reconstructed images with a fast computation speed remains a challenge. Recently, deep learning methods have attracted a lot of attention for encouraging reconstruction results, but they are lack of proper interpretability for neural networks. In this work, in order to enable high-quality image reconstruction for the parallel magnetic resonance imaging, we design the network structure from the perspective of sparse iterative reconstruction and enhance it with the residual structure. Experimental results on a public knee dataset indicate that, as compared with the state-of-the-art deep learning-based and optimization-based methods, the proposed network achieves lower error in reconstruction and is more robust under different samplings.

Low-Rank and Sparse Recovery of Human Gait Data

Due to occlusion or detached markers, information can often be lost while capturing human motion with optical tracking systems. Based on three natural properties of human gait movement, this study presents two different approaches to recover corrupted motion data. These properties are used to define a reconstruction model combining low-rank matrix completion of the measured data with a group-sparsity prior on the marker trajectories mapped in the frequency domain. Unlike most existing approaches, the proposed methodology is fully unsupervised and does not need training data or kinematic information of the user. We evaluated our methods on four different gait datasets with various gap lengths and compared their performance with a state-of-the-art approach using principal component analysis (PCA). Our results showed recovering missing data more precisely, with a reduction of at least 2 mm in mean reconstruction error compared to the literature method. When a small number of marker trajectories is available, our findings showed a reduction of more than 14 mm for the mean reconstruction error compared to the literature approach.

Compressed MRI reconstruction exploiting a rotation-invariant total variation discretization

Inspired by the first-order method of Malitsky and Pock, we propose a new variational framework for compressed MR image reconstruction which introduces the application of a rotation-invariant discretization of total variation functional into MR imaging while exploiting BM3D frame as a sparsifying transform. In the first step, we provide theoretical and numerical analysis establishing the exceptional rotation-invariance property of this total variation functional and observe its superiority over other well-known variational regularization terms in both upright and rotated imaging setups. Thereupon, the proposed MRI reconstruction model is presented as a constrained optimization problem, however, we do not use conventional ADMM-type algorithms designed for constrained problems to obtain a solution, but rather we tailor the linesearch-equipped method of Malitsky and Pock to our model, which was originally proposed for unconstrained problems. As attested by numerical experiments, this framework significantly outperforms various state-of-the-art algorithms from variational methods to adaptive and learning approaches and in particular, it eliminates the stagnating behavior of a previous work on BM3D-MRI which compromised the solution beyond a certain iteration.

ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Compressive sensing (CS) is an effective technique for reconstructing image from a small amount of sampled data. It has been widely applied in medical imaging, remote sensing, image compression, etc. In this paper, we propose two versions of a novel deep learning architecture, dubbed as ADMM-CSNet, by combining the traditional model-based CS method and data-driven deep learning method for image reconstruction from sparsely sampled measurements. We first consider a generalized CS model for image reconstruction with undetermined regularizations in undetermined transform domains, and then two efficient solvers using Alternating Direction Method of Multipliers (ADMM) algorithm for optimizing the model are proposed. We further unroll and generalize the ADMM algorithm to be two deep architectures, in which all parameters of the CS model and the ADMM algorithm are discriminatively learned by end-to-end training. For both applications of fast CS complex-valued MR imaging and CS imaging of real-valued natural images, the proposed ADMM-CSNet achieved favorable reconstruction accuracy in fast computational speed compared with the traditional and the other deep learning methods.

A deep error correction network for compressed sensing MRI

BACKGROUND

CS-MRI (compressed sensing for magnetic resonance imaging) exploits image sparsity properties to reconstruct MRI from very few Fourier k-space measurements. Due to imperfect modelings in the inverse imaging, state-of-the-art CS-MRI methods tend to leave structural reconstruction errors. Compensating such errors in the reconstruction could help further improve the reconstruction quality.

RESULTS

In this work, we propose a DECN (deep error correction network) for CS-MRI. The DECN model consists of three parts, which we refer to as modules: a guide, or template, module, an error correction module, and a data fidelity module. Existing CS-MRI algorithms can serve as the template module for guiding the reconstruction. Using this template as a guide, the error correction module learns a CNN (convolutional neural network) to map the k-space data in a way that adjusts for the reconstruction error of the template image. We propose a deep error correction network. Our experimental results show the proposed DECN CS-MRI reconstruction framework can considerably improve upon existing inversion algorithms by supplementing with an error-correcting CNN.

CONCLUSIONS

In the proposed a deep error correction framework, any off-the-shelf CS-MRI algorithm can be used as template generation. Then a deep neural network is used to compensate reconstruction errors. The promising experimental results validate the effectiveness and utility of the proposed framework.

Analysis of the Quantization Noise in Discrete Wavelet Transform Filters for 3D Medical Imaging

Denoising and compression of 2D and 3D images are important problems in modern medical imaging systems. Discrete wavelet transform (DWT) is used to solve them in practice. We analyze the quantization noise effect in coefficients of DWT filters for 3D medical imaging in this paper. The method for wavelet filters coefficients quantizing is proposed, which allows minimizing resources in hardware implementation by simplifying rounding operations. We develop the method for estimating the maximum error of 3D grayscale and color images DWT with various bits per color (BPC). The dependence of the peak signal-to-noise ratio (PSNR) of the images processing result on wavelet used, the effective bit-width of filters coefficients and BPC is revealed. We derive formulas for determining the minimum bit-width of wavelet filters coefficients that provide a high (PSNR ≥ 40 dB for images with 8 BPC, for example) and maximum (PSNR = ∞ dB) quality of 3D medical imaging by DWT depending on wavelet used. The experiments of 3D tomographic images processing confirmed the accuracy of theoretical analysis. All data are presented in the fixed-point format in the proposed method of 3D medical images DWT. It is making possible efficient, from the point of view of hardware and time resources, the implementation for image denoising and compression on modern devices such as field-programmable gate arrays and application-specific integrated circuits.

Incorporating prior knowledge via volumetric deep residual network to optimize the reconstruction of sparsely sampled MRI

For sparse sampling that accelerates magnetic resonance (MR) image acquisition, non-linear reconstruction algorithms have been developed, which incorporated patient specific a prior information. More generic a prior information could be acquired via deep learning and utilized for image reconstruction. In this study, we developed a volumetric hierarchical deep residual convolutional neural network, referred to as T-Net, to provide a data-driven end-to-end mapping from sparsely sampled MR images to fully sampled MR images, where cartilage MR images were acquired using an Ultra-short TE sequence and retrospectively undersampled using pseudo-random Cartesian and radial acquisition schemes. The network had a hierarchical architecture that promoted the sparsity of feature maps and increased the receptive field, which were valuable for signal synthesis and artifact suppression. Relatively dense local connections and global shortcuts were established to facilitate residual learning and compensate for details lost in hierarchical processing. Additionally, volumetric processing was adopted to fully exploit spatial continuity in three-dimensional space. Data consistency was further enforced. The network was trained with 336 three-dimensional images (each consisting of 32 slices) and tested by 24 images. The incorporation of a priori information acquired via deep learning facilitated high acceleration factors (as high as 8) while maintaining high image fidelity (quantitatively evaluated using the structural similarity index measurement). The proposed T-Net had an improved performance as compared to several state-of-the-art networks.

An Adaptive Space-Filling Curve Trajectory for Ordering 3D Datasets to 1D: Application to Brain Magnetic Resonance Imaging Data for Classification

In this work, we develop an adaptive, near-optimal, 3-Dimensional (3D) to 1D ordering methodology for brain magnetic resonance imaging (MRI) data, using a space-filling curve (SFC) trajectory, which is adaptive to brain’s shape as captured by MRI. We present the pseudocode of the heuristics for developing the SFC trajectory. We apply this trajectory to functional MRI brain activation maps from a schizophrenia study, compress the data, obtain features, and perform classification of schizophrenia patients vs. normal controls. We compare the classification results with those of a linear ordering trajectory, which has been the traditional method for ordering 3D MRI data to 1D. We report that the adaptive SFC trajectory-based classification performance is superior than the linear ordering trajectory-based classification.

A Deep Information Sharing Network for Multi-Contrast Compressed Sensing MRI Reconstruction

Compressed sensing (CS) theory can accelerate multi-contrast magnetic resonance imaging (MRI) by sampling fewer measurements within each contrast. However, conventional optimization-based reconstruction models suffer several limitations, including a strict assumption of shared sparse support, time-consuming optimization, and "shallow" models with difficulties in encoding the patterns contained in massive MRI data. In this paper, we propose the first deep learning model for multi-contrast CS-MRI reconstruction. We achieve information sharing through feature sharing units, which significantly reduces the number of model parameters. The feature sharing unit combines with a data fidelity unit to comprise an inference block, which are then cascaded with dense connections, allowing for efficient information transmission across different depths of the network. Experiments on various multi-contrast MRI datasets show that the proposed model outperforms both state-of-the-art single-contrast and multi-contrast MRI methods in accuracy and efficiency. We demonstrate that improved reconstruction quality can bring benefits to subsequent medical image analysis. Furthermore, the robustness of the proposed model to misregistration shows its potential in real MRI applications.

A cosparse analysis model with combined redundant systems for MRI reconstruction

PURPOSE

Magnetic resonance imaging (MRI) is widely used due to its noninvasive and nonionizing properties. However, MRI requires a long scanning time. In this paper, our goal is to reconstruct a high-quality MR image from its sampled k-space data to accelerate the data acquisition in MRI.

METHODS

We propose a cosparse analysis model with combined redundant systems to fully exploit the sparsity of MR images. Two fixed redundant systems are used to characterize different structures, namely, the wavelet tight frame and Gabor frame. An alternating iteration scheme is used for reconstruction with simple implementation and good performance.

RESULTS

The proposed method is tested on two MR images under three sampling patterns with sampling ratios ranging from 10% to 60%. The results show that the proposed method outperforms other state-of-the-art MRI reconstruction methods in terms of both subjective visual quality and objective quantitative measurement. For instance, for brain images under random sampling with a ratio of 10%, compared to the other three methods, the proposed method improves the peak signal-to-noise ratio (PSNR) by more than 9 dB.

CONCLUSIONS

To better characterize different sparsities of different structures of MRI, a cosparse analysis model combining the wavelet tight frame and Gabor frame is proposed. A partial ℓ 2 norm regularization is leveraged to obtain the optimal solution in a lower dimension. Compared to other state-of-the-art MRI reconstruction methods, the proposed method improves the reconstruction quality of MRI, especially highly undersampled MRI.

Orthogonal tensor dictionary learning for accelerated dynamic MRI

A direct application of the compressed sensing (CS) theory to dynamic magnetic resonance imaging (MRI) reconstruction needs vectorization or matricization of the dynamic MRI data, which is composed of a stack of 2D images and can be naturally regarded as a tensor. This 1D/2D model may destroy the inherent spatial structure property of the data. An alternative way to exploit the multidimensional structure in dynamic MRI is to employ tensor decomposition for dictionary learning, that is, learning multiple dictionaries along each dimension (mode) and sparsely representing the multidimensional data with respect to the Kronecker product of these dictionaries. In this work, we introduce a novel tensor dictionary learning method under an orthonormal constraint on the elementary matrix of the tensor dictionary for dynamic MRI reconstruction. The proposed algorithm alternates sparse coding, tensor dictionary learning, and updating reconstruction, and each corresponding subproblem is efficiently solved by a closed-form solution. Numerical experiments on phantom and synthetic data show significant improvements in reconstruction accuracy and computational efficiency obtained by the proposed scheme over the existing method that uses the 1D/2D model with overcomplete dictionary learning. Graphical abstract Fig. 1 Comparison between (a) the traditional method and (b) the proposed method based on dictionary learning for dynamic MRI reconstruction.

Patient-attentive sequential strategy for perimetry-based visual field acquisition

Perimetry is a non-invasive clinical psychometric examination used for diagnosing ophthalmic and neurological conditions. At its core, perimetry relies on a subject pressing a button whenever they see a visual stimulus within their field of view. This sequential process then yields a 2D visual field image that is critical for clinical use. Perimetry is painfully slow however, with examinations lasting 7-8 minutes per eye. Maintaining high levels of concentration during that time is exhausting for the patient and negatively affects the acquired visual field. We introduce PASS, a novel perimetry testing strategy, based on reinforcement learning, that requires fewer locations in order to effectively estimate 2D visual fields. PASS uses a selection policy that determines what locations should be tested in order to reconstruct the complete visual field as accurately as possible, and then separately reconstructs the visual field from sparse observations. Furthermore, PASS is patient-specific and non-greedy. It adaptively selects what locations to query based on the patient's answers to previous queries, and the locations are jointly selected to maximize the quality of the final reconstruction. In our experiments, we show that PASS outperforms state-of-the-art methods, leading to more accurate reconstructions while reducing between 30% and 70% the duration of the patient examination.