Real-time deep artifact suppression using recurrent U-Nets for low-latency cardiac MRI

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Background Subtraction Angiography with Deep Learning Using Multi-frame Spatiotemporal Angiographic Input

Catheter Digital Subtraction Angiography (DSA) is markedly degraded by all voluntary, respiratory, or cardiac motion artifact that occurs during the exam acquisition. Prior efforts directed toward improving DSA images with machine learning have focused on extracting vessels from individual, isolated 2D angiographic frames. In this work, we introduce improved 2D + t deep learning models that leverage the rich temporal information in angiographic timeseries. A total of 516 cerebral angiograms were collected with 8784 individual series. We utilized feature-based computer vision algorithms to separate the database into "motionless" and "motion-degraded" subsets. Motion measured from the "motion degraded" category was then used to create a realistic, but synthetic, motion-augmented dataset suitable for training 2D U-Net, 3D U-Net, SegResNet, and UNETR models. Quantitative results on a hold-out test set demonstrate that the 3D U-Net outperforms competing 2D U-Net architectures, with substantially reduced motion artifacts when compared to DSA. In comparison to single-frame 2D U-Net, the 3D U-Net utilizing 16 input frames achieves a reduced RMSE (35.77 ± 15.02 vs 23.14 ± 9.56, p < 0.0001; mean ± std dev) and an improved Multi-Scale SSIM (0.86 ± 0.08 vs 0.93 ± 0.05, p < 0.0001). The 3D U-Net also performs favorably in comparison to alternative convolutional and transformer-based architectures (U-Net RMSE 23.20 ± 7.55 vs SegResNet 23.99 ± 7.81, p < 0.0001, and UNETR 25.42 ± 7.79, p < 0.0001, mean ± std dev). These results demonstrate that multi-frame temporal information can boost performance of motion-resistant Background Subtraction Deep Learning algorithms, and we have presented a neuroangiography domain-specific synthetic affine motion augmentation pipeline that can be utilized to generate suitable datasets for supervised training of 3D (2d + t) architectures.

New clinical opportunities of low-field MRI: heart, lung, body, and musculoskeletal

Contemporary whole-body low-field MRI scanners (< 1 T) present new and exciting opportunities for improved body imaging. The fundamental reason is that the reduced off-resonance and reduced SAR provide substantially increased flexibility in the design of MRI pulse sequences. Promising body applications include lung parenchyma imaging, imaging adjacent to metallic implants, cardiac imaging, and dynamic imaging in general. The lower cost of such systems may make MRI favorable for screening high-risk populations and population health research, and the more open configurations allowed may prove favorable for obese subjects and for pregnant women. This article summarizes promising body applications for contemporary whole-body low-field MRI systems, with a focus on new platforms developed within the past 5 years. This is an active area of research, and one can expect many improvements as MRI physicists fully explore the landscape of pulse sequences that are feasible, and as clinicians apply these to patient populations.

Complementation-reinforced network for integrated reconstruction and segmentation of pulmonary gas MRI with high acceleration

BACKGROUND

Hyperpolarized (HP) gas MRI enables the clear visualization of lung structure and function. Clinically relevant biomarkers, such as ventilated defect percentage (VDP) derived from this modality can quantify lung ventilation function. However, long imaging time leads to image quality degradation and causes discomfort to the patients. Although accelerating MRI by undersampling k-space data is available, accurate reconstruction and segmentation of lung images are quite challenging at high acceleration factors.

PURPOSE

To simultaneously improve the performance of reconstruction and segmentation of pulmonary gas MRI at high acceleration factors by effectively utilizing the complementary information in different tasks.

METHODS

A complementation-reinforced network is proposed, which takes the undersampled images as input and outputs both the reconstructed images and the segmentation results of lung ventilation defects. The proposed network comprises a reconstruction branch and a segmentation branch. To effectively exploit the complementary information, several strategies are designed in the proposed network. Firstly, both branches adopt the encoder-decoder architecture, and their encoders are designed to share convolutional weights for facilitating knowledge transfer. Secondly, a designed feature-selecting block discriminately feeds shared features into decoders of both branches, which can adaptively pick suitable features for each task. Thirdly, the segmentation branch incorporates the lung mask obtained from the reconstructed images to enhance the accuracy of the segmentation results. Lastly, the proposed network is optimized by a tailored loss function that efficiently combines and balances these two tasks, in order to achieve mutual benefits.

RESULTS

Experimental results on the pulmonary HP 129 Xe MRI dataset (including 43 healthy subjects and 42 patients) show that the proposed network outperforms state-of-the-art methods at high acceleration factors (4, 5, and 6). The peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and Dice score of the proposed network are enhanced to 30.89, 0.875, and 0.892, respectively. Additionally, the VDP obtained from the proposed network has good correlations with that obtained from fully sampled images (r = 0.984). At the highest acceleration factor of 6, the proposed network promotes PSNR, SSIM, and Dice score by 7.79%, 5.39%, and 9.52%, respectively, in comparison to the single-task models.

CONCLUSION

The proposed method effectively enhances the reconstruction and segmentation performance at high acceleration factors up to 6. It facilitates fast and high-quality lung imaging and segmentation, and provides valuable support in the clinical diagnosis of lung diseases.

HyperSLICE: HyperBand optimized spiral for low-latency interactive cardiac examination

PURPOSE

Interactive cardiac MRI is used for fast scan planning and MR-guided interventions. However, the requirement for real-time acquisition and near-real-time visualization constrains the achievable spatio-temporal resolution. This study aims to improve interactive imaging resolution through optimization of undersampled spiral sampling and leveraging of deep learning for low-latency reconstruction (deep artifact suppression).

METHODS

A variable density spiral trajectory was parametrized and optimized via HyperBand to provide the best candidate trajectory for rapid deep artifact suppression. Training data consisted of 692 breath-held CINEs. The developed interactive sequence was tested in simulations and prospectively in 13 subjects (10 for image evaluation, 2 during catheterization, 1 during exercise). In the prospective study, the optimized framework-HyperSLICE- was compared with conventional Cartesian real-time and breath-hold CINE imaging in terms quantitative and qualitative image metrics. Statistical differences were tested using Friedman chi-squared tests with post hoc Nemenyi test (p < 0.05).

RESULTS

In simulations the normalized RMS error, peak SNR, structural similarity, and Laplacian energy were all statistically significantly higher using optimized spiral compared to radial and uniform spiral sampling, particularly after scan plan changes (structural similarity: 0.71 vs. 0.45 and 0.43). Prospectively, HyperSLICE enabled a higher spatial and temporal resolution than conventional Cartesian real-time imaging. The pipeline was demonstrated in patients during catheter pull back, showing sufficiently fast reconstruction for interactive imaging.

CONCLUSION

HyperSLICE enables high spatial and temporal resolution interactive imaging. Optimizing the spiral sampling enabled better overall image quality and superior handling of image transitions compared with radial and uniform spiral trajectories.

A deep unrolled neural network for real-time MRI-guided brain intervention

Accurate navigation and targeting are critical for neurological interventions including biopsy and deep brain stimulation. Real-time image guidance further improves surgical planning and MRI is ideally suited for both pre- and intra-operative imaging. However, balancing spatial and temporal resolution is a major challenge for real-time interventional MRI (i-MRI). Here, we proposed a deep unrolled neural network, dubbed as LSFP-Net, for real-time i-MRI reconstruction. By integrating LSFP-Net and a custom-designed, MR-compatible interventional device into a 3 T MRI scanner, a real-time MRI-guided brain intervention system is proposed. The performance of the system was evaluated using phantom and cadaver studies. 2D/3D real-time i-MRI was achieved with temporal resolutions of 80/732.8 ms, latencies of 0.4/3.66 s including data communication, processing and reconstruction time, and in-plane spatial resolution of 1 × 1 mm2. The results demonstrated that the proposed method enables real-time monitoring of the remote-controlled brain intervention, and showed the potential to be readily integrated into diagnostic scanners for image-guided neurosurgery.

A dual-stage partially interpretable neural network for joint suppression of bSSFP banding and flow artifacts in non-phase-cycled cine imaging

PURPOSE

To develop a partially interpretable neural network for joint suppression of banding and flow artifacts in non-phase-cycled bSSFP cine imaging.

METHODS

A dual-stage neural network consisting of a voxel-identification (VI) sub-network and artifact-suppression (AS) sub-network is proposed. The VI sub-network provides identification of artifacts, which guides artifact suppression and improves interpretability. The AS sub-network reduces banding and flow artifacts. Short-axis cine images of 12 frequency offsets from 28 healthy subjects were used to train and test the dual-stage network. An additional 77 patients were retrospectively enrolled to evaluate its clinical generalizability. For healthy subjects, artifact suppression performance was analyzed by comparison with traditional phase cycling. The partial interpretability provided by the VI sub-network was analyzed via correlation analysis. Generalizability was evaluated for cine obtained with different sequence parameters and scanners. For patients, artifact suppression performance and partial interpretability of the network were qualitatively evaluated by 3 clinicians. Cardiac function before and after artifact suppression was assessed via left ventricular ejection fraction (LVEF).

RESULTS

For the healthy subjects, visual inspection and quantitative analysis found a considerable reduction of banding and flow artifacts by the proposed network. Compared with traditional phase cycling, the proposed network improved flow artifact scores (4.57 ± 0.23 vs 3.40 ± 0.38, P = 0.002) and overall image quality (4.33 ± 0.22 vs 3.60 ± 0.38, P = 0.002). The VI sub-network well identified the location of banding and flow artifacts in the original movie and significantly correlated with the change of signal intensities in these regions. Changes of imaging parameters or the scanner did not cause a significant change of overall image quality relative to the baseline dataset, suggesting a good generalizability. For the patients, qualitative analysis showed a significant improvement of banding artifacts (4.01 ± 0.50 vs 2.77 ± 0.40, P < 0.001), flow artifacts (4.22 ± 0.38 vs 2.97 ± 0.57, P < 0.001), and image quality (3.91 ± 0.45 vs 2.60 ± 0.43, P < 0.001) relative to the original cine. The artifact suppression slightly reduced the LVEF (mean bias = -1.25%, P = 0.01).

CONCLUSIONS

The dual-stage network simultaneously reduces banding and flow artifacts in bSSFP cine imaging with a partial interpretability, sparing the need for sequence modification. The method can be easily deployed in a clinical setting to identify artifacts and improve cine image quality.

Interventional cardiovascular magnetic resonance: state-of-the-art

Transcatheter cardiovascular interventions increasingly rely on advanced imaging. X-ray fluoroscopy provides excellent visualization of catheters and devices, but poor visualization of anatomy. In contrast, magnetic resonance imaging (MRI) provides excellent visualization of anatomy and can generate real-time imaging with frame rates similar to X-ray fluoroscopy. Realization of MRI as a primary imaging modality for cardiovascular interventions has been slow, largely because existing guidewires, catheters and other devices create imaging artifacts and can heat dangerously. Nonetheless, numerous clinical centers have started interventional cardiovascular magnetic resonance (iCMR) programs for invasive hemodynamic studies or electrophysiology procedures to leverage the clear advantages of MRI tissue characterization, to quantify cardiac chamber function and flow, and to avoid ionizing radiation exposure. Clinical implementation of more complex cardiovascular interventions has been challenging because catheters and other tools require re-engineering for safety and conspicuity in the iCMR environment. However, recent innovations in scanner and interventional device technology, in particular availability of high performance low-field MRI scanners could be the inflection point, enabling a new generation of iCMR procedures. In this review we review these technical considerations, summarize contemporary clinical iCMR experience, and consider potential future applications.

A low-rank deep image prior reconstruction for free-breathing ungated spiral functional CMR at 0.55 T and 1.5 T

OBJECTIVE

This study combines a deep image prior with low-rank subspace modeling to enable real-time (free-breathing and ungated) functional cardiac imaging on a commercial 0.55 T scanner.

MATERIALS AND METHODS

The proposed low-rank deep image prior (LR-DIP) uses two u-nets to generate spatial and temporal basis functions that are combined to yield dynamic images, with no need for additional training data. Simulations and scans in 13 healthy subjects were performed at 0.55 T and 1.5 T using a golden angle spiral bSSFP sequence with images reconstructed using [Formula: see text]-ESPIRiT, low-rank plus sparse (L + S) matrix completion, and LR-DIP. Cartesian breath-held ECG-gated cine images were acquired for reference at 1.5 T. Two cardiothoracic radiologists rated images on a 1-5 scale for various categories, and LV function measurements were compared.

RESULTS

LR-DIP yielded the lowest errors in simulations, especially at high acceleration factors (R [Formula: see text] 8). LR-DIP ejection fraction measurements agreed with 1.5 T reference values (mean bias - 0.3% at 0.55 T and - 0.2% at 1.5 T). Compared to reference images, LR-DIP images received similar ratings at 1.5 T (all categories above 3.9) and slightly lower at 0.55 T (above 3.4).

CONCLUSION

Feasibility of real-time functional cardiac imaging using a low-rank deep image prior reconstruction was demonstrated in healthy subjects on a commercial 0.55 T scanner.

Quantifying myocardial perfusion during MR-guided interventions without exogenous contrast agents: intra-arterial spin labeling

PURPOSE

To test intra-arterial spin labeling (iASL) using active guiding catheters for myocardial perfusion measurements during magnetic resonance (MR)-guided interventions in a pig study.

METHODS

In this work, a single-loop radiofrequency (RF) coil at the tip of a 6F active coronary catheter was used as a transmit coil for local spin labeling. The transmit magnetic RF field (B1) of the coil and the labeling efficiency were determined, and iASL was tested in two pigs after the catheter was engaged in the aortic root, the ostium of the left coronary artery (LCA) under MR-guidance. The iASL effect was assessed by the signal difference between spin-labeling On and control (spin-labeling OFF) images, and in a cross-correlation between ON/Off states of spin-labeling a binary labeling paradigm. In addition, quantitative myocardial perfusion was calculated from the iASL experiments.

RESULTS

The maximum B1 in the vicinity of the catheter coil was 2.1 µT. A strong local labeling effect with a labeling efficiency of 0.45 was achieved with iASL both in vitro and in vivo. In both pigs, the proximal myocardial segments supplied by the LCA showed significant labelling effect up to distances of 60 mm from the aortic root with a relative signal difference of (3.14 ± 2.89)% in the first and (3.50 ± 1.25)% in the second animal. The mean correlation coefficients were R = 0.63 ± 0.22 and 0.42 ± 0.16, respectively. The corresponding computed myocardial perfusion values in this region of the myocardium were similar to those obtained with contrast perfusion methods ((1.2 ± 1.1) mL/min/g and (0.8 ± 0.6) mL/min/g).

CONCLUSION

The proposed iASL method demonstrates the feasibility of selective myocardial perfusion measurements during MR-guided coronary interventions, which with further technical improvements may provide an alternative to exogenous contrast-based perfusion. Due to the invasive nature of the iASL method, it can potentially be used in concert with MRI-guided coronary angioplasty.

Respiratory Motion Management in Abdominal MRI: Radiology In Training

A 96-year-old woman had a suboptimal evaluation of liver observations at abdominal MRI due to significant respiratory motion. State-of-the-art strategies to minimize respiratory motion during clinical abdominal MRI are discussed.

FReSCO: Flow Reconstruction and Segmentation for low-latency Cardiac Output monitoring using deep artifact suppression and segmentation

PURPOSE

Real-time monitoring of cardiac output (CO) requires low-latency reconstruction and segmentation of real-time phase-contrast MR, which has previously been difficult to perform. Here we propose a deep learning framework for "FReSCO" (Flow Reconstruction and Segmentation for low latency Cardiac Output monitoring).

METHODS

Deep artifact suppression and segmentation U-Nets were independently trained. Breath-hold spiral phase-contrast MR data (N = 516) were synthetically undersampled using a variable-density spiral sampling pattern and gridded to create aliased data for training of the artifact suppression U-net. A subset of the data (N = 96) was segmented and used to train the segmentation U-net. Real-time spiral phase-contrast MR was prospectively acquired and then reconstructed and segmented using the trained models (FReSCO) at low latency at the scanner in 10 healthy subjects during rest, exercise, and recovery periods. Cardiac output obtained via FReSCO was compared with a reference rest CO and rest and exercise compressed-sensing CO.

RESULTS

The FReSCO framework was demonstrated prospectively at the scanner. Beat-to-beat heartrate, stroke volume, and CO could be visualized with a mean latency of 622 ms. No significant differences were noted when compared with reference at rest (bias = -0.21 ± 0.50 L/min, p = 0.246) or compressed sensing at peak exercise (bias = 0.12 ± 0.48 L/min, p = 0.458).

CONCLUSIONS

The FReSCO framework was successfully demonstrated for real-time monitoring of CO during exercise and could provide a convenient tool for assessment of the hemodynamic response to a range of stressors.

Dual-Domain Reconstruction Network with V-Net and K-Net for Fast MRI

PURPOSE

To introduce a dual-domain reconstruction network with V-Net and K-Net for accurate MR image reconstruction from undersampled k-space data.

METHODS

Most state-of-the-art reconstruction methods apply U-Net or cascaded U-Nets in the image domain and/or k-space domain. Nevertheless, these methods have the following problems: (1) directly applying U-Net in the k-space domain is not optimal for extracting features; (2) classical image-domain-oriented U-Net is heavyweighted and hence inefficient when cascaded many times to yield good reconstruction accuracy; (3) classical image-domain-oriented U-Net does not make full use of information of the encoder network for extracting features in the decoder network; and (4) existing methods are ineffective in simultaneously extracting and fusing features in the image domain and its dual k-space domain. To tackle these problems, we present 3 different methods: (1) V-Net, an image-domain encoder-decoder subnetwork that is more lightweight for cascading and effective in fully utilizing features in the encoder for decoding; (2) K-Net, a k-space domain subnetwork that is more suitable for extracting hierarchical features in the k-space domain, and (3) KV-Net, a dual-domain reconstruction network in which V-Nets and K-Nets are effectively combined and cascaded.

RESULTS

Extensive experimental results on the fastMRI dataset demonstrate that the proposed KV-Net can reconstruct high-quality images and outperform state-of-the-art approaches with fewer parameters.

CONCLUSIONS

To reconstruct images effectively and efficiently from incomplete k-space data, we have presented a dual-domain KV-Net to combine K-Nets and V-Nets. The KV-Net achieves better results with 9% and 5% parameters than comparable methods (XPD-Net and i-RIM).

A System for Real-Time, Online Mixed-Reality Visualization of Cardiac Magnetic Resonance Images

Image-guided cardiovascular interventions are rapidly evolving procedures that necessitate imaging systems capable of rapid data acquisition and low-latency image reconstruction and visualization. Compared to alternative modalities, Magnetic Resonance Imaging (MRI) is attractive for guidance in complex interventional settings thanks to excellent soft tissue contrast and large fields-of-view without exposure to ionizing radiation. However, most clinically deployed MRI sequences and visualization pipelines exhibit poor latency characteristics, and spatial integration of complex anatomy and device orientation can be challenging on conventional 2D displays. This work demonstrates a proof-of-concept system linking real-time cardiac MR image acquisition, online low-latency reconstruction, and a stereoscopic display to support further development in real-time MR-guided intervention. Data are acquired using an undersampled, radial trajectory and reconstructed via parallelized through-time radial generalized autocalibrating partially parallel acquisition (GRAPPA) implemented on graphics processing units. Images are rendered for display in a stereoscopic mixed-reality head-mounted display. The system is successfully tested by imaging standard cardiac views in healthy volunteers. Datasets comprised of one slice (46 ms), two slices (92 ms), and three slices (138 ms) are collected, with the acquisition time of each listed in parentheses. Images are displayed with latencies of 42 ms/frame or less for all three conditions. Volumetric data are acquired at one volume per heartbeat with acquisition times of 467 ms and 588 ms when 8 and 12 partitions are acquired, respectively. Volumes are displayed with a latency of 286 ms or less. The faster-than-acquisition latencies for both planar and volumetric display enable real-time 3D visualization of the heart.