1
|
Li Y, Ruhm L, Wang Z, Zhao R, Anderson A, Arnold P, Huesmann G, Henning A, Lam F. Joint learning of nonlinear representation and projection for fast constrained MRSI reconstruction. Magn Reson Med 2024. [PMID: 39233507 DOI: 10.1002/mrm.30276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2024] [Revised: 07/16/2024] [Accepted: 08/12/2024] [Indexed: 09/06/2024]
Abstract
PURPOSE To develop and evaluate a novel method for computationally efficient reconstruction from noisy MR spectroscopic imaging (MRSI) data. METHODS The proposed method features (a) a novel strategy that jointly learns a nonlinear low-dimensional representation of high-dimensional spectroscopic signals and a neural-network-based projector to recover the low-dimensional embeddings from noisy/limited data; (b) a formulation that integrates the forward encoding model, a regularizer exploiting the learned representation, and a complementary spatial constraint; and (c) a highly efficient algorithm enabled by the learned projector within an alternating direction method of multipliers (ADMM) framework, circumventing the computationally expensive network inversion subproblem. RESULTS The proposed method has been evaluated using simulations as well as in vivo 1 $$ {}^1 $$ H and 31 $$ {}^{31} $$ P MRSI data, demonstrating improved performance over state-of-the-art methods, with about 6× $$ \times $$ fewer averages needed than standard Fourier reconstruction for similar metabolite estimation variances and up to 100× $$ \times $$ reduction in processing time compared to a prior neural network constrained reconstruction method. Computational and theoretical analyses were performed to offer further insights into the effectiveness of the proposed method. CONCLUSION A novel method was developed for fast, high-SNR spatiospectral reconstruction from noisy MRSI data. We expect our method to be useful for enhancing the quality of MRSI or other high-dimensional spatiospectral imaging data.
Collapse
Affiliation(s)
- Yahang Li
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
| | - Loreen Ruhm
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Texas, USA
- High-Field Magnetic Resonance Center, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
| | - Zepeng Wang
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
| | - Ruiyang Zhao
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
| | - Aaron Anderson
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Carle Neuroscience Institute, Carle Foundation Hospital, Urbana, Illinois, USA
| | - Paul Arnold
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Carle Neuroscience Institute, Carle Foundation Hospital, Urbana, Illinois, USA
| | - Graham Huesmann
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Carle Neuroscience Institute, Carle Foundation Hospital, Urbana, Illinois, USA
| | - Anke Henning
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Texas, USA
- High-Field Magnetic Resonance Center, Max Planck Institute for Biological Cybernetics, Tübingen, Germany
| | - Fan Lam
- Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
- Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois, USA
| |
Collapse
|
2
|
Bugler H, Berto R, Souza R, Harris AD. Frequency and phase correction of GABA-edited magnetic resonance spectroscopy using complex-valued convolutional neural networks. Magn Reson Imaging 2024; 111:186-195. [PMID: 38744351 DOI: 10.1016/j.mri.2024.05.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Revised: 05/09/2024] [Accepted: 05/10/2024] [Indexed: 05/16/2024]
Abstract
PURPOSE To determine the significance of complex-valued inputs and complex-valued convolutions compared to real-valued inputs and real-valued convolutions in convolutional neural networks (CNNs) for frequency and phase correction (FPC) of GABA-edited magnetic resonance spectroscopy (MRS) data. METHODS An ablation study using simulated data was performed to determine the most effective input (real or complex) and convolution type (real or complex) to predict frequency and phase shifts in GABA-edited MEGA-PRESS data using CNNs. The best CNN model was subsequently compared using both simulated and in vivo data to two recently proposed deep learning (DL) methods for FPC of GABA-edited MRS. All methods were trained using the same experimental setup and evaluated using the signal-to-noise ratio (SNR) and linewidth of the GABA peak, choline artifact, and by visually assessing the reconstructed final difference spectrum. Statistical significance was assessed using the Wilcoxon signed rank test. RESULTS The ablation study showed that using complex values for the input represented by real and imaginary channels in our model input tensor, with complex convolutions was most effective for FPC. Overall, in the comparative study using simulated data, our CC-CNN model (that received complex-valued inputs with complex convolutions) outperformed the other models as evaluated by the mean absolute error. CONCLUSION Our results indicate that the optimal CNN configuration for GABA-edited MRS FPC uses a complex-valued input and complex convolutions. Overall, this model outperformed existing DL models.
Collapse
Affiliation(s)
- Hanna Bugler
- Department of Biomedical Engineering, University of Calgary, Canada; Department of Radiology, University of Calgary, Canada; Hotchkiss Brain Institute, University of Calgary,Canada; Alberta Children's Hospital Research Institute, University of Calgary, Canada.
| | - Rodrigo Berto
- Department of Biomedical Engineering, University of Calgary, Canada; Department of Radiology, University of Calgary, Canada; Hotchkiss Brain Institute, University of Calgary,Canada; Alberta Children's Hospital Research Institute, University of Calgary, Canada
| | - Roberto Souza
- Hotchkiss Brain Institute, University of Calgary,Canada; Department of Electrical and Software Engineering, University of Calgary, Canada
| | - Ashley D Harris
- Department of Radiology, University of Calgary, Canada; Hotchkiss Brain Institute, University of Calgary,Canada; Alberta Children's Hospital Research Institute, University of Calgary, Canada
| |
Collapse
|
3
|
Wang J, Salerno M. Deep learning-based rapid image reconstruction and motion correction for high-resolution cartesian first-pass myocardial perfusion imaging at 3T. Magn Reson Med 2024; 92:1104-1114. [PMID: 38576068 DOI: 10.1002/mrm.30106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 03/19/2024] [Accepted: 03/19/2024] [Indexed: 04/06/2024]
Abstract
PURPOSE To develop and evaluate a deep learning (DL) -based rapid image reconstruction and motion correction technique for high-resolution Cartesian first-pass myocardial perfusion imaging at 3T with whole-heart coverage for both single-slice (SS) and simultaneous multi-slice (SMS) acquisitions. METHODS 3D physics-driven unrolled network architectures were utilized for the reconstruction of high-resolution Cartesian perfusion imaging. The SS and SMS multiband (MB) = 2 networks were trained from 135 slices from 20 subjects. Structural similarity index (SSIM), peak SNR (PSNR), and normalized RMS error (NRMSE) were assessed, and prospective images were blindly graded by two experienced cardiologists (5, excellent; 1, poor). For respiratory motion correction, a 2D U-Net based motion corrected network was proposed, and the temporal fidelity and second-order derivative were calculated to assess the performance of the motion correction. RESULTS Excellent performance was demonstrated in the proposed technique with high SSIM and PSNR, and low NRMSE. Image quality scores were (4.3 [4.3, 4.4], 4.5 [4.4, 4.6], 4.3 [4.3, 4.4], and 4.5 [4.3, 4.5]) for SS DL and SS L1-SENSE, MB = 2 DL and MB = 2 SMS-L1-SENSE, respectively, showing no statistically significant difference (p > 0.05 for SS and SMS) between (SMS)-L1-SENSE and the proposed DL technique. The network inference time was around 4 s per dynamic perfusion series with 40 frames while the time of (SMS)-L1-SENSE with GPU acceleration was approximately 30 min. CONCLUSION The proposed DL-based image reconstruction and motion correction technique enabled rapid and high-quality reconstruction for SS and SMS MB = 2 high-resolution Cartesian first-pass perfusion imaging at 3T.
Collapse
Affiliation(s)
- Junyu Wang
- Department of Cardiovascular Medicine, Stanford University, Stanford, California, USA
| | - Michael Salerno
- Department of Cardiovascular Medicine, Stanford University, Stanford, California, USA
- Department of Radiology, Cardiovascular Imaging, Stanford University, Stanford, California, USA
| |
Collapse
|
4
|
Wang S, Wu R, Jia S, Diakite A, Li C, Liu Q, Zheng H, Ying L. Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning. Magn Reson Med 2024; 92:496-518. [PMID: 38624162 DOI: 10.1002/mrm.30105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 03/19/2024] [Accepted: 03/20/2024] [Indexed: 04/17/2024]
Abstract
Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.
Collapse
Affiliation(s)
- Shanshan Wang
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Ruoyou Wu
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Sen Jia
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Alou Diakite
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Cheng Li
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Qiegen Liu
- Department of Electronic Information Engineering, Nanchang University, Nanchang, China
| | - Hairong Zheng
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Leslie Ying
- Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, New York, USA
| |
Collapse
|
5
|
Sharma R, Tsiamyrtzis P, Webb AG, Leiss EL, Tsekos NV. Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI. MAGMA (NEW YORK, N.Y.) 2024; 37:507-528. [PMID: 37989921 DOI: 10.1007/s10334-023-01127-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 09/30/2023] [Accepted: 10/16/2023] [Indexed: 11/23/2023]
Abstract
OBJECTIVE This study aims to assess the statistical significance of training parameters in 240 dense UNets (DUNets) used for enhancing low Signal-to-Noise Ratio (SNR) and undersampled MRI in various acquisition protocols. The objective is to determine the validity of differences between different DUNet configurations and their impact on image quality metrics. MATERIALS AND METHODS To achieve this, we trained all DUNets using the same learning rate and number of epochs, with variations in 5 acquisition protocols, 24 loss function weightings, and 2 ground truths. We calculated evaluation metrics for two metric regions of interest (ROI). We employed both Analysis of Variance (ANOVA) and Mixed Effects Model (MEM) to assess the statistical significance of the independent parameters, aiming to compare their efficacy in revealing differences and interactions among fixed parameters. RESULTS ANOVA analysis showed that, except for the acquisition protocol, fixed variables were statistically insignificant. In contrast, MEM analysis revealed that all fixed parameters and their interactions held statistical significance. This emphasizes the need for advanced statistical analysis in comparative studies, where MEM can uncover finer distinctions often overlooked by ANOVA. DISCUSSION These findings highlight the importance of utilizing appropriate statistical analysis when comparing different deep learning models. Additionally, the surprising effectiveness of the UNet architecture in enhancing various acquisition protocols underscores the potential for developing improved methods for characterizing and training deep learning models. This study serves as a stepping stone toward enhancing the transparency and comparability of deep learning techniques for medical imaging applications.
Collapse
Affiliation(s)
- Rishabh Sharma
- Medical Robotics and Imaging Lab, Department of Computer Science, 501, Philip G. Hoffman Hall, University of Houston, 4800 Calhoun Road, Houston, TX, 77204, USA
| | - Panagiotis Tsiamyrtzis
- Department of Mechanical Engineering, Politecnico Di Milano, Milan, Italy
- Department of Statistics, Athens University of Economics and Business, Athens, Greece
| | - Andrew G Webb
- C.J. Gorter Center for High Field MRI, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Ernst L Leiss
- Department of Computer Science, University of Houston, Houston, TX, USA
| | - Nikolaos V Tsekos
- Medical Robotics and Imaging Lab, Department of Computer Science, 501, Philip G. Hoffman Hall, University of Houston, 4800 Calhoun Road, Houston, TX, 77204, USA.
| |
Collapse
|
6
|
Baker RR, Muthurangu V, Rega M, Walsh SB, Steeden JA. Rapid 2D 23Na MRI of the calf using a denoising convolutional neural network. Magn Reson Imaging 2024; 110:184-194. [PMID: 38642779 DOI: 10.1016/j.mri.2024.04.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Revised: 04/12/2024] [Accepted: 04/17/2024] [Indexed: 04/22/2024]
Abstract
PURPOSE 23Na MRI can be used to quantify in-vivo tissue sodium concentration (TSC), but the inherently low 23Na signal leads to long scan times and/or noisy or low-resolution images. Reconstruction algorithms such as compressed sensing (CS) have been proposed to mitigate low signal-to-noise ratio (SNR); although, these can result in unnatural images, suboptimal denoising and long processing times. Recently, machine learning has been increasingly used to denoise 1H MRI acquisitions; however, this approach typically requires large volumes of high-quality training data, which is not readily available for 23Na MRI. Here, we propose using 1H data to train a denoising convolutional neural network (CNN), which we subsequently demonstrate on prospective 23Na images of the calf. METHODS 1893 1H fat-saturated transverse slices of the knee from the open-source fastMRI dataset were used to train denoising CNNs for different levels of noise. Synthetic low SNR images were generated by adding gaussian noise to the high-quality 1H k-space data before reconstruction to create paired training data. For prospective testing, 23Na images of the calf were acquired in 10 healthy volunteers with a total of 150 averages over ten minutes, which were used as a reference throughout the study. From this data, images with fewer averages were retrospectively reconstructed using a non-uniform fast Fourier transform (NUFFT) as well as CS, with the NUFFT images subsequently denoised using the trained CNN. RESULTS CNNs were successfully applied to 23Na images reconstructed with 50, 40 and 30 averages. Muscle and skin apparent TSC quantification from CNN-denoised images were equivalent to those from CS images, with <0.9 mM bias compared to reference values. Estimated SNR was significantly higher in CNN-denoised images compared to NUFFT, CS and reference images. Quantitative edge sharpness was equivalent for all images. For subjective image quality ranking, CNN-denoised images ranked equally best with reference images and significantly better than NUFFT and CS images. CONCLUSION Denoising CNNs trained on 1H data can be successfully applied to 23Na images of the calf; thus, allowing scan time to be reduced from ten minutes to two minutes with little impact on image quality or apparent TSC quantification accuracy.
Collapse
Affiliation(s)
- Rebecca R Baker
- UCL Centre for Medical Imaging, University College London, London, UK; UCL Centre for Translational Cardiovascular Imaging, University College London, London, UK.
| | - Vivek Muthurangu
- UCL Centre for Translational Cardiovascular Imaging, University College London, London, UK.
| | - Marilena Rega
- Institute of Nuclear Medicine, University College Hospital, London, UK.
| | - Stephen B Walsh
- Department of Renal Medicine, University College London, London, UK.
| | - Jennifer A Steeden
- UCL Centre for Translational Cardiovascular Imaging, University College London, London, UK.
| |
Collapse
|
7
|
Karakuzu A, Boudreau M, Stikov N. Reproducible Research Practices in Magnetic Resonance Neuroimaging: A Review Informed by Advanced Language Models. Magn Reson Med Sci 2024; 23:252-267. [PMID: 38897936 PMCID: PMC11234949 DOI: 10.2463/mrms.rev.2023-0174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/21/2024] Open
Abstract
MRI has progressed significantly with the introduction of advanced computational methods and novel imaging techniques, but their wider adoption hinges on their reproducibility. This concise review synthesizes reproducible research insights from recent MRI articles to examine the current state of reproducibility in neuroimaging, highlighting key trends and challenges. It also provides a custom generative pretrained transformer (GPT) model, designed specifically for aiding in an automated analysis and synthesis of information pertaining to the reproducibility insights associated with the articles at the core of this review.
Collapse
Affiliation(s)
- Agah Karakuzu
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montréal, Montréal, Quebec, Canada
- Montréal Heart Institute, Montréal, Quebec, Canada
| | - Mathieu Boudreau
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montréal, Montréal, Quebec, Canada
| | - Nikola Stikov
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montréal, Montréal, Quebec, Canada
- Montréal Heart Institute, Montréal, Quebec, Canada
- Center for Advanced Interdisciplinary Research, Ss. Cyril and Methodius University, Skopje, North Macedonia
| |
Collapse
|
8
|
Poimala J, Cox B, Hauptmann A. Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography. PHOTOACOUSTICS 2024; 37:100597. [PMID: 38425677 PMCID: PMC10901832 DOI: 10.1016/j.pacs.2024.100597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/15/2023] [Accepted: 02/16/2024] [Indexed: 03/02/2024]
Abstract
Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.
Collapse
Affiliation(s)
- Jenni Poimala
- Research Unit of Mathematical Sciences, University of Oulu, Finland
| | - Ben Cox
- Department of Medical Physics and Biomedical Engineering, University College London, UK
| | - Andreas Hauptmann
- Research Unit of Mathematical Sciences, University of Oulu, Finland
- Department of Computer Science, University College London, UK
| |
Collapse
|
9
|
Hewlett M, Petrov I, Johnson PM, Drangova M. Deep-learning-based motion correction using multichannel MRI data: a study using simulated artifacts in the fastMRI dataset. NMR IN BIOMEDICINE 2024:e5179. [PMID: 38808752 DOI: 10.1002/nbm.5179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 04/21/2024] [Accepted: 04/29/2024] [Indexed: 05/30/2024]
Abstract
Deep learning presents a generalizable solution for motion correction requiring no pulse sequence modifications or additional hardware, but previous networks have all been applied to coil-combined data. Multichannel MRI data provide a degree of spatial encoding that may be useful for motion correction. We hypothesize that incorporating deep learning for motion correction prior to coil combination will improve results. A conditional generative adversarial network was trained using simulated rigid motion artifacts in brain images acquired at multiple sites with multiple contrasts (not limited to healthy subjects). We compared the performance of deep-learning-based motion correction on individual channel images (single-channel model) with that performed after coil combination (channel-combined model). We also investigate simultaneous motion correction of all channel data from an image volume (multichannel model). The single-channel model significantly (p < 0.0001) improved mean absolute error, with an average 50.9% improvement compared with the uncorrected images. This was significantly (p < 0.0001) better than the 36.3% improvement achieved by the channel-combined model (conventional approach). The multichannel model provided no significant improvement in quantitative measures of image quality compared with the uncorrected images. Results were independent of the presence of pathology, and generalizable to a new center unseen during training. Performing motion correction on single-channel images prior to coil combination provided an improvement in performance compared with conventional deep-learning-based motion correction. Improved deep learning methods for retrospective correction of motion-affected MR images could reduce the need for repeat scans if applied in a clinical setting.
Collapse
Affiliation(s)
- Miriam Hewlett
- Robarts Research Institute, The University of Western Ontario, London, Ontario, Canada
- Department of Medical Biophysics, The University of Western Ontario, London, Ontario, Canada
| | - Ivailo Petrov
- Robarts Research Institute, The University of Western Ontario, London, Ontario, Canada
| | - Patricia M Johnson
- Department of Radiology, New York Medicine School of Medicine, New York, New York, USA
| | - Maria Drangova
- Robarts Research Institute, The University of Western Ontario, London, Ontario, Canada
- Department of Medical Biophysics, The University of Western Ontario, London, Ontario, Canada
| |
Collapse
|
10
|
Yuan T, Yang J, Chi J, Yu T, Liu F. A cross-domain complex convolution neural network for undersampled magnetic resonance image reconstruction. Magn Reson Imaging 2024; 108:86-97. [PMID: 38331053 DOI: 10.1016/j.mri.2024.02.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 02/01/2024] [Accepted: 02/05/2024] [Indexed: 02/10/2024]
Abstract
To introduce a new cross-domain complex convolution neural network for accurate MR image reconstruction from undersampled k-space data. Most reconstruction methods utilize neural networks or cascade neural networks in either the image domain and/or the k-space domain. However, these methods encounter several challenges: 1) Applying neural networks directly in the k-space domain is suboptimal for feature extraction; 2) Classic image-domain networks have difficulty in fully extracting texture features; and 3) Existing cross-domain methods still face challenges in extracting and fusing features from both image and k-space domains simultaneously. In this work, we propose a novel deep-learning-based 2-D single-coil complex-valued MR reconstruction network termed TEID-Net. TEID-Net integrates three modules: 1) TE-Net, an image-domain-based sub-network designed to enhance contrast in input features by incorporating a Texture Enhancement Module; 2) ID-Net, an intermediate-domain sub-network tailored to operate in the image-Fourier space, with the specific goal of reducing aliasing artifacts realized by leveraging the superior incoherence property of the decoupled one-dimensional signals; and 3) TEID-Net, a cross-domain reconstruction network in which ID-Nets and TE-Nets are combined and cascaded to boost the quality of image reconstruction further. Extensive experiments have been conducted on the fastMRI and Calgary-Campinas datasets. Results demonstrate the effectiveness of the proposed TEID-Net in mitigating undersampling-induced artifacts and producing high-quality image reconstructions, outperforming several state-of-the-art methods while utilizing fewer network parameters. The cross-domain TEID-Net excels in restoring tissue structures and intricate texture details. The results illustrate that TEID-Net is particularly well-suited for regular Cartesian undersampling scenarios.
Collapse
Affiliation(s)
- Tengfei Yuan
- College of Electronics and Information, Qingdao University, Qingdao, Shandong, China
| | - Jie Yang
- College of Mechanical and Electrical Engineering, Qingdao University, Qingdao, Shandong, China
| | - Jieru Chi
- College of Electronics and Information, Qingdao University, Qingdao, Shandong, China.
| | - Teng Yu
- College of Electronics and Information, Qingdao University, Qingdao, Shandong, China
| | - Feng Liu
- School of Electrical Engineering and Computer Science, University of Queensland, Brisbane, Brisbane, Australia
| |
Collapse
|
11
|
Li Z, Xiao S, Wang C, Li H, Zhao X, Duan C, Zhou Q, Rao Q, Fang Y, Xie J, Shi L, Guo F, Ye C, Zhou X. Encoding Enhanced Complex CNN for Accurate and Highly Accelerated MRI. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:1828-1840. [PMID: 38194397 DOI: 10.1109/tmi.2024.3351211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2024]
Abstract
Magnetic resonance imaging (MRI) using hyperpolarized noble gases provides a way to visualize the structure and function of human lung, but the long imaging time limits its broad research and clinical applications. Deep learning has demonstrated great potential for accelerating MRI by reconstructing images from undersampled data. However, most existing deep convolutional neural networks (CNN) directly apply square convolution to k-space data without considering the inherent properties of k-space sampling, limiting k-space learning efficiency and image reconstruction quality. In this work, we propose an encoding enhanced (EN2) complex CNN for highly undersampled pulmonary MRI reconstruction. EN2 complex CNN employs convolution along either the frequency or phase-encoding direction, resembling the mechanisms of k-space sampling, to maximize the utilization of the encoding correlation and integrity within a row or column of k-space. We also employ complex convolution to learn rich representations from the complex k-space data. In addition, we develop a feature-strengthened modularized unit to further boost the reconstruction performance. Experiments demonstrate that our approach can accurately reconstruct hyperpolarized 129Xe and 1H lung MRI from 6-fold undersampled k-space data and provide lung function measurements with minimal biases compared with fully sampled images. These results demonstrate the effectiveness of the proposed algorithmic components and indicate that the proposed approach could be used for accelerated pulmonary MRI in research and clinical lung disease patient care.
Collapse
|
12
|
Morssy A, Frean MR, Teal PD. Informed Adaptive Sensing. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024; 46:3230-3241. [PMID: 38064331 DOI: 10.1109/tpami.2023.3340990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
For many inverse problems, the data on which the solution is based is acquired sequentially. We present an approach to the solution of such inverse problems where a sensor can be directed (or otherwise reconfigured on the fly) to acquire a particular measurement. An example problem is magnetic resonance image reconstruction. We use an estimate of mutual information derived from an empirical conditional distribution provided by a generative model to guide our measurement acquisition given measurements acquired so far. The conditionally generated data is a set of samples which are representative of the plausible solutions that satisfy the acquired measurements. We present experiments on toy and real world data sets. We focus on image data but we demonstrate that the method is applicable to a broader class of problems. We also show how a learned model such as a deep neural network can be leveraged to allow generalisation to unseen data. Our informed adaptive sensing method outperforms random sampling, variance based sampling, sparsity based methods, and compressed sensing.
Collapse
|
13
|
Jacobs L, Mandija S, Liu H, van den Berg CAT, Sbrizzi A, Maspero M. Generalizable synthetic MRI with physics-informed convolutional networks. Med Phys 2024; 51:3348-3359. [PMID: 38063208 DOI: 10.1002/mp.16884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 11/20/2023] [Accepted: 11/28/2023] [Indexed: 05/08/2024] Open
Abstract
BACKGROUND Magnetic resonance imaging (MRI) provides state-of-the-art image quality for neuroimaging, consisting of multiple separately acquired contrasts. Synthetic MRI aims to accelerate examinations by synthesizing any desirable contrast from a single acquisition. PURPOSE We developed a physics-informed deep learning-based method to synthesize multiple brain MRI contrasts from a single 5-min acquisition and investigate its ability to generalize to arbitrary contrasts. METHODS A dataset of 55 subjects acquired with a clinical MRI protocol and a 5-min transient-state sequence was used. The model, based on a generative adversarial network, maps data acquired from the five-minute scan to "effective" quantitative parameter maps (q*-maps), feeding the generated PD, T1, and T2 maps into a signal model to synthesize four clinical contrasts (proton density-weighted, T1-weighted, T2-weighted, and T2-weighted fluid-attenuated inversion recovery), from which losses are computed. The synthetic contrasts are compared to an end-to-end deep learning-based method proposed by literature. The generalizability of the proposed method is investigated for five volunteers by synthesizing three contrasts unseen during training and comparing these to ground truth acquisitions via qualitative assessment and contrast-to-noise ratio (CNR) assessment. RESULTS The physics-informed method matched the quality of the end-to-end method for the four standard contrasts, with structural similarity metrics above0.75 ± 0.08 $0.75\pm 0.08$ ( ± $\pm$ std), peak signal-to-noise ratios above22.4 ± 1.9 $22.4\pm 1.9$ , representing a portion of compact lesions comparable to standard MRI. Additionally, the physics-informed method enabled contrast adjustment, and similar signal contrast and comparable CNRs to the ground truth acquisitions for three sequences unseen during model training. CONCLUSIONS The study demonstrated the feasibility of physics-informed, deep learning-based synthetic MRI to generate high-quality contrasts and generalize to contrasts beyond the training data. This technology has the potential to accelerate neuroimaging protocols.
Collapse
Affiliation(s)
- Luuk Jacobs
- Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
- Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht, Utrecht, The Netherlands
| | - Stefano Mandija
- Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
- Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht, Utrecht, The Netherlands
| | - Hongyan Liu
- Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
- Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht, Utrecht, The Netherlands
| | - Cornelis A T van den Berg
- Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
- Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht, Utrecht, The Netherlands
| | - Alessandro Sbrizzi
- Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
- Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht, Utrecht, The Netherlands
| | - Matteo Maspero
- Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
- Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht, Utrecht, The Netherlands
| |
Collapse
|
14
|
Li Z, An K, Yu H, Luo F, Pan J, Wang S, Zhang J, Wu W, Chang D. Spectrum learning for super-resolution tomographic reconstruction. Phys Med Biol 2024; 69:085018. [PMID: 38373346 DOI: 10.1088/1361-6560/ad2a94] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 02/19/2024] [Indexed: 02/21/2024]
Abstract
Objective. Computed Tomography (CT) has been widely used in industrial high-resolution non-destructive testing. However, it is difficult to obtain high-resolution images for large-scale objects due to their physical limitations. The objective is to develop an improved super-resolution technique that preserves small structures and details while efficiently capturing high-frequency information.Approach. The study proposes a new deep learning based method called spectrum learning (SPEAR) network for CT images super-resolution. This approach leverages both global information in the image domain and high-frequency information in the frequency domain. The SPEAR network is designed to reconstruct high-resolution images from low-resolution inputs by considering not only the main body of the images but also the small structures and other details. The symmetric property of the spectrum is exploited to reduce weight parameters in the frequency domain. Additionally, a spectrum loss is introduced to enforce the preservation of both high-frequency components and global information.Main results. The network is trained using pairs of low-resolution and high-resolution CT images, and it is fine-tuned using additional low-dose and normal-dose CT image pairs. The experimental results demonstrate that the proposed SPEAR network outperforms state-of-the-art networks in terms of image reconstruction quality. The approach successfully preserves high-frequency information and small structures, leading to better results compared to existing methods. The network's ability to generate high-resolution images from low-resolution inputs, even in cases of low-dose CT images, showcases its effectiveness in maintaining image quality.Significance. The proposed SPEAR network's ability to simultaneously capture global information and high-frequency details addresses the limitations of existing methods, resulting in more accurate and informative image reconstructions. This advancement can have substantial implications for various industries and medical diagnoses relying on accurate imaging.
Collapse
Affiliation(s)
- Zirong Li
- The School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Guangdong, People's Republic of China
| | - Kang An
- The Key Laboratory of Optoelectronic Technology and Systems, ICT Research Center, Ministry of Education, Chongqing University, Chongqing, People's Republic of China
| | - Hengyong Yu
- The Department of Electrical and Computer Engineering, University of Massachusetts Lowell, Lowell, MA, United States of America
| | - Fulin Luo
- The College of Computer Science, Chongqing University, Chongqing, People's Republic of China
| | - Jiayi Pan
- The School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Guangdong, People's Republic of China
| | - Shaoyu Wang
- The Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, People's Republic of China
| | - Jianjia Zhang
- The School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Guangdong, People's Republic of China
| | - Weiwen Wu
- The School of Biomedical Engineering, Shenzhen Campus of Sun Yat-sen University, Guangdong, People's Republic of China
- The Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou, People's Republic of China
| | - Dingyue Chang
- The China Academy of Engineering Physics, Institute of Materials, Mianyang 621700, People's Republic of China
| |
Collapse
|
15
|
Hajim WI, Zainudin S, Mohd Daud K, Alheeti K. Optimized models and deep learning methods for drug response prediction in cancer treatments: a review. PeerJ Comput Sci 2024; 10:e1903. [PMID: 38660174 PMCID: PMC11042005 DOI: 10.7717/peerj-cs.1903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 01/31/2024] [Indexed: 04/26/2024]
Abstract
Recent advancements in deep learning (DL) have played a crucial role in aiding experts to develop personalized healthcare services, particularly in drug response prediction (DRP) for cancer patients. The DL's techniques contribution to this field is significant, and they have proven indispensable in the medical field. This review aims to analyze the diverse effectiveness of various DL models in making these predictions, drawing on research published from 2017 to 2023. We utilized the VOS-Viewer 1.6.18 software to create a word cloud from the titles and abstracts of the selected studies. This study offers insights into the focus areas within DL models used for drug response. The word cloud revealed a strong link between certain keywords and grouped themes, highlighting terms such as deep learning, machine learning, precision medicine, precision oncology, drug response prediction, and personalized medicine. In order to achieve an advance in DRP using DL, the researchers need to work on enhancing the models' generalizability and interoperability. It is also crucial to develop models that not only accurately represent various architectures but also simplify these architectures, balancing the complexity with the predictive capabilities. In the future, researchers should try to combine methods that make DL models easier to understand; this will make DRP reviews more open and help doctors trust the decisions made by DL models in cancer DRP.
Collapse
Affiliation(s)
- Wesam Ibrahim Hajim
- Department of Applied Geology, College of Sciences, Tirkit University, Tikrit, Salah ad Din, Iraq
- Center for Artificial Intelligence Technology, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Selangor, Malaysia
| | - Suhaila Zainudin
- Center for Artificial Intelligence Technology, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Selangor, Malaysia
| | - Kauthar Mohd Daud
- Center for Artificial Intelligence Technology, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Selangor, Malaysia
| | - Khattab Alheeti
- Department of Computer Networking Systems, College of Computer Sciences and Information Technology, University of Anbar, Al Anbar, Ramadi, Iraq
| |
Collapse
|
16
|
Wang J, Awad M, Zhou R, Wang Z, Wang X, Feng X, Yang Y, Meyer C, Kramer CM, Salerno M. High-resolution spiral real-time cardiac cine imaging with deep learning-based rapid image reconstruction and quantification. NMR IN BIOMEDICINE 2024; 37:e5051. [PMID: 37926525 DOI: 10.1002/nbm.5051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 08/20/2023] [Accepted: 09/08/2023] [Indexed: 11/07/2023]
Abstract
The objective of the current study was to develop and evaluate a DEep learning-based rapid Spiral Image REconstruction (DESIRE) and deep learning (DL)-based segmentation approach to quantify the left ventricular ejection fraction (LVEF) for high-resolution spiral real-time cine imaging, including 2D balanced steady-state free precession imaging at 1.5 T and gradient echo (GRE) imaging at 1.5 and 3 T. A 3D U-Net-based image reconstruction network and 2D U-Net-based image segmentation network were proposed and evaluated. Low-rank plus sparse (L+S) served as the reference for the image reconstruction network and manual contouring of the left ventricle was the reference of the segmentation network. To assess the image reconstruction quality, structural similarity index, peak signal-to-noise ratio, normalized root-mean-square error, and blind grading by two experienced cardiologists (5: excellent; 1: poor) were performed. To assess the segmentation performance, quantification of the LVEF on GRE imaging at 3 T was compared with the quantification from manual contouring. Excellent performance was demonstrated by the proposed technique. In terms of image quality, there was no difference between L+S and the proposed DESIRE technique. For quantification analysis, the proposed DL method was not different to the manual segmentation method (p > 0.05) in terms of quantification of LVEF. The reconstruction time for DESIRE was ~32 s (including nonuniform fast Fourier transform [NUFFT]) per dynamic series (40 frames), while the reconstruction time of L+S with GPU acceleration was approximately 3 min. The DL segmentation takes less than 5 s. In conclusion, the proposed DL-based image reconstruction and quantification techniques enabled 1-min image reconstruction for the whole heart and quantification with automatic reconstruction and quantification of the left ventricle function for high-resolution spiral real-time cine imaging with excellent performance.
Collapse
Affiliation(s)
- Junyu Wang
- Department of Medicine, Cardiovascular Medicine, Stanford University, Stanford, California, USA
| | - Marina Awad
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Ruixi Zhou
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
| | - Zhixing Wang
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
- Department of Radiation Oncology, City of Hope, Duarte, California, USA
| | - Xitong Wang
- Department of Bioengineering, Stanford University, Stanford, California, USA
| | - Xue Feng
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Yang Yang
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, California, USA
| | - Craig Meyer
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Christopher M Kramer
- Department of Medicine, Division of Cardiovascular, University of Virginia Health System, Charlottesville, Virginia, USA
- Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Virginia, USA
| | - Michael Salerno
- Department of Medicine, Cardiovascular Medicine, Stanford University, Stanford, California, USA
- Department of Radiology, Cardiovascular Imaging, Stanford University, Stanford, California, USA
| |
Collapse
|
17
|
Hossain MB, Shinde RK, Oh S, Kwon KC, Kim N. A Systematic Review and Identification of the Challenges of Deep Learning Techniques for Undersampled Magnetic Resonance Image Reconstruction. SENSORS (BASEL, SWITZERLAND) 2024; 24:753. [PMID: 38339469 PMCID: PMC10856856 DOI: 10.3390/s24030753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 01/05/2024] [Accepted: 01/22/2024] [Indexed: 02/12/2024]
Abstract
Deep learning (DL) in magnetic resonance imaging (MRI) shows excellent performance in image reconstruction from undersampled k-space data. Artifact-free and high-quality MRI reconstruction is essential for ensuring accurate diagnosis, supporting clinical decision-making, enhancing patient safety, facilitating efficient workflows, and contributing to the validity of research studies and clinical trials. Recently, deep learning has demonstrated several advantages over conventional MRI reconstruction methods. Conventional methods rely on manual feature engineering to capture complex patterns and are usually computationally demanding due to their iterative nature. Conversely, DL methods use neural networks with hundreds of thousands of parameters and automatically learn relevant features and representations directly from the data. Nevertheless, there are some limitations to DL-based techniques concerning MRI reconstruction tasks, such as the need for large, labeled datasets, the possibility of overfitting, and the complexity of model training. Researchers are striving to develop DL models that are more efficient, adaptable, and capable of providing valuable information for medical practitioners. We provide a comprehensive overview of the current developments and clinical uses by focusing on state-of-the-art DL architectures and tools used in MRI reconstruction. This study has three objectives. Our main objective is to describe how various DL designs have changed over time and talk about cutting-edge tactics, including their advantages and disadvantages. Hence, data pre- and post-processing approaches are assessed using publicly available MRI datasets and source codes. Secondly, this work aims to provide an extensive overview of the ongoing research on transformers and deep convolutional neural networks for rapid MRI reconstruction. Thirdly, we discuss several network training strategies, like supervised, unsupervised, transfer learning, and federated learning for rapid and efficient MRI reconstruction. Consequently, this article provides significant resources for future improvement of MRI data pre-processing and fast image reconstruction.
Collapse
Affiliation(s)
- Md. Biddut Hossain
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea; (M.B.H.); (R.K.S.)
| | - Rupali Kiran Shinde
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea; (M.B.H.); (R.K.S.)
| | - Sukhoon Oh
- Research Equipment Operation Department, Korea Basic Science Institute, Cheongju-si 28119, Chungcheongbuk-do, Republic of Korea;
| | - Ki-Chul Kwon
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea; (M.B.H.); (R.K.S.)
| | - Nam Kim
- School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Republic of Korea; (M.B.H.); (R.K.S.)
| |
Collapse
|
18
|
Noordman CR, Yakar D, Bosma J, Simonis FFJ, Huisman H. Complexities of deep learning-based undersampled MR image reconstruction. Eur Radiol Exp 2023; 7:58. [PMID: 37789241 PMCID: PMC10547669 DOI: 10.1186/s41747-023-00372-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 08/01/2023] [Indexed: 10/05/2023] Open
Abstract
Artificial intelligence has opened a new path of innovation in magnetic resonance (MR) image reconstruction of undersampled k-space acquisitions. This review offers readers an analysis of the current deep learning-based MR image reconstruction methods. The literature in this field shows exponential growth, both in volume and complexity, as the capabilities of machine learning in solving inverse problems such as image reconstruction are explored. We review the latest developments, aiming to assist researchers and radiologists who are developing new methods or seeking to provide valuable feedback. We shed light on key concepts by exploring the technical intricacies of MR image reconstruction, highlighting the importance of raw datasets and the difficulty of evaluating diagnostic value using standard metrics.Relevance statement Increasingly complex algorithms output reconstructed images that are difficult to assess for robustness and diagnostic quality, necessitating high-quality datasets and collaboration with radiologists.Key points• Deep learning-based image reconstruction algorithms are increasing both in complexity and performance.• The evaluation of reconstructed images may mistake perceived image quality for diagnostic value.• Collaboration with radiologists is crucial for advancing deep learning technology.
Collapse
Affiliation(s)
- Constant Richard Noordman
- Diagnostic Image Analysis Group, Department of Medical Imaging, Radboud University Medical Center, Nijmegen, 6525 GA, The Netherlands.
| | - Derya Yakar
- Medical Imaging Center, Departments of Radiology, Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, Groningen, 9700 RB, The Netherlands
| | - Joeran Bosma
- Diagnostic Image Analysis Group, Department of Medical Imaging, Radboud University Medical Center, Nijmegen, 6525 GA, The Netherlands
| | | | - Henkjan Huisman
- Diagnostic Image Analysis Group, Department of Medical Imaging, Radboud University Medical Center, Nijmegen, 6525 GA, The Netherlands
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, 7030, Norway
| |
Collapse
|
19
|
Bi W, Xv J, Song M, Hao X, Gao D, Qi F. Linear fine-tuning: a linear transformation based transfer strategy for deep MRI reconstruction. Front Neurosci 2023; 17:1202143. [PMID: 37409107 PMCID: PMC10318193 DOI: 10.3389/fnins.2023.1202143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 06/05/2023] [Indexed: 07/07/2023] Open
Abstract
Introduction Fine-tuning (FT) is a generally adopted transfer learning method for deep learning-based magnetic resonance imaging (MRI) reconstruction. In this approach, the reconstruction model is initialized with pre-trained weights derived from a source domain with ample data and subsequently updated with limited data from the target domain. However, the direct full-weight update strategy can pose the risk of "catastrophic forgetting" and overfitting, hindering its effectiveness. The goal of this study is to develop a zero-weight update transfer strategy to preserve pre-trained generic knowledge and reduce overfitting. Methods Based on the commonality between the source and target domains, we assume a linear transformation relationship of the optimal model weights from the source domain to the target domain. Accordingly, we propose a novel transfer strategy, linear fine-tuning (LFT), which introduces scaling and shifting (SS) factors into the pre-trained model. In contrast to FT, LFT only updates SS factors in the transfer phase, while the pre-trained weights remain fixed. Results To evaluate the proposed LFT, we designed three different transfer scenarios and conducted a comparative analysis of FT, LFT, and other methods at various sampling rates and data volumes. In the transfer scenario between different contrasts, LFT outperforms typical transfer strategies at various sampling rates and considerably reduces artifacts on reconstructed images. In transfer scenarios between different slice directions or anatomical structures, LFT surpasses the FT method, particularly when the target domain contains a decreasing number of training images, with a maximum improvement of up to 2.06 dB (5.89%) in peak signal-to-noise ratio. Discussion The LFT strategy shows great potential to address the issues of "catastrophic forgetting" and overfitting in transfer scenarios for MRI reconstruction, while reducing the reliance on the amount of data in the target domain. Linear fine-tuning is expected to shorten the development cycle of reconstruction models for adapting complicated clinical scenarios, thereby enhancing the clinical applicability of deep MRI reconstruction.
Collapse
Affiliation(s)
- Wanqing Bi
- The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China
| | - Jianan Xv
- The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China
| | - Mengdie Song
- The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China
| | - Xiaohan Hao
- The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China
- Fuqing Medical Co., Ltd., Hefei, Anhui, China
| | - Dayong Gao
- Department of Mechanical Engineering, University of Washington, Seattle, WA, United States
| | - Fulang Qi
- The Centers for Biomedical Engineering, University of Science and Technology of China, Hefei, Anhui, China
| |
Collapse
|
20
|
Güngör A, Dar SU, Öztürk Ş, Korkmaz Y, Bedel HA, Elmas G, Ozbey M, Çukur T. Adaptive diffusion priors for accelerated MRI reconstruction. Med Image Anal 2023; 88:102872. [PMID: 37384951 DOI: 10.1016/j.media.2023.102872] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 04/13/2023] [Accepted: 06/12/2023] [Indexed: 07/01/2023]
Abstract
Deep MRI reconstruction is commonly performed with conditional models that de-alias undersampled acquisitions to recover images consistent with fully-sampled data. Since conditional models are trained with knowledge of the imaging operator, they can show poor generalization across variable operators. Unconditional models instead learn generative image priors decoupled from the operator to improve reliability against domain shifts related to the imaging operator. Recent diffusion models are particularly promising given their high sample fidelity. Nevertheless, inference with a static image prior can perform suboptimally. Here we propose the first adaptive diffusion prior for MRI reconstruction, AdaDiff, to improve performance and reliability against domain shifts. AdaDiff leverages an efficient diffusion prior trained via adversarial mapping over large reverse diffusion steps. A two-phase reconstruction is executed following training: a rapid-diffusion phase that produces an initial reconstruction with the trained prior, and an adaptation phase that further refines the result by updating the prior to minimize data-consistency loss. Demonstrations on multi-contrast brain MRI clearly indicate that AdaDiff outperforms competing conditional and unconditional methods under domain shifts, and achieves superior or on par within-domain performance.
Collapse
Affiliation(s)
- Alper Güngör
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey; ASELSAN Research Center, Ankara 06200, Turkey
| | - Salman Uh Dar
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey; Department of Internal Medicine III, Heidelberg University Hospital, Heidelberg 69120, Germany
| | - Şaban Öztürk
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey; Department of Electrical and Electronics Engineering, Amasya University, Amasya 05100, Turkey
| | - Yilmaz Korkmaz
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey
| | - Hasan A Bedel
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey
| | - Gokberk Elmas
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey
| | - Muzaffer Ozbey
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey
| | - Tolga Çukur
- Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara 06800, Turkey; Neuroscience Program, Bilkent University, Ankara 06800, Turkey.
| |
Collapse
|
21
|
Jin Z, Xiang QS. Improving accelerated MRI by deep learning with sparsified complex data. Magn Reson Med 2023; 89:1825-1838. [PMID: 36480017 DOI: 10.1002/mrm.29556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2022] [Revised: 10/23/2022] [Accepted: 11/22/2022] [Indexed: 12/13/2022]
Abstract
PURPOSE To obtain high-quality accelerated MR images with complex-valued reconstruction from undersampled k-space data. METHODS The MRI scans from human subjects were retrospectively undersampled with a regular pattern using skipped phase encoding, leading to ghosts in zero-filling reconstruction. A complex difference transform along the phase-encoding direction was applied in image domain to yield sparsified complex-valued edge maps. These sparse edge maps were used to train a complex-valued U-type convolutional neural network (SCU-Net) for deghosting. A k-space inverse filtering was performed on the predicted deghosted complex edge maps from SCU-Net to obtain final complex images. The SCU-Net was compared with other algorithms including zero-filling, GRAPPA, RAKI, finite difference complex U-type convolutional neural network (FDCU-Net), and CU-Net, both qualitatively and quantitatively, using such metrics as structural similarity index, peak SNR, and normalized mean square error. RESULTS The SCU-Net was found to be effective in deghosting aliased edge maps even at high acceleration factors. High-quality complex images were obtained by performing an inverse filtering on deghosted edge maps. The SCU-Net compared favorably with other algorithms. CONCLUSION Using sparsified complex data, SCU-Net offers higher reconstruction quality for regularly undersampled k-space data. The proposed method is especially useful for phase-sensitive MRI applications.
Collapse
Affiliation(s)
- Zhaoyang Jin
- Machine Learning and I-health International Cooperation Base of Zhejiang Province, School of Automation, Hangzhou Dianzi University, Hangzhou, People's Republic of China
| | - Qing-San Xiang
- Department of Radiology, University of British Columbia, British Columbia, Vancouver, Canada
| |
Collapse
|
22
|
Ouchi S, Ito S. Efficient complex-valued image reconstruction for compressed sensing MRI using single real-valued convolutional neural network. Magn Reson Imaging 2023; 101:13-24. [PMID: 36965835 DOI: 10.1016/j.mri.2023.03.011] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 03/19/2023] [Accepted: 03/21/2023] [Indexed: 03/27/2023]
Affiliation(s)
- Shohei Ouchi
- Department of Information and Control Systems Science, Graduate School of Engineering, Utsunomiya University, 7-1-2 Yoto, Utsunomiya, Tochigi 321-8585, Japan; Japan Society for the Promotion of Science, Japan.
| | - Satoshi Ito
- Department of Information and Control Systems Science, Graduate School of Engineering, Utsunomiya University, 7-1-2 Yoto, Utsunomiya, Tochigi 321-8585, Japan.
| |
Collapse
|
23
|
Deveshwar N, Rajagopal A, Sahin S, Shimron E, Larson PEZ. Synthesizing Complex-Valued Multicoil MRI Data from Magnitude-Only Images. Bioengineering (Basel) 2023; 10:bioengineering10030358. [PMID: 36978749 PMCID: PMC10045391 DOI: 10.3390/bioengineering10030358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 03/08/2023] [Accepted: 03/12/2023] [Indexed: 03/18/2023] Open
Abstract
Despite the proliferation of deep learning techniques for accelerated MRI acquisition and enhanced image reconstruction, the construction of large and diverse MRI datasets continues to pose a barrier to effective clinical translation of these technologies. One major challenge is in collecting the MRI raw data (required for image reconstruction) from clinical scanning, as only magnitude images are typically saved and used for clinical assessment and diagnosis. The image phase and multi-channel RF coil information are not retained when magnitude-only images are saved in clinical imaging archives. Additionally, preprocessing used for data in clinical imaging can lead to biased results. While several groups have begun concerted efforts to collect large amounts of MRI raw data, current databases are limited in the diversity of anatomy, pathology, annotations, and acquisition types they contain. To address this, we present a method for synthesizing realistic MR data from magnitude-only data, allowing for the use of diverse data from clinical imaging archives in advanced MRI reconstruction development. Our method uses a conditional GAN-based framework to generate synthetic phase images from input magnitude images. We then applied ESPIRiT to derive RF coil sensitivity maps from fully sampled real data to generate multi-coil data. The synthetic data generation method was evaluated by comparing image reconstruction results from training Variational Networks either with real data or synthetic data. We demonstrate that the Variational Network trained on synthetic MRI data from our method, consisting of GAN-derived synthetic phase and multi-coil information, outperformed Variational Networks trained on data with synthetic phase generated using current state-of-the-art methods. Additionally, we demonstrate that the Variational Networks trained with synthetic k-space data from our method perform comparably to image reconstruction networks trained on undersampled real k-space data.
Collapse
Affiliation(s)
- Nikhil Deveshwar
- UC Berkeley-UCSF Graduate Program in Bioengineering, Berkeley, CA 94701, USA
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94701, USA
| | - Abhejit Rajagopal
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
| | - Sule Sahin
- UC Berkeley-UCSF Graduate Program in Bioengineering, Berkeley, CA 94701, USA
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
| | - Efrat Shimron
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94701, USA
| | - Peder E. Z. Larson
- UC Berkeley-UCSF Graduate Program in Bioengineering, Berkeley, CA 94701, USA
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA 94016, USA
- Correspondence:
| |
Collapse
|
24
|
Oscanoa JA, Middione MJ, Alkan C, Yurt M, Loecher M, Vasanawala SS, Ennis DB. Deep Learning-Based Reconstruction for Cardiac MRI: A Review. Bioengineering (Basel) 2023; 10:334. [PMID: 36978725 PMCID: PMC10044915 DOI: 10.3390/bioengineering10030334] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 03/03/2023] [Accepted: 03/03/2023] [Indexed: 03/09/2023] Open
Abstract
Cardiac magnetic resonance (CMR) is an essential clinical tool for the assessment of cardiovascular disease. Deep learning (DL) has recently revolutionized the field through image reconstruction techniques that allow unprecedented data undersampling rates. These fast acquisitions have the potential to considerably impact the diagnosis and treatment of cardiovascular disease. Herein, we provide a comprehensive review of DL-based reconstruction methods for CMR. We place special emphasis on state-of-the-art unrolled networks, which are heavily based on a conventional image reconstruction framework. We review the main DL-based methods and connect them to the relevant conventional reconstruction theory. Next, we review several methods developed to tackle specific challenges that arise from the characteristics of CMR data. Then, we focus on DL-based methods developed for specific CMR applications, including flow imaging, late gadolinium enhancement, and quantitative tissue characterization. Finally, we discuss the pitfalls and future outlook of DL-based reconstructions in CMR, focusing on the robustness, interpretability, clinical deployment, and potential for new methods.
Collapse
Affiliation(s)
- Julio A. Oscanoa
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
| | | | - Cagan Alkan
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA
| | - Mahmut Yurt
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA
| | - Michael Loecher
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
| | | | - Daniel B. Ennis
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
| |
Collapse
|
25
|
Dawood P, Breuer F, Stebani J, Burd P, Homolya I, Oberberger J, Jakob PM, Blaimer M. Iterative training of robust k-space interpolation networks for improved image reconstruction with limited scan specific training samples. Magn Reson Med 2023; 89:812-827. [PMID: 36226661 DOI: 10.1002/mrm.29482] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 09/12/2022] [Accepted: 09/20/2022] [Indexed: 12/13/2022]
Abstract
PURPOSE To evaluate an iterative learning approach for enhanced performance of robust artificial-neural-networks for k-space interpolation (RAKI), when only a limited amount of training data (auto-calibration signals [ACS]) are available for accelerated standard 2D imaging. METHODS In a first step, the RAKI model was tailored for the case of limited training data amount. In the iterative learning approach (termed iterative RAKI [iRAKI]), the tailored RAKI model is initially trained using original and augmented ACS obtained from a linear parallel imaging reconstruction. Subsequently, the RAKI convolution filters are refined iteratively using original and augmented ACS extracted from the previous RAKI reconstruction. Evaluation was carried out on 200 retrospectively undersampled in vivo datasets from the fastMRI neuro database with different contrast settings. RESULTS For limited training data (18 and 22 ACS lines for R = 4 and R = 5, respectively), iRAKI outperforms standard RAKI by reducing residual artifacts and yields better noise suppression when compared to standard parallel imaging, underlined by quantitative reconstruction quality metrics. Additionally, iRAKI shows better performance than both GRAPPA and standard RAKI in case of pre-scan calibration with varying contrast between training- and undersampled data. CONCLUSION RAKI benefits from the iterative learning approach, which preserves the noise suppression feature, but requires less original training data for the accurate reconstruction of standard 2D images thereby improving net acceleration.
Collapse
Affiliation(s)
- Peter Dawood
- Department of Physics, University of Würzburg, Würzburg, Germany
| | - Felix Breuer
- Magnetic Resonance and X-ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, Division Development Center X-Ray Technology, Würzburg, Germany
| | - Jannik Stebani
- Magnetic Resonance and X-ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, Division Development Center X-Ray Technology, Würzburg, Germany
| | - Paul Burd
- Institute for Theoretical Physics and Astrophysics, University of Würzburg, Würzburg, Germany
| | - István Homolya
- Brain Imaging Centre, Research Centre for Natural Sciences, Budapest, Hungary
| | - Johannes Oberberger
- Department of Internal Medicine I, University Hospital Würzburg, Würzburg, Germany
| | - Peter M Jakob
- Department of Physics, University of Würzburg, Würzburg, Germany
| | - Martin Blaimer
- Magnetic Resonance and X-ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, Division Development Center X-Ray Technology, Würzburg, Germany
| |
Collapse
|
26
|
Blumenthal M, Luo G, Schilling M, Holme HCM, Uecker M. Deep, deep learning with BART. Magn Reson Med 2023; 89:678-693. [PMID: 36254526 PMCID: PMC10898647 DOI: 10.1002/mrm.29485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Revised: 08/26/2022] [Accepted: 09/20/2022] [Indexed: 12/13/2022]
Abstract
PURPOSE To develop a deep-learning-based image reconstruction framework for reproducible research in MRI. METHODS The BART toolbox offers a rich set of implementations of calibration and reconstruction algorithms for parallel imaging and compressed sensing. In this work, BART was extended by a nonlinear operator framework that provides automatic differentiation to allow computation of gradients. Existing MRI-specific operators of BART, such as the nonuniform fast Fourier transform, are directly integrated into this framework and are complemented by common building blocks used in neural networks. To evaluate the use of the framework for advanced deep-learning-based reconstruction, two state-of-the-art unrolled reconstruction networks, namely the Variational Network and MoDL, were implemented. RESULTS State-of-the-art deep image-reconstruction networks can be constructed and trained using BART's gradient-based optimization algorithms. The BART implementation achieves a similar performance in terms of training time and reconstruction quality compared to the original implementations based on TensorFlow. CONCLUSION By integrating nonlinear operators and neural networks into BART, we provide a general framework for deep-learning-based reconstruction in MRI.
Collapse
Affiliation(s)
- Moritz Blumenthal
- Institute for Diagnostic and Interventional Radiology,
University Medical Center Göttingen, Göttingen, Germany
| | - Guanxiong Luo
- Institute for Diagnostic and Interventional Radiology,
University Medical Center Göttingen, Göttingen, Germany
| | - Martin Schilling
- Institute for Diagnostic and Interventional Radiology,
University Medical Center Göttingen, Göttingen, Germany
| | | | - Martin Uecker
- Institute for Diagnostic and Interventional Radiology,
University Medical Center Göttingen, Göttingen, Germany
- Institute of Biomedical Imaging, Graz University of
Technology, Graz, Austria
- German Centre for Cardiovascular Research (DZHK),Partner
Site Göttingen, Göttingen, Germany
- Cluster of Excellence “Multiscale Bioimaging: from
Molecular Machines to Networks of Excitable Cells” (MBExC), University of
Göttingen, Germany
- BioTechMed-Graz, Graz, Austria
| |
Collapse
|
27
|
Basty N, Thanaj M, Cule M, Sorokin EP, Liu Y, Thomas EL, Bell JD, Whitcher B. Artifact-free fat-water separation in Dixon MRI using deep learning. JOURNAL OF BIG DATA 2023; 10:4. [PMID: 36686622 PMCID: PMC9835035 DOI: 10.1186/s40537-022-00677-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/04/2022] [Accepted: 12/25/2022] [Indexed: 06/17/2023]
Abstract
Chemical-shift encoded MRI (CSE-MRI) is a widely used technique for the study of body composition and metabolic disorders, where derived fat and water signals enable the quantification of adipose tissue and muscle. The UK Biobank is acquiring whole-body Dixon MRI (a specific implementation of CSE-MRI) for over 100,000 participants. Current processing methods associated with large whole-body volumes are time intensive and prone to artifacts during fat-water separation performed by the scanner, making quantitative analysis challenging. The most common artifacts are fat-water swaps, where the labels are inverted at the voxel level. It is common for researchers to discard swapped data (generally around 10%), which is wasteful and may lead to unintended biases. Given the large number of whole-body Dixon MRI acquisitions in the UK Biobank, thousands of swaps are expected to be present in the fat and water volumes from image reconstruction performed on the scanner. If they go undetected, errors will propagate into processes such as organ segmentation, and dilute the results in population-based analyses. There is a clear need for a robust method to accurately separate fat and water volumes in big data collections like the UK Biobank. We formulate fat-water separation as a style transfer problem, where swap-free fat and water volumes are predicted from the acquired Dixon MRI data using a conditional generative adversarial network, and introduce a new loss function for the generator model. Our method is able to predict highly accurate fat and water volumes free from artifacts in the UK Biobank. We show that our model separates fat and water volumes using either single input (in-phase only) or dual input (in-phase and opposed-phase) data, with the latter producing superior results. Our proposed method enables faster and more accurate downstream analysis of body composition from Dixon MRI in population studies by eliminating the need for visual inspection or discarding data due to fat-water swaps. Supplementary Information The online version contains supplementary material available at 10.1186/s40537-022-00677-1.
Collapse
Affiliation(s)
- Nicolas Basty
- Research Centre for Optimal Health, University of Westminster, London, UK
| | - Marjola Thanaj
- Research Centre for Optimal Health, University of Westminster, London, UK
| | | | | | - Yi Liu
- Calico Life Sciences LLC, South San Francisco, USA
| | - E. Louise Thomas
- Research Centre for Optimal Health, University of Westminster, London, UK
| | - Jimmy D. Bell
- Research Centre for Optimal Health, University of Westminster, London, UK
| | - Brandon Whitcher
- Research Centre for Optimal Health, University of Westminster, London, UK
| |
Collapse
|
28
|
Li C, Liu S, Wang Z, Yuan G. Classifying epileptic phase-amplitude coupling in SEEG using complex-valued convolutional neural network. Front Physiol 2023; 13:1085530. [PMID: 36685186 PMCID: PMC9849379 DOI: 10.3389/fphys.2022.1085530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 12/20/2022] [Indexed: 01/06/2023] Open
Abstract
EEG phase-amplitude coupling (PAC), the amplitude of high-frequency oscillations modulated by the phase of low-frequency oscillations (LFOs), is a useful biomarker to localize epileptogenic tissue. It is commonly represented in a comodulogram of coupling strength but without coupled phase information. The phase-amplitude coupling is also found in the normal brain, and it is difficult to discriminate pathological phase-amplitude couplings from normal ones. This study proposes a novel approach based on complex-valued phase-amplitude coupling (CV-PAC) for classifying epileptic phase-amplitude coupling. The CV-PAC combines both the coupling strengths and the coupled phases of low-frequency oscillations. The complex-valued convolutional neural network (CV-CNN) is then used to classify epileptic CV-PAC. Stereo-electroencephalography (SEEG) recordings from nine intractable epilepsy patients were analyzed. The leave-one-out cross-validation is performed, and the area-under-curve (AUC) value is used as the indicator of the performance of different measures. Our result shows that the area-under-curve value is .92 for classifying epileptic CV-PAC using CV-CNN. The area-under-curve value decreases to .89, .80, and .88 while using traditional convolutional neural networks, support vector machine, and random forest, respectively. The phases of delta (1-4 Hz) and alpha (8-10 Hz) bands are different between epileptic and normal CV-PAC. The phase information of CV-PAC is important for improving classification performance. The proposed approach of CV-PAC/CV-CNN promises to identify more accurate epileptic brain activities for potential surgical intervention.
Collapse
Affiliation(s)
- Chunsheng Li
- Department of Biomedical Engineering, School of Electrical Engineering, Shenyang University of Technology, Shenyang, China,*Correspondence: Chunsheng Li,
| | - Shiyue Liu
- Department of Biomedical Engineering, School of Electrical Engineering, Shenyang University of Technology, Shenyang, China
| | - Zeyu Wang
- Department of Biomedical Engineering, School of Electrical Engineering, Shenyang University of Technology, Shenyang, China,Department of Electrical Engineering and Information Systems, University of Pannonia, Veszprem, Hungary
| | - Guanqian Yuan
- Department of Neurosurgery, General Hospital of Northern Theater Command, Shenyang, China
| |
Collapse
|
29
|
Takeshima H. [[MRI] 2. Recent Research on MR Image Reconstruction Using Artificial Intelligence]. Nihon Hoshasen Gijutsu Gakkai Zasshi 2023; 79:863-869. [PMID: 37599072 DOI: 10.6009/jjrt.2023-2236] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/22/2023]
Affiliation(s)
- Hidenori Takeshima
- Imaging Modality Group, Advanced Technology Research Department, Research and Development Center, Canon Medical Systems Corporation
| |
Collapse
|
30
|
Guo D, Zeng G, Fu H, Wang Z, Yang Y, Qu X. A Joint Group Sparsity-based deep learning for multi-contrast MRI reconstruction. JOURNAL OF MAGNETIC RESONANCE (SAN DIEGO, CALIF. : 1997) 2023; 346:107354. [PMID: 36527935 DOI: 10.1016/j.jmr.2022.107354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 11/24/2022] [Accepted: 12/03/2022] [Indexed: 06/17/2023]
Abstract
Multi-contrast magnetic resonance imaging (MRI) can provide richer diagnosis information. The data acquisition time, however, is increased than single-contrast imaging. To reduce this time, k-space undersampling is an effective way but a smart reconstruction algorithm is required to remove undersampling image artifacts. Traditional algorithms commonly explore the similarity of multi-contrast images through joint sparsity. However, these algorithms are time-consuming due to the iterative process and require adjusting hyperparameters manually. Recently, data-driven deep learning successfully overcome these limitations but the reconstruction error still needs to be further reduced. Here, we propose a Joint Group Sparsity-based Network (JGSN) for multi-contrast MRI reconstruction, which unrolls the iterative process of the joint sparsity algorithm. The designed network includes data consistency modules, learnable sparse transform modules, and joint group sparsity constraint modules. In particular, weights of different contrasts in the transform module are shared to reduce network parameters without sacrificing the quality of reconstruction. The experiments were performed on the retrospective undersampled brain and knee data. Experimental results on in vivo brain data and knee data show that our method consistently outperforms the state-of-the-art methods under different sampling patterns.
Collapse
Affiliation(s)
- Di Guo
- School of Computer and Information Engineering, Fujian Engineering Research Center for Medical Data Mining and Application, Xiamen University of Technology, Xiamen, China
| | - Gushan Zeng
- School of Computer and Information Engineering, Fujian Engineering Research Center for Medical Data Mining and Application, Xiamen University of Technology, Xiamen, China
| | - Hao Fu
- School of Computer and Information Engineering, Fujian Engineering Research Center for Medical Data Mining and Application, Xiamen University of Technology, Xiamen, China
| | - Zi Wang
- Department of Electronic Science, Fujian Provincial Key Laboratory of Plasma and Magnetic Resonance, National Institute for Data Science in Health and Medicine, Xiamen University, China
| | - Yonggui Yang
- Department of Radiology, The Second Affiliated Hospital of Xiamen Medical College, Xiamen, China
| | - Xiaobo Qu
- Department of Electronic Science, Fujian Provincial Key Laboratory of Plasma and Magnetic Resonance, National Institute for Data Science in Health and Medicine, Xiamen University, China.
| |
Collapse
|
31
|
[Deep parallel MRI reconstruction based on a complex-valued loss function]. NAN FANG YI KE DA XUE XUE BAO = JOURNAL OF SOUTHERN MEDICAL UNIVERSITY 2022; 42:1755-1764. [PMID: 36651242 PMCID: PMC9878414 DOI: 10.12122/j.issn.1673-4254.2022.12.02] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
Abstract
OBJECTIVE To propose a new method for fast MRI reconstruction based on deep learning in parallel MRI data using a new loss function defined as the summation of the mean squared errors of the magnitude and phase. METHODS The multicoil image data were combined into single-coil image data to eliminate the correlation between noises and used as a label in the training process. Considering the importance of the phase information in some applications, where the phase information was lost when combining multicoil data using sum of square method, a new loss function was introduced, defined as the weighted sum of the mean squared error (MSE) of the magnitude and phase. The single weight in the loss function was used to balance the importance of the magnitude and phase in different applications. To validate the proposed method, real brain and knee data in FastMRI dataset were used for training and testing. We also compared this proposed method with two other methods that used MSE or mean absolute error (MAE) as a loss function. RESULTS The experimental results showed that the proposed method was capable of accurate reconstruction of multicoil MR images with significantly reduced artifacts compared with the other two methods. Quantitative analysis showed that the propose method increased the peak signal-to-noise ratio (PSNR) of the reconstructed images by about 1 dB. CONCLUSION The proposed deep MRI reconstruction method using a new loss function to fit the noise in parallel MRI data can accelerate MRI reconstruction and significantly improve the quality of the reconstructed images.
Collapse
|
32
|
⊥-loss: A symmetric loss function for magnetic resonance imaging reconstruction and image registration with deep learning. Med Image Anal 2022; 80:102509. [DOI: 10.1016/j.media.2022.102509] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 04/08/2022] [Accepted: 06/01/2022] [Indexed: 02/07/2023]
|
33
|
Jain P, Pradeep CS, Sinha N. The Complex-valued PD-net for MRI reconstruction of knee images. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:2093-2096. [PMID: 36085925 DOI: 10.1109/embc48229.2022.9872016] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
MRI reconstruction is the fundamental task of obtaining diagnostic quality images from MRI sensor data and is an active area of research for improving accuracy, speed and memory requirements of the process. Complex-valued neural networks have previously achieved superior MRI reconstructions compared to real-valued nets. But those works operated in the image domain to denoise poor quality reconstructions of the raw sensor (k-space) data. Also small-scale or proprietary datasets with few clinical images or raw k-space volumes were used in these works, and none of the works use publicly available large-scale raw k-space datasets. Recent studies have shown that cross-domain neural networks for MRI reconstruction, or networks which leverage information from both k-space and image domains, have better potential than single-domain networks which operate only in one domain. We study the effects of complex-valued operations on a top-performing cross-domain neural network for MRI reconstruction called the Primal-Dual net, or PD-net. The PD-net is a fully convolutional architecture that takes input as raw k-space data and outputs the reconstructions, thus performing both the inversion and denoising tasks. We experiment with the publicly available, large-scale fastMRI single-coil knee dataset having 973 train volumes and 199 validation volumes. Our proposed method (Complex PD-net) achieves PSNR and SSIM of 33.3 dB and 0.8033 respectively, compared to 32.13 dB and 0.728 obtained by PD-net. Our Complex PD-net achieves 10.3% higher SSIM with just over 50% of the total parameters w.r.t. the SOTA methodology.
Collapse
|
34
|
Karkalousos D, Noteboom S, Hulst HE, Vos FM, Caan MWA. Assessment of data consistency through cascades of independently recurrent inference machines for fast and robust accelerated MRI reconstruction. Phys Med Biol 2022; 67. [DOI: 10.1088/1361-6560/ac6cc2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 05/04/2022] [Indexed: 11/11/2022]
Abstract
Abstract
Objective. Machine Learning methods can learn how to reconstruct magnetic resonance images (MRI) and thereby accelerate acquisition, which is of paramount importance to the clinical workflow. Physics-informed networks incorporate the forward model of accelerated MRI reconstruction in the learning process. With increasing network complexity, robustness is not ensured when reconstructing data unseen during training. We aim to embed data consistency (DC) in deep networks while balancing the degree of network complexity. While doing so, we will assess whether either explicit or implicit enforcement of DC in varying network architectures is preferred to optimize performance. Approach. We propose a scheme called Cascades of Independently Recurrent Inference Machines (CIRIM) to assess DC through unrolled optimization. Herein we assess DC both implicitly by gradient descent and explicitly by a designed term. Extensive comparison of the CIRIM to compressed sensing as well as other Machine Learning methods is performed: the End-to-End Variational Network (E2EVN), CascadeNet, KIKINet, LPDNet, RIM, IRIM, and UNet. Models were trained and evaluated on T1-weighted and FLAIR contrast brain data, and T2-weighted knee data. Both 1D and 2D undersampling patterns were evaluated. Robustness was tested by reconstructing 7.5× prospectively undersampled 3D FLAIR MRI data of multiple sclerosis (MS) patients with white matter lesions. Main results. The CIRIM performed best when implicitly enforcing DC, while the E2EVN required an explicit DC formulation. Through its cascades, the CIRIM was able to score higher on structural similarity and PSNR compared to other methods, in particular under heterogeneous imaging conditions. In reconstructing MS patient data, prospectively acquired with a sampling pattern unseen during model training, the CIRIM maintained lesion contrast while efficiently denoising the images. Significance. The CIRIM showed highly promising generalization capabilities maintaining a very fair trade-off between reconstructed image quality and fast reconstruction times, which is crucial in the clinical workflow.
Collapse
|
35
|
Wang J, Weller DS, Kramer CM, Salerno M. DEep learning-based rapid Spiral Image REconstruction (DESIRE) for high-resolution spiral first-pass myocardial perfusion imaging. NMR IN BIOMEDICINE 2022; 35:e4661. [PMID: 34939246 DOI: 10.1002/nbm.4661] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Revised: 11/01/2021] [Accepted: 11/16/2021] [Indexed: 06/14/2023]
Abstract
The objective of the current study was to develop and evaluate a DEep learning-based rapid Spiral Image REconstruction (DESIRE) technique for high-resolution spiral first-pass myocardial perfusion imaging with whole-heart coverage, to provide fast and accurate image reconstruction for both single-slice (SS) and simultaneous multislice (SMS) acquisitions. Three-dimensional U-Net-based image enhancement architectures were evaluated for high-resolution spiral perfusion imaging at 3 T. The SS and SMS MB = 2 networks were trained on SS perfusion images from 156 slices from 20 subjects. Structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and normalized root mean square error (NRMSE) were assessed, and prospective images were blindly graded by two experienced cardiologists (5: excellent; 1: poor). Excellent performance was demonstrated for the proposed technique. For SS, SSIM, PSNR, and NRMSE were 0.977 [0.972, 0.982], 42.113 [40.174, 43.493] dB, and 0.102 [0.080, 0.125], respectively, for the best network. For SMS MB = 2 retrospective data, SSIM, PSNR, and NRMSE were 0.961 [0.950, 0.969], 40.834 [39.619, 42.004] dB, and 0.107 [0.086, 0.133], respectively, for the best network. The image quality scores were 4.5 [4.1, 4.8], 4.5 [4.3, 4.6], 3.5 [3.3, 4], and 3.5 [3.3, 3.8] for SS DESIRE, SS L1-SPIRiT, MB = 2 DESIRE, and MB = 2 SMS-slice-L1-SPIRiT, respectively, showing no statistically significant difference (p = 1 and p = 1 for SS and SMS, respectively) between L1-SPIRiT and the proposed DESIRE technique. The network inference time was ~100 ms per dynamic perfusion series with DESIRE, while the reconstruction time of L1-SPIRiT with GPU acceleration was ~ 30 min. It was concluded that DESIRE enabled fast and high-quality image reconstruction for both SS and SMS MB = 2 whole-heart high-resolution spiral perfusion imaging.
Collapse
Affiliation(s)
- Junyu Wang
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Daniel S Weller
- Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, Virginia, USA
| | - Christopher M Kramer
- Department of Medicine, Cardiovascular Division, University of Virginia Health System, Charlottesville, Virginia, USA
- Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Virginia, USA
| | - Michael Salerno
- Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia, USA
- Department of Medicine, Cardiovascular Division, University of Virginia Health System, Charlottesville, Virginia, USA
- Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Virginia, USA
- Departments of Medicine and Radiology, Stanford University Medical Center, Stanford, California, USA
| |
Collapse
|
36
|
Li Y, Wang Z, Lam F. SNR Enhancement for Multi-TE MRSI Using Joint Low-Dimensional Model and Spatial Constraints. IEEE Trans Biomed Eng 2022; 69:3087-3097. [PMID: 35320082 DOI: 10.1109/tbme.2022.3161417] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
We present a novel method to enhance the SNR for multi-TE MR spectroscopic imaging (MRSI) data by integrating learned nonlinear low-dimensional model and spatial constraints. A deep complex convolutional autoen-coder (DCCAE) was developed to learn a nonlinear low-dimensional representation of the high-dimensional multi-TE 1 H spectroscopy signals. The learned model significantly reduces the data dimension thus serving as an effective constraint for noise reduction. A reconstruction formulation was proposed to integrate the spatiospectral encoding model, the learned model, and a spatial constraint for an SNR-enhancing reconstruction from multi-TE data. The proposed method has been evaluated using both numerical simulations and in vivo brain MRSI experiments. The superior denoising performance of the proposed over alternative methods was demonstrated, both qualitatively and quantitatively. In vivo multi-TE data was used to assess the improved metabolite quantification reproducibility and accuracy achieved by the proposed method. We expect the proposed SNR-enhancing reconstruction to enable faster and/or higher-resolution multi-TE 1 H-MRSI of the brain, potentially useful for various clinical applications.
Collapse
|
37
|
Yurt M, Özbey M, UH Dar S, Tinaz B, Oguz KK, Çukur T. Progressively Volumetrized Deep Generative Models for Data-Efficient Contextual Learning of MR Image Recovery. Med Image Anal 2022; 78:102429. [DOI: 10.1016/j.media.2022.102429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Revised: 03/14/2022] [Accepted: 03/18/2022] [Indexed: 10/18/2022]
|
38
|
Jung S, Yun J, Kim DY, Kim D. Improved multi‐echo gradient echo myelin water fraction mapping using complex‐valued neural network analysis. Magn Reson Med 2022; 88:492-500. [DOI: 10.1002/mrm.29192] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Revised: 01/19/2022] [Accepted: 01/19/2022] [Indexed: 01/20/2023]
Affiliation(s)
- Soozy Jung
- Department of Electrical and Electronic Engineering Yonsei University Seoul Republic of Korea
| | - JiSu Yun
- Department of Electrical and Electronic Engineering Yonsei University Seoul Republic of Korea
| | - Deog Young Kim
- Department and Research Institute of Rehabilitation Medicine Yonsei University College of Medicine Seoul Republic of Korea
| | - Dong‐Hyun Kim
- Department of Electrical and Electronic Engineering Yonsei University Seoul Republic of Korea
| |
Collapse
|
39
|
Duan C, Xiong Y, Cheng K, Xiao S, Lyu J, Wang C, Bian X, Zhang J, Zhang D, Chen L, Zhou X, Lou X. Accelerating susceptibility-weighted imaging with deep learning by complex-valued convolutional neural network (ComplexNet): validation in clinical brain imaging. Eur Radiol 2022; 32:5679-5687. [PMID: 35182203 DOI: 10.1007/s00330-022-08638-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2021] [Revised: 12/15/2021] [Accepted: 01/11/2022] [Indexed: 11/30/2022]
Abstract
OBJECTIVES Susceptibility-weighted imaging (SWI) is crucial for the characterization of intracranial hemorrhage and mineralization, but has the drawback of long acquisition times. We aimed to propose a deep learning model to accelerate SWI, and evaluate the clinical feasibility of this approach. METHODS A complex-valued convolutional neural network (ComplexNet) was developed to reconstruct high-quality SWI from highly accelerated k-space data. ComplexNet can leverage the inherently complex-valued nature of SWI data and learn richer representations by using complex-valued network. SWI data were acquired from 117 participants who underwent clinical brain MRI examination between 2019 and 2021, including patients with tumor, stroke, hemorrhage, traumatic brain injury, etc. Reconstruction quality was evaluated using quantitative image metrics and image quality scores, including overall image quality, signal-to-noise ratio, sharpness, and artifacts. RESULTS The average reconstruction time of ComplexNet was 19 ms per section (1.33 s per participant). ComplexNet achieved significantly improved quantitative image metrics compared to a conventional compressed sensing method and a real-valued network with acceleration rates of 5 and 8 (p < 0.001). Meanwhile, there was no significant difference between fully sampled and ComplexNet approaches in terms of overall image quality and artifacts (p > 0.05) at both acceleration rates. Furthermore, ComplexNet showed comparable diagnostic performance to the fully sampled SWI for visualizing a wide range of pathology, including hemorrhage, cerebral microbleeds, and brain tumor. CONCLUSIONS ComplexNet can effectively accelerate SWI while providing superior performance in terms of overall image quality and visualization of pathology for routine clinical brain imaging. KEY POINTS • The complex-valued convolutional neural network (ComplexNet) allowed fast and high-quality reconstruction of highly accelerated SWI data, with an average reconstruction time of 19 ms per section. • ComplexNet achieved significantly improved quantitative image metrics compared to a conventional compressed sensing method and a real-valued network with acceleration rates of 5 and 8 (p < 0.001). • ComplexNet showed comparable diagnostic performance to the fully sampled SWI for visualizing a wide range of pathology, including hemorrhage, cerebral microbleeds, and brain tumor.
Collapse
Affiliation(s)
- Caohui Duan
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Yongqin Xiong
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Kun Cheng
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Sa Xiao
- Department of Neurosurgery, Chinese PLA General Hospital, 28 Fuxing Road, Beijing, 100853, People's Republic of China
| | - Jinhao Lyu
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Cheng Wang
- Department of Neurosurgery, Chinese PLA General Hospital, 28 Fuxing Road, Beijing, 100853, People's Republic of China
| | - Xiangbing Bian
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Jing Zhang
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Dekang Zhang
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Ling Chen
- Department of Neurosurgery, Chinese PLA General Hospital, 28 Fuxing Road, Beijing, 100853, People's Republic of China
| | - Xin Zhou
- Key Laboratory of Magnetic Resonance in Biological Systems, State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, National Center for Magnetic Resonance in Wuhan, Wuhan Institute of Physics and Mathematics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences-Wuhan National Laboratory for Optoelectronics, Wuhan, 430071, People's Republic of China
| | - Xin Lou
- Department of Radiology, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China.
| |
Collapse
|
40
|
Wang S, Cao G, Wang Y, Liao S, Wang Q, Shi J, Li C, Shen D. Review and Prospect: Artificial Intelligence in Advanced Medical Imaging. FRONTIERS IN RADIOLOGY 2021; 1:781868. [PMID: 37492170 PMCID: PMC10365109 DOI: 10.3389/fradi.2021.781868] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Accepted: 11/08/2021] [Indexed: 07/27/2023]
Abstract
Artificial intelligence (AI) as an emerging technology is gaining momentum in medical imaging. Recently, deep learning-based AI techniques have been actively investigated in medical imaging, and its potential applications range from data acquisition and image reconstruction to image analysis and understanding. In this review, we focus on the use of deep learning in image reconstruction for advanced medical imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Particularly, recent deep learning-based methods for image reconstruction will be emphasized, in accordance with their methodology designs and performances in handling volumetric imaging data. It is expected that this review can help relevant researchers understand how to adapt AI for medical imaging and which advantages can be achieved with the assistance of AI.
Collapse
Affiliation(s)
- Shanshan Wang
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences (CAS), Shenzhen, China
- Pengcheng Laboratrory, Shenzhen, China
| | - Guohua Cao
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - Yan Wang
- School of Computer Science, Sichuan University, Chengdu, China
| | - Shu Liao
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| | - Qian Wang
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - Jun Shi
- School of Communication and Information Engineering, Shanghai University, Shanghai, China
| | - Cheng Li
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences (CAS), Shenzhen, China
| | - Dinggang Shen
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
| |
Collapse
|
41
|
Xiao L, Liu Y, Yi Z, Zhao Y, Xie L, Cao P, Leong ATL, Wu EX. Partial Fourier reconstruction of complex MR images using complex-valued convolutional neural networks. Magn Reson Med 2021; 87:999-1014. [PMID: 34611904 DOI: 10.1002/mrm.29033] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2021] [Revised: 08/23/2021] [Accepted: 09/16/2021] [Indexed: 12/16/2022]
Abstract
PURPOSE To provide a complex-valued deep learning approach for partial Fourier (PF) reconstruction of complex MR images. METHODS Conventional PF reconstruction methods, such as projection onto convex sets (POCS), uses low-resolution image phase information from the central symmetrically sampled k-space for image reconstruction. However, this smooth phase constraint undermines the phase estimation accuracy in presence of rapid local phase variations, causing image artifacts and limiting the extent of PF reconstruction. Using both magnitude and phase characteristics in big complex image datasets, we propose a complex-valued deep learning approach with an unrolled network architecture for PF reconstruction that iteratively reconstructs PF sampled data and enforces data consistency. We evaluate our approach for reconstructing both spin-echo and gradient-echo data. RESULTS The proposed method outperformed the iterative POCS PF reconstruction method. It produced better artifact suppression and recovery of both image magnitude and phase details in presence of local phase changes. No noise amplification was observed even for highly PF reconstruction. Moreover, the network trained on axial brain data could reconstruct sagittal and coronal brain and knee data. This method could be extended to 2D PF reconstruction and joint multi-slice PF reconstruction. CONCLUSION Our proposed method can effectively reconstruct MR data even at low PF fractions, yielding high-fidelity magnitude and phase images. It presents a valuable alternative to conventional PF reconstruction, especially for phase-sensitive 2D or 3D MRI applications.
Collapse
Affiliation(s)
- Linfang Xiao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Yilong Liu
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Zheyuan Yi
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Yujiao Zhao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Linshan Xie
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Peibei Cao
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Alex T L Leong
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| | - Ed X Wu
- Laboratory of Biomedical Imaging and Signal Processing, The University of Hong Kong, Hong Kong SAR, People's Republic of China.,Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, People's Republic of China
| |
Collapse
|
42
|
Jaubert O, Montalt‐Tordera J, Knight D, Coghlan GJ, Arridge S, Steeden JA, Muthurangu V. Real-time deep artifact suppression using recurrent U-Nets for low-latency cardiac MRI. Magn Reson Med 2021; 86:1904-1916. [PMID: 34032308 PMCID: PMC8613539 DOI: 10.1002/mrm.28834] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2020] [Revised: 03/22/2021] [Accepted: 04/17/2021] [Indexed: 12/11/2022]
Abstract
PURPOSE Real-time low latency MRI is performed to guide various cardiac interventions. Real-time acquisitions often require iterative image reconstruction strategies, which lead to long reconstruction times. In this study, we aim to reconstruct highly undersampled radial real-time data with low latency using deep learning. METHODS A 2D U-Net with convolutional long short-term memory layers is proposed to exploit spatial and preceding temporal information to reconstruct highly accelerated tiny golden radial data with low latency. The network was trained using a dataset of breath-hold CINE data (including 770 time series from 7 different orientations). Synthetic paired data were created by retrospectively undersampling the magnitude images, and the network was trained to recover the target images. In the spirit of interventional imaging, the network was trained and tested for varying acceleration rates and orientations. Data were prospectively acquired and reconstructed in real time in 1 healthy subject interactively and in 3 patients who underwent catheterization. Images were visually compared to sliding window and compressed sensing reconstructions and a conventional Cartesian real-time sequence. RESULTS The proposed network generalized well to different acceleration rates and unseen orientations for all considered metrics in simulated data (less than 4% reduction in structural similarity index compared to similar acceleration and orientation-specific networks). The proposed reconstruction was demonstrated interactively, successfully depicting catheters in vivo with low latency (39 ms, including 19 ms for deep artifact suppression) and an image quality comparing favorably to other reconstructions. CONCLUSION Deep artifact suppression was successfully demonstrated in the time-critical application of non-Cartesian real-time interventional cardiac MR.
Collapse
Affiliation(s)
- Olivier Jaubert
- Department of Computer ScienceUniversity College LondonLondonUnited Kingdom
- UCL Centre for Translational Cardiovascular ImagingUniversity College LondonLondonUnited Kingdom
| | - Javier Montalt‐Tordera
- UCL Centre for Translational Cardiovascular ImagingUniversity College LondonLondonUnited Kingdom
| | - Dan Knight
- UCL Centre for Translational Cardiovascular ImagingUniversity College LondonLondonUnited Kingdom
- Department of CardiologyRoyal Free London NHS Foundation TrustLondonUnited Kingdom
| | - Gerry J. Coghlan
- UCL Centre for Translational Cardiovascular ImagingUniversity College LondonLondonUnited Kingdom
- Department of CardiologyRoyal Free London NHS Foundation TrustLondonUnited Kingdom
| | - Simon Arridge
- Department of Computer ScienceUniversity College LondonLondonUnited Kingdom
| | - Jennifer A. Steeden
- UCL Centre for Translational Cardiovascular ImagingUniversity College LondonLondonUnited Kingdom
| | - Vivek Muthurangu
- UCL Centre for Translational Cardiovascular ImagingUniversity College LondonLondonUnited Kingdom
| |
Collapse
|
43
|
Jaubert O, Steeden J, Montalt-Tordera J, Arridge S, Kowalik GT, Muthurangu V. Deep artifact suppression for spiral real-time phase contrast cardiac magnetic resonance imaging in congenital heart disease. Magn Reson Imaging 2021; 83:125-132. [PMID: 34419611 DOI: 10.1016/j.mri.2021.08.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2021] [Revised: 08/12/2021] [Accepted: 08/12/2021] [Indexed: 10/20/2022]
Abstract
PURPOSE Real-time spiral phase contrast MR (PCMR) enables rapid free-breathing assessment of flow. Target spatial and temporal resolutions require high acceleration rates often leading to long reconstruction times. Here we propose a deep artifact suppression framework for fast and accurate flow quantification. METHODS U-Nets were trained for deep artifact suppression using 520 breath-hold gated spiral PCMR aortic datasets collected in congenital heart disease patients. Two spiral trajectories (uniform and perturbed) and two losses (Mean Absolute Error -MAE- and average structural similarity index measurement -SSIM-) were compared in synthetic data in terms of MAE, peak SNR (PSNR) and SSIM. Perturbed spiral PCMR was prospectively acquired in 20 patients. Stroke Volume (SV), peak mean velocity and edge sharpness measurements were compared to Compressed Sensing (CS) and Cartesian reference. RESULTS In synthetic data, perturbed spiral consistently outperformed uniform spiral for the different image metrics. U-Net MAE showed better MAE and PSNR while U-Net SSIM showed higher SSIM based metrics. In-vivo, there were no significant differences in SV between any of the real-time reconstructions and the reference standard Cartesian data. However, U-Net SSIM had better image sharpness and lower biases for peak velocity when compared to U-Net MAE. Reconstruction of 96 frames took ~59 s for CS and 3.9 s for U-Nets. CONCLUSION Deep artifact suppression of complex valued images using an SSIM based loss was successfully demonstrated in a cohort of congenital heart disease patients for fast and accurate flow quantification.
Collapse
Affiliation(s)
- Olivier Jaubert
- Department of Computer Science, University College London, London WC1E 6BT, United Kingdom; Centre for Cardiovascular Imaging, Institute of Cardiovascular Science, University College London, London WC1N 1EH, United Kingdom.
| | - Jennifer Steeden
- Centre for Cardiovascular Imaging, Institute of Cardiovascular Science, University College London, London WC1N 1EH, United Kingdom
| | - Javier Montalt-Tordera
- Centre for Cardiovascular Imaging, Institute of Cardiovascular Science, University College London, London WC1N 1EH, United Kingdom
| | - Simon Arridge
- Department of Computer Science, University College London, London WC1E 6BT, United Kingdom
| | - Grzegorz Tomasz Kowalik
- Centre for Cardiovascular Imaging, Institute of Cardiovascular Science, University College London, London WC1N 1EH, United Kingdom
| | - Vivek Muthurangu
- Centre for Cardiovascular Imaging, Institute of Cardiovascular Science, University College London, London WC1N 1EH, United Kingdom
| |
Collapse
|
44
|
Lv J, Li G, Tong X, Chen W, Huang J, Wang C, Yang G. Transfer learning enhanced generative adversarial networks for multi-channel MRI reconstruction. Comput Biol Med 2021; 134:104504. [PMID: 34062366 DOI: 10.1016/j.compbiomed.2021.104504] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Revised: 05/17/2021] [Accepted: 05/17/2021] [Indexed: 12/23/2022]
Abstract
Deep learning based generative adversarial networks (GAN) can effectively perform image reconstruction with under-sampled MR data. In general, a large number of training samples are required to improve the reconstruction performance of a certain model. However, in real clinical applications, it is difficult to obtain tens of thousands of raw patient data to train the model since saving k-space data is not in the routine clinical flow. Therefore, enhancing the generalizability of a network based on small samples is urgently needed. In this study, three novel applications were explored based on parallel imaging combined with the GAN model (PI-GAN) and transfer learning. The model was pre-trained with public Calgary brain images and then fine-tuned for use in (1) patients with tumors in our center; (2) different anatomies, including knee and liver; (3) different k-space sampling masks with acceleration factors (AFs) of 2 and 6. As for the brain tumor dataset, the transfer learning results could remove the artifacts found in PI-GAN and yield smoother brain edges. The transfer learning results for the knee and liver were superior to those of the PI-GAN model trained with its own dataset using a smaller number of training cases. However, the learning procedure converged more slowly in the knee datasets compared to the learning in the brain tumor datasets. The reconstruction performance was improved by transfer learning both in the models with AFs of 2 and 6. Of these two models, the one with AF = 2 showed better results. The results also showed that transfer learning with the pre-trained model could solve the problem of inconsistency between the training and test datasets and facilitate generalization to unseen data.
Collapse
Affiliation(s)
- Jun Lv
- School of Computer and Control Engineering, Yantai University, Yantai, China
| | - Guangyuan Li
- School of Computer and Control Engineering, Yantai University, Yantai, China
| | - Xiangrong Tong
- School of Computer and Control Engineering, Yantai University, Yantai, China
| | | | - Jiahao Huang
- School of Optics and Photonics, Beijing Institute of Technology, Beijing, China
| | - Chengyan Wang
- Human Phenome Institute, Fudan University, Shanghai, China.
| | - Guang Yang
- Cardiovascular Research Centre, Royal Brompton Hospital, London, SW3 6NP, UK; National Heart and Lung Institute, Imperial College London, London, SW7 2AZ, UK.
| |
Collapse
|