1
|
Wang S, Wu R, Jia S, Diakite A, Li C, Liu Q, Zheng H, Ying L. Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning. Magn Reson Med 2024; 92:496-518. [PMID: 38624162 DOI: 10.1002/mrm.30105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 03/19/2024] [Accepted: 03/20/2024] [Indexed: 04/17/2024]
Abstract
Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.
Collapse
Affiliation(s)
- Shanshan Wang
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Ruoyou Wu
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Sen Jia
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Alou Diakite
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Cheng Li
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Qiegen Liu
- Department of Electronic Information Engineering, Nanchang University, Nanchang, China
| | - Hairong Zheng
- Paul C Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Leslie Ying
- Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, New York, USA
| |
Collapse
|
2
|
Heckel R, Jacob M, Chaudhari A, Perlman O, Shimron E. Deep learning for accelerated and robust MRI reconstruction. MAGMA (NEW YORK, N.Y.) 2024; 37:335-368. [PMID: 39042206 DOI: 10.1007/s10334-024-01173-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Revised: 05/24/2024] [Accepted: 05/28/2024] [Indexed: 07/24/2024]
Abstract
Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.
Collapse
Affiliation(s)
- Reinhard Heckel
- Department of computer engineering, Technical University of Munich, Munich, Germany
| | - Mathews Jacob
- Department of Electrical and Computer Engineering, University of Iowa, Iowa, 52242, IA, USA
| | - Akshay Chaudhari
- Department of Radiology, Stanford University, Stanford, 94305, CA, USA
- Department of Biomedical Data Science, Stanford University, Stanford, 94305, CA, USA
| | - Or Perlman
- Department of Biomedical Engineering, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Efrat Shimron
- Department of Electrical and Computer Engineering, Technion-Israel Institute of Technology, Haifa, 3200004, Israel.
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, 3200004, Israel.
| |
Collapse
|
3
|
Zhang C, Piccini D, Demirel OB, Bonanno G, Roy CW, Yaman B, Moeller S, Shenoy C, Stuber M, Akçakaya M. Large-scale 3D non-Cartesian coronary MRI reconstruction using distributed memory-efficient physics-guided deep learning with limited training data. MAGMA (NEW YORK, N.Y.) 2024; 37:429-438. [PMID: 38743377 DOI: 10.1007/s10334-024-01157-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 02/19/2024] [Accepted: 03/13/2024] [Indexed: 05/16/2024]
Abstract
OBJECT To enable high-quality physics-guided deep learning (PG-DL) reconstruction of large-scale 3D non-Cartesian coronary MRI by overcoming challenges of hardware limitations and limited training data availability. MATERIALS AND METHODS While PG-DL has emerged as a powerful image reconstruction method, its application to large-scale 3D non-Cartesian MRI is hindered by hardware limitations and limited availability of training data. We combine several recent advances in deep learning and MRI reconstruction to tackle the former challenge, and we further propose a 2.5D reconstruction using 2D convolutional neural networks, which treat 3D volumes as batches of 2D images to train the network with a limited amount of training data. Both 3D and 2.5D variants of the PG-DL networks were compared to conventional methods for high-resolution 3D kooshball coronary MRI. RESULTS Proposed PG-DL reconstructions of 3D non-Cartesian coronary MRI with 3D and 2.5D processing outperformed all conventional methods both quantitatively and qualitatively in terms of image assessment by an experienced cardiologist. The 2.5D variant further improved vessel sharpness compared to 3D processing, and scored higher in terms of qualitative image quality. DISCUSSION PG-DL reconstruction of large-scale 3D non-Cartesian MRI without compromising image size or network complexity is achieved, and the proposed 2.5D processing enables high-quality reconstruction with limited training data.
Collapse
Affiliation(s)
- Chi Zhang
- Electrical and Computer Engineering, University of Minnesota, 200 Union Street S.E., Minneapolis, MN, 55455, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
| | - Davide Piccini
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- Advanced Clinical Imaging Technology, Siemens Healthineers International, Lausanne, Switzerland
| | - Omer Burak Demirel
- Electrical and Computer Engineering, University of Minnesota, 200 Union Street S.E., Minneapolis, MN, 55455, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
| | - Gabriele Bonanno
- Advanced Clinical Imaging Technology, Siemens Healthineers International, Lausanne, Switzerland
| | - Christopher W Roy
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Burhaneddin Yaman
- Electrical and Computer Engineering, University of Minnesota, 200 Union Street S.E., Minneapolis, MN, 55455, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
| | - Steen Moeller
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
| | - Chetan Shenoy
- Department of Medicine (Cardiology), University of Minnesota, Minneapolis, MN, 55455, USA
| | - Matthias Stuber
- Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- Center for Biomedical Imaging, Lausanne, Switzerland
| | - Mehmet Akçakaya
- Electrical and Computer Engineering, University of Minnesota, 200 Union Street S.E., Minneapolis, MN, 55455, USA.
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
4
|
Gao C, Ming Z, Nguyen KL, Pang J, Bedayat A, Dale BM, Zhong X, Finn JP. Ferumoxytol-Enhanced Cardiac Cine MRI Reconstruction Using a Variable-Splitting Spatiotemporal Network. J Magn Reson Imaging 2024. [PMID: 38436994 DOI: 10.1002/jmri.29295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 01/29/2024] [Accepted: 01/31/2024] [Indexed: 03/05/2024] Open
Abstract
BACKGROUND Balanced steady-state free precession (bSSFP) imaging is commonly used in cardiac cine MRI but prone to image artifacts. Ferumoxytol-enhanced (FE) gradient echo (GRE) has been proposed as an alternative. Utilizing the abundance of bSSFP images to develop a computationally efficient network that is applicable to FE GRE cine would benefit future network development. PURPOSE To develop a variable-splitting spatiotemporal network (VSNet) for image reconstruction, trained on bSSFP cine images and applicable to FE GRE cine images. STUDY TYPE Retrospective and prospective. SUBJECTS 41 patients (26 female, 53 ± 19 y/o) for network training, 31 patients (19 female, 49 ± 17 y/o) and 5 healthy subjects (5 female, 30 ± 7 y/o) for testing. FIELD STRENGTH/SEQUENCE 1.5T and 3T, bSSFP and GRE. ASSESSMENT VSNet was compared to VSNet with total variation loss, compressed sensing and low rank methods for 14× accelerated data. The GRAPPA×2/×3 images served as the reference. Peak signal-to-noise-ratio (PSNR), structural similarity index (SSIM), left ventricular (LV) and right ventricular (RV) end-diastolic volume (EDV), end-systolic volume (ESV), and ejection fraction (EF) were measured. Qualitative image ranking and scoring were independently performed by three readers. Latent scores were calculated based on scores of each method relative to the reference. STATISTICS Linear mixed-effects regression, Tukey method, Fleiss' Kappa, Bland-Altman analysis, and Bayesian categorical cumulative probit model. A P-value <0.05 was considered statistically significant. RESULTS VSNet achieved significantly higher PSNR (32.7 ± 0.2), SSIM (0.880 ± 0.004), rank (2.14 ± 0.06), and latent scores (-1.72 ± 0.22) compared to other methods (rank >2.90, latent score < -2.63). Fleiss' Kappa was 0.52 for scoring and 0.61 for ranking. VSNet showed no significantly different LV and RV ESV (P = 0.938) and EF (P = 0.143) measurements, but statistically significant different (2.62 mL) EDV measurements compared to the reference. CONCLUSION VSNet produced the highest image quality and the most accurate functional measurements for FE GRE cine images among the tested 14× accelerated reconstruction methods. LEVEL OF EVIDENCE 3 TECHNICAL EFFICACY: Stage 1.
Collapse
Affiliation(s)
- Chang Gao
- Department of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, California, USA
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
| | - Zhengyang Ming
- Department of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, California, USA
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
| | - Kim-Lien Nguyen
- Department of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, California, USA
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
- Division of Cardiology, University of California Los Angeles and VA Greater Los Angeles Healthcare System, Los Angeles, California, USA
| | - Jianing Pang
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Chicago, Illinois, USA
| | - Arash Bedayat
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
| | - Brian M Dale
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Cary, North Carliona, USA
| | - Xiaodong Zhong
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Los Angeles, California, USA
| | - J Paul Finn
- Department of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, California, USA
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
| |
Collapse
|
5
|
Qu B, Zhang J, Kang T, Lin J, Lin M, She H, Wu Q, Wang M, Zheng G. Radial magnetic resonance image reconstruction with a deep unrolled projected fast iterative soft-thresholding network. Comput Biol Med 2024; 168:107707. [PMID: 38000244 DOI: 10.1016/j.compbiomed.2023.107707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Revised: 10/31/2023] [Accepted: 11/15/2023] [Indexed: 11/26/2023]
Abstract
Radially sampling of magnetic resonance imaging (MRI) is an effective way to accelerate the imaging. How to preserve the image details in reconstruction is always challenging. In this work, a deep unrolled neural network is designed to emulate the iterative sparse image reconstruction process of a projected fast soft-threshold algorithm (pFISTA). The proposed method, an unrolled pFISTA network for Deep Radial MRI (pFISTA-DR), include the preprocessing module to refine coil sensitivity maps and initial reconstructed image, the learnable convolution filters to extract image feature maps, and adaptive threshold to robustly remove image artifacts. Experimental results show that, among the compared methods, pFISTA-DR provides the best reconstruction and achieved the highest PSNR, the highest SSIM and the lowest reconstruction errors.
Collapse
Affiliation(s)
- Biao Qu
- Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China
| | - Jialue Zhang
- Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China; Fujian Provincial Key Laboratory of Plasma and Magnetic Resonance, Department of Electronic Science, Xiamen University, China
| | - Taishan Kang
- Department of Radiology, Zhongshan Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China
| | - Jianzhong Lin
- Department of Radiology, Zhongshan Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China
| | - Meijin Lin
- Department of Applied Marine Physics & Engineering, College of Ocean and Earth Sciences, Xiamen University, Xiamen, China
| | - Huajun She
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Qingxia Wu
- Department of Medical Imaging, Henan Provincial People's Hospital, Zhengzhou, China
| | - Meiyun Wang
- Department of Medical Imaging, Henan Provincial People's Hospital, Zhengzhou, China; Laboratory of Brain Science and Brain-Like Intelligence Technology, Institute for Integrated Medical Science and Engineering, Henan Academy of Sciences, Zhengzhou, China
| | - Gaofeng Zheng
- Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China.
| |
Collapse
|
6
|
Wood G, Pedersen AU, Kunze KP, Neji R, Hajhosseiny R, Wetzl J, Yoon SS, Schmidt M, Nørgaard BL, Prieto C, Botnar RM, Kim WY. Automated detection of cardiac rest period for trigger delay calculation for image-based navigator coronary magnetic resonance angiography. J Cardiovasc Magn Reson 2023; 25:52. [PMID: 37779192 PMCID: PMC10544388 DOI: 10.1186/s12968-023-00962-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 09/12/2023] [Indexed: 10/03/2023] Open
Abstract
BACKGROUND Coronary magnetic resonance angiography (coronary MRA) is increasingly being considered as a clinically viable method to investigate coronary artery disease (CAD). Accurate determination of the trigger delay to place the acquisition window within the quiescent part of the cardiac cycle is critical for coronary MRA in order to reduce cardiac motion. This is currently reliant on operator-led decision making, which can negatively affect consistency of scan acquisition. Recently developed deep learning (DL) derived software may overcome these issues by automation of cardiac rest period detection. METHODS Thirty individuals (female, n = 10) were investigated using a 0.9 mm isotropic image-navigator (iNAV)-based motion-corrected coronary MRA sequence. Each individual was scanned three times utilising different strategies for determination of the optimal trigger delay: (1) the DL software, (2) an experienced operator decision, and (3) a previously utilised formula for determining the trigger delay. Methodologies were compared using custom-made analysis software to assess visible coronary vessel length and coronary vessel sharpness for the entire vessel length and the first 4 cm of each vessel. RESULTS There was no difference in image quality between any of the methodologies for determination of the optimal trigger delay, as assessed by visible coronary vessel length, coronary vessel sharpness for each entire vessel and vessel sharpness for the first 4 cm of the left mainstem, left anterior descending or right coronary arteries. However, vessel length of the left circumflex was slightly greater using the formula method. The time taken to calculate the trigger delay was significantly lower for the DL-method as compared to the operator-led approach (106 ± 38.0 s vs 168 ± 39.2 s, p < 0.01, 95% CI of difference 25.5-98.1 s). CONCLUSIONS Deep learning-derived automated software can effectively and efficiently determine the optimal trigger delay for acquisition of coronary MRA and thus may simplify workflow and improve reproducibility.
Collapse
Affiliation(s)
- Gregory Wood
- Department of Cardiology, Aarhus University Hospital, Palle Juul Jensens Boulevard 99, 8200, Aarhus N, Denmark.
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark.
| | - Alexandra Uglebjerg Pedersen
- Department of Cardiology, Aarhus University Hospital, Palle Juul Jensens Boulevard 99, 8200, Aarhus N, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Karl P Kunze
- MR Research Collaborations, Siemens Healthcare Limited, Camberley, UK
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| | - Radhouene Neji
- MR Research Collaborations, Siemens Healthcare Limited, Camberley, UK
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
| | - Reza Hajhosseiny
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Jens Wetzl
- Cardiovascular MR Predevelopment, Siemens Healthcare GmbH, Erlangen, Germany
| | - Seung Su Yoon
- Cardiovascular MR Predevelopment, Siemens Healthcare GmbH, Erlangen, Germany
| | - Michaela Schmidt
- Cardiovascular MR Predevelopment, Siemens Healthcare GmbH, Erlangen, Germany
| | - Bjarne Linde Nørgaard
- Department of Cardiology, Aarhus University Hospital, Palle Juul Jensens Boulevard 99, 8200, Aarhus N, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Claudia Prieto
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
- Escuela de Ingeniería, Pontificia Universidad Católica de Chile, Santiago, Chile
- Millenium Institute for Intelligent Healthcare Engineering, Santiago, Chile
| | - René M Botnar
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK
- Escuela de Ingeniería, Pontificia Universidad Católica de Chile, Santiago, Chile
- Instituto de Ingeniería Biológica y Médica, Pontificia Universidad Católica de Chile, Santiago, Chile
- Institute for Advanced Study, Technical University of Munich, Garching, Germany
- Millenium Institute for Intelligent Healthcare Engineering, Santiago, Chile
| | - Won Yong Kim
- Department of Cardiology, Aarhus University Hospital, Palle Juul Jensens Boulevard 99, 8200, Aarhus N, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| |
Collapse
|
7
|
Chen Y, Guo H, Dong P, Li Y, Zhang Z, Mao N, Chu T, Sun Z, Wang F, Feng Z, Wang H, Ma H. Feasibility of 3.0 T balanced fast field echo non-contrast-enhanced whole-heart coronary magnetic resonance angiography. Cardiovasc Diagn Ther 2023; 13:51-60. [PMID: 36864952 PMCID: PMC9971310 DOI: 10.21037/cdt-22-487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 12/02/2022] [Indexed: 01/01/2023]
Abstract
Background Coronary artery disease (CAD) is one of the most common diseases seriously harmful to human health caused by atherosclerosis. Besides coronary computed tomography angiography (CCTA) and invasive coronary angiography (ICA), coronary magnetic resonance angiography (CMRA) has become an alternative examination. The purpose of this study was to prospectively evaluate the feasibility of 3.0 T free-breathing whole-heart non-contrast-enhanced coronary magnetic resonance angiography (NCE-CMRA). Methods After Institutional Review Board approval, the NCE-CMRA data sets of 29 patients acquired successfully at 3.0 T were evaluated independently by two blinded readers for visualization and image quality of coronary arteries using the subjective quality grade. The acquisition times were recorded in the meantime. A part of the patients had undergone CCTA, we represented stenosis by scores and used the Kappa to evaluate the consistency between CCTA and NCE-CMRA. Results Six patients did not get diagnostic image quality because of severe artifacts. The image quality score assessed by both radiologists is 3.2±0.7, which means the NCE-CMRA can show the coronary arteries excellently. The main vessels of the coronary artery on NCE-CMRA images are considered reliably assessable. The acquisition time of NCE-CMRA, is 8.8±1.2 min. The Kappa of CCTA and NCE-CMRA on detecting stenosis is 0.842 (P<0.001). Conclusions The NCE-CMRA results in reliable image quality and visualization parameters of coronary arteries within a short scan time. The NCE-CMRA and CCTA have a good agreement for detecting stenosis.
Collapse
Affiliation(s)
- Yang Chen
- Department of Medical Imaging, Weifang Medical University, Weifang, China
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Hao Guo
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Peng Dong
- Department of Medical Imaging, Weifang Medical University, Weifang, China
| | - Yue Li
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Zhongsheng Zhang
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Ning Mao
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Tongpeng Chu
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Zehua Sun
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Fang Wang
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Zhiqiang Feng
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Huaying Wang
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| | - Heng Ma
- Department of Radiology, Qingdao University and Yantai Yuhuangding Hospital, Yantai, China
| |
Collapse
|
8
|
Chen Z, Pawar K, Ekanayake M, Pain C, Zhong S, Egan GF. Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges. J Digit Imaging 2023; 36:204-230. [PMID: 36323914 PMCID: PMC9984670 DOI: 10.1007/s10278-022-00721-9] [Citation(s) in RCA: 23] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 09/09/2022] [Accepted: 10/17/2022] [Indexed: 11/06/2022] Open
Abstract
Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.
Collapse
Affiliation(s)
- Zhaolin Chen
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia.
- Department of Data Science and AI, Monash University, Melbourne, VIC, Australia.
| | - Kamlesh Pawar
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
| | - Mevan Ekanayake
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC, Australia
| | - Cameron Pain
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC, Australia
| | - Shenjun Zhong
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- National Imaging Facility, Brisbane, QLD, Australia
| | - Gary F Egan
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
9
|
Gao C, Ghodrati V, Shih SF, Wu HH, Liu Y, Nickel MD, Vahle T, Dale B, Sai V, Felker E, Surawech C, Miao Q, Finn JP, Zhong X, Hu P. Undersampling artifact reduction for free-breathing 3D stack-of-radial MRI based on a deep adversarial learning network. Magn Reson Imaging 2023; 95:70-79. [PMID: 36270417 PMCID: PMC10163826 DOI: 10.1016/j.mri.2022.10.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 10/06/2022] [Accepted: 10/14/2022] [Indexed: 11/05/2022]
Abstract
PURPOSE Stack-of-radial MRI allows free-breathing abdominal scans, however, it requires relatively long acquisition time. Undersampling reduces scan time but can cause streaking artifacts and degrade image quality. This study developed deep learning networks with adversarial loss and evaluated the performance of reducing streaking artifacts and preserving perceptual image sharpness. METHODS A 3D generative adversarial network (GAN) was developed for reducing streaking artifacts in stack-of-radial abdominal scans. Training and validation datasets were self-gated to 5 respiratory states to reduce motion artifacts and to effectively augment the data. The network used a combination of three loss functions to constrain the anatomy and preserve image quality: adversarial loss, mean-squared-error loss and structural similarity index loss. The performance of the network was investigated for 3-5 times undersampled data from 2 institutions. The performance of the GAN for 5 times accelerated images was compared with a 3D U-Net and evaluated using quantitative NMSE, SSIM and region of interest (ROI) measurements as well as qualitative scores of radiologists. RESULTS The 3D GAN showed similar NMSE (0.0657 vs. 0.0559, p = 0.5217) and significantly higher SSIM (0.841 vs. 0.798, p < 0.0001) compared to U-Net. ROI analysis showed GAN removed streaks in both the background air and the tissue and was not significantly different from the reference mean and variations. Radiologists' scores showed GAN had a significant improvement of 1.6 point (p = 0.004) on a 4-point scale in streaking score while no significant difference in sharpness score compared to the input. CONCLUSION 3D GAN removes streaking artifacts and preserves perceptual image details.
Collapse
Affiliation(s)
- Chang Gao
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | - Vahid Ghodrati
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | - Shu-Fu Shih
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Department of Bioengineering, University of California Los Angeles, Los Angeles, CA, United States
| | - Holden H Wu
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States; Department of Bioengineering, University of California Los Angeles, Los Angeles, CA, United States
| | - Yongkai Liu
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | | | - Thomas Vahle
- MR Application Predevelopment, Siemens Healthcare GmbH, Erlangen, Germany
| | - Brian Dale
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Cary, NC, United States
| | - Victor Sai
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States
| | - Ely Felker
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States
| | - Chuthaporn Surawech
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Department of Radiology, Division of Diagnostic Radiology, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital, Bangkok, Thailand
| | - Qi Miao
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Department of Radiology, The First Affiliated Hospital of China Medical University, Shenyang, Liaoning Province, China
| | - J Paul Finn
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | - Xiaodong Zhong
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Los Angeles, CA, United States
| | - Peng Hu
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States.
| |
Collapse
|
10
|
Ramzi Z, G R C, Starck JL, Ciuciu P. NC-PDNet: A Density-Compensated Unrolled Network for 2D and 3D Non-Cartesian MRI Reconstruction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1625-1638. [PMID: 35041598 DOI: 10.1109/tmi.2022.3144619] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Deep Learning has become a very promising avenue for magnetic resonance image (MRI) reconstruction. In this work, we explore the potential of unrolled networks for non-Cartesian acquisition settings. We design the NC-PDNet (Non-Cartesian Primal Dual Netwok), the first density-compensated (DCp) unrolled neural network, and validate the need for its key components via an ablation study. Moreover, we conduct some generalizability experiments to test this network in out-of-distribution settings, for example training on knee data and validating on brain data. The results show that NC-PDNet outperforms baseline (U-Net, Deep image prior) models both visually and quantitatively in all settings. In particular, in the 2D multi-coil acquisition scenario, the NC-PDNet provides up to a 1.2 dB improvement in peak signal-to-noise ratio (PSNR) over baseline networks, while also allowing a gain of at least 1dB in PSNR in generalization settings. We provide the open-source implementation of NC-PDNet, and in particular the Non-uniform Fourier Transform in TensorFlow, tested on 2D multi-coil and 3D single-coil k-space data.
Collapse
|
11
|
Kleineisel J, Heidenreich JF, Eirich P, Petri N, Köstler H, Petritsch B, Bley TA, Wech T. Real-time cardiac MRI using an undersampled spiral k-space trajectory and a reconstruction based on a variational network. Magn Reson Med 2022; 88:2167-2178. [PMID: 35692042 DOI: 10.1002/mrm.29357] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 05/17/2022] [Accepted: 05/20/2022] [Indexed: 11/06/2022]
Abstract
PURPOSE Cardiac MRI represents the gold standard to determine myocardial function. However, the current clinical standard protocol, a segmented Cartesian acquisition, is time-consuming and can lead to compromised image quality in the case of arrhythmia or dyspnea. In this article, a machine learning-based reconstruction of undersampled spiral k-space data is presented to enable free breathing real-time cardiac MRI with good image quality and short reconstruction times. METHODS Data were acquired in free breathing with a 2D spiral trajectory corrected by the gradient system transfer function. Undersampled data were reconstructed by a variational network (VN), which was specifically adapted to the non-Cartesian sampling pattern. The network was trained with data from 11 subjects. Subsequently, the imaging technique was validated in 14 subjects by quantifying the difference to a segmented reference acquisition, an expert reader study, and by comparing derived volumes and functional parameters with values obtained using the current clinical gold standard. RESULTS The scan time for the entire heart was below 1 min. The VN reconstructed data in about 0.9 s per image, which is considerably shorter than conventional model-based approaches. The VN furthermore performed better than a U-Net and not inferior to a low-rank plus sparse model in terms of achieved image quality. Functional parameters agreed, on average, with reference data. CONCLUSIONS The proposed VN method enables real-time cardiac imaging with both high spatial and temporal resolution in free breathing and with short reconstruction time.
Collapse
Affiliation(s)
- Jonas Kleineisel
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Julius F Heidenreich
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Philipp Eirich
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany.,Comprehensive Heart Failure Center, University Hospital Würzburg, Würzburg, Germany
| | - Nils Petri
- Department of Internal Medicine I, University Hospital Würzburg, Würzburg, Germany
| | - Herbert Köstler
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Bernhard Petritsch
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Thorsten A Bley
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Tobias Wech
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| |
Collapse
|
12
|
Androulakis E, Mohiaddin R, Bratis K. Magnetic resonance coronary angiography in the era of multimodality imaging. Clin Radiol 2022; 77:e489-e499. [DOI: 10.1016/j.crad.2022.03.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2021] [Accepted: 03/09/2022] [Indexed: 11/28/2022]
|
13
|
Yurt M, Özbey M, UH Dar S, Tinaz B, Oguz KK, Çukur T. Progressively Volumetrized Deep Generative Models for Data-Efficient Contextual Learning of MR Image Recovery. Med Image Anal 2022; 78:102429. [DOI: 10.1016/j.media.2022.102429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Revised: 03/14/2022] [Accepted: 03/18/2022] [Indexed: 10/18/2022]
|
14
|
Chen Z, Chen Y, Xie Y, Li D, Christodoulou AG. Data-Consistent non-Cartesian deep subspace learning for efficient dynamic MR image reconstruction. PROCEEDINGS. IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING 2022; 2022:10.1109/isbi52829.2022.9761497. [PMID: 35572068 PMCID: PMC9104888 DOI: 10.1109/isbi52829.2022.9761497] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
Non-Cartesian sampling with subspace-constrained image reconstruction is a popular approach to dynamic MRI, but slow iterative reconstruction limits its clinical application. Data-consistent (DC) deep learning can accelerate reconstruction with good image quality, but has not been formulated for non-Cartesian subspace imaging. In this study, we propose a DC non-Cartesian deep subspace learning framework for fast, accurate dynamic MR image reconstruction. Four novel DC formulations are developed and evaluated: two gradient decent approaches, a directly solved approach, and a conjugate gradient approach. We applied a U-Net model with and without DC layers to reconstruct T1-weighted images for cardiac MR Multitasking (an advanced multidimensional imaging method), comparing our results to the iteratively reconstructed reference. Experimental results show that the proposed framework significantly improves reconstruction accuracy over the U-Net model without DC, while significantly accelerating the reconstruction over conventional iterative reconstruction.
Collapse
Affiliation(s)
- Zihao Chen
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, USA
- Department of Bioengineering, UCLA, Los Angeles, USA
| | - Yuhua Chen
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, USA
- Department of Bioengineering, UCLA, Los Angeles, USA
| | - Yibin Xie
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, USA
| | - Debiao Li
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, USA
- Department of Bioengineering, UCLA, Los Angeles, USA
| | - Anthony G Christodoulou
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, USA
- Department of Bioengineering, UCLA, Los Angeles, USA
| |
Collapse
|
15
|
Image Features of Magnetic Resonance Angiography under Deep Learning in Exploring the Effect of Comprehensive Rehabilitation Nursing on the Neurological Function Recovery of Patients with Acute Stroke. CONTRAST MEDIA & MOLECULAR IMAGING 2021; 2021:1197728. [PMID: 34602911 PMCID: PMC8449730 DOI: 10.1155/2021/1197728] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/25/2021] [Revised: 08/28/2021] [Accepted: 08/31/2021] [Indexed: 11/18/2022]
Abstract
This study was to explore the effects of imaging characteristics of magnetic resonance angiography (MRA) based on deep learning on the comprehensive rehabilitation nursing on the neurological recovery of patients with acute stroke. In this study, 84 patients with acute stroke who were treated in hospital were selected as the research objects, and they were rolled into a control group (routine care) and an experimental group (comprehensive rehabilitation care). The dense dilated block-convolution neural network (DD-CNN) algorithm under deep learning for cerebrovascular was adopted to assess the effect of comprehensive rehabilitation care on the neurological recovery of patients with acute stroke. The results showed that the Berg scale scores, Fugl-Meyer scores, and Functional Independence Measure (FIM) scores of the experimental group of patients after 6 weeks and 12 weeks of comprehensive rehabilitation nursing were greatly different from those before treatment, showing statistical differences (P < 0.05). Compared with conventional magnetic resonance imaging (MRI) images, MRA images based on CNN algorithm, Dense Net algorithm, and DD-CNN algorithm can more clearly show the patient's cerebral artery occlusion. The average dice similarity coefficient (DSC) values of CNN algorithm, Dense Net algorithm, and DD-CNN algorithm were determined to be 84.3%, 95.7%, and 97.8%, respectively; the average sensitivity (Sen) values of the three algorithms were 76.1%, 95.4%, and 96.8%, respectively; and the average accuracy (Acc) values were 87.9%, 96.3%, and 97.9%, respectively. Thus, there were statistically obvious differences among the three algorithms in terms of average values of DSC, Sen, and Acc (P < 0.05). The MRA images processed by the DD-CNN algorithm showed that the degree of neurological recovery of the experimental group was observably greater than that of the control group, and the difference was statistically obvious (P < 0.05). In short, the image features of MRA based on the deep learning DD-CNN algorithm showed good application value in studying the effect of comprehensive rehabilitation nursing on the neurological recovery of patients with acute stroke, and it was worthy of promotion.
Collapse
|
16
|
Hajhosseiny R, Munoz C, Cruz G, Khamis R, Kim WY, Prieto C, Botnar RM. Coronary Magnetic Resonance Angiography in Chronic Coronary Syndromes. Front Cardiovasc Med 2021; 8:682924. [PMID: 34485397 PMCID: PMC8416045 DOI: 10.3389/fcvm.2021.682924] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Accepted: 07/23/2021] [Indexed: 01/14/2023] Open
Abstract
Cardiovascular disease is the leading cause of mortality worldwide, with atherosclerotic coronary artery disease (CAD) accounting for the majority of cases. X-ray coronary angiography and computed tomography coronary angiography (CCTA) are the imaging modalities of choice for the assessment of CAD. However, the use of ionising radiation and iodinated contrast agents remain drawbacks. There is therefore a clinical need for an alternative modality for the early identification and longitudinal monitoring of CAD without these associated drawbacks. Coronary magnetic resonance angiography (CMRA) could be a potential alternative for the detection and monitoring of coronary arterial stenosis, without exposing patients to ionising radiation or iodinated contrast agents. Further advantages include its versatility, excellent soft tissue characterisation and suitability for repeat imaging. Despite the early promise of CMRA, widespread clinical utilisation remains limited due to long and unpredictable scan times, onerous scan planning, lower spatial resolution, as well as motion related image quality degradation. The past decade has brought about a resurgence in CMRA technology, with significant leaps in image acceleration, respiratory and cardiac motion estimation and advanced motion corrected or motion-resolved image reconstruction. With the advent of artificial intelligence, great advances are also seen in deep learning-based motion estimation, undersampled and super-resolution reconstruction promising further improvements of CMRA. This has enabled high spatial resolution (1 mm isotropic), 3D whole heart CMRA in a clinically feasible and reliable acquisition time of under 10 min. Furthermore, latest super-resolution image reconstruction approaches which are currently under evaluation promise acquisitions as short as 1 min. In this review, we will explore the recent technological advances that are designed to bring CMRA closer to clinical reality.
Collapse
Affiliation(s)
- Reza Hajhosseiny
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
- National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Camila Munoz
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
| | - Gastao Cruz
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
| | - Ramzi Khamis
- National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Won Yong Kim
- Department of Cardiology and Institute of Clinical Medicine, Aarhus University Hospital, Skejby, Denmark
| | - Claudia Prieto
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
- Escuela de Ingeniería, Pontificia Universidad Catolica de Chile, Santiago, Chile
| | - René M. Botnar
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
- Escuela de Ingeniería, Pontificia Universidad Catolica de Chile, Santiago, Chile
- Instituto de Ingeniería Biologica y Medica, Pontificia Universidad Catolica de Chile, Santiago, Chile
| |
Collapse
|
17
|
Chandra SS, Bran Lorenzana M, Liu X, Liu S, Bollmann S, Crozier S. Deep learning in magnetic resonance image reconstruction. J Med Imaging Radiat Oncol 2021; 65:564-577. [PMID: 34254448 DOI: 10.1111/1754-9485.13276] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Accepted: 06/10/2021] [Indexed: 11/26/2022]
Abstract
Magnetic resonance (MR) imaging visualises soft tissue contrast in exquisite detail without harmful ionising radiation. In this work, we provide a state-of-the-art review on the use of deep learning in MR image reconstruction from different image acquisition types involving compressed sensing techniques, parallel image acquisition and multi-contrast imaging. Publications with deep learning-based image reconstruction for MR imaging were identified from the literature (PubMed and Google Scholar), and a comprehensive description of each of the works was provided. A detailed comparison that highlights the differences, the data used and the performance of each of these works were also made. A discussion of the potential use cases for each of these methods is provided. The sparse image reconstruction methods were found to be most popular in using deep learning for improved performance, accelerating acquisitions by around 4-8 times. Multi-contrast image reconstruction methods rely on at least one pre-acquired image, but can achieve 16-fold, and even up to 32- to 50-fold acceleration depending on the set-up. Parallel imaging provides frameworks to be integrated in many of these methods for additional speed-up potential. The successful use of compressed sensing techniques and multi-contrast imaging with deep learning and parallel acquisition methods could yield significant MR acquisition speed-ups within clinical routines in the near future.
Collapse
Affiliation(s)
- Shekhar S Chandra
- School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Queensland, Australia
| | - Marlon Bran Lorenzana
- School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Queensland, Australia
| | - Xinwen Liu
- School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Queensland, Australia
| | - Siyu Liu
- School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Queensland, Australia
| | - Steffen Bollmann
- School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Queensland, Australia
| | - Stuart Crozier
- School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Queensland, Australia
| |
Collapse
|
18
|
Liu F, Kijowski R, El Fakhri G, Feng L. Magnetic resonance parameter mapping using model-guided self-supervised deep learning. Magn Reson Med 2021; 85:3211-3226. [PMID: 33464652 PMCID: PMC9185837 DOI: 10.1002/mrm.28659] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Revised: 11/15/2020] [Accepted: 12/07/2020] [Indexed: 12/25/2022]
Abstract
PURPOSE To develop a model-guided self-supervised deep learning MRI reconstruction framework called reference-free latent map extraction (RELAX) for rapid quantitative MR parameter mapping. METHODS Two physical models are incorporated for network training in RELAX, including the inherent MR imaging model and a quantitative model that is used to fit parameters in quantitative MRI. By enforcing these physical model constraints, RELAX eliminates the need for full sampled reference data sets that are required in standard supervised learning. Meanwhile, RELAX also enables direct reconstruction of corresponding MR parameter maps from undersampled k-space. Generic sparsity constraints used in conventional iterative reconstruction, such as the total variation constraint, can be additionally included in the RELAX framework to improve reconstruction quality. The performance of RELAX was tested for accelerated T1 and T2 mapping in both simulated and actually acquired MRI data sets and was compared with supervised learning and conventional constrained reconstruction for suppressing noise and/or undersampling-induced artifacts. RESULTS In the simulated data sets, RELAX generated good T1 /T2 maps in the presence of noise and/or undersampling artifacts, comparable to artifact/noise-free ground truth. The inclusion of a spatial total variation constraint helps improve image quality. For the in vivo T1 /T2 mapping data sets, RELAX achieved superior reconstruction quality compared with conventional iterative reconstruction, and similar reconstruction performance to supervised deep learning reconstruction. CONCLUSION This work has demonstrated the initial feasibility of rapid quantitative MR parameter mapping based on self-supervised deep learning. The RELAX framework may also be further extended to other quantitative MRI applications by incorporating corresponding quantitative imaging models.
Collapse
Affiliation(s)
- Fang Liu
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Richard Kijowski
- Department of Radiology, University of Wisconsin-Madison, Madison, Wisconsin, USA
| | - Georges El Fakhri
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Li Feng
- Biomedical Engineering and Imaging Institute and Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, USA
| |
Collapse
|
19
|
Kofler A, Haltmeier M, Schaeffter T, Kolbitsch C. An end-to-end-trainable iterative network architecture for accelerated radial multi-coil 2D cine MR image reconstruction. Med Phys 2021; 48:2412-2425. [PMID: 33651398 DOI: 10.1002/mp.14809] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2020] [Revised: 02/11/2021] [Accepted: 02/18/2021] [Indexed: 11/06/2022] Open
Abstract
PURPOSE Iterative convolutional neural networks (CNNs) which resemble unrolled learned iterative schemes have shown to consistently deliver state-of-the-art results for image reconstruction problems across different imaging modalities. However, because these methods include the forward model in the architecture, their applicability is often restricted to either relatively small reconstruction problems or to problems with operators which are computationally cheap to compute. As a consequence, they have not been applied to dynamic non-Cartesian multi-coil reconstruction problems so far. METHODS In this work, we propose a CNN architecture for image reconstruction of accelerated 2D radial cine MRI with multiple receiver coils. The network is based on a computationally light CNN component and a subsequent conjugate gradient (CG) method which can be jointly trained end-to-end using an efficient training strategy. We investigate the proposed training strategy and compare our method with other well-known reconstruction techniques with learned and non-learned regularization methods. RESULTS Our proposed method outperforms all other methods based on non-learned regularization. Further, it performs similar or better than a CNN-based method employing a 3D U-Net and a method using adaptive dictionary learning. In addition, we empirically demonstrate that even by training the network with only iteration, it is possible to increase the length of the network at test time and further improve the results. CONCLUSIONS End-to-end training allows to highly reduce the number of trainable parameters of and stabilize the reconstruction network. Further, because it is possible to change the length of the network at the test time, the need to find a compromise between the complexity of the CNN-block and the number of iterations in each CG-block becomes irrelevant.
Collapse
Affiliation(s)
- Andreas Kofler
- Physikalisch-Technische Bundesanstalt (PTB), Braunschweig, Berlin, 10587, Germany
| | - Markus Haltmeier
- Department of Mathematics, University of Innsbruck, Innsbruck, 6020, Austria
| | - Tobias Schaeffter
- Physikalisch-Technische Bundesanstalt (PTB), Braunschweig, Berlin, 10587, Germany.,School of Imaging Sciences and Biomedical Engineering, King's College London, London, SE1 7EH, UK.,Department of Biomedical Engineering, Technical University of Berlin, Berlin, 10623, Germany
| | - Christoph Kolbitsch
- Physikalisch-Technische Bundesanstalt (PTB), Braunschweig, Berlin, 10587, Germany.,School of Imaging Sciences and Biomedical Engineering, King's College London, London, SE1 7EH, UK
| |
Collapse
|
20
|
Zhang Y, She H, Du YP. Dynamic MRI of the abdomen using parallel non-Cartesian convolutional recurrent neural networks. Magn Reson Med 2021; 86:964-973. [PMID: 33749023 DOI: 10.1002/mrm.28774] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2020] [Revised: 02/25/2021] [Accepted: 02/25/2021] [Indexed: 11/10/2022]
Abstract
PURPOSE To improve the image quality and reduce computational time for the reconstruction of undersampled non-Cartesian abdominal dynamic parallel MR data using the deep learning approach. METHODS An algorithm of parallel non-Cartesian convolutional recurrent neural networks (PNCRNNs) was developed to enable the use of the redundant information in both spatial and temporal domains, and achieve data fidelity for the reconstruction of non-Cartesian parallel MR data. The performance of PNCRNNs was evaluated for various acceleration rates, motion patterns, and imaging applications in comparison with that of the state-of-the-art algorithms of dynamic imaging, including extra-dimensional golden-angle radial sparse parallel MRI (XD-GRASP), low-rank plus sparse matrix decomposition (L+S), blind compressive sensing (BCS), and 3D convolutional neural networks (3D CNNs). RESULTS PNCRNNs increased the peak SNR of 9.07 dB compared with XD-GRASP, 9.26 dB compared with L+S, 3.48 dB compared with BCS, and 3.14 dB compared with 3D CNN at R = 16. The reconstruction time was 18 ms for each bin, which was two orders faster than that of XD-GRASP, L+S, and BCS. PNCRNNs provided good reconstruction for various motion patterns, k-space trajectories, and imaging applications. CONCLUSION The proposed PNCRNN provides substantial improvement of the image quality for dynamic golden-angle radial imaging of the abdomen in comparison with XD-GRASP, L+S, BCS, and 3D CNN. The reconstruction time of PNCRNN can be as fast as 50 bins per second, due to the use of the highly computational efficient Toeplitz approach.
Collapse
Affiliation(s)
- Yufei Zhang
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Huajun She
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Yiping P Du
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
21
|
Liu F, Kijowski R, Feng L, El Fakhri G. High-performance rapid MR parameter mapping using model-based deep adversarial learning. Magn Reson Imaging 2020; 74:152-160. [PMID: 32980503 PMCID: PMC7669737 DOI: 10.1016/j.mri.2020.09.021] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2020] [Revised: 08/27/2020] [Accepted: 09/21/2020] [Indexed: 02/01/2023]
Abstract
PURPOSE To develop and evaluate a deep adversarial learning-based image reconstruction approach for rapid and efficient MR parameter mapping. METHODS The proposed method provides an image reconstruction framework by combining the end-to-end convolutional neural network (CNN) mapping, adversarial learning, and MR physical models. The CNN performs direct image-to-parameter mapping by transforming a series of undersampled images directly into MR parameter maps. Adversarial learning is used to improve image sharpness and enable better texture restoration during the image-to-parameter conversion. An additional pathway concerning the MR signal model is added between the estimated parameter maps and undersampled k-space data to ensure the data consistency during network training. The proposed framework was evaluated on T2 mapping of the brain and the knee at an acceleration rate R = 8 and was compared with other state-of-the-art reconstruction methods. Global and regional quantitative assessments were performed to demonstrate the reconstruction performance of the proposed method. RESULTS The proposed adversarial learning approach achieved accurate T2 mapping up to R = 8 in brain and knee joint image datasets. Compared to conventional reconstruction approaches that exploit image sparsity and low-rankness, the proposed method yielded lower errors and higher similarity to the reference and better image sharpness in the T2 estimation. The quantitative metrics were normalized root mean square error of 3.6% for brain and 7.3% for knee, structural similarity index of 85.1% for brain and 83.2% for knee, and tenengrad measures of 9.2% for brain and 10.1% for the knee. The adversarial approach also achieved better performance for maintaining greater image texture and sharpness in comparison to the CNN approach without adversarial learning. CONCLUSION The proposed framework by incorporating the efficient end-to-end CNN mapping, adversarial learning, and physical model enforced data consistency is a promising approach for rapid and efficient reconstruction of quantitative MR parameters.
Collapse
Affiliation(s)
- Fang Liu
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
| | - Richard Kijowski
- Department of Radiology, University of Wisconsin-Madison, Madison, WI, USA
| | - Li Feng
- Biomedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, USA
| | - Georges El Fakhri
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|