1
|
Sui Z, Palaniappan P, Brenner J, Paganelli C, Kurz C, Landry G, Riboldi M. Intra-frame motion deterioration effects and deep-learning-based compensation in MR-guided radiotherapy. Med Phys 2024; 51:1899-1917. [PMID: 37665948 DOI: 10.1002/mp.16702] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 07/07/2023] [Accepted: 07/31/2023] [Indexed: 09/06/2023] Open
Abstract
BACKGROUND Current commercially available hybrid magnetic resonance linear accelerators (MR-Linac) use 2D+t cine MR imaging to provide intra-fractional motion monitoring. However, given the limited temporal resolution of cine MR imaging, target intra-frame motion deterioration effects, resulting in effective time latency and motion artifacts in the image domain, can be appreciable, especially in the case of fast breathing. PURPOSE The aim of this work is to investigate intra-frame motion deterioration effects in MR-guided radiotherapy (MRgRT) by simulating the motion-corrupted image acquisition, and to explore the feasibility of deep-learning-based compensation approaches, relying on the intra-frame motion information which is spatially and temporally encoded in the raw data (k-space). METHODS An intra-frame motion model was defined to simulate motion-corrupted MR images, with 4D anthropomorphic digital phantoms being exploited to provide ground truth 2D+t cine MR sequences. A total number of 10 digital phantoms were generated for lung cancer patients, with randomly selected eight patients for training or validation and the remaining two for testing. The simulation code served as the data generator, and a dedicated motion pattern perturbation scheme was proposed to build the intra-frame motion database, where three degrees of freedom were designed to guarantee the diversity of intra-frame motion trajectories, enabling a thorough exploration in the domain of the potential anatomical structure positions. U-Nets with three types of loss functions: L1 or L2 loss defined in image or Fourier domain, referred to as NNImgLoss-L1 , NNFloss-L1 and NNL2-Loss were trained to extract information from the motion-corrupted image and used to estimate the ground truth final-position image, corresponding to the end of the acquisition. Images before and after compensation were evaluated in terms of (i) image mean-squared error (MSE) and mean absolute error (MAE), and (ii) accuracy of gross tumor volume (GTV) contouring, based on optical-flow image registration. RESULTS Image degradation caused by intra-frame motion was observed: for a linearly and fully acquired Cartesian readout k-space trajectory, intra-frame motion resulted in an imaging latency of approximately 50% of the acquisition time; in comparison, the motion artifacts exhibited only a negligible contribution to the overall geometric errors. All three compensation models led to a decrease in image MSE/MAE and GTV position offset compared to the motion-corrupted image. In the investigated testing dataset for GTV contouring, the average dice similarity coefficients (DSC) improved from 88% to 96%, and the 95th percentile Hausdorff distance (HD95 ) dropped from 4.8 mm to 2.1 mm. Different models showed slight performance variations across different intra-frame motion amplitude categories: NNImgLoss-L1 excelled for small/medium amplitudes, whereas NNFloss-L1 demonstrated higher DSC median values at larger amplitudes. The saliency maps of the motion-corrupted image highlighted the major contribution of the later acquired k-space data, as well as the edges of the moving anatomical structures at their final positions, during the model inference stage. CONCLUSIONS Our results demonstrate the deep-learning-based approaches have the potential to compensate for intra-frame motion by utilizing the later acquired data to drive the convergence of the earlier acquired k-space components.
Collapse
Affiliation(s)
- Zhuojie Sui
- Department of Medical Physics, Faculty of Physics, Ludwig-Maximilians-Universität München, Garching, Germany
| | - Prasannakumar Palaniappan
- Department of Medical Physics, Faculty of Physics, Ludwig-Maximilians-Universität München, Garching, Germany
| | - Jakob Brenner
- Department of Medical Physics, Faculty of Physics, Ludwig-Maximilians-Universität München, Garching, Germany
| | - Chiara Paganelli
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy
| | - Christopher Kurz
- Department of Radiation Oncology, University Hospital, LMU Munich, Munich, Germany
| | - Guillaume Landry
- Department of Radiation Oncology, University Hospital, LMU Munich, Munich, Germany
| | - Marco Riboldi
- Department of Medical Physics, Faculty of Physics, Ludwig-Maximilians-Universität München, Garching, Germany
| |
Collapse
|
2
|
Pan F, Fan Q, Xie H, Bai C, Zhang Z, Chen H, Yang L, Zhou X, Bao Q, Liu C. Correction of Arterial-Phase Motion Artifacts in Gadoxetic Acid-Enhanced Liver MRI Using an Innovative Unsupervised Network. Bioengineering (Basel) 2023; 10:1192. [PMID: 37892922 PMCID: PMC10604307 DOI: 10.3390/bioengineering10101192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 09/30/2023] [Accepted: 10/10/2023] [Indexed: 10/29/2023] Open
Abstract
This study aims to propose and evaluate DR-CycleGAN, a disentangled unsupervised network by introducing a novel content-consistency loss, for removing arterial-phase motion artifacts in gadoxetic acid-enhanced liver MRI examinations. From June 2020 to July 2021, gadoxetic acid-enhanced liver MRI data were retrospectively collected in this center to establish training and testing datasets. Motion artifacts were semi-quantitatively assessed using a five-point Likert scale (1 = no artifact, 2 = mild, 3 = moderate, 4 = severe, and 5 = non-diagnostic) and quantitatively evaluated using the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR). The datasets comprised a training dataset (308 examinations, including 58 examinations with artifact grade = 1 and 250 examinations with artifact grade ≥ 2), a paired test dataset (320 examinations, including 160 examinations with artifact grade = 1 and paired 160 examinations with simulated motion artifacts of grade ≥ 2), and an unpaired test dataset (474 examinations with artifact grade ranging from 1 to 5). The performance of DR-CycleGAN was evaluated and compared with a state-of-the-art network, Cycle-MedGAN V2.0. As a result, in the paired test dataset, DR-CycleGAN demonstrated significantly higher SSIM and PSNR values and lower motion artifact grades compared to Cycle-MedGAN V2.0 (0.89 ± 0.07 vs. 0.84 ± 0.09, 32.88 ± 2.11 vs. 30.81 ± 2.64, and 2.7 ± 0.7 vs. 3.0 ± 0.9, respectively; p < 0.001 each). In the unpaired test dataset, DR-CycleGAN also exhibited a superior motion artifact correction performance, resulting in a significant decrease in motion artifact grades from 2.9 ± 1.3 to 2.0 ± 0.6 compared to Cycle-MedGAN V2.0 (to 2.4 ± 0.9, p < 0.001). In conclusion, DR-CycleGAN effectively reduces motion artifacts in the arterial phase images of gadoxetic acid-enhanced liver MRI examinations, offering the potential to enhance image quality.
Collapse
Affiliation(s)
- Feng Pan
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China; (F.P.); (Q.F.); (H.C.); (L.Y.)
| | - Qianqian Fan
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China; (F.P.); (Q.F.); (H.C.); (L.Y.)
| | - Han Xie
- State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China; (H.X.); (Z.Z.); (X.Z.)
| | - Chongxin Bai
- School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China;
| | - Zhi Zhang
- State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China; (H.X.); (Z.Z.); (X.Z.)
| | - Hebing Chen
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China; (F.P.); (Q.F.); (H.C.); (L.Y.)
| | - Lian Yang
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China; (F.P.); (Q.F.); (H.C.); (L.Y.)
| | - Xin Zhou
- State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China; (H.X.); (Z.Z.); (X.Z.)
- University of Chinese Academy of Sciences, Beijing 100864, China
- Optics Valley Laboratory, Wuhan 430074, China
| | - Qingjia Bao
- State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China; (H.X.); (Z.Z.); (X.Z.)
| | - Chaoyang Liu
- State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China; (H.X.); (Z.Z.); (X.Z.)
- University of Chinese Academy of Sciences, Beijing 100864, China
- Optics Valley Laboratory, Wuhan 430074, China
| |
Collapse
|
3
|
Curtis AD, Mertens AJ, Cheng HLM. A predictive signal model for dynamic cardiac magnetic resonance imaging. Sci Rep 2023; 13:10296. [PMID: 37357251 DOI: 10.1038/s41598-023-37475-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Accepted: 06/22/2023] [Indexed: 06/27/2023] Open
Abstract
Robust dynamic cardiac magnetic resonance imaging (MRI) has been a long-standing endeavor-as real-time imaging can provide information on the temporal signatures of disease we currently cannot assess-with the past decade seeing remarkable advances in acceleration using compressed sensing (CS) and artificial intelligence (AI). However, substantial limitations to real-time imaging remain and reconstruction quality is not always guaranteed. To improve reconstruction fidelity in dynamic cardiac MRI, we propose a novel predictive signal model that uses a priori statistics to adaptively predict temporal cardiac dynamics. By using a small training set obtained from the same patient, the new signal model can achieve robust dynamic cardiac MRI in the presence of irregular cardiac rhythm. Evaluation on simulated irregular cardiac dynamics and prospectively undersampled clinical cardiac MRI data demonstrate improved reconstruction quality for two reconstruction frameworks: Kalman filter and CS. The predictive model also works with different undersampling patterns (cartesian, radial, spiral) and can serve as a versatile foundation for robust dynamic cardiac MRI.
Collapse
Affiliation(s)
- Aaron D Curtis
- The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
- Translational Biology & Engineering Program, Ted Rogers Centre for Heart Research, Toronto, Canada
| | - Alexander J Mertens
- The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
- Translational Biology & Engineering Program, Ted Rogers Centre for Heart Research, Toronto, Canada
| | - Hai-Ling Margaret Cheng
- The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada.
- Translational Biology & Engineering Program, Ted Rogers Centre for Heart Research, Toronto, Canada.
- Institute of Biomedical Engineering, University of Toronto, 661 University Avenue, Room 1443, Toronto, Ontario, M5G 1M1, Canada.
| |
Collapse
|
4
|
Neofytou AP, Neji R, Kowalik GT, Mooiweer R, Wong J, Fotaki A, Ferreira J, Evans C, Bosio F, Mughal N, Razavi R, Pushparajah K, Roujol S. Retrospective motion correction through multi-average k-space data elimination (REMAKE) for free-breathing cardiac cine imaging. Magn Reson Med 2023; 89:2242-2254. [PMID: 36763898 PMCID: PMC10952356 DOI: 10.1002/mrm.29613] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 01/06/2023] [Accepted: 01/20/2023] [Indexed: 02/12/2023]
Abstract
PURPOSE To develop a motion-robust reconstruction technique for free-breathing cine imaging with multiple averages. METHOD Retrospective motion correction through multiple average k-space data elimination (REMAKE) was developed using iterative removal of k-space segments (from individual k-space samples) that contribute most to motion corruption while combining any remaining segments across multiple signal averages. A variant of REMAKE, termed REMAKE+, was developed to address any losses in SNR due to k-space information removal. With REMAKE+, multiple reconstructions using different initial conditions were performed, co-registered, and averaged. Both techniques were validated against clinical "standard" signal averaging reconstruction in a static phantom (with simulated motion) and 15 patients undergoing free-breathing cine imaging with multiple averages. Quantitative analysis of myocardial sharpness, blood/myocardial SNR, myocardial-blood contrast-to-noise ratio (CNR), as well as subjective assessment of image quality and rate of diagnostic quality images were performed. RESULTS In phantom, motion artifacts using "standard" (RMS error [RMSE]: 2.2 ± 0.5) were substantially reduced using REMAKE/REMAKE+ (RMSE: 1.5 ± 0.4/1.0 ± 0.4, p < 0.01). In patients, REMAKE/REMAKE+ led to higher myocardial sharpness (0.79 ± 0.09/0.79 ± 0.1 vs. 0.74 ± 0.12 for "standard", p = 0.004/0.04), higher image quality (1.8 ± 0.2/1.9 ± 0.2 vs. 1.6 ± 0.4 for "standard", p = 0.02/0.008), and a higher rate of diagnostic quality images (99%/100% vs. 94% for "standard"). Blood/myocardial SNR for "standard" (94 ± 30/33 ± 10) was higher vs. REMAKE (80 ± 25/28 ± 8, p = 0.002/0.005) and tended to be lower vs. REMAKE+ (105 ± 33/36 ± 12, p = 0.02/0.06). Myocardial-blood CNR for "standard" (61 ± 22) was higher vs. REMAKE (53 ± 19, p = 0.003) and lower vs. REMAKE+ (69 ± 24, p = 0.007). CONCLUSIONS Compared to "standard" signal averaging reconstruction, REMAKE and REMAKE+ provide improved myocardial sharpness, image quality, and rate of diagnostic quality images.
Collapse
Affiliation(s)
- Alexander Paul Neofytou
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Radhouene Neji
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- MR Research CollaborationsSiemens Healthcare LimitedNewton House, Sir William Siemens Square, Frimley, CamberleySurreyUK
| | - Grzegorz Tomasz Kowalik
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Ronald Mooiweer
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- MR Research CollaborationsSiemens Healthcare LimitedNewton House, Sir William Siemens Square, Frimley, CamberleySurreyUK
| | - James Wong
- Department of Paediatric CardiologyEvelina London Children's HospitalLondonUK
| | - Anastasia Fotaki
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Joana Ferreira
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Carl Evans
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Filippo Bosio
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Nabila Mughal
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Reza Razavi
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Kuberan Pushparajah
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- Department of Paediatric CardiologyEvelina London Children's HospitalLondonUK
| | - Sébastien Roujol
- School of Biomedical Engineering and Imaging Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| |
Collapse
|
5
|
Jafari M, Shoeibi A, Khodatars M, Ghassemi N, Moridian P, Alizadehsani R, Khosravi A, Ling SH, Delfan N, Zhang YD, Wang SH, Gorriz JM, Alinejad-Rokny H, Acharya UR. Automated diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging using deep learning models: A review. Comput Biol Med 2023; 160:106998. [PMID: 37182422 DOI: 10.1016/j.compbiomed.2023.106998] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 03/01/2023] [Accepted: 04/28/2023] [Indexed: 05/16/2023]
Abstract
In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. At early stages, CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMRI) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians as each scan has many slices of data, and the contrast of it might be low. To address these issues, deep learning (DL) techniques have been employed in the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. The following presents research to detect CVDs using CMR images and the most significant DL methods. Another section discussed the challenges in diagnosing CVDs from CMRI data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. Finally, the most important findings of this study are presented in the conclusion section.
Collapse
Affiliation(s)
- Mahboobeh Jafari
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Afshin Shoeibi
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia; Data Science and Computational Intelligence Institute, University of Granada, Spain.
| | - Marjane Khodatars
- Data Science and Computational Intelligence Institute, University of Granada, Spain
| | - Navid Ghassemi
- Internship in BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Parisa Moridian
- Data Science and Computational Intelligence Institute, University of Granada, Spain
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Australia
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Australia
| | - Sai Ho Ling
- Faculty of Engineering and IT, University of Technology Sydney (UTS), Australia
| | - Niloufar Delfan
- Faculty of Computer Engineering, Dept. of Artificial Intelligence Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Yu-Dong Zhang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
| | - Shui-Hua Wang
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK
| | - Juan M Gorriz
- Data Science and Computational Intelligence Institute, University of Granada, Spain; Department of Psychiatry, University of Cambridge, UK
| | - Hamid Alinejad-Rokny
- BioMedical Machine Learning Lab, The Graduate School of Biomedical Engineering, UNSW Sydney, Sydney, NSW, 2052, Australia; UNSW Data Science Hub, The University of New South Wales, Sydney, NSW, 2052, Australia; Health Data Analytics Program, Centre for Applied Artificial Intelligence, Macquarie University, Sydney, 2109, Australia
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia; Dept. of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan
| |
Collapse
|
6
|
Chen J, Chen S, Wee L, Dekker A, Bermejo I. Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review. Phys Med Biol 2023; 68. [PMID: 36753766 DOI: 10.1088/1361-6560/acba74] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Accepted: 02/08/2023] [Indexed: 02/10/2023]
Abstract
Purpose. There is a growing number of publications on the application of unpaired image-to-image (I2I) translation in medical imaging. However, a systematic review covering the current state of this topic for medical physicists is lacking. The aim of this article is to provide a comprehensive review of current challenges and opportunities for medical physicists and engineers to apply I2I translation in practice.Methods and materials. The PubMed electronic database was searched using terms referring to unpaired (unsupervised), I2I translation, and medical imaging. This review has been reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. From each full-text article, we extracted information extracted regarding technical and clinical applications of methods, Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) study type, performance of algorithm and accessibility of source code and pre-trained models.Results. Among 461 unique records, 55 full-text articles were included in the review. The major technical applications described in the selected literature are segmentation (26 studies), unpaired domain adaptation (18 studies), and denoising (8 studies). In terms of clinical applications, unpaired I2I translation has been used for automatic contouring of regions of interest in MRI, CT, x-ray and ultrasound images, fast MRI or low dose CT imaging, CT or MRI only based radiotherapy planning, etc Only 5 studies validated their models using an independent test set and none were externally validated by independent researchers. Finally, 12 articles published their source code and only one study published their pre-trained models.Conclusion. I2I translation of medical images offers a range of valuable applications for medical physicists. However, the scarcity of external validation studies of I2I models and the shortage of publicly available pre-trained models limits the immediate applicability of the proposed methods in practice.
Collapse
Affiliation(s)
- Junhua Chen
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, 6229 ET, The Netherlands
| | - Shenlun Chen
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, 6229 ET, The Netherlands
| | - Leonard Wee
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, 6229 ET, The Netherlands
| | - Andre Dekker
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, 6229 ET, The Netherlands
| | - Inigo Bermejo
- Department of Radiation Oncology (MAASTRO), GROW School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Maastricht, 6229 ET, The Netherlands
| |
Collapse
|
7
|
Chen Z, Pawar K, Ekanayake M, Pain C, Zhong S, Egan GF. Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges. J Digit Imaging 2023; 36:204-230. [PMID: 36323914 PMCID: PMC9984670 DOI: 10.1007/s10278-022-00721-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 09/09/2022] [Accepted: 10/17/2022] [Indexed: 11/06/2022] Open
Abstract
Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.
Collapse
Affiliation(s)
- Zhaolin Chen
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia.
- Department of Data Science and AI, Monash University, Melbourne, VIC, Australia.
| | - Kamlesh Pawar
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
| | - Mevan Ekanayake
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC, Australia
| | - Cameron Pain
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC, Australia
| | - Shenjun Zhong
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- National Imaging Facility, Brisbane, QLD, Australia
| | - Gary F Egan
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
8
|
Gao C, Ghodrati V, Shih SF, Wu HH, Liu Y, Nickel MD, Vahle T, Dale B, Sai V, Felker E, Surawech C, Miao Q, Finn JP, Zhong X, Hu P. Undersampling artifact reduction for free-breathing 3D stack-of-radial MRI based on a deep adversarial learning network. Magn Reson Imaging 2023; 95:70-79. [PMID: 36270417 PMCID: PMC10163826 DOI: 10.1016/j.mri.2022.10.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 10/06/2022] [Accepted: 10/14/2022] [Indexed: 11/05/2022]
Abstract
PURPOSE Stack-of-radial MRI allows free-breathing abdominal scans, however, it requires relatively long acquisition time. Undersampling reduces scan time but can cause streaking artifacts and degrade image quality. This study developed deep learning networks with adversarial loss and evaluated the performance of reducing streaking artifacts and preserving perceptual image sharpness. METHODS A 3D generative adversarial network (GAN) was developed for reducing streaking artifacts in stack-of-radial abdominal scans. Training and validation datasets were self-gated to 5 respiratory states to reduce motion artifacts and to effectively augment the data. The network used a combination of three loss functions to constrain the anatomy and preserve image quality: adversarial loss, mean-squared-error loss and structural similarity index loss. The performance of the network was investigated for 3-5 times undersampled data from 2 institutions. The performance of the GAN for 5 times accelerated images was compared with a 3D U-Net and evaluated using quantitative NMSE, SSIM and region of interest (ROI) measurements as well as qualitative scores of radiologists. RESULTS The 3D GAN showed similar NMSE (0.0657 vs. 0.0559, p = 0.5217) and significantly higher SSIM (0.841 vs. 0.798, p < 0.0001) compared to U-Net. ROI analysis showed GAN removed streaks in both the background air and the tissue and was not significantly different from the reference mean and variations. Radiologists' scores showed GAN had a significant improvement of 1.6 point (p = 0.004) on a 4-point scale in streaking score while no significant difference in sharpness score compared to the input. CONCLUSION 3D GAN removes streaking artifacts and preserves perceptual image details.
Collapse
Affiliation(s)
- Chang Gao
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | - Vahid Ghodrati
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | - Shu-Fu Shih
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Department of Bioengineering, University of California Los Angeles, Los Angeles, CA, United States
| | - Holden H Wu
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States; Department of Bioengineering, University of California Los Angeles, Los Angeles, CA, United States
| | - Yongkai Liu
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | | | - Thomas Vahle
- MR Application Predevelopment, Siemens Healthcare GmbH, Erlangen, Germany
| | - Brian Dale
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Cary, NC, United States
| | - Victor Sai
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States
| | - Ely Felker
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States
| | - Chuthaporn Surawech
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Department of Radiology, Division of Diagnostic Radiology, Faculty of Medicine, Chulalongkorn University and King Chulalongkorn Memorial Hospital, Bangkok, Thailand
| | - Qi Miao
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Department of Radiology, The First Affiliated Hospital of China Medical University, Shenyang, Liaoning Province, China
| | - J Paul Finn
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States
| | - Xiaodong Zhong
- MR R&D Collaborations, Siemens Medical Solutions USA, Inc., Los Angeles, CA, United States
| | - Peng Hu
- Department of Radiological Sciences, University of California Los Angeles, Los Angeles, CA, United States; Inter-Departmental Graduate Program of Physics and Biology in Medicine, University of California Los Angeles, Los Angeles, CA, United States.
| |
Collapse
|
9
|
Li H, Bhatt M, Qu Z, Zhang S, Hartel MC, Khademhosseini A, Cloutier G. Deep learning in ultrasound elastography imaging: A review. Med Phys 2022; 49:5993-6018. [PMID: 35842833 DOI: 10.1002/mp.15856] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2021] [Revised: 02/04/2022] [Accepted: 07/06/2022] [Indexed: 11/11/2022] Open
Abstract
It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Hongliang Li
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada.,Institute of Biomedical Engineering, University of Montreal, Montréal, Québec, Canada
| | - Manish Bhatt
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada
| | - Zhen Qu
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada
| | - Shiming Zhang
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Martin C Hartel
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Ali Khademhosseini
- California Nanosystems Institute, University of California, Los Angeles, California, USA
| | - Guy Cloutier
- Laboratory of Biorheology and Medical Ultrasonics, University of Montreal Hospital Research Center, Montréal, Québec, Canada.,Institute of Biomedical Engineering, University of Montreal, Montréal, Québec, Canada.,Department of Radiology, Radio-Oncology and Nuclear Medicine, University of Montreal, Montréal, Québec, Canada
| |
Collapse
|
10
|
Ismail TF, Strugnell W, Coletti C, Božić-Iven M, Weingärtner S, Hammernik K, Correia T, Küstner T. Cardiac MR: From Theory to Practice. Front Cardiovasc Med 2022; 9:826283. [PMID: 35310962 PMCID: PMC8927633 DOI: 10.3389/fcvm.2022.826283] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 01/17/2022] [Indexed: 01/10/2023] Open
Abstract
Cardiovascular disease (CVD) is the leading single cause of morbidity and mortality, causing over 17. 9 million deaths worldwide per year with associated costs of over $800 billion. Improving prevention, diagnosis, and treatment of CVD is therefore a global priority. Cardiovascular magnetic resonance (CMR) has emerged as a clinically important technique for the assessment of cardiovascular anatomy, function, perfusion, and viability. However, diversity and complexity of imaging, reconstruction and analysis methods pose some limitations to the widespread use of CMR. Especially in view of recent developments in the field of machine learning that provide novel solutions to address existing problems, it is necessary to bridge the gap between the clinical and scientific communities. This review covers five essential aspects of CMR to provide a comprehensive overview ranging from CVDs to CMR pulse sequence design, acquisition protocols, motion handling, image reconstruction and quantitative analysis of the obtained data. (1) The basic MR physics of CMR is introduced. Basic pulse sequence building blocks that are commonly used in CMR imaging are presented. Sequences containing these building blocks are formed for parametric mapping and functional imaging techniques. Commonly perceived artifacts and potential countermeasures are discussed for these methods. (2) CMR methods for identifying CVDs are illustrated. Basic anatomy and functional processes are described to understand the cardiac pathologies and how they can be captured by CMR imaging. (3) The planning and conduct of a complete CMR exam which is targeted for the respective pathology is shown. Building blocks are illustrated to create an efficient and patient-centered workflow. Further strategies to cope with challenging patients are discussed. (4) Imaging acceleration and reconstruction techniques are presented that enable acquisition of spatial, temporal, and parametric dynamics of the cardiac cycle. The handling of respiratory and cardiac motion strategies as well as their integration into the reconstruction processes is showcased. (5) Recent advances on deep learning-based reconstructions for this purpose are summarized. Furthermore, an overview of novel deep learning image segmentation and analysis methods is provided with a focus on automatic, fast and reliable extraction of biomarkers and parameters of clinical relevance.
Collapse
Affiliation(s)
- Tevfik F. Ismail
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
- Cardiology Department, Guy's and St Thomas' Hospital, London, United Kingdom
| | - Wendy Strugnell
- Queensland X-Ray, Mater Hospital Brisbane, Brisbane, QLD, Australia
| | - Chiara Coletti
- Magnetic Resonance Systems Lab, Delft University of Technology, Delft, Netherlands
| | - Maša Božić-Iven
- Magnetic Resonance Systems Lab, Delft University of Technology, Delft, Netherlands
- Computer Assisted Clinical Medicine, Heidelberg University, Mannheim, Germany
| | | | - Kerstin Hammernik
- Lab for AI in Medicine, Technical University of Munich, Munich, Germany
- Department of Computing, Imperial College London, London, United Kingdom
| | - Teresa Correia
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom
- Centre of Marine Sciences, Faro, Portugal
| | - Thomas Küstner
- Medical Image and Data Analysis (MIDAS.lab), Department of Diagnostic and Interventional Radiology, University Hospital of Tübingen, Tübingen, Germany
| |
Collapse
|
11
|
Lyu Q, Shan H, Xie Y, Kwan AC, Otaki Y, Kuronuma K, Li D, Wang G. Cine Cardiac MRI Motion Artifact Reduction Using a Recurrent Neural Network. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:2170-2181. [PMID: 33856986 PMCID: PMC8376223 DOI: 10.1109/tmi.2021.3073381] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
Cine cardiac magnetic resonance imaging (MRI) is widely used for the diagnosis of cardiac diseases thanks to its ability to present cardiovascular features in excellent contrast. As compared to computed tomography (CT), MRI, however, requires a long scan time, which inevitably induces motion artifacts and causes patients' discomfort. Thus, there has been a strong clinical motivation to develop techniques to reduce both the scan time and motion artifacts. Given its successful applications in other medical imaging tasks such as MRI super-resolution and CT metal artifact reduction, deep learning is a promising approach for cardiac MRI motion artifact reduction. In this paper, we propose a novel recurrent generative adversarial network model for cardiac MRI motion artifact reduction. This model utilizes bi-directional convolutional long short-term memory (ConvLSTM) and multi-scale convolutions to improve the performance of the proposed network, in which bi-directional ConvLSTMs handle long-range temporal features while multi-scale convolutions gather both local and global features. We demonstrate a decent generalizability of the proposed method thanks to the novel architecture of our deep network that captures the essential relationship of cardiovascular dynamics. Indeed, our extensive experiments show that our method achieves better image quality for cine cardiac MRI images than existing state-of-the-art methods. In addition, our method can generate reliable missing intermediate frames based on their adjacent frames, improving the temporal resolution of cine cardiac MRI sequences.
Collapse
|
12
|
Ghodrati V, Bydder M, Bedayat A, Prosper A, Yoshida T, Nguyen KL, Finn JP, Hu P. Temporally aware volumetric generative adversarial network-based MR image reconstruction with simultaneous respiratory motion compensation: Initial feasibility in 3D dynamic cine cardiac MRI. Magn Reson Med 2021; 86:2666-2683. [PMID: 34254363 PMCID: PMC10172149 DOI: 10.1002/mrm.28912] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Revised: 06/02/2021] [Accepted: 06/12/2021] [Indexed: 12/26/2022]
Abstract
PURPOSE Develop a novel three-dimensional (3D) generative adversarial network (GAN)-based technique for simultaneous image reconstruction and respiratory motion compensation of 4D MRI. Our goal was to enable high-acceleration factors 10.7X-15.8X, while maintaining robust and diagnostic image quality superior to state-of-the-art self-gating (SG) compressed sensing wavelet (CS-WV) reconstruction at lower acceleration factors 3.5X-7.9X. METHODS Our GAN was trained based on pixel-wise content loss functions, adversarial loss function, and a novel data-driven temporal aware loss function to maintain anatomical accuracy and temporal coherence. Besides image reconstruction, our network also performs respiratory motion compensation for free-breathing scans. A novel progressive growing-based strategy was adapted to make the training process possible for the proposed GAN-based structure. The proposed method was developed and thoroughly evaluated qualitatively and quantitatively based on 3D cardiac cine data from 42 patients. RESULTS Our proposed method achieved significantly better scores in general image quality and image artifacts at 10.7X-15.8X acceleration than the SG CS-WV approach at 3.5X-7.9X acceleration (4.53 ± 0.540 vs. 3.13 ± 0.681 for general image quality, 4.12 ± 0.429 vs. 2.97 ± 0.434 for image artifacts, P < .05 for both). No spurious anatomical structures were observed in our images. The proposed method enabled similar cardiac-function quantification as conventional SG CS-WV. The proposed method achieved faster central processing unit-based image reconstruction (6 s/cardiac phase) than the SG CS-WV (312 s/cardiac phase). CONCLUSION The proposed method showed promising potential for high-resolution (1 mm3 ) free-breathing 4D MR data acquisition with simultaneous respiratory motion compensation and fast reconstruction time.
Collapse
Affiliation(s)
- Vahid Ghodrati
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA.,Biomedical Physics Inter-Departmental Graduate Program, University of California, Los Angeles, California, USA
| | - Mark Bydder
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA
| | - Arash Bedayat
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA
| | - Ashley Prosper
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA
| | - Takegawa Yoshida
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA
| | - Kim-Lien Nguyen
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA.,Biomedical Physics Inter-Departmental Graduate Program, University of California, Los Angeles, California, USA.,Department of Medicine (Cardiology), David Geffen School of Medicine, University of California, Los Angeles, California, USA
| | - J Paul Finn
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA
| | - Peng Hu
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles, California, USA.,Biomedical Physics Inter-Departmental Graduate Program, University of California, Los Angeles, California, USA
| |
Collapse
|