1
|
Gulamhussene G, Rak M, Bashkanov O, Joeres F, Omari J, Pech M, Hansen C. Transfer-learning is a key ingredient to fast deep learning-based 4D liver MRI reconstruction. Sci Rep 2023; 13:11227. [PMID: 37433827 DOI: 10.1038/s41598-023-38073-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Accepted: 07/02/2023] [Indexed: 07/13/2023] Open
Abstract
Time-resolved volumetric magnetic resonance imaging (4D MRI) could be used to address organ motion in image-guided interventions like tumor ablation. Current 4D reconstruction techniques are unsuitable for most interventional settings because they are limited to specific breathing phases, lack temporal/spatial resolution, and have long prior acquisitions or reconstruction times. Deep learning-based (DL) 4D MRI approaches promise to overcome these shortcomings but are sensitive to domain shift. This work shows that transfer learning (TL) combined with an ensembling strategy can help alleviate this key challenge. We evaluate four approaches: pre-trained models from the source domain, models directly trained from scratch on target domain data, models fine-tuned from a pre-trained model and an ensemble of fine-tuned models. For that the data base was split into 16 source and 4 target domain subjects. Comparing ensemble of fine-tuned models (N = 10) with directly learned models, we report significant improvements (P < 0.001) of the root mean squared error (RMSE) of up to 12% and the mean displacement (MDISP) of up to 17.5%. The smaller the target domain data amount, the larger the effect. This shows that TL + Ens significantly reduces beforehand acquisition time and improves reconstruction quality, rendering it a key component in making 4D MRI clinically feasible for the first time in the context of 4D organ motion models of the liver and beyond.
Collapse
Affiliation(s)
- Gino Gulamhussene
- Otto-von-Guericke University Magdeburg, Faculty of Computer Science, 39106, Magdeburg, Germany.
| | - Marko Rak
- Otto-von-Guericke University Magdeburg, Faculty of Computer Science, 39106, Magdeburg, Germany
| | - Oleksii Bashkanov
- Otto-von-Guericke University Magdeburg, Faculty of Computer Science, 39106, Magdeburg, Germany
| | - Fabian Joeres
- Otto-von-Guericke University Magdeburg, Faculty of Computer Science, 39106, Magdeburg, Germany
| | - Jazan Omari
- Department of Radiology and Nuclear Medicine, University Hospital Magdeburg, 39120, Magdeburg, Germany
| | - Maciej Pech
- Department of Radiology and Nuclear Medicine, University Hospital Magdeburg, 39120, Magdeburg, Germany
| | - Christian Hansen
- Otto-von-Guericke University Magdeburg, Faculty of Computer Science, 39106, Magdeburg, Germany.
| |
Collapse
|
2
|
Gulamhussene G, Meyer A, Rak M, Bashkanov O, Omari J, Pech M, Hansen C. Predicting 4D liver MRI for MR-guided interventions. Comput Med Imaging Graph 2022; 101:102122. [PMID: 36122484 DOI: 10.1016/j.compmedimag.2022.102122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 06/06/2022] [Accepted: 08/18/2022] [Indexed: 01/27/2023]
Abstract
Organ motion poses an unresolved challenge in image-guided interventions like radiation therapy, biopsies or tumor ablation. In the pursuit of solving this problem, the research field of time-resolved volumetric magnetic resonance imaging (4D MRI) has evolved. However, current techniques are unsuitable for most interventional settings because they lack sufficient temporal and/or spatial resolution or have long acquisition times. In this work, we propose a novel approach for real-time, high-resolution 4D MRI with large fields of view for MR-guided interventions. To this end, we propose a network-agnostic, end-to-end trainable, deep learning formulation that enables the prediction of a 4D liver MRI with respiratory states from a live 2D navigator MRI. Our method can be used in two ways: First, it can reconstruct high quality fast (near real-time) 4D MRI with high resolution (209×128×128 matrix size with isotropic 1.8mm voxel size and 0.6s/volume) given a dynamic interventional 2D navigator slice for guidance during an intervention. Second, it can be used for retrospective 4D reconstruction with a temporal resolution of below 0.2s/volume for motion analysis and use in radiation therapy. We report a mean target registration error (TRE) of 1.19±0.74mm, which is below voxel size. We compare our results with a state-of-the-art retrospective 4D MRI reconstruction. Visual evaluation shows comparable quality. We compare different network architectures within our formulation. We show that small training sizes with short acquisition times down to 2 min can already achieve promising results and 24 min are sufficient for high quality results. Because our method can be readily combined with earlier time reducing methods, acquisition time can be further decreased while also limiting quality loss. We show that an end-to-end, deep learning formulation is highly promising for 4D MRI reconstruction.
Collapse
Affiliation(s)
- Gino Gulamhussene
- Otto-von-Guericke University, Faculty of Computer Science, Universitätsplatz 2, Magdeburg, 39106, Saxony-Anhalt, Germany.
| | - Anneke Meyer
- Otto-von-Guericke University, Faculty of Computer Science, Universitätsplatz 2, Magdeburg, 39106, Saxony-Anhalt, Germany
| | - Marko Rak
- Otto-von-Guericke University, Faculty of Computer Science, Universitätsplatz 2, Magdeburg, 39106, Saxony-Anhalt, Germany
| | - Oleksii Bashkanov
- Otto-von-Guericke University, Faculty of Computer Science, Universitätsplatz 2, Magdeburg, 39106, Saxony-Anhalt, Germany
| | - Jazan Omari
- University Hospital Magdeburg, Department of Radiology and Nuclear Medicine, Leipziger Straße 44, Magdeburg, 39120, Saxony-Anhalt, Germany
| | - Maciej Pech
- University Hospital Magdeburg, Department of Radiology and Nuclear Medicine, Leipziger Straße 44, Magdeburg, 39120, Saxony-Anhalt, Germany
| | - Christian Hansen
- Otto-von-Guericke University, Faculty of Computer Science, Universitätsplatz 2, Magdeburg, 39106, Saxony-Anhalt, Germany
| |
Collapse
|
3
|
Kanberoglu B, Das D, Nair P, Turaga P, Frakes D. An Optical Flow-Based Approach for Minimally Divergent Velocimetry Data Interpolation. Int J Biomed Imaging 2019; 2019:9435163. [PMID: 30863431 PMCID: PMC6378004 DOI: 10.1155/2019/9435163] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2018] [Revised: 12/04/2018] [Accepted: 12/10/2018] [Indexed: 12/03/2022] Open
Abstract
Three-dimensional (3D) biomedical image sets are often acquired with in-plane pixel spacings that are far less than the out-of-plane spacings between images. The resultant anisotropy, which can be detrimental in many applications, can be decreased using image interpolation. Optical flow and/or other registration-based interpolators have proven useful in such interpolation roles in the past. When acquired images are comprised of signals that describe the flow velocity of fluids, additional information is available to guide the interpolation process. In this paper, we present an optical-flow based framework for image interpolation that also minimizes resultant divergence in the interpolated data.
Collapse
Affiliation(s)
- Berkay Kanberoglu
- School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, 85281, USA
| | - Dhritiman Das
- Department of Computer Science, Technical University of Munich, Munich, 80333, Germany
| | - Priya Nair
- School of Biological and Health Systems Engineering, Arizona State University, Tempe, 85281, USA
| | - Pavan Turaga
- School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, 85281, USA
- School of Arts, Media and Engineering, Arizona State University, Tempe, 85281, USA
| | - David Frakes
- School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, 85281, USA
- School of Biological and Health Systems Engineering, Arizona State University, Tempe, 85281, USA
| |
Collapse
|