1
|
Hattori M, Tsubakiya H, Lee SH, Kanai T, Suzuki K, Yuasa T. A deep-learning-based scatter correction with water equivalent path length map for digital radiography. Radiol Phys Technol 2024; 17:488-503. [PMID: 38696086 DOI: 10.1007/s12194-024-00807-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Revised: 04/20/2024] [Accepted: 04/22/2024] [Indexed: 05/27/2024]
Abstract
We proposed a new deep learning (DL) model for accurate scatter correction in digital radiography. The proposed network featured a pixel-wise water equivalent path length (WEPL) map of subjects with diverse sizes and 3D inner structures. The proposed U-Net model comprises two concatenated modules: one for generating a WEPL map and the other for predicting scatter using the WEPL map as auxiliary information. First, 3D CT images were used as numerical phantoms for training and validation, generating observed and scattered images by Monte Carlo simulation, and WEPL maps using Siddon's algorithm. Then, we optimised the model without overfitting. Next, we validated the proposed model's performance by comparing it with other DL models. The proposed model obtained scatter-corrected images with a peak signal-to-noise ratio of 44.24 ± 2.89 dB and a structural similarity index measure of 0.9987 ± 0.0004, which were higher than other DL models. Finally, scatter fractions (SFs) were compared with other DL models using an actual phantom to confirm practicality. Among DL models, the proposed model showed the smallest deviation from measured SF values. Furthermore, using an actual radiograph containing an acrylic object, the contrast-to-noise ratio (CNR) of the proposed model and the anti-scatter grid were compared. The CNR of the images corrected using the proposed model are 16% and 82% higher than those of the raw and grid-applied images, respectively. The advantage of the proposed method is that no actual radiography system is required for collecting training dataset, as the dataset is created from CT images using Monte Carlo simulation.
Collapse
Affiliation(s)
- Masayuki Hattori
- Graduate School of Science and Engineering, Yamagata University, Yonezawa, 992-8510, Japan.
- Department of Radiology, Yamagata University Hospital, Yamagata, 990-9585, Japan.
| | - Hisato Tsubakiya
- Graduate School of Science and Engineering, Yamagata University, Yonezawa, 992-8510, Japan
| | - Sung-Hyun Lee
- Department of Heavy Particle Medical Science, Graduate School of Medicine, Yamagata University, Yamagata, 990-9585, Japan
| | - Takayuki Kanai
- Department of Heavy Particle Medical Science, Graduate School of Medicine, Yamagata University, Yamagata, 990-9585, Japan
- Department of Radiation Oncology, Tokyo Women's Medical University, Shinjuku, Tokyo, Japan
| | - Koji Suzuki
- Department of Radiology, Yamagata University Hospital, Yamagata, 990-9585, Japan
| | - Tetsuya Yuasa
- Graduate School of Science and Engineering, Yamagata University, Yonezawa, 992-8510, Japan
| |
Collapse
|
2
|
Lu X, Liang X, Liu W, Miao X, Guan X. ReeGAN: MRI image edge-preserving synthesis based on GANs trained with misaligned data. Med Biol Eng Comput 2024; 62:1851-1868. [PMID: 38396277 DOI: 10.1007/s11517-024-03035-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 01/27/2024] [Indexed: 02/25/2024]
Abstract
As a crucial medical examination technique, different modalities of magnetic resonance imaging (MRI) complement each other, offering multi-angle and multi-dimensional insights into the body's internal information. Therefore, research on MRI cross-modality conversion is of great significance, and many innovative techniques have been explored. However, most methods are trained on well-aligned data, and the impact of misaligned data has not received sufficient attention. Additionally, many methods focus on transforming the entire image and ignore crucial edge information. To address these challenges, we propose a generative adversarial network based on multi-feature fusion, which effectively preserves edge information while training on noisy data. Notably, we consider images with limited range random transformations as noisy labels and use an additional small auxiliary registration network to help the generator adapt to the noise distribution. Moreover, we inject auxiliary edge information to improve the quality of synthesized target modality images. Our goal is to find the best solution for cross-modality conversion. Comprehensive experiments and ablation studies demonstrate the effectiveness of the proposed method.
Collapse
Affiliation(s)
- Xiangjiang Lu
- Guangxi Key Lab of Multi-Source Information Mining & Security, School of Computer Science and Engineering & School of Software, Guangxi Normal University, Guilin, 541004, China.
| | - Xiaoshuang Liang
- Guangxi Key Lab of Multi-Source Information Mining & Security, School of Computer Science and Engineering & School of Software, Guangxi Normal University, Guilin, 541004, China
| | - Wenjing Liu
- Guangxi Key Lab of Multi-Source Information Mining & Security, School of Computer Science and Engineering & School of Software, Guangxi Normal University, Guilin, 541004, China
| | - Xiuxia Miao
- Guangxi Key Lab of Multi-Source Information Mining & Security, School of Computer Science and Engineering & School of Software, Guangxi Normal University, Guilin, 541004, China
| | - Xianglong Guan
- Guangxi Key Lab of Multi-Source Information Mining & Security, School of Computer Science and Engineering & School of Software, Guangxi Normal University, Guilin, 541004, China
| |
Collapse
|
3
|
Dai X, Ma N, Du L, Wang X, Ju Z, Jie C, Gong H, Ge R, Yu W, Qu B. Application of MR images in radiotherapy planning for brain tumor based on deep learning. Int J Neurosci 2024:1-11. [PMID: 38712669 DOI: 10.1080/00207454.2024.2352784] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Accepted: 05/03/2024] [Indexed: 05/08/2024]
Abstract
PURPOSE Explore the function and dose calculation accuracy of MRI images in radiotherapy planning through deep learning methods. METHODS 131 brain tumor patients undergoing radiotherapy with previous MR and CT images were recruited for this study. A new series of MRI from the aligned MR was firstly registered to CT images strictly using MIM software and then resampled. A deep learning method (U-NET) was used to establish a MRI-to-CT conversion model, for which 105 patient images were used as the training set and 26 patient images were used as the tuning set. Data from additional 8 patients were collected as the test set, and the accuracy of the model was evaluated from a dosimetric standpoint. RESULTS Comparing the synthetic CT images with the original CT images, the difference in dosimetric parameters D98, D95, D2 and Dmean of PTV in 8 patients was less than 0.5%. The gamma passed rates of PTV and whole body volume were: 1%/1 mm: 93.96%±6.75%, 2%/2 mm: 99.87%±0.30%, 3%/3 mm: 100.00%±0.00%; and 1%/1 mm: 99.14%±0.80%, 2%/2 mm: 99.92%±0.08%, 3%/3 mm: 99.99%±0.01%. CONCLUSION MR images can be used both in delineation and treatment efficacy evaluation and in dose calculation. Using the deep learning way to convert MR image to CT image is a viable method and can be further used in dose calculation.
Collapse
Affiliation(s)
- Xiangkun Dai
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | - Na Ma
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
- School of Biological Science and Medical Engineering, Beihang, University, Beijing, China
| | - Lehui Du
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | | | - Zhongjian Ju
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | - Chuanbin Jie
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | - Hanshun Gong
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | - Ruigang Ge
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | - Wei Yu
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| | - Baolin Qu
- Department of Radiotherapy, First Medical Center of PLA General Hospital, Beijing, China
| |
Collapse
|
4
|
Dayarathna S, Islam KT, Uribe S, Yang G, Hayat M, Chen Z. Deep learning based synthesis of MRI, CT and PET: Review and analysis. Med Image Anal 2024; 92:103046. [PMID: 38052145 DOI: 10.1016/j.media.2023.103046] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Revised: 11/14/2023] [Accepted: 11/29/2023] [Indexed: 12/07/2023]
Abstract
Medical image synthesis represents a critical area of research in clinical decision-making, aiming to overcome the challenges associated with acquiring multiple image modalities for an accurate clinical workflow. This approach proves beneficial in estimating an image of a desired modality from a given source modality among the most common medical imaging contrasts, such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Positron Emission Tomography (PET). However, translating between two image modalities presents difficulties due to the complex and non-linear domain mappings. Deep learning-based generative modelling has exhibited superior performance in synthetic image contrast applications compared to conventional image synthesis methods. This survey comprehensively reviews deep learning-based medical imaging translation from 2018 to 2023 on pseudo-CT, synthetic MR, and synthetic PET. We provide an overview of synthetic contrasts in medical imaging and the most frequently employed deep learning networks for medical image synthesis. Additionally, we conduct a detailed analysis of each synthesis method, focusing on their diverse model designs based on input domains and network architectures. We also analyse novel network architectures, ranging from conventional CNNs to the recent Transformer and Diffusion models. This analysis includes comparing loss functions, available datasets and anatomical regions, and image quality assessments and performance in other downstream tasks. Finally, we discuss the challenges and identify solutions within the literature, suggesting possible future directions. We hope that the insights offered in this survey paper will serve as a valuable roadmap for researchers in the field of medical image synthesis.
Collapse
Affiliation(s)
- Sanuwani Dayarathna
- Department of Data Science and AI, Faculty of Information Technology, Monash University, Clayton VIC 3800, Australia.
| | | | - Sergio Uribe
- Department of Medical Imaging and Radiation Sciences, Faculty of Medicine, Monash University, Clayton VIC 3800, Australia
| | - Guang Yang
- Bioengineering Department and Imperial-X, Imperial College London, W12 7SL, United Kingdom
| | - Munawar Hayat
- Department of Data Science and AI, Faculty of Information Technology, Monash University, Clayton VIC 3800, Australia
| | - Zhaolin Chen
- Department of Data Science and AI, Faculty of Information Technology, Monash University, Clayton VIC 3800, Australia; Monash Biomedical Imaging, Clayton VIC 3800, Australia
| |
Collapse
|
5
|
Grexa I, Iván ZZ, Migh E, Kovács F, Bolck HA, Zheng X, Mund A, Moshkov N, Miczán V, Koos K, Horvath P. SuperCUT, an unsupervised multimodal image registration with deep learning for biomedical microscopy. Brief Bioinform 2024; 25:bbae029. [PMID: 38483256 PMCID: PMC10938542 DOI: 10.1093/bib/bbae029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 12/20/2023] [Accepted: 01/08/2024] [Indexed: 03/17/2024] Open
Abstract
Numerous imaging techniques are available for observing and interrogating biological samples, and several of them can be used consecutively to enable correlative analysis of different image modalities with varying resolutions and the inclusion of structural or molecular information. Achieving accurate registration of multimodal images is essential for the correlative analysis process, but it remains a challenging computer vision task with no widely accepted solution. Moreover, supervised registration methods require annotated data produced by experts, which is limited. To address this challenge, we propose a general unsupervised pipeline for multimodal image registration using deep learning. We provide a comprehensive evaluation of the proposed pipeline versus the current state-of-the-art image registration and style transfer methods on four types of biological problems utilizing different microscopy modalities. We found that style transfer of modality domains paired with fully unsupervised training leads to comparable image registration accuracy to supervised methods and, most importantly, does not require human intervention.
Collapse
Affiliation(s)
- Istvan Grexa
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
- Doctoral School of Interdisciplinary Medicine, University of Szeged, Korányi fasor 10, Szeged 6720 Hungary
| | - Zsanett Zsófia Iván
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
- Doctoral School of Biology, University of Szeged, Közép fasor 52, Szeged 6726 Hungary
| | - Ede Migh
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
| | - Ferenc Kovács
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
- Single-Cell Technologies Ltd, Temesvári körút 62, Szeged 6726, Hungary
| | - Hella A Bolck
- Department of Pathology and Molecular Pathology, University Hospital Zürich, Zürich, Schmelzbergstrasse 12 8091, Switzerland
| | - Xiang Zheng
- Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Tuborg Havnevej 19 2900 Hellerup, Denmark
| | - Andreas Mund
- Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Tuborg Havnevej 19 2900 Hellerup, Denmark
| | - Nikita Moshkov
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
| | - Vivien Miczán
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
| | - Krisztian Koos
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
| | - Peter Horvath
- Synthetic and Systems Biology Unit, Biological Research Centre (BRC), Temesvári körút 62, Szeged 6726
- Single-Cell Technologies Ltd, Temesvári körút 62, Szeged 6726, Hungary
- Institute for Molecular Medicine Finland (FIMM), Helsinki Institute of Life Science, University of Helsinki, Tukholmankatu 8, Helsinki 00014, Finland
- Institute of AI for Health, Helmholtz Zentrum München, Ingolstädter Landstraße 1, 85764 Oberschleißheim Neuherberg, Germany
| |
Collapse
|
6
|
Vukadinovic M, Kwan AC, Li D, Ouyang D. GANcMRI: Cardiac magnetic resonance video generation and physiologic guidance using latent space prompting. PROCEEDINGS OF MACHINE LEARNING RESEARCH 2023; 225:594-606. [PMID: 38213931 PMCID: PMC10783442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 01/13/2024]
Abstract
Generative artificial intelligence can be applied to medical imaging on tasks such as privacy-preserving image generation and superresolution and denoising of existing images. Few prior approaches have used cardiac magnetic resonance imaging (cMRI) as a modality given the complexity of videos (the addition of the temporal dimension) as well as the limited scale of publicly available datasets. We introduce GANcMRI, a generative adversarial network that can synthesize cMRI videos with physiological guidance based on latent space prompting. GANcMRI uses a StyleGAN framework to learn the latent space from individual video frames and leverages the timedependent trajectory between end-systolic and end-diastolic frames in the latent space to predict progression and generate motion over time. We proposed various methods for modeling latent time-dependent trajectories and found that our Frame-to-frame approach generates the best motion and video quality. GANcMRI generated high-quality cMRI image frames that are indistinguishable by cardiologists, however, artifacts in video generation allow cardiologists to still recognize the difference between real and generated videos. The generated cMRI videos can be prompted to apply physiologybased adjustments which produces clinically relevant phenotypes recognizable by cardiologists. GANcMRI has many potential applications such as data augmentation, education, anomaly detection, and preoperative planning.
Collapse
|
7
|
Osman AFI, Tamam NM. Contrast-enhanced MRI synthesis using dense-dilated residual convolutions based 3D network toward elimination of gadolinium in neuro-oncology. J Appl Clin Med Phys 2023; 24:e14120. [PMID: 37552487 PMCID: PMC10691635 DOI: 10.1002/acm2.14120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Revised: 07/20/2023] [Accepted: 08/01/2023] [Indexed: 08/09/2023] Open
Abstract
Recent studies have raised broad safety and health concerns about using of gadolinium contrast agents during magnetic resonance imaging (MRI) to enhance identification of active tumors. In this paper, we developed a deep learning-based method for three-dimensional (3D) contrast-enhanced T1-weighted (T1) image synthesis from contrast-free image(s). The MR images of 1251 patients with glioma from the RSNA-ASNR-MICCAI BraTS Challenge 2021 dataset were used in this study. A 3D dense-dilated residual U-Net (DD-Res U-Net) was developed for contrast-enhanced T1 image synthesis from contrast-free image(s). The model was trained on a randomly split training set (n = 800) using a customized loss function and validated on a validation set (n = 200) to improve its generalizability. The generated images were quantitatively assessed against the ground-truth on a test set (n = 251) using the mean absolute error (MAE), mean-squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), normalized mutual information (NMI), and Hausdorff distance (HDD) metrics. We also performed a qualitative visual similarity assessment between the synthetic and ground-truth images. The effectiveness of the proposed model was compared with a 3D U-Net baseline model and existing deep learning-based methods in the literature. Our proposed DD-Res U-Net model achieved promising performance for contrast-enhanced T1 synthesis in both quantitative metrics and perceptual evaluation on the test set (n = 251). Analysis of results on the whole brain region showed a PSNR (in dB) of 29.882 ± 5.924, a SSIM of 0.901 ± 0.071, a MAE of 0.018 ± 0.013, a MSE of 0.002 ± 0.002, a HDD of 2.329 ± 9.623, and a NMI of 1.352 ± 0.091 when using only T1 as input; and a PSNR (in dB) of 30.284 ± 4.934, a SSIM of 0.915 ± 0.063, a MAE of 0.017 ± 0.013, a MSE of 0.001 ± 0.002, a HDD of 1.323 ± 3.551, and a NMI of 1.364 ± 0.089 when combining T1 with other MRI sequences. Compared to the U-Net baseline model, our model revealed superior performance. Our model demonstrated excellent capability in generating synthetic contrast-enhanced T1 images from contrast-free MR image(s) of the whole brain region when using multiple contrast-free images as input. Without incorporating tumor mask information during network training, its performance was inferior in the tumor regions compared to the whole brain which requires further improvements to replace the gadolinium administration in neuro-oncology.
Collapse
Affiliation(s)
| | - Nissren M. Tamam
- Department of PhysicsCollege of SciencePrincess Nourah bint Abdulrahman UniversityRiyadhSaudi Arabia
| |
Collapse
|
8
|
周 家, 郭 红, 陈 红. [Deep learning method for magnetic resonance imaging fluid-attenuated inversion recovery image synthesis]. SHENG WU YI XUE GONG CHENG XUE ZA ZHI = JOURNAL OF BIOMEDICAL ENGINEERING = SHENGWU YIXUE GONGCHENGXUE ZAZHI 2023; 40:903-911. [PMID: 37879919 PMCID: PMC10600433 DOI: 10.7507/1001-5515.202302012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 08/19/2023] [Indexed: 10/27/2023]
Abstract
Magnetic resonance imaging(MRI) can obtain multi-modal images with different contrast, which provides rich information for clinical diagnosis. However, some contrast images are not scanned or the quality of the acquired images cannot meet the diagnostic requirements due to the difficulty of patient's cooperation or the limitation of scanning conditions. Image synthesis techniques have become a method to compensate for such image deficiencies. In recent years, deep learning has been widely used in the field of MRI synthesis. In this paper, a synthesis network based on multi-modal fusion is proposed, which firstly uses a feature encoder to encode the features of multiple unimodal images separately, and then fuses the features of different modal images through a feature fusion module, and finally generates the target modal image. The similarity measure between the target image and the predicted image in the network is improved by introducing a dynamic weighted combined loss function based on the spatial domain and K-space domain. After experimental validation and quantitative comparison, the multi-modal fusion deep learning network proposed in this paper can effectively synthesize high-quality MRI fluid-attenuated inversion recovery (FLAIR) images. In summary, the method proposed in this paper can reduce MRI scanning time of the patient, as well as solve the clinical problem of missing FLAIR images or image quality that is difficult to meet diagnostic requirements.
Collapse
Affiliation(s)
- 家柠 周
- 沈阳工业大学 电气工程学院(沈阳 110870)School of Electrical Engineering, Shenyang University of Technology, Shenyang 110870, P. R. China
| | - 红宇 郭
- 沈阳工业大学 电气工程学院(沈阳 110870)School of Electrical Engineering, Shenyang University of Technology, Shenyang 110870, P. R. China
- 东软医疗系统股份有限公司(沈阳 110167)Neusoft Medical System Co. Ltd, Shenyang 110167, P. R. China
| | - 红 陈
- 沈阳工业大学 电气工程学院(沈阳 110870)School of Electrical Engineering, Shenyang University of Technology, Shenyang 110870, P. R. China
| |
Collapse
|
9
|
Bianconi A, Rossi LF, Bonada M, Zeppa P, Nico E, De Marco R, Lacroce P, Cofano F, Bruno F, Morana G, Melcarne A, Ruda R, Mainardi L, Fiaschi P, Garbossa D, Morra L. Deep learning-based algorithm for postoperative glioblastoma MRI segmentation: a promising new tool for tumor burden assessment. Brain Inform 2023; 10:26. [PMID: 37801128 PMCID: PMC10558414 DOI: 10.1186/s40708-023-00207-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 09/16/2023] [Indexed: 10/07/2023] Open
Abstract
OBJECTIVE Clinical and surgical decisions for glioblastoma patients depend on a tumor imaging-based evaluation. Artificial Intelligence (AI) can be applied to magnetic resonance imaging (MRI) assessment to support clinical practice, surgery planning and prognostic predictions. In a real-world context, the current obstacles for AI are low-quality imaging and postoperative reliability. The aim of this study is to train an automatic algorithm for glioblastoma segmentation on a clinical MRI dataset and to obtain reliable results both pre- and post-operatively. METHODS The dataset used for this study comprises 237 (71 preoperative and 166 postoperative) MRIs from 71 patients affected by a histologically confirmed Grade IV Glioma. The implemented U-Net architecture was trained by transfer learning to perform the segmentation task on postoperative MRIs. The training was carried out first on BraTS2021 dataset for preoperative segmentation. Performance is evaluated using DICE score (DS) and Hausdorff 95% (H95). RESULTS In preoperative scenario, overall DS is 91.09 (± 0.60) and H95 is 8.35 (± 1.12), considering tumor core, enhancing tumor and whole tumor (ET and edema). In postoperative context, overall DS is 72.31 (± 2.88) and H95 is 23.43 (± 7.24), considering resection cavity (RC), gross tumor volume (GTV) and whole tumor (WT). Remarkably, the RC segmentation obtained a mean DS of 63.52 (± 8.90) in postoperative MRIs. CONCLUSIONS The performances achieved by the algorithm are consistent with previous literature for both pre-operative and post-operative glioblastoma's MRI evaluation. Through the proposed algorithm, it is possible to reduce the impact of low-quality images and missing sequences.
Collapse
Affiliation(s)
- Andrea Bianconi
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy.
| | | | - Marta Bonada
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Pietro Zeppa
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Elsa Nico
- Department of Neurosurgery, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, Phoenix, AZ, USA
| | - Raffaele De Marco
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | | | - Fabio Cofano
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Francesco Bruno
- Neurooncology, Department of Neuroscience, University of Turin, Turin, Italy
| | - Giovanni Morana
- Neuroradiology, Department of Neuroscience, University of Turin, Turin, Italy
| | - Antonio Melcarne
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Roberta Ruda
- Neurooncology, Department of Neuroscience, University of Turin, Turin, Italy
| | - Luca Mainardi
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy
| | - Pietro Fiaschi
- IRCCS Ospedale Policlinico S. Martino, Genoa, Italy
- Dipartimento di Neuroscienze, Riabilitazione, Oftalmologia, Genetica e Scienze Materno-Infantili, Univeristy of Genoa, Genoa, Italy
| | - Diego Garbossa
- Neurosurgery, Department of Neuroscience, University of Turin, via Cherasco 15, 10126, Turin, Italy
| | - Lia Morra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, Turin, Italy
| |
Collapse
|
10
|
Chan K, Maralani PJ, Moody AR, Khademi A. Synthesis of diffusion-weighted MRI scalar maps from FLAIR volumes using generative adversarial networks. Front Neuroinform 2023; 17:1197330. [PMID: 37603783 PMCID: PMC10436214 DOI: 10.3389/fninf.2023.1197330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Accepted: 07/18/2023] [Indexed: 08/23/2023] Open
Abstract
Introduction Acquisition and pre-processing pipelines for diffusion-weighted imaging (DWI) volumes are resource- and time-consuming. Generating synthetic DWI scalar maps from commonly acquired brain MRI sequences such as fluid-attenuated inversion recovery (FLAIR) could be useful for supplementing datasets. In this work we design and compare GAN-based image translation models for generating DWI scalar maps from FLAIR MRI for the first time. Methods We evaluate a pix2pix model, two modified CycleGANs using paired and unpaired data, and a convolutional autoencoder in synthesizing DWI fractional anisotropy (FA) and mean diffusivity (MD) from whole FLAIR volumes. In total, 420 FLAIR and DWI volumes (11,957 images) from multi-center dementia and vascular disease cohorts were used for training/testing. Generated images were evaluated using two groups of metrics: (1) human perception metrics including peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), (2) structural metrics including a newly proposed histogram similarity (Hist-KL) metric and mean squared error (MSE). Results Pix2pix demonstrated the best performance both quantitatively and qualitatively with mean PSNR, SSIM, and MSE metrics of 23.41 dB, 0.8, 0.004, respectively for MD generation, and 24.05 dB, 0.78, 0.004, respectively for FA generation. The new histogram similarity metric demonstrated sensitivity to differences in fine details between generated and real images with mean pix2pix MD and FA Hist-KL metrics of 11.73 and 3.74, respectively. Detailed analysis of clinically relevant regions of white matter (WM) and gray matter (GM) in the pix2pix images also showed strong significant (p < 0.001) correlations between real and synthetic FA values in both tissue types (R = 0.714 for GM, R = 0.877 for WM). Discussion/conclusion Our results show that pix2pix's FA and MD models had significantly better structural similarity of tissue structures and fine details than other models, including WM tracts and CSF spaces, between real and generated images. Regional analysis of synthetic volumes showed that synthetic DWI images can not only be used to supplement clinical datasets, but demonstrates potential utility in bypassing or correcting registration in data pre-processing.
Collapse
Affiliation(s)
- Karissa Chan
- Electrical, Computer and Biomedical Engineering Department, Toronto Metropolitan University, Toronto, ON, Canada
- Institute of Biomedical Engineering, Science and Technology (iBEST), Toronto, ON, Canada
| | - Pejman Jabehdar Maralani
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Alan R. Moody
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - April Khademi
- Electrical, Computer and Biomedical Engineering Department, Toronto Metropolitan University, Toronto, ON, Canada
- Institute of Biomedical Engineering, Science and Technology (iBEST), Toronto, ON, Canada
- Keenan Research Center, St. Michael’s Hospital, Toronto, ON, Canada
| |
Collapse
|
11
|
Wahid KA, Xu J, El-Habashy D, Khamis Y, Abobakr M, McDonald B, O’ Connell N, Thill D, Ahmed S, Sharafi CS, Preston K, Salzillo TC, Mohamed ASR, He R, Cho N, Christodouleas J, Fuller CD, Naser MA. Deep-learning-based generation of synthetic 6-minute MRI from 2-minute MRI for use in head and neck cancer radiotherapy. Front Oncol 2022; 12:975902. [PMID: 36425548 PMCID: PMC9679225 DOI: 10.3389/fonc.2022.975902] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 10/21/2022] [Indexed: 11/10/2022] Open
Abstract
BackgroundQuick magnetic resonance imaging (MRI) scans with low contrast-to-noise ratio are typically acquired for daily MRI-guided radiotherapy setup. However, for patients with head and neck (HN) cancer, these images are often insufficient for discriminating target volumes and organs at risk (OARs). In this study, we investigated a deep learning (DL) approach to generate high-quality synthetic images from low-quality images.MethodsWe used 108 unique HN image sets of paired 2-minute T2-weighted scans (2mMRI) and 6-minute T2-weighted scans (6mMRI). 90 image sets (~20,000 slices) were used to train a 2-dimensional generative adversarial DL model that utilized 2mMRI as input and 6mMRI as output. Eighteen image sets were used to test model performance. Similarity metrics, including the mean squared error (MSE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) were calculated between normalized synthetic 6mMRI and ground-truth 6mMRI for all test cases. In addition, a previously trained OAR DL auto-segmentation model was used to segment the right parotid gland, left parotid gland, and mandible on all test case images. Dice similarity coefficients (DSC) were calculated between 2mMRI and either ground-truth 6mMRI or synthetic 6mMRI for each OAR; two one-sided t-tests were applied between the ground-truth and synthetic 6mMRI to determine equivalence. Finally, a visual Turing test using paired ground-truth and synthetic 6mMRI was performed using three clinician observers; the percentage of images that were correctly identified was compared to random chance using proportion equivalence tests.ResultsThe median similarity metrics across the whole images were 0.19, 0.93, and 33.14 for MSE, SSIM, and PSNR, respectively. The median of DSCs comparing ground-truth vs. synthetic 6mMRI auto-segmented OARs were 0.86 vs. 0.85, 0.84 vs. 0.84, and 0.82 vs. 0.85 for the right parotid gland, left parotid gland, and mandible, respectively (equivalence p<0.05 for all OARs). The percent of images correctly identified was equivalent to chance (p<0.05 for all observers).ConclusionsUsing 2mMRI inputs, we demonstrate that DL-generated synthetic 6mMRI outputs have high similarity to ground-truth 6mMRI, but further improvements can be made. Our study facilitates the clinical incorporation of synthetic MRI in MRI-guided radiotherapy.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | | | - Dina El-Habashy
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Clinical Oncology and Nuclear Medicine, Menoufia University, Shebin Elkom, Egypt
| | - Yomna Khamis
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Clinical Oncology and Nuclear Medicine, Faculty of Medicine, Alexandria University, Alexandria, Egypt
| | - Moamen Abobakr
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Brigid McDonald
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | | | | | - Sara Ahmed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Christina Setareh Sharafi
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Kathryn Preston
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Travis C. Salzillo
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Abdallah S. R. Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | | | | | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
- *Correspondence: Clifton D. Fuller, ; Mohamed A. Naser,
| | - Mohamed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
- *Correspondence: Clifton D. Fuller, ; Mohamed A. Naser,
| |
Collapse
|
12
|
Zhou Q, Zou H. A layer-wise fusion network incorporating self-supervised learning for multimodal MR image synthesis. Front Genet 2022; 13:937042. [PMID: 36017492 PMCID: PMC9396279 DOI: 10.3389/fgene.2022.937042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Magnetic resonance (MR) imaging plays an important role in medical diagnosis and treatment; different modalities of MR images can provide rich and complementary information to improve the accuracy of diagnosis. However, due to the limitations of scanning time and medical conditions, certain modalities of MR may be unavailable or of low quality in clinical practice. In this study, we propose a new multimodal MR image synthesis network to generate missing MR images. The proposed model comprises three stages: feature extraction, feature fusion, and image generation. During feature extraction, 2D and 3D self-supervised pretext tasks are introduced to pre-train the backbone for better representations of each modality. Then, a channel attention mechanism is used when fusing features so that the network can adaptively weigh different fusion operations to learn common representations of all modalities. Finally, a generative adversarial network is considered as the basic framework to generate images, in which a feature-level edge information loss is combined with the pixel-wise loss to ensure consistency between the synthesized and real images in terms of anatomical characteristics. 2D and 3D self-supervised pre-training can have better performance on feature extraction to retain more details in the synthetic images. Moreover, the proposed multimodal attention feature fusion block (MAFFB) in the well-designed layer-wise fusion strategy can model both common and unique information in all modalities, consistent with the clinical analysis. We also perform an interpretability analysis to confirm the rationality and effectiveness of our method. The experimental results demonstrate that our method can be applied in both single-modal and multimodal synthesis with high robustness and outperforms other state-of-the-art approaches objectively and subjectively.
Collapse
|