Feasibility of Synthetic Computed Tomography Images Generated from Magnetic Resonance Imaging Scans Using Various Deep Learning Methods in the Planning of Radiation Therapy for Prostate Cancer

CycleSeg: Simultaneous synthetic CT generation and unsupervised segmentation for MR-only radiotherapy treatment planning of prostate cancer

BACKGROUND

MR-only radiotherapy treatment planning is an attractive alternative to conventional workflow, reducing scan time and ionizing radiation. It is crucial to derive the electron density map or synthetic CT (sCT) from MR data to perform dose calculations to enable MR-only treatment planning. Automatic segmentation of relevant organs in MR images can accelerate the process by preventing the time-consuming manual contouring step. However, the segmentation label is available only for CT data in many cases.

PURPOSE

We propose CycleSeg, a unified framework that generates sCT and corresponding segmentation from MR images without access to MR segmentation labels METHODS: CycleSeg utilizes the CycleGAN formulation to perform unpaired synthesis of sCT and image alignment. To enable MR (sCT) segmentation, CycleSeg incorporates unsupervised domain adaptation by using a pseudo-labeling approach with feature alignment in semantic segmentation space. In contrast to previous approaches that perform segmentation on MR data, CycleSeg could perform segmentation on both MR and sCT. Experiments were performed with data from prostate cancer patients, with 78/7/10 subjects in the training/validation/test sets, respectively.

RESULTS

CycleSeg showed the best sCT generation results, with the lowest mean absolute error of 102.2 and the lowest Fréchet inception distance of 13.0. CycleSeg also performed best on MR segmentation, with the highest average dice score of 81.0 and 81.1 for MR and sCT segmentation, respectively. Ablation experiments confirmed the contribution of the proposed components of CycleSeg.

CONCLUSION

CycleSeg effectively synthesized CT and performed segmentation on MR images of prostate cancer patients. Thus, CycleSeg has the potential to expedite MR-only radiotherapy treatment planning, reducing the prescribed scans and manual segmentation effort, and increasing throughput.

Channel-wise attention enhanced and structural similarity constrained cycleGAN for effective synthetic CT generation from head and neck MRI images

BACKGROUND

Magnetic resonance imaging (MRI) plays an increasingly important role in radiotherapy, enhancing the accuracy of target and organs at risk delineation, but the absence of electron density information limits its further clinical application. Therefore, the aim of this study is to develop and evaluate a novel unsupervised network (cycleSimulationGAN) for unpaired MR-to-CT synthesis.

METHODS

The proposed cycleSimulationGAN in this work integrates contour consistency loss function and channel-wise attention mechanism to synthesize high-quality CT-like images. Specially, the proposed cycleSimulationGAN constrains the structural similarity between the synthetic and input images for better structural retention characteristics. Additionally, we propose to equip a novel channel-wise attention mechanism based on the traditional generator of GAN to enhance the feature representation capability of deep network and extract more effective features. The mean absolute error (MAE) of Hounsfield Units (HU), peak signal-to-noise ratio (PSNR), root-mean-square error (RMSE) and structural similarity index (SSIM) were calculated between synthetic CT (sCT) and ground truth (GT) CT images to quantify the overall sCT performance.

RESULTS

One hundred and sixty nasopharyngeal carcinoma (NPC) patients who underwent volumetric-modulated arc radiotherapy (VMAT) were enrolled in this study. The generated sCT of our method were more consistent with the GT compared with other methods in terms of visual inspection. The average MAE, RMSE, PSNR, and SSIM calculated over twenty patients were 61.88 ± 1.42, 116.85 ± 3.42, 36.23 ± 0.52 and 0.985 ± 0.002 for the proposed method. The four image quality assessment metrics were significantly improved by our approach compared to conventional cycleGAN, the proposed cycleSimulationGAN produces significantly better synthetic results except for SSIM in bone.

CONCLUSIONS

We developed a novel cycleSimulationGAN model that can effectively create sCT images, making them comparable to GT images, which could potentially benefit the MRI-based treatment planning.

Machine Learning for Medical Image Translation: A Systematic Review

BACKGROUND

CT scans are often the first and only form of brain imaging that is performed to inform treatment plans for neurological patients due to its time- and cost-effective nature. However, MR images give a more detailed picture of tissue structure and characteristics and are more likely to pick up abnormalities and lesions. The purpose of this paper is to review studies which use deep learning methods to generate synthetic medical images of modalities such as MRI and CT.

METHODS

A literature search was performed in March 2023, and relevant articles were selected and analyzed. The year of publication, dataset size, input modality, synthesized modality, deep learning architecture, motivations, and evaluation methods were analyzed.

RESULTS

A total of 103 studies were included in this review, all of which were published since 2017. Of these, 74% of studies investigated MRI to CT synthesis, and the remaining studies investigated CT to MRI, Cross MRI, PET to CT, and MRI to PET. Additionally, 58% of studies were motivated by synthesizing CT scans from MRI to perform MRI-only radiation therapy. Other motivations included synthesizing scans to aid diagnosis and completing datasets by synthesizing missing scans.

CONCLUSIONS

Considerably more research has been carried out on MRI to CT synthesis, despite CT to MRI synthesis yielding specific benefits. A limitation on medical image synthesis is that medical datasets, especially paired datasets of different modalities, are lacking in size and availability; it is therefore recommended that a global consortium be developed to obtain and make available more datasets for use. Finally, it is recommended that work be carried out to establish all uses of the synthesis of medical scans in clinical practice and discover which evaluation methods are suitable for assessing the synthesized images for these needs.

Comprehensive dose evaluation of a Deep Learning based synthetic Computed Tomography algorithm for pelvic Magnetic Resonance-only radiotherapy

BACKGROUND AND PURPOSE

Magnetic Resonance (MR)-only radiotherapy enables the use of MR without the uncertainty of MR-Computed Tomography (CT) registration. This requires a synthetic CT (sCT) for dose calculations, which can be facilitated by a novel Zero Echo Time (ZTE) sequence where bones are visible and images are acquired in 65 seconds. This study evaluated the dose calculation accuracy for pelvic sites of a ZTE-based Deep Learning sCT algorithm developed by GE Healthcare.

MATERIALS AND METHODS

ZTE and CT images were acquired in 56 pelvic radiotherapy patients in the radiotherapy position. A 2D U-net convolutional neural network was trained using pairs of deformably registered CT and ZTE images from 36 patients. In the remaining 20 patients the dosimetric accuracy of the sCT was assessed using cylindrical dummy Planning Target Volumes (PTVs) positioned at four different central axial locations, as well as the clinical treatment plans (for prostate (n=10), rectum (n=4) and anus (n=6) cancers). The sCT was rigidly and deformably registered, the plan recalculated and the doses compared using mean differences and gamma analysis.

RESULTS

Mean dose differences to the PTV D98% were ≤ 0.5% for all dummy PTVs and clinical plans (rigid registration). Mean gamma pass rates at 1%/1 mm were 98.0 ± 0.4% (rigid) and 100.0 ± 0.0% (deformable), 96.5 ± 0.8% and 99.8 ± 0.1%, and 95.4 ± 0.6% and 99.4 ± 0.4% for the clinical prostate, rectum and anus plans respectively.

CONCLUSIONS

A ZTE-based sCT algorithm with high dose accuracy throughout the pelvis has been developed. This suggests the algorithm is sufficiently accurate for MR-only radiotherapy for all pelvic sites.

Deep Learning in MRI-guided Radiation Therapy: A Systematic Review

MRI-guided radiation therapy (MRgRT) offers a precise and adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed. MRI-guided radiation therapy offers a precise, adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed with emphasis placed on underlying methods. Studies are further categorized into the areas of segmentation, synthesis, radiomics, and real time MRI. Finally, clinical implications, current challenges, and future directions are discussed.

Synthetic computed tomography generation for abdominal adaptive radiotherapy using low-field magnetic resonance imaging

Background and Purpose

Magnetic Resonance guided Radiotherapy (MRgRT) still needs the acquisition of Computed Tomography (CT) images and co-registration between CT and Magnetic Resonance Imaging (MRI). The generation of synthetic CT (sCT) images from the MR data can overcome this limitation. In this study we aim to propose a Deep Learning (DL) based approach for sCT image generation for abdominal Radiotherapy using low field MR images.

Materials and methods

CT and MR images were collected from 76 patients treated on abdominal sites. U-Net and conditional Generative Adversarial Network (cGAN) architectures were used to generate sCT images. Additionally, sCT images composed of only six bulk densities were generated with the aim of having a Simplified sCT.Radiotherapy plans calculated using the generated images were compared to the original plan in terms of gamma pass rate and Dose Volume Histogram (DVH) parameters.

Results

sCT images were generated in 2 s and 2.5 s with U-Net and cGAN architectures respectively.Gamma pass rates for 2%/2mm and 3%/3mm criteria were 91% and 95% respectively. Dose differences within 1% for DVH parameters on the target volume and organs at risk were obtained.

Conclusion

U-Net and cGAN architectures are able to generate abdominal sCT images fast and accurately from low field MRI.

Generation and Evaluation of Synthetic Computed Tomography (CT) from Cone-Beam CT (CBCT) by Incorporating Feature-Driven Loss into Intensity-Based Loss Functions in Deep Convolutional Neural Network

Simple Summary

Despite numerous benefits of cone-beam computed tomography (CBCT), its applications to radiotherapy were limited mainly due to degraded image quality. Recently, enhancing the CBCT image quality by generating synthetic CT image by deep convolutional neural network (CNN) has become frequent. Most of the previous works, however, generated synthetic CT with simple, classical intensity-driven loss in network training, while not specifying a full-package of verifications. This work trained the network by combining feature- and intensity-driven losses and attempted to demonstrate clinical relevance of the synthetic CT images by assessing both image similarity and dose calculating accuracy throughout a commercial Monte-Carlo.

Abstract

Deep convolutional neural network (CNN) helped enhance image quality of cone-beam computed tomography (CBCT) by generating synthetic CT. Most of the previous works, however, trained network by intensity-based loss functions, possibly undermining to promote image feature similarity. The verifications were not sufficient to demonstrate clinical applicability, either. This work investigated the effect of variable loss functions combining feature- and intensity-driven losses in synthetic CT generation, followed by strengthening the verification of generated images in both image similarity and dosimetry accuracy. The proposed strategy highlighted the feature-driven quantification in (1) training the network by perceptual loss, besides L1 and structural similarity (SSIM) losses regarding anatomical similarity, and (2) evaluating image similarity by feature mapping ratio (FMR), besides conventional metrics. In addition, the synthetic CT images were assessed in terms of dose calculating accuracy by a commercial Monte-Carlo algorithm. The network was trained with 50 paired CBCT-CT scans acquired at the same CT simulator and treatment unit to constrain environmental factors any other than loss functions. For 10 independent cases, incorporating perceptual loss into L1 and SSIM losses outperformed the other combinations, which enhanced FMR of image similarity by 10%, and the dose calculating accuracy by 1–2% of gamma passing rate in 1%/1mm criterion.

Improving the clinical workflow of a MR-Linac by dosimetric evaluation of synthetic CT

Adaptive radiotherapy performed on the daily magnetic resonance imaging (MRI) is an option to improve the treatment quality. In the adapt-to-shape workflow of 1.5-T MR-Linac, the contours of structures are adjusted on the basis of patient daily MRI, and the adapted plan is recalculated on the MRI-based synthetic computed tomography (syCT) generated by bulk density assignment. Because dosimetric accuracy of this strategy is a priority and requires evaluation, this study aims to explore the usefulness of adding an assessment of dosimetric errors associated with recalculation on syCT to the clinical workflow. Sixty-one patients, with various tumor sites, treated using a 1.5-T MR-Linac were included in this study. In Monaco V5.4, the target and organs at risk (OARs) were contoured, and a reference CT plan that contains information about the outlined contours, their average electron density (ED), and the priority of ED assignment was generated. To evaluate the dosimetric error of syCT caused by the inherent approximation within bulk density assignment, the reference CT plan was recalculated on the syCT obtained from the reference CT by forcing all contoured structures to their mean ED defined on the reference plan. The dose–volume histogram (DVH) and dose distribution of the CT and syCT plan were compared. The causes of dosimetric discrepancies were investigated, and the reference plan was reworked to minimize errors if needed. For 54 patients, gamma analysis of the dose distribution on syCT and CT show a median pass rate of 99.7% and 98.5% with the criteria of 3%/3 mm and 2%/2 mm, respectively. DVH difference of targets and OARs remained less than 1.5% or 1 Gy. For the remaining patients, factors (i.e., inappropriate ED assignments) influenced the dosimetric agreement of the syCT vs. CT reference DVH by up to 21%. The causes of the errors were promptly identified, and the DVH dosimetry was realigned except for two lung treatments for which a significant discrepancy remained. The recalculation on the syCT obtained from the planning CT is a powerful tool to assess and decrease the minimal error committed during the adaptive plan on the MRI-based syCT.

Synthetic 18F-FDG PET Image Generation Using a Combination of Biomathematical Modeling and Machine Learning

Simple Summary

Training computer-assisted algorithms on medical images, particularly 18F-fluorodeoxyglucose (18F-FDG) positron emission tomography (PET) due to its excellent diagnostic accuracy, is difficult, considering small/fragmented samples or privacy concerns. In computer-vision, deep learning-based models, unlike the conventional data augmentation methods, are highly sought after for creating massive medical samples. For this reason, we developed a multi-scale computational framework to generate synthetic 18F-FDG PET images similar to the real ones in different stages of solid tumor growth and angiogenesis. The framework is developed based on the bio-physiological phenomena of FDG radiotracer uptake in solid tumors using a biomathematical model and a generative adversarial network (GAN)-based architecture. The non-invasive augmented 18F-FDG PET images can be used in clinical practice without the need to manage the patient data. In addition, our spatiotemporal mathematical model can calculate the distribution of various radiopharmaceuticals in different tumor-associated vasculatures.

Abstract

No previous works have attempted to combine generative adversarial network (GAN) architectures and the biomathematical modeling of positron emission tomography (PET) radiotracer uptake in tumors to generate extra training samples. Here, we developed a novel computational model to produce synthetic 18F-fluorodeoxyglucose (18F-FDG) PET images of solid tumors in different stages of progression and angiogenesis. First, a comprehensive biomathematical model is employed for creating tumor-induced angiogenesis, intravascular and extravascular fluid flow, as well as modeling of the transport phenomena and reaction processes of 18F-FDG in a tumor microenvironment. Then, a deep convolutional GAN (DCGAN) model is employed for producing synthetic PET images using 170 input images of 18F-FDG uptake in each of 10 different tumor microvascular networks. The interstitial fluid parameters and spatiotemporal distribution of 18F-FDG uptake in tumor and healthy tissues have been compared against previously published numerical and experimental studies, indicating the accuracy of the model. The structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) of the generated PET sample and the experimental one are 0.72 and 28.53, respectively. Our results demonstrate that a combination of biomathematical modeling and GAN-based augmentation models provides a robust framework for the non-invasive and accurate generation of synthetic PET images of solid tumors in different stages.